jyapayne/swig
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'vadz-doxygen'

Browse source 
This is the Doxygen work begun in Google Summer of Code projects 2008
and 2012 and subsequently improved by numerous contributors.

* vadz-doxygen: (314 commits)
  Add changes entry for Doxygen support
  Add some missing doctype tyemaps
  Doxygen warnings cleanup
  Move doxygen warning numbers
  Add Python doxygen example
  Doxygen example
  Add Doxygen to include paths
  Doxygen source rename
  More merge fixes from doxygen branches
  Correct python example headers
  Correct source code headers
  Another merge fix from doxygen branches
  Java enums output format fixes
  Add omitted doxygen_parsing_enums testcase
  PEP8 conformance for comment verifier module
  Clean up merge problem
  Doxygen html tweaks
  Update html chapter numbering for added Doxygen chapter
  Fixes to makechap.py to detect ill-formed headers
  html fixes for Doxygen
  Add missing CPlusPlus17.html file
  Format files to unix format
  Doxygen testcase tweak to match that in the html docs
  Doxygen html documentation updates and corrections
  Remove doxygen Examples subdirectory
  Beautify doxygen source code
  Code formatting fixes in doxygen code
  Remove unused doxygen code
  new_node refactor
  Various merge fixes in doxygen branches
  Unused variable warning fix
  Fix wrongly resetting indent after formulae in Doxygen comments
  Add support for doxygen:alias feature
  Get rid of meaningless return type of DoxygenParser methods
  Return enum, not untyped int, when classifying Doxygen commands
  Get rid of unnecessary "typedef enum" in C++ code
  Use slash, not backslash, in "C/C++" in the documentation
  Replace literal "<" with "&lt;" in HTML documentation
  Fix broken link to java.sun.com in Doxygen documentation
  Fix using com.sun.tools.javadoc package under macOS
  Fix error reporting for special characters in Doxygen parsing code
  Switch Python Doxygen unit tests to use inspect.getdoc()
  Use correct separator in Java class path under Windows.
  Remove executable permission from appveyor.yml.
  Use JAVA_HOME value in configure to detect Java.
  Display JAVA_HOME value in "make java_version".
  Fix harmless MSVC warning in DoxygenTranslator code.
  Reset "_last" for all but first enum elements.
  Don't duplicate Javadoc from global enum Doxygen comments twice.
  Move Doxygen comments concatenation from the parser to the lexer.
  Fix shift/reduce conflicts in Doxygen pre/post comment parsing.
  Rewrote part of the grammar dealing with Doxygen comments for enums.
  No changes, just remove spurious white space only differences.
  Move Doxygen comment mangling from the parser to the lexer.
  Merge "-builtin" autodoc bugs workarounds from master into test.
  Quote JAVA_HOME variable value in Java test suite makefile.
  Remove unused C_COMMENT_STRING terminal from the grammar.
  Fix missing returns in the Doxygen test suite code.
  Fix trimming whitespace from Doxygen comments.
  Remove code not doing anything from PyDocConverter.
  Remove unused <sstream> header.
  Remove unreferenced struct declaration.
  Remove unused Swig_warn() function.
  Remove any whitespace before ignored Doxygen commands.
  Remove trailing space from one of Doxygen tests.
  Fix autodoc strings generated in Python builtin case and the test.
  Fix Doxygen unit test in Python "-builtin" case.
  Use class docstrings in "-builtin" Python case.
  Don't indent Doxygen doc strings in generated Python code.
  Add a possibility to flexibly ignore custom Doxygen tags.
  Stop completely ignoring many Doxygen comments.
  Fix structural Doxygen comment recognition in the parser.
  No changes, just make checking for Doxygen structural tags more sane.
  Use "//", not "#", for comments in SWIG input.
  Allow upper case letters and digits in Doxygen words.
  Pass the node the Doxygen comment is attached to to DoxygenParser.
  Get rid of findCommand() which duplicaed commandBelongs().
  Recognize unknown Doxygen tags correctly.
  No real changes, just pass original command to commandBelongs().
  Describe Doxygen-specific %features in a single place.
  Give warnings for unknown Doxygen commands in Doxygen parser.
  Document the return type when translating Doxygen @return to Python.
  Fix translated Doxygen comments for overloaded functions in Python.
  Also merge Doxygen comments for overloaded constructors in Python.
  Allow using enum elements as default values for Python functions.
  Don't always use "*args" for all Python wrapper functions.
  No real changes, just make PYTHON::check_kwargs() const.
  Refactor: move makeParameterName() to common Language base class.
  Remove long line wrapping from Python parameter list generation code.
  Simplify and make more efficient building Python docstrings.
  Translate Doxygen code blocks to Sphinx code blocks.
  Add a simple test of multiple parameters to Doxygen test suite.
  Make Python parameters types hyperlinks in the doc strings.
  Make Language::classLookup() and enumLookup() static.
  Fix arguments of @param, @return etc translations to Python.
  Remove unused method from PyDocConverter.
  No real changes, just remove an unnecessary variable.
  Preserve relative indentation when parsing Doxygen comments.
  Use Sphinx-friendly formatting for overloaded functions documentation.
  Add poor man trailing white space detection to Doxygen Python tests.
  ...
This commit is contained in:
William S Fulton 2018-06-07 08:13:10 +01:00
commit 33921666a1
123 changed files with 12964 additions and 1344 deletions
Download patch file Download diff file Expand all files Collapse all files

3
Source/CParse/cparse.h
View file

@ -28,6 +28,7 @@ extern "C" {
extern int cparse_cplusplusout;
extern int cparse_start_line;
extern String *cparse_unknown_directive;
extern int scan_doxygen_comments;
extern void Swig_cparse_cplusplus(int);
extern void Swig_cparse_cplusplusout(int);
@ -61,7 +62,7 @@ extern "C" {
extern void cparse_normalize_void(Node *);
extern Parm *Swig_cparse_parm(String *s);
extern ParmList *Swig_cparse_parms(String *s, Node *file_line_node);
extern Node *new_node(const_String_or_char_ptr tag);
extern Node *Swig_cparse_new_node(const_String_or_char_ptr tag);
/* templ.c */
extern int Swig_cparse_template_expand(Node *n, String *rname, ParmList *tparms, Symtab *tscope);

121
Source/CParse/cscanner.c
View file

@ -50,6 +50,57 @@ static int last_brace = 0;
static int last_id = 0;
static int rename_active = 0;
/* Doxygen comments scanning */
int scan_doxygen_comments = 0;
int isStructuralDoxygen(String *s) {
static const char* const structuralTags[] = {
"addtogroup",
"callgraph",
"callergraph",
"category",
"def",
"defgroup",
"dir",
"example",
"file",
"headerfile",
"internal",
"mainpage",
"name",
"nosubgrouping",
"overload",
"package",
"page",
"protocol",
"relates",
"relatesalso",
"showinitializer",
"weakgroup",
};
unsigned n;
char *slashPointer = Strchr(s, '\\');
char *atPointer = Strchr(s,'@');
if (slashPointer == NULL && atPointer == NULL)
return 0;
else if(slashPointer == NULL)
slashPointer = atPointer;
slashPointer++; /* skip backslash or at sign */
for (n = 0; n < sizeof(structuralTags)/sizeof(structuralTags[0]); n++) {
const size_t len = strlen(structuralTags[n]);
if (strncmp(slashPointer, structuralTags[n], len) == 0) {
/* Take care to avoid false positives with prefixes of other tags. */
if (slashPointer[len] == '\0' || isspace(slashPointer[len]))
return 1;
}
}
return 0;
}
/* -----------------------------------------------------------------------------
* Swig_cparse_cplusplus()
* ----------------------------------------------------------------------------- */
@ -367,10 +418,70 @@ static int yylook(void) {
case SWIG_TOKEN_COMMENT:
{
String *cmt = Scanner_text(scan);
char *loc = Char(cmt);
if ((strncmp(loc,"/*@SWIG",7) == 0) && (loc[Len(cmt)-3] == '@')) {
Scanner_locator(scan, cmt);
typedef enum {
DOX_COMMENT_PRE = -1,
DOX_COMMENT_NONE,
DOX_COMMENT_POST
} comment_kind_t;
comment_kind_t existing_comment = DOX_COMMENT_NONE;
/* Concatenate or skip all consecutive comments at once. */
do {
String *cmt = Scanner_text(scan);
char *loc = Char(cmt);
if ((strncmp(loc, "/*@SWIG", 7) == 0) && (loc[Len(cmt)-3] == '@')) {
Scanner_locator(scan, cmt);
}
if (scan_doxygen_comments) { /* else just skip this node, to avoid crashes in parser module*/
/* Check for all possible Doxygen comment start markers while ignoring
comments starting with a row of asterisks or slashes just as
Doxygen itself does. */
if (Len(cmt) > 3 && loc[0] == '/' &&
((loc[1] == '/' && ((loc[2] == '/' && loc[3] != '/') || loc[2] == '!')) ||
(loc[1] == '*' && ((loc[2] == '*' && loc[3] != '*') || loc[2] == '!')))) {
comment_kind_t this_comment = loc[3] == '<' ? DOX_COMMENT_POST : DOX_COMMENT_PRE;
if (existing_comment != DOX_COMMENT_NONE && this_comment != existing_comment) {
/* We can't concatenate together Doxygen pre- and post-comments. */
break;
}
if (this_comment == DOX_COMMENT_POST || !isStructuralDoxygen(loc)) {
String *str;
int begin = this_comment == DOX_COMMENT_POST ? 4 : 3;
int end = Len(cmt);
if (loc[end - 1] == '/' && loc[end - 2] == '*') {
end -= 2;
}
str = NewStringWithSize(loc + begin, end - begin);
if (existing_comment == DOX_COMMENT_NONE) {
yylval.str = str;
Setline(yylval.str, Scanner_start_line(scan));
Setfile(yylval.str, Scanner_file(scan));
} else {
Append(yylval.str, str);
}
existing_comment = this_comment;
}
}
}
do {
tok = Scanner_token(scan);
} while (tok == SWIG_TOKEN_ENDLINE);
} while (tok == SWIG_TOKEN_COMMENT);
Scanner_pushtoken(scan, tok, Scanner_text(scan));
switch (existing_comment) {
case DOX_COMMENT_PRE:
return DOXYGENSTRING;
case DOX_COMMENT_NONE:
break;
case DOX_COMMENT_POST:
return DOXYGENPOSTSTRING;
}
}
break;
@ -907,6 +1018,8 @@ int yylex(void) {
return (ID);
case POUND:
return yylex();
case SWIG_TOKEN_COMMENT:
return yylex();
default:
return (l);
}

198
Source/CParse/parser.y
View file

@ -64,6 +64,13 @@ static int template_reduce = 0;
static int cparse_externc = 0;
int ignore_nested_classes = 0;
int kwargs_supported = 0;
/* -----------------------------------------------------------------------------
* Doxygen Comment Globals
* ----------------------------------------------------------------------------- */
static String *currentDeclComment = NULL; /* Comment of C/C++ declaration. */
static Node *previousNode = NULL; /* Pointer to the previous node (for post comments) */
static Node *currentNode = NULL; /* Pointer to the current node (for post comments) */
/* -----------------------------------------------------------------------------
* Assist Functions
* ----------------------------------------------------------------------------- */
@ -75,6 +82,14 @@ static void yyerror (const char *e) {
(void)e;
}
static Node *new_node(const_String_or_char_ptr tag) {
Node *n = Swig_cparse_new_node(tag);
/* Remember the previous node in case it will need a post-comment */
previousNode = currentNode;
currentNode = n;
return n;
}
/* Copies a node. Does not copy tree links or symbol table data (except for
sym:name) */
@ -159,6 +174,36 @@ static Node *copy_node(Node *n) {
return nn;
}
static void set_comment(Node *n, String *comment) {
String *name;
Parm *p;
if (!n || !comment)
return;
if (Getattr(n, "doxygen"))
Append(Getattr(n, "doxygen"), comment);
else {
Setattr(n, "doxygen", comment);
/* This is the first comment, populate it with @params, if any */
p = Getattr(n, "parms");
while (p) {
if (Getattr(p, "doxygen"))
Printv(comment, "\n@param ", Getattr(p, "name"), Getattr(p, "doxygen"), NIL);
p=nextSibling(p);
}
}
/* Append same comment to every generated overload */
name = Getattr(n, "name");
if (!name)
return;
n = nextSibling(n);
while (n && Getattr(n, "name") && Strcmp(Getattr(n, "name"), name) == 0) {
Setattr(n, "doxygen", comment);
n = nextSibling(n);
}
}
/* -----------------------------------------------------------------------------
* Variables
* ----------------------------------------------------------------------------- */
@ -1553,6 +1598,9 @@ static String *add_qualifier_to_declarator(SwigType *type, SwigType *qualifier)
%token <str> CONVERSIONOPERATOR
%token PARSETYPE PARSEPARM PARSEPARMS
%token <str> DOXYGENSTRING
%token <str> DOXYGENPOSTSTRING
%left CAST
%left QUESTIONMARK
%left LOR
@ -1579,11 +1627,11 @@ static String *add_qualifier_to_declarator(SwigType *type, SwigType *qualifier)
/* C declarations */
%type <node> c_declaration c_decl c_decl_tail c_enum_key c_enum_inherit c_enum_decl c_enum_forward_decl c_constructor_decl;
%type <node> enumlist edecl;
%type <node> enumlist enumlist_tail enumlist_item edecl_with_dox edecl;
/* C++ declarations */
%type <node> cpp_declaration cpp_class_decl cpp_forward_class_decl cpp_template_decl cpp_alternate_rettype;
%type <node> cpp_members cpp_member;
%type <node> cpp_members cpp_member cpp_member_no_dox;
%type <node> cpp_constructor_decl cpp_destructor_decl cpp_protection_decl cpp_conversion_operator cpp_static_assert;
%type <node> cpp_swig_directive cpp_template_possible cpp_opt_declarators ;
%type <node> cpp_using_decl cpp_namespace_decl cpp_catch_decl cpp_lambda_decl;
@ -1595,7 +1643,7 @@ static String *add_qualifier_to_declarator(SwigType *type, SwigType *qualifier)
%type <id> storage_class extern_string;
%type <pl> parms ptail rawparms varargs_parms ;
%type <pl> templateparameters templateparameterstail;
%type <p> parm valparm rawvalparms valparms valptail ;
%type <p> parm_no_dox parm valparm rawvalparms valparms valptail ;
%type <p> typemap_parm tm_list tm_tail ;
%type <p> templateparameter ;
%type <id> templcpptype cpptype classkey classkeyopt access_specifier;
@ -1628,7 +1676,6 @@ static String *add_qualifier_to_declarator(SwigType *type, SwigType *qualifier)
%type <node> featattr;
%type <node> lambda_introducer lambda_body;
%type <pl> lambda_tail;
%type <node> optional_constant_directive;
%type <str> virt_specifier_seq virt_specifier_seq_opt;
%%
@ -1675,9 +1722,24 @@ program : interface {
interface : interface declaration {
/* add declaration to end of linked list (the declaration isn't always a single declaration, sometimes it is a linked list itself) */
if (currentDeclComment != NULL) {
set_comment($2, currentDeclComment);
currentDeclComment = NULL;
}
appendChild($1,$2);
$$ = $1;
}
| interface DOXYGENSTRING {
currentDeclComment = $2;
$$ = $1;
}
| interface DOXYGENPOSTSTRING {
Node *node = lastChild($1);
if (node) {
set_comment(node, $2);
}
$$ = $1;
}
| empty {
$$ = new_node("top");
}
@ -4553,7 +4615,7 @@ cpp_members : cpp_member cpp_members {
/* A class member. May be data or a function. Static or virtual as well */
cpp_member : c_declaration { $$ = $1; }
cpp_member_no_dox : c_declaration { $$ = $1; }
| cpp_constructor_decl {
$$ = $1;
if (extendmode && current_class) {
@ -4587,6 +4649,18 @@ cpp_member : c_declaration { $$ = $1; }
| fragment_directive {$$ = $1; }
| types_directive {$$ = $1; }
| SEMI { $$ = 0; }
cpp_member : cpp_member_no_dox {
$$ = $1;
}
| DOXYGENSTRING cpp_member_no_dox {
$$ = $2;
set_comment($2, $1);
}
| cpp_member_no_dox DOXYGENPOSTSTRING {
$$ = $1;
set_comment($1, $2);
}
;
/* Possibly a constructor */
@ -4974,20 +5048,31 @@ rawparms : parm ptail {
set_nextSibling($1,$2);
$$ = $1;
}
| empty { $$ = 0; }
| empty {
$$ = 0;
previousNode = currentNode;
currentNode=0;
}
;
ptail : COMMA parm ptail {
set_nextSibling($2,$3);
$$ = $2;
}
| COMMA DOXYGENPOSTSTRING parm ptail {
set_comment(previousNode, $2);
set_nextSibling($3, $4);
$$ = $3;
}
| empty { $$ = 0; }
;
parm : rawtype parameter_declarator {
parm_no_dox : rawtype parameter_declarator {
SwigType_push($1,$2.type);
$$ = NewParmWithoutFileLineInfo($1,$2.id);
previousNode = currentNode;
currentNode = $$;
Setfile($$,cparse_file);
Setline($$,cparse_line);
if ($2.defarg) {
@ -4997,6 +5082,8 @@ parm : rawtype parameter_declarator {
| TEMPLATE LESSTHAN cpptype GREATERTHAN cpptype idcolon def_args {
$$ = NewParmWithoutFileLineInfo(NewStringf("template<class> %s %s", $5,$6), 0);
previousNode = currentNode;
currentNode = $$;
Setfile($$,cparse_file);
Setline($$,cparse_line);
if ($7.val) {
@ -5006,11 +5093,26 @@ parm : rawtype parameter_declarator {
| PERIOD PERIOD PERIOD {
SwigType *t = NewString("v(...)");
$$ = NewParmWithoutFileLineInfo(t, 0);
previousNode = currentNode;
currentNode = $$;
Setfile($$,cparse_file);
Setline($$,cparse_line);
}
;
parm : parm_no_dox {
$$ = $1;
}
| DOXYGENSTRING parm_no_dox {
$$ = $2;
set_comment($2, $1);
}
| parm_no_dox DOXYGENPOSTSTRING {
$$ = $1;
set_comment($1, $2);
}
;
valparms : rawvalparms {
Parm *p;
$$ = $1;
@ -6217,28 +6319,70 @@ ename : identifier { $$ = $1; }
| empty { $$ = (char *) 0;}
;
optional_constant_directive : constant_directive { $$ = $1; }
| empty { $$ = 0; }
;
optional_ignored_define
: constant_directive
| empty
;
optional_ignored_define_after_comma
: empty
| COMMA
| COMMA constant_directive
;
/* Enum lists - any #define macros (constant directives) within the enum list are ignored. Trailing commas accepted. */
enumlist : enumlist COMMA optional_constant_directive edecl optional_constant_directive {
Node *leftSibling = Getattr($1,"_last");
set_nextSibling(leftSibling,$4);
Setattr($1,"_last",$4);
$$ = $1;
}
| enumlist COMMA optional_constant_directive {
$$ = $1;
}
| optional_constant_directive edecl optional_constant_directive {
Setattr($2,"_last",$2);
$$ = $2;
}
| optional_constant_directive {
$$ = 0;
}
;
enumlist : enumlist_item optional_ignored_define_after_comma {
Setattr($1,"_last",$1);
$$ = $1;
}
| enumlist_item enumlist_tail optional_ignored_define_after_comma {
set_nextSibling($1, $2);
Setattr($1,"_last",Getattr($2,"_last"));
Setattr($2,"_last",NULL);
$$ = $1;
}
| optional_ignored_define {
$$ = 0;
}
;
enumlist_tail : COMMA enumlist_item {
Setattr($2,"_last",$2);
$$ = $2;
}
| enumlist_tail COMMA enumlist_item {
set_nextSibling(Getattr($1,"_last"), $3);
Setattr($1,"_last",$3);
$$ = $1;
}
;
enumlist_item : optional_ignored_define edecl_with_dox optional_ignored_define {
$$ = $2;
}
;
edecl_with_dox : edecl {
$$ = $1;
}
| DOXYGENSTRING edecl {
$$ = $2;
set_comment($2, $1);
}
| edecl DOXYGENPOSTSTRING {
$$ = $1;
set_comment($1, $2);
}
| DOXYGENPOSTSTRING edecl {
$$ = $2;
set_comment(previousNode, $1);
}
| DOXYGENPOSTSTRING edecl DOXYGENPOSTSTRING {
$$ = $2;
set_comment(previousNode, $1);
set_comment($2, $3);
}
;
edecl : identifier {
SwigType *type = NewSwigType(T_INT);

4
Source/CParse/util.c
View file

@ -112,12 +112,12 @@ void cparse_normalize_void(Node *n) {
}
/* -----------------------------------------------------------------------------
* new_node()
* Swig_cparse_new_node()
*
* Create an empty parse node, setting file and line number information
* ----------------------------------------------------------------------------- */
Node *new_node(const_String_or_char_ptr tag) {
Node *Swig_cparse_new_node(const_String_or_char_ptr tag) {
Node *n = NewHash();
set_nodeType(n,tag);
Setfile(n,cparse_file);