diff --git a/Doc/Devel/index.html b/Doc/Devel/index.html index 35aff4ab1..752532070 100644 --- a/Doc/Devel/index.html +++ b/Doc/Devel/index.html @@ -19,6 +19,7 @@ The following documentation describe the internal APIs used by SWIG. These may
  • Command line arguments
  • Parse tree navigation and manipulation
  • Parameter and Parameter list handling functions +
  • Generic C/C++ Scanner interface
    diff --git a/Doc/Devel/scanner.html b/Doc/Devel/scanner.html new file mode 100644 index 000000000..28bd3af52 --- /dev/null +++ b/Doc/Devel/scanner.html @@ -0,0 +1,275 @@ + + +SWIG C Scanner + + + +
    +

    SWIG C/C++ Scanning

    + +

    +David M. Beazley
    +dave-swig@dabeaz.com
    +January 11, 2007
    + + +

    + +

    Introduction

    + +This document describes functions that can be used to tokenize C/C++ +input text. These functions are relatively low-level and are meant to +be used in the implementation of scanners that can be plugged into yacc or used for +other purposes. For instance, the preprocessor uses these functions to evaluate and test +constant expressions. + +

    +All of these functions are declared in Source/Swig/swigscan.h. This API is considered to be stable. + +

    Creation and Deletion of Scanner

    + +The following functions are used to create and destroy a scanner object. More than one scanner object can be created and used +as necessary. + +

    +Scanner *NewScanner() + +

    +Creates a new scanner object. The scanner contains initially contains no text. To feed text to the scanner use Scanner_push(). +
    + +

    +Scanner *DelScanner() + +

    +Deletes a scanner object. +
    + +

    Scanner Functions

    + +

    +void Scanner_clear(Scanner *s) +

    +Clears all text from the scanner. This can be used to reset a scanner to its initial state, ready to receive new input text. +
    + +

    +void Scanner_push(Scanner *s, String *text) +

    +Pushes an input string into the scanner. Subsequent tokens will be +returned from the new string. If the scanner is already processing a +string, the pushed string takes precedence--in effect, interrupting +the scanning of the previous string. This behavior is used to +implement certain SWIG features such as the %inline +directive. Once the pushed string has been completely scanned, the +scanner will return to scanning the previous string (if any). The +scanning of text relies upon the DOH file interface to strings +(Getc(), Ungetc(), etc.). Prior to calling this +function, the input string should be set so that its file pointer is +in the location where you want scanning to begin. You may have to +use Seek() to set the file pointer back to the beginning of a +string prior to using this function. +
    + +

    +void Scanner_pushtoken(Scanner *s, int tokvalue, String_or_char *val) +

    +Pushes a token into the scanner. This exact token will be returned by the next call to Scanner_token(). +tokvalue is the integer token value to return and val is the token text to return. This +function is only used to handle very special parsing cases. For instance, if you need the scanner to +return a ficticious token into order to enter a special parsing case. +
    + +

    +int Scanner_token(Scanner *s) + +

    +Returns the next token. An integer token code is returned (see table below) on success. If no more input text is +available 0 is returned. If a scanning error occurred, -1 is returned. In this case, error information can be +obtained using Scanner_errinfo(). +
    + +

    +String *Scanner_text(Scanner *s) +

    +Returns the scanned text corresponding to the last token returned by Scanner_token(). The returned string +is only valid until the next call to Scanner_token(). If you need to save it, make a copy. +
    + +

    +void Scanner_skip_line(Scanner *s) +

    +Skips to the end of the current line. The text skipped can be obtained using Scanner_text() afterwards. +
    + +

    +void Scanner_skip_balanced(Scanner *s, int startchar, int endchar) +

    +Skips to the end of a block of text denoted by starting and ending characters. For example, { and }. The +function is smart about how it skips text. String literals and comments are ignored. The function also is aware of nesting. The +skipped text can be obtained using Scanner_text() afterwards. Returns 0 on success, -1 if no matching endchar could be found. +
    + + +

    +void Scanner_set_location(Scanner *s, int startchar, int endchar) +

    +Changes the current filename and line number of the scanner.< +
    + +

    +String *Scanner_get_file(Scanner *s) +

    +Gets the current filename associated with text in the scanner. +
    + +

    +int Scanner_get_line(Scanner *s) +

    +Gets the current line number associated with text in the scanner. +
    + +

    +void Scanner_idstart(Scanner *s, char *idchar) +

    +Sets additional characters (other than the C default) that may be used to start C identifiers. idchar is a string +containing the characters (e.g., "%@"). The purpose of this function is to up special keywords such as "%module" or "@directive" as +simple identifiers. +
    + +

    +String *Scanner_errmsg(Scanner *s) +

    +Returns the error message associated with the last scanner error (if any). This will only return a meaningful result +if Scanner_token() returned -1. +
    + +

    +int Scanner_errline(Scanner *s) +

    +Returns the line number associated with the last scanner error (if any). This will only return a meaningful result +if Scanner_token() returned -1. The line number usually corresponds to the starting line number of a particular +token (e.g., for unterminated strings, comments, etc.). +
    + +

    +int Scanner_isoperator(int tokval) +

    +A convenience function that returns 0 or 1 depending on whether tokval is a valid C/C++ operator (i.e., a candidate for +operator overloading). +
    + + +

    Token Codes

    + +The following table shows token codes returned by the scanner. These are integer codes returned by +the Scanner_token() function. + +
    +
    +Token code                   C Token
    +-------------------------    -------------
    +SWIG_TOKEN_LPAREN            (
    +SWIG_TOKEN_RPAREN            )
    +SWIG_TOKEN_SEMI              ; 
    +SWIG_TOKEN_COMMA             , 
    +SWIG_TOKEN_STAR              * 
    +SWIG_TOKEN_TIMES             *
    +SWIG_TOKEN_LBRACE            { 
    +SWIG_TOKEN_RBRACE            } 
    +SWIG_TOKEN_EQUAL             = 
    +SWIG_TOKEN_EQUALTO           == 
    +SWIG_TOKEN_NOTEQUAL          != 
    +SWIG_TOKEN_PLUS              + 
    +SWIG_TOKEN_MINUS             - 
    +SWIG_TOKEN_AND               & 
    +SWIG_TOKEN_LAND              && 
    +SWIG_TOKEN_OR                | 
    +SWIG_TOKEN_LOR               || 
    +SWIG_TOKEN_XOR               ^ 
    +SWIG_TOKEN_LESSTHAN          < 
    +SWIG_TOKEN_GREATERTHAN       > 
    +SWIG_TOKEN_LTEQUAL           <= 
    +SWIG_TOKEN_GTEQUAL           >= 
    +SWIG_TOKEN_NOT               ~ 
    +SWIG_TOKEN_LNOT              ! 
    +SWIG_TOKEN_LBRACKET          [ 
    +SWIG_TOKEN_RBRACKET          ] 
    +SWIG_TOKEN_SLASH             / 
    +SWIG_TOKEN_DIVIDE            /
    +SWIG_TOKEN_BACKSLASH         \ 
    +SWIG_TOKEN_POUND             # 
    +SWIG_TOKEN_PERCENT           % 
    +SWIG_TOKEN_MODULO            %
    +SWIG_TOKEN_COLON             : 
    +SWIG_TOKEN_DCOLON            :: 
    +SWIG_TOKEN_DCOLONSTAR        ::*
    +SWIG_TOKEN_LSHIFT            << 
    +SWIG_TOKEN_RSHIFT            >> 
    +SWIG_TOKEN_QUESTION          ? 
    +SWIG_TOKEN_PLUSPLUS          ++ 
    +SWIG_TOKEN_MINUSMINUS        -- 
    +SWIG_TOKEN_PLUSEQUAL         += 
    +SWIG_TOKEN_MINUSEQUAL        -= 
    +SWIG_TOKEN_TIMESEQUAL        *= 
    +SWIG_TOKEN_DIVEQUAL          /= 
    +SWIG_TOKEN_ANDEQUAL          &= 
    +SWIG_TOKEN_OREQUAL           |= 
    +SWIG_TOKEN_XOREQUAL          ^= 
    +SWIG_TOKEN_LSEQUAL           <<= 
    +SWIG_TOKEN_RSEQUAL           >>= 
    +SWIG_TOKEN_MODEQUAL          %= 
    +SWIG_TOKEN_ARROW             -> 
    +SWIG_TOKEN_ARROWSTAR         ->* 
    +SWIG_TOKEN_PERIOD            . 
    +SWIG_TOKEN_AT                @ 
    +SWIG_TOKEN_DOLLAR            $ 
    +SWIG_TOKEN_ENDLINE           Literal newline
    +SWIG_TOKEN_ID                identifer 
    +SWIG_TOKEN_FLOAT             Floating point with F suffix (e.g., 3.1415F)
    +SWIG_TOKEN_DOUBLE            Floating point (e.g., 3.1415 )
    +SWIG_TOKEN_INT               Integer (e.g., 314)
    +SWIG_TOKEN_UINT              Unsigned integer (e.g., 314U)
    +SWIG_TOKEN_LONG              Long integer (e.g., 314L) 
    +SWIG_TOKEN_ULONG             Unsigned long integer (e.g., 314UL)
    +SWIG_TOKEN_LONGLONG          Long long integer (e.g., 314LL )
    +SWIG_TOKEN_ULONGLONG         Unsigned long long integer (e.g., 314ULL) 
    +SWIG_TOKEN_CHAR              Character literal in single quotes ('c')
    +SWIG_TOKEN_STRING            String literal in double quotes ("str")
    +SWIG_TOKEN_RSTRING           Reverse quote string (`str`)
    +SWIG_TOKEN_CODEBLOCK         SWIG code literal block %{ ... %}
    +SWIG_TOKEN_COMMENT           C or C++ comment  (// or /* ... */)
    +SWIG_TOKEN_ILLEGAL           Illegal character
    +
    +
    + +Notes + + + + + + + + + + diff --git a/Source/Preprocessor/cpp.c b/Source/Preprocessor/cpp.c index 255b5fb8e..e0c36a6f9 100644 --- a/Source/Preprocessor/cpp.c +++ b/Source/Preprocessor/cpp.c @@ -27,7 +27,7 @@ static int imported_depth = 0; /* Depth of %imported files */ static int single_include = 1; /* Only include each file once */ static Hash *included_files = 0; static List *dependencies = 0; -static SwigScanner *id_scan = 0; +static Scanner *id_scan = 0; static int error_as_warning = 0; /* Understand the cpp #error directive as a special #warning */ /* Test a character to see if it starts an identifier */ @@ -185,7 +185,7 @@ void Preprocessor_init(void) { Preprocessor_expr_init(); /* Initialize the expression evaluator */ included_files = NewHash(); - id_scan = NewSwigScanner();; + id_scan = NewScanner();; } @@ -225,7 +225,7 @@ void Preprocessor_delete(void) { Delete(cpp); Delete(included_files); Preprocessor_expr_delete(); - DelSwigScanner(id_scan); + DelScanner(id_scan); Delete(dependencies); @@ -1150,15 +1150,15 @@ DOH *Preprocessor_replace(DOH *s) { static int checkpp_id(DOH *s) { int c; int hastok = 0; - SwigScanner *scan = id_scan; + Scanner *scan = id_scan; Seek(s, 0, SEEK_SET); - SwigScanner_clear(scan); + Scanner_clear(scan); s = Copy(s); Seek(s, SEEK_SET, 0); - SwigScanner_push(scan, s); - while ((c = SwigScanner_token(scan))) { + Scanner_push(scan, s); + while ((c = Scanner_token(scan))) { hastok = 1; if ((c == SWIG_TOKEN_ID) || (c == SWIG_TOKEN_LBRACE) || (c == SWIG_TOKEN_RBRACE)) return 1; diff --git a/Source/Preprocessor/expr.c b/Source/Preprocessor/expr.c index 3b1d794f4..6113851b9 100644 --- a/Source/Preprocessor/expr.c +++ b/Source/Preprocessor/expr.c @@ -13,7 +13,7 @@ char cvsroot_expr_c[] = "$Id$"; #include "swig.h" #include "preprocessor.h" -static SwigScanner *scan = 0; +static Scanner *scan = 0; typedef struct { int op; @@ -222,12 +222,12 @@ void Preprocessor_expr_init(void) { if (!expr_init) init_precedence(); if (!scan) - scan = NewSwigScanner(); + scan = NewScanner(); } void Preprocessor_expr_delete(void) { - SwigScanner_clear(scan); - DelSwigScanner(scan); + Scanner_clear(scan); + DelScanner(scan); } @@ -235,11 +235,11 @@ void Preprocessor_expr_delete(void) { * Tokenizer * ----------------------------------------------------------------------------- */ -static int expr_token(SwigScanner * s) { +static int expr_token(Scanner * s) { int t; while (1) { - t = SwigScanner_token(s); - if (!((t == SWIG_TOKEN_BACKSLASH) || (t == SWIG_TOKEN_ENDLINE))) + t = Scanner_token(s); + if (!((t == SWIG_TOKEN_BACKSLASH) || (t == SWIG_TOKEN_ENDLINE) || (t == SWIG_TOKEN_COMMENT))) break; } return t; @@ -262,8 +262,8 @@ int Preprocessor_expr(DOH *s, int *error) { Seek(s, 0, SEEK_SET); /* Printf(stdout,"evaluating : '%s'\n", s); */ *error = 0; - SwigScanner_clear(scan); - SwigScanner_push(scan, s); + Scanner_clear(scan); + Scanner_push(scan, s); /* Put initial state onto the stack */ stack[sp].op = EXPR_TOP; @@ -282,10 +282,10 @@ int Preprocessor_expr(DOH *s, int *error) { } if ((token == SWIG_TOKEN_INT) || (token == SWIG_TOKEN_UINT) || (token == SWIG_TOKEN_LONG) || (token == SWIG_TOKEN_ULONG)) { /* A number. Reduce EXPR_TOP to an EXPR_VALUE */ - char *c = Char(SwigScanner_text(scan)); + char *c = Char(Scanner_text(scan)); stack[sp].value = (long) strtol(c, 0, 0); stack[sp].svalue = 0; - /* stack[sp].value = (long) atol(Char(SwigScanner_text(scan))); */ + /* stack[sp].value = (long) atol(Char(Scanner_text(scan))); */ stack[sp].op = EXPR_VALUE; } else if (token == SWIG_TOKEN_PLUS) { } else if ((token == SWIG_TOKEN_MINUS) || (token == SWIG_TOKEN_LNOT) || (token == SWIG_TOKEN_NOT)) { @@ -302,7 +302,7 @@ int Preprocessor_expr(DOH *s, int *error) { stack[sp].svalue = 0; } else if (token == SWIG_TOKEN_ENDLINE) { } else if ((token == SWIG_TOKEN_STRING)) { - stack[sp].svalue = NewString(SwigScanner_text(scan)); + stack[sp].svalue = NewString(Scanner_text(scan)); stack[sp].op = EXPR_VALUE; } else if ((token == SWIG_TOKEN_ID)) { stack[sp].value = 0; diff --git a/Source/Swig/scanner.c b/Source/Swig/scanner.c index 0515a8646..d41e0a1f4 100644 --- a/Source/Swig/scanner.c +++ b/Source/Swig/scanner.c @@ -15,28 +15,30 @@ char cvsroot_scanner_c[] = "$Id$"; #include "swig.h" #include -struct SwigScanner { +struct Scanner { String *text; /* Current token value */ - List *scanobjs; /* Objects being scanned */ + List *scanobjs; /* Objects being scanned */ String *str; /* Current object being scanned */ - char *idstart; /* Optional identifier start characters */ - int nexttoken; /* Next token to be returned */ - int start_line; /* Starting line of certain declarations */ - int string_start; - int line; - int yylen; /* Length of text pushed into text */ + char *idstart; /* Optional identifier start characters */ + int nexttoken; /* Next token to be returned */ + int start_line; /* Starting line of certain declarations */ + int string_start; + int line; + int yylen; /* Length of text pushed into text */ String *file; + String *error; /* Last error message (if any) */ + int error_line; /* Error line number */ }; /* ----------------------------------------------------------------------------- - * NewSwigScanner() + * NewScanner() * * Create a new scanner object * ----------------------------------------------------------------------------- */ -SwigScanner *NewSwigScanner() { - SwigScanner *s; - s = (SwigScanner *) malloc(sizeof(SwigScanner)); +Scanner *NewScanner() { + Scanner *s; + s = (Scanner *) malloc(sizeof(Scanner)); s->line = 1; s->file = 0; s->nexttoken = -1; @@ -47,34 +49,39 @@ SwigScanner *NewSwigScanner() { s->scanobjs = NewList(); s->text = NewStringEmpty(); s->str = 0; + s->error = 0; return s; } /* ----------------------------------------------------------------------------- - * DelSwigScanner() + * DelScanner() * * Delete a scanner object. * ----------------------------------------------------------------------------- */ -void DelSwigScanner(SwigScanner * s) { +void DelScanner(Scanner * s) { assert(s); Delete(s->scanobjs); Delete(s->text); Delete(s->file); + Delete(s->error); + Delete(s->str); free(s); } /* ----------------------------------------------------------------------------- - * SwigScanner_clear() + * Scanner_clear() * * Clear the contents of a scanner object. * ----------------------------------------------------------------------------- */ -void SwigScanner_clear(SwigScanner * s) { +void Scanner_clear(Scanner * s) { assert(s); Delete(s->str); Clear(s->text); Clear(s->scanobjs); + Delete(s->error); + s->error = 0; s->line = 1; s->nexttoken = -1; s->start_line = 0; @@ -83,13 +90,13 @@ void SwigScanner_clear(SwigScanner * s) { } /* ----------------------------------------------------------------------------- - * SwigScanner_push() + * Scanner_push() * * Push some new text into the scanner. The scanner will start parsing this text * immediately before returning to the old text. * ----------------------------------------------------------------------------- */ -void SwigScanner_push(SwigScanner * s, String *txt) { +void Scanner_push(Scanner * s, String *txt) { assert(s && txt); Push(s->scanobjs, txt); if (s->str) @@ -100,55 +107,57 @@ void SwigScanner_push(SwigScanner * s, String *txt) { } /* ----------------------------------------------------------------------------- - * SwigScanner_pushtoken() + * Scanner_pushtoken() * * Push a token into the scanner. This token will be returned on the next - * call to SwigScanner_token(). + * call to Scanner_token(). * ----------------------------------------------------------------------------- */ -void SwigScanner_pushtoken(SwigScanner * s, int nt) { +void Scanner_pushtoken(Scanner * s, int nt, String_or_char *val) { assert(s); assert((nt >= 0) && (nt < SWIG_MAXTOKENS)); s->nexttoken = nt; + Clear(s->text); + Append(s->text,val); } /* ----------------------------------------------------------------------------- - * SwigScanner_set_location() + * Scanner_set_location() * * Set the file and line number location of the scanner. * ----------------------------------------------------------------------------- */ -void SwigScanner_set_location(SwigScanner * s, String *file, int line) { +void Scanner_set_location(Scanner * s, String *file, int line) { Setline(s->str, line); Setfile(s->str, file); } /* ----------------------------------------------------------------------------- - * SwigScanner_get_file() + * Scanner_get_file() * * Get the current file. * ----------------------------------------------------------------------------- */ -String *SwigScanner_get_file(SwigScanner * s) { +String *Scanner_get_file(Scanner * s) { return Getfile(s->str); } /* ----------------------------------------------------------------------------- - * SwigScanner_get_line() + * Scanner_get_line() * * Get the current line number * ----------------------------------------------------------------------------- */ -int SwigScanner_get_line(SwigScanner * s) { +int Scanner_get_line(Scanner * s) { return Getline(s->str); } /* ----------------------------------------------------------------------------- - * SwigScanner_idstart() + * Scanner_idstart() * * Change the set of additional characters that can be used to start an identifier. * ----------------------------------------------------------------------------- */ -void SwigScanner_idstart(SwigScanner * s, char *id) { +void Scanner_idstart(Scanner * s, char *id) { s->idstart = Swig_copy_string(id); } @@ -157,7 +166,7 @@ void SwigScanner_idstart(SwigScanner * s, char *id) { * * Returns the next character from the scanner or 0 if end of the string. * ----------------------------------------------------------------------------- */ -static char nextchar(SwigScanner * s) { +static char nextchar(Scanner * s) { char c[2] = { 0, 0 }; int nc; if (!s->str) @@ -182,12 +191,40 @@ static char nextchar(SwigScanner * s) { return c[0]; } +/* ----------------------------------------------------------------------------- + * set_error() + * + * Sets error information on the scanner. + * ----------------------------------------------------------------------------- */ + +static void set_error(Scanner *s, int line, String_or_char *msg) { + s->error_line = line; + s->error = NewString(msg); +} + +/* ----------------------------------------------------------------------------- + * Scanner_errmsg() + * Scanner_errline() + * + * Returns error information (if any) + * ----------------------------------------------------------------------------- */ + +String * +Scanner_errmsg(Scanner *s) { + return s->error; +} + +int +Scanner_errline(Scanner *s) { + return s->error_line; +} + /* ----------------------------------------------------------------------------- * retract() * * Retract n characters * ----------------------------------------------------------------------------- */ -static void retract(SwigScanner * s, int n) { +static void retract(Scanner * s, int n) { int i, l; char *str; @@ -198,21 +235,146 @@ static void retract(SwigScanner * s, int n) { if (str[l - 1] == '\n') { s->line--; } - /* // Ungetc(str[l-1],s->str); */ Seek(s->str, -1, SEEK_CUR); Delitem(s->text, DOH_END); } } +/* ----------------------------------------------------------------------------- + * get_escape() + * + * Get escape sequence. Called when a backslash is found in a string + * ----------------------------------------------------------------------------- */ + +static void get_escape(Scanner *s) { + int result = 0; + int state = 0; + int c; + + while (1) { + c = nextchar(s); + if (c == 0) + break; + switch (state) { + case 0: + if (c == 'n') { + Delitem(s->text, DOH_END); + Append(s->text,"\n"); + return; + } + if (c == 'r') { + Delitem(s->text, DOH_END); + Append(s->text,"\r"); + return; + } + if (c == 't') { + Delitem(s->text, DOH_END); + Append(s->text,"\t"); + return; + } + if (c == 'a') { + Delitem(s->text, DOH_END); + Append(s->text,"\a"); + return; + } + if (c == 'b') { + Delitem(s->text, DOH_END); + Append(s->text,"\b"); + return; + } + if (c == 'f') { + Delitem(s->text, DOH_END); + Append(s->text,"\f"); + return; + } + if (c == '\\') { + Delitem(s->text, DOH_END); + Append(s->text,"\\"); + return; + } + if (c == 'v') { + Delitem(s->text, DOH_END); + Append(s->text,"\v"); + return; + } + if (c == 'e') { + Delitem(s->text, DOH_END); + Append(s->text,"\033"); + return; + } + if (c == '\'') { + Delitem(s->text, DOH_END); + Append(s->text,"\'"); + return; + } + if (c == '\"') { + Delitem(s->text, DOH_END); + Append(s->text,"\""); + return; + } + if (c == '\n') { + Delitem(s->text, DOH_END); + return; + } + if (isdigit(c)) { + state = 10; + result = (c - '0'); + Delitem(s->text, DOH_END); + } else if (c == 'x') { + state = 20; + Delitem(s->text, DOH_END); + } else { + char tmp[3]; + tmp[0] = '\\'; + tmp[1] = c; + tmp[2] = 0; + Delitem(s->text, DOH_END); + Append(s->text, tmp); + return; + } + break; + case 10: + if (!isdigit(c)) { + char tmp[2]; + retract(s,1); + tmp[0] = (char) result; + tmp[1] = 0; + Append(s->text, tmp); + return; + } + result = (result << 3) + (c - '0'); + Delitem(s->text, DOH_END); + break; + case 20: + if (!isxdigit(c)) { + char tmp[2]; + retract(s,1); + tmp[0] = (char) result; + tmp[1] = 0; + Append(s->text,tmp); + return; + } + if (isdigit(c)) + result = (result << 4) + (c - '0'); + else + result = (result << 4) + (10 + tolower(c) - 'a'); + Delitem(s->text, DOH_END); + break; + } + } + return; +} + /* ----------------------------------------------------------------------------- * look() * * Return the raw value of the next token. * ----------------------------------------------------------------------------- */ -static int look(SwigScanner * s) { +static int look(Scanner * s) { int state; int c = 0; + int comment_start = 0; state = 0; Clear(s->text); @@ -259,7 +421,7 @@ static int look(SwigScanner * s) { else if (c == ',') return SWIG_TOKEN_COMMA; else if (c == '*') - return SWIG_TOKEN_STAR; + state = 220; else if (c == '}') return SWIG_TOKEN_RBRACE; else if (c == '{') @@ -267,15 +429,15 @@ static int look(SwigScanner * s) { else if (c == '=') state = 33; else if (c == '+') - return SWIG_TOKEN_PLUS; + state = 200; else if (c == '-') - return SWIG_TOKEN_MINUS; + state = 210; else if (c == '&') state = 31; else if (c == '|') state = 32; else if (c == '^') - return SWIG_TOKEN_XOR; + state = 230; else if (c == '<') state = 60; else if (c == '>') @@ -296,11 +458,15 @@ static int look(SwigScanner * s) { return SWIG_TOKEN_DOLLAR; else if (c == '#') return SWIG_TOKEN_POUND; + else if (c == '?') + return SWIG_TOKEN_QUESTION; /* Look for multi-character sequences */ - else if (c == '/') + else if (c == '/') { state = 1; /* Comment (maybe) */ + comment_start = s->line; + } else if (c == '\"') { state = 2; /* Possibly a string */ s->string_start = s->line; @@ -334,14 +500,15 @@ static int look(SwigScanner * s) { Clear(s->text); Setline(s->text, Getline(s->str)); Setfile(s->text, Getfile(s->str)); - - Append(s->text, " "); + Append(s->text, "//"); } else if (c == '*') { state = 11; /* C style comment */ Clear(s->text); Setline(s->text, Getline(s->str)); Setfile(s->text, Getfile(s->str)); - Append(s->text, " "); + Append(s->text, "/*"); + } else if (c == '=') { + return SWIG_TOKEN_DIVEQUAL; } else { retract(s, 1); return SWIG_TOKEN_SLASH; @@ -349,19 +516,20 @@ static int look(SwigScanner * s) { break; case 10: /* C++ style comment */ if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated comment",comment_start); */ - return 0; + set_error(s,comment_start,"Unterminated comment"); + return SWIG_TOKEN_ERROR; } if (c == '\n') { - return SWIG_TOKEN_ENDLINE; + retract(s,1); + return SWIG_TOKEN_COMMENT; } else { state = 10; } break; case 11: /* C style comment block */ if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated comment",comment_start); */ - return 0; + set_error(s,comment_start,"Unterminated comment"); + return SWIG_TOKEN_ERROR; } if (c == '*') { state = 12; @@ -371,14 +539,13 @@ static int look(SwigScanner * s) { break; case 12: /* Still in C style comment */ if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated comment",comment_start); */ - return 0; + set_error(s,comment_start,"Unterminated comment"); + return SWIG_TOKEN_ERROR; } if (c == '*') { state = 12; } else if (c == '/') { - Clear(s->text); - state = 0; + return SWIG_TOKEN_COMMENT; } else { state = 11; } @@ -386,23 +553,16 @@ static int look(SwigScanner * s) { case 2: /* Processing a string */ if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated string", string_start); */ - return 0; + set_error(s,s->string_start, "Unterminated string"); + return SWIG_TOKEN_ERROR; } if (c == '\"') { return SWIG_TOKEN_STRING; } else if (c == '\\') { - state = 21; /* Possibly an escape sequence. */ - break; + get_escape(s); } else state = 2; break; - case 21: /* An escape sequence. get next character, then go - back to processing strings */ - if ((c = nextchar(s)) == 0) - return 0; - state = 2; - break; case 3: /* Maybe a not equals */ if ((c = nextchar(s)) == 0) @@ -415,11 +575,13 @@ static int look(SwigScanner * s) { } break; - case 31: /* AND or Logical AND */ + case 31: /* AND or Logical AND or ANDEQUAL */ if ((c = nextchar(s)) == 0) return SWIG_TOKEN_AND; else if (c == '&') return SWIG_TOKEN_LAND; + else if (c == '=') + return SWIG_TOKEN_ANDEQUAL; else { retract(s, 1); return SWIG_TOKEN_AND; @@ -431,6 +593,8 @@ static int look(SwigScanner * s) { return SWIG_TOKEN_OR; else if (c == '|') return SWIG_TOKEN_LOR; + else if (c == '=') + return SWIG_TOKEN_OREQUAL; else { retract(s, 1); return SWIG_TOKEN_OR; @@ -457,9 +621,11 @@ static int look(SwigScanner * s) { Setline(s->text, Getline(s->str)); Setfile(s->text, Getfile(s->str)); s->start_line = s->line; - } else if (strchr(s->idstart, '%') && ((isalpha(c)) || (c == '_'))) + } else if (strchr(s->idstart, '%') && ((isalpha(c)) || (c == '_'))) { state = 7; - else { + } else if (c == '=') { + return SWIG_TOKEN_MODEQUAL; + } else { retract(s, 1); return SWIG_TOKEN_PERCENT; } @@ -467,15 +633,15 @@ static int look(SwigScanner * s) { case 40: /* Process an include block */ if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated code block.", start_line); */ - return 0; + set_error(s,s->start_line,"Unterminated code block"); + return SWIG_TOKEN_ERROR; } if (c == '%') state = 41; break; case 41: /* Still processing include block */ if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated code block.", start_line); */ + set_error(s,s->start_line,"Unterminated code block"); return 0; } if (c == '}') { @@ -492,18 +658,29 @@ static int look(SwigScanner * s) { if ((c = nextchar(s)) == 0) return SWIG_TOKEN_COLON; if (c == ':') - return SWIG_TOKEN_DCOLON; + state = 50; else { retract(s, 1); return SWIG_TOKEN_COLON; } break; + case 50: /* DCOLON, DCOLONSTAR */ + if ((c = nextchar(s)) == 0) + return SWIG_TOKEN_DCOLON; + else if (c == '*') + return SWIG_TOKEN_DCOLONSTAR; + else { + retract(s, 1); + return SWIG_TOKEN_DCOLON; + } + break; + case 60: /* shift operators */ if ((c = nextchar(s)) == 0) return SWIG_TOKEN_LESSTHAN; if (c == '<') - return SWIG_TOKEN_LSHIFT; + state = 240; else if (c == '=') return SWIG_TOKEN_LTEQUAL; else { @@ -515,7 +692,7 @@ static int look(SwigScanner * s) { if ((c = nextchar(s)) == 0) return SWIG_TOKEN_GREATERTHAN; if (c == '>') - return SWIG_TOKEN_RSHIFT; + state = 250; else if (c == '=') return SWIG_TOKEN_GTEQUAL; else { @@ -703,21 +880,14 @@ static int look(SwigScanner * s) { /* A character constant */ case 9: if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated character constant", string_start); */ - return 0; + set_error(s,s->string_start,"Unterminated character constant"); + return SWIG_TOKEN_ERROR; } if (c == '\'') { return (SWIG_TOKEN_CHAR); - } else if (c == '\\') - state = 91; - break; - - case 91: - if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated character constant", string_start); */ - return 0; + } else if (c == '\\') { + get_escape(s); } - state = 9; break; /* A period or maybe a floating point number */ @@ -733,13 +903,98 @@ static int look(SwigScanner * s) { } break; + case 200: /* PLUS, PLUSPLUS, PLUSEQUAL */ + if ((c = nextchar(s)) == 0) + return SWIG_TOKEN_PLUS; + else if (c == '+') + return SWIG_TOKEN_PLUSPLUS; + else if (c == '=') + return SWIG_TOKEN_PLUSEQUAL; + else { + retract(s, 1); + return SWIG_TOKEN_PLUS; + } + break; + + case 210: /* MINUS, MINUSMINUS, MINUSEQUAL, ARROW */ + if ((c = nextchar(s)) == 0) + return SWIG_TOKEN_MINUS; + else if (c == '+') + return SWIG_TOKEN_MINUSMINUS; + else if (c == '=') + return SWIG_TOKEN_MINUSEQUAL; + else if (c == '>') + state = 211; + else { + retract(s, 1); + return SWIG_TOKEN_MINUS; + } + break; + + case 211: /* ARROW, ARROWSTAR */ + if ((c = nextchar(s)) == 0) + return SWIG_TOKEN_ARROW; + else if (c == '*') + return SWIG_TOKEN_ARROWSTAR; + else { + retract(s, 1); + return SWIG_TOKEN_ARROW; + } + break; + + + case 220: /* STAR, TIMESEQUAL */ + if ((c = nextchar(s)) == 0) + return SWIG_TOKEN_STAR; + else if (c == '=') + return SWIG_TOKEN_TIMESEQUAL; + else { + retract(s, 1); + return SWIG_TOKEN_STAR; + } + break; + + case 230: /* XOR, XOREQUAL */ + if ((c = nextchar(s)) == 0) + return SWIG_TOKEN_XOR; + else if (c == '=') + return SWIG_TOKEN_XOREQUAL; + else { + retract(s, 1); + return SWIG_TOKEN_XOR; + } + break; + + case 240: /* LSHIFT, LSEQUAL */ + if ((c = nextchar(s)) == 0) + return SWIG_TOKEN_LSHIFT; + else if (c == '=') + return SWIG_TOKEN_LSEQUAL; + else { + retract(s, 1); + return SWIG_TOKEN_LSHIFT; + } + break; + + case 250: /* RSHIFT, RSEQUAL */ + if ((c = nextchar(s)) == 0) + return SWIG_TOKEN_RSHIFT; + else if (c == '=') + return SWIG_TOKEN_RSEQUAL; + else { + retract(s, 1); + return SWIG_TOKEN_RSHIFT; + } + break; + + /* An illegal character */ /* Reverse string */ case 900: if ((c = nextchar(s)) == 0) { - /* add_error(0,"Unterminated character constant", string_start); */ - return 0; + set_error(s,s->string_start,"Unterminated character constant"); + return SWIG_TOKEN_ERROR; } if (c == '`') { return (SWIG_TOKEN_RSTRING); @@ -753,40 +1008,41 @@ static int look(SwigScanner * s) { } /* ----------------------------------------------------------------------------- - * SwigScanner_token() + * Scanner_token() * * Real entry point to return the next token. Returns 0 if at end of input. * ----------------------------------------------------------------------------- */ -int SwigScanner_token(SwigScanner * s) { +int Scanner_token(Scanner * s) { int t; - Clear(s->text); + Delete(s->error); if (s->nexttoken >= 0) { t = s->nexttoken; s->nexttoken = -1; return t; } + Clear(s->text); t = look(s); return t; } /* ----------------------------------------------------------------------------- - * SwigScanner_text() + * Scanner_text() * * Return the lexene associated with the last returned token. * ----------------------------------------------------------------------------- */ -String *SwigScanner_text(SwigScanner * s) { +String *Scanner_text(Scanner * s) { return s->text; } /* ----------------------------------------------------------------------------- - * SwigScanner_skip_line() + * Scanner_skip_line() * * Skips to the end of a line * ----------------------------------------------------------------------------- */ -void SwigScanner_skip_line(SwigScanner * s) { +void Scanner_skip_line(Scanner * s) { char c; int done = 0; Clear(s->text); @@ -804,13 +1060,13 @@ void SwigScanner_skip_line(SwigScanner * s) { } /* ----------------------------------------------------------------------------- - * SwigScanner_skip_balanced() + * Scanner_skip_balanced() * * Skips a piece of code enclosed in begin/end symbols such as '{...}' or * (...). Ignores symbols inside comments or strings. * ----------------------------------------------------------------------------- */ -int SwigScanner_skip_balanced(SwigScanner * s, int startchar, int endchar) { +int Scanner_skip_balanced(Scanner * s, int startchar, int endchar) { char c; int num_levels = 1; int l; @@ -890,3 +1146,16 @@ int SwigScanner_skip_balanced(SwigScanner * s, int startchar, int endchar) { } return 0; } + +/* ----------------------------------------------------------------------------- + * Scanner_isoperator() + * + * Returns 0 or 1 depending on whether or not a token corresponds to a C/C++ + * operator. + * ----------------------------------------------------------------------------- */ + +int +Scanner_isoperator(int tokval) { + if (tokval >= 100) return 1; + return 0; +} diff --git a/Source/Swig/swig.h b/Source/Swig/swig.h index dcb425228..7e2e70fe2 100644 --- a/Source/Swig/swig.h +++ b/Source/Swig/swig.h @@ -103,75 +103,7 @@ extern "C" { /* --- Scanner Interface --- */ - typedef struct SwigScanner SwigScanner; - - extern SwigScanner *NewSwigScanner(); - extern void DelSwigScanner(SwigScanner *); - extern void SwigScanner_clear(SwigScanner *); - extern void SwigScanner_push(SwigScanner *, String *); - extern void SwigScanner_pushtoken(SwigScanner *, int); - extern int SwigScanner_token(SwigScanner *); - extern String *SwigScanner_text(SwigScanner *); - extern void SwigScanner_skip_line(SwigScanner *); - extern int SwigScanner_skip_balanced(SwigScanner *, int startchar, int endchar); - extern void SwigScanner_set_location(SwigScanner *, String *file, int line); - extern String *SwigScanner_get_file(SwigScanner *); - extern int SwigScanner_get_line(SwigScanner *); - extern void SwigScanner_idstart(SwigScanner *, char *idchar); - -#define SWIG_MAXTOKENS 512 -#define SWIG_TOKEN_LPAREN 1 -#define SWIG_TOKEN_RPAREN 2 -#define SWIG_TOKEN_SEMI 3 -#define SWIG_TOKEN_COMMA 4 -#define SWIG_TOKEN_STAR 5 -#define SWIG_TOKEN_LBRACE 6 -#define SWIG_TOKEN_RBRACE 7 -#define SWIG_TOKEN_EQUAL 8 -#define SWIG_TOKEN_EQUALTO 9 -#define SWIG_TOKEN_NOTEQUAL 10 -#define SWIG_TOKEN_PLUS 11 -#define SWIG_TOKEN_MINUS 12 -#define SWIG_TOKEN_AND 13 -#define SWIG_TOKEN_LAND 14 -#define SWIG_TOKEN_OR 15 -#define SWIG_TOKEN_LOR 16 -#define SWIG_TOKEN_XOR 17 -#define SWIG_TOKEN_LESSTHAN 18 -#define SWIG_TOKEN_GREATERTHAN 19 -#define SWIG_TOKEN_LTEQUAL 20 -#define SWIG_TOKEN_GTEQUAL 21 -#define SWIG_TOKEN_NOT 22 -#define SWIG_TOKEN_LNOT 23 -#define SWIG_TOKEN_LBRACKET 24 -#define SWIG_TOKEN_RBRACKET 25 -#define SWIG_TOKEN_SLASH 26 -#define SWIG_TOKEN_BACKSLASH 27 -#define SWIG_TOKEN_ENDLINE 28 -#define SWIG_TOKEN_STRING 29 -#define SWIG_TOKEN_POUND 30 -#define SWIG_TOKEN_PERCENT 31 -#define SWIG_TOKEN_COLON 32 -#define SWIG_TOKEN_DCOLON 33 -#define SWIG_TOKEN_LSHIFT 34 -#define SWIG_TOKEN_RSHIFT 35 -#define SWIG_TOKEN_ID 36 -#define SWIG_TOKEN_FLOAT 37 -#define SWIG_TOKEN_DOUBLE 38 -#define SWIG_TOKEN_INT 39 -#define SWIG_TOKEN_UINT 40 -#define SWIG_TOKEN_LONG 41 -#define SWIG_TOKEN_ULONG 42 -#define SWIG_TOKEN_CHAR 43 -#define SWIG_TOKEN_PERIOD 44 -#define SWIG_TOKEN_AT 45 -#define SWIG_TOKEN_DOLLAR 46 -#define SWIG_TOKEN_CODEBLOCK 47 -#define SWIG_TOKEN_RSTRING 48 -#define SWIG_TOKEN_LONGLONG 49 -#define SWIG_TOKEN_ULONGLONG 50 -#define SWIG_TOKEN_ILLEGAL 98 -#define SWIG_TOKEN_LAST 99 +#include "swigscan.h" /* --- Functions for manipulating the string-based type encoding --- */ diff --git a/Source/Swig/swigscan.h b/Source/Swig/swigscan.h new file mode 100644 index 000000000..49ed99763 --- /dev/null +++ b/Source/Swig/swigscan.h @@ -0,0 +1,106 @@ +/* ----------------------------------------------------------------------------- + * See the LICENSE file for information on copyright, usage and redistribution + * of SWIG, and the README file for authors - http://www.swig.org/release.html. + * + * swigscan.h + * + * C/C++ scanner. + * ----------------------------------------------------------------------------- */ + +/* $Id: swig.h 9633 2007-01-10 23:43:07Z beazley $ */ + +typedef struct Scanner Scanner; + +extern Scanner *NewScanner(); +extern void DelScanner(Scanner *); +extern void Scanner_clear(Scanner *); +extern void Scanner_push(Scanner *, String *); +extern void Scanner_pushtoken(Scanner *, int, String_or_char *value); +extern int Scanner_token(Scanner *); +extern String *Scanner_text(Scanner *); +extern void Scanner_skip_line(Scanner *); +extern int Scanner_skip_balanced(Scanner *, int startchar, int endchar); +extern void Scanner_set_location(Scanner *, String *file, int line); +extern String *Scanner_get_file(Scanner *); +extern int Scanner_get_line(Scanner *); +extern void Scanner_idstart(Scanner *, char *idchar); +extern String *Scanner_errmsg(Scanner *); +extern int Scanner_errline(Scanner *); +extern int Scanner_isoperator(int tokval); + +/* Note: Tokens in range 100+ are for C/C++ operators */ + +#define SWIG_MAXTOKENS 200 +#define SWIG_TOKEN_LPAREN 1 /* ( */ +#define SWIG_TOKEN_RPAREN 2 /* ) */ +#define SWIG_TOKEN_SEMI 3 /* ; */ +#define SWIG_TOKEN_LBRACE 4 /* { */ +#define SWIG_TOKEN_RBRACE 5 /* } */ +#define SWIG_TOKEN_LBRACKET 6 /* [ */ +#define SWIG_TOKEN_RBRACKET 7 /* ] */ +#define SWIG_TOKEN_BACKSLASH 8 /* \ */ +#define SWIG_TOKEN_ENDLINE 9 /* \n */ +#define SWIG_TOKEN_STRING 10 /* "str" */ +#define SWIG_TOKEN_POUND 11 /* # */ +#define SWIG_TOKEN_COLON 12 /* : */ +#define SWIG_TOKEN_DCOLON 13 /* :: */ +#define SWIG_TOKEN_DCOLONSTAR 14 /* ::* */ +#define SWIG_TOKEN_ID 15 /* identifer */ +#define SWIG_TOKEN_FLOAT 16 /* 3.1415F */ +#define SWIG_TOKEN_DOUBLE 17 /* 3.1415 */ +#define SWIG_TOKEN_INT 18 /* 314 */ +#define SWIG_TOKEN_UINT 19 /* 314U */ +#define SWIG_TOKEN_LONG 20 /* 314L */ +#define SWIG_TOKEN_ULONG 21 /* 314UL */ +#define SWIG_TOKEN_CHAR 22 /* 'charconst' */ +#define SWIG_TOKEN_PERIOD 23 /* . */ +#define SWIG_TOKEN_AT 24 /* @ */ +#define SWIG_TOKEN_DOLLAR 25 /* $ */ +#define SWIG_TOKEN_CODEBLOCK 26 /* %{ ... %} ... */ +#define SWIG_TOKEN_RSTRING 27 /* `charconst` */ +#define SWIG_TOKEN_LONGLONG 28 /* 314LL */ +#define SWIG_TOKEN_ULONGLONG 29 /* 314ULL */ +#define SWIG_TOKEN_QUESTION 30 /* ? */ +#define SWIG_TOKEN_COMMENT 31 /* C or C++ comment */ +#define SWIG_TOKEN_ILLEGAL 99 +#define SWIG_TOKEN_ERROR -1 + +#define SWIG_TOKEN_COMMA 101 /* , */ +#define SWIG_TOKEN_STAR 102 /* * */ +#define SWIG_TOKEN_TIMES 102 /* * */ +#define SWIG_TOKEN_EQUAL 103 /* = */ +#define SWIG_TOKEN_EQUALTO 104 /* == */ +#define SWIG_TOKEN_NOTEQUAL 105 /* != */ +#define SWIG_TOKEN_PLUS 106 /* + */ +#define SWIG_TOKEN_MINUS 107 /* - */ +#define SWIG_TOKEN_AND 108 /* & */ +#define SWIG_TOKEN_LAND 109 /* && */ +#define SWIG_TOKEN_OR 110 /* | */ +#define SWIG_TOKEN_LOR 111 /* || */ +#define SWIG_TOKEN_XOR 112 /* ^ */ +#define SWIG_TOKEN_LESSTHAN 113 /* < */ +#define SWIG_TOKEN_GREATERTHAN 114 /* > */ +#define SWIG_TOKEN_LTEQUAL 115 /* <= */ +#define SWIG_TOKEN_GTEQUAL 116 /* >= */ +#define SWIG_TOKEN_NOT 117 /* ~ */ +#define SWIG_TOKEN_LNOT 118 /* ! */ +#define SWIG_TOKEN_SLASH 119 /* / */ +#define SWIG_TOKEN_DIVIDE 119 /* / */ +#define SWIG_TOKEN_PERCENT 120 /* % */ +#define SWIG_TOKEN_MODULO 120 /* % */ +#define SWIG_TOKEN_LSHIFT 121 /* << */ +#define SWIG_TOKEN_RSHIFT 122 /* >> */ +#define SWIG_TOKEN_PLUSPLUS 123 /* ++ */ +#define SWIG_TOKEN_MINUSMINUS 124 /* -- */ +#define SWIG_TOKEN_PLUSEQUAL 125 /* += */ +#define SWIG_TOKEN_MINUSEQUAL 126 /* -= */ +#define SWIG_TOKEN_TIMESEQUAL 127 /* *= */ +#define SWIG_TOKEN_DIVEQUAL 128 /* /= */ +#define SWIG_TOKEN_ANDEQUAL 129 /* &= */ +#define SWIG_TOKEN_OREQUAL 130 /* |= */ +#define SWIG_TOKEN_XOREQUAL 131 /* ^= */ +#define SWIG_TOKEN_LSEQUAL 132 /* <<= */ +#define SWIG_TOKEN_RSEQUAL 133 /* >>= */ +#define SWIG_TOKEN_MODEQUAL 134 /* %= */ +#define SWIG_TOKEN_ARROW 135 /* -> */ +#define SWIG_TOKEN_ARROWSTAR 136 /* ->* */