jyapayne/swig
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

API cleanup. Scanner interface

Browse source 
git-svn-id: https://swig.svn.sourceforge.net/svnroot/swig/trunk@9635 626c5289-ae23-0410-ae9c-e8d60b6d4f22
This commit is contained in:
Dave Beazley 2007-01-12 01:44:16 +00:00
commit 6ffd26cc1e
7 changed files with 762 additions and 179 deletions
Download patch file Download diff file Expand all files Collapse all files

1
Doc/Devel/index.html
View file

@ -19,6 +19,7 @@ The following documentation describe the internal APIs used by SWIG. These may
<li><a href="cmdopt.html">Command line arguments</a>
<li><a href="tree.html">Parse tree navigation and manipulation</a>
<li><a href="parm.html">Parameter and Parameter list handling functions</a>
<li><a href="scanner.html">Generic C/C++ Scanner interface</a>
</ul>
<hr>

275
Doc/Devel/scanner.html Normal file
View file

@ -0,0 +1,275 @@
<html>
<head>
<title>SWIG C Scanner</title>
</head>
<body>
<center>
<h1>SWIG C/C++ Scanning</h1>
<p>
David M. Beazley <br>
dave-swig@dabeaz.com<br>
January 11, 2007<br>
</b>
</center>
<h2>Introduction</h2>
This document describes functions that can be used to tokenize C/C++
input text. These functions are relatively low-level and are meant to
be used in the implementation of scanners that can be plugged into yacc or used for
other purposes. For instance, the preprocessor uses these functions to evaluate and test
constant expressions.
<p>
All of these functions are declared in <tt>Source/Swig/swigscan.h</tt>. This API is considered to be stable.
<h2>Creation and Deletion of Scanner</h2>
The following functions are used to create and destroy a scanner object. More than one scanner object can be created and used
as necessary.
<p>
<b><tt>Scanner *NewScanner()</tt></b>
<blockquote>
Creates a new scanner object. The scanner contains initially contains no text. To feed text to the scanner use <tt>Scanner_push()</tt>.
</blockquote>
<p>
<b><tt>Scanner *DelScanner()</tt></b>
<blockquote>
Deletes a scanner object.
</blockquote>
<h2>Scanner Functions</h2>
<p>
<b><tt>void Scanner_clear(Scanner *s)</tt></b>
<blockquote>
Clears all text from the scanner. This can be used to reset a scanner to its initial state, ready to receive new input text.
</blockquote>
<p>
<b><tt>void Scanner_push(Scanner *s, String *text)</tt></b>
<blockquote>
Pushes an input string into the scanner. Subsequent tokens will be
returned from the new string. If the scanner is already processing a
string, the pushed string takes precedence--in effect, interrupting
the scanning of the previous string. This behavior is used to
implement certain SWIG features such as the <tt>%inline</tt>
directive. Once the pushed string has been completely scanned, the
scanner will return to scanning the previous string (if any). The
scanning of text relies upon the DOH file interface to strings
(<tt>Getc()</tt>, <tt>Ungetc()</tt>, etc.). Prior to calling this
function, the input string should be set so that its file pointer is
in the location where you want scanning to begin. You may have to
use <tt>Seek()</tt> to set the file pointer back to the beginning of a
string prior to using this function.
</blockquote>
<p>
<b><tt>void Scanner_pushtoken(Scanner *s, int tokvalue, String_or_char *val)</tt></b>
<blockquote>
Pushes a token into the scanner. This exact token will be returned by the next call to <tt>Scanner_token()</tt>.
<tt>tokvalue</tt> is the integer token value to return and <tt>val</tt> is the token text to return. This
function is only used to handle very special parsing cases. For instance, if you need the scanner to
return a ficticious token into order to enter a special parsing case.
</blockquote>
<p>
<b><tt>int Scanner_token(Scanner *s)</tt></b>
<blockquote>
Returns the next token. An integer token code is returned (see table below) on success. If no more input text is
available 0 is returned. If a scanning error occurred, -1 is returned. In this case, error information can be
obtained using <tt>Scanner_errinfo()</tt>.
</blockquote>
<p>
<b><tt>String *Scanner_text(Scanner *s)</tt></b>
<blockquote>
Returns the scanned text corresponding to the last token returned by <tt>Scanner_token()</tt>. The returned string
is only valid until the next call to <tt>Scanner_token()</tt>. If you need to save it, make a copy.
</blockquote>
<p>
<b><tt>void Scanner_skip_line(Scanner *s)</tt></b>
<blockquote>
Skips to the end of the current line. The text skipped can be obtained using <tt>Scanner_text()</tt> afterwards.
</blockquote>
<p>
<b><tt>void Scanner_skip_balanced(Scanner *s, int startchar, int endchar)</tt></b>
<blockquote>
Skips to the end of a block of text denoted by starting and ending characters. For example, <tt>{</tt> and <tt>}</tt>. The
function is smart about how it skips text. String literals and comments are ignored. The function also is aware of nesting. The
skipped text can be obtained using <tt>Scanner_text()</tt> afterwards. Returns 0 on success, -1 if no matching <tt>endchar</tt> could be found.
</blockquote>
<p>
<b><tt>void Scanner_set_location(Scanner *s, int startchar, int endchar)</tt></b>
<blockquote>
Changes the current filename and line number of the scanner.<
</blockquote>
<p>
<b><tt>String *Scanner_get_file(Scanner *s)</tt></b>
<blockquote>
Gets the current filename associated with text in the scanner.
</blockquote>
<p>
<b><tt>int Scanner_get_line(Scanner *s)</tt></b>
<blockquote>
Gets the current line number associated with text in the scanner.
</blockquote>
<p>
<b><tt>void Scanner_idstart(Scanner *s, char *idchar)</tt></b>
<blockquote>
Sets additional characters (other than the C default) that may be used to start C identifiers. <tt>idchar</tt> is a string
containing the characters (e.g., "%@"). The purpose of this function is to up special keywords such as "%module" or "@directive" as
simple identifiers.
</blockquote>
<p>
<b><tt>String *Scanner_errmsg(Scanner *s)</tt></b>
<blockquote>
Returns the error message associated with the last scanner error (if any). This will only return a meaningful result
if <tt>Scanner_token()</tt> returned -1.
</blockquote>
<p>
<b><tt>int Scanner_errline(Scanner *s)</tt></b>
<blockquote>
Returns the line number associated with the last scanner error (if any). This will only return a meaningful result
if <tt>Scanner_token()</tt> returned -1. The line number usually corresponds to the starting line number of a particular
token (e.g., for unterminated strings, comments, etc.).
</blockquote>
<p>
<b><tt>int Scanner_isoperator(int tokval)</tt></b>
<blockquote>
A convenience function that returns 0 or 1 depending on whether <tt>tokval</tt> is a valid C/C++ operator (i.e., a candidate for
operator overloading).
</blockquote>
<h2>Token Codes</h2>
The following table shows token codes returned by the scanner. These are integer codes returned by
the <tt>Scanner_token()</tt> function.
<blockquote>
<pre>
Token code C Token
------------------------- -------------
SWIG_TOKEN_LPAREN (
SWIG_TOKEN_RPAREN )
SWIG_TOKEN_SEMI ;
SWIG_TOKEN_COMMA ,
SWIG_TOKEN_STAR *
SWIG_TOKEN_TIMES *
SWIG_TOKEN_LBRACE {
SWIG_TOKEN_RBRACE }
SWIG_TOKEN_EQUAL =
SWIG_TOKEN_EQUALTO ==
SWIG_TOKEN_NOTEQUAL !=
SWIG_TOKEN_PLUS +
SWIG_TOKEN_MINUS -
SWIG_TOKEN_AND &amp;
SWIG_TOKEN_LAND &amp;&amp;
SWIG_TOKEN_OR |
SWIG_TOKEN_LOR ||
SWIG_TOKEN_XOR ^
SWIG_TOKEN_LESSTHAN &lt;
SWIG_TOKEN_GREATERTHAN >
SWIG_TOKEN_LTEQUAL &lt;=
SWIG_TOKEN_GTEQUAL >=
SWIG_TOKEN_NOT ~
SWIG_TOKEN_LNOT !
SWIG_TOKEN_LBRACKET [
SWIG_TOKEN_RBRACKET ]
SWIG_TOKEN_SLASH /
SWIG_TOKEN_DIVIDE /
SWIG_TOKEN_BACKSLASH \
SWIG_TOKEN_POUND #
SWIG_TOKEN_PERCENT %
SWIG_TOKEN_MODULO %
SWIG_TOKEN_COLON :
SWIG_TOKEN_DCOLON ::
SWIG_TOKEN_DCOLONSTAR ::*
SWIG_TOKEN_LSHIFT &lt;&lt;
SWIG_TOKEN_RSHIFT >>
SWIG_TOKEN_QUESTION ?
SWIG_TOKEN_PLUSPLUS ++
SWIG_TOKEN_MINUSMINUS --
SWIG_TOKEN_PLUSEQUAL +=
SWIG_TOKEN_MINUSEQUAL -=
SWIG_TOKEN_TIMESEQUAL *=
SWIG_TOKEN_DIVEQUAL /=
SWIG_TOKEN_ANDEQUAL &amp;=
SWIG_TOKEN_OREQUAL |=
SWIG_TOKEN_XOREQUAL ^=
SWIG_TOKEN_LSEQUAL &lt;&lt;=
SWIG_TOKEN_RSEQUAL >>=
SWIG_TOKEN_MODEQUAL %=
SWIG_TOKEN_ARROW ->
SWIG_TOKEN_ARROWSTAR ->*
SWIG_TOKEN_PERIOD .
SWIG_TOKEN_AT @
SWIG_TOKEN_DOLLAR $
SWIG_TOKEN_ENDLINE Literal newline
SWIG_TOKEN_ID identifer
SWIG_TOKEN_FLOAT Floating point with F suffix (e.g., 3.1415F)
SWIG_TOKEN_DOUBLE Floating point (e.g., 3.1415 )
SWIG_TOKEN_INT Integer (e.g., 314)
SWIG_TOKEN_UINT Unsigned integer (e.g., 314U)
SWIG_TOKEN_LONG Long integer (e.g., 314L)
SWIG_TOKEN_ULONG Unsigned long integer (e.g., 314UL)
SWIG_TOKEN_LONGLONG Long long integer (e.g., 314LL )
SWIG_TOKEN_ULONGLONG Unsigned long long integer (e.g., 314ULL)
SWIG_TOKEN_CHAR Character literal in single quotes ('c')
SWIG_TOKEN_STRING String literal in double quotes ("str")
SWIG_TOKEN_RSTRING Reverse quote string (`str`)
SWIG_TOKEN_CODEBLOCK SWIG code literal block %{ ... %}
SWIG_TOKEN_COMMENT C or C++ comment (// or /* ... */)
SWIG_TOKEN_ILLEGAL Illegal character
</pre>
</blockquote>
<b>Notes</b>
<ul>
<li>When more than one token code exist for the same token text, those codes are identical (e.g., <tt>SWIG_TOKEN_STAR</tt> and <tt>SWIG_TOKEN_TIMES</tt>).
<p>
<li>
String literals are returned in their exact representation in which escape codes (if any) have been interpreted.
<p>
<li>
All C identifiers and keywords are simply returned as <tt>SWIG_TOKEN_ID</tt>. To check for specific keywords, you will need to
add extra checking on the returned text.
<p>
<li>C and C++ comments include the comment starting and ending text (e.g., "//", "/*").
<p>
<li>The maximum token integer value is found in the constant <tt>SWIG_MAXTOKENS</tt>. This can be used if you wanted to create
an array or table for the purposes of remapping tokens to a different set of codes. For instance, if you are
using these functions to write a yacc-compatible lexer.
</ul>
</body>
</html>

14
Source/Preprocessor/cpp.c
View file

@ -27,7 +27,7 @@ static int imported_depth = 0; /* Depth of %imported files */
static int single_include = 1; /* Only include each file once */
static Hash *included_files = 0;
static List *dependencies = 0;
static SwigScanner *id_scan = 0;
static Scanner *id_scan = 0;
static int error_as_warning = 0; /* Understand the cpp #error directive as a special #warning */
/* Test a character to see if it starts an identifier */
@ -185,7 +185,7 @@ void Preprocessor_init(void) {
Preprocessor_expr_init(); /* Initialize the expression evaluator */
included_files = NewHash();
id_scan = NewSwigScanner();;
id_scan = NewScanner();;
}
@ -225,7 +225,7 @@ void Preprocessor_delete(void) {
Delete(cpp);
Delete(included_files);
Preprocessor_expr_delete();
DelSwigScanner(id_scan);
DelScanner(id_scan);
Delete(dependencies);
@ -1150,15 +1150,15 @@ DOH *Preprocessor_replace(DOH *s) {
static int checkpp_id(DOH *s) {
int c;
int hastok = 0;
SwigScanner *scan = id_scan;
Scanner *scan = id_scan;
Seek(s, 0, SEEK_SET);
SwigScanner_clear(scan);
Scanner_clear(scan);
s = Copy(s);
Seek(s, SEEK_SET, 0);
SwigScanner_push(scan, s);
while ((c = SwigScanner_token(scan))) {
Scanner_push(scan, s);
while ((c = Scanner_token(scan))) {
hastok = 1;
if ((c == SWIG_TOKEN_ID) || (c == SWIG_TOKEN_LBRACE) || (c == SWIG_TOKEN_RBRACE))
return 1;

24
Source/Preprocessor/expr.c
View file

@ -13,7 +13,7 @@ char cvsroot_expr_c[] = "$Id$";
#include "swig.h"
#include "preprocessor.h"
static SwigScanner *scan = 0;
static Scanner *scan = 0;
typedef struct {
int op;
@ -222,12 +222,12 @@ void Preprocessor_expr_init(void) {
if (!expr_init)
init_precedence();
if (!scan)
scan = NewSwigScanner();
scan = NewScanner();
}
void Preprocessor_expr_delete(void) {
SwigScanner_clear(scan);
DelSwigScanner(scan);
Scanner_clear(scan);
DelScanner(scan);
}
@ -235,11 +235,11 @@ void Preprocessor_expr_delete(void) {
* Tokenizer
* ----------------------------------------------------------------------------- */
static int expr_token(SwigScanner * s) {
static int expr_token(Scanner * s) {
int t;
while (1) {
t = SwigScanner_token(s);
if (!((t == SWIG_TOKEN_BACKSLASH) || (t == SWIG_TOKEN_ENDLINE)))
t = Scanner_token(s);
if (!((t == SWIG_TOKEN_BACKSLASH) || (t == SWIG_TOKEN_ENDLINE) || (t == SWIG_TOKEN_COMMENT)))
break;
}
return t;
@ -262,8 +262,8 @@ int Preprocessor_expr(DOH *s, int *error) {
Seek(s, 0, SEEK_SET);
/* Printf(stdout,"evaluating : '%s'\n", s); */
*error = 0;
SwigScanner_clear(scan);
SwigScanner_push(scan, s);
Scanner_clear(scan);
Scanner_push(scan, s);
/* Put initial state onto the stack */
stack[sp].op = EXPR_TOP;
@ -282,10 +282,10 @@ int Preprocessor_expr(DOH *s, int *error) {
}
if ((token == SWIG_TOKEN_INT) || (token == SWIG_TOKEN_UINT) || (token == SWIG_TOKEN_LONG) || (token == SWIG_TOKEN_ULONG)) {
/* A number. Reduce EXPR_TOP to an EXPR_VALUE */
char *c = Char(SwigScanner_text(scan));
char *c = Char(Scanner_text(scan));
stack[sp].value = (long) strtol(c, 0, 0);
stack[sp].svalue = 0;
/* stack[sp].value = (long) atol(Char(SwigScanner_text(scan))); */
/* stack[sp].value = (long) atol(Char(Scanner_text(scan))); */
stack[sp].op = EXPR_VALUE;
} else if (token == SWIG_TOKEN_PLUS) {
} else if ((token == SWIG_TOKEN_MINUS) || (token == SWIG_TOKEN_LNOT) || (token == SWIG_TOKEN_NOT)) {
@ -302,7 +302,7 @@ int Preprocessor_expr(DOH *s, int *error) {
stack[sp].svalue = 0;
} else if (token == SWIG_TOKEN_ENDLINE) {
} else if ((token == SWIG_TOKEN_STRING)) {
stack[sp].svalue = NewString(SwigScanner_text(scan));
stack[sp].svalue = NewString(Scanner_text(scan));
stack[sp].op = EXPR_VALUE;
} else if ((token == SWIG_TOKEN_ID)) {
stack[sp].value = 0;

451
Source/Swig/scanner.c
View file

@ -15,28 +15,30 @@ char cvsroot_scanner_c[] = "$Id$";
#include "swig.h"
#include <ctype.h>
struct SwigScanner {
struct Scanner {
String *text; /* Current token value */
List *scanobjs; /* Objects being scanned */
List *scanobjs; /* Objects being scanned */
String *str; /* Current object being scanned */
char *idstart; /* Optional identifier start characters */
int nexttoken; /* Next token to be returned */
int start_line; /* Starting line of certain declarations */
int string_start;
int line;
int yylen; /* Length of text pushed into text */
char *idstart; /* Optional identifier start characters */
int nexttoken; /* Next token to be returned */
int start_line; /* Starting line of certain declarations */
int string_start;
int line;
int yylen; /* Length of text pushed into text */
String *file;
String *error; /* Last error message (if any) */
int error_line; /* Error line number */
};
/* -----------------------------------------------------------------------------
* NewSwigScanner()
* NewScanner()
*
* Create a new scanner object
* ----------------------------------------------------------------------------- */
SwigScanner *NewSwigScanner() {
SwigScanner *s;
s = (SwigScanner *) malloc(sizeof(SwigScanner));
Scanner *NewScanner() {
Scanner *s;
s = (Scanner *) malloc(sizeof(Scanner));
s->line = 1;
s->file = 0;
s->nexttoken = -1;
@ -47,34 +49,39 @@ SwigScanner *NewSwigScanner() {
s->scanobjs = NewList();
s->text = NewStringEmpty();
s->str = 0;
s->error = 0;
return s;
}
/* -----------------------------------------------------------------------------
* DelSwigScanner()
* DelScanner()
*
* Delete a scanner object.
* ----------------------------------------------------------------------------- */
void DelSwigScanner(SwigScanner * s) {
void DelScanner(Scanner * s) {
assert(s);
Delete(s->scanobjs);
Delete(s->text);
Delete(s->file);
Delete(s->error);
Delete(s->str);
free(s);
}
/* -----------------------------------------------------------------------------
* SwigScanner_clear()
* Scanner_clear()
*
* Clear the contents of a scanner object.
* ----------------------------------------------------------------------------- */
void SwigScanner_clear(SwigScanner * s) {
void Scanner_clear(Scanner * s) {
assert(s);
Delete(s->str);
Clear(s->text);
Clear(s->scanobjs);
Delete(s->error);
s->error = 0;
s->line = 1;
s->nexttoken = -1;
s->start_line = 0;
@ -83,13 +90,13 @@ void SwigScanner_clear(SwigScanner * s) {
}
/* -----------------------------------------------------------------------------
* SwigScanner_push()
* Scanner_push()
*
* Push some new text into the scanner. The scanner will start parsing this text
* immediately before returning to the old text.
* ----------------------------------------------------------------------------- */
void SwigScanner_push(SwigScanner * s, String *txt) {
void Scanner_push(Scanner * s, String *txt) {
assert(s && txt);
Push(s->scanobjs, txt);
if (s->str)
@ -100,55 +107,57 @@ void SwigScanner_push(SwigScanner * s, String *txt) {
}
/* -----------------------------------------------------------------------------
* SwigScanner_pushtoken()
* Scanner_pushtoken()
*
* Push a token into the scanner. This token will be returned on the next
* call to SwigScanner_token().
* call to Scanner_token().
* ----------------------------------------------------------------------------- */
void SwigScanner_pushtoken(SwigScanner * s, int nt) {
void Scanner_pushtoken(Scanner * s, int nt, String_or_char *val) {
assert(s);
assert((nt >= 0) && (nt < SWIG_MAXTOKENS));
s->nexttoken = nt;
Clear(s->text);
Append(s->text,val);
}
/* -----------------------------------------------------------------------------
* SwigScanner_set_location()
* Scanner_set_location()
*
* Set the file and line number location of the scanner.
* ----------------------------------------------------------------------------- */
void SwigScanner_set_location(SwigScanner * s, String *file, int line) {
void Scanner_set_location(Scanner * s, String *file, int line) {
Setline(s->str, line);
Setfile(s->str, file);
}
/* -----------------------------------------------------------------------------
* SwigScanner_get_file()
* Scanner_get_file()
*
* Get the current file.
* ----------------------------------------------------------------------------- */
String *SwigScanner_get_file(SwigScanner * s) {
String *Scanner_get_file(Scanner * s) {
return Getfile(s->str);
}
/* -----------------------------------------------------------------------------
* SwigScanner_get_line()
* Scanner_get_line()
*
* Get the current line number
* ----------------------------------------------------------------------------- */
int SwigScanner_get_line(SwigScanner * s) {
int Scanner_get_line(Scanner * s) {
return Getline(s->str);
}
/* -----------------------------------------------------------------------------
* SwigScanner_idstart()
* Scanner_idstart()
*
* Change the set of additional characters that can be used to start an identifier.
* ----------------------------------------------------------------------------- */
void SwigScanner_idstart(SwigScanner * s, char *id) {
void Scanner_idstart(Scanner * s, char *id) {
s->idstart = Swig_copy_string(id);
}
@ -157,7 +166,7 @@ void SwigScanner_idstart(SwigScanner * s, char *id) {
*
* Returns the next character from the scanner or 0 if end of the string.
* ----------------------------------------------------------------------------- */
static char nextchar(SwigScanner * s) {
static char nextchar(Scanner * s) {
char c[2] = { 0, 0 };
int nc;
if (!s->str)
@ -182,12 +191,40 @@ static char nextchar(SwigScanner * s) {
return c[0];
}
/* -----------------------------------------------------------------------------
* set_error()
*
* Sets error information on the scanner.
* ----------------------------------------------------------------------------- */
static void set_error(Scanner *s, int line, String_or_char *msg) {
s->error_line = line;
s->error = NewString(msg);
}
/* -----------------------------------------------------------------------------
* Scanner_errmsg()
* Scanner_errline()
*
* Returns error information (if any)
* ----------------------------------------------------------------------------- */
String *
Scanner_errmsg(Scanner *s) {
return s->error;
}
int
Scanner_errline(Scanner *s) {
return s->error_line;
}
/* -----------------------------------------------------------------------------
* retract()
*
* Retract n characters
* ----------------------------------------------------------------------------- */
static void retract(SwigScanner * s, int n) {
static void retract(Scanner * s, int n) {
int i, l;
char *str;
@ -198,21 +235,146 @@ static void retract(SwigScanner * s, int n) {
if (str[l - 1] == '\n') {
s->line--;
}
/* // Ungetc(str[l-1],s->str); */
Seek(s->str, -1, SEEK_CUR);
Delitem(s->text, DOH_END);
}
}
/* -----------------------------------------------------------------------------
* get_escape()
*
* Get escape sequence. Called when a backslash is found in a string
* ----------------------------------------------------------------------------- */
static void get_escape(Scanner *s) {
int result = 0;
int state = 0;
int c;
while (1) {
c = nextchar(s);
if (c == 0)
break;
switch (state) {
case 0:
if (c == 'n') {
Delitem(s->text, DOH_END);
Append(s->text,"\n");
return;
}
if (c == 'r') {
Delitem(s->text, DOH_END);
Append(s->text,"\r");
return;
}
if (c == 't') {
Delitem(s->text, DOH_END);
Append(s->text,"\t");
return;
}
if (c == 'a') {
Delitem(s->text, DOH_END);
Append(s->text,"\a");
return;
}
if (c == 'b') {
Delitem(s->text, DOH_END);
Append(s->text,"\b");
return;
}
if (c == 'f') {
Delitem(s->text, DOH_END);
Append(s->text,"\f");
return;
}
if (c == '\\') {
Delitem(s->text, DOH_END);
Append(s->text,"\\");
return;
}
if (c == 'v') {
Delitem(s->text, DOH_END);
Append(s->text,"\v");
return;
}
if (c == 'e') {
Delitem(s->text, DOH_END);
Append(s->text,"\033");
return;
}
if (c == '\'') {
Delitem(s->text, DOH_END);
Append(s->text,"\'");
return;
}
if (c == '\"') {
Delitem(s->text, DOH_END);
Append(s->text,"\"");
return;
}
if (c == '\n') {
Delitem(s->text, DOH_END);
return;
}
if (isdigit(c)) {
state = 10;
result = (c - '0');
Delitem(s->text, DOH_END);
} else if (c == 'x') {
state = 20;
Delitem(s->text, DOH_END);
} else {
char tmp[3];
tmp[0] = '\\';
tmp[1] = c;
tmp[2] = 0;
Delitem(s->text, DOH_END);
Append(s->text, tmp);
return;
}
break;
case 10:
if (!isdigit(c)) {
char tmp[2];
retract(s,1);
tmp[0] = (char) result;
tmp[1] = 0;
Append(s->text, tmp);
return;
}
result = (result << 3) + (c - '0');
Delitem(s->text, DOH_END);
break;
case 20:
if (!isxdigit(c)) {
char tmp[2];
retract(s,1);
tmp[0] = (char) result;
tmp[1] = 0;
Append(s->text,tmp);
return;
}
if (isdigit(c))
result = (result << 4) + (c - '0');
else
result = (result << 4) + (10 + tolower(c) - 'a');
Delitem(s->text, DOH_END);
break;
}
}
return;
}
/* -----------------------------------------------------------------------------
* look()
*
* Return the raw value of the next token.
* ----------------------------------------------------------------------------- */
static int look(SwigScanner * s) {
static int look(Scanner * s) {
int state;
int c = 0;
int comment_start = 0;
state = 0;
Clear(s->text);
@ -259,7 +421,7 @@ static int look(SwigScanner * s) {
else if (c == ',')
return SWIG_TOKEN_COMMA;
else if (c == '*')
return SWIG_TOKEN_STAR;
state = 220;
else if (c == '}')
return SWIG_TOKEN_RBRACE;
else if (c == '{')
@ -267,15 +429,15 @@ static int look(SwigScanner * s) {
else if (c == '=')
state = 33;
else if (c == '+')
return SWIG_TOKEN_PLUS;
state = 200;
else if (c == '-')
return SWIG_TOKEN_MINUS;
state = 210;
else if (c == '&')
state = 31;
else if (c == '|')
state = 32;
else if (c == '^')
return SWIG_TOKEN_XOR;
state = 230;
else if (c == '<')
state = 60;
else if (c == '>')
@ -296,11 +458,15 @@ static int look(SwigScanner * s) {
return SWIG_TOKEN_DOLLAR;
else if (c == '#')
return SWIG_TOKEN_POUND;
else if (c == '?')
return SWIG_TOKEN_QUESTION;
/* Look for multi-character sequences */
else if (c == '/')
else if (c == '/') {
state = 1; /* Comment (maybe) */
comment_start = s->line;
}
else if (c == '\"') {
state = 2; /* Possibly a string */
s->string_start = s->line;
@ -334,14 +500,15 @@ static int look(SwigScanner * s) {
Clear(s->text);
Setline(s->text, Getline(s->str));
Setfile(s->text, Getfile(s->str));
Append(s->text, " ");
Append(s->text, "//");
} else if (c == '*') {
state = 11; /* C style comment */
Clear(s->text);
Setline(s->text, Getline(s->str));
Setfile(s->text, Getfile(s->str));
Append(s->text, " ");
Append(s->text, "/*");
} else if (c == '=') {
return SWIG_TOKEN_DIVEQUAL;
} else {
retract(s, 1);
return SWIG_TOKEN_SLASH;
@ -349,19 +516,20 @@ static int look(SwigScanner * s) {
break;
case 10: /* C++ style comment */
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated comment",comment_start); */
return 0;
set_error(s,comment_start,"Unterminated comment");
return SWIG_TOKEN_ERROR;
}
if (c == '\n') {
return SWIG_TOKEN_ENDLINE;
retract(s,1);
return SWIG_TOKEN_COMMENT;
} else {
state = 10;
}
break;
case 11: /* C style comment block */
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated comment",comment_start); */
return 0;
set_error(s,comment_start,"Unterminated comment");
return SWIG_TOKEN_ERROR;
}
if (c == '*') {
state = 12;
@ -371,14 +539,13 @@ static int look(SwigScanner * s) {
break;
case 12: /* Still in C style comment */
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated comment",comment_start); */
return 0;
set_error(s,comment_start,"Unterminated comment");
return SWIG_TOKEN_ERROR;
}
if (c == '*') {
state = 12;
} else if (c == '/') {
Clear(s->text);
state = 0;
return SWIG_TOKEN_COMMENT;
} else {
state = 11;
}
@ -386,23 +553,16 @@ static int look(SwigScanner * s) {
case 2: /* Processing a string */
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated string", string_start); */
return 0;
set_error(s,s->string_start, "Unterminated string");
return SWIG_TOKEN_ERROR;
}
if (c == '\"') {
return SWIG_TOKEN_STRING;
} else if (c == '\\') {
state = 21; /* Possibly an escape sequence. */
break;
get_escape(s);
} else
state = 2;
break;
case 21: /* An escape sequence. get next character, then go
back to processing strings */
if ((c = nextchar(s)) == 0)
return 0;
state = 2;
break;
case 3: /* Maybe a not equals */
if ((c = nextchar(s)) == 0)
@ -415,11 +575,13 @@ static int look(SwigScanner * s) {
}
break;
case 31: /* AND or Logical AND */
case 31: /* AND or Logical AND or ANDEQUAL */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_AND;
else if (c == '&')
return SWIG_TOKEN_LAND;
else if (c == '=')
return SWIG_TOKEN_ANDEQUAL;
else {
retract(s, 1);
return SWIG_TOKEN_AND;
@ -431,6 +593,8 @@ static int look(SwigScanner * s) {
return SWIG_TOKEN_OR;
else if (c == '|')
return SWIG_TOKEN_LOR;
else if (c == '=')
return SWIG_TOKEN_OREQUAL;
else {
retract(s, 1);
return SWIG_TOKEN_OR;
@ -457,9 +621,11 @@ static int look(SwigScanner * s) {
Setline(s->text, Getline(s->str));
Setfile(s->text, Getfile(s->str));
s->start_line = s->line;
} else if (strchr(s->idstart, '%') && ((isalpha(c)) || (c == '_')))
} else if (strchr(s->idstart, '%') && ((isalpha(c)) || (c == '_'))) {
state = 7;
else {
} else if (c == '=') {
return SWIG_TOKEN_MODEQUAL;
} else {
retract(s, 1);
return SWIG_TOKEN_PERCENT;
}
@ -467,15 +633,15 @@ static int look(SwigScanner * s) {
case 40: /* Process an include block */
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated code block.", start_line); */
return 0;
set_error(s,s->start_line,"Unterminated code block");
return SWIG_TOKEN_ERROR;
}
if (c == '%')
state = 41;
break;
case 41: /* Still processing include block */
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated code block.", start_line); */
set_error(s,s->start_line,"Unterminated code block");
return 0;
}
if (c == '}') {
@ -492,18 +658,29 @@ static int look(SwigScanner * s) {
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_COLON;
if (c == ':')
return SWIG_TOKEN_DCOLON;
state = 50;
else {
retract(s, 1);
return SWIG_TOKEN_COLON;
}
break;
case 50: /* DCOLON, DCOLONSTAR */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_DCOLON;
else if (c == '*')
return SWIG_TOKEN_DCOLONSTAR;
else {
retract(s, 1);
return SWIG_TOKEN_DCOLON;
}
break;
case 60: /* shift operators */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_LESSTHAN;
if (c == '<')
return SWIG_TOKEN_LSHIFT;
state = 240;
else if (c == '=')
return SWIG_TOKEN_LTEQUAL;
else {
@ -515,7 +692,7 @@ static int look(SwigScanner * s) {
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_GREATERTHAN;
if (c == '>')
return SWIG_TOKEN_RSHIFT;
state = 250;
else if (c == '=')
return SWIG_TOKEN_GTEQUAL;
else {
@ -703,21 +880,14 @@ static int look(SwigScanner * s) {
/* A character constant */
case 9:
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated character constant", string_start); */
return 0;
set_error(s,s->string_start,"Unterminated character constant");
return SWIG_TOKEN_ERROR;
}
if (c == '\'') {
return (SWIG_TOKEN_CHAR);
} else if (c == '\\')
state = 91;
break;
case 91:
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated character constant", string_start); */
return 0;
} else if (c == '\\') {
get_escape(s);
}
state = 9;
break;
/* A period or maybe a floating point number */
@ -733,13 +903,98 @@ static int look(SwigScanner * s) {
}
break;
case 200: /* PLUS, PLUSPLUS, PLUSEQUAL */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_PLUS;
else if (c == '+')
return SWIG_TOKEN_PLUSPLUS;
else if (c == '=')
return SWIG_TOKEN_PLUSEQUAL;
else {
retract(s, 1);
return SWIG_TOKEN_PLUS;
}
break;
case 210: /* MINUS, MINUSMINUS, MINUSEQUAL, ARROW */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_MINUS;
else if (c == '+')
return SWIG_TOKEN_MINUSMINUS;
else if (c == '=')
return SWIG_TOKEN_MINUSEQUAL;
else if (c == '>')
state = 211;
else {
retract(s, 1);
return SWIG_TOKEN_MINUS;
}
break;
case 211: /* ARROW, ARROWSTAR */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_ARROW;
else if (c == '*')
return SWIG_TOKEN_ARROWSTAR;
else {
retract(s, 1);
return SWIG_TOKEN_ARROW;
}
break;
case 220: /* STAR, TIMESEQUAL */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_STAR;
else if (c == '=')
return SWIG_TOKEN_TIMESEQUAL;
else {
retract(s, 1);
return SWIG_TOKEN_STAR;
}
break;
case 230: /* XOR, XOREQUAL */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_XOR;
else if (c == '=')
return SWIG_TOKEN_XOREQUAL;
else {
retract(s, 1);
return SWIG_TOKEN_XOR;
}
break;
case 240: /* LSHIFT, LSEQUAL */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_LSHIFT;
else if (c == '=')
return SWIG_TOKEN_LSEQUAL;
else {
retract(s, 1);
return SWIG_TOKEN_LSHIFT;
}
break;
case 250: /* RSHIFT, RSEQUAL */
if ((c = nextchar(s)) == 0)
return SWIG_TOKEN_RSHIFT;
else if (c == '=')
return SWIG_TOKEN_RSEQUAL;
else {
retract(s, 1);
return SWIG_TOKEN_RSHIFT;
}
break;
/* An illegal character */
/* Reverse string */
case 900:
if ((c = nextchar(s)) == 0) {
/* add_error(0,"Unterminated character constant", string_start); */
return 0;
set_error(s,s->string_start,"Unterminated character constant");
return SWIG_TOKEN_ERROR;
}
if (c == '`') {
return (SWIG_TOKEN_RSTRING);
@ -753,40 +1008,41 @@ static int look(SwigScanner * s) {
}
/* -----------------------------------------------------------------------------
* SwigScanner_token()
* Scanner_token()
*
* Real entry point to return the next token. Returns 0 if at end of input.
* ----------------------------------------------------------------------------- */
int SwigScanner_token(SwigScanner * s) {
int Scanner_token(Scanner * s) {
int t;
Clear(s->text);
Delete(s->error);
if (s->nexttoken >= 0) {
t = s->nexttoken;
s->nexttoken = -1;
return t;
}
Clear(s->text);
t = look(s);
return t;
}
/* -----------------------------------------------------------------------------
* SwigScanner_text()
* Scanner_text()
*
* Return the lexene associated with the last returned token.
* ----------------------------------------------------------------------------- */
String *SwigScanner_text(SwigScanner * s) {
String *Scanner_text(Scanner * s) {
return s->text;
}
/* -----------------------------------------------------------------------------
* SwigScanner_skip_line()
* Scanner_skip_line()
*
* Skips to the end of a line
* ----------------------------------------------------------------------------- */
void SwigScanner_skip_line(SwigScanner * s) {
void Scanner_skip_line(Scanner * s) {
char c;
int done = 0;
Clear(s->text);
@ -804,13 +1060,13 @@ void SwigScanner_skip_line(SwigScanner * s) {
}
/* -----------------------------------------------------------------------------
* SwigScanner_skip_balanced()
* Scanner_skip_balanced()
*
* Skips a piece of code enclosed in begin/end symbols such as '{...}' or
* (...). Ignores symbols inside comments or strings.
* ----------------------------------------------------------------------------- */
int SwigScanner_skip_balanced(SwigScanner * s, int startchar, int endchar) {
int Scanner_skip_balanced(Scanner * s, int startchar, int endchar) {
char c;
int num_levels = 1;
int l;
@ -890,3 +1146,16 @@ int SwigScanner_skip_balanced(SwigScanner * s, int startchar, int endchar) {
}
return 0;
}
/* -----------------------------------------------------------------------------
* Scanner_isoperator()
*
* Returns 0 or 1 depending on whether or not a token corresponds to a C/C++
* operator.
* ----------------------------------------------------------------------------- */
int
Scanner_isoperator(int tokval) {
if (tokval >= 100) return 1;
return 0;
}

70
Source/Swig/swig.h
View file

@ -103,75 +103,7 @@ extern "C" {
/* --- Scanner Interface --- */
typedef struct SwigScanner SwigScanner;
extern SwigScanner *NewSwigScanner();
extern void DelSwigScanner(SwigScanner *);
extern void SwigScanner_clear(SwigScanner *);
extern void SwigScanner_push(SwigScanner *, String *);
extern void SwigScanner_pushtoken(SwigScanner *, int);
extern int SwigScanner_token(SwigScanner *);
extern String *SwigScanner_text(SwigScanner *);
extern void SwigScanner_skip_line(SwigScanner *);
extern int SwigScanner_skip_balanced(SwigScanner *, int startchar, int endchar);
extern void SwigScanner_set_location(SwigScanner *, String *file, int line);
extern String *SwigScanner_get_file(SwigScanner *);
extern int SwigScanner_get_line(SwigScanner *);
extern void SwigScanner_idstart(SwigScanner *, char *idchar);
#define SWIG_MAXTOKENS 512
#define SWIG_TOKEN_LPAREN 1
#define SWIG_TOKEN_RPAREN 2
#define SWIG_TOKEN_SEMI 3
#define SWIG_TOKEN_COMMA 4
#define SWIG_TOKEN_STAR 5
#define SWIG_TOKEN_LBRACE 6
#define SWIG_TOKEN_RBRACE 7
#define SWIG_TOKEN_EQUAL 8
#define SWIG_TOKEN_EQUALTO 9
#define SWIG_TOKEN_NOTEQUAL 10
#define SWIG_TOKEN_PLUS 11
#define SWIG_TOKEN_MINUS 12
#define SWIG_TOKEN_AND 13
#define SWIG_TOKEN_LAND 14
#define SWIG_TOKEN_OR 15
#define SWIG_TOKEN_LOR 16
#define SWIG_TOKEN_XOR 17
#define SWIG_TOKEN_LESSTHAN 18
#define SWIG_TOKEN_GREATERTHAN 19
#define SWIG_TOKEN_LTEQUAL 20
#define SWIG_TOKEN_GTEQUAL 21
#define SWIG_TOKEN_NOT 22
#define SWIG_TOKEN_LNOT 23
#define SWIG_TOKEN_LBRACKET 24
#define SWIG_TOKEN_RBRACKET 25
#define SWIG_TOKEN_SLASH 26
#define SWIG_TOKEN_BACKSLASH 27
#define SWIG_TOKEN_ENDLINE 28
#define SWIG_TOKEN_STRING 29
#define SWIG_TOKEN_POUND 30
#define SWIG_TOKEN_PERCENT 31
#define SWIG_TOKEN_COLON 32
#define SWIG_TOKEN_DCOLON 33
#define SWIG_TOKEN_LSHIFT 34
#define SWIG_TOKEN_RSHIFT 35
#define SWIG_TOKEN_ID 36
#define SWIG_TOKEN_FLOAT 37
#define SWIG_TOKEN_DOUBLE 38
#define SWIG_TOKEN_INT 39
#define SWIG_TOKEN_UINT 40
#define SWIG_TOKEN_LONG 41
#define SWIG_TOKEN_ULONG 42
#define SWIG_TOKEN_CHAR 43
#define SWIG_TOKEN_PERIOD 44
#define SWIG_TOKEN_AT 45
#define SWIG_TOKEN_DOLLAR 46
#define SWIG_TOKEN_CODEBLOCK 47
#define SWIG_TOKEN_RSTRING 48
#define SWIG_TOKEN_LONGLONG 49
#define SWIG_TOKEN_ULONGLONG 50
#define SWIG_TOKEN_ILLEGAL 98
#define SWIG_TOKEN_LAST 99
#include "swigscan.h"
/* --- Functions for manipulating the string-based type encoding --- */

106
Source/Swig/swigscan.h Normal file
View file

@ -0,0 +1,106 @@
/* -----------------------------------------------------------------------------
* See the LICENSE file for information on copyright, usage and redistribution
* of SWIG, and the README file for authors - http://www.swig.org/release.html.
*
* swigscan.h
*
* C/C++ scanner.
* ----------------------------------------------------------------------------- */
/* $Id: swig.h 9633 2007-01-10 23:43:07Z beazley $ */
typedef struct Scanner Scanner;
extern Scanner *NewScanner();
extern void DelScanner(Scanner *);
extern void Scanner_clear(Scanner *);
extern void Scanner_push(Scanner *, String *);
extern void Scanner_pushtoken(Scanner *, int, String_or_char *value);
extern int Scanner_token(Scanner *);
extern String *Scanner_text(Scanner *);
extern void Scanner_skip_line(Scanner *);
extern int Scanner_skip_balanced(Scanner *, int startchar, int endchar);
extern void Scanner_set_location(Scanner *, String *file, int line);
extern String *Scanner_get_file(Scanner *);
extern int Scanner_get_line(Scanner *);
extern void Scanner_idstart(Scanner *, char *idchar);
extern String *Scanner_errmsg(Scanner *);
extern int Scanner_errline(Scanner *);
extern int Scanner_isoperator(int tokval);
/* Note: Tokens in range 100+ are for C/C++ operators */
#define SWIG_MAXTOKENS 200
#define SWIG_TOKEN_LPAREN 1 /* ( */
#define SWIG_TOKEN_RPAREN 2 /* ) */
#define SWIG_TOKEN_SEMI 3 /* ; */
#define SWIG_TOKEN_LBRACE 4 /* { */
#define SWIG_TOKEN_RBRACE 5 /* } */
#define SWIG_TOKEN_LBRACKET 6 /* [ */
#define SWIG_TOKEN_RBRACKET 7 /* ] */
#define SWIG_TOKEN_BACKSLASH 8 /* \ */
#define SWIG_TOKEN_ENDLINE 9 /* \n */
#define SWIG_TOKEN_STRING 10 /* "str" */
#define SWIG_TOKEN_POUND 11 /* # */
#define SWIG_TOKEN_COLON 12 /* : */
#define SWIG_TOKEN_DCOLON 13 /* :: */
#define SWIG_TOKEN_DCOLONSTAR 14 /* ::* */
#define SWIG_TOKEN_ID 15 /* identifer */
#define SWIG_TOKEN_FLOAT 16 /* 3.1415F */
#define SWIG_TOKEN_DOUBLE 17 /* 3.1415 */
#define SWIG_TOKEN_INT 18 /* 314 */
#define SWIG_TOKEN_UINT 19 /* 314U */
#define SWIG_TOKEN_LONG 20 /* 314L */
#define SWIG_TOKEN_ULONG 21 /* 314UL */
#define SWIG_TOKEN_CHAR 22 /* 'charconst' */
#define SWIG_TOKEN_PERIOD 23 /* . */
#define SWIG_TOKEN_AT 24 /* @ */
#define SWIG_TOKEN_DOLLAR 25 /* $ */
#define SWIG_TOKEN_CODEBLOCK 26 /* %{ ... %} ... */
#define SWIG_TOKEN_RSTRING 27 /* `charconst` */
#define SWIG_TOKEN_LONGLONG 28 /* 314LL */
#define SWIG_TOKEN_ULONGLONG 29 /* 314ULL */
#define SWIG_TOKEN_QUESTION 30 /* ? */
#define SWIG_TOKEN_COMMENT 31 /* C or C++ comment */
#define SWIG_TOKEN_ILLEGAL 99
#define SWIG_TOKEN_ERROR -1
#define SWIG_TOKEN_COMMA 101 /* , */
#define SWIG_TOKEN_STAR 102 /* * */
#define SWIG_TOKEN_TIMES 102 /* * */
#define SWIG_TOKEN_EQUAL 103 /* = */
#define SWIG_TOKEN_EQUALTO 104 /* == */
#define SWIG_TOKEN_NOTEQUAL 105 /* != */
#define SWIG_TOKEN_PLUS 106 /* + */
#define SWIG_TOKEN_MINUS 107 /* - */
#define SWIG_TOKEN_AND 108 /* & */
#define SWIG_TOKEN_LAND 109 /* && */
#define SWIG_TOKEN_OR 110 /* | */
#define SWIG_TOKEN_LOR 111 /* || */
#define SWIG_TOKEN_XOR 112 /* ^ */
#define SWIG_TOKEN_LESSTHAN 113 /* < */
#define SWIG_TOKEN_GREATERTHAN 114 /* > */
#define SWIG_TOKEN_LTEQUAL 115 /* <= */
#define SWIG_TOKEN_GTEQUAL 116 /* >= */
#define SWIG_TOKEN_NOT 117 /* ~ */
#define SWIG_TOKEN_LNOT 118 /* ! */
#define SWIG_TOKEN_SLASH 119 /* / */
#define SWIG_TOKEN_DIVIDE 119 /* / */
#define SWIG_TOKEN_PERCENT 120 /* % */
#define SWIG_TOKEN_MODULO 120 /* % */
#define SWIG_TOKEN_LSHIFT 121 /* << */
#define SWIG_TOKEN_RSHIFT 122 /* >> */
#define SWIG_TOKEN_PLUSPLUS 123 /* ++ */
#define SWIG_TOKEN_MINUSMINUS 124 /* -- */
#define SWIG_TOKEN_PLUSEQUAL 125 /* += */
#define SWIG_TOKEN_MINUSEQUAL 126 /* -= */
#define SWIG_TOKEN_TIMESEQUAL 127 /* *= */
#define SWIG_TOKEN_DIVEQUAL 128 /* /= */
#define SWIG_TOKEN_ANDEQUAL 129 /* &= */
#define SWIG_TOKEN_OREQUAL 130 /* |= */
#define SWIG_TOKEN_XOREQUAL 131 /* ^= */
#define SWIG_TOKEN_LSEQUAL 132 /* <<= */
#define SWIG_TOKEN_RSEQUAL 133 /* >>= */
#define SWIG_TOKEN_MODEQUAL 134 /* %= */
#define SWIG_TOKEN_ARROW 135 /* -> */
#define SWIG_TOKEN_ARROWSTAR 136 /* ->* */