Additional doc fixes.

This commit is contained in:
Maggie Mari 2012-08-20 17:11:26 -05:00
commit c7cbebab7b
5 changed files with 176 additions and 150 deletions

View file

@ -267,55 +267,46 @@ ignore the captured match:
.. code-block:: python
# Check if any of the regexes matched and yield
# the appropriate result.
if comment_match:
comment = comment_match.group(0)
string = string[len(comment):]
For numbers, we yield the captured match, converted to a float and
tagged with the appropriate token type:
.. code-block:: python
# Check if any of the regexes matched and yield
# the appropriate result.
if comment_match:
comment = comment_match.group(0)
string = string[len(comment):]
elif number_match:
number = number_match.group(0)
yield NumberToken(float(number))
string = string[len(number):]
# For numbers, we yield the captured match, converted to a float and
# tagged with the appropriate token type:
The identifier case is a little more complex. We have to check for
keywords to decide whether we have captured an identifier or a keyword:
elif number_match:
number = number_match.group(0)
yield NumberToken(float(number))
string = string[len(number):]
.. code-block:: python
# The identifier case is a little more complex. We have to check for
# keywords to decide whether we have captured an identifier or a keyword:
elif identifier_match:
identifier = identifier_match.group(0)
# Check if we matched a keyword.
if identifier == 'def':
yield DefToken()
elif identifier == 'extern':
yield ExternToken()
else:
yield IdentifierToken(identifier)
string = string[len(identifier):]
elif identifier_match:
identifier = identifier_match.group(0)
# Check if we matched a keyword.
if identifier == 'def':
yield DefToken()
elif identifier == 'extern':
yield ExternToken()
else:
yield IdentifierToken(identifier)
string = string[len(identifier):]
Finally, if we haven't recognized a comment, a number of an identifier,
we yield the current character as an "unknown character" token. This is
used, for example, for operators like ``+`` or ``*``:
# Finally, if we haven't recognized a comment, a number of an identifier,
# we yield the current character as an "unknown character" token. This is
# used, for example, for operators like ``+`` or ``*``:
else: # Yield the unknown character.
yield CharacterToken(string[0])
string = string[1:]
.. code-block:: python
else: # Yield the unknown character.
yield CharacterToken(string[0])
string = string[1:]
# Once we're done with the loop, we return a final end-of-file token:
Once we're done with the loop, we return a final end-of-file token:
.. code-block:: python
yield EOFToken()
yield EOFToken()