Additional doc fixes.
This commit is contained in:
parent
67ee2792bf
commit
c7cbebab7b
5 changed files with 176 additions and 150 deletions
|
|
@ -267,55 +267,46 @@ ignore the captured match:
|
|||
|
||||
.. code-block:: python
|
||||
|
||||
# Check if any of the regexes matched and yield
|
||||
# the appropriate result.
|
||||
if comment_match:
|
||||
comment = comment_match.group(0)
|
||||
string = string[len(comment):]
|
||||
|
||||
For numbers, we yield the captured match, converted to a float and
|
||||
tagged with the appropriate token type:
|
||||
|
||||
.. code-block:: python
|
||||
# Check if any of the regexes matched and yield
|
||||
# the appropriate result.
|
||||
if comment_match:
|
||||
comment = comment_match.group(0)
|
||||
string = string[len(comment):]
|
||||
|
||||
elif number_match:
|
||||
number = number_match.group(0)
|
||||
yield NumberToken(float(number))
|
||||
string = string[len(number):]
|
||||
# For numbers, we yield the captured match, converted to a float and
|
||||
# tagged with the appropriate token type:
|
||||
|
||||
The identifier case is a little more complex. We have to check for
|
||||
keywords to decide whether we have captured an identifier or a keyword:
|
||||
elif number_match:
|
||||
number = number_match.group(0)
|
||||
yield NumberToken(float(number))
|
||||
string = string[len(number):]
|
||||
|
||||
.. code-block:: python
|
||||
# The identifier case is a little more complex. We have to check for
|
||||
# keywords to decide whether we have captured an identifier or a keyword:
|
||||
|
||||
elif identifier_match:
|
||||
identifier = identifier_match.group(0)
|
||||
# Check if we matched a keyword.
|
||||
if identifier == 'def':
|
||||
yield DefToken()
|
||||
elif identifier == 'extern':
|
||||
yield ExternToken()
|
||||
else:
|
||||
yield IdentifierToken(identifier)
|
||||
string = string[len(identifier):]
|
||||
elif identifier_match:
|
||||
identifier = identifier_match.group(0)
|
||||
# Check if we matched a keyword.
|
||||
if identifier == 'def':
|
||||
yield DefToken()
|
||||
elif identifier == 'extern':
|
||||
yield ExternToken()
|
||||
else:
|
||||
yield IdentifierToken(identifier)
|
||||
string = string[len(identifier):]
|
||||
|
||||
|
||||
Finally, if we haven't recognized a comment, a number of an identifier,
|
||||
we yield the current character as an "unknown character" token. This is
|
||||
used, for example, for operators like ``+`` or ``*``:
|
||||
# Finally, if we haven't recognized a comment, a number of an identifier,
|
||||
# we yield the current character as an "unknown character" token. This is
|
||||
# used, for example, for operators like ``+`` or ``*``:
|
||||
|
||||
else: # Yield the unknown character.
|
||||
yield CharacterToken(string[0])
|
||||
string = string[1:]
|
||||
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
else: # Yield the unknown character.
|
||||
yield CharacterToken(string[0])
|
||||
string = string[1:]
|
||||
# Once we're done with the loop, we return a final end-of-file token:
|
||||
|
||||
|
||||
Once we're done with the loop, we return a final end-of-file token:
|
||||
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
yield EOFToken()
|
||||
yield EOFToken()
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue