added a new macro tutorial (#9588)
This commit is contained in:
Arne Döring
Andreas Rumpf
parent
6ff596d4f8
commit
30480605c1
3 changed files with 357 additions and 394 deletions
347
doc/tut3.rst
Normal file
347
doc/tut3.rst
Normal file
|
|
@ -0,0 +1,347 @@
|
|||
=======================
|
||||
Nim Tutorial (Part III)
|
||||
=======================
|
||||
|
||||
:Author: Arne Döring
|
||||
:Version: |nimversion|
|
||||
|
||||
.. contents::
|
||||
|
||||
|
||||
Introduction
|
||||
============
|
||||
|
||||
"With Great Power Comes Great Responsibility." -- Spider Man's Uncle
|
||||
|
||||
This document is a tutorial for the macros of the *Nim* programming
|
||||
language. A macro enables to formulate and distribute Nim syntax tree
|
||||
transformations as a normal library. The arguments of a macro are
|
||||
passed a syntax tree, and its job is it to create another syntax tree
|
||||
for the compiler. The way this works in Nim is, whenever the compiler
|
||||
encounters a call expression to a macro, The compiler evaluates the
|
||||
macro at compile time with the syntax tree from the invocation, and
|
||||
then it replaces the call to the macro by the result of the macro.
|
||||
|
||||
Examples of things that can be implemented in macros:
|
||||
|
||||
* An assert macro that prints both sides of a comparison operator, if
|
||||
the assertion fails. ``myAssert(a == b)`` that converts to
|
||||
``if a != b: quit($a " != " $b)``
|
||||
|
||||
* A debug macro that prints the value and the name of the symbol.
|
||||
``myDebugEcho(a)`` that converts to ``echo "a: ", a``
|
||||
|
||||
* Symbolic differentiation of an expression.
|
||||
``diff(a*pow(x,3) + b*pow(x,2) + c*x + d, x)`` that converts to
|
||||
``3*a*pow(x,2) + 2*a*x + c``
|
||||
|
||||
Macro Arguments
|
||||
---------------
|
||||
|
||||
The types of macro arguments have two faces. One face is used for
|
||||
the overload resolution, and the other face is used for the semantic
|
||||
checking of the macro implementation. For example
|
||||
``macro foo(arg: int)`` will be called in an expression ``foo(x)``, if
|
||||
``x`` is of type int, but for the semantic checking of the macro
|
||||
implementation, ``arg`` has the type ``NimNode``, not ``int`` as you might
|
||||
expect, because ``x`` will be passed as a symbol (``NimNode``), not as
|
||||
an integer. There are two ways to pass arguments to a macro, either typed or
|
||||
untyped.
|
||||
|
||||
Untyped Arguments
|
||||
-----------------
|
||||
|
||||
Untyped macro arguments are passed to the macro before they are
|
||||
semantically checked. This means the syntax tree that is passed down
|
||||
to the macro does not need to make sense for Nim yet, the only
|
||||
limitation for the syntax in an untyped macro argument is, it needs to
|
||||
be parseable by the Nim parser. The semantic of this syntax is
|
||||
entirely up to the macro implementation. In this case the macro is
|
||||
responsible to implement its own semantic checking on the
|
||||
argument. The upside for untyped arguments is, the syntax tree is
|
||||
quite predictable and less complex than for typed arguments. Untyped
|
||||
arguments have the type ``untyped`` in arguments list.
|
||||
|
||||
Typed Arguments
|
||||
---------------
|
||||
|
||||
For typed arguments, the semantic checker runs on the argument and
|
||||
does transformations on it, before it is passed to the macro. Here
|
||||
identifier nodes will already be resolved as symbols, implicit type
|
||||
conversions are visible in the tree as calls, templates will be
|
||||
expanded and probably most importantly, nodes have type information.
|
||||
Typed arguments can have the type ``typed`` in the arguments list.
|
||||
But all other types, such as ``int``, ``float`` or ``MyObjectType``
|
||||
are typed arguments as well, and they will be passed to the macro as a
|
||||
syntax tree.
|
||||
|
||||
Static Arguments
|
||||
----------------
|
||||
|
||||
Static arguments are a way to pass values as values and not as syntax
|
||||
tree nodes to a macro. For example for ``macro foo(arg: static[int])``
|
||||
in the expression ``foo(x)``, ``x`` needs to be an integer constant,
|
||||
but in the macro body ``arg`` is just like a normal parameter of type
|
||||
``int``.
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
import macros
|
||||
|
||||
macro myMacro(arg: static[int]): untyped =
|
||||
echo arg # just an int (7), not ``NimNode``
|
||||
|
||||
myMacro(1 + 2 * 3)
|
||||
|
||||
|
||||
Code blocks as arguments
|
||||
------------------------
|
||||
|
||||
In Nim it is possible to pass the last argument of a call expression in a
|
||||
separate code block with indentation. For example the following code
|
||||
example is a valid (but not a recommended) way to call ``echo``:
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
echo "Hello ":
|
||||
let a = "Wor"
|
||||
let b = "ld!"
|
||||
a & b
|
||||
|
||||
For macros this way of calling is useful for example to implement an
|
||||
embedded domain specific language. Syntax trees of arbitrary
|
||||
complexity can be passed to macros with this notation.
|
||||
|
||||
The Syntax Tree
|
||||
---------------
|
||||
|
||||
In order to build a Nim syntax tree one needs to know how Nim source
|
||||
code is represented as a syntax tree, and how such a tree needs to
|
||||
look like so that the Nim compiler will understand it. The nodes of the
|
||||
Nim syntax tree are documented in the `macros <macros.html>`_ module.
|
||||
But a probably more interesting and interactive way to explore the Nim
|
||||
syntax tree is with ``macros.treeRepr``, it converts a syntax tree
|
||||
into a multi line string for printing on the console. It can be used
|
||||
to explore how the argument expressions are represented in tree form
|
||||
and for debug printing of generated syntax tree. ``dumpTree`` is a
|
||||
predefined macro that just prints its argument in tree representation,
|
||||
but does nothing else. Here is an example of such a tree representation:
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
dumpTree:
|
||||
var mt: MyType = MyType(a:123.456, b:"abcdef")
|
||||
|
||||
# output:
|
||||
# StmtList
|
||||
# VarSection
|
||||
# IdentDefs
|
||||
# Ident "mt"
|
||||
# Ident "MyType"
|
||||
# ObjConstr
|
||||
# Ident "MyType"
|
||||
# ExprColonExpr
|
||||
# Ident "a"
|
||||
# FloatLit 123.456
|
||||
# ExprColonExpr
|
||||
# Ident "b"
|
||||
# StrLit "abcdef"
|
||||
|
||||
Custom sematic checking
|
||||
-----------------------
|
||||
|
||||
The first thing that a macro should do with its arguments is to check
|
||||
if the argument is in the correct form. Not every type of wrong input
|
||||
needs to be caught here, but anything that could cause a crash during
|
||||
macro evaluation should be caught and create a nice error message.
|
||||
``macros.expectKind`` and ``macros.expectLen`` are a good start. If
|
||||
the checks need to be more complex, arbitrary error messages can
|
||||
be created with the ``macros.error`` proc.
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
macro myAssert(arg: untyped): untyped =
|
||||
arg.expectKind nnkInfix
|
||||
|
||||
Generating Code
|
||||
---------------
|
||||
|
||||
There are two ways to generate the code. Either by creating the syntax
|
||||
tree with expressions that contain a lot of calls to ``newTree`` and
|
||||
``newLit``, or with ``quote do:`` expressions. The first option offers
|
||||
the best low level control for the syntax tree generation, but the
|
||||
second option is much less verbose. If you choose to create the syntax
|
||||
tree with calls to ``newTree`` and ``newLit`` the macro
|
||||
``marcos.dumpAstGen`` can help you with the verbosity. ``quote do:``
|
||||
allows you to write the code that you want to generate literally,
|
||||
backticks are used to insert code from ``NimNode`` symbols into the
|
||||
generated expression. This means that you can't use backticks within
|
||||
``quote do:`` for anything else than injecting symbols. Make sure to
|
||||
inject only symbols of type ``NimNode`` into the generated syntax
|
||||
tree. You can use ``newLit`` to convert arbitrary values into
|
||||
expressions trees of type ``NimNode`` so that it is safe to inject
|
||||
them into the tree.
|
||||
|
||||
|
||||
.. code-block:: nim
|
||||
:test: "nim c $1"
|
||||
|
||||
import macros
|
||||
|
||||
type
|
||||
MyType = object
|
||||
a: float
|
||||
b: string
|
||||
|
||||
macro myMacro(arg: untyped): untyped =
|
||||
var mt: MyType = MyType(a:123.456, b:"abcdef")
|
||||
|
||||
# ...
|
||||
|
||||
let mtLit = newLit(mt)
|
||||
|
||||
result = quote do:
|
||||
echo `arg`
|
||||
echo `mtLit`
|
||||
|
||||
myMacro("Hallo")
|
||||
|
||||
The call to ``myMacro`` will generate the following code:
|
||||
|
||||
.. code-block:: nim
|
||||
echo "Hallo"
|
||||
echo MyType(a: 123.456'f64, b: "abcdef")
|
||||
|
||||
|
||||
Building your first macro
|
||||
-------------------------
|
||||
|
||||
To give a footstart to writing macros we will show now how to
|
||||
implement the ``myDebug`` macro mentioned earlier. The first thing to
|
||||
do is to build a simple example of the macro usage, and then just
|
||||
print the argument. This way it is possible to get an idea of a
|
||||
correct argument should be look like.
|
||||
|
||||
.. code-block:: nim
|
||||
:test: "nim c $1"
|
||||
|
||||
import macros
|
||||
|
||||
macro myAssert(arg: untyped): untyped =
|
||||
echo arg.treeRepr
|
||||
|
||||
let a = 1
|
||||
let b = 2
|
||||
|
||||
myAssert(a != b)
|
||||
|
||||
.. code-block::
|
||||
|
||||
Infix
|
||||
Ident "!="
|
||||
Ident "a"
|
||||
Ident "b"
|
||||
|
||||
|
||||
From the output it is possible to see that the information that the
|
||||
argument is an infix operator (node kind is "Infix"), as well as that the two
|
||||
operands are at index 1 and 2. With this information the actual
|
||||
macro can be written.
|
||||
|
||||
.. code-block:: nim
|
||||
:test: "nim c $1"
|
||||
|
||||
import macros
|
||||
|
||||
macro myAssert(arg: untyped): untyped =
|
||||
# all node kind identifiers are prefixed with "nnk"
|
||||
arg.expectKind nnkInfix
|
||||
arg.expectLen 3
|
||||
# operator as string literal
|
||||
let op = newLit(" " & arg[0].repr & " ")
|
||||
let lhs = arg[1]
|
||||
let rhs = arg[2]
|
||||
|
||||
result = quote do:
|
||||
if not `arg`:
|
||||
raise newException(AssertionError,$`lhs` & `op` & $`rhs`)
|
||||
|
||||
let a = 1
|
||||
let b = 2
|
||||
|
||||
myAssert(a != b)
|
||||
myAssert(a == b)
|
||||
|
||||
|
||||
This is the code that will be generated. To debug what the macro
|
||||
actually generated, the statement ``echo result.repr`` can be used, in
|
||||
the last line of the macro. It is also the statement that has been
|
||||
used to get this output.
|
||||
|
||||
.. code-block:: nim
|
||||
if not (a != b):
|
||||
raise newException(AssertionError, $a & " != " & $b)
|
||||
|
||||
With Power Comes Responsibility
|
||||
-------------------------------
|
||||
|
||||
Macros are very powerful. A good advice is to use them as little as
|
||||
possible, but as much as necessary. Macros can change the semantics of
|
||||
expressions, making the code incomprehensible for anybody who does not
|
||||
know exactly what the macro does with it. So whenever a macro is not
|
||||
necessary and the same logic can be implemented using templates or
|
||||
generics, it is probably better not to use a macro. And when a macro
|
||||
is used for something, the macro should better have a well written
|
||||
documentation. For all the people who claim to write only perfectly
|
||||
self-explanatory code: when it comes to macros, the implementation is
|
||||
not enough for documentation.
|
||||
|
||||
Limitations
|
||||
-----------
|
||||
|
||||
Since macros are evaluated in the compiler in the NimVM, macros share
|
||||
all the limitations of the NimVM. They have to be implemented in pure Nim
|
||||
code. Macros can start external processes on the shell, but they
|
||||
cannot call C functions except from those that are built in the
|
||||
compiler.
|
||||
|
||||
|
||||
More Examples
|
||||
=============
|
||||
|
||||
This tutorial can only cover the basics of the macro system. There are
|
||||
macros out there that could be an inspiration for you of what is
|
||||
possible with it.
|
||||
|
||||
|
||||
Strformat
|
||||
---------
|
||||
|
||||
In the Nim standard library, the ``strformat`` library provides a
|
||||
macro that parses a string literal at compile time. Parsing a string
|
||||
in a macro like here is generally not recommended. The parsed AST
|
||||
cannot have type information, and parsing implemented on the VM is
|
||||
generally not very fast. Working on AST nodes is almost always the
|
||||
recommended way. But still ``strformat`` is a good example for a
|
||||
practical use case for a macro that is slightly more complex that the
|
||||
``assert`` macro.
|
||||
|
||||
`Strformat <https://github.com/nim-lang/Nim/blob/5845716df8c96157a047c2bd6bcdd795a7a2b9b1/lib/pure/strformat.nim#L280>`_
|
||||
|
||||
Ast Pattern Matching
|
||||
--------------------
|
||||
|
||||
Ast Pattern Matching is a macro library to aid in writing complex
|
||||
macros. This can be seen as a good example of how to repurpose the
|
||||
Nim syntax tree with new semantics.
|
||||
|
||||
`Ast Pattern Matching <https://github.com/krux02/ast-pattern-matching>`_
|
||||
|
||||
OpenGL Sandbox
|
||||
--------------
|
||||
|
||||
This project has a working Nim to GLSL compiler written entirely in
|
||||
macros. It scans recursively though all used function symbols to
|
||||
compile them so that cross library functions can be executed on the GPU.
|
||||
|
||||
`OpenGL Sandbox <https://github.com/krux02/opengl-sandbox>`_
|
||||
Loading…
Add table
Add a link
Reference in a new issue