WIP: an API for VM replay global state support

doc/intern.txt

@@ -268,7 +268,52 @@ severe way. Plenty of different solutions have been proposed:
 
 Since we adopt the "replay the top level statements" idea, the natural
 solution to this problem is to emit pseudo top level statements that
-reflect the mutations done to the global variable.
+reflect the mutations done to the global variable. However, this is
+MUCH harder than it sounds, for example ``squeaknim`` uses this
+snippet:
+
+.. code-block:: nim
+  apicall.add(") module: '" & dllName & "'>\C" &
+              "\t^self externalCallFailed\C!\C\C")
+  stCode.add(st & "\C\t\"Generated by NimSqueak\"\C\t" & apicall)
+
+We can "replay" ``stCode.add`` only if the values of ``st``
+and ``apicall`` are known. And even then a hash table's ``add`` with its
+hashing mechanism is too hard to replay.
+
+In practice, things are worse still, consider ``someGlobal[i][j].add arg``.
+We only know the root is ``someGlobal`` but the concrete path to the data
+is unknown as is the value that is added. We could compute a "diff" between
+the global states and use that to compute a symbol patchset, but this is
+quite some work, expensive to do at runtime (it would need to run after
+every module has been compiled) and also would break for hash tables.
+
+We need an API that hides the complex aliasing problems by not relying
+on Nim's global variables. The obvious solution is to use string keys
+instead of global variables:
+
+.. code-block:: nim
+
+  proc cachePut*(key: string; value: string)
+  proc cacheGet*(key: string): string
+
+However, the values being strings/json is quite problematic: Many
+lookup tables that are built at compiletime embed *proc vars* and
+types which have no obvious string representation... Seems like
+AST diffing is still the best idea as it will not require to use
+an alien API and works with some existing Nimble packages, at least.
+
+On the other hand, in Nim's future I would like to replace the VM
+by native code. A diff algorithm wouldn't work for that.
+Instead the native code would work with an API like ``put``, ``get``:
+
+.. code-block:: nim
+
+  proc cachePut*(key: string; value: NimNode)
+  proc cacheGet*(key: string): NimNode
+
+The API should embrace the AST diffing notion: See the
+module ``macrocache`` for the final details.