destructors: spec reflects reality, =sink is here to stay

doc/destructors.rst

@@ -52,17 +52,12 @@ written as:
       for i in 0..<a.len:
         a.data[i] = b.data[i]
 
-  proc `=move`*[T](a, b: var myseq[T]) =
+  proc `=sink`*[T](a: var myseq[T]; b: myseq[T]) =
-    # do nothing for self-assignments:
+    # move assignment
-    if a.data == b.data: return
     `=destroy`(a)
     a.len = b.len
     a.cap = b.cap
     a.data = b.data
-    # b's elements have been stolen so ensure that the
-    # destructor for b does nothing:
-    b.data = nil
-    b.len = 0
 
   proc add*[T](x: var myseq[T]; y: sink T) =
     if x.len >= x.cap: resize(x)
@@ -128,38 +123,41 @@ The general pattern in ``=destroy`` looks like:
 
 
 
-`=move` hook
+`=sink` hook
 ------------
 
-A `=move` hook moves an object around, the resources are stolen from the source
+A `=sink` hook moves an object around, the resources are stolen from the source
-and passed to the destination. It must be ensured that source's destructor does
+and passed to the destination. It is ensured that source's destructor does
-not free the resources afterwards.
+not free the resources afterwards by setting the object to its default value
+(the value the object's state started in). Setting an object ``x`` back to its
+default value is written as ``wasMoved(x)``.
 
 The prototype of this hook for a type ``T`` needs to be:
 
 .. code-block:: nim
 
-  proc `=move`(dest, source: var T)
+  proc `=sink`(dest: var T; source: T)
 
 
-The general pattern in ``=move`` looks like:
+The general pattern in ``=sink`` looks like:
 
 .. code-block:: nim
 
-  proc `=move`(dest, source: var T) =
+  proc `=sink`(dest: var T; source: T) =
-    # protect against self-assignments:
-    if dest.field != source.field:
     `=destroy`(dest)
     dest.field = source.field
-      source.field = nil
 
 
+**Note**: ``=sink`` does not need to check for self-assignments.
+How self-assignments are handled is explained later in this document.
+
 
 `=` (copy) hook
 ---------------
 
 The ordinary assignment in Nim conceptually copies the values. The ``=`` hook
-is called for assignments that couldn't be transformed into moves.
+is called for assignments that couldn't be transformed into ``=sink``
+operations.
 
 The prototype of this hook for a type ``T`` needs to be:
 
@@ -197,11 +195,11 @@ Swap
 ====
 
 The need to check for self-assignments and also the need to destroy previous
-objects inside ``=`` and ``=move`` is a strong indicator to treat ``system.swap``
+objects inside ``=`` and ``=sink`` is a strong indicator to treat
-as a builtin primitive of its own that simply swaps every field in the involved
+``system.swap`` as a builtin primitive of its own that simply swaps every
-objects via ``copyMem`` or a comparable mechanism.
+field in the involved objects via ``copyMem`` or a comparable mechanism.
 In other words, ``swap(a, b)`` is **not** implemented
-as ``let tmp = move(a); b = move(a); a = move(tmp)``!
+as ``let tmp = move(a); b = move(a); a = move(tmp)``.
 
 This has further consequences:
 
@@ -214,8 +212,12 @@ Sink parameters
 ===============
 
 To move a variable into a collection usually ``sink`` parameters are involved.
-A location that is passed to a ``sink`` parameters should not be used afterwards.
+A location that is passed to a ``sink`` parameter should not be used afterwards.
-This is ensured by a static analysis over a control flow graph. A sink parameter
+This is ensured by a static analysis over a control flow graph. If it cannot be
+proven to be the last usage of the location, a copy is done instead and this
+copy is then passed to the sink parameter.
+
+A sink parameter
 *may* be consumed once in the proc's body but doesn't have to be consumed at all.
 The reason for this is that signatures
 like ``proc put(t: var Table; k: sink Key, v: sink Value)`` should be possible
@@ -250,63 +252,10 @@ An implementation is allowed, but not required to implement even more move
 optimizations (and the current implementation does not).
 
 
-Self assignments
-================
-
-Unfortunately this document departs significantly from
-the older design as specified here, https://github.com/nim-lang/Nim/wiki/Destructors.
-The reason is that under the old design so called "self assignments" could not work.
-
-
-.. code-block:: nim
-
-  proc select(cond: bool; a, b: sink string): string =
-    if cond:
-      result = a # moves a into result
-    else:
-      result = b # moves b into result
-
-  proc main =
-    var x = "abc"
-    var y = "xyz"
-
-    # possible self-assignment:
-    x = select(rand() < 0.5, x, y)
-    # 'select' must communicate what parameter has been
-    # consumed. We cannot simply generate:
-    # (select(...); wasMoved(x); wasMoved(y))
-
-Consequence: ``sink`` parameters for objects that have a non-trivial destructor
-must be passed as by-pointer under the hood. A further advantage is that parameters
-are never destroyed, only variables are. The caller's location passed to
-a ``sink`` parameter has to be destroyed by the caller and does not burden
-the callee.
-
-
-Const temporaries
-=================
-
-Constant literals like ``nil`` cannot be easily be ``=moved``'d. The solution
-is to pass a temporary location that contains ``nil`` to the sink location.
-In other words, ``var T`` can only bind to locations, but ``sink T`` can bind
-to values.
-
-For example:
-
-.. code-block:: nim
-
-  var x: owned ref T = nil
-  # gets turned into by the compiler:
-  var tmp = nil
-  move(x, tmp)
-
 
 Rewrite rules
 =============
 
-**Note**: A function call ``f()`` is always the "last read" of the involved
-temporary location and so covered under the more general rewrite rules.
-
 **Note**: There are two different allowed implementation strategies:
 
 1. The produced ``finally`` section can be a single section that is wrapped
@@ -324,17 +273,28 @@ not destroyed at the scope exit, but at the proc exit.
   finally: `=destroy`(x)
 
 
-  f(...)
+  g(f(...))
-  ------------------------    (function-call)
+  ------------------------    (nested-function-call)
-  (let tmp;
+  g(let tmp;
   bitwiseCopy tmp, f(...);
   tmp)
   finally: `=destroy`(tmp)
 
 
+  x = f(...)
+  ------------------------    (function-sink)
+  `=sink`(x, f(...))
+
+
   x = lastReadOf z
   ------------------          (move-optimization)
-  `=move`(x, z)
+  `=sink`(x, z)
+  wasMoved(z)
+
+
+  v = v
+  ------------------   (self-assignment-removal)
+  discard "nop"
 
 
   x = y
@@ -342,62 +302,106 @@ not destroyed at the scope exit, but at the proc exit.
   `=`(x, y)
 
 
-  x = move y
+  f_sink(g())
-  ------------------          (enforced-move)
+  -----------------------     (call-to-sink)
-  `=move`(x, y)
+  f_sink(g())
 
 
   f_sink(notLastReadOf y)
-  -----------------------     (copy-to-sink)
+  --------------------------     (copy-to-sink)
-  (let tmp; `=`(tmp, y); f_sink(tmp))
+  (let tmp; `=`(tmp, y);
-  finally: `=destroy`(tmp)
+  f_sink(tmp))
 
 
-  f_sink(move y)
+  f_sink(lastReadOf y)
-  -----------------------     (enforced-move-to-sink)
+  -----------------------     (move-to-sink)
-  (let tmp; `=move`(tmp, y); f_sink(tmp))
+  f_sink(y)
-  finally: `=destroy`(tmp)
+  wasMoved(y)
 
 
+Object and array construction
+=============================
 
-Cursor variables
+Object and array construction is treated as a function call where the
+function has ``sink`` parameters.
+
+
+Destructor removal
+==================
+
+``wasMoved(x);`` followed by a `=destroy(x)` operation cancel each other
+out. An implementation is encouraged to exploit this in order to improve
+efficiency and code sizes.
+
+
+Self assignments
 ================
 
-There is an additional rewrite rule for so called "cursor" variables.
+``=sink`` in combination with ``wasMoved`` can handle self-assignments but
-A cursor variable is a variable that is only used for navigation inside
+it's subtle.
-a data structure. The otherwise implied copies (or moves) and destructions
-can be avoided altogether for cursor variables:
 
-::
+The simple case of ``x = x`` cannot be turned
+into ``=sink(x, x); wasMoved(x)`` because that would lose ``x``'s value.
+The solution is that simple self-assignments are simply transformed into
+an empty statement that does nothing.
 
-  var x {.cursor.}: T
+The complex case looks like a variant of ``x = f(x)``, we consider
-  x = path(z)
+``x = select(rand() < 0.5, x, y)`` here:
-  stmts
-  --------------------------  (cursor-var)
-  x = bitwiseCopy(path z)
-  stmts
-  # x is not destroyed.
 
 
-``stmts`` must not mutate ``z`` nor ``x``. All assignments to ``x`` must be
-of the form ``path(z)`` but the ``z`` can differ. Neither ``z`` nor ``x``
-can be aliased; this implies the addresses of these locations must not be
-used explicitly.
-
-The current implementation does not compute cursor variables but supports
-the ``.cursor`` pragma annotation. Cursor variables are respected and
-simply trusted: No checking is performed that no mutations or aliasing
-occurs.
-
-Cursor variables are commonly used in ``iterator`` implementations:
-
 .. code-block:: nim
 
-  iterator nonEmptyItems(x: seq[string]): string =
+  proc select(cond: bool; a, b: sink string): string =
-    for i in 0..high(x):
+    if cond:
-      let it {.cursor.} = x[i] # no string copies, no destruction of 'it'
+      result = a # moves a into result
-      if it.len > 0:
+    else:
-        yield it
+      result = b # moves b into result
+
+  proc main =
+    var x = "abc"
+    var y = "xyz"
+    # possible self-assignment:
+    x = select(true, x, y)
+
+
+Is transformed into:
+
+
+.. code-block:: nim
+
+  proc select(cond: bool; a, b: sink string): string =
+    try:
+      if cond:
+        `=sink`(result, a)
+        wasMoved(a)
+      else:
+        `=sink`(result, b)
+        wasMoved(b)
+    finally:
+      `=destroy`(b)
+      `=destroy`(a)
+
+  proc main =
+    var
+      x: string
+      y: string
+    try:
+      `=sink`(x, "abc")
+      `=sink`(y, "xyz")
+      `=sink`(x, select(true,
+        let blitTmp = x
+        wasMoved(x)
+        blitTmp,
+        let blitTmp = y
+        wasMoved(y)
+        blitTmp))
+      echo [x]
+    finally:
+      `=destroy`(y)
+      `=destroy`(x)
+
+As can be manually verified, this transformation is correct for
+self-assignments.
 
 
 Lent type
@@ -425,14 +429,14 @@ for expressions of type ``lent T`` or of type ``var T``.
   proc construct(kids: sink seq[Tree]): Tree =
     result = Tree(kids: kids)
     # converted into:
-    `=move`(result.kids, kids)
+    `=sink`(result.kids, kids); wasMoved(kids)
 
   proc `[]`*(x: Tree; i: int): lent Tree =
     result = x.kids[i]
     # borrows from 'x', this is transformed into:
     result = addr x.kids[i]
     # This means 'lent' is like 'var T' a hidden pointer.
-    # Unlike 'var' this cannot be used to mutate the object.
+    # Unlike 'var' this hidden pointer cannot be used to mutate the object.
 
   iterator children*(t: Tree): lent Tree =
     for x in t.kids: yield x
@@ -459,11 +463,10 @@ Let ``W`` be an ``owned ref`` type. Conceptually its hooks look like:
 
   proc `=`(x: var W; y: W) {.error: "owned refs can only be moved".}
 
-  proc `=move`(x, y: var W) =
+  proc `=sink`(x: var W; y: W) =
-    if x != y:
     `=destroy`(x)
     bitwiseCopy x, y # raw pointer copy
-      y = nil
+
 
 Let ``U`` be an unowned ``ref`` type. Conceptually its hooks look like:
 
@@ -479,9 +482,8 @@ Let ``U`` be an unowned ``ref`` type. Conceptually its hooks look like:
     if x != nil: dec x.refcount
     bitwiseCopy x, y # raw pointer copy
 
-  proc `=move`(x, y: var U) =
+  proc `=sink`(x: var U, y: U) {.error.}
-    # Note: Moves are the same as assignments.
+  # Note: Moves are not available.
-    `=`(x, y)
 
 
 Hook lifting
@@ -490,7 +492,7 @@ Hook lifting
 The hooks of a tuple type ``(A, B, ...)`` are generated by lifting the
 hooks of the involved types ``A``, ``B``, ... to the tuple type. In
 other words, a copy ``x = y`` is implemented
-as ``x[0] = y[0]; x[1] = y[1]; ...``, likewise for ``=move`` and ``=destroy``.
+as ``x[0] = y[0]; x[1] = y[1]; ...``, likewise for ``=sink`` and ``=destroy``.
 
 Other value-based compound types like ``object`` and ``array`` are handled
 correspondingly. For ``object`` however, the compiler generated hooks
@@ -529,6 +531,8 @@ have been derived from the rewrite rules and are as follows:
   hooks are generated for ``typeof(x)``.
 - In ``x = ...`` (assignment) hooks are generated for ``typeof(x)``.
 - In ``f(...)`` (function call) hooks are generated for ``typeof(f(...))``.
+- For every sink parameter ``x: sink T`` the hooks are generated
+  for ``typeof(x)``.
 
 
 nodestroy pragma