documentation improvements; GC_step improved

doc/gc.txt

@@ -2,11 +2,15 @@
 Nimrod's Garbage Collector
 ==========================
 
+:Author: Andreas Rumpf
+:Version: |nimrodversion|
+
+
 Introduction
 ============
 
-This document describes how the GC works and how to tune it for (soft)
+This document describes how the GC works and how to tune it for
-realtime systems.
+(soft) `realtime systems`:idx:.
 
 The basic algorithm is *Deferrent Reference Counting* with cycle detection.
 References on the stack are not counted for better performance (and easier C
@@ -15,22 +19,49 @@ delta-subgraph of the heap that changed since its last run.
 
 
 The GC is only triggered in a memory allocation operation. It it not triggered
-by some timer or background thread.
+by some timer and does not run in a background thread.
+
+The cycle collector can be en-/disabled independently from the other parts of
+the GC with ``GC_enableMarkAndSweep`` and ``GC_disableMarkAndSweep``.
+
+To force a full collection call ``GC_fullCollect``. Note that it is generally
+better to let the GC do its work and not enforce a full collection.
 
 
 Realtime support
 ================
 
-To enable realtime support, the switch `useRealtimeGC`:idx: needs to be
+To enable realtime support, the symbol `useRealtimeGC`:idx: needs to be
 defined. With this switch the GC supports the following operations:
 
 .. code-block:: nimrod
   proc GC_setMaxPause*(MaxPauseInUs: int)
   proc GC_step*(us: int, strongAdvice = false)
 
-After calling ``GC_setMaxPause`` any GC run tries to finish within 
+The unit of the parameters ``MaxPauseInUs`` and ``us`` is microseconds.
-``MaxPauseInUs`` microseconds. XXX complete documentation
 
+These two procs are the two modus operandi of the realtime GC: 
+
+(1) GC_SetMaxPause Mode
+
+    You can call ``GC_SetMaxPause`` at program startup and then each triggered
+    GC run tries to not take longer than ``MaxPause`` time. However, it is 
+    possible (and common) that the work is nevertheless not evenly distributed 
+    as each call to ``new`` can trigger the GC and thus take  ``MaxPause`` 
+    time.
+
+(2) GC_step Mode
+
+    This allows the GC to perform some work for up to ``us`` time. This is
+    useful to call in a main loop to ensure the GC can do its work. To    
+    bind all GC activity to a ``GC_step`` call, deactivate the GC with 
+    ``GC_disable`` at program startup.
+
+These procs provide a "best effort" realtime guarantee; in particular the
+cycle collector is not aware of deadlines yet. Deactivate it to get more
+predictable realtime behaviour. Tests show that a 2ms max pause
+time will be met in almost all cases on modern CPUs unless the cycle collector
+is triggered.
 
 
 Time measurement
@@ -53,6 +84,11 @@ later versions of the collector.
 
 
 Tweaking the GC
-===============
+---------------
+
+The collector checks whether there is still time left for its work after 
+every ``workPackage``'th iteration. This is currently set to 100 which means
+that up to 100 objects are traversed and freed before it checks again. Thus
+``workPackage`` affects the timing granularity and may need to be tweaked in
+highly specialized environments or for older hardware.
 
-To be written. 

doc/lib.txt

@@ -201,6 +201,10 @@ Internet Protocols and Support
 * `ftpclient <ftpclient.html>`_
   This module implements an FTP client.
 
+* `cookies <cookies.html>`_
+  This module contains helper procs for parsing and generating cookies.
+
+
 Parsers
 -------
 

doc/nimrodc.txt

@@ -386,7 +386,7 @@ input management. To start Nimrod in interactive mode use the command
 ``nimrod i``. To quit use the ``quit()`` command. To determine whether an input
 line is an incomplete statement to be continued these rules are used:
 
-1. The line ends with ``[-+*/\\<>!\?\|%&$@~,;:=#^]\s*$`` (operator symbol  followed by optional whitespace).
+1. The line ends with ``[-+*/\\<>!\?\|%&$@~,;:=#^]\s*$`` (operator symbol followed by optional whitespace).
 2. The line starts with a space (indentation).
 3. The line is within a triple quoted string literal. However, the detection 
    does not work if the line contains more than one ``"""``.
@@ -407,6 +407,13 @@ So to generate code for an `AVR`:idx: processor use this command::
   
   nimrod c --cpu:avr --os:standalone --gc:none -d:useMalloc --genScript x.nim
 
+
+Nimrod for realtime systems
+===========================
+
+See the documentation of Nimrod's soft realtime `GC <gc.html>`_ for further 
+information.
+
 
 Debugging with Nimrod
 =====================

lib/system.nim

@@ -807,7 +807,6 @@ const
   taintMode = compileOption("taintmode")
 
 when taintMode:
-  # XXX use a compile time option for it!
   type TaintedString* = distinct string ## a distinct string type that 
                                         ## is `tainted`:idx:. It is an alias for
                                         ## ``string`` if the taint mode is not
@@ -1486,8 +1485,9 @@ type
     gcOptimizeTime,    ## optimize for speed
     gcOptimizeSpace    ## optimize for memory footprint
 
-proc GC_setStrategy*(strategy: TGC_Strategy) {.rtl.}
+proc GC_setStrategy*(strategy: TGC_Strategy) {.rtl, deprecated.}
   ## tells the GC the desired strategy for the application.
+  ## **Deprecated** since version 0.8.14. This has always been a nop.
 
 proc GC_enableMarkAndSweep*() {.rtl.}
 proc GC_disableMarkAndSweep*() {.rtl.}

lib/system/gc.nim

@@ -856,15 +856,11 @@ when withRealtime:
 
   proc GC_step(gch: var TGcHeap, us: int, strongAdvice: bool) =
     acquire(gch)
-    var oldThreshold = gch.cycleThreshold
-    # disable cycle collection:
-    gch.cycleThreshold = high(gch.cycleThreshold)-1
     gch.maxPause = us.toNano
-    if strongAdvice:
+    if (gch.zct.len >= ZctThreshold or (cycleGC and
-      if gch.recGcLock == 0: collectCTBody(gch)
+        getOccupiedMem(gch.region)>=gch.cycleThreshold) or alwaysGC) or 
-    else:
+        strongAdvice:
-      collectCT(gch)
+      collectCTBody(gch)
-    gch.cycleThreshold = oldThreshold
     release(gch)
 
   proc GC_step*(us: int, strongAdvice = false) = GC_step(gch, us, strongAdvice)

todo.txt

@@ -1,8 +1,6 @@
 version 0.9.0
 =============
 
-- complete GC's documentation
-- make ``cookies`` part of the stdlib's documentation
 - make templates hygienic by default
 - ``=`` should be overloadable; requires specialization for ``=``
 - fix remaining generics bugs

web/news.txt

@@ -13,12 +13,15 @@ Bugfixes
 - Fixed a bug where the compiler would "optimize away" valid constant parts of
   a string concatenation.
 - Fixed a bug concerning implicit type conversions in ``case`` statements.
+- Fixed a serious code generation bug that caused ``algorithm.sort`` to
+  produce segmentation faults.
 
 
 Library Additions
 -----------------
 
 - Added the (already existing) module ``htmlgen`` to the documentation.
+- Added the (already existing) module ``cookies`` to the documentation.
 - Added ``system.shallow`` that can be used to speed up string and sequence
   assignments.
 - Added ``system.eval`` that can execute an anonymous block of code at
@@ -58,9 +61,10 @@ Changes affecting backwards compatibility
   ``PNimrodNode`` which unfortunately breaks the old macro system.
 - ``pegs.@`` has been renamed to ``pegs.!*`` and ``pegs.@@`` has been renamed
   to ``pegs.!*\`` as ``@`` operators now have different precedence.
-- the type ``proc`` (without any params or return type) is now considered a
+- The type ``proc`` (without any params or return type) is now considered a
   type class matching all proc types. Use ``proc ()`` to get the old meaning
   denoting a proc expecing no arguments and returing no value.
+- Deprecated ``system.GC_setStrategy``.
 
 
 Compiler Additions

web/nimrod.ini

@@ -41,7 +41,7 @@ srcdoc: "pure/json;pure/base64;pure/scgi;pure/redis;impure/graphics"
 srcdoc: "impure/rdstdin;wrappers/zmq;wrappers/sphinx"
 srcdoc: "pure/collections/tables;pure/collections/sets;pure/collections/lists"
 srcdoc: "pure/collections/intsets;pure/collections/queues;pure/encodings"
-srcdoc: "pure/events;pure/collections/sequtils;pure/irc;ecmas/dom"
+srcdoc: "pure/events;pure/collections/sequtils;pure/irc;ecmas/dom;pure/cookies"
 srcdoc: "pure/ftpclient;pure/memfiles;pure/subexes;pure/collections/critbits"
 srcdoc: "pure/asyncio;pure/actors;core/locks;pure/oids;pure/endians"