* Allow an escape hatch for platform specific flags (of which there are many,
for example MAP_POPULATE itself is a Linux-only thing, not other Unix).
Continue with same defaults as before in this commit, but that really
should be changed to *not* include MAP_POPULATE. While pre-faulting
all the pages can be useful sometimes *if* you know you're going to
access all the data, it is highly unlikely to be what users expect the
default to be. For some things all that up front work is 1000s of
times slower than being lazy/on-demand/only ever faulting parts of the
file. Even the MAP_POPULATE fan who originally in 2014 committed to
this file defaulted it to off (but turned it always-on as a "temporary"
work around for some long since gone/mutated compiler issue).
Anyway, at least something like this `mapFlags` gives users the ability
to override the poor default choice or activate any other idiosyncratic
platform-specific features.
* Use simple, efficient default flags, but also accept whatever the open/mapMem
caller specifies. Save flags in MemFile so they can be used in `resize`.
This field should not need exporting like the others -- callers can always
save whatever values they pass -- but we include a cautionary comment in
case anyone ever asks for a `*` there, as well as for documentation.
Also make documentation for ``mapFlags`` in open more likely to inspire care.
Updating accessors are also provided since the idea of this change is to
allow "updating" operations external to the module which are by their very
nature closely tied to module internals (as well as to OS interface details).
operations (like eg., resizing a file and re-mapping) do not need
to worry about race conditions of re-opened paths, renamed parent
directories and that sort of thing. Operating directly on already
open handles is both safer and more efficient than relying upon the
stability of filesystem paths.
* Add MemMapFileStream
* Added tests
* Fixed bug in memfiles (zero index for string)
* Added flush to changelog
* Attempt to fix Win's nuances
* Fix attempt to fix
* Continue...
* And again...
* Reworked tests (all for win on Win)
* Fixes in flush (Win)
* Replace fn vars to consts
* Added the attempts parameter to the flush
* Replace while to for
* Move to memfiles
* Use Natural instead of uint
* Better error messages for append mode. Handle specific cases.
* Fix 2 problems. First, 0 is a valid fd on Unix (easily gotten if user first
closes all fds and then starts using memfiles). Use -1 instead for an invalid
fd. Second, it is best practice to conserve open fds on Unix and file handles
on Windows. These handles are not needed unless the user wants to remap the
memory with ``mapMem`` (or a hypothetical future ``proc resize``). Adding a
new bool param ``allowRemap=false`` to ``memfiles.open`` solves this cleanly
in a "mostly" backward compatible way. This is only "mostly" because the
default ``false`` case does not keep unneeded resources allocated, but that
most sensible default means that any ``mapMem`` callers need to fix all their
open calls to have allowRemap=true, as this PR also does for tmemfiles2.nim.
* Include backwards compatibility note.
type, then converter, then 3 iterators lowest- to highest-level
(also fastest to slowest) including a new intermediate iterator
lines(MemFile, buffer) that is more like readLine(File) in case
that helps anyone port code.
Add doc comments.
Also have toString just use newString+c_memcpy instead of
currently fragile toNimStr which Araq wants a separate PR for.
