* add bindParams to db_sqlite
* no need typeinfo
* remove extro spaces
* reduce bindParams to two branches,raise DbError
* Update lib/impure/db_sqlite.nim
* change bindParams to macro,accept varargs[untyped] as params
* change bind blob val to openArray[byte]
* remove unused err type
* explicitly using i32 param
* using import std/private/since
* SQLITE_OK to right hand
* bindParam val int using bindParam overload
* copy data by default
* change exec to template
* remove SqlPrepared procs unused varargs
* fix setupquery for prepared,reset first for exec prepared,add bindNull for literal nil
Co-authored-by: alaviss <leorize+oss@disroot.org>
* add insert,tryInsert unify for postgres that need pk name
* add ReadDbEffect to new procs
* add .since and changelog
* change since to 1.3
* Update lib/impure/db_postgres.nim
Co-authored-by: bung87 <crc32@qq.com>
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
Co-authored-by: alaviss <leorize+oss@disroot.org>
* Allow let to not have value when using importc
This allows a let statement with the `{.importc.}` pragma to not be
initialised with a value. This allows us to declare C constants as Nim
lets without putting the value in the Nim code (which can lead to
errors, and requires us to go looking for the value). Fixes#14253
* Proper fix and documentation + changelog entry
* Improve testcase with one from timotheecour
* Add test to verify it working with macros
* do not track 'raise Defect' in the .raises: [] clause anymore
* --panics:on maps 'raise Defect' to an unrecoverable fatal error
* make tests green again
* update the documentation too
* added bit operations based on bit slices, clarified documentation, made non-mutating versions of mask ops
* Added since annotations, some runnable examples
* Added mask()/masked() functions, changed internal workings of mask ops to use new bit* funcs
* Changelog updated for new bitops improvements
* Reorganization, added runnable examples
* Documentation adjustments
* Add incltrl for since annotation
* Fix masked() impl
* Fix mask() return type
* Don't call toUnsigned on already unsigned types
* Remove improper `var T` for flipMasked()
* Fix return types for flipMasked()
* Slight syntactic cleanup for *masked ops
* Added tests for bitslice operations, new mask() operation, non-mutating mask ops
* Fix setmasked() var T issue
* More comprehensive tests
* Fix runnable example for bitsliced()
* Fix runnable example for mask()
* added high level sendTo and recvFrom to std/asyncnet; tests were also added.
* add .since annotation, a changelog entry and fixed to standard library style guide.
* Improved asserts msgs and added notes for use with UDP sockets
* fix#14064 xmltree should allow create text node with raw text(non-escaped) eg. html style element's text
* change xnRawText to VerbatimText,newRawText to newVerbatimText ,add since anotation
* change changelog_1_2_0.md latest date
* move change log
Co-authored-by: bung87 <crc32@qq.com>
* * relativePath(foo) now works
* relativePath(rel, abs) and relativePath(abs, rel) now work (fixes#13222)
* relativePath, absolutePath, getCurrentDir now available in more targets (eg: vm, nodejs etc)
* fix bug: isAbsolutePath now works with -d:js; add tests
* workaround https://github.com/nim-lang/Nim/issues/13469
* remove `relativePath(path)` overload for now
* add back changelog after rebase
* Add deques.peekFirst/Last(var Deque[T]) -> var T
* Add changelog entry for deques.peekFirst/Last var T overloads
* Add since annotation to peekFirst/peekLast
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* io: make file descriptors non-inheritable by default
This prevents file descriptors/handles leakage to child processes
that might cause issues like running out of file descriptors, or potential
security issues like leaking a file descriptor to a restricted file.
While this breaks backward compatibility, I'm rather certain that not
many programs (if any) actually make use of this implementation detail.
A new API `setInheritable` is provided for the few that actually want to
use this functionality.
* io: disable inheritance at file creation time for supported platforms
Some platforms provide extension to fopen-family of functions to allow
for disabling descriptor inheritance atomically during File creation.
This guards against possible leaks when a child process is spawned
before we managed to disable the file descriptor inheritance
(ie. in a multi-threaded program).
* net, nativesockets: make sockets non inheritable by default
With this commit, sockets will no longer leak to child processes when
you don't want it to. Should solves a lot of "address in use" that might
occur when your server has just restarted.
All APIs that create sockets in these modules now expose a `inheritable`
flag that allow users to toggle inheritance for the resulting sockets.
An implementation of `setInheritance()` is also provided for SocketHandle.
While atomically disabling inheritance at creation time is supported on
Windows, it's only implemented by native winsock2, which is too much for
now. This support can be implemented in a future patch.
* posix: add F_DUPFD_CLOEXEC
This command duplicates file descriptor with close-on-exec flag set.
Defined in POSIX.1-2008.
* ioselectors_kqueue: don't leak file descriptors
File descriptors internally used by ioselectors on BSD/OSX are now
shielded from leakage.
* posix: add O_CLOEXEC
This flag allows file descriptors to be open() with close-on-exec flag
set atomically.
This flag is specified in POSIX.1-2008
* tfdleak: test for selectors leakage
Also simplified the test by using handle-type agnostic APIs to test for
validity.
* ioselectors_epoll: mark all fd created close-on-exec
File descriptors from ioselectors should no longer leaks on Linux.
* tfdleak: don't check for selector leakage on Windows
The getFd proc for ioselectors_select returns a hardcoded -1
* io: add NoInheritFlag at compile time
* io: add support for ioctl-based close-on-exec
This allows for the flag to be set/unset in one syscall. While the
performance gains might be negliable, we have one less failure point
to deal with.
* tfdleak: add a test for setInheritable
* stdlib: add nimInheritHandles to restore old behaviors
* memfiles: make file handle not inheritable by default for posix
* io: setInheritable now operates on OS file handle
On Windows, the native handle is the only thing that's inheritable, thus
we can assume that users of this function will already have the handle
available to them. This also allows users to pass down file descriptors
from memfiles on Windows with ease, should that be desired.
With this, nativesockets.setInheritable can be made much simpler.
* changelog: clarify
* nativesockets: document setInheritable return value
* posix_utils: atomically disable fd inheritance for mkstemp
* Unwind just the "pseudorandom probing" (whole hash-code-keyed variable
stride double hashing) part of recent sets & tables changes (which has
still been causing bugs over a month later (e.g., two days ago
https://github.com/nim-lang/Nim/issues/13794) as well as still having
several "figure this out" implementation question comments in them (see
just diffs of this PR).
This topic has been discussed in many places:
https://github.com/nim-lang/Nim/issues/13393https://github.com/nim-lang/Nim/pull/13418https://github.com/nim-lang/Nim/pull/13440https://github.com/nim-lang/Nim/issues/13794
Alternative/non-mandatory stronger integer hashes (or vice-versa opt-in
identity hashes) are a better solution that is more general (no illusion
of one hard-coded sequence solving all problems) while retaining the
virtues of linear probing such as cache obliviousness and age-less tables
under delete-heavy workloads (still untested after a month of this change).
The only real solution for truly adversarial keys is a hash keyed off of
data unobservable to attackers. That all fits better with a few families
of user-pluggable/define-switchable hashes which can be provided in a
separate PR more about `hashes.nim`.
This PR carefully preserves the better (but still hard coded!) probing
of the `intsets` and other recent fixes like `move` annotations, hash
order invariant tests, `intsets.missingOrExcl` fixing, and the move of
`rightSize` into `hashcommon.nim`.
* Fix `data.len` -> `dataLen` problem.
* This is an alternate resolution to https://github.com/nim-lang/Nim/issues/13393
(which arguably could be resolved outside the stdlib).
Add version1 of Wang Yi's hash specialized to 8 byte integers. This gives
simple help to users having trouble with overly colliding hash(key)s. I.e.,
A) `import hashes; proc hash(x: myInt): Hash = hashWangYi1(int(x))`
in the instantiation context of a `HashSet` or `Table`
or
B) more globally, compile with `nim c -d:hashWangYi1`.
No hash can be all things to all use cases, but this one is A) vetted to
scramble well by the SMHasher test suite (a necessarily limited but far
more thorough test than prior proposals here), B) only a few ALU ops on
many common CPUs, and C) possesses an easy via "grade school multi-digit
multiplication" fall back for weaker deployment contexts.
Some people might want to stampede ahead unbridled, but my view is that a
good plan is to
A) include this in the stdlib for a release or three to let people try it
on various key sets nim-core could realistically never access/test
(maybe mentioning it in the changelog so people actually try it out),
B) have them report problems (if any),
C) if all seems good, make the stdlib more novice friendly by adding
`hashIdentity(x)=x` and changing the default `hash() = hashWangYi1`
with some `when defined` rearranging so users can `-d:hashIdentity`
if they want the old behavior back.
This plan is compatible with any number of competing integer hashes if
people want to add them. I would strongly recommend they all *at least*
pass the SMHasher suite since the idea here is to become more friendly to
novices who do not generally understand hashing failure modes.
* Re-organize to work around `when nimvm` limitations; Add some tests; Add
a changelog.md entry.
* Add less than 64-bit CPU when fork.
* Fix decl instead of call typo.
* First attempt at fixing range error on 32-bit platforms; Still do the
arithmetic in doubled up 64-bit, but truncate the hash to the lower
32-bits, but then still return `uint64` to be the same. So, type
correct but truncated hash value. Update `thashes.nim` as well.
* A second try at making 32-bit mode CI work.
* Use a more systematic identifier convention than Wang Yi's code.
* Fix test that was wrong for as long as `toHashSet` used `rightSize` (a
very long time, I think). `$a`/`$b` depend on iteration order which
varies with table range reduced hash order which varies with range for
some `hash()`. With 3 elements, 3!=6 is small and we've just gotten
lucky with past experimental `hash()` changes. An alternate fix here
would be to not stringify but use the HashSet operators, but it is not
clear that doesn't alter the "spirit" of the test.
* Fix another stringified test depending upon hash order.
* Oops - revert the string-keyed test.
* Fix another stringify test depending on hash order.
* Add a better than always zero `defined(js)` branch.
* It turns out to be easy to just work all in `BigInt` inside JS and thus
guarantee the same low order bits of output hashes (for `isSafeInteger`
input numbers). Since `hashWangYi1` output bits are equally random in
all their bits, this means that tables will be safely scrambled for table
sizes up to 2**32 or 4 gigaentries which is probably fine, as long as the
integer keys are all < 2**53 (also likely fine). (I'm unsure why the
infidelity with C/C++ back ends cut off is 32, not 53 bits.)
Since HashSet & Table only use the low order bits, a quick corollary of
this is that `$` on most int-keyed sets/tables will be the same in all
the various back ends which seems a nice-to-have trait.
* These string hash tests fail for me locally. Maybe this is what causes
the CI hang for testament pcat collections?
* Oops. That failure was from me manually patching string hash in hashes. Revert.
* Import more test improvements from https://github.com/nim-lang/Nim/pull/13410
* Fix bug where I swapped order when reverting the test. Ack.
* Oh, just accept either order like more and more hash tests.
* Iterate in the same order.
* `return` inside `emit` made us skip `popFrame` causing weird troubles.
* Oops - do Windows branch also.
* `nimV1hash` -> multiply-mnemonic, type-scoped `nimIntHash1` (mnemonic
resolutions are "1 == identity", 1 for Nim Version 1, 1 for
first/simplest/fastest in a series of possibilities. Should be very
easy to remember.)
* Re-organize `when nimvm` logic to be a strict `when`-`else`.
* Merge other changes.
* Lift constants to a common area.
* Fall back to identity hash when `BigInt` is unavailable.
* Increase timeout slightly (probably just real-time perturbation of CI
system performance).
* Improve jsconsole adding the rest of the stable api as documented on the standard at https://developer.mozilla.org/docs/Web/API/Console
* Improve jsconsole, add runnableexamples
* Simplify jsconsole
* new minor feature: macros for proc types, to be documented
* Finished the implementation and added tests
* [skip ci] Describe the new custom pragmas in the manual and the changelog
Co-authored-by: Zahary Karadjov <zahary@gmail.com>
* stacktraces can now show custom runtime msgs
* improve tests/stdlib/tstackframes.nim
* fix test for --gc:arc
* test --stacktraceMsgs:on and --stacktraceMsgs:off
* --stacktracemsgs:off by default
