Unwind just the "pseudorandom probing" part of recent sets,tables changes (#13816)

* Unwind just the "pseudorandom probing" (whole hash-code-keyed variable
stride double hashing) part of recent sets & tables changes (which has
still been causing bugs over a month later (e.g., two days ago
https://github.com/nim-lang/Nim/issues/13794) as well as still having
several "figure this out" implementation question comments in them (see
just diffs of this PR).

This topic has been discussed in many places:
  https://github.com/nim-lang/Nim/issues/13393
  https://github.com/nim-lang/Nim/pull/13418
  https://github.com/nim-lang/Nim/pull/13440
  https://github.com/nim-lang/Nim/issues/13794

Alternative/non-mandatory stronger integer hashes (or vice-versa opt-in
identity hashes) are a better solution that is more general (no illusion
of one hard-coded sequence solving all problems) while retaining the
virtues of linear probing such as cache obliviousness and age-less tables
under delete-heavy workloads (still untested after a month of this change).

The only real solution for truly adversarial keys is a hash keyed off of
data unobservable to attackers.  That all fits better with a few families
of user-pluggable/define-switchable hashes which can be provided in a
separate PR more about `hashes.nim`.

This PR carefully preserves the better (but still hard coded!) probing
of the  `intsets` and other recent fixes like `move` annotations, hash
order invariant tests, `intsets.missingOrExcl` fixing, and the move of
`rightSize` into `hashcommon.nim`.

* Fix `data.len` -> `dataLen` problem.
This commit is contained in:
c-blake 2020-03-31 13:18:45 -04:00 • committed by GitHub
commit b1aa3b1eea
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 99 additions and 175 deletions

View file

@ -46,20 +46,16 @@ type
IntSet* = object ## An efficient set of `int` implemented as a sparse bit set.
elems: int # only valid for small numbers
counter, max: int
countDeleted: int
head: PTrunk
data: TrunkSeq
a: array[0..33, int] # profiling shows that 34 elements are enough
proc mustRehash[T](t: T): bool {.inline.} =
# FACTOR between hashcommon.mustRehash, intsets.mustRehash
let counter2 = t.counter + t.countDeleted
let length = t.max + 1
assert length > counter2
result = (length * 2 < counter2 * 3) or (length - counter2 < 4)
assert length > t.counter
result = (length * 2 < t.counter * 3) or (length - t.counter < 4)
proc nextTry(h, maxHash: Hash, perturb: var Hash): Hash {.inline.} =
# FACTOR between hashcommon.nextTry, intsets.nextTry
const PERTURB_SHIFT = 5
var perturb2 = cast[uint](perturb) shr PERTURB_SHIFT
perturb = cast[Hash](perturb2)