custom integer literals bugfixes (#17499)

* custom integer literals bugfixes

* make nimsuggest compile again

changelog.md

@@ -266,7 +266,7 @@
 - The unary minus in `-1` is now part of the integer literal, it is now parsed as a single token.
   This implies that edge cases like `-128'i8` finally work correctly.
 
-- Custom numeric literals are now supported.
+- Custom numeric literals (e.g. `-128'bignum`) are now supported.
 
 
 ## Compiler changes

compiler/lexer.nim

@@ -345,13 +345,14 @@ proc getNumber(L: var Lexer, result: var Token) =
   result.tokType = tkIntLit   # int literal until we know better
   result.literal = ""
   result.base = base10
-  let startpos = L.bufpos
+  tokenBegin(result, L.bufpos)
-  tokenBegin(result, startpos)
 
   var isPositive = true
   if L.buf[L.bufpos] == '-':
     eatChar(L, result)
-    isPositive = true
+    isPositive = false
+
+  let startpos = L.bufpos
 
   template setNumber(field, value) =
     field = (if isPositive: value else: -value)
@@ -419,7 +420,7 @@ proc getNumber(L: var Lexer, result: var Token) =
         suffixAsLower.add toLowerAscii(c)
         inc postPos
         if L.buf[postPos] notin SymChars+{'_'}: break
-      case suffix
+      case suffixAsLower
       of "f", "f32": result.tokType = tkFloat32Lit
       of "d", "f64": result.tokType = tkFloat64Lit
       of "f128": result.tokType = tkFloat128Lit

doc/manual.rst

@@ -490,7 +490,7 @@ this. Another reason is that Nim can thus support `array[char, int]` or
 type is used for Unicode characters, it can represent any Unicode character.
 `Rune` is declared in the `unicode module <unicode.html>`_.
 
-A character literal that  does not end in ``'`` interpreted as ``'`` if there
+A character literal that does not end in ``'`` is interpreted as ``'`` if there
 is a preceeding backtick token. There must be no whitespace between the preceeding
 backtick token and the character literal. This special rule ensures that a declaration
 like ``proc `'customLiteral`(s: string)`` is valid. See also
@@ -538,6 +538,9 @@ Numeric literals have the form::
 
   CUSTOM_NUMERIC_LIT = (FLOAT_LIT | DEC_LIT | OCT_LIT | BIN_LIT) '\'' CUSTOM_NUMERIC_SUFFIX
 
+  # CUSTOM_NUMERIC_SUFFIX is any Nim identifier that is not
+  # a pre-defined type suffix.
+
 
 As can be seen in the productions, numeric literals can contain underscores
 for readability. Integer and floating-point literals may be given in decimal (no
@@ -653,6 +656,7 @@ the case that additional parameters are passed to the callee:
   var x = 5'u4(123)
 
 Custom numeric literals are covered by the grammar rule named `CUSTOM_NUMERIC_LIT`.
+A custom numeric literal is a single token.
 
 
 Operators

tests/lexer/tunary_minus.nim

@@ -51,6 +51,9 @@ template main =
     doAssert x() == minusOne:
       "unable to handle negatives after semi-colon"
 
+    doAssert -0b111 == -7
+    doAssert -0xff == -255
+
   block: # check when a minus (-) is an unary op
     doAssert -one == minusOne:
       "unable to a negative prior to identifier"