jyapayne/Nim
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #2 from nim-lang/devel

Browse source 
Update
This commit is contained in:
Daniil Yarancev 2018-06-05 21:25:45 +03:00 • committed by GitHub
commit 6426413598
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
598 changed files with 32385 additions and 27063 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.gitlab-ci.yml
View file

@ -8,7 +8,7 @@ stages:
.linux_set_path: &linux_set_path_def
before_script:
- export PATH=$(pwd)/bin:$PATH
- export PATH=$(pwd)/bin${PATH:+:$PATH}
tags:
- linux

2
.travis.yml
View file

@ -30,7 +30,7 @@ before_script:
- sh build.sh
- cd ..
- sed -i -e 's,cc = gcc,cc = clang,' config/nim.cfg
- export PATH=$(pwd)/bin:$PATH
- export PATH=$(pwd)/bin${PATH:+:$PATH}
script:
- nim c koch
- ./koch boot

1
appveyor.yml
View file

@ -24,6 +24,7 @@ environment:
# platform: x86
install:
- ps: Install-Product node 8 # node 8 or later is required to test js async stuff
- MKDIR %CD%\DIST
- MKDIR %CD%\DIST\PCRE
- nuget install pcre -Verbosity quiet -Version 8.33.0.1 -OutputDirectory %CD%\DIST\PCRE

306
changelog.md
View file

@ -1,192 +1,154 @@
## v0.18.0 - dd/mm/yyyy
## v0.19.X - XX/XX/2018
### Changes affecting backwards compatibility
- The stdlib module ``future`` has been renamed to ``sugar``.
- ``macros.callsite`` is now deprecated. Since the introduction of ``varargs``
parameters this became unnecessary.
- Anonymous tuples with a single element can now be written as ``(1,)`` with a
trailing comma. The underlying AST is ``nnkTupleConst(newLit 1)`` for this
example. ``nnkTupleConstr`` is a new node kind your macros need to be able
to deal with!
- Indexing into a ``cstring`` for the JS target is now mapped
to ``charCodeAt``.
- Assignments that would "slice" an object into its supertype are now prevented
at runtime. Use ``ref object`` with inheritance rather than ``object`` with
inheritance to prevent this issue.
- The ``not nil`` type annotation now has to be enabled explicitly
via ``{.experimental: "notnil"}`` as we are still not pleased with how this
feature works with Nim's containers.
- The parser now warns about inconsistent spacing around binary operators as
these can easily be confused with unary operators. This warning will likely
become an error in the future.
- Arrays of char cannot be converted to ``cstring`` anymore, pointers to
arrays of char can! This means ``$`` for arrays can finally exist
in ``system.nim`` and do the right thing.
- ``echo`` now works with strings that contain ``\0`` (the binary zero is not
shown) and ``nil`` strings are equal to empty strings.
- JSON: Deprecated `getBVal`, `getFNum`, and `getNum` in favour to
`getBool`, `getFloat`, `getBiggestInt`. Also `getInt` procedure was added.
- `reExtended` is no longer default for the `re` constructor in the `re`
module.
- The overloading rules changed slightly so that constrained generics are
preferred over unconstrained generics. (Bug #6526)
- It is now possible to forward declare object types so that mutually
recursive types can be created across module boundaries. See
[package level objects](https://nim-lang.org/docs/manual.html#package-level-objects)
for more information.
- The **unary** ``<`` is now deprecated, for ``.. <`` use ``..<`` for other usages
use the ``pred`` proc.
- We changed how array accesses "from backwards" like ``a[^1]`` or ``a[0..^1]`` are
implemented. These are now implemented purely in ``system.nim`` without compiler
support. There is a new "heterogenous" slice type ``system.HSlice`` that takes 2
generic parameters which can be ``BackwardsIndex`` indices. ``BackwardsIndex`` is
produced by ``system.^``.
This means if you overload ``[]`` or ``[]=`` you need to ensure they also work
with ``system.BackwardsIndex`` (if applicable for the accessors).
- ``mod`` and bitwise ``and`` do not produce ``range`` subtypes anymore. This
turned out to be more harmful than helpful and the language is simpler
without this special typing rule.
- Added ``algorithm.rotateLeft``.
- ``rationals.toRational`` now uses an algorithm based on continued fractions.
This means its results are more precise and it can't run into an infinite loop
anymore.
- Added ``typetraits.$`` as an alias for ``typetraits.name``.
- ``os.getEnv`` now takes an optional ``default`` parameter that tells ``getEnv``
what to return if the environment variable does not exist.
- Bodies of ``for`` loops now get their own scope:
```nim
# now compiles:
for i in 0..4:
let i = i + 1
echo i
```
#### Breaking changes in the standard library
- The parsing rules of ``if`` expressions were changed so that multiple
statements are allowed in the branches. We found few code examples that
now fail because of this change, but here is one:
- ``re.split`` for empty regular expressions now yields every character in
the string which is what other programming languages chose to do.
- The returned tuple of ``system.instantiationInfo`` now has a third field
containing the column of the instantiation.
```nim
t[ti] = if exp_negative: '-' else: '+'; inc(ti)
```
- ``cookies.setCookie` no longer assumes UTC for the expiration date.
- ``strutils.formatEng`` does not distinguish between ``nil`` and ``""``
strings anymore for its ``unit`` parameter. Instead the space is controlled
by a new parameter ``useUnitSpace``.
This now needs to be written as:
- ``proc `-`*(a, b: Time): int64`` in the ``times`` module has changed return type
to ``times.Duration`` in order to support higher time resolutions.
The proc is no longer deprecated.
- ``posix.Timeval.tv_sec`` has changed type to ``posix.Time``.
```nim
t[ti] = (if exp_negative: '-' else: '+'); inc(ti)
```
- ``math.`mod` `` for floats now behaves the same as ``mod`` for integers
(previously it used floor division like Python). Use ``math.floorMod`` for the old behavior.
- To make Nim even more robust the system iterators ``..`` and ``countup``
now only accept a single generic type ``T``. This means the following code
doesn't die with an "out of range" error anymore:
#### Breaking changes in the compiler
```nim
var b = 5.Natural
var a = -5
for i in a..b:
echo i
```
- The undocumented ``#? braces`` parsing mode was removed.
- The undocumented PHP backend was removed.
- ``formatFloat``/``formatBiggestFloat`` now support formatting floats with zero
precision digits. The previous ``precision = 0`` behavior (default formatting)
is now available via ``precision = -1``.
- The ``nim doc`` command is now an alias for ``nim doc2``, the second version of
the documentation generator. The old version 1 can still be accessed
via the new ``nim doc0`` command.
- Added ``system.getStackTraceEntries`` that allows you to access the stack
trace in a structured manner without string parsing.
- Added ``sequtils.mapLiterals`` for easier construction of array and tuple
literals.
- Added ``parseutils.parseSaturatedNatural``.
- ``atomic`` and ``generic`` are no longer keywords in Nim. ``generic`` used to be
an alias for ``concept``, ``atomic`` was not used for anything.
- Moved from stdlib into Nimble packages:
- [``basic2d``](https://github.com/nim-lang/basic2d)
_deprecated: use ``glm``, ``arraymancer``, ``neo``, or another package instead_
- [``basic3d``](https://github.com/nim-lang/basic3d)
_deprecated: use ``glm``, ``arraymancer``, ``neo``, or another package instead_
- [``gentabs``](https://github.com/lcrees/gentabs)
- [``libuv``](https://github.com/lcrees/libuv)
- [``numeric``](https://github.com/lcrees/polynumeric)
- [``poly``](https://github.com/lcrees/polynumeric)
- [``pdcurses``](https://github.com/lcrees/pdcurses)
- [``romans``](https://github.com/lcrees/romans)
### Library additions
- Added ``system.runnableExamples`` to make examples in Nim's documentation easier
to write and test. The examples are tested as the last step of
``nim doc``.
- Nim's ``rst2html`` command now supports the testing of code snippets via an RST
extension that we called ``:test:``::
- ``re.split`` now also supports the ``maxsplit`` parameter for consistency
with ``strutils.split``.
- Added ``system.toOpenArray`` in order to support zero-copy slicing
operations. This is currently not yet available for the JavaScript target.
- Added ``getCurrentDir``, ``findExe``, ``cpDir`` and ``mvDir`` procs to
``nimscript``.
- The ``times`` module now supports up to nanosecond time resolution when available.
- Added the type ``times.Duration`` for representing fixed durations of time.
- Added the proc ``times.convert`` for converting between different time units,
e.g days to seconds.
- Added the proc ``algorithm.binarySearch[T, K]`` with the ```cmp``` parameter.
- Added the proc ``algorithm.upperBound``.
- Added inverse hyperbolic functions, ``math.arcsinh``, ``math.arccosh`` and ``math.arctanh`` procs.
- Added cotangent, secant and cosecant procs ``math.cot``, ``math.sec`` and ``math.csc``; and their hyperbolic, inverse and inverse hyperbolic functions, ``math.coth``, ``math.sech``, ``math.csch``, ``math.arccot``, ``math.arcsec``, ``math.arccsc``, ``math.arccoth``, ``math.arcsech`` and ``math.arccsch`` procs.
- Added the procs ``math.floorMod`` and ``math.floorDiv`` for floor based integer division.
- Added the procs ``rationals.`div```, ``rationals.`mod```, ``rationals.floorDiv`` and ``rationals.floorMod`` for rationals.
- Added the proc ``math.prod`` for product of elements in openArray.
```rst
.. code-block:: nim
:test:
# shows how the 'if' statement works
if true: echo "yes"
```
- The ``[]`` proc for strings now raises an ``IndexError`` exception when
the specified slice is out of bounds. See issue
[#6223](https://github.com/nim-lang/Nim/issues/6223) for more details.
You can use ``substr(str, start, finish)`` to get the old behaviour back,
see [this commit](https://github.com/nim-lang/nimbot/commit/98cc031a27ea89947daa7f0bb536bcf86462941f) for an example.
- ``strutils.split`` and ``strutils.rsplit`` with an empty string and a
separator now returns that empty string.
See issue [#4377](https://github.com/nim-lang/Nim/issues/4377).
- The experimental overloading of the dot ``.`` operators now take
an ``untyped``` parameter as the field name, it used to be
a ``static[string]``. You can use ``when defined(nimNewDot)`` to make
your code work with both old and new Nim versions.
See [special-operators](https://nim-lang.org/docs/manual.html#special-operators)
for more information.
- Added ``macros.unpackVarargs``.
- The memory manager now uses a variant of the TLSF algorithm that has much
better memory fragmentation behaviour. According
to [http://www.gii.upv.es/tlsf/](http://www.gii.upv.es/tlsf/) the maximum
fragmentation measured is lower than 25%. As a nice bonus ``alloc`` and
``dealloc`` became O(1) operations.
- The behavior of ``$`` has been changed for all standard library collections. The
collection-to-string implementations now perform proper quoting and escaping of
strings and chars.
- The ``random`` procs in ``random.nim`` have all been deprecated. Instead use
the new ``rand`` procs. The module now exports the state of the random
number generator as type ``Rand`` so multiple threads can easily use their
own random number generators that do not require locking. For more information
about this rename see issue [#6934](https://github.com/nim-lang/Nim/issues/6934)
- The compiler is now more consistent in its treatment of ambiguous symbols:
Types that shadow procs and vice versa are marked as ambiguous (bug #6693).
- ``yield`` (or ``await`` which is mapped to ``yield``) never worked reliably
in an array, seq or object constructor and is now prevented at compile-time.
- For string formatting / interpolation a new module
called [strformat](https://nim-lang.org/docs/strformat.html) has been added
to the stdlib.
- codegenDecl pragma now works for the JavaScript backend. It returns an empty string for
function return type placeholders.
- Asynchronous programming for the JavaScript backend using the `asyncjs` module.
- Extra semantic checks for procs with noreturn pragma: return type is not allowed,
statements after call to noreturn procs are no longer allowed.
- Noreturn proc calls and raising exceptions branches are now skipped during common type
deduction in if and case expressions. The following code snippets now compile:
```nim
import strutils
let str = "Y"
let a = case str:
of "Y": true
of "N": false
else: raise newException(ValueError, "Invalid boolean")
let b = case str:
of nil, "": raise newException(ValueError, "Invalid boolean")
elif str.startsWith("Y"): true
elif str.startsWith("N"): false
else: false
let c = if str == "Y": true
elif str == "N": false
else:
echo "invalid bool"
quit("this is the end")
```
- Proc [toCountTable](https://nim-lang.org/docs/tables.html#toCountTable,openArray[A]) now produces a `CountTable` with values correspoding to the number of occurrences of the key in the input. It used to produce a table with all values set to `1`.
### Library changes
Counting occurrences in a sequence used to be:
- ``macros.astGenRepr``, ``macros.lispRepr`` and ``macros.treeRepr``
now escapes the content of string literals consistently.
- ``macros.NimSym`` and ``macros.NimIdent`` is now deprecated in favor
of the more general ``NimNode``.
- ``macros.getImpl`` now includes the pragmas of types, instead of omitting them.
- ``macros.hasCustomPragma`` and ``macros.getCustomPragmaVal`` now
also support ``ref`` and ``ptr`` types, pragmas on types and variant
fields.
- ``system.SomeReal`` is now called ``SomeFloat`` for consistency and
correctness.
- ``algorithm.smartBinarySearch`` and ``algorithm.binarySearch`` is
now joined in ``binarySearch``. ``smartbinarySearch`` is now
deprecated.
- The `terminal` module now exports additional procs for generating ANSI color
codes as strings.
- Added the parameter ``val`` for the ``CritBitTree[int].inc`` proc.
- An exception raised from ``test`` block of ``unittest`` now shows its type in
the error message
- The proc ``tgamma`` was renamed to ``gamma``. ``tgamma`` is deprecated.
```nim
let mySeq = @[1, 2, 1, 3, 1, 4]
var myCounter = initCountTable[int]()
### Language additions
for item in mySeq:
myCounter.inc item
```
- Dot calls combined with explicit generic instantiations can now be written
as ``x.y[:z]`` which is transformed into ``y[z](x)`` by the parser.
- ``func`` is now an alias for ``proc {.noSideEffect.}``.
- In order to make ``for`` loops and iterators more flexible to use Nim now
supports so called "for-loop macros". See
the `manual <manual.html#macros-for-loop-macros>`_ for more details.
Now, you can simply do:
### Language changes
```nim
let
mySeq = @[1, 2, 1, 3, 1, 4]
myCounter = mySeq.toCountTable()
```
- The `importcpp` pragma now allows importing the listed fields of generic
C++ types. Support for numeric parameters have also been added through
the use of `static[T]` types.
(#6415)
- Added support for casting between integers of same bitsize in VM (compile time and nimscript).
This allow to among other things to reinterpret signed integers as unsigned.
- Native C++ exceptions can now be imported with `importcpp` pragma.
Imported exceptions can be raised and caught just like Nim exceptions.
More details in language manual.
- ``nil`` for strings/seqs is finally gone. Instead the default value for
these is ``"" / @[]``.
- Accessing the binary zero terminator in Nim's native strings
is now invalid. Internally a Nim string still has the trailing zero for
zero-copy interoperability with ``cstring``. Compile your code with the
new switch ``--laxStrings:on`` if you need a transition period.
- The command syntax now supports keyword arguments after the first comma.
- Thread-local variables can now be declared inside procs. This implies all
the effects of the `global` pragma.
- Nim now supports `except` clause in the export statement.
### Tool changes
- ``jsondoc2`` has been renamed ``jsondoc``, similar to how ``doc2`` was renamed
``doc``. The old ``jsondoc`` can still be invoked with ``jsondoc0``.
### Compiler changes
- The VM's instruction count limit was raised to 1 billion instructions in
order to support more complex computations at compile-time.
- Support for hot code reloading has been implemented for the JavaScript
target. To use it, compile your code with `--hotCodeReloading:on` and use a
helper library such as LiveReload or BrowserSync.
- A new compiler option `--cppCompileToNamespace` puts the generated C++ code
into the namespace "Nim" in order to avoid naming conflicts with existing
C++ code. This is done for all Nim code - internal and exported.
- Added ``macros.getProjectPath`` and ``ospaths.putEnv`` procs to Nim's virtual
machine.
- The ``deadCodeElim`` option is now always turned on and the switch has no
effect anymore, but is recognized for backwards compatibility.
- ``experimental`` is now a pragma / command line switch that can enable specific
language extensions, it is not an all-or-nothing switch anymore.
### Bugfixes

2
ci/build.sh
View file

@ -6,7 +6,7 @@ cd csources
sh build.sh
cd ..
# Add Nim to the PATH
export PATH=$(pwd)/bin:$PATH
export PATH=$(pwd)/bin${PATH:+:$PATH}
# Bootstrap.
nim -v
nim c koch

2
ci/deps.sh
View file

@ -7,7 +7,7 @@ apt-get install -y -qq build-essential git libcurl4-openssl-dev libsdl1.2-dev li
gcc -v
export PATH=$(pwd)/bin:$PATH
export PATH=$(pwd)/bin${PATH:+:$PATH}
# Nimble deps
nim e install_nimble.nims

6
compiler/aliases.nim
View file

@ -34,10 +34,10 @@ proc isPartOfAux(n: PNode, b: PType, marker: var IntSet): TAnalysisResult =
of nkOfBranch, nkElse:
result = isPartOfAux(lastSon(n.sons[i]), b, marker)
if result == arYes: return
else: internalError("isPartOfAux(record case branch)")
else: discard "isPartOfAux(record case branch)"
of nkSym:
result = isPartOfAux(n.sym.typ, b, marker)
else: internalError(n.info, "isPartOfAux()")
else: discard
proc isPartOfAux(a, b: PType, marker: var IntSet): TAnalysisResult =
result = arNo
@ -49,7 +49,7 @@ proc isPartOfAux(a, b: PType, marker: var IntSet): TAnalysisResult =
if a.sons[0] != nil:
result = isPartOfAux(a.sons[0].skipTypes(skipPtrs), b, marker)
if result == arNo: result = isPartOfAux(a.n, b, marker)
of tyGenericInst, tyDistinct, tyAlias:
of tyGenericInst, tyDistinct, tyAlias, tySink:
result = isPartOfAux(lastSon(a), b, marker)
of tyArray, tySet, tyTuple:
for i in countup(0, sonsLen(a) - 1):

128
compiler/ast.nim
View file

@ -10,7 +10,7 @@
# abstract syntax tree + symbol table
import
msgs, hashes, nversion, options, strutils, securehash, ropes, idents,
msgs, hashes, nversion, options, strutils, std / sha1, ropes, idents,
intsets, idgen
type
@ -221,7 +221,8 @@ type
nkGotoState, # used for the state machine (for iterators)
nkState, # give a label to a code section (for iterators)
nkBreakState, # special break statement for easier code generation
nkFuncDef # a func
nkFuncDef, # a func
nkTupleConstr # a tuple constructor
TNodeKinds* = set[TNodeKind]
@ -261,7 +262,8 @@ type
# variable is a thread variable
sfCompileTime, # proc can be evaluated at compile time
sfConstructor, # proc is a C++ constructor
sfDeadCodeElim, # dead code elimination for the module is turned on
sfDispatcher, # copied method symbol is the dispatcher
# deprecated and unused, except for the con
sfBorrow, # proc is borrowed
sfInfixCall, # symbol needs infix call syntax in target language;
# for interfacing with C++, JS
@ -274,10 +276,9 @@ type
TSymFlags* = set[TSymFlag]
const
sfDispatcher* = sfDeadCodeElim # copied method symbol is the dispatcher
sfNoInit* = sfMainModule # don't generate code to init the variable
sfImmediate* = sfDeadCodeElim
sfImmediate* = sfDispatcher
# macro or template is immediately expanded
# without considering any possible overloads
sfAllUntyped* = sfVolatile # macro or template is immediately expanded \
@ -305,6 +306,7 @@ const
sfEscapes* = sfProcvar # param escapes
sfBase* = sfDiscriminant
sfIsSelf* = sfOverriden # param is 'self'
sfCustomPragma* = sfRegister # symbol is custom pragma template
const
# getting ready for the future expr/stmt merge
@ -354,7 +356,7 @@ type
tyInt, tyInt8, tyInt16, tyInt32, tyInt64, # signed integers
tyFloat, tyFloat32, tyFloat64, tyFloat128,
tyUInt, tyUInt8, tyUInt16, tyUInt32, tyUInt64,
tyOptAsRef, tyUnused1, tyUnused2,
tyOptAsRef, tySink, tyLent,
tyVarargs,
tyUnused,
tyProxy # used as errornous type (for idetools)
@ -631,16 +633,17 @@ type
mCpuEndian, mHostOS, mHostCPU, mBuildOS, mBuildCPU, mAppType,
mNaN, mInf, mNegInf,
mCompileOption, mCompileOptionArg,
mNLen, mNChild, mNSetChild, mNAdd, mNAddMultiple, mNDel, mNKind,
mNLen, mNChild, mNSetChild, mNAdd, mNAddMultiple, mNDel,
mNKind, mNSymKind
mNIntVal, mNFloatVal, mNSymbol, mNIdent, mNGetType, mNStrVal, mNSetIntVal,
mNSetFloatVal, mNSetSymbol, mNSetIdent, mNSetType, mNSetStrVal, mNLineInfo,
mNNewNimNode, mNCopyNimNode, mNCopyNimTree, mStrToIdent, mIdentToStr,
mNNewNimNode, mNCopyNimNode, mNCopyNimTree, mStrToIdent,
mNBindSym, mLocals, mNCallSite,
mEqIdent, mEqNimrodNode, mSameNodeType, mGetImpl,
mNHint, mNWarning, mNError,
mInstantiationInfo, mGetTypeInfo, mNGenSym,
mNimvm, mIntDefine, mStrDefine, mRunnableExamples,
mException
mException, mBuiltinType
# things that we can evaluate safely at compile time, even if not asked for it:
const
@ -731,7 +734,7 @@ type
locOther # location is something other
TLocFlag* = enum
lfIndirect, # backend introduced a pointer
lfFullExternalName, # only used when 'gCmd == cmdPretty': Indicates
lfFullExternalName, # only used when 'conf.cmd == cmdPretty': Indicates
# that the symbol has been imported via 'importc: "fullname"' and
# no format string.
lfNoDeepCopy, # no need for a deep copy
@ -846,6 +849,8 @@ type
constraint*: PNode # additional constraints like 'lit|result'; also
# misused for the codegenDecl pragma in the hope
# it won't cause problems
# for skModule the string literal to output for
# deprecated modules.
when defined(nimsuggest):
allUsages*: seq[TLineInfo]
@ -940,13 +945,13 @@ const
tyGenericParam}
StructuralEquivTypes*: TTypeKinds = {tyNil, tyTuple, tyArray,
tySet, tyRange, tyPtr, tyRef, tyVar, tySequence, tyProc, tyOpenArray,
tySet, tyRange, tyPtr, tyRef, tyVar, tyLent, tySequence, tyProc, tyOpenArray,
tyVarargs}
ConcreteTypes*: TTypeKinds = { # types of the expr that may occur in::
# var x = expr
tyBool, tyChar, tyEnum, tyArray, tyObject,
tySet, tyTuple, tyRange, tyPtr, tyRef, tyVar, tySequence, tyProc,
tySet, tyTuple, tyRange, tyPtr, tyRef, tyVar, tyLent, tySequence, tyProc,
tyPointer,
tyOpenArray, tyString, tyCString, tyInt..tyInt64, tyFloat..tyFloat128,
tyUInt..tyUInt64}
@ -978,7 +983,9 @@ const
nkIdentKinds* = {nkIdent, nkSym, nkAccQuoted, nkOpenSymChoice,
nkClosedSymChoice}
nkPragmaCallKinds* = {nkExprColonExpr, nkCall, nkCallStrLit}
nkLiterals* = {nkCharLit..nkTripleStrLit}
nkFloatLiterals* = {nkFloatLit..nkFloat128Lit}
nkLambdaKinds* = {nkLambda, nkDo}
declarativeDefs* = {nkProcDef, nkFuncDef, nkMethodDef, nkIteratorDef, nkConverterDef}
procDefs* = nkLambdaKinds + declarativeDefs
@ -1035,9 +1042,9 @@ proc newNode*(kind: TNodeKind): PNode =
new(result)
result.kind = kind
#result.info = UnknownLineInfo() inlined:
result.info.fileIndex = int32(-1)
result.info.fileIndex = InvalidFileIdx
result.info.col = int16(-1)
result.info.line = int16(-1)
result.info.line = uint16(0)
when defined(useNodeIds):
result.id = gNodeId
if result.id == nodeIdToDebug:
@ -1071,14 +1078,14 @@ template previouslyInferred*(t: PType): PType =
if t.sons.len > 1: t.lastSon else: nil
proc newSym*(symKind: TSymKind, name: PIdent, owner: PSym,
info: TLineInfo): PSym =
info: TLineInfo; options: TOptions = {}): PSym =
# generates a symbol and initializes the hash field too
new(result)
result.name = name
result.kind = symKind
result.flags = {}
result.info = info
result.options = gOptions
result.options = options
result.owner = owner
result.offset = -1
result.id = getID()
@ -1088,7 +1095,7 @@ proc newSym*(symKind: TSymKind, name: PIdent, owner: PSym,
# writeStacktrace()
# MessageOut(name.s & " has id: " & toString(result.id))
var emptyNode* = newNode(nkEmpty)
var emptyNode* = newNode(nkEmpty) # XXX global variable here!
# There is a single empty node that is shared! Do not overwrite it!
proc isMetaType*(t: PType): bool =
@ -1109,13 +1116,13 @@ proc linkTo*(s: PSym, t: PType): PSym {.discardable.} =
s.typ = t
result = s
template fileIdx*(c: PSym): int32 =
template fileIdx*(c: PSym): FileIndex =
# XXX: this should be used only on module symbols
c.position.int32
c.position.FileIndex
template filename*(c: PSym): string =
# XXX: this should be used only on module symbols
c.position.int32.toFilename
c.position.FileIndex.toFilename
proc appendToModule*(m: PSym, n: PNode) =
## The compiler will use this internally to add nodes that will be
@ -1318,7 +1325,7 @@ proc copyType*(t: PType, owner: PSym, keepId: bool): PType =
proc exactReplica*(t: PType): PType = copyType(t, t.owner, true)
proc copySym*(s: PSym, keepId: bool = false): PSym =
result = newSym(s.kind, s.name, s.owner, s.info)
result = newSym(s.kind, s.name, s.owner, s.info, s.options)
#result.ast = nil # BUGFIX; was: s.ast which made problems
result.typ = s.typ
if keepId:
@ -1337,8 +1344,9 @@ proc copySym*(s: PSym, keepId: bool = false): PSym =
if result.kind in {skVar, skLet, skField}:
result.guard = s.guard
proc createModuleAlias*(s: PSym, newIdent: PIdent, info: TLineInfo): PSym =
result = newSym(s.kind, newIdent, s.owner, info)
proc createModuleAlias*(s: PSym, newIdent: PIdent, info: TLineInfo;
options: TOptions): PSym =
result = newSym(s.kind, newIdent, s.owner, info, options)
# keep ID!
result.ast = s.ast
result.id = s.id
@ -1427,7 +1435,7 @@ proc propagateToOwner*(owner, elem: PType) =
owner.flags.incl tfHasMeta
if tfHasAsgn in elem.flags:
let o2 = owner.skipTypes({tyGenericInst, tyAlias})
let o2 = owner.skipTypes({tyGenericInst, tyAlias, tySink})
if o2.kind in {tyTuple, tyObject, tyArray,
tySequence, tyOpt, tySet, tyDistinct}:
o2.flags.incl tfHasAsgn
@ -1435,7 +1443,7 @@ proc propagateToOwner*(owner, elem: PType) =
if owner.kind notin {tyProc, tyGenericInst, tyGenericBody,
tyGenericInvocation, tyPtr}:
let elemB = elem.skipTypes({tyGenericInst, tyAlias})
let elemB = elem.skipTypes({tyGenericInst, tyAlias, tySink})
if elemB.isGCedMem or tfHasGCedMem in elemB.flags:
# for simplicity, we propagate this flag even to generics. We then
# ensure this doesn't bite us in sempass2.
@ -1474,7 +1482,7 @@ proc copyNode*(src: PNode): PNode =
echo "COMES FROM ", src.id
case src.kind
of nkCharLit..nkUInt64Lit: result.intVal = src.intVal
of nkFloatLit..nkFloat128Lit: result.floatVal = src.floatVal
of nkFloatLiterals: result.floatVal = src.floatVal
of nkSym: result.sym = src.sym
of nkIdent: result.ident = src.ident
of nkStrLit..nkTripleStrLit: result.strVal = src.strVal
@ -1493,7 +1501,7 @@ proc shallowCopy*(src: PNode): PNode =
echo "COMES FROM ", src.id
case src.kind
of nkCharLit..nkUInt64Lit: result.intVal = src.intVal
of nkFloatLit..nkFloat128Lit: result.floatVal = src.floatVal
of nkFloatLiterals: result.floatVal = src.floatVal
of nkSym: result.sym = src.sym
of nkIdent: result.ident = src.ident
of nkStrLit..nkTripleStrLit: result.strVal = src.strVal
@ -1513,7 +1521,7 @@ proc copyTree*(src: PNode): PNode =
echo "COMES FROM ", src.id
case src.kind
of nkCharLit..nkUInt64Lit: result.intVal = src.intVal
of nkFloatLit..nkFloat128Lit: result.floatVal = src.floatVal
of nkFloatLiterals: result.floatVal = src.floatVal
of nkSym: result.sym = src.sym
of nkIdent: result.ident = src.ident
of nkStrLit..nkTripleStrLit: result.strVal = src.strVal
@ -1557,15 +1565,17 @@ proc getInt*(a: PNode): BiggestInt =
case a.kind
of nkCharLit..nkUInt64Lit: result = a.intVal
else:
internalError(a.info, "getInt")
result = 0
#internalError(a.info, "getInt")
doAssert false, "getInt"
#result = 0
proc getFloat*(a: PNode): BiggestFloat =
case a.kind
of nkFloatLit..nkFloat128Lit: result = a.floatVal
of nkFloatLiterals: result = a.floatVal
else:
internalError(a.info, "getFloat")
result = 0.0
doAssert false, "getFloat"
#internalError(a.info, "getFloat")
#result = 0.0
proc getStr*(a: PNode): string =
case a.kind
@ -1574,16 +1584,18 @@ proc getStr*(a: PNode): string =
# let's hope this fixes more problems than it creates:
result = nil
else:
internalError(a.info, "getStr")
result = ""
doAssert false, "getStr"
#internalError(a.info, "getStr")
#result = ""
proc getStrOrChar*(a: PNode): string =
case a.kind
of nkStrLit..nkTripleStrLit: result = a.strVal
of nkCharLit..nkUInt64Lit: result = $chr(int(a.intVal))
else:
internalError(a.info, "getStrOrChar")
result = ""
doAssert false, "getStrOrChar"
#internalError(a.info, "getStrOrChar")
#result = ""
proc isGenericRoutine*(s: PSym): bool =
case s.kind
@ -1608,6 +1620,19 @@ proc originatingModule*(s: PSym): PSym =
proc isRoutine*(s: PSym): bool {.inline.} =
result = s.kind in skProcKinds
proc isCompileTimeProc*(s: PSym): bool {.inline.} =
result = s.kind == skMacro or
s.kind == skProc and sfCompileTime in s.flags
proc requiredParams*(s: PSym): int =
# Returns the number of required params (without default values)
# XXX: Perhaps we can store this in the `offset` field of the
# symbol instead?
for i in 1 ..< s.typ.len:
if s.typ.n[i].sym.ast != nil:
return i - 1
return s.typ.len - 1
proc hasPattern*(s: PSym): bool {.inline.} =
result = isRoutine(s) and s.ast.sons[patternPos].kind != nkEmpty
@ -1615,7 +1640,7 @@ iterator items*(n: PNode): PNode =
for i in 0..<n.safeLen: yield n.sons[i]
iterator pairs*(n: PNode): tuple[i: int, n: PNode] =
for i in 0..<n.len: yield (i, n.sons[i])
for i in 0..<n.safeLen: yield (i, n.sons[i])
proc isAtom*(n: PNode): bool {.inline.} =
result = n.kind >= nkNone and n.kind <= nkNilLit
@ -1655,6 +1680,33 @@ proc toObject*(typ: PType): PType =
if result.kind == tyRef:
result = result.lastSon
proc isException*(t: PType): bool =
# check if `y` is object type and it inherits from Exception
assert(t != nil)
if t.kind != tyObject:
return false
var base = t
while base != nil:
if base.sym != nil and base.sym.magic == mException:
return true
base = base.lastSon
return false
proc isImportedException*(t: PType; conf: ConfigRef): bool =
assert(t != nil)
if optNoCppExceptions in conf.globalOptions:
return false
let base = t.skipTypes({tyAlias, tyPtr, tyDistinct, tyGenericInst})
if base.sym != nil and sfCompileToCpp in base.sym.flags:
result = true
proc isInfixAs*(n: PNode): bool =
return n.kind == nkInfix and n[0].kind == nkIdent and n[0].ident.id == getIdent("as").id
proc findUnresolvedStatic*(n: PNode): PNode =
if n.kind == nkSym and n.typ.kind == tyStatic and n.typ.n == nil:
return n

24
compiler/astalgo.nim
View file

@ -69,22 +69,22 @@ proc debug*(n: PNode) {.deprecated.}
template mdbg*: bool {.dirty.} =
when compiles(c.module):
c.module.fileIdx == gProjectMainIdx
c.module.fileIdx.int32 == c.config.projectMainIdx
elif compiles(c.c.module):
c.c.module.fileIdx == gProjectMainIdx
c.c.module.fileIdx.int32 == c.c.config.projectMainIdx
elif compiles(m.c.module):
m.c.module.fileIdx == gProjectMainIdx
m.c.module.fileIdx.int32 == m.c.config.projectMainIdx
elif compiles(cl.c.module):
cl.c.module.fileIdx == gProjectMainIdx
cl.c.module.fileIdx.int32 == cl.c.config.projectMainIdx
elif compiles(p):
when compiles(p.lex):
p.lex.fileIdx == gProjectMainIdx
p.lex.fileIdx.int32 == p.lex.config.projectMainIdx
else:
p.module.module.fileIdx == gProjectMainIdx
p.module.module.fileIdx.int32 == p.config.projectMainIdx
elif compiles(m.module.fileIdx):
m.module.fileIdx == gProjectMainIdx
m.module.fileIdx.int32 == m.config.projectMainIdx
elif compiles(L.fileIdx):
L.fileIdx == gProjectMainIdx
L.fileIdx.int32 == L.config.projectMainIdx
else:
error()
@ -180,7 +180,7 @@ proc lookupInRecord(n: PNode, field: PIdent): PSym =
result = lookupInRecord(n.sons[i], field)
if result != nil: return
of nkRecCase:
if (n.sons[0].kind != nkSym): internalError(n.info, "lookupInRecord")
if (n.sons[0].kind != nkSym): return nil
result = lookupInRecord(n.sons[0], field)
if result != nil: return
for i in countup(1, sonsLen(n) - 1):
@ -188,10 +188,10 @@ proc lookupInRecord(n: PNode, field: PIdent): PSym =
of nkOfBranch, nkElse:
result = lookupInRecord(lastSon(n.sons[i]), field)
if result != nil: return
else: internalError(n.info, "lookupInRecord(record case branch)")
else: return nil
of nkSym:
if n.sym.name.id == field.id: result = n.sym
else: internalError(n.info, "lookupInRecord()")
else: return nil
proc getModule(s: PSym): PSym =
result = s
@ -203,7 +203,7 @@ proc getSymFromList(list: PNode, ident: PIdent, start: int = 0): PSym =
if list.sons[i].kind == nkSym:
result = list.sons[i].sym
if result.name.id == ident.id: return
else: internalError(list.info, "getSymFromList")
else: return nil
result = nil
proc hashNode(p: RootRef): Hash =

26
compiler/bitsets.nim
View file

@ -28,6 +28,7 @@ proc bitSetExcl*(x: var TBitSet, elem: BiggestInt)
proc bitSetIn*(x: TBitSet, e: BiggestInt): bool
proc bitSetEquals*(x, y: TBitSet): bool
proc bitSetContains*(x, y: TBitSet): bool
proc bitSetCard*(x: TBitSet): BiggestInt
# implementation
proc bitSetIn(x: TBitSet, e: BiggestInt): bool =
@ -69,3 +70,28 @@ proc bitSetContains(x, y: TBitSet): bool =
if (x[i] and not y[i]) != int8(0):
return false
result = true
# Number of set bits for all values of int8
const populationCount: array[low(int8)..high(int8), int8] = block:
var arr: array[low(int8)..high(int8), int8]
proc countSetBits(x: int8): int8 =
return
( x and 0b00000001'i8) +
((x and 0b00000010'i8) shr 1) +
((x and 0b00000100'i8) shr 2) +
((x and 0b00001000'i8) shr 3) +
((x and 0b00010000'i8) shr 4) +
((x and 0b00100000'i8) shr 5) +
((x and 0b01000000'i8) shr 6) +
((x and 0b10000000'i8) shr 7)
for it in low(int8)..high(int8):
arr[it] = countSetBits(it)
arr
proc bitSetCard(x: TBitSet): BiggestInt =
for it in x:
result.inc populationCount[it]

72
compiler/ccgcalls.nim
View file

@ -32,7 +32,7 @@ proc fixupCall(p: BProc, le, ri: PNode, d: var TLoc,
if d.k == locNone: getTemp(p, typ.sons[0], d, needsInit=true)
elif d.k notin {locTemp} and not hasNoInit(ri):
# reset before pass as 'result' var:
resetLoc(p, d)
discard "resetLoc(p, d)"
add(pl, addrLoc(d))
add(pl, ~");$n")
line(p, cpsStmts, pl)
@ -71,7 +71,7 @@ proc isInCurrentFrame(p: BProc, n: PNode): bool =
if n.sym.kind in {skVar, skResult, skTemp, skLet} and p.prc != nil:
result = p.prc.id == n.sym.owner.id
of nkDotExpr, nkBracketExpr:
if skipTypes(n.sons[0].typ, abstractInst).kind notin {tyVar,tyPtr,tyRef}:
if skipTypes(n.sons[0].typ, abstractInst).kind notin {tyVar,tyLent,tyPtr,tyRef}:
result = isInCurrentFrame(p, n.sons[0])
of nkHiddenStdConv, nkHiddenSubConv, nkConv:
result = isInCurrentFrame(p, n.sons[1])
@ -83,6 +83,8 @@ proc isInCurrentFrame(p: BProc, n: PNode): bool =
result = isInCurrentFrame(p, n.sons[0])
else: discard
proc genIndexCheck(p: BProc; arr, idx: TLoc)
proc openArrayLoc(p: BProc, n: PNode): Rope =
var a: TLoc
@ -93,18 +95,28 @@ proc openArrayLoc(p: BProc, n: PNode): Rope =
initLocExpr(p, q[1], a)
initLocExpr(p, q[2], b)
initLocExpr(p, q[3], c)
let fmt =
case skipTypes(a.t, abstractVar+{tyPtr}).kind
of tyOpenArray, tyVarargs, tyArray:
"($1)+($2), ($3)-($2)+1"
of tyString, tySequence:
if skipTypes(n.typ, abstractInst).kind == tyVar and
not compileToCpp(p.module):
"(*$1)->data+($2), ($3)-($2)+1"
else:
"$1->data+($2), ($3)-($2)+1"
else: (internalError("openArrayLoc: " & typeToString(a.t)); "")
result = fmt % [rdLoc(a), rdLoc(b), rdLoc(c)]
# but first produce the required index checks:
if optBoundsCheck in p.options:
genIndexCheck(p, a, b)
genIndexCheck(p, a, c)
let ty = skipTypes(a.t, abstractVar+{tyPtr})
case ty.kind
of tyArray:
let first = firstOrd(ty)
if first == 0:
result = "($1)+($2), ($3)-($2)+1" % [rdLoc(a), rdLoc(b), rdLoc(c)]
else:
result = "($1)+(($2)-($4)), ($3)-($2)+1" % [rdLoc(a), rdLoc(b), rdLoc(c), intLiteral(first)]
of tyOpenArray, tyVarargs:
result = "($1)+($2), ($3)-($2)+1" % [rdLoc(a), rdLoc(b), rdLoc(c)]
of tyString, tySequence:
if skipTypes(n.typ, abstractInst).kind == tyVar and
not compileToCpp(p.module):
result = "(*$1)->data+($2), ($3)-($2)+1" % [rdLoc(a), rdLoc(b), rdLoc(c)]
else:
result = "$1->data+($2), ($3)-($2)+1" % [rdLoc(a), rdLoc(b), rdLoc(c)]
else:
internalError(p.config, "openArrayLoc: " & typeToString(a.t))
else:
initLocExpr(p, n, a)
case skipTypes(a.t, abstractVar).kind
@ -113,25 +125,25 @@ proc openArrayLoc(p: BProc, n: PNode): Rope =
of tyString, tySequence:
if skipTypes(n.typ, abstractInst).kind == tyVar and
not compileToCpp(p.module):
result = "(*$1)->data, (*$1)->$2" % [a.rdLoc, lenField(p)]
result = "(*$1)->data, (*$1 ? (*$1)->$2 : 0)" % [a.rdLoc, lenField(p)]
else:
result = "$1->data, $1->$2" % [a.rdLoc, lenField(p)]
result = "$1->data, ($1 ? $1->$2 : 0)" % [a.rdLoc, lenField(p)]
of tyArray:
result = "$1, $2" % [rdLoc(a), rope(lengthOrd(a.t))]
of tyPtr, tyRef:
case lastSon(a.t).kind
of tyString, tySequence:
result = "(*$1)->data, (*$1)->$2" % [a.rdLoc, lenField(p)]
result = "(*$1)->data, (*$1 ? (*$1)->$2 : 0)" % [a.rdLoc, lenField(p)]
of tyArray:
result = "$1, $2" % [rdLoc(a), rope(lengthOrd(lastSon(a.t)))]
else:
internalError("openArrayLoc: " & typeToString(a.t))
else: internalError("openArrayLoc: " & typeToString(a.t))
internalError(p.config, "openArrayLoc: " & typeToString(a.t))
else: internalError(p.config, "openArrayLoc: " & typeToString(a.t))
proc genArgStringToCString(p: BProc, n: PNode): Rope {.inline.} =
var a: TLoc
initLocExpr(p, n.sons[0], a)
result = "$1->data" % [a.rdLoc]
result = "($1 ? $1->data : (NCSTRING)\"\")" % [a.rdLoc]
proc genArg(p: BProc, n: PNode, param: PSym; call: PNode): Rope =
var a: TLoc
@ -228,7 +240,7 @@ proc genClosureCall(p: BProc, le, ri: PNode, d: var TLoc) =
getTemp(p, typ.sons[0], d, needsInit=true)
elif d.k notin {locTemp} and not hasNoInit(ri):
# reset before pass as 'result' var:
resetLoc(p, d)
discard "resetLoc(p, d)"
add(pl, addrLoc(d))
genCallPattern()
else:
@ -261,7 +273,7 @@ proc genOtherArg(p: BProc; ri: PNode; i: int; typ: PType): Rope =
result = genArgNoParam(p, ri.sons[i]) #, typ.n.sons[i].sym)
else:
if tfVarargs notin typ.flags:
localError(ri.info, "wrong argument count")
localError(p.config, ri.info, "wrong argument count")
result = nil
else:
result = genArgNoParam(p, ri.sons[i])
@ -325,13 +337,13 @@ proc genThisArg(p: BProc; ri: PNode; i: int; typ: PType): Rope =
# for better or worse c2nim translates the 'this' argument to a 'var T'.
# However manual wrappers may also use 'ptr T'. In any case we support both
# for convenience.
internalAssert i < sonsLen(typ)
internalAssert p.config, i < sonsLen(typ)
assert(typ.n.sons[i].kind == nkSym)
# if the parameter is lying (tyVar) and thus we required an additional deref,
# skip the deref:
var ri = ri[i]
while ri.kind == nkObjDownConv: ri = ri[0]
let t = typ.sons[i].skipTypes({tyGenericInst, tyAlias})
let t = typ.sons[i].skipTypes({tyGenericInst, tyAlias, tySink})
if t.kind == tyVar:
let x = if ri.kind == nkHiddenAddr: ri[0] else: ri
if x.typ.kind == tyPtr:
@ -382,7 +394,7 @@ proc genPatternCall(p: BProc; ri: PNode; pat: string; typ: PType): Rope =
result.add genOtherArg(p, ri, k, typ)
result.add(~")")
else:
localError(ri.info, "call expression expected for C++ pattern")
localError(p.config, ri.info, "call expression expected for C++ pattern")
inc i
elif pat[i+1] == '.':
result.add genThisArg(p, ri, j, typ)
@ -420,7 +432,7 @@ proc genInfixCall(p: BProc, le, ri: PNode, d: var TLoc) =
assert(sonsLen(typ) == sonsLen(typ.n))
# don't call '$' here for efficiency:
let pat = ri.sons[0].sym.loc.r.data
internalAssert pat != nil
internalAssert p.config, pat != nil
if pat.contains({'#', '(', '@', '\''}):
var pl = genPatternCall(p, ri, pat, typ)
# simpler version of 'fixupCall' that works with the pl+params combination:
@ -469,7 +481,7 @@ proc genNamedParamCall(p: BProc, ri: PNode, d: var TLoc) =
# don't call '$' here for efficiency:
let pat = ri.sons[0].sym.loc.r.data
internalAssert pat != nil
internalAssert p.config, pat != nil
var start = 3
if ' ' in pat:
start = 1
@ -489,7 +501,7 @@ proc genNamedParamCall(p: BProc, ri: PNode, d: var TLoc) =
for i in countup(start, length-1):
assert(sonsLen(typ) == sonsLen(typ.n))
if i >= sonsLen(typ):
internalError(ri.info, "varargs for objective C method?")
internalError(p.config, ri.info, "varargs for objective C method?")
assert(typ.n.sons[i].kind == nkSym)
var param = typ.n.sons[i].sym
add(pl, ~" ")
@ -527,7 +539,7 @@ proc genNamedParamCall(p: BProc, ri: PNode, d: var TLoc) =
line(p, cpsStmts, pl)
proc genCall(p: BProc, e: PNode, d: var TLoc) =
if e.sons[0].typ.skipTypes({tyGenericInst, tyAlias}).callConv == ccClosure:
if e.sons[0].typ.skipTypes({tyGenericInst, tyAlias, tySink}).callConv == ccClosure:
genClosureCall(p, nil, e, d)
elif e.sons[0].kind == nkSym and sfInfixCall in e.sons[0].sym.flags:
genInfixCall(p, nil, e, d)
@ -538,7 +550,7 @@ proc genCall(p: BProc, e: PNode, d: var TLoc) =
postStmtActions(p)
proc genAsgnCall(p: BProc, le, ri: PNode, d: var TLoc) =
if ri.sons[0].typ.skipTypes({tyGenericInst, tyAlias}).callConv == ccClosure:
if ri.sons[0].typ.skipTypes({tyGenericInst, tyAlias, tySink}).callConv == ccClosure:
genClosureCall(p, le, ri, d)
elif ri.sons[0].kind == nkSym and sfInfixCall in ri.sons[0].sym.flags:
genInfixCall(p, le, ri, d)

432
compiler/ccgexprs.nim
View file

File diff suppressed because it is too large Load diff

91
compiler/ccgliterals.nim Normal file
View file

@ -0,0 +1,91 @@
#
#
# The Nim Compiler
# (c) Copyright 2018 Andreas Rumpf
#
# See the file "copying.txt", included in this
# distribution, for details about the copyright.
#
## This include file contains the logic to produce constant string
## and seq literals. The code here is responsible that
## ``const x = ["a", "b"]`` works without hidden runtime creation code.
## The price is that seqs and strings are not purely a library
## implementation.
template detectVersion(field, corename) =
if m.g.field == 0:
let core = getCompilerProc(m.g.graph, corename)
if core == nil or core.kind != skConst:
m.g.field = 1
else:
m.g.field = int ast.getInt(core.ast)
result = m.g.field
proc detectStrVersion(m: BModule): int =
detectVersion(strVersion, "nimStrVersion")
proc detectSeqVersion(m: BModule): int =
detectVersion(seqVersion, "nimSeqVersion")
# ----- Version 1: GC'ed strings and seqs --------------------------------
proc genStringLiteralDataOnlyV1(m: BModule, s: string): Rope =
discard cgsym(m, "TGenericSeq")
result = getTempName(m)
addf(m.s[cfsData], "STRING_LITERAL($1, $2, $3);$n",
[result, makeCString(s), rope(len(s))])
proc genStringLiteralV1(m: BModule; n: PNode): Rope =
if s.isNil:
result = ropecg(m, "((#NimStringDesc*) NIM_NIL)", [])
else:
let id = nodeTableTestOrSet(m.dataCache, n, m.labels)
if id == m.labels:
# string literal not found in the cache:
result = ropecg(m, "((#NimStringDesc*) &$1)",
[genStringLiteralDataOnlyV1(m, n.strVal)])
else:
result = ropecg(m, "((#NimStringDesc*) &$1$2)",
[m.tmpBase, rope(id)])
# ------ Version 2: destructor based strings and seqs -----------------------
proc genStringLiteralDataOnlyV2(m: BModule, s: string): Rope =
result = getTempName(m)
addf(m.s[cfsData], " static const NIM_CHAR $1[$2] = $3;$n",
[result, rope(len(s)+1), makeCString(s)])
proc genStringLiteralV2(m: BModule; n: PNode): Rope =
let id = nodeTableTestOrSet(m.dataCache, n, m.labels)
if id == m.labels:
# string literal not found in the cache:
let pureLit = genStringLiteralDataOnlyV2(m, n.strVal)
result = getTempName(m)
addf(m.s[cfsData], "static const #NimStringV2 $1 = {$2, $2, $3};$n",
[result, rope(len(n.strVal)+1), pureLit])
else:
result = m.tmpBase & rope(id)
# ------ Version selector ---------------------------------------------------
proc genStringLiteralDataOnly(m: BModule; s: string; info: TLineInfo): Rope =
case detectStrVersion(m)
of 0, 1: result = genStringLiteralDataOnlyV1(m, s)
of 2: result = genStringLiteralDataOnlyV2(m, s)
else:
localError(m.config, info, "cannot determine how to produce code for string literal")
proc genStringLiteralFromData(m: BModule; data: Rope; info: TLineInfo): Rope =
result = ropecg(m, "((#NimStringDesc*) &$1)",
[data])
proc genNilStringLiteral(m: BModule; info: TLineInfo): Rope =
result = ropecg(m, "((#NimStringDesc*) NIM_NIL)", [])
proc genStringLiteral(m: BModule; n: PNode): Rope =
case detectStrVersion(m)
of 0, 1: result = genStringLiteralV1(m, n)
of 2: result = genStringLiteralV2(m, n)
else:
localError(m.config, n.info, "cannot determine how to produce code for string literal")

34
compiler/ccgmerge.nim
View file

@ -45,28 +45,28 @@ const
]
NimMergeEndMark = "/*\tNIM_merge_END:*/"
proc genSectionStart*(fs: TCFileSection): Rope =
if compilationCachePresent:
proc genSectionStart*(fs: TCFileSection; conf: ConfigRef): Rope =
if compilationCachePresent(conf):
result = rope(tnl)
add(result, "/*\t")
add(result, CFileSectionNames[fs])
add(result, ":*/")
add(result, tnl)
proc genSectionEnd*(fs: TCFileSection): Rope =
if compilationCachePresent:
proc genSectionEnd*(fs: TCFileSection; conf: ConfigRef): Rope =
if compilationCachePresent(conf):
result = rope(NimMergeEndMark & tnl)
proc genSectionStart*(ps: TCProcSection): Rope =
if compilationCachePresent:
proc genSectionStart*(ps: TCProcSection; conf: ConfigRef): Rope =
if compilationCachePresent(conf):
result = rope(tnl)
add(result, "/*\t")
add(result, CProcSectionNames[ps])
add(result, ":*/")
add(result, tnl)
proc genSectionEnd*(ps: TCProcSection): Rope =
if compilationCachePresent:
proc genSectionEnd*(ps: TCProcSection; conf: ConfigRef): Rope =
if compilationCachePresent(conf):
result = rope(NimMergeEndMark & tnl)
proc writeTypeCache(a: TypeCache, s: var string) =
@ -96,7 +96,7 @@ proc writeIntSet(a: IntSet, s: var string) =
s.add('}')
proc genMergeInfo*(m: BModule): Rope =
if not compilationCachePresent: return nil
if not compilationCachePresent(m.config): return nil
var s = "/*\tNIM_merge_INFO:"
s.add(tnl)
s.add("typeCache:{")
@ -161,7 +161,7 @@ proc readVerbatimSection(L: var TBaseLexer): Rope =
buf = L.buf
r.add(tnl)
of '\0':
internalError("ccgmerge: expected: " & NimMergeEndMark)
doAssert(false, "ccgmerge: expected: " & NimMergeEndMark)
break
else:
if atEndMark(buf, pos):
@ -179,7 +179,7 @@ proc readKey(L: var TBaseLexer, result: var string) =
while buf[pos] in IdentChars:
result.add(buf[pos])
inc pos
if buf[pos] != ':': internalError("ccgmerge: ':' expected")
if buf[pos] != ':': doAssert(false, "ccgmerge: ':' expected")
L.bufpos = pos + 1 # skip ':'
proc newFakeType(id: int): PType =
@ -187,12 +187,12 @@ proc newFakeType(id: int): PType =
result.id = id
proc readTypeCache(L: var TBaseLexer, result: var TypeCache) =
if ^L.bufpos != '{': internalError("ccgmerge: '{' expected")
if ^L.bufpos != '{': doAssert(false, "ccgmerge: '{' expected")
inc L.bufpos
while ^L.bufpos != '}':
skipWhite(L)
var key = decodeStr(L.buf, L.bufpos)
if ^L.bufpos != ':': internalError("ccgmerge: ':' expected")
if ^L.bufpos != ':': doAssert(false, "ccgmerge: ':' expected")
inc L.bufpos
var value = decodeStr(L.buf, L.bufpos)
# XXX implement me
@ -201,7 +201,7 @@ proc readTypeCache(L: var TBaseLexer, result: var TypeCache) =
inc L.bufpos
proc readIntSet(L: var TBaseLexer, result: var IntSet) =
if ^L.bufpos != '{': internalError("ccgmerge: '{' expected")
if ^L.bufpos != '{': doAssert(false, "ccgmerge: '{' expected")
inc L.bufpos
while ^L.bufpos != '}':
skipWhite(L)
@ -225,7 +225,7 @@ proc processMergeInfo(L: var TBaseLexer, m: BModule) =
of "labels": m.labels = decodeVInt(L.buf, L.bufpos)
of "flags":
m.flags = cast[set[CodegenFlag]](decodeVInt(L.buf, L.bufpos) != 0)
else: internalError("ccgmerge: unknown key: " & k)
else: doAssert(false, "ccgmerge: unknown key: " & k)
when not defined(nimhygiene):
{.pragma: inject.}
@ -275,9 +275,9 @@ proc readMergeSections(cfilename: string, m: var TMergeSections) =
if sectionB >= 0 and sectionB <= high(TCProcSection).int:
m.p[TCProcSection(sectionB)] = verbatim
else:
internalError("ccgmerge: unknown section: " & k)
doAssert(false, "ccgmerge: unknown section: " & k)
else:
internalError("ccgmerge: '*/' expected")
doAssert(false, "ccgmerge: '*/' expected")
proc mergeRequired*(m: BModule): bool =
for i in cfsHeaders..cfsProcs:

350
compiler/ccgstmts.nim
View file

@ -16,13 +16,17 @@ const
# above X strings a hash-switch for strings is generated
proc registerGcRoot(p: BProc, v: PSym) =
if gSelectedGC in {gcMarkAndSweep, gcGenerational, gcV2, gcRefc} and
if p.config.selectedGC in {gcMarkAndSweep, gcGenerational, gcV2, gcRefc} and
containsGarbageCollectedRef(v.loc.t):
# we register a specialized marked proc here; this has the advantage
# that it works out of the box for thread local storage then :-)
let prc = genTraverseProcForGlobal(p.module, v, v.info)
appcg(p.module, p.module.initProc.procSec(cpsInit),
"#nimRegisterGlobalMarker($1);$n", [prc])
if sfThread in v.flags:
appcg(p.module, p.module.initProc.procSec(cpsInit),
"#nimRegisterThreadLocalMarker($1);$n", [prc])
else:
appcg(p.module, p.module.initProc.procSec(cpsInit),
"#nimRegisterGlobalMarker($1);$n", [prc])
proc isAssignedImmediately(n: PNode): bool {.inline.} =
if n.kind == nkEmpty: return false
@ -33,15 +37,19 @@ proc isAssignedImmediately(n: PNode): bool {.inline.} =
return false
result = true
proc inExceptBlockLen(p: BProc): int =
for x in p.nestedTryStmts:
if x.inExcept: result.inc
proc genVarTuple(p: BProc, n: PNode) =
var tup, field: TLoc
if n.kind != nkVarTuple: internalError(n.info, "genVarTuple")
if n.kind != nkVarTuple: internalError(p.config, n.info, "genVarTuple")
var L = sonsLen(n)
# if we have a something that's been captured, use the lowering instead:
for i in countup(0, L-3):
if n[i].kind != nkSym:
genStmts(p, lowerTupleUnpacking(n, p.prc))
genStmts(p, lowerTupleUnpacking(p.module.g.graph, n, p.prc))
return
genLineDir(p, n)
@ -62,7 +70,7 @@ proc genVarTuple(p: BProc, n: PNode) =
if t.kind == tyTuple:
field.r = "$1.Field$2" % [rdLoc(tup), rope(i)]
else:
if t.n.sons[i].kind != nkSym: internalError(n.info, "genVarTuple")
if t.n.sons[i].kind != nkSym: internalError(p.config, n.info, "genVarTuple")
field.r = "$1.$2" % [rdLoc(tup), mangleRecFieldName(p.module, t.n.sons[i].sym, t)]
putLocIntoDest(p, v.loc, field)
@ -92,7 +100,7 @@ proc startBlock(p: BProc, start: FormatStr = "{$n",
setLen(p.blocks, result + 1)
p.blocks[result].id = p.labels
p.blocks[result].nestedTryStmts = p.nestedTryStmts.len.int16
p.blocks[result].nestedExceptStmts = p.inExceptBlock.int16
p.blocks[result].nestedExceptStmts = p.inExceptBlockLen.int16
proc assignLabel(b: var TBlock): Rope {.inline.} =
b.label = "LA" & b.id.rope
@ -117,7 +125,7 @@ proc endBlock(p: BProc, blockEnd: Rope) =
proc endBlock(p: BProc) =
let topBlock = p.blocks.len - 1
var blockEnd = if p.blocks[topBlock].label != nil:
rfmt(nil, "} $1: ;$n", p.blocks[topBlock].label)
ropecg(p.module, "} $1: ;$n", p.blocks[topBlock].label)
else:
~"}$n"
let frameLen = p.blocks[topBlock].frameLen
@ -141,7 +149,7 @@ template preserveBreakIdx(body: untyped): untyped =
p.breakIdx = oldBreakIdx
proc genState(p: BProc, n: PNode) =
internalAssert n.len == 1
internalAssert p.config, n.len == 1
let n0 = n[0]
if n0.kind == nkIntLit:
let idx = n.sons[0].intVal
@ -149,6 +157,39 @@ proc genState(p: BProc, n: PNode) =
elif n0.kind == nkStrLit:
linefmt(p, cpsStmts, "$1: ;$n", n0.strVal.rope)
proc blockLeaveActions(p: BProc, howManyTrys, howManyExcepts: int) =
# Called by return and break stmts.
# Deals with issues faced when jumping out of try/except/finally stmts,
var stack = newSeq[tuple[n: PNode, inExcept: bool]](0)
for i in countup(1, howManyTrys):
let tryStmt = p.nestedTryStmts.pop
if not p.module.compileToCpp or optNoCppExceptions in p.config.globalOptions:
# Pop safe points generated by try
if not tryStmt.inExcept:
linefmt(p, cpsStmts, "#popSafePoint();$n")
# Pop this try-stmt of the list of nested trys
# so we don't infinite recurse on it in the next step.
stack.add(tryStmt)
# Find finally-stmt for this try-stmt
# and generate a copy of its sons
var finallyStmt = lastSon(tryStmt.n)
if finallyStmt.kind == nkFinally:
genStmts(p, finallyStmt.sons[0])
# push old elements again:
for i in countdown(howManyTrys-1, 0):
p.nestedTryStmts.add(stack[i])
if not p.module.compileToCpp or optNoCppExceptions in p.config.globalOptions:
# Pop exceptions that was handled by the
# except-blocks we are in
for i in countdown(howManyExcepts-1, 0):
linefmt(p, cpsStmts, "#popCurrentException();$n")
proc genGotoState(p: BProc, n: PNode) =
# we resist the temptation to translate it into duff's device as it later
# will be translated into computed gotos anyway for GCC at least:
@ -159,7 +200,11 @@ proc genGotoState(p: BProc, n: PNode) =
initLocExpr(p, n.sons[0], a)
lineF(p, cpsStmts, "switch ($1) {$n", [rdLoc(a)])
p.beforeRetNeeded = true
lineF(p, cpsStmts, "case -1: goto BeforeRet_;$n", [])
lineF(p, cpsStmts, "case -1:$n", [])
blockLeaveActions(p,
howManyTrys = p.nestedTryStmts.len,
howManyExcepts = p.inExceptBlockLen)
lineF(p, cpsStmts, " goto BeforeRet_;$n", [])
var statesCounter = lastOrd(n.sons[0].typ)
if n.len >= 2 and n[1].kind == nkIntLit:
statesCounter = n[1].intVal
@ -169,23 +214,21 @@ proc genGotoState(p: BProc, n: PNode) =
lineF(p, cpsStmts, "case $2: goto $1$2;$n", [prefix, rope(i)])
lineF(p, cpsStmts, "}$n", [])
proc genBreakState(p: BProc, n: PNode) =
proc genBreakState(p: BProc, n: PNode, d: var TLoc) =
var a: TLoc
initLoc(d, locExpr, n, OnUnknown)
if n.sons[0].kind == nkClosure:
# XXX this produces quite inefficient code!
initLocExpr(p, n.sons[0].sons[1], a)
lineF(p, cpsStmts, "if (((NI*) $1)[1] < 0) break;$n", [rdLoc(a)])
d.r = "(((NI*) $1)[1] < 0)" % [rdLoc(a)]
else:
initLocExpr(p, n.sons[0], a)
# the environment is guaranteed to contain the 'state' field at offset 1:
lineF(p, cpsStmts, "if ((((NI*) $1.ClE_0)[1]) < 0) break;$n", [rdLoc(a)])
# lineF(p, cpsStmts, "if (($1) < 0) break;$n", [rdLoc(a)])
proc genVarPrototypeAux(m: BModule, n: PNode)
d.r = "((((NI*) $1.ClE_0)[1]) < 0)" % [rdLoc(a)]
proc genGotoVar(p: BProc; value: PNode) =
if value.kind notin {nkCharLit..nkUInt64Lit}:
localError(value.info, "'goto' target must be a literal value")
localError(p.config, value.info, "'goto' target must be a literal value")
else:
lineF(p, cpsStmts, "goto NIMSTATE_$#;$n", [value.intVal.rope])
@ -217,7 +260,7 @@ proc genSingleVar(p: BProc, a: PNode) =
# Alternative construction using default constructor (which may zeromem):
# if sfImportc notin v.flags: constructLoc(p.module.preInitProc, v.loc)
if sfExportc in v.flags and p.module.g.generatedHeader != nil:
genVarPrototypeAux(p.module.g.generatedHeader, vn)
genVarPrototype(p.module.g.generatedHeader, vn)
registerGcRoot(p, v)
else:
let value = a.sons[2]
@ -239,7 +282,10 @@ proc genSingleVar(p: BProc, a: PNode) =
if params != nil: params.add(~", ")
assert(sonsLen(typ) == sonsLen(typ.n))
add(params, genOtherArg(p, value, i, typ))
lineF(p, cpsStmts, "$#($#);$n", [decl, params])
if params == nil:
lineF(p, cpsStmts, "$#;$n", [decl])
else:
lineF(p, cpsStmts, "$#($#);$n", [decl, params])
else:
initLocExprSingleUse(p, value, tmp)
lineF(p, cpsStmts, "$# = $#;$n", [decl, tmp.rdLoc])
@ -260,28 +306,16 @@ proc genClosureVar(p: BProc, a: PNode) =
loadInto(p, a.sons[0], a.sons[2], v)
proc genVarStmt(p: BProc, n: PNode) =
for i in countup(0, sonsLen(n) - 1):
var a = n.sons[i]
if a.kind == nkCommentStmt: continue
if a.kind == nkIdentDefs:
# can be a lifted var nowadays ...
if a.sons[0].kind == nkSym:
genSingleVar(p, a)
else:
genClosureVar(p, a)
else:
genVarTuple(p, a)
proc genConstStmt(p: BProc, t: PNode) =
for i in countup(0, sonsLen(t) - 1):
var it = t.sons[i]
for it in n.sons:
if it.kind == nkCommentStmt: continue
if it.kind != nkConstDef: internalError(t.info, "genConstStmt")
var c = it.sons[0].sym
if c.typ.containsCompileTimeOnly: continue
elif c.typ.kind in ConstantDataTypes and lfNoDecl notin c.loc.flags and
c.ast.len != 0:
if not emitLazily(c): requestConstImpl(p, c)
if it.kind == nkIdentDefs:
# can be a lifted var nowadays ...
if it.sons[0].kind == nkSym:
genSingleVar(p, it)
else:
genClosureVar(p, it)
else:
genVarTuple(p, it)
proc genIf(p: BProc, n: PNode, d: var TLoc) =
#
@ -302,10 +336,9 @@ proc genIf(p: BProc, n: PNode, d: var TLoc) =
getTemp(p, n.typ, d)
genLineDir(p, n)
let lend = getLabel(p)
for i in countup(0, sonsLen(n) - 1):
for it in n.sons:
# bug #4230: avoid false sharing between branches:
if d.k == locTemp and isEmptyType(n.typ): d.k = locNone
let it = n.sons[i]
if it.len == 2:
when newScopeForIf: startBlock(p)
initLocExprSingleUse(p, it.sons[0], a)
@ -329,47 +362,9 @@ proc genIf(p: BProc, n: PNode, d: var TLoc) =
startBlock(p)
expr(p, it.sons[0], d)
endBlock(p)
else: internalError(n.info, "genIf()")
else: internalError(p.config, n.info, "genIf()")
if sonsLen(n) > 1: fixLabel(p, lend)
proc blockLeaveActions(p: BProc, howManyTrys, howManyExcepts: int) =
# Called by return and break stmts.
# Deals with issues faced when jumping out of try/except/finally stmts,
var stack: seq[PNode]
newSeq(stack, 0)
var alreadyPoppedCnt = p.inExceptBlock
for i in countup(1, howManyTrys):
if not p.module.compileToCpp or optNoCppExceptions in gGlobalOptions:
# Pop safe points generated by try
if alreadyPoppedCnt > 0:
dec alreadyPoppedCnt
else:
linefmt(p, cpsStmts, "#popSafePoint();$n")
# Pop this try-stmt of the list of nested trys
# so we don't infinite recurse on it in the next step.
var tryStmt = p.nestedTryStmts.pop
stack.add(tryStmt)
# Find finally-stmt for this try-stmt
# and generate a copy of its sons
var finallyStmt = lastSon(tryStmt)
if finallyStmt.kind == nkFinally:
genStmts(p, finallyStmt.sons[0])
# push old elements again:
for i in countdown(howManyTrys-1, 0):
p.nestedTryStmts.add(stack[i])
if not p.module.compileToCpp or optNoCppExceptions in gGlobalOptions:
# Pop exceptions that was handled by the
# except-blocks we are in
for i in countdown(howManyExcepts-1, 0):
linefmt(p, cpsStmts, "#popCurrentException();$n")
proc genReturnStmt(p: BProc, t: PNode) =
if nfPreventCg in t.flags: return
p.beforeRetNeeded = true
@ -377,7 +372,7 @@ proc genReturnStmt(p: BProc, t: PNode) =
if (t.sons[0].kind != nkEmpty): genStmts(p, t.sons[0])
blockLeaveActions(p,
howManyTrys = p.nestedTryStmts.len,
howManyExcepts = p.inExceptBlock)
howManyExcepts = p.inExceptBlockLen)
if (p.finallySafePoints.len > 0):
# If we're in a finally block, and we came here by exception
# consume it before we return.
@ -391,7 +386,7 @@ proc genGotoForCase(p: BProc; caseStmt: PNode) =
let it = caseStmt.sons[i]
for j in 0 .. it.len-2:
if it.sons[j].kind == nkRange:
localError(it.info, "range notation not available for computed goto")
localError(p.config, it.info, "range notation not available for computed goto")
return
let val = getOrdValue(it.sons[j])
lineF(p, cpsStmts, "NIMSTATE_$#:$n", [val.rope])
@ -406,19 +401,19 @@ proc genComputedGoto(p: BProc; n: PNode) =
let it = n.sons[i]
if it.kind == nkCaseStmt:
if lastSon(it).kind != nkOfBranch:
localError(it.info,
localError(p.config, it.info,
"case statement must be exhaustive for computed goto"); return
casePos = i
let aSize = lengthOrd(it.sons[0].typ)
if aSize > 10_000:
localError(it.info,
localError(p.config, it.info,
"case statement has too many cases for computed goto"); return
arraySize = aSize.int
if firstOrd(it.sons[0].typ) != 0:
localError(it.info,
localError(p.config, it.info,
"case statement has to start at 0 for computed goto"); return
if casePos < 0:
localError(n.info, "no case statement found for computed goto"); return
localError(p.config, n.info, "no case statement found for computed goto"); return
var id = p.labels+1
inc p.labels, arraySize+1
let tmp = "TMP$1_" % [id.rope]
@ -452,7 +447,7 @@ proc genComputedGoto(p: BProc; n: PNode) =
let it = caseStmt.sons[i]
for j in 0 .. it.len-2:
if it.sons[j].kind == nkRange:
localError(it.info, "range notation not available for computed goto")
localError(p.config, it.info, "range notation not available for computed goto")
return
let val = getOrdValue(it.sons[j])
lineF(p, cpsStmts, "TMP$#_:$n", [intLiteral(val+id+1)])
@ -556,33 +551,38 @@ proc genBreakStmt(p: BProc, t: PNode) =
# an unnamed 'break' can only break a loop after 'transf' pass:
while idx >= 0 and not p.blocks[idx].isLoop: dec idx
if idx < 0 or not p.blocks[idx].isLoop:
internalError(t.info, "no loop to break")
internalError(p.config, t.info, "no loop to break")
let label = assignLabel(p.blocks[idx])
blockLeaveActions(p,
p.nestedTryStmts.len - p.blocks[idx].nestedTryStmts,
p.inExceptBlock - p.blocks[idx].nestedExceptStmts)
p.inExceptBlockLen - p.blocks[idx].nestedExceptStmts)
genLineDir(p, t)
lineF(p, cpsStmts, "goto $1;$n", [label])
proc genRaiseStmt(p: BProc, t: PNode) =
if p.inExceptBlock > 0:
if p.module.compileToCpp:
discard cgsym(p.module, "popCurrentExceptionEx")
if p.nestedTryStmts.len > 0 and p.nestedTryStmts[^1].inExcept:
# if the current try stmt have a finally block,
# we must execute it before reraising
var finallyBlock = p.nestedTryStmts[p.nestedTryStmts.len - 1].lastSon
var finallyBlock = p.nestedTryStmts[^1].n[^1]
if finallyBlock.kind == nkFinally:
genSimpleBlock(p, finallyBlock.sons[0])
if t.sons[0].kind != nkEmpty:
genSimpleBlock(p, finallyBlock[0])
if t[0].kind != nkEmpty:
var a: TLoc
initLocExpr(p, t.sons[0], a)
initLocExprSingleUse(p, t[0], a)
var e = rdLoc(a)
var typ = skipTypes(t.sons[0].typ, abstractPtrs)
var typ = skipTypes(t[0].typ, abstractPtrs)
genLineDir(p, t)
lineCg(p, cpsStmts, "#raiseException((#Exception*)$1, $2);$n",
[e, makeCString(typ.sym.name.s)])
if isImportedException(typ, p.config):
lineF(p, cpsStmts, "throw $1;$n", [e])
else:
lineCg(p, cpsStmts, "#raiseException((#Exception*)$1, $2);$n",
[e, makeCString(typ.sym.name.s)])
else:
genLineDir(p, t)
# reraise the last exception:
if p.module.compileToCpp and optNoCppExceptions notin gGlobalOptions:
if p.module.compileToCpp and optNoCppExceptions notin p.config.globalOptions:
line(p, cpsStmts, ~"throw;$n")
else:
linefmt(p, cpsStmts, "#reraiseException();$n")
@ -775,91 +775,75 @@ proc genCase(p: BProc, t: PNode, d: var TLoc) =
else:
genOrdinalCase(p, t, d)
proc genTryCpp(p: BProc, t: PNode, d: var TLoc) =
# code to generate:
#
# XXX: There should be a standard dispatch algorithm
# that's used both here and with multi-methods
#
# try
# {
# myDiv(4, 9);
# } catch (NimException& exp) {
# if (isObj(exp, EIO) {
# ...
# } else if (isObj(exp, ESystem) {
# ...
# finallyPart()
# raise;
# } else {
# // general handler
# }
# }
# finallyPart();
# } catch (NimExceptionType1&) {
# body
# } catch (NimExceptionType2&) {
# finallyPart()
# raise;
# }
# catch(...) {
# general_handler_body
# }
# finallyPart();
template genExceptBranchBody(body: PNode) {.dirty.} =
if optStackTrace in p.options:
linefmt(p, cpsStmts, "#setFrame((TFrame*)&FR_);$n")
expr(p, body, d)
if not isEmptyType(t.typ) and d.k == locNone:
getTemp(p, t.typ, d)
genLineDir(p, t)
let exc = getTempName(p.module)
if getCompilerProc("Exception") != nil:
discard cgsym(p.module, "Exception")
else:
discard cgsym(p.module, "E_Base")
add(p.nestedTryStmts, t)
discard cgsym(p.module, "popCurrentExceptionEx")
add(p.nestedTryStmts, (t, false))
startBlock(p, "try {$n")
expr(p, t.sons[0], d)
let length = sonsLen(t)
endBlock(p, ropecg(p.module, "} catch (NimException& $1) {$n", [exc]))
if optStackTrace in p.options:
linefmt(p, cpsStmts, "#setFrame((TFrame*)&FR_);$n")
inc p.inExceptBlock
var i = 1
expr(p, t[0], d)
endBlock(p)
var catchAllPresent = false
while (i < length) and (t.sons[i].kind == nkExceptBranch):
p.nestedTryStmts[^1].inExcept = true
for i in 1..<t.len:
if t[i].kind != nkExceptBranch: break
# bug #4230: avoid false sharing between branches:
if d.k == locTemp and isEmptyType(t.typ): d.k = locNone
let blen = sonsLen(t.sons[i])
if i > 1: addf(p.s(cpsStmts), "else ", [])
if blen == 1:
if t[i].len == 1:
# general except section:
catchAllPresent = true
startBlock(p)
expr(p, t.sons[i].sons[0], d)
linefmt(p, cpsStmts, "#popCurrentException();$n")
startBlock(p, "catch (...) {$n")
genExceptBranchBody(t[i][0])
endBlock(p)
else:
var orExpr: Rope = nil
for j in countup(0, blen - 2):
assert(t.sons[i].sons[j].kind == nkType)
if orExpr != nil: add(orExpr, "||")
appcg(p.module, orExpr,
"#isObj($1.exp->m_type, $2)",
[exc, genTypeInfo(p.module, t[i][j].typ, t[i][j].info)])
lineF(p, cpsStmts, "if ($1) ", [orExpr])
startBlock(p)
expr(p, t.sons[i].sons[blen-1], d)
linefmt(p, cpsStmts, "#popCurrentException();$n")
endBlock(p)
inc(i)
for j in 0..t[i].len-2:
if t[i][j].isInfixAs():
let exvar = t[i][j][2] # ex1 in `except ExceptType as ex1:`
fillLoc(exvar.sym.loc, locTemp, exvar, mangleLocalName(p, exvar.sym), OnUnknown)
startBlock(p, "catch ($1& $2) {$n", getTypeDesc(p.module, t[i][j][1].typ), rdLoc(exvar.sym.loc))
else:
startBlock(p, "catch ($1&) {$n", getTypeDesc(p.module, t[i][j].typ))
genExceptBranchBody(t[i][^1]) # exception handler body will duplicated for every type
endBlock(p)
# reraise the exception if there was no catch all
# and none of the handlers matched
if not catchAllPresent:
if i > 1: lineF(p, cpsStmts, "else ", [])
startBlock(p)
var finallyBlock = t.lastSon
if finallyBlock.kind == nkFinally:
#expr(p, finallyBlock.sons[0], d)
genStmts(p, finallyBlock.sons[0])
discard pop(p.nestedTryStmts)
if not catchAllPresent and t[^1].kind == nkFinally:
# finally requires catch all presence
startBlock(p, "catch (...) {$n")
genSimpleBlock(p, t[^1][0])
line(p, cpsStmts, ~"throw;$n")
endBlock(p)
lineF(p, cpsStmts, "}$n", []) # end of catch block
dec p.inExceptBlock
discard pop(p.nestedTryStmts)
if (i < length) and (t.sons[i].kind == nkFinally):
genSimpleBlock(p, t.sons[i].sons[0])
if t[^1].kind == nkFinally:
genSimpleBlock(p, t[^1][0])
proc genTry(p: BProc, t: PNode, d: var TLoc) =
# code to generate:
@ -895,23 +879,20 @@ proc genTry(p: BProc, t: PNode, d: var TLoc) =
p.module.includeHeader("<setjmp.h>")
genLineDir(p, t)
var safePoint = getTempName(p.module)
if getCompilerProc("Exception") != nil:
discard cgsym(p.module, "Exception")
else:
discard cgsym(p.module, "E_Base")
discard cgsym(p.module, "Exception")
linefmt(p, cpsLocals, "#TSafePoint $1;$n", safePoint)
linefmt(p, cpsStmts, "#pushSafePoint(&$1);$n", safePoint)
if isDefined("nimStdSetjmp"):
if isDefined(p.config, "nimStdSetjmp"):
linefmt(p, cpsStmts, "$1.status = setjmp($1.context);$n", safePoint)
elif isDefined("nimSigSetjmp"):
elif isDefined(p.config, "nimSigSetjmp"):
linefmt(p, cpsStmts, "$1.status = sigsetjmp($1.context, 0);$n", safePoint)
elif isDefined("nimRawSetjmp"):
elif isDefined(p.config, "nimRawSetjmp"):
linefmt(p, cpsStmts, "$1.status = _setjmp($1.context);$n", safePoint)
else:
linefmt(p, cpsStmts, "$1.status = setjmp($1.context);$n", safePoint)
startBlock(p, "if ($1.status == 0) {$n", [safePoint])
var length = sonsLen(t)
add(p.nestedTryStmts, t)
add(p.nestedTryStmts, (t, false))
expr(p, t.sons[0], d)
linefmt(p, cpsStmts, "#popSafePoint();$n")
endBlock(p)
@ -919,7 +900,7 @@ proc genTry(p: BProc, t: PNode, d: var TLoc) =
linefmt(p, cpsStmts, "#popSafePoint();$n")
if optStackTrace in p.options:
linefmt(p, cpsStmts, "#setFrame((TFrame*)&FR_);$n")
inc p.inExceptBlock
p.nestedTryStmts[^1].inExcept = true
var i = 1
while (i < length) and (t.sons[i].kind == nkExceptBranch):
# bug #4230: avoid false sharing between branches:
@ -950,7 +931,6 @@ proc genTry(p: BProc, t: PNode, d: var TLoc) =
linefmt(p, cpsStmts, "#popCurrentException();$n")
endBlock(p)
inc(i)
dec p.inExceptBlock
discard pop(p.nestedTryStmts)
endBlock(p) # end of else block
if i < length and t.sons[i].kind == nkFinally:
@ -961,19 +941,20 @@ proc genTry(p: BProc, t: PNode, d: var TLoc) =
proc genAsmOrEmitStmt(p: BProc, t: PNode, isAsmStmt=false): Rope =
var res = ""
for i in countup(0, sonsLen(t) - 1):
case t.sons[i].kind
for it in t.sons:
case it.kind
of nkStrLit..nkTripleStrLit:
res.add(t.sons[i].strVal)
res.add(it.strVal)
of nkSym:
var sym = t.sons[i].sym
var sym = it.sym
if sym.kind in {skProc, skFunc, skIterator, skMethod}:
var a: TLoc
initLocExpr(p, t.sons[i], a)
initLocExpr(p, it, a)
res.add($rdLoc(a))
elif sym.kind == skType:
res.add($getTypeDesc(p.module, sym.typ))
else:
discard getTypeDesc(p.module, skipTypes(sym.typ, abstractPtrs))
var r = sym.loc.r
if r == nil:
# if no name has already been given,
@ -982,12 +963,12 @@ proc genAsmOrEmitStmt(p: BProc, t: PNode, isAsmStmt=false): Rope =
sym.loc.r = r # but be consequent!
res.add($r)
of nkTypeOfExpr:
res.add($getTypeDesc(p.module, t.sons[i].typ))
res.add($getTypeDesc(p.module, it.typ))
else:
discard getTypeDesc(p.module, skipTypes(it.typ, abstractPtrs))
var a: TLoc
initLocExpr(p, t.sons[i], a)
initLocExpr(p, it, a)
res.add($a.rdLoc)
#internalError(t.sons[i].info, "genAsmOrEmitStmt()")
if isAsmStmt and hasGnuAsm in CC[cCompiler].props:
for x in splitLines(res):
@ -1032,7 +1013,7 @@ proc genEmit(p: BProc, t: PNode) =
if p.prc == nil:
# top level emit pragma?
let section = determineSection(t[1])
genCLineDir(p.module.s[section], t.info)
genCLineDir(p.module.s[section], t.info, p.config)
add(p.module.s[section], s)
else:
genLineDir(p, t)
@ -1063,8 +1044,7 @@ proc genWatchpoint(p: BProc, n: PNode) =
genTypeInfo(p.module, typ, n.info)])
proc genPragma(p: BProc, n: PNode) =
for i in countup(0, sonsLen(n) - 1):
var it = n.sons[i]
for it in n.sons:
case whichPragma(it)
of wEmit: genEmit(p, it)
of wBreakpoint: genBreakPoint(p, it)
@ -1159,4 +1139,4 @@ proc genAsgn(p: BProc, e: PNode, fastAsgn: bool) =
proc genStmts(p: BProc, t: PNode) =
var a: TLoc
expr(p, t, a)
internalAssert a.k in {locNone, locTemp, locLocalVar}
internalAssert p.config, a.k in {locNone, locTemp, locLocalVar}

12
compiler/ccgthreadvars.nim
View file

@ -12,11 +12,11 @@
# included from cgen.nim
proc emulatedThreadVars(): bool =
result = {optThreads, optTlsEmulation} <= gGlobalOptions
proc emulatedThreadVars(conf: ConfigRef): bool =
result = {optThreads, optTlsEmulation} <= conf.globalOptions
proc accessThreadLocalVar(p: BProc, s: PSym) =
if emulatedThreadVars() and not p.threadVarAccessed:
if emulatedThreadVars(p.config) and not p.threadVarAccessed:
p.threadVarAccessed = true
incl p.module.flags, usesThreadVars
addf(p.procSec(cpsLocals), "\tNimThreadVars* NimTV_;$n", [])
@ -37,7 +37,7 @@ var
# made to be one.
proc declareThreadVar(m: BModule, s: PSym, isExtern: bool) =
if emulatedThreadVars():
if emulatedThreadVars(m.config):
# we gather all thread locals var into a struct; we need to allocate
# storage for that somehow, can't use the thread local storage
# allocator for it :-(
@ -46,7 +46,7 @@ proc declareThreadVar(m: BModule, s: PSym, isExtern: bool) =
addf(nimtv, "$1 $2;$n", [getTypeDesc(m, s.loc.t), s.loc.r])
else:
if isExtern: add(m.s[cfsVars], "extern ")
if optThreads in gGlobalOptions: add(m.s[cfsVars], "NIM_THREADVAR ")
if optThreads in m.config.globalOptions: add(m.s[cfsVars], "NIM_THREADVAR ")
add(m.s[cfsVars], getTypeDesc(m, s.loc.t))
addf(m.s[cfsVars], " $1;$n", [s.loc.r])
@ -57,7 +57,7 @@ proc generateThreadLocalStorage(m: BModule) =
proc generateThreadVarsSize(m: BModule) =
if nimtv != nil:
let externc = if gCmd == cmdCompileToCpp or
let externc = if m.config.cmd == cmdCompileToCpp or
sfCompileToCpp in m.module.flags: "extern \"C\" "
else: ""
addf(m.s[cfsProcs],

40
compiler/ccgtrav.nim
View file

@ -17,11 +17,11 @@ type
p: BProc
visitorFrmt: string
proc genTraverseProc(c: var TTraversalClosure, accessor: Rope, typ: PType)
proc genTraverseProc(c: TTraversalClosure, accessor: Rope, typ: PType)
proc genCaseRange(p: BProc, branch: PNode)
proc getTemp(p: BProc, t: PType, result: var TLoc; needsInit=false)
proc genTraverseProc(c: var TTraversalClosure, accessor: Rope, n: PNode;
proc genTraverseProc(c: TTraversalClosure, accessor: Rope, n: PNode;
typ: PType) =
if n == nil: return
case n.kind
@ -29,12 +29,12 @@ proc genTraverseProc(c: var TTraversalClosure, accessor: Rope, n: PNode;
for i in countup(0, sonsLen(n) - 1):
genTraverseProc(c, accessor, n.sons[i], typ)
of nkRecCase:
if (n.sons[0].kind != nkSym): internalError(n.info, "genTraverseProc")
if (n.sons[0].kind != nkSym): internalError(c.p.config, n.info, "genTraverseProc")
var p = c.p
let disc = n.sons[0].sym
if disc.loc.r == nil: fillObjectFields(c.p.module, typ)
if disc.loc.t == nil:
internalError(n.info, "genTraverseProc()")
internalError(c.p.config, n.info, "genTraverseProc()")
lineF(p, cpsStmts, "switch ($1.$2) {$n", [accessor, disc.loc.r])
for i in countup(1, sonsLen(n) - 1):
let branch = n.sons[i]
@ -51,9 +51,9 @@ proc genTraverseProc(c: var TTraversalClosure, accessor: Rope, n: PNode;
if field.typ.kind == tyVoid: return
if field.loc.r == nil: fillObjectFields(c.p.module, typ)
if field.loc.t == nil:
internalError(n.info, "genTraverseProc()")
internalError(c.p.config, n.info, "genTraverseProc()")
genTraverseProc(c, "$1.$2" % [accessor, field.loc.r], field.loc.t)
else: internalError(n.info, "genTraverseProc()")
else: internalError(c.p.config, n.info, "genTraverseProc()")
proc parentObj(accessor: Rope; m: BModule): Rope {.inline.} =
if not m.compileToCpp:
@ -61,22 +61,23 @@ proc parentObj(accessor: Rope; m: BModule): Rope {.inline.} =
else:
result = accessor
proc genTraverseProc(c: var TTraversalClosure, accessor: Rope, typ: PType) =
proc genTraverseProc(c: TTraversalClosure, accessor: Rope, typ: PType) =
if typ == nil: return
var p = c.p
case typ.kind
of tyGenericInst, tyGenericBody, tyTypeDesc, tyAlias, tyDistinct, tyInferred:
of tyGenericInst, tyGenericBody, tyTypeDesc, tyAlias, tyDistinct, tyInferred,
tySink:
genTraverseProc(c, accessor, lastSon(typ))
of tyArray:
let arraySize = lengthOrd(typ.sons[0])
var i: TLoc
getTemp(p, getSysType(tyInt), i)
getTemp(p, getSysType(c.p.module.g.graph, unknownLineInfo(), tyInt), i)
let oldCode = p.s(cpsStmts)
linefmt(p, cpsStmts, "for ($1 = 0; $1 < $2; $1++) {$n",
i.r, arraySize.rope)
let oldLen = p.s(cpsStmts).len
genTraverseProc(c, rfmt(nil, "$1[$2]", accessor, i.r), typ.sons[1])
genTraverseProc(c, ropecg(c.p.module, "$1[$2]", accessor, i.r), typ.sons[1])
if p.s(cpsStmts).len == oldLen:
# do not emit dummy long loops for faster debug builds:
p.s(cpsStmts) = oldCode
@ -91,20 +92,20 @@ proc genTraverseProc(c: var TTraversalClosure, accessor: Rope, typ: PType) =
of tyTuple:
let typ = getUniqueType(typ)
for i in countup(0, sonsLen(typ) - 1):
genTraverseProc(c, rfmt(nil, "$1.Field$2", accessor, i.rope), typ.sons[i])
genTraverseProc(c, ropecg(c.p.module, "$1.Field$2", accessor, i.rope), typ.sons[i])
of tyRef, tyString, tySequence:
lineCg(p, cpsStmts, c.visitorFrmt, accessor)
of tyProc:
if typ.callConv == ccClosure:
lineCg(p, cpsStmts, c.visitorFrmt, rfmt(nil, "$1.ClE_0", accessor))
lineCg(p, cpsStmts, c.visitorFrmt, ropecg(c.p.module, "$1.ClE_0", accessor))
else:
discard
proc genTraverseProcSeq(c: var TTraversalClosure, accessor: Rope, typ: PType) =
proc genTraverseProcSeq(c: TTraversalClosure, accessor: Rope, typ: PType) =
var p = c.p
assert typ.kind == tySequence
var i: TLoc
getTemp(p, getSysType(tyInt), i)
getTemp(p, getSysType(c.p.module.g.graph, unknownLineInfo(), tyInt), i)
let oldCode = p.s(cpsStmts)
lineF(p, cpsStmts, "for ($1 = 0; $1 < $2->$3; $1++) {$n",
[i.r, accessor, rope(if c.p.module.compileToCpp: "len" else: "Sup.len")])
@ -116,18 +117,13 @@ proc genTraverseProcSeq(c: var TTraversalClosure, accessor: Rope, typ: PType) =
else:
lineF(p, cpsStmts, "}$n", [])
proc genTraverseProc(m: BModule, origTyp: PType; sig: SigHash;
reason: TTypeInfoReason): Rope =
proc genTraverseProc(m: BModule, origTyp: PType; sig: SigHash): Rope =
var c: TTraversalClosure
var p = newProc(nil, m)
result = "Marker_" & getTypeName(m, origTyp, sig)
var typ = origTyp.skipTypes(abstractInst)
if typ.kind == tyOpt: typ = optLowering(typ)
case reason
of tiNew: c.visitorFrmt = "#nimGCvisit((void*)$1, op);$n"
else: assert false
let header = "static N_NIMCALL(void, $1)(void* p, NI op)" % [result]
let t = getTypeDesc(m, typ)
@ -135,6 +131,8 @@ proc genTraverseProc(m: BModule, origTyp: PType; sig: SigHash;
lineF(p, cpsInit, "a = ($1)p;$n", [t])
c.p = p
c.visitorFrmt = "#nimGCvisit((void*)$1, op);$n"
assert typ.kind != tyTypeDesc
if typ.kind == tySequence:
genTraverseProcSeq(c, "a".rope, typ)
@ -159,7 +157,7 @@ proc genTraverseProcForGlobal(m: BModule, s: PSym; info: TLineInfo): Rope =
var sLoc = s.loc.r
result = getTempName(m)
if sfThread in s.flags and emulatedThreadVars():
if sfThread in s.flags and emulatedThreadVars(m.config):
accessThreadLocalVar(p, s)
sLoc = "NimTV_->" & sLoc

176
compiler/ccgtypes.nim
View file

@ -14,6 +14,8 @@
import sighashes
from lowerings import createObj
proc genProcHeader(m: BModule, prc: PSym): Rope
proc isKeyword(w: PIdent): bool =
# Nim and C++ share some keywords
# it's more efficient to test the whole Nim keywords range
@ -42,38 +44,11 @@ when false:
assert p.kind == skPackage
result = gDebugInfo.register(p.name.s, m.name.s)
proc idOrSig(m: BModule; s: PSym): Rope =
if s.kind in routineKinds and s.typ != nil:
# signatures for exported routines are reliable enough to
# produce a unique name and this means produced C++ is more stable wrt
# Nim changes:
let sig = hashProc(s)
result = rope($sig)
#let m = if s.typ.callConv != ccInline: findPendingModule(m, s) else: m
let counter = m.sigConflicts.getOrDefault(sig)
#if sigs == "_jckmNePK3i2MFnWwZlp6Lg" and s.name.s == "contains":
# echo "counter ", counter, " ", s.id
if counter != 0:
result.add "_" & rope(counter+1)
# this minor hack is necessary to make tests/collections/thashes compile.
# The inlined hash function's original module is ambiguous so we end up
# generating duplicate names otherwise:
if s.typ.callConv == ccInline:
result.add rope(m.module.name.s)
m.sigConflicts.inc(sig)
else:
let sig = hashNonProc(s)
result = rope($sig)
let counter = m.sigConflicts.getOrDefault(sig)
if counter != 0:
result.add "_" & rope(counter+1)
m.sigConflicts.inc(sig)
proc mangleName(m: BModule; s: PSym): Rope =
result = s.loc.r
if result == nil:
result = s.name.s.mangle.rope
add(result, m.idOrSig(s))
add(result, idOrSig(s, m.module.name.s, m.sigConflicts))
s.loc.r = result
writeMangledName(m.ndi, s)
@ -119,7 +94,7 @@ proc scopeMangledParam(p: BProc; param: PSym) =
const
irrelevantForBackend = {tyGenericBody, tyGenericInst, tyGenericInvocation,
tyDistinct, tyRange, tyStatic, tyAlias, tyInferred}
tyDistinct, tyRange, tyStatic, tyAlias, tySink, tyInferred}
proc typeName(typ: PType): Rope =
let typ = typ.skipTypes(irrelevantForBackend)
@ -139,7 +114,7 @@ proc getTypeName(m: BModule; typ: PType; sig: SigHash): Rope =
t = t.lastSon
else:
break
let typ = if typ.kind == tyAlias: typ.lastSon else: typ
let typ = if typ.kind in {tyAlias, tySink}: typ.lastSon else: typ
if typ.loc.r == nil:
typ.loc.r = typ.typeName & $sig
else:
@ -147,7 +122,7 @@ proc getTypeName(m: BModule; typ: PType; sig: SigHash): Rope =
# check consistency:
assert($typ.loc.r == $(typ.typeName & $sig))
result = typ.loc.r
if result == nil: internalError("getTypeName: " & $typ.kind)
if result == nil: internalError(m.config, "getTypeName: " & $typ.kind)
proc mapSetType(typ: PType): TCTypeKind =
case int(getSize(typ))
@ -167,10 +142,10 @@ proc mapType(typ: PType): TCTypeKind =
of tyOpenArray, tyArray, tyVarargs: result = ctArray
of tyObject, tyTuple: result = ctStruct
of tyUserTypeClasses:
internalAssert typ.isResolvedUserTypeClass
doAssert typ.isResolvedUserTypeClass
return mapType(typ.lastSon)
of tyGenericBody, tyGenericInst, tyGenericParam, tyDistinct, tyOrdinal,
tyTypeDesc, tyAlias, tyInferred:
tyTypeDesc, tyAlias, tySink, tyInferred:
result = mapType(lastSon(typ))
of tyEnum:
if firstOrd(typ) < 0:
@ -181,9 +156,9 @@ proc mapType(typ: PType): TCTypeKind =
of 2: result = ctUInt16
of 4: result = ctInt32
of 8: result = ctInt64
else: internalError("mapType")
else: result = ctInt32
of tyRange: result = mapType(typ.sons[0])
of tyPtr, tyVar, tyRef, tyOptAsRef:
of tyPtr, tyVar, tyLent, tyRef, tyOptAsRef:
var base = skipTypes(typ.lastSon, typedescInst)
case base.kind
of tyOpenArray, tyArray, tyVarargs: result = ctPtrToArray
@ -208,8 +183,8 @@ proc mapType(typ: PType): TCTypeKind =
result = TCTypeKind(ord(typ.kind) - ord(tyInt) + ord(ctInt))
of tyStatic:
if typ.n != nil: result = mapType(lastSon typ)
else: internalError("mapType")
else: internalError("mapType")
else: doAssert(false, "mapType")
else: doAssert(false, "mapType")
proc mapReturnType(typ: PType): TCTypeKind =
#if skipTypes(typ, typedescInst).kind == tyArray: result = ctPtr
@ -242,7 +217,7 @@ proc isInvalidReturnType(rettype: PType): bool =
case mapType(rettype)
of ctArray:
result = not (skipTypes(rettype, typedescInst).kind in
{tyVar, tyRef, tyPtr})
{tyVar, tyLent, tyRef, tyPtr})
of ctStruct:
let t = skipTypes(rettype, typedescInst)
if rettype.isImportedCppType or t.isImportedCppType: return false
@ -264,13 +239,9 @@ proc cacheGetType(tab: TypeCache; sig: SigHash): Rope =
result = tab.getOrDefault(sig)
proc addAbiCheck(m: BModule, t: PType, name: Rope) =
if isDefined("checkabi"):
if isDefined(m.config, "checkabi"):
addf(m.s[cfsTypeInfo], "NIM_CHECK_SIZE($1, $2);$n", [name, rope(getSize(t))])
proc getTempName(m: BModule): Rope =
result = m.tmpBase & rope(m.labels)
inc m.labels
proc ccgIntroducedPtr(s: PSym): bool =
var pt = skipTypes(s.typ, typedescInst)
assert skResult != s.kind
@ -316,8 +287,13 @@ proc getSimpleTypeDesc(m: BModule, typ: PType): Rope =
of tyPointer:
result = typeNameOrLiteral(m, typ, "void*")
of tyString:
discard cgsym(m, "NimStringDesc")
result = typeNameOrLiteral(m, typ, "NimStringDesc*")
case detectStrVersion(m)
of 2:
discard cgsym(m, "string")
result = typeNameOrLiteral(m, typ, "NimStringV2")
else:
discard cgsym(m, "NimStringDesc")
result = typeNameOrLiteral(m, typ, "NimStringDesc*")
of tyCString: result = typeNameOrLiteral(m, typ, "NCSTRING")
of tyBool: result = typeNameOrLiteral(m, typ, "NIM_BOOL")
of tyChar: result = typeNameOrLiteral(m, typ, "NIM_CHAR")
@ -327,8 +303,8 @@ proc getSimpleTypeDesc(m: BModule, typ: PType): Rope =
of tyDistinct, tyRange, tyOrdinal: result = getSimpleTypeDesc(m, typ.sons[0])
of tyStatic:
if typ.n != nil: result = getSimpleTypeDesc(m, lastSon typ)
else: internalError("tyStatic for getSimpleTypeDesc")
of tyGenericInst, tyAlias:
else: internalError(m.config, "tyStatic for getSimpleTypeDesc")
of tyGenericInst, tyAlias, tySink:
result = getSimpleTypeDesc(m, lastSon typ)
else: result = nil
@ -348,7 +324,7 @@ proc getTypePre(m: BModule, typ: PType; sig: SigHash): Rope =
if result == nil: result = cacheGetType(m.typeCache, sig)
proc structOrUnion(t: PType): Rope =
let t = t.skipTypes({tyAlias})
let t = t.skipTypes({tyAlias, tySink})
(if tfUnion in t.flags: rope("union") else: rope("struct"))
proc getForwardStructFormat(m: BModule): string =
@ -368,8 +344,10 @@ proc getTypeForward(m: BModule, typ: PType; sig: SigHash): Rope =
if not isImportedType(concrete):
addf(m.s[cfsForwardTypes], getForwardStructFormat(m),
[structOrUnion(typ), result])
else:
pushType(m, concrete)
doAssert m.forwTypeCache[sig] == result
else: internalError("getTypeForward(" & $typ.kind & ')')
else: internalError(m.config, "getTypeForward(" & $typ.kind & ')')
proc getTypeDescWeak(m: BModule; t: PType; check: var IntSet): Rope =
## like getTypeDescAux but creates only a *weak* dependency. In other words
@ -396,7 +374,7 @@ proc getTypeDescWeak(m: BModule; t: PType; check: var IntSet): Rope =
result = getTypeDescAux(m, t, check)
proc paramStorageLoc(param: PSym): TStorageLoc =
if param.typ.skipTypes({tyVar, tyTypeDesc}).kind notin {
if param.typ.skipTypes({tyVar, tyLent, tyTypeDesc}).kind notin {
tyArray, tyOpenArray, tyVarargs}:
result = OnStack
else:
@ -411,7 +389,7 @@ proc genProcParams(m: BModule, t: PType, rettype, params: var Rope,
else:
rettype = getTypeDescAux(m, t.sons[0], check)
for i in countup(1, sonsLen(t.n) - 1):
if t.n.sons[i].kind != nkSym: internalError(t.n.info, "genProcParams")
if t.n.sons[i].kind != nkSym: internalError(m.config, t.n.info, "genProcParams")
var param = t.n.sons[i].sym
if isCompileTimeOnly(param.typ): continue
if params != nil: add(params, ~", ")
@ -430,11 +408,11 @@ proc genProcParams(m: BModule, t: PType, rettype, params: var Rope,
add(params, param.loc.r)
# declare the len field for open arrays:
var arr = param.typ
if arr.kind == tyVar: arr = arr.sons[0]
if arr.kind in {tyVar, tyLent}: arr = arr.lastSon
var j = 0
while arr.kind in {tyOpenArray, tyVarargs}:
# this fixes the 'sort' bug:
if param.typ.kind == tyVar: param.loc.storage = OnUnknown
if param.typ.kind in {tyVar, tyLent}: param.loc.storage = OnUnknown
# need to pass hidden parameter:
addf(params, ", NI $1Len_$2", [param.loc.r, j.rope])
inc(j)
@ -464,7 +442,7 @@ proc mangleRecFieldName(m: BModule; field: PSym, rectype: PType): Rope =
result = field.loc.r
else:
result = rope(mangleField(m, field.name))
if result == nil: internalError(field.info, "mangleRecFieldName")
if result == nil: internalError(m.config, field.info, "mangleRecFieldName")
proc genRecordFieldsAux(m: BModule, n: PNode,
accessExpr: Rope, rectype: PType,
@ -475,7 +453,7 @@ proc genRecordFieldsAux(m: BModule, n: PNode,
for i in countup(0, sonsLen(n) - 1):
add(result, genRecordFieldsAux(m, n.sons[i], accessExpr, rectype, check))
of nkRecCase:
if n.sons[0].kind != nkSym: internalError(n.info, "genRecordFieldsAux")
if n.sons[0].kind != nkSym: internalError(m.config, n.info, "genRecordFieldsAux")
add(result, genRecordFieldsAux(m, n.sons[0], accessExpr, rectype, check))
let uname = rope(mangle(n.sons[0].sym.name.s) & 'U')
let ae = if accessExpr != nil: "$1.$2" % [accessExpr, uname]
@ -496,14 +474,14 @@ proc genRecordFieldsAux(m: BModule, n: PNode,
if hasAttribute in CC[cCompiler].props:
add(unionBody, "struct __attribute__((__packed__)){" )
else:
addf(unionBody, "#pragma pack(1)$nstruct{", [])
addf(unionBody, "#pragma pack(push, 1)$nstruct{", [])
add(unionBody, a)
addf(unionBody, "} $1;$n", [sname])
if tfPacked in rectype.flags and hasAttribute notin CC[cCompiler].props:
addf(unionBody, "#pragma pack(pop)$n", [])
else:
add(unionBody, genRecordFieldsAux(m, k, ae, rectype, check))
else: internalError("genRecordFieldsAux(record case branch)")
else: internalError(m.config, "genRecordFieldsAux(record case branch)")
if unionBody != nil:
addf(result, "union{$n$1} $2;$n", [unionBody, uname])
of nkSym:
@ -531,7 +509,7 @@ proc genRecordFieldsAux(m: BModule, n: PNode,
# don't use fieldType here because we need the
# tyGenericInst for C++ template support
addf(result, "$1 $2;$n", [getTypeDescAux(m, field.loc.t, check), sname])
else: internalError(n.info, "genRecordFieldsAux()")
else: internalError(m.config, n.info, "genRecordFieldsAux()")
proc getRecordFields(m: BModule, typ: PType, check: var IntSet): Rope =
result = genRecordFieldsAux(m, typ.n, nil, typ, check)
@ -551,7 +529,7 @@ proc getRecordDesc(m: BModule, typ: PType, name: Rope,
if hasAttribute in CC[cCompiler].props:
result = structOrUnion(typ) & " __attribute__((__packed__))"
else:
result = "#pragma pack(1)" & tnl & structOrUnion(typ)
result = "#pragma pack(push, 1)" & tnl & structOrUnion(typ)
else:
result = structOrUnion(typ)
@ -569,6 +547,16 @@ proc getRecordDesc(m: BModule, typ: PType, name: Rope,
elif m.compileToCpp:
appcg(m, result, " : public $1 {$n",
[getTypeDescAux(m, typ.sons[0].skipTypes(skipPtrs), check)])
if typ.isException:
appcg(m, result, "virtual void raise() {throw *this;}$n") # required for polymorphic exceptions
if typ.sym.magic == mException:
# Add cleanup destructor to Exception base class
appcg(m, result, "~$1() {if(this->raise_id) popCurrentExceptionEx(this->raise_id);}$n", [name])
# hack: forward declare popCurrentExceptionEx() on top of type description,
# proper request to generate popCurrentExceptionEx not possible for 2 reasons:
# generated function will be below declared Exception type and circular dependency
# between Exception and popCurrentExceptionEx function
result = genProcHeader(m, magicsys.getCompilerProc(m.g.graph, "popCurrentExceptionEx")) & ";" & rnl & result
hasField = true
else:
appcg(m, result, " {$n $1 Sup;$n",
@ -615,8 +603,10 @@ proc scanCppGenericSlot(pat: string, cursor, outIdx, outStars: var int): bool =
return false
proc resolveStarsInCppType(typ: PType, idx, stars: int): PType =
# XXX: we should catch this earlier and report it as a semantic error
if idx >= typ.len: internalError "invalid apostrophe type parameter index"
# Make sure the index refers to one of the generic params of the type.
# XXX: we should catch this earlier and report it as a semantic error.
if idx >= typ.len:
doAssert false, "invalid apostrophe type parameter index"
result = typ.sons[idx]
for i in 1..stars:
@ -629,7 +619,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet): Rope =
var t = origTyp.skipTypes(irrelevantForBackend)
if containsOrIncl(check, t.id):
if not (isImportedCppType(origTyp) or isImportedCppType(t)):
internalError("cannot generate C type for: " & typeToString(origTyp))
internalError(m.config, "cannot generate C type for: " & typeToString(origTyp))
# XXX: this BUG is hard to fix -> we need to introduce helper structs,
# but determining when this needs to be done is hard. We should split
# C type generation into an analysis and a code generation phase somehow.
@ -641,7 +631,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet): Rope =
excl(check, t.id)
return
case t.kind
of tyRef, tyOptAsRef, tyPtr, tyVar:
of tyRef, tyOptAsRef, tyPtr, tyVar, tyLent:
var star = if t.kind == tyVar and tfVarIsPtr notin origTyp.flags and
compileToCpp(m): "&" else: "*"
var et = origTyp.skipTypes(abstractInst).lastSon
@ -661,7 +651,6 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet): Rope =
let name = getTypeForward(m, et, hashType et)
result = name & star
m.typeCache[sig] = result
pushType(m, et)
of tySequence:
# no restriction! We have a forward declaration for structs
let name = getTypeForward(m, et, hashType et)
@ -708,7 +697,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet): Rope =
of 2: addf(m.s[cfsTypes], "typedef NU16 $1;$n", [result])
of 4: addf(m.s[cfsTypes], "typedef NI32 $1;$n", [result])
of 8: addf(m.s[cfsTypes], "typedef NI64 $1;$n", [result])
else: internalError(t.sym.info, "getTypeDescAux: enum")
else: internalError(m.config, t.sym.info, "getTypeDescAux: enum")
when false:
let owner = hashOwner(t.sym)
if not gDebugInfo.hasEnum(t.sym.name.s, t.sym.info.line, owner):
@ -818,6 +807,9 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet): Rope =
let typeInSlot = resolveStarsInCppType(origTyp, idx + 1, stars)
if typeInSlot == nil or typeInSlot.kind == tyVoid:
result.add(~"void")
elif typeInSlot.kind == tyStatic:
internalAssert m.config, typeInSlot.n != nil
result.add typeInSlot.n.renderTree
else:
result.add getTypeDescAux(m, typeInSlot, check)
else:
@ -834,6 +826,13 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet): Rope =
# always call for sideeffects:
assert t.kind != tyTuple
discard getRecordDesc(m, t, result, check)
# The resulting type will include commas and these won't play well
# with the C macros for defining procs such as N_NIMCALL. We must
# create a typedef for the type and use it in the proc signature:
let typedefName = ~"TY" & $sig
addf(m.s[cfsTypes], "typedef $1 $2;$n", [result, typedefName])
m.typeCache[sig] = typedefName
result = typedefName
else:
when false:
if t.sym != nil and t.sym.name.s == "KeyValuePair":
@ -872,11 +871,11 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet): Rope =
of 1, 2, 4, 8: addf(m.s[cfsTypes], "typedef NU$2 $1;$n", [result, rope(s*8)])
else: addf(m.s[cfsTypes], "typedef NU8 $1[$2];$n",
[result, rope(getSize(t))])
of tyGenericInst, tyDistinct, tyOrdinal, tyTypeDesc, tyAlias,
of tyGenericInst, tyDistinct, tyOrdinal, tyTypeDesc, tyAlias, tySink,
tyUserTypeClass, tyUserTypeClassInst, tyInferred:
result = getTypeDescAux(m, lastSon(t), check)
else:
internalError("getTypeDescAux(" & $t.kind & ')')
internalError(m.config, "getTypeDescAux(" & $t.kind & ')')
result = nil
# fixes bug #145:
excl(check, t.id)
@ -916,7 +915,7 @@ template cgDeclFrmt*(s: PSym): string = s.constraint.strVal
proc genProcHeader(m: BModule, prc: PSym): Rope =
var
rettype, params: Rope
genCLineDir(result, prc.info)
genCLineDir(result, prc.info, m.config)
# using static is needed for inline procs
if lfExportLib in prc.loc.flags:
if isHeaderFile in m.flags:
@ -969,8 +968,9 @@ proc genTypeInfoAuxBase(m: BModule; typ, origType: PType;
if flags != 0:
addf(m.s[cfsTypeInit3], "$1.flags = $2;$n", [name, rope(flags)])
discard cgsym(m, "TNimType")
if isDefined("nimTypeNames"):
var typename = typeToString(origType, preferName)
if isDefined(m.config, "nimTypeNames"):
var typename = typeToString(if origType.typeInst != nil: origType.typeInst
else: origType, preferName)
if typename == "ref object" and origType.skipTypes(skipPtrs).sym != nil:
typename = "anon ref object from " & $origType.skipTypes(skipPtrs).sym.info
addf(m.s[cfsTypeInit3], "$1.name = $2;$n",
@ -1000,7 +1000,7 @@ proc discriminatorTableName(m: BModule, objtype: PType, d: PSym): Rope =
while lookupInRecord(objtype.n, d.name) == nil:
objtype = objtype.sons[0]
if objtype.sym == nil:
internalError(d.info, "anonymous obj with discriminator")
internalError(m.config, d.info, "anonymous obj with discriminator")
result = "NimDT_$1_$2" % [rope($hashType(objtype)), rope(d.name.s.mangle)]
proc discriminatorTableDecl(m: BModule, objtype: PType, d: PSym): Rope =
@ -1034,7 +1034,7 @@ proc genObjectFields(m: BModule, typ, origType: PType, n: PNode, expr: Rope;
assert L > 0
if field.loc.r == nil: fillObjectFields(m, typ)
if field.loc.t == nil:
internalError(n.info, "genObjectFields")
internalError(m.config, n.info, "genObjectFields")
addf(m.s[cfsTypeInit3], "$1.kind = 3;$n" &
"$1.offset = offsetof($2, $3);$n" & "$1.typ = $4;$n" &
"$1.name = $5;$n" & "$1.sons = &$6[0];$n" &
@ -1050,7 +1050,7 @@ proc genObjectFields(m: BModule, typ, origType: PType, n: PNode, expr: Rope;
case b.kind
of nkOfBranch:
if sonsLen(b) < 2:
internalError(b.info, "genObjectFields; nkOfBranch broken")
internalError(m.config, b.info, "genObjectFields; nkOfBranch broken")
for j in countup(0, sonsLen(b) - 2):
if b.sons[j].kind == nkRange:
var x = int(getOrdValue(b.sons[j].sons[0]))
@ -1064,23 +1064,23 @@ proc genObjectFields(m: BModule, typ, origType: PType, n: PNode, expr: Rope;
of nkElse:
addf(m.s[cfsTypeInit3], "$1[$2] = &$3;$n",
[tmp, rope(L), tmp2])
else: internalError(n.info, "genObjectFields(nkRecCase)")
else: internalError(m.config, n.info, "genObjectFields(nkRecCase)")
of nkSym:
var field = n.sym
if field.bitsize == 0:
if field.loc.r == nil: fillObjectFields(m, typ)
if field.loc.t == nil:
internalError(n.info, "genObjectFields")
internalError(m.config, n.info, "genObjectFields")
addf(m.s[cfsTypeInit3], "$1.kind = 1;$n" &
"$1.offset = offsetof($2, $3);$n" & "$1.typ = $4;$n" &
"$1.name = $5;$n", [expr, getTypeDesc(m, origType),
field.loc.r, genTypeInfo(m, field.typ, info), makeCString(field.name.s)])
else: internalError(n.info, "genObjectFields")
else: internalError(m.config, n.info, "genObjectFields")
proc genObjectInfo(m: BModule, typ, origType: PType, name: Rope; info: TLineInfo) =
if typ.kind == tyObject:
if incompleteType(typ):
localError(info, "request for RTTI generation for incomplete object: " &
localError(m.config, info, "request for RTTI generation for incomplete object: " &
typeToString(typ))
genTypeInfoAux(m, typ, origType, name, info)
else:
@ -1171,19 +1171,15 @@ proc genSetInfo(m: BModule, typ: PType, name: Rope; info: TLineInfo) =
proc genArrayInfo(m: BModule, typ: PType, name: Rope; info: TLineInfo) =
genTypeInfoAuxBase(m, typ, typ, name, genTypeInfo(m, typ.sons[1], info), info)
proc fakeClosureType(owner: PSym): PType =
proc fakeClosureType(m: BModule; owner: PSym): PType =
# we generate the same RTTI as for a tuple[pointer, ref tuple[]]
result = newType(tyTuple, owner)
result.rawAddSon(newType(tyPointer, owner))
var r = newType(tyRef, owner)
let obj = createObj(owner, owner.info, final=false)
let obj = createObj(m.g.graph, owner, owner.info, final=false)
r.rawAddSon(obj)
result.rawAddSon(r)
type
TTypeInfoReason = enum ## for what do we need the type info?
tiNew, ## for 'new'
include ccgtrav
proc genDeepCopyProc(m: BModule; s: PSym; result: Rope) =
@ -1227,24 +1223,24 @@ proc genTypeInfo(m: BModule, t: PType; info: TLineInfo): Rope =
m.g.typeInfoMarker[sig] = result
case t.kind
of tyEmpty, tyVoid: result = rope"0"
of tyPointer, tyBool, tyChar, tyCString, tyString, tyInt..tyUInt64, tyVar:
of tyPointer, tyBool, tyChar, tyCString, tyString, tyInt..tyUInt64, tyVar, tyLent:
genTypeInfoAuxBase(m, t, t, result, rope"0", info)
of tyStatic:
if t.n != nil: result = genTypeInfo(m, lastSon t, info)
else: internalError("genTypeInfo(" & $t.kind & ')')
else: internalError(m.config, "genTypeInfo(" & $t.kind & ')')
of tyUserTypeClasses:
internalAssert t.isResolvedUserTypeClass
internalAssert m.config, t.isResolvedUserTypeClass
return genTypeInfo(m, t.lastSon, info)
of tyProc:
if t.callConv != ccClosure:
genTypeInfoAuxBase(m, t, t, result, rope"0", info)
else:
let x = fakeClosureType(t.owner)
let x = fakeClosureType(m, t.owner)
genTupleInfo(m, x, x, result, info)
of tySequence, tyRef, tyOptAsRef:
genTypeInfoAux(m, t, t, result, info)
if gSelectedGC >= gcMarkAndSweep:
let markerProc = genTraverseProc(m, origType, sig, tiNew)
if m.config.selectedGC >= gcMarkAndSweep:
let markerProc = genTraverseProc(m, origType, sig)
addf(m.s[cfsTypeInit3], "$1.marker = $2;$n", [result, markerProc])
of tyPtr, tyRange: genTypeInfoAux(m, t, t, result, info)
of tyArray: genArrayInfo(m, t, result, info)
@ -1257,7 +1253,7 @@ proc genTypeInfo(m: BModule, t: PType; info: TLineInfo): Rope =
# BUGFIX: use consistently RTTI without proper field names; otherwise
# results are not deterministic!
genTupleInfo(m, t, origType, result, info)
else: internalError("genTypeInfo(" & $t.kind & ')')
else: internalError(m.config, "genTypeInfo(" & $t.kind & ')')
if t.deepCopy != nil:
genDeepCopyProc(m, t.deepCopy, result)
elif origType.deepCopy != nil:

10
compiler/ccgutils.nim
View file

@ -53,7 +53,7 @@ proc hashString*(s: string): BiggestInt =
result = a
var
gTypeTable: array[TTypeKind, TIdTable]
gTypeTable: array[TTypeKind, TIdTable] # XXX globals here
gCanonicalTypes: array[TTypeKind, PType]
proc initTypeTables() =
@ -110,13 +110,13 @@ proc getUniqueType*(key: PType): PType =
of tyDistinct:
if key.deepCopy != nil: result = key
else: result = getUniqueType(lastSon(key))
of tyGenericInst, tyOrdinal, tyStatic, tyAlias, tyInferred:
of tyGenericInst, tyOrdinal, tyStatic, tyAlias, tySink, tyInferred:
result = getUniqueType(lastSon(key))
#let obj = lastSon(key)
#if obj.sym != nil and obj.sym.name.s == "TOption":
# echo "for ", typeToString(key), " I returned "
# debug result
of tyPtr, tyRef, tyVar:
of tyPtr, tyRef, tyVar, tyLent:
let elemType = lastSon(key)
if elemType.kind in {tyBool, tyChar, tyInt..tyUInt64}:
# no canonicalization for integral types, so that e.g. ``ptr pid_t`` is
@ -211,8 +211,4 @@ proc mangle*(name: string): string =
if requiresUnderscore:
result.add "_"
proc emitLazily*(s: PSym): bool {.inline.} =
result = optDeadCodeElim in gGlobalOptions or
sfDeadCodeElim in getModule(s).flags
initTypeTables()

280
compiler/cgen.nim
View file

@ -11,7 +11,7 @@
import
ast, astalgo, hashes, trees, platform, magicsys, extccomp, options, intsets,
nversion, nimsets, msgs, securehash, bitsets, idents, types,
nversion, nimsets, msgs, std / sha1, bitsets, idents, types,
ccgutils, os, ropes, math, passes, rodread, wordrecg, treetab, cgmeth,
condsyms, rodutils, renderer, idgen, cgendata, ccgmerge, semfold, aliases,
lowerings, semparallel, tables, sets, ndi
@ -19,6 +19,7 @@ import
import strutils except `%` # collides with ropes.`%`
from modulegraphs import ModuleGraph
from configuration import warnGcMem, errXMustBeCompileTime, hintDependency, errGenerated
import dynlib
when not declared(dynlib.libCandidates):
@ -92,6 +93,7 @@ proc useHeader(m: BModule, sym: PSym) =
proc cgsym(m: BModule, name: string): Rope
proc ropecg(m: BModule, frmt: FormatStr, args: varargs[Rope]): Rope =
assert m != nil
var i = 0
var length = len(frmt)
result = nil
@ -115,15 +117,15 @@ proc ropecg(m: BModule, frmt: FormatStr, args: varargs[Rope]): Rope =
if i >= length or not (frmt[i] in {'0'..'9'}): break
num = j
if j > high(args) + 1:
internalError("ropes: invalid format string $" & $j)
internalError(m.config, "ropes: invalid format string $" & $j)
add(result, args[j-1])
of 'n':
if optLineDir notin gOptions: add(result, rnl)
if optLineDir notin m.config.options: add(result, rnl)
inc(i)
of 'N':
add(result, rnl)
inc(i)
else: internalError("ropes: invalid format string $" & frmt[i])
else: internalError(m.config, "ropes: invalid format string $" & frmt[i])
elif frmt[i] == '#' and frmt[i+1] in IdentStartChars:
inc(i)
var j = i
@ -145,15 +147,10 @@ proc ropecg(m: BModule, frmt: FormatStr, args: varargs[Rope]): Rope =
if i - 1 >= start:
add(result, substr(frmt, start, i - 1))
template rfmt(m: BModule, fmt: string, args: varargs[Rope]): untyped =
ropecg(m, fmt, args)
var indent = "\t".rope
proc indentLine(p: BProc, r: Rope): Rope =
result = r
for i in countup(0, p.blocks.len-1):
prepend(result, indent)
prepend(result, "\t".rope)
proc appcg(m: BModule, c: var Rope, frmt: FormatStr,
args: varargs[Rope]) =
@ -189,14 +186,14 @@ proc safeLineNm(info: TLineInfo): int =
result = toLinenumber(info)
if result < 0: result = 0 # negative numbers are not allowed in #line
proc genCLineDir(r: var Rope, filename: string, line: int) =
proc genCLineDir(r: var Rope, filename: string, line: int; conf: ConfigRef) =
assert line >= 0
if optLineDir in gOptions:
if optLineDir in conf.options:
addf(r, "$N#line $2 $1$N",
[rope(makeSingleLineCString(filename)), rope(line)])
proc genCLineDir(r: var Rope, info: TLineInfo) =
genCLineDir(r, info.toFullPath, info.safeLineNm)
proc genCLineDir(r: var Rope, info: TLineInfo; conf: ConfigRef) =
genCLineDir(r, info.toFullPath, info.safeLineNm, conf)
proc freshLineInfo(p: BProc; info: TLineInfo): bool =
if p.lastLineInfo.line != info.line or
@ -213,9 +210,9 @@ proc genLineDir(p: BProc, t: PNode) =
tt = tt.sons[1]
let line = tt.info.safeLineNm
if optEmbedOrigSrc in gGlobalOptions:
add(p.s(cpsStmts), ~"//" & tt.info.sourceLine & rnl)
genCLineDir(p.s(cpsStmts), tt.info.toFullPath, line)
if optEmbedOrigSrc in p.config.globalOptions:
add(p.s(cpsStmts), ~"//" & sourceLine(p.config, tt.info) & rnl)
genCLineDir(p.s(cpsStmts), tt.info.toFullPath, line, p.config)
if ({optStackTrace, optEndb} * p.options == {optStackTrace, optEndb}) and
(p.prc == nil or sfPure notin p.prc.flags):
if freshLineInfo(p, tt.info):
@ -223,22 +220,27 @@ proc genLineDir(p: BProc, t: PNode) =
line.rope, makeCString(toFilename(tt.info)))
elif ({optLineTrace, optStackTrace} * p.options ==
{optLineTrace, optStackTrace}) and
(p.prc == nil or sfPure notin p.prc.flags) and tt.info.fileIndex >= 0:
(p.prc == nil or sfPure notin p.prc.flags) and tt.info.fileIndex != InvalidFileIDX:
if freshLineInfo(p, tt.info):
linefmt(p, cpsStmts, "nimln_($1, $2);$n",
line.rope, tt.info.quotedFilename)
line.rope, quotedFilename(p.config, tt.info))
proc postStmtActions(p: BProc) {.inline.} =
add(p.s(cpsStmts), p.module.injectStmt)
proc accessThreadLocalVar(p: BProc, s: PSym)
proc emulatedThreadVars(): bool {.inline.}
proc emulatedThreadVars(conf: ConfigRef): bool {.inline.}
proc genProc(m: BModule, prc: PSym)
template compileToCpp(m: BModule): untyped =
gCmd == cmdCompileToCpp or sfCompileToCpp in m.module.flags
m.config.cmd == cmdCompileToCpp or sfCompileToCpp in m.module.flags
include "ccgtypes.nim"
proc getTempName(m: BModule): Rope =
result = m.tmpBase & rope(m.labels)
inc m.labels
include ccgliterals
include ccgtypes
# ------------------------------ Manager of temporaries ------------------
@ -260,6 +262,11 @@ proc rdCharLoc(a: TLoc): Rope =
proc genObjectInit(p: BProc, section: TCProcSection, t: PType, a: TLoc,
takeAddr: bool) =
if p.module.compileToCpp and t.isException and not isDefined(p.config, "noCppExceptions"):
# init vtable in Exception object for polymorphic exceptions
includeHeader(p.module, "<new>")
linefmt(p, section, "new ($1) $2;$n", rdLoc(a), getTypeDesc(p.module, t))
case analyseObjectWithTypeField(t)
of frNone:
discard
@ -312,8 +319,10 @@ proc resetLoc(p: BProc, loc: var TLoc) =
genObjectInit(p, cpsStmts, loc.t, loc, true)
else:
useStringh(p.module)
# array passed as argument decayed into pointer, bug #7332
# so we use getTypeDesc here rather than rdLoc(loc)
linefmt(p, cpsStmts, "memset((void*)$1, 0, sizeof($2));$n",
addrLoc(loc), rdLoc(loc))
addrLoc(loc), getTypeDesc(p.module, loc.t))
# XXX: We can be extra clever here and call memset only
# on the bytes following the m_type field?
genObjectInit(p, cpsStmts, loc.t, loc, true)
@ -330,7 +339,7 @@ proc constructLoc(p: BProc, loc: TLoc, isTemp = false) =
if not isImportedCppType(typ):
useStringh(p.module)
linefmt(p, cpsStmts, "memset((void*)$1, 0, sizeof($2));$n",
addrLoc(loc), rdLoc(loc))
addrLoc(loc), getTypeDesc(p.module, typ))
genObjectInit(p, cpsStmts, loc.t, loc, true)
proc initLocalVar(p: BProc, v: PSym, immediateAsgn: bool) =
@ -361,7 +370,7 @@ proc getIntTemp(p: BProc, result: var TLoc) =
linefmt(p, cpsLocals, "NI $1;$n", result.r)
result.k = locTemp
result.storage = OnStack
result.lode = lodeTyp getSysType(tyInt)
result.lode = lodeTyp getSysType(p.module.g.graph, unknownLineInfo(), tyInt)
result.flags = {}
proc initGCFrame(p: BProc): Rope =
@ -405,7 +414,7 @@ proc assignLocalVar(p: BProc, n: PNode) =
#assert(s.loc.k == locNone) # not yet assigned
# this need not be fulfilled for inline procs; they are regenerated
# for each module that uses them!
let nl = if optLineDir in gOptions: "" else: tnl
let nl = if optLineDir in p.config.options: "" else: tnl
let decl = localVarDecl(p, n) & ";" & nl
line(p, cpsLocals, decl)
localDebugInfo(p, n.sym)
@ -499,24 +508,24 @@ proc initFrame(p: BProc, procname, filename: Rope): Rope =
discard cgsym(p.module, "nimFrame")
if p.maxFrameLen > 0:
discard cgsym(p.module, "VarSlot")
result = rfmt(nil, "\tnimfrs_($1, $2, $3, $4);$n",
result = ropecg(p.module, "\tnimfrs_($1, $2, $3, $4);$n",
procname, filename, p.maxFrameLen.rope,
p.blocks[0].frameLen.rope)
else:
result = rfmt(nil, "\tnimfr_($1, $2);$n", procname, filename)
result = ropecg(p.module, "\tnimfr_($1, $2);$n", procname, filename)
proc initFrameNoDebug(p: BProc; frame, procname, filename: Rope; line: int): Rope =
discard cgsym(p.module, "nimFrame")
addf(p.blocks[0].sections[cpsLocals], "TFrame $1;$n", [frame])
result = rfmt(nil, "\t$1.procname = $2; $1.filename = $3; " &
result = ropecg(p.module, "\t$1.procname = $2; $1.filename = $3; " &
" $1.line = $4; $1.len = -1; nimFrame(&$1);$n",
frame, procname, filename, rope(line))
proc deinitFrameNoDebug(p: BProc; frame: Rope): Rope =
result = rfmt(p.module, "\t#popFrameOfAddr(&$1);$n", frame)
result = ropecg(p.module, "\t#popFrameOfAddr(&$1);$n", frame)
proc deinitFrame(p: BProc): Rope =
result = rfmt(p.module, "\t#popFrame();$n")
result = ropecg(p.module, "\t#popFrame();$n")
include ccgexprs
@ -539,16 +548,18 @@ proc loadDynamicLib(m: BModule, lib: PLib) =
if lib.path.kind in {nkStrLit..nkTripleStrLit}:
var s: TStringSeq = @[]
libCandidates(lib.path.strVal, s)
rawMessage(hintDependency, lib.path.strVal)
rawMessage(m.config, hintDependency, lib.path.strVal)
var loadlib: Rope = nil
for i in countup(0, high(s)):
inc(m.labels)
if i > 0: add(loadlib, "||")
appcg(m, loadlib, "($1 = #nimLoadLibrary((#NimStringDesc*) &$2))$n",
[tmp, getStrLit(m, s[i])])
let n = newStrNode(nkStrLit, s[i])
n.info = lib.path.info
appcg(m, loadlib, "($1 = #nimLoadLibrary($2))$n",
[tmp, genStringLiteral(m, n)])
appcg(m, m.s[cfsDynLibInit],
"if (!($1)) #nimLoadLibraryError((#NimStringDesc*) &$2);$n",
[loadlib, getStrLit(m, lib.path.strVal)])
"if (!($1)) #nimLoadLibraryError($2);$n",
[loadlib, genStringLiteral(m, lib.path)])
else:
var p = newProc(nil, m)
p.options = p.options - {optStackTrace, optEndb}
@ -561,7 +572,7 @@ proc loadDynamicLib(m: BModule, lib: PLib) =
"if (!($1 = #nimLoadLibrary($2))) #nimLoadLibraryError($2);$n",
[tmp, rdLoc(dest)])
if lib.name == nil: internalError("loadDynamicLib")
if lib.name == nil: internalError(m.config, "loadDynamicLib")
proc mangleDynLibProc(sym: PSym): Rope =
if sfCompilerProc in sym.flags:
@ -592,14 +603,14 @@ proc symInDynamicLib(m: BModule, sym: PSym) =
[tmp, getTypeDesc(m, sym.typ), params, makeCString($extname)]
var last = lastSon(n)
if last.kind == nkHiddenStdConv: last = last.sons[1]
internalAssert(last.kind == nkStrLit)
internalAssert(m.config, last.kind == nkStrLit)
let idx = last.strVal
if idx.len == 0:
add(m.initProc.s(cpsStmts), load)
elif idx.len == 1 and idx[0] in {'0'..'9'}:
add(m.extensionLoaders[idx[0]], load)
else:
internalError(sym.info, "wrong index: " & idx)
internalError(m.config, sym.info, "wrong index: " & idx)
else:
appcg(m, m.s[cfsDynLibInit],
"\t$1 = ($2) #nimGetProcAddr($3, $4);$n",
@ -625,18 +636,18 @@ proc symInDynamicLibPartial(m: BModule, sym: PSym) =
sym.typ.sym = nil # generate a new name
proc cgsym(m: BModule, name: string): Rope =
let sym = magicsys.getCompilerProc(name)
let sym = magicsys.getCompilerProc(m.g.graph, name)
if sym != nil:
case sym.kind
of skProc, skFunc, skMethod, skConverter, skIterator: genProc(m, sym)
of skVar, skResult, skLet: genVarPrototype(m, newSymNode sym)
of skType: discard getTypeDesc(m, sym.typ)
else: internalError("cgsym: " & name & ": " & $sym.kind)
else: internalError(m.config, "cgsym: " & name & ": " & $sym.kind)
else:
# we used to exclude the system module from this check, but for DLL
# generation support this sloppyness leads to hard to detect bugs, so
# we're picky here for the system module too:
rawMessage(errSystemNeeds, name)
rawMessage(m.config, errGenerated, "system module needs: " & name)
result = sym.loc.r
proc generateHeaders(m: BModule) =
@ -662,12 +673,18 @@ proc generateHeaders(m: BModule) =
add(m.s[cfsHeaders], "#undef powerpc" & tnl)
add(m.s[cfsHeaders], "#undef unix" & tnl)
proc openNamespaceNim(): Rope =
result.add("namespace Nim {" & tnl)
proc closeNamespaceNim(): Rope =
result.add("}" & tnl)
proc closureSetup(p: BProc, prc: PSym) =
if tfCapturesEnv notin prc.typ.flags: return
# prc.ast[paramsPos].last contains the type we're after:
var ls = lastSon(prc.ast[paramsPos])
if ls.kind != nkSym:
internalError(prc.info, "closure generation failed")
internalError(p.config, prc.info, "closure generation failed")
var env = ls.sym
#echo "created environment: ", env.id, " for ", prc.name.s
assignLocalVar(p, ls)
@ -707,7 +724,7 @@ proc genProcAux(m: BModule, prc: PSym) =
assert(prc.ast != nil)
if sfPure notin prc.flags and prc.typ.sons[0] != nil:
if resultPos >= prc.ast.len:
internalError(prc.info, "proc has no result symbol")
internalError(m.config, prc.info, "proc has no result symbol")
let resNode = prc.ast.sons[resultPos]
let res = resNode.sym # get result symbol
if not isInvalidReturnType(prc.typ.sons[0]):
@ -722,10 +739,11 @@ proc genProcAux(m: BModule, prc: PSym) =
assignLocalVar(p, resNode)
assert(res.loc.r != nil)
initLocalVar(p, res, immediateAsgn=false)
returnStmt = rfmt(nil, "\treturn $1;$n", rdLoc(res.loc))
returnStmt = ropecg(p.module, "\treturn $1;$n", rdLoc(res.loc))
else:
fillResult(resNode)
assignParam(p, res)
if sfNoInit notin prc.flags: resetLoc(p, res.loc)
if skipTypes(res.typ, abstractInst).kind == tyArray:
#incl(res.loc.flags, lfIndirect)
res.loc.storage = OnUnknown
@ -743,15 +761,15 @@ proc genProcAux(m: BModule, prc: PSym) =
if sfPure in prc.flags:
if hasDeclspec in extccomp.CC[extccomp.cCompiler].props:
header = "__declspec(naked) " & header
generatedProc = rfmt(nil, "$N$1 {$n$2$3$4}$N$N",
generatedProc = ropecg(p.module, "$N$1 {$n$2$3$4}$N$N",
header, p.s(cpsLocals), p.s(cpsInit), p.s(cpsStmts))
else:
generatedProc = rfmt(nil, "$N$1 {$N", header)
generatedProc = ropecg(p.module, "$N$1 {$N", header)
add(generatedProc, initGCFrame(p))
if optStackTrace in prc.options:
add(generatedProc, p.s(cpsLocals))
var procname = makeCString(prc.name.s)
add(generatedProc, initFrame(p, procname, prc.info.quotedFilename))
add(generatedProc, initFrame(p, procname, quotedFilename(p.config, prc.info)))
else:
add(generatedProc, p.s(cpsLocals))
if optProfiler in prc.options:
@ -770,10 +788,10 @@ proc genProcAux(m: BModule, prc: PSym) =
proc requiresExternC(m: BModule; sym: PSym): bool {.inline.} =
result = (sfCompileToCpp in m.module.flags and
sfCompileToCpp notin sym.getModule().flags and
gCmd != cmdCompileToCpp) or (
m.config.cmd != cmdCompileToCpp) or (
sym.flags * {sfImportc, sfInfixCall, sfCompilerProc} == {sfImportc} and
sym.magic == mNone and
gCmd == cmdCompileToCpp)
m.config.cmd == cmdCompileToCpp)
proc genProcPrototype(m: BModule, sym: PSym) =
useHeader(m, sym)
@ -781,7 +799,7 @@ proc genProcPrototype(m: BModule, sym: PSym) =
if lfDynamicLib in sym.loc.flags:
if getModule(sym).id != m.module.id and
not containsOrIncl(m.declaredThings, sym.id):
add(m.s[cfsVars], rfmt(nil, "extern $1 $2;$n",
add(m.s[cfsVars], ropecg(m, "extern $1 $2;$n",
getTypeDesc(m, sym.loc.t), mangleDynLibProc(sym)))
elif not containsOrIncl(m.declaredProtos, sym.id):
var header = genProcHeader(m, sym)
@ -793,7 +811,7 @@ proc genProcPrototype(m: BModule, sym: PSym) =
header.add(" __attribute__((naked))")
if sfNoReturn in sym.flags and hasAttribute in CC[cCompiler].props:
header.add(" __attribute__((noreturn))")
add(m.s[cfsProcHeaders], rfmt(nil, "$1;$n", header))
add(m.s[cfsProcHeaders], ropecg(m, "$1;$n", header))
proc genProcNoForward(m: BModule, prc: PSym) =
if lfImportCompilerProc in prc.loc.flags:
@ -879,7 +897,7 @@ proc genProc(m: BModule, prc: PSym) =
if not containsOrIncl(m.g.generatedHeader.declaredThings, prc.id):
genProcAux(m.g.generatedHeader, prc)
proc genVarPrototypeAux(m: BModule, n: PNode) =
proc genVarPrototype(m: BModule, n: PNode) =
#assert(sfGlobal in sym.flags)
let sym = n.sym
useHeader(m, sym)
@ -899,17 +917,14 @@ proc genVarPrototypeAux(m: BModule, n: PNode) =
if sfVolatile in sym.flags: add(m.s[cfsVars], " volatile")
addf(m.s[cfsVars], " $1;$n", [sym.loc.r])
proc genVarPrototype(m: BModule, n: PNode) =
genVarPrototypeAux(m, n)
proc addIntTypes(result: var Rope) {.inline.} =
proc addIntTypes(result: var Rope; conf: ConfigRef) {.inline.} =
addf(result, "#define NIM_NEW_MANGLING_RULES" & tnl &
"#define NIM_INTBITS $1" & tnl, [
platform.CPU[targetCPU].intSize.rope])
if optUseNimNamespace in conf.globalOptions: result.add("#define USE_NIM_NAMESPACE" & tnl)
proc getCopyright(cfile: Cfile): Rope =
const copyrightYear = "2017"
if optCompileOnly in gGlobalOptions:
proc getCopyright(conf: ConfigRef; cfile: Cfile): Rope =
if optCompileOnly in conf.globalOptions:
result = ("/* Generated by Nim Compiler v$1 */$N" &
"/* (c) " & copyrightYear & " Andreas Rumpf */$N" &
"/* The generated code is subject to the original license. */$N") %
@ -924,11 +939,11 @@ proc getCopyright(cfile: Cfile): Rope =
rope(platform.OS[targetOS].name),
rope(platform.CPU[targetCPU].name),
rope(extccomp.CC[extccomp.cCompiler].name),
rope(getCompileCFileCmd(cfile))]
rope(getCompileCFileCmd(conf, cfile))]
proc getFileHeader(cfile: Cfile): Rope =
result = getCopyright(cfile)
addIntTypes(result)
proc getFileHeader(conf: ConfigRef; cfile: Cfile): Rope =
result = getCopyright(conf, cfile)
addIntTypes(result, conf)
proc genFilenames(m: BModule): Rope =
discard cgsym(m, "dbgRegisterFilename")
@ -1033,8 +1048,8 @@ proc genMainProc(m: BModule) =
var nimMain, otherMain: FormatStr
if platform.targetOS == osWindows and
gGlobalOptions * {optGenGuiApp, optGenDynLib} != {}:
if optGenGuiApp in gGlobalOptions:
m.config.globalOptions * {optGenGuiApp, optGenDynLib} != {}:
if optGenGuiApp in m.config.globalOptions:
nimMain = WinNimMain
otherMain = WinCMain
else:
@ -1044,7 +1059,7 @@ proc genMainProc(m: BModule) =
elif platform.targetOS == osGenode:
nimMain = GenodeNimMain
otherMain = ComponentConstruct
elif optGenDynLib in gGlobalOptions:
elif optGenDynLib in m.config.globalOptions:
nimMain = PosixNimDllMain
otherMain = PosixCDllMain
elif platform.targetOS == osStandalone:
@ -1054,16 +1069,16 @@ proc genMainProc(m: BModule) =
nimMain = PosixNimMain
otherMain = PosixCMain
if m.g.breakpoints != nil: discard cgsym(m, "dbgRegisterBreakpoint")
if optEndb in gOptions:
if optEndb in m.config.options:
m.g.breakpoints.add(m.genFilenames)
let initStackBottomCall =
if platform.targetOS == osStandalone or gSelectedGC == gcNone: "".rope
if platform.targetOS == osStandalone or m.config.selectedGC == gcNone: "".rope
else: ropecg(m, "\t#initStackBottomWith((void *)&inner);$N")
inc(m.labels)
appcg(m, m.s[cfsProcs], PreMainBody, [
m.g.mainDatInit, m.g.breakpoints, m.g.otherModsInit,
if emulatedThreadVars() and platform.targetOS != osStandalone:
if emulatedThreadVars(m.config) and platform.targetOS != osStandalone:
ropecg(m, "\t#initThreadVarsEmulation();$N")
else:
"".rope,
@ -1071,8 +1086,12 @@ proc genMainProc(m: BModule) =
appcg(m, m.s[cfsProcs], nimMain,
[m.g.mainModInit, initStackBottomCall, rope(m.labels)])
if optNoMain notin gGlobalOptions:
if optNoMain notin m.config.globalOptions:
if optUseNimNamespace in m.config.globalOptions:
m.s[cfsProcs].add closeNamespaceNim() & "using namespace Nim;" & tnl
appcg(m, m.s[cfsProcs], otherMain, [])
if optUseNimNamespace in m.config.globalOptions: m.s[cfsProcs].add openNamespaceNim()
proc getSomeInitName(m: PSym, suffix: string): Rope =
assert m.kind == skModule
@ -1080,7 +1099,7 @@ proc getSomeInitName(m: PSym, suffix: string): Rope =
if {sfSystemModule, sfMainModule} * m.flags == {}:
result = m.owner.name.s.mangle.rope
result.add "_"
result.add m.name.s
result.add m.name.s.mangle
result.add suffix
proc getInitName(m: PSym): Rope =
@ -1096,8 +1115,8 @@ proc registerModuleToMain(g: BModuleList; m: PSym) =
var
init = m.getInitName
datInit = m.getDatInitName
addf(g.mainModProcs, "NIM_EXTERNC N_NOINLINE(void, $1)(void);$N", [init])
addf(g.mainModProcs, "NIM_EXTERNC N_NOINLINE(void, $1)(void);$N", [datInit])
addf(g.mainModProcs, "N_LIB_PRIVATE N_NIMCALL(void, $1)(void);$N", [init])
addf(g.mainModProcs, "N_LIB_PRIVATE N_NIMCALL(void, $1)(void);$N", [datInit])
if sfSystemModule notin m.flags:
addf(g.mainDatInit, "\t$1();$N", [datInit])
let initCall = "\t$1();$N" % [init]
@ -1108,7 +1127,7 @@ proc registerModuleToMain(g: BModuleList; m: PSym) =
proc genInitCode(m: BModule) =
var initname = getInitName(m.module)
var prc = "NIM_EXTERNC N_NOINLINE(void, $1)(void) {$N" % [initname]
var prc = "N_LIB_PRIVATE N_NIMCALL(void, $1)(void) {$N" % [initname]
if m.typeNodes > 0:
appcg(m, m.s[cfsTypeInit1], "static #TNimNode $1[$2];$n",
[m.typeNodesName, rope(m.typeNodes)])
@ -1118,11 +1137,11 @@ proc genInitCode(m: BModule) =
add(prc, initGCFrame(m.initProc))
add(prc, genSectionStart(cpsLocals))
add(prc, genSectionStart(cpsLocals, m.config))
add(prc, m.preInitProc.s(cpsLocals))
add(prc, m.initProc.s(cpsLocals))
add(prc, m.postInitProc.s(cpsLocals))
add(prc, genSectionEnd(cpsLocals))
add(prc, genSectionEnd(cpsLocals, m.config))
if optStackTrace in m.initProc.options and frameDeclared notin m.flags:
# BUT: the generated init code might depend on a current frame, so
@ -1130,33 +1149,33 @@ proc genInitCode(m: BModule) =
incl m.flags, frameDeclared
if preventStackTrace notin m.flags:
var procname = makeCString(m.module.name.s)
add(prc, initFrame(m.initProc, procname, m.module.info.quotedFilename))
add(prc, initFrame(m.initProc, procname, quotedFilename(m.config, m.module.info)))
else:
add(prc, ~"\tTFrame FR_; FR_.len = 0;$N")
add(prc, genSectionStart(cpsInit))
add(prc, genSectionStart(cpsInit, m.config))
add(prc, m.preInitProc.s(cpsInit))
add(prc, m.initProc.s(cpsInit))
add(prc, m.postInitProc.s(cpsInit))
add(prc, genSectionEnd(cpsInit))
add(prc, genSectionEnd(cpsInit, m.config))
add(prc, genSectionStart(cpsStmts))
add(prc, genSectionStart(cpsStmts, m.config))
add(prc, m.preInitProc.s(cpsStmts))
add(prc, m.initProc.s(cpsStmts))
add(prc, m.postInitProc.s(cpsStmts))
add(prc, genSectionEnd(cpsStmts))
add(prc, genSectionEnd(cpsStmts, m.config))
if optStackTrace in m.initProc.options and preventStackTrace notin m.flags:
add(prc, deinitFrame(m.initProc))
add(prc, deinitGCFrame(m.initProc))
addf(prc, "}$N$N", [])
prc.addf("NIM_EXTERNC N_NOINLINE(void, $1)(void) {$N",
prc.addf("N_LIB_PRIVATE N_NIMCALL(void, $1)(void) {$N",
[getDatInitName(m.module)])
for i in cfsTypeInit1..cfsDynLibInit:
add(prc, genSectionStart(i))
add(prc, genSectionStart(i, m.config))
add(prc, m.s[i])
add(prc, genSectionEnd(i))
add(prc, genSectionEnd(i, m.config))
addf(prc, "}$N$N", [])
# we cannot simply add the init proc to ``m.s[cfsProcs]`` anymore because
@ -1171,16 +1190,19 @@ proc genInitCode(m: BModule) =
add(m.s[cfsInitProc], ex)
proc genModule(m: BModule, cfile: Cfile): Rope =
result = getFileHeader(cfile)
result = getFileHeader(m.config, cfile)
result.add(genMergeInfo(m))
generateThreadLocalStorage(m)
generateHeaders(m)
for i in countup(cfsHeaders, cfsProcs):
add(result, genSectionStart(i))
add(result, genSectionStart(i, m.config))
add(result, m.s[i])
add(result, genSectionEnd(i))
add(result, genSectionEnd(i, m.config))
if optUseNimNamespace in m.config.globalOptions and i == cfsHeaders:
result.add openNamespaceNim()
add(result, m.s[cfsInitProc])
if optUseNimNamespace in m.config.globalOptions: result.add closeNamespaceNim()
proc newPreInitProc(m: BModule): BProc =
result = newProc(nil, m)
@ -1193,10 +1215,12 @@ proc newPostInitProc(m: BModule): BProc =
result.labels = 200_000
proc initProcOptions(m: BModule): TOptions =
if sfSystemModule in m.module.flags: gOptions-{optStackTrace} else: gOptions
let opts = m.config.options
if sfSystemModule in m.module.flags: opts-{optStackTrace} else: opts
proc rawNewModule(g: BModuleList; module: PSym, filename: string): BModule =
new(result)
result.g = g
result.tmpBase = rope("TM" & $hashOwner(module) & "_")
result.headerFiles = @[]
result.declaredThings = initIntSet()
@ -1217,14 +1241,13 @@ proc rawNewModule(g: BModuleList; module: PSym, filename: string): BModule =
result.forwardedProcs = @[]
result.typeNodesName = getTempName(result)
result.nimTypesName = getTempName(result)
result.g = g
# no line tracing for the init sections of the system module so that we
# don't generate a TFrame which can confuse the stack botton initialization:
if sfSystemModule in module.flags:
incl result.flags, preventStackTrace
excl(result.preInitProc.options, optStackTrace)
excl(result.postInitProc.options, optStackTrace)
let ndiName = if optCDebug in gGlobalOptions: changeFileExt(completeCFilePath(filename), "ndi")
let ndiName = if optCDebug in g.config.globalOptions: changeFileExt(completeCFilePath(g.config, filename), "ndi")
else: ""
open(result.ndi, ndiName)
@ -1277,7 +1300,7 @@ proc resetCgenModules*(g: BModuleList) =
for m in cgenModules(g): resetModule(m)
proc rawNewModule(g: BModuleList; module: PSym): BModule =
result = rawNewModule(g, module, module.position.int32.toFullPath)
result = rawNewModule(g, module, module.position.FileIndex.toFullPath)
proc newModule(g: BModuleList; module: PSym): BModule =
# we should create only one cgen module for each module sym
@ -1285,22 +1308,19 @@ proc newModule(g: BModuleList; module: PSym): BModule =
growCache g.modules, module.position
g.modules[module.position] = result
if (optDeadCodeElim in gGlobalOptions):
if (sfDeadCodeElim in module.flags):
internalError("added pending module twice: " & module.filename)
template injectG(config) {.dirty.} =
template injectG() {.dirty.} =
if graph.backend == nil:
graph.backend = newModuleList(config)
graph.backend = newModuleList(graph)
let g = BModuleList(graph.backend)
proc myOpen(graph: ModuleGraph; module: PSym; cache: IdentCache): PPassContext =
injectG(graph.config)
injectG()
result = newModule(g, module)
if optGenIndex in gGlobalOptions and g.generatedHeader == nil:
let f = if graph.config.headerFile.len > 0: graph.config.headerFile else: gProjectFull
if optGenIndex in graph.config.globalOptions and g.generatedHeader == nil:
let f = if graph.config.headerFile.len > 0: graph.config.headerFile
else: graph.config.projectFull
g.generatedHeader = rawNewModule(g, module,
changeFileExt(completeCFilePath(f), hExt))
changeFileExt(completeCFilePath(graph.config, f), hExt))
incl g.generatedHeader.flags, isHeaderFile
proc writeHeader(m: BModule) =
@ -1311,41 +1331,44 @@ proc writeHeader(m: BModule) =
var guard = "__$1__" % [m.filename.splitFile.name.rope]
result.addf("#ifndef $1$n#define $1$n", [guard])
addIntTypes(result)
addIntTypes(result, m.config)
generateHeaders(m)
generateThreadLocalStorage(m)
for i in countup(cfsHeaders, cfsProcs):
add(result, genSectionStart(i))
add(result, genSectionStart(i, m.config))
add(result, m.s[i])
add(result, genSectionEnd(i))
add(result, genSectionEnd(i, m.config))
if optUseNimNamespace in m.config.globalOptions and i == cfsHeaders: result.add openNamespaceNim()
add(result, m.s[cfsInitProc])
if optGenDynLib in gGlobalOptions:
if optGenDynLib in m.config.globalOptions:
result.add("N_LIB_IMPORT ")
result.addf("N_CDECL(void, NimMain)(void);$n", [])
if optUseNimNamespace in m.config.globalOptions: result.add closeNamespaceNim()
result.addf("#endif /* $1 */$n", [guard])
writeRope(result, m.filename)
proc getCFile(m: BModule): string =
let ext =
if m.compileToCpp: ".cpp"
elif gCmd == cmdCompileToOC or sfCompileToObjC in m.module.flags: ".m"
elif m.config.cmd == cmdCompileToOC or sfCompileToObjC in m.module.flags: ".m"
else: ".c"
result = changeFileExt(completeCFilePath(m.cfilename.withPackageName), ext)
result = changeFileExt(completeCFilePath(m.config, withPackageName(m.config, m.cfilename)), ext)
proc myOpenCached(graph: ModuleGraph; module: PSym, rd: PRodReader): PPassContext =
injectG(graph.config)
injectG()
var m = newModule(g, module)
readMergeInfo(getCFile(m), m)
result = m
proc myProcess(b: PPassContext, n: PNode): PNode =
result = n
if b == nil or passes.skipCodegen(n): return
if b == nil: return
var m = BModule(b)
if passes.skipCodegen(m.config, n): return
m.initProc.options = initProcOptions(m)
softRnl = if optLineDir in gOptions: noRnl else: rnl
softRnl = if optLineDir in m.config.options: noRnl else: rnl
genStmts(m.initProc, n)
proc finishModule(m: BModule) =
@ -1355,18 +1378,18 @@ proc finishModule(m: BModule) =
# a ``for`` loop here
var prc = m.forwardedProcs[i]
if sfForward in prc.flags:
internalError(prc.info, "still forwarded: " & prc.name.s)
internalError(m.config, prc.info, "still forwarded: " & prc.name.s)
genProcNoForward(m, prc)
inc(i)
assert(m.g.forwardedProcsCounter >= i)
dec(m.g.forwardedProcsCounter, i)
setLen(m.forwardedProcs, 0)
proc shouldRecompile(code: Rope, cfile: Cfile): bool =
proc shouldRecompile(m: BModule; code: Rope, cfile: Cfile): bool =
result = true
if optForceFullMake notin gGlobalOptions:
if optForceFullMake notin m.config.globalOptions:
if not equalsFile(code, cfile.cname):
if isDefined("nimdiff"):
if isDefined(m.config, "nimdiff"):
if fileExists(cfile.cname):
copyFile(cfile.cname, cfile.cname & ".backup")
echo "diff ", cfile.cname, ".backup ", cfile.cname
@ -1389,7 +1412,7 @@ proc writeModule(m: BModule, pending: bool) =
# generate code for the init statements of the module:
let cfile = getCFile(m)
if m.rd == nil or optForceFullMake in gGlobalOptions:
if m.rd == nil or optForceFullMake in m.config.globalOptions:
genInitCode(m)
finishTypeDescriptions(m)
if sfMainModule in m.module.flags:
@ -1397,35 +1420,35 @@ proc writeModule(m: BModule, pending: bool) =
add(m.s[cfsProcHeaders], m.g.mainModProcs)
generateThreadVarsSize(m)
var cf = Cfile(cname: cfile, obj: completeCFilePath(toObjFile(cfile)), flags: {})
var cf = Cfile(cname: cfile, obj: completeCFilePath(m.config, toObjFile(m.config, cfile)), flags: {})
var code = genModule(m, cf)
when hasTinyCBackend:
if gCmd == cmdRun:
if conf.cmd == cmdRun:
tccgen.compileCCode($code)
return
if not shouldRecompile(code, cf): cf.flags = {CfileFlag.Cached}
addFileToCompile(cf)
if not shouldRecompile(m, code, cf): cf.flags = {CfileFlag.Cached}
addFileToCompile(m.config, cf)
elif pending and mergeRequired(m) and sfMainModule notin m.module.flags:
let cf = Cfile(cname: cfile, obj: completeCFilePath(toObjFile(cfile)), flags: {})
let cf = Cfile(cname: cfile, obj: completeCFilePath(m.config, toObjFile(m.config, cfile)), flags: {})
mergeFiles(cfile, m)
genInitCode(m)
finishTypeDescriptions(m)
var code = genModule(m, cf)
writeRope(code, cfile)
addFileToCompile(cf)
addFileToCompile(m.config, cf)
else:
# Consider: first compilation compiles ``system.nim`` and produces
# ``system.c`` but then compilation fails due to an error. This means
# that ``system.o`` is missing, so we need to call the C compiler for it:
var cf = Cfile(cname: cfile, obj: completeCFilePath(toObjFile(cfile)), flags: {})
var cf = Cfile(cname: cfile, obj: completeCFilePath(m.config, toObjFile(m.config, cfile)), flags: {})
if not existsFile(cf.obj): cf.flags = {CfileFlag.Cached}
addFileToCompile(cf)
addFileToCompile(m.config, cf)
close(m.ndi)
proc updateCachedModule(m: BModule) =
let cfile = getCFile(m)
var cf = Cfile(cname: cfile, obj: completeCFilePath(toObjFile(cfile)), flags: {})
var cf = Cfile(cname: cfile, obj: completeCFilePath(m.config, toObjFile(m.config, cfile)), flags: {})
if mergeRequired(m) and sfMainModule notin m.module.flags:
mergeFiles(cfile, m)
@ -1436,12 +1459,13 @@ proc updateCachedModule(m: BModule) =
writeRope(code, cfile)
else:
cf.flags = {CfileFlag.Cached}
addFileToCompile(cf)
addFileToCompile(m.config, cf)
proc myClose(graph: ModuleGraph; b: PPassContext, n: PNode): PNode =
result = n
if b == nil or passes.skipCodegen(n): return
if b == nil: return
var m = BModule(b)
if passes.skipCodegen(m.config, n): return
# if the module is cached, we don't regenerate the main proc
# nor the dispatchers? But if the dispatchers changed?
# XXX emit the dispatchers into its own .c file?
@ -1475,7 +1499,7 @@ proc cgenWriteModules*(backend: RootRef, config: ConfigRef) =
m.updateCachedModule
else:
m.writeModule(pending=true)
writeMapping(g.mapping)
writeMapping(config, g.mapping)
if g.generatedHeader != nil: writeHeader(g.generatedHeader)
const cgenPass* = makePass(myOpen, myOpenCached, myProcess, myClose)

22
compiler/cgendata.nim
View file

@ -14,6 +14,7 @@ import
tables, ndi
from msgs import TLineInfo
from modulegraphs import ModuleGraph
type
TLabel* = Rope # for the C generator a label is just a rope
@ -70,11 +71,10 @@ type
threadVarAccessed*: bool # true if the proc already accessed some threadvar
lastLineInfo*: TLineInfo # to avoid generating excessive 'nimln' statements
currLineInfo*: TLineInfo # AST codegen will make this superfluous
nestedTryStmts*: seq[PNode] # in how many nested try statements we are
# (the vars must be volatile then)
inExceptBlock*: int # are we currently inside an except block?
# leaving such scopes by raise or by return must
# execute any applicable finally blocks
nestedTryStmts*: seq[tuple[n: PNode, inExcept: bool]]
# in how many nested try statements we are
# (the vars must be volatile then)
# bool is true when are in the except part of a try block
finallySafePoints*: seq[Rope] # For correctly cleaning up exceptions when
# using return in finally statements
labels*: Natural # for generating unique labels in the C proc
@ -117,6 +117,8 @@ type
breakpoints*: Rope # later the breakpoints are inserted into the main proc
typeInfoMarker*: TypeCache
config*: ConfigRef
graph*: ModuleGraph
strVersion*, seqVersion*: int # version of the string/seq implementation to use
TCGen = object of TPassContext # represents a C source file
s*: TCFileSections # sections of the C file
@ -148,6 +150,9 @@ type
g*: BModuleList
ndi*: NdiFile
template config*(m: BModule): ConfigRef = m.g.config
template config*(p: BProc): ConfigRef = p.module.g.config
proc includeHeader*(this: BModule; header: string) =
if not this.headerFiles.contains header:
this.headerFiles.add header
@ -165,14 +170,15 @@ proc newProc*(prc: PSym, module: BModule): BProc =
result.prc = prc
result.module = module
if prc != nil: result.options = prc.options
else: result.options = gOptions
else: result.options = module.config.options
newSeq(result.blocks, 1)
result.nestedTryStmts = @[]
result.finallySafePoints = @[]
result.sigConflicts = initCountTable[string]()
proc newModuleList*(config: ConfigRef): BModuleList =
BModuleList(modules: @[], typeInfoMarker: initTable[SigHash, Rope](), config: config)
proc newModuleList*(g: ModuleGraph): BModuleList =
BModuleList(modules: @[], typeInfoMarker: initTable[SigHash, Rope](), config: g.config,
graph: g)
iterator cgenModules*(g: BModuleList): BModule =
for i in 0..high(g.modules):

47
compiler/cgmeth.nim
View file

@ -11,9 +11,9 @@
import
intsets, options, ast, astalgo, msgs, idents, renderer, types, magicsys,
sempass2, strutils, modulegraphs
sempass2, strutils, modulegraphs, configuration
proc genConv(n: PNode, d: PType, downcast: bool): PNode =
proc genConv(n: PNode, d: PType, downcast: bool; conf: ConfigRef): PNode =
var dest = skipTypes(d, abstractPtrs)
var source = skipTypes(n.typ, abstractPtrs)
if (source.kind == tyObject) and (dest.kind == tyObject):
@ -24,12 +24,12 @@ proc genConv(n: PNode, d: PType, downcast: bool): PNode =
elif diff < 0:
result = newNodeIT(nkObjUpConv, n.info, d)
addSon(result, n)
if downcast: internalError(n.info, "cgmeth.genConv: no upcast allowed")
if downcast: internalError(conf, n.info, "cgmeth.genConv: no upcast allowed")
elif diff > 0:
result = newNodeIT(nkObjDownConv, n.info, d)
addSon(result, n)
if not downcast:
internalError(n.info, "cgmeth.genConv: no downcast allowed")
internalError(conf, n.info, "cgmeth.genConv: no downcast allowed")
else:
result = n
else:
@ -42,7 +42,7 @@ proc getDispatcher*(s: PSym): PSym =
let disp = dispn.sym
if sfDispatcher in disp.flags: result = disp
proc methodCall*(n: PNode): PNode =
proc methodCall*(n: PNode; conf: ConfigRef): PNode =
result = n
# replace ordinary method by dispatcher method:
let disp = getDispatcher(result.sons[0].sym)
@ -50,9 +50,9 @@ proc methodCall*(n: PNode): PNode =
result.sons[0].sym = disp
# change the arguments to up/downcasts to fit the dispatcher's parameters:
for i in countup(1, sonsLen(result)-1):
result.sons[i] = genConv(result.sons[i], disp.typ.sons[i], true)
result.sons[i] = genConv(result.sons[i], disp.typ.sons[i], true, conf)
else:
localError(n.info, "'" & $result.sons[0] & "' lacks a dispatcher")
localError(conf, n.info, "'" & $result.sons[0] & "' lacks a dispatcher")
type
MethodResult = enum No, Invalid, Yes
@ -68,7 +68,7 @@ proc sameMethodBucket(a, b: PSym): MethodResult =
while true:
aa = skipTypes(aa, {tyGenericInst, tyAlias})
bb = skipTypes(bb, {tyGenericInst, tyAlias})
if aa.kind == bb.kind and aa.kind in {tyVar, tyPtr, tyRef}:
if aa.kind == bb.kind and aa.kind in {tyVar, tyPtr, tyRef, tyLent}:
aa = aa.lastSon
bb = bb.lastSon
else:
@ -130,7 +130,7 @@ proc createDispatcher(s: PSym): PSym =
attachDispatcher(disp, newSymNode(disp))
return disp
proc fixupDispatcher(meth, disp: PSym) =
proc fixupDispatcher(meth, disp: PSym; conf: ConfigRef) =
# We may have constructed the dispatcher from a method prototype
# and need to augment the incomplete dispatcher with information
# from later definitions, particularly the resultPos slot. Also,
@ -149,7 +149,7 @@ proc fixupDispatcher(meth, disp: PSym) =
disp.typ.lockLevel = meth.typ.lockLevel
elif meth.typ.lockLevel != UnspecifiedLockLevel and
meth.typ.lockLevel != disp.typ.lockLevel:
message(meth.info, warnLockLevel,
message(conf, meth.info, warnLockLevel,
"method has lock level $1, but another method has $2" %
[$meth.typ.lockLevel, $disp.typ.lockLevel])
# XXX The following code silences a duplicate warning in
@ -166,13 +166,13 @@ proc methodDef*(g: ModuleGraph; s: PSym, fromCache: bool) =
of Yes:
add(g.methods[i].methods, s)
attachDispatcher(s, lastSon(disp.ast))
fixupDispatcher(s, disp)
fixupDispatcher(s, disp, g.config)
#echo "fixup ", disp.name.s, " ", disp.id
when useEffectSystem: checkMethodEffects(disp, s)
when useEffectSystem: checkMethodEffects(g, disp, s)
if {sfBase, sfFromGeneric} * s.flags == {sfBase} and
g.methods[i].methods[0] != s:
# already exists due to forwarding definition?
localError(s.info, "method is not a base")
localError(g.config, s.info, "method is not a base")
return
of No: discard
of Invalid:
@ -183,10 +183,10 @@ proc methodDef*(g: ModuleGraph; s: PSym, fromCache: bool) =
#if fromCache:
# internalError(s.info, "no method dispatcher found")
if witness != nil:
localError(s.info, "invalid declaration order; cannot attach '" & s.name.s &
localError(g.config, s.info, "invalid declaration order; cannot attach '" & s.name.s &
"' to method defined here: " & $witness.info)
elif sfBase notin s.flags:
message(s.info, warnUseBase)
message(g.config, s.info, warnUseBase)
proc relevantCol(methods: TSymSeq, col: int): bool =
# returns true iff the position is relevant
@ -225,32 +225,33 @@ proc sortBucket(a: var TSymSeq, relevantCols: IntSet) =
a[j] = v
if h == 1: break
proc genDispatcher(methods: TSymSeq, relevantCols: IntSet): PSym =
proc genDispatcher(g: ModuleGraph; methods: TSymSeq, relevantCols: IntSet): PSym =
var base = lastSon(methods[0].ast).sym
result = base
var paramLen = sonsLen(base.typ)
var nilchecks = newNodeI(nkStmtList, base.info)
var disp = newNodeI(nkIfStmt, base.info)
var ands = getSysSym("and")
var iss = getSysSym("of")
var ands = getSysSym(g, unknownLineInfo(), "and")
var iss = getSysSym(g, unknownLineInfo(), "of")
let boolType = getSysType(g, unknownLineInfo(), tyBool)
for col in countup(1, paramLen - 1):
if contains(relevantCols, col):
let param = base.typ.n.sons[col].sym
if param.typ.skipTypes(abstractInst).kind in {tyRef, tyPtr}:
addSon(nilchecks, newTree(nkCall,
newSymNode(getCompilerProc"chckNilDisp"), newSymNode(param)))
newSymNode(getCompilerProc(g, "chckNilDisp")), newSymNode(param)))
for meth in countup(0, high(methods)):
var curr = methods[meth] # generate condition:
var cond: PNode = nil
for col in countup(1, paramLen - 1):
if contains(relevantCols, col):
var isn = newNodeIT(nkCall, base.info, getSysType(tyBool))
var isn = newNodeIT(nkCall, base.info, boolType)
addSon(isn, newSymNode(iss))
let param = base.typ.n.sons[col].sym
addSon(isn, newSymNode(param))
addSon(isn, newNodeIT(nkType, base.info, curr.typ.sons[col]))
if cond != nil:
var a = newNodeIT(nkCall, base.info, getSysType(tyBool))
var a = newNodeIT(nkCall, base.info, boolType)
addSon(a, newSymNode(ands))
addSon(a, cond)
addSon(a, isn)
@ -262,7 +263,7 @@ proc genDispatcher(methods: TSymSeq, relevantCols: IntSet): PSym =
addSon(call, newSymNode(curr))
for col in countup(1, paramLen - 1):
addSon(call, genConv(newSymNode(base.typ.n.sons[col].sym),
curr.typ.sons[col], false))
curr.typ.sons[col], false, g.config))
var ret: PNode
if retTyp != nil:
var a = newNodeI(nkFastAsgn, base.info)
@ -290,4 +291,4 @@ proc generateMethodDispatchers*(g: ModuleGraph): PNode =
if relevantCol(g.methods[bucket].methods, col): incl(relevantCols, col)
sortBucket(g.methods[bucket].methods, relevantCols)
addSon(result,
newSymNode(genDispatcher(g.methods[bucket].methods, relevantCols)))
newSymNode(genDispatcher(g, g.methods[bucket].methods, relevantCols)))

1287
compiler/closureiters.nim Normal file
View file

File diff suppressed because it is too large Load diff

860
compiler/commands.nim
View file

File diff suppressed because it is too large Load diff

78
compiler/condsyms.nim
View file

@ -12,78 +12,30 @@
import
strtabs, platform, strutils, idents
# We need to use a StringTableRef here as defined symbols are always guaranteed
# to be style insensitive. Otherwise hell would break lose.
var gSymbols: StringTableRef
const
catNone = "false"
proc defineSymbol*(symbol: string, value: string = "true") =
gSymbols[symbol] = value
proc defineSymbol*(symbols: StringTableRef; symbol: string, value: string = "true") =
symbols[symbol] = value
proc undefSymbol*(symbol: string) =
gSymbols[symbol] = catNone
proc undefSymbol*(symbols: StringTableRef; symbol: string) =
symbols[symbol] = catNone
proc isDefined*(symbol: string): bool =
if gSymbols.hasKey(symbol):
result = gSymbols[symbol] != catNone
elif cmpIgnoreStyle(symbol, CPU[targetCPU].name) == 0:
result = true
elif cmpIgnoreStyle(symbol, platform.OS[targetOS].name) == 0:
result = true
else:
case symbol.normalize
of "x86": result = targetCPU == cpuI386
of "itanium": result = targetCPU == cpuIa64
of "x8664": result = targetCPU == cpuAmd64
of "posix", "unix":
result = targetOS in {osLinux, osMorphos, osSkyos, osIrix, osPalmos,
osQnx, osAtari, osAix,
osHaiku, osVxWorks, osSolaris, osNetbsd,
osFreebsd, osOpenbsd, osDragonfly, osMacosx,
osAndroid}
of "linux":
result = targetOS in {osLinux, osAndroid}
of "bsd":
result = targetOS in {osNetbsd, osFreebsd, osOpenbsd, osDragonfly}
of "emulatedthreadvars":
result = platform.OS[targetOS].props.contains(ospLacksThreadVars)
of "msdos": result = targetOS == osDos
of "mswindows", "win32": result = targetOS == osWindows
of "macintosh": result = targetOS in {osMacos, osMacosx}
of "sunos": result = targetOS == osSolaris
of "littleendian": result = CPU[targetCPU].endian == platform.littleEndian
of "bigendian": result = CPU[targetCPU].endian == platform.bigEndian
of "cpu8": result = CPU[targetCPU].bit == 8
of "cpu16": result = CPU[targetCPU].bit == 16
of "cpu32": result = CPU[targetCPU].bit == 32
of "cpu64": result = CPU[targetCPU].bit == 64
of "nimrawsetjmp":
result = targetOS in {osSolaris, osNetbsd, osFreebsd, osOpenbsd,
osDragonfly, osMacosx}
else: discard
#proc lookupSymbol*(symbols: StringTableRef; symbol: string): string =
# result = if isDefined(symbol): gSymbols[symbol] else: nil
proc isDefined*(symbol: PIdent): bool = isDefined(symbol.s)
proc lookupSymbol*(symbol: string): string =
result = if isDefined(symbol): gSymbols[symbol] else: nil
proc lookupSymbol*(symbol: PIdent): string = lookupSymbol(symbol.s)
iterator definedSymbolNames*: string =
for key, val in pairs(gSymbols):
iterator definedSymbolNames*(symbols: StringTableRef): string =
for key, val in pairs(symbols):
if val != catNone: yield key
proc countDefinedSymbols*(): int =
proc countDefinedSymbols*(symbols: StringTableRef): int =
result = 0
for key, val in pairs(gSymbols):
for key, val in pairs(symbols):
if val != catNone: inc(result)
proc initDefines*() =
gSymbols = newStringTable(modeStyleInsensitive)
defineSymbol("nimrod") # 'nimrod' is always defined
proc initDefines*(symbols: StringTableRef) =
# for bootstrapping purposes and old code:
template defineSymbol(s) = symbols.defineSymbol(s)
defineSymbol("nimhygiene")
defineSymbol("niminheritable")
defineSymbol("nimmixin")
@ -112,3 +64,9 @@ proc initDefines*() =
defineSymbol("nimNewRoof")
defineSymbol("nimHasRunnableExamples")
defineSymbol("nimNewDot")
defineSymbol("nimHasNilChecks")
defineSymbol("nimSymKind")
defineSymbol("nimVmEqIdent")
defineSymbol("nimNoNil")
defineSymbol("nimNoZeroTerminator")
defineSymbol("nimNotNil")

360
compiler/configuration.nim Normal file
View file

@ -0,0 +1,360 @@
#
#
# The Nim Compiler
# (c) Copyright 2018 Andreas Rumpf
#
# See the file "copying.txt", included in this
# distribution, for details about the copyright.
#
## This module contains the rather excessive configuration object that
## needs to be passed around to everything so that the compiler becomes
## more useful as a library.
import tables
const
explanationsBaseUrl* = "https://nim-lang.org/docs/manual"
type
TMsgKind* = enum
errUnknown, errInternal, errIllFormedAstX, errCannotOpenFile,
errXExpected,
errGridTableNotImplemented,
errGeneralParseError,
errNewSectionExpected,
errInvalidDirectiveX,
errGenerated,
errUser,
warnCannotOpenFile,
warnOctalEscape, warnXIsNeverRead, warnXmightNotBeenInit,
warnDeprecated, warnConfigDeprecated,
warnSmallLshouldNotBeUsed, warnUnknownMagic, warnRedefinitionOfLabel,
warnUnknownSubstitutionX, warnLanguageXNotSupported,
warnFieldXNotSupported, warnCommentXIgnored,
warnTypelessParam,
warnUseBase, warnWriteToForeignHeap, warnUnsafeCode,
warnEachIdentIsTuple, warnShadowIdent,
warnProveInit, warnProveField, warnProveIndex, warnGcUnsafe, warnGcUnsafe2,
warnUninit, warnGcMem, warnDestructor, warnLockLevel, warnResultShadowed,
warnInconsistentSpacing, warnUser,
hintSuccess, hintSuccessX,
hintLineTooLong, hintXDeclaredButNotUsed, hintConvToBaseNotNeeded,
hintConvFromXtoItselfNotNeeded, hintExprAlwaysX, hintQuitCalled,
hintProcessing, hintCodeBegin, hintCodeEnd, hintConf, hintPath,
hintConditionAlwaysTrue, hintName, hintPattern,
hintExecuting, hintLinking, hintDependency,
hintSource, hintPerformance, hintStackTrace, hintGCStats,
hintUser, hintUserRaw
const
MsgKindToStr*: array[TMsgKind, string] = [
errUnknown: "unknown error",
errInternal: "internal error: $1",
errIllFormedAstX: "illformed AST: $1",
errCannotOpenFile: "cannot open '$1'",
errXExpected: "'$1' expected",
errGridTableNotImplemented: "grid table is not implemented",
errGeneralParseError: "general parse error",
errNewSectionExpected: "new section expected",
errInvalidDirectiveX: "invalid directive: '$1'",
errGenerated: "$1",
errUser: "$1",
warnCannotOpenFile: "cannot open '$1'",
warnOctalEscape: "octal escape sequences do not exist; leading zero is ignored",
warnXIsNeverRead: "'$1' is never read",
warnXmightNotBeenInit: "'$1' might not have been initialized",
warnDeprecated: "$1 is deprecated",
warnConfigDeprecated: "config file '$1' is deprecated",
warnSmallLshouldNotBeUsed: "'l' should not be used as an identifier; may look like '1' (one)",
warnUnknownMagic: "unknown magic '$1' might crash the compiler",
warnRedefinitionOfLabel: "redefinition of label '$1'",
warnUnknownSubstitutionX: "unknown substitution '$1'",
warnLanguageXNotSupported: "language '$1' not supported",
warnFieldXNotSupported: "field '$1' not supported",
warnCommentXIgnored: "comment '$1' ignored",
warnTypelessParam: "'$1' has no type. Typeless parameters are deprecated; only allowed for 'template'",
warnUseBase: "use {.base.} for base methods; baseless methods are deprecated",
warnWriteToForeignHeap: "write to foreign heap",
warnUnsafeCode: "unsafe code: '$1'",
warnEachIdentIsTuple: "each identifier is a tuple",
warnShadowIdent: "shadowed identifier: '$1'",
warnProveInit: "Cannot prove that '$1' is initialized. This will become a compile time error in the future.",
warnProveField: "cannot prove that field '$1' is accessible",
warnProveIndex: "cannot prove index '$1' is valid",
warnGcUnsafe: "not GC-safe: '$1'",
warnGcUnsafe2: "$1",
warnUninit: "'$1' might not have been initialized",
warnGcMem: "'$1' uses GC'ed memory",
warnDestructor: "usage of a type with a destructor in a non destructible context. This will become a compile time error in the future.",
warnLockLevel: "$1",
warnResultShadowed: "Special variable 'result' is shadowed.",
warnInconsistentSpacing: "Number of spaces around '$#' is not consistent",
warnUser: "$1",
hintSuccess: "operation successful",
hintSuccessX: "operation successful ($# lines compiled; $# sec total; $#; $#)",
hintLineTooLong: "line too long",
hintXDeclaredButNotUsed: "'$1' is declared but not used",
hintConvToBaseNotNeeded: "conversion to base object is not needed",
hintConvFromXtoItselfNotNeeded: "conversion from $1 to itself is pointless",
hintExprAlwaysX: "expression evaluates always to '$1'",
hintQuitCalled: "quit() called",
hintProcessing: "$1",
hintCodeBegin: "generated code listing:",
hintCodeEnd: "end of listing",
hintConf: "used config file '$1'",
hintPath: "added path: '$1'",
hintConditionAlwaysTrue: "condition is always true: '$1'",
hintName: "name should be: '$1'",
hintPattern: "$1",
hintExecuting: "$1",
hintLinking: "",
hintDependency: "$1",
hintSource: "$1",
hintPerformance: "$1",
hintStackTrace: "$1",
hintGCStats: "$1",
hintUser: "$1",
hintUserRaw: "$1"]
const
WarningsToStr* = ["CannotOpenFile", "OctalEscape",
"XIsNeverRead", "XmightNotBeenInit",
"Deprecated", "ConfigDeprecated",
"SmallLshouldNotBeUsed", "UnknownMagic",
"RedefinitionOfLabel", "UnknownSubstitutionX",
"LanguageXNotSupported", "FieldXNotSupported",
"CommentXIgnored",
"TypelessParam", "UseBase", "WriteToForeignHeap",
"UnsafeCode", "EachIdentIsTuple", "ShadowIdent",
"ProveInit", "ProveField", "ProveIndex", "GcUnsafe", "GcUnsafe2", "Uninit",
"GcMem", "Destructor", "LockLevel", "ResultShadowed",
"Spacing", "User"]
HintsToStr* = ["Success", "SuccessX", "LineTooLong",
"XDeclaredButNotUsed", "ConvToBaseNotNeeded", "ConvFromXtoItselfNotNeeded",
"ExprAlwaysX", "QuitCalled", "Processing", "CodeBegin", "CodeEnd", "Conf",
"Path", "CondTrue", "Name", "Pattern", "Exec", "Link", "Dependency",
"Source", "Performance", "StackTrace", "GCStats",
"User", "UserRaw"]
const
fatalMin* = errUnknown
fatalMax* = errInternal
errMin* = errUnknown
errMax* = errUser
warnMin* = warnCannotOpenFile
warnMax* = pred(hintSuccess)
hintMin* = hintSuccess
hintMax* = high(TMsgKind)
static:
doAssert HintsToStr.len == ord(hintMax) - ord(hintMin) + 1
doAssert WarningsToStr.len == ord(warnMax) - ord(warnMin) + 1
type
TNoteKind* = range[warnMin..hintMax] # "notes" are warnings or hints
TNoteKinds* = set[TNoteKind]
const
NotesVerbosity*: array[0..3, TNoteKinds] = [
{low(TNoteKind)..high(TNoteKind)} - {warnShadowIdent, warnUninit,
warnProveField, warnProveIndex,
warnGcUnsafe,
hintSuccessX, hintPath, hintConf,
hintProcessing, hintPattern,
hintDependency,
hintExecuting, hintLinking,
hintCodeBegin, hintCodeEnd,
hintSource, hintStackTrace,
hintGCStats},
{low(TNoteKind)..high(TNoteKind)} - {warnShadowIdent, warnUninit,
warnProveField, warnProveIndex,
warnGcUnsafe,
hintPath,
hintDependency,
hintCodeBegin, hintCodeEnd,
hintSource, hintStackTrace,
hintGCStats},
{low(TNoteKind)..high(TNoteKind)} - {hintStackTrace, warnUninit},
{low(TNoteKind)..high(TNoteKind)}]
const
errXMustBeCompileTime* = "'$1' can only be used in compile-time context"
errArgsNeedRunOption* = "arguments can only be given if the '--run' option is selected"
#[
errStringLiteralExpected: "string literal expected",
errIntLiteralExpected: "integer literal expected",
errIdentifierExpected: "identifier expected, but found '$1'",
errNewlineExpected: "newline expected, but found '$1'",
errInvalidModuleName: "invalid module name: '$1'",
errOnOrOffExpected: "'on' or 'off' expected",
errNoneSpeedOrSizeExpected: "'none', 'speed' or 'size' expected",
errInvalidPragma: "invalid pragma",
errUnknownPragma: "unknown pragma: '$1'",
errAtPopWithoutPush: "'pop' without a 'push' pragma",
errEmptyAsm: "empty asm statement",
errInvalidIndentation: "invalid indentation",
errNoReturnWithReturnTypeNotAllowed: "routines with NoReturn pragma are not allowed to have return type",
errAttemptToRedefine: ,
errStmtInvalidAfterReturn: "statement not allowed after 'return', 'break', 'raise', 'continue' or proc call with noreturn pragma",
errStmtExpected: "statement expected",
errInvalidLabel: "'$1' is no label",
errInvalidCmdLineOption: "invalid command line option: '$1'",
errCmdLineArgExpected: "argument for command line option expected: '$1'",
errCmdLineNoArgExpected: "invalid argument for command line option: '$1'",
errInvalidVarSubstitution: "invalid variable substitution in '$1'",
errUnknownVar: "unknown variable: '$1'",
errUnknownCcompiler: "unknown C compiler: '$1'",
errOnOrOffExpectedButXFound: "'on' or 'off' expected, but '$1' found",
errOnOffOrListExpectedButXFound: "'on', 'off' or 'list' expected, but '$1' found",
errGenOutExpectedButXFound: "'c', 'c++' or 'yaml' expected, but '$1' found",
,
errInvalidMultipleAsgn: "multiple assignment is not allowed",
errColonOrEqualsExpected: "':' or '=' expected, but found '$1'",
errUndeclaredField: "undeclared field: '$1'",
errUndeclaredRoutine: "attempting to call undeclared routine: '$1'",
errUseQualifier: "ambiguous identifier: '$1' -- use a qualifier",
errTypeExpected: "type expected",
errSystemNeeds: "system module needs '$1'",
errExecutionOfProgramFailed: "execution of an external program failed: '$1'",
errNotOverloadable: ,
errInvalidArgForX: "invalid argument for '$1'",
errStmtHasNoEffect: "statement has no effect",
,
errXExpectsArrayType: "'$1' expects an array type",
errIteratorCannotBeInstantiated: "'$1' cannot be instantiated because its body has not been compiled yet",
errExprXAmbiguous: "expression '$1' ambiguous in this context",
errConstantDivisionByZero: ,
errOrdinalOrFloatTypeExpected: "ordinal or float type expected",
errOverOrUnderflow: ,
errCannotEvalXBecauseIncompletelyDefined: ,
errChrExpectsRange0_255: "'chr' expects an int in the range 0..255",
errDynlibRequiresExportc: "'dynlib' requires 'exportc'",
errNilAccess: "attempt to access a nil address",
errIndexOutOfBounds: "index out of bounds",
errIndexTypesDoNotMatch: "index types do not match",
errBracketsInvalidForType: "'[]' operator invalid for this type",
errValueOutOfSetBounds: "value out of set bounds",
errFieldNotInit: "field '$1' not initialized",
errExprXCannotBeCalled: "expression '$1' cannot be called",
errExprHasNoType: "expression has no type",
errExprXHasNoType:,
errCastNotInSafeMode: "'cast' not allowed in safe mode",
errExprCannotBeCastToX: ,
errCommaOrParRiExpected: "',' or ')' expected",
errCurlyLeOrParLeExpected: "'{' or '(' expected",
errSectionExpected: "section ('type', 'proc', etc.) expected",
errRangeExpected: "range expected",
errMagicOnlyInSystem: "'magic' only allowed in system module",
errPowerOfTwoExpected: "power of two expected",
errStringMayNotBeEmpty: "string literal may not be empty",
errCallConvExpected: "calling convention expected",
errProcOnlyOneCallConv: "a proc can only have one calling convention",
errSymbolMustBeImported: "symbol must be imported if 'lib' pragma is used",
errExprMustBeBool: "expression must be of type 'bool'",
errConstExprExpected: "constant expression expected",
errDuplicateCaseLabel: "duplicate case label",
errRangeIsEmpty: "range is empty",
,
errSelectorMustBeOrdinal: "selector must be of an ordinal type",
errOrdXMustNotBeNegative: "ord($1) must not be negative",
errLenXinvalid: "len($1) must be less than 32768",
errTypeXhasUnknownSize: "type '$1' has unknown size",
errConstNeedsConstExpr: "a constant can only be initialized with a constant expression",
errConstNeedsValue: "a constant needs a value",
errResultCannotBeOpenArray: "the result type cannot be on open array",
errSizeTooBig: "computing the type's size produced an overflow",
errInheritanceOnlyWithEnums: "inheritance only works with an enum",
errIllegalRecursionInTypeX:,
errCannotInstantiateX: "cannot instantiate: '$1'",
errTypeMismatch: "type mismatch: got <",
errButExpected: "but expected one of: ",
errButExpectedX: "but expected '$1'",
errAmbiguousCallXYZ: "ambiguous call; both $1 and $2 match for: $3",
errWrongNumberOfArguments: "wrong number of arguments",
errWrongNumberOfArgumentsInCall: "wrong number of arguments in call to '$1'",
errMissingGenericParamsForTemplate: "'$1' has unspecified generic parameters",
errXCannotBePassedToProcVar: ,
,
errImplOfXexpected: ,
errIllegalConvFromXtoY: ,
errCannotBindXTwice: "cannot bind parameter '$1' twice",
errInvalidOrderInArrayConstructor: ,
errInvalidOrderInEnumX: "invalid order in enum '$1'",
errEnumXHasHoles: "enum '$1' has holes",
errExceptExpected: "'except' or 'finally' expected",
errInvalidTry: "after catch all 'except' or 'finally' no section may follow",
errOptionExpected: ,
errXisNoLabel: "'$1' is not a label",
errNotAllCasesCovered: "not all cases are covered",
errUnknownSubstitionVar: "unknown substitution variable: '$1'",
errComplexStmtRequiresInd: "complex statement requires indentation",
errXisNotCallable: "'$1' is not callable",
errNoPragmasAllowedForX: "no pragmas allowed for $1",
,
errInvalidParamKindX: "invalid param kind: '$1'",
errDefaultArgumentInvalid: "default argument invalid",
errNamedParamHasToBeIdent: "named parameter has to be an identifier",
errNoReturnTypeForX: "no return type allowed for $1",
errConvNeedsOneArg: "a type conversion needs exactly one argument",
errInvalidPragmaX: ,
errXNotAllowedHere: "$1 not allowed here",
errXisNoType: "invalid type: '$1'",
errCircumNeedsPointer: "'[]' needs a pointer or reference type",
errInvalidExpression: "invalid expression",
errInvalidExpressionX: "invalid expression: '$1'",
errEnumHasNoValueX: "enum has no value '$1'",
,
errNoCommand: "no command given",
errInvalidCommandX: "invalid command: '$1'",
errXNeedsParamObjectType: ,
errTemplateInstantiationTooNested: "template instantiation too nested, try --evalTemplateLimit:N",
errMacroInstantiationTooNested: "macro instantiation too nested, try --evalMacroLimit:N",
errInstantiationFrom: "template/generic instantiation from here",
errInvalidIndexValueForTuple: "invalid index value for tuple subscript",
errCommandExpectsFilename: "command expects a filename argument",
errMainModuleMustBeSpecified: "please, specify a main module in the project configuration file",
errXExpected: "'$1' expected",
,
errCastToANonConcreteType: "cannot cast to a non concrete type: '$1'",
errInvalidSectionStart: "invalid section start",
errGridTableNotImplemented: "grid table is not implemented",
errGeneralParseError: "general parse error",
errNewSectionExpected: "new section expected",
errWhitespaceExpected: "whitespace expected, got '$1'",
errXisNoValidIndexFile: "'$1' is no valid index file",
errCannotRenderX: "cannot render reStructuredText element '$1'",
errVarVarTypeNotAllowed: ,
errInstantiateXExplicitly: "instantiate '$1' explicitly",
errOnlyACallOpCanBeDelegator: ,
errUsingNoSymbol: "'$1' is not a variable, constant or a proc name",
errMacroBodyDependsOnGenericTypes: "the macro body cannot be compiled, " &
"because the parameter '$1' has a generic type",
errDestructorNotGenericEnough: "Destructor signature is too specific. " &
"A destructor must be associated will all instantiations of a generic type",
errInlineIteratorsAsProcParams: "inline iterators can be used as parameters only for " &
"templates, macros and other inline iterators",
errXExpectsTwoArguments: "'$1' expects two arguments",
errXExpectsObjectTypes: "'$1' expects object types",
errXcanNeverBeOfThisSubtype: "'$1' can never be of this subtype",
errTooManyIterations: "interpretation requires too many iterations; " &
"if you are sure this is not a bug in your code edit " &
"compiler/vmdef.MaxLoopIterations and rebuild the compiler",
errFieldXNotFound: "field '$1' cannot be found",
errInvalidConversionFromTypeX: "invalid conversion from type '$1'",
errAssertionFailed: "assertion failed",
errCannotGenerateCodeForX: "cannot generate code for '$1'",
errXRequiresOneArgument: "$1 requires one parameter",
errUnhandledExceptionX: "unhandled exception: $1",
errCyclicTree: "macro returned a cyclic abstract syntax tree",
errXisNoMacroOrTemplate: "'$1' is no macro or template",
errXhasSideEffects: "'$1' can have side effects",
errWrongSymbolX:,
errIllegalCaptureX: "illegal capture '$1'",
errXCannotBeClosure: "'$1' cannot have 'closure' calling convention",
,
]#

6
compiler/depends.nim
View file

@ -19,6 +19,7 @@ proc generateDot*(project: string)
type
TGen = object of TPassContext
module*: PSym
config: ConfigRef
PGen = ref TGen
var gDotGraph: Rope # the generated DOT file; we need a global variable
@ -33,10 +34,10 @@ proc addDotDependency(c: PPassContext, n: PNode): PNode =
case n.kind
of nkImportStmt:
for i in countup(0, sonsLen(n) - 1):
var imported = getModuleName(n.sons[i])
var imported = getModuleName(g.config, n.sons[i])
addDependencyAux(g.module.name.s, imported)
of nkFromStmt, nkImportExceptStmt:
var imported = getModuleName(n.sons[0])
var imported = getModuleName(g.config, n.sons[0])
addDependencyAux(g.module.name.s, imported)
of nkStmtList, nkBlockStmt, nkStmtListExpr, nkBlockExpr:
for i in countup(0, sonsLen(n) - 1): discard addDotDependency(c, n.sons[i])
@ -52,6 +53,7 @@ proc myOpen(graph: ModuleGraph; module: PSym; cache: IdentCache): PPassContext =
var g: PGen
new(g)
g.module = module
g.config = graph.config
result = g
const gendependPass* = makePass(open = myOpen, process = addDotDependency)

194
compiler/destroyer.nim
View file

@ -89,11 +89,35 @@
## tmp.bar))
## destroy(tmp.bar)
## destroy(tmp.x); destroy(tmp.y)
##
##[
From https://github.com/nim-lang/Nim/wiki/Destructors
Rule Pattern Transformed into
---- ------- ----------------
1.1 var x: T; stmts var x: T; try stmts
finally: `=destroy`(x)
1.2 var x: sink T; stmts var x: sink T; stmts; ensureEmpty(x)
2 x = f() `=sink`(x, f())
3 x = lastReadOf z `=sink`(x, z)
4.1 y = sinkParam `=sink`(y, sinkParam)
4.2 x = y `=`(x, y) # a copy
5.1 f_sink(g()) f_sink(g())
5.2 f_sink(y) f_sink(copy y); # copy unless we can see it's the last read
5.3 f_sink(move y) f_sink(y); reset(y) # explicit moves empties 'y'
5.4 f_noSink(g()) var tmp = bitwiseCopy(g()); f(tmp); `=destroy`(tmp)
Remarks: Rule 1.2 is not yet implemented because ``sink`` is currently
not allowed as a local variable.
``move`` builtin needs to be implemented.
]##
import
intsets, ast, astalgo, msgs, renderer, magicsys, types, idents, trees,
strutils, options, dfa, lowerings, rodread
strutils, options, dfa, lowerings, rodread, tables, modulegraphs,
configuration
const
InterestingSyms = {skVar, skResult, skLet}
@ -106,6 +130,15 @@ type
tmpObj: PType
tmp: PSym
destroys, topLevelVars: PNode
toDropBit: Table[int, PSym]
graph: ModuleGraph
proc getTemp(c: var Con; typ: PType; info: TLineInfo): PNode =
# XXX why are temps fields in an object here?
let f = newSym(skField, getIdent(":d" & $c.tmpObj.n.len), c.owner, info)
f.typ = typ
rawAddField c.tmpObj, f
result = rawDirectAccess(c.tmp, f)
proc isHarmlessVar*(s: PSym; c: Con): bool =
# 's' is harmless if it used only once and its
@ -174,7 +207,7 @@ proc patchHead(n: PNode) =
if n[1].typ.isNil:
# XXX toptree crashes without this workaround. Figure out why.
return
let t = n[1].typ.skipTypes({tyVar, tyGenericInst, tyAlias, tyInferred})
let t = n[1].typ.skipTypes({tyVar, tyLent, tyGenericInst, tyAlias, tySink, tyInferred})
template patch(op, field) =
if s.name.s == op and field != nil and field != s:
n.sons[0].sym = field
@ -191,47 +224,118 @@ proc patchHead(s: PSym) =
template genOp(opr, opname) =
let op = opr
if op == nil:
globalError(dest.info, "internal error: '" & opname & "' operator not found for type " & typeToString(t))
globalError(c.graph.config, dest.info, "internal error: '" & opname & "' operator not found for type " & typeToString(t))
elif op.ast[genericParamsPos].kind != nkEmpty:
globalError(dest.info, "internal error: '" & opname & "' operator is generic")
globalError(c.graph.config, dest.info, "internal error: '" & opname & "' operator is generic")
patchHead op
result = newTree(nkCall, newSymNode(op), newTree(nkHiddenAddr, dest))
proc genSink(t: PType; dest: PNode): PNode =
let t = t.skipTypes({tyGenericInst, tyAlias})
proc genSink(c: Con; t: PType; dest: PNode): PNode =
let t = t.skipTypes({tyGenericInst, tyAlias, tySink})
genOp(if t.sink != nil: t.sink else: t.assignment, "=sink")
proc genCopy(t: PType; dest: PNode): PNode =
let t = t.skipTypes({tyGenericInst, tyAlias})
proc genCopy(c: Con; t: PType; dest: PNode): PNode =
let t = t.skipTypes({tyGenericInst, tyAlias, tySink})
genOp(t.assignment, "=")
proc genDestroy(t: PType; dest: PNode): PNode =
let t = t.skipTypes({tyGenericInst, tyAlias})
proc genDestroy(c: Con; t: PType; dest: PNode): PNode =
let t = t.skipTypes({tyGenericInst, tyAlias, tySink})
genOp(t.destructor, "=destroy")
proc addTopVar(c: var Con; v: PNode) =
c.topLevelVars.add newTree(nkIdentDefs, v, emptyNode, emptyNode)
proc dropBit(c: var Con; s: PSym): PSym =
result = c.toDropBit.getOrDefault(s.id)
assert result != nil
proc registerDropBit(c: var Con; s: PSym) =
let result = newSym(skTemp, getIdent(s.name.s & "_AliveBit"), c.owner, s.info)
result.typ = getSysType(c.graph, s.info, tyBool)
let trueVal = newIntTypeNode(nkIntLit, 1, result.typ)
c.topLevelVars.add newTree(nkIdentDefs, newSymNode result, emptyNode, trueVal)
c.toDropBit[s.id] = result
# generate:
# if not sinkParam_AliveBit: `=destroy`(sinkParam)
c.destroys.add newTree(nkIfStmt,
newTree(nkElifBranch, newSymNode result, genDestroy(c, s.typ, newSymNode s)))
proc p(n: PNode; c: var Con): PNode
template recurse(n, dest) =
for i in 0..<n.len:
dest.add p(n[i], c)
proc isSinkParam(s: PSym): bool {.inline.} =
result = s.kind == skParam and s.typ.kind == tySink
const constrExprs = nkCallKinds+{nkObjConstr}
proc destructiveMoveSink(n: PNode; c: var Con): PNode =
# generate: (chckMove(sinkParam_AliveBit); sinkParam_AliveBit = false; sinkParam)
result = newNodeIT(nkStmtListExpr, n.info, n.typ)
let bit = newSymNode dropBit(c, n.sym)
if optMoveCheck in c.owner.options:
result.add callCodegenProc(c.graph, "chckMove", bit)
result.add newTree(nkAsgn, bit,
newIntTypeNode(nkIntLit, 0, getSysType(c.graph, n.info, tyBool)))
result.add n
proc moveOrCopy(dest, ri: PNode; c: var Con): PNode =
if ri.kind in nkCallKinds+{nkObjConstr}:
result = genSink(ri.typ, dest)
if ri.kind in constrExprs:
result = genSink(c, ri.typ, dest)
# watch out and no not transform 'ri' twice if it's a call:
let ri2 = copyNode(ri)
recurse(ri, ri2)
result.add ri2
elif ri.kind == nkSym and isHarmlessVar(ri.sym, c):
result = genSink(ri.typ, dest)
result = genSink(c, ri.typ, dest)
result.add p(ri, c)
elif ri.kind == nkSym and isSinkParam(ri.sym):
result = genSink(c, ri.typ, dest)
result.add destructiveMoveSink(ri, c)
else:
result = genCopy(ri.typ, dest)
result = genCopy(c, ri.typ, dest)
result.add p(ri, c)
proc passCopyToSink(n: PNode; c: var Con): PNode =
result = newNodeIT(nkStmtListExpr, n.info, n.typ)
let tmp = getTemp(c, n.typ, n.info)
if hasDestructor(n.typ):
var m = genCopy(c, n.typ, tmp)
m.add p(n, c)
result.add m
message(c.graph.config, n.info, hintPerformance,
"passing '$1' to a sink parameter introduces an implicit copy; " &
"use 'move($1)' to prevent it" % $n)
else:
result.add newTree(nkAsgn, tmp, p(n, c))
result.add tmp
proc genReset(n: PNode; c: var Con): PNode =
result = newNodeI(nkCall, n.info)
result.add(newSymNode(createMagic(c.graph, "reset", mReset)))
# The mReset builtin does not take the address:
result.add n
proc destructiveMoveVar(n: PNode; c: var Con): PNode =
# generate: (let tmp = v; reset(v); tmp)
result = newNodeIT(nkStmtListExpr, n.info, n.typ)
var temp = newSym(skLet, getIdent("blitTmp"), c.owner, n.info)
var v = newNodeI(nkLetSection, n.info)
let tempAsNode = newSymNode(temp)
var vpart = newNodeI(nkIdentDefs, tempAsNode.info, 3)
vpart.sons[0] = tempAsNode
vpart.sons[1] = ast.emptyNode
vpart.sons[2] = n
add(v, vpart)
result.add v
result.add genReset(n, c)
result.add tempAsNode
proc p(n: PNode; c: var Con): PNode =
case n.kind
of nkVarSection, nkLetSection:
@ -243,7 +347,7 @@ proc p(n: PNode; c: var Con): PNode =
let L = it.len-1
let ri = it[L]
if it.kind == nkVarTuple and hasDestructor(ri.typ):
let x = lowerTupleUnpacking(it, c.owner)
let x = lowerTupleUnpacking(c.graph, it, c.owner)
result.add p(x, c)
elif it.kind == nkIdentDefs and hasDestructor(it[0].typ):
for j in 0..L-2:
@ -252,7 +356,7 @@ proc p(n: PNode; c: var Con): PNode =
# move the variable declaration to the top of the frame:
c.addTopVar v
# make sure it's destroyed at the end of the proc:
c.destroys.add genDestroy(v.typ, v)
c.destroys.add genDestroy(c, v.typ, v)
if ri.kind != nkEmpty:
let r = moveOrCopy(v, ri, c)
result.add r
@ -266,22 +370,45 @@ proc p(n: PNode; c: var Con): PNode =
varSection.add itCopy
result.add varSection
of nkCallKinds:
let parameters = n[0].typ
let L = if parameters != nil: parameters.len else: 0
for i in 1 ..< n.len:
let arg = n[i]
if i < L and parameters[i].kind == tySink:
if arg.kind in nkCallKinds:
# recurse but skip the call expression in order to prevent
# destructor injections: Rule 5.1 is different from rule 5.4!
let a = copyNode(arg)
recurse(arg, a)
n.sons[i] = a
elif arg.kind in {nkObjConstr, nkCharLit..nkFloat128Lit}:
discard "object construction to sink parameter: nothing to do"
elif arg.kind == nkSym and isHarmlessVar(arg.sym, c):
# if x is a variable and it its last read we eliminate its
# destructor invokation, but don't. We need to reset its memory
# to disable its destructor which we have not elided:
n.sons[i] = destructiveMoveVar(arg, c)
elif arg.kind == nkSym and isSinkParam(arg.sym):
# mark the sink parameter as used:
n.sons[i] = destructiveMoveSink(arg, c)
else:
# an object that is not temporary but passed to a 'sink' parameter
# results in a copy.
n.sons[i] = passCopyToSink(arg, c)
else:
n.sons[i] = p(arg, c)
if n.typ != nil and hasDestructor(n.typ):
discard "produce temp creation"
result = newNodeIT(nkStmtListExpr, n.info, n.typ)
let f = newSym(skField, getIdent(":d" & $c.tmpObj.n.len), c.owner, n.info)
f.typ = n.typ
rawAddField c.tmpObj, f
var m = genSink(n.typ, rawDirectAccess(c.tmp, f))
var call = copyNode(n)
recurse(n, call)
m.add call
result.add m
result.add rawDirectAccess(c.tmp, f)
c.destroys.add genDestroy(n.typ, rawDirectAccess(c.tmp, f))
let tmp = getTemp(c, n.typ, n.info)
var sinkExpr = genSink(c, n.typ, tmp)
sinkExpr.add n
result.add sinkExpr
result.add tmp
c.destroys.add genDestroy(c, n.typ, tmp)
else:
result = copyNode(n)
recurse(n, result)
result = n
of nkAsgn, nkFastAsgn:
if hasDestructor(n[0].typ):
result = moveOrCopy(n[0], n[1], c)
@ -295,22 +422,29 @@ proc p(n: PNode; c: var Con): PNode =
result = copyNode(n)
recurse(n, result)
proc injectDestructorCalls*(owner: PSym; n: PNode): PNode =
proc injectDestructorCalls*(g: ModuleGraph; owner: PSym; n: PNode): PNode =
when defined(nimDebugDestroys):
echo "injecting into ", n
var c: Con
c.owner = owner
c.tmp = newSym(skTemp, getIdent":d", owner, n.info)
c.tmpObj = createObj(owner, n.info)
c.tmpObj = createObj(g, owner, n.info)
c.tmp.typ = c.tmpObj
c.destroys = newNodeI(nkStmtList, n.info)
c.topLevelVars = newNodeI(nkVarSection, n.info)
c.toDropBit = initTable[int, PSym]()
c.graph = g
let cfg = constructCfg(owner, n)
shallowCopy(c.g, cfg)
c.jumpTargets = initIntSet()
for i in 0..<c.g.len:
if c.g[i].kind in {goto, fork}:
c.jumpTargets.incl(i+c.g[i].dest)
if owner.kind in {skProc, skFunc, skMethod, skIterator, skConverter}:
let params = owner.typ.n
for i in 1 ..< params.len:
let param = params[i].sym
if param.typ.kind == tySink: registerDropBit(c, param)
let body = p(n, c)
if c.tmp.typ.n.len > 0:
c.addTopVar(newSymNode c.tmp)

20
compiler/dfa.nim
View file

@ -23,7 +23,7 @@
## "A Graph–Free Approach to Data–Flow Analysis" by Markus Mohnen.
## https://link.springer.com/content/pdf/10.1007/3-540-45937-5_6.pdf
import ast, astalgo, types, intsets, tables, msgs
import ast, astalgo, types, intsets, tables, msgs, options
type
InstrKind* = enum
@ -102,14 +102,14 @@ proc genLabel(c: Con): TPosition =
proc jmpBack(c: var Con, n: PNode, p = TPosition(0)) =
let dist = p.int - c.code.len
internalAssert(-0x7fff < dist and dist < 0x7fff)
doAssert(-0x7fff < dist and dist < 0x7fff)
c.code.add Instr(n: n, kind: goto, dest: dist)
proc patch(c: var Con, p: TPosition) =
# patch with current index
let p = p.int
let diff = c.code.len - p
internalAssert(-0x7fff < diff and diff < 0x7fff)
doAssert(-0x7fff < diff and diff < 0x7fff)
c.code[p].dest = diff
proc popBlock(c: var Con; oldLen: int) =
@ -160,7 +160,7 @@ proc genBreak(c: var Con; n: PNode) =
if c.blocks[i].label == n.sons[0].sym:
c.blocks[i].fixups.add L1
return
globalError(n.info, errGenerated, "VM problem: cannot find 'break' target")
#globalError(n.info, "VM problem: cannot find 'break' target")
else:
c.blocks[c.blocks.high].fixups.add L1
@ -323,7 +323,7 @@ proc gen(c: var Con; n: PNode) =
of nkBreakStmt: genBreak(c, n)
of nkTryStmt: genTry(c, n)
of nkStmtList, nkStmtListExpr, nkChckRangeF, nkChckRange64, nkChckRange,
nkBracket, nkCurly, nkPar, nkClosure, nkObjConstr:
nkBracket, nkCurly, nkPar, nkTupleConstr, nkClosure, nkObjConstr:
for x in n: gen(c, x)
of nkPragmaBlock: gen(c, n.lastSon)
of nkDiscardStmt: gen(c, n.sons[0])
@ -334,7 +334,7 @@ proc gen(c: var Con; n: PNode) =
of nkVarSection, nkLetSection: genVarSection(c, n)
else: discard
proc dfa(code: seq[Instr]) =
proc dfa(code: seq[Instr]; conf: ConfigRef) =
var u = newSeq[IntSet](code.len) # usages
var d = newSeq[IntSet](code.len) # defs
var c = newSeq[IntSet](code.len) # consumed
@ -426,17 +426,17 @@ proc dfa(code: seq[Instr]) =
of use, useWithinCall:
let s = code[i].sym
if s.id notin d[i]:
localError(code[i].n.info, "usage of uninitialized variable: " & s.name.s)
localError(conf, code[i].n.info, "usage of uninitialized variable: " & s.name.s)
if s.id in c[i]:
localError(code[i].n.info, "usage of an already consumed variable: " & s.name.s)
localError(conf, code[i].n.info, "usage of an already consumed variable: " & s.name.s)
else: discard
proc dataflowAnalysis*(s: PSym; body: PNode) =
proc dataflowAnalysis*(s: PSym; body: PNode; conf: ConfigRef) =
var c = Con(code: @[], blocks: @[])
gen(c, body)
when defined(useDfa) and defined(debugDfa): echoCfg(c.code)
dfa(c.code)
dfa(c.code, conf)
proc constructCfg*(s: PSym; body: PNode): ControlFlowGraph =
## constructs a control flow graph for ``body``.

238
compiler/docgen.nim
View file

@ -16,7 +16,7 @@ import
wordrecg, syntaxes, renderer, lexer, packages/docutils/rstast,
packages/docutils/rst, packages/docutils/rstgen, times,
packages/docutils/highlite, sempass2, json, xmltree, cgi,
typesrenderer, astalgo, modulepaths
typesrenderer, astalgo, modulepaths, configuration
type
TSections = array[TSymKind, Rope]
@ -29,6 +29,7 @@ type
jArray: JsonNode
types: TStrTable
isPureRst: bool
conf*: ConfigRef
PDoc* = ref TDocumentor ## Alias to type less.
@ -53,42 +54,47 @@ proc attachToType(d: PDoc; p: PSym): PSym =
if params.len > 0: check(0)
for i in 2..<params.len: check(i)
proc compilerMsgHandler(filename: string, line, col: int,
msgKind: rst.MsgKind, arg: string) {.procvar.} =
# translate msg kind:
var k: msgs.TMsgKind
case msgKind
of meCannotOpenFile: k = errCannotOpenFile
of meExpected: k = errXExpected
of meGridTableNotImplemented: k = errGridTableNotImplemented
of meNewSectionExpected: k = errNewSectionExpected
of meGeneralParseError: k = errGeneralParseError
of meInvalidDirective: k = errInvalidDirectiveX
of mwRedefinitionOfLabel: k = warnRedefinitionOfLabel
of mwUnknownSubstitution: k = warnUnknownSubstitutionX
of mwUnsupportedLanguage: k = warnLanguageXNotSupported
of mwUnsupportedField: k = warnFieldXNotSupported
globalError(newLineInfo(filename, line, col), k, arg)
template declareClosures =
proc compilerMsgHandler(filename: string, line, col: int,
msgKind: rst.MsgKind, arg: string) {.procvar.} =
# translate msg kind:
var k: TMsgKind
case msgKind
of meCannotOpenFile: k = errCannotOpenFile
of meExpected: k = errXExpected
of meGridTableNotImplemented: k = errGridTableNotImplemented
of meNewSectionExpected: k = errNewSectionExpected
of meGeneralParseError: k = errGeneralParseError
of meInvalidDirective: k = errInvalidDirectiveX
of mwRedefinitionOfLabel: k = warnRedefinitionOfLabel
of mwUnknownSubstitution: k = warnUnknownSubstitutionX
of mwUnsupportedLanguage: k = warnLanguageXNotSupported
of mwUnsupportedField: k = warnFieldXNotSupported
globalError(conf, newLineInfo(conf, filename, line, col), k, arg)
proc docgenFindFile(s: string): string {.procvar.} =
result = options.findFile(s)
if result.len == 0:
result = getCurrentDir() / s
if not existsFile(result): result = ""
proc docgenFindFile(s: string): string {.procvar.} =
result = options.findFile(conf, s)
if result.len == 0:
result = getCurrentDir() / s
if not existsFile(result): result = ""
proc parseRst(text, filename: string,
line, column: int, hasToc: var bool,
rstOptions: RstParseOptions): PRstNode =
rstOptions: RstParseOptions;
conf: ConfigRef): PRstNode =
declareClosures()
result = rstParse(text, filename, line, column, hasToc, rstOptions,
docgenFindFile, compilerMsgHandler)
proc newDocumentor*(filename: string, config: StringTableRef): PDoc =
proc newDocumentor*(filename: string, conf: ConfigRef): PDoc =
declareClosures()
new(result)
initRstGenerator(result[], (if gCmd != cmdRst2tex: outHtml else: outLatex),
options.gConfigVars, filename, {roSupportRawDirective},
result.conf = conf
initRstGenerator(result[], (if conf.cmd != cmdRst2tex: outHtml else: outLatex),
conf.configVars, filename, {roSupportRawDirective},
docgenFindFile, compilerMsgHandler)
if config.hasKey("doc.googleAnalytics"):
if conf.configVars.hasKey("doc.googleAnalytics"):
result.analytics = """
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
@ -100,7 +106,7 @@ proc newDocumentor*(filename: string, config: StringTableRef): PDoc =
ga('send', 'pageview');
</script>
""" % [config.getOrDefault"doc.googleAnalytics"]
""" % [conf.configVars.getOrDefault"doc.googleAnalytics"]
else:
result.analytics = ""
@ -109,10 +115,10 @@ proc newDocumentor*(filename: string, config: StringTableRef): PDoc =
result.jArray = newJArray()
initStrTable result.types
result.onTestSnippet = proc (d: var RstGenerator; filename, cmd: string; status: int; content: string) =
localError(newLineInfo(d.filename, -1, -1), warnUser, "only 'rst2html' supports the ':test:' attribute")
localError(conf, newLineInfo(conf, d.filename, -1, -1), warnUser, "only 'rst2html' supports the ':test:' attribute")
proc dispA(dest: var Rope, xml, tex: string, args: openArray[Rope]) =
if gCmd != cmdRst2tex: addf(dest, xml, args)
proc dispA(conf: ConfigRef; dest: var Rope, xml, tex: string, args: openArray[Rope]) =
if conf.cmd != cmdRst2tex: addf(dest, xml, args)
else: addf(dest, tex, args)
proc getVarIdx(varnames: openArray[string], id: string): int =
@ -121,7 +127,8 @@ proc getVarIdx(varnames: openArray[string], id: string): int =
return i
result = -1
proc ropeFormatNamedVars(frmt: FormatStr, varnames: openArray[string],
proc ropeFormatNamedVars(conf: ConfigRef; frmt: FormatStr,
varnames: openArray[string],
varvalues: openArray[Rope]): Rope =
var i = 0
var L = len(frmt)
@ -144,7 +151,8 @@ proc ropeFormatNamedVars(frmt: FormatStr, varnames: openArray[string],
j = (j * 10) + ord(frmt[i]) - ord('0')
inc(i)
if (i > L + 0 - 1) or not (frmt[i] in {'0'..'9'}): break
if j > high(varvalues) + 1: internalError("ropeFormatNamedVars")
if j > high(varvalues) + 1:
rawMessage(conf, errGenerated, "Invalid format string; too many $s: " & frmt)
num = j
add(result, varvalues[j - 1])
of 'A'..'Z', 'a'..'z', '\x80'..'\xFF':
@ -155,20 +163,23 @@ proc ropeFormatNamedVars(frmt: FormatStr, varnames: openArray[string],
if not (frmt[i] in {'A'..'Z', '_', 'a'..'z', '\x80'..'\xFF'}): break
var idx = getVarIdx(varnames, id)
if idx >= 0: add(result, varvalues[idx])
else: rawMessage(errUnknownSubstitionVar, id)
else: rawMessage(conf, errGenerated, "unknown substition variable: " & id)
of '{':
var id = ""
inc(i)
while frmt[i] != '}':
if frmt[i] == '\0': rawMessage(errTokenExpected, "}")
while i < frmt.len and frmt[i] != '}':
add(id, frmt[i])
inc(i)
inc(i) # skip }
# search for the variable:
var idx = getVarIdx(varnames, id)
if i >= frmt.len:
rawMessage(conf, errGenerated, "expected closing '}'")
else:
inc(i) # skip }
# search for the variable:
let idx = getVarIdx(varnames, id)
if idx >= 0: add(result, varvalues[idx])
else: rawMessage(errUnknownSubstitionVar, id)
else: internalError("ropeFormatNamedVars")
else: rawMessage(conf, errGenerated, "unknown substition variable: " & id)
else:
add(result, "$")
var start = i
while i < L:
if frmt[i] != '$': inc(i)
@ -181,7 +192,7 @@ proc genComment(d: PDoc, n: PNode): string =
if n.comment != nil:
renderRstToOut(d[], parseRst(n.comment, toFilename(n.info),
toLinenumber(n.info), toColumn(n.info),
dummyHasToc, d.options), result)
dummyHasToc, d.options, d.conf), result)
proc genRecComment(d: PDoc, n: PNode): Rope =
if n == nil: return nil
@ -220,45 +231,46 @@ proc nodeToHighlightedHtml(d: PDoc; n: PNode; result: var Rope; renderFlags: TRe
of tkEof:
break
of tkComment:
dispA(result, "<span class=\"Comment\">$1</span>", "\\spanComment{$1}",
dispA(d.conf, result, "<span class=\"Comment\">$1</span>", "\\spanComment{$1}",
[rope(esc(d.target, literal))])
of tokKeywordLow..tokKeywordHigh:
dispA(result, "<span class=\"Keyword\">$1</span>", "\\spanKeyword{$1}",
dispA(d.conf, result, "<span class=\"Keyword\">$1</span>", "\\spanKeyword{$1}",
[rope(literal)])
of tkOpr:
dispA(result, "<span class=\"Operator\">$1</span>", "\\spanOperator{$1}",
dispA(d.conf, result, "<span class=\"Operator\">$1</span>", "\\spanOperator{$1}",
[rope(esc(d.target, literal))])
of tkStrLit..tkTripleStrLit:
dispA(result, "<span class=\"StringLit\">$1</span>",
dispA(d.conf, result, "<span class=\"StringLit\">$1</span>",
"\\spanStringLit{$1}", [rope(esc(d.target, literal))])
of tkCharLit:
dispA(result, "<span class=\"CharLit\">$1</span>", "\\spanCharLit{$1}",
dispA(d.conf, result, "<span class=\"CharLit\">$1</span>", "\\spanCharLit{$1}",
[rope(esc(d.target, literal))])
of tkIntLit..tkUInt64Lit:
dispA(result, "<span class=\"DecNumber\">$1</span>",
dispA(d.conf, result, "<span class=\"DecNumber\">$1</span>",
"\\spanDecNumber{$1}", [rope(esc(d.target, literal))])
of tkFloatLit..tkFloat128Lit:
dispA(result, "<span class=\"FloatNumber\">$1</span>",
dispA(d.conf, result, "<span class=\"FloatNumber\">$1</span>",
"\\spanFloatNumber{$1}", [rope(esc(d.target, literal))])
of tkSymbol:
dispA(result, "<span class=\"Identifier\">$1</span>",
dispA(d.conf, result, "<span class=\"Identifier\">$1</span>",
"\\spanIdentifier{$1}", [rope(esc(d.target, literal))])
of tkSpaces, tkInvalid:
add(result, literal)
of tkCurlyDotLe:
dispA(result, """<span class="Other pragmabegin">$1</span><div class="pragma">""",
dispA(d.conf, result, """<span class="Other pragmabegin">$1</span><div class="pragma">""",
"\\spanOther{$1}",
[rope(esc(d.target, literal))])
of tkCurlyDotRi:
dispA(result, "</div><span class=\"Other pragmaend\">$1</span>",
dispA(d.conf, result, "</div><span class=\"Other pragmaend\">$1</span>",
"\\spanOther{$1}",
[rope(esc(d.target, literal))])
of tkParLe, tkParRi, tkBracketLe, tkBracketRi, tkCurlyLe, tkCurlyRi,
tkBracketDotLe, tkBracketDotRi, tkParDotLe,
tkParDotRi, tkComma, tkSemiColon, tkColon, tkEquals, tkDot, tkDotDot,
tkAccent, tkColonColon,
tkGStrLit, tkGTripleStrLit, tkInfixOpr, tkPrefixOpr, tkPostfixOpr:
dispA(result, "<span class=\"Other\">$1</span>", "\\spanOther{$1}",
tkGStrLit, tkGTripleStrLit, tkInfixOpr, tkPrefixOpr, tkPostfixOpr,
tkBracketLeColon:
dispA(d.conf, result, "<span class=\"Other\">$1</span>", "\\spanOther{$1}",
[rope(esc(d.target, literal))])
proc getAllRunnableExamples(d: PDoc; n: PNode; dest: var Rope) =
@ -266,7 +278,7 @@ proc getAllRunnableExamples(d: PDoc; n: PNode; dest: var Rope) =
of nkCallKinds:
if n[0].kind == nkSym and n[0].sym.magic == mRunnableExamples and
n.len >= 2 and n.lastSon.kind == nkStmtList:
dispA(dest, "\n<strong class=\"examples_text\">$1</strong>\n",
dispA(d.conf, dest, "\n<strong class=\"examples_text\">$1</strong>\n",
"\n\\textbf{$1}\n", [rope"Examples:"])
inc d.listingCounter
let id = $d.listingCounter
@ -335,7 +347,6 @@ proc getName(d: PDoc, n: PNode, splitAfter = -1): string =
of nkOpenSymChoice, nkClosedSymChoice:
result = getName(d, n[0], splitAfter)
else:
internalError(n.info, "getName()")
result = ""
proc getNameIdent(n: PNode): PIdent =
@ -365,7 +376,6 @@ proc getRstName(n: PNode): PRstNode =
of nkOpenSymChoice, nkClosedSymChoice:
result = getRstName(n[0])
else:
internalError(n.info, "getRstName()")
result = nil
proc newUniquePlainSymbol(d: PDoc, original: string): string =
@ -489,22 +499,22 @@ proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind) =
symbolOrIdEncRope = encodeUrl(symbolOrId).rope
var seeSrcRope: Rope = nil
let docItemSeeSrc = getConfigVar("doc.item.seesrc")
let docItemSeeSrc = getConfigVar(d.conf, "doc.item.seesrc")
if docItemSeeSrc.len > 0:
let cwd = getCurrentDir().canonicalizePath()
let cwd = canonicalizePath(d.conf, getCurrentDir())
var path = n.info.toFullPath
if path.startsWith(cwd):
path = path[cwd.len+1 .. ^1].replace('\\', '/')
let gitUrl = getConfigVar("git.url")
let gitUrl = getConfigVar(d.conf, "git.url")
if gitUrl.len > 0:
var commit = getConfigVar("git.commit")
var commit = getConfigVar(d.conf, "git.commit")
if commit.len == 0: commit = "master"
dispA(seeSrcRope, "$1", "", [ropeFormatNamedVars(docItemSeeSrc,
dispA(d.conf, seeSrcRope, "$1", "", [ropeFormatNamedVars(d.conf, docItemSeeSrc,
["path", "line", "url", "commit"], [rope path,
rope($n.info.line), rope gitUrl,
rope commit])])
add(d.section[k], ropeFormatNamedVars(getConfigVar("doc.item"),
add(d.section[k], ropeFormatNamedVars(d.conf, getConfigVar(d.conf, "doc.item"),
["name", "header", "desc", "itemID", "header_plain", "itemSym",
"itemSymOrID", "itemSymEnc", "itemSymOrIDEnc", "seeSrc"],
[nameRope, result, comm, itemIDRope, plainNameRope, plainSymbolRope,
@ -515,7 +525,7 @@ proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind) =
let att = attachToType(d, nameNode.sym)
if att != nil:
attype = rope esc(d.target, att.name.s)
add(d.toc[k], ropeFormatNamedVars(getConfigVar("doc.item.toc"),
add(d.toc[k], ropeFormatNamedVars(d.conf, getConfigVar(d.conf, "doc.item.toc"),
["name", "header", "desc", "itemID", "header_plain", "itemSym",
"itemSymOrID", "itemSymEnc", "itemSymOrIDEnc", "attype"],
[rope(getName(d, nameNode, d.splitAfter)), result, comm,
@ -543,7 +553,7 @@ proc genJsonItem(d: PDoc, n, nameNode: PNode, k: TSymKind): JsonNode =
initTokRender(r, n, {renderNoBody, renderNoComments, renderDocComments})
result = %{ "name": %name, "type": %($k), "line": %n.info.line,
result = %{ "name": %name, "type": %($k), "line": %n.info.line.int,
"col": %n.info.col}
if comm != nil and comm != "":
result["description"] = %comm
@ -568,9 +578,9 @@ proc traceDeps(d: PDoc, it: PNode) =
traceDeps(d, a)
else:
if d.section[k] != nil: add(d.section[k], ", ")
dispA(d.section[k],
dispA(d.conf, d.section[k],
"<a class=\"reference external\" href=\"$1.html\">$1</a>",
"$1", [rope(getModuleName(it))])
"$1", [rope(getModuleName(d.conf, it))])
proc generateDoc*(d: PDoc, n: PNode) =
case n.kind
@ -617,7 +627,7 @@ proc generateJson*(d: PDoc, n: PNode) =
of nkCommentStmt:
if n.comment != nil and startsWith(n.comment, "##"):
let stripped = n.comment.substr(2).strip
d.add %{ "comment": %stripped, "line": %n.info.line,
d.add %{ "comment": %stripped, "line": %n.info.line.int,
"col": %n.info.col }
of nkProcDef:
when useEffectSystem: documentRaises(n)
@ -703,10 +713,10 @@ proc genSection(d: PDoc, kind: TSymKind) =
]
if d.section[kind] == nil: return
var title = sectionNames[kind].rope
d.section[kind] = ropeFormatNamedVars(getConfigVar("doc.section"), [
d.section[kind] = ropeFormatNamedVars(d.conf, getConfigVar(d.conf, "doc.section"), [
"sectionid", "sectionTitle", "sectionTitleID", "content"], [
ord(kind).rope, title, rope(ord(kind) + 50), d.section[kind]])
d.toc[kind] = ropeFormatNamedVars(getConfigVar("doc.section.toc"), [
d.toc[kind] = ropeFormatNamedVars(d.conf, getConfigVar(d.conf, "doc.section.toc"), [
"sectionid", "sectionTitle", "sectionTitleID", "content"], [
ord(kind).rope, title, rope(ord(kind) + 50), d.toc[kind]])
@ -722,7 +732,7 @@ proc genOutFile(d: PDoc): Rope =
genSection(d, i)
add(toc, d.toc[i])
if toc != nil:
toc = ropeFormatNamedVars(getConfigVar("doc.toc"), ["content"], [toc])
toc = ropeFormatNamedVars(d.conf, getConfigVar(d.conf, "doc.toc"), ["content"], [toc])
for i in countup(low(TSymKind), high(TSymKind)): add(code, d.section[i])
# Extract the title. Non API modules generate an entry in the index table.
@ -736,13 +746,13 @@ proc genOutFile(d: PDoc): Rope =
let bodyname = if d.hasToc and not d.isPureRst: "doc.body_toc_group"
elif d.hasToc: "doc.body_toc"
else: "doc.body_no_toc"
content = ropeFormatNamedVars(getConfigVar(bodyname), ["title",
content = ropeFormatNamedVars(d.conf, getConfigVar(d.conf, bodyname), ["title",
"tableofcontents", "moduledesc", "date", "time", "content"],
[title.rope, toc, d.modDesc, rope(getDateStr()),
rope(getClockStr()), code])
if optCompileOnly notin gGlobalOptions:
if optCompileOnly notin d.conf.globalOptions:
# XXX what is this hack doing here? 'optCompileOnly' means raw output!?
code = ropeFormatNamedVars(getConfigVar("doc.file"), ["title",
code = ropeFormatNamedVars(d.conf, getConfigVar(d.conf, "doc.file"), ["title",
"tableofcontents", "moduledesc", "date", "time",
"content", "author", "version", "analytics"],
[title.rope, toc, d.modDesc, rope(getDateStr()),
@ -753,60 +763,60 @@ proc genOutFile(d: PDoc): Rope =
result = code
proc generateIndex*(d: PDoc) =
if optGenIndex in gGlobalOptions:
writeIndexFile(d[], splitFile(options.outFile).dir /
if optGenIndex in d.conf.globalOptions:
writeIndexFile(d[], splitFile(d.conf.outFile).dir /
splitFile(d.filename).name & IndexExt)
proc getOutFile2(filename, ext, dir: string): string =
if gWholeProject:
let d = if options.outFile != "": options.outFile else: dir
proc getOutFile2(conf: ConfigRef; filename, ext, dir: string): string =
if optWholeProject in conf.globalOptions:
let d = if conf.outFile != "": conf.outFile else: dir
createDir(d)
result = d / changeFileExt(filename, ext)
else:
result = getOutFile(filename, ext)
result = getOutFile(conf, filename, ext)
proc writeOutput*(d: PDoc, filename, outExt: string, useWarning = false) =
var content = genOutFile(d)
if optStdout in gGlobalOptions:
if optStdout in d.conf.globalOptions:
writeRope(stdout, content)
else:
writeRope(content, getOutFile2(filename, outExt, "htmldocs"), useWarning)
writeRope(content, getOutFile2(d.conf, filename, outExt, "htmldocs"), useWarning)
proc writeOutputJson*(d: PDoc, filename, outExt: string,
useWarning = false) =
let content = %*{"orig": d.filename,
"nimble": getPackageName(d.filename),
"nimble": getPackageName(d.conf, d.filename),
"entries": d.jArray}
if optStdout in gGlobalOptions:
if optStdout in d.conf.globalOptions:
write(stdout, $content)
else:
var f: File
if open(f, getOutFile2(splitFile(filename).name,
if open(f, getOutFile2(d.conf, splitFile(filename).name,
outExt, "jsondocs"), fmWrite):
write(f, $content)
close(f)
else:
discard "fixme: error report"
proc commandDoc*() =
var ast = parseFile(gProjectMainIdx, newIdentCache())
proc commandDoc*(conf: ConfigRef) =
var ast = parseFile(conf.projectMainIdx.FileIndex, newIdentCache(), conf)
if ast == nil: return
var d = newDocumentor(gProjectFull, options.gConfigVars)
var d = newDocumentor(conf.projectFull, conf)
d.hasToc = true
generateDoc(d, ast)
writeOutput(d, gProjectFull, HtmlExt)
writeOutput(d, conf.projectFull, HtmlExt)
generateIndex(d)
proc commandRstAux(filename, outExt: string) =
proc commandRstAux(conf: ConfigRef; filename, outExt: string) =
var filen = addFileExt(filename, "txt")
var d = newDocumentor(filen, options.gConfigVars)
var d = newDocumentor(filen, conf)
d.onTestSnippet = proc (d: var RstGenerator; filename, cmd: string;
status: int; content: string) =
var outp: string
if filename.len == 0:
inc(d.id)
let nameOnly = splitFile(d.filename).name
let subdir = getNimcacheDir() / nameOnly
let subdir = getNimcacheDir(conf) / nameOnly
createDir(subdir)
outp = subdir / (nameOnly & "_snippet_" & $d.id & ".nim")
elif isAbsolute(filename):
@ -815,14 +825,14 @@ proc commandRstAux(filename, outExt: string) =
# Nim's convention: every path is relative to the file it was written in:
outp = splitFile(d.filename).dir / filename
writeFile(outp, content)
let cmd = unescape(cmd) % quoteShell(outp)
rawMessage(hintExecuting, cmd)
let cmd = cmd % quoteShell(outp)
rawMessage(conf, hintExecuting, cmd)
if execShellCmd(cmd) != status:
rawMessage(errExecutionOfProgramFailed, cmd)
rawMessage(conf, errGenerated, "executing of external program failed: " & cmd)
d.isPureRst = true
var rst = parseRst(readFile(filen), filen, 0, 1, d.hasToc,
{roSupportRawDirective})
{roSupportRawDirective}, conf)
var modDesc = newStringOfCap(30_000)
#d.modDesc = newMutableRope(30_000)
renderRstToOut(d[], rst, modDesc)
@ -831,50 +841,50 @@ proc commandRstAux(filename, outExt: string) =
writeOutput(d, filename, outExt)
generateIndex(d)
proc commandRst2Html*() =
commandRstAux(gProjectFull, HtmlExt)
proc commandRst2Html*(conf: ConfigRef) =
commandRstAux(conf, conf.projectFull, HtmlExt)
proc commandRst2TeX*() =
proc commandRst2TeX*(conf: ConfigRef) =
splitter = "\\-"
commandRstAux(gProjectFull, TexExt)
commandRstAux(conf, conf.projectFull, TexExt)
proc commandJson*() =
var ast = parseFile(gProjectMainIdx, newIdentCache())
proc commandJson*(conf: ConfigRef) =
var ast = parseFile(conf.projectMainIdx.FileIndex, newIdentCache(), conf)
if ast == nil: return
var d = newDocumentor(gProjectFull, options.gConfigVars)
var d = newDocumentor(conf.projectFull, conf)
d.hasToc = true
generateJson(d, ast)
let json = d.jArray
let content = rope(pretty(json))
if optStdout in gGlobalOptions:
if optStdout in d.conf.globalOptions:
writeRope(stdout, content)
else:
#echo getOutFile(gProjectFull, JsonExt)
writeRope(content, getOutFile(gProjectFull, JsonExt), useWarning = false)
writeRope(content, getOutFile(conf, conf.projectFull, JsonExt), useWarning = false)
proc commandTags*() =
var ast = parseFile(gProjectMainIdx, newIdentCache())
proc commandTags*(conf: ConfigRef) =
var ast = parseFile(conf.projectMainIdx.FileIndex, newIdentCache(), conf)
if ast == nil: return
var d = newDocumentor(gProjectFull, options.gConfigVars)
var d = newDocumentor(conf.projectFull, conf)
d.hasToc = true
var
content: Rope
generateTags(d, ast, content)
if optStdout in gGlobalOptions:
if optStdout in d.conf.globalOptions:
writeRope(stdout, content)
else:
#echo getOutFile(gProjectFull, TagsExt)
writeRope(content, getOutFile(gProjectFull, TagsExt), useWarning = false)
writeRope(content, getOutFile(conf, conf.projectFull, TagsExt), useWarning = false)
proc commandBuildIndex*() =
var content = mergeIndexes(gProjectFull).rope
proc commandBuildIndex*(conf: ConfigRef) =
var content = mergeIndexes(conf.projectFull).rope
let code = ropeFormatNamedVars(getConfigVar("doc.file"), ["title",
let code = ropeFormatNamedVars(conf, getConfigVar(conf, "doc.file"), ["title",
"tableofcontents", "moduledesc", "date", "time",
"content", "author", "version", "analytics"],
["Index".rope, nil, nil, rope(getDateStr()),
rope(getClockStr()), content, nil, nil, nil])
# no analytics because context is not available
writeRope(code, getOutFile("theindex", HtmlExt))
writeRope(code, getOutFile(conf, "theindex", HtmlExt))

6
compiler/docgen2.nim
View file

@ -25,8 +25,8 @@ template closeImpl(body: untyped) {.dirty.} =
var g = PGen(p)
let useWarning = sfMainModule notin g.module.flags
#echo g.module.name.s, " ", g.module.owner.id, " ", gMainPackageId
if (g.module.owner.id == gMainPackageId and gWholeProject) or
sfMainModule in g.module.flags:
if (g.module.owner.id == gMainPackageId and optWholeProject in g.doc.conf.globalOptions) or
sfMainModule in g.module.flags:
body
try:
generateIndex(g.doc)
@ -55,7 +55,7 @@ proc myOpen(graph: ModuleGraph; module: PSym; cache: IdentCache): PPassContext =
var g: PGen
new(g)
g.module = module
var d = newDocumentor(module.filename, options.gConfigVars)
var d = newDocumentor(module.filename, graph.config)
d.hasToc = true
g.doc = d
result = g

24
compiler/evalffi.nim
View file

@ -86,10 +86,10 @@ proc mapType(t: ast.PType): ptr libffi.TType =
else: result = nil
of tyFloat, tyFloat64: result = addr libffi.type_double
of tyFloat32: result = addr libffi.type_float
of tyVar, tyPointer, tyPtr, tyRef, tyCString, tySequence, tyString, tyExpr,
of tyVar, tyLent, tyPointer, tyPtr, tyRef, tyCString, tySequence, tyString, tyExpr,
tyStmt, tyTypeDesc, tyProc, tyArray, tyStatic, tyNil:
result = addr libffi.type_pointer
of tyDistinct, tyAlias:
of tyDistinct, tyAlias, tySink:
result = mapType(t.sons[0])
else:
result = nil
@ -112,12 +112,12 @@ template `+!`(x, y: untyped): untyped =
proc packSize(v: PNode, typ: PType): int =
## computes the size of the blob
case typ.kind
of tyPtr, tyRef, tyVar:
of tyPtr, tyRef, tyVar, tyLent:
if v.kind in {nkNilLit, nkPtrLit}:
result = sizeof(pointer)
else:
result = sizeof(pointer) + packSize(v.sons[0], typ.lastSon)
of tyDistinct, tyGenericInst, tyAlias:
of tyDistinct, tyGenericInst, tyAlias, tySink:
result = packSize(v, typ.sons[0])
of tyArray:
# consider: ptr array[0..1000_000, int] which is common for interfacing;
@ -151,7 +151,7 @@ proc getField(n: PNode; position: int): PSym =
else: discard
proc packObject(x: PNode, typ: PType, res: pointer) =
internalAssert x.kind in {nkObjConstr, nkPar}
internalAssert x.kind in {nkObjConstr, nkPar, nkTupleConstr}
# compute the field's offsets:
discard typ.getSize
for i in countup(ord(x.kind == nkObjConstr), sonsLen(x) - 1):
@ -209,7 +209,7 @@ proc pack(v: PNode, typ: PType, res: pointer) =
awr(cstring, cstring(v.strVal))
else:
globalError(v.info, "cannot map pointer/proc value to FFI")
of tyPtr, tyRef, tyVar:
of tyPtr, tyRef, tyVar, tyLent:
if v.kind == nkNilLit:
# nothing to do since the memory is 0 initialized anyway
discard
@ -231,7 +231,7 @@ proc pack(v: PNode, typ: PType, res: pointer) =
packObject(v, typ, res)
of tyNil:
discard
of tyDistinct, tyGenericInst, tyAlias:
of tyDistinct, tyGenericInst, tyAlias, tySink:
pack(v, typ.sons[0], res)
else:
globalError(v.info, "cannot map value to FFI " & typeToString(v.typ))
@ -260,14 +260,14 @@ proc unpackObject(x: pointer, typ: PType, n: PNode): PNode =
# iterate over any actual field of 'n' ... if n is nil we need to create
# the nkPar node:
if n.isNil:
result = newNode(nkPar)
result = newNode(nkTupleConstr)
result.typ = typ
if typ.n.isNil:
internalError("cannot unpack unnamed tuple")
unpackObjectAdd(x, typ.n, result)
else:
result = n
if result.kind notin {nkObjConstr, nkPar}:
if result.kind notin {nkObjConstr, nkPar, nkTupleConstr}:
globalError(n.info, "cannot map value from FFI")
if typ.n.isNil:
globalError(n.info, "cannot unpack unnamed tuple")
@ -364,7 +364,7 @@ proc unpack(x: pointer, typ: PType, n: PNode): PNode =
result = n
else:
awi(nkPtrLit, cast[ByteAddress](p))
of tyPtr, tyRef, tyVar:
of tyPtr, tyRef, tyVar, tyLent:
let p = rd(pointer, x)
if p.isNil:
setNil()
@ -388,14 +388,14 @@ proc unpack(x: pointer, typ: PType, n: PNode): PNode =
aws(nkStrLit, $p)
of tyNil:
setNil()
of tyDistinct, tyGenericInst, tyAlias:
of tyDistinct, tyGenericInst, tyAlias, tySink:
result = unpack(x, typ.lastSon, n)
else:
# XXX what to do with 'array' here?
globalError(n.info, "cannot map value from FFI " & typeToString(typ))
proc fficast*(x: PNode, destTyp: PType): PNode =
if x.kind == nkPtrLit and x.typ.kind in {tyPtr, tyRef, tyVar, tyPointer,
if x.kind == nkPtrLit and x.typ.kind in {tyPtr, tyRef, tyVar, tyLent, tyPointer,
tyProc, tyCString, tyString,
tySequence}:
result = newNodeIT(x.kind, x.info, destTyp)

37
compiler/evaltempl.nim
View file

@ -14,11 +14,12 @@ import
rodread
type
TemplCtx {.pure, final.} = object
TemplCtx = object
owner, genSymOwner: PSym
instLines: bool # use the instantiation lines numbers
mapping: TIdTable # every gensym'ed symbol needs to be mapped to some
# new symbol
config: ConfigRef
proc copyNode(ctx: TemplCtx, a, b: PNode): PNode =
result = copyNode(a)
@ -42,7 +43,7 @@ proc evalTemplateAux(templ, actual: PNode, c: var TemplCtx, result: PNode) =
s.kind == skType and s.typ != nil and s.typ.kind == tyGenericParam:
handleParam actual.sons[s.owner.typ.len + s.position - 1]
else:
internalAssert sfGenSym in s.flags or s.kind == skType
internalAssert c.config, sfGenSym in s.flags or s.kind == skType
var x = PSym(idTableGet(c.mapping, s))
if x == nil:
x = copySym(s, false)
@ -59,11 +60,16 @@ proc evalTemplateAux(templ, actual: PNode, c: var TemplCtx, result: PNode) =
evalTemplateAux(templ.sons[i], actual, c, res)
result.add res
proc evalTemplateArgs(n: PNode, s: PSym; fromHlo: bool): PNode =
const
errWrongNumberOfArguments = "wrong number of arguments"
errMissingGenericParamsForTemplate = "'$1' has unspecified generic parameters"
errTemplateInstantiationTooNested = "template instantiation too nested"
proc evalTemplateArgs(n: PNode, s: PSym; conf: ConfigRef; fromHlo: bool): PNode =
# if the template has zero arguments, it can be called without ``()``
# `n` is then a nkSym or something similar
var totalParams = case n.kind
of nkCall, nkInfix, nkPrefix, nkPostfix, nkCommand, nkCallStrLit: n.len-1
of nkCallKinds: n.len-1
else: 0
var
@ -82,10 +88,10 @@ proc evalTemplateArgs(n: PNode, s: PSym; fromHlo: bool): PNode =
if givenRegularParams < 0: givenRegularParams = 0
if totalParams > expectedRegularParams + genericParams:
globalError(n.info, errWrongNumberOfArguments)
globalError(conf, n.info, errWrongNumberOfArguments)
if totalParams < genericParams:
globalError(n.info, errMissingGenericParamsForTemplate,
globalError(conf, n.info, errMissingGenericParamsForTemplate %
n.renderTree)
result = newNodeI(nkArgList, n.info)
@ -97,7 +103,7 @@ proc evalTemplateArgs(n: PNode, s: PSym; fromHlo: bool): PNode =
for i in givenRegularParams+1 .. expectedRegularParams:
let default = s.typ.n.sons[i].sym.ast
if default.isNil or default.kind == nkEmpty:
localError(n.info, errWrongNumberOfArguments)
localError(conf, n.info, errWrongNumberOfArguments)
addSon(result, ast.emptyNode)
else:
addSon(result, default.copyTree)
@ -106,8 +112,9 @@ proc evalTemplateArgs(n: PNode, s: PSym; fromHlo: bool): PNode =
for i in 1 .. genericParams:
result.addSon n.sons[givenRegularParams + i]
var evalTemplateCounter* = 0
# to prevent endless recursion in templates instantiation
# to prevent endless recursion in template instantiation
const evalTemplateLimit* = 1000
var evalTemplateCounter* = 0 # XXX remove this global
proc wrapInComesFrom*(info: TLineInfo; sym: PSym; res: PNode): PNode =
when true:
@ -131,17 +138,19 @@ proc wrapInComesFrom*(info: TLineInfo; sym: PSym; res: PNode): PNode =
result.add res
result.typ = res.typ
proc evalTemplate*(n: PNode, tmpl, genSymOwner: PSym; fromHlo=false): PNode =
proc evalTemplate*(n: PNode, tmpl, genSymOwner: PSym;
conf: ConfigRef; fromHlo=false): PNode =
inc(evalTemplateCounter)
if evalTemplateCounter > 100:
globalError(n.info, errTemplateInstantiationTooNested)
if evalTemplateCounter > evalTemplateLimit:
globalError(conf, n.info, errTemplateInstantiationTooNested)
result = n
# replace each param by the corresponding node:
var args = evalTemplateArgs(n, tmpl, fromHlo)
var args = evalTemplateArgs(n, tmpl, conf, fromHlo)
var ctx: TemplCtx
ctx.owner = tmpl
ctx.genSymOwner = genSymOwner
ctx.config = conf
initIdTable(ctx.mapping)
let body = tmpl.getBody
@ -150,7 +159,7 @@ proc evalTemplate*(n: PNode, tmpl, genSymOwner: PSym; fromHlo=false): PNode =
evalTemplateAux(body, args, ctx, result)
if result.len == 1: result = result.sons[0]
else:
localError(result.info, errIllFormedAstX,
localError(conf, result.info, "illformed AST: " &
renderTree(result, {renderNoComments}))
else:
result = copyNode(body)

389
compiler/extccomp.nim
View file

@ -14,7 +14,7 @@
import
ropes, os, strutils, osproc, platform, condsyms, options, msgs,
securehash, streams
configuration, std / sha1, streams
#from debuginfo import writeDebugInfo
@ -176,10 +176,10 @@ compiler bcc:
result = (
name: "bcc",
objExt: "obj",
optSpeed: " -O2 -6 ",
optSpeed: " -O3 -6 ",
optSize: " -O1 -6 ",
compilerExe: "bcc32",
cppCompiler: "",
compilerExe: "bcc32c",
cppCompiler: "cpp32c",
compileTmpl: "-c $options $include -o$objfile $file",
buildGui: " -tW",
buildDll: " -tWD",
@ -193,7 +193,9 @@ compiler bcc:
pic: "",
asmStmtFrmt: "__asm{$n$1$n}$n",
structStmtFmt: "$1 $2",
props: {hasCpp})
props: {hasSwitchRange, hasComputedGoto, hasCpp, hasGcGuard,
hasAttribute})
# Digital Mars C Compiler
compiler dmc:
@ -374,80 +376,81 @@ proc nameToCC*(name: string): TSystemCC =
return i
result = ccNone
proc getConfigVar(c: TSystemCC, suffix: string): string =
proc getConfigVar(conf: ConfigRef; c: TSystemCC, suffix: string): string =
# use ``cpu.os.cc`` for cross compilation, unless ``--compileOnly`` is given
# for niminst support
let fullSuffix =
if gCmd == cmdCompileToCpp:
if conf.cmd == cmdCompileToCpp:
".cpp" & suffix
elif gCmd == cmdCompileToOC:
elif conf.cmd == cmdCompileToOC:
".objc" & suffix
elif gCmd == cmdCompileToJS:
elif conf.cmd == cmdCompileToJS:
".js" & suffix
else:
suffix
if (platform.hostOS != targetOS or platform.hostCPU != targetCPU) and
optCompileOnly notin gGlobalOptions:
optCompileOnly notin conf.globalOptions:
let fullCCname = platform.CPU[targetCPU].name & '.' &
platform.OS[targetOS].name & '.' &
CC[c].name & fullSuffix
result = getConfigVar(fullCCname)
result = getConfigVar(conf, fullCCname)
if result.len == 0:
# not overriden for this cross compilation setting?
result = getConfigVar(CC[c].name & fullSuffix)
result = getConfigVar(conf, CC[c].name & fullSuffix)
else:
result = getConfigVar(CC[c].name & fullSuffix)
result = getConfigVar(conf, CC[c].name & fullSuffix)
proc setCC*(ccname: string) =
proc setCC*(conf: ConfigRef; ccname: string; info: TLineInfo) =
cCompiler = nameToCC(ccname)
if cCompiler == ccNone: rawMessage(errUnknownCcompiler, ccname)
compileOptions = getConfigVar(cCompiler, ".options.always")
if cCompiler == ccNone:
localError(conf, info, "unknown C compiler: '$1'" % ccname)
compileOptions = getConfigVar(conf, cCompiler, ".options.always")
linkOptions = ""
ccompilerpath = getConfigVar(cCompiler, ".path")
for i in countup(low(CC), high(CC)): undefSymbol(CC[i].name)
defineSymbol(CC[cCompiler].name)
ccompilerpath = getConfigVar(conf, cCompiler, ".path")
for i in countup(low(CC), high(CC)): undefSymbol(conf.symbols, CC[i].name)
defineSymbol(conf.symbols, CC[cCompiler].name)
proc addOpt(dest: var string, src: string) =
if len(dest) == 0 or dest[len(dest)-1] != ' ': add(dest, " ")
add(dest, src)
proc addLinkOption*(option: string) =
proc addLinkOption*(conf: ConfigRef; option: string) =
addOpt(linkOptions, option)
proc addCompileOption*(option: string) =
proc addCompileOption*(conf: ConfigRef; option: string) =
if strutils.find(compileOptions, option, 0) < 0:
addOpt(compileOptions, option)
proc addLinkOptionCmd*(option: string) =
proc addLinkOptionCmd*(conf: ConfigRef; option: string) =
addOpt(linkOptionsCmd, option)
proc addCompileOptionCmd*(option: string) =
proc addCompileOptionCmd*(conf: ConfigRef; option: string) =
compileOptionsCmd.add(option)
proc initVars*() =
proc initVars*(conf: ConfigRef) =
# we need to define the symbol here, because ``CC`` may have never been set!
for i in countup(low(CC), high(CC)): undefSymbol(CC[i].name)
defineSymbol(CC[cCompiler].name)
addCompileOption(getConfigVar(cCompiler, ".options.always"))
for i in countup(low(CC), high(CC)): undefSymbol(conf.symbols, CC[i].name)
defineSymbol(conf.symbols, CC[cCompiler].name)
addCompileOption(conf, getConfigVar(conf, cCompiler, ".options.always"))
#addLinkOption(getConfigVar(cCompiler, ".options.linker"))
if len(ccompilerpath) == 0:
ccompilerpath = getConfigVar(cCompiler, ".path")
ccompilerpath = getConfigVar(conf, cCompiler, ".path")
proc completeCFilePath*(cfile: string, createSubDir: bool = true): string =
result = completeGeneratedFilePath(cfile, createSubDir)
proc completeCFilePath*(conf: ConfigRef; cfile: string, createSubDir: bool = true): string =
result = completeGeneratedFilePath(conf, cfile, createSubDir)
proc toObjFile*(filename: string): string =
proc toObjFile*(conf: ConfigRef; filename: string): string =
# Object file for compilation
#if filename.endsWith(".cpp"):
# result = changeFileExt(filename, "cpp." & CC[cCompiler].objExt)
#else:
result = changeFileExt(filename, CC[cCompiler].objExt)
proc addFileToCompile*(cf: Cfile) =
proc addFileToCompile*(conf: ConfigRef; cf: Cfile) =
toCompile.add(cf)
proc resetCompilationLists* =
proc resetCompilationLists*(conf: ConfigRef) =
toCompile.setLen 0
## XXX: we must associate these with their originating module
# when the module is loaded/unloaded it adds/removes its items
@ -455,108 +458,112 @@ proc resetCompilationLists* =
# Maybe we can do that in checkDep on the other hand?
externalToLink.setLen 0
proc addExternalFileToLink*(filename: string) =
proc addExternalFileToLink*(conf: ConfigRef; filename: string) =
externalToLink.insert(filename, 0)
proc execWithEcho(cmd: string, msg = hintExecuting): int =
rawMessage(msg, cmd)
proc execWithEcho(conf: ConfigRef; cmd: string, msg = hintExecuting): int =
rawMessage(conf, msg, cmd)
result = execCmd(cmd)
proc execExternalProgram*(cmd: string, msg = hintExecuting) =
if execWithEcho(cmd, msg) != 0:
rawMessage(errExecutionOfProgramFailed, cmd)
proc execExternalProgram*(conf: ConfigRef; cmd: string, msg = hintExecuting) =
if execWithEcho(conf, cmd, msg) != 0:
rawMessage(conf, errGenerated, "execution of an external program failed: '$1'" %
cmd)
proc generateScript(projectFile: string, script: Rope) =
proc generateScript(conf: ConfigRef; projectFile: string, script: Rope) =
let (dir, name, ext) = splitFile(projectFile)
writeRope(script, dir / addFileExt("compile_" & name,
writeRope(script, getNimcacheDir(conf) / addFileExt("compile_" & name,
platform.OS[targetOS].scriptExt))
copyFile(conf.libpath / "nimbase.h", getNimcacheDir(conf) / "nimbase.h")
proc getOptSpeed(c: TSystemCC): string =
result = getConfigVar(c, ".options.speed")
proc getOptSpeed(conf: ConfigRef; c: TSystemCC): string =
result = getConfigVar(conf, c, ".options.speed")
if result == "":
result = CC[c].optSpeed # use default settings from this file
proc getDebug(c: TSystemCC): string =
result = getConfigVar(c, ".options.debug")
proc getDebug(conf: ConfigRef; c: TSystemCC): string =
result = getConfigVar(conf, c, ".options.debug")
if result == "":
result = CC[c].debug # use default settings from this file
proc getOptSize(c: TSystemCC): string =
result = getConfigVar(c, ".options.size")
proc getOptSize(conf: ConfigRef; c: TSystemCC): string =
result = getConfigVar(conf, c, ".options.size")
if result == "":
result = CC[c].optSize # use default settings from this file
proc noAbsolutePaths: bool {.inline.} =
proc noAbsolutePaths(conf: ConfigRef): bool {.inline.} =
# We used to check current OS != specified OS, but this makes no sense
# really: Cross compilation from Linux to Linux for example is entirely
# reasonable.
# `optGenMapping` is included here for niminst.
result = gGlobalOptions * {optGenScript, optGenMapping} != {}
result = conf.globalOptions * {optGenScript, optGenMapping} != {}
proc cFileSpecificOptions(cfilename: string): string =
proc cFileSpecificOptions(conf: ConfigRef; cfilename: string): string =
result = compileOptions
for option in compileOptionsCmd:
if strutils.find(result, option, 0) < 0:
addOpt(result, option)
var trunk = splitFile(cfilename).name
if optCDebug in gGlobalOptions:
var key = trunk & ".debug"
if existsConfigVar(key): addOpt(result, getConfigVar(key))
else: addOpt(result, getDebug(cCompiler))
if optOptimizeSpeed in gOptions:
var key = trunk & ".speed"
if existsConfigVar(key): addOpt(result, getConfigVar(key))
else: addOpt(result, getOptSpeed(cCompiler))
elif optOptimizeSize in gOptions:
var key = trunk & ".size"
if existsConfigVar(key): addOpt(result, getConfigVar(key))
else: addOpt(result, getOptSize(cCompiler))
var key = trunk & ".always"
if existsConfigVar(key): addOpt(result, getConfigVar(key))
let trunk = splitFile(cfilename).name
if optCDebug in conf.globalOptions:
let key = trunk & ".debug"
if existsConfigVar(conf, key): addOpt(result, getConfigVar(conf, key))
else: addOpt(result, getDebug(conf, cCompiler))
if optOptimizeSpeed in conf.options:
let key = trunk & ".speed"
if existsConfigVar(conf, key): addOpt(result, getConfigVar(conf, key))
else: addOpt(result, getOptSpeed(conf, cCompiler))
elif optOptimizeSize in conf.options:
let key = trunk & ".size"
if existsConfigVar(conf, key): addOpt(result, getConfigVar(conf, key))
else: addOpt(result, getOptSize(conf, cCompiler))
let key = trunk & ".always"
if existsConfigVar(conf, key): addOpt(result, getConfigVar(conf, key))
proc getCompileOptions: string =
result = cFileSpecificOptions("__dummy__")
proc getCompileOptions(conf: ConfigRef): string =
result = cFileSpecificOptions(conf, "__dummy__")
proc getLinkOptions: string =
proc getLinkOptions(conf: ConfigRef): string =
result = linkOptions & " " & linkOptionsCmd & " "
for linkedLib in items(cLinkedLibs):
result.add(CC[cCompiler].linkLibCmd % linkedLib.quoteShell)
for libDir in items(cLibs):
result.add(join([CC[cCompiler].linkDirCmd, libDir.quoteShell]))
proc needsExeExt(): bool {.inline.} =
result = (optGenScript in gGlobalOptions and targetOS == osWindows) or
proc needsExeExt(conf: ConfigRef): bool {.inline.} =
result = (optGenScript in conf.globalOptions and targetOS == osWindows) or
(platform.hostOS == osWindows)
proc getCompilerExe(compiler: TSystemCC; cfile: string): string =
result = if gCmd == cmdCompileToCpp and not cfile.endsWith(".c"):
proc getCompilerExe(conf: ConfigRef; compiler: TSystemCC; cfile: string): string =
result = if conf.cmd == cmdCompileToCpp and not cfile.endsWith(".c"):
CC[compiler].cppCompiler
else:
CC[compiler].compilerExe
if result.len == 0:
rawMessage(errCompilerDoesntSupportTarget, CC[compiler].name)
rawMessage(conf, errGenerated,
"Compiler '$1' doesn't support the requested target" %
CC[compiler].name)
proc getLinkerExe(compiler: TSystemCC): string =
proc getLinkerExe(conf: ConfigRef; compiler: TSystemCC): string =
result = if CC[compiler].linkerExe.len > 0: CC[compiler].linkerExe
elif gMixedMode and gCmd != cmdCompileToCpp: CC[compiler].cppCompiler
else: compiler.getCompilerExe("")
elif gMixedMode and conf.cmd != cmdCompileToCpp: CC[compiler].cppCompiler
else: getCompilerExe(conf, compiler, "")
proc getCompileCFileCmd*(cfile: Cfile): string =
proc getCompileCFileCmd*(conf: ConfigRef; cfile: Cfile): string =
var c = cCompiler
var options = cFileSpecificOptions(cfile.cname)
var exe = getConfigVar(c, ".exe")
if exe.len == 0: exe = c.getCompilerExe(cfile.cname)
var options = cFileSpecificOptions(conf, cfile.cname)
var exe = getConfigVar(conf, c, ".exe")
if exe.len == 0: exe = getCompilerExe(conf, c, cfile.cname)
if needsExeExt(): exe = addFileExt(exe, "exe")
if optGenDynLib in gGlobalOptions and
if needsExeExt(conf): exe = addFileExt(exe, "exe")
if optGenDynLib in conf.globalOptions and
ospNeedsPIC in platform.OS[targetOS].props:
add(options, ' ' & CC[c].pic)
var includeCmd, compilePattern: string
if not noAbsolutePaths():
if not noAbsolutePaths(conf):
# compute include paths:
includeCmd = CC[c].includeCmd & quoteShell(libpath)
includeCmd = CC[c].includeCmd & quoteShell(conf.libpath)
for includeDir in items(cIncludes):
includeCmd.add(join([CC[c].includeCmd, includeDir.quoteShell]))
@ -564,18 +571,18 @@ proc getCompileCFileCmd*(cfile: Cfile): string =
compilePattern = joinPath(ccompilerpath, exe)
else:
includeCmd = ""
compilePattern = c.getCompilerExe(cfile.cname)
compilePattern = getCompilerExe(conf, c, cfile.cname)
var cf = if noAbsolutePaths(): extractFilename(cfile.cname)
var cf = if noAbsolutePaths(conf): extractFilename(cfile.cname)
else: cfile.cname
var objfile =
if cfile.obj.len == 0:
if not cfile.flags.contains(CfileFlag.External) or noAbsolutePaths():
toObjFile(cf)
if not cfile.flags.contains(CfileFlag.External) or noAbsolutePaths(conf):
toObjFile(conf, cf)
else:
completeCFilePath(toObjFile(cf))
elif noAbsolutePaths():
completeCFilePath(conf, toObjFile(conf, cf))
elif noAbsolutePaths(conf):
extractFilename(cfile.obj)
else:
cfile.obj
@ -584,30 +591,30 @@ proc getCompileCFileCmd*(cfile: Cfile): string =
cf = quoteShell(cf)
result = quoteShell(compilePattern % [
"file", cf, "objfile", objfile, "options", options,
"include", includeCmd, "nim", getPrefixDir(),
"nim", getPrefixDir(), "lib", libpath])
"include", includeCmd, "nim", getPrefixDir(conf),
"nim", getPrefixDir(conf), "lib", conf.libpath])
add(result, ' ')
addf(result, CC[c].compileTmpl, [
"file", cf, "objfile", objfile,
"options", options, "include", includeCmd,
"nim", quoteShell(getPrefixDir()),
"nim", quoteShell(getPrefixDir()),
"lib", quoteShell(libpath)])
"nim", quoteShell(getPrefixDir(conf)),
"nim", quoteShell(getPrefixDir(conf)),
"lib", quoteShell(conf.libpath)])
proc footprint(cfile: Cfile): SecureHash =
proc footprint(conf: ConfigRef; cfile: Cfile): SecureHash =
result = secureHash(
$secureHashFile(cfile.cname) &
platform.OS[targetOS].name &
platform.CPU[targetCPU].name &
extccomp.CC[extccomp.cCompiler].name &
getCompileCFileCmd(cfile))
getCompileCFileCmd(conf, cfile))
proc externalFileChanged(cfile: Cfile): bool =
if gCmd notin {cmdCompileToC, cmdCompileToCpp, cmdCompileToOC, cmdCompileToLLVM}:
proc externalFileChanged(conf: ConfigRef; cfile: Cfile): bool =
if conf.cmd notin {cmdCompileToC, cmdCompileToCpp, cmdCompileToOC, cmdCompileToLLVM}:
return false
var hashFile = toGeneratedFile(cfile.cname.withPackageName, "sha1")
var currentHash = footprint(cfile)
var hashFile = toGeneratedFile(conf, conf.withPackageName(cfile.cname), "sha1")
var currentHash = footprint(conf, cfile)
var f: File
if open(f, hashFile, fmRead):
let oldHash = parseSecureHash(f.readLine())
@ -620,133 +627,137 @@ proc externalFileChanged(cfile: Cfile): bool =
f.writeLine($currentHash)
close(f)
proc addExternalFileToCompile*(c: var Cfile) =
if optForceFullMake notin gGlobalOptions and not externalFileChanged(c):
proc addExternalFileToCompile*(conf: ConfigRef; c: var Cfile) =
if optForceFullMake notin conf.globalOptions and not externalFileChanged(conf, c):
c.flags.incl CfileFlag.Cached
toCompile.add(c)
proc addExternalFileToCompile*(filename: string) =
proc addExternalFileToCompile*(conf: ConfigRef; filename: string) =
var c = Cfile(cname: filename,
obj: toObjFile(completeCFilePath(changeFileExt(filename, ""), false)),
obj: toObjFile(conf, completeCFilePath(conf, changeFileExt(filename, ""), false)),
flags: {CfileFlag.External})
addExternalFileToCompile(c)
addExternalFileToCompile(conf, c)
proc compileCFile(list: CFileList, script: var Rope, cmds: var TStringSeq,
proc compileCFile(conf: ConfigRef; list: CFileList, script: var Rope, cmds: var TStringSeq,
prettyCmds: var TStringSeq) =
for it in list:
# call the C compiler for the .c file:
if it.flags.contains(CfileFlag.Cached): continue
var compileCmd = getCompileCFileCmd(it)
if optCompileOnly notin gGlobalOptions:
var compileCmd = getCompileCFileCmd(conf, it)
if optCompileOnly notin conf.globalOptions:
add(cmds, compileCmd)
let (_, name, _) = splitFile(it.cname)
add(prettyCmds, "CC: " & name)
if optGenScript in gGlobalOptions:
if optGenScript in conf.globalOptions:
add(script, compileCmd)
add(script, tnl)
proc getLinkCmd(projectfile, objfiles: string): string =
if optGenStaticLib in gGlobalOptions:
proc getLinkCmd(conf: ConfigRef; projectfile, objfiles: string): string =
if optGenStaticLib in conf.globalOptions:
var libname: string
if options.outFile.len > 0:
libname = options.outFile.expandTilde
if conf.outFile.len > 0:
libname = conf.outFile.expandTilde
if not libname.isAbsolute():
libname = getCurrentDir() / libname
else:
libname = (libNameTmpl() % splitFile(gProjectName).name)
libname = (libNameTmpl() % splitFile(conf.projectName).name)
result = CC[cCompiler].buildLib % ["libfile", libname,
"objfiles", objfiles]
else:
var linkerExe = getConfigVar(cCompiler, ".linkerexe")
if len(linkerExe) == 0: linkerExe = cCompiler.getLinkerExe
var linkerExe = getConfigVar(conf, cCompiler, ".linkerexe")
if len(linkerExe) == 0: linkerExe = getLinkerExe(conf, cCompiler)
# bug #6452: We must not use ``quoteShell`` here for ``linkerExe``
if needsExeExt(): linkerExe = addFileExt(linkerExe, "exe")
if noAbsolutePaths(): result = linkerExe
if needsExeExt(conf): linkerExe = addFileExt(linkerExe, "exe")
if noAbsolutePaths(conf): result = linkerExe
else: result = joinPath(ccompilerpath, linkerExe)
let buildgui = if optGenGuiApp in gGlobalOptions: CC[cCompiler].buildGui
let buildgui = if optGenGuiApp in conf.globalOptions: CC[cCompiler].buildGui
else: ""
var exefile, builddll: string
if optGenDynLib in gGlobalOptions:
if optGenDynLib in conf.globalOptions:
exefile = platform.OS[targetOS].dllFrmt % splitFile(projectfile).name
builddll = CC[cCompiler].buildDll
else:
exefile = splitFile(projectfile).name & platform.OS[targetOS].exeExt
builddll = ""
if options.outFile.len > 0:
exefile = options.outFile.expandTilde
if conf.outFile.len > 0:
exefile = conf.outFile.expandTilde
if not exefile.isAbsolute():
exefile = getCurrentDir() / exefile
if not noAbsolutePaths():
if not noAbsolutePaths(conf):
if not exefile.isAbsolute():
exefile = joinPath(splitFile(projectfile).dir, exefile)
when false:
if optCDebug in gGlobalOptions:
if optCDebug in conf.globalOptions:
writeDebugInfo(exefile.changeFileExt("ndb"))
exefile = quoteShell(exefile)
let linkOptions = getLinkOptions() & " " &
getConfigVar(cCompiler, ".options.linker")
var linkTmpl = getConfigVar(cCompiler, ".linkTmpl")
let linkOptions = getLinkOptions(conf) & " " &
getConfigVar(conf, cCompiler, ".options.linker")
var linkTmpl = getConfigVar(conf, cCompiler, ".linkTmpl")
if linkTmpl.len == 0:
linkTmpl = CC[cCompiler].linkTmpl
result = quoteShell(result % ["builddll", builddll,
"buildgui", buildgui, "options", linkOptions, "objfiles", objfiles,
"exefile", exefile, "nim", getPrefixDir(), "lib", libpath])
"exefile", exefile, "nim", getPrefixDir(conf), "lib", conf.libpath])
result.add ' '
addf(result, linkTmpl, ["builddll", builddll,
"buildgui", buildgui, "options", linkOptions,
"objfiles", objfiles, "exefile", exefile,
"nim", quoteShell(getPrefixDir()),
"lib", quoteShell(libpath)])
"nim", quoteShell(getPrefixDir(conf)),
"lib", quoteShell(conf.libpath)])
template tryExceptOSErrorMessage(errorPrefix: string = "", body: untyped): typed =
template tryExceptOSErrorMessage(conf: ConfigRef; errorPrefix: string = "", body: untyped): typed =
try:
body
except OSError:
let ose = (ref OSError)(getCurrentException())
if errorPrefix.len > 0:
rawMessage(errGenerated, errorPrefix & " " & ose.msg & " " & $ose.errorCode)
rawMessage(conf, errGenerated, errorPrefix & " " & ose.msg & " " & $ose.errorCode)
else:
rawMessage(errExecutionOfProgramFailed, ose.msg & " " & $ose.errorCode)
rawMessage(conf, errGenerated, "execution of an external program failed: '$1'" %
(ose.msg & " " & $ose.errorCode))
raise
proc execLinkCmd(linkCmd: string) =
tryExceptOSErrorMessage("invocation of external linker program failed."):
execExternalProgram(linkCmd,
if optListCmd in gGlobalOptions or gVerbosity > 1: hintExecuting else: hintLinking)
proc execLinkCmd(conf: ConfigRef; linkCmd: string) =
tryExceptOSErrorMessage(conf, "invocation of external linker program failed."):
execExternalProgram(conf, linkCmd,
if optListCmd in conf.globalOptions or conf.verbosity > 1: hintExecuting else: hintLinking)
proc execCmdsInParallel(cmds: seq[string]; prettyCb: proc (idx: int)) =
proc execCmdsInParallel(conf: ConfigRef; cmds: seq[string]; prettyCb: proc (idx: int)) =
let runCb = proc (idx: int, p: Process) =
let exitCode = p.peekExitCode
if exitCode != 0:
rawMessage(errGenerated, "execution of an external compiler program '" &
rawMessage(conf, errGenerated, "execution of an external compiler program '" &
cmds[idx] & "' failed with exit code: " & $exitCode & "\n\n" &
p.outputStream.readAll.strip)
if gNumberOfProcessors == 0: gNumberOfProcessors = countProcessors()
if conf.numberOfProcessors == 0: conf.numberOfProcessors = countProcessors()
var res = 0
if gNumberOfProcessors <= 1:
if conf.numberOfProcessors <= 1:
for i in countup(0, high(cmds)):
tryExceptOSErrorMessage("invocation of external compiler program failed."):
res = execWithEcho(cmds[i])
if res != 0: rawMessage(errExecutionOfProgramFailed, cmds[i])
tryExceptOSErrorMessage(conf, "invocation of external compiler program failed."):
res = execWithEcho(conf, cmds[i])
if res != 0:
rawMessage(conf, errGenerated, "execution of an external program failed: '$1'" %
cmds[i])
else:
tryExceptOSErrorMessage("invocation of external compiler program failed."):
if optListCmd in gGlobalOptions or gVerbosity > 1:
tryExceptOSErrorMessage(conf, "invocation of external compiler program failed."):
if optListCmd in conf.globalOptions or conf.verbosity > 1:
res = execProcesses(cmds, {poEchoCmd, poStdErrToStdOut, poUsePath},
gNumberOfProcessors, afterRunEvent=runCb)
elif gVerbosity == 1:
conf.numberOfProcessors, afterRunEvent=runCb)
elif conf.verbosity == 1:
res = execProcesses(cmds, {poStdErrToStdOut, poUsePath},
gNumberOfProcessors, prettyCb, afterRunEvent=runCb)
conf.numberOfProcessors, prettyCb, afterRunEvent=runCb)
else:
res = execProcesses(cmds, {poStdErrToStdOut, poUsePath},
gNumberOfProcessors, afterRunEvent=runCb)
conf.numberOfProcessors, afterRunEvent=runCb)
if res != 0:
if gNumberOfProcessors <= 1:
rawMessage(errExecutionOfProgramFailed, cmds.join())
if conf.numberOfProcessors <= 1:
rawMessage(conf, errGenerated, "execution of an external program failed: '$1'" %
cmds.join())
proc callCCompiler*(projectfile: string) =
proc callCCompiler*(conf: ConfigRef; projectfile: string) =
var
linkCmd: string
if gGlobalOptions * {optCompileOnly, optGenScript} == {optCompileOnly}:
if conf.globalOptions * {optCompileOnly, optGenScript} == {optCompileOnly}:
return # speed up that call if only compiling and no script shall be
# generated
#var c = cCompiler
@ -755,36 +766,36 @@ proc callCCompiler*(projectfile: string) =
var prettyCmds: TStringSeq = @[]
let prettyCb = proc (idx: int) =
echo prettyCmds[idx]
compileCFile(toCompile, script, cmds, prettyCmds)
if optCompileOnly notin gGlobalOptions:
execCmdsInParallel(cmds, prettyCb)
if optNoLinking notin gGlobalOptions:
compileCFile(conf, toCompile, script, cmds, prettyCmds)
if optCompileOnly notin conf.globalOptions:
execCmdsInParallel(conf, cmds, prettyCb)
if optNoLinking notin conf.globalOptions:
# call the linker:
var objfiles = ""
for it in externalToLink:
let objFile = if noAbsolutePaths(): it.extractFilename else: it
let objFile = if noAbsolutePaths(conf): it.extractFilename else: it
add(objfiles, ' ')
add(objfiles, quoteShell(
addFileExt(objFile, CC[cCompiler].objExt)))
for x in toCompile:
let objFile = if noAbsolutePaths(): x.obj.extractFilename else: x.obj
let objFile = if noAbsolutePaths(conf): x.obj.extractFilename else: x.obj
add(objfiles, ' ')
add(objfiles, quoteShell(objFile))
linkCmd = getLinkCmd(projectfile, objfiles)
if optCompileOnly notin gGlobalOptions:
execLinkCmd(linkCmd)
linkCmd = getLinkCmd(conf, projectfile, objfiles)
if optCompileOnly notin conf.globalOptions:
execLinkCmd(conf, linkCmd)
else:
linkCmd = ""
if optGenScript in gGlobalOptions:
if optGenScript in conf.globalOptions:
add(script, linkCmd)
add(script, tnl)
generateScript(projectfile, script)
generateScript(conf, projectfile, script)
#from json import escapeJson
import json
proc writeJsonBuildInstructions*(projectfile: string) =
proc writeJsonBuildInstructions*(conf: ConfigRef; projectfile: string) =
template lit(x: untyped) = f.write x
template str(x: untyped) =
when compiles(escapeJson(x, buf)):
@ -794,11 +805,11 @@ proc writeJsonBuildInstructions*(projectfile: string) =
else:
f.write escapeJson(x)
proc cfiles(f: File; buf: var string; clist: CfileList, isExternal: bool) =
proc cfiles(conf: ConfigRef; f: File; buf: var string; clist: CfileList, isExternal: bool) =
var pastStart = false
for it in clist:
if CfileFlag.Cached in it.flags: continue
let compileCmd = getCompileCFileCmd(it)
let compileCmd = getCompileCFileCmd(conf, it)
if pastStart: lit "],\L"
lit "["
str it.cname
@ -807,11 +818,11 @@ proc writeJsonBuildInstructions*(projectfile: string) =
pastStart = true
lit "]\L"
proc linkfiles(f: File; buf, objfiles: var string; clist: CfileList;
proc linkfiles(conf: ConfigRef; f: File; buf, objfiles: var string; clist: CfileList;
llist: seq[string]) =
var pastStart = false
for it in llist:
let objfile = if noAbsolutePaths(): it.extractFilename
let objfile = if noAbsolutePaths(conf): it.extractFilename
else: it
let objstr = addFileExt(objfile, CC[cCompiler].objExt)
add(objfiles, ' ')
@ -832,25 +843,25 @@ proc writeJsonBuildInstructions*(projectfile: string) =
var buf = newStringOfCap(50)
let file = projectfile.splitFile.name
let jsonFile = toGeneratedFile(file, "json")
let jsonFile = toGeneratedFile(conf, file, "json")
var f: File
if open(f, jsonFile, fmWrite):
lit "{\"compile\":[\L"
cfiles(f, buf, toCompile, false)
cfiles(conf, f, buf, toCompile, false)
lit "],\L\"link\":[\L"
var objfiles = ""
# XXX add every file here that is to link
linkfiles(f, buf, objfiles, toCompile, externalToLink)
linkfiles(conf, f, buf, objfiles, toCompile, externalToLink)
lit "],\L\"linkcmd\": "
str getLinkCmd(projectfile, objfiles)
str getLinkCmd(conf, projectfile, objfiles)
lit "\L}\L"
close(f)
proc runJsonBuildInstructions*(projectfile: string) =
proc runJsonBuildInstructions*(conf: ConfigRef; projectfile: string) =
let file = projectfile.splitFile.name
let jsonFile = toGeneratedFile(file, "json")
let jsonFile = toGeneratedFile(conf, file, "json")
try:
let data = json.parseFile(jsonFile)
let toCompile = data["compile"]
@ -867,32 +878,32 @@ proc runJsonBuildInstructions*(projectfile: string) =
let prettyCb = proc (idx: int) =
echo prettyCmds[idx]
execCmdsInParallel(cmds, prettyCb)
execCmdsInParallel(conf, cmds, prettyCb)
let linkCmd = data["linkcmd"]
doAssert linkCmd.kind == JString
execLinkCmd(linkCmd.getStr)
execLinkCmd(conf, linkCmd.getStr)
except:
echo getCurrentException().getStackTrace()
quit "error evaluating JSON file: " & jsonFile
proc genMappingFiles(list: CFileList): Rope =
proc genMappingFiles(conf: ConfigRef; list: CFileList): Rope =
for it in list:
addf(result, "--file:r\"$1\"$N", [rope(it.cname)])
proc writeMapping*(gSymbolMapping: Rope) =
if optGenMapping notin gGlobalOptions: return
proc writeMapping*(conf: ConfigRef; symbolMapping: Rope) =
if optGenMapping notin conf.globalOptions: return
var code = rope("[C_Files]\n")
add(code, genMappingFiles(toCompile))
add(code, genMappingFiles(conf, toCompile))
add(code, "\n[C_Compiler]\nFlags=")
add(code, strutils.escape(getCompileOptions()))
add(code, strutils.escape(getCompileOptions(conf)))
add(code, "\n[Linker]\nFlags=")
add(code, strutils.escape(getLinkOptions() & " " &
getConfigVar(cCompiler, ".options.linker")))
add(code, strutils.escape(getLinkOptions(conf) & " " &
getConfigVar(conf, cCompiler, ".options.linker")))
add(code, "\n[Environment]\nlibpath=")
add(code, strutils.escape(libpath))
add(code, strutils.escape(conf.libpath))
addf(code, "\n[Symbols]$n$1", [gSymbolMapping])
writeRope(code, joinPath(gProjectPath, "mapping.txt"))
addf(code, "\n[Symbols]$n$1", [symbolMapping])
writeRope(code, joinPath(conf.projectPath, "mapping.txt"))

57
compiler/filter_tmpl.nim
View file

@ -27,7 +27,7 @@ type
emit, conc, toStr: string
curly, bracket, par: int
pendingExprLine: bool
config: ConfigRef
const
PatternChars = {'a'..'z', 'A'..'Z', '0'..'9', '\x80'..'\xFF', '.', '_'}
@ -35,16 +35,15 @@ const
proc newLine(p: var TTmplParser) =
llStreamWrite(p.outp, repeat(')', p.emitPar))
p.emitPar = 0
if p.info.line > int16(1): llStreamWrite(p.outp, "\n")
if p.info.line > uint16(1): llStreamWrite(p.outp, "\n")
if p.pendingExprLine:
llStreamWrite(p.outp, spaces(2))
p.pendingExprLine = false
proc scanPar(p: var TTmplParser, d: int) =
var i = d
while true:
while i < p.x.len:
case p.x[i]
of '\0': break
of '(': inc(p.par)
of ')': dec(p.par)
of '[': inc(p.bracket)
@ -63,16 +62,19 @@ const
proc parseLine(p: var TTmplParser) =
var j = 0
while p.x[j] == ' ': inc(j)
if p.x[0] == p.nimDirective and p.x[1] == '?':
let len = p.x.len
while j < len and p.x[j] == ' ': inc(j)
if len >= 2 and p.x[0] == p.nimDirective and p.x[1] == '?':
newLine(p)
elif p.x[j] == p.nimDirective:
elif j < len and p.x[j] == p.nimDirective:
newLine(p)
inc(j)
while p.x[j] == ' ': inc(j)
while j < len and p.x[j] == ' ': inc(j)
let d = j
var keyw = ""
while p.x[j] in PatternChars:
while j < len and p.x[j] in PatternChars:
add(keyw, p.x[j])
inc(j)
@ -84,7 +86,7 @@ proc parseLine(p: var TTmplParser) =
dec(p.indent, 2)
else:
p.info.col = int16(j)
localError(p.info, errXNotAllowedHere, "end")
localError(p.config, p.info, "'end' does not close a control flow construct")
llStreamWrite(p.outp, spaces(p.indent))
llStreamWrite(p.outp, "#end")
of "if", "when", "try", "while", "for", "block", "case", "proc", "iterator",
@ -126,10 +128,8 @@ proc parseLine(p: var TTmplParser) =
llStreamWrite(p.outp, "(\"")
inc(p.emitPar)
p.state = psTempl
while true:
while j < len:
case p.x[j]
of '\0':
break
of '\x01'..'\x1F', '\x80'..'\xFF':
llStreamWrite(p.outp, "\\x")
llStreamWrite(p.outp, toHex(ord(p.x[j]), 2))
@ -156,11 +156,8 @@ proc parseLine(p: var TTmplParser) =
llStreamWrite(p.outp, '(')
inc(j)
var curly = 0
while true:
while j < len:
case p.x[j]
of '\0':
localError(p.info, errXExpected, "}")
break
of '{':
inc(j)
inc(curly)
@ -173,6 +170,9 @@ proc parseLine(p: var TTmplParser) =
else:
llStreamWrite(p.outp, p.x[j])
inc(j)
if curly > 0:
localError(p.config, p.info, "expected closing '}'")
break
llStreamWrite(p.outp, ')')
llStreamWrite(p.outp, p.conc)
llStreamWrite(p.outp, '\"')
@ -181,7 +181,7 @@ proc parseLine(p: var TTmplParser) =
llStreamWrite(p.outp, p.conc)
llStreamWrite(p.outp, p.toStr)
llStreamWrite(p.outp, '(')
while p.x[j] in PatternChars:
while j < len and p.x[j] in PatternChars:
llStreamWrite(p.outp, p.x[j])
inc(j)
llStreamWrite(p.outp, ')')
@ -193,28 +193,29 @@ proc parseLine(p: var TTmplParser) =
inc(j)
else:
p.info.col = int16(j)
localError(p.info, errInvalidExpression, "$")
localError(p.config, p.info, "invalid expression")
else:
llStreamWrite(p.outp, p.x[j])
inc(j)
llStreamWrite(p.outp, "\\n\"")
proc filterTmpl*(stdin: PLLStream, filename: string, call: PNode): PLLStream =
proc filterTmpl*(stdin: PLLStream, filename: string, call: PNode; conf: ConfigRef): PLLStream =
var p: TTmplParser
p.info = newLineInfo(filename, 0, 0)
p.config = conf
p.info = newLineInfo(conf, filename, 0, 0)
p.outp = llStreamOpen("")
p.inp = stdin
p.subsChar = charArg(call, "subschar", 1, '$')
p.nimDirective = charArg(call, "metachar", 2, '#')
p.emit = strArg(call, "emit", 3, "result.add")
p.conc = strArg(call, "conc", 4, " & ")
p.toStr = strArg(call, "tostring", 5, "$")
p.subsChar = charArg(conf, call, "subschar", 1, '$')
p.nimDirective = charArg(conf, call, "metachar", 2, '#')
p.emit = strArg(conf, call, "emit", 3, "result.add")
p.conc = strArg(conf, call, "conc", 4, " & ")
p.toStr = strArg(conf, call, "tostring", 5, "$")
p.x = newStringOfCap(120)
# do not process the first line which contains the directive:
if llStreamReadLine(p.inp, p.x):
p.info.line = p.info.line + int16(1)
p.info.line = p.info.line + 1'u16
while llStreamReadLine(p.inp, p.x):
p.info.line = p.info.line + int16(1)
p.info.line = p.info.line + 1'u16
parseLine(p)
newLine(p)
result = p.outp

43
compiler/filters.nim
View file

@ -13,43 +13,44 @@ import
llstream, os, wordrecg, idents, strutils, ast, astalgo, msgs, options,
renderer
proc invalidPragma(n: PNode) =
localError(n.info, errXNotAllowedHere, renderTree(n, {renderNoComments}))
proc invalidPragma(conf: ConfigRef; n: PNode) =
localError(conf, n.info,
"'$1' not allowed here" % renderTree(n, {renderNoComments}))
proc getArg(n: PNode, name: string, pos: int): PNode =
proc getArg(conf: ConfigRef; n: PNode, name: string, pos: int): PNode =
result = nil
if n.kind in {nkEmpty..nkNilLit}: return
for i in countup(1, sonsLen(n) - 1):
if n.sons[i].kind == nkExprEqExpr:
if n.sons[i].sons[0].kind != nkIdent: invalidPragma(n)
if n.sons[i].sons[0].kind != nkIdent: invalidPragma(conf, n)
if cmpIgnoreStyle(n.sons[i].sons[0].ident.s, name) == 0:
return n.sons[i].sons[1]
elif i == pos:
return n.sons[i]
proc charArg*(n: PNode, name: string, pos: int, default: char): char =
var x = getArg(n, name, pos)
proc charArg*(conf: ConfigRef; n: PNode, name: string, pos: int, default: char): char =
var x = getArg(conf, n, name, pos)
if x == nil: result = default
elif x.kind == nkCharLit: result = chr(int(x.intVal))
else: invalidPragma(n)
else: invalidPragma(conf, n)
proc strArg*(n: PNode, name: string, pos: int, default: string): string =
var x = getArg(n, name, pos)
proc strArg*(conf: ConfigRef; n: PNode, name: string, pos: int, default: string): string =
var x = getArg(conf, n, name, pos)
if x == nil: result = default
elif x.kind in {nkStrLit..nkTripleStrLit}: result = x.strVal
else: invalidPragma(n)
else: invalidPragma(conf, n)
proc boolArg*(n: PNode, name: string, pos: int, default: bool): bool =
var x = getArg(n, name, pos)
proc boolArg*(conf: ConfigRef; n: PNode, name: string, pos: int, default: bool): bool =
var x = getArg(conf, n, name, pos)
if x == nil: result = default
elif x.kind == nkIdent and cmpIgnoreStyle(x.ident.s, "true") == 0: result = true
elif x.kind == nkIdent and cmpIgnoreStyle(x.ident.s, "false") == 0: result = false
else: invalidPragma(n)
else: invalidPragma(conf, n)
proc filterStrip*(stdin: PLLStream, filename: string, call: PNode): PLLStream =
var pattern = strArg(call, "startswith", 1, "")
var leading = boolArg(call, "leading", 2, true)
var trailing = boolArg(call, "trailing", 3, true)
proc filterStrip*(conf: ConfigRef; stdin: PLLStream, filename: string, call: PNode): PLLStream =
var pattern = strArg(conf, call, "startswith", 1, "")
var leading = boolArg(conf, call, "leading", 2, true)
var trailing = boolArg(conf, call, "trailing", 3, true)
result = llStreamOpen("")
var line = newStringOfCap(80)
while llStreamReadLine(stdin, line):
@ -60,10 +61,10 @@ proc filterStrip*(stdin: PLLStream, filename: string, call: PNode): PLLStream =
llStreamWriteln(result, line)
llStreamClose(stdin)
proc filterReplace*(stdin: PLLStream, filename: string, call: PNode): PLLStream =
var sub = strArg(call, "sub", 1, "")
if len(sub) == 0: invalidPragma(call)
var by = strArg(call, "by", 2, "")
proc filterReplace*(conf: ConfigRef; stdin: PLLStream, filename: string, call: PNode): PLLStream =
var sub = strArg(conf, call, "sub", 1, "")
if len(sub) == 0: invalidPragma(conf, call)
var by = strArg(conf, call, "by", 2, "")
result = llStreamOpen("")
var line = newStringOfCap(80)
while llStreamReadLine(stdin, line):

6
compiler/gorgeimpl.nim
View file

@ -9,7 +9,7 @@
## Module that implements ``gorge`` for the compiler.
import msgs, securehash, os, osproc, streams, strutils, options
import msgs, std / sha1, os, osproc, streams, strutils, options
proc readOutput(p: Process): (string, int) =
result[0] = ""
@ -21,11 +21,11 @@ proc readOutput(p: Process): (string, int) =
result[0].setLen(result[0].len - "\n".len)
result[1] = p.waitForExit
proc opGorge*(cmd, input, cache: string, info: TLineInfo): (string, int) =
proc opGorge*(cmd, input, cache: string, info: TLineInfo; conf: ConfigRef): (string, int) =
let workingDir = parentDir(info.toFullPath)
if cache.len > 0:# and optForceFullMake notin gGlobalOptions:
let h = secureHash(cmd & "\t" & input & "\t" & cache)
let filename = options.toGeneratedFile("gorge_" & $h, "txt")
let filename = options.toGeneratedFile(conf, "gorge_" & $h, "txt")
var f: File
if open(f, filename):
result = (f.readAll, 0)

246
compiler/guards.nim
View file

@ -10,7 +10,7 @@
## This module implements the 'implies' relation for guards.
import ast, astalgo, msgs, magicsys, nimsets, trees, types, renderer, idents,
saturate
saturate, modulegraphs, options, configuration
const
someEq = {mEqI, mEqF64, mEqEnum, mEqCh, mEqB, mEqRef, mEqProc,
@ -83,18 +83,25 @@ proc isLetLocation(m: PNode, isApprox: bool): bool =
proc interestingCaseExpr*(m: PNode): bool = isLetLocation(m, true)
let
opLe = createMagic("<=", mLeI)
opLt = createMagic("<", mLtI)
opAnd = createMagic("and", mAnd)
opOr = createMagic("or", mOr)
opIsNil = createMagic("isnil", mIsNil)
opEq = createMagic("==", mEqI)
opAdd = createMagic("+", mAddI)
opSub = createMagic("-", mSubI)
opMul = createMagic("*", mMulI)
opDiv = createMagic("div", mDivI)
opLen = createMagic("len", mLengthSeq)
type
Operators* = object
opNot, opContains, opLe, opLt, opAnd, opOr, opIsNil, opEq: PSym
opAdd, opSub, opMul, opDiv, opLen: PSym
proc initOperators*(g: ModuleGraph): Operators =
result.opLe = createMagic(g, "<=", mLeI)
result.opLt = createMagic(g, "<", mLtI)
result.opAnd = createMagic(g, "and", mAnd)
result.opOr = createMagic(g, "or", mOr)
result.opIsNil = createMagic(g, "isnil", mIsNil)
result.opEq = createMagic(g, "==", mEqI)
result.opAdd = createMagic(g, "+", mAddI)
result.opSub = createMagic(g, "-", mSubI)
result.opMul = createMagic(g, "*", mMulI)
result.opDiv = createMagic(g, "div", mDivI)
result.opLen = createMagic(g, "len", mLengthSeq)
result.opNot = createMagic(g, "not", mNot)
result.opContains = createMagic(g, "contains", mInSet)
proc swapArgs(fact: PNode, newOp: PSym): PNode =
result = newNodeI(nkCall, fact.info, 3)
@ -102,16 +109,16 @@ proc swapArgs(fact: PNode, newOp: PSym): PNode =
result.sons[1] = fact.sons[2]
result.sons[2] = fact.sons[1]
proc neg(n: PNode): PNode =
proc neg(n: PNode; o: Operators): PNode =
if n == nil: return nil
case n.getMagic
of mNot:
result = n.sons[1]
of someLt:
# not (a < b) == a >= b == b <= a
result = swapArgs(n, opLe)
result = swapArgs(n, o.opLe)
of someLe:
result = swapArgs(n, opLt)
result = swapArgs(n, o.opLt)
of mInSet:
if n.sons[1].kind != nkCurly: return nil
let t = n.sons[2].typ.skipTypes(abstractInst)
@ -133,11 +140,11 @@ proc neg(n: PNode): PNode =
of mOr:
# not (a or b) --> not a and not b
let
a = n.sons[1].neg
b = n.sons[2].neg
a = n.sons[1].neg(o)
b = n.sons[2].neg(o)
if a != nil and b != nil:
result = newNodeI(nkCall, n.info, 3)
result.sons[0] = newSymNode(opAnd)
result.sons[0] = newSymNode(o.opAnd)
result.sons[1] = a
result.sons[2] = b
elif a != nil:
@ -147,7 +154,7 @@ proc neg(n: PNode): PNode =
else:
# leave not (a == 4) as it is
result = newNodeI(nkCall, n.info, 2)
result.sons[0] = newSymNode(opNot)
result.sons[0] = newSymNode(o.opNot)
result.sons[1] = n
proc buildCall(op: PSym; a: PNode): PNode =
@ -181,7 +188,7 @@ proc `|div|`(a, b: PNode): PNode =
if a.kind in {nkCharLit..nkUInt64Lit}: result.intVal = a.intVal div b.intVal
else: result.floatVal = a.floatVal / b.floatVal
proc negate(a, b, res: PNode): PNode =
proc negate(a, b, res: PNode; o: Operators): PNode =
if b.kind in {nkCharLit..nkUInt64Lit} and b.intVal != low(BiggestInt):
var b = copyNode(b)
b.intVal = -b.intVal
@ -189,11 +196,11 @@ proc negate(a, b, res: PNode): PNode =
b.intVal = b.intVal |+| a.intVal
result = b
else:
result = buildCall(opAdd, a, b)
result = buildCall(o.opAdd, a, b)
elif b.kind in {nkFloatLit..nkFloat64Lit}:
var b = copyNode(b)
b.floatVal = -b.floatVal
result = buildCall(opAdd, a, b)
result = buildCall(o.opAdd, a, b)
else:
result = res
@ -205,7 +212,7 @@ proc lowBound*(x: PNode): PNode =
result = nkIntLit.newIntNode(firstOrd(x.typ))
result.info = x.info
proc highBound*(x: PNode): PNode =
proc highBound*(x: PNode; o: Operators): PNode =
let typ = x.typ.skipTypes(abstractInst)
result = if typ.kind == tyArray:
nkIntLit.newIntNode(lastOrd(typ))
@ -213,23 +220,23 @@ proc highBound*(x: PNode): PNode =
x.sym.kind == skConst:
nkIntLit.newIntNode(x.sym.ast.len-1)
else:
opAdd.buildCall(opLen.buildCall(x), minusOne())
o.opAdd.buildCall(o.opLen.buildCall(x), minusOne())
result.info = x.info
proc reassociation(n: PNode): PNode =
proc reassociation(n: PNode; o: Operators): PNode =
result = n
# (foo+5)+5 --> foo+10; same for '*'
case result.getMagic
of someAdd:
if result[2].isValue and
result[1].getMagic in someAdd and result[1][2].isValue:
result = opAdd.buildCall(result[1][1], result[1][2] |+| result[2])
result = o.opAdd.buildCall(result[1][1], result[1][2] |+| result[2])
if result[2].intVal == 0:
result = result[1]
of someMul:
if result[2].isValue and
result[1].getMagic in someMul and result[1][2].isValue:
result = opMul.buildCall(result[1][1], result[1][2] |*| result[2])
result = o.opMul.buildCall(result[1][1], result[1][2] |*| result[2])
if result[2].intVal == 1:
result = result[1]
elif result[2].intVal == 0:
@ -243,12 +250,12 @@ proc pred(n: PNode): PNode =
else:
result = n
proc canon*(n: PNode): PNode =
proc canon*(n: PNode; o: Operators): PNode =
# XXX for now only the new code in 'semparallel' uses this
if n.safeLen >= 1:
result = shallowCopy(n)
for i in 0 ..< n.len:
result.sons[i] = canon(n.sons[i])
result.sons[i] = canon(n.sons[i], o)
elif n.kind == nkSym and n.sym.kind == skLet and
n.sym.ast.getMagic in (someEq + someAdd + someMul + someMin +
someMax + someHigh + {mUnaryLt} + someSub + someLen + someDiv):
@ -263,24 +270,24 @@ proc canon*(n: PNode): PNode =
# (4 + foo) + 2 --> (foo + 4) + 2
of someHigh:
# high == len+(-1)
result = opAdd.buildCall(opLen.buildCall(result[1]), minusOne())
result = o.opAdd.buildCall(o.opLen.buildCall(result[1]), minusOne())
of mUnaryLt:
result = buildCall(opAdd, result[1], minusOne())
result = buildCall(o.opAdd, result[1], minusOne())
of someSub:
# x - 4 --> x + (-4)
result = negate(result[1], result[2], result)
result = negate(result[1], result[2], result, o)
of someLen:
result.sons[0] = opLen.newSymNode
result.sons[0] = o.opLen.newSymNode
of someLt:
# x < y same as x <= y-1:
let y = n[2].canon
let y = n[2].canon(o)
let p = pred(y)
let minus = if p != y: p else: opAdd.buildCall(y, minusOne()).canon
result = opLe.buildCall(n[1].canon, minus)
let minus = if p != y: p else: o.opAdd.buildCall(y, minusOne()).canon(o)
result = o.opLe.buildCall(n[1].canon(o), minus)
else: discard
result = skipConv(result)
result = reassociation(result)
result = reassociation(result, o)
# most important rule: (x-4) <= a.len --> x <= a.len+4
case result.getMagic
of someLe:
@ -291,10 +298,10 @@ proc canon*(n: PNode): PNode =
case x.getMagic
of someSub:
result = buildCall(result[0].sym, x[1],
reassociation(opAdd.buildCall(y, x[2])))
reassociation(o.opAdd.buildCall(y, x[2]), o))
of someAdd:
# Rule A:
let plus = negate(y, x[2], nil).reassociation
let plus = negate(y, x[2], nil, o).reassociation(o)
if plus != nil: result = buildCall(result[0].sym, x[1], plus)
else: discard
elif y.kind in nkCallKinds and y.len == 3 and y[2].isValue and
@ -303,9 +310,9 @@ proc canon*(n: PNode): PNode =
case y.getMagic
of someSub:
result = buildCall(result[0].sym, y[1],
reassociation(opAdd.buildCall(x, y[2])))
reassociation(o.opAdd.buildCall(x, y[2]), o))
of someAdd:
let plus = negate(x, y[2], nil).reassociation
let plus = negate(x, y[2], nil, o).reassociation(o)
# ensure that Rule A will not trigger afterwards with the
# additional 'not isLetLocation' constraint:
if plus != nil and not isLetLocation(x, true):
@ -323,15 +330,15 @@ proc canon*(n: PNode): PNode =
result.sons[2] = y[1]
else: discard
proc `+@`*(a: PNode; b: BiggestInt): PNode =
canon(if b != 0: opAdd.buildCall(a, nkIntLit.newIntNode(b)) else: a)
proc buildAdd*(a: PNode; b: BiggestInt; o: Operators): PNode =
canon(if b != 0: o.opAdd.buildCall(a, nkIntLit.newIntNode(b)) else: a, o)
proc usefulFact(n: PNode): PNode =
proc usefulFact(n: PNode; o: Operators): PNode =
case n.getMagic
of someEq:
if skipConv(n.sons[2]).kind == nkNilLit and (
isLetLocation(n.sons[1], false) or isVar(n.sons[1])):
result = opIsNil.buildCall(n.sons[1])
result = o.opIsNil.buildCall(n.sons[1])
else:
if isLetLocation(n.sons[1], true) or isLetLocation(n.sons[2], true):
# XXX algebraic simplifications! 'i-1 < a.len' --> 'i < a.len+1'
@ -351,11 +358,11 @@ proc usefulFact(n: PNode): PNode =
result = n
of mAnd:
let
a = usefulFact(n.sons[1])
b = usefulFact(n.sons[2])
a = usefulFact(n.sons[1], o)
b = usefulFact(n.sons[2], o)
if a != nil and b != nil:
result = newNodeI(nkCall, n.info, 3)
result.sons[0] = newSymNode(opAnd)
result.sons[0] = newSymNode(o.opAnd)
result.sons[1] = a
result.sons[2] = b
elif a != nil:
@ -363,9 +370,9 @@ proc usefulFact(n: PNode): PNode =
elif b != nil:
result = b
of mNot:
let a = usefulFact(n.sons[1])
let a = usefulFact(n.sons[1], o)
if a != nil:
result = a.neg
result = a.neg(o)
of mOr:
# 'or' sucks! (p.isNil or q.isNil) --> hard to do anything
# with that knowledge...
@ -374,14 +381,14 @@ proc usefulFact(n: PNode): PNode =
# (x == 3) or (y == 2) ---> not ( not (x==3) and not (y == 2))
# not (x != 3 and y != 2)
let
a = usefulFact(n.sons[1]).neg
b = usefulFact(n.sons[2]).neg
a = usefulFact(n.sons[1], o).neg(o)
b = usefulFact(n.sons[2], o).neg(o)
if a != nil and b != nil:
result = newNodeI(nkCall, n.info, 3)
result.sons[0] = newSymNode(opAnd)
result.sons[0] = newSymNode(o.opAnd)
result.sons[1] = a
result.sons[2] = b
result = result.neg
result = result.neg(o)
elif n.kind == nkSym and n.sym.kind == skLet:
# consider:
# let a = 2 < x
@ -389,32 +396,34 @@ proc usefulFact(n: PNode): PNode =
# ...
# We make can easily replace 'a' by '2 < x' here:
if n.sym.ast != nil:
result = usefulFact(n.sym.ast)
result = usefulFact(n.sym.ast, o)
elif n.kind == nkStmtListExpr:
result = usefulFact(n.lastSon)
result = usefulFact(n.lastSon, o)
type
TModel* = seq[PNode] # the "knowledge base"
TModel* = object
s*: seq[PNode] # the "knowledge base"
o*: Operators
proc addFact*(m: var TModel, nn: PNode) =
let n = usefulFact(nn)
if n != nil: m.add n
let n = usefulFact(nn, m.o)
if n != nil: m.s.add n
proc addFactNeg*(m: var TModel, n: PNode) =
let n = n.neg
let n = n.neg(m.o)
if n != nil: addFact(m, n)
proc canonOpr(opr: PSym): PSym =
case opr.magic
of someEq: result = opEq
of someLe: result = opLe
of someLt: result = opLt
of someLen: result = opLen
of someAdd: result = opAdd
of someSub: result = opSub
of someMul: result = opMul
of someDiv: result = opDiv
else: result = opr
proc sameOpr(a, b: PSym): bool =
case a.magic
of someEq: result = b.magic in someEq
of someLe: result = b.magic in someLe
of someLt: result = b.magic in someLt
of someLen: result = b.magic in someLen
of someAdd: result = b.magic in someAdd
of someSub: result = b.magic in someSub
of someMul: result = b.magic in someMul
of someDiv: result = b.magic in someDiv
else: result = a == b
proc sameTree*(a, b: PNode): bool =
result = false
@ -425,7 +434,7 @@ proc sameTree*(a, b: PNode): bool =
of nkSym:
result = a.sym == b.sym
if not result and a.sym.magic != mNone:
result = a.sym.magic == b.sym.magic or canonOpr(a.sym) == canonOpr(b.sym)
result = a.sym.magic == b.sym.magic or sameOpr(a.sym, b.sym)
of nkIdent: result = a.ident.id == b.ident.id
of nkCharLit..nkInt64Lit: result = a.intVal == b.intVal
of nkFloatLit..nkFloat64Lit: result = a.floatVal == b.floatVal
@ -462,8 +471,8 @@ proc invalidateFacts*(m: var TModel, n: PNode) =
# The same mechanism could be used for more complex data stored on the heap;
# procs that 'write: []' cannot invalidate 'n.kind' for instance. In fact, we
# could CSE these expressions then and help C's optimizer.
for i in 0..high(m):
if m[i] != nil and m[i].hasSubTree(n): m[i] = nil
for i in 0..high(m.s):
if m.s[i] != nil and m.s[i].hasSubTree(n): m.s[i] = nil
proc valuesUnequal(a, b: PNode): bool =
if a.isValue and b.isValue:
@ -486,7 +495,7 @@ proc impliesEq(fact, eq: PNode): TImplication =
if sameTree(fact.sons[2], eq.sons[loc]) and isValue(eq.sons[val]):
if inSet(fact.sons[1], eq.sons[val]): result = impYes
else: result = impNo
of mNot, mOr, mAnd: internalError(eq.info, "impliesEq")
of mNot, mOr, mAnd: assert(false, "impliesEq")
else: discard
proc leImpliesIn(x, c, aSet: PNode): TImplication =
@ -549,7 +558,7 @@ proc impliesIn(fact, loc, aSet: PNode): TImplication =
elif sameTree(fact.sons[2], loc):
# 4 < x --> 3 <= x
result = geImpliesIn(fact.sons[2], fact.sons[1].pred, aSet)
of mNot, mOr, mAnd: internalError(loc.info, "impliesIn")
of mNot, mOr, mAnd: assert(false, "impliesIn")
else: discard
proc valueIsNil(n: PNode): TImplication =
@ -567,11 +576,11 @@ proc impliesIsNil(fact, eq: PNode): TImplication =
result = valueIsNil(fact.sons[2].skipConv)
elif sameTree(fact.sons[2], eq.sons[1]):
result = valueIsNil(fact.sons[1].skipConv)
of mNot, mOr, mAnd: internalError(eq.info, "impliesIsNil")
of mNot, mOr, mAnd: assert(false, "impliesIsNil")
else: discard
proc impliesGe(fact, x, c: PNode): TImplication =
internalAssert isLocation(x)
assert isLocation(x)
case fact.sons[0].sym.magic
of someEq:
if sameTree(fact.sons[1], x):
@ -603,7 +612,7 @@ proc impliesGe(fact, x, c: PNode): TImplication =
# fact: 3 <= x; question: x >= 2 ? --> true iff 2 <= 3
if isValue(fact.sons[1]) and isValue(c):
if leValue(c, fact.sons[1]): result = impYes
of mNot, mOr, mAnd: internalError(x.info, "impliesGe")
of mNot, mOr, mAnd: assert(false, "impliesGe")
else: discard
proc impliesLe(fact, x, c: PNode): TImplication =
@ -643,7 +652,7 @@ proc impliesLe(fact, x, c: PNode): TImplication =
if isValue(fact.sons[1]) and isValue(c):
if leValue(c, fact.sons[1].pred): result = impNo
of mNot, mOr, mAnd: internalError(x.info, "impliesLe")
of mNot, mOr, mAnd: assert(false, "impliesLe")
else: discard
proc impliesLt(fact, x, c: PNode): TImplication =
@ -707,14 +716,14 @@ proc factImplies(fact, prop: PNode): TImplication =
proc doesImply*(facts: TModel, prop: PNode): TImplication =
assert prop.kind in nkCallKinds
for f in facts:
for f in facts.s:
# facts can be invalidated, in which case they are 'nil':
if not f.isNil:
result = f.factImplies(prop)
if result != impUnknown: return
proc impliesNotNil*(facts: TModel, arg: PNode): TImplication =
result = doesImply(facts, opIsNil.buildCall(arg).neg)
proc impliesNotNil*(m: TModel, arg: PNode): TImplication =
result = doesImply(m, m.o.opIsNil.buildCall(arg).neg(m.o))
proc simpleSlice*(a, b: PNode): BiggestInt =
# returns 'c' if a..b matches (i+c)..(i+c), -1 otherwise. (i)..(i) is matched
@ -768,8 +777,10 @@ macro `=~`(x: PNode, pat: untyped): bool =
var conds = newTree(nnkBracket)
m(x, pat, conds)
when declared(macros.toNimIdent):
result = nestList(toNimIdent"and", conds)
when compiles(nestList(ident"and", conds)):
result = nestList(ident"and", conds)
#elif declared(macros.toNimIdent):
# result = nestList(toNimIdent"and", conds)
else:
result = nestList(!"and", conds)
@ -815,20 +826,20 @@ proc ple(m: TModel; a, b: PNode): TImplication =
if a.getMagic in someMul and a[2].isValue and a[1].getMagic in someDiv and
a[1][2].isValue:
# simplify (x div 4) * 2 <= y to x div (c div d) <= y
if ple(m, buildCall(opDiv, a[1][1], `|div|`(a[1][2], a[2])), b) == impYes:
if ple(m, buildCall(m.o.opDiv, a[1][1], `|div|`(a[1][2], a[2])), b) == impYes:
return impYes
# x*3 + x == x*4. It follows that:
# x*3 + y <= x*4 if y <= x and 3 <= 4
if a =~ x*dc + y and b =~ x2*ec:
if sameTree(x, x2):
let ec1 = opAdd.buildCall(ec, minusOne())
let ec1 = m.o.opAdd.buildCall(ec, minusOne())
if x >=? 1 and ec >=? 1 and dc >=? 1 and dc <=? ec1 and y <=? x:
return impYes
elif a =~ x*dc and b =~ x2*ec + y:
#echo "BUG cam ehrer e ", a, " <=? ", b
if sameTree(x, x2):
let ec1 = opAdd.buildCall(ec, minusOne())
let ec1 = m.o.opAdd.buildCall(ec, minusOne())
if x >=? 1 and ec >=? 1 and dc >=? 1 and dc <=? ec1 and y <=? zero():
return impYes
@ -861,9 +872,9 @@ proc ple(m: TModel; a, b: PNode): TImplication =
# use the knowledge base:
return pleViaModel(m, a, b)
#return doesImply(m, opLe.buildCall(a, b))
#return doesImply(m, o.opLe.buildCall(a, b))
type TReplacements = seq[tuple[a,b: PNode]]
type TReplacements = seq[tuple[a, b: PNode]]
proc replaceSubTree(n, x, by: PNode): PNode =
if sameTree(n, x):
@ -881,11 +892,11 @@ proc applyReplacements(n: PNode; rep: TReplacements): PNode =
proc pleViaModelRec(m: var TModel; a, b: PNode): TImplication =
# now check for inferrable facts: a <= b and b <= c implies a <= c
for i in 0..m.high:
let fact = m[i]
for i in 0..m.s.high:
let fact = m.s[i]
if fact != nil and fact.getMagic in someLe:
# mark as used:
m[i] = nil
m.s[i] = nil
# i <= len-100
# i <=? len-1
# --> true if (len-100) <= (len-1)
@ -917,7 +928,7 @@ proc pleViaModelRec(m: var TModel; a, b: PNode): TImplication =
proc pleViaModel(model: TModel; aa, bb: PNode): TImplication =
# compute replacements:
var replacements: TReplacements = @[]
for fact in model:
for fact in model.s:
if fact != nil and fact.getMagic in someEq:
let a = fact[1]
let b = fact[2]
@ -927,12 +938,13 @@ proc pleViaModel(model: TModel; aa, bb: PNode): TImplication =
var a = aa
var b = bb
if replacements.len > 0:
m = @[]
m.s = @[]
m.o = model.o
# make the other facts consistent:
for fact in model:
for fact in model.s:
if fact != nil and fact.getMagic notin someEq:
# XXX 'canon' should not be necessary here, but it is
m.add applyReplacements(fact, replacements).canon
m.s.add applyReplacements(fact, replacements).canon(m.o)
a = applyReplacements(aa, replacements)
b = applyReplacements(bb, replacements)
else:
@ -941,31 +953,31 @@ proc pleViaModel(model: TModel; aa, bb: PNode): TImplication =
result = pleViaModelRec(m, a, b)
proc proveLe*(m: TModel; a, b: PNode): TImplication =
let x = canon(opLe.buildCall(a, b))
let x = canon(m.o.opLe.buildCall(a, b), m.o)
#echo "ROOT ", renderTree(x[1]), " <=? ", renderTree(x[2])
result = ple(m, x[1], x[2])
if result == impUnknown:
# try an alternative: a <= b iff not (b < a) iff not (b+1 <= a):
let y = canon(opLe.buildCall(opAdd.buildCall(b, one()), a))
let y = canon(m.o.opLe.buildCall(m.o.opAdd.buildCall(b, one()), a), m.o)
result = ~ple(m, y[1], y[2])
proc addFactLe*(m: var TModel; a, b: PNode) =
m.add canon(opLe.buildCall(a, b))
m.s.add canon(m.o.opLe.buildCall(a, b), m.o)
proc settype(n: PNode): PType =
result = newType(tySet, n.typ.owner)
addSonSkipIntLit(result, n.typ)
proc buildOf(it, loc: PNode): PNode =
proc buildOf(it, loc: PNode; o: Operators): PNode =
var s = newNodeI(nkCurly, it.info, it.len-1)
s.typ = settype(loc)
for i in 0..it.len-2: s.sons[i] = it.sons[i]
result = newNodeI(nkCall, it.info, 3)
result.sons[0] = newSymNode(opContains)
result.sons[0] = newSymNode(o.opContains)
result.sons[1] = s
result.sons[2] = loc
proc buildElse(n: PNode): PNode =
proc buildElse(n: PNode; o: Operators): PNode =
var s = newNodeIT(nkCurly, n.info, settype(n.sons[0]))
for i in 1..n.len-2:
let branch = n.sons[i]
@ -973,23 +985,23 @@ proc buildElse(n: PNode): PNode =
for j in 0..branch.len-2:
s.add(branch.sons[j])
result = newNodeI(nkCall, n.info, 3)
result.sons[0] = newSymNode(opContains)
result.sons[0] = newSymNode(o.opContains)
result.sons[1] = s
result.sons[2] = n.sons[0]
proc addDiscriminantFact*(m: var TModel, n: PNode) =
var fact = newNodeI(nkCall, n.info, 3)
fact.sons[0] = newSymNode(opEq)
fact.sons[0] = newSymNode(m.o.opEq)
fact.sons[1] = n.sons[0]
fact.sons[2] = n.sons[1]
m.add fact
m.s.add fact
proc addAsgnFact*(m: var TModel, key, value: PNode) =
var fact = newNodeI(nkCall, key.info, 3)
fact.sons[0] = newSymNode(opEq)
fact.sons[0] = newSymNode(m.o.opEq)
fact.sons[1] = key
fact.sons[2] = value
m.add fact
m.s.add fact
proc sameSubexprs*(m: TModel; a, b: PNode): bool =
# This should be used to check whether two *path expressions* refer to the
@ -1002,7 +1014,7 @@ proc sameSubexprs*(m: TModel; a, b: PNode): bool =
# However, nil checking requires exactly the same mechanism! But for now
# we simply use sameTree and live with the unsoundness of the analysis.
var check = newNodeI(nkCall, a.info, 3)
check.sons[0] = newSymNode(opEq)
check.sons[0] = newSymNode(m.o.opEq)
check.sons[1] = a
check.sons[2] = b
result = m.doesImply(check) == impYes
@ -1010,11 +1022,11 @@ proc sameSubexprs*(m: TModel; a, b: PNode): bool =
proc addCaseBranchFacts*(m: var TModel, n: PNode, i: int) =
let branch = n.sons[i]
if branch.kind == nkOfBranch:
m.add buildOf(branch, n.sons[0])
m.s.add buildOf(branch, n.sons[0], m.o)
else:
m.add n.buildElse.neg
m.s.add n.buildElse(m.o).neg(m.o)
proc buildProperFieldCheck(access, check: PNode): PNode =
proc buildProperFieldCheck(access, check: PNode; o: Operators): PNode =
if check.sons[1].kind == nkCurly:
result = copyTree(check)
if access.kind == nkDotExpr:
@ -1026,10 +1038,10 @@ proc buildProperFieldCheck(access, check: PNode): PNode =
else:
# it is some 'not'
assert check.getMagic == mNot
result = buildProperFieldCheck(access, check.sons[1]).neg
result = buildProperFieldCheck(access, check.sons[1], o).neg(o)
proc checkFieldAccess*(m: TModel, n: PNode) =
proc checkFieldAccess*(m: TModel, n: PNode; conf: ConfigRef) =
for i in 1..n.len-1:
let check = buildProperFieldCheck(n.sons[0], n.sons[i])
let check = buildProperFieldCheck(n.sons[0], n.sons[i], m.o)
if check != nil and m.doesImply(check) != impYes:
message(n.info, warnProveField, renderTree(n.sons[0])); break
message(conf, n.info, warnProveField, renderTree(n.sons[0])); break

20
compiler/hlo.nim
View file

@ -12,12 +12,12 @@
proc hlo(c: PContext, n: PNode): PNode
proc evalPattern(c: PContext, n, orig: PNode): PNode =
internalAssert n.kind == nkCall and n.sons[0].kind == nkSym
internalAssert c.config, n.kind == nkCall and n.sons[0].kind == nkSym
# we need to ensure that the resulting AST is semchecked. However, it's
# aweful to semcheck before macro invocation, so we don't and treat
# templates and macros as immediate in this context.
var rule: string
if optHints in gOptions and hintPattern in gNotes:
if optHints in c.config.options and hintPattern in c.config.notes:
rule = renderTree(n, {renderNoComments})
let s = n.sons[0].sym
case s.kind
@ -27,8 +27,8 @@ proc evalPattern(c: PContext, n, orig: PNode): PNode =
result = semTemplateExpr(c, n, s, {efFromHlo})
else:
result = semDirectOp(c, n, {})
if optHints in gOptions and hintPattern in gNotes:
message(orig.info, hintPattern, rule & " --> '" &
if optHints in c.config.options and hintPattern in c.config.notes:
message(c.config, orig.info, hintPattern, rule & " --> '" &
renderTree(result, {renderNoComments}) & "'")
proc applyPatterns(c: PContext, n: PNode): PNode =
@ -44,8 +44,8 @@ proc applyPatterns(c: PContext, n: PNode): PNode =
assert x.kind in {nkStmtList, nkCall}
# better be safe than sorry, so check evalTemplateCounter too:
inc(evalTemplateCounter)
if evalTemplateCounter > 100:
globalError(n.info, errTemplateInstantiationTooNested)
if evalTemplateCounter > evalTemplateLimit:
globalError(c.config, n.info, "template instantiation too nested")
# deactivate this pattern:
c.patterns[i] = nil
if x.kind == nkStmtList:
@ -86,18 +86,18 @@ proc hlo(c: PContext, n: PNode): PNode =
else:
result = fitNode(c, n.typ, result, n.info)
# optimization has been applied so check again:
result = commonOptimizations(c.module, result)
result = commonOptimizations(c.graph, c.module, result)
result = hlo(c, result)
result = commonOptimizations(c.module, result)
result = commonOptimizations(c.graph, c.module, result)
proc hloBody(c: PContext, n: PNode): PNode =
# fast exit:
if c.patterns.len == 0 or optPatterns notin gOptions: return n
if c.patterns.len == 0 or optPatterns notin c.config.options: return n
c.hloLoopDetector = 0
result = hlo(c, n)
proc hloStmt(c: PContext, n: PNode): PNode =
# fast exit:
if c.patterns.len == 0 or optPatterns notin gOptions: return n
if c.patterns.len == 0 or optPatterns notin c.config.options: return n
c.hloLoopDetector = 0
result = hlo(c, n)

2
compiler/idents.nim
View file

@ -37,7 +37,7 @@ proc resetIdentCache*() =
for i in low(legacy.buckets)..high(legacy.buckets):
legacy.buckets[i] = nil
proc cmpIgnoreStyle(a, b: cstring, blen: int): int =
proc cmpIgnoreStyle*(a, b: cstring, blen: int): int =
if a[0] != b[0]: return 1
var i = 0
var j = 0

12
compiler/idgen.nim
View file

@ -36,20 +36,20 @@ proc setId*(id: int) {.inline.} =
proc idSynchronizationPoint*(idRange: int) =
gFrontEndId = (gFrontEndId div idRange + 1) * idRange + 1
proc toGid(f: string): string =
proc toGid(conf: ConfigRef; f: string): string =
# we used to use ``f.addFileExt("gid")`` (aka ``$project.gid``), but this
# will cause strange bugs if multiple projects are in the same folder, so
# we simply use a project independent name:
result = options.completeGeneratedFilePath("nim.gid")
result = options.completeGeneratedFilePath(conf, "nim.gid")
proc saveMaxIds*(project: string) =
var f = open(project.toGid, fmWrite)
proc saveMaxIds*(conf: ConfigRef; project: string) =
var f = open(toGid(conf, project), fmWrite)
f.writeLine($gFrontEndId)
f.close()
proc loadMaxIds*(project: string) =
proc loadMaxIds*(conf: ConfigRef; project: string) =
var f: File
if open(f, project.toGid, fmRead):
if open(f, toGid(conf, project), fmRead):
var line = newStringOfCap(20)
if f.readLine(line):
var frontEndId = parseInt(line)

51
compiler/importer.nim
View file

@ -11,11 +11,18 @@
import
intsets, strutils, os, ast, astalgo, msgs, options, idents, rodread, lookups,
semdata, passes, renderer, modulepaths
semdata, passes, renderer, modulepaths, sigmatch, configuration
proc evalImport*(c: PContext, n: PNode): PNode
proc evalFrom*(c: PContext, n: PNode): PNode
proc readExceptSet*(c: PContext, n: PNode): IntSet =
assert n.kind in {nkImportExceptStmt, nkExportExceptStmt}
result = initIntSet()
for i in 1 ..< n.len:
let ident = lookups.considerQuotedIdent(c.config, n[i])
result.incl(ident.id)
proc importPureEnumField*(c: PContext; s: PSym) =
var check = strTableGet(c.importTable.symbols, s.name)
if check == nil:
@ -40,7 +47,7 @@ proc rawImportSymbol(c: PContext, s: PSym) =
for j in countup(0, sonsLen(etyp.n) - 1):
var e = etyp.n.sons[j].sym
if e.kind != skEnumField:
internalError(s.info, "rawImportSymbol")
internalError(c.config, s.info, "rawImportSymbol")
# BUGFIX: because of aliases for enums the symbol may already
# have been put into the symbol table
# BUGFIX: but only iff they are the same symbols!
@ -62,14 +69,14 @@ proc rawImportSymbol(c: PContext, s: PSym) =
if hasPattern(s): addPattern(c, s)
proc importSymbol(c: PContext, n: PNode, fromMod: PSym) =
let ident = lookups.considerQuotedIdent(n)
let ident = lookups.considerQuotedIdent(c.config, n)
let s = strTableGet(fromMod.tab, ident)
if s == nil:
errorUndeclaredIdentifier(c, n.info, ident.s)
else:
if s.kind == skStub: loadStub(s)
if s.kind notin ExportableSymKinds:
internalError(n.info, "importSymbol: 2")
internalError(c.config, n.info, "importSymbol: 2")
# for an enumeration we have to add all identifiers
case s.kind
of skProcKinds:
@ -77,7 +84,7 @@ proc importSymbol(c: PContext, n: PNode, fromMod: PSym) =
var it: TIdentIter
var e = initIdentIter(it, fromMod.tab, s.name)
while e != nil:
if e.name.id != s.name.id: internalError(n.info, "importSymbol: 3")
if e.name.id != s.name.id: internalError(c.config, n.info, "importSymbol: 3")
rawImportSymbol(c, e)
e = nextIdentIter(it, fromMod.tab)
else: rawImportSymbol(c, s)
@ -89,7 +96,7 @@ proc importAllSymbolsExcept(c: PContext, fromMod: PSym, exceptSet: IntSet) =
if s.kind != skModule:
if s.kind != skEnumField:
if s.kind notin ExportableSymKinds:
internalError(s.info, "importAllSymbols: " & $s.kind)
internalError(c.config, s.info, "importAllSymbols: " & $s.kind)
if exceptSet.isNil or s.name.id notin exceptSet:
rawImportSymbol(c, s)
s = nextIter(i, fromMod.tab)
@ -110,22 +117,23 @@ proc importForwarded(c: PContext, n: PNode, exceptSet: IntSet) =
elif exceptSet.isNil or s.name.id notin exceptSet:
rawImportSymbol(c, s)
of nkExportExceptStmt:
localError(n.info, errGenerated, "'export except' not implemented")
localError(c.config, n.info, "'export except' not implemented")
else:
for i in 0..safeLen(n)-1:
importForwarded(c, n.sons[i], exceptSet)
proc importModuleAs(n: PNode, realModule: PSym): PSym =
proc importModuleAs(c: PContext; n: PNode, realModule: PSym): PSym =
result = realModule
if n.kind != nkImportAs: discard
elif n.len != 2 or n.sons[1].kind != nkIdent:
localError(n.info, errGenerated, "module alias must be an identifier")
localError(c.config, n.info, "module alias must be an identifier")
elif n.sons[1].ident.id != realModule.name.id:
# some misguided guy will write 'import abc.foo as foo' ...
result = createModuleAlias(realModule, n.sons[1].ident, realModule.info)
result = createModuleAlias(realModule, n.sons[1].ident, realModule.info,
c.config.options)
proc myImportModule(c: PContext, n: PNode): PSym =
var f = checkModuleName(n)
var f = checkModuleName(c.config, n)
if f != InvalidFileIDX:
let L = c.graph.importStack.len
let recursion = c.graph.importStack.find(f)
@ -138,7 +146,7 @@ proc myImportModule(c: PContext, n: PNode): PSym =
err.add toFullPath(c.graph.importStack[i]) & " imports " &
toFullPath(c.graph.importStack[i+1])
c.recursiveDep = err
result = importModuleAs(n, gImportModule(c.graph, c.module, f, c.cache))
result = importModuleAs(c, n, gImportModule(c.graph, c.module, f, c.cache))
#echo "set back to ", L
c.graph.importStack.setLen(L)
# we cannot perform this check reliably because of
@ -146,10 +154,13 @@ proc myImportModule(c: PContext, n: PNode): PSym =
when true:
if result.info.fileIndex == c.module.info.fileIndex and
result.info.fileIndex == n.info.fileIndex:
localError(n.info, errGenerated, "A module cannot import itself")
localError(c.config, n.info, "A module cannot import itself")
if sfDeprecated in result.flags:
message(n.info, warnDeprecated, result.name.s)
#suggestSym(n.info, result, false)
if result.constraint != nil:
message(c.config, n.info, warnDeprecated, result.constraint.strVal & "; " & result.name.s)
else:
message(c.config, n.info, warnDeprecated, result.name.s)
suggestSym(c.config, n.info, result, c.graph.usageSym, false)
proc impMod(c: PContext; it: PNode) =
let m = myImportModule(c, it)
@ -179,7 +190,7 @@ proc evalImport(c: PContext, n: PNode): PNode =
proc evalFrom(c: PContext, n: PNode): PNode =
result = n
checkMinSonsLen(n, 2)
checkMinSonsLen(n, 2, c.config)
var m = myImportModule(c, n.sons[0])
if m != nil:
n.sons[0] = newSymNode(m)
@ -190,14 +201,10 @@ proc evalFrom(c: PContext, n: PNode): PNode =
proc evalImportExcept*(c: PContext, n: PNode): PNode =
result = n
checkMinSonsLen(n, 2)
checkMinSonsLen(n, 2, c.config)
var m = myImportModule(c, n.sons[0])
if m != nil:
n.sons[0] = newSymNode(m)
addDecl(c, m, n.info) # add symbol to symbol table of module
var exceptSet = initIntSet()
for i in countup(1, sonsLen(n) - 1):
let ident = lookups.considerQuotedIdent(n.sons[i])
exceptSet.incl(ident.id)
importAllSymbolsExcept(c, m, exceptSet)
importAllSymbolsExcept(c, m, readExceptSet(c, n))
#importForwarded(c, m.ast, exceptSet)

2
compiler/installer.ini
View file

@ -6,7 +6,7 @@ Name: "Nim"
Version: "$version"
Platforms: """
windows: i386;amd64
linux: i386;amd64;powerpc64;arm;sparc;mips;mipsel;mips64;mips64el;powerpc;powerpc64el;arm64
linux: i386;amd64;powerpc64;arm;sparc;mips;mipsel;mips64;mips64el;powerpc;powerpc64el;arm64;riscv64
macosx: i386;amd64;powerpc64
solaris: i386;amd64;sparc;sparc64
freebsd: i386;amd64

840
compiler/jsgen.nim
View file

File diff suppressed because it is too large Load diff

41
compiler/jstypes.nim
View file

@ -34,11 +34,11 @@ proc genObjectFields(p: PProc, typ: PType, n: PNode): Rope =
s = genTypeInfo(p, field.typ)
result = ("{kind: 1, offset: \"$1\", len: 0, " &
"typ: $2, name: $3, sons: null}") %
[mangleName(field, p.target), s,
[mangleName(p.module, field), s,
makeJSString(field.name.s)]
of nkRecCase:
length = sonsLen(n)
if (n.sons[0].kind != nkSym): internalError(n.info, "genObjectFields")
if (n.sons[0].kind != nkSym): internalError(p.config, n.info, "genObjectFields")
field = n.sons[0].sym
s = genTypeInfo(p, field.typ)
for i in countup(1, length - 1):
@ -47,7 +47,7 @@ proc genObjectFields(p: PProc, typ: PType, n: PNode): Rope =
case b.kind
of nkOfBranch:
if sonsLen(b) < 2:
internalError(b.info, "genObjectFields; nkOfBranch broken")
internalError(p.config, b.info, "genObjectFields; nkOfBranch broken")
for j in countup(0, sonsLen(b) - 2):
if u != nil: add(u, ", ")
if b.sons[j].kind == nkRange:
@ -57,15 +57,15 @@ proc genObjectFields(p: PProc, typ: PType, n: PNode): Rope =
add(u, rope(getOrdValue(b.sons[j])))
of nkElse:
u = rope(lengthOrd(field.typ))
else: internalError(n.info, "genObjectFields(nkRecCase)")
else: internalError(p.config, n.info, "genObjectFields(nkRecCase)")
if result != nil: add(result, ", " & tnl)
addf(result, "[setConstr($1), $2]",
[u, genObjectFields(p, typ, lastSon(b))])
result = ("{kind: 3, offset: \"$1\", len: $3, " &
"typ: $2, name: $4, sons: [$5]}") % [
mangleName(field, p.target), s,
mangleName(p.module, field), s,
rope(lengthOrd(field.typ)), makeJSString(field.name.s), result]
else: internalError(n.info, "genObjectFields")
else: internalError(p.config, n.info, "genObjectFields")
proc objHasTypeField(t: PType): bool {.inline.} =
tfInheritable in t.flags or t.sons[0] != nil
@ -104,7 +104,7 @@ proc genEnumInfo(p: PProc, typ: PType, name: Rope) =
let length = sonsLen(typ.n)
var s: Rope = nil
for i in countup(0, length - 1):
if (typ.n.sons[i].kind != nkSym): internalError(typ.n.info, "genEnumInfo")
if (typ.n.sons[i].kind != nkSym): internalError(p.config, typ.n.info, "genEnumInfo")
let field = typ.n.sons[i].sym
if i > 0: add(s, ", " & tnl)
let extName = if field.ast == nil: field.name.s else: field.ast.strVal
@ -121,27 +121,8 @@ proc genEnumInfo(p: PProc, typ: PType, name: Rope) =
addf(p.g.typeInfo, "$1.base = $2;$n",
[name, genTypeInfo(p, typ.sons[0])])
proc genEnumInfoPHP(p: PProc; t: PType): Rope =
let t = t.skipTypes({tyGenericInst, tyDistinct, tyAlias})
result = "$$NTI$1" % [rope(t.id)]
p.declareGlobal(t.id, result)
if containsOrIncl(p.g.typeInfoGenerated, t.id): return
let length = sonsLen(t.n)
var s: Rope = nil
for i in countup(0, length - 1):
if (t.n.sons[i].kind != nkSym): internalError(t.n.info, "genEnumInfo")
let field = t.n.sons[i].sym
if i > 0: add(s, ", " & tnl)
let extName = if field.ast == nil: field.name.s else: field.ast.strVal
addf(s, "$# => $#$n",
[rope(field.position), makeJSString(extName)])
prepend(p.g.typeInfo, "$$$# = $#;$n" % [result, s])
proc genTypeInfo(p: PProc, typ: PType): Rope =
if p.target == targetPHP:
return makeJSString(typeToString(typ, preferModuleInfo))
let t = typ.skipTypes({tyGenericInst, tyDistinct, tyAlias})
let t = typ.skipTypes({tyGenericInst, tyDistinct, tyAlias, tySink})
result = "NTI$1" % [rope(t.id)]
if containsOrIncl(p.g.typeInfoGenerated, t.id): return
case t.kind
@ -152,7 +133,7 @@ proc genTypeInfo(p: PProc, typ: PType): Rope =
"var $1 = {size: 0,kind: $2,base: null,node: null,finalizer: null};$n" %
[result, rope(ord(t.kind))]
prepend(p.g.typeInfo, s)
of tyVar, tyRef, tyPtr, tySequence, tyRange, tySet:
of tyVar, tyLent, tyRef, tyPtr, tySequence, tyRange, tySet:
var s =
"var $1 = {size: 0,kind: $2,base: null,node: null,finalizer: null};$n" %
[result, rope(ord(t.kind))]
@ -171,5 +152,5 @@ proc genTypeInfo(p: PProc, typ: PType): Rope =
of tyTuple: genTupleInfo(p, t, result)
of tyStatic:
if t.n != nil: result = genTypeInfo(p, lastSon t)
else: internalError("genTypeInfo(" & $t.kind & ')')
else: internalError("genTypeInfo(" & $t.kind & ')')
else: internalError(p.config, "genTypeInfo(" & $t.kind & ')')
else: internalError(p.config, "genTypeInfo(" & $t.kind & ')')

245
compiler/lambdalifting.nim
View file

@ -7,11 +7,11 @@
# distribution, for details about the copyright.
#
# This include file implements lambda lifting for the transformator.
# This file implements lambda lifting for the transformator.
import
intsets, strutils, options, ast, astalgo, trees, treetab, msgs, os,
idents, renderer, types, magicsys, rodread, lowerings, tables
intsets, strutils, options, ast, astalgo, trees, treetab, msgs,
idents, renderer, types, magicsys, rodread, lowerings, tables, modulegraphs
discard """
The basic approach is that captured vars need to be put on the heap and
@ -125,32 +125,32 @@ proc newCall(a: PSym, b: PNode): PNode =
result.add newSymNode(a)
result.add b
proc createStateType(iter: PSym): PType =
proc createClosureIterStateType*(g: ModuleGraph; iter: PSym): PType =
var n = newNodeI(nkRange, iter.info)
addSon(n, newIntNode(nkIntLit, -1))
addSon(n, newIntNode(nkIntLit, 0))
result = newType(tyRange, iter)
result.n = n
var intType = nilOrSysInt()
var intType = nilOrSysInt(g)
if intType.isNil: intType = newType(tyInt, iter)
rawAddSon(result, intType)
proc createStateField(iter: PSym): PSym =
proc createStateField(g: ModuleGraph; iter: PSym): PSym =
result = newSym(skField, getIdent(":state"), iter, iter.info)
result.typ = createStateType(iter)
result.typ = createClosureIterStateType(g, iter)
proc createEnvObj(owner: PSym; info: TLineInfo): PType =
proc createEnvObj(g: ModuleGraph; owner: PSym; info: TLineInfo): PType =
# YYY meh, just add the state field for every closure for now, it's too
# hard to figure out if it comes from a closure iterator:
result = createObj(owner, info, final=false)
rawAddField(result, createStateField(owner))
result = createObj(g, owner, info, final=false)
rawAddField(result, createStateField(g, owner))
proc getIterResult(iter: PSym): PSym =
proc getClosureIterResult*(iter: PSym): PSym =
if resultPos < iter.ast.len:
result = iter.ast.sons[resultPos].sym
else:
# XXX a bit hacky:
result = newSym(skResult, getIdent":result", iter, iter.info)
result = newSym(skResult, getIdent":result", iter, iter.info, {})
result.typ = iter.typ.sons[0]
incl(result.flags, sfUsed)
iter.ast.add newSymNode(result)
@ -166,7 +166,7 @@ proc addHiddenParam(routine: PSym, param: PSym) =
assert sfFromGeneric in param.flags
#echo "produced environment: ", param.id, " for ", routine.id
proc getHiddenParam(routine: PSym): PSym =
proc getHiddenParam(g: ModuleGraph; routine: PSym): PSym =
let params = routine.ast.sons[paramsPos]
let hidden = lastSon(params)
if hidden.kind == nkSym and hidden.sym.kind == skParam and hidden.sym.name.s == paramName:
@ -174,7 +174,7 @@ proc getHiddenParam(routine: PSym): PSym =
assert sfFromGeneric in result.flags
else:
# writeStackTrace()
localError(routine.info, "internal error: could not find env param for " & routine.name.s)
localError(g.config, routine.info, "internal error: could not find env param for " & routine.name.s)
result = routine
proc getEnvParam*(routine: PSym): PSym =
@ -186,11 +186,12 @@ proc getEnvParam*(routine: PSym): PSym =
proc interestingVar(s: PSym): bool {.inline.} =
result = s.kind in {skVar, skLet, skTemp, skForVar, skParam, skResult} and
sfGlobal notin s.flags
sfGlobal notin s.flags and
s.typ.kind notin {tyStatic, tyTypeDesc}
proc illegalCapture(s: PSym): bool {.inline.} =
result = skipTypes(s.typ, abstractInst).kind in
{tyVar, tyOpenArray, tyVarargs} or
{tyVar, tyOpenArray, tyVarargs, tyLent} or
s.kind == skResult
proc isInnerProc(s: PSym): bool =
@ -207,14 +208,14 @@ proc newAsgnStmt(le, ri: PNode, info: TLineInfo): PNode =
result.sons[0] = le
result.sons[1] = ri
proc makeClosure*(prc: PSym; env: PNode; info: TLineInfo): PNode =
proc makeClosure*(g: ModuleGraph; prc: PSym; env: PNode; info: TLineInfo): PNode =
result = newNodeIT(nkClosure, info, prc.typ)
result.add(newSymNode(prc))
if env == nil:
result.add(newNodeIT(nkNilLit, info, getSysType(tyNil)))
result.add(newNodeIT(nkNilLit, info, getSysType(g, info, tyNil)))
else:
if env.skipConv.kind == nkClosure:
localError(info, "internal error: taking closure of closure")
localError(g.config, info, "internal error: taking closure of closure")
result.add(env)
proc interestingIterVar(s: PSym): bool {.inline.} =
@ -226,23 +227,23 @@ proc interestingIterVar(s: PSym): bool {.inline.} =
template isIterator*(owner: PSym): bool =
owner.kind == skIterator and owner.typ.callConv == ccClosure
proc liftingHarmful(owner: PSym): bool {.inline.} =
proc liftingHarmful(conf: ConfigRef; owner: PSym): bool {.inline.} =
## lambda lifting can be harmful for JS-like code generators.
let isCompileTime = sfCompileTime in owner.flags or owner.kind == skMacro
result = gCmd in {cmdCompileToPHP, cmdCompileToJS} and not isCompileTime
result = conf.cmd == cmdCompileToJS and not isCompileTime
proc liftIterSym*(n: PNode; owner: PSym): PNode =
proc liftIterSym*(g: ModuleGraph; n: PNode; owner: PSym): PNode =
# transforms (iter) to (let env = newClosure[iter](); (iter, env))
if liftingHarmful(owner): return n
if liftingHarmful(g.config, owner): return n
let iter = n.sym
assert iter.isIterator
result = newNodeIT(nkStmtListExpr, n.info, n.typ)
let hp = getHiddenParam(iter)
let hp = getHiddenParam(g, iter)
var env: PNode
if owner.isIterator:
let it = getHiddenParam(owner)
let it = getHiddenParam(g, owner)
addUniqueField(it.typ.sons[0], hp)
env = indirectAccess(newSymNode(it), hp, hp.info)
else:
@ -254,11 +255,11 @@ proc liftIterSym*(n: PNode; owner: PSym): PNode =
addVar(v, env)
result.add(v)
# add 'new' statement:
result.add newCall(getSysSym"internalNew", env)
result.add makeClosure(iter, env, n.info)
result.add newCall(getSysSym(g, n.info, "internalNew"), env)
result.add makeClosure(g, iter, env, n.info)
proc freshVarForClosureIter*(s, owner: PSym): PNode =
let envParam = getHiddenParam(owner)
proc freshVarForClosureIter*(g: ModuleGraph; s, owner: PSym): PNode =
let envParam = getHiddenParam(g, owner)
let obj = envParam.typ.lastSon
addField(obj, s)
@ -268,15 +269,19 @@ proc freshVarForClosureIter*(s, owner: PSym): PNode =
if field != nil:
result = rawIndirectAccess(access, field, s.info)
else:
localError(s.info, "internal error: cannot generate fresh variable")
localError(g.config, s.info, "internal error: cannot generate fresh variable")
result = access
# ------------------ new stuff -------------------------------------------
proc markAsClosure(owner: PSym; n: PNode) =
proc markAsClosure(g: ModuleGraph; owner: PSym; n: PNode) =
let s = n.sym
if illegalCapture(s) or owner.typ.callConv notin {ccClosure, ccDefault}:
localError(n.info, errIllegalCaptureX, s.name.s)
if illegalCapture(s):
localError(g.config, n.info, "illegal capture '$1' of type <$2> which is declared here: $3" %
[s.name.s, typeToString(s.typ), $s.info])
elif owner.typ.callConv notin {ccClosure, ccDefault}:
localError(g.config, n.info, "illegal capture '$1' because '$2' has the calling convention: <$3>" %
[s.name.s, owner.name.s, CallingConvToStr[owner.typ.callConv]])
incl(owner.typ.flags, tfCapturesEnv)
owner.typ.callConv = ccClosure
@ -285,12 +290,14 @@ type
processed, capturedVars: IntSet
ownerToType: Table[int, PType]
somethingToDo: bool
graph: ModuleGraph
proc initDetectionPass(fn: PSym): DetectionPass =
proc initDetectionPass(g: ModuleGraph; fn: PSym): DetectionPass =
result.processed = initIntSet()
result.capturedVars = initIntSet()
result.ownerToType = initTable[int, PType]()
result.processed.incl(fn.id)
result.graph = g
discard """
proc outer =
@ -307,7 +314,7 @@ proc getEnvTypeForOwner(c: var DetectionPass; owner: PSym;
result = c.ownerToType.getOrDefault(owner.id)
if result.isNil:
result = newType(tyRef, owner)
let obj = createEnvObj(owner, info)
let obj = createEnvObj(c.graph, owner, info)
rawAddSon(result, obj)
c.ownerToType[owner.id] = result
@ -316,13 +323,13 @@ proc createUpField(c: var DetectionPass; dest, dep: PSym; info: TLineInfo) =
let obj = refObj.lastSon
let fieldType = c.getEnvTypeForOwner(dep, info) #getHiddenParam(dep).typ
if refObj == fieldType:
localError(dep.info, "internal error: invalid up reference computed")
localError(c.graph.config, dep.info, "internal error: invalid up reference computed")
let upIdent = getIdent(upName)
let upField = lookupInRecord(obj.n, upIdent)
if upField != nil:
if upField.typ != fieldType:
localError(dep.info, "internal error: up references do not agree")
localError(c.graph.config, dep.info, "internal error: up references do not agree")
else:
let result = newSym(skField, upIdent, obj.owner, obj.owner.info)
result.typ = fieldType
@ -364,7 +371,7 @@ proc addClosureParam(c: var DetectionPass; fn: PSym; info: TLineInfo) =
cp.typ = t
addHiddenParam(fn, cp)
elif cp.typ != t and fn.kind != skIterator:
localError(fn.info, "internal error: inconsistent environment type")
localError(c.graph.config, fn.info, "internal error: inconsistent environment type")
#echo "adding closure to ", fn.name.s
proc detectCapturedVars(n: PNode; owner: PSym; c: var DetectionPass) =
@ -390,9 +397,13 @@ proc detectCapturedVars(n: PNode; owner: PSym; c: var DetectionPass) =
addClosureParam(c, owner, n.info)
if interestingIterVar(s):
if not c.capturedVars.containsOrIncl(s.id):
let obj = getHiddenParam(owner).typ.lastSon
let obj = getHiddenParam(c.graph, owner).typ.lastSon
#let obj = c.getEnvTypeForOwner(s.owner).lastSon
addField(obj, s)
if s.name == getIdent(":state"):
obj.n[0].sym.id = -s.id
else:
addField(obj, s)
# but always return because the rest of the proc is only relevant when
# ow != owner:
return
@ -410,7 +421,7 @@ proc detectCapturedVars(n: PNode; owner: PSym; c: var DetectionPass) =
"""
# mark 'owner' as taking a closure:
c.somethingToDo = true
markAsClosure(owner, n)
markAsClosure(c.graph, owner, n)
addClosureParam(c, owner, n.info)
#echo "capturing ", n.info
# variable 's' is actually captured:
@ -435,7 +446,7 @@ proc detectCapturedVars(n: PNode; owner: PSym; c: var DetectionPass) =
"""
let up = w.skipGenericOwner
#echo "up for ", w.name.s, " up ", up.name.s
markAsClosure(w, n)
markAsClosure(c.graph, w, n)
addClosureParam(c, w, n.info) # , ow
createUpField(c, w, up, n.info)
w = up
@ -462,10 +473,10 @@ proc initLiftingPass(fn: PSym): LiftingPass =
result.processed.incl(fn.id)
result.envVars = initTable[int, PNode]()
proc accessViaEnvParam(n: PNode; owner: PSym): PNode =
proc accessViaEnvParam(g: ModuleGraph; n: PNode; owner: PSym): PNode =
let s = n.sym
# Type based expression construction for simplicity:
let envParam = getHiddenParam(owner)
let envParam = getHiddenParam(g, owner)
if not envParam.isNil:
var access = newSymNode(envParam)
while true:
@ -477,7 +488,7 @@ proc accessViaEnvParam(n: PNode; owner: PSym): PNode =
let upField = lookupInRecord(obj.n, getIdent(upName))
if upField == nil: break
access = rawIndirectAccess(access, upField, n.info)
localError(n.info, "internal error: environment misses: " & s.name.s)
localError(g.config, n.info, "internal error: environment misses: " & s.name.s)
result = n
proc newEnvVar(owner: PSym; typ: PType): PNode =
@ -496,22 +507,22 @@ proc newEnvVar(owner: PSym; typ: PType): PNode =
proc setupEnvVar(owner: PSym; d: DetectionPass;
c: var LiftingPass): PNode =
if owner.isIterator:
return getHiddenParam(owner).newSymNode
return getHiddenParam(d.graph, owner).newSymNode
result = c.envvars.getOrDefault(owner.id)
if result.isNil:
let envVarType = d.ownerToType.getOrDefault(owner.id)
if envVarType.isNil:
localError owner.info, "internal error: could not determine closure type"
localError d.graph.config, owner.info, "internal error: could not determine closure type"
result = newEnvVar(owner, envVarType)
c.envVars[owner.id] = result
proc getUpViaParam(owner: PSym): PNode =
let p = getHiddenParam(owner)
proc getUpViaParam(g: ModuleGraph; owner: PSym): PNode =
let p = getHiddenParam(g, owner)
result = p.newSymNode
if owner.isIterator:
let upField = lookupInRecord(p.typ.lastSon.n, getIdent(upName))
if upField == nil:
localError(owner.info, "could not find up reference for closure iter")
localError(g.config, owner.info, "could not find up reference for closure iter")
else:
result = rawIndirectAccess(result, upField, p.info)
@ -521,7 +532,7 @@ proc rawClosureCreation(owner: PSym;
var env: PNode
if owner.isIterator:
env = getHiddenParam(owner).newSymNode
env = getHiddenParam(d.graph, owner).newSymNode
else:
env = setupEnvVar(owner, d, c)
if env.kind == nkSym:
@ -529,7 +540,7 @@ proc rawClosureCreation(owner: PSym;
addVar(v, env)
result.add(v)
# add 'new' statement:
result.add(newCall(getSysSym"internalNew", env))
result.add(newCall(getSysSym(d.graph, env.info, "internalNew"), env))
# add assignment statements for captured parameters:
for i in 1..<owner.typ.n.len:
let local = owner.typ.n[i].sym
@ -540,7 +551,7 @@ proc rawClosureCreation(owner: PSym;
let upField = lookupInRecord(env.typ.lastSon.n, getIdent(upName))
if upField != nil:
let up = getUpViaParam(owner)
let up = getUpViaParam(d.graph, owner)
if up != nil and upField.typ == up.typ:
result.add(newAsgnStmt(rawIndirectAccess(env, upField, env.info),
up, env.info))
@ -548,7 +559,7 @@ proc rawClosureCreation(owner: PSym;
# result.add(newAsgnStmt(rawIndirectAccess(env, upField, env.info),
# oldenv, env.info))
else:
localError(env.info, "internal error: cannot create up reference")
localError(d.graph.config, env.info, "internal error: cannot create up reference")
proc closureCreationForIter(iter: PNode;
d: DetectionPass; c: var LiftingPass): PNode =
@ -556,10 +567,10 @@ proc closureCreationForIter(iter: PNode;
let owner = iter.sym.skipGenericOwner
var v = newSym(skVar, getIdent(envName), owner, iter.info)
incl(v.flags, sfShadowed)
v.typ = getHiddenParam(iter.sym).typ
v.typ = getHiddenParam(d.graph, iter.sym).typ
var vnode: PNode
if owner.isIterator:
let it = getHiddenParam(owner)
let it = getHiddenParam(d.graph, owner)
addUniqueField(it.typ.sons[0], v)
vnode = indirectAccess(newSymNode(it), v, v.info)
else:
@ -567,7 +578,7 @@ proc closureCreationForIter(iter: PNode;
var vs = newNodeI(nkVarSection, iter.info)
addVar(vs, vnode)
result.add(vs)
result.add(newCall(getSysSym"internalNew", vnode))
result.add(newCall(getSysSym(d.graph, iter.info, "internalNew"), vnode))
let upField = lookupInRecord(v.typ.lastSon.n, getIdent(upName))
if upField != nil:
@ -576,8 +587,8 @@ proc closureCreationForIter(iter: PNode;
result.add(newAsgnStmt(rawIndirectAccess(vnode, upField, iter.info),
u, iter.info))
else:
localError(iter.info, "internal error: cannot create up reference for iter")
result.add makeClosure(iter.sym, vnode, iter.info)
localError(d.graph.config, iter.info, "internal error: cannot create up reference for iter")
result.add makeClosure(d.graph, iter.sym, vnode, iter.info)
proc accessViaEnvVar(n: PNode; owner: PSym; d: DetectionPass;
c: var LiftingPass): PNode =
@ -587,11 +598,11 @@ proc accessViaEnvVar(n: PNode; owner: PSym; d: DetectionPass;
if field != nil:
result = rawIndirectAccess(access, field, n.info)
else:
localError(n.info, "internal error: not part of closure object type")
localError(d.graph.config, n.info, "internal error: not part of closure object type")
result = n
proc getStateField(owner: PSym): PSym =
getHiddenParam(owner).typ.sons[0].n.sons[0].sym
proc getStateField*(g: ModuleGraph; owner: PSym): PSym =
getHiddenParam(g, owner).typ.sons[0].n.sons[0].sym
proc liftCapturedVars(n: PNode; owner: PSym; d: DetectionPass;
c: var LiftingPass): PNode
@ -599,8 +610,8 @@ proc liftCapturedVars(n: PNode; owner: PSym; d: DetectionPass;
proc transformYield(n: PNode; owner: PSym; d: DetectionPass;
c: var LiftingPass): PNode =
if c.inContainer > 0:
localError(n.info, "invalid control flow: 'yield' within a constructor")
let state = getStateField(owner)
localError(d.graph.config, n.info, "invalid control flow: 'yield' within a constructor")
let state = getStateField(d.graph, owner)
assert state != nil
assert state.typ != nil
assert state.typ.n != nil
@ -610,20 +621,22 @@ proc transformYield(n: PNode; owner: PSym; d: DetectionPass;
var stateAsgnStmt = newNodeI(nkAsgn, n.info)
stateAsgnStmt.add(rawIndirectAccess(newSymNode(getEnvParam(owner)),
state, n.info))
stateAsgnStmt.add(newIntTypeNode(nkIntLit, stateNo, getSysType(tyInt)))
stateAsgnStmt.add(newIntTypeNode(nkIntLit, stateNo,
getSysType(d.graph, n.info, tyInt)))
var retStmt = newNodeI(nkReturnStmt, n.info)
if n.sons[0].kind != nkEmpty:
var a = newNodeI(nkAsgn, n.sons[0].info)
var retVal = liftCapturedVars(n.sons[0], owner, d, c)
addSon(a, newSymNode(getIterResult(owner)))
addSon(a, newSymNode(getClosureIterResult(owner)))
addSon(a, retVal)
retStmt.add(a)
else:
retStmt.add(emptyNode)
var stateLabelStmt = newNodeI(nkState, n.info)
stateLabelStmt.add(newIntTypeNode(nkIntLit, stateNo, getSysType(tyInt)))
stateLabelStmt.add(newIntTypeNode(nkIntLit, stateNo,
getSysType(d.graph, n.info, tyInt)))
result = newNodeI(nkStmtList, n.info)
result.add(stateAsgnStmt)
@ -632,16 +645,16 @@ proc transformYield(n: PNode; owner: PSym; d: DetectionPass;
proc transformReturn(n: PNode; owner: PSym; d: DetectionPass;
c: var LiftingPass): PNode =
let state = getStateField(owner)
let state = getStateField(d.graph, owner)
result = newNodeI(nkStmtList, n.info)
var stateAsgnStmt = newNodeI(nkAsgn, n.info)
stateAsgnStmt.add(rawIndirectAccess(newSymNode(getEnvParam(owner)),
state, n.info))
stateAsgnStmt.add(newIntTypeNode(nkIntLit, -1, getSysType(tyInt)))
stateAsgnStmt.add(newIntTypeNode(nkIntLit, -1, getSysType(d.graph, n.info, tyInt)))
result.add(stateAsgnStmt)
result.add(n)
proc wrapIterBody(n: PNode; owner: PSym): PNode =
proc wrapIterBody(g: ModuleGraph; n: PNode; owner: PSym): PNode =
if not owner.isIterator: return n
when false:
# unfortunately control flow is still convoluted and we can end up
@ -655,7 +668,7 @@ proc wrapIterBody(n: PNode; owner: PSym): PNode =
let info = n.info
result = newNodeI(nkStmtList, info)
var gs = newNodeI(nkGotoState, info)
gs.add(rawIndirectAccess(newSymNode(owner.getHiddenParam), getStateField(owner), info))
gs.add(rawIndirectAccess(newSymNode(getHiddenParam(g, owner)), getStateField(g, owner), info))
result.add(gs)
var state0 = newNodeI(nkState, info)
state0.add(newIntNode(nkIntLit, 0))
@ -664,9 +677,9 @@ proc wrapIterBody(n: PNode; owner: PSym): PNode =
result.add(n)
var stateAsgnStmt = newNodeI(nkAsgn, info)
stateAsgnStmt.add(rawIndirectAccess(newSymNode(owner.getHiddenParam),
getStateField(owner), info))
stateAsgnStmt.add(newIntTypeNode(nkIntLit, -1, getSysType(tyInt)))
stateAsgnStmt.add(rawIndirectAccess(newSymNode(getHiddenParam(g, owner)),
getStateField(g, owner), info))
stateAsgnStmt.add(newIntTypeNode(nkIntLit, -1, getSysType(g, info, tyInt)))
result.add(stateAsgnStmt)
proc symToClosure(n: PNode; owner: PSym; d: DetectionPass;
@ -674,25 +687,25 @@ proc symToClosure(n: PNode; owner: PSym; d: DetectionPass;
let s = n.sym
if s == owner:
# recursive calls go through (lambda, hiddenParam):
let available = getHiddenParam(owner)
result = makeClosure(s, available.newSymNode, n.info)
let available = getHiddenParam(d.graph, owner)
result = makeClosure(d.graph, s, available.newSymNode, n.info)
elif s.isIterator:
result = closureCreationForIter(n, d, c)
elif s.skipGenericOwner == owner:
# direct dependency, so use the outer's env variable:
result = makeClosure(s, setupEnvVar(owner, d, c), n.info)
result = makeClosure(d.graph, s, setupEnvVar(owner, d, c), n.info)
else:
let available = getHiddenParam(owner)
let wanted = getHiddenParam(s).typ
let available = getHiddenParam(d.graph, owner)
let wanted = getHiddenParam(d.graph, s).typ
# ugh: call through some other inner proc;
var access = newSymNode(available)
while true:
if access.typ == wanted:
return makeClosure(s, access, n.info)
return makeClosure(d.graph, s, access, n.info)
let obj = access.typ.sons[0]
let upField = lookupInRecord(obj.n, getIdent(upName))
if upField == nil:
localError(n.info, "internal error: no environment found")
localError(d.graph.config, n.info, "internal error: no environment found")
return n
access = rawIndirectAccess(access, upField, n.info)
@ -708,7 +721,9 @@ proc liftCapturedVars(n: PNode; owner: PSym; d: DetectionPass;
# echo renderTree(s.getBody, {renderIds})
let oldInContainer = c.inContainer
c.inContainer = 0
let body = wrapIterBody(liftCapturedVars(s.getBody, s, d, c), s)
var body = liftCapturedVars(s.getBody, s, d, c)
if oldIterTransf in d.graph.config.features:
body = wrapIterBody(d.graph, body, s)
if c.envvars.getOrDefault(s.id).isNil:
s.ast.sons[bodyPos] = body
else:
@ -718,9 +733,9 @@ proc liftCapturedVars(n: PNode; owner: PSym; d: DetectionPass;
result = symToClosure(n, owner, d, c)
elif s.id in d.capturedVars:
if s.owner != owner:
result = accessViaEnvParam(n, owner)
result = accessViaEnvParam(d.graph, n, owner)
elif owner.isIterator and interestingIterVar(s):
result = accessViaEnvParam(n, owner)
result = accessViaEnvParam(d.graph, n, owner)
else:
result = accessViaEnvVar(n, owner, d, c)
of nkEmpty..pred(nkSym), succ(nkSym)..nkNilLit, nkComesFrom,
@ -751,9 +766,9 @@ proc liftCapturedVars(n: PNode; owner: PSym; d: DetectionPass;
if n[1].kind == nkClosure: result = n[1]
else:
if owner.isIterator:
if n.kind == nkYieldStmt:
if oldIterTransf in d.graph.config.features and n.kind == nkYieldStmt:
return transformYield(n, owner, d, c)
elif n.kind == nkReturnStmt:
elif oldIterTransf in d.graph.config.features and n.kind == nkReturnStmt:
return transformReturn(n, owner, d, c)
elif nfLL in n.flags:
# special case 'when nimVm' due to bug #3636:
@ -786,8 +801,8 @@ proc semCaptureSym*(s, owner: PSym) =
# since the analysis is not entirely correct, we don't set 'tfCapturesEnv'
# here
proc liftIterToProc*(fn: PSym; body: PNode; ptrType: PType): PNode =
var d = initDetectionPass(fn)
proc liftIterToProc*(g: ModuleGraph; fn: PSym; body: PNode; ptrType: PType): PNode =
var d = initDetectionPass(g, fn)
var c = initLiftingPass(fn)
# pretend 'fn' is a closure iterator for the analysis:
let oldKind = fn.kind
@ -796,11 +811,11 @@ proc liftIterToProc*(fn: PSym; body: PNode; ptrType: PType): PNode =
fn.typ.callConv = ccClosure
d.ownerToType[fn.id] = ptrType
detectCapturedVars(body, fn, d)
result = wrapIterBody(liftCapturedVars(body, fn, d, c), fn)
result = wrapIterBody(g, liftCapturedVars(body, fn, d, c), fn)
fn.kind = oldKind
fn.typ.callConv = oldCC
proc liftLambdas*(fn: PSym, body: PNode; tooEarly: var bool): PNode =
proc liftLambdas*(g: ModuleGraph; fn: PSym, body: PNode; tooEarly: var bool): PNode =
# XXX gCmd == cmdCompileToJS does not suffice! The compiletime stuff needs
# the transformation even when compiling to JS ...
@ -808,23 +823,26 @@ proc liftLambdas*(fn: PSym, body: PNode; tooEarly: var bool): PNode =
let isCompileTime = sfCompileTime in fn.flags or fn.kind == skMacro
if body.kind == nkEmpty or (
gCmd in {cmdCompileToPHP, cmdCompileToJS} and not isCompileTime) or
g.config.cmd == cmdCompileToJS and not isCompileTime) or
fn.skipGenericOwner.kind != skModule:
# ignore forward declaration:
result = body
tooEarly = true
else:
var d = initDetectionPass(fn)
var d = initDetectionPass(g, fn)
detectCapturedVars(body, fn, d)
if not d.somethingToDo and fn.isIterator:
addClosureParam(d, fn, body.info)
d.somethingToDo = true
if d.somethingToDo:
var c = initLiftingPass(fn)
var newBody = liftCapturedVars(body, fn, d, c)
result = liftCapturedVars(body, fn, d, c)
if c.envvars.getOrDefault(fn.id) != nil:
newBody = newTree(nkStmtList, rawClosureCreation(fn, d, c), newBody)
result = wrapIterBody(newBody, fn)
result = newTree(nkStmtList, rawClosureCreation(fn, d, c), result)
if oldIterTransf in g.config.features:
result = wrapIterBody(g, result, fn)
else:
result = body
#if fn.name.s == "get2":
@ -832,15 +850,12 @@ proc liftLambdas*(fn: PSym, body: PNode; tooEarly: var bool): PNode =
# echo renderTree(result, {renderIds})
proc liftLambdasForTopLevel*(module: PSym, body: PNode): PNode =
if body.kind == nkEmpty or gCmd == cmdCompileToJS:
result = body
else:
# XXX implement it properly
result = body
# XXX implement it properly
result = body
# ------------------- iterator transformation --------------------------------
proc liftForLoop*(body: PNode; owner: PSym): PNode =
proc liftForLoop*(g: ModuleGraph; body: PNode; owner: PSym): PNode =
# problem ahead: the iterator could be invoked indirectly, but then
# we don't know what environment to create here:
#
@ -865,13 +880,14 @@ proc liftForLoop*(body: PNode; owner: PSym): PNode =
cl = createClosure()
while true:
let i = foo(cl)
nkBreakState(cl.state)
if (nkBreakState(cl.state)):
break
...
"""
if liftingHarmful(owner): return body
if liftingHarmful(g.config, owner): return body
var L = body.len
if not (body.kind == nkForStmt and body[L-2].kind in nkCallKinds):
localError(body.info, "ignored invalid for loop")
localError(g.config, body.info, "ignored invalid for loop")
return body
var call = body[L-2]
@ -884,7 +900,7 @@ proc liftForLoop*(body: PNode; owner: PSym): PNode =
# createClosure()
let iter = op.sym
let hp = getHiddenParam(iter)
let hp = getHiddenParam(g, iter)
env = newSym(skLet, iter.name, owner, body.info)
env.typ = hp.typ
env.flags = hp.flags
@ -893,7 +909,7 @@ proc liftForLoop*(body: PNode; owner: PSym): PNode =
addVar(v, newSymNode(env))
result.add(v)
# add 'new' statement:
result.add(newCall(getSysSym"internalNew", env.newSymNode))
result.add(newCall(getSysSym(g, env.info, "internalNew"), env.newSymNode))
elif op.kind == nkStmtListExpr:
let closure = op.lastSon
if closure.kind == nkClosure:
@ -903,7 +919,7 @@ proc liftForLoop*(body: PNode; owner: PSym): PNode =
var loopBody = newNodeI(nkStmtList, body.info, 3)
var whileLoop = newNodeI(nkWhileStmt, body.info, 2)
whileLoop.sons[0] = newIntTypeNode(nkIntLit, 1, getSysType(tyBool))
whileLoop.sons[0] = newIntTypeNode(nkIntLit, 1, getSysType(g, body.info, tyBool))
whileLoop.sons[1] = loopBody
result.add whileLoop
@ -918,12 +934,23 @@ proc liftForLoop*(body: PNode; owner: PSym): PNode =
addSon(vpart, ast.emptyNode) # no explicit type
if not env.isNil:
call.sons[0] = makeClosure(call.sons[0].sym, env.newSymNode, body.info)
call.sons[0] = makeClosure(g, call.sons[0].sym, env.newSymNode, body.info)
addSon(vpart, call)
addSon(v2, vpart)
loopBody.sons[0] = v2
var bs = newNodeI(nkBreakState, body.info)
bs.addSon(call.sons[0])
loopBody.sons[1] = bs
let ibs = newNode(nkIfStmt)
let elifBranch = newNode(nkElifBranch)
elifBranch.add(bs)
let br = newNode(nkBreakStmt)
br.add(emptyNode)
elifBranch.add(br)
ibs.add(elifBranch)
loopBody.sons[1] = ibs
loopBody.sons[2] = body[L-1]

172
compiler/lexer.nim
View file

@ -17,7 +17,7 @@
import
hashes, options, msgs, strutils, platform, idents, nimlexbase, llstream,
wordrecg
wordrecg, configuration
const
MaxLineLength* = 80 # lines longer than this lead to a warning
@ -60,7 +60,7 @@ type
tkCurlyDotLe, tkCurlyDotRi, # {. and .}
tkParDotLe, tkParDotRi, # (. and .)
tkComma, tkSemiColon,
tkColon, tkColonColon, tkEquals, tkDot, tkDotDot,
tkColon, tkColonColon, tkEquals, tkDot, tkDotDot, tkBracketLeColon,
tkOpr, tkComment, tkAccent,
tkSpaces, tkInfixOpr, tkPrefixOpr, tkPostfixOpr
@ -98,7 +98,7 @@ const
"tkTripleStrLit", "tkGStrLit", "tkGTripleStrLit", "tkCharLit", "(",
")", "[", "]", "{", "}", "[.", ".]", "{.", ".}", "(.", ".)",
",", ";",
":", "::", "=", ".", "..",
":", "::", "=", ".", "..", "[:",
"tkOpr", "tkComment", "`",
"tkSpaces", "tkInfixOpr",
"tkPrefixOpr", "tkPostfixOpr"]
@ -131,9 +131,9 @@ type
# like 0b01 or r"\L" are unaffected
commentOffsetA*, commentOffsetB*: int
TErrorHandler* = proc (info: TLineInfo; msg: TMsgKind; arg: string)
TErrorHandler* = proc (conf: ConfigRef; info: TLineInfo; msg: TMsgKind; arg: string)
TLexer* = object of TBaseLexer
fileIdx*: int32
fileIdx*: FileIndex
indentAhead*: int # if > 0 an indendation has already been read
# this is needed because scanning comments
# needs so much look-ahead
@ -144,13 +144,12 @@ type
cache*: IdentCache
when defined(nimsuggest):
previousToken: TLineInfo
config*: ConfigRef
when defined(nimpretty):
var
gIndentationWidth*: int
var gLinesCompiled*: int # all lines that have been compiled
proc getLineInfo*(L: TLexer, tok: TToken): TLineInfo {.inline.} =
result = newLineInfo(L.fileIdx, tok.line, tok.col)
when defined(nimpretty):
@ -165,13 +164,12 @@ proc isKeyword*(kind: TTokType): bool =
template ones(n): untyped = ((1 shl n)-1) # for utf-8 conversion
proc isNimIdentifier*(s: string): bool =
if s[0] in SymStartChars:
let sLen = s.len
if sLen > 0 and s[0] in SymStartChars:
var i = 1
var sLen = s.len
while i < sLen:
if s[i] == '_':
inc(i)
if s[i] notin SymChars: return
if s[i] == '_': inc(i)
if i < sLen and s[i] notin SymChars: return
inc(i)
result = true
@ -192,8 +190,8 @@ proc prettyTok*(tok: TToken): string =
if isKeyword(tok.tokType): result = "keyword " & tok.ident.s
else: result = tokToStr(tok)
proc printTok*(tok: TToken) =
msgWriteln($tok.line & ":" & $tok.col & "\t" &
proc printTok*(conf: ConfigRef; tok: TToken) =
msgWriteln(conf, $tok.line & ":" & $tok.col & "\t" &
TokTypeToStr[tok.tokType] & " " & tokToStr(tok))
proc initToken*(L: var TToken) =
@ -222,8 +220,8 @@ proc fillToken(L: var TToken) =
L.commentOffsetA = 0
L.commentOffsetB = 0
proc openLexer*(lex: var TLexer, fileIdx: int32, inputstream: PLLStream;
cache: IdentCache) =
proc openLexer*(lex: var TLexer, fileIdx: FileIndex, inputstream: PLLStream;
cache: IdentCache; config: ConfigRef) =
openBaseLexer(lex, inputstream)
lex.fileIdx = fileidx
lex.indentAhead = - 1
@ -232,13 +230,15 @@ proc openLexer*(lex: var TLexer, fileIdx: int32, inputstream: PLLStream;
lex.cache = cache
when defined(nimsuggest):
lex.previousToken.fileIndex = fileIdx
lex.config = config
proc openLexer*(lex: var TLexer, filename: string, inputstream: PLLStream;
cache: IdentCache) =
openLexer(lex, filename.fileInfoIdx, inputstream, cache)
cache: IdentCache; config: ConfigRef) =
openLexer(lex, fileInfoIdx(config, filename), inputstream, cache, config)
proc closeLexer*(lex: var TLexer) =
inc(gLinesCompiled, lex.lineNumber)
if lex.config != nil:
inc(lex.config.linesCompiled, lex.lineNumber)
closeBaseLexer(lex)
proc getLineInfo(L: TLexer): TLineInfo =
@ -246,9 +246,9 @@ proc getLineInfo(L: TLexer): TLineInfo =
proc dispMessage(L: TLexer; info: TLineInfo; msg: TMsgKind; arg: string) =
if L.errorHandler.isNil:
msgs.message(info, msg, arg)
msgs.message(L.config, info, msg, arg)
else:
L.errorHandler(info, msg, arg)
L.errorHandler(L.config, info, msg, arg)
proc lexMessage*(L: TLexer, msg: TMsgKind, arg = "") =
L.dispMessage(getLineInfo(L), msg, arg)
@ -274,7 +274,7 @@ template tokenEnd(tok, pos) {.dirty.} =
when defined(nimsuggest):
let colB = getColNumber(L, pos)+1
if L.fileIdx == gTrackPos.fileIndex and gTrackPos.col in colA..colB and
L.lineNumber == gTrackPos.line and gIdeCmd in {ideSug, ideCon}:
L.lineNumber == gTrackPos.line.int and L.config.ideCmd in {ideSug, ideCon}:
L.cursor = CursorPosition.InToken
gTrackPos.col = colA.int16
colA = 0
@ -285,9 +285,9 @@ template tokenEndIgnore(tok, pos) =
when defined(nimsuggest):
let colB = getColNumber(L, pos)
if L.fileIdx == gTrackPos.fileIndex and gTrackPos.col in colA..colB and
L.lineNumber == gTrackPos.line and gIdeCmd in {ideSug, ideCon}:
L.lineNumber == gTrackPos.line.int and L.config.ideCmd in {ideSug, ideCon}:
gTrackPos.fileIndex = trackPosInvalidFileIdx
gTrackPos.line = -1
gTrackPos.line = 0'u16
colA = 0
when defined(nimpretty):
tok.offsetB = L.offsetBase + pos
@ -299,7 +299,7 @@ template tokenEndPrevious(tok, pos) =
# the cursor in a string literal or comment:
let colB = getColNumber(L, pos)
if L.fileIdx == gTrackPos.fileIndex and gTrackPos.col in colA..colB and
L.lineNumber == gTrackPos.line and gIdeCmd in {ideSug, ideCon}:
L.lineNumber == gTrackPos.line.int and L.config.ideCmd in {ideSug, ideCon}:
L.cursor = CursorPosition.BeforeToken
gTrackPos = L.previousToken
gTrackPosAttached = true
@ -311,12 +311,12 @@ template tokenEndPrevious(tok, pos) =
# We need to parse the largest uint literal without overflow checks
proc unsafeParseUInt(s: string, b: var BiggestInt, start = 0): int =
var i = start
if s[i] in {'0'..'9'}:
if i < s.len and s[i] in {'0'..'9'}:
b = 0
while s[i] in {'0'..'9'}:
while i < s.len and s[i] in {'0'..'9'}:
b = b * 10 + (ord(s[i]) - ord('0'))
inc(i)
while s[i] == '_': inc(i) # underscores are allowed and ignored
while i < s.len and s[i] == '_': inc(i) # underscores are allowed and ignored
result = i - start
{.pop.} # overflowChecks
@ -341,7 +341,8 @@ proc getNumber(L: var TLexer, result: var TToken) =
break
if buf[pos] == '_':
if buf[pos+1] notin chars:
lexMessage(L, errInvalidToken, "_")
lexMessage(L, errGenerated,
"only single underscores may occur in a token: '__' is invalid")
break
add(tok.literal, '_')
inc(pos)
@ -355,7 +356,7 @@ proc getNumber(L: var TLexer, result: var TToken) =
inc(pos)
L.bufpos = pos
proc lexMessageLitNum(L: var TLexer, msg: TMsgKind, startpos: int) =
proc lexMessageLitNum(L: var TLexer, msg: string, startpos: int) =
# Used to get slightly human friendlier err messages.
# Note: the erroneous 'O' char in the character set is intentional
const literalishChars = {'A'..'F', 'a'..'f', '0'..'9', 'X', 'x', 'o', 'O',
@ -376,7 +377,7 @@ proc getNumber(L: var TLexer, result: var TToken) =
add(t.literal, L.buf[L.bufpos])
matchChars(L, t, {'0'..'9'})
L.bufpos = msgPos
lexMessage(L, msg, t.literal)
lexMessage(L, errGenerated, msg % t.literal)
var
startpos, endpos: int
@ -398,7 +399,7 @@ proc getNumber(L: var TLexer, result: var TToken) =
eatChar(L, result, '0')
case L.buf[L.bufpos]
of 'O':
lexMessageLitNum(L, errInvalidNumberOctalCode, startpos)
lexMessageLitNum(L, "$1 is not a valid number; did you mean octal? Then use one of '0o', '0c' or '0C'.", startpos)
of 'x', 'X':
eatChar(L, result, 'x')
matchUnderscoreChars(L, result, {'0'..'9', 'a'..'f', 'A'..'F'})
@ -409,7 +410,7 @@ proc getNumber(L: var TLexer, result: var TToken) =
eatChar(L, result, 'b')
matchUnderscoreChars(L, result, {'0'..'1'})
else:
internalError(getLineInfo(L), "getNumber")
internalError(L.config, getLineInfo(L), "getNumber")
else:
matchUnderscoreChars(L, result, {'0'..'9'})
if (L.buf[L.bufpos] == '.') and (L.buf[L.bufpos + 1] in {'0'..'9'}):
@ -464,7 +465,7 @@ proc getNumber(L: var TLexer, result: var TToken) =
result.tokType = tkInt8Lit
inc(postPos)
else:
lexMessageLitNum(L, errInvalidNumber, startpos)
lexMessageLitNum(L, "invalid number: '$1'", startpos)
of 'u', 'U':
inc(postPos)
if (L.buf[postPos] == '6') and (L.buf[postPos + 1] == '4'):
@ -482,12 +483,12 @@ proc getNumber(L: var TLexer, result: var TToken) =
else:
result.tokType = tkUIntLit
else:
lexMessageLitNum(L, errInvalidNumber, startpos)
lexMessageLitNum(L, "invalid number: '$1'", startpos)
# Is there still a literalish char awaiting? Then it's an error!
if L.buf[postPos] in literalishChars or
(L.buf[postPos] == '.' and L.buf[postPos + 1] in {'0'..'9'}):
lexMessageLitNum(L, errInvalidNumber, startpos)
lexMessageLitNum(L, "invalid number: '$1'", startpos)
# Third stage, extract actual number
L.bufpos = startpos # restore position
@ -528,7 +529,7 @@ proc getNumber(L: var TLexer, result: var TToken) =
else:
break
else:
internalError(getLineInfo(L), "getNumber")
internalError(L.config, getLineInfo(L), "getNumber")
case result.tokType
of tkIntLit, tkInt64Lit: result.iNumber = xi
@ -545,7 +546,7 @@ proc getNumber(L: var TLexer, result: var TToken) =
# XXX: Test this on big endian machine!
of tkFloat64Lit, tkFloatLit:
result.fNumber = (cast[PFloat64](addr(xi)))[]
else: internalError(getLineInfo(L), "getNumber")
else: internalError(L.config, getLineInfo(L), "getNumber")
# Bounds checks. Non decimal literals are allowed to overflow the range of
# the datatype as long as their pattern don't overflow _bitwise_, hence
@ -561,7 +562,7 @@ proc getNumber(L: var TLexer, result: var TToken) =
if outOfRange:
#echo "out of range num: ", result.iNumber, " vs ", xi
lexMessageLitNum(L, errNumberOutOfRange, startpos)
lexMessageLitNum(L, "number out of range: '$1'", startpos)
else:
case result.tokType
@ -577,19 +578,20 @@ proc getNumber(L: var TLexer, result: var TToken) =
result.iNumber = parseBiggestInt(result.literal)
# Explicit bounds checks
let outOfRange = case result.tokType:
of tkInt8Lit: (result.iNumber < int8.low or result.iNumber > int8.high)
of tkUInt8Lit: (result.iNumber < BiggestInt(uint8.low) or
result.iNumber > BiggestInt(uint8.high))
of tkInt16Lit: (result.iNumber < int16.low or result.iNumber > int16.high)
of tkUInt16Lit: (result.iNumber < BiggestInt(uint16.low) or
result.iNumber > BiggestInt(uint16.high))
of tkInt32Lit: (result.iNumber < int32.low or result.iNumber > int32.high)
of tkUInt32Lit: (result.iNumber < BiggestInt(uint32.low) or
result.iNumber > BiggestInt(uint32.high))
else: false
let outOfRange =
case result.tokType
of tkInt8Lit: (result.iNumber < int8.low or result.iNumber > int8.high)
of tkUInt8Lit: (result.iNumber < BiggestInt(uint8.low) or
result.iNumber > BiggestInt(uint8.high))
of tkInt16Lit: (result.iNumber < int16.low or result.iNumber > int16.high)
of tkUInt16Lit: (result.iNumber < BiggestInt(uint16.low) or
result.iNumber > BiggestInt(uint16.high))
of tkInt32Lit: (result.iNumber < int32.low or result.iNumber > int32.high)
of tkUInt32Lit: (result.iNumber < BiggestInt(uint32.low) or
result.iNumber > BiggestInt(uint32.high))
else: false
if outOfRange: lexMessageLitNum(L, errNumberOutOfRange, startpos)
if outOfRange: lexMessageLitNum(L, "number out of range: '$1'", startpos)
# Promote int literal to int64? Not always necessary, but more consistent
if result.tokType == tkIntLit:
@ -597,9 +599,9 @@ proc getNumber(L: var TLexer, result: var TToken) =
result.tokType = tkInt64Lit
except ValueError:
lexMessageLitNum(L, errInvalidNumber, startpos)
lexMessageLitNum(L, "invalid number: '$1'", startpos)
except OverflowError, RangeError:
lexMessageLitNum(L, errNumberOutOfRange, startpos)
lexMessageLitNum(L, "number out of range: '$1'", startpos)
tokenEnd(result, postPos-1)
L.bufpos = postPos
@ -625,7 +627,16 @@ proc getEscapedChar(L: var TLexer, tok: var TToken) =
inc(L.bufpos) # skip '\'
case L.buf[L.bufpos]
of 'n', 'N':
if tok.tokType == tkCharLit: lexMessage(L, errNnotAllowedInCharacter)
if L.config.oldNewlines:
if tok.tokType == tkCharLit:
lexMessage(L, errGenerated, "\\n not allowed in character literal")
add(tok.literal, tnl)
else:
add(tok.literal, '\L')
inc(L.bufpos)
of 'p', 'P':
if tok.tokType == tkCharLit:
lexMessage(L, errGenerated, "\\p not allowed in character literal")
add(tok.literal, tnl)
inc(L.bufpos)
of 'r', 'R', 'c', 'C':
@ -687,8 +698,8 @@ proc getEscapedChar(L: var TLexer, tok: var TToken) =
var xi = 0
handleDecChars(L, xi)
if (xi <= 255): add(tok.literal, chr(xi))
else: lexMessage(L, errInvalidCharacterConstant)
else: lexMessage(L, errInvalidCharacterConstant)
else: lexMessage(L, errGenerated, "invalid character constant")
else: lexMessage(L, errGenerated, "invalid character constant")
proc newString(s: cstring, len: int): string =
## XXX, how come there is no support for this?
@ -703,7 +714,7 @@ proc handleCRLF(L: var TLexer, pos: int): int =
if col > MaxLineLength:
lexMessagePos(L, hintLineTooLong, pos)
if optEmbedOrigSrc in gGlobalOptions:
if optEmbedOrigSrc in L.config.globalOptions:
let lineStart = cast[ByteAddress](L.buf) + L.lineStart
let line = newString(cast[cstring](lineStart), col)
addSourceLine(L.fileIdx, line)
@ -747,12 +758,12 @@ proc getString(L: var TLexer, tok: var TToken, rawMode: bool) =
tokenEndIgnore(tok, pos)
pos = handleCRLF(L, pos)
buf = L.buf
add(tok.literal, tnl)
add(tok.literal, "\n")
of nimlexbase.EndOfFile:
tokenEndIgnore(tok, pos)
var line2 = L.lineNumber
L.lineNumber = line
lexMessagePos(L, errClosingTripleQuoteExpected, L.lineStart)
lexMessagePos(L, errGenerated, L.lineStart, "closing \"\"\" expected, but end of file reached")
L.lineNumber = line2
L.bufpos = pos
break
@ -775,7 +786,7 @@ proc getString(L: var TLexer, tok: var TToken, rawMode: bool) =
break
elif c in {CR, LF, nimlexbase.EndOfFile}:
tokenEndIgnore(tok, pos)
lexMessage(L, errClosingQuoteExpected)
lexMessage(L, errGenerated, "closing \" expected")
break
elif (c == '\\') and not rawMode:
L.bufpos = pos
@ -791,12 +802,13 @@ proc getCharacter(L: var TLexer, tok: var TToken) =
inc(L.bufpos) # skip '
var c = L.buf[L.bufpos]
case c
of '\0'..pred(' '), '\'': lexMessage(L, errInvalidCharacterConstant)
of '\0'..pred(' '), '\'': lexMessage(L, errGenerated, "invalid character literal")
of '\\': getEscapedChar(L, tok)
else:
tok.literal = $c
inc(L.bufpos)
if L.buf[L.bufpos] != '\'': lexMessage(L, errMissingFinalQuote)
if L.buf[L.bufpos] != '\'':
lexMessage(L, errGenerated, "missing closing ' for character literal")
tokenEndIgnore(tok, L.bufpos)
inc(L.bufpos) # skip '
@ -817,7 +829,7 @@ proc getSymbol(L: var TLexer, tok: var TToken) =
inc(pos)
of '_':
if buf[pos+1] notin SymChars:
lexMessage(L, errInvalidToken, "_")
lexMessage(L, errGenerated, "invalid token: trailing underscore")
break
inc(pos)
else: break
@ -995,13 +1007,17 @@ proc skip(L: var TLexer, tok: var TToken) =
var buf = L.buf
tokenBegin(tok, pos)
tok.strongSpaceA = 0
when defined(nimpretty):
var hasComment = false
tok.commentOffsetA = L.offsetBase + pos
tok.commentOffsetB = tok.commentOffsetA
while true:
case buf[pos]
of ' ':
inc(pos)
inc(tok.strongSpaceA)
of '\t':
if not L.allowTabs: lexMessagePos(L, errTabulatorsAreNotAllowed, pos)
if not L.allowTabs: lexMessagePos(L, errGenerated, pos, "tabulators are not allowed")
inc(pos)
of CR, LF:
tokenEndPrevious(tok, pos)
@ -1013,6 +1029,7 @@ proc skip(L: var TLexer, tok: var TToken) =
inc(pos)
inc(indent)
elif buf[pos] == '#' and buf[pos+1] == '[':
when defined(nimpretty): hasComment = true
skipMultiLineComment(L, tok, pos+2, false)
pos = L.bufpos
buf = L.buf
@ -1026,14 +1043,11 @@ proc skip(L: var TLexer, tok: var TToken) =
of '#':
# do not skip documentation comment:
if buf[pos+1] == '#': break
when defined(nimpretty):
tok.commentOffsetA = L.offsetBase + pos
when defined(nimpretty): hasComment = true
if buf[pos+1] == '[':
skipMultiLineComment(L, tok, pos+2, false)
pos = L.bufpos
buf = L.buf
when defined(nimpretty):
tok.commentOffsetB = L.offsetBase + pos
else:
tokenBegin(tok, pos)
while buf[pos] notin {CR, LF, nimlexbase.EndOfFile}: inc(pos)
@ -1045,6 +1059,9 @@ proc skip(L: var TLexer, tok: var TToken) =
tokenEndPrevious(tok, pos-1)
L.bufpos = pos
when defined(nimpretty):
if hasComment:
tok.commentOffsetB = L.offsetBase + pos
tok.tokType = tkComment
if gIndentationWidth <= 0:
gIndentationWidth = tok.indent
@ -1053,7 +1070,7 @@ proc rawGetTok*(L: var TLexer, tok: var TToken) =
when defined(nimsuggest):
# we attach the cursor to the last *strong* token
if tok.tokType notin weakTokens:
L.previousToken.line = tok.line.int16
L.previousToken.line = tok.line.uint16
L.previousToken.col = tok.col.int16
when defined(nimsuggest):
@ -1066,6 +1083,10 @@ proc rawGetTok*(L: var TLexer, tok: var TToken) =
else:
tok.indent = -1
skip(L, tok)
when defined(nimpretty):
if tok.tokType == tkComment:
L.indentAhead = L.currLineIndent
return
var c = L.buf[L.bufpos]
tok.line = L.lineNumber
tok.col = getColNumber(L, L.bufpos)
@ -1101,7 +1122,7 @@ proc rawGetTok*(L: var TLexer, tok: var TToken) =
tok.tokType = tkParLe
when defined(nimsuggest):
if L.fileIdx == gTrackPos.fileIndex and tok.col < gTrackPos.col and
tok.line == gTrackPos.line and gIdeCmd == ideCon:
tok.line == gTrackPos.line.int and L.config.ideCmd == ideCon:
gTrackPos.col = tok.col.int16
of ')':
tok.tokType = tkParRi
@ -1111,6 +1132,9 @@ proc rawGetTok*(L: var TLexer, tok: var TToken) =
if L.buf[L.bufpos] == '.' and L.buf[L.bufpos+1] != '.':
tok.tokType = tkBracketDotLe
inc(L.bufpos)
elif L.buf[L.bufpos] == ':':
tok.tokType = tkBracketLeColon
inc(L.bufpos)
else:
tok.tokType = tkBracketLe
of ']':
@ -1119,7 +1143,7 @@ proc rawGetTok*(L: var TLexer, tok: var TToken) =
of '.':
when defined(nimsuggest):
if L.fileIdx == gTrackPos.fileIndex and tok.col+1 == gTrackPos.col and
tok.line == gTrackPos.line and gIdeCmd == ideSug:
tok.line == gTrackPos.line.int and L.config.ideCmd == ideSug:
tok.tokType = tkDot
L.cursor = CursorPosition.InToken
gTrackPos.col = tok.col.int16
@ -1161,7 +1185,7 @@ proc rawGetTok*(L: var TLexer, tok: var TToken) =
else:
tok.literal = $c
tok.tokType = tkInvalid
lexMessage(L, errInvalidToken, c & " (\\" & $(ord(c)) & ')')
lexMessage(L, errGenerated, "invalid token: " & c & " (\\" & $(ord(c)) & ')')
of '\"':
# check for extended raw string literal:
var rawMode = L.bufpos > 0 and L.buf[L.bufpos-1] in SymChars
@ -1178,7 +1202,7 @@ proc rawGetTok*(L: var TLexer, tok: var TToken) =
getNumber(L, tok)
let c = L.buf[L.bufpos]
if c in SymChars+{'_'}:
lexMessage(L, errInvalidToken, c & " (\\" & $(ord(c)) & ')')
lexMessage(L, errGenerated, "invalid token: no whitespace between number and identifier")
else:
if c in OpChars:
getOperator(L, tok)
@ -1188,6 +1212,6 @@ proc rawGetTok*(L: var TLexer, tok: var TToken) =
else:
tok.literal = $c
tok.tokType = tkInvalid
lexMessage(L, errInvalidToken, c & " (\\" & $(ord(c)) & ')')
lexMessage(L, errGenerated, "invalid token: " & c & " (\\" & $(ord(c)) & ')')
inc(L.bufpos)
atTokenEnd()

6
compiler/liftlocals.nim
View file

@ -52,17 +52,17 @@ proc lookupParam(params, dest: PNode): PSym =
if params[i].kind == nkSym and params[i].sym.name.id == dest.ident.id:
return params[i].sym
proc liftLocalsIfRequested*(prc: PSym; n: PNode): PNode =
proc liftLocalsIfRequested*(prc: PSym; n: PNode; conf: ConfigRef): PNode =
let liftDest = getPragmaVal(prc.ast, wLiftLocals)
if liftDest == nil: return n
let partialParam = lookupParam(prc.typ.n, liftDest)
if partialParam.isNil:
localError(liftDest.info, "'$1' is not a parameter of '$2'" %
localError(conf, liftDest.info, "'$1' is not a parameter of '$2'" %
[$liftDest, prc.name.s])
return n
let objType = partialParam.typ.skipTypes(abstractPtrs)
if objType.kind != tyObject or tfPartial notin objType.flags:
localError(liftDest.info, "parameter '$1' is not a pointer to a partial object" % $liftDest)
localError(conf, liftDest.info, "parameter '$1' is not a pointer to a partial object" % $liftDest)
return n
var c = Ctx(partialParam: partialParam, objType: objType)
let w = newTree(nkStmtList, n)

120
compiler/lookups.nim
View file

@ -11,23 +11,23 @@
import
intsets, ast, astalgo, idents, semdata, types, msgs, options, rodread,
renderer, wordrecg, idgen, nimfix.prettybase
renderer, wordrecg, idgen, nimfix.prettybase, configuration, strutils
proc ensureNoMissingOrUnusedSymbols(scope: PScope)
proc ensureNoMissingOrUnusedSymbols(c: PContext; scope: PScope)
proc noidentError(n, origin: PNode) =
proc noidentError(conf: ConfigRef; n, origin: PNode) =
var m = ""
if origin != nil:
m.add "in expression '" & origin.renderTree & "': "
m.add "identifier expected, but found '" & n.renderTree & "'"
localError(n.info, m)
localError(conf, n.info, m)
proc considerQuotedIdent*(n: PNode, origin: PNode = nil): PIdent =
proc considerQuotedIdent*(conf: ConfigRef; n: PNode, origin: PNode = nil): PIdent =
## Retrieve a PIdent from a PNode, taking into account accent nodes.
## ``origin`` can be nil. If it is not nil, it is used for a better
## error message.
template handleError(n, origin: PNode) =
noidentError(n, origin)
noidentError(conf, n, origin)
result = getIdent"<Error>"
case n.kind
@ -36,7 +36,7 @@ proc considerQuotedIdent*(n: PNode, origin: PNode = nil): PIdent =
of nkAccQuoted:
case n.len
of 0: handleError(n, origin)
of 1: result = considerQuotedIdent(n.sons[0], origin)
of 1: result = considerQuotedIdent(conf, n.sons[0], origin)
else:
var id = ""
for i in 0..<n.len:
@ -44,6 +44,7 @@ proc considerQuotedIdent*(n: PNode, origin: PNode = nil): PIdent =
case x.kind
of nkIdent: id.add(x.ident.s)
of nkSym: id.add(x.sym.name.s)
of nkLiterals - nkFloatLiterals: id.add(x.renderTree)
else: handleError(n, origin)
result = getIdent(id)
of nkOpenSymChoice, nkClosedSymChoice:
@ -70,7 +71,7 @@ proc rawCloseScope*(c: PContext) =
c.currentScope = c.currentScope.parent
proc closeScope*(c: PContext) =
ensureNoMissingOrUnusedSymbols(c.currentScope)
ensureNoMissingOrUnusedSymbols(c, c.currentScope)
rawCloseScope(c)
iterator walkScopes*(scope: PScope): PScope =
@ -79,15 +80,15 @@ iterator walkScopes*(scope: PScope): PScope =
yield current
current = current.parent
proc skipAlias*(s: PSym; n: PNode): PSym =
proc skipAlias*(s: PSym; n: PNode; conf: ConfigRef): PSym =
if s == nil or s.kind != skAlias:
result = s
else:
result = s.owner
if gCmd == cmdPretty:
if conf.cmd == cmdPretty:
prettybase.replaceDeprecated(n.info, s, result)
else:
message(n.info, warnDeprecated, "use " & result.name.s & " instead; " &
message(conf, n.info, warnDeprecated, "use " & result.name.s & " instead; " &
s.name.s)
proc localSearchInScope*(c: PContext, s: PIdent): PSym =
@ -124,14 +125,14 @@ proc errorSym*(c: PContext, n: PNode): PSym =
# ensure that 'considerQuotedIdent' can't fail:
if m.kind == nkDotExpr: m = m.sons[1]
let ident = if m.kind in {nkIdent, nkSym, nkAccQuoted}:
considerQuotedIdent(m)
considerQuotedIdent(c.config, m)
else:
getIdent("err:" & renderTree(m))
result = newSym(skError, ident, getCurrOwner(c), n.info)
result = newSym(skError, ident, getCurrOwner(c), n.info, {})
result.typ = errorType(c)
incl(result.flags, sfDiscardable)
# pretend it's imported from some unknown module to prevent cascading errors:
if gCmd != cmdInteractive and c.compilesContextId == 0:
if c.config.cmd != cmdInteractive and c.compilesContextId == 0:
c.importTable.addSym(result)
type
@ -153,7 +154,7 @@ proc getSymRepr*(s: PSym): string =
else:
result = s.name.s
proc ensureNoMissingOrUnusedSymbols(scope: PScope) =
proc ensureNoMissingOrUnusedSymbols(c: PContext; scope: PScope) =
# check if all symbols have been used and defined:
var it: TTabIter
var s = initTabIter(it, scope.symbols)
@ -163,54 +164,53 @@ proc ensureNoMissingOrUnusedSymbols(scope: PScope) =
# too many 'implementation of X' errors are annoying
# and slow 'suggest' down:
if missingImpls == 0:
localError(s.info, errImplOfXexpected, getSymRepr(s))
localError(c.config, s.info, "implementation of '$1' expected" % getSymRepr(s))
inc missingImpls
elif {sfUsed, sfExported} * s.flags == {} and optHints in s.options:
# BUGFIX: check options in s!
if s.kind notin {skForVar, skParam, skMethod, skUnknown, skGenericParam}:
# XXX: implicit type params are currently skTypes
# maybe they can be made skGenericParam as well.
if s.typ != nil and tfImplicitTypeParam notin s.typ.flags and
s.typ.kind != tyGenericParam:
message(s.info, hintXDeclaredButNotUsed, getSymRepr(s))
message(c.config, s.info, hintXDeclaredButNotUsed, getSymRepr(s))
s = nextIter(it, scope.symbols)
proc wrongRedefinition*(info: TLineInfo, s: string) =
if gCmd != cmdInteractive:
localError(info, errAttemptToRedefine, s)
proc wrongRedefinition*(c: PContext; info: TLineInfo, s: string) =
if c.config.cmd != cmdInteractive:
localError(c.config, info, "redefinition of '$1'" % s)
proc addDecl*(c: PContext, sym: PSym, info: TLineInfo) =
if not c.currentScope.addUniqueSym(sym):
wrongRedefinition(info, sym.name.s)
wrongRedefinition(c, info, sym.name.s)
proc addDecl*(c: PContext, sym: PSym) =
if not c.currentScope.addUniqueSym(sym):
wrongRedefinition(sym.info, sym.name.s)
wrongRedefinition(c, sym.info, sym.name.s)
proc addPrelimDecl*(c: PContext, sym: PSym) =
discard c.currentScope.addUniqueSym(sym)
proc addDeclAt*(scope: PScope, sym: PSym) =
proc addDeclAt*(c: PContext; scope: PScope, sym: PSym) =
if not scope.addUniqueSym(sym):
wrongRedefinition(sym.info, sym.name.s)
wrongRedefinition(c, sym.info, sym.name.s)
proc addInterfaceDeclAux(c: PContext, sym: PSym) =
if sfExported in sym.flags:
# add to interface:
if c.module != nil: strTableAdd(c.module.tab, sym)
else: internalError(sym.info, "addInterfaceDeclAux")
else: internalError(c.config, sym.info, "addInterfaceDeclAux")
proc addInterfaceDeclAt*(c: PContext, scope: PScope, sym: PSym) =
addDeclAt(scope, sym)
addDeclAt(c, scope, sym)
addInterfaceDeclAux(c, sym)
proc addOverloadableSymAt*(scope: PScope, fn: PSym) =
proc addOverloadableSymAt*(c: PContext; scope: PScope, fn: PSym) =
if fn.kind notin OverloadableSyms:
internalError(fn.info, "addOverloadableSymAt")
internalError(c.config, fn.info, "addOverloadableSymAt")
return
let check = strTableGet(scope.symbols, fn.name)
if check != nil and check.kind notin OverloadableSyms:
wrongRedefinition(fn.info, fn.name.s)
wrongRedefinition(c, fn.info, fn.name.s)
else:
scope.addSym(fn)
@ -221,12 +221,10 @@ proc addInterfaceDecl*(c: PContext, sym: PSym) =
proc addInterfaceOverloadableSymAt*(c: PContext, scope: PScope, sym: PSym) =
# it adds the symbol to the interface if appropriate
addOverloadableSymAt(scope, sym)
addOverloadableSymAt(c, scope, sym)
addInterfaceDeclAux(c, sym)
when defined(nimfix):
import strutils
# when we cannot find the identifier, retry with a changed identifer:
proc altSpelling(x: PIdent): PIdent =
case x.s[0]
@ -254,7 +252,7 @@ proc errorUseQualifier*(c: PContext; info: TLineInfo; s: PSym) =
err.add candidate.owner.name.s & "." & candidate.name.s
candidate = nextIdentIter(ti, c.importTable.symbols)
inc i
localError(info, errGenerated, err)
localError(c.config, info, errGenerated, err)
proc errorUndeclaredIdentifier*(c: PContext; info: TLineInfo; name: string) =
var err = "undeclared identifier: '" & name & "'"
@ -263,13 +261,13 @@ proc errorUndeclaredIdentifier*(c: PContext; info: TLineInfo; name: string) =
err.add c.recursiveDep
# prevent excessive errors for 'nim check'
c.recursiveDep = nil
localError(info, errGenerated, err)
localError(c.config, info, errGenerated, err)
proc lookUp*(c: PContext, n: PNode): PSym =
# Looks up a symbol. Generates an error in case of nil.
case n.kind
of nkIdent:
result = searchInScopes(c, n.ident).skipAlias(n)
result = searchInScopes(c, n.ident).skipAlias(n, c.config)
if result == nil:
fixSpelling(n, n.ident, searchInScopes)
errorUndeclaredIdentifier(c, n.info, n.ident.s)
@ -277,14 +275,14 @@ proc lookUp*(c: PContext, n: PNode): PSym =
of nkSym:
result = n.sym
of nkAccQuoted:
var ident = considerQuotedIdent(n)
result = searchInScopes(c, ident).skipAlias(n)
var ident = considerQuotedIdent(c.config, n)
result = searchInScopes(c, ident).skipAlias(n, c.config)
if result == nil:
fixSpelling(n, ident, searchInScopes)
errorUndeclaredIdentifier(c, n.info, ident.s)
result = errorSym(c, n)
else:
internalError(n.info, "lookUp")
internalError(c.config, n.info, "lookUp")
return
if contains(c.ambiguousSymbols, result.id):
errorUseQualifier(c, n.info, result)
@ -298,11 +296,11 @@ proc qualifiedLookUp*(c: PContext, n: PNode, flags: set[TLookupFlag]): PSym =
const allExceptModule = {low(TSymKind)..high(TSymKind)}-{skModule,skPackage}
case n.kind
of nkIdent, nkAccQuoted:
var ident = considerQuotedIdent(n)
var ident = considerQuotedIdent(c.config, n)
if checkModule in flags:
result = searchInScopes(c, ident).skipAlias(n)
result = searchInScopes(c, ident).skipAlias(n, c.config)
else:
result = searchInScopes(c, ident, allExceptModule).skipAlias(n)
result = searchInScopes(c, ident, allExceptModule).skipAlias(n, c.config)
if result == nil and checkPureEnumFields in flags:
result = strTableGet(c.pureEnumFields, ident)
if result == nil and checkUndeclared in flags:
@ -324,12 +322,12 @@ proc qualifiedLookUp*(c: PContext, n: PNode, flags: set[TLookupFlag]): PSym =
if n.sons[1].kind == nkIdent:
ident = n.sons[1].ident
elif n.sons[1].kind == nkAccQuoted:
ident = considerQuotedIdent(n.sons[1])
ident = considerQuotedIdent(c.config, n.sons[1])
if ident != nil:
if m == c.module:
result = strTableGet(c.topLevelScope.symbols, ident).skipAlias(n)
result = strTableGet(c.topLevelScope.symbols, ident).skipAlias(n, c.config)
else:
result = strTableGet(m.tab, ident).skipAlias(n)
result = strTableGet(m.tab, ident).skipAlias(n, c.config)
if result == nil and checkUndeclared in flags:
fixSpelling(n.sons[1], ident, searchInScopes)
errorUndeclaredIdentifier(c, n.sons[1].info, ident.s)
@ -338,7 +336,7 @@ proc qualifiedLookUp*(c: PContext, n: PNode, flags: set[TLookupFlag]): PSym =
result = n.sons[1].sym
elif checkUndeclared in flags and
n.sons[1].kind notin {nkOpenSymChoice, nkClosedSymChoice}:
localError(n.sons[1].info, errIdentifierExpected,
localError(c.config, n.sons[1].info, "identifier expected, but got: " &
renderTree(n.sons[1]))
result = errorSym(c, n.sons[1])
else:
@ -348,11 +346,11 @@ proc qualifiedLookUp*(c: PContext, n: PNode, flags: set[TLookupFlag]): PSym =
proc initOverloadIter*(o: var TOverloadIter, c: PContext, n: PNode): PSym =
case n.kind
of nkIdent, nkAccQuoted:
var ident = considerQuotedIdent(n)
var ident = considerQuotedIdent(c.config, n)
o.scope = c.currentScope
o.mode = oimNoQualifier
while true:
result = initIdentIter(o.it, o.scope.symbols, ident).skipAlias(n)
result = initIdentIter(o.it, o.scope.symbols, ident).skipAlias(n, c.config)
if result != nil:
break
else:
@ -369,17 +367,17 @@ proc initOverloadIter*(o: var TOverloadIter, c: PContext, n: PNode): PSym =
if n.sons[1].kind == nkIdent:
ident = n.sons[1].ident
elif n.sons[1].kind == nkAccQuoted:
ident = considerQuotedIdent(n.sons[1], n)
ident = considerQuotedIdent(c.config, n.sons[1], n)
if ident != nil:
if o.m == c.module:
# a module may access its private members:
result = initIdentIter(o.it, c.topLevelScope.symbols,
ident).skipAlias(n)
ident).skipAlias(n, c.config)
o.mode = oimSelfModule
else:
result = initIdentIter(o.it, o.m.tab, ident).skipAlias(n)
result = initIdentIter(o.it, o.m.tab, ident).skipAlias(n, c.config)
else:
noidentError(n.sons[1], n)
noidentError(c.config, n.sons[1], n)
result = errorSym(c, n.sons[1])
of nkClosedSymChoice, nkOpenSymChoice:
o.mode = oimSymChoice
@ -407,18 +405,18 @@ proc nextOverloadIter*(o: var TOverloadIter, c: PContext, n: PNode): PSym =
result = nil
of oimNoQualifier:
if o.scope != nil:
result = nextIdentIter(o.it, o.scope.symbols).skipAlias(n)
result = nextIdentIter(o.it, o.scope.symbols).skipAlias(n, c.config)
while result == nil:
o.scope = o.scope.parent
if o.scope == nil: break
result = initIdentIter(o.it, o.scope.symbols, o.it.name).skipAlias(n)
result = initIdentIter(o.it, o.scope.symbols, o.it.name).skipAlias(n, c.config)
# BUGFIX: o.it.name <-> n.ident
else:
result = nil
of oimSelfModule:
result = nextIdentIter(o.it, c.topLevelScope.symbols).skipAlias(n)
result = nextIdentIter(o.it, c.topLevelScope.symbols).skipAlias(n, c.config)
of oimOtherModule:
result = nextIdentIter(o.it, o.m.tab).skipAlias(n)
result = nextIdentIter(o.it, o.m.tab).skipAlias(n, c.config)
of oimSymChoice:
if o.symChoiceIndex < sonsLen(n):
result = n.sons[o.symChoiceIndex].sym
@ -429,19 +427,19 @@ proc nextOverloadIter*(o: var TOverloadIter, c: PContext, n: PNode): PSym =
o.mode = oimSymChoiceLocalLookup
o.scope = c.currentScope
result = firstIdentExcluding(o.it, o.scope.symbols,
n.sons[0].sym.name, o.inSymChoice).skipAlias(n)
n.sons[0].sym.name, o.inSymChoice).skipAlias(n, c.config)
while result == nil:
o.scope = o.scope.parent
if o.scope == nil: break
result = firstIdentExcluding(o.it, o.scope.symbols,
n.sons[0].sym.name, o.inSymChoice).skipAlias(n)
n.sons[0].sym.name, o.inSymChoice).skipAlias(n, c.config)
of oimSymChoiceLocalLookup:
result = nextIdentExcluding(o.it, o.scope.symbols, o.inSymChoice).skipAlias(n)
result = nextIdentExcluding(o.it, o.scope.symbols, o.inSymChoice).skipAlias(n, c.config)
while result == nil:
o.scope = o.scope.parent
if o.scope == nil: break
result = firstIdentExcluding(o.it, o.scope.symbols,
n.sons[0].sym.name, o.inSymChoice).skipAlias(n)
n.sons[0].sym.name, o.inSymChoice).skipAlias(n, c.config)
if result != nil and result.kind == skStub: loadStub(result)
@ -455,5 +453,3 @@ proc pickSym*(c: PContext, n: PNode; kinds: set[TSymKind];
else: return nil # ambiguous
a = nextOverloadIter(o, c, n)
proc isInfixAs*(n: PNode): bool =
return n.kind == nkInfix and considerQuotedIdent(n[0]).s == "as"

172
compiler/lowerings.nim
View file

@ -12,18 +12,18 @@
const
genPrefix* = ":tmp" # prefix for generated names
import ast, astalgo, types, idents, magicsys, msgs, options
import ast, astalgo, types, idents, magicsys, msgs, options, modulegraphs
from trees import getMagic
proc newDeref*(n: PNode): PNode {.inline.} =
result = newNodeIT(nkHiddenDeref, n.info, n.typ.sons[0])
addSon(result, n)
proc newTupleAccess*(tup: PNode, i: int): PNode =
proc newTupleAccess*(g: ModuleGraph; tup: PNode, i: int): PNode =
result = newNodeIT(nkBracketExpr, tup.info, tup.typ.skipTypes(
abstractInst).sons[i])
addSon(result, copyTree(tup))
var lit = newNodeIT(nkIntLit, tup.info, getSysType(tyInt))
var lit = newNodeIT(nkIntLit, tup.info, getSysType(g, tup.info, tyInt))
lit.intVal = i
addSon(result, lit)
@ -44,12 +44,12 @@ proc newFastAsgnStmt(le, ri: PNode): PNode =
result.sons[0] = le
result.sons[1] = ri
proc lowerTupleUnpacking*(n: PNode; owner: PSym): PNode =
proc lowerTupleUnpacking*(g: ModuleGraph; n: PNode; owner: PSym): PNode =
assert n.kind == nkVarTuple
let value = n.lastSon
result = newNodeI(nkStmtList, n.info)
var temp = newSym(skTemp, getIdent(genPrefix), owner, value.info)
var temp = newSym(skTemp, getIdent(genPrefix), owner, value.info, g.config.options)
temp.typ = skipTypes(value.typ, abstractInst)
incl(temp.flags, sfFromGeneric)
@ -61,7 +61,7 @@ proc lowerTupleUnpacking*(n: PNode; owner: PSym): PNode =
result.add newAsgnStmt(tempAsNode, value)
for i in 0 .. n.len-3:
if n.sons[i].kind == nkSym: v.addVar(n.sons[i])
result.add newAsgnStmt(n.sons[i], newTupleAccess(tempAsNode, i))
result.add newAsgnStmt(n.sons[i], newTupleAccess(g, tempAsNode, i))
proc newTupleAccessRaw*(tup: PNode, i: int): PNode =
result = newNodeI(nkBracketExpr, tup.info)
@ -77,7 +77,7 @@ proc lowerTupleUnpackingForAsgn*(n: PNode; owner: PSym): PNode =
let value = n.lastSon
result = newNodeI(nkStmtList, n.info)
var temp = newSym(skLet, getIdent("_"), owner, value.info)
var temp = newSym(skLet, getIdent("_"), owner, value.info, owner.options)
var v = newNodeI(nkLetSection, value.info)
let tempAsNode = newSymNode(temp) #newIdentNode(getIdent(genPrefix & $temp.id), value.info)
@ -95,7 +95,7 @@ proc lowerTupleUnpackingForAsgn*(n: PNode; owner: PSym): PNode =
proc lowerSwap*(n: PNode; owner: PSym): PNode =
result = newNodeI(nkStmtList, n.info)
# note: cannot use 'skTemp' here cause we really need the copy for the VM :-(
var temp = newSym(skVar, getIdent(genPrefix), owner, n.info)
var temp = newSym(skVar, getIdent(genPrefix), owner, n.info, owner.options)
temp.typ = n.sons[1].typ
incl(temp.flags, sfFromGeneric)
@ -112,16 +112,16 @@ proc lowerSwap*(n: PNode; owner: PSym): PNode =
result.add newFastAsgnStmt(n[1], n[2])
result.add newFastAsgnStmt(n[2], tempAsNode)
proc createObj*(owner: PSym, info: TLineInfo; final=true): PType =
proc createObj*(g: ModuleGraph; owner: PSym, info: TLineInfo; final=true): PType =
result = newType(tyObject, owner)
if final:
rawAddSon(result, nil)
incl result.flags, tfFinal
else:
rawAddSon(result, getCompilerProc("RootObj").typ)
rawAddSon(result, getCompilerProc(g, "RootObj").typ)
result.n = newNodeI(nkRecList, info)
let s = newSym(skType, getIdent("Env_" & info.toFilename),
owner, info)
owner, info, owner.options)
incl s.flags, sfAnon
s.typ = result
result.sym = s
@ -174,7 +174,8 @@ proc lookupInRecord(n: PNode, id: int): PSym =
proc addField*(obj: PType; s: PSym) =
# because of 'gensym' support, we have to mangle the name with its ID.
# This is hacky but the clean solution is much more complex than it looks.
var field = newSym(skField, getIdent(s.name.s & $obj.n.len), s.owner, s.info)
var field = newSym(skField, getIdent(s.name.s & $obj.n.len), s.owner, s.info,
s.options)
field.id = -s.id
let t = skipIntLit(s.typ)
field.typ = t
@ -185,7 +186,8 @@ proc addField*(obj: PType; s: PSym) =
proc addUniqueField*(obj: PType; s: PSym): PSym {.discardable.} =
result = lookupInRecord(obj.n, s.id)
if result == nil:
var field = newSym(skField, getIdent(s.name.s & $obj.n.len), s.owner, s.info)
var field = newSym(skField, getIdent(s.name.s & $obj.n.len), s.owner, s.info,
s.options)
field.id = -s.id
let t = skipIntLit(s.typ)
field.typ = t
@ -218,7 +220,7 @@ proc indirectAccess*(a: PNode, b: int, info: TLineInfo): PNode =
#if field == nil:
# echo "FIELD ", b
# debug deref.typ
internalAssert field != nil
assert field != nil
addSon(deref, a)
result = newNodeI(nkDotExpr, info)
addSon(result, deref)
@ -242,7 +244,7 @@ proc indirectAccess(a: PNode, b: string, info: TLineInfo): PNode =
#if field == nil:
# echo "FIELD ", b
# debug deref.typ
internalAssert field != nil
assert field != nil
addSon(deref, a)
result = newNodeI(nkDotExpr, info)
addSon(result, deref)
@ -278,12 +280,12 @@ proc genDeref*(n: PNode): PNode =
n.typ.skipTypes(abstractInst).sons[0])
result.add n
proc callCodegenProc*(name: string, arg1: PNode;
proc callCodegenProc*(g: ModuleGraph; name: string, arg1: PNode;
arg2, arg3, optionalArgs: PNode = nil): PNode =
result = newNodeI(nkCall, arg1.info)
let sym = magicsys.getCompilerProc(name)
let sym = magicsys.getCompilerProc(g, name)
if sym == nil:
localError(arg1.info, errSystemNeeds, name)
localError(g.config, arg1.info, "system module needs: " & name)
else:
result.add newSymNode(sym)
result.add arg1
@ -330,12 +332,13 @@ proc typeNeedsNoDeepCopy(t: PType): bool =
# note that seq[T] is fine, but 'var seq[T]' is not, so we need to skip 'var'
# for the stricter check and likewise we can skip 'seq' for a less
# strict check:
if t.kind in {tyVar, tySequence}: t = t.sons[0]
if t.kind in {tyVar, tyLent, tySequence}: t = t.lastSon
result = not containsGarbageCollectedRef(t)
proc addLocalVar(varSection, varInit: PNode; owner: PSym; typ: PType;
proc addLocalVar(g: ModuleGraph; varSection, varInit: PNode; owner: PSym; typ: PType;
v: PNode; useShallowCopy=false): PSym =
result = newSym(skTemp, getIdent(genPrefix), owner, varSection.info)
result = newSym(skTemp, getIdent(genPrefix), owner, varSection.info,
owner.options)
result.typ = typ
incl(result.flags, sfFromGeneric)
@ -349,7 +352,7 @@ proc addLocalVar(varSection, varInit: PNode; owner: PSym; typ: PType;
varInit.add newFastAsgnStmt(newSymNode(result), v)
else:
let deepCopyCall = newNodeI(nkCall, varInit.info, 3)
deepCopyCall.sons[0] = newSymNode(getSysMagic("deepCopy", mDeepCopy))
deepCopyCall.sons[0] = newSymNode(getSysMagic(g, varSection.info, "deepCopy", mDeepCopy))
deepCopyCall.sons[1] = newSymNode(result)
deepCopyCall.sons[2] = v
varInit.add deepCopyCall
@ -384,23 +387,23 @@ stmtList:
"""
proc createWrapperProc(f: PNode; threadParam, argsParam: PSym;
proc createWrapperProc(g: ModuleGraph; f: PNode; threadParam, argsParam: PSym;
varSection, varInit, call, barrier, fv: PNode;
spawnKind: TSpawnResult): PSym =
var body = newNodeI(nkStmtList, f.info)
var threadLocalBarrier: PSym
if barrier != nil:
var varSection2 = newNodeI(nkVarSection, barrier.info)
threadLocalBarrier = addLocalVar(varSection2, nil, argsParam.owner,
threadLocalBarrier = addLocalVar(g, varSection2, nil, argsParam.owner,
barrier.typ, barrier)
body.add varSection2
body.add callCodegenProc("barrierEnter", threadLocalBarrier.newSymNode)
body.add callCodegenProc(g, "barrierEnter", threadLocalBarrier.newSymNode)
var threadLocalProm: PSym
if spawnKind == srByVar:
threadLocalProm = addLocalVar(varSection, nil, argsParam.owner, fv.typ, fv)
threadLocalProm = addLocalVar(g, varSection, nil, argsParam.owner, fv.typ, fv)
elif fv != nil:
internalAssert fv.typ.kind == tyGenericInst
threadLocalProm = addLocalVar(varSection, nil, argsParam.owner, fv.typ, fv)
internalAssert g.config, fv.typ.kind == tyGenericInst
threadLocalProm = addLocalVar(g, varSection, nil, argsParam.owner, fv.typ, fv)
body.add varSection
body.add varInit
if fv != nil and spawnKind != srByVar:
@ -409,30 +412,30 @@ proc createWrapperProc(f: PNode; threadParam, argsParam: PSym;
body.add newAsgnStmt(indirectAccess(threadLocalProm.newSymNode,
"owner", fv.info), threadParam.newSymNode)
body.add callCodegenProc("nimArgsPassingDone", threadParam.newSymNode)
body.add callCodegenProc(g, "nimArgsPassingDone", threadParam.newSymNode)
if spawnKind == srByVar:
body.add newAsgnStmt(genDeref(threadLocalProm.newSymNode), call)
elif fv != nil:
let fk = fv.typ.sons[1].flowVarKind
if fk == fvInvalid:
localError(f.info, "cannot create a flowVar of type: " &
localError(g.config, f.info, "cannot create a flowVar of type: " &
typeToString(fv.typ.sons[1]))
body.add newAsgnStmt(indirectAccess(threadLocalProm.newSymNode,
if fk == fvGC: "data" else: "blob", fv.info), call)
if fk == fvGC:
let incRefCall = newNodeI(nkCall, fv.info, 2)
incRefCall.sons[0] = newSymNode(getSysMagic("GCref", mGCref))
incRefCall.sons[0] = newSymNode(getSysMagic(g, fv.info, "GCref", mGCref))
incRefCall.sons[1] = indirectAccess(threadLocalProm.newSymNode,
"data", fv.info)
body.add incRefCall
if barrier == nil:
# by now 'fv' is shared and thus might have beeen overwritten! we need
# to use the thread-local view instead:
body.add callCodegenProc("nimFlowVarSignal", threadLocalProm.newSymNode)
body.add callCodegenProc(g, "nimFlowVarSignal", threadLocalProm.newSymNode)
else:
body.add call
if barrier != nil:
body.add callCodegenProc("barrierLeave", threadLocalBarrier.newSymNode)
body.add callCodegenProc(g, "barrierLeave", threadLocalBarrier.newSymNode)
var params = newNodeI(nkFormalParams, f.info)
params.add emptyNode
@ -449,7 +452,8 @@ proc createWrapperProc(f: PNode; threadParam, argsParam: PSym;
t.n.add argsParam.newSymNode
let name = (if f.kind == nkSym: f.sym.name.s else: genPrefix) & "Wrapper"
result = newSym(skProc, getIdent(name), argsParam.owner, f.info)
result = newSym(skProc, getIdent(name), argsParam.owner, f.info,
argsParam.options)
result.ast = newProcNode(nkProcDef, f.info, body, params, newSymNode(result))
result.typ = t
@ -460,7 +464,7 @@ proc createCastExpr(argsParam: PSym; objType: PType): PNode =
result.typ = newType(tyPtr, objType.owner)
result.typ.rawAddSon(objType)
proc setupArgsForConcurrency(n: PNode; objType: PType; scratchObj: PSym,
proc setupArgsForConcurrency(g: ModuleGraph; n: PNode; objType: PType; scratchObj: PSym,
castExpr, call,
varSection, varInit, result: PNode) =
let formals = n[0].typ.n
@ -469,18 +473,18 @@ proc setupArgsForConcurrency(n: PNode; objType: PType; scratchObj: PSym,
# we pick n's type here, which hopefully is 'tyArray' and not
# 'tyOpenArray':
var argType = n[i].typ.skipTypes(abstractInst)
if i < formals.len and formals[i].typ.kind == tyVar:
localError(n[i].info, "'spawn'ed function cannot have a 'var' parameter")
if i < formals.len and formals[i].typ.kind in {tyVar, tyLent}:
localError(g.config, n[i].info, "'spawn'ed function cannot have a 'var' parameter")
#elif containsTyRef(argType):
# localError(n[i].info, "'spawn'ed function cannot refer to 'ref'/closure")
let fieldname = if i < formals.len: formals[i].sym.name else: tmpName
var field = newSym(skField, fieldname, objType.owner, n.info)
var field = newSym(skField, fieldname, objType.owner, n.info, g.config.options)
field.typ = argType
objType.addField(field)
result.add newFastAsgnStmt(newDotExpr(scratchObj, field), n[i])
let temp = addLocalVar(varSection, varInit, objType.owner, argType,
let temp = addLocalVar(g, varSection, varInit, objType.owner, argType,
indirectAccess(castExpr, field, n.info))
call.add(newSymNode(temp))
@ -501,20 +505,20 @@ proc getRoot*(n: PNode): PSym =
if getMagic(n) == mSlice: result = getRoot(n.sons[1])
else: discard
proc newIntLit*(value: BiggestInt): PNode =
proc newIntLit*(g: ModuleGraph; info: TLineInfo; value: BiggestInt): PNode =
result = nkIntLit.newIntNode(value)
result.typ = getSysType(tyInt)
result.typ = getSysType(g, info, tyInt)
proc genHigh*(n: PNode): PNode =
proc genHigh*(g: ModuleGraph; n: PNode): PNode =
if skipTypes(n.typ, abstractVar).kind == tyArray:
result = newIntLit(lastOrd(skipTypes(n.typ, abstractVar)))
result = newIntLit(g, n.info, lastOrd(skipTypes(n.typ, abstractVar)))
else:
result = newNodeI(nkCall, n.info, 2)
result.typ = getSysType(tyInt)
result.sons[0] = newSymNode(getSysMagic("high", mHigh))
result.typ = getSysType(g, n.info, tyInt)
result.sons[0] = newSymNode(getSysMagic(g, n.info, "high", mHigh))
result.sons[1] = n
proc setupArgsForParallelism(n: PNode; objType: PType; scratchObj: PSym;
proc setupArgsForParallelism(g: ModuleGraph; n: PNode; objType: PType; scratchObj: PSym;
castExpr, call,
varSection, varInit, result: PNode) =
let formals = n[0].typ.n
@ -529,16 +533,16 @@ proc setupArgsForParallelism(n: PNode; objType: PType; scratchObj: PSym;
# localError(n.info, "'spawn'ed function cannot refer to 'ref'/closure")
let fieldname = if i < formals.len: formals[i].sym.name else: tmpName
var field = newSym(skField, fieldname, objType.owner, n.info)
var field = newSym(skField, fieldname, objType.owner, n.info, g.config.options)
if argType.kind in {tyVarargs, tyOpenArray}:
# important special case: we always create a zero-copy slice:
let slice = newNodeI(nkCall, n.info, 4)
slice.typ = n.typ
slice.sons[0] = newSymNode(createMagic("slice", mSlice))
slice.sons[0].typ = getSysType(tyInt) # fake type
var fieldB = newSym(skField, tmpName, objType.owner, n.info)
fieldB.typ = getSysType(tyInt)
slice.sons[0] = newSymNode(createMagic(g, "slice", mSlice))
slice.sons[0].typ = getSysType(g, n.info, tyInt) # fake type
var fieldB = newSym(skField, tmpName, objType.owner, n.info, g.config.options)
fieldB.typ = getSysType(g, n.info, tyInt)
objType.addField(fieldB)
if getMagic(n) == mSlice:
@ -547,13 +551,13 @@ proc setupArgsForParallelism(n: PNode; objType: PType; scratchObj: PSym;
objType.addField(field)
result.add newFastAsgnStmt(newDotExpr(scratchObj, field), a)
var fieldA = newSym(skField, tmpName, objType.owner, n.info)
fieldA.typ = getSysType(tyInt)
var fieldA = newSym(skField, tmpName, objType.owner, n.info, g.config.options)
fieldA.typ = getSysType(g, n.info, tyInt)
objType.addField(fieldA)
result.add newFastAsgnStmt(newDotExpr(scratchObj, fieldA), n[2])
result.add newFastAsgnStmt(newDotExpr(scratchObj, fieldB), n[3])
let threadLocal = addLocalVar(varSection,nil, objType.owner, fieldA.typ,
let threadLocal = addLocalVar(g, varSection,nil, objType.owner, fieldA.typ,
indirectAccess(castExpr, fieldA, n.info),
useShallowCopy=true)
slice.sons[2] = threadLocal.newSymNode
@ -562,13 +566,13 @@ proc setupArgsForParallelism(n: PNode; objType: PType; scratchObj: PSym;
field.typ = a.typ
objType.addField(field)
result.add newFastAsgnStmt(newDotExpr(scratchObj, field), a)
result.add newFastAsgnStmt(newDotExpr(scratchObj, fieldB), genHigh(n))
result.add newFastAsgnStmt(newDotExpr(scratchObj, fieldB), genHigh(g, n))
slice.sons[2] = newIntLit(0)
slice.sons[2] = newIntLit(g, n.info, 0)
# the array itself does not need to go through a thread local variable:
slice.sons[1] = genDeref(indirectAccess(castExpr, field, n.info))
let threadLocal = addLocalVar(varSection,nil, objType.owner, fieldB.typ,
let threadLocal = addLocalVar(g, varSection,nil, objType.owner, fieldB.typ,
indirectAccess(castExpr, fieldB, n.info),
useShallowCopy=true)
slice.sons[3] = threadLocal.newSymNode
@ -580,7 +584,7 @@ proc setupArgsForParallelism(n: PNode; objType: PType; scratchObj: PSym;
field.typ = a.typ
objType.addField(field)
result.add newFastAsgnStmt(newDotExpr(scratchObj, field), a)
let threadLocal = addLocalVar(varSection,nil, objType.owner, field.typ,
let threadLocal = addLocalVar(g, varSection,nil, objType.owner, field.typ,
indirectAccess(castExpr, field, n.info),
useShallowCopy=true)
call.add(genDeref(threadLocal.newSymNode))
@ -589,13 +593,13 @@ proc setupArgsForParallelism(n: PNode; objType: PType; scratchObj: PSym;
field.typ = argType
objType.addField(field)
result.add newFastAsgnStmt(newDotExpr(scratchObj, field), n)
let threadLocal = addLocalVar(varSection, varInit,
let threadLocal = addLocalVar(g, varSection, varInit,
objType.owner, field.typ,
indirectAccess(castExpr, field, n.info),
useShallowCopy=true)
call.add(threadLocal.newSymNode)
proc wrapProcForSpawn*(owner: PSym; spawnExpr: PNode; retType: PType;
proc wrapProcForSpawn*(g: ModuleGraph; owner: PSym; spawnExpr: PNode; retType: PType;
barrier, dest: PNode = nil): PNode =
# if 'barrier' != nil, then it is in a 'parallel' section and we
# generate quite different code
@ -603,35 +607,35 @@ proc wrapProcForSpawn*(owner: PSym; spawnExpr: PNode; retType: PType;
let spawnKind = spawnResult(retType, barrier!=nil)
case spawnKind
of srVoid:
internalAssert dest == nil
internalAssert g.config, dest == nil
result = newNodeI(nkStmtList, n.info)
of srFlowVar:
internalAssert dest == nil
internalAssert g.config, dest == nil
result = newNodeIT(nkStmtListExpr, n.info, retType)
of srByVar:
if dest == nil: localError(n.info, "'spawn' must not be discarded")
if dest == nil: localError(g.config, n.info, "'spawn' must not be discarded")
result = newNodeI(nkStmtList, n.info)
if n.kind notin nkCallKinds:
localError(n.info, "'spawn' takes a call expression")
localError(g.config, n.info, "'spawn' takes a call expression")
return
if optThreadAnalysis in gGlobalOptions:
if optThreadAnalysis in g.config.globalOptions:
if {tfThread, tfNoSideEffect} * n[0].typ.flags == {}:
localError(n.info, "'spawn' takes a GC safe call expression")
localError(g.config, n.info, "'spawn' takes a GC safe call expression")
var
threadParam = newSym(skParam, getIdent"thread", owner, n.info)
argsParam = newSym(skParam, getIdent"args", owner, n.info)
threadParam = newSym(skParam, getIdent"thread", owner, n.info, g.config.options)
argsParam = newSym(skParam, getIdent"args", owner, n.info, g.config.options)
block:
let ptrType = getSysType(tyPointer)
let ptrType = getSysType(g, n.info, tyPointer)
threadParam.typ = ptrType
argsParam.typ = ptrType
argsParam.position = 1
var objType = createObj(owner, n.info)
var objType = createObj(g, owner, n.info)
incl(objType.flags, tfFinal)
let castExpr = createCastExpr(argsParam, objType)
var scratchObj = newSym(skVar, getIdent"scratch", owner, n.info)
var scratchObj = newSym(skVar, getIdent"scratch", owner, n.info, g.config.options)
block:
scratchObj.typ = objType
incl(scratchObj.flags, sfFromGeneric)
@ -644,36 +648,36 @@ proc wrapProcForSpawn*(owner: PSym; spawnExpr: PNode; retType: PType;
# templates and macros are in fact valid here due to the nature of
# the transformation:
if fn.kind == nkClosure:
localError(n.info, "closure in spawn environment is not allowed")
localError(g.config, n.info, "closure in spawn environment is not allowed")
if not (fn.kind == nkSym and fn.sym.kind in {skProc, skTemplate, skMacro,
skFunc, skMethod, skConverter}):
# for indirect calls we pass the function pointer in the scratchObj
var argType = n[0].typ.skipTypes(abstractInst)
var field = newSym(skField, getIdent"fn", owner, n.info)
var field = newSym(skField, getIdent"fn", owner, n.info, g.config.options)
field.typ = argType
objType.addField(field)
result.add newFastAsgnStmt(newDotExpr(scratchObj, field), n[0])
fn = indirectAccess(castExpr, field, n.info)
elif fn.kind == nkSym and fn.sym.kind == skIterator:
localError(n.info, "iterator in spawn environment is not allowed")
localError(g.config, n.info, "iterator in spawn environment is not allowed")
elif fn.typ.callConv == ccClosure:
localError(n.info, "closure in spawn environment is not allowed")
localError(g.config, n.info, "closure in spawn environment is not allowed")
call.add(fn)
var varSection = newNodeI(nkVarSection, n.info)
var varInit = newNodeI(nkStmtList, n.info)
if barrier.isNil:
setupArgsForConcurrency(n, objType, scratchObj, castExpr, call,
setupArgsForConcurrency(g, n, objType, scratchObj, castExpr, call,
varSection, varInit, result)
else:
setupArgsForParallelism(n, objType, scratchObj, castExpr, call,
setupArgsForParallelism(g, n, objType, scratchObj, castExpr, call,
varSection, varInit, result)
var barrierAsExpr: PNode = nil
if barrier != nil:
let typ = newType(tyPtr, owner)
typ.rawAddSon(magicsys.getCompilerProc("Barrier").typ)
var field = newSym(skField, getIdent"barrier", owner, n.info)
typ.rawAddSon(magicsys.getCompilerProc(g, "Barrier").typ)
var field = newSym(skField, getIdent"barrier", owner, n.info, g.config.options)
field.typ = typ
objType.addField(field)
result.add newFastAsgnStmt(newDotExpr(scratchObj, field), barrier)
@ -681,7 +685,7 @@ proc wrapProcForSpawn*(owner: PSym; spawnExpr: PNode; retType: PType;
var fvField, fvAsExpr: PNode = nil
if spawnKind == srFlowVar:
var field = newSym(skField, getIdent"fv", owner, n.info)
var field = newSym(skField, getIdent"fv", owner, n.info, g.config.options)
field.typ = retType
objType.addField(field)
fvField = newDotExpr(scratchObj, field)
@ -689,20 +693,20 @@ proc wrapProcForSpawn*(owner: PSym; spawnExpr: PNode; retType: PType;
# create flowVar:
result.add newFastAsgnStmt(fvField, callProc(spawnExpr[^1]))
if barrier == nil:
result.add callCodegenProc("nimFlowVarCreateSemaphore", fvField)
result.add callCodegenProc(g, "nimFlowVarCreateSemaphore", fvField)
elif spawnKind == srByVar:
var field = newSym(skField, getIdent"fv", owner, n.info)
var field = newSym(skField, getIdent"fv", owner, n.info, g.config.options)
field.typ = newType(tyPtr, objType.owner)
field.typ.rawAddSon(retType)
objType.addField(field)
fvAsExpr = indirectAccess(castExpr, field, n.info)
result.add newFastAsgnStmt(newDotExpr(scratchObj, field), genAddrOf(dest))
let wrapper = createWrapperProc(fn, threadParam, argsParam,
let wrapper = createWrapperProc(g, fn, threadParam, argsParam,
varSection, varInit, call,
barrierAsExpr, fvAsExpr, spawnKind)
result.add callCodegenProc("nimSpawn" & $spawnExpr.len, wrapper.newSymNode,
result.add callCodegenProc(g, "nimSpawn" & $spawnExpr.len, wrapper.newSymNode,
genAddrOf(scratchObj.newSymNode), nil, spawnExpr)
if spawnKind == srFlowVar: result.add fvField

194
compiler/magicsys.nim
View file

@ -10,63 +10,62 @@
# Built-in types and compilerprocs are registered here.
import
ast, astalgo, hashes, msgs, platform, nversion, times, idents, rodread
ast, astalgo, hashes, msgs, platform, nversion, times, idents, rodread,
modulegraphs
var systemModule*: PSym
export createMagic
var
gSysTypes: array[TTypeKind, PType]
compilerprocs: TStrTable
exposed: TStrTable
proc nilOrSysInt*(g: ModuleGraph): PType = g.sysTypes[tyInt]
proc nilOrSysInt*: PType = gSysTypes[tyInt]
proc registerSysType*(g: ModuleGraph; t: PType) =
if g.sysTypes[t.kind] == nil: g.sysTypes[t.kind] = t
proc registerSysType*(t: PType) =
if gSysTypes[t.kind] == nil: gSysTypes[t.kind] = t
proc newSysType(kind: TTypeKind, size: int): PType =
result = newType(kind, systemModule)
proc newSysType(g: ModuleGraph; kind: TTypeKind, size: int): PType =
result = newType(kind, g.systemModule)
result.size = size
result.align = size.int16
proc getSysSym*(name: string): PSym =
result = strTableGet(systemModule.tab, getIdent(name))
proc getSysSym*(g: ModuleGraph; info: TLineInfo; name: string): PSym =
result = strTableGet(g.systemModule.tab, getIdent(name))
if result == nil:
rawMessage(errSystemNeeds, name)
result = newSym(skError, getIdent(name), systemModule, systemModule.info)
result.typ = newType(tyError, systemModule)
localError(g.config, info, "system module needs: " & name)
result = newSym(skError, getIdent(name), g.systemModule, g.systemModule.info, {})
result.typ = newType(tyError, g.systemModule)
if result.kind == skStub: loadStub(result)
if result.kind == skAlias: result = result.owner
proc createMagic*(name: string, m: TMagic): PSym =
result = newSym(skProc, getIdent(name), nil, unknownLineInfo())
result.magic = m
when false:
proc createMagic*(g: ModuleGraph; name: string, m: TMagic): PSym =
result = newSym(skProc, getIdent(name), nil, unknownLineInfo())
result.magic = m
let
opNot* = createMagic("not", mNot)
opContains* = createMagic("contains", mInSet)
when false:
let
opNot* = createMagic("not", mNot)
opContains* = createMagic("contains", mInSet)
proc getSysMagic*(name: string, m: TMagic): PSym =
proc getSysMagic*(g: ModuleGraph; info: TLineInfo; name: string, m: TMagic): PSym =
var ti: TIdentIter
let id = getIdent(name)
var r = initIdentIter(ti, systemModule.tab, id)
var r = initIdentIter(ti, g.systemModule.tab, id)
while r != nil:
if r.kind == skStub: loadStub(r)
if r.magic == m:
# prefer the tyInt variant:
if r.typ.sons[0] != nil and r.typ.sons[0].kind == tyInt: return r
result = r
r = nextIdentIter(ti, systemModule.tab)
r = nextIdentIter(ti, g.systemModule.tab)
if result != nil: return result
rawMessage(errSystemNeeds, name)
result = newSym(skError, id, systemModule, systemModule.info)
result.typ = newType(tyError, systemModule)
localError(g.config, info, "system module needs: " & name)
result = newSym(skError, id, g.systemModule, g.systemModule.info, {})
result.typ = newType(tyError, g.systemModule)
proc sysTypeFromName*(name: string): PType =
result = getSysSym(name).typ
proc sysTypeFromName*(g: ModuleGraph; info: TLineInfo; name: string): PType =
result = getSysSym(g, info, name).typ
proc getSysType*(kind: TTypeKind): PType =
result = gSysTypes[kind]
proc getSysType*(g: ModuleGraph; info: TLineInfo; kind: TTypeKind): PType =
template sysTypeFromName(s: string): untyped = sysTypeFromName(g, info, s)
result = g.sysTypes[kind]
if result == nil:
case kind
of tyInt: result = sysTypeFromName("int")
@ -88,51 +87,49 @@ proc getSysType*(kind: TTypeKind): PType =
of tyString: result = sysTypeFromName("string")
of tyCString: result = sysTypeFromName("cstring")
of tyPointer: result = sysTypeFromName("pointer")
of tyNil: result = newSysType(tyNil, ptrSize)
else: internalError("request for typekind: " & $kind)
gSysTypes[kind] = result
of tyNil: result = newSysType(g, tyNil, ptrSize)
else: internalError(g.config, "request for typekind: " & $kind)
g.sysTypes[kind] = result
if result.kind != kind:
internalError("wanted: " & $kind & " got: " & $result.kind)
if result == nil: internalError("type not found: " & $kind)
internalError(g.config, "wanted: " & $kind & " got: " & $result.kind)
if result == nil: internalError(g.config, "type not found: " & $kind)
var
intTypeCache: array[-5..64, PType]
proc resetSysTypes*(g: ModuleGraph) =
g.systemModule = nil
initStrTable(g.compilerprocs)
initStrTable(g.exposed)
for i in low(g.sysTypes)..high(g.sysTypes):
g.sysTypes[i] = nil
proc resetSysTypes* =
systemModule = nil
initStrTable(compilerprocs)
initStrTable(exposed)
for i in low(gSysTypes)..high(gSysTypes):
gSysTypes[i] = nil
for i in low(g.intTypeCache)..high(g.intTypeCache):
g.intTypeCache[i] = nil
for i in low(intTypeCache)..high(intTypeCache):
intTypeCache[i] = nil
proc getIntLitType*(literal: PNode): PType =
proc getIntLitType*(g: ModuleGraph; literal: PNode): PType =
# we cache some common integer literal types for performance:
let value = literal.intVal
if value >= low(intTypeCache) and value <= high(intTypeCache):
result = intTypeCache[value.int]
if value >= low(g.intTypeCache) and value <= high(g.intTypeCache):
result = g.intTypeCache[value.int]
if result == nil:
let ti = getSysType(tyInt)
let ti = getSysType(g, literal.info, tyInt)
result = copyType(ti, ti.owner, false)
result.n = literal
intTypeCache[value.int] = result
g.intTypeCache[value.int] = result
else:
let ti = getSysType(tyInt)
let ti = getSysType(g, literal.info, tyInt)
result = copyType(ti, ti.owner, false)
result.n = literal
proc getFloatLitType*(literal: PNode): PType =
proc getFloatLitType*(g: ModuleGraph; literal: PNode): PType =
# for now we do not cache these:
result = newSysType(tyFloat, size=8)
result = newSysType(g, tyFloat, size=8)
result.n = literal
proc skipIntLit*(t: PType): PType {.inline.} =
if t.n != nil:
if t.kind in {tyInt, tyFloat}:
return getSysType(t.kind)
result = t
if t.n != nil and t.kind in {tyInt, tyFloat}:
result = copyType(t, t.owner, false)
result.n = nil
else:
result = t
proc addSonSkipIntLit*(father, son: PType) =
if isNil(father.sons): father.sons = @[]
@ -140,60 +137,59 @@ proc addSonSkipIntLit*(father, son: PType) =
add(father.sons, s)
propagateToOwner(father, s)
proc setIntLitType*(result: PNode) =
proc setIntLitType*(g: ModuleGraph; result: PNode) =
let i = result.intVal
case platform.intSize
of 8: result.typ = getIntLitType(result)
of 8: result.typ = getIntLitType(g, result)
of 4:
if i >= low(int32) and i <= high(int32):
result.typ = getIntLitType(result)
result.typ = getIntLitType(g, result)
else:
result.typ = getSysType(tyInt64)
result.typ = getSysType(g, result.info, tyInt64)
of 2:
if i >= low(int16) and i <= high(int16):
result.typ = getIntLitType(result)
result.typ = getIntLitType(g, result)
elif i >= low(int32) and i <= high(int32):
result.typ = getSysType(tyInt32)
result.typ = getSysType(g, result.info, tyInt32)
else:
result.typ = getSysType(tyInt64)
result.typ = getSysType(g, result.info, tyInt64)
of 1:
# 8 bit CPUs are insane ...
if i >= low(int8) and i <= high(int8):
result.typ = getIntLitType(result)
result.typ = getIntLitType(g, result)
elif i >= low(int16) and i <= high(int16):
result.typ = getSysType(tyInt16)
result.typ = getSysType(g, result.info, tyInt16)
elif i >= low(int32) and i <= high(int32):
result.typ = getSysType(tyInt32)
result.typ = getSysType(g, result.info, tyInt32)
else:
result.typ = getSysType(tyInt64)
else: internalError(result.info, "invalid int size")
proc getCompilerProc*(name: string): PSym =
let ident = getIdent(name)
result = strTableGet(compilerprocs, ident)
if result == nil:
result = strTableGet(rodCompilerprocs, ident)
if result != nil:
strTableAdd(compilerprocs, result)
if result.kind == skStub: loadStub(result)
if result.kind == skAlias: result = result.owner
proc registerCompilerProc*(s: PSym) =
strTableAdd(compilerprocs, s)
proc registerNimScriptSymbol*(s: PSym) =
# Nimscript symbols must be al unique:
let conflict = strTableGet(exposed, s.name)
if conflict == nil:
strTableAdd(exposed, s)
result.typ = getSysType(g, result.info, tyInt64)
else:
localError(s.info, "symbol conflicts with other .exportNims symbol at: " &
$conflict.info)
internalError(g.config, result.info, "invalid int size")
proc getNimScriptSymbol*(name: string): PSym =
strTableGet(exposed, getIdent(name))
proc getCompilerProc*(g: ModuleGraph; name: string): PSym =
let ident = getIdent(name)
result = strTableGet(g.compilerprocs, ident)
when false:
if result == nil:
result = strTableGet(g.rodCompilerprocs, ident)
if result != nil:
strTableAdd(g.compilerprocs, result)
if result.kind == skStub: loadStub(result)
if result.kind == skAlias: result = result.owner
proc resetNimScriptSymbols*() = initStrTable(exposed)
proc registerCompilerProc*(g: ModuleGraph; s: PSym) =
strTableAdd(g.compilerprocs, s)
initStrTable(compilerprocs)
initStrTable(exposed)
proc registerNimScriptSymbol*(g: ModuleGraph; s: PSym) =
# Nimscript symbols must be al unique:
let conflict = strTableGet(g.exposed, s.name)
if conflict == nil:
strTableAdd(g.exposed, s)
else:
localError(g.config, s.info,
"symbol conflicts with other .exportNims symbol at: " & $conflict.info)
proc getNimScriptSymbol*(g: ModuleGraph; name: string): PSym =
strTableGet(g.exposed, getIdent(name))
proc resetNimScriptSymbols*(g: ModuleGraph) = initStrTable(g.exposed)

252
compiler/main.nim
View file

@ -16,12 +16,12 @@ import
cgen, jsgen, json, nversion,
platform, nimconf, importer, passaux, depends, vm, vmdef, types, idgen,
docgen2, service, parser, modules, ccgutils, sigmatch, ropes,
modulegraphs, tables
modulegraphs, tables, rod, configuration
from magicsys import systemModule, resetSysTypes
from magicsys import resetSysTypes
proc rodPass =
if gSymbolFiles in {enabledSf, writeOnlySf}:
proc rodPass(g: ModuleGraph) =
if g.config.symbolFiles in {enabledSf, writeOnlySf}:
registerPass(rodwritePass)
proc codegenPass =
@ -35,7 +35,7 @@ proc writeDepsFile(g: ModuleGraph; project: string) =
let f = open(changeFileExt(project, "deps"), fmWrite)
for m in g.modules:
if m != nil:
f.writeLine(toFullPath(m.position.int32))
f.writeLine(toFullPath(m.position.FileIndex))
for k in g.inclToMod.keys:
if g.getModule(k).isNil: # don't repeat includes which are also modules
f.writeLine(k.toFullPath)
@ -46,55 +46,55 @@ proc commandGenDepend(graph: ModuleGraph; cache: IdentCache) =
registerPass(gendependPass)
#registerPass(cleanupPass)
compileProject(graph, cache)
writeDepsFile(graph, gProjectFull)
generateDot(gProjectFull)
execExternalProgram("dot -Tpng -o" & changeFileExt(gProjectFull, "png") &
' ' & changeFileExt(gProjectFull, "dot"))
let project = graph.config.projectFull
writeDepsFile(graph, project)
generateDot(project)
execExternalProgram(graph.config, "dot -Tpng -o" & changeFileExt(project, "png") &
' ' & changeFileExt(project, "dot"))
proc commandCheck(graph: ModuleGraph; cache: IdentCache) =
msgs.gErrorMax = high(int) # do not stop after first error
defineSymbol("nimcheck")
graph.config.errorMax = high(int) # do not stop after first error
defineSymbol(graph.config.symbols, "nimcheck")
semanticPasses() # use an empty backend for semantic checking only
rodPass()
rodPass(graph)
compileProject(graph, cache)
proc commandDoc2(graph: ModuleGraph; cache: IdentCache; json: bool) =
msgs.gErrorMax = high(int) # do not stop after first error
graph.config.errorMax = high(int) # do not stop after first error
semanticPasses()
if json: registerPass(docgen2JsonPass)
else: registerPass(docgen2Pass)
#registerPass(cleanupPass())
compileProject(graph, cache)
finishDoc2Pass(gProjectName)
finishDoc2Pass(graph.config.projectName)
proc commandCompileToC(graph: ModuleGraph; cache: IdentCache) =
extccomp.initVars()
let conf = graph.config
extccomp.initVars(conf)
semanticPasses()
registerPass(cgenPass)
rodPass()
rodPass(graph)
#registerPass(cleanupPass())
compileProject(graph, cache)
cgenWriteModules(graph.backend, graph.config)
if gCmd != cmdRun:
let proj = changeFileExt(gProjectFull, "")
extccomp.callCCompiler(proj)
extccomp.writeJsonBuildInstructions(proj)
if optGenScript in gGlobalOptions:
writeDepsFile(graph, toGeneratedFile(proj, ""))
cgenWriteModules(graph.backend, conf)
if conf.cmd != cmdRun:
let proj = changeFileExt(conf.projectFull, "")
extccomp.callCCompiler(conf, proj)
extccomp.writeJsonBuildInstructions(conf, proj)
if optGenScript in graph.config.globalOptions:
writeDepsFile(graph, toGeneratedFile(conf, proj, ""))
proc commandJsonScript(graph: ModuleGraph; cache: IdentCache) =
let proj = changeFileExt(gProjectFull, "")
extccomp.runJsonBuildInstructions(proj)
let proj = changeFileExt(graph.config.projectFull, "")
extccomp.runJsonBuildInstructions(graph.config, proj)
proc commandCompileToJS(graph: ModuleGraph; cache: IdentCache) =
#incl(gGlobalOptions, optSafeCode)
setTarget(osJS, cpuJS)
#initDefines()
defineSymbol("nimrod") # 'nimrod' is always defined
defineSymbol("ecmascript") # For backward compatibility
defineSymbol("js")
if gCmd == cmdCompileToPHP: defineSymbol("nimphp")
defineSymbol(graph.config.symbols, "ecmascript") # For backward compatibility
defineSymbol(graph.config.symbols, "js")
semanticPasses()
registerPass(JSgenPass)
compileProject(graph, cache)
@ -102,19 +102,19 @@ proc commandCompileToJS(graph: ModuleGraph; cache: IdentCache) =
proc interactivePasses(graph: ModuleGraph; cache: IdentCache) =
#incl(gGlobalOptions, optSafeCode)
#setTarget(osNimrodVM, cpuNimrodVM)
initDefines()
defineSymbol("nimscript")
when hasFFI: defineSymbol("nimffi")
initDefines(graph.config.symbols)
defineSymbol(graph.config.symbols, "nimscript")
when hasFFI: defineSymbol(graph.config.symbols, "nimffi")
registerPass(verbosePass)
registerPass(semPass)
registerPass(evalPass)
proc commandInteractive(graph: ModuleGraph; cache: IdentCache) =
msgs.gErrorMax = high(int) # do not stop after first error
graph.config.errorMax = high(int) # do not stop after first error
interactivePasses(graph, cache)
compileSystemModule(graph, cache)
if commandArgs.len > 0:
discard graph.compileModule(fileInfoIdx(gProjectFull), cache, {})
if graph.config.commandArgs.len > 0:
discard graph.compileModule(fileInfoIdx(graph.config, graph.config.projectFull), cache, {})
else:
var m = graph.makeStdinModule()
incl(m.flags, sfMainModule)
@ -126,178 +126,171 @@ proc evalNim(graph: ModuleGraph; nodes: PNode, module: PSym; cache: IdentCache)
carryPasses(graph, nodes, module, cache, evalPasses)
proc commandEval(graph: ModuleGraph; cache: IdentCache; exp: string) =
if systemModule == nil:
if graph.systemModule == nil:
interactivePasses(graph, cache)
compileSystemModule(graph, cache)
let echoExp = "echo \"eval\\t\", " & "repr(" & exp & ")"
evalNim(graph, echoExp.parseString(cache), makeStdinModule(graph), cache)
evalNim(graph, echoExp.parseString(cache, graph.config),
makeStdinModule(graph), cache)
proc commandScan(cache: IdentCache) =
var f = addFileExt(mainCommandArg(), NimExt)
proc commandScan(cache: IdentCache, config: ConfigRef) =
var f = addFileExt(mainCommandArg(config), NimExt)
var stream = llStreamOpen(f, fmRead)
if stream != nil:
var
L: TLexer
tok: TToken
initToken(tok)
openLexer(L, f, stream, cache)
openLexer(L, f, stream, cache, config)
while true:
rawGetTok(L, tok)
printTok(tok)
printTok(config, tok)
if tok.tokType == tkEof: break
closeLexer(L)
else:
rawMessage(errCannotOpenFile, f)
rawMessage(config, errGenerated, "cannot open file: " & f)
const
SimulateCaasMemReset = false
PrintRopeCacheStats = false
proc mainCommand*(graph: ModuleGraph; cache: IdentCache) =
when SimulateCaasMemReset:
gGlobalOptions.incl(optCaasEnabled)
let conf = graph.config
setupModuleCache()
# In "nim serve" scenario, each command must reset the registered passes
clearPasses()
gLastCmdTime = epochTime()
searchPaths.add(options.libpath)
when false: # gProjectFull.len != 0:
# current path is always looked first for modules
prependStr(searchPaths, gProjectPath)
conf.lastCmdTime = epochTime()
conf.searchPaths.add(conf.libpath)
setId(100)
case command.normalize
case conf.command.normalize
of "c", "cc", "compile", "compiletoc":
# compile means compileToC currently
gCmd = cmdCompileToC
conf.cmd = cmdCompileToC
commandCompileToC(graph, cache)
of "cpp", "compiletocpp":
gCmd = cmdCompileToCpp
defineSymbol("cpp")
conf.cmd = cmdCompileToCpp
defineSymbol(graph.config.symbols, "cpp")
commandCompileToC(graph, cache)
of "objc", "compiletooc":
gCmd = cmdCompileToOC
defineSymbol("objc")
conf.cmd = cmdCompileToOC
defineSymbol(graph.config.symbols, "objc")
commandCompileToC(graph, cache)
of "run":
gCmd = cmdRun
conf.cmd = cmdRun
when hasTinyCBackend:
extccomp.setCC("tcc")
commandCompileToC(graph, cache)
else:
rawMessage(errInvalidCommandX, command)
rawMessage(conf, errGenerated, "'run' command not available; rebuild with -d:tinyc")
of "js", "compiletojs":
gCmd = cmdCompileToJS
commandCompileToJS(graph, cache)
of "php":
gCmd = cmdCompileToPHP
conf.cmd = cmdCompileToJS
commandCompileToJS(graph, cache)
of "doc0":
wantMainModule()
gCmd = cmdDoc
loadConfigs(DocConfig, cache)
commandDoc()
wantMainModule(conf)
conf.cmd = cmdDoc
loadConfigs(DocConfig, cache, conf)
commandDoc(conf)
of "doc2", "doc":
gCmd = cmdDoc
loadConfigs(DocConfig, cache)
defineSymbol("nimdoc")
conf.cmd = cmdDoc
loadConfigs(DocConfig, cache, conf)
defineSymbol(conf.symbols, "nimdoc")
commandDoc2(graph, cache, false)
of "rst2html":
gCmd = cmdRst2html
loadConfigs(DocConfig, cache)
commandRst2Html()
conf.cmd = cmdRst2html
loadConfigs(DocConfig, cache, conf)
commandRst2Html(conf)
of "rst2tex":
gCmd = cmdRst2tex
loadConfigs(DocTexConfig, cache)
commandRst2TeX()
of "jsondoc":
wantMainModule()
gCmd = cmdDoc
loadConfigs(DocConfig, cache)
wantMainModule()
defineSymbol("nimdoc")
commandJson()
of "jsondoc2":
gCmd = cmdDoc
loadConfigs(DocConfig, cache)
wantMainModule()
defineSymbol("nimdoc")
conf.cmd = cmdRst2tex
loadConfigs(DocTexConfig, cache, conf)
commandRst2TeX(conf)
of "jsondoc0":
wantMainModule(conf)
conf.cmd = cmdDoc
loadConfigs(DocConfig, cache, conf)
wantMainModule(conf)
defineSymbol(conf.symbols, "nimdoc")
commandJson(conf)
of "jsondoc2", "jsondoc":
conf.cmd = cmdDoc
loadConfigs(DocConfig, cache, conf)
wantMainModule(conf)
defineSymbol(conf.symbols, "nimdoc")
commandDoc2(graph, cache, true)
of "ctags":
wantMainModule()
gCmd = cmdDoc
loadConfigs(DocConfig, cache)
wantMainModule()
defineSymbol("nimdoc")
commandTags()
wantMainModule(conf)
conf.cmd = cmdDoc
loadConfigs(DocConfig, cache, conf)
defineSymbol(conf.symbols, "nimdoc")
commandTags(conf)
of "buildindex":
gCmd = cmdDoc
loadConfigs(DocConfig, cache)
commandBuildIndex()
conf.cmd = cmdDoc
loadConfigs(DocConfig, cache, conf)
commandBuildIndex(conf)
of "gendepend":
gCmd = cmdGenDepend
conf.cmd = cmdGenDepend
commandGenDepend(graph, cache)
of "dump":
gCmd = cmdDump
if getConfigVar("dump.format") == "json":
wantMainModule()
conf.cmd = cmdDump
if getConfigVar(conf, "dump.format") == "json":
wantMainModule(conf)
var definedSymbols = newJArray()
for s in definedSymbolNames(): definedSymbols.elems.add(%s)
for s in definedSymbolNames(conf.symbols): definedSymbols.elems.add(%s)
var libpaths = newJArray()
for dir in searchPaths: libpaths.elems.add(%dir)
for dir in conf.searchPaths: libpaths.elems.add(%dir)
var dumpdata = % [
(key: "version", val: %VersionAsString),
(key: "project_path", val: %gProjectFull),
(key: "project_path", val: %conf.projectFull),
(key: "defined_symbols", val: definedSymbols),
(key: "lib_paths", val: libpaths)
]
msgWriteln($dumpdata, {msgStdout, msgSkipHook})
msgWriteln(conf, $dumpdata, {msgStdout, msgSkipHook})
else:
msgWriteln("-- list of currently defined symbols --",
msgWriteln(conf, "-- list of currently defined symbols --",
{msgStdout, msgSkipHook})
for s in definedSymbolNames(): msgWriteln(s, {msgStdout, msgSkipHook})
msgWriteln("-- end of list --", {msgStdout, msgSkipHook})
for s in definedSymbolNames(conf.symbols): msgWriteln(conf, s, {msgStdout, msgSkipHook})
msgWriteln(conf, "-- end of list --", {msgStdout, msgSkipHook})
for it in searchPaths: msgWriteln(it)
for it in conf.searchPaths: msgWriteln(conf, it)
of "check":
gCmd = cmdCheck
conf.cmd = cmdCheck
commandCheck(graph, cache)
of "parse":
gCmd = cmdParse
wantMainModule()
discard parseFile(gProjectMainIdx, cache)
conf.cmd = cmdParse
wantMainModule(conf)
discard parseFile(FileIndex conf.projectMainIdx, cache, conf)
of "scan":
gCmd = cmdScan
wantMainModule()
commandScan(cache)
msgWriteln("Beware: Indentation tokens depend on the parser's state!")
conf.cmd = cmdScan
wantMainModule(conf)
commandScan(cache, conf)
msgWriteln(conf, "Beware: Indentation tokens depend on the parser's state!")
of "secret":
gCmd = cmdInteractive
conf.cmd = cmdInteractive
commandInteractive(graph, cache)
of "e":
commandEval(graph, cache, mainCommandArg())
commandEval(graph, cache, mainCommandArg(conf))
of "nop", "help":
# prevent the "success" message:
gCmd = cmdDump
conf.cmd = cmdDump
of "jsonscript":
gCmd = cmdJsonScript
conf.cmd = cmdJsonScript
commandJsonScript(graph, cache)
else:
rawMessage(errInvalidCommandX, command)
rawMessage(conf, errGenerated, "invalid command: " & conf.command)
if msgs.gErrorCounter == 0 and
gCmd notin {cmdInterpret, cmdRun, cmdDump}:
if conf.errorCounter == 0 and
conf.cmd notin {cmdInterpret, cmdRun, cmdDump}:
when declared(system.getMaxMem):
let usedMem = formatSize(getMaxMem()) & " peakmem"
else:
let usedMem = formatSize(getTotalMem())
rawMessage(hintSuccessX, [$gLinesCompiled,
formatFloat(epochTime() - gLastCmdTime, ffDecimal, 3),
rawMessage(conf, hintSuccessX, [$conf.linesCompiled,
formatFloat(epochTime() - conf.lastCmdTime, ffDecimal, 3),
usedMem,
if condSyms.isDefined("release"): "Release Build"
if isDefined(conf, "release"): "Release Build"
else: "Debug Build"])
when PrintRopeCacheStats:
@ -308,9 +301,6 @@ proc mainCommand*(graph: ModuleGraph; cache: IdentCache) =
echo " efficiency: ", formatFloat(1-(gCacheMisses.float/gCacheTries.float),
ffDecimal, 3)
when SimulateCaasMemReset:
resetMemory()
resetAttributes(conf)
resetAttributes()
proc mainCommand*() = mainCommand(newModuleGraph(newConfigRef()), newIdentCache())
#proc mainCommand*() = mainCommand(newModuleGraph(newConfigRef()), newIdentCache())

57
compiler/modulegraphs.nim
View file

@ -25,7 +25,7 @@
## - Its dependent module stays the same.
##
import ast, intsets, tables, options
import ast, intsets, tables, options, rod, msgs, hashes, idents
type
ModuleGraph* = ref object
@ -34,67 +34,88 @@ type
deps*: IntSet # the dependency graph or potentially its transitive closure.
suggestMode*: bool # whether we are in nimsuggest mode or not.
invalidTransitiveClosure: bool
inclToMod*: Table[int32, int32] # mapping of include file to the
# first module that included it
importStack*: seq[int32] # The current import stack. Used for detecting recursive
# module dependencies.
inclToMod*: Table[FileIndex, FileIndex] # mapping of include file to the
# first module that included it
importStack*: seq[FileIndex] # The current import stack. Used for detecting recursive
# module dependencies.
backend*: RootRef # minor hack so that a backend can extend this easily
config*: ConfigRef
doStopCompile*: proc(): bool {.closure.}
usageSym*: PSym # for nimsuggest
owners*: seq[PSym]
methods*: seq[tuple[methods: TSymSeq, dispatcher: PSym]]
systemModule*: PSym
sysTypes*: array[TTypeKind, PType]
compilerprocs*: TStrTable
exposed*: TStrTable
intTypeCache*: array[-5..64, PType]
opContains*, opNot*: PSym
proc hash*(x: FileIndex): Hash {.borrow.}
{.this: g.}
proc stopCompile*(g: ModuleGraph): bool {.inline.} =
result = doStopCompile != nil and doStopCompile()
proc createMagic*(g: ModuleGraph; name: string, m: TMagic): PSym =
result = newSym(skProc, getIdent(name), nil, unknownLineInfo(), {})
result.magic = m
proc newModuleGraph*(config: ConfigRef = nil): ModuleGraph =
result = ModuleGraph()
initStrTable(result.packageSyms)
result.deps = initIntSet()
result.modules = @[]
result.importStack = @[]
result.inclToMod = initTable[int32, int32]()
result.inclToMod = initTable[FileIndex, FileIndex]()
if config.isNil:
result.config = newConfigRef()
else:
result.config = config
result.owners = @[]
result.methods = @[]
initStrTable(result.compilerprocs)
initStrTable(result.exposed)
result.opNot = createMagic(result, "not", mNot)
result.opContains = createMagic(result, "contains", mInSet)
proc resetAllModules*(g: ModuleGraph) =
initStrTable(packageSyms)
deps = initIntSet()
modules = @[]
importStack = @[]
inclToMod = initTable[int32, int32]()
inclToMod = initTable[FileIndex, FileIndex]()
usageSym = nil
owners = @[]
methods = @[]
initStrTable(compilerprocs)
initStrTable(exposed)
proc getModule*(g: ModuleGraph; fileIdx: int32): PSym =
if fileIdx >= 0 and fileIdx < modules.len:
result = modules[fileIdx]
proc getModule*(g: ModuleGraph; fileIdx: FileIndex): PSym =
if fileIdx.int32 >= 0 and fileIdx.int32 < modules.len:
result = modules[fileIdx.int32]
proc dependsOn(a, b: int): int {.inline.} = (a shl 15) + b
proc addDep*(g: ModuleGraph; m: PSym, dep: int32) =
proc addDep*(g: ModuleGraph; m: PSym, dep: FileIndex) =
assert m.position == m.info.fileIndex.int32
addModuleDep(m.info.fileIndex, dep, isIncludeFile = false)
if suggestMode:
deps.incl m.position.dependsOn(dep)
deps.incl m.position.dependsOn(dep.int)
# we compute the transitive closure later when quering the graph lazily.
# this improve efficiency quite a lot:
#invalidTransitiveClosure = true
proc addIncludeDep*(g: ModuleGraph; module, includeFile: int32) =
proc addIncludeDep*(g: ModuleGraph; module, includeFile: FileIndex) =
addModuleDep(module, includeFile, isIncludeFile = true)
discard hasKeyOrPut(inclToMod, includeFile, module)
proc parentModule*(g: ModuleGraph; fileIdx: int32): int32 =
proc parentModule*(g: ModuleGraph; fileIdx: FileIndex): FileIndex =
## returns 'fileIdx' if the file belonging to this index is
## directly used as a module or else the module that first
## references this include file.
if fileIdx >= 0 and fileIdx < modules.len and modules[fileIdx] != nil:
if fileIdx.int32 >= 0 and fileIdx.int32 < modules.len and modules[fileIdx.int32] != nil:
result = fileIdx
else:
result = inclToMod.getOrDefault(fileIdx)
@ -108,11 +129,11 @@ proc transitiveClosure(g: var IntSet; n: int) =
if g.contains(i.dependsOn(k)) and g.contains(k.dependsOn(j)):
g.incl i.dependsOn(j)
proc markDirty*(g: ModuleGraph; fileIdx: int32) =
proc markDirty*(g: ModuleGraph; fileIdx: FileIndex) =
let m = getModule fileIdx
if m != nil: incl m.flags, sfDirty
proc markClientsDirty*(g: ModuleGraph; fileIdx: int32) =
proc markClientsDirty*(g: ModuleGraph; fileIdx: FileIndex) =
# we need to mark its dependent modules D as dirty right away because after
# nimsuggest is done with this module, the module's dirty flag will be
# cleared but D still needs to be remembered as 'dirty'.
@ -123,7 +144,7 @@ proc markClientsDirty*(g: ModuleGraph; fileIdx: int32) =
# every module that *depends* on this file is also dirty:
for i in 0i32..<modules.len.int32:
let m = modules[i]
if m != nil and deps.contains(i.dependsOn(fileIdx)):
if m != nil and deps.contains(i.dependsOn(fileIdx.int)):
incl m.flags, sfDirty
proc isDirty*(g: ModuleGraph; m: PSym): bool =

225
compiler/modulepaths.nim
View file

@ -11,38 +11,39 @@ import ast, renderer, strutils, msgs, options, idents, os
import nimblecmd
const
considerParentDirs = not defined(noParentProjects)
considerNimbleDirs = not defined(noNimbleDirs)
when false:
const
considerParentDirs = not defined(noParentProjects)
considerNimbleDirs = not defined(noNimbleDirs)
proc findInNimbleDir(pkg, subdir, dir: string): string =
var best = ""
var bestv = ""
for k, p in os.walkDir(dir, relative=true):
if k == pcDir and p.len > pkg.len+1 and
p[pkg.len] == '-' and p.startsWith(pkg):
let (_, a) = getPathVersion(p)
if bestv.len == 0 or bestv < a:
bestv = a
best = dir / p
proc findInNimbleDir(pkg, subdir, dir: string): string =
var best = ""
var bestv = ""
for k, p in os.walkDir(dir, relative=true):
if k == pcDir and p.len > pkg.len+1 and
p[pkg.len] == '-' and p.startsWith(pkg):
let (_, a) = getPathVersion(p)
if bestv.len == 0 or bestv < a:
bestv = a
best = dir / p
if best.len > 0:
var f: File
if open(f, best / changeFileExt(pkg, ".nimble-link")):
# the second line contains what we're interested in, see:
# https://github.com/nim-lang/nimble#nimble-link
var override = ""
discard readLine(f, override)
discard readLine(f, override)
close(f)
if not override.isAbsolute():
best = best / override
else:
best = override
let f = if subdir.len == 0: pkg else: subdir
let res = addFileExt(best / f, "nim")
if best.len > 0 and fileExists(res):
result = res
if best.len > 0:
var f: File
if open(f, best / changeFileExt(pkg, ".nimble-link")):
# the second line contains what we're interested in, see:
# https://github.com/nim-lang/nimble#nimble-link
var override = ""
discard readLine(f, override)
discard readLine(f, override)
close(f)
if not override.isAbsolute():
best = best / override
else:
best = override
let f = if subdir.len == 0: pkg else: subdir
let res = addFileExt(best / f, "nim")
if best.len > 0 and fileExists(res):
result = res
const stdlibDirs = [
"pure", "core", "arch",
@ -51,76 +52,77 @@ const stdlibDirs = [
"wrappers", "wrappers/linenoise",
"windows", "posix", "js"]
proc resolveDollar(project, source, pkg, subdir: string; info: TLineInfo): string =
template attempt(a) =
let x = addFileExt(a, "nim")
if fileExists(x): return x
when false:
proc resolveDollar(project, source, pkg, subdir: string; info: TLineInfo): string =
template attempt(a) =
let x = addFileExt(a, "nim")
if fileExists(x): return x
case pkg
of "stdlib":
if subdir.len == 0:
return options.libpath
case pkg
of "stdlib":
if subdir.len == 0:
return options.libpath
else:
for candidate in stdlibDirs:
attempt(options.libpath / candidate / subdir)
of "root":
let root = project.splitFile.dir
if subdir.len == 0:
return root
else:
attempt(root / subdir)
else:
for candidate in stdlibDirs:
attempt(options.libpath / candidate / subdir)
of "root":
let root = project.splitFile.dir
if subdir.len == 0:
return root
else:
attempt(root / subdir)
else:
when considerParentDirs:
var p = parentDir(source.splitFile.dir)
# support 'import $karax':
let f = if subdir.len == 0: pkg else: subdir
when considerParentDirs:
var p = parentDir(source.splitFile.dir)
# support 'import $karax':
let f = if subdir.len == 0: pkg else: subdir
while p.len > 0:
let dir = p / pkg
if dirExists(dir):
attempt(dir / f)
# 2nd attempt: try to use 'karax/karax'
attempt(dir / pkg / f)
# 3rd attempt: try to use 'karax/src/karax'
attempt(dir / "src" / f)
attempt(dir / "src" / pkg / f)
p = parentDir(p)
while p.len > 0:
let dir = p / pkg
if dirExists(dir):
attempt(dir / f)
# 2nd attempt: try to use 'karax/karax'
attempt(dir / pkg / f)
# 3rd attempt: try to use 'karax/src/karax'
attempt(dir / "src" / f)
attempt(dir / "src" / pkg / f)
p = parentDir(p)
when considerNimbleDirs:
if not options.gNoNimblePath:
var nimbleDir = getEnv("NIMBLE_DIR")
if nimbleDir.len == 0: nimbleDir = getHomeDir() / ".nimble"
result = findInNimbleDir(pkg, subdir, nimbleDir / "pkgs")
if result.len > 0: return result
when not defined(windows):
result = findInNimbleDir(pkg, subdir, "/opt/nimble/pkgs")
when considerNimbleDirs:
if not options.gNoNimblePath:
var nimbleDir = getEnv("NIMBLE_DIR")
if nimbleDir.len == 0: nimbleDir = getHomeDir() / ".nimble"
result = findInNimbleDir(pkg, subdir, nimbleDir / "pkgs")
if result.len > 0: return result
when not defined(windows):
result = findInNimbleDir(pkg, subdir, "/opt/nimble/pkgs")
if result.len > 0: return result
proc scriptableImport(pkg, sub: string; info: TLineInfo): string =
result = resolveDollar(gProjectFull, info.toFullPath(), pkg, sub, info)
if result.isNil: result = ""
proc scriptableImport(pkg, sub: string; info: TLineInfo): string =
result = resolveDollar(gProjectFull, info.toFullPath(), pkg, sub, info)
if result.isNil: result = ""
proc lookupPackage(pkg, subdir: PNode): string =
let sub = if subdir != nil: renderTree(subdir, {renderNoComments}).replace(" ") else: ""
case pkg.kind
of nkStrLit, nkRStrLit, nkTripleStrLit:
result = scriptableImport(pkg.strVal, sub, pkg.info)
of nkIdent:
result = scriptableImport(pkg.ident.s, sub, pkg.info)
else:
localError(pkg.info, "package name must be an identifier or string literal")
result = ""
proc lookupPackage(pkg, subdir: PNode): string =
let sub = if subdir != nil: renderTree(subdir, {renderNoComments}).replace(" ") else: ""
case pkg.kind
of nkStrLit, nkRStrLit, nkTripleStrLit:
result = scriptableImport(pkg.strVal, sub, pkg.info)
of nkIdent:
result = scriptableImport(pkg.ident.s, sub, pkg.info)
else:
localError(pkg.info, "package name must be an identifier or string literal")
result = ""
proc getModuleName*(n: PNode): string =
proc getModuleName*(conf: ConfigRef; n: PNode): string =
# This returns a short relative module name without the nim extension
# e.g. like "system", "importer" or "somepath/module"
# The proc won't perform any checks that the path is actually valid
case n.kind
of nkStrLit, nkRStrLit, nkTripleStrLit:
try:
result = pathSubs(n.strVal, n.info.toFullPath().splitFile().dir)
result = pathSubs(conf, n.strVal, n.info.toFullPath().splitFile().dir)
except ValueError:
localError(n.info, "invalid path: " & n.strVal)
localError(conf, n.info, "invalid path: " & n.strVal)
result = n.strVal
of nkIdent:
result = n.ident.s
@ -135,40 +137,51 @@ proc getModuleName*(n: PNode): string =
n.sons[0] = n.sons[1]
n.sons[1] = n.sons[2]
n.sons.setLen(2)
return getModuleName(n.sons[0])
if n1.kind == nkPrefix and n1[0].kind == nkIdent and n1[0].ident.s == "$":
if n0.kind == nkIdent and n0.ident.s == "/":
result = lookupPackage(n1[1], n[2])
else:
localError(n.info, "only '/' supported with $package notation")
result = ""
return getModuleName(conf, n.sons[0])
when false:
if n1.kind == nkPrefix and n1[0].kind == nkIdent and n1[0].ident.s == "$":
if n0.kind == nkIdent and n0.ident.s == "/":
result = lookupPackage(n1[1], n[2])
else:
localError(n.info, "only '/' supported with $package notation")
result = ""
else:
let modname = getModuleName(conf, n[2])
if $n1 == "std":
template attempt(a) =
let x = addFileExt(a, "nim")
if fileExists(x): return x
for candidate in stdlibDirs:
attempt(conf.libpath / candidate / modname)
# hacky way to implement 'x / y /../ z':
result = getModuleName(n1)
result = getModuleName(conf, n1)
result.add renderTree(n0, {renderNoComments})
result.add getModuleName(n[2])
result.add modname
of nkPrefix:
if n.sons[0].kind == nkIdent and n.sons[0].ident.s == "$":
result = lookupPackage(n[1], nil)
else:
# hacky way to implement 'x / y /../ z':
result = renderTree(n, {renderNoComments}).replace(" ")
when false:
if n.sons[0].kind == nkIdent and n.sons[0].ident.s == "$":
result = lookupPackage(n[1], nil)
else:
discard
# hacky way to implement 'x / y /../ z':
result = renderTree(n, {renderNoComments}).replace(" ")
of nkDotExpr:
result = renderTree(n, {renderNoComments}).replace(".", "/")
of nkImportAs:
result = getModuleName(n.sons[0])
result = getModuleName(conf, n.sons[0])
else:
localError(n.info, errGenerated, "invalid module name: '$1'" % n.renderTree)
localError(conf, n.info, "invalid module name: '$1'" % n.renderTree)
result = ""
proc checkModuleName*(n: PNode; doLocalError=true): int32 =
proc checkModuleName*(conf: ConfigRef; n: PNode; doLocalError=true): FileIndex =
# This returns the full canonical path for a given module import
let modulename = n.getModuleName
let fullPath = findModule(modulename, n.info.toFullPath)
let modulename = getModuleName(conf, n)
let fullPath = findModule(conf, modulename, n.info.toFullPath)
if fullPath.len == 0:
if doLocalError:
let m = if modulename.len > 0: modulename else: $n
localError(n.info, errCannotOpenFile, m)
localError(conf, n.info, "cannot open file: " & m)
result = InvalidFileIDX
else:
result = fullPath.fileInfoIdx
result = fileInfoIdx(conf, fullPath)

187
compiler/modules.nim
View file

@ -10,131 +10,27 @@
## Implements the module handling, including the caching of modules.
import
ast, astalgo, magicsys, securehash, rodread, msgs, cgendata, sigmatch, options,
idents, os, lexer, idgen, passes, syntaxes, llstream, modulegraphs
ast, astalgo, magicsys, std / sha1, rodread, msgs, cgendata, sigmatch, options,
idents, os, lexer, idgen, passes, syntaxes, llstream, modulegraphs, rod,
configuration
when false:
type
TNeedRecompile* = enum Maybe, No, Yes, Probing, Recompiled
THashStatus* = enum hashNotTaken, hashCached, hashHasChanged, hashNotChanged
proc resetSystemArtifacts*(g: ModuleGraph) =
magicsys.resetSysTypes(g)
TModuleInMemory* = object
hash*: SecureHash
deps*: seq[int32] ## XXX: slurped files are currently not tracked
needsRecompile*: TNeedRecompile
hashStatus*: THashStatus
var
gCompiledModules: seq[PSym] = @[]
gMemCacheData*: seq[TModuleInMemory] = @[]
## XXX: we should implement recycling of file IDs
## if the user keeps renaming modules, the file IDs will keep growing
gFuzzyGraphChecking*: bool # nimsuggest uses this. XXX figure out why.
proc hashChanged(fileIdx: int32): bool =
internalAssert fileIdx >= 0 and fileIdx < gMemCacheData.len
template updateStatus =
gMemCacheData[fileIdx].hashStatus = if result: hashHasChanged
else: hashNotChanged
# echo "TESTING Hash: ", fileIdx.toFilename, " ", result
case gMemCacheData[fileIdx].hashStatus
of hashHasChanged:
result = true
of hashNotChanged:
result = false
of hashCached:
let newHash = secureHashFile(fileIdx.toFullPath)
result = newHash != gMemCacheData[fileIdx].hash
gMemCacheData[fileIdx].hash = newHash
updateStatus()
of hashNotTaken:
gMemCacheData[fileIdx].hash = secureHashFile(fileIdx.toFullPath)
result = true
updateStatus()
proc doHash(fileIdx: int32) =
if gMemCacheData[fileIdx].hashStatus == hashNotTaken:
# echo "FIRST Hash: ", fileIdx.ToFilename
gMemCacheData[fileIdx].hash = secureHashFile(fileIdx.toFullPath)
proc resetModule*(fileIdx: int32) =
# echo "HARD RESETTING ", fileIdx.toFilename
if fileIdx <% gMemCacheData.len:
gMemCacheData[fileIdx].needsRecompile = Yes
if fileIdx <% gCompiledModules.len:
gCompiledModules[fileIdx] = nil
if fileIdx <% cgendata.gModules.len:
cgendata.gModules[fileIdx] = nil
proc resetModule*(module: PSym) =
let conflict = getModule(module.position.int32)
if conflict == nil: return
doAssert conflict == module
resetModule(module.position.int32)
initStrTable(module.tab)
proc resetAllModules* =
for i in 0..gCompiledModules.high:
if gCompiledModules[i] != nil:
resetModule(i.int32)
resetPackageCache()
# for m in cgenModules(): echo "CGEN MODULE FOUND"
proc resetAllModulesHard* =
resetPackageCache()
gCompiledModules.setLen 0
gMemCacheData.setLen 0
magicsys.resetSysTypes()
# XXX
#gOwners = @[]
proc checkDepMem(fileIdx: int32): TNeedRecompile =
template markDirty =
resetModule(fileIdx)
return Yes
if gFuzzyGraphChecking:
if gMemCacheData[fileIdx].needsRecompile != Maybe:
return gMemCacheData[fileIdx].needsRecompile
else:
# cycle detection: We claim that a cycle does no harm.
if gMemCacheData[fileIdx].needsRecompile == Probing:
return No
if optForceFullMake in gGlobalOptions or hashChanged(fileIdx):
markDirty()
if gMemCacheData[fileIdx].deps != nil:
gMemCacheData[fileIdx].needsRecompile = Probing
for dep in gMemCacheData[fileIdx].deps:
let d = checkDepMem(dep)
if d in {Yes, Recompiled}:
# echo fileIdx.toFilename, " depends on ", dep.toFilename, " ", d
markDirty()
gMemCacheData[fileIdx].needsRecompile = No
return No
proc resetSystemArtifacts*() =
magicsys.resetSysTypes()
proc newModule(graph: ModuleGraph; fileIdx: int32): PSym =
proc newModule(graph: ModuleGraph; fileIdx: FileIndex): PSym =
# We cannot call ``newSym`` here, because we have to circumvent the ID
# mechanism, which we do in order to assign each module a persistent ID.
new(result)
result.id = - 1 # for better error checking
result.id = -1 # for better error checking
result.kind = skModule
let filename = fileIdx.toFullPath
result.name = getIdent(splitFile(filename).name)
if not isNimIdentifier(result.name.s):
rawMessage(errInvalidModuleName, result.name.s)
rawMessage(graph.config, errGenerated, "invalid module name: " & result.name.s)
result.info = newLineInfo(fileIdx, 1, 1)
let
pck = getPackageName(filename)
pck = getPackageName(graph.config, filename)
pck2 = if pck.len > 0: pck else: "unknown"
pack = getIdent(pck2)
var packSym = graph.packageSyms.strTableGet(pack)
@ -144,9 +40,9 @@ proc newModule(graph: ModuleGraph; fileIdx: int32): PSym =
graph.packageSyms.strTableAdd(packSym)
result.owner = packSym
result.position = fileIdx
result.position = int fileIdx
growCache graph.modules, fileIdx
growCache graph.modules, int fileIdx
graph.modules[result.position] = result
incl(result.flags, sfUsed)
@ -154,11 +50,11 @@ proc newModule(graph: ModuleGraph; fileIdx: int32): PSym =
strTableAdd(result.tab, result) # a module knows itself
let existing = strTableGet(packSym.tab, result.name)
if existing != nil and existing.info.fileIndex != result.info.fileIndex:
localError(result.info, "module names need to be unique per Nimble package; module clashes with " & existing.info.fileIndex.toFullPath)
localError(graph.config, result.info, "module names need to be unique per Nimble package; module clashes with " & existing.info.fileIndex.toFullPath)
# strTableIncl() for error corrections:
discard strTableIncl(packSym.tab, result)
proc compileModule*(graph: ModuleGraph; fileIdx: int32; cache: IdentCache, flags: TSymFlags): PSym =
proc compileModule*(graph: ModuleGraph; fileIdx: FileIndex; cache: IdentCache, flags: TSymFlags): PSym =
result = graph.getModule(fileIdx)
if result == nil:
#growCache gMemCacheData, fileIdx
@ -169,15 +65,17 @@ proc compileModule*(graph: ModuleGraph; fileIdx: int32; cache: IdentCache, flags
if sfMainModule in result.flags:
gMainPackageId = result.owner.id
if gCmd in {cmdCompileToC, cmdCompileToCpp, cmdCheck, cmdIdeTools}:
rd = handleSymbolFile(result, cache)
if result.id < 0:
internalError("handleSymbolFile should have set the module's ID")
return
else:
result.id = getID()
when false:
if conf.cmd in {cmdCompileToC, cmdCompileToCpp, cmdCheck, cmdIdeTools}:
rd = handleSymbolFile(result, cache)
if result.id < 0:
internalError("handleSymbolFile should have set the module's ID")
return
else:
discard
result.id = getModuleId(fileIdx, toFullPath(fileIdx))
discard processModule(graph, result,
if sfMainModule in flags and gProjectIsStdin: stdin.llStreamOpen else: nil,
if sfMainModule in flags and graph.config.projectIsStdin: stdin.llStreamOpen else: nil,
rd, cache)
#if optCaasEnabled in gGlobalOptions:
# gMemCacheData[fileIdx].needsRecompile = Recompiled
@ -188,7 +86,7 @@ proc compileModule*(graph: ModuleGraph; fileIdx: int32; cache: IdentCache, flags
initStrTable(result.tab)
result.ast = nil
discard processModule(graph, result,
if sfMainModule in flags and gProjectIsStdin: stdin.llStreamOpen else: nil,
if sfMainModule in flags and graph.config.projectIsStdin: stdin.llStreamOpen else: nil,
nil, cache)
graph.markClientsDirty(fileIdx)
when false:
@ -197,41 +95,44 @@ proc compileModule*(graph: ModuleGraph; fileIdx: int32; cache: IdentCache, flags
else:
result = gCompiledModules[fileIdx]
proc importModule*(graph: ModuleGraph; s: PSym, fileIdx: int32;
proc importModule*(graph: ModuleGraph; s: PSym, fileIdx: FileIndex;
cache: IdentCache): PSym {.procvar.} =
# this is called by the semantic checking phase
assert graph.config != nil
result = compileModule(graph, fileIdx, cache, {})
graph.addDep(s, fileIdx)
#if sfSystemModule in result.flags:
# localError(result.info, errAttemptToRedefine, result.name.s)
# restore the notes for outer module:
gNotes = if s.owner.id == gMainPackageId: gMainPackageNotes
else: ForeignPackageNotes
graph.config.notes =
if s.owner.id == gMainPackageId: graph.config.mainPackageNotes
else: graph.config.foreignPackageNotes
proc includeModule*(graph: ModuleGraph; s: PSym, fileIdx: int32;
proc includeModule*(graph: ModuleGraph; s: PSym, fileIdx: FileIndex;
cache: IdentCache): PNode {.procvar.} =
result = syntaxes.parseFile(fileIdx, cache)
result = syntaxes.parseFile(fileIdx, cache, graph.config)
graph.addDep(s, fileIdx)
graph.addIncludeDep(s.position.int32, fileIdx)
graph.addIncludeDep(s.position.FileIndex, fileIdx)
proc compileSystemModule*(graph: ModuleGraph; cache: IdentCache) =
if magicsys.systemModule == nil:
systemFileIdx = fileInfoIdx(options.libpath/"system.nim")
if graph.systemModule == nil:
systemFileIdx = fileInfoIdx(graph.config, graph.config.libpath / "system.nim")
discard graph.compileModule(systemFileIdx, cache, {sfSystemModule})
proc wantMainModule* =
if gProjectFull.len == 0:
fatal(gCmdLineInfo, errCommandExpectsFilename)
gProjectMainIdx = addFileExt(gProjectFull, NimExt).fileInfoIdx
proc wantMainModule*(conf: ConfigRef) =
if conf.projectFull.len == 0:
fatal(conf, newLineInfo(conf, "command line", 1, 1), errGenerated, "command expects a filename")
conf.projectMainIdx = int32 fileInfoIdx(conf, addFileExt(conf.projectFull, NimExt))
passes.gIncludeFile = includeModule
passes.gImportModule = importModule
proc compileProject*(graph: ModuleGraph; cache: IdentCache;
projectFileIdx = -1'i32) =
wantMainModule()
let systemFileIdx = fileInfoIdx(options.libpath / "system.nim")
let projectFile = if projectFileIdx < 0: gProjectMainIdx else: projectFileIdx
projectFileIdx = InvalidFileIDX) =
let conf = graph.config
wantMainModule(conf)
let systemFileIdx = fileInfoIdx(conf, conf.libpath / "system.nim")
let projectFile = if projectFileIdx == InvalidFileIDX: FileIndex(conf.projectMainIdx) else: projectFileIdx
graph.importStack.add projectFile
if projectFile == systemFileIdx:
discard graph.compileModule(projectFile, cache, {sfMainModule, sfSystemModule})
@ -240,7 +141,7 @@ proc compileProject*(graph: ModuleGraph; cache: IdentCache;
discard graph.compileModule(projectFile, cache, {sfMainModule})
proc makeModule*(graph: ModuleGraph; filename: string): PSym =
result = graph.newModule(fileInfoIdx filename)
result = graph.newModule(fileInfoIdx(graph.config, filename))
result.id = getID()
proc makeStdinModule*(graph: ModuleGraph): PSym = graph.makeModule"stdin"

844
compiler/msgs.nim
View file

File diff suppressed because it is too large Load diff

97
compiler/nim.nim
View file

@ -21,7 +21,7 @@ when defined(i386) and defined(windows) and defined(vcc):
import
commands, lexer, condsyms, options, msgs, nversion, nimconf, ropes,
extccomp, strutils, os, osproc, platform, main, parseopt, service,
nodejs, scriptconfig, idents, modulegraphs
nodejs, scriptconfig, idents, modulegraphs, configuration
when hasTinyCBackend:
import tccgen
@ -37,77 +37,70 @@ proc prependCurDir(f: string): string =
else:
result = f
proc handleCmdLine(cache: IdentCache; config: ConfigRef) =
proc handleCmdLine(cache: IdentCache; conf: ConfigRef) =
condsyms.initDefines(conf.symbols)
if paramCount() == 0:
writeCommandLineUsage()
writeCommandLineUsage(conf, conf.helpWritten)
else:
# Process command line arguments:
processCmdLine(passCmd1, "")
if gProjectName == "-":
gProjectName = "stdinfile"
gProjectFull = "stdinfile"
gProjectPath = canonicalizePath getCurrentDir()
gProjectIsStdin = true
elif gProjectName != "":
processCmdLine(passCmd1, "", conf)
if conf.projectName == "-":
conf.projectName = "stdinfile"
conf.projectFull = "stdinfile"
conf.projectPath = canonicalizePath(conf, getCurrentDir())
conf.projectIsStdin = true
elif conf.projectName != "":
try:
gProjectFull = canonicalizePath(gProjectName)
conf.projectFull = canonicalizePath(conf, conf.projectName)
except OSError:
gProjectFull = gProjectName
let p = splitFile(gProjectFull)
conf.projectFull = conf.projectName
let p = splitFile(conf.projectFull)
let dir = if p.dir.len > 0: p.dir else: getCurrentDir()
gProjectPath = canonicalizePath dir
gProjectName = p.name
conf.projectPath = canonicalizePath(conf, dir)
conf.projectName = p.name
else:
gProjectPath = canonicalizePath getCurrentDir()
loadConfigs(DefaultConfig, config) # load all config files
let scriptFile = gProjectFull.changeFileExt("nims")
conf.projectPath = canonicalizePath(conf, getCurrentDir())
loadConfigs(DefaultConfig, conf) # load all config files
let scriptFile = conf.projectFull.changeFileExt("nims")
if fileExists(scriptFile):
runNimScript(cache, scriptFile, freshDefines=false, config)
runNimScript(cache, scriptFile, freshDefines=false, conf)
# 'nim foo.nims' means to just run the NimScript file and do nothing more:
if scriptFile == gProjectFull: return
elif fileExists(gProjectPath / "config.nims"):
if scriptFile == conf.projectFull: return
elif fileExists(conf.projectPath / "config.nims"):
# directory wide NimScript file
runNimScript(cache, gProjectPath / "config.nims", freshDefines=false, config)
runNimScript(cache, conf.projectPath / "config.nims", freshDefines=false, conf)
# now process command line arguments again, because some options in the
# command line can overwite the config file's settings
extccomp.initVars()
processCmdLine(passCmd2, "")
if options.command == "":
rawMessage(errNoCommand, command)
mainCommand(newModuleGraph(config), cache)
if optHints in gOptions and hintGCStats in gNotes: echo(GC_getStatistics())
extccomp.initVars(conf)
processCmdLine(passCmd2, "", conf)
if conf.command == "":
rawMessage(conf, errGenerated, "command missing")
mainCommand(newModuleGraph(conf), cache)
if optHints in conf.options and hintGCStats in conf.notes: echo(GC_getStatistics())
#echo(GC_getStatistics())
if msgs.gErrorCounter == 0:
if conf.errorCounter == 0:
when hasTinyCBackend:
if gCmd == cmdRun:
tccgen.run(commands.arguments)
if optRun in gGlobalOptions:
if gCmd == cmdCompileToJS:
if conf.cmd == cmdRun:
tccgen.run(conf.arguments)
if optRun in conf.globalOptions:
if conf.cmd == cmdCompileToJS:
var ex: string
if options.outFile.len > 0:
ex = options.outFile.prependCurDir.quoteShell
if conf.outFile.len > 0:
ex = conf.outFile.prependCurDir.quoteShell
else:
ex = quoteShell(
completeCFilePath(changeFileExt(gProjectFull, "js").prependCurDir))
execExternalProgram(findNodeJs() & " " & ex & ' ' & commands.arguments)
elif gCmd == cmdCompileToPHP:
var ex: string
if options.outFile.len > 0:
ex = options.outFile.prependCurDir.quoteShell
else:
ex = quoteShell(
completeCFilePath(changeFileExt(gProjectFull, "php").prependCurDir))
execExternalProgram("php " & ex & ' ' & commands.arguments)
completeCFilePath(conf, changeFileExt(conf.projectFull, "js").prependCurDir))
execExternalProgram(conf, findNodeJs() & " " & ex & ' ' & conf.arguments)
else:
var binPath: string
if options.outFile.len > 0:
if conf.outFile.len > 0:
# If the user specified an outFile path, use that directly.
binPath = options.outFile.prependCurDir
binPath = conf.outFile.prependCurDir
else:
# Figure out ourselves a valid binary name.
binPath = changeFileExt(gProjectFull, ExeExt).prependCurDir
binPath = changeFileExt(conf.projectFull, ExeExt).prependCurDir
var ex = quoteShell(binPath)
execExternalProgram(ex & ' ' & commands.arguments)
execExternalProgram(conf, ex & ' ' & conf.arguments)
when declared(GC_setMaxPause):
GC_setMaxPause 2_000
@ -115,10 +108,10 @@ when declared(GC_setMaxPause):
when compileOption("gc", "v2") or compileOption("gc", "refc"):
# the new correct mark&sweet collector is too slow :-/
GC_disableMarkAndSweep()
condsyms.initDefines()
when not defined(selftest):
handleCmdLine(newIdentCache(), newConfigRef())
let conf = newConfigRef()
handleCmdLine(newIdentCache(), conf)
when declared(GC_setMaxPause):
echo GC_getStatistics()
msgQuit(int8(msgs.gErrorCounter > 0))
msgQuit(int8(conf.errorCounter > 0))

33
compiler/nimblecmd.nim
View file

@ -9,11 +9,12 @@
## Implements some helper procs for Nimble (Nim's package manager) support.
import parseutils, strutils, strtabs, os, options, msgs, sequtils
import parseutils, strutils, strtabs, os, options, msgs, sequtils,
configuration
proc addPath*(path: string, info: TLineInfo) =
if not options.searchPaths.contains(path):
options.searchPaths.insert(path, 0)
proc addPath*(conf: ConfigRef; path: string, info: TLineInfo) =
if not conf.searchPaths.contains(path):
conf.searchPaths.insert(path, 0)
type
Version* = distinct string
@ -84,7 +85,7 @@ proc getPathVersion*(p: string): tuple[name, version: string] =
result.name = p[0 .. sepIdx - 1]
result.version = p.substr(sepIdx + 1)
proc addPackage(packages: StringTableRef, p: string; info: TLineInfo) =
proc addPackage(conf: ConfigRef; packages: StringTableRef, p: string; info: TLineInfo) =
let (name, ver) = getPathVersion(p)
if isValidVersion(ver):
let version = newVersion(ver)
@ -92,14 +93,14 @@ proc addPackage(packages: StringTableRef, p: string; info: TLineInfo) =
(not packages.hasKey(name)):
packages[name] = $version
else:
localError(info, "invalid package name: " & p)
localError(conf, info, "invalid package name: " & p)
iterator chosen(packages: StringTableRef): string =
for key, val in pairs(packages):
let res = if val.len == 0: key else: key & '-' & val
yield res
proc addNimblePath(p: string, info: TLineInfo) =
proc addNimblePath(conf: ConfigRef; p: string, info: TLineInfo) =
var path = p
let nimbleLinks = toSeq(walkPattern(p / "*.nimble-link"))
if nimbleLinks.len > 0:
@ -111,23 +112,23 @@ proc addNimblePath(p: string, info: TLineInfo) =
if not path.isAbsolute():
path = p / path
if not contains(options.searchPaths, path):
message(info, hintPath, path)
options.lazyPaths.insert(path, 0)
if not contains(conf.searchPaths, path):
message(conf, info, hintPath, path)
conf.lazyPaths.insert(path, 0)
proc addPathRec(dir: string, info: TLineInfo) =
proc addPathRec(conf: ConfigRef; dir: string, info: TLineInfo) =
var packages = newStringTable(modeStyleInsensitive)
var pos = dir.len-1
if dir[pos] in {DirSep, AltSep}: inc(pos)
for k,p in os.walkDir(dir):
if k == pcDir and p[pos] != '.':
addPackage(packages, p, info)
addPackage(conf, packages, p, info)
for p in packages.chosen:
addNimblePath(p, info)
addNimblePath(conf, p, info)
proc nimblePath*(path: string, info: TLineInfo) =
addPathRec(path, info)
addNimblePath(path, info)
proc nimblePath*(conf: ConfigRef; path: string, info: TLineInfo) =
addPathRec(conf, path, info)
addNimblePath(conf, path, info)
when isMainModule:
proc v(s: string): Version = s.newVersion

156
compiler/nimconf.nim
View file

@ -11,7 +11,7 @@
import
llstream, nversion, commands, os, strutils, msgs, platform, condsyms, lexer,
options, idents, wordrecg, strtabs
options, idents, wordrecg, strtabs, configuration
# ---------------- configuration file parser -----------------------------
# we use Nim's scanner here to save space and work
@ -27,12 +27,12 @@ proc parseAtom(L: var TLexer, tok: var TToken; config: ConfigRef): bool =
ppGetTok(L, tok)
result = parseExpr(L, tok, config)
if tok.tokType == tkParRi: ppGetTok(L, tok)
else: lexMessage(L, errTokenExpected, "\')\'")
else: lexMessage(L, errGenerated, "expected closing ')'")
elif tok.ident.id == ord(wNot):
ppGetTok(L, tok)
result = not parseAtom(L, tok, config)
else:
result = isDefined(tok.ident)
result = isDefined(config, tok.ident.s)
ppGetTok(L, tok)
proc parseAndExpr(L: var TLexer, tok: var TToken; config: ConfigRef): bool =
@ -53,12 +53,12 @@ proc evalppIf(L: var TLexer, tok: var TToken; config: ConfigRef): bool =
ppGetTok(L, tok) # skip 'if' or 'elif'
result = parseExpr(L, tok, config)
if tok.tokType == tkColon: ppGetTok(L, tok)
else: lexMessage(L, errTokenExpected, "\':\'")
else: lexMessage(L, errGenerated, "expected ':'")
var condStack: seq[bool] = @[]
#var condStack: seq[bool] = @[]
proc doEnd(L: var TLexer, tok: var TToken) =
if high(condStack) < 0: lexMessage(L, errTokenExpected, "@if")
proc doEnd(L: var TLexer, tok: var TToken; condStack: var seq[bool]) =
if high(condStack) < 0: lexMessage(L, errGenerated, "expected @if")
ppGetTok(L, tok) # skip 'end'
setLen(condStack, high(condStack))
@ -66,20 +66,22 @@ type
TJumpDest = enum
jdEndif, jdElseEndif
proc jumpToDirective(L: var TLexer, tok: var TToken, dest: TJumpDest; config: ConfigRef)
proc doElse(L: var TLexer, tok: var TToken; config: ConfigRef) =
if high(condStack) < 0: lexMessage(L, errTokenExpected, "@if")
proc jumpToDirective(L: var TLexer, tok: var TToken, dest: TJumpDest; config: ConfigRef;
condStack: var seq[bool])
proc doElse(L: var TLexer, tok: var TToken; config: ConfigRef; condStack: var seq[bool]) =
if high(condStack) < 0: lexMessage(L, errGenerated, "expected @if")
ppGetTok(L, tok)
if tok.tokType == tkColon: ppGetTok(L, tok)
if condStack[high(condStack)]: jumpToDirective(L, tok, jdEndif, config)
if condStack[high(condStack)]: jumpToDirective(L, tok, jdEndif, config, condStack)
proc doElif(L: var TLexer, tok: var TToken; config: ConfigRef) =
if high(condStack) < 0: lexMessage(L, errTokenExpected, "@if")
proc doElif(L: var TLexer, tok: var TToken; config: ConfigRef; condStack: var seq[bool]) =
if high(condStack) < 0: lexMessage(L, errGenerated, "expected @if")
var res = evalppIf(L, tok, config)
if condStack[high(condStack)] or not res: jumpToDirective(L, tok, jdElseEndif, config)
if condStack[high(condStack)] or not res: jumpToDirective(L, tok, jdElseEndif, config, condStack)
else: condStack[high(condStack)] = true
proc jumpToDirective(L: var TLexer, tok: var TToken, dest: TJumpDest; config: ConfigRef) =
proc jumpToDirective(L: var TLexer, tok: var TToken, dest: TJumpDest; config: ConfigRef;
condStack: var seq[bool]) =
var nestedIfs = 0
while true:
if tok.ident != nil and tok.ident.s == "@":
@ -89,39 +91,39 @@ proc jumpToDirective(L: var TLexer, tok: var TToken, dest: TJumpDest; config: Co
inc(nestedIfs)
of wElse:
if dest == jdElseEndif and nestedIfs == 0:
doElse(L, tok, config)
doElse(L, tok, config, condStack)
break
of wElif:
if dest == jdElseEndif and nestedIfs == 0:
doElif(L, tok, config)
doElif(L, tok, config, condStack)
break
of wEnd:
if nestedIfs == 0:
doEnd(L, tok)
doEnd(L, tok, condStack)
break
if nestedIfs > 0: dec(nestedIfs)
else:
discard
ppGetTok(L, tok)
elif tok.tokType == tkEof:
lexMessage(L, errTokenExpected, "@end")
lexMessage(L, errGenerated, "expected @end")
else:
ppGetTok(L, tok)
proc parseDirective(L: var TLexer, tok: var TToken; config: ConfigRef) =
proc parseDirective(L: var TLexer, tok: var TToken; config: ConfigRef; condStack: var seq[bool]) =
ppGetTok(L, tok) # skip @
case whichKeyword(tok.ident)
of wIf:
setLen(condStack, len(condStack) + 1)
let res = evalppIf(L, tok, config)
condStack[high(condStack)] = res
if not res: jumpToDirective(L, tok, jdElseEndif, config)
of wElif: doElif(L, tok, config)
of wElse: doElse(L, tok, config)
of wEnd: doEnd(L, tok)
if not res: jumpToDirective(L, tok, jdElseEndif, config, condStack)
of wElif: doElif(L, tok, config, condStack)
of wElse: doElse(L, tok, config, condStack)
of wEnd: doEnd(L, tok, condStack)
of wWrite:
ppGetTok(L, tok)
msgs.msgWriteln(strtabs.`%`(tokToStr(tok), options.gConfigVars,
msgs.msgWriteln(config, strtabs.`%`(tokToStr(tok), config.configVars,
{useEnvironment, useKey}))
ppGetTok(L, tok)
else:
@ -144,60 +146,63 @@ proc parseDirective(L: var TLexer, tok: var TToken; config: ConfigRef) =
ppGetTok(L, tok)
os.putEnv(key, os.getEnv(key) & tokToStr(tok))
ppGetTok(L, tok)
else: lexMessage(L, errInvalidDirectiveX, tokToStr(tok))
else:
lexMessage(L, errGenerated, "invalid directive: '$1'" % tokToStr(tok))
proc confTok(L: var TLexer, tok: var TToken; config: ConfigRef) =
proc confTok(L: var TLexer, tok: var TToken; config: ConfigRef; condStack: var seq[bool]) =
ppGetTok(L, tok)
while tok.ident != nil and tok.ident.s == "@":
parseDirective(L, tok, config) # else: give the token to the parser
parseDirective(L, tok, config, condStack) # else: give the token to the parser
proc checkSymbol(L: TLexer, tok: TToken) =
if tok.tokType notin {tkSymbol..tkInt64Lit, tkStrLit..tkTripleStrLit}:
lexMessage(L, errIdentifierExpected, tokToStr(tok))
lexMessage(L, errGenerated, "expected identifier, but got: " & tokToStr(tok))
proc parseAssignment(L: var TLexer, tok: var TToken; config: ConfigRef) =
proc parseAssignment(L: var TLexer, tok: var TToken;
config: ConfigRef; condStack: var seq[bool]) =
if tok.ident.s == "-" or tok.ident.s == "--":
confTok(L, tok, config) # skip unnecessary prefix
confTok(L, tok, config, condStack) # skip unnecessary prefix
var info = getLineInfo(L, tok) # save for later in case of an error
checkSymbol(L, tok)
var s = tokToStr(tok)
confTok(L, tok, config) # skip symbol
confTok(L, tok, config, condStack) # skip symbol
var val = ""
while tok.tokType == tkDot:
add(s, '.')
confTok(L, tok, config)
confTok(L, tok, config, condStack)
checkSymbol(L, tok)
add(s, tokToStr(tok))
confTok(L, tok, config)
confTok(L, tok, config, condStack)
if tok.tokType == tkBracketLe:
# BUGFIX: val, not s!
# BUGFIX: do not copy '['!
confTok(L, tok, config)
confTok(L, tok, config, condStack)
checkSymbol(L, tok)
add(val, tokToStr(tok))
confTok(L, tok, config)
if tok.tokType == tkBracketRi: confTok(L, tok, config)
else: lexMessage(L, errTokenExpected, "']'")
confTok(L, tok, config, condStack)
if tok.tokType == tkBracketRi: confTok(L, tok, config, condStack)
else: lexMessage(L, errGenerated, "expected closing ']'")
add(val, ']')
let percent = tok.ident != nil and tok.ident.s == "%="
if tok.tokType in {tkColon, tkEquals} or percent:
if len(val) > 0: add(val, ':')
confTok(L, tok, config) # skip ':' or '=' or '%'
confTok(L, tok, config, condStack) # skip ':' or '=' or '%'
checkSymbol(L, tok)
add(val, tokToStr(tok))
confTok(L, tok, config) # skip symbol
confTok(L, tok, config, condStack) # skip symbol
while tok.ident != nil and tok.ident.s == "&":
confTok(L, tok, config)
confTok(L, tok, config, condStack)
checkSymbol(L, tok)
add(val, tokToStr(tok))
confTok(L, tok, config)
confTok(L, tok, config, condStack)
if percent:
processSwitch(s, strtabs.`%`(val, options.gConfigVars,
processSwitch(s, strtabs.`%`(val, config.configVars,
{useEnvironment, useEmpty}), passPP, info, config)
else:
processSwitch(s, val, passPP, info, config)
proc readConfigFile(filename: string; cache: IdentCache; config: ConfigRef) =
proc readConfigFile(
filename: string; cache: IdentCache; config: ConfigRef): bool =
var
L: TLexer
tok: TToken
@ -205,54 +210,61 @@ proc readConfigFile(filename: string; cache: IdentCache; config: ConfigRef) =
stream = llStreamOpen(filename, fmRead)
if stream != nil:
initToken(tok)
openLexer(L, filename, stream, cache)
openLexer(L, filename, stream, cache, config)
tok.tokType = tkEof # to avoid a pointless warning
confTok(L, tok, config) # read in the first token
while tok.tokType != tkEof: parseAssignment(L, tok, config)
if len(condStack) > 0: lexMessage(L, errTokenExpected, "@end")
var condStack: seq[bool] = @[]
confTok(L, tok, config, condStack) # read in the first token
while tok.tokType != tkEof: parseAssignment(L, tok, config, condStack)
if len(condStack) > 0: lexMessage(L, errGenerated, "expected @end")
closeLexer(L)
rawMessage(hintConf, filename)
return true
proc getUserConfigPath(filename: string): string =
result = joinPath(getConfigDir(), filename)
proc getSystemConfigPath(filename: string): string =
proc getSystemConfigPath(conf: ConfigRef; filename: string): string =
# try standard configuration file (installation did not distribute files
# the UNIX way)
let p = getPrefixDir()
let p = getPrefixDir(conf)
result = joinPath([p, "config", filename])
when defined(unix):
if not existsFile(result): result = joinPath([p, "etc", filename])
if not existsFile(result): result = "/etc/" & filename
proc loadConfigs*(cfg: string; cache: IdentCache; config: ConfigRef = nil) =
setDefaultLibpath()
proc loadConfigs*(cfg: string; cache: IdentCache; conf: ConfigRef) =
setDefaultLibpath(conf)
var configFiles = newSeq[string]()
if optSkipConfigFile notin gGlobalOptions:
readConfigFile(getSystemConfigPath(cfg), cache, config)
template readConfigFile(path: string) =
let configPath = path
if readConfigFile(configPath, cache, conf):
add(configFiles, configPath)
if optSkipUserConfigFile notin gGlobalOptions:
readConfigFile(getUserConfigPath(cfg), cache, config)
if optSkipConfigFile notin conf.globalOptions:
readConfigFile(getSystemConfigPath(conf, cfg))
var pd = if gProjectPath.len > 0: gProjectPath else: getCurrentDir()
if optSkipParentConfigFiles notin gGlobalOptions:
if optSkipUserConfigFile notin conf.globalOptions:
readConfigFile(getUserConfigPath(cfg))
let pd = if conf.projectPath.len > 0: conf.projectPath else: getCurrentDir()
if optSkipParentConfigFiles notin conf.globalOptions:
for dir in parentDirs(pd, fromRoot=true, inclusive=false):
readConfigFile(dir / cfg, cache, config)
readConfigFile(dir / cfg)
if optSkipProjConfigFile notin gGlobalOptions:
readConfigFile(pd / cfg, cache, config)
if optSkipProjConfigFile notin conf.globalOptions:
readConfigFile(pd / cfg)
if gProjectName.len != 0:
if conf.projectName.len != 0:
# new project wide config file:
var projectConfig = changeFileExt(gProjectFull, "nimcfg")
var projectConfig = changeFileExt(conf.projectFull, "nimcfg")
if not fileExists(projectConfig):
projectConfig = changeFileExt(gProjectFull, "nim.cfg")
if not fileExists(projectConfig):
projectConfig = changeFileExt(gProjectFull, "nimrod.cfg")
if fileExists(projectConfig):
rawMessage(warnDeprecated, projectConfig)
readConfigFile(projectConfig, cache, config)
projectConfig = changeFileExt(conf.projectFull, "nim.cfg")
readConfigFile(projectConfig)
proc loadConfigs*(cfg: string; config: ConfigRef = nil) =
for filename in configFiles:
rawMessage(conf, hintConf, filename)
proc loadConfigs*(cfg: string; conf: ConfigRef) =
# for backwards compatibility only.
loadConfigs(cfg, newIdentCache(), config)
loadConfigs(cfg, newIdentCache(), conf)

1
compiler/nimeval.nim
View file

@ -20,6 +20,7 @@ proc execute*(program: string) =
initDefines()
defineSymbol("nimrodvm")
defineSymbol("nimscript")
when hasFFI: defineSymbol("nimffi")
registerPass(verbosePass)
registerPass(semPass)

16
compiler/nimfix/nimfix.nim
View file

@ -38,7 +38,7 @@ In addition, all command line options of Nim are supported.
proc mainCommand =
registerPass verbosePass
registerPass semPass
gCmd = cmdPretty
conf.cmd = cmdPretty
searchPaths.add options.libpath
if gProjectFull.len != 0:
# current path is always looked first for modules
@ -47,7 +47,7 @@ proc mainCommand =
compileProject(newModuleGraph(), newIdentCache())
pretty.overwriteFiles()
proc processCmdLine*(pass: TCmdLinePass, cmd: string) =
proc processCmdLine*(pass: TCmdLinePass, cmd: string, config: ConfigRef) =
var p = parseopt.initOptParser(cmd)
var argsCount = 0
gOnlyMainfile = true
@ -76,16 +76,16 @@ proc processCmdLine*(pass: TCmdLinePass, cmd: string) =
of "wholeproject": gOnlyMainfile = false
of "besteffort": msgs.gErrorMax = high(int) # don't stop after first error
else:
processSwitch(pass, p)
processSwitch(pass, p, config)
of cmdArgument:
options.gProjectName = unixToNativePath(p.key)
# if processArgument(pass, p, argsCount): break
proc handleCmdLine() =
proc handleCmdLine(config: ConfigRef) =
if paramCount() == 0:
stdout.writeLine(Usage)
else:
processCmdLine(passCmd1, "")
processCmdLine(passCmd1, "", config)
if gProjectName != "":
try:
gProjectFull = canonicalizePath(gProjectName)
@ -96,11 +96,11 @@ proc handleCmdLine() =
gProjectName = p.name
else:
gProjectPath = getCurrentDir()
loadConfigs(DefaultConfig) # load all config files
loadConfigs(DefaultConfig, config) # load all config files
# now process command line arguments again, because some options in the
# command line can overwite the config file's settings
extccomp.initVars()
processCmdLine(passCmd2, "")
processCmdLine(passCmd2, "", config)
mainCommand()
when compileOption("gc", "v2") or compileOption("gc", "refc"):
@ -108,4 +108,4 @@ when compileOption("gc", "v2") or compileOption("gc", "refc"):
condsyms.initDefines()
defineSymbol "nimfix"
handleCmdline()
handleCmdline newConfigRef()

33
compiler/nimfix/pretty.nim
View file

@ -13,8 +13,9 @@
import
strutils, os, intsets, strtabs
import compiler/options, compiler/ast, compiler/astalgo, compiler/msgs,
compiler/semdata, compiler/nimfix/prettybase, compiler/ropes, compiler/idents
import ".." / [options, ast, astalgo, msgs, semdata, ropes, idents,
configuration]
import prettybase
type
StyleCheck* {.pure.} = enum None, Warn, Auto
@ -24,11 +25,11 @@ var
gStyleCheck*: StyleCheck
gCheckExtern*, gOnlyMainfile*: bool
proc overwriteFiles*() =
let doStrip = options.getConfigVar("pretty.strip").normalize == "on"
proc overwriteFiles*(conf: ConfigRef) =
let doStrip = options.getConfigVar(conf, "pretty.strip").normalize == "on"
for i in 0 .. high(gSourceFiles):
if gSourceFiles[i].dirty and not gSourceFiles[i].isNimfixFile and
(not gOnlyMainfile or gSourceFiles[i].fileIdx == gProjectMainIdx):
(not gOnlyMainfile or gSourceFiles[i].fileIdx == conf.projectMainIdx.FileIndex):
let newFile = if gOverWrite: gSourceFiles[i].fullpath
else: gSourceFiles[i].fullpath.changeFileExt(".pretty.nim")
try:
@ -41,7 +42,7 @@ proc overwriteFiles*() =
f.write(gSourceFiles[i].newline)
f.close
except IOError:
rawMessage(errCannotOpenFile, newFile)
rawMessage(conf, errGenerated, "cannot open file: " & newFile)
proc `=~`(s: string, a: openArray[string]): bool =
for x in a:
@ -95,7 +96,7 @@ proc beautifyName(s: string, k: TSymKind): string =
proc replaceInFile(info: TLineInfo; newName: string) =
loadFile(info)
let line = gSourceFiles[info.fileIndex].lines[info.line-1]
let line = gSourceFiles[info.fileIndex.int].lines[info.line.int-1]
var first = min(info.col.int, line.len)
if first < 0: return
#inc first, skipIgnoreCase(line, "proc ", first)
@ -107,28 +108,28 @@ proc replaceInFile(info: TLineInfo; newName: string) =
if differ(line, first, last, newName):
# last-first+1 != newName.len or
var x = line.substr(0, first-1) & newName & line.substr(last+1)
system.shallowCopy(gSourceFiles[info.fileIndex].lines[info.line-1], x)
gSourceFiles[info.fileIndex].dirty = true
system.shallowCopy(gSourceFiles[info.fileIndex.int].lines[info.line.int-1], x)
gSourceFiles[info.fileIndex.int].dirty = true
proc checkStyle(info: TLineInfo, s: string, k: TSymKind; sym: PSym) =
proc checkStyle(conf: ConfigRef; info: TLineInfo, s: string, k: TSymKind; sym: PSym) =
let beau = beautifyName(s, k)
if s != beau:
if gStyleCheck == StyleCheck.Auto:
sym.name = getIdent(beau)
replaceInFile(info, beau)
else:
message(info, hintName, beau)
message(conf, info, hintName, beau)
proc styleCheckDefImpl(info: TLineInfo; s: PSym; k: TSymKind) =
proc styleCheckDefImpl(conf: ConfigRef; info: TLineInfo; s: PSym; k: TSymKind) =
# operators stay as they are:
if k in {skResult, skTemp} or s.name.s[0] notin prettybase.Letters: return
if k in {skType, skGenericParam} and sfAnon in s.flags: return
if {sfImportc, sfExportc} * s.flags == {} or gCheckExtern:
checkStyle(info, s.name.s, k, s)
checkStyle(conf, info, s.name.s, k, s)
template styleCheckDef*(info: TLineInfo; s: PSym; k: TSymKind) =
when defined(nimfix):
if gStyleCheck != StyleCheck.None: styleCheckDefImpl(info, s, k)
if gStyleCheck != StyleCheck.None: styleCheckDefImpl(conf, info, s, k)
template styleCheckDef*(info: TLineInfo; s: PSym) =
styleCheckDef(info, s, s.kind)
@ -136,7 +137,7 @@ template styleCheckDef*(s: PSym) =
styleCheckDef(s.info, s, s.kind)
proc styleCheckUseImpl(info: TLineInfo; s: PSym) =
if info.fileIndex < 0: return
if info.fileIndex.int < 0: return
# we simply convert it to what it looks like in the definition
# for consistency
@ -151,4 +152,4 @@ proc styleCheckUseImpl(info: TLineInfo; s: PSym) =
template styleCheckUse*(info: TLineInfo; s: PSym) =
when defined(nimfix):
if gStyleCheck != StyleCheck.None: styleCheckUseImpl(info, s)
if gStyleCheck != StyleCheck.None: styleCheckUseImpl(conf, info, s)

18
compiler/nimfix/prettybase.nim
View file

@ -8,7 +8,7 @@
#
import strutils, lexbase, streams
import compiler/ast, compiler/msgs, compiler/idents
import ".." / [ast, msgs, idents]
from os import splitFile
type
@ -16,13 +16,13 @@ type
lines*: seq[string]
dirty*, isNimfixFile*: bool
fullpath*, newline*: string
fileIdx*: int32
fileIdx*: FileIndex
var
gSourceFiles*: seq[TSourceFile] = @[]
proc loadFile*(info: TLineInfo) =
let i = info.fileIndex
let i = info.fileIndex.int
if i >= gSourceFiles.len:
gSourceFiles.setLen(i+1)
if gSourceFiles[i].lines.isNil:
@ -64,7 +64,7 @@ proc differ*(line: string, a, b: int, x: string): bool =
proc replaceDeprecated*(info: TLineInfo; oldSym, newSym: PIdent) =
loadFile(info)
let line = gSourceFiles[info.fileIndex].lines[info.line-1]
let line = gSourceFiles[info.fileIndex.int32].lines[info.line.int-1]
var first = min(info.col.int, line.len)
if first < 0: return
#inc first, skipIgnoreCase(line, "proc ", first)
@ -75,8 +75,8 @@ proc replaceDeprecated*(info: TLineInfo; oldSym, newSym: PIdent) =
let last = first+identLen(line, first)-1
if cmpIgnoreStyle(line[first..last], oldSym.s) == 0:
var x = line.substr(0, first-1) & newSym.s & line.substr(last+1)
system.shallowCopy(gSourceFiles[info.fileIndex].lines[info.line-1], x)
gSourceFiles[info.fileIndex].dirty = true
system.shallowCopy(gSourceFiles[info.fileIndex.int32].lines[info.line.int-1], x)
gSourceFiles[info.fileIndex.int32].dirty = true
#if newSym.s == "File": writeStackTrace()
proc replaceDeprecated*(info: TLineInfo; oldSym, newSym: PSym) =
@ -85,10 +85,10 @@ proc replaceDeprecated*(info: TLineInfo; oldSym, newSym: PSym) =
proc replaceComment*(info: TLineInfo) =
loadFile(info)
let line = gSourceFiles[info.fileIndex].lines[info.line-1]
let line = gSourceFiles[info.fileIndex.int32].lines[info.line.int-1]
var first = info.col.int
if line[first] != '#': inc first
var x = line.substr(0, first-1) & "discard " & line.substr(first+1).escape
system.shallowCopy(gSourceFiles[info.fileIndex].lines[info.line-1], x)
gSourceFiles[info.fileIndex].dirty = true
system.shallowCopy(gSourceFiles[info.fileIndex.int32].lines[info.line.int-1], x)
gSourceFiles[info.fileIndex.int32].dirty = true

72
compiler/nimsets.nim
View file

@ -12,27 +12,10 @@
import
ast, astalgo, trees, nversion, msgs, platform, bitsets, types, renderer
proc toBitSet*(s: PNode, b: var TBitSet)
# this function is used for case statement checking:
proc overlap*(a, b: PNode): bool
proc inSet*(s: PNode, elem: PNode): bool
proc someInSet*(s: PNode, a, b: PNode): bool
proc emptyRange*(a, b: PNode): bool
proc setHasRange*(s: PNode): bool
# returns true if set contains a range (needed by the code generator)
# these are used for constant folding:
proc unionSets*(a, b: PNode): PNode
proc diffSets*(a, b: PNode): PNode
proc intersectSets*(a, b: PNode): PNode
proc symdiffSets*(a, b: PNode): PNode
proc containsSets*(a, b: PNode): bool
proc equalSets*(a, b: PNode): bool
proc cardSet*(s: PNode): BiggestInt
# implementation
proc inSet(s: PNode, elem: PNode): bool =
proc inSet*(s: PNode, elem: PNode): bool =
assert s.kind == nkCurly
if s.kind != nkCurly:
internalError(s.info, "inSet")
#internalError(s.info, "inSet")
return false
for i in countup(0, sonsLen(s) - 1):
if s.sons[i].kind == nkRange:
@ -44,7 +27,7 @@ proc inSet(s: PNode, elem: PNode): bool =
return true
result = false
proc overlap(a, b: PNode): bool =
proc overlap*(a, b: PNode): bool =
if a.kind == nkRange:
if b.kind == nkRange:
# X..Y and C..D overlap iff (X <= D and C <= Y)
@ -58,10 +41,11 @@ proc overlap(a, b: PNode): bool =
else:
result = sameValue(a, b)
proc someInSet(s: PNode, a, b: PNode): bool =
proc someInSet*(s: PNode, a, b: PNode): bool =
# checks if some element of a..b is in the set s
assert s.kind == nkCurly
if s.kind != nkCurly:
internalError(s.info, "SomeInSet")
#internalError(s.info, "SomeInSet")
return false
for i in countup(0, sonsLen(s) - 1):
if s.sons[i].kind == nkRange:
@ -74,7 +58,7 @@ proc someInSet(s: PNode, a, b: PNode): bool =
return true
result = false
proc toBitSet(s: PNode, b: var TBitSet) =
proc toBitSet*(s: PNode, b: var TBitSet) =
var first, j: BiggestInt
first = firstOrd(s.typ.sons[0])
bitSetInit(b, int(getSize(s.typ)))
@ -87,7 +71,7 @@ proc toBitSet(s: PNode, b: var TBitSet) =
else:
bitSetIncl(b, getOrdValue(s.sons[i]) - first)
proc toTreeSet(s: TBitSet, settype: PType, info: TLineInfo): PNode =
proc toTreeSet*(s: TBitSet, settype: PType, info: TLineInfo): PNode =
var
a, b, e, first: BiggestInt # a, b are interval borders
elemType: PType
@ -128,18 +112,18 @@ template nodeSetOp(a, b: PNode, op: untyped) {.dirty.} =
op(x, y)
result = toTreeSet(x, a.typ, a.info)
proc unionSets(a, b: PNode): PNode = nodeSetOp(a, b, bitSetUnion)
proc diffSets(a, b: PNode): PNode = nodeSetOp(a, b, bitSetDiff)
proc intersectSets(a, b: PNode): PNode = nodeSetOp(a, b, bitSetIntersect)
proc symdiffSets(a, b: PNode): PNode = nodeSetOp(a, b, bitSetSymDiff)
proc unionSets*(a, b: PNode): PNode = nodeSetOp(a, b, bitSetUnion)
proc diffSets*(a, b: PNode): PNode = nodeSetOp(a, b, bitSetDiff)
proc intersectSets*(a, b: PNode): PNode = nodeSetOp(a, b, bitSetIntersect)
proc symdiffSets*(a, b: PNode): PNode = nodeSetOp(a, b, bitSetSymDiff)
proc containsSets(a, b: PNode): bool =
proc containsSets*(a, b: PNode): bool =
var x, y: TBitSet
toBitSet(a, x)
toBitSet(b, y)
result = bitSetContains(x, y)
proc equalSets(a, b: PNode): bool =
proc equalSets*(a, b: PNode): bool =
var x, y: TBitSet
toBitSet(a, x)
toBitSet(b, y)
@ -151,26 +135,24 @@ proc complement*(a: PNode): PNode =
for i in countup(0, high(x)): x[i] = not x[i]
result = toTreeSet(x, a.typ, a.info)
proc cardSet(s: PNode): BiggestInt =
# here we can do better than converting it into a compact set
# we just count the elements directly
result = 0
for i in countup(0, sonsLen(s) - 1):
if s.sons[i].kind == nkRange:
result = result + getOrdValue(s.sons[i].sons[1]) -
getOrdValue(s.sons[i].sons[0]) + 1
else:
inc(result)
proc deduplicate*(a: PNode): PNode =
var x: TBitSet
toBitSet(a, x)
result = toTreeSet(x, a.typ, a.info)
proc setHasRange(s: PNode): bool =
proc cardSet*(a: PNode): BiggestInt =
var x: TBitSet
toBitSet(a, x)
result = bitSetCard(x)
proc setHasRange*(s: PNode): bool =
assert s.kind == nkCurly
if s.kind != nkCurly:
internalError(s.info, "SetHasRange")
return false
for i in countup(0, sonsLen(s) - 1):
if s.sons[i].kind == nkRange:
return true
result = false
proc emptyRange(a, b: PNode): bool =
proc emptyRange*(a, b: PNode): bool =
result = not leValue(a, b) # a > b iff not (a <= b)

3
compiler/nversion.nim
View file

@ -15,3 +15,6 @@ const
VersionAsString* = system.NimVersion
RodFileVersion* = "1223" # modify this if the rod-format changes!
NimCompilerApiVersion* = 1 ## Check for the existance of this before accessing it
## as older versions of the compiler API do not
## declare this.

447
compiler/options.nim
View file

@ -8,7 +8,9 @@
#
import
os, strutils, strtabs, osproc, sets
os, strutils, strtabs, osproc, sets, configuration, platform
from terminal import isatty
const
hasTinyCBackend* = defined(tinyc)
@ -17,14 +19,14 @@ const
hasFFI* = defined(useFFI)
newScopeForIf* = true
useCaas* = not defined(noCaas)
noTimeMachine* = defined(avoidTimeMachine) and defined(macosx)
copyrightYear* = "2018"
type # please make sure we have under 32 options
# (improves code efficiency a lot!)
TOption* = enum # **keep binary compatible**
optNone, optObjCheck, optFieldCheck, optRangeCheck, optBoundsCheck,
optOverflowCheck, optNilCheck,
optNaNCheck, optInfCheck,
optNaNCheck, optInfCheck, optMoveCheck,
optAssert, optLineDir, optWarns, optHints,
optOptimizeSpeed, optOptimizeSize, optStackTrace, # stack tracing support
optLineTrace, # line tracing support (includes stack tracing)
@ -35,11 +37,14 @@ type # please make sure we have under 32 options
optImplicitStatic, # optimization: implicit at compile time
# evaluation
optPatterns, # en/disable pattern matching
optMemTracker
optMemTracker,
optHotCodeReloading,
optLaxStrings
TOptions* = set[TOption]
TGlobalOption* = enum # **keep binary compatible**
gloptNone, optForceFullMake, optDeadCodeElim,
gloptNone, optForceFullMake,
optDeadCodeElimUnused, # deprecated, always on
optListCmd, optCompileOnly, optNoLinking,
optCDebug, # turn on debugging information
optGenDynLib, # generate a dynamic library
@ -67,6 +72,11 @@ type # please make sure we have under 32 options
optIdeTerse # idetools: use terse descriptions
optNoCppExceptions # use C exception handling even with CPP
optExcessiveStackTrace # fully qualified module filenames
optWholeProject # for 'doc2': output any dependency
optListFullPaths
optNoNimblePath
optDynlibOverrideAll
optUseNimNamespace
TGlobalOptions* = set[TGlobalOption]
@ -79,7 +89,6 @@ type
# **keep binary compatible**
cmdNone, cmdCompileToC, cmdCompileToCpp, cmdCompileToOC,
cmdCompileToJS,
cmdCompileToPHP,
cmdCompileToLLVM, cmdInterpret, cmdPretty, cmdDoc,
cmdGenDepend, cmdDump,
cmdCheck, # semantic checking for whole project
@ -101,67 +110,187 @@ type
ideNone, ideSug, ideCon, ideDef, ideUse, ideDus, ideChk, ideMod,
ideHighlight, ideOutline, ideKnown, ideMsg
Feature* = enum ## experimental features
implicitDeref,
dotOperators,
callOperator,
parallel,
destructor,
notnil,
oldIterTransf
SymbolFilesOption* = enum
disabledSf, enabledSf, writeOnlySf, readOnlySf, v2Sf
ConfigRef* = ref object ## eventually all global configuration should be moved here
linesCompiled*: int # all lines that have been compiled
options*: TOptions
globalOptions*: TGlobalOptions
exitcode*: int8
cmd*: TCommands # the command
selectedGC*: TGCMode # the selected GC
verbosity*: int # how verbose the compiler is
numberOfProcessors*: int # number of processors
evalExpr*: string # expression for idetools --eval
lastCmdTime*: float # when caas is enabled, we measure each command
symbolFiles*: SymbolFilesOption
cppDefines*: HashSet[string]
headerFile*: string
features*: set[Feature]
arguments*: string ## the arguments to be passed to the program that
## should be run
helpWritten*: bool
ideCmd*: IdeCmd
oldNewlines*: bool
enableNotes*: TNoteKinds
disableNotes*: TNoteKinds
foreignPackageNotes*: TNoteKinds
notes*: TNoteKinds
mainPackageNotes*: TNoteKinds
errorCounter*: int
hintCounter*: int
warnCounter*: int
errorMax*: int
configVars*: StringTableRef
symbols*: StringTableRef ## We need to use a StringTableRef here as defined
## symbols are always guaranteed to be style
## insensitive. Otherwise hell would break lose.
packageCache*: StringTableRef
searchPaths*: seq[string]
lazyPaths*: seq[string]
outFile*, prefixDir*, libpath*, nimcacheDir*: string
dllOverrides, moduleOverrides*: StringTableRef
projectName*: string # holds a name like 'nim'
projectPath*: string # holds a path like /home/alice/projects/nim/compiler/
projectFull*: string # projectPath/projectName
projectIsStdin*: bool # whether we're compiling from stdin
projectMainIdx*: int32 # the canonical path id of the main module
command*: string # the main command (e.g. cc, check, scan, etc)
commandArgs*: seq[string] # any arguments after the main command
keepComments*: bool # whether the parser needs to keep comments
implicitImports*: seq[string] # modules that are to be implicitly imported
implicitIncludes*: seq[string] # modules that are to be implicitly included
docSeeSrcUrl*: string # if empty, no seeSrc will be generated. \
# The string uses the formatting variables `path` and `line`.
const oldExperimentalFeatures* = {implicitDeref, dotOperators, callOperator, parallel}
const
ChecksOptions* = {optObjCheck, optFieldCheck, optRangeCheck, optNilCheck,
optOverflowCheck, optBoundsCheck, optAssert, optNaNCheck, optInfCheck,
optMoveCheck}
DefaultOptions* = {optObjCheck, optFieldCheck, optRangeCheck,
optBoundsCheck, optOverflowCheck, optAssert, optWarns,
optHints, optStackTrace, optLineTrace,
optPatterns, optNilCheck, optMoveCheck}
DefaultGlobalOptions* = {optThreadAnalysis}
template newPackageCache*(): untyped =
newStringTable(when FileSystemCaseSensitive:
modeCaseInsensitive
else:
modeCaseSensitive)
proc newConfigRef*(): ConfigRef =
result = ConfigRef(cppDefines: initSet[string](),
headerFile: "")
result = ConfigRef(
selectedGC: gcRefc,
verbosity: 1,
options: DefaultOptions,
globalOptions: DefaultGlobalOptions,
evalExpr: "",
cppDefines: initSet[string](),
headerFile: "", features: {}, foreignPackageNotes: {hintProcessing, warnUnknownMagic,
hintQuitCalled, hintExecuting},
notes: NotesVerbosity[1], mainPackageNotes: NotesVerbosity[1],
configVars: newStringTable(modeStyleInsensitive),
symbols: newStringTable(modeStyleInsensitive),
packageCache: newPackageCache(),
searchPaths: @[],
lazyPaths: @[],
outFile: "", prefixDir: "", libpath: "", nimcacheDir: "",
dllOverrides: newStringTable(modeCaseInsensitive),
moduleOverrides: newStringTable(modeStyleInsensitive),
projectName: "", # holds a name like 'nim'
projectPath: "", # holds a path like /home/alice/projects/nim/compiler/
projectFull: "", # projectPath/projectName
projectIsStdin: false, # whether we're compiling from stdin
projectMainIdx: 0'i32, # the canonical path id of the main module
command: "", # the main command (e.g. cc, check, scan, etc)
commandArgs: @[], # any arguments after the main command
keepComments: true, # whether the parser needs to keep comments
implicitImports: @[], # modules that are to be implicitly imported
implicitIncludes: @[], # modules that are to be implicitly included
docSeeSrcUrl: "",
arguments: ""
)
# enable colors by default on terminals
if terminal.isatty(stderr):
incl(result.globalOptions, optUseColors)
proc newPartialConfigRef*(): ConfigRef =
## create a new ConfigRef that is only good enough for error reporting.
result = ConfigRef(
selectedGC: gcRefc,
verbosity: 1,
options: DefaultOptions,
globalOptions: DefaultGlobalOptions,
foreignPackageNotes: {hintProcessing, warnUnknownMagic,
hintQuitCalled, hintExecuting},
notes: NotesVerbosity[1], mainPackageNotes: NotesVerbosity[1])
proc cppDefine*(c: ConfigRef; define: string) =
c.cppDefines.incl define
var
gIdeCmd*: IdeCmd
proc isDefined*(conf: ConfigRef; symbol: string): bool =
if conf.symbols.hasKey(symbol):
result = conf.symbols[symbol] != "false"
elif cmpIgnoreStyle(symbol, CPU[targetCPU].name) == 0:
result = true
elif cmpIgnoreStyle(symbol, platform.OS[targetOS].name) == 0:
result = true
else:
case symbol.normalize
of "x86": result = targetCPU == cpuI386
of "itanium": result = targetCPU == cpuIa64
of "x8664": result = targetCPU == cpuAmd64
of "posix", "unix":
result = targetOS in {osLinux, osMorphos, osSkyos, osIrix, osPalmos,
osQnx, osAtari, osAix,
osHaiku, osVxWorks, osSolaris, osNetbsd,
osFreebsd, osOpenbsd, osDragonfly, osMacosx,
osAndroid}
of "linux":
result = targetOS in {osLinux, osAndroid}
of "bsd":
result = targetOS in {osNetbsd, osFreebsd, osOpenbsd, osDragonfly}
of "emulatedthreadvars":
result = platform.OS[targetOS].props.contains(ospLacksThreadVars)
of "msdos": result = targetOS == osDos
of "mswindows", "win32": result = targetOS == osWindows
of "macintosh": result = targetOS in {osMacos, osMacosx}
of "sunos": result = targetOS == osSolaris
of "littleendian": result = CPU[targetCPU].endian == platform.littleEndian
of "bigendian": result = CPU[targetCPU].endian == platform.bigEndian
of "cpu8": result = CPU[targetCPU].bit == 8
of "cpu16": result = CPU[targetCPU].bit == 16
of "cpu32": result = CPU[targetCPU].bit == 32
of "cpu64": result = CPU[targetCPU].bit == 64
of "nimrawsetjmp":
result = targetOS in {osSolaris, osNetbsd, osFreebsd, osOpenbsd,
osDragonfly, osMacosx}
else: discard
const
ChecksOptions* = {optObjCheck, optFieldCheck, optRangeCheck, optNilCheck,
optOverflowCheck, optBoundsCheck, optAssert, optNaNCheck, optInfCheck}
proc importantComments*(conf: ConfigRef): bool {.inline.} = conf.cmd in {cmdDoc, cmdIdeTools}
proc usesNativeGC*(conf: ConfigRef): bool {.inline.} = conf.selectedGC >= gcRefc
var
gOptions*: TOptions = {optObjCheck, optFieldCheck, optRangeCheck,
optBoundsCheck, optOverflowCheck, optAssert, optWarns,
optHints, optStackTrace, optLineTrace,
optPatterns, optNilCheck}
gGlobalOptions*: TGlobalOptions = {optThreadAnalysis}
gExitcode*: int8
gCmd*: TCommands = cmdNone # the command
gSelectedGC* = gcRefc # the selected GC
searchPaths*: seq[string] = @[]
lazyPaths*: seq[string] = @[]
outFile*: string = ""
docSeeSrcUrl*: string = "" # if empty, no seeSrc will be generated. \
# The string uses the formatting variables `path` and `line`.
#headerFile*: string = ""
gVerbosity* = 1 # how verbose the compiler is
gNumberOfProcessors*: int # number of processors
gWholeProject*: bool # for 'doc2': output any dependency
gEvalExpr* = "" # expression for idetools --eval
gLastCmdTime*: float # when caas is enabled, we measure each command
gListFullPaths*: bool
gPreciseStack*: bool = false
gNoNimblePath* = false
gExperimentalMode*: bool
newDestructors*: bool
gDynlibOverrideAll*: bool
type
SymbolFilesOption* = enum
disabledSf, enabledSf, writeOnlySf, readOnlySf
var gSymbolFiles*: SymbolFilesOption
proc importantComments*(): bool {.inline.} = gCmd in {cmdDoc, cmdIdeTools}
proc usesNativeGC*(): bool {.inline.} = gSelectedGC >= gcRefc
template preciseStack*(): bool = gPreciseStack
template compilationCachePresent*: untyped =
gSymbolFiles in {enabledSf, writeOnlySf}
template compilationCachePresent*(conf: ConfigRef): untyped =
conf.symbolFiles in {enabledSf, writeOnlySf}
# {optCaasEnabled, optSymbolFiles} * gGlobalOptions != {}
template optPreserveOrigSource*: untyped =
optEmbedOrigSrc in gGlobalOptions
template optPreserveOrigSource*(conf: ConfigRef): untyped =
optEmbedOrigSrc in conf.globalOptions
const
genSubDir* = "nimcache"
@ -176,82 +305,62 @@ const
DocConfig* = "nimdoc.cfg"
DocTexConfig* = "nimdoc.tex.cfg"
# additional configuration variables:
var
gConfigVars* = newStringTable(modeStyleInsensitive)
gDllOverrides = newStringTable(modeCaseInsensitive)
gModuleOverrides* = newStringTable(modeStyleInsensitive)
gPrefixDir* = "" # Overrides the default prefix dir in getPrefixDir proc.
libpath* = ""
gProjectName* = "" # holds a name like 'nim'
gProjectPath* = "" # holds a path like /home/alice/projects/nim/compiler/
gProjectFull* = "" # projectPath/projectName
gProjectIsStdin* = false # whether we're compiling from stdin
gProjectMainIdx*: int32 # the canonical path id of the main module
nimcacheDir* = ""
command* = "" # the main command (e.g. cc, check, scan, etc)
commandArgs*: seq[string] = @[] # any arguments after the main command
gKeepComments*: bool = true # whether the parser needs to keep comments
implicitImports*: seq[string] = @[] # modules that are to be implicitly imported
implicitIncludes*: seq[string] = @[] # modules that are to be implicitly included
const oKeepVariableNames* = true
template compilingLib*: bool =
template compilingLib*(conf: ConfigRef): bool =
gGlobalOptions * {optGenGuiApp, optGenDynLib} != {}
proc mainCommandArg*: string =
proc mainCommandArg*(conf: ConfigRef): string =
## This is intended for commands like check or parse
## which will work on the main project file unless
## explicitly given a specific file argument
if commandArgs.len > 0:
result = commandArgs[0]
if conf.commandArgs.len > 0:
result = conf.commandArgs[0]
else:
result = gProjectName
result = conf.projectName
proc existsConfigVar*(key: string): bool =
result = hasKey(gConfigVars, key)
proc existsConfigVar*(conf: ConfigRef; key: string): bool =
result = hasKey(conf.configVars, key)
proc getConfigVar*(key: string): string =
result = gConfigVars.getOrDefault key
proc getConfigVar*(conf: ConfigRef; key: string): string =
result = conf.configVars.getOrDefault key
proc setConfigVar*(key, val: string) =
gConfigVars[key] = val
proc setConfigVar*(conf: ConfigRef; key, val: string) =
conf.configVars[key] = val
proc getOutFile*(filename, ext: string): string =
if options.outFile != "": result = options.outFile
proc getOutFile*(conf: ConfigRef; filename, ext: string): string =
if conf.outFile != "": result = conf.outFile
else: result = changeFileExt(filename, ext)
proc getPrefixDir*(): string =
proc getPrefixDir*(conf: ConfigRef): string =
## Gets the prefix dir, usually the parent directory where the binary resides.
##
## This is overridden by some tools (namely nimsuggest) via the ``gPrefixDir``
## This is overridden by some tools (namely nimsuggest) via the ``conf.prefixDir``
## global.
if gPrefixDir != "": result = gPrefixDir
else:
result = splitPath(getAppDir()).head
if conf.prefixDir != "": result = conf.prefixDir
else: result = splitPath(getAppDir()).head
proc setDefaultLibpath*() =
proc setDefaultLibpath*(conf: ConfigRef) =
# set default value (can be overwritten):
if libpath == "":
if conf.libpath == "":
# choose default libpath:
var prefix = getPrefixDir()
var prefix = getPrefixDir(conf)
when defined(posix):
if prefix == "/usr": libpath = "/usr/lib/nim"
elif prefix == "/usr/local": libpath = "/usr/local/lib/nim"
else: libpath = joinPath(prefix, "lib")
else: libpath = joinPath(prefix, "lib")
if prefix == "/usr": conf.libpath = "/usr/lib/nim"
elif prefix == "/usr/local": conf.libpath = "/usr/local/lib/nim"
else: conf.libpath = joinPath(prefix, "lib")
else: conf.libpath = joinPath(prefix, "lib")
# Special rule to support other tools (nimble) which import the compiler
# modules and make use of them.
let realNimPath = findExe("nim")
# Find out if $nim/../../lib/system.nim exists.
let parentNimLibPath = realNimPath.parentDir().parentDir() / "lib"
if not fileExists(libpath / "system.nim") and
let parentNimLibPath = realNimPath.parentDir.parentDir / "lib"
if not fileExists(conf.libpath / "system.nim") and
fileExists(parentNimlibPath / "system.nim"):
libpath = parentNimLibPath
conf.libpath = parentNimLibPath
proc canonicalizePath*(path: string): string =
proc canonicalizePath*(conf: ConfigRef; path: string): string =
# on Windows, 'expandFilename' calls getFullPathName which doesn't do
# case corrections, so we have to use this convoluted way of retrieving
# the true filename (see tests/modules and Nimble uses 'import Uri' instead
@ -263,12 +372,12 @@ proc canonicalizePath*(path: string): string =
else:
result = path.expandFilename
proc shortenDir*(dir: string): string =
proc shortenDir*(conf: ConfigRef; dir: string): string =
## returns the interesting part of a dir
var prefix = gProjectPath & DirSep
var prefix = conf.projectPath & DirSep
if startsWith(dir, prefix):
return substr(dir, len(prefix))
prefix = getPrefixDir() & DirSep
prefix = getPrefixDir(conf) & DirSep
if startsWith(dir, prefix):
return substr(dir, len(prefix))
result = dir
@ -279,114 +388,89 @@ proc removeTrailingDirSep*(path: string): string =
else:
result = path
proc disableNimblePath*() =
gNoNimblePath = true
lazyPaths.setLen(0)
proc disableNimblePath*(conf: ConfigRef) =
incl conf.globalOptions, optNoNimblePath
conf.lazyPaths.setLen(0)
include packagehandling
proc getNimcacheDir*: string =
result = if nimcacheDir.len > 0: nimcacheDir else: gProjectPath.shortenDir /
genSubDir
proc getNimcacheDir*(conf: ConfigRef): string =
result = if conf.nimcacheDir.len > 0: conf.nimcacheDir
else: shortenDir(conf, conf.projectPath) / genSubDir
proc pathSubs*(p, config: string): string =
proc pathSubs*(conf: ConfigRef; p, config: string): string =
let home = removeTrailingDirSep(os.getHomeDir())
result = unixToNativePath(p % [
"nim", getPrefixDir(),
"lib", libpath,
"nim", getPrefixDir(conf),
"lib", conf.libpath,
"home", home,
"config", config,
"projectname", options.gProjectName,
"projectpath", options.gProjectPath,
"projectdir", options.gProjectPath,
"nimcache", getNimcacheDir()])
"projectname", conf.projectName,
"projectpath", conf.projectPath,
"projectdir", conf.projectPath,
"nimcache", getNimcacheDir(conf)])
if "~/" in result:
result = result.replace("~/", home & '/')
proc toGeneratedFile*(path, ext: string): string =
proc toGeneratedFile*(conf: ConfigRef; path, ext: string): string =
## converts "/home/a/mymodule.nim", "rod" to "/home/a/nimcache/mymodule.rod"
var (head, tail) = splitPath(path)
#if len(head) > 0: head = shortenDir(head & dirSep)
result = joinPath([getNimcacheDir(), changeFileExt(tail, ext)])
result = joinPath([getNimcacheDir(conf), changeFileExt(tail, ext)])
#echo "toGeneratedFile(", path, ", ", ext, ") = ", result
when noTimeMachine:
var alreadyExcludedDirs = initSet[string]()
proc excludeDirFromTimeMachine(dir: string) {.raises: [].} =
## Calls a macosx command on the directory to exclude it from backups.
##
## The macosx tmutil command is invoked to mark the specified path as an
## item to be excluded from time machine backups. If a path already exists
## with files before excluding it, newer files won't be added to the
## directory, but previous files won't be removed from the backup until the
## user deletes that directory.
##
## The whole proc is optional and will ignore all kinds of errors. The only
## way to be sure that it works is to call ``tmutil isexcluded path``.
if alreadyExcludedDirs.contains(dir): return
alreadyExcludedDirs.incl(dir)
try:
var p = startProcess("/usr/bin/tmutil", args = ["addexclusion", dir])
discard p.waitForExit
p.close
except Exception:
discard
proc completeGeneratedFilePath*(f: string, createSubDir: bool = true): string =
proc completeGeneratedFilePath*(conf: ConfigRef; f: string, createSubDir: bool = true): string =
var (head, tail) = splitPath(f)
#if len(head) > 0: head = removeTrailingDirSep(shortenDir(head & dirSep))
var subdir = getNimcacheDir() # / head
var subdir = getNimcacheDir(conf) # / head
if createSubDir:
try:
createDir(subdir)
when noTimeMachine:
excludeDirFromTimeMachine(subdir)
except OSError:
writeLine(stdout, "cannot create directory: " & subdir)
quit(1)
result = joinPath(subdir, tail)
#echo "completeGeneratedFilePath(", f, ") = ", result
proc rawFindFile(f: string): string =
for it in searchPaths:
proc rawFindFile(conf: ConfigRef; f: string): string =
for it in conf.searchPaths:
result = joinPath(it, f)
if existsFile(result):
return result.canonicalizePath
return canonicalizePath(conf, result)
result = ""
proc rawFindFile2(f: string): string =
for i, it in lazyPaths:
proc rawFindFile2(conf: ConfigRef; f: string): string =
for i, it in conf.lazyPaths:
result = joinPath(it, f)
if existsFile(result):
# bring to front
for j in countDown(i,1):
swap(lazyPaths[j], lazyPaths[j-1])
swap(conf.lazyPaths[j], conf.lazyPaths[j-1])
return result.canonicalizePath
return canonicalizePath(conf, result)
result = ""
template patchModule() {.dirty.} =
if result.len > 0 and gModuleOverrides.len > 0:
let key = getPackageName(result) & "_" & splitFile(result).name
if gModuleOverrides.hasKey(key):
let ov = gModuleOverrides[key]
template patchModule(conf: ConfigRef) {.dirty.} =
if result.len > 0 and conf.moduleOverrides.len > 0:
let key = getPackageName(conf, result) & "_" & splitFile(result).name
if conf.moduleOverrides.hasKey(key):
let ov = conf.moduleOverrides[key]
if ov.len > 0: result = ov
proc findFile*(f: string): string {.procvar.} =
proc findFile*(conf: ConfigRef; f: string): string {.procvar.} =
if f.isAbsolute:
result = if f.existsFile: f else: ""
else:
result = f.rawFindFile
result = rawFindFile(conf, f)
if result.len == 0:
result = f.toLowerAscii.rawFindFile
result = rawFindFile(conf, f.toLowerAscii)
if result.len == 0:
result = f.rawFindFile2
result = rawFindFile2(conf, f)
if result.len == 0:
result = f.toLowerAscii.rawFindFile2
patchModule()
result = rawFindFile2(conf, f.toLowerAscii)
patchModule(conf)
proc findModule*(modulename, currentModule: string): string =
proc findModule*(conf: ConfigRef; modulename, currentModule: string): string =
# returns path to module
when defined(nimfix):
# '.nimfix' modules are preferred over '.nim' modules so that specialized
@ -396,16 +480,16 @@ proc findModule*(modulename, currentModule: string): string =
let currentPath = currentModule.splitFile.dir
result = currentPath / m
if not existsFile(result):
result = findFile(m)
result = findFile(conf, m)
if existsFile(result): return result
let m = addFileExt(modulename, NimExt)
let currentPath = currentModule.splitFile.dir
result = currentPath / m
if not existsFile(result):
result = findFile(m)
patchModule()
result = findFile(conf, m)
patchModule(conf)
proc findProjectNimFile*(pkg: string): string =
proc findProjectNimFile*(conf: ConfigRef; pkg: string): string =
const extensions = [".nims", ".cfg", ".nimcfg", ".nimble"]
var candidates: seq[string] = @[]
for k, f in os.walkDir(pkg, relative=true):
@ -430,25 +514,12 @@ proc canonDynlibName(s: string): string =
else:
result = s.substr(start)
proc inclDynlibOverride*(lib: string) =
gDllOverrides[lib.canonDynlibName] = "true"
proc inclDynlibOverride*(conf: ConfigRef; lib: string) =
conf.dllOverrides[lib.canonDynlibName] = "true"
proc isDynlibOverride*(lib: string): bool =
result = gDynlibOverrideAll or gDllOverrides.hasKey(lib.canonDynlibName)
proc binaryStrSearch*(x: openArray[string], y: string): int =
var a = 0
var b = len(x) - 1
while a <= b:
var mid = (a + b) div 2
var c = cmpIgnoreCase(x[mid], y)
if c < 0:
a = mid + 1
elif c > 0:
b = mid - 1
else:
return mid
result = - 1
proc isDynlibOverride*(conf: ConfigRef; lib: string): bool =
result = optDynlibOverrideAll in conf.globalOptions or
conf.dllOverrides.hasKey(lib.canonDynlibName)
proc parseIdeCmd*(s: string): IdeCmd =
case s:

23
compiler/packagehandling.nim
View file

@ -15,23 +15,16 @@ iterator myParentDirs(p: string): string =
if current.len == 0: break
yield current
template newPackageCache(): untyped =
newStringTable(when FileSystemCaseSensitive:
modeCaseInsensitive
else:
modeCaseSensitive)
proc resetPackageCache*(conf: ConfigRef) =
conf.packageCache = newPackageCache()
var packageCache = newPackageCache()
proc resetPackageCache*() = packageCache = newPackageCache()
proc getPackageName*(path: string): string =
proc getPackageName*(conf: ConfigRef; path: string): string =
var parents = 0
block packageSearch:
for d in myParentDirs(path):
if packageCache.hasKey(d):
if conf.packageCache.hasKey(d):
#echo "from cache ", d, " |", packageCache[d], "|", path.splitFile.name
return packageCache[d]
return conf.packageCache[d]
inc parents
for file in walkFiles(d / "*.nimble"):
result = file.splitFile.name
@ -43,12 +36,12 @@ proc getPackageName*(path: string): string =
if result.isNil: result = ""
for d in myParentDirs(path):
#echo "set cache ", d, " |", result, "|", parents
packageCache[d] = result
conf.packageCache[d] = result
dec parents
if parents <= 0: break
proc withPackageName*(path: string): string =
let x = path.getPackageName
proc withPackageName*(conf: ConfigRef; path: string): string =
let x = getPackageName(conf, path)
if x.len == 0:
result = path
else:

93
compiler/parampatterns.nim
View file

@ -10,7 +10,8 @@
## This module implements the pattern matching features for term rewriting
## macro support.
import strutils, ast, astalgo, types, msgs, idents, renderer, wordrecg, trees
import strutils, ast, astalgo, types, msgs, idents, renderer, wordrecg, trees,
options
# we precompile the pattern here for efficiency into some internal
# stack based VM :-) Why? Because it's fun; I did no benchmarks to see if that
@ -41,8 +42,8 @@ type
const
MaxStackSize* = 64 ## max required stack size by the VM
proc patternError(n: PNode) =
localError(n.info, errIllFormedAstX, renderTree(n, {renderNoComments}))
proc patternError(n: PNode; conf: ConfigRef) =
localError(conf, n.info, "illformed AST: " & renderTree(n, {renderNoComments}))
proc add(code: var TPatternCode, op: TOpcode) {.inline.} =
add(code, chr(ord(op)))
@ -53,42 +54,42 @@ proc whichAlias*(p: PSym): TAliasRequest =
else:
result = aqNone
proc compileConstraints(p: PNode, result: var TPatternCode) =
proc compileConstraints(p: PNode, result: var TPatternCode; conf: ConfigRef) =
case p.kind
of nkCallKinds:
if p.sons[0].kind != nkIdent:
patternError(p.sons[0])
patternError(p.sons[0], conf)
return
let op = p.sons[0].ident
if p.len == 3:
if op.s == "|" or op.id == ord(wOr):
compileConstraints(p.sons[1], result)
compileConstraints(p.sons[2], result)
compileConstraints(p.sons[1], result, conf)
compileConstraints(p.sons[2], result, conf)
result.add(ppOr)
elif op.s == "&" or op.id == ord(wAnd):
compileConstraints(p.sons[1], result)
compileConstraints(p.sons[2], result)
compileConstraints(p.sons[1], result, conf)
compileConstraints(p.sons[2], result, conf)
result.add(ppAnd)
else:
patternError(p)
patternError(p, conf)
elif p.len == 2 and (op.s == "~" or op.id == ord(wNot)):
compileConstraints(p.sons[1], result)
compileConstraints(p.sons[1], result, conf)
result.add(ppNot)
else:
patternError(p)
patternError(p, conf)
of nkAccQuoted, nkPar:
if p.len == 1:
compileConstraints(p.sons[0], result)
compileConstraints(p.sons[0], result, conf)
else:
patternError(p)
patternError(p, conf)
of nkIdent:
let spec = p.ident.s.normalize
case spec
of "atom": result.add(ppAtom)
of "lit": result.add(ppLit)
of "sym": result.add(ppSym)
of "atom": result.add(ppAtom)
of "lit": result.add(ppLit)
of "sym": result.add(ppSym)
of "ident": result.add(ppIdent)
of "call": result.add(ppCall)
of "call": result.add(ppCall)
of "alias": result[0] = chr(aqShouldAlias.ord)
of "noalias": result[0] = chr(aqNoAlias.ord)
of "lvalue": result.add(ppLValue)
@ -97,24 +98,24 @@ proc compileConstraints(p: PNode, result: var TPatternCode) =
of "nosideeffect": result.add(ppNoSideEffect)
else:
# check all symkinds:
internalAssert int(high(TSymKind)) < 255
internalAssert conf, int(high(TSymKind)) < 255
for i in low(TSymKind)..high(TSymKind):
if cmpIgnoreStyle(($i).substr(2), spec) == 0:
result.add(ppSymKind)
result.add(chr(i.ord))
return
# check all nodekinds:
internalAssert int(high(TNodeKind)) < 255
internalAssert conf, int(high(TNodeKind)) < 255
for i in low(TNodeKind)..high(TNodeKind):
if cmpIgnoreStyle($i, spec) == 0:
result.add(ppNodeKind)
result.add(chr(i.ord))
return
patternError(p)
patternError(p, conf)
else:
patternError(p)
patternError(p, conf)
proc semNodeKindConstraints*(p: PNode): PNode =
proc semNodeKindConstraints*(p: PNode; conf: ConfigRef): PNode =
## does semantic checking for a node kind pattern and compiles it into an
## efficient internal format.
assert p.kind == nkCurlyExpr
@ -123,11 +124,11 @@ proc semNodeKindConstraints*(p: PNode): PNode =
result.strVal.add(chr(aqNone.ord))
if p.len >= 2:
for i in 1..<p.len:
compileConstraints(p.sons[i], result.strVal)
compileConstraints(p.sons[i], result.strVal, conf)
if result.strVal.len > MaxStackSize-1:
internalError(p.info, "parameter pattern too complex")
internalError(conf, p.info, "parameter pattern too complex")
else:
patternError(p)
patternError(p, conf)
result.strVal.add(ppEof)
type
@ -178,6 +179,36 @@ type
arDiscriminant, # is a discriminant
arStrange # it is a strange beast like 'typedesc[var T]'
proc exprRoot*(n: PNode): PSym =
var it = n
while true:
case it.kind
of nkSym: return it.sym
of nkHiddenDeref, nkDerefExpr:
if it[0].typ.skipTypes(abstractInst).kind in {tyPtr, tyRef}:
# 'ptr' is unsafe anyway and 'ref' is always on the heap,
# so allow these derefs:
break
else:
it = it[0]
of nkDotExpr, nkBracketExpr, nkHiddenAddr,
nkObjUpConv, nkObjDownConv, nkCheckedFieldExpr:
it = it[0]
of nkHiddenStdConv, nkHiddenSubConv, nkConv:
it = it[1]
of nkStmtList, nkStmtListExpr:
if it.len > 0 and it.typ != nil: it = it.lastSon
else: break
of nkCallKinds:
if it.typ != nil and it.typ.kind == tyVar and it.len > 1:
# See RFC #7373, calls returning 'var T' are assumed to
# return a view into the first argument (if there is one):
it = it[1]
else:
break
else:
break
proc isAssignable*(owner: PSym, n: PNode; isUnsafeAddr=false): TAssignableResult =
## 'owner' can be nil!
result = arNone
@ -189,7 +220,7 @@ proc isAssignable*(owner: PSym, n: PNode; isUnsafeAddr=false): TAssignableResult
let kinds = if isUnsafeAddr: {skVar, skResult, skTemp, skParam, skLet}
else: {skVar, skResult, skTemp}
if n.sym.kind in kinds:
if owner != nil and owner.id == n.sym.owner.id and
if owner != nil and owner == n.sym.owner and
sfGlobal notin n.sym.flags:
result = arLocalLValue
else:
@ -205,7 +236,8 @@ proc isAssignable*(owner: PSym, n: PNode; isUnsafeAddr=false): TAssignableResult
result = arLValue
else:
result = isAssignable(owner, n.sons[0], isUnsafeAddr)
if result != arNone and sfDiscriminant in n.sons[1].sym.flags:
if result != arNone and n[1].kind == nkSym and
sfDiscriminant in n[1].sym.flags:
result = arDiscriminant
of nkBracketExpr:
if skipTypes(n.sons[0].typ, abstractInst-{tyTypeDesc}).kind in
@ -222,7 +254,10 @@ proc isAssignable*(owner: PSym, n: PNode; isUnsafeAddr=false): TAssignableResult
elif compareTypes(n.typ, n.sons[1].typ, dcEqIgnoreDistinct):
# types that are equal modulo distinction preserve l-value:
result = isAssignable(owner, n.sons[1], isUnsafeAddr)
of nkHiddenDeref, nkDerefExpr, nkHiddenAddr:
of nkHiddenDeref:
if n[0].typ.kind == tyLent: result = arDiscriminant
else: result = arLValue
of nkDerefExpr, nkHiddenAddr:
result = arLValue
of nkObjUpConv, nkObjDownConv, nkCheckedFieldExpr:
result = isAssignable(owner, n.sons[0], isUnsafeAddr)

164
compiler/parser.nim
View file

@ -17,6 +17,8 @@
# In fact the grammar is generated from this file:
when isMainModule:
# Leave a note in grammar.txt that it is generated:
#| # This file is generated by compiler/parser.nim.
import pegs
var outp = open("doc/grammar.txt", fmWrite)
for line in lines("compiler/parser.nim"):
@ -25,7 +27,7 @@ when isMainModule:
outp.close
import
llstream, lexer, idents, strutils, ast, astalgo, msgs
llstream, lexer, idents, strutils, ast, astalgo, msgs, options, configuration
type
TParser* = object # A TParser object represents a file that
@ -81,21 +83,21 @@ proc getTok(p: var TParser) =
rawGetTok(p.lex, p.tok)
p.hasProgress = true
proc openParser*(p: var TParser, fileIdx: int32, inputStream: PLLStream,
cache: IdentCache;
proc openParser*(p: var TParser, fileIdx: FileIndex, inputStream: PLLStream,
cache: IdentCache; config: ConfigRef;
strongSpaces=false) =
## Open a parser, using the given arguments to set up its internal state.
##
initToken(p.tok)
openLexer(p.lex, fileIdx, inputStream, cache)
openLexer(p.lex, fileIdx, inputStream, cache, config)
getTok(p) # read the first token
p.firstTok = true
p.strongSpaces = strongSpaces
proc openParser*(p: var TParser, filename: string, inputStream: PLLStream,
cache: IdentCache;
cache: IdentCache; config: ConfigRef;
strongSpaces=false) =
openParser(p, filename.fileInfoIdx, inputStream, cache, strongSpaces)
openParser(p, fileInfoIdx(config, filename), inputStream, cache, config, strongSpaces)
proc closeParser(p: var TParser) =
## Close a parser, freeing up its resources.
@ -105,9 +107,13 @@ proc parMessage(p: TParser, msg: TMsgKind, arg = "") =
## Produce and emit the parser message `arg` to output.
lexMessageTok(p.lex, msg, p.tok, arg)
proc parMessage(p: TParser, msg: TMsgKind, tok: TToken) =
proc parMessage(p: TParser, msg: string, tok: TToken) =
## Produce and emit a parser message to output about the token `tok`
parMessage(p, msg, prettyTok(tok))
parMessage(p, errGenerated, msg % prettyTok(tok))
proc parMessage(p: TParser, arg: string) =
## Produce and emit the parser message `arg` to output.
lexMessageTok(p.lex, errGenerated, p.tok, arg)
template withInd(p, body: untyped) =
let oldInd = p.currInd
@ -123,7 +129,13 @@ proc rawSkipComment(p: var TParser, node: PNode) =
if p.tok.tokType == tkComment:
if node != nil:
if node.comment == nil: node.comment = ""
add(node.comment, p.tok.literal)
when defined(nimpretty):
if p.tok.commentOffsetB > p.tok.commentOffsetA:
add node.comment, fileSection(p.lex.fileIdx, p.tok.commentOffsetA, p.tok.commentOffsetB)
else:
add node.comment, p.tok.literal
else:
add(node.comment, p.tok.literal)
else:
parMessage(p, errInternal, "skipComment")
getTok(p)
@ -134,6 +146,12 @@ proc skipComment(p: var TParser, node: PNode) =
proc flexComment(p: var TParser, node: PNode) =
if p.tok.indent < 0 or realInd(p): rawSkipComment(p, node)
const
errInvalidIndentation = "invalid indentation"
errIdentifierExpected = "identifier expected, but got '$1'"
errExprExpected = "expression expected, but found '$1'"
errTokenExpected = "'$1' expected"
proc skipInd(p: var TParser) =
if p.tok.indent >= 0:
if not realInd(p): parMessage(p, errInvalidIndentation)
@ -152,11 +170,11 @@ proc getTokNoInd(p: var TParser) =
proc expectIdentOrKeyw(p: TParser) =
if p.tok.tokType != tkSymbol and not isKeyword(p.tok.tokType):
lexMessage(p.lex, errIdentifierExpected, prettyTok(p.tok))
lexMessage(p.lex, errGenerated, errIdentifierExpected % prettyTok(p.tok))
proc expectIdent(p: TParser) =
if p.tok.tokType != tkSymbol:
lexMessage(p.lex, errIdentifierExpected, prettyTok(p.tok))
lexMessage(p.lex, errGenerated, errIdentifierExpected % prettyTok(p.tok))
proc eat(p: var TParser, tokType: TTokType) =
## Move the parser to the next token if the current token is of type
@ -164,7 +182,8 @@ proc eat(p: var TParser, tokType: TTokType) =
if p.tok.tokType == tokType:
getTok(p)
else:
lexMessageTok(p.lex, errTokenExpected, p.tok, TokTypeToStr[tokType])
lexMessage(p.lex, errGenerated,
"expected: '" & TokTypeToStr[tokType] & "', but got: '" & prettyTok(p.tok) & "'")
proc parLineInfo(p: TParser): TLineInfo =
## Retrieve the line information associated with the parser's current state.
@ -260,13 +279,9 @@ proc isUnary(p: TParser): bool =
proc checkBinary(p: TParser) {.inline.} =
## Check if the current parser token is a binary operator.
# we don't check '..' here as that's too annoying
if p.strongSpaces and p.tok.tokType == tkOpr:
if p.tok.tokType == tkOpr:
if p.tok.strongSpaceB > 0 and p.tok.strongSpaceA != p.tok.strongSpaceB:
parMessage(p, errGenerated,
"Number of spaces around '$#' not consistent" %
prettyTok(p.tok))
elif p.tok.strongSpaceA notin {0,1,2,4,8}:
parMessage(p, errGenerated, "Number of spaces must be 0,1,2,4 or 8")
parMessage(p, warnInconsistentSpacing, prettyTok(p.tok))
#| module = stmt ^* (';' / IND{=})
#|
@ -387,20 +402,6 @@ proc exprList(p: var TParser, endTok: TTokType, result: PNode) =
getTok(p)
optInd(p, a)
proc dotExpr(p: var TParser, a: PNode): PNode =
#| dotExpr = expr '.' optInd symbol
var info = p.parLineInfo
getTok(p)
result = newNodeI(nkDotExpr, info)
optInd(p, result)
addSon(result, a)
addSon(result, parseSymbol(p, smAfterDot))
proc qualifiedIdent(p: var TParser): PNode =
#| qualifiedIdent = symbol ('.' optInd symbol)?
result = parseSymbol(p)
if p.tok.tokType == tkDot: result = dotExpr(p, result)
proc exprColonEqExprListAux(p: var TParser, endTok: TTokType, result: PNode) =
assert(endTok in {tkCurlyRi, tkCurlyDotRi, tkBracketRi, tkParRi})
getTok(p)
@ -411,6 +412,9 @@ proc exprColonEqExprListAux(p: var TParser, endTok: TTokType, result: PNode) =
addSon(result, a)
if p.tok.tokType != tkComma: break
getTok(p)
# (1,) produces a tuple expression
if endTok == tkParRi and p.tok.tokType == tkParRi and result.kind == nkPar:
result.kind = nkTupleConstr
skipComment(p, a)
optPar(p)
eat(p, endTok)
@ -421,6 +425,33 @@ proc exprColonEqExprList(p: var TParser, kind: TNodeKind,
result = newNodeP(kind, p)
exprColonEqExprListAux(p, endTok, result)
proc dotExpr(p: var TParser, a: PNode): PNode =
#| dotExpr = expr '.' optInd (symbol | '[:' exprList ']')
#| explicitGenericInstantiation = '[:' exprList ']' ( '(' exprColonEqExpr ')' )?
var info = p.parLineInfo
getTok(p)
result = newNodeI(nkDotExpr, info)
optInd(p, result)
addSon(result, a)
addSon(result, parseSymbol(p, smAfterDot))
if p.tok.tokType == tkBracketLeColon and p.tok.strongSpaceA <= 0:
var x = newNodeI(nkBracketExpr, p.parLineInfo)
# rewrite 'x.y[:z]()' to 'y[z](x)'
x.add result[1]
exprList(p, tkBracketRi, x)
eat(p, tkBracketRi)
var y = newNodeI(nkCall, p.parLineInfo)
y.add x
y.add result[0]
if p.tok.tokType == tkParLe and p.tok.strongSpaceA <= 0:
exprColonEqExprListAux(p, tkParRi, y)
result = y
proc qualifiedIdent(p: var TParser): PNode =
#| qualifiedIdent = symbol ('.' optInd symbol)?
result = parseSymbol(p)
if p.tok.tokType == tkDot: result = dotExpr(p, result)
proc setOrTableConstr(p: var TParser): PNode =
#| setOrTableConstr = '{' ((exprColonEqExpr comma)* | ':' ) '}'
result = newNodeP(nkCurly, p)
@ -551,6 +582,9 @@ proc parsePar(p: var TParser): PNode =
if p.tok.tokType == tkComma:
getTok(p)
skipComment(p, a)
# (1,) produces a tuple expression:
if p.tok.tokType == tkParRi:
result.kind = nkTupleConstr
# progress guaranteed
while p.tok.tokType != tkParRi and p.tok.tokType != tkEof:
var a = exprColonEqExpr(p)
@ -679,10 +713,17 @@ proc namedParams(p: var TParser, callee: PNode,
# progress guaranteed
exprColonEqExprListAux(p, endTok, result)
proc commandParam(p: var TParser): PNode =
proc commandParam(p: var TParser, isFirstParam: var bool): PNode =
result = parseExpr(p)
if p.tok.tokType == tkDo:
result = postExprBlocks(p, result)
elif p.tok.tokType == tkEquals and not isFirstParam:
let lhs = result
result = newNodeP(nkExprEqExpr, p)
getTok(p)
addSon(result, lhs)
addSon(result, parseExpr(p))
isFirstParam = false
proc primarySuffix(p: var TParser, r: PNode, baseIndent: int): PNode =
#| primarySuffix = '(' (exprColonEqExpr comma?)* ')' doBlocks?
@ -717,17 +758,19 @@ proc primarySuffix(p: var TParser, r: PNode, baseIndent: int): PNode =
# progress guaranteed
somePar()
result = namedParams(p, result, nkCurlyExpr, tkCurlyRi)
of tkSymbol, tkAccent, tkIntLit..tkCharLit, tkNil, tkCast, tkAddr, tkType:
if p.inPragma == 0:
of tkSymbol, tkAccent, tkIntLit..tkCharLit, tkNil, tkCast, tkAddr, tkType,
tkOpr, tkDotDot:
if p.inPragma == 0 and (isUnary(p) or p.tok.tokType notin {tkOpr, tkDotDot}):
# actually parsing {.push hints:off.} as {.push(hints:off).} is a sweet
# solution, but pragmas.nim can't handle that
let a = result
result = newNodeP(nkCommand, p)
addSon(result, a)
var isFirstParam = true
when true:
# progress NOT guaranteed
p.hasProgress = false
addSon result, commandParam(p)
addSon result, commandParam(p, isFirstParam)
if not p.hasProgress: break
else:
while p.tok.tokType != tkEof:
@ -818,7 +861,6 @@ proc parseIfExpr(p: var TParser, kind: TNodeKind): PNode =
if realInd(p):
p.currInd = p.tok.indent
wasIndented = true
echo result.info, " yes ", p.currInd
addSon(branch, parseExpr(p))
result.add branch
while sameInd(p) or not wasIndented:
@ -855,7 +897,7 @@ proc parsePragma(p: var TParser): PNode =
skipComment(p, a)
optPar(p)
if p.tok.tokType in {tkCurlyDotRi, tkCurlyRi}: getTok(p)
else: parMessage(p, errTokenExpected, ".}")
else: parMessage(p, "expected '.}'")
dec p.inPragma
proc identVis(p: var TParser; allowDot=false): PNode =
@ -914,15 +956,15 @@ proc parseIdentColonEquals(p: var TParser, flags: TDeclaredIdentFlags): PNode =
optInd(p, result)
addSon(result, parseTypeDesc(p))
else:
addSon(result, ast.emptyNode)
addSon(result, newNodeP(nkEmpty, p))
if p.tok.tokType != tkEquals and withBothOptional notin flags:
parMessage(p, errColonOrEqualsExpected, p.tok)
parMessage(p, "':' or '=' expected, but got '$1'", p.tok)
if p.tok.tokType == tkEquals:
getTok(p)
optInd(p, result)
addSon(result, parseExpr(p))
else:
addSon(result, ast.emptyNode)
addSon(result, newNodeP(nkEmpty, p))
proc parseTuple(p: var TParser, indentAllowed = false): PNode =
#| inlTupleDecl = 'tuple'
@ -962,6 +1004,8 @@ proc parseTuple(p: var TParser, indentAllowed = false): PNode =
parMessage(p, errIdentifierExpected, p.tok)
break
if not sameInd(p): break
elif p.tok.tokType == tkParLe:
parMessage(p, errGenerated, "the syntax for tuple types is 'tuple[...]', not 'tuple(...)'")
else:
result = newNodeP(nkTupleClassTy, p)
@ -983,8 +1027,11 @@ proc parseParamList(p: var TParser, retColon = true): PNode =
a = parseIdentColonEquals(p, {withBothOptional, withPragma})
of tkParRi:
break
of tkVar:
parMessage(p, errGenerated, "the syntax is 'parameter: var T', not 'var parameter: T'")
break
else:
parMessage(p, errTokenExpected, ")")
parMessage(p, "expected closing ')'")
break
addSon(result, a)
if p.tok.tokType notin {tkComma, tkSemiColon}: break
@ -1061,6 +1108,7 @@ proc parseTypeDescKAux(p: var TParser, kind: TNodeKind,
#| distinct = 'distinct' optInd typeDesc
result = newNodeP(kind, p)
getTok(p)
if p.tok.indent != -1 and p.tok.indent <= p.currInd: return
optInd(p, result)
if not isOperator(p.tok) and isExprStart(p):
addSon(result, primary(p, mode))
@ -1144,7 +1192,7 @@ proc primary(p: var TParser, mode: TPrimaryMode): PNode =
if mode == pmTypeDef:
result = parseTypeClass(p)
else:
parMessage(p, errInvalidToken, p.tok)
parMessage(p, "the 'concept' keyword is only valid in 'type' sections")
of tkStatic:
let info = parLineInfo(p)
getTokNoInd(p)
@ -1254,7 +1302,7 @@ proc postExprBlocks(p: var TParser, x: PNode): PNode =
if nextBlock.kind == nkElse: break
else:
if openingParams.kind != nkEmpty:
parMessage(p, errTokenExpected, ":")
parMessage(p, "expected ':'")
proc parseExprStmt(p: var TParser): PNode =
#| exprStmt = simpleExpr
@ -1274,17 +1322,18 @@ proc parseExprStmt(p: var TParser): PNode =
addSon(result, b)
else:
# simpleExpr parsed 'p a' from 'p a, b'?
var isFirstParam = false
if p.tok.indent < 0 and p.tok.tokType == tkComma and a.kind == nkCommand:
result = a
while true:
getTok(p)
optInd(p, result)
addSon(result, commandParam(p))
addSon(result, commandParam(p, isFirstParam))
if p.tok.tokType != tkComma: break
elif p.tok.indent < 0 and isExprStart(p):
result = newNode(nkCommand, a.info, @[a])
while true:
addSon(result, commandParam(p))
addSon(result, commandParam(p, isFirstParam))
if p.tok.tokType != tkComma: break
getTok(p)
optInd(p, result)
@ -1489,7 +1538,7 @@ proc parseTry(p: var TParser; isExpr: bool): PNode =
addSon(b, parseStmt(p))
addSon(result, b)
if b.kind == nkFinally: break
if b == nil: parMessage(p, errTokenExpected, "except")
if b == nil: parMessage(p, "expected 'except'")
proc parseExceptBlock(p: var TParser, kind: TNodeKind): PNode =
#| exceptBlock = 'except' colcom stmt
@ -1544,7 +1593,7 @@ proc parseAsm(p: var TParser): PNode =
of tkTripleStrLit: addSon(result,
newStrNodeP(nkTripleStrLit, p.tok.literal, p))
else:
parMessage(p, errStringLiteralExpected)
parMessage(p, "the 'asm' statement takes a string literal")
addSon(result, ast.emptyNode)
return
getTok(p)
@ -1675,7 +1724,7 @@ proc parseSection(p: var TParser, kind: TNodeKind,
parMessage(p, errIdentifierExpected, p.tok)
proc parseConstant(p: var TParser): PNode =
#| constant = identWithPragma (colon typedesc)? '=' optInd expr indAndComment
#| constant = identWithPragma (colon typeDesc)? '=' optInd expr indAndComment
result = newNodeP(nkConstDef, p)
addSon(result, identWithPragma(p))
if p.tok.tokType == tkColon:
@ -1723,7 +1772,7 @@ proc parseEnum(p: var TParser): PNode =
p.tok.tokType == tkEof:
break
if result.len <= 1:
lexMessageTok(p.lex, errIdentifierExpected, p.tok, prettyTok(p.tok))
parMessage(p, errIdentifierExpected, p.tok)
proc parseObjectPart(p: var TParser): PNode
proc parseObjectWhen(p: var TParser): PNode =
@ -2086,7 +2135,7 @@ proc parseStmt(p: var TParser): PNode =
case p.tok.tokType
of tkIf, tkWhile, tkCase, tkTry, tkFor, tkBlock, tkAsm, tkProc, tkFunc,
tkIterator, tkMacro, tkType, tkConst, tkWhen, tkVar:
parMessage(p, errComplexStmtRequiresInd)
parMessage(p, "complex statement requires indentation")
result = ast.emptyNode
else:
if p.inSemiStmtList > 0:
@ -2130,7 +2179,12 @@ proc parseTopLevelStmt(p: var TParser): PNode =
if p.tok.indent != 0:
if p.firstTok and p.tok.indent < 0: discard
elif p.tok.tokType != tkSemiColon:
parMessage(p, errInvalidIndentation)
# special casing for better error messages:
if p.tok.tokType == tkOpr and p.tok.ident.s == "*":
parMessage(p, errGenerated,
"invalid indentation; an export marker '*' follows the declared identifier")
else:
parMessage(p, errInvalidIndentation)
p.firstTok = false
case p.tok.tokType
of tkSemiColon:
@ -2144,8 +2198,8 @@ proc parseTopLevelStmt(p: var TParser): PNode =
if result.kind == nkEmpty: parMessage(p, errExprExpected, p.tok)
break
proc parseString*(s: string; cache: IdentCache; filename: string = "";
line: int = 0;
proc parseString*(s: string; cache: IdentCache; config: ConfigRef;
filename: string = ""; line: int = 0;
errorHandler: TErrorHandler = nil): PNode =
## Parses a string into an AST, returning the top node.
## `filename` and `line`, although optional, provide info so that the
@ -2158,7 +2212,7 @@ proc parseString*(s: string; cache: IdentCache; filename: string = "";
# XXX for now the builtin 'parseStmt/Expr' functions do not know about strong
# spaces...
parser.lex.errorHandler = errorHandler
openParser(parser, filename, stream, cache, false)
openParser(parser, filename, stream, cache, config, false)
result = parser.parseAll
closeParser(parser)

18
compiler/passaux.nim
View file

@ -10,22 +10,26 @@
## implements some little helper passes
import
strutils, ast, astalgo, passes, idents, msgs, options, idgen
strutils, ast, astalgo, passes, idents, msgs, options, idgen, configuration
from modulegraphs import ModuleGraph
type
VerboseRef = ref object of TPassContext
config: ConfigRef
proc verboseOpen(graph: ModuleGraph; s: PSym; cache: IdentCache): PPassContext =
#MessageOut('compiling ' + s.name.s);
result = nil # we don't need a context
rawMessage(hintProcessing, s.name.s)
result = VerboseRef(config: graph.config)
rawMessage(graph.config, hintProcessing, s.name.s)
proc verboseProcess(context: PPassContext, n: PNode): PNode =
result = n
if context != nil: internalError("logpass: context is not nil")
if gVerbosity == 3:
let v = VerboseRef(context)
if v.config.verbosity == 3:
# system.nim deactivates all hints, for verbosity:3 we want the processing
# messages nonetheless, so we activate them again unconditionally:
incl(msgs.gNotes, hintProcessing)
message(n.info, hintProcessing, $idgen.gFrontendId)
incl(v.config.notes, hintProcessing)
message(v.config, n.info, hintProcessing, $idgen.gFrontendId)
const verbosePass* = makePass(open = verboseOpen, process = verboseProcess)

86
compiler/passes.nim
View file

@ -7,13 +7,14 @@
# distribution, for details about the copyright.
#
# This module implements the passes functionality. A pass must implement the
# `TPass` interface.
## This module implements the passes functionality. A pass must implement the
## `TPass` interface.
import
strutils, options, ast, astalgo, llstream, msgs, platform, os,
condsyms, idents, renderer, types, extccomp, math, magicsys, nversion,
nimsets, syntaxes, times, rodread, idgen, modulegraphs, reorder
nimsets, syntaxes, times, rodread, idgen, modulegraphs, reorder, rod,
configuration
type
@ -29,7 +30,8 @@ type
TPassProcess* = proc (p: PPassContext, topLevelStmt: PNode): PNode {.nimcall.}
TPass* = tuple[open: TPassOpen, openCached: TPassOpenCached,
process: TPassProcess, close: TPassClose]
process: TPassProcess, close: TPassClose,
isFrontend: bool]
TPassData* = tuple[input: PNode, closeOutput: PNode]
TPasses* = openArray[TPass]
@ -41,38 +43,26 @@ type
proc makePass*(open: TPassOpen = nil,
openCached: TPassOpenCached = nil,
process: TPassProcess = nil,
close: TPassClose = nil): TPass =
close: TPassClose = nil,
isFrontend = false): TPass =
result.open = open
result.openCached = openCached
result.close = close
result.process = process
result.isFrontend = isFrontend
# the semantic checker needs these:
var
gImportModule*: proc (graph: ModuleGraph; m: PSym, fileIdx: int32; cache: IdentCache): PSym {.nimcall.}
gIncludeFile*: proc (graph: ModuleGraph; m: PSym, fileIdx: int32; cache: IdentCache): PNode {.nimcall.}
gImportModule*: proc (graph: ModuleGraph; m: PSym, fileIdx: FileIndex; cache: IdentCache): PSym {.nimcall.}
gIncludeFile*: proc (graph: ModuleGraph; m: PSym, fileIdx: FileIndex; cache: IdentCache): PNode {.nimcall.}
# implementation
proc skipCodegen*(n: PNode): bool {.inline.} =
proc skipCodegen*(config: ConfigRef; n: PNode): bool {.inline.} =
# can be used by codegen passes to determine whether they should do
# something with `n`. Currently, this ignores `n` and uses the global
# error count instead.
result = msgs.gErrorCounter > 0
proc astNeeded*(s: PSym): bool =
# The ``rodwrite`` module uses this to determine if the body of a proc
# needs to be stored. The passes manager frees s.sons[codePos] when
# appropriate to free the procedure body's memory. This is important
# to keep memory usage down.
if (s.kind in {skMethod, skProc, skFunc}) and
({sfCompilerProc, sfCompileTime} * s.flags == {}) and
(s.typ.callConv != ccInline) and
(s.ast.sons[genericParamsPos].kind == nkEmpty):
result = false
# XXX this doesn't really make sense with excessive CTFE
else:
result = true
result = config.errorCounter > 0
const
maxPasses = 10
@ -150,20 +140,21 @@ proc closePassesCached(graph: ModuleGraph; a: var TPassContextArray) =
m = gPasses[i].close(graph, a[i], m)
a[i] = nil # free the memory here
proc resolveMod(module, relativeTo: string): int32 =
let fullPath = findModule(module, relativeTo)
proc resolveMod(conf: ConfigRef; module, relativeTo: string): FileIndex =
let fullPath = findModule(conf, module, relativeTo)
if fullPath.len == 0:
result = InvalidFileIDX
else:
result = fullPath.fileInfoIdx
result = fileInfoIdx(conf, fullPath)
proc processImplicits(implicits: seq[string], nodeKind: TNodeKind,
proc processImplicits(conf: ConfigRef; implicits: seq[string], nodeKind: TNodeKind,
a: var TPassContextArray; m: PSym) =
# XXX fixme this should actually be relative to the config file!
let gCmdLineInfo = newLineInfo(FileIndex(0), 1, 1)
let relativeTo = m.info.toFullPath
for module in items(implicits):
# implicit imports should not lead to a module importing itself
if m.position != resolveMod(module, relativeTo):
if m.position != resolveMod(conf, module, relativeTo).int32:
var importStmt = newNodeI(nodeKind, gCmdLineInfo)
var str = newStrNode(nkStrLit, module)
str.info = gCmdLineInfo
@ -178,26 +169,55 @@ proc processModule*(graph: ModuleGraph; module: PSym, stream: PLLStream,
a: TPassContextArray
s: PLLStream
fileIdx = module.fileIdx
if rd == nil:
if module.id < 0:
# new module caching mechanism:
for i in 0..<gPassesLen:
if not isNil(gPasses[i].open) and not gPasses[i].isFrontend:
a[i] = gPasses[i].open(graph, module, cache)
else:
a[i] = nil
var stmtIndex = 0
var doContinue = true
while doContinue:
let n = loadNode(module, stmtIndex)
if n == nil or graph.stopCompile(): break
#if n.kind == nkImportStmt:
# echo "yes and it's ", n
inc stmtIndex
var m = n
for i in 0..<gPassesLen:
if not isNil(gPasses[i].process) and not gPasses[i].isFrontend:
m = gPasses[i].process(a[i], m)
if isNil(m):
doContinue = false
break
var m: PNode = nil
for i in 0..<gPassesLen:
if not isNil(gPasses[i].close) and not gPasses[i].isFrontend:
m = gPasses[i].close(graph, a[i], m)
a[i] = nil
elif rd == nil:
openPasses(graph, a, module, cache)
if stream == nil:
let filename = fileIdx.toFullPathConsiderDirty
s = llStreamOpen(filename, fmRead)
if s == nil:
rawMessage(errCannotOpenFile, filename)
rawMessage(graph.config, errCannotOpenFile, filename)
return false
else:
s = stream
while true:
openParsers(p, fileIdx, s, cache)
openParsers(p, fileIdx, s, cache, graph.config)
if sfSystemModule notin module.flags:
# XXX what about caching? no processing then? what if I change the
# modules to include between compilation runs? we'd need to track that
# in ROD files. I think we should enable this feature only
# for the interactive mode.
processImplicits implicitImports, nkImportStmt, a, module
processImplicits implicitIncludes, nkIncludeStmt, a, module
processImplicits graph.config, graph.config.implicitImports, nkImportStmt, a, module
processImplicits graph.config, graph.config.implicitIncludes, nkIncludeStmt, a, module
while true:
if graph.stopCompile(): break

4
compiler/patterns.nim
View file

@ -150,7 +150,7 @@ proc matches(c: PPatternContext, p, n: PNode): bool =
of "*": result = matchNested(c, p, n, rpn=false)
of "**": result = matchNested(c, p, n, rpn=true)
of "~": result = not matches(c, p.sons[1], n)
else: internalError(p.info, "invalid pattern")
else: doAssert(false, "invalid pattern")
# template {add(a, `&` * b)}(a: string{noalias}, b: varargs[string]) =
# add(a, b)
elif p.kind == nkCurlyExpr:
@ -289,7 +289,7 @@ proc applyRule*(c: PContext, s: PSym, n: PNode): PNode =
# constraint not fulfilled:
if not ok: return nil
markUsed(n.info, s, c.graph.usageSym)
markUsed(c.config, n.info, s, c.graph.usageSym)
if ctx.subMatch:
assert m.len == 3
m.sons[1] = result

1790
compiler/pbraces.nim
View file

File diff suppressed because it is too large Load diff

5
compiler/platform.nim
View file

@ -176,7 +176,7 @@ type
cpuNone, cpuI386, cpuM68k, cpuAlpha, cpuPowerpc, cpuPowerpc64,
cpuPowerpc64el, cpuSparc, cpuVm, cpuIa64, cpuAmd64, cpuMips, cpuMipsel,
cpuArm, cpuArm64, cpuJS, cpuNimrodVM, cpuAVR, cpuMSP430, cpuSparc64,
cpuMips64, cpuMips64el
cpuMips64, cpuMips64el, cpuRiscV64
type
TEndian* = enum
@ -207,7 +207,8 @@ const
(name: "msp430", intSize: 16, endian: littleEndian, floatSize: 32, bit: 16),
(name: "sparc64", intSize: 64, endian: bigEndian, floatSize: 64, bit: 64),
(name: "mips64", intSize: 64, endian: bigEndian, floatSize: 64, bit: 64),
(name: "mips64el", intSize: 64, endian: littleEndian, floatSize: 64, bit: 64)]
(name: "mips64el", intSize: 64, endian: littleEndian, floatSize: 64, bit: 64),
(name: "riscv64", intSize: 64, endian: littleEndian, floatSize: 64, bit: 64)]
var
targetCPU*, hostCPU*: TSystemCPU

2
compiler/plugins/active.nim
View file

@ -10,4 +10,4 @@
## Include file that imports all plugins that are active.
import
locals.locals, itersgen
locals / locals, itersgen

17
compiler/plugins/itersgen.nim
View file

@ -9,30 +9,29 @@
## Plugin to transform an inline iterator into a data structure.
import compiler/pluginsupport, compiler/ast, compiler/astalgo,
compiler/magicsys, compiler/lookups, compiler/semdata,
compiler/lambdalifting, compiler/rodread, compiler/msgs
import ".." / [pluginsupport, ast, astalgo,
magicsys, lookups, semdata,
lambdalifting, rodread, msgs]
proc iterToProcImpl(c: PContext, n: PNode): PNode =
result = newNodeI(nkStmtList, n.info)
let iter = n[1]
if iter.kind != nkSym or iter.sym.kind != skIterator:
localError(iter.info, "first argument needs to be an iterator")
localError(c.config, iter.info, "first argument needs to be an iterator")
return
if n[2].typ.isNil:
localError(n[2].info, "second argument needs to be a type")
localError(c.config, n[2].info, "second argument needs to be a type")
return
if n[3].kind != nkIdent:
localError(n[3].info, "third argument needs to be an identifier")
localError(c.config, n[3].info, "third argument needs to be an identifier")
return
let t = n[2].typ.skipTypes({tyTypeDesc, tyGenericInst})
if t.kind notin {tyRef, tyPtr} or t.lastSon.kind != tyObject:
localError(n[2].info,
localError(c.config, n[2].info,
"type must be a non-generic ref|ptr to object with state field")
return
let body = liftIterToProc(iter.sym, iter.sym.getBody, t)
let body = liftIterToProc(c.graph, iter.sym, iter.sym.getBody, t)
let prc = newSym(skProc, n[3].ident, iter.sym.owner, iter.sym.info)
prc.typ = copyType(iter.sym.typ, prc, false)

4
compiler/plugins/locals/locals.nim
View file

@ -9,8 +9,8 @@
## The builtin 'system.locals' implemented as a plugin.
import compiler/pluginsupport, compiler/ast, compiler/astalgo,
compiler/magicsys, compiler/lookups, compiler/semdata, compiler/lowerings
import "../../" / [pluginsupport, ast, astalgo,
magicsys, lookups, semdata, lowerings]
proc semLocals(c: PContext, n: PNode): PNode =
var counter = 0

651
compiler/pragmas.nim
View file

File diff suppressed because it is too large Load diff

21
compiler/procfind.nim
View file

@ -14,14 +14,12 @@ import
ast, astalgo, msgs, semdata, types, trees, strutils
proc equalGenericParams(procA, procB: PNode): bool =
if sonsLen(procA) != sonsLen(procB): return
if sonsLen(procA) != sonsLen(procB): return false
for i in countup(0, sonsLen(procA) - 1):
if procA.sons[i].kind != nkSym:
internalError(procA.info, "equalGenericParams")
return
return false
if procB.sons[i].kind != nkSym:
internalError(procB.info, "equalGenericParams")
return
return false
let a = procA.sons[i].sym
let b = procB.sons[i].sym
if a.name.id != b.name.id or
@ -57,7 +55,7 @@ proc searchForProcOld*(c: PContext, scope: PScope, fn: PSym): PSym =
of paramsEqual:
return
of paramsIncompatible:
localError(fn.info, errNotOverloadable, fn.name.s)
localError(c.config, fn.info, "overloaded '$1' leads to ambiguous calls" % fn.name.s)
return
of paramsNotEqual:
discard
@ -66,30 +64,25 @@ proc searchForProcOld*(c: PContext, scope: PScope, fn: PSym): PSym =
proc searchForProcNew(c: PContext, scope: PScope, fn: PSym): PSym =
const flags = {ExactGenericParams, ExactTypeDescValues,
ExactConstraints, IgnoreCC}
var it: TIdentIter
result = initIdentIter(it, scope.symbols, fn.name)
while result != nil:
if result.kind == fn.kind and sameType(result.typ, fn.typ, flags):
if result.kind == fn.kind: #and sameType(result.typ, fn.typ, flags):
case equalParams(result.typ.n, fn.typ.n)
of paramsEqual:
if (sfExported notin result.flags) and (sfExported in fn.flags):
let message = ("public implementation '$1' has non-public " &
"forward declaration in $2") %
[getProcHeader(result), $result.info]
localError(fn.info, errGenerated, message)
localError(c.config, fn.info, message)
return
of paramsIncompatible:
localError(fn.info, errNotOverloadable, fn.name.s)
localError(c.config, fn.info, "overloaded '$1' leads to ambiguous calls" % fn.name.s)
return
of paramsNotEqual:
discard
result = nextIdentIter(it, scope.symbols)
return nil
proc searchForProc*(c: PContext, scope: PScope, fn: PSym): PSym =
result = searchForProcNew(c, scope, fn)
when false:

118
compiler/renderer.nim
View file

@ -10,7 +10,7 @@
# This module implements the renderer of the standard Nim representation.
import
lexer, options, idents, strutils, ast, msgs
lexer, options, idents, strutils, ast, msgs, configuration
type
TRenderFlag* = enum
@ -39,8 +39,8 @@ type
inPragma: int
when defined(nimpretty):
pendingNewlineCount: int
origContent: string
fid*: FileIndex
config*: ConfigRef
# We render the source code in a two phases: The first
# determines how long the subtree will likely be, the second
@ -91,7 +91,7 @@ const
MaxLineLen = 80
LineCommentColumn = 30
proc initSrcGen(g: var TSrcGen, renderFlags: TRenderFlags) =
proc initSrcGen(g: var TSrcGen, renderFlags: TRenderFlags; config: ConfigRef) =
g.comStack = @[]
g.tokens = @[]
g.indent = 0
@ -103,6 +103,7 @@ proc initSrcGen(g: var TSrcGen, renderFlags: TRenderFlags) =
g.pendingNL = -1
g.pendingWhitespace = -1
g.inGenericParams = false
g.config = config
proc addTok(g: var TSrcGen, kind: TTokType, s: string) =
var length = len(g.tokens)
@ -173,34 +174,14 @@ proc put(g: var TSrcGen, kind: TTokType, s: string) =
else:
g.pendingWhitespace = s.len
proc toNimChar(c: char): string =
case c
of '\0': result = "\\x00" # not "\\0" to avoid ambiguous cases like "\\012".
of '\a': result = "\\a" # \x07
of '\b': result = "\\b" # \x08
of '\t': result = "\\t" # \x09
of '\L': result = "\\L" # \x0A
of '\v': result = "\\v" # \x0B
of '\f': result = "\\f" # \x0C
of '\c': result = "\\c" # \x0D
of '\e': result = "\\e" # \x1B
of '\x01'..'\x06', '\x0E'..'\x1A', '\x1C'..'\x1F', '\x80'..'\xFF':
result = "\\x" & strutils.toHex(ord(c), 2)
of '\'', '\"', '\\': result = '\\' & c
else: result = c & ""
proc makeNimString(s: string): string =
result = "\""
for i in countup(0, len(s)-1): add(result, toNimChar(s[i]))
add(result, '\"')
proc putComment(g: var TSrcGen, s: string) =
if s.isNil: return
var i = 0
let hi = len(s) - 1
var isCode = (len(s) >= 2) and (s[1] != ' ')
var ind = g.lineLen
var com = "## "
while true:
while i <= hi:
case s[i]
of '\0':
break
@ -208,7 +189,7 @@ proc putComment(g: var TSrcGen, s: string) =
put(g, tkComment, com)
com = "## "
inc(i)
if s[i] == '\x0A': inc(i)
if i < s.len and s[i] == '\x0A': inc(i)
optNL(g, ind)
of '\x0A':
put(g, tkComment, com)
@ -223,12 +204,12 @@ proc putComment(g: var TSrcGen, s: string) =
# gets too long:
# compute length of the following word:
var j = i
while s[j] > ' ': inc(j)
while j <= hi and s[j] > ' ': inc(j)
if not isCode and (g.lineLen + (j - i) > MaxLineLen):
put(g, tkComment, com)
optNL(g, ind)
com = "## "
while s[i] > ' ':
while i <= hi and s[i] > ' ':
add(com, s[i])
inc(i)
put(g, tkComment, com)
@ -237,8 +218,9 @@ proc putComment(g: var TSrcGen, s: string) =
proc maxLineLength(s: string): int =
if s.isNil: return 0
var i = 0
let hi = len(s) - 1
var lineLen = 0
while true:
while i <= hi:
case s[i]
of '\0':
break
@ -257,7 +239,7 @@ proc maxLineLength(s: string): int =
proc putRawStr(g: var TSrcGen, kind: TTokType, s: string) =
var i = 0
var hi = len(s) - 1
let hi = len(s) - 1
var str = ""
while i <= hi:
case s[i]
@ -325,8 +307,8 @@ proc lsub(g: TSrcGen; n: PNode): int
proc litAux(g: TSrcGen; n: PNode, x: BiggestInt, size: int): string =
proc skip(t: PType): PType =
result = t
while result.kind in {tyGenericInst, tyRange, tyVar, tyDistinct,
tyOrdinal, tyAlias}:
while result.kind in {tyGenericInst, tyRange, tyVar, tyLent, tyDistinct,
tyOrdinal, tyAlias, tySink}:
result = lastSon(result)
if n.typ != nil and n.typ.skip.kind in {tyBool, tyEnum}:
let enumfields = n.typ.skip.n
@ -349,22 +331,25 @@ proc ulitAux(g: TSrcGen; n: PNode, x: BiggestInt, size: int): string =
proc atom(g: TSrcGen; n: PNode): string =
when defined(nimpretty):
let comment = if n.info.commentOffsetA < n.info.commentOffsetB:
" " & substr(g.origContent, n.info.commentOffsetA, n.info.commentOffsetB)
" " & fileSection(g.fid, n.info.commentOffsetA, n.info.commentOffsetB)
else:
""
if n.info.offsetA <= n.info.offsetB:
# for some constructed tokens this can not be the case and we're better
# off to not mess with the offset then.
return substr(g.origContent, n.info.offsetA, n.info.offsetB) & comment
return fileSection(g.fid, n.info.offsetA, n.info.offsetB) & comment
var f: float32
case n.kind
of nkEmpty: result = ""
of nkIdent: result = n.ident.s
of nkSym: result = n.sym.name.s
of nkStrLit: result = makeNimString(n.strVal)
of nkRStrLit: result = "r\"" & replace(n.strVal, "\"", "\"\"") & '\"'
of nkStrLit: result = ""; result.addQuoted(n.strVal)
of nkRStrLit: result = "r\"" & replace(n.strVal, "\"", "\"\"") & '\"'
of nkTripleStrLit: result = "\"\"\"" & n.strVal & "\"\"\""
of nkCharLit: result = '\'' & toNimChar(chr(int(n.intVal))) & '\''
of nkCharLit:
result = "\'"
result.addEscapedChar(chr(int(n.intVal)));
result.add '\''
of nkIntLit: result = litAux(g, n, n.intVal, 4)
of nkInt8Lit: result = litAux(g, n, n.intVal, 1) & "\'i8"
of nkInt16Lit: result = litAux(g, n, n.intVal, 2) & "\'i16"
@ -394,7 +379,7 @@ proc atom(g: TSrcGen; n: PNode): string =
if (n.typ != nil) and (n.typ.sym != nil): result = n.typ.sym.name.s
else: result = "[type node]"
else:
internalError("rnimsyn.atom " & $n.kind)
internalError(g.config, "rnimsyn.atom " & $n.kind)
result = ""
proc lcomma(g: TSrcGen; n: PNode, start: int = 0, theEnd: int = - 1): int =
@ -432,6 +417,9 @@ proc lsub(g: TSrcGen; n: PNode): int =
of nkCommand: result = lsub(g, n.sons[0]) + lcomma(g, n, 1) + 1
of nkExprEqExpr, nkAsgn, nkFastAsgn: result = lsons(g, n) + 3
of nkPar, nkCurly, nkBracket, nkClosure: result = lcomma(g, n) + 2
of nkTupleConstr:
# assume the trailing comma:
result = lcomma(g, n) + 3
of nkArgList: result = lcomma(g, n)
of nkTableConstr:
result = if n.len > 0: lcomma(g, n) + 2 else: len("{:}")
@ -898,6 +886,14 @@ proc gsub(g: var TSrcGen, n: PNode, c: TContext) =
put(g, tkBracketLe, "[")
gcomma(g, n, 2)
put(g, tkBracketRi, "]")
elif n.len > 1 and n.lastSon.kind == nkStmtList:
gsub(g, n[0])
if n.len > 2:
put(g, tkParLe, "(")
gcomma(g, n, 1, -2)
put(g, tkParRi, ")")
put(g, tkColon, ":")
gsub(g, n, n.len-1)
else:
if sonsLen(n) >= 1: gsub(g, n.sons[0])
put(g, tkParLe, "(")
@ -994,6 +990,11 @@ proc gsub(g: var TSrcGen, n: PNode, c: TContext) =
put(g, tkParLe, "(")
gcomma(g, n, c)
put(g, tkParRi, ")")
of nkTupleConstr:
put(g, tkParLe, "(")
gcomma(g, n, c)
if n.len == 1: put(g, tkComma, ",")
put(g, tkParRi, ")")
of nkCurly:
put(g, tkCurlyLe, "{")
gcomma(g, n, c)
@ -1067,6 +1068,7 @@ proc gsub(g: var TSrcGen, n: PNode, c: TContext) =
if n.len > 1:
let opr = if n[0].kind == nkIdent: n[0].ident
elif n[0].kind == nkSym: n[0].sym.name
elif n[0].kind in {nkOpenSymChoice, nkClosedSymChoice}: n[0][0].sym.name
else: nil
if n[1].kind == nkPrefix or (opr != nil and renderer.isKeyword(opr)):
put(g, tkSpaces, Space)
@ -1411,22 +1413,32 @@ proc gsub(g: var TSrcGen, n: PNode, c: TContext) =
put(g, tkParLe, "(ComesFrom|")
gsub(g, n, 0)
put(g, tkParRi, ")")
of nkGotoState, nkState:
of nkGotoState:
var c: TContext
initContext c
putWithSpace g, tkSymbol, if n.kind == nkState: "state" else: "goto"
putWithSpace g, tkSymbol, "goto"
gsons(g, n, c)
of nkState:
var c: TContext
initContext c
putWithSpace g, tkSymbol, "state"
gsub(g, n[0], c)
putWithSpace(g, tkColon, ":")
indentNL(g)
gsons(g, n, c, 1)
dedent(g)
of nkBreakState:
put(g, tkTuple, "breakstate")
of nkTypeClassTy:
gTypeClassTy(g, n)
else:
#nkNone, nkExplicitTypeListCall:
internalError(n.info, "rnimsyn.gsub(" & $n.kind & ')')
internalError(g.config, n.info, "rnimsyn.gsub(" & $n.kind & ')')
proc renderTree*(n: PNode, renderFlags: TRenderFlags = {}): string =
var g: TSrcGen
initSrcGen(g, renderFlags)
initSrcGen(g, renderFlags, newPartialConfigRef())
# do not indent the initial statement list so that
# writeFile("file.nim", repr n)
# produces working Nim code:
@ -1439,17 +1451,13 @@ proc renderTree*(n: PNode, renderFlags: TRenderFlags = {}): string =
proc `$`*(n: PNode): string = n.renderTree
proc renderModule*(n: PNode, infile, outfile: string,
renderFlags: TRenderFlags = {}) =
renderFlags: TRenderFlags = {};
fid = FileIndex(-1)) =
var
f: File
g: TSrcGen
initSrcGen(g, renderFlags)
when defined(nimpretty):
try:
g.origContent = readFile(infile)
except IOError:
rawMessage(errCannotOpenFile, infile)
initSrcGen(g, renderFlags, newPartialConfigRef())
g.fid = fid
for i in countup(0, sonsLen(n) - 1):
gsub(g, n.sons[i])
optNL(g)
@ -1458,16 +1466,14 @@ proc renderModule*(n: PNode, infile, outfile: string,
nkCommentStmt: putNL(g)
else: discard
gcoms(g)
if optStdout in gGlobalOptions:
write(stdout, g.buf)
elif open(f, outfile, fmWrite):
if open(f, outfile, fmWrite):
write(f, g.buf)
close(f)
else:
rawMessage(errCannotOpenFile, outfile)
rawMessage(g.config, errGenerated, "cannot open file: " & outfile)
proc initTokRender*(r: var TSrcGen, n: PNode, renderFlags: TRenderFlags = {}) =
initSrcGen(r, renderFlags)
initSrcGen(r, renderFlags, newPartialConfigRef())
gsub(r, n)
proc getNextTok*(r: var TSrcGen, kind: var TTokType, literal: var string) =

39
compiler/reorder.nim
View file

@ -1,7 +1,8 @@
import
intsets, ast, idents, algorithm, renderer, parser, ospaths, strutils,
sequtils, msgs, modulegraphs, syntaxes, options, modulepaths, tables
import
intsets, ast, idents, algorithm, renderer, parser, ospaths, strutils,
sequtils, msgs, modulegraphs, syntaxes, options, modulepaths, tables,
configuration
type
DepN = ref object
@ -135,13 +136,13 @@ proc hasIncludes(n:PNode): bool =
if a.kind == nkIncludeStmt:
return true
proc includeModule*(graph: ModuleGraph; s: PSym, fileIdx: int32;
proc includeModule*(graph: ModuleGraph; s: PSym, fileIdx: FileIndex;
cache: IdentCache): PNode {.procvar.} =
result = syntaxes.parseFile(fileIdx, cache)
result = syntaxes.parseFile(fileIdx, cache, graph.config)
graph.addDep(s, fileIdx)
graph.addIncludeDep(s.position.int32, fileIdx)
graph.addIncludeDep(FileIndex s.position, fileIdx)
proc expandIncludes(graph: ModuleGraph, module: PSym, n: PNode,
proc expandIncludes(graph: ModuleGraph, module: PSym, n: PNode,
modulePath: string, includedFiles: var IntSet,
cache: IdentCache): PNode =
# Parses includes and injects them in the current tree
@ -151,15 +152,15 @@ proc expandIncludes(graph: ModuleGraph, module: PSym, n: PNode,
for a in n:
if a.kind == nkIncludeStmt:
for i in 0..<a.len:
var f = checkModuleName(a.sons[i])
var f = checkModuleName(graph.config, a.sons[i])
if f != InvalidFileIDX:
if containsOrIncl(includedFiles, f):
localError(a.info, errRecursiveDependencyX, f.toFilename)
if containsOrIncl(includedFiles, f.int):
localError(graph.config, a.info, "recursive dependency: '$1'" % f.toFilename)
else:
let nn = includeModule(graph, module, f, cache)
let nnn = expandIncludes(graph, module, nn, modulePath,
let nnn = expandIncludes(graph, module, nn, modulePath,
includedFiles, cache)
excl(includedFiles, f)
excl(includedFiles, f.int)
for b in nnn:
result.add b
else:
@ -189,7 +190,7 @@ proc haveSameKind(dns: seq[DepN]): bool =
if dn.pnode.kind != kind:
return false
proc mergeSections(comps: seq[seq[DepN]], res: PNode) =
proc mergeSections(conf: ConfigRef; comps: seq[seq[DepN]], res: PNode) =
# Merges typeSections and ConstSections when they form
# a strong component (ex: circular type definition)
for c in comps:
@ -229,7 +230,7 @@ proc mergeSections(comps: seq[seq[DepN]], res: PNode) =
wmsg &= "line " & $cs[^1].pnode.info.line &
" depends on line " & $cs[j].pnode.info.line &
": " & cs[^1].expls[ci] & "\n"
message(cs[0].pnode.info, warnUser, wmsg)
message(conf, cs[0].pnode.info, warnUser, wmsg)
var i = 0
while i < cs.len:
@ -273,9 +274,9 @@ proc hasCommand(n: PNode): bool =
of nkStmtList, nkStmtListExpr, nkWhenStmt, nkElifBranch, nkElse,
nkStaticStmt, nkLetSection, nkConstSection, nkVarSection,
nkIdentDefs:
for a in n:
if a.hasCommand:
return true
for a in n:
if a.hasCommand:
return true
else:
return false
@ -430,7 +431,7 @@ proc reorder*(graph: ModuleGraph, n: PNode, module: PSym, cache: IdentCache): PN
return n
var includedFiles = initIntSet()
let mpath = module.fileIdx.toFullPath
let n = expandIncludes(graph, module, n, mpath,
let n = expandIncludes(graph, module, n, mpath,
includedFiles, cache).splitSections
result = newNodeI(nkStmtList, n.info)
var deps = newSeq[(IntSet, IntSet)](n.len)
@ -441,4 +442,4 @@ proc reorder*(graph: ModuleGraph, n: PNode, module: PSym, cache: IdentCache): PN
var g = buildGraph(n, deps)
let comps = getStrongComponents(g)
mergeSections(comps, result)
mergeSections(graph.config, comps, result)

26
compiler/rod.nim Normal file
View file

@ -0,0 +1,26 @@
#
#
# The Nim Compiler
# (c) Copyright 2017 Andreas Rumpf
#
# See the file "copying.txt", included in this
# distribution, for details about the copyright.
#
## This module implements the canonalization for the various caching mechanisms.
import ast, idgen, msgs
when not defined(nimSymbolfiles):
template setupModuleCache* = discard
template storeNode*(module: PSym; n: PNode) = discard
template loadNode*(module: PSym; index: var int): PNode = PNode(nil)
template getModuleId*(fileIdx: FileIndex; fullpath: string): int = getID()
template addModuleDep*(module, fileIdx: FileIndex; isIncludeFile: bool) = discard
template storeRemaining*(module: PSym) = discard
else:
include rodimpl

947
compiler/rodimpl.nim Normal file
View file

@ -0,0 +1,947 @@
#
#
# The Nim Compiler
# (c) Copyright 2018 Andreas Rumpf
#
# See the file "copying.txt", included in this
# distribution, for details about the copyright.
#
## This module implements the new compilation cache.
import strutils, os, intsets, tables, ropes, db_sqlite, msgs, options, types,
renderer, rodutils, std / sha1, idents, astalgo, magicsys
## Todo:
## - Implement the 'import' replay logic so that the codegen runs over
## dependent modules.
## - Make conditional symbols and the configuration part of a module's
## dependencies.
## - Test multi methods.
## - Implement the limited VM support based on sets.
## - Depencency computation should use signature hashes in order to
## avoid recompiling dependent modules.
var db: DbConn
proc hashFileCached(fileIdx: int32; fullpath: string): string =
result = msgs.getHash(fileIdx)
if result.len == 0:
result = $secureHashFile(fullpath)
msgs.setHash(fileIdx, result)
proc needsRecompile(fileIdx: int32; fullpath: string; cycleCheck: var IntSet): bool =
let root = db.getRow(sql"select id, fullhash from filenames where fullpath = ?",
fullpath)
if root[0].len == 0: return true
if root[1] != hashFileCached(fileIdx, fullpath):
return true
# cycle detection: assume "not changed" is correct.
if cycleCheck.containsOrIncl(fileIdx):
return false
# check dependencies (recursively):
for row in db.fastRows(sql"select fullpath from filenames where id in (select dependency from deps where module = ?)",
root[0]):
let dep = row[0]
if needsRecompile(dep.fileInfoIdx, dep, cycleCheck):
return true
return false
proc getModuleId*(fileIdx: int32; fullpath: string): int =
if gSymbolFiles != v2Sf: return getID()
let module = db.getRow(
sql"select id, fullHash from modules where fullpath = ?", fullpath)
let currentFullhash = hashFileCached(fileIdx, fullpath)
if module[0].len == 0:
result = int db.insertID(sql"insert into modules(fullpath, interfHash, fullHash) values (?, ?, ?)",
fullpath, "", currentFullhash)
else:
result = parseInt(module[0])
if currentFullhash == module[1]:
# not changed, so use the cached AST (even if it might be wrong
# due to its dependencies):
doAssert(result != 0)
var cycleCheck = initIntSet()
if not needsRecompile(fileIdx, fullpath, cycleCheck):
return -result
db.exec(sql"update modules set fullHash = ? where id = ?", currentFullhash, module[0])
db.exec(sql"delete from deps where module = ?", module[0])
db.exec(sql"delete from types where module = ?", module[0])
db.exec(sql"delete from syms where module = ?", module[0])
db.exec(sql"delete from toplevelstmts where module = ?", module[0])
db.exec(sql"delete from statics where module = ?", module[0])
type
TRodWriter = object
module: PSym
sstack: seq[PSym] # a stack of symbols to process
tstack: seq[PType] # a stack of types to process
tmarks, smarks: IntSet
forwardedSyms: seq[PSym]
PRodWriter = var TRodWriter
proc initRodWriter(module: PSym): TRodWriter =
result = TRodWriter(module: module, sstack: @[], tstack: @[],
tmarks: initIntSet(), smarks: initIntSet(), forwardedSyms: @[])
when false:
proc getDefines(): string =
result = ""
for d in definedSymbolNames():
if result.len != 0: add(result, " ")
add(result, d)
const
rodNL = "\L"
proc pushType(w: PRodWriter, t: PType) =
if not containsOrIncl(w.tmarks, t.id):
w.tstack.add(t)
proc pushSym(w: PRodWriter, s: PSym) =
if not containsOrIncl(w.smarks, s.id):
w.sstack.add(s)
proc toDbFileId(fileIdx: int32): int =
if fileIdx == -1: return -1
let fullpath = fileIdx.toFullPath
let row = db.getRow(sql"select id, fullhash from filenames where fullpath = ?",
fullpath)
let id = row[0]
let fullhash = hashFileCached(fileIdx, fullpath)
if id.len == 0:
result = int db.insertID(sql"insert into filenames(fullpath, fullhash) values (?, ?)",
fullpath, fullhash)
else:
if row[1] != fullhash:
db.exec(sql"update filenames set fullhash = ? where fullpath = ?", fullhash, fullpath)
result = parseInt(id)
proc fromDbFileId(dbId: int): int32 =
if dbId == -1: return -1
let fullpath = db.getValue(sql"select fullpath from filenames where id = ?", dbId)
doAssert fullpath.len > 0, "cannot find file name for DB ID " & $dbId
result = fileInfoIdx(fullpath)
proc encodeNode(w: PRodWriter, fInfo: TLineInfo, n: PNode,
result: var string) =
if n == nil:
# nil nodes have to be stored too:
result.add("()")
return
result.add('(')
encodeVInt(ord(n.kind), result)
# we do not write comments for now
# Line information takes easily 20% or more of the filesize! Therefore we
# omit line information if it is the same as the parent's line information:
if fInfo.fileIndex != n.info.fileIndex:
result.add('?')
encodeVInt(n.info.col, result)
result.add(',')
encodeVInt(n.info.line, result)
result.add(',')
encodeVInt(toDbFileId(n.info.fileIndex), result)
elif fInfo.line != n.info.line:
result.add('?')
encodeVInt(n.info.col, result)
result.add(',')
encodeVInt(n.info.line, result)
elif fInfo.col != n.info.col:
result.add('?')
encodeVInt(n.info.col, result)
# No need to output the file index, as this is the serialization of one
# file.
let f = n.flags * PersistentNodeFlags
if f != {}:
result.add('$')
encodeVInt(cast[int32](f), result)
if n.typ != nil:
result.add('^')
encodeVInt(n.typ.id, result)
pushType(w, n.typ)
case n.kind
of nkCharLit..nkUInt64Lit:
if n.intVal != 0:
result.add('!')
encodeVBiggestInt(n.intVal, result)
of nkFloatLit..nkFloat64Lit:
if n.floatVal != 0.0:
result.add('!')
encodeStr($n.floatVal, result)
of nkStrLit..nkTripleStrLit:
if n.strVal != "":
result.add('!')
encodeStr(n.strVal, result)
of nkIdent:
result.add('!')
encodeStr(n.ident.s, result)
of nkSym:
result.add('!')
encodeVInt(n.sym.id, result)
pushSym(w, n.sym)
else:
for i in countup(0, sonsLen(n) - 1):
encodeNode(w, n.info, n.sons[i], result)
add(result, ')')
proc encodeLoc(w: PRodWriter, loc: TLoc, result: var string) =
var oldLen = result.len
result.add('<')
if loc.k != low(loc.k): encodeVInt(ord(loc.k), result)
if loc.storage != low(loc.storage):
add(result, '*')
encodeVInt(ord(loc.storage), result)
if loc.flags != {}:
add(result, '$')
encodeVInt(cast[int32](loc.flags), result)
if loc.lode != nil:
add(result, '^')
encodeNode(w, unknownLineInfo(), loc.lode, result)
#encodeVInt(cast[int32](loc.t.id), result)
#pushType(w, loc.t)
if loc.r != nil:
add(result, '!')
encodeStr($loc.r, result)
if oldLen + 1 == result.len:
# no data was necessary, so remove the '<' again:
setLen(result, oldLen)
else:
add(result, '>')
proc encodeType(w: PRodWriter, t: PType, result: var string) =
if t == nil:
# nil nodes have to be stored too:
result.add("[]")
return
# we need no surrounding [] here because the type is in a line of its own
if t.kind == tyForward: internalError("encodeType: tyForward")
# for the new rodfile viewer we use a preceding [ so that the data section
# can easily be disambiguated:
add(result, '[')
encodeVInt(ord(t.kind), result)
add(result, '+')
encodeVInt(t.id, result)
if t.n != nil:
encodeNode(w, w.module.info, t.n, result)
if t.flags != {}:
add(result, '$')
encodeVInt(cast[int32](t.flags), result)
if t.callConv != low(t.callConv):
add(result, '?')
encodeVInt(ord(t.callConv), result)
if t.owner != nil:
add(result, '*')
encodeVInt(t.owner.id, result)
pushSym(w, t.owner)
if t.sym != nil:
add(result, '&')
encodeVInt(t.sym.id, result)
pushSym(w, t.sym)
if t.size != - 1:
add(result, '/')
encodeVBiggestInt(t.size, result)
if t.align != 2:
add(result, '=')
encodeVInt(t.align, result)
if t.lockLevel.ord != UnspecifiedLockLevel.ord:
add(result, '\14')
encodeVInt(t.lockLevel.int16, result)
if t.destructor != nil and t.destructor.id != 0:
add(result, '\15')
encodeVInt(t.destructor.id, result)
pushSym(w, t.destructor)
if t.deepCopy != nil:
add(result, '\16')
encodeVInt(t.deepcopy.id, result)
pushSym(w, t.deepcopy)
if t.assignment != nil:
add(result, '\17')
encodeVInt(t.assignment.id, result)
pushSym(w, t.assignment)
if t.sink != nil:
add(result, '\18')
encodeVInt(t.sink.id, result)
pushSym(w, t.sink)
for i, s in items(t.methods):
add(result, '\19')
encodeVInt(i, result)
add(result, '\20')
encodeVInt(s.id, result)
pushSym(w, s)
encodeLoc(w, t.loc, result)
for i in countup(0, sonsLen(t) - 1):
if t.sons[i] == nil:
add(result, "^()")
else:
add(result, '^')
encodeVInt(t.sons[i].id, result)
pushType(w, t.sons[i])
proc encodeLib(w: PRodWriter, lib: PLib, info: TLineInfo, result: var string) =
add(result, '|')
encodeVInt(ord(lib.kind), result)
add(result, '|')
encodeStr($lib.name, result)
add(result, '|')
encodeNode(w, info, lib.path, result)
proc encodeInstantiations(w: PRodWriter; s: seq[PInstantiation];
result: var string) =
for t in s:
result.add('\15')
encodeVInt(t.sym.id, result)
pushSym(w, t.sym)
for tt in t.concreteTypes:
result.add('\17')
encodeVInt(tt.id, result)
pushType(w, tt)
result.add('\20')
encodeVInt(t.compilesId, result)
proc encodeSym(w: PRodWriter, s: PSym, result: var string) =
if s == nil:
# nil nodes have to be stored too:
result.add("{}")
return
# we need no surrounding {} here because the symbol is in a line of its own
encodeVInt(ord(s.kind), result)
result.add('+')
encodeVInt(s.id, result)
result.add('&')
encodeStr(s.name.s, result)
if s.typ != nil:
result.add('^')
encodeVInt(s.typ.id, result)
pushType(w, s.typ)
result.add('?')
if s.info.col != -1'i16: encodeVInt(s.info.col, result)
result.add(',')
if s.info.line != -1'i16: encodeVInt(s.info.line, result)
result.add(',')
encodeVInt(toDbFileId(s.info.fileIndex), result)
if s.owner != nil:
result.add('*')
encodeVInt(s.owner.id, result)
pushSym(w, s.owner)
if s.flags != {}:
result.add('$')
encodeVInt(cast[int32](s.flags), result)
if s.magic != mNone:
result.add('@')
encodeVInt(ord(s.magic), result)
result.add('!')
encodeVInt(cast[int32](s.options), result)
if s.position != 0:
result.add('%')
encodeVInt(s.position, result)
if s.offset != - 1:
result.add('`')
encodeVInt(s.offset, result)
encodeLoc(w, s.loc, result)
if s.annex != nil: encodeLib(w, s.annex, s.info, result)
if s.constraint != nil:
add(result, '#')
encodeNode(w, unknownLineInfo(), s.constraint, result)
case s.kind
of skType, skGenericParam:
for t in s.typeInstCache:
result.add('\14')
encodeVInt(t.id, result)
pushType(w, t)
of routineKinds:
encodeInstantiations(w, s.procInstCache, result)
if s.gcUnsafetyReason != nil:
result.add('\16')
encodeVInt(s.gcUnsafetyReason.id, result)
pushSym(w, s.gcUnsafetyReason)
of skModule, skPackage:
encodeInstantiations(w, s.usedGenerics, result)
# we don't serialize:
#tab*: TStrTable # interface table for modules
of skLet, skVar, skField, skForVar:
if s.guard != nil:
result.add('\18')
encodeVInt(s.guard.id, result)
pushSym(w, s.guard)
if s.bitsize != 0:
result.add('\19')
encodeVInt(s.bitsize, result)
else: discard
# lazy loading will soon reload the ast lazily, so the ast needs to be
# the last entry of a symbol:
if s.ast != nil:
# we used to attempt to save space here by only storing a dummy AST if
# it is not necessary, but Nim's heavy compile-time evaluation features
# make that unfeasible nowadays:
encodeNode(w, s.info, s.ast, result)
proc storeSym(w: PRodWriter; s: PSym) =
if sfForward in s.flags and s.kind != skModule:
w.forwardedSyms.add s
return
var buf = newStringOfCap(160)
encodeSym(w, s, buf)
# XXX only store the name for exported symbols in order to speed up lookup
# times once we enable the skStub logic.
db.exec(sql"insert into syms(nimid, module, name, data, exported) values (?, ?, ?, ?, ?)",
s.id, abs(w.module.id), s.name.s, buf, ord(sfExported in s.flags))
proc storeType(w: PRodWriter; t: PType) =
var buf = newStringOfCap(160)
encodeType(w, t, buf)
db.exec(sql"insert into types(nimid, module, data) values (?, ?, ?)",
t.id, abs(w.module.id), buf)
var w = initRodWriter(nil)
proc storeNode*(module: PSym; n: PNode) =
if gSymbolFiles != v2Sf: return
w.module = module
var buf = newStringOfCap(160)
encodeNode(w, module.info, n, buf)
db.exec(sql"insert into toplevelstmts(module, position, data) values (?, ?, ?)",
abs(module.id), module.offset, buf)
inc module.offset
var i = 0
while true:
if i > 10_000:
quit "loop never ends!"
if w.sstack.len > 0:
let s = w.sstack.pop()
when false:
echo "popped ", s.name.s, " ", s.id
storeSym(w, s)
elif w.tstack.len > 0:
let t = w.tstack.pop()
storeType(w, t)
when false:
echo "popped type ", typeToString(t), " ", t.id
else:
break
inc i
proc storeRemaining*(module: PSym) =
if gSymbolFiles != v2Sf: return
w.module = module
for s in w.forwardedSyms:
assert sfForward notin s.flags
storeSym(w, s)
w.forwardedSyms.setLen 0
# ---------------- decoder -----------------------------------
type
TRodReader = object
module: PSym
#sstack: seq[(PSym, ptr PSym)] # a stack of symbols to process
#tstack: seq[(PType, ptr PType)] # a stack of types to process
#tmarks, smarks: IntSet
syms: Table[int, PSym] ## XXX make this more efficients
types: Table[int, PType]
cache: IdentCache
BlobReader = object
s: string
pos: int
PRodReader = var TRodReader
proc initRodReader(cache: IdentCache): TRodReader =
TRodReader(module: nil,
syms: initTable[int, PSym](), types: initTable[int, PType](),
cache: cache)
var gr = initRodReader(newIdentCache())
using
r: PRodReader
b: var BlobReader
proc loadSym(r; id: int, info: TLineInfo): PSym
proc loadType(r; id: int, info: TLineInfo): PType
proc decodeLineInfo(r; b; info: var TLineInfo) =
if b.s[b.pos] == '?':
inc(b.pos)
if b.s[b.pos] == ',': info.col = -1'i16
else: info.col = int16(decodeVInt(b.s, b.pos))
if b.s[b.pos] == ',':
inc(b.pos)
if b.s[b.pos] == ',': info.line = -1'i16
else: info.line = int16(decodeVInt(b.s, b.pos))
if b.s[b.pos] == ',':
inc(b.pos)
info.fileIndex = fromDbFileId(decodeVInt(b.s, b.pos))
proc skipNode(b) =
assert b.s[b.pos] == '('
var par = 0
var pos = b.pos+1
while true:
case b.s[pos]
of ')':
if par == 0: break
dec par
of '(': inc par
else: discard
inc pos
b.pos = pos+1 # skip ')'
proc decodeNodeLazyBody(r; b; fInfo: TLineInfo,
belongsTo: PSym): PNode =
result = nil
if b.s[b.pos] == '(':
inc(b.pos)
if b.s[b.pos] == ')':
inc(b.pos)
return # nil node
result = newNodeI(TNodeKind(decodeVInt(b.s, b.pos)), fInfo)
decodeLineInfo(r, b, result.info)
if b.s[b.pos] == '$':
inc(b.pos)
result.flags = cast[TNodeFlags](int32(decodeVInt(b.s, b.pos)))
if b.s[b.pos] == '^':
inc(b.pos)
var id = decodeVInt(b.s, b.pos)
result.typ = loadType(r, id, result.info)
case result.kind
of nkCharLit..nkUInt64Lit:
if b.s[b.pos] == '!':
inc(b.pos)
result.intVal = decodeVBiggestInt(b.s, b.pos)
of nkFloatLit..nkFloat64Lit:
if b.s[b.pos] == '!':
inc(b.pos)
var fl = decodeStr(b.s, b.pos)
result.floatVal = parseFloat(fl)
of nkStrLit..nkTripleStrLit:
if b.s[b.pos] == '!':
inc(b.pos)
result.strVal = decodeStr(b.s, b.pos)
else:
result.strVal = ""
of nkIdent:
if b.s[b.pos] == '!':
inc(b.pos)
var fl = decodeStr(b.s, b.pos)
result.ident = r.cache.getIdent(fl)
else:
internalError(result.info, "decodeNode: nkIdent")
of nkSym:
if b.s[b.pos] == '!':
inc(b.pos)
var id = decodeVInt(b.s, b.pos)
result.sym = loadSym(r, id, result.info)
else:
internalError(result.info, "decodeNode: nkSym")
else:
var i = 0
while b.s[b.pos] != ')':
when false:
if belongsTo != nil and i == bodyPos:
addSonNilAllowed(result, nil)
belongsTo.offset = b.pos
skipNode(b)
else:
discard
addSonNilAllowed(result, decodeNodeLazyBody(r, b, result.info, nil))
inc i
if b.s[b.pos] == ')': inc(b.pos)
else: internalError(result.info, "decodeNode: ')' missing")
else:
internalError(fInfo, "decodeNode: '(' missing " & $b.pos)
proc decodeNode(r; b; fInfo: TLineInfo): PNode =
result = decodeNodeLazyBody(r, b, fInfo, nil)
proc decodeLoc(r; b; loc: var TLoc, info: TLineInfo) =
if b.s[b.pos] == '<':
inc(b.pos)
if b.s[b.pos] in {'0'..'9', 'a'..'z', 'A'..'Z'}:
loc.k = TLocKind(decodeVInt(b.s, b.pos))
else:
loc.k = low(loc.k)
if b.s[b.pos] == '*':
inc(b.pos)
loc.storage = TStorageLoc(decodeVInt(b.s, b.pos))
else:
loc.storage = low(loc.storage)
if b.s[b.pos] == '$':
inc(b.pos)
loc.flags = cast[TLocFlags](int32(decodeVInt(b.s, b.pos)))
else:
loc.flags = {}
if b.s[b.pos] == '^':
inc(b.pos)
loc.lode = decodeNode(r, b, info)
# rrGetType(b, decodeVInt(b.s, b.pos), info)
else:
loc.lode = nil
if b.s[b.pos] == '!':
inc(b.pos)
loc.r = rope(decodeStr(b.s, b.pos))
else:
loc.r = nil
if b.s[b.pos] == '>': inc(b.pos)
else: internalError(info, "decodeLoc " & b.s[b.pos])
proc loadBlob(query: SqlQuery; id: int): BlobReader =
let blob = db.getValue(query, id)
if blob.len == 0:
internalError("symbolfiles: cannot find ID " & $ id)
result = BlobReader(pos: 0)
shallowCopy(result.s, blob)
proc loadType(r; id: int; info: TLineInfo): PType =
result = r.types.getOrDefault(id)
if result != nil: return result
var b = loadBlob(sql"select data from types where nimid = ?", id)
if b.s[b.pos] == '[':
inc(b.pos)
if b.s[b.pos] == ']':
inc(b.pos)
return # nil type
new(result)
result.kind = TTypeKind(decodeVInt(b.s, b.pos))
if b.s[b.pos] == '+':
inc(b.pos)
result.id = decodeVInt(b.s, b.pos)
setId(result.id)
#if debugIds: registerID(result)
else:
internalError(info, "decodeType: no id")
# here this also avoids endless recursion for recursive type
r.types[result.id] = result
if b.s[b.pos] == '(': result.n = decodeNode(r, b, unknownLineInfo())
if b.s[b.pos] == '$':
inc(b.pos)
result.flags = cast[TTypeFlags](int32(decodeVInt(b.s, b.pos)))
if b.s[b.pos] == '?':
inc(b.pos)
result.callConv = TCallingConvention(decodeVInt(b.s, b.pos))
if b.s[b.pos] == '*':
inc(b.pos)
result.owner = loadSym(r, decodeVInt(b.s, b.pos), info)
if b.s[b.pos] == '&':
inc(b.pos)
result.sym = loadSym(r, decodeVInt(b.s, b.pos), info)
if b.s[b.pos] == '/':
inc(b.pos)
result.size = decodeVInt(b.s, b.pos)
else:
result.size = -1
if b.s[b.pos] == '=':
inc(b.pos)
result.align = decodeVInt(b.s, b.pos).int16
else:
result.align = 2
if b.s[b.pos] == '\14':
inc(b.pos)
result.lockLevel = decodeVInt(b.s, b.pos).TLockLevel
else:
result.lockLevel = UnspecifiedLockLevel
if b.s[b.pos] == '\15':
inc(b.pos)
result.destructor = loadSym(r, decodeVInt(b.s, b.pos), info)
if b.s[b.pos] == '\16':
inc(b.pos)
result.deepCopy = loadSym(r, decodeVInt(b.s, b.pos), info)
if b.s[b.pos] == '\17':
inc(b.pos)
result.assignment = loadSym(r, decodeVInt(b.s, b.pos), info)
if b.s[b.pos] == '\18':
inc(b.pos)
result.sink = loadSym(r, decodeVInt(b.s, b.pos), info)
while b.s[b.pos] == '\19':
inc(b.pos)
let x = decodeVInt(b.s, b.pos)
doAssert b.s[b.pos] == '\20'
inc(b.pos)
let y = loadSym(r, decodeVInt(b.s, b.pos), info)
result.methods.safeAdd((x, y))
decodeLoc(r, b, result.loc, info)
while b.s[b.pos] == '^':
inc(b.pos)
if b.s[b.pos] == '(':
inc(b.pos)
if b.s[b.pos] == ')': inc(b.pos)
else: internalError(info, "decodeType ^(" & b.s[b.pos])
rawAddSon(result, nil)
else:
var d = decodeVInt(b.s, b.pos)
rawAddSon(result, loadType(r, d, info))
proc decodeLib(r; b; info: TLineInfo): PLib =
result = nil
if b.s[b.pos] == '|':
new(result)
inc(b.pos)
result.kind = TLibKind(decodeVInt(b.s, b.pos))
if b.s[b.pos] != '|': internalError("decodeLib: 1")
inc(b.pos)
result.name = rope(decodeStr(b.s, b.pos))
if b.s[b.pos] != '|': internalError("decodeLib: 2")
inc(b.pos)
result.path = decodeNode(r, b, info)
proc decodeInstantiations(r; b; info: TLineInfo;
s: var seq[PInstantiation]) =
while b.s[b.pos] == '\15':
inc(b.pos)
var ii: PInstantiation
new ii
ii.sym = loadSym(r, decodeVInt(b.s, b.pos), info)
ii.concreteTypes = @[]
while b.s[b.pos] == '\17':
inc(b.pos)
ii.concreteTypes.add loadType(r, decodeVInt(b.s, b.pos), info)
if b.s[b.pos] == '\20':
inc(b.pos)
ii.compilesId = decodeVInt(b.s, b.pos)
s.safeAdd ii
proc loadSymFromBlob(r; b; info: TLineInfo): PSym =
if b.s[b.pos] == '{':
inc(b.pos)
if b.s[b.pos] == '}':
inc(b.pos)
return # nil sym
var k = TSymKind(decodeVInt(b.s, b.pos))
var id: int
if b.s[b.pos] == '+':
inc(b.pos)
id = decodeVInt(b.s, b.pos)
setId(id)
else:
internalError(info, "decodeSym: no id")
var ident: PIdent
if b.s[b.pos] == '&':
inc(b.pos)
ident = r.cache.getIdent(decodeStr(b.s, b.pos))
else:
internalError(info, "decodeSym: no ident")
#echo "decoding: {", ident.s
new(result)
result.id = id
result.kind = k
result.name = ident # read the rest of the symbol description:
r.syms[result.id] = result
if b.s[b.pos] == '^':
inc(b.pos)
result.typ = loadType(r, decodeVInt(b.s, b.pos), info)
decodeLineInfo(r, b, result.info)
if b.s[b.pos] == '*':
inc(b.pos)
result.owner = loadSym(r, decodeVInt(b.s, b.pos), result.info)
if b.s[b.pos] == '$':
inc(b.pos)
result.flags = cast[TSymFlags](int32(decodeVInt(b.s, b.pos)))
if b.s[b.pos] == '@':
inc(b.pos)
result.magic = TMagic(decodeVInt(b.s, b.pos))
if b.s[b.pos] == '!':
inc(b.pos)
result.options = cast[TOptions](int32(decodeVInt(b.s, b.pos)))
else:
result.options = r.module.options
if b.s[b.pos] == '%':
inc(b.pos)
result.position = decodeVInt(b.s, b.pos)
if b.s[b.pos] == '`':
inc(b.pos)
result.offset = decodeVInt(b.s, b.pos)
else:
result.offset = - 1
decodeLoc(r, b, result.loc, result.info)
result.annex = decodeLib(r, b, info)
if b.s[b.pos] == '#':
inc(b.pos)
result.constraint = decodeNode(r, b, unknownLineInfo())
case result.kind
of skType, skGenericParam:
while b.s[b.pos] == '\14':
inc(b.pos)
result.typeInstCache.safeAdd loadType(r, decodeVInt(b.s, b.pos), result.info)
of routineKinds:
decodeInstantiations(r, b, result.info, result.procInstCache)
if b.s[b.pos] == '\16':
inc(b.pos)
result.gcUnsafetyReason = loadSym(r, decodeVInt(b.s, b.pos), result.info)
of skModule, skPackage:
decodeInstantiations(r, b, result.info, result.usedGenerics)
of skLet, skVar, skField, skForVar:
if b.s[b.pos] == '\18':
inc(b.pos)
result.guard = loadSym(r, decodeVInt(b.s, b.pos), result.info)
if b.s[b.pos] == '\19':
inc(b.pos)
result.bitsize = decodeVInt(b.s, b.pos).int16
else: discard
if b.s[b.pos] == '(':
#if result.kind in routineKinds:
# result.ast = decodeNodeLazyBody(b, result.info, result)
#else:
result.ast = decodeNode(r, b, result.info)
if sfCompilerProc in result.flags:
registerCompilerProc(result)
#echo "loading ", result.name.s
proc loadSym(r; id: int; info: TLineInfo): PSym =
result = r.syms.getOrDefault(id)
if result != nil: return result
var b = loadBlob(sql"select data from syms where nimid = ?", id)
result = loadSymFromBlob(r, b, info)
doAssert id == result.id, "symbol ID is not consistent!"
proc loadModuleSymTab(r; module: PSym) =
## goal: fill module.tab
gr.syms[module.id] = module
for row in db.fastRows(sql"select nimid, data from syms where module = ? and exported = 1", abs(module.id)):
let id = parseInt(row[0])
var s = r.syms.getOrDefault(id)
if s == nil:
var b = BlobReader(pos: 0)
shallowCopy(b.s, row[1])
s = loadSymFromBlob(r, b, module.info)
assert s != nil
strTableAdd(module.tab, s)
if sfSystemModule in module.flags:
magicsys.systemModule = module
proc loadNode*(module: PSym; index: int): PNode =
assert gSymbolFiles == v2Sf
if index == 0:
loadModuleSymTab(gr, module)
#index = parseInt db.getValue(
# sql"select min(id) from toplevelstmts where module = ?", abs module.id)
var b = BlobReader(pos: 0)
b.s = db.getValue(sql"select data from toplevelstmts where position = ? and module = ?",
index, abs module.id)
if b.s.len == 0:
db.exec(sql"insert into controlblock(idgen) values (?)", gFrontEndId)
return nil # end marker
gr.module = module
result = decodeNode(gr, b, module.info)
proc addModuleDep*(module, fileIdx: int32; isIncludeFile: bool) =
if gSymbolFiles != v2Sf: return
let a = toDbFileId(module)
let b = toDbFileId(fileIdx)
db.exec(sql"insert into deps(module, dependency, isIncludeFile) values (?, ?, ?)",
a, b, ord(isIncludeFile))
# --------------- Database model ---------------------------------------------
proc createDb() =
db.exec(sql"""
create table if not exists controlblock(
idgen integer not null
);
""")
db.exec(sql"""
create table if not exists filenames(
id integer primary key,
fullpath varchar(8000) not null,
fullHash varchar(256) not null
);
""")
db.exec sql"create index if not exists FilenameIx on filenames(fullpath);"
db.exec(sql"""
create table if not exists modules(
id integer primary key,
fullpath varchar(8000) not null,
interfHash varchar(256) not null,
fullHash varchar(256) not null,
created timestamp not null default (DATETIME('now'))
);""")
db.exec(sql"""create unique index if not exists SymNameIx on modules(fullpath);""")
db.exec(sql"""
create table if not exists deps(
id integer primary key,
module integer not null,
dependency integer not null,
isIncludeFile integer not null,
foreign key (module) references filenames(id),
foreign key (dependency) references filenames(id)
);""")
db.exec(sql"""create index if not exists DepsIx on deps(module);""")
db.exec(sql"""
create table if not exists types(
id integer primary key,
nimid integer not null,
module integer not null,
data blob not null,
foreign key (module) references module(id)
);
""")
db.exec sql"create index TypeByModuleIdx on types(module);"
db.exec sql"create index TypeByNimIdIdx on types(nimid);"
db.exec(sql"""
create table if not exists syms(
id integer primary key,
nimid integer not null,
module integer not null,
name varchar(256) not null,
data blob not null,
exported int not null,
foreign key (module) references module(id)
);
""")
db.exec sql"create index if not exists SymNameIx on syms(name);"
db.exec sql"create index SymByNameAndModuleIdx on syms(name, module);"
db.exec sql"create index SymByModuleIdx on syms(module);"
db.exec sql"create index SymByNimIdIdx on syms(nimid);"
db.exec(sql"""
create table if not exists toplevelstmts(
id integer primary key,
position integer not null,
module integer not null,
data blob not null,
foreign key (module) references module(id)
);
""")
db.exec sql"create index TopLevelStmtByModuleIdx on toplevelstmts(module);"
db.exec sql"create index TopLevelStmtByPositionIdx on toplevelstmts(position);"
db.exec(sql"""
create table if not exists statics(
id integer primary key,
module integer not null,
data blob not null,
foreign key (module) references module(id)
);
""")
db.exec sql"create index StaticsByModuleIdx on toplevelstmts(module);"
db.exec sql"insert into controlblock(idgen) values (0)"
proc setupModuleCache* =
if gSymbolFiles != v2Sf: return
let dbfile = getNimcacheDir() / "rodfiles.db"
if not fileExists(dbfile):
db = open(connection=dbfile, user="nim", password="",
database="nim")
createDb()
else:
db = open(connection=dbfile, user="nim", password="",
database="nim")
db.exec(sql"pragma journal_mode=off")
db.exec(sql"pragma SYNCHRONOUS=off")
db.exec(sql"pragma LOCKING_MODE=exclusive")
let lastId = db.getValue(sql"select max(idgen) from controlblock")
if lastId.len > 0:
idgen.setId(parseInt lastId)

179
compiler/rodread.nim
View file

@ -90,7 +90,8 @@
import
os, options, strutils, nversion, ast, astalgo, msgs, platform, condsyms,
ropes, idents, securehash, idgen, types, rodutils, memfiles, tables
ropes, idents, std / sha1, idgen, types, rodutils, memfiles, tables,
configuration
type
TReasonForRecompile* = enum ## all the reasons that can trigger recompilation
@ -126,8 +127,8 @@ type
s: cstring # mmap'ed file contents
options: TOptions
reason: TReasonForRecompile
modDeps: seq[int32]
files: seq[int32]
modDeps: seq[FileIndex]
files: seq[FileIndex]
dataIdx: int # offset of start of data section
convertersIdx: int # offset of start of converters section
initIdx, interfIdx, compilerProcsIdx, methodsIdx: int
@ -143,6 +144,7 @@ type
origFile: string
inViewMode: bool
cache*: IdentCache
config: ConfigRef
PRodReader* = ref TRodReader
@ -163,11 +165,11 @@ proc decodeLineInfo(r: PRodReader, info: var TLineInfo) =
else: info.col = int16(decodeVInt(r.s, r.pos))
if r.s[r.pos] == ',':
inc(r.pos)
if r.s[r.pos] == ',': info.line = -1'i16
else: info.line = int16(decodeVInt(r.s, r.pos))
if r.s[r.pos] == ',': info.line = 0'u16
else: info.line = uint16(decodeVInt(r.s, r.pos))
if r.s[r.pos] == ',':
inc(r.pos)
info = newLineInfo(r.files[decodeVInt(r.s, r.pos)], info.line, info.col)
info = newLineInfo(r.files[decodeVInt(r.s, r.pos)], int info.line, info.col)
proc skipNode(r: PRodReader) =
assert r.s[r.pos] == '('
@ -222,14 +224,14 @@ proc decodeNodeLazyBody(r: PRodReader, fInfo: TLineInfo,
var fl = decodeStr(r.s, r.pos)
result.ident = r.cache.getIdent(fl)
else:
internalError(result.info, "decodeNode: nkIdent")
internalError(r.config, result.info, "decodeNode: nkIdent")
of nkSym:
if r.s[r.pos] == '!':
inc(r.pos)
var id = decodeVInt(r.s, r.pos)
result.sym = rrGetSym(r, id, result.info)
else:
internalError(result.info, "decodeNode: nkSym")
internalError(r.config, result.info, "decodeNode: nkSym")
else:
var i = 0
while r.s[r.pos] != ')':
@ -241,9 +243,9 @@ proc decodeNodeLazyBody(r: PRodReader, fInfo: TLineInfo,
addSonNilAllowed(result, decodeNodeLazyBody(r, result.info, nil))
inc i
if r.s[r.pos] == ')': inc(r.pos)
else: internalError(result.info, "decodeNode: ')' missing")
else: internalError(r.config, result.info, "decodeNode: ')' missing")
else:
internalError(fInfo, "decodeNode: '(' missing " & $r.pos)
internalError(r.config, fInfo, "decodeNode: '(' missing " & $r.pos)
proc decodeNode(r: PRodReader, fInfo: TLineInfo): PNode =
result = decodeNodeLazyBody(r, fInfo, nil)
@ -277,7 +279,7 @@ proc decodeLoc(r: PRodReader, loc: var TLoc, info: TLineInfo) =
else:
loc.r = nil
if r.s[r.pos] == '>': inc(r.pos)
else: internalError(info, "decodeLoc " & r.s[r.pos])
else: internalError(r.config, info, "decodeLoc " & r.s[r.pos])
proc decodeType(r: PRodReader, info: TLineInfo): PType =
result = nil
@ -294,7 +296,7 @@ proc decodeType(r: PRodReader, info: TLineInfo): PType =
setId(result.id)
if debugIds: registerID(result)
else:
internalError(info, "decodeType: no id")
internalError(r.config, info, "decodeType: no id")
# here this also avoids endless recursion for recursive type
idTablePut(gTypeTable, result, result)
if r.s[r.pos] == '(': result.n = decodeNode(r, unknownLineInfo())
@ -345,14 +347,14 @@ proc decodeType(r: PRodReader, info: TLineInfo): PType =
doAssert r.s[r.pos] == '\20'
inc(r.pos)
let y = rrGetSym(r, decodeVInt(r.s, r.pos), info)
result.methods.safeAdd((x, y))
result.methods.add((x, y))
decodeLoc(r, result.loc, info)
while r.s[r.pos] == '^':
inc(r.pos)
if r.s[r.pos] == '(':
inc(r.pos)
if r.s[r.pos] == ')': inc(r.pos)
else: internalError(info, "decodeType ^(" & r.s[r.pos])
else: internalError(r.config, info, "decodeType ^(" & r.s[r.pos])
rawAddSon(result, nil)
else:
var d = decodeVInt(r.s, r.pos)
@ -364,10 +366,10 @@ proc decodeLib(r: PRodReader, info: TLineInfo): PLib =
new(result)
inc(r.pos)
result.kind = TLibKind(decodeVInt(r.s, r.pos))
if r.s[r.pos] != '|': internalError("decodeLib: 1")
if r.s[r.pos] != '|': internalError(r.config, "decodeLib: 1")
inc(r.pos)
result.name = rope(decodeStr(r.s, r.pos))
if r.s[r.pos] != '|': internalError("decodeLib: 2")
if r.s[r.pos] != '|': internalError(r.config, "decodeLib: 2")
inc(r.pos)
result.path = decodeNode(r, info)
@ -385,7 +387,7 @@ proc decodeInstantiations(r: PRodReader; info: TLineInfo;
if r.s[r.pos] == '\20':
inc(r.pos)
ii.compilesId = decodeVInt(r.s, r.pos)
s.safeAdd ii
s.add ii
proc decodeSym(r: PRodReader, info: TLineInfo): PSym =
var
@ -403,12 +405,12 @@ proc decodeSym(r: PRodReader, info: TLineInfo): PSym =
id = decodeVInt(r.s, r.pos)
setId(id)
else:
internalError(info, "decodeSym: no id")
internalError(r.config, info, "decodeSym: no id")
if r.s[r.pos] == '&':
inc(r.pos)
ident = r.cache.getIdent(decodeStr(r.s, r.pos))
else:
internalError(info, "decodeSym: no ident")
internalError(r.config, info, "decodeSym: no ident")
#echo "decoding: {", ident.s
result = r.syms.getOrDefault(id)
if result == nil:
@ -417,7 +419,7 @@ proc decodeSym(r: PRodReader, info: TLineInfo): PSym =
r.syms[result.id] = result
if debugIds: registerID(result)
elif result.id != id:
internalError(info, "decodeSym: wrong id")
internalError(r.config, info, "decodeSym: wrong id")
elif result.kind != skStub and not r.inViewMode:
# we already loaded the symbol
return
@ -465,7 +467,7 @@ proc decodeSym(r: PRodReader, info: TLineInfo): PSym =
of skType, skGenericParam:
while r.s[r.pos] == '\14':
inc(r.pos)
result.typeInstCache.safeAdd rrGetType(r, decodeVInt(r.s, r.pos), result.info)
result.typeInstCache.add rrGetType(r, decodeVInt(r.s, r.pos), result.info)
of routineKinds:
decodeInstantiations(r, result.info, result.procInstCache)
if r.s[r.pos] == '\16':
@ -512,7 +514,7 @@ proc skipSection(r: PRodReader) =
else: discard
inc(r.pos)
else:
internalError("skipSection " & $r.line)
internalError(r.config, "skipSection " & $r.line)
proc rdWord(r: PRodReader): string =
result = ""
@ -530,7 +532,7 @@ proc newStub(r: PRodReader, name: string, id: int): PSym =
if debugIds: registerID(result)
proc processInterf(r: PRodReader, module: PSym) =
if r.interfIdx == 0: internalError("processInterf")
if r.interfIdx == 0: internalError(r.config, "processInterf")
r.pos = r.interfIdx
while (r.s[r.pos] > '\x0A') and (r.s[r.pos] != ')'):
var w = decodeStr(r.s, r.pos)
@ -543,7 +545,7 @@ proc processInterf(r: PRodReader, module: PSym) =
r.syms[s.id] = s
proc processCompilerProcs(r: PRodReader, module: PSym) =
if r.compilerProcsIdx == 0: internalError("processCompilerProcs")
if r.compilerProcsIdx == 0: internalError(r.config, "processCompilerProcs")
r.pos = r.compilerProcsIdx
while (r.s[r.pos] > '\x0A') and (r.s[r.pos] != ')'):
var w = decodeStr(r.s, r.pos)
@ -586,7 +588,7 @@ proc cmdChangeTriggersRecompilation(old, new: TCommands): bool =
# new command forces us to consider it here :-)
case old
of cmdCompileToC, cmdCompileToCpp, cmdCompileToOC,
cmdCompileToJS, cmdCompileToPHP, cmdCompileToLLVM:
cmdCompileToJS, cmdCompileToLLVM:
if new in {cmdDoc, cmdCheck, cmdIdeTools, cmdPretty, cmdDef,
cmdInteractive}:
return false
@ -621,26 +623,26 @@ proc processRodFile(r: PRodReader, hash: SecureHash) =
of "OPTIONS":
inc(r.pos) # skip ':'
r.options = cast[TOptions](int32(decodeVInt(r.s, r.pos)))
if options.gOptions != r.options: r.reason = rrOptions
if r.config.options != r.options: r.reason = rrOptions
of "GOPTIONS":
inc(r.pos) # skip ':'
var dep = cast[TGlobalOptions](int32(decodeVInt(r.s, r.pos)))
if gGlobalOptions-harmlessOptions != dep-harmlessOptions:
if r.config.globalOptions-harmlessOptions != dep-harmlessOptions:
r.reason = rrOptions
of "CMD":
inc(r.pos) # skip ':'
var dep = cast[TCommands](int32(decodeVInt(r.s, r.pos)))
if cmdChangeTriggersRecompilation(dep, gCmd): r.reason = rrOptions
if cmdChangeTriggersRecompilation(dep, r.config.cmd): r.reason = rrOptions
of "DEFINES":
inc(r.pos) # skip ':'
d = 0
while r.s[r.pos] > '\x0A':
w = decodeStr(r.s, r.pos)
inc(d)
if not condsyms.isDefined(r.cache.getIdent(w)):
if not isDefined(r.config, w):
r.reason = rrDefines #MessageOut('not defined, but should: ' + w);
if r.s[r.pos] == ' ': inc(r.pos)
if d != countDefinedSymbols(): r.reason = rrDefines
if d != countDefinedSymbols(r.config.symbols): r.reason = rrDefines
of "FILES":
inc(r.pos, 2) # skip "(\10"
inc(r.line)
@ -648,7 +650,7 @@ proc processRodFile(r: PRodReader, hash: SecureHash) =
let finalPath = decodeStr(r.s, r.pos)
#let resolvedPath = relativePath.findModule(r.origFile)
#let finalPath = if resolvedPath.len > 0: resolvedPath else: relativePath
r.files.add(finalPath.fileInfoIdx)
r.files.add(fileInfoIdx(r.config, finalPath))
inc(r.pos) # skip #10
inc(r.line)
if r.s[r.pos] == ')': inc(r.pos)
@ -696,7 +698,7 @@ proc processRodFile(r: PRodReader, hash: SecureHash) =
r.initIdx = r.pos + 2 # "(\10"
skipSection(r)
else:
internalError("invalid section: '" & section &
internalError(r.config, "invalid section: '" & section &
"' at " & $r.line & " in " & r.filename)
#MsgWriteln("skipping section: " & section &
# " at " & $r.line & " in " & r.filename)
@ -712,9 +714,11 @@ proc startsWith(buf: cstring, token: string, pos = 0): bool =
result = s == token.len
proc newRodReader(modfilename: string, hash: SecureHash,
readerIndex: int; cache: IdentCache): PRodReader =
readerIndex: int; cache: IdentCache;
config: ConfigRef): PRodReader =
new(result)
result.cache = cache
result.config = config
try:
result.memfile = memfiles.open(modfilename)
except OSError:
@ -757,13 +761,13 @@ proc rrGetType(r: PRodReader, id: int, info: TLineInfo): PType =
# load the type:
var oldPos = r.pos
var d = iiTableGet(r.index.tab, id)
if d == InvalidKey: internalError(info, "rrGetType")
if d == InvalidKey: internalError(r.config, info, "rrGetType")
r.pos = d + r.dataIdx
result = decodeType(r, info)
r.pos = oldPos
type
TFileModuleRec{.final.} = object
TFileModuleRec = object
filename*: string
reason*: TReasonForRecompile
rd*: PRodReader
@ -776,7 +780,7 @@ var gMods*: TFileModuleMap = @[]
proc decodeSymSafePos(rd: PRodReader, offset: int, info: TLineInfo): PSym =
# all compiled modules
if rd.dataIdx == 0: internalError(info, "dataIdx == 0")
if rd.dataIdx == 0: internalError(rd.config, info, "dataIdx == 0")
var oldPos = rd.pos
rd.pos = offset + rd.dataIdx
result = decodeSym(rd, info)
@ -811,8 +815,9 @@ proc rrGetSym(r: PRodReader, id: int, info: TLineInfo): PSym =
if moduleID < 0:
var x = ""
encodeVInt(id, x)
internalError(info, "missing from both indexes: +" & x)
internalError(r.config, info, "missing from both indexes: +" & x)
var rd = getReader(moduleID)
doAssert rd != nil
d = iiTableGet(rd.index.tab, id)
if d != InvalidKey:
result = decodeSymSafePos(rd, d, info)
@ -820,14 +825,14 @@ proc rrGetSym(r: PRodReader, id: int, info: TLineInfo): PSym =
var x = ""
encodeVInt(id, x)
when false: findSomeWhere(id)
internalError(info, "rrGetSym: no reader found: +" & x)
internalError(r.config, info, "rrGetSym: no reader found: +" & x)
else:
# own symbol:
result = decodeSymSafePos(r, d, info)
if result != nil and result.kind == skStub: rawLoadStub(result)
proc loadInitSection*(r: PRodReader): PNode =
if r.initIdx == 0 or r.dataIdx == 0: internalError("loadInitSection")
if r.initIdx == 0 or r.dataIdx == 0: internalError(r.config, "loadInitSection")
var oldPos = r.pos
r.pos = r.initIdx
result = newNode(nkStmtList)
@ -844,7 +849,7 @@ proc loadConverters(r: PRodReader) =
# We have to ensure that no exported converter is a stub anymore, and the
# import mechanism takes care of the rest.
if r.convertersIdx == 0 or r.dataIdx == 0:
internalError("importConverters")
internalError(r.config, "importConverters")
r.pos = r.convertersIdx
while r.s[r.pos] > '\x0A':
var d = decodeVInt(r.s, r.pos)
@ -853,36 +858,36 @@ proc loadConverters(r: PRodReader) =
proc loadMethods(r: PRodReader) =
if r.methodsIdx == 0 or r.dataIdx == 0:
internalError("loadMethods")
internalError(r.config, "loadMethods")
r.pos = r.methodsIdx
while r.s[r.pos] > '\x0A':
var d = decodeVInt(r.s, r.pos)
r.methods.add(rrGetSym(r, d, unknownLineInfo()))
if r.s[r.pos] == ' ': inc(r.pos)
proc getHash*(fileIdx: int32): SecureHash =
if fileIdx <% gMods.len and gMods[fileIdx].hashDone:
return gMods[fileIdx].hash
proc getHash*(fileIdx: FileIndex): SecureHash =
if fileIdx.int32 <% gMods.len and gMods[fileIdx.int32].hashDone:
return gMods[fileIdx.int32].hash
result = secureHashFile(fileIdx.toFullPath)
if fileIdx >= gMods.len: setLen(gMods, fileIdx+1)
gMods[fileIdx].hash = result
if fileIdx.int32 >= gMods.len: setLen(gMods, fileIdx.int32+1)
gMods[fileIdx.int32].hash = result
template growCache*(cache, pos) =
if cache.len <= pos: cache.setLen(pos+1)
proc checkDep(fileIdx: int32; cache: IdentCache): TReasonForRecompile =
proc checkDep(fileIdx: FileIndex; cache: IdentCache; conf: ConfigRef): TReasonForRecompile =
assert fileIdx != InvalidFileIDX
growCache gMods, fileIdx
if gMods[fileIdx].reason != rrEmpty:
growCache gMods, fileIdx.int32
if gMods[fileIdx.int32].reason != rrEmpty:
# reason has already been computed for this module:
return gMods[fileIdx].reason
return gMods[fileIdx.int32].reason
let filename = fileIdx.toFilename
var hash = getHash(fileIdx)
gMods[fileIdx].reason = rrNone # we need to set it here to avoid cycles
gMods[fileIdx.int32].reason = rrNone # we need to set it here to avoid cycles
result = rrNone
var rodfile = toGeneratedFile(filename.withPackageName, RodExt)
var r = newRodReader(rodfile, hash, fileIdx, cache)
var rodfile = toGeneratedFile(conf, conf.withPackageName(filename), RodExt)
var r = newRodReader(rodfile, hash, fileIdx.int32, cache, conf)
if r == nil:
result = (if existsFile(rodfile): rrRodInvalid else: rrRodDoesNotExist)
else:
@ -893,32 +898,31 @@ proc checkDep(fileIdx: int32; cache: IdentCache): TReasonForRecompile =
# NOTE: we need to process the entire module graph so that no ID will
# be used twice! However, compilation speed does not suffer much from
# this, since results are cached.
var res = checkDep(systemFileIdx, cache)
var res = checkDep(systemFileIdx, cache, conf)
if res != rrNone: result = rrModDeps
for i in countup(0, high(r.modDeps)):
res = checkDep(r.modDeps[i], cache)
res = checkDep(r.modDeps[i], cache, conf)
if res != rrNone:
result = rrModDeps
# we cannot break here, because of side-effects of `checkDep`
if result != rrNone:
rawMessage(hintProcessing, reasonToFrmt[result] % filename)
if result != rrNone or optForceFullMake in gGlobalOptions:
rawMessage(conf, hintProcessing, reasonToFrmt[result] % filename)
if result != rrNone or optForceFullMake in conf.globalOptions:
# recompilation is necessary:
if r != nil: memfiles.close(r.memfile)
r = nil
gMods[fileIdx].rd = r
gMods[fileIdx].reason = result # now we know better
gMods[fileIdx.int32].rd = r
gMods[fileIdx.int32].reason = result # now we know better
proc handleSymbolFile*(module: PSym; cache: IdentCache): PRodReader =
let fileIdx = module.fileIdx
if gSymbolFiles in {disabledSf, writeOnlySf}:
proc handleSymbolFile*(module: PSym; cache: IdentCache; conf: ConfigRef): PRodReader =
if conf.symbolFiles in {disabledSf, writeOnlySf, v2Sf}:
module.id = getID()
return nil
idgen.loadMaxIds(options.gProjectPath / options.gProjectName)
discard checkDep(fileIdx, cache)
if gMods[fileIdx].reason == rrEmpty: internalError("handleSymbolFile")
result = gMods[fileIdx].rd
idgen.loadMaxIds(conf, conf.projectPath / conf.projectName)
let fileIdx = module.fileIdx
discard checkDep(fileIdx, cache, conf)
#if gMods[fileIdx.int32].reason == rrEmpty: internalError("handleSymbolFile")
result = gMods[fileIdx.int32].rd
if result != nil:
module.id = result.moduleID
result.syms[module.id] = module
@ -930,19 +934,20 @@ proc handleSymbolFile*(module: PSym; cache: IdentCache): PRodReader =
module.id = getID()
proc rawLoadStub(s: PSym) =
if s.kind != skStub: internalError("loadStub")
assert s.kind == skStub
#if s.kind != skStub: internalError("loadStub")
var rd = gMods[s.position].rd
var theId = s.id # used for later check
var d = iiTableGet(rd.index.tab, s.id)
if d == InvalidKey: internalError("loadStub: invalid key")
#if d == InvalidKey: internalError("loadStub: invalid key")
var rs = decodeSymSafePos(rd, d, unknownLineInfo())
if rs != s:
#echo "rs: ", toHex(cast[int](rs.position), int.sizeof * 2),
# "\ns: ", toHex(cast[int](s.position), int.sizeof * 2)
internalError(rs.info, "loadStub: wrong symbol")
elif rs.id != theId:
internalError(rs.info, "loadStub: wrong ID")
#MessageOut('loaded stub: ' + s.name.s);
when false:
if rs != s:
#echo "rs: ", toHex(cast[int](rs.position), int.sizeof * 2),
# "\ns: ", toHex(cast[int](s.position), int.sizeof * 2)
internalError(rs.info, "loadStub: wrong symbol")
elif rs.id != theId:
internalError(rs.info, "loadStub: wrong ID")
proc loadStub*(s: PSym) =
## loads the stub symbol `s`.
@ -1030,9 +1035,8 @@ proc writeSym(f: File; s: PSym) =
if s.magic != mNone:
f.write('@')
f.write($s.magic)
if s.options != gOptions:
f.write('!')
f.write($s.options)
f.write('!')
f.write($s.options)
if s.position != 0:
f.write('%')
f.write($s.position)
@ -1083,9 +1087,10 @@ proc writeType(f: File; t: PType) =
f.write("]\n")
proc viewFile(rodfile: string) =
var r = newRodReader(rodfile, secureHash(""), 0, newIdentCache())
let conf = newConfigRef()
var r = newRodReader(rodfile, secureHash(""), 0, newIdentCache(), conf)
if r == nil:
rawMessage(errGenerated, "cannot open file (or maybe wrong version):" &
rawMessage(conf, errGenerated, "cannot open file (or maybe wrong version):" &
rodfile)
return
r.inViewMode = true
@ -1133,9 +1138,9 @@ proc viewFile(rodfile: string) =
outf.write("FILES(\n")
while r.s[r.pos] != ')':
let relativePath = decodeStr(r.s, r.pos)
let resolvedPath = relativePath.findModule(r.origFile)
let resolvedPath = findModule(conf, relativePath, r.origFile)
let finalPath = if resolvedPath.len > 0: resolvedPath else: relativePath
r.files.add(finalPath.fileInfoIdx)
r.files.add(fileInfoIdx(conf, finalPath))
inc(r.pos) # skip #10
inc(r.line)
outf.writeLine finalPath
@ -1152,7 +1157,7 @@ proc viewFile(rodfile: string) =
if r.s[r.pos] == '\x0A':
inc(r.pos)
inc(r.line)
outf.write(w, " ", inclHash, "\n")
outf.write(w.int32, " ", inclHash, "\n")
if r.s[r.pos] == ')': inc(r.pos)
outf.write(")\n")
of "DEPS":
@ -1162,7 +1167,7 @@ proc viewFile(rodfile: string) =
let v = int32(decodeVInt(r.s, r.pos))
r.modDeps.add(r.files[v])
if r.s[r.pos] == ' ': inc(r.pos)
outf.write(" ", r.files[v])
outf.write(" ", r.files[v].int32)
outf.write("\n")
of "INTERF", "COMPILERPROCS":
inc r.pos, 2
@ -1227,7 +1232,7 @@ proc viewFile(rodfile: string) =
if r.s[r.pos] == ')': inc r.pos
outf.write("<not supported by viewer>)\n")
else:
internalError("invalid section: '" & section &
internalError(r.config, "invalid section: '" & section &
"' at " & $r.line & " in " & r.filename)
skipSection(r)
if r.s[r.pos] == '\x0A':
@ -1236,4 +1241,4 @@ proc viewFile(rodfile: string) =
outf.close
when isMainModule:
viewFile(paramStr(1).addFileExt(rodExt))
viewFile(paramStr(1).addFileExt(RodExt))

20
compiler/rodutils.nim
View file

@ -8,18 +8,22 @@
#
## Serialization utilities for the compiler.
import strutils
import strutils, math
proc c_snprintf(s: cstring; n:uint; frmt: cstring): cint {.importc: "snprintf", header: "<stdio.h>", nodecl, varargs.}
proc toStrMaxPrecision*(f: BiggestFloat, literalPostfix = ""): string =
if f != f:
case classify(f)
of fcNaN:
result = "NAN"
elif f == 0.0:
of fcNegZero:
result = "-0.0" & literalPostfix
of fcZero:
result = "0.0" & literalPostfix
elif f == 0.5 * f:
if f > 0.0: result = "INF"
else: result = "-INF"
of fcInf:
result = "INF"
of fcNegInf:
result = "-INF"
else:
when defined(nimNoArrayToCstringConversion):
result = newString(81)
@ -63,6 +67,8 @@ proc decodeStr*(s: cstring, pos: var int): string =
const
chars = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
{.push overflowChecks: off.}
# since negative numbers require a leading '-' they use up 1 byte. Thus we
# subtract/add `vintDelta` here to save space for small negative numbers
# which are common in ROD files:
@ -127,6 +133,8 @@ proc decodeVInt*(s: cstring, pos: var int): int =
proc decodeVBiggestInt*(s: cstring, pos: var int): BiggestInt =
decodeIntImpl()
{.pop.}
iterator decodeVIntArray*(s: cstring): int =
var i = 0
while s[i] != '\0':

69
compiler/rodwrite.nim
View file

@ -13,7 +13,7 @@
import
intsets, os, options, strutils, nversion, ast, astalgo, msgs, platform,
condsyms, ropes, idents, securehash, rodread, passes, idgen,
condsyms, ropes, idents, std / sha1, rodread, passes, idgen,
rodutils, modulepaths
from modulegraphs import ModuleGraph
@ -37,12 +37,13 @@ type
files: TStringSeq
origFile: string
cache: IdentCache
config: ConfigRef
PRodWriter = ref TRodWriter
proc getDefines(): string =
proc getDefines(conf: ConfigRef): string =
result = ""
for d in definedSymbolNames():
for d in definedSymbolNames(conf.symbols):
if result.len != 0: add(result, " ")
add(result, d)
@ -55,10 +56,12 @@ proc fileIdx(w: PRodWriter, filename: string): int =
w.files[result] = filename
template filename*(w: PRodWriter): string =
w.module.filename
toFilename(FileIndex w.module.position)
proc newRodWriter(hash: SecureHash, module: PSym; cache: IdentCache): PRodWriter =
proc newRodWriter(hash: SecureHash, module: PSym; cache: IdentCache;
config: ConfigRef): PRodWriter =
new(result)
result.config = config
result.sstack = @[]
result.tstack = @[]
initIiTable(result.index.tab)
@ -67,8 +70,8 @@ proc newRodWriter(hash: SecureHash, module: PSym; cache: IdentCache): PRodWriter
result.imports.r = ""
result.hash = hash
result.module = module
result.defines = getDefines()
result.options = options.gOptions
result.defines = getDefines(config)
result.options = config.options
result.files = @[]
result.inclDeps = ""
result.modDeps = ""
@ -83,14 +86,14 @@ proc newRodWriter(hash: SecureHash, module: PSym; cache: IdentCache): PRodWriter
proc addModDep(w: PRodWriter, dep: string; info: TLineInfo) =
if w.modDeps.len != 0: add(w.modDeps, ' ')
let resolved = dep.findModule(info.toFullPath)
let resolved = findModule(w.config, dep, info.toFullPath)
encodeVInt(fileIdx(w, resolved), w.modDeps)
const
rodNL = "\x0A"
proc addInclDep(w: PRodWriter, dep: string; info: TLineInfo) =
let resolved = dep.findModule(info.toFullPath)
let resolved = findModule(w.config, dep, info.toFullPath)
encodeVInt(fileIdx(w, resolved), w.inclDeps)
add(w.inclDeps, " ")
encodeStr($secureHashFile(resolved), w.inclDeps)
@ -125,14 +128,14 @@ proc encodeNode(w: PRodWriter, fInfo: TLineInfo, n: PNode,
result.add('?')
encodeVInt(n.info.col, result)
result.add(',')
encodeVInt(n.info.line, result)
encodeVInt(int n.info.line, result)
result.add(',')
encodeVInt(fileIdx(w, toFullPath(n.info)), result)
elif fInfo.line != n.info.line:
result.add('?')
encodeVInt(n.info.col, result)
result.add(',')
encodeVInt(n.info.line, result)
encodeVInt(int n.info.line, result)
elif fInfo.col != n.info.col:
result.add('?')
encodeVInt(n.info.col, result)
@ -201,7 +204,7 @@ proc encodeType(w: PRodWriter, t: PType, result: var string) =
result.add("[]")
return
# we need no surrounding [] here because the type is in a line of its own
if t.kind == tyForward: internalError("encodeType: tyForward")
if t.kind == tyForward: internalError(w.config, "encodeType: tyForward")
# for the new rodfile viewer we use a preceding [ so that the data section
# can easily be disambiguated:
add(result, '[')
@ -303,7 +306,7 @@ proc encodeSym(w: PRodWriter, s: PSym, result: var string) =
result.add('?')
if s.info.col != -1'i16: encodeVInt(s.info.col, result)
result.add(',')
if s.info.line != -1'i16: encodeVInt(s.info.line, result)
if s.info.line != 0'u16: encodeVInt(int s.info.line, result)
result.add(',')
encodeVInt(fileIdx(w, toFullPath(s.info)), result)
if s.owner != nil:
@ -390,9 +393,9 @@ proc symStack(w: PRodWriter): int =
inc result
elif iiTableGet(w.index.tab, s.id) == InvalidKey:
var m = getModule(s)
if m == nil and s.kind != skPackage and sfGenSym notin s.flags:
internalError("symStack: module nil: " & s.name.s)
if s.kind == skPackage or {sfFromGeneric, sfGenSym} * s.flags != {} or m.id == w.module.id:
#if m == nil and s.kind != skPackage and sfGenSym notin s.flags:
# internalError("symStack: module nil: " & s.name.s & " " & $s.kind & " ID " & $s.id)
if m == nil or s.kind == skPackage or {sfFromGeneric, sfGenSym} * s.flags != {} or m.id == w.module.id:
# put definition in here
var L = w.data.len
addToIndex(w.index, s.id, L)
@ -411,7 +414,7 @@ proc symStack(w: PRodWriter): int =
add(w.compilerProcs, ' ')
encodeVInt(s.id, w.compilerProcs)
add(w.compilerProcs, rodNL)
if s.kind == skConverter or hasPattern(s):
if s.kind == skConverter or (s.ast != nil and hasPattern(s)):
if w.converters.len != 0: add(w.converters, ' ')
encodeVInt(s.id, w.converters)
if s.kind == skMethod and sfDispatcher notin s.flags:
@ -454,7 +457,7 @@ proc processStacks(w: PRodWriter, finalPass: bool) =
oldS = slen
oldT = tlen
if finalPass and (oldS != 0 or oldT != 0):
internalError("could not serialize some forwarded symbols/types")
internalError(w.config, "could not serialize some forwarded symbols/types")
proc rawAddInterfaceSym(w: PRodWriter, s: PSym) =
pushSym(w, s)
@ -476,8 +479,8 @@ proc addStmt(w: PRodWriter, n: PNode) =
proc writeRod(w: PRodWriter) =
processStacks(w, true)
var f: File
if not open(f, completeGeneratedFilePath(changeFileExt(
w.filename.withPackageName, RodExt)),
if not open(f, completeGeneratedFilePath(w.config, changeFileExt(
withPackageName(w.config, w.filename), RodExt)),
fmWrite):
#echo "couldn't write rod file for: ", w.filename
return
@ -506,12 +509,12 @@ proc writeRod(w: PRodWriter) =
f.write(rodNL)
var goptions = "GOPTIONS:"
encodeVInt(cast[int32](gGlobalOptions), goptions)
encodeVInt(cast[int32](w.config.globalOptions), goptions)
f.write(goptions)
f.write(rodNL)
var cmd = "CMD:"
encodeVInt(cast[int32](gCmd), cmd)
encodeVInt(cast[int32](w.config.cmd), cmd)
f.write(cmd)
f.write(rodNL)
@ -587,17 +590,17 @@ proc process(c: PPassContext, n: PNode): PNode =
of nkProcDef, nkFuncDef, nkIteratorDef, nkConverterDef,
nkTemplateDef, nkMacroDef:
let s = n.sons[namePos].sym
if s == nil: internalError(n.info, "rodwrite.process")
if s == nil: internalError(w.config, n.info, "rodwrite.process")
if n.sons[bodyPos] == nil:
internalError(n.info, "rodwrite.process: body is nil")
internalError(w.config, n.info, "rodwrite.process: body is nil")
if n.sons[bodyPos].kind != nkEmpty or s.magic != mNone or
sfForward notin s.flags:
addInterfaceSym(w, s)
of nkMethodDef:
let s = n.sons[namePos].sym
if s == nil: internalError(n.info, "rodwrite.process")
if s == nil: internalError(w.config, n.info, "rodwrite.process")
if n.sons[bodyPos] == nil:
internalError(n.info, "rodwrite.process: body is nil")
internalError(w.config, n.info, "rodwrite.process: body is nil")
if n.sons[bodyPos].kind != nkEmpty or s.magic != mNone or
sfForward notin s.flags:
pushSym(w, s)
@ -612,7 +615,7 @@ proc process(c: PPassContext, n: PNode): PNode =
for i in countup(0, sonsLen(n) - 1):
var a = n.sons[i]
if a.kind == nkCommentStmt: continue
if a.sons[0].kind != nkSym: internalError(a.info, "rodwrite.process")
if a.sons[0].kind != nkSym: internalError(w.config, a.info, "rodwrite.process")
var s = a.sons[0].sym
addInterfaceSym(w, s)
# this takes care of enum fields too
@ -627,22 +630,22 @@ proc process(c: PPassContext, n: PNode): PNode =
# end
of nkImportStmt:
for i in countup(0, sonsLen(n) - 1):
addModDep(w, getModuleName(n.sons[i]), n.info)
addModDep(w, getModuleName(w.config, n.sons[i]), n.info)
addStmt(w, n)
of nkFromStmt, nkImportExceptStmt:
addModDep(w, getModuleName(n.sons[0]), n.info)
addModDep(w, getModuleName(w.config, n.sons[0]), n.info)
addStmt(w, n)
of nkIncludeStmt:
for i in countup(0, sonsLen(n) - 1):
addInclDep(w, getModuleName(n.sons[i]), n.info)
addInclDep(w, getModuleName(w.config, n.sons[i]), n.info)
of nkPragma:
addStmt(w, n)
else:
discard
proc myOpen(g: ModuleGraph; module: PSym; cache: IdentCache): PPassContext =
if module.id < 0: internalError("rodwrite: module ID not set")
var w = newRodWriter(module.fileIdx.getHash, module, cache)
if module.id < 0: internalError(g.config, "rodwrite: module ID not set")
var w = newRodWriter(rodread.getHash FileIndex module.position, module, cache, g.config)
rawAddInterfaceSym(w, module)
result = w
@ -650,7 +653,7 @@ proc myClose(graph: ModuleGraph; c: PPassContext, n: PNode): PNode =
result = process(c, n)
var w = PRodWriter(c)
writeRod(w)
idgen.saveMaxIds(options.gProjectPath / options.gProjectName)
idgen.saveMaxIds(graph.config, graph.config.projectPath / graph.config.projectName)
const rodwritePass* = makePass(open = myOpen, close = myClose, process = process)

6
compiler/ropes.nim
View file

@ -188,11 +188,11 @@ iterator leaves*(r: Rope): string =
var stack = @[r]
while stack.len > 0:
var it = stack.pop
while isNil(it.data):
while it.left != nil:
assert it.right != nil
stack.add(it.right)
it = it.left
assert(it != nil)
assert(it.data != nil)
yield it.data
iterator items*(r: Rope): char =
@ -251,7 +251,7 @@ proc `%`*(frmt: FormatStr, args: openArray[Rope]): Rope =
while true:
j = j * 10 + ord(frmt[i]) - ord('0')
inc(i)
if frmt[i] notin {'0'..'9'}: break
if i >= frmt.len or frmt[i] notin {'0'..'9'}: break
num = j
if j > high(args) + 1:
errorHandler(rInvalidFormatStr, $(j))

75
compiler/scriptconfig.nim
View file

@ -13,7 +13,7 @@
import
ast, modules, idents, passes, passaux, condsyms,
options, nimconf, sem, semdata, llstream, vm, vmdef, commands, msgs,
os, times, osproc, wordrecg, strtabs, modulegraphs
os, times, osproc, wordrecg, strtabs, modulegraphs, configuration
# we support 'cmpIgnoreStyle' natively for efficiency:
from strutils import cmpIgnoreStyle, contains
@ -26,11 +26,12 @@ proc listDirs(a: VmArgs, filter: set[PathComponent]) =
setResult(a, result)
proc setupVM*(module: PSym; cache: IdentCache; scriptName: string;
config: ConfigRef = nil): PEvalContext =
graph: ModuleGraph): PEvalContext =
# For Nimble we need to export 'setupVM'.
result = newCtx(module, cache)
result = newCtx(module, cache, graph)
result.mode = emRepl
registerAdditionalOps(result)
let conf = graph.config
# captured vars:
var errorMsg: string
@ -70,11 +71,16 @@ proc setupVM*(module: PSym; cache: IdentCache; scriptName: string;
setResult(a, os.getCurrentDir())
cbos moveFile:
os.moveFile(getString(a, 0), getString(a, 1))
cbos moveDir:
os.moveDir(getString(a, 0), getString(a, 1))
cbos copyFile:
os.copyFile(getString(a, 0), getString(a, 1))
cbos copyDir:
os.copyDir(getString(a, 0), getString(a, 1))
cbos getLastModificationTime:
# depends on Time's implementation!
setResult(a, int64(getLastModificationTime(getString(a, 0))))
setResult(a, getLastModificationTime(getString(a, 0)).toUnix)
cbos findExe:
setResult(a, os.findExe(getString(a, 0)))
cbos rawExec:
setResult(a, osproc.execCmd getString(a, 0))
@ -83,6 +89,8 @@ proc setupVM*(module: PSym; cache: IdentCache; scriptName: string;
setResult(a, os.getEnv(a.getString 0, a.getString 1))
cbconf existsEnv:
setResult(a, os.existsEnv(a.getString 0))
cbconf putEnv:
os.putEnv(a.getString 0, a.getString 1)
cbconf dirExists:
setResult(a, os.dirExists(a.getString 0))
cbconf fileExists:
@ -91,13 +99,13 @@ proc setupVM*(module: PSym; cache: IdentCache; scriptName: string;
cbconf thisDir:
setResult(a, vthisDir)
cbconf put:
options.setConfigVar(getString(a, 0), getString(a, 1))
options.setConfigVar(conf, getString(a, 0), getString(a, 1))
cbconf get:
setResult(a, options.getConfigVar(a.getString 0))
setResult(a, options.getConfigVar(conf, a.getString 0))
cbconf exists:
setResult(a, options.existsConfigVar(a.getString 0))
setResult(a, options.existsConfigVar(conf, a.getString 0))
cbconf nimcacheDir:
setResult(a, options.getNimcacheDir())
setResult(a, options.getNimcacheDir(conf))
cbconf paramStr:
setResult(a, os.paramStr(int a.getInt 0))
cbconf paramCount:
@ -107,67 +115,66 @@ proc setupVM*(module: PSym; cache: IdentCache; scriptName: string;
cbconf cmpIgnoreCase:
setResult(a, strutils.cmpIgnoreCase(a.getString 0, a.getString 1))
cbconf setCommand:
options.command = a.getString 0
conf.command = a.getString 0
let arg = a.getString 1
if arg.len > 0:
gProjectName = arg
conf.projectName = arg
let path =
if gProjectName.isAbsolute: gProjectName
else: gProjectPath / gProjectName
if conf.projectName.isAbsolute: conf.projectName
else: conf.projectPath / conf.projectName
try:
gProjectFull = canonicalizePath(path)
conf.projectFull = canonicalizePath(conf, path)
except OSError:
gProjectFull = path
conf.projectFull = path
cbconf getCommand:
setResult(a, options.command)
setResult(a, conf.command)
cbconf switch:
processSwitch(a.getString 0, a.getString 1, passPP, module.info)
processSwitch(a.getString 0, a.getString 1, passPP, module.info, conf)
cbconf hintImpl:
processSpecificNote(a.getString 0, wHint, passPP, module.info,
a.getString 1)
a.getString 1, conf)
cbconf warningImpl:
processSpecificNote(a.getString 0, wWarning, passPP, module.info,
a.getString 1)
a.getString 1, conf)
cbconf patchFile:
let key = a.getString(0) & "_" & a.getString(1)
var val = a.getString(2).addFileExt(NimExt)
if {'$', '~'} in val:
val = pathSubs(val, vthisDir)
val = pathSubs(conf, val, vthisDir)
elif not isAbsolute(val):
val = vthisDir / val
gModuleOverrides[key] = val
conf.moduleOverrides[key] = val
cbconf selfExe:
setResult(a, os.getAppFilename())
cbconf cppDefine:
if config != nil:
options.cppDefine(config, a.getString(0))
options.cppDefine(conf, a.getString(0))
proc runNimScript*(cache: IdentCache; scriptName: string;
freshDefines=true; config: ConfigRef=nil) =
rawMessage(hintConf, scriptName)
freshDefines=true; conf: ConfigRef) =
rawMessage(conf, hintConf, scriptName)
passes.gIncludeFile = includeModule
passes.gImportModule = importModule
let graph = newModuleGraph(config)
if freshDefines: initDefines()
let graph = newModuleGraph(conf)
if freshDefines: initDefines(conf.symbols)
defineSymbol("nimscript")
defineSymbol("nimconfig")
defineSymbol(conf.symbols, "nimscript")
defineSymbol(conf.symbols, "nimconfig")
registerPass(semPass)
registerPass(evalPass)
searchPaths.add(options.libpath)
conf.searchPaths.add(conf.libpath)
var m = graph.makeModule(scriptName)
incl(m.flags, sfMainModule)
vm.globalCtx = setupVM(m, cache, scriptName, config)
vm.globalCtx = setupVM(m, cache, scriptName, graph)
graph.compileSystemModule(cache)
discard graph.processModule(m, llStreamOpen(scriptName, fmRead), nil, cache)
# ensure we load 'system.nim' again for the real non-config stuff!
resetSystemArtifacts()
resetSystemArtifacts(graph)
vm.globalCtx = nil
# do not remove the defined symbols
#initDefines()
undefSymbol("nimscript")
undefSymbol("nimconfig")
undefSymbol(conf.symbols, "nimscript")
undefSymbol(conf.symbols, "nimconfig")

166
compiler/sem.nim
View file

@ -16,7 +16,7 @@ import
procfind, lookups, rodread, pragmas, passes, semdata, semtypinst, sigmatch,
intsets, transf, vmdef, vm, idgen, aliases, cgmeth, lambdalifting,
evaltempl, patterns, parampatterns, sempass2, nimfix.pretty, semmacrosanity,
semparallel, lowerings, pluginsupport, plugins.active
semparallel, lowerings, pluginsupport, plugins.active, rod, configuration
from modulegraphs import ModuleGraph
@ -33,11 +33,12 @@ proc semExprNoDeref(c: PContext, n: PNode, flags: TExprFlags = {}): PNode
proc semProcBody(c: PContext, n: PNode): PNode
proc fitNode(c: PContext, formal: PType, arg: PNode; info: TLineInfo): PNode
proc changeType(n: PNode, newType: PType, check: bool)
proc changeType(c: PContext; n: PNode, newType: PType, check: bool)
proc semLambda(c: PContext, n: PNode, flags: TExprFlags): PNode
proc semTypeNode(c: PContext, n: PNode, prev: PType): PType
proc semStmt(c: PContext, n: PNode): PNode
proc semOpAux(c: PContext, n: PNode)
proc semParamList(c: PContext, n, genericParams: PNode, s: PSym)
proc addParams(c: PContext, n: PNode, kind: TSymKind)
proc maybeAddResult(c: PContext, s: PSym, n: PNode)
@ -64,14 +65,14 @@ template semIdeForTemplateOrGeneric(c: PContext; n: PNode;
# templates perform some quick check whether the cursor is actually in
# the generic or template.
when defined(nimsuggest):
if gCmd == cmdIdeTools and requiresCheck:
if c.config.cmd == cmdIdeTools and requiresCheck:
#if optIdeDebug in gGlobalOptions:
# echo "passing to safeSemExpr: ", renderTree(n)
discard safeSemExpr(c, n)
proc fitNode(c: PContext, formal: PType, arg: PNode; info: TLineInfo): PNode =
if arg.typ.isNil:
localError(arg.info, errExprXHasNoType,
localError(c.config, arg.info, "expression has no type: " &
renderTree(arg, {renderNoComments}))
# error correction:
result = copyTree(arg)
@ -79,14 +80,14 @@ proc fitNode(c: PContext, formal: PType, arg: PNode; info: TLineInfo): PNode =
else:
result = indexTypesMatch(c, formal, arg.typ, arg)
if result == nil:
typeMismatch(info, formal, arg.typ)
typeMismatch(c.config, info, formal, arg.typ)
# error correction:
result = copyTree(arg)
result.typ = formal
else:
let x = result.skipConv
if x.kind == nkPar and formal.kind != tyExpr:
changeType(x, formal, check=true)
if x.kind in {nkPar, nkTupleConstr} and formal.kind != tyExpr:
changeType(c, x, formal, check=true)
else:
result = skipHiddenSubConv(result)
#result.typ = takeType(formal, arg.typ)
@ -102,8 +103,8 @@ proc commonType*(x, y: PType): PType =
# if expressions, etc.:
if x == nil: return x
if y == nil: return y
var a = skipTypes(x, {tyGenericInst, tyAlias})
var b = skipTypes(y, {tyGenericInst, tyAlias})
var a = skipTypes(x, {tyGenericInst, tyAlias, tySink})
var b = skipTypes(y, {tyGenericInst, tyAlias, tySink})
result = x
if a.kind in {tyExpr, tyNil}: result = y
elif b.kind in {tyExpr, tyNil}: result = x
@ -153,14 +154,17 @@ proc commonType*(x, y: PType): PType =
if a.kind in {tyRef, tyPtr}:
k = a.kind
if b.kind != a.kind: return x
a = a.lastSon
b = b.lastSon
# bug #7601, array construction of ptr generic
a = a.lastSon.skipTypes({tyGenericInst})
b = b.lastSon.skipTypes({tyGenericInst})
if a.kind == tyObject and b.kind == tyObject:
result = commonSuperclass(a, b)
# this will trigger an error later:
if result.isNil or result == a: return x
if result == b: return y
if k != tyNone:
# bug #7906, tyRef/tyPtr + tyGenericInst of ref/ptr object ->
# ill-formed AST, no need for additional tyRef/tyPtr
if k != tyNone and x.kind != tyGenericInst:
let r = result
result = newType(k, r.owner)
result.addSonSkipIntLit(r)
@ -179,7 +183,7 @@ proc commonType*(x: PType, y: PNode): PType =
commonType(x, y.typ)
proc newSymS(kind: TSymKind, n: PNode, c: PContext): PSym =
result = newSym(kind, considerQuotedIdent(n), getCurrOwner(c), n.info)
result = newSym(kind, considerQuotedIdent(c.config, n), getCurrOwner(c), n.info)
when defined(nimsuggest):
suggestDecl(c, n, result)
@ -191,7 +195,7 @@ proc newSymG*(kind: TSymKind, n: PNode, c: PContext): PSym =
# and sfGenSym in n.sym.flags:
result = n.sym
if result.kind != kind:
localError(n.info, "cannot use symbol of kind '" &
localError(c.config, n.info, "cannot use symbol of kind '" &
$result.kind & "' as a '" & $kind & "'")
if sfGenSym in result.flags and result.kind notin {skTemplate, skMacro, skParam}:
# declarative context, so produce a fresh gensym:
@ -203,7 +207,7 @@ proc newSymG*(kind: TSymKind, n: PNode, c: PContext): PSym =
# template; we must fix it here: see #909
result.owner = getCurrOwner(c)
else:
result = newSym(kind, considerQuotedIdent(n), getCurrOwner(c), n.info)
result = newSym(kind, considerQuotedIdent(c.config, n), getCurrOwner(c), n.info)
#if kind in {skForVar, skLet, skVar} and result.owner.kind == skModule:
# incl(result.flags, sfGlobal)
when defined(nimsuggest):
@ -215,15 +219,20 @@ proc semIdentVis(c: PContext, kind: TSymKind, n: PNode,
proc semIdentWithPragma(c: PContext, kind: TSymKind, n: PNode,
allowed: TSymFlags): PSym
proc typeAllowedCheck(info: TLineInfo; typ: PType; kind: TSymKind) =
let t = typeAllowed(typ, kind)
proc typeAllowedCheck(conf: ConfigRef; info: TLineInfo; typ: PType; kind: TSymKind;
flags: TTypeAllowedFlags = {}) =
let t = typeAllowed(typ, kind, flags)
if t != nil:
if t == typ: localError(info, "invalid type: '" & typeToString(typ) & "'")
else: localError(info, "invalid type: '" & typeToString(t) &
"' in this context: '" & typeToString(typ) & "'")
if t == typ:
localError(conf, info, "invalid type: '" & typeToString(typ) &
"' for " & substr($kind, 2).toLowerAscii)
else:
localError(conf, info, "invalid type: '" & typeToString(t) &
"' in this context: '" & typeToString(typ) &
"' for " & substr($kind, 2).toLowerAscii)
proc paramsTypeCheck(c: PContext, typ: PType) {.inline.} =
typeAllowedCheck(typ.n.info, typ, skProc)
typeAllowedCheck(c.config, typ.n.info, typ, skProc)
proc expectMacroOrTemplateCall(c: PContext, n: PNode): PSym
proc semDirectOp(c: PContext, n: PNode, flags: TExprFlags): PNode
@ -276,10 +285,10 @@ proc fixupTypeAfterEval(c: PContext, evaluated, eOrig: PNode): PNode =
result = evaluated
let expectedType = eOrig.typ.skipTypes({tyStatic})
if hasCycle(result):
globalError(eOrig.info, "the resulting AST is cyclic and cannot be processed further")
globalError(c.config, eOrig.info, "the resulting AST is cyclic and cannot be processed further")
result = errorNode(c, eOrig)
else:
semmacrosanity.annotateType(result, expectedType)
semmacrosanity.annotateType(result, expectedType, c.config)
else:
result = semExprWithType(c, evaluated)
#result = fitNode(c, e.typ, result) inlined with special case:
@ -296,18 +305,18 @@ proc tryConstExpr(c: PContext, n: PNode): PNode =
var e = semExprWithType(c, n)
if e == nil: return
result = getConstExpr(c.module, e)
result = getConstExpr(c.module, e, c.graph)
if result != nil: return
let oldErrorCount = msgs.gErrorCounter
let oldErrorMax = msgs.gErrorMax
let oldErrorCount = c.config.errorCounter
let oldErrorMax = c.config.errorMax
let oldErrorOutputs = errorOutputs
errorOutputs = {}
msgs.gErrorMax = high(int)
c.config.errorMax = high(int)
try:
result = evalConstExpr(c.module, c.cache, e)
result = evalConstExpr(c.module, c.cache, c.graph, e)
if result == nil or result.kind == nkEmpty:
result = nil
else:
@ -316,26 +325,29 @@ proc tryConstExpr(c: PContext, n: PNode): PNode =
except ERecoverableError:
result = nil
msgs.gErrorCounter = oldErrorCount
msgs.gErrorMax = oldErrorMax
c.config.errorCounter = oldErrorCount
c.config.errorMax = oldErrorMax
errorOutputs = oldErrorOutputs
const
errConstExprExpected = "constant expression expected"
proc semConstExpr(c: PContext, n: PNode): PNode =
var e = semExprWithType(c, n)
if e == nil:
localError(n.info, errConstExprExpected)
localError(c.config, n.info, errConstExprExpected)
return n
result = getConstExpr(c.module, e)
result = getConstExpr(c.module, e, c.graph)
if result == nil:
#if e.kind == nkEmpty: globalError(n.info, errConstExprExpected)
result = evalConstExpr(c.module, c.cache, e)
result = evalConstExpr(c.module, c.cache, c.graph, e)
if result == nil or result.kind == nkEmpty:
if e.info != n.info:
pushInfoContext(n.info)
localError(e.info, errConstExprExpected)
localError(c.config, e.info, errConstExprExpected)
popInfoContext()
else:
localError(e.info, errConstExprExpected)
localError(c.config, e.info, errConstExprExpected)
# error correction:
result = e
else:
@ -350,7 +362,7 @@ proc semExprFlagDispatched(c: PContext, n: PNode, flags: TExprFlags): PNode =
else:
result = semExprWithType(c, n, flags)
if efPreferStatic in flags:
var evaluated = getConstExpr(c.module, result)
var evaluated = getConstExpr(c.module, result, c.graph)
if evaluated != nil: return evaluated
evaluated = evalAtCompileTime(c, result)
if evaluated != nil: return evaluated
@ -372,8 +384,8 @@ proc semAfterMacroCall(c: PContext, call, macroResult: PNode,
## reassigned, and binding the unbound identifiers that the macro output
## contains.
inc(evalTemplateCounter)
if evalTemplateCounter > 100:
globalError(s.info, errTemplateInstantiationTooNested)
if evalTemplateCounter > evalTemplateLimit:
globalError(c.config, s.info, "template instantiation too nested")
c.friendModules.add(s.owner.getModule)
result = macroResult
@ -415,43 +427,46 @@ proc semAfterMacroCall(c: PContext, call, macroResult: PNode,
dec(evalTemplateCounter)
discard c.friendModules.pop()
const
errMissingGenericParamsForTemplate = "'$1' has unspecified generic parameters"
proc semMacroExpr(c: PContext, n, nOrig: PNode, sym: PSym,
flags: TExprFlags = {}): PNode =
pushInfoContext(nOrig.info)
markUsed(n.info, sym, c.graph.usageSym)
markUsed(c.config, n.info, sym, c.graph.usageSym)
styleCheckUse(n.info, sym)
if sym == c.p.owner:
globalError(n.info, errRecursiveDependencyX, sym.name.s)
globalError(c.config, n.info, "recursive dependency: '$1'" % sym.name.s)
let genericParams = if sfImmediate in sym.flags: 0
else: sym.ast[genericParamsPos].len
let suppliedParams = max(n.safeLen - 1, 0)
if suppliedParams < genericParams:
globalError(n.info, errMissingGenericParamsForTemplate, n.renderTree)
globalError(c.config, n.info, errMissingGenericParamsForTemplate % n.renderTree)
#if c.evalContext == nil:
# c.evalContext = c.createEvalContext(emStatic)
result = evalMacroCall(c.module, c.cache, n, nOrig, sym)
result = evalMacroCall(c.module, c.cache, c.graph, n, nOrig, sym)
if efNoSemCheck notin flags:
result = semAfterMacroCall(c, n, result, sym, flags)
result = wrapInComesFrom(nOrig.info, sym, result)
popInfoContext()
proc forceBool(c: PContext, n: PNode): PNode =
result = fitNode(c, getSysType(tyBool), n, n.info)
result = fitNode(c, getSysType(c.graph, n.info, tyBool), n, n.info)
if result == nil: result = n
proc semConstBoolExpr(c: PContext, n: PNode): PNode =
let nn = semExprWithType(c, n)
result = fitNode(c, getSysType(tyBool), nn, nn.info)
result = fitNode(c, getSysType(c.graph, n.info, tyBool), nn, nn.info)
if result == nil:
localError(n.info, errConstExprExpected)
localError(c.config, n.info, errConstExprExpected)
return nn
result = getConstExpr(c.module, result)
result = getConstExpr(c.module, result, c.graph)
if result == nil:
localError(n.info, errConstExprExpected)
localError(c.config, n.info, errConstExprExpected)
result = nn
proc semGenericStmt(c: PContext, n: PNode): PNode
@ -464,14 +479,14 @@ proc addCodeForGenerics(c: PContext, n: PNode) =
var prc = c.generics[i].inst.sym
if prc.kind in {skProc, skFunc, skMethod, skConverter} and prc.magic == mNone:
if prc.ast == nil or prc.ast.sons[bodyPos] == nil:
internalError(prc.info, "no code for " & prc.name.s)
internalError(c.config, prc.info, "no code for " & prc.name.s)
else:
addSon(n, prc.ast)
c.lastGenericIdx = c.generics.len
proc myOpen(graph: ModuleGraph; module: PSym; cache: IdentCache): PPassContext =
var c = newContext(graph, module, cache)
if c.p != nil: internalError(module.info, "sem.myOpen")
if c.p != nil: internalError(graph.config, module.info, "sem.myOpen")
c.semConstExpr = semConstExpr
c.semExpr = semExpr
c.semTryExpr = tryExpr
@ -489,14 +504,14 @@ proc myOpen(graph: ModuleGraph; module: PSym; cache: IdentCache): PPassContext =
c.importTable = openScope(c)
c.importTable.addSym(module) # a module knows itself
if sfSystemModule in module.flags:
magicsys.systemModule = module # set global variable!
graph.systemModule = module
c.topLevelScope = openScope(c)
# don't be verbose unless the module belongs to the main package:
if module.owner.id == gMainPackageId:
gNotes = gMainPackageNotes
graph.config.notes = graph.config.mainPackageNotes
else:
if gMainPackageNotes == {}: gMainPackageNotes = gNotes
gNotes = ForeignPackageNotes
if graph.config.mainPackageNotes == {}: graph.config.mainPackageNotes = graph.config.notes
graph.config.notes = graph.config.foreignPackageNotes
result = c
proc myOpenCached(graph: ModuleGraph; module: PSym; rd: PRodReader): PPassContext =
@ -505,30 +520,30 @@ proc myOpenCached(graph: ModuleGraph; module: PSym; rd: PRodReader): PPassContex
proc replayMethodDefs(graph: ModuleGraph; rd: PRodReader) =
for m in items(rd.methods): methodDef(graph, m, true)
proc isImportSystemStmt(n: PNode): bool =
if magicsys.systemModule == nil: return false
proc isImportSystemStmt(g: ModuleGraph; n: PNode): bool =
if g.systemModule == nil: return false
case n.kind
of nkImportStmt:
for x in n:
if x.kind == nkIdent:
let f = checkModuleName(x, false)
if f == magicsys.systemModule.info.fileIndex:
let f = checkModuleName(g.config, x, false)
if f == g.systemModule.info.fileIndex:
return true
of nkImportExceptStmt, nkFromStmt:
if n[0].kind == nkIdent:
let f = checkModuleName(n[0], false)
if f == magicsys.systemModule.info.fileIndex:
let f = checkModuleName(g.config, n[0], false)
if f == g.systemModule.info.fileIndex:
return true
else: discard
proc semStmtAndGenerateGenerics(c: PContext, n: PNode): PNode =
if n.kind == nkDefer:
localError(n.info, "defer statement not supported at top level")
if c.topStmts == 0 and not isImportSystemStmt(n):
localError(c.config, n.info, "defer statement not supported at top level")
if c.topStmts == 0 and not isImportSystemStmt(c.graph, n):
if sfSystemModule notin c.module.flags and
n.kind notin {nkEmpty, nkCommentStmt}:
c.importTable.addSym magicsys.systemModule # import the "System" identifier
importAllSymbols(c, magicsys.systemModule)
c.importTable.addSym c.graph.systemModule # import the "System" identifier
importAllSymbols(c, c.graph.systemModule)
inc c.topStmts
else:
inc c.topStmts
@ -550,11 +565,11 @@ proc semStmtAndGenerateGenerics(c: PContext, n: PNode): PNode =
if result.kind != nkEmpty: addSon(a, result)
result = a
result = hloStmt(c, result)
if gCmd == cmdInteractive and not isEmptyType(result.typ):
if c.config.cmd == cmdInteractive and not isEmptyType(result.typ):
result = buildEchoStmt(c, result)
if gCmd == cmdIdeTools:
if c.config.cmd == cmdIdeTools:
appendToModule(c.module, result)
result = transformStmt(c.module, result)
result = transformStmt(c.graph, c.module, result)
proc recoverContext(c: PContext) =
# clean up in case of a semantic error: We clean up the stacks, etc. This is
@ -567,7 +582,7 @@ proc recoverContext(c: PContext) =
proc myProcess(context: PPassContext, n: PNode): PNode =
var c = PContext(context)
# no need for an expensive 'try' if we stop after the first error anyway:
if msgs.gErrorMax <= 1:
if c.config.errorMax <= 1:
result = semStmtAndGenerateGenerics(c, n)
else:
let oldContextLen = msgs.getInfoContextLen()
@ -583,15 +598,16 @@ proc myProcess(context: PPassContext, n: PNode): PNode =
result = nil
else:
result = ast.emptyNode
#if gCmd == cmdIdeTools: findSuggest(c, n)
#if c.config.cmd == cmdIdeTools: findSuggest(c, n)
rod.storeNode(c.module, result)
proc testExamples(c: PContext) =
let inp = toFullPath(c.module.info)
let outp = inp.changeFileExt"" & "_examples.nim"
renderModule(c.runnableExamples, inp, outp)
let backend = if isDefined("js"): "js"
elif isDefined("cpp"): "cpp"
elif isDefined("objc"): "objc"
let backend = if isDefined(c.config, "js"): "js"
elif isDefined(c.config, "cpp"): "cpp"
elif isDefined(c.config, "objc"): "objc"
else: "c"
if os.execShellCmd("nim " & backend & " -r " & outp) != 0:
quit "[Examples] failed"
@ -599,13 +615,13 @@ proc testExamples(c: PContext) =
proc myClose(graph: ModuleGraph; context: PPassContext, n: PNode): PNode =
var c = PContext(context)
if gCmd == cmdIdeTools and not c.suggestionsMade:
if c.config.cmd == cmdIdeTools and not c.suggestionsMade:
suggestSentinel(c)
closeScope(c) # close module's scope
rawCloseScope(c) # imported symbols; don't check for unused ones!
result = newNode(nkStmtList)
if n != nil:
internalError(n.info, "n is not nil") #result := n;
internalError(c.config, n.info, "n is not nil") #result := n;
addCodeForGenerics(c, result)
if c.module.ast != nil:
result.add(c.module.ast)
@ -613,6 +629,8 @@ proc myClose(graph: ModuleGraph; context: PPassContext, n: PNode): PNode =
replayMethodDefs(graph, c.rd)
popOwner(c)
popProcCon(c)
storeRemaining(c.module)
if c.runnableExamples != nil: testExamples(c)
const semPass* = makePass(myOpen, myOpenCached, myProcess, myClose)
const semPass* = makePass(myOpen, myOpenCached, myProcess, myClose,
isFrontend = true)

60
compiler/semasgn.nim
View file

@ -33,7 +33,7 @@ proc at(a, i: PNode, elemType: PType): PNode =
proc liftBodyTup(c: var TLiftCtx; t: PType; body, x, y: PNode) =
for i in 0 ..< t.len:
let lit = lowerings.newIntLit(i)
let lit = lowerings.newIntLit(c.c.graph, x.info, i)
liftBodyAux(c, t.sons[i], body, x.at(lit, t.sons[i]), y.at(lit, t.sons[i]))
proc dotField(x: PNode, f: PSym): PNode =
@ -66,15 +66,15 @@ proc liftBodyObj(c: var TLiftCtx; n, body, x, y: PNode) =
liftBodyObj(c, n[i].lastSon, branch.sons[L-1], x, y)
caseStmt.add(branch)
body.add(caseStmt)
localError(c.info, "cannot lift assignment operator to 'case' object")
localError(c.c.config, c.info, "cannot lift assignment operator to 'case' object")
of nkRecList:
for t in items(n): liftBodyObj(c, t, body, x, y)
else:
illFormedAstLocal(n)
illFormedAstLocal(n, c.c.config)
proc genAddr(c: PContext; x: PNode): PNode =
if x.kind == nkHiddenDeref:
checkSonsLen(x, 1)
checkSonsLen(x, 1, c.config)
result = x.sons[0]
else:
result = newNodeIT(nkHiddenAddr, x.info, makeVarType(c, x.typ))
@ -82,7 +82,7 @@ proc genAddr(c: PContext; x: PNode): PNode =
proc newAsgnCall(c: PContext; op: PSym; x, y: PNode): PNode =
if sfError in op.flags:
localError(x.info, errWrongSymbolX, op.name.s)
localError(c.config, x.info, "usage of '$1' is a user-defined error" % op.name.s)
result = newNodeI(nkCall, x.info)
result.add newSymNode(op)
result.add genAddr(c, x)
@ -101,7 +101,7 @@ proc newOpCall(op: PSym; x: PNode): PNode =
proc destructorCall(c: PContext; op: PSym; x: PNode): PNode =
result = newNodeIT(nkCall, x.info, op.typ.sons[0])
result.add(newSymNode(op))
if newDestructors:
if destructor in c.features:
result.add genAddr(c, x)
else:
result.add x
@ -124,7 +124,7 @@ proc considerAsgnOrSink(c: var TLiftCtx; t: PType; body, x, y: PNode;
op = field
if op == nil:
op = liftBody(c.c, t, c.kind, c.info)
markUsed(c.info, op, c.c.graph.usageSym)
markUsed(c.c.config, c.info, op, c.c.graph.usageSym)
styleCheckUse(c.info, op)
body.add newAsgnCall(c.c, op, x, y)
result = true
@ -134,7 +134,7 @@ proc considerOverloadedOp(c: var TLiftCtx; t: PType; body, x, y: PNode): bool =
of attachedDestructor:
let op = t.destructor
if op != nil:
markUsed(c.info, op, c.c.graph.usageSym)
markUsed(c.c.config, c.info, op, c.c.graph.usageSym)
styleCheckUse(c.info, op)
body.add destructorCall(c.c, op, x)
result = true
@ -145,7 +145,7 @@ proc considerOverloadedOp(c: var TLiftCtx; t: PType; body, x, y: PNode): bool =
of attachedDeepCopy:
let op = t.deepCopy
if op != nil:
markUsed(c.info, op, c.c.graph.usageSym)
markUsed(c.c.config, c.info, op, c.c.graph.usageSym)
styleCheckUse(c.info, op)
body.add newDeepCopyCall(op, x, y)
result = true
@ -163,37 +163,37 @@ proc addVar(father, v, value: PNode) =
proc declareCounter(c: var TLiftCtx; body: PNode; first: BiggestInt): PNode =
var temp = newSym(skTemp, getIdent(lowerings.genPrefix), c.fn, c.info)
temp.typ = getSysType(tyInt)
temp.typ = getSysType(c.c.graph, body.info, tyInt)
incl(temp.flags, sfFromGeneric)
var v = newNodeI(nkVarSection, c.info)
result = newSymNode(temp)
v.addVar(result, lowerings.newIntLit(first))
v.addVar(result, lowerings.newIntLit(c.c.graph, body.info, first))
body.add v
proc genBuiltin(magic: TMagic; name: string; i: PNode): PNode =
proc genBuiltin(g: ModuleGraph; magic: TMagic; name: string; i: PNode): PNode =
result = newNodeI(nkCall, i.info)
result.add createMagic(name, magic).newSymNode
result.add createMagic(g, name, magic).newSymNode
result.add i
proc genWhileLoop(c: var TLiftCtx; i, dest: PNode): PNode =
result = newNodeI(nkWhileStmt, c.info, 2)
let cmp = genBuiltin(mLeI, "<=", i)
cmp.add genHigh(dest)
cmp.typ = getSysType(tyBool)
let cmp = genBuiltin(c.c.graph, mLeI, "<=", i)
cmp.add genHigh(c.c.graph, dest)
cmp.typ = getSysType(c.c.graph, c.info, tyBool)
result.sons[0] = cmp
result.sons[1] = newNodeI(nkStmtList, c.info)
proc addIncStmt(body, i: PNode) =
let incCall = genBuiltin(mInc, "inc", i)
incCall.add lowerings.newIntLit(1)
proc addIncStmt(c: var TLiftCtx; body, i: PNode) =
let incCall = genBuiltin(c.c.graph, mInc, "inc", i)
incCall.add lowerings.newIntLit(c.c.graph, c.info, 1)
body.add incCall
proc newSeqCall(c: PContext; x, y: PNode): PNode =
# don't call genAddr(c, x) here:
result = genBuiltin(mNewSeq, "newSeq", x)
let lenCall = genBuiltin(mLengthSeq, "len", y)
lenCall.typ = getSysType(tyInt)
result = genBuiltin(c.graph, mNewSeq, "newSeq", x)
let lenCall = genBuiltin(c.graph, mLengthSeq, "len", y)
lenCall.typ = getSysType(c.graph, x.info, tyInt)
result.add lenCall
proc liftBodyAux(c: var TLiftCtx; t: PType; body, x, y: PNode) =
@ -212,7 +212,7 @@ proc liftBodyAux(c: var TLiftCtx; t: PType; body, x, y: PNode) =
let elemType = t.lastSon
liftBodyAux(c, elemType, whileLoop.sons[1], x.at(i, elemType),
y.at(i, elemType))
addIncStmt(whileLoop.sons[1], i)
addIncStmt(c, whileLoop.sons[1], i)
body.add whileLoop
else:
defaultOp(c, t, body, x, y)
@ -231,20 +231,20 @@ proc liftBodyAux(c: var TLiftCtx; t: PType; body, x, y: PNode) =
# have to go through some indirection; we delegate this to the codegen:
let call = newNodeI(nkCall, c.info, 2)
call.typ = t
call.sons[0] = newSymNode(createMagic("deepCopy", mDeepCopy))
call.sons[0] = newSymNode(createMagic(c.c.graph, "deepCopy", mDeepCopy))
call.sons[1] = y
body.add newAsgnStmt(x, call)
of tyVarargs, tyOpenArray:
localError(c.info, errGenerated, "cannot copy openArray")
localError(c.c.config, c.info, "cannot copy openArray")
of tyFromExpr, tyProxy, tyBuiltInTypeClass, tyUserTypeClass,
tyUserTypeClassInst, tyCompositeTypeClass, tyAnd, tyOr, tyNot, tyAnything,
tyGenericParam, tyGenericBody, tyNil, tyExpr, tyStmt,
tyTypeDesc, tyGenericInvocation, tyForward:
internalError(c.info, "assignment requested for type: " & typeToString(t))
internalError(c.c.config, c.info, "assignment requested for type: " & typeToString(t))
of tyOrdinal, tyRange, tyInferred,
tyGenericInst, tyStatic, tyVar, tyAlias:
tyGenericInst, tyStatic, tyVar, tyLent, tyAlias, tySink:
liftBodyAux(c, lastSon(t), body, x, y)
of tyUnused, tyOptAsRef, tyUnused1, tyUnused2: internalError("liftBodyAux")
of tyUnused, tyOptAsRef: internalError(c.c.config, "liftBodyAux")
proc newProcType(info: TLineInfo; owner: PSym): PType =
result = newType(tyProc, owner)
@ -306,7 +306,7 @@ proc liftBody(c: PContext; typ: PType; kind: TTypeAttachedOp;
proc getAsgnOrLiftBody(c: PContext; typ: PType; info: TLineInfo): PSym =
let t = typ.skipTypes({tyGenericInst, tyVar, tyAlias})
let t = typ.skipTypes({tyGenericInst, tyVar, tyLent, tyAlias, tySink})
result = t.assignment
if result.isNil:
result = liftBody(c, t, attachedAsgn, info)
@ -319,7 +319,7 @@ proc liftTypeBoundOps*(c: PContext; typ: PType; info: TLineInfo) =
## In the semantic pass this is called in strategic places
## to ensure we lift assignment, destructors and moves properly.
## The later 'destroyer' pass depends on it.
if not newDestructors or not hasDestructor(typ): return
if destructor notin c.features or not hasDestructor(typ): return
when false:
# do not produce wrong liftings while we're still instantiating generics:
# now disabled; breaks topttree.nim!

140
compiler/semcall.nim
View file

@ -59,7 +59,8 @@ proc pickBestCandidate(c: PContext, headSymbol: PNode,
filter: TSymKinds,
best, alt: var TCandidate,
errors: var CandidateErrors,
diagnosticsFlag = false) =
diagnosticsFlag: bool,
errorsEnabled: bool) =
var o: TOverloadIter
var sym = initOverloadIter(o, c, headSymbol)
var scope = o.lastOverloadScope
@ -68,6 +69,7 @@ proc pickBestCandidate(c: PContext, headSymbol: PNode,
# This can occur in cases like 'init(a, 1, (var b = new(Type2); b))'
let counterInitial = c.currentScope.symbols.counter
var syms: seq[tuple[s: PSym, scope: int]]
var noSyms = true
var nextSymIndex = 0
while sym != nil:
if sym.kind in filter:
@ -102,17 +104,19 @@ proc pickBestCandidate(c: PContext, headSymbol: PNode,
var cmp = cmpCandidates(best, z)
if cmp < 0: best = z # x is better than the best so far
elif cmp == 0: alt = z # x is as good as the best so far
elif errors != nil or z.diagnostics != nil:
elif errorsEnabled or z.diagnosticsEnabled:
errors.safeAdd(CandidateError(
sym: sym,
unmatchedVarParam: int z.mutabilityProblem,
firstMismatch: z.firstMismatch,
diagnostics: z.diagnostics))
else:
# Symbol table has been modified. Restart and pre-calculate all syms
# before any further candidate init and compare. SLOW, but rare case.
syms = initCandidateSymbols(c, headSymbol, initialBinding, filter,
best, alt, o, diagnosticsFlag)
if syms == nil:
noSyms = false
if noSyms:
sym = nextOverloadIter(o, c, headSymbol)
scope = o.lastOverloadScope
elif nextSymIndex < syms.len:
@ -123,6 +127,20 @@ proc pickBestCandidate(c: PContext, headSymbol: PNode,
else:
break
proc effectProblem(f, a: PType; result: var string) =
if f.kind == tyProc and a.kind == tyProc:
if tfThread in f.flags and tfThread notin a.flags:
result.add "\n This expression is not GC-safe. Annotate the " &
"proc with {.gcsafe.} to get extended error information."
elif tfNoSideEffect in f.flags and tfNoSideEffect notin a.flags:
result.add "\n This expression can have side effects. Annotate the " &
"proc with {.noSideEffect.} to get extended error information."
proc renderNotLValue(n: PNode): string =
result = $n
if n.kind in {nkHiddenStdConv, nkHiddenSubConv, nkHiddenCallConv} and n.len == 2:
result = typeToString(n.typ.skipTypes(abstractVar)) & "(" & result & ")"
proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
(TPreferedDesc, string) =
var prefer = preferName
@ -154,32 +172,61 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
else:
add(candidates, err.sym.getProcHeader(prefer))
add(candidates, "\n")
if err.firstMismatch != 0 and n.len > 1:
let cond = n.len > 2
if cond:
candidates.add(" first type mismatch at position: " & $err.firstMismatch &
"\n required type: ")
var wanted, got: PType = nil
if err.firstMismatch < err.sym.typ.len:
wanted = err.sym.typ.sons[err.firstMismatch]
if cond: candidates.add typeToString(wanted)
else:
if cond: candidates.add "none"
if err.firstMismatch < n.len:
if cond:
candidates.add "\n but expression '"
candidates.add renderTree(n[err.firstMismatch])
candidates.add "' is of type: "
got = n[err.firstMismatch].typ
if cond: candidates.add typeToString(got)
if wanted != nil and got != nil:
effectProblem(wanted, got, candidates)
if cond: candidates.add "\n"
if err.unmatchedVarParam != 0 and err.unmatchedVarParam < n.len:
add(candidates, "for a 'var' type a variable needs to be passed, but '" &
renderTree(n[err.unmatchedVarParam]) & "' is immutable\n")
candidates.add(" for a 'var' type a variable needs to be passed, but '" &
renderNotLValue(n[err.unmatchedVarParam]) &
"' is immutable\n")
for diag in err.diagnostics:
add(candidates, diag & "\n")
candidates.add(diag & "\n")
result = (prefer, candidates)
const
errTypeMismatch = "type mismatch: got <"
errButExpected = "but expected one of: "
errUndeclaredField = "undeclared field: '$1'"
errUndeclaredRoutine = "attempting to call undeclared routine: '$1'"
errAmbiguousCallXYZ = "ambiguous call; both $1 and $2 match for: $3"
proc notFoundError*(c: PContext, n: PNode, errors: CandidateErrors) =
# Gives a detailed error message; this is separated from semOverloadedCall,
# as semOverlodedCall is already pretty slow (and we need this information
# only in case of an error).
if errorOutputs == {}:
# fail fast:
globalError(n.info, errTypeMismatch, "")
if errors.isNil or errors.len == 0:
localError(n.info, errExprXCannotBeCalled, n[0].renderTree)
globalError(c.config, n.info, "type mismatch")
if errors.len == 0:
localError(c.config, n.info, "expression '$1' cannot be called" % n[0].renderTree)
return
let (prefer, candidates) = presentFailedCandidates(c, n, errors)
var result = msgKindToString(errTypeMismatch)
var result = errTypeMismatch
add(result, describeArgs(c, n, 1, prefer))
add(result, ')')
add(result, '>')
if candidates != "":
add(result, "\n" & msgKindToString(errButExpected) & "\n" & candidates)
localError(n.info, errGenerated, result & "\nexpression: " & $n)
add(result, "\n" & errButExpected & "\n" & candidates)
localError(c.config, n.info, result & "\nexpression: " & $n)
proc bracketNotFoundError(c: PContext; n: PNode) =
var errors: CandidateErrors = @[]
@ -189,22 +236,25 @@ proc bracketNotFoundError(c: PContext; n: PNode) =
while symx != nil:
if symx.kind in routineKinds:
errors.add(CandidateError(sym: symx,
unmatchedVarParam: 0,
diagnostics: nil))
unmatchedVarParam: 0, firstMismatch: 0,
diagnostics: nil,
enabled: false))
symx = nextOverloadIter(o, c, headSymbol)
if errors.len == 0:
localError(n.info, "could not resolve: " & $n)
localError(c.config, n.info, "could not resolve: " & $n)
else:
notFoundError(c, n, errors)
proc resolveOverloads(c: PContext, n, orig: PNode,
filter: TSymKinds, flags: TExprFlags,
errors: var CandidateErrors): TCandidate =
errors: var CandidateErrors,
errorsEnabled: bool): TCandidate =
var initialBinding: PNode
var alt: TCandidate
var f = n.sons[0]
if f.kind == nkBracketExpr:
# fill in the bindings:
semOpAux(c, f)
initialBinding = f
f = f.sons[0]
else:
@ -212,7 +262,8 @@ proc resolveOverloads(c: PContext, n, orig: PNode,
template pickBest(headSymbol) =
pickBestCandidate(c, headSymbol, n, orig, initialBinding,
filter, result, alt, errors, efExplain in flags)
filter, result, alt, errors, efExplain in flags,
errorsEnabled)
pickBest(f)
let overloadsState = result.state
@ -234,7 +285,7 @@ proc resolveOverloads(c: PContext, n, orig: PNode,
else: return
if nfDotField in n.flags:
internalAssert f.kind == nkIdent and n.sonsLen >= 2
internalAssert c.config, f.kind == nkIdent and n.len >= 2
# leave the op head symbol empty,
# we are going to try multiple variants
@ -263,13 +314,13 @@ proc resolveOverloads(c: PContext, n, orig: PNode,
if overloadsState == csEmpty and result.state == csEmpty:
if nfDotField in n.flags and nfExplicitCall notin n.flags:
localError(n.info, errUndeclaredField, considerQuotedIdent(f).s)
localError(c.config, n.info, errUndeclaredField % considerQuotedIdent(c.config, f, n).s)
else:
localError(n.info, errUndeclaredRoutine, considerQuotedIdent(f).s)
localError(c.config, n.info, errUndeclaredRoutine % considerQuotedIdent(c.config, f, n).s)
return
elif result.state != csMatch:
if nfExprCall in n.flags:
localError(n.info, errExprXCannotBeCalled,
localError(c.config, n.info, "expression '$1' cannot be called" %
renderTree(n, {renderNoComments}))
else:
if {nfDotField, nfDotSetter} * n.flags != {}:
@ -279,13 +330,13 @@ proc resolveOverloads(c: PContext, n, orig: PNode,
return
if alt.state == csMatch and cmpCandidates(result, alt) == 0 and
not sameMethodDispatcher(result.calleeSym, alt.calleeSym):
internalAssert result.state == csMatch
internalAssert c.config, result.state == csMatch
#writeMatches(result)
#writeMatches(alt)
if errorOutputs == {}:
# quick error message for performance of 'compiles' built-in:
globalError(n.info, errGenerated, "ambiguous call")
elif gErrorCounter == 0:
globalError(c.config, n.info, errGenerated, "ambiguous call")
elif c.config.errorCounter == 0:
# don't cascade errors
var args = "("
for i in countup(1, sonsLen(n) - 1):
@ -293,7 +344,7 @@ proc resolveOverloads(c: PContext, n, orig: PNode,
add(args, typeToString(n.sons[i].typ))
add(args, ")")
localError(n.info, errGenerated, msgKindToString(errAmbiguousCallXYZ) % [
localError(c.config, n.info, errAmbiguousCallXYZ % [
getProcHeader(result.calleeSym), getProcHeader(alt.calleeSym),
args])
@ -334,7 +385,7 @@ proc inferWithMetatype(c: PContext, formal: PType,
result.typ = generateTypeInstance(c, m.bindings, arg.info,
formal.skipTypes({tyCompositeTypeClass}))
else:
typeMismatch(arg.info, formal, arg.typ)
typeMismatch(c.config, arg.info, formal, arg.typ)
# error correction:
result = copyTree(arg)
result.typ = formal
@ -342,7 +393,7 @@ proc inferWithMetatype(c: PContext, formal: PType,
proc semResolvedCall(c: PContext, n: PNode, x: TCandidate): PNode =
assert x.state == csMatch
var finalCallee = x.calleeSym
markUsed(n.sons[0].info, finalCallee, c.graph.usageSym)
markUsed(c.config, n.sons[0].info, finalCallee, c.graph.usageSym)
styleCheckUse(n.sons[0].info, finalCallee)
assert finalCallee.ast != nil
if x.hasFauxMatch:
@ -368,7 +419,7 @@ proc semResolvedCall(c: PContext, n: PNode, x: TCandidate): PNode =
of skType:
x.call.add newSymNode(s, n.info)
else:
internalAssert false
internalAssert c.config, false
result = x.call
instGenericConvertersSons(c, result, x)
@ -377,7 +428,7 @@ proc semResolvedCall(c: PContext, n: PNode, x: TCandidate): PNode =
proc canDeref(n: PNode): bool {.inline.} =
result = n.len >= 2 and (let t = n[1].typ;
t != nil and t.skipTypes({tyGenericInst, tyAlias}).kind in {tyPtr, tyRef})
t != nil and t.skipTypes({tyGenericInst, tyAlias, tySink}).kind in {tyPtr, tyRef})
proc tryDeref(n: PNode): PNode =
result = newNodeI(nkHiddenDeref, n.info)
@ -386,18 +437,17 @@ proc tryDeref(n: PNode): PNode =
proc semOverloadedCall(c: PContext, n, nOrig: PNode,
filter: TSymKinds, flags: TExprFlags): PNode =
var errors: CandidateErrors = if efExplain in flags: @[]
else: nil
var r = resolveOverloads(c, n, nOrig, filter, flags, errors)
var errors: CandidateErrors = if efExplain in flags: @[] else: nil
var r = resolveOverloads(c, n, nOrig, filter, flags, errors, efExplain in flags)
if r.state == csMatch:
# this may be triggered, when the explain pragma is used
if errors.len > 0:
let (_, candidates) = presentFailedCandidates(c, n, errors)
message(n.info, hintUserRaw,
message(c.config, n.info, hintUserRaw,
"Non-matching candidates for " & renderTree(n) & "\n" &
candidates)
result = semResolvedCall(c, n, r)
elif experimentalMode(c) and canDeref(n):
elif implicitDeref in c.features and canDeref(n):
# try to deref the first argument and then try overloading resolution again:
#
# XXX: why is this here?
@ -406,7 +456,7 @@ proc semOverloadedCall(c: PContext, n, nOrig: PNode,
# into sigmatch with hidden conversion produced there
#
n.sons[1] = n.sons[1].tryDeref
var r = resolveOverloads(c, n, nOrig, filter, flags, errors)
var r = resolveOverloads(c, n, nOrig, filter, flags, errors, efExplain in flags)
if r.state == csMatch: result = semResolvedCall(c, n, r)
else:
# get rid of the deref again for a better error message:
@ -426,8 +476,8 @@ proc semOverloadedCall(c: PContext, n, nOrig: PNode,
else:
notFoundError(c, n, errors)
proc explicitGenericInstError(n: PNode): PNode =
localError(n.info, errCannotInstantiateX, renderTree(n))
proc explicitGenericInstError(c: PContext; n: PNode): PNode =
localError(c.config, n.info, errCannotInstantiateX % renderTree(n))
result = n
proc explicitGenericSym(c: PContext, n: PNode, s: PSym): PNode =
@ -442,7 +492,7 @@ proc explicitGenericSym(c: PContext, n: PNode, s: PSym): PNode =
if tm in {isNone, isConvertible}: return nil
var newInst = generateInstance(c, s, m.bindings, n.info)
newInst.typ.flags.excl tfUnresolved
markUsed(n.info, s, c.graph.usageSym)
markUsed(c.config, n.info, s, c.graph.usageSym)
styleCheckUse(n.info, s)
result = newSymNode(newInst, n.info)
@ -458,11 +508,11 @@ proc explicitGenericInstantiation(c: PContext, n: PNode, s: PSym): PNode =
# number of generic type parameters:
if safeLen(s.ast.sons[genericParamsPos]) != n.len-1:
let expected = safeLen(s.ast.sons[genericParamsPos])
localError(n.info, errGenerated, "cannot instantiate: " & renderTree(n) &
"; got " & $(n.len-1) & " type(s) but expected " & $expected)
localError(c.config, n.info, errGenerated, "cannot instantiate: '" & renderTree(n) &
"'; got " & $(n.len-1) & " type(s) but expected " & $expected)
return n
result = explicitGenericSym(c, n, s)
if result == nil: result = explicitGenericInstError(n)
if result == nil: result = explicitGenericInstError(c, n)
elif a.kind in {nkClosedSymChoice, nkOpenSymChoice}:
# choose the generic proc with the proper number of type parameters.
# XXX I think this could be improved by reusing sigmatch.paramTypesMatch.
@ -480,10 +530,10 @@ proc explicitGenericInstantiation(c: PContext, n: PNode, s: PSym): PNode =
# get rid of nkClosedSymChoice if not ambiguous:
if result.len == 1 and a.kind == nkClosedSymChoice:
result = result[0]
elif result.len == 0: result = explicitGenericInstError(n)
# candidateCount != 1: return explicitGenericInstError(n)
elif result.len == 0: result = explicitGenericInstError(c, n)
# candidateCount != 1: return explicitGenericInstError(c, n)
else:
result = explicitGenericInstError(n)
result = explicitGenericInstError(c, n)
proc searchForBorrowProc(c: PContext, startScope: PScope, fn: PSym): PSym =
# Searchs for the fn in the symbol table. If the parameter lists are suitable

74
compiler/semdata.nim
View file

@ -14,7 +14,7 @@ import
wordrecg,
ropes, msgs, platform, os, condsyms, idents, renderer, types, extccomp, math,
magicsys, nversion, nimsets, parser, times, passes, rodread, vmdef,
modulegraphs
modulegraphs, configuration
type
TOptionEntry* = object # entries to put on a stack for pragma parsing
@ -63,7 +63,7 @@ type
# to the user.
efWantStmt, efAllowStmt, efDetermineType, efExplain,
efAllowDestructor, efWantValue, efOperand, efNoSemCheck,
efNoProcvarCheck, efNoEvaluateGeneric, efInCall, efFromHlo,
efNoEvaluateGeneric, efInCall, efFromHlo
TExprFlags* = set[TExprFlag]
@ -89,6 +89,7 @@ type
ambiguousSymbols*: IntSet # ids of all ambiguous symbols (cannot
# store this info in the syms themselves!)
inGenericContext*: int # > 0 if we are in a generic type
inStaticContext*: int # > 0 if we are inside a static: block
inUnrolledContext*: int # > 0 if we are unrolling a loop
compilesContextId*: int # > 0 if we are in a ``compiles`` magic
compilesContextIdGenerator*: int
@ -130,6 +131,7 @@ type
signatures*: TStrTable
recursiveDep*: string
suggestionsMade*: bool
features*: set[Feature]
inTypeContext*: int
typesWithOps*: seq[(PType, PType)] #\
# We need to instantiate the type bound ops lazily after
@ -138,13 +140,15 @@ type
# would otherwise fail.
runnableExamples*: PNode
template config*(c: PContext): ConfigRef = c.graph.config
proc makeInstPair*(s: PSym, inst: PInstantiation): TInstantiationPair =
result.genericSym = s
result.inst = inst
proc filename*(c: PContext): string =
# the module's filename
return c.module.filename
return toFilename(FileIndex c.module.position)
proc scopeDepth*(c: PContext): int {.inline.} =
result = if c.currentScope != nil: c.currentScope.depthLevel
@ -163,7 +167,7 @@ proc pushOwner*(c: PContext; owner: PSym) =
proc popOwner*(c: PContext) =
var length = len(c.graph.owners)
if length > 0: setLen(c.graph.owners, length - 1)
else: internalError("popOwner")
else: internalError(c.config, "popOwner")
proc lastOptionEntry*(c: PContext): POptionEntry =
result = c.optionStack[^1]
@ -199,19 +203,19 @@ proc considerGenSyms*(c: PContext; n: PNode) =
for i in 0..<n.safeLen:
considerGenSyms(c, n.sons[i])
proc newOptionEntry*(): POptionEntry =
proc newOptionEntry*(conf: ConfigRef): POptionEntry =
new(result)
result.options = gOptions
result.options = conf.options
result.defaultCC = ccDefault
result.dynlib = nil
result.notes = gNotes
result.notes = conf.notes
proc newContext*(graph: ModuleGraph; module: PSym; cache: IdentCache): PContext =
new(result)
result.ambiguousSymbols = initIntSet()
result.optionStack = @[]
result.libs = @[]
result.optionStack.add(newOptionEntry())
result.optionStack.add(newOptionEntry(graph.config))
result.module = module
result.friendModules = @[module]
result.converters = @[]
@ -225,7 +229,7 @@ proc newContext*(graph: ModuleGraph; module: PSym; cache: IdentCache): PContext
result.graph = graph
initStrTable(result.signatures)
result.typesWithOps = @[]
result.features = graph.config.features
proc inclSym(sq: var TSymSeq, s: PSym) =
var L = len(sq)
@ -254,36 +258,37 @@ proc newTypeS*(kind: TTypeKind, c: PContext): PType =
proc makePtrType*(c: PContext, baseType: PType): PType =
result = newTypeS(tyPtr, c)
addSonSkipIntLit(result, baseType.assertNotNil)
addSonSkipIntLit(result, baseType)
proc makeTypeWithModifier*(c: PContext,
modifier: TTypeKind,
baseType: PType): PType =
assert modifier in {tyVar, tyPtr, tyRef, tyStatic, tyTypeDesc}
assert modifier in {tyVar, tyLent, tyPtr, tyRef, tyStatic, tyTypeDesc}
if modifier in {tyVar, tyTypeDesc} and baseType.kind == modifier:
if modifier in {tyVar, tyLent, tyTypeDesc} and baseType.kind == modifier:
result = baseType
else:
result = newTypeS(modifier, c)
addSonSkipIntLit(result, baseType.assertNotNil)
addSonSkipIntLit(result, baseType)
proc makeVarType*(c: PContext, baseType: PType): PType =
if baseType.kind == tyVar:
proc makeVarType*(c: PContext, baseType: PType; kind = tyVar): PType =
if baseType.kind == kind:
result = baseType
else:
result = newTypeS(tyVar, c)
addSonSkipIntLit(result, baseType.assertNotNil)
result = newTypeS(kind, c)
addSonSkipIntLit(result, baseType)
proc makeTypeDesc*(c: PContext, typ: PType): PType =
if typ.kind == tyTypeDesc:
result = typ
else:
result = newTypeS(tyTypeDesc, c)
result.addSonSkipIntLit(typ.assertNotNil)
result.addSonSkipIntLit(typ)
proc makeTypeSymNode*(c: PContext, typ: PType, info: TLineInfo): PNode =
let typedesc = makeTypeDesc(c, typ)
let sym = newSym(skType, c.cache.idAnon, getCurrOwner(c), info).linkTo(typedesc)
let sym = newSym(skType, c.cache.idAnon, getCurrOwner(c), info,
c.config.options).linkTo(typedesc)
return newSymNode(sym, info)
proc makeTypeFromExpr*(c: PContext, n: PNode): PType =
@ -338,21 +343,20 @@ proc makeNotType*(c: PContext, t1: PType): PType =
result.flags.incl(t1.flags * {tfHasStatic})
result.flags.incl tfHasMeta
proc nMinusOne*(n: PNode): PNode =
proc nMinusOne(c: PContext; n: PNode): PNode =
result = newNode(nkCall, n.info, @[
newSymNode(getSysMagic("pred", mPred)),
n])
newSymNode(getSysMagic(c.graph, n.info, "pred", mPred)), n])
# Remember to fix the procs below this one when you make changes!
proc makeRangeWithStaticExpr*(c: PContext, n: PNode): PType =
let intType = getSysType(tyInt)
let intType = getSysType(c.graph, n.info, tyInt)
result = newTypeS(tyRange, c)
result.sons = @[intType]
if n.typ != nil and n.typ.n == nil:
result.flags.incl tfUnresolved
result.n = newNode(nkRange, n.info, @[
newIntTypeNode(nkIntLit, 0, intType),
makeStaticExpr(c, n.nMinusOne)])
makeStaticExpr(c, nMinusOne(c, n))])
template rangeHasUnresolvedStatic*(t: PType): bool =
tfUnresolved in t.flags
@ -371,7 +375,8 @@ proc fillTypeS*(dest: PType, kind: TTypeKind, c: PContext) =
dest.size = - 1
proc makeRangeType*(c: PContext; first, last: BiggestInt;
info: TLineInfo; intType = getSysType(tyInt)): PType =
info: TLineInfo; intType: PType = nil): PType =
let intType = if intType != nil: intType else: getSysType(c.graph, info, tyInt)
var n = newNodeI(nkRange, info)
addSon(n, newIntTypeNode(nkIntLit, first, intType))
addSon(n, newIntTypeNode(nkIntLit, last, intType))
@ -384,20 +389,17 @@ proc markIndirect*(c: PContext, s: PSym) {.inline.} =
incl(s.flags, sfAddrTaken)
# XXX add to 'c' for global analysis
proc illFormedAst*(n: PNode) =
globalError(n.info, errIllFormedAstX, renderTree(n, {renderNoComments}))
proc illFormedAst*(n: PNode; conf: ConfigRef) =
globalError(conf, n.info, errIllFormedAstX, renderTree(n, {renderNoComments}))
proc illFormedAstLocal*(n: PNode) =
localError(n.info, errIllFormedAstX, renderTree(n, {renderNoComments}))
proc illFormedAstLocal*(n: PNode; conf: ConfigRef) =
localError(conf, n.info, errIllFormedAstX, renderTree(n, {renderNoComments}))
proc checkSonsLen*(n: PNode, length: int) =
if sonsLen(n) != length: illFormedAst(n)
proc checkSonsLen*(n: PNode, length: int; conf: ConfigRef) =
if sonsLen(n) != length: illFormedAst(n, conf)
proc checkMinSonsLen*(n: PNode, length: int) =
if sonsLen(n) < length: illFormedAst(n)
proc checkMinSonsLen*(n: PNode, length: int; conf: ConfigRef) =
if sonsLen(n) < length: illFormedAst(n, conf)
proc isTopLevel*(c: PContext): bool {.inline.} =
result = c.currentScope.depthLevel <= 2
proc experimentalMode*(c: PContext): bool {.inline.} =
result = gExperimentalMode or sfExperimental in c.module.flags

702
compiler/semexprs.nim
View file

File diff suppressed because it is too large Load diff

27
compiler/semfields.nim
View file

@ -16,16 +16,17 @@ type
tupleIndex: int
field: PSym
replaceByFieldName: bool
c: PContext
proc instFieldLoopBody(c: TFieldInstCtx, n: PNode, forLoop: PNode): PNode =
case n.kind
of nkEmpty..pred(nkIdent), succ(nkSym)..nkNilLit: result = n
of nkIdent, nkSym:
result = n
let ident = considerQuotedIdent(n)
let ident = considerQuotedIdent(c.c.config, n)
var L = sonsLen(forLoop)
if c.replaceByFieldName:
if ident.id == considerQuotedIdent(forLoop[0]).id:
if ident.id == considerQuotedIdent(c.c.config, forLoop[0]).id:
let fieldName = if c.tupleType.isNil: c.field.name.s
elif c.tupleType.n.isNil: "Field" & $c.tupleIndex
else: c.tupleType.n.sons[c.tupleIndex].sym.name.s
@ -33,7 +34,7 @@ proc instFieldLoopBody(c: TFieldInstCtx, n: PNode, forLoop: PNode): PNode =
return
# other fields:
for i in ord(c.replaceByFieldName)..L-3:
if ident.id == considerQuotedIdent(forLoop[i]).id:
if ident.id == considerQuotedIdent(c.c.config, forLoop[i]).id:
var call = forLoop.sons[L-2]
var tupl = call.sons[i+1-ord(c.replaceByFieldName)]
if c.field.isNil:
@ -47,7 +48,7 @@ proc instFieldLoopBody(c: TFieldInstCtx, n: PNode, forLoop: PNode): PNode =
break
else:
if n.kind == nkContinueStmt:
localError(n.info, errGenerated,
localError(c.c.config, n.info,
"'continue' not supported in a 'fields' loop")
result = copyNode(n)
newSons(result, sonsLen(n))
@ -63,6 +64,7 @@ proc semForObjectFields(c: TFieldsCtx, typ, forLoop, father: PNode) =
case typ.kind
of nkSym:
var fc: TFieldInstCtx # either 'tup[i]' or 'field' is valid
fc.c = c.c
fc.field = typ.sym
fc.replaceByFieldName = c.m == mFieldPairs
openScope(c.c)
@ -76,7 +78,7 @@ proc semForObjectFields(c: TFieldsCtx, typ, forLoop, father: PNode) =
let L = forLoop.len
let call = forLoop.sons[L-2]
if call.len > 2:
localError(forLoop.info, errGenerated,
localError(c.c.config, forLoop.info,
"parallel 'fields' iterator does not work for 'case' objects")
return
# iterate over the selector:
@ -99,17 +101,17 @@ proc semForObjectFields(c: TFieldsCtx, typ, forLoop, father: PNode) =
of nkRecList:
for t in items(typ): semForObjectFields(c, t, forLoop, father)
else:
illFormedAstLocal(typ)
illFormedAstLocal(typ, c.c.config)
proc semForFields(c: PContext, n: PNode, m: TMagic): PNode =
# so that 'break' etc. work as expected, we produce
# a 'while true: stmt; break' loop ...
result = newNodeI(nkWhileStmt, n.info, 2)
var trueSymbol = strTableGet(magicsys.systemModule.tab, getIdent"true")
var trueSymbol = strTableGet(c.graph.systemModule.tab, getIdent"true")
if trueSymbol == nil:
localError(n.info, errSystemNeeds, "true")
localError(c.config, n.info, "system needs: 'true'")
trueSymbol = newSym(skUnknown, getIdent"true", getCurrOwner(c), n.info)
trueSymbol.typ = getSysType(tyBool)
trueSymbol.typ = getSysType(c.graph, n.info, tyBool)
result.sons[0] = newSymNode(trueSymbol, n.info)
var stmts = newNodeI(nkStmtList, n.info)
@ -118,18 +120,18 @@ proc semForFields(c: PContext, n: PNode, m: TMagic): PNode =
var length = sonsLen(n)
var call = n.sons[length-2]
if length-2 != sonsLen(call)-1 + ord(m==mFieldPairs):
localError(n.info, errWrongNumberOfVariables)
localError(c.config, n.info, errWrongNumberOfVariables)
return result
const skippedTypesForFields = abstractVar - {tyTypeDesc} + tyUserTypeClasses
var tupleTypeA = skipTypes(call.sons[1].typ, skippedTypesForFields)
if tupleTypeA.kind notin {tyTuple, tyObject}:
localError(n.info, errGenerated, "no object or tuple type")
localError(c.config, n.info, errGenerated, "no object or tuple type")
return result
for i in 1..call.len-1:
var tupleTypeB = skipTypes(call.sons[i].typ, skippedTypesForFields)
if not sameType(tupleTypeA, tupleTypeB):
typeMismatch(call.sons[i].info, tupleTypeA, tupleTypeB)
typeMismatch(c.config, call.sons[i].info, tupleTypeA, tupleTypeB)
inc(c.p.nestedLoopCounter)
if tupleTypeA.kind == tyTuple:
@ -139,6 +141,7 @@ proc semForFields(c: PContext, n: PNode, m: TMagic): PNode =
var fc: TFieldInstCtx
fc.tupleType = tupleTypeA
fc.tupleIndex = i
fc.c = c
fc.replaceByFieldName = m == mFieldPairs
var body = instFieldLoopBody(fc, loopBody, n)
inc c.inUnrolledContext

530
compiler/semfold.nim
View file

@ -13,24 +13,18 @@
import
strutils, options, ast, astalgo, trees, treetab, nimsets, times,
nversion, platform, math, msgs, os, condsyms, idents, renderer, types,
commands, magicsys, saturate
commands, magicsys, modulegraphs, strtabs
proc getConstExpr*(m: PSym, n: PNode): PNode
# evaluates the constant expression or returns nil if it is no constant
# expression
proc evalOp*(m: TMagic, n, a, b, c: PNode): PNode
proc leValueConv*(a, b: PNode): bool
proc newIntNodeT*(intVal: BiggestInt, n: PNode): PNode
proc newFloatNodeT(floatVal: BiggestFloat, n: PNode): PNode
proc newStrNodeT*(strVal: string, n: PNode): PNode
# implementation
proc newIntNodeT(intVal: BiggestInt, n: PNode): PNode =
proc newIntNodeT*(intVal: BiggestInt, n: PNode; g: ModuleGraph): PNode =
case skipTypes(n.typ, abstractVarRange).kind
of tyInt:
result = newIntNode(nkIntLit, intVal)
result.typ = getIntLitType(result)
# See bug #6989. 'pred' et al only produce an int literal type if the
# original type was 'int', not a distinct int etc.
if n.typ.kind == tyInt:
result.typ = getIntLitType(g, result)
else:
result.typ = n.typ
# hrm, this is not correct: 1 + high(int) shouldn't produce tyInt64 ...
#setIntLitType(result)
of tyChar:
@ -41,20 +35,82 @@ proc newIntNodeT(intVal: BiggestInt, n: PNode): PNode =
result.typ = n.typ
result.info = n.info
proc newFloatNodeT(floatVal: BiggestFloat, n: PNode): PNode =
proc newFloatNodeT*(floatVal: BiggestFloat, n: PNode; g: ModuleGraph): PNode =
result = newFloatNode(nkFloatLit, floatVal)
if skipTypes(n.typ, abstractVarRange).kind == tyFloat:
result.typ = getFloatLitType(result)
result.typ = getFloatLitType(g, result)
else:
result.typ = n.typ
result.info = n.info
proc newStrNodeT(strVal: string, n: PNode): PNode =
proc newStrNodeT*(strVal: string, n: PNode; g: ModuleGraph): PNode =
result = newStrNode(nkStrLit, strVal)
result.typ = n.typ
result.info = n.info
proc ordinalValToString*(a: PNode): string =
proc getConstExpr*(m: PSym, n: PNode; g: ModuleGraph): PNode
# evaluates the constant expression or returns nil if it is no constant
# expression
proc evalOp*(m: TMagic, n, a, b, c: PNode; g: ModuleGraph): PNode
proc checkInRange(n: PNode, res: BiggestInt): bool =
if res in firstOrd(n.typ)..lastOrd(n.typ):
result = true
proc foldAdd(a, b: BiggestInt, n: PNode; g: ModuleGraph): PNode =
let res = a +% b
if ((res xor a) >= 0'i64 or (res xor b) >= 0'i64) and
checkInRange(n, res):
result = newIntNodeT(res, n, g)
proc foldSub*(a, b: BiggestInt, n: PNode; g: ModuleGraph): PNode =
let res = a -% b
if ((res xor a) >= 0'i64 or (res xor not b) >= 0'i64) and
checkInRange(n, res):
result = newIntNodeT(res, n, g)
proc foldAbs*(a: BiggestInt, n: PNode; g: ModuleGraph): PNode =
if a != firstOrd(n.typ):
result = newIntNodeT(a, n, g)
proc foldMod*(a, b: BiggestInt, n: PNode; g: ModuleGraph): PNode =
if b != 0'i64:
result = newIntNodeT(a mod b, n, g)
proc foldModU*(a, b: BiggestInt, n: PNode; g: ModuleGraph): PNode =
if b != 0'i64:
result = newIntNodeT(a %% b, n, g)
proc foldDiv*(a, b: BiggestInt, n: PNode; g: ModuleGraph): PNode =
if b != 0'i64 and (a != firstOrd(n.typ) or b != -1'i64):
result = newIntNodeT(a div b, n, g)
proc foldDivU*(a, b: BiggestInt, n: PNode; g: ModuleGraph): PNode =
if b != 0'i64:
result = newIntNodeT(a /% b, n, g)
proc foldMul*(a, b: BiggestInt, n: PNode; g: ModuleGraph): PNode =
let res = a *% b
let floatProd = toBiggestFloat(a) * toBiggestFloat(b)
let resAsFloat = toBiggestFloat(res)
# Fast path for normal case: small multiplicands, and no info
# is lost in either method.
if resAsFloat == floatProd and checkInRange(n, res):
return newIntNodeT(res, n, g)
# Somebody somewhere lost info. Close enough, or way off? Note
# that a != 0 and b != 0 (else resAsFloat == floatProd == 0).
# The difference either is or isn't significant compared to the
# true value (of which floatProd is a good approximation).
# abs(diff)/abs(prod) <= 1/32 iff
# 32 * abs(diff) <= abs(prod) -- 5 good bits is "close enough"
if 32.0 * abs(resAsFloat - floatProd) <= abs(floatProd) and
checkInRange(n, res):
return newIntNodeT(res, n, g)
proc ordinalValToString*(a: PNode; g: ModuleGraph): string =
# because $ has the param ordinal[T], `a` is not necessarily an enum, but an
# ordinal
var x = getInt(a)
@ -66,14 +122,14 @@ proc ordinalValToString*(a: PNode): string =
of tyEnum:
var n = t.n
for i in countup(0, sonsLen(n) - 1):
if n.sons[i].kind != nkSym: internalError(a.info, "ordinalValToString")
if n.sons[i].kind != nkSym: internalError(g.config, a.info, "ordinalValToString")
var field = n.sons[i].sym
if field.position == x:
if field.ast == nil:
return field.name.s
else:
return field.ast.strVal
internalError(a.info, "no symbol for ordinal value: " & $x)
internalError(g.config, a.info, "no symbol for ordinal value: " & $x)
else:
result = $x
@ -92,12 +148,12 @@ proc pickIntRange(a, b: PType): PType =
proc isIntRangeOrLit(t: PType): bool =
result = isIntRange(t) or isIntLit(t)
proc makeRange(typ: PType, first, last: BiggestInt): PType =
proc makeRange(typ: PType, first, last: BiggestInt; g: ModuleGraph): PType =
let minA = min(first, last)
let maxA = max(first, last)
let lowerNode = newIntNode(nkIntLit, minA)
if typ.kind == tyInt and minA == maxA:
result = getIntLitType(lowerNode)
result = getIntLitType(g, lowerNode)
elif typ.kind in {tyUint, tyUInt64}:
# these are not ordinal types, so you get no subrange type for these:
result = typ
@ -109,7 +165,7 @@ proc makeRange(typ: PType, first, last: BiggestInt): PType =
result.n = n
addSonSkipIntLit(result, skipTypes(typ, {tyRange}))
proc makeRangeF(typ: PType, first, last: BiggestFloat): PType =
proc makeRangeF(typ: PType, first, last: BiggestFloat; g: ModuleGraph): PType =
var n = newNode(nkRange)
addSon(n, newFloatNode(nkFloatLit, min(first.float, last.float)))
addSon(n, newFloatNode(nkFloatLit, max(first.float, last.float)))
@ -119,9 +175,9 @@ proc makeRangeF(typ: PType, first, last: BiggestFloat): PType =
proc evalIs(n, a: PNode): PNode =
# XXX: This should use the standard isOpImpl
internalAssert a.kind == nkSym and a.sym.kind == skType
internalAssert n.sonsLen == 3 and
n[2].kind in {nkStrLit..nkTripleStrLit, nkType}
#internalAssert a.kind == nkSym and a.sym.kind == skType
#internalAssert n.sonsLen == 3 and
# n[2].kind in {nkStrLit..nkTripleStrLit, nkType}
let t1 = a.sym.typ
@ -145,125 +201,113 @@ proc evalIs(n, a: PNode): PNode =
result = newIntNode(nkIntLit, ord(match))
result.typ = n.typ
proc evalOp(m: TMagic, n, a, b, c: PNode): PNode =
proc evalOp(m: TMagic, n, a, b, c: PNode; g: ModuleGraph): PNode =
# b and c may be nil
result = nil
case m
of mOrd: result = newIntNodeT(getOrdValue(a), n)
of mChr: result = newIntNodeT(getInt(a), n)
of mUnaryMinusI, mUnaryMinusI64: result = newIntNodeT(- getInt(a), n)
of mUnaryMinusF64: result = newFloatNodeT(- getFloat(a), n)
of mNot: result = newIntNodeT(1 - getInt(a), n)
of mCard: result = newIntNodeT(nimsets.cardSet(a), n)
of mBitnotI: result = newIntNodeT(not getInt(a), n)
of mLengthArray: result = newIntNodeT(lengthOrd(a.typ), n)
of mOrd: result = newIntNodeT(getOrdValue(a), n, g)
of mChr: result = newIntNodeT(getInt(a), n, g)
of mUnaryMinusI, mUnaryMinusI64: result = newIntNodeT(- getInt(a), n, g)
of mUnaryMinusF64: result = newFloatNodeT(- getFloat(a), n, g)
of mNot: result = newIntNodeT(1 - getInt(a), n, g)
of mCard: result = newIntNodeT(nimsets.cardSet(a), n, g)
of mBitnotI: result = newIntNodeT(not getInt(a), n, g)
of mLengthArray: result = newIntNodeT(lengthOrd(a.typ), n, g)
of mLengthSeq, mLengthOpenArray, mXLenSeq, mLengthStr, mXLenStr:
if a.kind == nkNilLit:
result = newIntNodeT(0, n)
result = newIntNodeT(0, n, g)
elif a.kind in {nkStrLit..nkTripleStrLit}:
result = newIntNodeT(len a.strVal, n)
result = newIntNodeT(len a.strVal, n, g)
else:
result = newIntNodeT(sonsLen(a), n) # BUGFIX
result = newIntNodeT(sonsLen(a), n, g)
of mUnaryPlusI, mUnaryPlusF64: result = a # throw `+` away
of mToFloat, mToBiggestFloat:
result = newFloatNodeT(toFloat(int(getInt(a))), n)
of mToInt, mToBiggestInt: result = newIntNodeT(system.toInt(getFloat(a)), n)
of mAbsF64: result = newFloatNodeT(abs(getFloat(a)), n)
of mAbsI:
if getInt(a) >= 0: result = a
else: result = newIntNodeT(- getInt(a), n)
result = newFloatNodeT(toFloat(int(getInt(a))), n, g)
# XXX: Hides overflow/underflow
of mToInt, mToBiggestInt: result = newIntNodeT(system.toInt(getFloat(a)), n, g)
of mAbsF64: result = newFloatNodeT(abs(getFloat(a)), n, g)
of mAbsI: result = foldAbs(getInt(a), n, g)
of mZe8ToI, mZe8ToI64, mZe16ToI, mZe16ToI64, mZe32ToI64, mZeIToI64:
# byte(-128) = 1...1..1000_0000'64 --> 0...0..1000_0000'64
result = newIntNodeT(getInt(a) and (`shl`(1, getSize(a.typ) * 8) - 1), n)
of mToU8: result = newIntNodeT(getInt(a) and 0x000000FF, n)
of mToU16: result = newIntNodeT(getInt(a) and 0x0000FFFF, n)
of mToU32: result = newIntNodeT(getInt(a) and 0x00000000FFFFFFFF'i64, n)
of mUnaryLt: result = newIntNodeT(getOrdValue(a) |-| 1, n)
of mSucc: result = newIntNodeT(getOrdValue(a) |+| getInt(b), n)
of mPred: result = newIntNodeT(getOrdValue(a) |-| getInt(b), n)
of mAddI: result = newIntNodeT(getInt(a) |+| getInt(b), n)
of mSubI: result = newIntNodeT(getInt(a) |-| getInt(b), n)
of mMulI: result = newIntNodeT(getInt(a) |*| getInt(b), n)
result = newIntNodeT(getInt(a) and (`shl`(1, getSize(a.typ) * 8) - 1), n, g)
of mToU8: result = newIntNodeT(getInt(a) and 0x000000FF, n, g)
of mToU16: result = newIntNodeT(getInt(a) and 0x0000FFFF, n, g)
of mToU32: result = newIntNodeT(getInt(a) and 0x00000000FFFFFFFF'i64, n, g)
of mUnaryLt: result = foldSub(getOrdValue(a), 1, n, g)
of mSucc: result = foldAdd(getOrdValue(a), getInt(b), n, g)
of mPred: result = foldSub(getOrdValue(a), getInt(b), n, g)
of mAddI: result = foldAdd(getInt(a), getInt(b), n, g)
of mSubI: result = foldSub(getInt(a), getInt(b), n, g)
of mMulI: result = foldMul(getInt(a), getInt(b), n, g)
of mMinI:
if getInt(a) > getInt(b): result = newIntNodeT(getInt(b), n)
else: result = newIntNodeT(getInt(a), n)
if getInt(a) > getInt(b): result = newIntNodeT(getInt(b), n, g)
else: result = newIntNodeT(getInt(a), n, g)
of mMaxI:
if getInt(a) > getInt(b): result = newIntNodeT(getInt(a), n)
else: result = newIntNodeT(getInt(b), n)
if getInt(a) > getInt(b): result = newIntNodeT(getInt(a), n, g)
else: result = newIntNodeT(getInt(b), n, g)
of mShlI:
case skipTypes(n.typ, abstractRange).kind
of tyInt8: result = newIntNodeT(int8(getInt(a)) shl int8(getInt(b)), n)
of tyInt16: result = newIntNodeT(int16(getInt(a)) shl int16(getInt(b)), n)
of tyInt32: result = newIntNodeT(int32(getInt(a)) shl int32(getInt(b)), n)
of tyInt8: result = newIntNodeT(int8(getInt(a)) shl int8(getInt(b)), n, g)
of tyInt16: result = newIntNodeT(int16(getInt(a)) shl int16(getInt(b)), n, g)
of tyInt32: result = newIntNodeT(int32(getInt(a)) shl int32(getInt(b)), n, g)
of tyInt64, tyInt, tyUInt..tyUInt64:
result = newIntNodeT(`shl`(getInt(a), getInt(b)), n)
else: internalError(n.info, "constant folding for shl")
result = newIntNodeT(`shl`(getInt(a), getInt(b)), n, g)
else: internalError(g.config, n.info, "constant folding for shl")
of mShrI:
case skipTypes(n.typ, abstractRange).kind
of tyInt8: result = newIntNodeT(int8(getInt(a)) shr int8(getInt(b)), n)
of tyInt16: result = newIntNodeT(int16(getInt(a)) shr int16(getInt(b)), n)
of tyInt32: result = newIntNodeT(int32(getInt(a)) shr int32(getInt(b)), n)
of tyInt8: result = newIntNodeT(int8(getInt(a)) shr int8(getInt(b)), n, g)
of tyInt16: result = newIntNodeT(int16(getInt(a)) shr int16(getInt(b)), n, g)
of tyInt32: result = newIntNodeT(int32(getInt(a)) shr int32(getInt(b)), n, g)
of tyInt64, tyInt, tyUInt..tyUInt64:
result = newIntNodeT(`shr`(getInt(a), getInt(b)), n)
else: internalError(n.info, "constant folding for shr")
of mDivI:
let y = getInt(b)
if y != 0:
result = newIntNodeT(`|div|`(getInt(a), y), n)
of mModI:
let y = getInt(b)
if y != 0:
result = newIntNodeT(`|mod|`(getInt(a), y), n)
of mAddF64: result = newFloatNodeT(getFloat(a) + getFloat(b), n)
of mSubF64: result = newFloatNodeT(getFloat(a) - getFloat(b), n)
of mMulF64: result = newFloatNodeT(getFloat(a) * getFloat(b), n)
result = newIntNodeT(`shr`(getInt(a), getInt(b)), n, g)
else: internalError(g.config, n.info, "constant folding for shr")
of mDivI: result = foldDiv(getInt(a), getInt(b), n, g)
of mModI: result = foldMod(getInt(a), getInt(b), n, g)
of mAddF64: result = newFloatNodeT(getFloat(a) + getFloat(b), n, g)
of mSubF64: result = newFloatNodeT(getFloat(a) - getFloat(b), n, g)
of mMulF64: result = newFloatNodeT(getFloat(a) * getFloat(b), n, g)
of mDivF64:
if getFloat(b) == 0.0:
if getFloat(a) == 0.0: result = newFloatNodeT(NaN, n)
else: result = newFloatNodeT(Inf, n)
if getFloat(a) == 0.0: result = newFloatNodeT(NaN, n, g)
elif getFloat(b).classify == fcNegZero: result = newFloatNodeT(-Inf, n, g)
else: result = newFloatNodeT(Inf, n, g)
else:
result = newFloatNodeT(getFloat(a) / getFloat(b), n)
result = newFloatNodeT(getFloat(a) / getFloat(b), n, g)
of mMaxF64:
if getFloat(a) > getFloat(b): result = newFloatNodeT(getFloat(a), n)
else: result = newFloatNodeT(getFloat(b), n)
if getFloat(a) > getFloat(b): result = newFloatNodeT(getFloat(a), n, g)
else: result = newFloatNodeT(getFloat(b), n, g)
of mMinF64:
if getFloat(a) > getFloat(b): result = newFloatNodeT(getFloat(b), n)
else: result = newFloatNodeT(getFloat(a), n)
of mIsNil: result = newIntNodeT(ord(a.kind == nkNilLit), n)
if getFloat(a) > getFloat(b): result = newFloatNodeT(getFloat(b), n, g)
else: result = newFloatNodeT(getFloat(a), n, g)
of mIsNil: result = newIntNodeT(ord(a.kind == nkNilLit), n, g)
of mLtI, mLtB, mLtEnum, mLtCh:
result = newIntNodeT(ord(getOrdValue(a) < getOrdValue(b)), n)
result = newIntNodeT(ord(getOrdValue(a) < getOrdValue(b)), n, g)
of mLeI, mLeB, mLeEnum, mLeCh:
result = newIntNodeT(ord(getOrdValue(a) <= getOrdValue(b)), n)
result = newIntNodeT(ord(getOrdValue(a) <= getOrdValue(b)), n, g)
of mEqI, mEqB, mEqEnum, mEqCh:
result = newIntNodeT(ord(getOrdValue(a) == getOrdValue(b)), n)
of mLtF64: result = newIntNodeT(ord(getFloat(a) < getFloat(b)), n)
of mLeF64: result = newIntNodeT(ord(getFloat(a) <= getFloat(b)), n)
of mEqF64: result = newIntNodeT(ord(getFloat(a) == getFloat(b)), n)
of mLtStr: result = newIntNodeT(ord(getStr(a) < getStr(b)), n)
of mLeStr: result = newIntNodeT(ord(getStr(a) <= getStr(b)), n)
of mEqStr: result = newIntNodeT(ord(getStr(a) == getStr(b)), n)
result = newIntNodeT(ord(getOrdValue(a) == getOrdValue(b)), n, g)
of mLtF64: result = newIntNodeT(ord(getFloat(a) < getFloat(b)), n, g)
of mLeF64: result = newIntNodeT(ord(getFloat(a) <= getFloat(b)), n, g)
of mEqF64: result = newIntNodeT(ord(getFloat(a) == getFloat(b)), n, g)
of mLtStr: result = newIntNodeT(ord(getStr(a) < getStr(b)), n, g)
of mLeStr: result = newIntNodeT(ord(getStr(a) <= getStr(b)), n, g)
of mEqStr: result = newIntNodeT(ord(getStr(a) == getStr(b)), n, g)
of mLtU, mLtU64:
result = newIntNodeT(ord(`<%`(getOrdValue(a), getOrdValue(b))), n)
result = newIntNodeT(ord(`<%`(getOrdValue(a), getOrdValue(b))), n, g)
of mLeU, mLeU64:
result = newIntNodeT(ord(`<=%`(getOrdValue(a), getOrdValue(b))), n)
of mBitandI, mAnd: result = newIntNodeT(a.getInt and b.getInt, n)
of mBitorI, mOr: result = newIntNodeT(getInt(a) or getInt(b), n)
of mBitxorI, mXor: result = newIntNodeT(a.getInt xor b.getInt, n)
of mAddU: result = newIntNodeT(`+%`(getInt(a), getInt(b)), n)
of mSubU: result = newIntNodeT(`-%`(getInt(a), getInt(b)), n)
of mMulU: result = newIntNodeT(`*%`(getInt(a), getInt(b)), n)
of mModU:
let y = getInt(b)
if y != 0:
result = newIntNodeT(`%%`(getInt(a), y), n)
of mDivU:
let y = getInt(b)
if y != 0:
result = newIntNodeT(`/%`(getInt(a), y), n)
of mLeSet: result = newIntNodeT(ord(containsSets(a, b)), n)
of mEqSet: result = newIntNodeT(ord(equalSets(a, b)), n)
result = newIntNodeT(ord(`<=%`(getOrdValue(a), getOrdValue(b))), n, g)
of mBitandI, mAnd: result = newIntNodeT(a.getInt and b.getInt, n, g)
of mBitorI, mOr: result = newIntNodeT(getInt(a) or getInt(b), n, g)
of mBitxorI, mXor: result = newIntNodeT(a.getInt xor b.getInt, n, g)
of mAddU: result = newIntNodeT(`+%`(getInt(a), getInt(b)), n, g)
of mSubU: result = newIntNodeT(`-%`(getInt(a), getInt(b)), n, g)
of mMulU: result = newIntNodeT(`*%`(getInt(a), getInt(b)), n, g)
of mModU: result = foldModU(getInt(a), getInt(b), n, g)
of mDivU: result = foldDivU(getInt(a), getInt(b), n, g)
of mLeSet: result = newIntNodeT(ord(containsSets(a, b)), n, g)
of mEqSet: result = newIntNodeT(ord(equalSets(a, b)), n, g)
of mLtSet:
result = newIntNodeT(ord(containsSets(a, b) and not equalSets(a, b)), n)
result = newIntNodeT(ord(containsSets(a, b) and not equalSets(a, b)), n, g)
of mMulSet:
result = nimsets.intersectSets(a, b)
result.info = n.info
@ -276,120 +320,125 @@ proc evalOp(m: TMagic, n, a, b, c: PNode): PNode =
of mSymDiffSet:
result = nimsets.symdiffSets(a, b)
result.info = n.info
of mConStrStr: result = newStrNodeT(getStrOrChar(a) & getStrOrChar(b), n)
of mInSet: result = newIntNodeT(ord(inSet(a, b)), n)
of mConStrStr: result = newStrNodeT(getStrOrChar(a) & getStrOrChar(b), n, g)
of mInSet: result = newIntNodeT(ord(inSet(a, b)), n, g)
of mRepr:
# BUGFIX: we cannot eval mRepr here for reasons that I forgot.
discard
of mIntToStr, mInt64ToStr: result = newStrNodeT($(getOrdValue(a)), n)
of mIntToStr, mInt64ToStr: result = newStrNodeT($(getOrdValue(a)), n, g)
of mBoolToStr:
if getOrdValue(a) == 0: result = newStrNodeT("false", n)
else: result = newStrNodeT("true", n)
of mCopyStr: result = newStrNodeT(substr(getStr(a), int(getOrdValue(b))), n)
if getOrdValue(a) == 0: result = newStrNodeT("false", n, g)
else: result = newStrNodeT("true", n, g)
of mCopyStr: result = newStrNodeT(substr(getStr(a), int(getOrdValue(b))), n, g)
of mCopyStrLast:
result = newStrNodeT(substr(getStr(a), int(getOrdValue(b)),
int(getOrdValue(c))), n)
of mFloatToStr: result = newStrNodeT($getFloat(a), n)
int(getOrdValue(c))), n, g)
of mFloatToStr: result = newStrNodeT($getFloat(a), n, g)
of mCStrToStr, mCharToStr:
if a.kind == nkBracket:
var s = ""
for b in a.sons:
s.add b.getStrOrChar
result = newStrNodeT(s, n)
result = newStrNodeT(s, n, g)
else:
result = newStrNodeT(getStrOrChar(a), n)
result = newStrNodeT(getStrOrChar(a), n, g)
of mStrToStr: result = a
of mEnumToStr: result = newStrNodeT(ordinalValToString(a), n)
of mEnumToStr: result = newStrNodeT(ordinalValToString(a, g), n, g)
of mArrToSeq:
result = copyTree(a)
result.typ = n.typ
of mCompileOption:
result = newIntNodeT(ord(commands.testCompileOption(a.getStr, n.info)), n)
result = newIntNodeT(ord(commands.testCompileOption(g.config, a.getStr, n.info)), n, g)
of mCompileOptionArg:
result = newIntNodeT(ord(
testCompileOptionArg(getStr(a), getStr(b), n.info)), n)
testCompileOptionArg(g.config, getStr(a), getStr(b), n.info)), n, g)
of mEqProc:
result = newIntNodeT(ord(
exprStructuralEquivalent(a, b, strictSymEquality=true)), n)
exprStructuralEquivalent(a, b, strictSymEquality=true)), n, g)
else: discard
proc getConstIfExpr(c: PSym, n: PNode): PNode =
proc getConstIfExpr(c: PSym, n: PNode; g: ModuleGraph): PNode =
result = nil
for i in countup(0, sonsLen(n) - 1):
var it = n.sons[i]
if it.len == 2:
var e = getConstExpr(c, it.sons[0])
var e = getConstExpr(c, it.sons[0], g)
if e == nil: return nil
if getOrdValue(e) != 0:
if result == nil:
result = getConstExpr(c, it.sons[1])
result = getConstExpr(c, it.sons[1], g)
if result == nil: return
elif it.len == 1:
if result == nil: result = getConstExpr(c, it.sons[0])
else: internalError(it.info, "getConstIfExpr()")
if result == nil: result = getConstExpr(c, it.sons[0], g)
else: internalError(g.config, it.info, "getConstIfExpr()")
proc leValueConv(a, b: PNode): bool =
proc leValueConv*(a, b: PNode): bool =
result = false
case a.kind
of nkCharLit..nkUInt64Lit:
case b.kind
of nkCharLit..nkUInt64Lit: result = a.intVal <= b.intVal
of nkFloatLit..nkFloat128Lit: result = a.intVal <= round(b.floatVal).int
else: internalError(a.info, "leValueConv")
else: result = false #internalError(a.info, "leValueConv")
of nkFloatLit..nkFloat128Lit:
case b.kind
of nkFloatLit..nkFloat128Lit: result = a.floatVal <= b.floatVal
of nkCharLit..nkUInt64Lit: result = a.floatVal <= toFloat(int(b.intVal))
else: internalError(a.info, "leValueConv")
else: internalError(a.info, "leValueConv")
else: result = false # internalError(a.info, "leValueConv")
else: result = false # internalError(a.info, "leValueConv")
proc magicCall(m: PSym, n: PNode): PNode =
proc magicCall(m: PSym, n: PNode; g: ModuleGraph): PNode =
if sonsLen(n) <= 1: return
var s = n.sons[0].sym
var a = getConstExpr(m, n.sons[1])
var a = getConstExpr(m, n.sons[1], g)
var b, c: PNode
if a == nil: return
if sonsLen(n) > 2:
b = getConstExpr(m, n.sons[2])
b = getConstExpr(m, n.sons[2], g)
if b == nil: return
if sonsLen(n) > 3:
c = getConstExpr(m, n.sons[3])
c = getConstExpr(m, n.sons[3], g)
if c == nil: return
result = evalOp(s.magic, n, a, b, c)
result = evalOp(s.magic, n, a, b, c, g)
proc getAppType(n: PNode): PNode =
if gGlobalOptions.contains(optGenDynLib):
result = newStrNodeT("lib", n)
elif gGlobalOptions.contains(optGenStaticLib):
result = newStrNodeT("staticlib", n)
elif gGlobalOptions.contains(optGenGuiApp):
result = newStrNodeT("gui", n)
proc getAppType(n: PNode; g: ModuleGraph): PNode =
if g.config.globalOptions.contains(optGenDynLib):
result = newStrNodeT("lib", n, g)
elif g.config.globalOptions.contains(optGenStaticLib):
result = newStrNodeT("staticlib", n, g)
elif g.config.globalOptions.contains(optGenGuiApp):
result = newStrNodeT("gui", n, g)
else:
result = newStrNodeT("console", n)
result = newStrNodeT("console", n, g)
proc rangeCheck(n: PNode, value: BiggestInt) =
if value < firstOrd(n.typ) or value > lastOrd(n.typ):
localError(n.info, errGenerated, "cannot convert " & $value &
proc rangeCheck(n: PNode, value: BiggestInt; g: ModuleGraph) =
var err = false
if n.typ.skipTypes({tyRange}).kind in {tyUInt..tyUInt64}:
err = value <% firstOrd(n.typ) or value >% lastOrd(n.typ, fixedUnsigned=true)
else:
err = value < firstOrd(n.typ) or value > lastOrd(n.typ)
if err:
localError(g.config, n.info, "cannot convert " & $value &
" to " & typeToString(n.typ))
proc foldConv*(n, a: PNode; check = false): PNode =
proc foldConv*(n, a: PNode; g: ModuleGraph; check = false): PNode =
# XXX range checks?
case skipTypes(n.typ, abstractRange).kind
of tyInt..tyInt64, tyUInt..tyUInt64:
case skipTypes(a.typ, abstractRange).kind
of tyFloat..tyFloat64:
result = newIntNodeT(int(getFloat(a)), n)
of tyChar: result = newIntNodeT(getOrdValue(a), n)
result = newIntNodeT(int(getFloat(a)), n, g)
of tyChar: result = newIntNodeT(getOrdValue(a), n, g)
else:
result = a
result.typ = n.typ
if check and result.kind in {nkCharLit..nkUInt64Lit}:
rangeCheck(n, result.intVal)
rangeCheck(n, result.intVal, g)
of tyFloat..tyFloat64:
case skipTypes(a.typ, abstractRange).kind
of tyInt..tyInt64, tyEnum, tyBool, tyChar:
result = newFloatNodeT(toBiggestFloat(getOrdValue(a)), n)
result = newFloatNodeT(toBiggestFloat(getOrdValue(a)), n, g)
else:
result = a
result.typ = n.typ
@ -399,47 +448,47 @@ proc foldConv*(n, a: PNode; check = false): PNode =
result = a
result.typ = n.typ
proc getArrayConstr(m: PSym, n: PNode): PNode =
proc getArrayConstr(m: PSym, n: PNode; g: ModuleGraph): PNode =
if n.kind == nkBracket:
result = n
else:
result = getConstExpr(m, n)
result = getConstExpr(m, n, g)
if result == nil: result = n
proc foldArrayAccess(m: PSym, n: PNode): PNode =
var x = getConstExpr(m, n.sons[0])
if x == nil or x.typ.skipTypes({tyGenericInst, tyAlias}).kind == tyTypeDesc:
proc foldArrayAccess(m: PSym, n: PNode; g: ModuleGraph): PNode =
var x = getConstExpr(m, n.sons[0], g)
if x == nil or x.typ.skipTypes({tyGenericInst, tyAlias, tySink}).kind == tyTypeDesc:
return
var y = getConstExpr(m, n.sons[1])
var y = getConstExpr(m, n.sons[1], g)
if y == nil: return
var idx = getOrdValue(y)
case x.kind
of nkPar:
of nkPar, nkTupleConstr:
if idx >= 0 and idx < sonsLen(x):
result = x.sons[int(idx)]
if result.kind == nkExprColonExpr: result = result.sons[1]
else:
localError(n.info, errIndexOutOfBounds)
localError(g.config, n.info, "index out of bounds: " & $n)
of nkBracket:
idx = idx - x.typ.firstOrd
if idx >= 0 and idx < x.len: result = x.sons[int(idx)]
else: localError(n.info, errIndexOutOfBounds)
else: localError(g.config, n.info, "index out of bounds: " & $n)
of nkStrLit..nkTripleStrLit:
result = newNodeIT(nkCharLit, x.info, n.typ)
if idx >= 0 and idx < len(x.strVal):
result.intVal = ord(x.strVal[int(idx)])
elif idx == len(x.strVal):
elif idx == len(x.strVal) and optLaxStrings in g.config.options:
discard
else:
localError(n.info, errIndexOutOfBounds)
localError(g.config, n.info, "index out of bounds: " & $n)
else: discard
proc foldFieldAccess(m: PSym, n: PNode): PNode =
proc foldFieldAccess(m: PSym, n: PNode; g: ModuleGraph): PNode =
# a real field access; proc calls have already been transformed
var x = getConstExpr(m, n.sons[0])
if x == nil or x.kind notin {nkObjConstr, nkPar}: return
var x = getConstExpr(m, n.sons[0], g)
if x == nil or x.kind notin {nkObjConstr, nkPar, nkTupleConstr}: return
var field = n.sons[1].sym
for i in countup(ord(x.kind == nkObjConstr), sonsLen(x) - 1):
@ -452,13 +501,13 @@ proc foldFieldAccess(m: PSym, n: PNode): PNode =
if it.sons[0].sym.name.id == field.name.id:
result = x.sons[i].sons[1]
return
localError(n.info, errFieldXNotFound, field.name.s)
localError(g.config, n.info, "field not found: " & field.name.s)
proc foldConStrStr(m: PSym, n: PNode): PNode =
proc foldConStrStr(m: PSym, n: PNode; g: ModuleGraph): PNode =
result = newNodeIT(nkStrLit, n.info, n.typ)
result.strVal = ""
for i in countup(1, sonsLen(n) - 1):
let a = getConstExpr(m, n.sons[i])
let a = getConstExpr(m, n.sons[i], g)
if a == nil: return nil
result.strVal.add(getStrOrChar(a))
@ -470,37 +519,55 @@ proc newSymNodeTypeDesc*(s: PSym; info: TLineInfo): PNode =
else:
result.typ = s.typ
proc getConstExpr(m: PSym, n: PNode): PNode =
proc getConstExpr(m: PSym, n: PNode; g: ModuleGraph): PNode =
result = nil
proc getSrcTimestamp(): DateTime =
try:
result = utc(fromUnix(parseInt(getEnv("SOURCE_DATE_EPOCH",
"not a number"))))
except ValueError:
# Environment variable malformed.
# https://reproducible-builds.org/specs/source-date-epoch/: "If the
# value is malformed, the build process SHOULD exit with a non-zero
# error code", which this doesn't do. This uses local time, because
# that maintains compatibility with existing usage.
result = local(getTime())
case n.kind
of nkSym:
var s = n.sym
case s.kind
of skEnumField:
result = newIntNodeT(s.position, n)
result = newIntNodeT(s.position, n, g)
of skConst:
case s.magic
of mIsMainModule: result = newIntNodeT(ord(sfMainModule in m.flags), n)
of mCompileDate: result = newStrNodeT(times.getDateStr(), n)
of mCompileTime: result = newStrNodeT(times.getClockStr(), n)
of mCpuEndian: result = newIntNodeT(ord(CPU[targetCPU].endian), n)
of mHostOS: result = newStrNodeT(toLowerAscii(platform.OS[targetOS].name), n)
of mHostCPU: result = newStrNodeT(platform.CPU[targetCPU].name.toLowerAscii, n)
of mBuildOS: result = newStrNodeT(toLowerAscii(platform.OS[platform.hostOS].name), n)
of mBuildCPU: result = newStrNodeT(platform.CPU[platform.hostCPU].name.toLowerAscii, n)
of mAppType: result = getAppType(n)
of mNaN: result = newFloatNodeT(NaN, n)
of mInf: result = newFloatNodeT(Inf, n)
of mNegInf: result = newFloatNodeT(NegInf, n)
of mIsMainModule: result = newIntNodeT(ord(sfMainModule in m.flags), n, g)
of mCompileDate: result = newStrNodeT(format(getSrcTimestamp(),
"yyyy-MM-dd"), n, g)
of mCompileTime: result = newStrNodeT(format(getSrcTimestamp(),
"HH:mm:ss"), n, g)
of mCpuEndian: result = newIntNodeT(ord(CPU[targetCPU].endian), n, g)
of mHostOS: result = newStrNodeT(toLowerAscii(platform.OS[targetOS].name), n, g)
of mHostCPU: result = newStrNodeT(platform.CPU[targetCPU].name.toLowerAscii, n, g)
of mBuildOS: result = newStrNodeT(toLowerAscii(platform.OS[platform.hostOS].name), n, g)
of mBuildCPU: result = newStrNodeT(platform.CPU[platform.hostCPU].name.toLowerAscii, n, g)
of mAppType: result = getAppType(n, g)
of mNaN: result = newFloatNodeT(NaN, n, g)
of mInf: result = newFloatNodeT(Inf, n, g)
of mNegInf: result = newFloatNodeT(NegInf, n, g)
of mIntDefine:
if isDefined(s.name):
result = newIntNodeT(lookupSymbol(s.name).parseInt, n)
if isDefined(g.config, s.name.s):
try:
result = newIntNodeT(g.config.symbols[s.name.s].parseInt, n, g)
except ValueError:
localError(g.config, n.info, "expression is not an integer literal")
of mStrDefine:
if isDefined(s.name):
result = newStrNodeT(lookupSymbol(s.name), n)
if isDefined(g.config, s.name.s):
result = newStrNodeT(g.config.symbols[s.name.s], n, g)
else:
result = copyTree(s.ast)
of {skProc, skFunc, skMethod}:
of skProc, skFunc, skMethod:
result = n
of skType:
# XXX gensym'ed symbols can come here and cannot be resolved. This is
@ -520,7 +587,7 @@ proc getConstExpr(m: PSym, n: PNode): PNode =
of nkCharLit..nkNilLit:
result = copyNode(n)
of nkIfExpr:
result = getConstIfExpr(m, n)
result = getConstIfExpr(m, n, g)
of nkCallKinds:
if n.sons[0].kind != nkSym: return
var s = n.sons[0].sym
@ -533,68 +600,67 @@ proc getConstExpr(m: PSym, n: PNode): PNode =
of mSizeOf:
var a = n.sons[1]
if computeSize(a.typ) < 0:
localError(a.info, errCannotEvalXBecauseIncompletelyDefined,
"sizeof")
localError(g.config, a.info, "cannot evaluate 'sizeof' because its type is not defined completely")
result = nil
elif skipTypes(a.typ, typedescInst+{tyRange}).kind in
IntegralTypes+NilableTypes+{tySet}:
#{tyArray,tyObject,tyTuple}:
result = newIntNodeT(getSize(a.typ), n)
result = newIntNodeT(getSize(a.typ), n, g)
else:
result = nil
# XXX: size computation for complex types is still wrong
of mLow:
result = newIntNodeT(firstOrd(n.sons[1].typ), n)
result = newIntNodeT(firstOrd(n.sons[1].typ), n, g)
of mHigh:
if skipTypes(n.sons[1].typ, abstractVar).kind notin
{tySequence, tyString, tyCString, tyOpenArray, tyVarargs}:
result = newIntNodeT(lastOrd(skipTypes(n[1].typ, abstractVar)), n)
result = newIntNodeT(lastOrd(skipTypes(n[1].typ, abstractVar)), n, g)
else:
var a = getArrayConstr(m, n.sons[1])
var a = getArrayConstr(m, n.sons[1], g)
if a.kind == nkBracket:
# we can optimize it away:
result = newIntNodeT(sonsLen(a)-1, n)
result = newIntNodeT(sonsLen(a)-1, n, g)
of mLengthOpenArray:
var a = getArrayConstr(m, n.sons[1])
var a = getArrayConstr(m, n.sons[1], g)
if a.kind == nkBracket:
# we can optimize it away! This fixes the bug ``len(134)``.
result = newIntNodeT(sonsLen(a), n)
result = newIntNodeT(sonsLen(a), n, g)
else:
result = magicCall(m, n)
result = magicCall(m, n, g)
of mLengthArray:
# It doesn't matter if the argument is const or not for mLengthArray.
# This fixes bug #544.
result = newIntNodeT(lengthOrd(n.sons[1].typ), n)
result = newIntNodeT(lengthOrd(n.sons[1].typ), n, g)
of mAstToStr:
result = newStrNodeT(renderTree(n[1], {renderNoComments}), n)
result = newStrNodeT(renderTree(n[1], {renderNoComments}), n, g)
of mConStrStr:
result = foldConStrStr(m, n)
result = foldConStrStr(m, n, g)
of mIs:
let a = getConstExpr(m, n[1])
let a = getConstExpr(m, n[1], g)
if a != nil and a.kind == nkSym and a.sym.kind == skType:
result = evalIs(n, a)
else:
result = magicCall(m, n)
result = magicCall(m, n, g)
except OverflowError:
localError(n.info, errOverOrUnderflow)
localError(g.config, n.info, "over- or underflow")
except DivByZeroError:
localError(n.info, errConstantDivisionByZero)
localError(g.config, n.info, "division by zero")
of nkAddr:
var a = getConstExpr(m, n.sons[0])
var a = getConstExpr(m, n.sons[0], g)
if a != nil:
result = n
n.sons[0] = a
of nkBracket:
result = copyTree(n)
for i in countup(0, sonsLen(n) - 1):
var a = getConstExpr(m, n.sons[i])
var a = getConstExpr(m, n.sons[i], g)
if a == nil: return nil
result.sons[i] = a
incl(result.flags, nfAllConst)
of nkRange:
var a = getConstExpr(m, n.sons[0])
var a = getConstExpr(m, n.sons[0], g)
if a == nil: return
var b = getConstExpr(m, n.sons[1])
var b = getConstExpr(m, n.sons[1], g)
if b == nil: return
result = copyNode(n)
addSon(result, a)
@ -602,7 +668,7 @@ proc getConstExpr(m: PSym, n: PNode): PNode =
of nkCurly:
result = copyTree(n)
for i in countup(0, sonsLen(n) - 1):
var a = getConstExpr(m, n.sons[i])
var a = getConstExpr(m, n.sons[i], g)
if a == nil: return nil
result.sons[i] = a
incl(result.flags, nfAllConst)
@ -613,50 +679,50 @@ proc getConstExpr(m: PSym, n: PNode): PNode =
# if a == nil: return nil
# result.sons[i].sons[1] = a
# incl(result.flags, nfAllConst)
of nkPar:
of nkPar, nkTupleConstr:
# tuple constructor
result = copyTree(n)
if (sonsLen(n) > 0) and (n.sons[0].kind == nkExprColonExpr):
for i in countup(0, sonsLen(n) - 1):
var a = getConstExpr(m, n.sons[i].sons[1])
var a = getConstExpr(m, n.sons[i].sons[1], g)
if a == nil: return nil
result.sons[i].sons[1] = a
else:
for i in countup(0, sonsLen(n) - 1):
var a = getConstExpr(m, n.sons[i])
var a = getConstExpr(m, n.sons[i], g)
if a == nil: return nil
result.sons[i] = a
incl(result.flags, nfAllConst)
of nkChckRangeF, nkChckRange64, nkChckRange:
var a = getConstExpr(m, n.sons[0])
var a = getConstExpr(m, n.sons[0], g)
if a == nil: return
if leValueConv(n.sons[1], a) and leValueConv(a, n.sons[2]):
result = a # a <= x and x <= b
result.typ = n.typ
else:
localError(n.info, errGenerated, `%`(
msgKindToString(errIllegalConvFromXtoY),
[typeToString(n.sons[0].typ), typeToString(n.typ)]))
localError(g.config, n.info,
"conversion from $1 to $2 is invalid" %
[typeToString(n.sons[0].typ), typeToString(n.typ)])
of nkStringToCString, nkCStringToString:
var a = getConstExpr(m, n.sons[0])
var a = getConstExpr(m, n.sons[0], g)
if a == nil: return
result = a
result.typ = n.typ
of nkHiddenStdConv, nkHiddenSubConv, nkConv:
var a = getConstExpr(m, n.sons[1])
var a = getConstExpr(m, n.sons[1], g)
if a == nil: return
result = foldConv(n, a, check=n.kind == nkHiddenStdConv)
result = foldConv(n, a, g, check=n.kind == nkHiddenStdConv)
of nkCast:
var a = getConstExpr(m, n.sons[1])
var a = getConstExpr(m, n.sons[1], g)
if a == nil: return
if n.typ != nil and n.typ.kind in NilableTypes:
# we allow compile-time 'cast' for pointer types:
result = a
result.typ = n.typ
of nkBracketExpr: result = foldArrayAccess(m, n)
of nkDotExpr: result = foldFieldAccess(m, n)
of nkBracketExpr: result = foldArrayAccess(m, n, g)
of nkDotExpr: result = foldFieldAccess(m, n, g)
of nkStmtListExpr:
if n.len == 2 and n[0].kind == nkComesFrom:
result = getConstExpr(m, n[1])
result = getConstExpr(m, n[1], g)
else:
discard

82
compiler/semgnrc.nim
View file

@ -17,13 +17,13 @@
# included from sem.nim
proc getIdentNode(n: PNode): PNode =
proc getIdentNode(c: PContext; n: PNode): PNode =
case n.kind
of nkPostfix: result = getIdentNode(n.sons[1])
of nkPragmaExpr: result = getIdentNode(n.sons[0])
of nkPostfix: result = getIdentNode(c, n.sons[1])
of nkPragmaExpr: result = getIdentNode(c, n.sons[0])
of nkIdent, nkAccQuoted, nkSym: result = n
else:
illFormedAst(n)
illFormedAst(n, c.config)
result = n
type
@ -103,8 +103,8 @@ proc semGenericStmtSymbol(c: PContext, n: PNode, s: PSym,
proc lookup(c: PContext, n: PNode, flags: TSemGenericFlags,
ctx: var GenericCtx): PNode =
result = n
let ident = considerQuotedIdent(n)
var s = searchInScopes(c, ident).skipAlias(n)
let ident = considerQuotedIdent(c.config, n)
var s = searchInScopes(c, ident).skipAlias(n, c.config)
if s == nil:
s = strTableGet(c.pureEnumFields, ident)
if s != nil and contains(c.ambiguousSymbols, s.id):
@ -140,8 +140,8 @@ proc fuzzyLookup(c: PContext, n: PNode, flags: TSemGenericFlags,
n.sons[0] = semGenericStmt(c, n.sons[0], flags, ctx)
result = n
let n = n[1]
let ident = considerQuotedIdent(n)
var s = searchInScopes(c, ident).skipAlias(n)
let ident = considerQuotedIdent(c.config, n)
var s = searchInScopes(c, ident).skipAlias(n, c.config)
if s != nil and s.kind in routineKinds:
isMacro = s.kind in {skTemplate, skMacro}
if withinBind in flags:
@ -158,7 +158,7 @@ proc fuzzyLookup(c: PContext, n: PNode, flags: TSemGenericFlags,
result = newDot(result, syms)
proc addTempDecl(c: PContext; n: PNode; kind: TSymKind) =
let s = newSymS(skUnknown, getIdentNode(n), c)
let s = newSymS(skUnknown, getIdentNode(c, n), c)
addPrelimDecl(c, s)
styleCheckDef(n.info, s, kind)
@ -169,7 +169,7 @@ proc semGenericStmt(c: PContext, n: PNode,
when defined(nimsuggest):
if withinTypeDesc in flags: inc c.inTypeContext
#if gCmd == cmdIdeTools: suggestStmt(c, n)
#if conf.cmd == cmdIdeTools: suggestStmt(c, n)
semIdeForTemplateOrGenericCheck(n, ctx.cursorInBody)
case n.kind
@ -201,13 +201,13 @@ proc semGenericStmt(c: PContext, n: PNode,
result = semMixinStmt(c, n, ctx.toMixin)
of nkCall, nkHiddenCallConv, nkInfix, nkPrefix, nkCommand, nkCallStrLit:
# check if it is an expression macro:
checkMinSonsLen(n, 1)
checkMinSonsLen(n, 1, c.config)
let fn = n.sons[0]
var s = qualifiedLookUp(c, fn, {})
if s == nil and
{withinMixin, withinConcept}*flags == {} and
fn.kind in {nkIdent, nkAccQuoted} and
considerQuotedIdent(fn).id notin ctx.toMixin:
considerQuotedIdent(c.config, fn).id notin ctx.toMixin:
errorUndeclaredIdentifier(c, n.info, fn.renderTree)
var first = int ord(withinConcept in flags)
@ -285,7 +285,7 @@ proc semGenericStmt(c: PContext, n: PNode,
withBracketExpr ctx, n.sons[0]:
result = semGenericStmt(c, result, flags, ctx)
of nkAsgn, nkFastAsgn:
checkSonsLen(n, 2)
checkSonsLen(n, 2, c.config)
let a = n.sons[0]
let b = n.sons[1]
@ -323,7 +323,7 @@ proc semGenericStmt(c: PContext, n: PNode,
n.sons[0] = semGenericStmt(c, n.sons[0], flags, ctx)
for i in countup(1, sonsLen(n)-1):
var a = n.sons[i]
checkMinSonsLen(a, 1)
checkMinSonsLen(a, 1, c.config)
var L = sonsLen(a)
for j in countup(0, L-2):
a.sons[j] = semGenericStmt(c, a.sons[j], flags, ctx)
@ -340,23 +340,23 @@ proc semGenericStmt(c: PContext, n: PNode,
closeScope(c)
closeScope(c)
of nkBlockStmt, nkBlockExpr, nkBlockType:
checkSonsLen(n, 2)
checkSonsLen(n, 2, c.config)
openScope(c)
if n.sons[0].kind != nkEmpty:
addTempDecl(c, n.sons[0], skLabel)
n.sons[1] = semGenericStmt(c, n.sons[1], flags, ctx)
closeScope(c)
of nkTryStmt:
checkMinSonsLen(n, 2)
checkMinSonsLen(n, 2, c.config)
n.sons[0] = semGenericStmtScope(c, n.sons[0], flags, ctx)
for i in countup(1, sonsLen(n)-1):
var a = n.sons[i]
checkMinSonsLen(a, 1)
checkMinSonsLen(a, 1, c.config)
var L = sonsLen(a)
openScope(c)
for j in countup(0, L-2):
if a.sons[j].isInfixAs():
addTempDecl(c, getIdentNode(a.sons[j][2]), skLet)
addTempDecl(c, getIdentNode(c, a.sons[j][2]), skLet)
a.sons[j].sons[1] = semGenericStmt(c, a.sons[j][1], flags+{withinTypeDesc}, ctx)
else:
a.sons[j] = semGenericStmt(c, a.sons[j], flags+{withinTypeDesc}, ctx)
@ -367,44 +367,44 @@ proc semGenericStmt(c: PContext, n: PNode,
for i in countup(0, sonsLen(n) - 1):
var a = n.sons[i]
if a.kind == nkCommentStmt: continue
if (a.kind != nkIdentDefs) and (a.kind != nkVarTuple): illFormedAst(a)
checkMinSonsLen(a, 3)
if (a.kind != nkIdentDefs) and (a.kind != nkVarTuple): illFormedAst(a, c.config)
checkMinSonsLen(a, 3, c.config)
var L = sonsLen(a)
a.sons[L-2] = semGenericStmt(c, a.sons[L-2], flags+{withinTypeDesc}, ctx)
a.sons[L-1] = semGenericStmt(c, a.sons[L-1], flags, ctx)
for j in countup(0, L-3):
addTempDecl(c, getIdentNode(a.sons[j]), skVar)
addTempDecl(c, getIdentNode(c, a.sons[j]), skVar)
of nkGenericParams:
for i in countup(0, sonsLen(n) - 1):
var a = n.sons[i]
if (a.kind != nkIdentDefs): illFormedAst(a)
checkMinSonsLen(a, 3)
if (a.kind != nkIdentDefs): illFormedAst(a, c.config)
checkMinSonsLen(a, 3, c.config)
var L = sonsLen(a)
a.sons[L-2] = semGenericStmt(c, a.sons[L-2], flags+{withinTypeDesc}, ctx)
# do not perform symbol lookup for default expressions
for j in countup(0, L-3):
addTempDecl(c, getIdentNode(a.sons[j]), skType)
addTempDecl(c, getIdentNode(c, a.sons[j]), skType)
of nkConstSection:
for i in countup(0, sonsLen(n) - 1):
var a = n.sons[i]
if a.kind == nkCommentStmt: continue
if (a.kind != nkConstDef): illFormedAst(a)
checkSonsLen(a, 3)
addTempDecl(c, getIdentNode(a.sons[0]), skConst)
if (a.kind != nkConstDef): illFormedAst(a, c.config)
checkSonsLen(a, 3, c.config)
addTempDecl(c, getIdentNode(c, a.sons[0]), skConst)
a.sons[1] = semGenericStmt(c, a.sons[1], flags+{withinTypeDesc}, ctx)
a.sons[2] = semGenericStmt(c, a.sons[2], flags, ctx)
of nkTypeSection:
for i in countup(0, sonsLen(n) - 1):
var a = n.sons[i]
if a.kind == nkCommentStmt: continue
if (a.kind != nkTypeDef): illFormedAst(a)
checkSonsLen(a, 3)
addTempDecl(c, getIdentNode(a.sons[0]), skType)
if (a.kind != nkTypeDef): illFormedAst(a, c.config)
checkSonsLen(a, 3, c.config)
addTempDecl(c, getIdentNode(c, a.sons[0]), skType)
for i in countup(0, sonsLen(n) - 1):
var a = n.sons[i]
if a.kind == nkCommentStmt: continue
if (a.kind != nkTypeDef): illFormedAst(a)
checkSonsLen(a, 3)
if (a.kind != nkTypeDef): illFormedAst(a, c.config)
checkSonsLen(a, 3, c.config)
if a.sons[1].kind != nkEmpty:
openScope(c)
a.sons[1] = semGenericStmt(c, a.sons[1], flags, ctx)
@ -421,28 +421,28 @@ proc semGenericStmt(c: PContext, n: PNode,
case n.sons[i].kind
of nkEnumFieldDef: a = n.sons[i].sons[0]
of nkIdent: a = n.sons[i]
else: illFormedAst(n)
addDecl(c, newSymS(skUnknown, getIdentNode(a), c))
else: illFormedAst(n, c.config)
addDecl(c, newSymS(skUnknown, getIdentNode(c, a), c))
of nkObjectTy, nkTupleTy, nkTupleClassTy:
discard
of nkFormalParams:
checkMinSonsLen(n, 1)
checkMinSonsLen(n, 1, c.config)
if n.sons[0].kind != nkEmpty:
n.sons[0] = semGenericStmt(c, n.sons[0], flags+{withinTypeDesc}, ctx)
for i in countup(1, sonsLen(n) - 1):
var a = n.sons[i]
if (a.kind != nkIdentDefs): illFormedAst(a)
checkMinSonsLen(a, 3)
if (a.kind != nkIdentDefs): illFormedAst(a, c.config)
checkMinSonsLen(a, 3, c.config)
var L = sonsLen(a)
a.sons[L-2] = semGenericStmt(c, a.sons[L-2], flags+{withinTypeDesc}, ctx)
a.sons[L-1] = semGenericStmt(c, a.sons[L-1], flags, ctx)
for j in countup(0, L-3):
addTempDecl(c, getIdentNode(a.sons[j]), skParam)
addTempDecl(c, getIdentNode(c, a.sons[j]), skParam)
of nkProcDef, nkMethodDef, nkConverterDef, nkMacroDef, nkTemplateDef,
nkFuncDef, nkIteratorDef, nkLambdaKinds:
checkSonsLen(n, bodyPos + 1)
checkSonsLen(n, bodyPos + 1, c.config)
if n.sons[namePos].kind != nkEmpty:
addTempDecl(c, getIdentNode(n.sons[0]), skProc)
addTempDecl(c, getIdentNode(c, n.sons[0]), skProc)
openScope(c)
n.sons[genericParamsPos] = semGenericStmt(c, n.sons[genericParamsPos],
flags, ctx)
@ -463,7 +463,7 @@ proc semGenericStmt(c: PContext, n: PNode,
closeScope(c)
of nkPragma, nkPragmaExpr: discard
of nkExprColonExpr, nkExprEqExpr:
checkMinSonsLen(n, 2)
checkMinSonsLen(n, 2, c.config)
result.sons[1] = semGenericStmt(c, n.sons[1], flags, ctx)
else:
for i in countup(0, sonsLen(n) - 1):

57
compiler/seminst.nim
View file

@ -54,10 +54,13 @@ proc pushProcCon*(c: PContext; owner: PSym) =
rawPushProcCon(c, owner)
rawHandleSelf(c, owner)
const
errCannotInstantiateX = "cannot instantiate: '$1'"
iterator instantiateGenericParamList(c: PContext, n: PNode, pt: TIdTable): PSym =
internalAssert n.kind == nkGenericParams
internalAssert c.config, n.kind == nkGenericParams
for i, a in n.pairs:
internalAssert a.kind == nkSym
internalAssert c.config, a.kind == nkSym
var q = a.sym
if q.typ.kind notin {tyTypeDesc, tyGenericParam, tyStatic}+tyTypeClasses:
continue
@ -71,10 +74,10 @@ iterator instantiateGenericParamList(c: PContext, n: PNode, pt: TIdTable): PSym
# later by semAsgn in return type inference scenario
t = q.typ
else:
localError(a.info, errCannotInstantiateX, s.name.s)
localError(c.config, a.info, errCannotInstantiateX % s.name.s)
t = errorType(c)
elif t.kind == tyGenericParam:
localError(a.info, errCannotInstantiateX, q.name.s)
localError(c.config, a.info, errCannotInstantiateX % q.name.s)
t = errorType(c)
elif t.kind == tyGenericInvocation:
#t = instGenericContainer(c, a, t)
@ -145,7 +148,7 @@ proc instantiateBody(c: PContext, n, params: PNode, result, orig: PSym) =
freshGenSyms(b, result, orig, symMap)
b = semProcBody(c, b)
b = hloBody(c, b)
n.sons[bodyPos] = transformBody(c.module, b, result)
n.sons[bodyPos] = transformBody(c.graph, c.module, b, result)
#echo "code instantiated ", result.name.s
excl(result.flags, sfForward)
dec c.inGenericInst
@ -174,6 +177,8 @@ proc sideEffectsCheck(c: PContext, s: PSym) =
proc instGenericContainer(c: PContext, info: TLineInfo, header: PType,
allowMetaTypes = false): PType =
internalAssert c.config, header.kind == tyGenericInvocation
var
typeMap: LayeredIdTable
cl: TReplTypeVars
@ -185,7 +190,35 @@ proc instGenericContainer(c: PContext, info: TLineInfo, header: PType,
cl.info = info
cl.c = c
cl.allowMetaTypes = allowMetaTypes
# We must add all generic params in scope, because the generic body
# may include tyFromExpr nodes depending on these generic params.
# XXX: This looks quite similar to the code in matchUserTypeClass,
# perhaps the code can be extracted in a shared function.
openScope(c)
let genericTyp = header.base
for i in 0 .. (genericTyp.len - 2):
let genParam = genericTyp[i]
var param: PSym
template paramSym(kind): untyped =
newSym(kind, genParam.sym.name, genericTyp.sym, genParam.sym.info)
if genParam.kind == tyStatic:
param = paramSym skConst
param.ast = header[i+1].n
param.typ = header[i+1]
else:
param = paramSym skType
param.typ = makeTypeDesc(c, header[i+1])
# this scope was not created by the user,
# unused params shoudn't be reported.
param.flags.incl sfUsed
addDecl(c, param)
result = replaceTypeVarsT(cl, header)
closeScope(c)
proc instantiateProcType(c: PContext, pt: TIdTable,
prc: PSym, info: TLineInfo) =
@ -203,14 +236,12 @@ proc instantiateProcType(c: PContext, pt: TIdTable,
# at this point semtypinst have to become part of sem, because it
# will need to use openScope, addDecl, etc.
#addDecl(c, prc)
pushInfoContext(info)
var typeMap = initLayeredTypeMap(pt)
var cl = initTypeVars(c, addr(typeMap), info, nil)
var result = instCopyType(cl, prc.typ)
let originalParams = result.n
result.n = originalParams.shallowCopy
for i in 1 ..< result.len:
# twrong_field_caching requires these 'resetIdTable' calls:
if i > 1:
@ -218,7 +249,7 @@ proc instantiateProcType(c: PContext, pt: TIdTable,
resetIdTable(cl.localCache)
result.sons[i] = replaceTypeVarsT(cl, result.sons[i])
propagateToOwner(result, result.sons[i])
internalAssert originalParams[i].kind == nkSym
internalAssert c.config, originalParams[i].kind == nkSym
when true:
let oldParam = originalParams[i].sym
let param = copySym(oldParam)
@ -255,9 +286,9 @@ proc generateInstance(c: PContext, fn: PSym, pt: TIdTable,
## The `pt` parameter is a type-unsafe mapping table used to link generic
## parameters to their concrete types within the generic instance.
# no need to instantiate generic templates/macros:
internalAssert fn.kind notin {skMacro, skTemplate}
internalAssert c.config, fn.kind notin {skMacro, skTemplate}
# generates an instantiated proc
if c.instCounter > 1000: internalError(fn.ast.info, "nesting too deep")
if c.instCounter > 1000: internalError(c.config, fn.ast.info, "nesting too deep")
inc(c.instCounter)
# careful! we copy the whole AST including the possibly nil body!
var n = copyTree(fn.ast)
@ -276,7 +307,7 @@ proc generateInstance(c: PContext, fn: PSym, pt: TIdTable,
openScope(c)
let gp = n.sons[genericParamsPos]
internalAssert gp.kind != nkEmpty
internalAssert c.config, gp.kind != nkEmpty
n.sons[namePos] = newSymNode(result)
pushInfoContext(info)
var entry = TInstantiation.new
@ -316,7 +347,9 @@ proc generateInstance(c: PContext, fn: PSym, pt: TIdTable,
if c.inGenericContext == 0:
instantiateBody(c, n, fn.typ.n, result, fn)
sideEffectsCheck(c, result)
paramsTypeCheck(c, result.typ)
if result.magic != mSlice:
# 'toOpenArray' is special and it is allowed to return 'openArray':
paramsTypeCheck(c, result.typ)
else:
result = oldPrc
popProcCon(c)

38
compiler/semmacrosanity.nim
View file

@ -10,7 +10,7 @@
## Implements type sanity checking for ASTs resulting from macros. Lots of
## room for improvement here.
import ast, astalgo, msgs, types
import ast, astalgo, msgs, types, options
proc ithField(n: PNode, field: var int): PSym =
result = nil
@ -20,7 +20,7 @@ proc ithField(n: PNode, field: var int): PSym =
result = ithField(n.sons[i], field)
if result != nil: return
of nkRecCase:
if n.sons[0].kind != nkSym: internalError(n.info, "ithField")
if n.sons[0].kind != nkSym: return
result = ithField(n.sons[0], field)
if result != nil: return
for i in countup(1, sonsLen(n) - 1):
@ -28,13 +28,13 @@ proc ithField(n: PNode, field: var int): PSym =
of nkOfBranch, nkElse:
result = ithField(lastSon(n.sons[i]), field)
if result != nil: return
else: internalError(n.info, "ithField(record case branch)")
else: discard
of nkSym:
if field == 0: result = n.sym
else: dec(field)
else: discard
proc annotateType*(n: PNode, t: PType) =
proc annotateType*(n: PNode, t: PType; conf: ConfigRef) =
let x = t.skipTypes(abstractInst+{tyRange})
# Note: x can be unequal to t and we need to be careful to use 't'
# to not to skip tyGenericInst
@ -46,50 +46,50 @@ proc annotateType*(n: PNode, t: PType) =
var j = i-1
let field = x.n.ithField(j)
if field.isNil:
globalError n.info, "invalid field at index " & $i
globalError conf, n.info, "invalid field at index " & $i
else:
internalAssert(n.sons[i].kind == nkExprColonExpr)
annotateType(n.sons[i].sons[1], field.typ)
of nkPar:
internalAssert(conf, n.sons[i].kind == nkExprColonExpr)
annotateType(n.sons[i].sons[1], field.typ, conf)
of nkPar, nkTupleConstr:
if x.kind == tyTuple:
n.typ = t
for i in 0 ..< n.len:
if i >= x.len: globalError n.info, "invalid field at index " & $i
else: annotateType(n.sons[i], x.sons[i])
if i >= x.len: globalError conf, n.info, "invalid field at index " & $i
else: annotateType(n.sons[i], x.sons[i], conf)
elif x.kind == tyProc and x.callConv == ccClosure:
n.typ = t
else:
globalError(n.info, "() must have a tuple type")
globalError(conf, n.info, "() must have a tuple type")
of nkBracket:
if x.kind in {tyArray, tySequence, tyOpenArray}:
n.typ = t
for m in n: annotateType(m, x.elemType)
for m in n: annotateType(m, x.elemType, conf)
else:
globalError(n.info, "[] must have some form of array type")
globalError(conf, n.info, "[] must have some form of array type")
of nkCurly:
if x.kind in {tySet}:
n.typ = t
for m in n: annotateType(m, x.elemType)
for m in n: annotateType(m, x.elemType, conf)
else:
globalError(n.info, "{} must have the set type")
globalError(conf, n.info, "{} must have the set type")
of nkFloatLit..nkFloat128Lit:
if x.kind in {tyFloat..tyFloat128}:
n.typ = t
else:
globalError(n.info, "float literal must have some float type")
globalError(conf, n.info, "float literal must have some float type")
of nkCharLit..nkUInt64Lit:
if x.kind in {tyInt..tyUInt64, tyBool, tyChar, tyEnum}:
n.typ = t
else:
globalError(n.info, "integer literal must have some int type")
globalError(conf, n.info, "integer literal must have some int type")
of nkStrLit..nkTripleStrLit:
if x.kind in {tyString, tyCString}:
n.typ = t
else:
globalError(n.info, "string literal must be of some string type")
globalError(conf, n.info, "string literal must be of some string type")
of nkNilLit:
if x.kind in NilableTypes:
n.typ = t
else:
globalError(n.info, "nil literal must be of some pointer type")
globalError(conf, n.info, "nil literal must be of some pointer type")
else: discard

73
compiler/semmagic.nim
View file

@ -16,7 +16,7 @@ proc semAddr(c: PContext; n: PNode; isUnsafeAddr=false): PNode =
if x.kind == nkSym:
x.sym.flags.incl(sfAddrTaken)
if isAssignable(c, x, isUnsafeAddr) notin {arLValue, arLocalLValue}:
localError(n.info, errExprHasNoAddress)
localError(c.config, n.info, errExprHasNoAddress)
result.add x
result.typ = makePtrType(c, x.typ)
@ -43,7 +43,7 @@ proc semArrGet(c: PContext; n: PNode; flags: TExprFlags): PNode =
let x = copyTree(n)
x.sons[0] = newIdentNode(getIdent"[]", n.info)
bracketNotFoundError(c, x)
#localError(n.info, "could not resolve: " & $n)
#localError(c.config, n.info, "could not resolve: " & $n)
result = n
proc semArrPut(c: PContext; n: PNode; flags: TExprFlags): PNode =
@ -64,25 +64,28 @@ proc semAsgnOpr(c: PContext; n: PNode): PNode =
proc semIsPartOf(c: PContext, n: PNode, flags: TExprFlags): PNode =
var r = isPartOf(n[1], n[2])
result = newIntNodeT(ord(r), n)
result = newIntNodeT(ord(r), n, c.graph)
proc expectIntLit(c: PContext, n: PNode): int =
let x = c.semConstExpr(c, n)
case x.kind
of nkIntLit..nkInt64Lit: result = int(x.intVal)
else: localError(n.info, errIntLiteralExpected)
else: localError(c.config, n.info, errIntLiteralExpected)
proc semInstantiationInfo(c: PContext, n: PNode): PNode =
result = newNodeIT(nkPar, n.info, n.typ)
result = newNodeIT(nkTupleConstr, n.info, n.typ)
let idx = expectIntLit(c, n.sons[1])
let useFullPaths = expectIntLit(c, n.sons[2])
let info = getInfoContext(idx)
var filename = newNodeIT(nkStrLit, n.info, getSysType(tyString))
var filename = newNodeIT(nkStrLit, n.info, getSysType(c.graph, n.info, tyString))
filename.strVal = if useFullPaths != 0: info.toFullPath else: info.toFilename
var line = newNodeIT(nkIntLit, n.info, getSysType(tyInt))
var line = newNodeIT(nkIntLit, n.info, getSysType(c.graph, n.info, tyInt))
line.intVal = toLinenumber(info)
var column = newNodeIT(nkIntLit, n.info, getSysType(c.graph, n.info, tyInt))
column.intVal = toColumn(info)
result.add(filename)
result.add(line)
result.add(column)
proc toNode(t: PType, i: TLineInfo): PNode =
result = newNodeIT(nkType, i, t)
@ -107,10 +110,10 @@ proc uninstantiate(t: PType): PType =
of tyCompositeTypeClass: uninstantiate t.sons[1]
else: t
proc evalTypeTrait(traitCall: PNode, operand: PType, context: PSym): PNode =
const skippedTypes = {tyTypeDesc, tyAlias}
proc evalTypeTrait(c: PContext; traitCall: PNode, operand: PType, context: PSym): PNode =
const skippedTypes = {tyTypeDesc, tyAlias, tySink}
let trait = traitCall[0]
internalAssert trait.kind == nkSym
internalAssert c.config, trait.kind == nkSym
var operand = operand.skipTypes(skippedTypes)
template operand2: PType =
@ -137,7 +140,7 @@ proc evalTypeTrait(traitCall: PNode, operand: PType, context: PSym): PNode =
of "genericHead":
var res = uninstantiate(operand)
if res == operand and res.kind notin tyMagicGenerics:
localError(traitCall.info,
localError(c.config, traitCall.info,
"genericHead expects a generic type. The given type was " &
typeToString(operand))
return newType(tyError, context).toNode(traitCall.info)
@ -145,22 +148,22 @@ proc evalTypeTrait(traitCall: PNode, operand: PType, context: PSym): PNode =
of "stripGenericParams":
result = uninstantiate(operand).toNode(traitCall.info)
of "supportsCopyMem":
let t = operand.skipTypes({tyVar, tyGenericInst, tyAlias, tyInferred})
let t = operand.skipTypes({tyVar, tyLent, tyGenericInst, tyAlias, tySink, tyInferred})
let complexObj = containsGarbageCollectedRef(t) or
hasDestructor(t)
result = newIntNodeT(ord(not complexObj), traitCall)
result = newIntNodeT(ord(not complexObj), traitCall, c.graph)
else:
localError(traitCall.info, "unknown trait")
localError(c.config, traitCall.info, "unknown trait")
result = emptyNode
proc semTypeTraits(c: PContext, n: PNode): PNode =
checkMinSonsLen(n, 2)
checkMinSonsLen(n, 2, c.config)
let t = n.sons[1].typ
internalAssert t != nil and t.kind == tyTypeDesc
internalAssert c.config, t != nil and t.kind == tyTypeDesc
if t.sonsLen > 0:
# This is either a type known to sem or a typedesc
# param to a regular proc (again, known at instantiation)
result = evalTypeTrait(n, t, getCurrOwner(c))
result = evalTypeTrait(c, n, t, getCurrOwner(c))
else:
# a typedesc variable, pass unmodified to evals
result = n
@ -173,7 +176,7 @@ proc semOrd(c: PContext, n: PNode): PNode =
elif parType.kind == tySet:
result.typ = makeRangeType(c, firstOrd(parType), lastOrd(parType), n.info)
else:
localError(n.info, errOrdinalTypeExpected)
localError(c.config, n.info, errOrdinalTypeExpected)
result.typ = errorType(c)
proc semBindSym(c: PContext, n: PNode): PNode =
@ -182,13 +185,13 @@ proc semBindSym(c: PContext, n: PNode): PNode =
let sl = semConstExpr(c, n.sons[1])
if sl.kind notin {nkStrLit, nkRStrLit, nkTripleStrLit}:
localError(n.sons[1].info, errStringLiteralExpected)
localError(c.config, n.sons[1].info, errStringLiteralExpected)
return errorNode(c, n)
let isMixin = semConstExpr(c, n.sons[2])
if isMixin.kind != nkIntLit or isMixin.intVal < 0 or
isMixin.intVal > high(TSymChoiceRule).int:
localError(n.sons[2].info, errConstExprExpected)
localError(c.config, n.sons[2].info, errConstExprExpected)
return errorNode(c, n)
let id = newIdentNode(getIdent(sl.strVal), n.info)
@ -196,6 +199,10 @@ proc semBindSym(c: PContext, n: PNode): PNode =
if s != nil:
# we need to mark all symbols:
var sc = symChoice(c, id, s, TSymChoiceRule(isMixin.intVal))
if not (c.inStaticContext > 0 or getCurrOwner(c).isCompileTimeProc):
# inside regular code, bindSym resolves to the sym-choice
# nodes (see tinspectsymbol)
return sc
result.add(sc)
else:
errorUndeclaredIdentifier(c, n.sons[1].info, sl.strVal)
@ -218,9 +225,9 @@ proc semOf(c: PContext, n: PNode): PNode =
let y = skipTypes(n.sons[2].typ, abstractPtrs-{tyTypeDesc})
if x.kind == tyTypeDesc or y.kind != tyTypeDesc:
localError(n.info, errXExpectsObjectTypes, "of")
localError(c.config, n.info, "'of' takes object types")
elif b.kind != tyObject or a.kind != tyObject:
localError(n.info, errXExpectsObjectTypes, "of")
localError(c.config, n.info, "'of' takes object types")
else:
let diff = inheritanceDiff(a, b)
# | returns: 0 iff `a` == `b`
@ -229,26 +236,26 @@ proc semOf(c: PContext, n: PNode): PNode =
# | returns: `maxint` iff `a` and `b` are not compatible at all
if diff <= 0:
# optimize to true:
message(n.info, hintConditionAlwaysTrue, renderTree(n))
message(c.config, n.info, hintConditionAlwaysTrue, renderTree(n))
result = newIntNode(nkIntLit, 1)
result.info = n.info
result.typ = getSysType(tyBool)
result.typ = getSysType(c.graph, n.info, tyBool)
return result
elif diff == high(int):
localError(n.info, errXcanNeverBeOfThisSubtype, typeToString(a))
localError(c.config, n.info, "'$1' cannot be of this subtype" % typeToString(a))
else:
localError(n.info, errXExpectsTwoArguments, "of")
n.typ = getSysType(tyBool)
localError(c.config, n.info, "'of' takes 2 arguments")
n.typ = getSysType(c.graph, n.info, tyBool)
result = n
proc magicsAfterOverloadResolution(c: PContext, n: PNode,
flags: TExprFlags): PNode =
case n[0].sym.magic
of mAddr:
checkSonsLen(n, 2)
checkSonsLen(n, 2, c.config)
result = semAddr(c, n.sons[1], n[0].sym.name.s == "unsafeAddr")
of mTypeOf:
checkSonsLen(n, 2)
checkSonsLen(n, 2, c.config)
result = semTypeOf(c, n.sons[1])
of mArrGet: result = semArrGet(c, n, flags)
of mArrPut: result = semArrPut(c, n, flags)
@ -260,8 +267,8 @@ proc magicsAfterOverloadResolution(c: PContext, n: PNode,
of mIsPartOf: result = semIsPartOf(c, n, flags)
of mTypeTrait: result = semTypeTraits(c, n)
of mAstToStr:
result = newStrNodeT(renderTree(n[1], {renderNoComments}), n)
result.typ = getSysType(tyString)
result = newStrNodeT(renderTree(n[1], {renderNoComments}), n, c.graph)
result.typ = getSysType(c.graph, n.info, tyString)
of mInstantiationInfo: result = semInstantiationInfo(c, n)
of mOrd: result = semOrd(c, n)
of mOf: result = semOf(c, n)
@ -274,11 +281,11 @@ proc magicsAfterOverloadResolution(c: PContext, n: PNode,
of mDotDot:
result = n
of mRoof:
localError(n.info, "builtin roof operator is not supported anymore")
localError(c.config, n.info, "builtin roof operator is not supported anymore")
of mPlugin:
let plugin = getPlugin(n[0].sym)
if plugin.isNil:
localError(n.info, "cannot find plugin " & n[0].sym.name.s)
localError(c.config, n.info, "cannot find plugin " & n[0].sym.name.s)
result = n
else:
result = plugin(c, n)

92
compiler/semobjconstr.nim
View file

@ -39,32 +39,32 @@ proc mergeInitStatus(existing: var InitStatus, newStatus: InitStatus) =
of initUnknown:
discard
proc invalidObjConstr(n: PNode) =
proc invalidObjConstr(c: PContext, n: PNode) =
if n.kind == nkInfix and n[0].kind == nkIdent and n[0].ident.s[0] == ':':
localError(n.info, "incorrect object construction syntax; use a space after the colon")
localError(c.config, n.info, "incorrect object construction syntax; use a space after the colon")
else:
localError(n.info, "incorrect object construction syntax")
localError(c.config, n.info, "incorrect object construction syntax")
proc locateFieldInInitExpr(field: PSym, initExpr: PNode): PNode =
proc locateFieldInInitExpr(c: PContext, field: PSym, initExpr: PNode): PNode =
# Returns the assignment nkExprColonExpr node or nil
let fieldId = field.name.id
for i in 1 ..< initExpr.len:
let assignment = initExpr[i]
if assignment.kind != nkExprColonExpr:
invalidObjConstr(assignment)
invalidObjConstr(c, assignment)
continue
if fieldId == considerQuotedIdent(assignment[0]).id:
if fieldId == considerQuotedIdent(c.config, assignment[0]).id:
return assignment
proc semConstrField(c: PContext, flags: TExprFlags,
field: PSym, initExpr: PNode): PNode =
let assignment = locateFieldInInitExpr(field, initExpr)
let assignment = locateFieldInInitExpr(c, field, initExpr)
if assignment != nil:
if nfSem in assignment.flags: return assignment[1]
if not fieldVisible(c, field):
localError(initExpr.info,
"the field '$1' is not accessible.", [field.name.s])
localError(c.config, initExpr.info,
"the field '$1' is not accessible." % [field.name.s])
return
var initValue = semExprFlagDispatched(c, assignment[1], flags)
@ -76,8 +76,10 @@ proc semConstrField(c: PContext, flags: TExprFlags,
return initValue
proc caseBranchMatchesExpr(branch, matched: PNode): bool =
for i in 0 .. (branch.len - 2):
if exprStructuralEquivalent(branch[i], matched):
for i in 0 .. branch.len-2:
if branch[i].kind == nkRange:
if overlap(branch[i], matched): return true
elif exprStructuralEquivalent(branch[i], matched):
return true
return false
@ -99,25 +101,25 @@ iterator directFieldsInRecList(recList: PNode): PNode =
if recList.kind == nkSym:
yield recList
else:
internalAssert recList.kind == nkRecList
doAssert recList.kind == nkRecList
for field in recList:
if field.kind != nkSym: continue
yield field
template quoteStr(s: string): string = "'" & s & "'"
proc fieldsPresentInInitExpr(fieldsRecList, initExpr: PNode): string =
proc fieldsPresentInInitExpr(c: PContext, fieldsRecList, initExpr: PNode): string =
result = ""
for field in directFieldsInRecList(fieldsRecList):
let assignment = locateFieldInInitExpr(field.sym, initExpr)
let assignment = locateFieldInInitExpr(c, field.sym, initExpr)
if assignment != nil:
if result.len != 0: result.add ", "
result.add field.sym.name.s.quoteStr
proc missingMandatoryFields(fieldsRecList, initExpr: PNode): string =
proc missingMandatoryFields(c: PContext, fieldsRecList, initExpr: PNode): string =
for r in directFieldsInRecList(fieldsRecList):
if {tfNotNil, tfNeedsInit} * r.sym.typ.flags != {}:
let assignment = locateFieldInInitExpr(r.sym, initExpr)
let assignment = locateFieldInInitExpr(c, r.sym, initExpr)
if assignment == nil:
if result == nil:
result = r.sym.name.s
@ -125,10 +127,10 @@ proc missingMandatoryFields(fieldsRecList, initExpr: PNode): string =
result.add ", "
result.add r.sym.name.s
proc checkForMissingFields(recList, initExpr: PNode) =
let missing = missingMandatoryFields(recList, initExpr)
proc checkForMissingFields(c: PContext, recList, initExpr: PNode) =
let missing = missingMandatoryFields(c, recList, initExpr)
if missing != nil:
localError(initExpr.info, "fields not initialized: $1.", [missing])
localError(c.config, initExpr.info, "fields not initialized: $1.", [missing])
proc semConstructFields(c: PContext, recNode: PNode,
initExpr: PNode, flags: TExprFlags): InitStatus =
@ -144,14 +146,14 @@ proc semConstructFields(c: PContext, recNode: PNode,
template fieldsPresentInBranch(branchIdx: int): string =
let branch = recNode[branchIdx]
let fields = branch[branch.len - 1]
fieldsPresentInInitExpr(fields, initExpr)
fieldsPresentInInitExpr(c, fields, initExpr)
template checkMissingFields(branchNode: PNode) =
let fields = branchNode[branchNode.len - 1]
checkForMissingFields(fields, initExpr)
checkForMissingFields(c, fields, initExpr)
let discriminator = recNode.sons[0];
internalAssert discriminator.kind == nkSym
let discriminator = recNode.sons[0]
internalAssert c.config, discriminator.kind == nkSym
var selectedBranch = -1
for i in 1 ..< recNode.len:
@ -162,9 +164,9 @@ proc semConstructFields(c: PContext, recNode: PNode,
if selectedBranch != -1:
let prevFields = fieldsPresentInBranch(selectedBranch)
let currentFields = fieldsPresentInBranch(i)
localError(initExpr.info,
"The fields ($1) and ($2) cannot be initialized together, " &
"because they are from conflicting branches in the case object.",
localError(c.config, initExpr.info,
("The fields '$1' and '$2' cannot be initialized together, " &
"because they are from conflicting branches in the case object.") %
[prevFields, currentFields])
result = initConflict
else:
@ -177,9 +179,9 @@ proc semConstructFields(c: PContext, recNode: PNode,
discriminator.sym, initExpr)
if discriminatorVal == nil:
let fields = fieldsPresentInBranch(selectedBranch)
localError(initExpr.info,
"you must provide a compile-time value for the discriminator '$1' " &
"in order to prove that it's safe to initialize $2.",
localError(c.config, initExpr.info,
("you must provide a compile-time value for the discriminator '$1' " &
"in order to prove that it's safe to initialize $2.") %
[discriminator.sym.name.s, fields])
mergeInitStatus(result, initNone)
else:
@ -187,7 +189,7 @@ proc semConstructFields(c: PContext, recNode: PNode,
template wrongBranchError(i) =
let fields = fieldsPresentInBranch(i)
localError(initExpr.info,
localError(c.config, initExpr.info,
"a case selecting discriminator '$1' with value '$2' " &
"appears in the object construction, but the field(s) $3 " &
"are in conflict with this value.",
@ -217,7 +219,7 @@ proc semConstructFields(c: PContext, recNode: PNode,
# value was given to the discrimator. We can assume that it will be
# initialized to zero and this will select a particular branch as
# a result:
let matchedBranch = recNode.pickCaseBranch newIntLit(0)
let matchedBranch = recNode.pickCaseBranch newIntLit(c.graph, initExpr.info, 0)
checkMissingFields matchedBranch
else:
result = initPartial
@ -237,20 +239,17 @@ proc semConstructFields(c: PContext, recNode: PNode,
result = if e != nil: initFull else: initNone
else:
internalAssert false
internalAssert c.config, false
proc semConstructType(c: PContext, initExpr: PNode,
t: PType, flags: TExprFlags): InitStatus =
var t = t
result = initUnknown
while true:
let status = semConstructFields(c, t.n, initExpr, flags)
mergeInitStatus(result, status)
if status in {initPartial, initNone, initUnknown}:
checkForMissingFields t.n, initExpr
checkForMissingFields c, t.n, initExpr
let base = t.sons[0]
if base == nil: break
t = skipTypes(base, skipPtrs)
@ -260,10 +259,14 @@ proc semObjConstr(c: PContext, n: PNode, flags: TExprFlags): PNode =
result = newNodeIT(nkObjConstr, n.info, t)
for child in n: result.add child
t = skipTypes(t, {tyGenericInst, tyAlias})
if t.kind == tyRef: t = skipTypes(t.sons[0], {tyGenericInst, tyAlias})
if t == nil:
localError(c.config, n.info, errGenerated, "object constructor needs an object type")
return
t = skipTypes(t, {tyGenericInst, tyAlias, tySink})
if t.kind == tyRef: t = skipTypes(t.sons[0], {tyGenericInst, tyAlias, tySink})
if t.kind != tyObject:
localError(n.info, errGenerated, "object constructor needs an object type")
localError(c.config, n.info, errGenerated, "object constructor needs an object type")
return
# Check if the object is fully initialized by recursively testing each
@ -290,17 +293,16 @@ proc semObjConstr(c: PContext, n: PNode, flags: TExprFlags): PNode =
let field = result[i]
if nfSem notin field.flags:
if field.kind != nkExprColonExpr:
invalidObjConstr(field)
invalidObjConstr(c, field)
continue
let id = considerQuotedIdent(field[0])
let id = considerQuotedIdent(c.config, field[0])
# This node was not processed. There are two possible reasons:
# 1) It was shadowed by a field with the same name on the left
for j in 1 ..< i:
let prevId = considerQuotedIdent(result[j][0])
let prevId = considerQuotedIdent(c.config, result[j][0])
if prevId.id == id.id:
localError(field.info, errFieldInitTwice, id.s)
localError(c.config, field.info, errFieldInitTwice % id.s)
return
# 2) No such field exists in the constructed type
localError(field.info, errUndeclaredFieldX, id.s)
localError(c.config, field.info, errUndeclaredFieldX % id.s)
return

Some files were not shown because too many files have changed in this diff Show more