Commit graph

513 commits

Author SHA1 Message Date
Andreas Rumpf
75f8e136e1 ccgexprs.nim: less confusing code layout 2019-03-04 15:41:07 +01:00
zah
ca4b971bc8 Initial version of the hot-code reloading support for native targets (#10729)
* squashed work by Zahary

* squashing a ton of useful history... otherwise rebasing on top of upstream Nim after commit 82c009a2cb would be impossible.

* Code review changes; Working test suite (without code reloading enabled)

* - documentation
- implemented the HCR test - almost works...
- fix the issue on Unix where for executable targets the source file for the main module of a project in nimcache was being overwritten with the binary itself (and thus the actual source code was lost)
- fixing embedded paths to shared objects on unix (the "lib" prefix was being prepended to the entire path instead of just the filename)
- other fixes
- removing unnecessary includes since that file is already included in chcks.nim which is in turn included in system.nim (and previously was getting imported in chcks.nim but then system.nim improts something... and that breaks HCR (perhaps it could be fixed but it would be nice not to import anything in system))

* fix for clang & C++ - explicitly casting a function pointer to void*
more stable mangling of parameter names when HCR is on
the length of the static arrays in the DatInit functions is now part of the name of the variables, so when they get resized they get also recreated
more stable mangling for inline functions - no longer depends on the module which first used them
work on the new complicated HCR test - turned surprisingly complex - WIP
test now successfully passes even when re-running `koch test` (previously when the nimcache wasn't cold that lead to errors)
better documentation
calling setStackBottomWith for PreMain
passes over the HcrInit/DatInit/Init calls of all modules are now in the proper order (first all of one type, then all of the next). Also typeinfo globals are registered (created) in a single pass before the DatInit pass (because of the way generic instantiations are handled)
Fix the test suite execution on macOs
fix for being able to query the program arguments when using HCR on posix!
other fixes

* Bugfix: Fix a compilation error in C++ mode when a function pointer
is converted to a raw pointer

* basic documentation for the new hot code reloading semantics

* Add change log entry

* Don't re-execute the top-level statements while reloading JS code

* fix a number of tests broken in a recent bugfix

* Review changes

* Added {.executeOnReload.} pragma that indicates top-level statements
  that should be executed on each reload. To make this work, I've modified
  the way the `if (hcr_init_) {...}` guards are produced in the init code.
  This still needs more work as the new guards seem to be inserted within
  the previously generated guards.

  This change also removes the need for `lastRegistedGlobal` in nimhcr.

* Implemented the `signatureHash` magic and the `hasModuleChanged` API
  depending on it (the actual logic is not imlemented yet).

* Add the "hcr" prefix to all HCR-related symbols in the system module.
  Added a new `hotcodereloading` module exporting the high-level API to
  the user.

  Besides being more hygienic, this was also required in order to make
  it possible to use macros in the high-level API. Without the split,
  `system` would have to import `macros`, which was going to produce
  the well-known init problems.

* Attempted to solve the "GC markers problem".

  Crashes were expected with the previous code, because the GC markers
  were compiled as normal procs are registered in the GC. When their
  module is unloaded, dangling pointers will remain in the GC tables.
  To solve this issue, I don't register any GC markers when HCR is on,
  but I add them to the HCR globals metadata and I use a single marker
  registed in nimhcr during the initialization of the system module that
  will be responsible for marking all globals.

* fix a compilation error

* - implemented the hasModuleChanged functionality
- tuples can be returned and broken into different vars in global scope
- added comments for the closnig scopes of the if statements in the init proc
- the new executeOnReload pragma works now!
- other fixes

* finally! fixing this hack in a proper way - declaring the destructor out of line (out of the class body) - we no longer need to forward-declare popCurrentExceptionEx

* Force full module parsing

This is a temporary hack that breaks some tests. I'll investigate
later how these can be fixed.

* tuples are now properly handled when global!

* these comments mess up the codegen in debug mode when $n is not actually a new line (or something like that) - these labels are intended only for GOTO labels anyway...

* "solved" the issue with the .pdb locks on windows when a binary is being debugged and hot code reloading is used at the same time

* fixes after rebasing...

* small fixes for the test

* better handling of globals! no more compiler crashes for locals with the global pragma, also simplified code around loops in global scope which have local vars (actually globals)

* we can now use the global pragma even for ... globals!

* the right output

* lets try those boehm GC tests

* after the test is ran it will be at its starting state - no git modifications

* clarification in the docs

* removed unnecessary line directives for forward declarations of functions - they were causing trouble with hot code reloading when no semantic change propagates to the main module but a line directive got changed and thus the main module had to be recompiled since the .c code had changed

* fixed bug! was inserting duplicate keys into the table and later was removing only 1 copy of all the duplicates (after a few reloads)

* no longer breaking into DatInit code when not supposed to

* fixes after rebasing

* yet more fixes after rebasing

* Update jssys.nim

* Rework the HCR path-handling logic

After reviewing the code more carefully, I've noticed that the old logic
will be broken when the user overrides the '--out:f' compiler option.

Besides fixing this issues, I took the opportunity to implement the
missing '--outdir:d' option.

Other changes:

* ./koch test won't overwrite any HCR and RTL builds located in nim/lib
* HCR and RTL are compiled with --threads:on by default

* Clean up the globals registration logic

* Handle non-flattened top-level stmtlists in JS as well

* The HCR is not supported with the Boehm GC yet

Also fixes some typos and the expected output of the HCR integration test

* The GC marker procs are now properly used as trampolines

* Fix the HCR integration test in release builds

* Fix ./koch tools

* this forward declaration doesn't seem to be necessary, and in fact breaks HCR because a 2nd function pointer is emitted for this externed/rtl func

* the forward declaration I removed in the last commit was actually necessary

* Attempt to make all tests green

* Fix tgenscript

* BAT file for running the HCR integration test on Windows [skip ci]

* Fix the docgen tests

* A final fix for Travis (hopefully)
2019-02-26 15:48:55 +01:00
Neelesh Chandola
9884129058 Tuple unpacking now works for for vars (#10152)
* Tuple unpacking now works for `for` vars
* Give error if length of tuple vars != length of tuple
* Fix error message showing wrong tuple length
* unpacking now works now for mutable items
* Update changelog
2019-02-23 14:01:34 +01:00
Andreas Rumpf
ea409fb15a first steps in implementing 'owned' pointers; undocumented, do not use 2019-02-23 08:44:08 +01:00
Andreas Rumpf
417d27c544 gc:destructors: progress 2019-02-21 12:19:06 +01:00
Andreas Rumpf
43d570c178 gc:destructors: make system/repr compile 2019-02-21 07:57:29 +01:00
Arne Döring
28394153ab 32 bit fixes (#10608) 2019-02-13 23:30:14 +01:00
Michał Zieliński
580f62220c Fixes #10568: Fix null pointer dereference in address computation.
According to https://hownot2code.com/2016/08/18/null-pointer-dereferencing-causes-undefined-behavior/ this was an undefined behaviour.
2019-02-06 10:13:44 +01:00
Andreas Rumpf
dee8e6e98a
gc: destructors is beginning to work (#10483)
* kochdocs.nim: code cleanup
* docgen: nicer indentation
* parser.nim: code cleanup
* fixes #10458
* make tests green again
* make =destroy mixins
* gc:destructors: produced C code is almost working
* --gc:destructors simple program compiles (but leaks memory)
* gc:destructors make examples compile in C++ mode
* destructors: string implementation bugfixes
* strs.nim: minor code cleanup
* destructors: builtin seqs are beginning to work
* remove debugging helpers
2019-01-29 15:12:16 +01:00
Araq
d69a7842fa fixes #7878 2019-01-14 12:36:34 +01:00
Andreas Rumpf
bb1ce398af
Merge pull request #9881 from timotheecour/pr_ref_9880
refs #9880 show index and bound in lots of `index out of bounds` errors
2018-12-11 21:49:08 +01:00
Timothee Cour
f3ecc15a94 refs #9880 show index and bound in lots of index out of bounds errors 2018-12-09 16:50:45 -08:00
Jacek Sieka
a65141fc0b daily cruft removal 2018-12-08 13:36:38 -06:00
Araq
af815c3c18 fixes #9864 [backport] 2018-12-05 10:58:18 +01:00
Jacek Sieka
07744b6e47 More dead code (#9835)
* remove more assignment flags
* needsComplexAssignment > containsGarbageCollectedRef
* remove OnStackShadowDup
2018-12-04 11:22:59 +01:00
Andreas Rumpf
7d82df20be gc:destructors further progress 2018-11-29 01:13:32 +01:00
Andreas Rumpf
eb919c35b2 fixes #9675 2018-11-26 10:41:26 +01:00
Jacek Sieka
87568830ab Remove dead code (#9777)
* gc/gc2: remove unused ref counting stuff
* also removes some false threading support - hasSharedHeap is always
false in gc/gc2
* gc: remove some threading cruft
* remove asgnRefNoCycle
* compiler: remove TLoc.dup (unused)
2018-11-26 09:47:39 +01:00
Jacek Sieka
2b2fc5c7e9 compiler: remove unused assignment flags (#9760) 2018-11-21 11:04:45 +01:00
cooldome
9b9f5dee0b Fixes discard bug in cpp codegen (#9747) [backport]
Fixes discard bug in cpp codegen
2018-11-19 08:44:49 +01:00
Araq
4bd9f32f33 --gc:destructors: hello world example compiles and runs 2018-11-16 00:13:49 +01:00
Andreas Rumpf
cb4d810659 make Nim take roughly 100MB less RAM for bootstrapping via a new compiler switch -d:leanCompiler; useful for the Raberry PI 2018-11-07 18:36:52 +01:00
Araq
e71410d74e language cleanup: the .unchecked pragma is dead 2018-11-06 13:55:03 +01:00
Araq
b3d80dac9c make tmissingbracket compile for C++; refs #7870 2018-10-30 16:52:33 +01:00
LemonBoy
b613092de8 Don't crash if GC_{un,}ref is called on nil (#9445)
Fixes #9442
2018-10-22 17:09:15 +02:00
Ștefan Talpalaru
10f5f67767 gogc: GCC-8.2.0 compatibility and other improvements (#9211)
- Go's write barriers are now plugged-in in all the relevant points
- "gcGo" is correctly classified by usesWriteBarrier()
- some gogc structures and functions now use golib wrappers to keep GCC
  version-specific conditions out of the compiler/stdlib code
- we no longer allow mixing the C malloc with Go's
- fix a problem with string copying
2018-10-11 22:15:17 +02:00
Araq
d48e964950 fixes #9281 2018-10-11 19:52:48 +02:00
LemonBoy
0ead36dae6 Unchecked arrays now have their own type (#9267) 2018-10-10 21:03:18 +02:00
Araq
c92fdb24c8 fixes #5015 2018-10-03 00:00:19 +02:00
Araq
bf85955802 fixes #7092 2018-10-02 23:45:27 +02:00
Araq
7935c4aa5a C++ codegen: emit correctly typed code for closures in 'const'; refs #7870 2018-10-02 19:35:24 +02:00
LemonBoy
a1083d7c43 Fix codegen for some set operations
Taking the LHS type when a temporary result value was needed lead to bad
code being generated if we get a tyRef.

Fixes #9098
2018-09-28 09:59:45 +02:00
Andreas Rumpf
c38a608c90 C codegen: support system.move and system.wasMoved 2018-09-24 16:00:57 +02:00
Andreas Rumpf
179791aa9a
Merge pull request #8746 from LemonBoy/nil-opts
nil strings are movable
2018-09-23 07:51:22 +02:00
LemonBoy
7a087ffb29 nil strings are movable 2018-09-22 20:47:49 +02:00
LemonBoy
6af6ca6351 Fix codegen for set[T] parameters
Sometimes sets are materialized as arrays and we must treat them as
such: the CPP backend is pickier than the C one and would sometimes
produce invalid code.

Fixes #8967
2018-09-15 11:55:11 +02:00
Araq
64517445ea even more strict isNil handling for strings/seqs in order to detect bugs 2018-08-22 12:35:46 +02:00
Andreas Rumpf
ddaf2e3805 some progress on destructors for builtin seqs 2018-08-20 17:59:47 +02:00
Andreas Rumpf
696c0c48c7 fixes merge conflict 2018-08-19 15:14:03 +02:00
Andreas Rumpf
792829ad25 exploit the fact that empty seqs don't have to allocate in the code generator 2018-08-18 16:59:59 +02:00
Emery Hemingway
2da0341b4d Genode fixes (#8501)
* Genode fixes

- wrap strings in "Genode::Cstring" when logging
- define SIGABRT for Genode
- disable GCC -fstack-protector
- use log RPC for fatal messages
- add --os:genode build to appveyor
- define paramStr and paramCount

* Select fixups for Genode POSIX
2018-08-16 18:12:37 +02:00
Andreas Rumpf
dae5450947 WIP: disallow 'nil' for strings and seqs 2018-08-13 11:41:33 +02:00
Araq
640b30a47f codegen improvement: map the empty string to 'nil' since it's now allowed to do that 2018-08-08 12:06:00 +02:00
andri lim
4d5cce9882 fixes #6255, add system.ashr arithmetic right shift (#8547) 2018-08-07 18:41:30 +02:00
Araq
7ac6462cbd make at least bootstrapping work 2018-08-05 14:17:24 +02:00
Andreas Rumpf
c57e320c94
fixes 7833 (#8533)
* fixes #7833; still to-do: fix setLen
* make tests green again
* also fixes setLen and string concats; refs #7833
* change formating to avoid a compiler warning
* emit the write barrier also for addChar
* fixes yet another regression
* make setLengthStr compile for the old version
* make growobjcrash complete earlier
2018-08-05 11:42:38 +02:00
Araq
f131867de7 emit the write barrier also for addChar 2018-08-04 16:56:35 +02:00
Araq
39192fa568 also fixes setLen and string concats; refs #7833 2018-07-31 15:46:00 +02:00
Araq
53566f7165 fixes #7833; still to-do: fix setLen 2018-07-30 23:27:01 +02:00
Andreas Rumpf
f485ebe162 --gc:destructors: next steps; WIP 2018-07-21 13:16:53 +02:00