Nim/lib/system/excpt.nim
zah ca4b971bc8 Initial version of the hot-code reloading support for native targets (#10729)
* squashed work by Zahary

* squashing a ton of useful history... otherwise rebasing on top of upstream Nim after commit 82c009a2cb would be impossible.

* Code review changes; Working test suite (without code reloading enabled)

* - documentation
- implemented the HCR test - almost works...
- fix the issue on Unix where for executable targets the source file for the main module of a project in nimcache was being overwritten with the binary itself (and thus the actual source code was lost)
- fixing embedded paths to shared objects on unix (the "lib" prefix was being prepended to the entire path instead of just the filename)
- other fixes
- removing unnecessary includes since that file is already included in chcks.nim which is in turn included in system.nim (and previously was getting imported in chcks.nim but then system.nim improts something... and that breaks HCR (perhaps it could be fixed but it would be nice not to import anything in system))

* fix for clang & C++ - explicitly casting a function pointer to void*
more stable mangling of parameter names when HCR is on
the length of the static arrays in the DatInit functions is now part of the name of the variables, so when they get resized they get also recreated
more stable mangling for inline functions - no longer depends on the module which first used them
work on the new complicated HCR test - turned surprisingly complex - WIP
test now successfully passes even when re-running `koch test` (previously when the nimcache wasn't cold that lead to errors)
better documentation
calling setStackBottomWith for PreMain
passes over the HcrInit/DatInit/Init calls of all modules are now in the proper order (first all of one type, then all of the next). Also typeinfo globals are registered (created) in a single pass before the DatInit pass (because of the way generic instantiations are handled)
Fix the test suite execution on macOs
fix for being able to query the program arguments when using HCR on posix!
other fixes

* Bugfix: Fix a compilation error in C++ mode when a function pointer
is converted to a raw pointer

* basic documentation for the new hot code reloading semantics

* Add change log entry

* Don't re-execute the top-level statements while reloading JS code

* fix a number of tests broken in a recent bugfix

* Review changes

* Added {.executeOnReload.} pragma that indicates top-level statements
  that should be executed on each reload. To make this work, I've modified
  the way the `if (hcr_init_) {...}` guards are produced in the init code.
  This still needs more work as the new guards seem to be inserted within
  the previously generated guards.

  This change also removes the need for `lastRegistedGlobal` in nimhcr.

* Implemented the `signatureHash` magic and the `hasModuleChanged` API
  depending on it (the actual logic is not imlemented yet).

* Add the "hcr" prefix to all HCR-related symbols in the system module.
  Added a new `hotcodereloading` module exporting the high-level API to
  the user.

  Besides being more hygienic, this was also required in order to make
  it possible to use macros in the high-level API. Without the split,
  `system` would have to import `macros`, which was going to produce
  the well-known init problems.

* Attempted to solve the "GC markers problem".

  Crashes were expected with the previous code, because the GC markers
  were compiled as normal procs are registered in the GC. When their
  module is unloaded, dangling pointers will remain in the GC tables.
  To solve this issue, I don't register any GC markers when HCR is on,
  but I add them to the HCR globals metadata and I use a single marker
  registed in nimhcr during the initialization of the system module that
  will be responsible for marking all globals.

* fix a compilation error

* - implemented the hasModuleChanged functionality
- tuples can be returned and broken into different vars in global scope
- added comments for the closnig scopes of the if statements in the init proc
- the new executeOnReload pragma works now!
- other fixes

* finally! fixing this hack in a proper way - declaring the destructor out of line (out of the class body) - we no longer need to forward-declare popCurrentExceptionEx

* Force full module parsing

This is a temporary hack that breaks some tests. I'll investigate
later how these can be fixed.

* tuples are now properly handled when global!

* these comments mess up the codegen in debug mode when $n is not actually a new line (or something like that) - these labels are intended only for GOTO labels anyway...

* "solved" the issue with the .pdb locks on windows when a binary is being debugged and hot code reloading is used at the same time

* fixes after rebasing...

* small fixes for the test

* better handling of globals! no more compiler crashes for locals with the global pragma, also simplified code around loops in global scope which have local vars (actually globals)

* we can now use the global pragma even for ... globals!

* the right output

* lets try those boehm GC tests

* after the test is ran it will be at its starting state - no git modifications

* clarification in the docs

* removed unnecessary line directives for forward declarations of functions - they were causing trouble with hot code reloading when no semantic change propagates to the main module but a line directive got changed and thus the main module had to be recompiled since the .c code had changed

* fixed bug! was inserting duplicate keys into the table and later was removing only 1 copy of all the duplicates (after a few reloads)

* no longer breaking into DatInit code when not supposed to

* fixes after rebasing

* yet more fixes after rebasing

* Update jssys.nim

* Rework the HCR path-handling logic

After reviewing the code more carefully, I've noticed that the old logic
will be broken when the user overrides the '--out:f' compiler option.

Besides fixing this issues, I took the opportunity to implement the
missing '--outdir:d' option.

Other changes:

* ./koch test won't overwrite any HCR and RTL builds located in nim/lib
* HCR and RTL are compiled with --threads:on by default

* Clean up the globals registration logic

* Handle non-flattened top-level stmtlists in JS as well

* The HCR is not supported with the Boehm GC yet

Also fixes some typos and the expected output of the HCR integration test

* The GC marker procs are now properly used as trampolines

* Fix the HCR integration test in release builds

* Fix ./koch tools

* this forward declaration doesn't seem to be necessary, and in fact breaks HCR because a 2nd function pointer is emitted for this externed/rtl func

* the forward declaration I removed in the last commit was actually necessary

* Attempt to make all tests green

* Fix tgenscript

* BAT file for running the HCR integration test on Windows [skip ci]

* Fix the docgen tests

* A final fix for Travis (hopefully)
2019-02-26 15:48:55 +01:00

553 lines
17 KiB
Nim

#
#
# Nim's Runtime Library
# (c) Copyright 2015 Andreas Rumpf
#
# See the file "copying.txt", included in this
# distribution, for details about the copyright.
#
# Exception handling code. Carefully coded so that tiny programs which do not
# use the heap (and nor exceptions) do not include the GC or memory allocator.
var
errorMessageWriter*: (proc(msg: string) {.tags: [WriteIOEffect], benign,
nimcall.})
## Function that will be called
## instead of stdmsg.write when printing stacktrace.
## Unstable API.
proc c_fwrite(buf: pointer, size, n: csize, f: CFilePtr): cint {.
importc: "fwrite", header: "<stdio.h>".}
proc rawWrite(f: CFilePtr, s: cstring) {.compilerproc, nonreloadable, hcrInline.} =
# we cannot throw an exception here!
discard c_fwrite(s, 1, s.len, f)
when not defined(windows) or not defined(guiapp):
proc writeToStdErr(msg: cstring) = rawWrite(cstderr, msg)
else:
proc MessageBoxA(hWnd: cint, lpText, lpCaption: cstring, uType: int): int32 {.
header: "<windows.h>", nodecl.}
proc writeToStdErr(msg: cstring) =
discard MessageBoxA(0, msg, nil, 0)
proc showErrorMessage(data: cstring) {.gcsafe.} =
if errorMessageWriter != nil:
errorMessageWriter($data)
else:
when defined(genode):
# stderr not available by default, use the LOG session
echo data
else:
writeToStdErr(data)
proc quitOrDebug() {.inline.} =
when not defined(endb):
quit(1)
else:
endbStep() # call the debugger
proc chckIndx(i, a, b: int): int {.inline, compilerproc, benign.}
proc chckRange(i, a, b: int): int {.inline, compilerproc, benign.}
proc chckRangeF(x, a, b: float): float {.inline, compilerproc, benign.}
proc chckNil(p: pointer) {.noinline, compilerproc, benign.}
type
GcFrame = ptr GcFrameHeader
GcFrameHeader {.compilerproc.} = object
len: int
prev: ptr GcFrameHeader
var
framePtr {.threadvar.}: PFrame
excHandler {.threadvar.}: PSafePoint
# list of exception handlers
# a global variable for the root of all try blocks
currException {.threadvar.}: ref Exception
raiseCounter {.threadvar.}: uint
gcFramePtr {.threadvar.}: GcFrame
type
FrameState = tuple[gcFramePtr: GcFrame, framePtr: PFrame,
excHandler: PSafePoint, currException: ref Exception]
proc getFrameState*(): FrameState {.compilerRtl, inl.} =
return (gcFramePtr, framePtr, excHandler, currException)
proc setFrameState*(state: FrameState) {.compilerRtl, inl.} =
gcFramePtr = state.gcFramePtr
framePtr = state.framePtr
excHandler = state.excHandler
currException = state.currException
proc getFrame*(): PFrame {.compilerRtl, inl.} = framePtr
proc popFrame {.compilerRtl, inl.} =
framePtr = framePtr.prev
when false:
proc popFrameOfAddr(s: PFrame) {.compilerRtl.} =
var it = framePtr
if it == s:
framePtr = framePtr.prev
else:
while it != nil:
if it == s:
framePtr = it.prev
break
it = it.prev
proc setFrame*(s: PFrame) {.compilerRtl, inl.} =
framePtr = s
proc getGcFrame*(): GcFrame {.compilerRtl, inl.} = gcFramePtr
proc popGcFrame*() {.compilerRtl, inl.} = gcFramePtr = gcFramePtr.prev
proc setGcFrame*(s: GcFrame) {.compilerRtl, inl.} = gcFramePtr = s
proc pushGcFrame*(s: GcFrame) {.compilerRtl, inl.} =
s.prev = gcFramePtr
zeroMem(cast[pointer](cast[int](s)+%sizeof(GcFrameHeader)), s.len*sizeof(pointer))
gcFramePtr = s
proc pushSafePoint(s: PSafePoint) {.compilerRtl, inl.} =
s.hasRaiseAction = false
s.prev = excHandler
excHandler = s
proc popSafePoint {.compilerRtl, inl.} =
excHandler = excHandler.prev
proc pushCurrentException(e: ref Exception) {.compilerRtl, inl.} =
e.up = currException
currException = e
proc popCurrentException {.compilerRtl, inl.} =
currException = currException.up
proc popCurrentExceptionEx(id: uint) {.compilerRtl.} =
# in cpp backend exceptions can pop-up in the different order they were raised, example #5628
if currException.raiseId == id:
currException = currException.up
else:
var cur = currException.up
var prev = currException
while cur != nil and cur.raiseId != id:
prev = cur
cur = cur.up
if cur == nil:
showErrorMessage("popCurrentExceptionEx() exception was not found in the exception stack. Aborting...")
quitOrDebug()
prev.up = cur.up
proc closureIterSetupExc(e: ref Exception) {.compilerproc, inline.} =
if not e.isNil:
currException = e
# some platforms have native support for stack traces:
const
nativeStackTraceSupported* = (defined(macosx) or defined(linux)) and
not NimStackTrace
hasSomeStackTrace = NimStackTrace or
defined(nativeStackTrace) and nativeStackTraceSupported
when defined(nativeStacktrace) and nativeStackTraceSupported:
type
TDl_info {.importc: "Dl_info", header: "<dlfcn.h>",
final, pure.} = object
dli_fname: cstring
dli_fbase: pointer
dli_sname: cstring
dli_saddr: pointer
proc backtrace(symbols: ptr pointer, size: int): int {.
importc: "backtrace", header: "<execinfo.h>".}
proc dladdr(addr1: pointer, info: ptr TDl_info): int {.
importc: "dladdr", header: "<dlfcn.h>".}
when not hasThreadSupport:
var
tempAddresses: array[0..127, pointer] # should not be alloc'd on stack
tempDlInfo: TDl_info
proc auxWriteStackTraceWithBacktrace(s: var string) =
when hasThreadSupport:
var
tempAddresses: array[0..127, pointer] # but better than a threadvar
tempDlInfo: TDl_info
# This is allowed to be expensive since it only happens during crashes
# (but this way you don't need manual stack tracing)
var size = backtrace(cast[ptr pointer](addr(tempAddresses)),
len(tempAddresses))
var enabled = false
for i in 0..size-1:
var dlresult = dladdr(tempAddresses[i], addr(tempDlInfo))
if enabled:
if dlresult != 0:
var oldLen = s.len
add(s, tempDlInfo.dli_fname)
if tempDlInfo.dli_sname != nil:
for k in 1..max(1, 25-(s.len-oldLen)): add(s, ' ')
add(s, tempDlInfo.dli_sname)
else:
add(s, '?')
add(s, "\n")
else:
if dlresult != 0 and tempDlInfo.dli_sname != nil and
c_strcmp(tempDlInfo.dli_sname, "signalHandler") == 0'i32:
# Once we're past signalHandler, we're at what the user is
# interested in
enabled = true
when not hasThreadSupport:
var
tempFrames: array[0..127, PFrame] # should not be alloc'd on stack
const
reraisedFromBegin = -10
reraisedFromEnd = -100
template reraisedFrom(z): untyped =
StackTraceEntry(procname: nil, line: z, filename: nil)
proc auxWriteStackTrace(f: PFrame; s: var seq[StackTraceEntry]) =
var
it = f
i = 0
while it != nil:
inc(i)
it = it.prev
var last = i-1
when true: # not defined(gcDestructors):
if s.len == 0:
s = newSeq[StackTraceEntry](i)
else:
last = s.len + i - 1
s.setLen(last+1)
it = f
while it != nil:
s[last] = StackTraceEntry(procname: it.procname,
line: it.line,
filename: it.filename)
it = it.prev
dec last
template addFrameEntry(s, f: untyped) =
var oldLen = s.len
add(s, f.filename)
if f.line > 0:
add(s, '(')
add(s, $f.line)
add(s, ')')
for k in 1..max(1, 25-(s.len-oldLen)): add(s, ' ')
add(s, f.procname)
add(s, "\n")
proc `$`(s: seq[StackTraceEntry]): string =
result = newStringOfCap(2000)
for i in 0 .. s.len-1:
if s[i].line == reraisedFromBegin: result.add "[[reraised from:\n"
elif s[i].line == reraisedFromEnd: result.add "]]\n"
else: addFrameEntry(result, s[i])
proc auxWriteStackTrace(f: PFrame, s: var string) =
when hasThreadSupport:
var
tempFrames: array[0..127, PFrame] # but better than a threadvar
const
firstCalls = 32
var
it = f
i = 0
total = 0
# setup long head:
while it != nil and i <= high(tempFrames)-firstCalls:
tempFrames[i] = it
inc(i)
inc(total)
it = it.prev
# go up the stack to count 'total':
var b = it
while it != nil:
inc(total)
it = it.prev
var skipped = 0
if total > len(tempFrames):
# skip N
skipped = total-i-firstCalls+1
for j in 1..skipped:
if b != nil: b = b.prev
# create '...' entry:
tempFrames[i] = nil
inc(i)
# setup short tail:
while b != nil and i <= high(tempFrames):
tempFrames[i] = b
inc(i)
b = b.prev
for j in countdown(i-1, 0):
if tempFrames[j] == nil:
add(s, "(")
add(s, $skipped)
add(s, " calls omitted) ...\n")
else:
addFrameEntry(s, tempFrames[j])
proc stackTraceAvailable*(): bool
when hasSomeStackTrace:
proc rawWriteStackTrace(s: var string) =
when NimStackTrace:
if framePtr == nil:
add(s, "No stack traceback available\n")
else:
add(s, "Traceback (most recent call last)\n")
auxWriteStackTrace(framePtr, s)
elif defined(nativeStackTrace) and nativeStackTraceSupported:
add(s, "Traceback from system (most recent call last)\n")
auxWriteStackTraceWithBacktrace(s)
else:
add(s, "No stack traceback available\n")
proc rawWriteStackTrace(s: var seq[StackTraceEntry]) =
when NimStackTrace:
auxWriteStackTrace(framePtr, s)
else:
s = @[]
proc stackTraceAvailable(): bool =
when NimStackTrace:
if framePtr == nil:
result = false
else:
result = true
elif defined(nativeStackTrace) and nativeStackTraceSupported:
result = true
else:
result = false
else:
proc stackTraceAvailable*(): bool = result = false
var onUnhandledException*: (proc (errorMsg: string) {.
nimcall.}) ## set this error \
## handler to override the existing behaviour on an unhandled exception.
## The default is to write a stacktrace to ``stderr`` and then call ``quit(1)``.
## Unstable API.
template unhandled(buf, body) =
if onUnhandledException != nil:
onUnhandledException($buf)
else:
body
proc raiseExceptionAux(e: ref Exception) =
if localRaiseHook != nil:
if not localRaiseHook(e): return
if globalRaiseHook != nil:
if not globalRaiseHook(e): return
when defined(cpp) and not defined(noCppExceptions):
pushCurrentException(e)
raiseCounter.inc
if raiseCounter == 0:
raiseCounter.inc # skip zero at overflow
e.raiseId = raiseCounter
{.emit: "`e`->raise();".}
elif defined(nimQuirky):
pushCurrentException(e)
else:
if excHandler != nil:
if not excHandler.hasRaiseAction or excHandler.raiseAction(e):
pushCurrentException(e)
c_longjmp(excHandler.context, 1)
else:
when hasSomeStackTrace:
var buf = newStringOfCap(2000)
if e.trace.len == 0: rawWriteStackTrace(buf)
else: add(buf, $e.trace)
add(buf, "Error: unhandled exception: ")
add(buf, e.msg)
add(buf, " [")
add(buf, $e.name)
add(buf, "]\n")
unhandled(buf):
showErrorMessage(buf)
quitOrDebug()
else:
# ugly, but avoids heap allocations :-)
template xadd(buf, s, slen) =
if L + slen < high(buf):
copyMem(addr(buf[L]), cstring(s), slen)
inc L, slen
template add(buf, s) =
xadd(buf, s, s.len)
var buf: array[0..2000, char]
var L = 0
if e.trace.len != 0:
add(buf, $e.trace) # gc allocation
add(buf, "Error: unhandled exception: ")
add(buf, e.msg)
add(buf, " [")
xadd(buf, e.name, e.name.len)
add(buf, "]\n")
when defined(nimNoArrayToCstringConversion):
template tbuf(): untyped = addr buf
else:
template tbuf(): untyped = buf
unhandled(tbuf()):
showErrorMessage(tbuf())
quitOrDebug()
proc raiseExceptionEx(e: ref Exception, ename, procname, filename: cstring, line: int) {.compilerRtl.} =
if e.name.isNil: e.name = ename
when hasSomeStackTrace:
if e.trace.len == 0:
rawWriteStackTrace(e.trace)
elif framePtr != nil:
e.trace.add reraisedFrom(reraisedFromBegin)
auxWriteStackTrace(framePtr, e.trace)
e.trace.add reraisedFrom(reraisedFromEnd)
else:
if procname != nil and filename != nil:
e.trace.add StackTraceEntry(procname: procname, filename: filename, line: line)
raiseExceptionAux(e)
proc raiseException(e: ref Exception, ename: cstring) {.compilerRtl.} =
raiseExceptionEx(e, ename, nil, nil, 0)
proc reraiseException() {.compilerRtl.} =
if currException == nil:
sysFatal(ReraiseError, "no exception to reraise")
else:
raiseExceptionAux(currException)
proc writeStackTrace() =
when hasSomeStackTrace:
var s = ""
rawWriteStackTrace(s)
cast[proc (s: cstring) {.noSideEffect, tags: [], nimcall.}](showErrorMessage)(s)
else:
cast[proc (s: cstring) {.noSideEffect, tags: [], nimcall.}](showErrorMessage)("No stack traceback available\n")
proc getStackTrace(): string =
when hasSomeStackTrace:
result = ""
rawWriteStackTrace(result)
else:
result = "No stack traceback available\n"
proc getStackTrace(e: ref Exception): string =
if not isNil(e):
result = $e.trace
else:
result = ""
proc getStackTraceEntries*(e: ref Exception): seq[StackTraceEntry] =
## Returns the attached stack trace to the exception ``e`` as
## a ``seq``. This is not yet available for the JS backend.
when not defined(gcDestructors):
shallowCopy(result, e.trace)
else:
result = move(e.trace)
const nimCallDepthLimit {.intdefine.} = 2000
proc callDepthLimitReached() {.noinline.} =
writeStackTrace()
showErrorMessage("Error: call depth limit reached in a debug build (" &
$nimCallDepthLimit & " function calls). You can change it with " &
"-d:nimCallDepthLimit=<int> but really try to avoid deep " &
"recursions instead.\n")
quitOrDebug()
proc nimFrame(s: PFrame) {.compilerRtl, inl.} =
s.calldepth = if framePtr == nil: 0 else: framePtr.calldepth+1
s.prev = framePtr
framePtr = s
if s.calldepth == nimCallDepthLimit: callDepthLimitReached()
when defined(endb):
var
dbgAborting: bool # whether the debugger wants to abort
when defined(cpp) and appType != "lib" and
not defined(js) and not defined(nimscript) and
hostOS != "standalone" and not defined(noCppExceptions):
proc setTerminate(handler: proc() {.noconv.})
{.importc: "std::set_terminate", header: "<exception>".}
setTerminate proc() {.noconv.} =
# Remove ourself as a handler, reinstalling the default handler.
setTerminate(nil)
when defined(genode):
# stderr not available by default, use the LOG session
echo currException.getStackTrace() & "Error: unhandled exception: " &
currException.msg & " [" & $currException.name & "]\n"
else:
writeToStdErr currException.getStackTrace() & "Error: unhandled exception: " &
currException.msg & " [" & $currException.name & "]\n"
quit 1
when not defined(noSignalHandler) and not defined(useNimRtl):
proc signalHandler(sign: cint) {.exportc: "signalHandler", noconv.} =
template processSignal(s, action: untyped) {.dirty.} =
if s == SIGINT: action("SIGINT: Interrupted by Ctrl-C.\n")
elif s == SIGSEGV:
action("SIGSEGV: Illegal storage access. (Attempt to read from nil?)\n")
elif s == SIGABRT:
when defined(endb):
if dbgAborting: return # the debugger wants to abort
action("SIGABRT: Abnormal termination.\n")
elif s == SIGFPE: action("SIGFPE: Arithmetic error.\n")
elif s == SIGILL: action("SIGILL: Illegal operation.\n")
elif s == SIGBUS:
action("SIGBUS: Illegal storage access. (Attempt to read from nil?)\n")
else:
block platformSpecificSignal:
when declared(SIGPIPE):
if s == SIGPIPE:
action("SIGPIPE: Pipe closed.\n")
break platformSpecificSignal
action("unknown signal\n")
# print stack trace and quit
when defined(memtracker):
logPendingOps()
when hasSomeStackTrace:
GC_disable()
var buf = newStringOfCap(2000)
rawWriteStackTrace(buf)
processSignal(sign, buf.add) # nice hu? currying a la Nim :-)
showErrorMessage(buf)
GC_enable()
else:
var msg: cstring
template asgn(y) =
msg = y
processSignal(sign, asgn)
showErrorMessage(msg)
when defined(endb): dbgAborting = true
quit(1) # always quit when SIGABRT
proc registerSignalHandler() =
c_signal(SIGINT, signalHandler)
c_signal(SIGSEGV, signalHandler)
c_signal(SIGABRT, signalHandler)
c_signal(SIGFPE, signalHandler)
c_signal(SIGILL, signalHandler)
c_signal(SIGBUS, signalHandler)
when declared(SIGPIPE):
c_signal(SIGPIPE, signalHandler)
registerSignalHandler() # call it in initialization section
proc setControlCHook(hook: proc () {.noconv.}) =
# ugly cast, but should work on all architectures:
type SignalHandler = proc (sign: cint) {.noconv, benign.}
c_signal(SIGINT, cast[SignalHandler](hook))
when not defined(noSignalHandler) and not defined(useNimRtl):
proc unsetControlCHook() =
# proc to unset a hook set by setControlCHook
c_signal(SIGINT, signalHandler)