jyapayne/swig
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

*** empty log message ***

Browse source 
git-svn-id: https://swig.svn.sourceforge.net/svnroot/swig/trunk@1071 626c5289-ae23-0410-ae9c-e8d60b6d4f22
This commit is contained in:
Dave Beazley 2001-03-31 15:38:00 +00:00
commit 252a70db63
1 changed files with 104 additions and 73 deletions
Download patch file Download diff file Expand all files Collapse all files

175
SWIG/Tools/WAD/Papers/usenix2001.tex
View file

@ -51,7 +51,7 @@ minus3pt\let\@listi\@listI}
\date{} \date{}
%make title bold and 14 pt font (Latex default is non-bold, 16 pt) %make title bold and 14 pt font (Latex default is non-bold, 16 pt)
\title{\Large \bf An Embedded Error Recovery and Debugging Mechanism for Scripting Language Extensions} \title{\Large \bf An Embedded Error Reporting and Debugging Mechanism for Scripting Language Extensions}
%for single author (just remove % characters) %for single author (just remove % characters)
\author{{David M.\ Beazley} \\ \author{{David M.\ Beazley} \\
@ -89,17 +89,17 @@ sophisticated application software. One of the most useful features
of these languages is their ability to easily interact with compiled of these languages is their ability to easily interact with compiled
languages such as C and C++. Although this mixed language approach languages such as C and C++. Although this mixed language approach
has many benefits, one of the greatest drawbacks is the complexity of has many benefits, one of the greatest drawbacks is the complexity of
debugging that results from using interpreted and compiled code in debugging that results from using interpreted and compiled code in the
the same application. In part, this is due to the fact that scripting same application. In part, this is due to the fact that scripting
language interpreters are unable to recover from catastrophic errors in language interpreters are unable to recover from catastrophic errors
compiled extension code. Furthermore, traditional C/C++ debuggers do in compiled extension code. Furthermore, traditional C/C++ debuggers
not provide a satisfactory degree of integration with interpreted do not provide a satisfactory degree of integration with interpreted
languages. This paper describes an experimental system in which fatal languages. This paper describes an experimental system in which fatal
extension errors such as segmentation faults, bus errors, and failed extension errors such as segmentation faults, bus errors, and failed
assertions are handled as scripting language exceptions. This system, assertions are handled as scripting language exceptions. This system,
which has been implemented as a general purpose shared library, which has been implemented as a general purpose shared library,
requires no modifications to the target scripting language, introduces requires no modifications to the target scripting language, introduces
no performance overhead, and simplifies the debugging of mixed no performance penalty, and simplifies the debugging of mixed
interpreted-compiled application software. interpreted-compiled application software.
} }
@ -130,7 +130,7 @@ A critical aspect of scripting-compiled code integration is the way in
which it departs from traditional C/C++ development. Rather than which it departs from traditional C/C++ development. Rather than
building large monolithic stand-alone applications, scripting building large monolithic stand-alone applications, scripting
languages strongly encourage the creation of modular software languages strongly encourage the creation of modular software
components. As a result, scripted software tends to be constructed as components. Because of this, scripted software tends to be constructed as
a mix of dynamically loadable libraries, scripts, and third-party a mix of dynamically loadable libraries, scripts, and third-party
extension modules. In this sense, one might argue that the benefits of extension modules. In this sense, one might argue that the benefits of
scripting are achieved at the expense of creating a somewhat more scripting are achieved at the expense of creating a somewhat more
@ -141,7 +141,7 @@ associated with debugging programs that utilize multiple languages,
dynamically loadable modules, and a sophisticated runtime environment. dynamically loadable modules, and a sophisticated runtime environment.
To address this problem, this paper describes an experimental system To address this problem, this paper describes an experimental system
known as WAD (Wrapped Application Debugger) in which an embedded error known as WAD (Wrapped Application Debugger) in which an embedded error
recovery and debugging mechanism is added to common scripting reporting and debugging mechanism is added to common scripting
languages. This system converts catastrophic signals such as languages. This system converts catastrophic signals such as
segmentation faults and failed assertions to exceptions that can be segmentation faults and failed assertions to exceptions that can be
handled by the scripting language interpreter. In doing so, it handled by the scripting language interpreter. In doing so, it
@ -171,8 +171,9 @@ NameError: doh
In this case, a programmer might be able to apply a fix simply based In this case, a programmer might be able to apply a fix simply based
on information in the traceback. Alternatively, if the problem is on information in the traceback. Alternatively, if the problem is
more complicated, a script-level debugger can be used to provide more information. In contrast, more complicated, a script-level debugger can be used to provide more
a failure in compiled extension code might produce the following result: information. In contrast, a failure in compiled extension code might
produce the following result:
\begin{verbatim} \begin{verbatim}
% python foo.py % python foo.py
@ -180,21 +181,20 @@ Segmentation Fault (core dumped)
\end{verbatim} \end{verbatim}
In this case, the user has no idea of what has happened other In this case, the user has no idea of what has happened other
than it appears to be ``very bad.'' Furthermore, script-level than it appears to be ``very bad.'' To make matters worse, script-level
debuggers are unable to identify the problem since they also crash debuggers are unable to identify the problem since they also crash
when the error occurs (they usually run in the same process as when the error occurs (they usually run in the same process as
the interpreter). A user might be able to narrow the source of the the interpreter). A user might be able to narrow the source of the
problem through trial-and-error techniques such as inserting print problem through trial-and-error techniques such as inserting print
statements or commenting out sections of script code. Unfortunately, statements or commenting out sections of script code. However,
neither of these techniques are very attractive for obvious reasons. neither of these techniques are very attractive for obvious reasons.
Alternatively, a user could run the application under the control of a Alternatively, a user could run the application under the control of a
traditional debugger such as gdb \cite{gdb}. Unfortunately, this also has traditional debugger such as gdb \cite{gdb}. Although this certainly provides
drawbacks. First, even though the debugger provides information about the error, some information about the error, the debugger mostly provides information about the
the debugger mostly provides information about the internal internal implementation of the scripting language interpreter.
implementation of the scripting language interpreter. Needless Needless to say, this isn't very useful nor does it provide much insight as to
to say, this isn't very useful nor does it provide much insight as to where the error might have occurred within a script. A related problem is that
where the error might have occurred within a script. Second,
the structure of a scripted application tends to be much more complex the structure of a scripted application tends to be much more complex
than a traditional stand-alone program. As a result, a user may not than a traditional stand-alone program. As a result, a user may not
have a good sense of how to actually attach a C/C++ debugger to their have a good sense of how to actually attach a C/C++ debugger to their
@ -204,22 +204,23 @@ connections. Because of this, it can be difficult to reproduce
and identify certain types of catastrophic errors (especially if they and identify certain types of catastrophic errors (especially if they
depend on timing or peculiar sequences of events). Finally, this approach depend on timing or peculiar sequences of events). Finally, this approach
assumes that a programmer has a C/C++ development environment installed on assumes that a programmer has a C/C++ development environment installed on
their machine and that they know how to use a low-level their machine and that they know how to use a low-level C source
debugger. Unfortunately, neither of these assumptions may hold in practice. debugger. Unfortunately, neither of these assumptions may hold in practice.
This is because scripting languages are often used to provide programmability to This is because scripting languages are often used to provide programmability to
applications in which end-users might write scripts, yet would not be expected applications in which end-users might write scripts, yet would not be expected
to write low-level C code. to write low-level C code.
Even if a traditional debugger such as gdb were modified to Even if a traditional debugger such as gdb were modified to provide
provide better integration with scripting languages, it is not clear better integration with scripting languages, it is not clear that this
that this would be the most natural solution to the problem. would be the most natural solution to the problem. For one, the whole
For one, the whole notion of having to run a separate debugging process to debug notion of having to run a separate debugging process to debug
extension code is unnatural when no such requirement exists for extension code is unnatural when no such requirement exists for
a script. Furthermore, even if such a debugger existed, an inexperienced user may not scripts. Furthermore, even if such a debugger existed, an
have the expertise or inclination to use it. Finally, inexperienced user may not have the expertise or inclination to use
obscure fatal errors may occur long after an application has been deployed. it. Finally, obscure fatal errors may occur long after an application
Unless the debugger is distributed along with the application in some manner, it will be has been deployed. Unless the debugger is distributed along with the
extraordinary difficult to obtain useful diagnostics when such errors occur. application in some manner, it will be extraordinary difficult to
obtain useful diagnostics when such errors occur.
\begin{figure*}[t] \begin{figure*}[t]
{\small {\small
@ -253,34 +254,34 @@ SegFault: [ C stack trace ]
\caption{Cross language traceback generated for a segmentation fault in a Python extension} \caption{Cross language traceback generated for a segmentation fault in a Python extension}
\end{figure*} \end{figure*}
The easiest solution to the debugging problem is The current solution to the debugging problem is to take a proactive approach and simply add as
to simply add as much error checking as possible. Although this is never much error checking as possible to extension code. Although this is never
a bad thing to do, it's usually not enough to completely eliminate the problem. a bad thing to do, it's usually not enough to completely eliminate the problem.
For one, scripting languages are sometimes used to control hundreds For one, scripting languages are sometimes used to control hundreds
of thousands to millions of lines of compiled code. In this case, it is improbable of thousands to millions of lines of compiled code. In this case, it is simply improbable
that a programmer will be able to foresee every conceivable error. that a programmer will be able to foresee every conceivable error.
Second, scripting languages are often used to put new user interfaces on legacy software. In this In addition, scripting languages are often used to put new user interfaces on legacy software. In this
case, scripting may introduce new modes of execution that cause a formerly ``bug-free'' case, scripting may introduce new modes of execution that cause a formerly ``bug-free''
application to fail in an unexpected manner. Finally, certain types application to fail in an unexpected manner. Finally, certain types
of errors such as floating-point exceptions can be particularly of errors such as floating-point exceptions can be particularly
difficult to eliminate because they might be generated algorithmically (e.g., difficult to eliminate because they might be generated algorithmically (e.g.,
as the result of a numerical method). Therefore, even when a programmer has worked hard to eliminate as the result of instability in a numerical method). Therefore, even when a programmer has worked hard to eliminate
crashes, there is always a small probability that a complex application crashes, there is always a small probability that a complex application
will fail. will fail.
\section{Embedded Error Recovery} \section{Embedded Error Reporting}
Rather than modifying an existing debugger to support scripting Rather than modifying an existing debugger to support scripting
languages, an alternative approach is to add a more powerful error languages, an alternative approach is to add a more powerful error
handling and recovery mechanism to the scripting language interpreter. handling and reporting mechanism to the scripting language
This approach has been implemented in the form of an interpreter. This approach has been implemented in the form of an
experimental system known as WAD. WAD experimental system known as WAD. WAD is packaged as dynamically
is packaged as dynamically loadable shared library that can either be loadable shared library that can either be loaded as a scripting
loaded as a scripting language extension or linked to existing language extension or linked to existing extension modules as a
extension modules as a library. The core of the system is generic and library. The core of the system is generic and requires no
requires no modifications to the scripting interpreter or existing modifications to the scripting interpreter or existing extension
extension modules. Furthermore, the system does not introduce a performance penalty as it modules. Furthermore, the system does not introduce a performance
does not rely upon program instrumentation or tracing. penalty as it does not rely upon program instrumentation or tracing.
WAD works by converting fatal signals such as SIGSEGV, WAD works by converting fatal signals such as SIGSEGV,
SIGBUS, SIGFPE, and SIGABRT into scripting language exceptions that contain SIGBUS, SIGFPE, and SIGABRT into scripting language exceptions that contain
@ -292,15 +293,15 @@ shown for Python and Tcl/Tk in Figures 1 and 2. In this case, the user
is given a very clear idea of what has happened without having is given a very clear idea of what has happened without having
to launch a separate debugger. to launch a separate debugger.
The advantage to this approach is that it provides The advantage to this approach is that it provides more seamless
more seamless integration between error handling integration between error handling in scripts and error handling in
in scripts and error handling in extensions. In addition, it eliminates extensions. In addition, it eliminates the most common debugging step
the most common debugging step that a developer is likely to perform that a developer is likely to perform in the event of a fatal
in the event of a fatal error--running a separate debugger on a core error--running a separate debugger on a core file and typing 'where'
file and typing 'where' to get a stack trace. Finally, this allows to get a stack trace. Finally, this allows end-users to provide
end-users to provide extension writers with useful debugging extension writers with useful debugging information since they can
information since they can supply a stack trace as opposed to a vague supply a stack trace as opposed to a vague complaint that the program
complaint that the program ``crashed.'' ``crashed.''
\begin{figure*}[t] \begin{figure*}[t]
\begin{picture}(400,250)(0,0) \begin{picture}(400,250)(0,0)
@ -632,7 +633,7 @@ The severity of this problem depends greatly on the architecture and
compiler. For example, on the SPARC, register windows effectively compiler. For example, on the SPARC, register windows effectively
solve the callee-save problem \cite{sparc}. In this case, each stack frame has its own solve the callee-save problem \cite{sparc}. In this case, each stack frame has its own
register window and the windows are flushed to the stack whenever a register window and the windows are flushed to the stack whenever a
signal occurs. Therefore, the recovery mechanism can examine the stack and signal occurs. Therefore, the recovery mechanism can simply examine the stack and
arrange to restore the registers to their proper values when control arrange to restore the registers to their proper values when control
is returned. Furthermore, certain conventions of the SPARC ABI resolve several related is returned. Furthermore, certain conventions of the SPARC ABI resolve several related
issues. For example, floating point registers are caller-saved issues. For example, floating point registers are caller-saved
@ -640,23 +641,40 @@ and the contents of the SPARC global registers are not guaranteed to be preserve
across procedure calls (in fact, they are not even saved by {\tt setjmp}). across procedure calls (in fact, they are not even saved by {\tt setjmp}).
On other platforms, the problem of register management becomes much On other platforms, the problem of register management becomes much
more interesting. One approach is to simply ignore the problem more interesting. In this case, a heuristic approach that examines
altogether and return to the interpreter with the registers in an the machine code for each function on the call stack can be used to
essentially random state. Surprisingly, this approach actually seems to work (although a considerable degree of determine where the registers might have been saved. This approach is
caution might be in order). used by gdb and other debuggers when they allow users to inspect
This is because the return of an error code tends to trigger register values within arbitrary stack frames \cite{gdb}. Even though
a cascade of procedure returns within the implementation of the interpreter. this sounds complicated to implement, the algorithm is greatly
As a result, the values of the registers are simply discarded and simplified by the fact that compilers usually generate code to store
overwritten with restored values as the interpreter unwinds itself and prepares to handle an the callee-save registers immediately upon the entry to each function.
exception. A better solution to this problem is to modify the recovery mechanism to discover and In addition, this code is highly regular and easy to examine. For instance, on
restore saved registers from the stack. Unfortunately, there is i386-Linux, the callee-save registers can be fully restored by simply
no standardized way to know exactly where the registers might have been saved. examining the first 12 bytes of the machine code for each function on
Therefore, a heuristic scheme that examines the machine code for each procedure would the stack.
have to be used to try and identify stack locations. This approach is used by gdb
and other debuggers when they allow users to inspect register values %
within arbitrary stack frames \cite{gdb}. However, this technique has % Include an example
not yet been implemented in WAD due to its obvious implementation difficulty and the %
fact that the WAD prototype has primarily been developed for the SPARC.
% more interesting. One approach is to simply ignore the problem
% altogether and return to the interpreter with the registers in an
% essentially random state. Surprisingly, this approach actually seems to work (although a considerable degree of
% caution might be in order).
% This is because the return of an error code tends to trigger
% a cascade of procedure returns within the implementation of the interpreter.
% As a result, the values of the registers are simply discarded and
% overwritten with restored values as the interpreter unwinds itself and prepares to handle an
% exception. A better solution to this problem is to modify the recovery mechanism to discover and
% restore saved registers from the stack. Unfortunately, there is
% no standardized way to know exactly where the registers might have been saved.
% Therefore, a heuristic scheme that examines the machine code for each procedure would
% have to be used to try and identify stack locations. This approach is used by gdb
% and other debuggers when they allow users to inspect register values
% within arbitrary stack frames \cite{gdb}. However, this technique has
% not yet been implemented in WAD due to its obvious implementation difficulty and the
% fact that the WAD prototype has primarily been developed for the SPARC.
As a fall-back, WAD can be configured to return control to a location As a fall-back, WAD can be configured to return control to a location
previously specified with {\tt setjmp}. Unfortunately, this either previously specified with {\tt setjmp}. Unfortunately, this either
@ -665,6 +683,12 @@ Although this kind of instrumentation can be facilitated by automatic
wrapper code generators, it is not a preferred solution and is wrapper code generators, it is not a preferred solution and is
not discussed further. not discussed further.
\section{Making WAD Easy to Use}
\section{Design and Portability Concerns}
\section{Implementation Details} \section{Implementation Details}
Currently, WAD is implemented in ANSI C and small amount of assembly Currently, WAD is implemented in ANSI C and small amount of assembly
@ -721,6 +745,9 @@ platforms. To further complicate matters, the recovery process itself is
not thread-safe (i.e., it is not possible to concurrently handle fatal errors not thread-safe (i.e., it is not possible to concurrently handle fatal errors
occurring different threads). occurring different threads).
% Add comment about global interpreter locks, free-threading
%
Third, certain types of errors may result in an unrecoverable crash. Third, certain types of errors may result in an unrecoverable crash.
For example, if an application overwrites the heap, it may destroy For example, if an application overwrites the heap, it may destroy
critical data structures within the interpreter. critical data structures within the interpreter.
@ -728,12 +755,16 @@ Similarly,
destruction of the call stack (via buffer overflow) makes it destruction of the call stack (via buffer overflow) makes it
impossible for the recovery mechanism to create a stack-trace and impossible for the recovery mechanism to create a stack-trace and
return to the interpreter. Although it might be possible to add a heuristic scheme for return to the interpreter. Although it might be possible to add a heuristic scheme for
recovering a partial stack trace such as backward stack tracing, no such feature has been implemented recovering a partial stack trace such as backward stack tracing, no such feature has yet been implemented
\cite{debug}. Finally, memory management problems such as \cite{debug}. Finally, memory management problems such as
double-freeing of heap allocated memory can cause a system to fail in double-freeing of heap allocated memory can cause a system to fail in
a way that bears little resemblance to the actual source of the a way that bears little resemblance to the actual source of the
problem. problem.
%
% Add comments about what WAD does in this case?
%
Finally, there are a number of issues that pertain Finally, there are a number of issues that pertain
to the interaction of the recovery mechanism with the interpreter. to the interaction of the recovery mechanism with the interpreter.
First, the recovery scheme is unable to return to procedures First, the recovery scheme is unable to return to procedures