WAD (Wrapped Application Debugger)
Author(s):
David M. Beazley (beazley@cs.uchicago.edu)
Copyright (C) 2001
University of Chicago
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
See the file COPYING for a complete copy of the LGPL.
$Header$
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!! DISCLAIMER !!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
THIS IS EXPERIMENTAL UNMAINTAINED RESEARCH SOFTWARE THAT REPRESENTS
WORK IN PROGRESS. IT IS NOT PORTABLE, IT HAS NOT BEEN EXHAUSTIVELY
TESTED, AND IT MIGHT NOT WORK AT ALL. PLEASE KEEP AWAY FROM SMALL
CHILDREN, PETS, NUCLEAR REACTORS, AIR-TRAFFIC CONTROL, AND VOTING
MACHINES.
0. Supported Platforms
This software is currently only known to work with 32-bit applications
on Sun Sparc Solaris 2.8 and recent i386-Linux systems. In addition,
there are numerous issues concerning the interaction of this software
with signal handling, thread libraries, and compilers. Please read
this entire document before proceeding.
1. Introduction
WAD is an embedded error-recovery mechanism that attempts to convert
fatal errors such as SIGSEGV, SIGBUS, and SIGFPE into sensible error
messages and exceptions. It is primarily designed to support
scripting language extension programming although it can also be used
with stand-alone C programs.
The primary goal of this system is to explore an alternative approach
to mixed scripting-compiled debugging. It requires no modifications
or recompilation of existing software. Therefore, it should be
relatively easy to try out. Feedback is welcome. Contributions and
modifications are even more welcome.
2. Compilation and Installation
WAD is not particularly portable and at this time, only two platforms
are supported: Sun Sparc Solaris and i386-Linux.
Installation is as follows:
./configure
make
make install
The build process creates the following shared libraries:
libwad.so - Standalone WAD. Can be linked with C/C++
programs.
libwadpy.so - Python WAD. Can be linked to Python extension
modules or imported on its own as 'import libwadpy'
libwadtcl.so - Tcl WAD. Can be linked to Tcl extension
modules or loaded as 'load libwadtcl.so'.
libwadpl.so - Perl WAD. Can be linked to Perl extension
modules or loaded as 'libwadpl'.
To install the libraries, simply type 'make install'. This copies the libraries
to /usr/local/lib (unless you modify the makefile).
Notes:
- The Sun version of WAD has only been tested when compiled with the
Sun Workshop C/C++ compilers. However WAD works with other programs
that have been compiled with gcc. If gcc is installed on your
machine, you may want to set the following environment variables
before running configure:
setenv CC cc
setenv CXX CC
./configure
- You may need to modify the Makefile to point to the installed locations
of various scripting language libraries if you have installed
them in non-traditional locations.
- The Linux version has only been tested with 2.2-12 and 2.2-14 kernels
and the RedHat 6.x distribution. Your mileage may vary. There
may be some compatibility issues related to glibc and other parts
of the system as well.
3. Using WAD
WAD has no functional API nor does it have any command line options so
it's pretty easy to describe---simply link the appropriate WAD library with
your C code. For example:
% cc blah.c -lwad
Once linked, fatal errors will now produce stack traces. For example:
% ./a.out seg
starting.
Segmentation fault.
#2 0x400571eb in __libc_start_main() in 'libc-start.c', line 90
#1 0x08048b39 in main(argc=0x2,argv=0xbffffce4) in 'debug.c', line 62
#0 0x080489b3 in seg_crash(n=0x0) in 'debug.c', line 9
/r0/beazley/Projects/WAD/Wad/debug.c, line 9
int *a = 0;
if (n > 0) seg_crash(n-1);
=> *a = 3;
return 1;
}
For scripting languages, WAD works in a similar manner--simply link
your scripting language extension module with the appropriate WAD library
(wadpy, wadtcl, wadpl). For example:
% ld -G $(OBJS) -lwadpy -o pymodule.so
When the scripting module is loaded into the interpreter, WAD should
automatically initialize.
4. Debugging Modes
Due to WAD's experimental nature, a number of debugging modes can be set
through the use of environment variables. These variables control WAD's
runtime behavior and cause the system to dump debugging information for
various stages of error recovery. A lot of this data is pretty ugly and
probably only of interest to you if you are trying to debug WAD itself.
WAD_DEBUG_SEGMENT - Displays information about the virtual memory
map and mapping of addresses to segments.
WAD_DEBUG_SYMBOL - Symbol table mapping.
WAD_DEBUG_OBJECT - Loading/Unloading of object files.
WAD_DEBUG_FILE - Loading/Unloading of raw files.
WAD_DEBUG_HOLD - Freezes WAD before it returns from the signal handler.
Useful if you need to attach a debugger to WAD itself.
WAD_DEBUG_STABS - Display stabs data.
WAD_DEBUG_RETURN - Display information about WAD return points.
WAD_DEBUG_SYMBOL_SEARCH - Display all symbols in the symbol table that are
searched.
WAD_DEBUG_UNWIND - Display information about stack unwinding.
WAD_DEBUG_SIGNAL - Display information about signal handling.
WAD_DEBUG_INIT - Print initialization information.
WAD_NOSTACK - Do NOT use an alternative signal handling stack.
This may be necessary on certain Linux systems when
threads are being used.
WAD_ONESHOT - Disable WAD signal handler after first signal has
been received.
WAD_DEBUG_MEMORY - Print information about WAD memory use.
WAD_DEBUG_STRINGS - Print information about WAD string manager.
5. Platform Specific Issues
General:
- WAD does not gracefully recover from errors that corrupt the call
stack (i.e., buffer overlow).
- Errors that destroy the process heap may or may not be recoverable
depending on what has been destroyed.
- WAD does not currently support 64 bit applications on any platform.
- If executables have been stripped, their symbol tables might not
have enough information to recover from errors. Therefore, if you
are using Python, Tcl, or Perl from a binary distribution, you
may want to rebuild non-stripped versions of these packages yourself.
- WAD only works with programs that utilize the ELF32 linking format
and stabs debugging data. Newer formats such as DWARF2 are not
supported at this time.
- WAD does not correctly report argument values for structures or
floating point numbers yet.
- Overly aggressive compiler optimization may lead to very strange
WAD output.
Solaris:
- WAD is extremely slow at collecting debugging information
from large applications.
Linux:
- The interaction of threads and signals are particularly problematic
on this platform and may cause WAD not to work at all. Here are
some specific thread-based issues that may arise:
1. WAD causes the program to crash immediately upon startup.
This appears to be caused by a bug in in the implementation
of sigaction() and the initialization of signals. This
only occurs if WAD is directly linked to an executable
using threads. It does not occur when WAD is dynamically
loaded into a threaded application.
2. Programs may lock up when an error occurs. This is sometimes
caused by an apparently broken implementation of sigaltstack().
One solution to this is to set the following environment
variable:
setenv WAD_NOSTACK
in which case the WAD signal handler will use the same
stack as the thread/process that generates the error.
3. WAD just crashes altogether and doesn't seem to do anything.
It appears that some versions of Linux threads do *not*
pass CPU context information correctly to signal handlers
defined in threaded programs. There is no known fix to
this at this time. Upgrade your system.
- WAD does not work if it is compiled as PIC code. The WAD libraries
should be compiled *without* the -fpic option.
- WAD has to rely upon a heuristic register recovery scheme when it
returns to scripting language interpreters. It seems to
work, but it relies upon a very specific compiler code generation
convention for saving registers in function prologues. It also
relies upon the register save conventions described in the Linux
Assembly HOWTO.
- If you are using WAD with pre-installed binaries for Python, Tcl,
and other scripting languages, it may not work correctly due to
stripped symbol tables. Most Linux installs such as Redhat strip
symbol tables from executables. This makes it difficult for WAD
to determine context correctly (although it may still work since
the dynamic loading symbol table is still available in most cases).
6. Language specific issues
If WAD is linked with a normal C/C++ program, errors simply produce a stack trace
that is printed on standard error.
Python:
WAD tries to raise a Python exception and return. At this time, the exception
merely contains a traceback string. However, in future versions, it may be
possible to access a complete exception object.
Tcl:
WAD returns a Tcl and places the stack trace into the Tcl variable $errorInfo.
The wish shell uses this to dump error information.
Perl:
Perl doesn't seem to have a very well-defined exception handling
mechanism. Standard functions tend to just exit. The WAD handler
produces a C stack trace and produces a Perl stack trace using some
code derived from the sigtrap module.
Note: 3/23/01 - Perl support is currently broken.
7. Testing and Examples
The Test directory contains some very simple code for testing WAD. In the
most simple form, compile the stand-along test program 'debug' as follows:
% cd Test
% make
Now, running it:
% debug
WAD debug program.
Usage: debug type
seg - Fail with an uninitialized pointer.
bus - Fail with a bus error.
abort - Fail with an assertion error.
math - Fail with a math error.
heap - Blow the process heap.
overflow - Buffer overflow on the stack.
% debug seg
WAD debug program.
Segmentation fault.
#2 0x400581eb in __libc_start_main() in 'libc-start.c', line 90
#1 0x08048b61 in main(argc=0x2,argv=0xbffffc54) in 'debug.c', line 85
#0 0x080489d0 in seg_crash() in 'debug.c', line 15
/r0/beazley/Projects/WAD/Test/debug.c, line 15
int seg_crash() {
int *a = 0;
=> *a = 3;
return 1;
}
Additional targets 'make python', 'make tcl', and 'make perl' are also available.
The scripts debug.py, debug.tcl, debug.pl can be used to test these extensions.
8. Documentation
No official documentation exists at this time. However, details
of WAD's design and implementation can be found in papers presented
at the Ninth International Python Conference and the 2000 USENIX
Technical Conference. Both papers can be obtained at:
http://systems.cs.uchicago.edu/wad
9. To-Do
If you find WAD to be interesting or useful, there are a variety of
ways to contribute. Here is the short to-do list:
- Better register management. Try to implement in a more portable
way. Add some support code for recovering local variables
that happen to be stored in registers.
- Add heuristic for recovering functions called through an
-fomit-frame-pointer compiler optimization scheme. This
can probably be determined by looking at the function preamble
machine code. Then one can back-trace to the calling function
and look at it's preamble.
- Continued clean up and modularization of the core. Many of the
internal APIs could be greatly improved.
- Support for ELF64 linking format.
- Support for DWARF2 debugging data.
- Improved support for stack-overflow and heap-corruption. Although WAD
probably won't be able to recover, it still might be able to produce some
informative diagnostics.
- Removal of printf() and other high-level library calls which may not
operate with a corrupted heap.
- Better integration with scripting languages.
- Support for new platforms.
- Support for new scripting languages.
Please contact me if you are interested in working on any of these projects.
Dave Beazley (beazley@cs.uchicago.edu)
June 24, 2001
b19e234375