Also fixes li_std_vector_enum testcase when run with -threads.
Patch supplied on swig-devel mailing list on 12 Sep with details...
==============================================
I just wanted to mention that I found a crash issue in bug..
I am using SWIG 2.0.11 with python and have –threads enabled. I have a C++ std::vector that I instantiate in SWIG with %template. I also have a method in a class that returns this vector. I also include std_vector.i, btw..
When I iterate like so:
children = Action.getActionList()
for child in children:
pass
Everything is fine..
When I iterate like this:
for child in Action.getActionList()
pass
Product crashes.
The problem is the following. This code gets called first:
SWIGINTERN PyObject *_wrap_delete_SwigPyIterator(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
PyObject *resultobj = 0;
swig::SwigPyIterator *arg1 = (swig::SwigPyIterator *) 0 ;
void *argp1 = 0 ;
int res1 = 0 ;
PyObject * obj0 = 0 ;
if(!PyArg_UnpackTuple(args,(char *)"delete_SwigPyIterator",1,1,&obj0)) SWIG_fail;
res1 = SWIG_ConvertPtr(obj0, &argp1,SWIGTYPE_p_swig__SwigPyIterator, SWIG_POINTER_DISOWN | 0 );
if (!SWIG_IsOK(res1)) {
SWIG_exception_fail(SWIG_ArgError(res1), "in method '" "delete_SwigPyIterator" "', argument " "1"" of type '" "swig::SwigPyIterator *""'");
}
arg1 = reinterpret_cast< swig::SwigPyIterator * >(argp1);
{
SWIG_PYTHON_THREAD_BEGIN_ALLOW;
delete arg1;
SWIG_PYTHON_THREAD_END_ALLOW;
}
resultobj = SWIG_Py_Void();
return resultobj;
fail:
return NULL;
}
Note the SWIG_PYTHON_THREAD_BEGIN_ALLOW/END_ALLOW. In between those two statements, we delete arg1. That in turn will eventually end up in this code:
namespace swig {
class SwigPtr_PyObject {
protected:
PyObject *_obj;
public:
… snip! …
~SwigPtr_PyObject()
{
Py_XDECREF(_obj);
}
Uh-oh! We call Py_XDECREF when we aren’t supposed to because we are in a SWIG_PYTHON_THREAD_BEGIN_ALLOW/END_ALLOW section!
This takes care of the issue:
namespace swig {
class SwigPtr_PyObject {
protected:
PyObject *_obj;
public:
… snip! …
~SwigPtr_PyObject()
{
SWIG_PYTHON_THREAD_BEGIN_BLOCK;
Py_XDECREF(_obj);
SWIG_PYTHON_THREAD_END_BLOCK;
}
There are several other methods in this class that use the Python API, but don’t have the BEGIN/END block defined. I’m not sure if they are required for all of them, but I believe they are..
I have attached a modified pyclasses.swg with what I believe are the correct changes. This code is from 2.0.11, but as far as I can tell, it’s the same as what is in 3.0.2…
Apologies for not doing more here (making/running tests, getting it in the code repository, etc..), but I’m under some pressure to get some unrelated things done…
Add %feature("doxygen:ignore:<command>") implementation, documentation and
test case.
This feature allows to use custom tags in C++ Doxygen comments for
C++-specific things that don't make sense in the context of the target
language and also allows to insert contents specific to the target language in
the C++ comments using (different) custom commands, which is very useful in
practice to explain the particularities of the API wrappers.
In addition to translating the comment itself, also document the type of the
object returned. This is consistent with generating not only :param: but also
:type: for the parameters documented using @param.
The most important change is to ensure that the "Overload" headers added by
PyDocConverter code are indented correctly, i.e. using the same indent level
as the comment itself, otherwise Sphinx directives such as :param: were not
parsed correctly.
Also add a vertical separator using reST "|" formatting character to increase
separation between the overloads.
This was done for plain functions/methods but not constructors, which don't
have "kind=function" attribute.
Just don't check for "kind" at all, the presence of "sym:overloaded" should be
good enough to indicate that it's an overloaded something.
Also add a test for overloaded constructors documentation.
Enum values are just (integer) constants in Python and so can be used as the
function default values just as well as literal numbers, account for this when
checking whether function parameters can be represented in Python.
Also rename is_primitive_defaultargs() to is_representable_as_pyargs() to
describe better what this function does.
Make the rules for combining explicitly specified docstring, autodoc one and
the one obtained by translating Doxygen comments implicit in the structure of
the code itself instead of writing complicated conditions checking them.
This results in small changes to the whitespace in the generated Python code
when using autodoc, but this makes it PEP 8-compliant, so it is the right
thing to do anyhow.
Also cache the docstring built from translated Doxygen comments. The existing
code seemed to intend to do it, but didn't, really. This helps with
performance generally speaking (-10% for a relatively big library using a lot
of Doxygen comments) and also makes debugging Doxygen translation code less
painful as it's executed only once instead of twice for each comment.
Finally, avoid putting "r", used for Python raw strings, into docstrings in C
code, it is really not needed there.
It is very useful to be able to click on the parameter types in a function
documentation to go to this parameter description, so always use hyperlink
markup for the parameters of class types, even if not all of them are
necessarily documented -- Sphinx doesn't seem to complain about links to
unknown classes.
For the parameter documentation to be really taken as such, in its entirety,
by Sphinx, it must be indented relative to the :param: tag. Do this by
appending an extra indent after every line of the output and work around the
unnecessary indent of the last line by removing the trailing whitespace.
This required updating the existing tests and removing the expected but not
present any more whitespace from them, but as trailing whitespace in the
documentation is at best insignificant (and at worst harmful) anyhow, this is
not a big price to pay for simpler translator code.
This is important to preserve the structure of the lists which appear
correctly in Python output without any additional effort if the indentation is
lost.
It is also makes the behaviour consistent for
/**
*
*
*/
comments and those without the asterisks in the middle lines, as now the
indentation is preserved in both cases while it was only preserved when the
asterisks were present previously.
Sphinx doesn't allow sections inside the function documentation and gives tons
of SEVERE warnings for them, so while just emphasizing the "Overload" header
is not ideal, it is better than before because at least the string "Overload"
itself appears in the Sphinx-generated output.
Show that there is trailing white space in the output in an ugly but
functional way, as without this it's impossible to determine what is the
actual change at all.
Using C++ types in documentation for Python users is more harmful than
useless, so use Python types whenever possible and allow defining "doctype"
typemap to customize this for the user-defined types.
Enum values are just (integer) constants in Python and so can be used as the
function default values just as well as literal numbers, account for this when
checking whether function parameters can be represented in Python.
Also rename is_primitive_defaultargs() to is_representable_as_pyargs() to
describe better what this function does.
reST is standard Python markup, so use *...*, ``...`` and so on instead of
_..._, '...' etc.
No other changes even though the mapping of some Doxygen tags to markup used
for them seems suspicions (e.g. \var almost certainly should be the same as
\em).
Use the more or less standard :param:, :type:, :return: and :raises: in the
function/methods descriptions.
Update the output expected from the Python tests accordingly.
This is much more readable than just dumping the expected output and the
actual result produced by SWIG, especially for longer Doxygen comments.
Remove the bunch of commented out code and also do not save the results
in the file, this is not very useful as it's not even clear which test the
files correspond to and annoying when these files are left over later. If
anybody really uses this functionality, it ought to be optional.
Update Doxygen-specific Python unit tests to work with the new indentation.
Update one of Doxygen-specific Java tests to still build with the new handling
of srcdir.
- Examples/Makefile.in rules use SRCDIR as the relative source directory
- ./config.status replicates Examples/ source directory tree in build
directory, and copies each Makefile to build directory, prefixed with
a header which sets SRCDIR to source directory
- Examples/test-suite/.../Makefile.in set SRCDIR from Autoconf-set srcdir
- Examples/test-suite/errors/Makefile.in needs to filter out source
directory from SWIG error messages
- Lua: embedded interpreters are passed location of run-time test
- Python: copy run-time scripts to build directory because of 2to3
conversion; import_packages example copies __init__.py from source
directory; test-suite sets SCRIPTDIR to location of run-time tests
- Javascript: binding.gyp renamed to binding.gyp.in so that $srcdir
can be substituted with SRCDIR; removed './' from require() statements
so that NODE_PATH can be used to point Node.js to build directory
- some of the %.clean rules in the test-suite Makefiles were using a single tab
as an empty rule, dangerous! I've replaced these with the safer '@exit 0'.
