Documentation for improved python import option
Docs for SFbug1297 patch (PR #7) closes #111
This commit is contained in:
Paweł Tomulik
William S Fulton
parent
44dd28950c
commit
f53bd5a1e1
1 changed files with 370 additions and 5 deletions
|
|
@ -105,6 +105,12 @@
|
|||
<li><a href="#Python_nn71">%feature("docstring")</a>
|
||||
</ul>
|
||||
<li><a href="#Python_nn72">Python Packages</a>
|
||||
<ul>
|
||||
<li><a href="#Python_modulepackage">%module(package="...") syntax</a>
|
||||
<li><a href="#Python_absrelimports">Absolute and relative imports</a></li>
|
||||
<li><a href="Python_absimport">Enforcing absolute import semantics</a></li>
|
||||
<li><a href="#Python_importfrominit">Importing from __init__.py</a></li>
|
||||
</ul>
|
||||
<li><a href="#Python_python3support">Python 3 Support</a>
|
||||
<ul>
|
||||
<li><a href="#Python_nn74">Function annotation</a>
|
||||
|
|
@ -5273,11 +5279,66 @@ with more than one line.
|
|||
|
||||
<H2><a name="Python_nn72"></a>34.11 Python Packages</H2>
|
||||
|
||||
<p>Python has concepts of modules and packages. Modules are separate units of
|
||||
code and may be grouped together to form a package. Packages may be nested,
|
||||
that is they may contain subpackages. This leads to tree-like hierarchy, with
|
||||
packages as intermediate nodes and modules as leaf nodes.</p>
|
||||
|
||||
<p>The hierarchy of python packages/modules follows the hierarchy of
|
||||
<tt>*.py</tt> files found in source tree (or, more generally, in python path).
|
||||
Normally, the developer creates new module by placing a <tt>*.py</tt> file
|
||||
somewhere under python path; the module is then named after that <tt>*.py</tt>
|
||||
file. A package is created by placing <tt>__init__.py</tt> file within a
|
||||
directory; the package is then named after that directory. For example, the
|
||||
following source tree:</p>
|
||||
|
||||
<div class="diagram">
|
||||
<pre>
|
||||
mod1.py
|
||||
pkg1/__init__.py
|
||||
pkg1/mod2.py
|
||||
pkg1/pkg2/__init__.py
|
||||
pkg1/pkg2/mod3.py
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
Using the <tt>package</tt> option of the <tt>%module</tt> directive
|
||||
allows you to specify what Python package that the module will be
|
||||
living in when installed.
|
||||
defines the following python packages and modules:
|
||||
</p>
|
||||
|
||||
<div class="diagram">
|
||||
<pre>
|
||||
pkg1 # package
|
||||
pkg1.pkg2 # package
|
||||
mod1 # module
|
||||
pkg1.mod2 # module
|
||||
pkg1.pkg2.mod3 # module
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
The purpose of <tt>__init__.py</tt> file is two-fold. First, existence of
|
||||
<tt>__init__.py</tt> in a directory informs python interpreter that this
|
||||
directory contains python package. Second, the code of <tt>__init__.py</tt> is
|
||||
loaded/executed automatically when the package is initialized (when it or its
|
||||
submodule/subpackage gets <tt>import</tt>'ed). By default, swig generates
|
||||
proxy python code – one <tt>*.py</tt> file for each <tt>*.i</tt>
|
||||
interface. The <tt>__init__.py</tt> files, however, are not generated by swig.
|
||||
They should be created by other means. Both files (module <tt>*.py</tt> and
|
||||
<tt>__init__.py</tt>) should be installed in appropriate destination
|
||||
directories in order to obtain desirable package/module hierarchy.
|
||||
</p>
|
||||
|
||||
<p>The way python defines its modules and packages impacts swig users. Some
|
||||
users may need to use special features such as <tt>package</tt> option of
|
||||
<tt>%module</tt> directive or import-related command-line options. These are
|
||||
explained in the following sections.</p>
|
||||
|
||||
<H3><a name="Python_modulepackage"></a>34.11.1 %module(package="...") syntax</H3>
|
||||
|
||||
<p>
|
||||
Using the <tt>package</tt> option of the <tt>%module</tt> directive allows you
|
||||
to specify a Python package that the module will be living in when installed.
|
||||
</p>
|
||||
|
||||
<div class="code">
|
||||
|
|
@ -5294,10 +5355,314 @@ if they live in separate Python packages then that won't work.
|
|||
However if the importee specifies what its package is with the
|
||||
<tt>%module</tt> option then the Python code generated for the
|
||||
importer will use that package name when importing the other module
|
||||
and also in base class declarations, etc. if the package name is
|
||||
different than its own.
|
||||
and also in base class declarations, etc..
|
||||
</p>
|
||||
|
||||
<p>Swig assumes, that the <tt>package</tt> option provided to <tt>%module</tt>
|
||||
together with the <tt>module</tt> name (that is, <tt>wx.xrc</tt> in the above
|
||||
example) forms a fully qualified (absolute) name of a module (in Python terms).
|
||||
This is important especially for Python 3, where absolute imports are used by
|
||||
default. It's up to you to place the generated module files (<tt>.py</tt>,
|
||||
<tt>.so</tt>) in proper subdirectories. For example, if you have
|
||||
interface <tt>foo.i</tt> with:
|
||||
</p>
|
||||
|
||||
<div class="code">
|
||||
<pre>
|
||||
%module(package="pkg1.pkg2") foo
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
then the resultant directory layout should be
|
||||
</p>
|
||||
|
||||
<div class="diagram">
|
||||
<pre>
|
||||
pkg1/
|
||||
pkg1/__init__.py
|
||||
pkg1/pkg2/__init__.py
|
||||
pkg1/pkg2/foo.py # (generated by swig)
|
||||
pkg1/pkg2/_foo.so # (shared library from C/C++ code generated by SWIG)
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<H3><a name="Python_absrelimports"></a>34.11.2 Absolute and relative imports</H3>
|
||||
|
||||
<p>Suppose, we have the following hierarchy of files:</p>
|
||||
|
||||
<div class="diagram">
|
||||
<pre>
|
||||
pkg1/
|
||||
pkg1/__init__.py
|
||||
pkg1/mod2.py
|
||||
pkg1/pkg2/__init__.py
|
||||
pkg1/pkg2/mod3.py
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>Let the contents of <tt>pkg1/pkg2/mod3.py</tt> be</p>
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
class M3: pass
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
We edit <tt>pkg1/mod2.py</tt> and want to import module of
|
||||
<tt>pkg1/pkg2/pkg3.py</tt> in order to derive from class <tt>M3</tt>. We can
|
||||
write appropriate python code in several ways, for example:
|
||||
</p>
|
||||
|
||||
<ol>
|
||||
<li><p>Using "<tt>import <></tt>" syntax with absolute package name:</p>
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
# pkg1/mod2.py
|
||||
import pkg1.pkg2.mod3
|
||||
class M2(pkg1.pkg2.mod3.M3): pass
|
||||
</pre>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
<li><p>Using "<tt>import <></tt>" syntax with package name relative to
|
||||
<tt>pkg1</tt> (only in python 2.7 and earlier):</p>
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
# pkg1/mod2.py
|
||||
import pkg2.mod3
|
||||
class M2(pkg2.mod3.M3): pass
|
||||
</pre>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
<li><p>Using "<tt>from <> import <></tt>" syntax (relative import
|
||||
syntax, only in python 2.5 and later):</p>
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
# pkg1/mod2.py
|
||||
from .pkg2 import mod3
|
||||
class M2(mod3.M3): pass
|
||||
</pre>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
<li><p>Other variants, for example the following construction in order to
|
||||
have the <tt>pkg2.mod3.M3</tt> symbol available in <tt>mod2</tt> as
|
||||
in point 2 above (but now under python 3):</p>
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
# pkg1/mod2.py
|
||||
from . import pkg2
|
||||
from .pkg2 import mod3
|
||||
class M2(pkg2.mod3.M3): pass
|
||||
</pre>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
</ol>
|
||||
|
||||
<p>Now suppose we have <tt>mod2.i</tt> with</p>
|
||||
|
||||
<div class="code">
|
||||
<pre>
|
||||
// mod2.i
|
||||
%module (package="pkg1") mod2
|
||||
%import "mod3.i"
|
||||
// ...
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>and <tt>mod3.i</tt> with</p>
|
||||
|
||||
<div class="code">
|
||||
<pre>
|
||||
// mod3.i
|
||||
%module (package="pkg1.pkg2") mod3
|
||||
// ...
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>By default, swig would generate <tt>mod2.py</tt> proxy file with
|
||||
<tt>import</tt> directive as in point 1. This may be changed with
|
||||
<tt>-relativeimport</tt> CLI option. The <tt>-relativeimport</tt> instructs
|
||||
swig to organize imports as in point 2 (for python 2.x) or as in point 4 (for
|
||||
python 3, that is when -py3 CLI option is enabled). In short, if you have
|
||||
<tt>mod2.i</tt> and <tt>mod3.i</tt> as above, then without
|
||||
<tt>-relativeimport</tt> swig will write</p>
|
||||
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
import pkg1.pkg2.mod3
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>to <tt>mod2.py</tt> proxy file, and with <tt>-relativeimport</tt> it will
|
||||
write</p>
|
||||
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
import pkg2.mod3
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>if <tt>-py3</tt> is not used, or</p>
|
||||
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
from . import pkg2
|
||||
import pkg1.pkg2.mod3
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>when <tt>-py3</tt> option is used.</p>
|
||||
|
||||
<p>You should avoid using relative imports and use absolute ones whenever
|
||||
possible. There are some cases, however, when relative imports may be
|
||||
necessary. The first example is, when some (legacy) python code refers entities
|
||||
imported by proxy files generated by swig, and it assumes that the proxy file
|
||||
uses relative imports. Second case is, when one puts import directives in
|
||||
<tt>__init__.py</tt> to import symbols from submodules or subpackages and the
|
||||
submodule depends on other submodules (discussed later).</p>
|
||||
|
||||
<H3><a name="Python_absimport"></a>34.11.3 Enforcing absolute import semantics</H3>
|
||||
|
||||
<p>As you may know, there is incompatibility in import semantics (for the
|
||||
<tt>import <></tt> syntax) between python 2 and 3. In python 2.4 and
|
||||
earlier it is not clear whether</p>
|
||||
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
import foo
|
||||
</pre>
|
||||
</div>
|
||||
<p>refers to a top-level module or to another module inside the current
|
||||
package. In python 3 it always refers to a top-level module
|
||||
(see <a href="http://www.python.org/dev/peps/pep-0328/">PEP 328</a>).
|
||||
To instruct python 2.5 through 2.7 to use new semantics (that is <tt>import
|
||||
foo</tt> is interpreted as absolute import), one have to put the following
|
||||
line
|
||||
</p>
|
||||
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
from __future__ import absolute_import
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>at the very beginning of his proxy <tt>*.py</tt> file. In swig, it may be
|
||||
accomplished with <tt>%pythonbegin</tt> directive.</p>
|
||||
|
||||
<H3><a name="Python_importfrominit"></a>34.11.4 Importing from __init__.py</H3>
|
||||
|
||||
<p>Imports in <tt>__init__.py</tt> are handy when you want to populate
|
||||
package's namespace with names imported from other modules. In swig-based
|
||||
projects this approach may also be used to split large piece of code into
|
||||
smaller modules, compile them in parallel and then re-assemble all stuff at
|
||||
python level by importing submodules' contents in <tt>__init__.py</tt>, for
|
||||
example.</p>
|
||||
|
||||
<p>Unfortunately import directives in <tt>__init__.py</tt> may cause troubles,
|
||||
especially if they refer to package's submodules. This is caused by the way
|
||||
python initializes packages. If you spot problems with imports from
|
||||
<tt>__init__.py</tt> try using <tt>-relativeimport</tt> option. Below we
|
||||
explain in detail one issue, for which the <tt>-relativeimport</tt> workaround
|
||||
may be helpful.</p>
|
||||
|
||||
<p>Consider the following example (python 3):</p>
|
||||
|
||||
<div class="diagram">
|
||||
<pre>
|
||||
pkg1/__init__.py # (empty)
|
||||
pkg1/pkg2/__init__.py # (imports something from bar.py)
|
||||
pkg1/pkg2/foo.py
|
||||
pkg1/pkg2/bar.py # (imports foo.py)
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>Let's the files' contents be:</p>
|
||||
|
||||
<ul>
|
||||
<li> <p>for <tt>pkg1/pkg2/__init__.py:</tt></p>
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
# pkg1/pkg2/__init__.py
|
||||
from .bar import Bar
|
||||
</pre>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
<li> <p>for <tt>pkg1/pkg2/foo.py:</tt></p>
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
# pkg1/pkg2/foo.py
|
||||
class Foo: pass
|
||||
</pre>
|
||||
</div>
|
||||
</li>
|
||||
|
||||
<li> <p>for <tt>pkg1/pkg2/foo.py:</tt></p>
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
# pkg1/pkg2/bar.py
|
||||
import pkg1.pkg2.foo
|
||||
class Bar(pkg1.pkg2.foo.Foo): pass
|
||||
</pre>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<p>Now one would simply do <tt>import pkg1.pkg2</tt>, but this usually fails:</p>
|
||||
|
||||
<div class="diagram">
|
||||
<pre>
|
||||
>>> import pkg1.pkg2
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in <module>
|
||||
File "./pkg1/pkg2/__init__.py", line 2, in <module>
|
||||
from .bar import Bar
|
||||
File "./pkg1/pkg2/bar.py", line 3, in <module>
|
||||
class Bar(pkg1.pkg2.foo.Foo): pass
|
||||
AttributeError: 'module' object has no attribute 'pkg2'
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>Surprisingly, if we execute the <tt>import pkg1.pkg2</tt> directive for the
|
||||
second time, it succeeds. The reason seems to be following: when python spots
|
||||
the <tt>from .bar import Bar</tt> directive in <tt>pkg1/pkg2/__init__.py</tt>
|
||||
it starts loading <tt>pkg1/pkg2/bar.py</tt>. This module imports
|
||||
<tt>pkg1.pkg2.foo</tt> in turn and tries to use <tt>pkg1.pkg2.foo.Foo</tt>, but
|
||||
the package <tt>pkg1</tt> is not fully initialized yet (the initialization
|
||||
procedure is actually in progress) and it seems like the effect of already seen
|
||||
directive <tt>import pkg1.pkg2.pkg3.foo</tt> is "delayed" or ignored. Exactly
|
||||
same may happen to a proxy module generated by swig.</p>
|
||||
|
||||
<p>It's observed, that a possible workaround for this case is to use relative
|
||||
import in <tt>pkg1/pkg2/bar.py</tt>. If we change <tt>bar.py</tt> to be:</p>
|
||||
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
from .pkg3 import foo
|
||||
class Bar(foo.Foo): pass
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>or</p>
|
||||
|
||||
<div class="targetlang">
|
||||
<pre>
|
||||
from . import pkg3
|
||||
from .pkg3 import foo
|
||||
class Bar(pkg3.foo.Foo): pass
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<p>then the example works again. With swig, you need to enable the
|
||||
<tt>-relativeimport</tt> option in order to have the above workaround in
|
||||
effect (note, that python 2 case also needs <tt>-relativeimport</tt>
|
||||
workaround).</p>
|
||||
|
||||
|
||||
<H2><a name="Python_python3support"></a>34.12 Python 3 Support</H2>
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue