Python function annotations removed from -py3 option.

Python function annotations containing C/C++ types are no longer
generated when using the -py3 option. Function annotations support
has been moved to a feature to provide finer grained control.
It can be turned on globally by adding:

  %feature("python:annotations", "c");

or by using the command line argument:

  -features python:annotations=c

The implementation is designed to be expandable to support different
annotations implementations. Future implementations could implement
something like the following for generating pure Python types:

  %feature("python:annotations", "python");

or typing module types to conform to PEP-484:

  %feature("python:annotations", "typing");

Closes #1561
Issue #735

CHANGES.current

@@ -7,6 +7,20 @@ the issue number to the end of the URL: https://github.com/swig/swig/issues/
 Version 4.1.0 (in progress)
 ===========================
 
+2022-02-27: wsfulton
+            [Python] #735 #1561 Function annotations containing C/C++ types are no longer
+            generated when using the -py3 option. Function annotations support has been
+            moved to a feature to provide finer grained control. It can be turned on
+            globally by adding:
+
+              %feature("python:annotations", "c");
+
+            or by using the command line argument:
+
+              -features python:annotations=c
+
+            *** POTENTIAL INCOMPATIBILITY ***
+
 2022-02-26: wsfulton
             #655 #1840 Add new warning WARN_LANG_USING_NAME_DIFFERENT to warn when a
             method introduced by a using declaration in a derived class cannot

Doc/Manual/Contents.html

@@ -1478,7 +1478,10 @@
 </ul>
 <li><a href="Python.html#Python_python3support">Python 3 Support</a>
 <ul>
-<li><a href="Python.html#Python_nn74">Function annotation</a>
+<li><a href="Python.html#Python_nn74">Function annotations</a>
+<ul>
+<li><a href="Python.html#Python_annotations_c">C/C++ annotation types</a>
+</ul>
 <li><a href="Python.html#Python_nn75">Buffer interface</a>
 <li><a href="Python.html#Python_nn76">Abstract base classes</a>
 <li><a href="Python.html#Python_nn77">Byte string output conversion</a>

Doc/Manual/Python.html

@@ -133,7 +133,10 @@
 </ul>
 <li><a href="#Python_python3support">Python 3 Support</a>
 <ul>
-<li><a href="#Python_nn74">Function annotation</a>
+<li><a href="#Python_nn74">Function annotations</a>
+<ul>
+<li><a href="#Python_annotations_c">C/C++ annotation types</a>
+</ul>
 <li><a href="#Python_nn75">Buffer interface</a>
 <li><a href="#Python_nn76">Abstract base classes</a>
 <li><a href="#Python_nn77">Byte string output conversion</a>
@@ -6753,37 +6756,64 @@ The following are Python 3 new features that are currently supported by
 SWIG.
 </p>
 
-<H3><a name="Python_nn74">33.12.1 Function annotation</a></H3>
+<H3><a name="Python_nn74">33.12.1 Function annotations</a></H3>
 
 
 <p>
-The <tt>-py3</tt> option will enable function annotation support. When used
-SWIG is able to generate proxy method definitions like this:
-</p>
-
-<div class="code"><pre>
-  def foo(self, bar : "int"=0) -&gt; "void" : ...
-</pre></div>
-
-<p>
-Also, even if without passing SWIG the <tt>-py3</tt> option, the parameter list
-still could be generated:
-</p>
-
-<div class="code"><pre>
-  def foo(self, bar=0): ...
-</pre></div>
-
-<p>
-But for overloaded function or method, the parameter list would fallback to
-<tt>*args</tt> or <tt>self, *args</tt>, and <tt>**kwargs</tt> may be append
-depend on whether you enabled the keyword argument. This fallback is due to
-all overloaded functions share the same function in SWIG generated proxy class.
-</p>
-
-<p>
-For detailed usage of function annotation, see
+Python 3 supports function annotations as defined in
 <a href="https://www.python.org/dev/peps/pep-3107/">PEP 3107</a>.
+Annotation support is via a <tt>python:annotations</tt>
+<a href="Customization.html#Customization_features">%feature directives</a>.
+SWIG currently supports one type of function annotation.
+</p>
+
+<H4><a name="Python_annotations_c">33.12.1.1 C/C++ annotation types</a></H4>
+
+
+<p>
+The <tt>%feature("python:annotations", "c")</tt> directive generates function annotations
+containing C/C++ types. For example:
+</p>
+
+<div class="code"><pre>
+%feature("python:annotations", "c") global_ints;
+int *global_ints(int &amp;ri);
+</pre></div>
+
+<p>
+The generated code then contains function annotations containing the C types:
+</p>
+
+<div class="targetlang"><pre>
+def global_ints(ri: "int &amp;") -&gt; "int *":
+    return _python_annotations_c.global_ints(ri)
+</pre></div>
+
+<p>
+There are some limitations with annotations support, for example, overloaded functions use
+<tt>*args</tt> or <tt>**kwargs</tt> when keyword arguments are enabled.
+The parameter names and types are then not shown. For example, with input:
+</p>
+
+<div class="code"><pre>
+int *global_overloaded(int &amp;ri);
+int *global_overloaded();
+</pre></div>
+
+<p>
+The generated Python function including annotations is shown below.
+Only the return type is annotated.
+</p>
+
+<div class="targetlang"><pre>
+def global_overloaded(*args) -&gt; "int *":
+    return _python_annotations_c.global_overloaded(*args)
+</pre></div>
+
+<p>
+<b>Compatibility Note:</b> SWIG-4.1.0 changed the way that function annotations are generated.
+Prior versions required the <tt>-py3</tt> option which enabled function annotation support
+containing C/C++ types instead of supporting <tt>%feature("python:annotations", "c")</tt>.
 </p>
 
 <H3><a name="Python_nn75">33.12.2 Buffer interface</a></H3>

Examples/test-suite/python/Makefile.in

@@ -48,6 +48,7 @@ CPP_TEST_CASES += \
 	li_std_wstring_inherit \
 	primitive_types \
 	python_abstractbase \
+	python_annotations_c \
 	python_append \
 	python_builtin \
 	python_destructor_exception \

Examples/test-suite/python/python_annotations_c_runme.py

@@ -0,0 +1,27 @@
+import sys
+
+if sys.version_info[0:2] >= (3, 2):
+    from python_annotations_c import *
+
+    anno = MakeShort.__annotations__
+    if anno != {'x': 'int', 'return': 'Space::Template< short >'}:
+        raise RuntimeError("annotations mismatch: {}".format(anno))
+
+    anno = global_ints.__annotations__
+    if anno != {'ri': 'int &', 't': 'TemplateShort', 'return': 'int *'}:
+        raise RuntimeError("annotations mismatch: {}".format(anno))
+
+    ts = MakeShort(10)
+
+    anno = MakeShort.__annotations__
+    if anno != {'x': 'int', 'return': 'Space::Template< short >'}:
+        raise RuntimeError("annotations mismatch: {}".format(anno))
+
+    anno = ts.mymethod.__annotations__
+    if anno != {'arg2': 'int', 'tt': 'TemplateShort', 'return': 'void'}:
+        raise RuntimeError("annotations mismatch: {}".format(anno))
+
+    # No annotations
+    anno = no_annotations.__annotations__
+    if anno != {}:
+        raise RuntimeError("annotations mismatch: {}".format(anno))

Examples/test-suite/python_annotations_c.i

@@ -0,0 +1,26 @@
+%module python_annotations_c
+
+// Tests the C/C++ annotations that were automatically added by using -py3 before swig-4.1.0
+// In swig-4.1.0 and later, the feature below is needed as annotations are no longer generated with -py3
+%feature("python:annotations", "c") mymethod;
+%feature("python:annotations", "c") makeT<short>;
+%feature("python:annotations", "c") global_ints;
+
+%inline %{
+namespace Space {
+template<class T>
+struct Template {
+  void mymethod(int, Template* tt) {}
+};
+}
+template<typename T>
+Space::Template<T> makeT(int x) {
+  return Space::Template<T>();
+};
+int *global_ints(int &ri, Space::Template<short> t) { return &ri; }
+int *global_overloaded(int &ri) { return &ri; }
+int *global_overloaded() { return NULL; }
+int *no_annotations(int &ri, const char *c) { return NULL; }
+%}
+%template(TemplateShort) Space::Template<short>;
+%template(MakeShort) makeT<short>;

Source/Modules/python.cxx

@@ -316,7 +316,6 @@ public:
 	  } else {
 	    Swig_arg_error();
 	  }
-	  /* end added */
 	} else if (strcmp(argv[i], "-globals") == 0) {
 	  if (argv[i + 1]) {
 	    global_name = NewString(argv[i + 1]);
@@ -2275,7 +2274,7 @@ public:
       return parms;
     }
 
-    bool funcanno = py3 ? true : false;
+    bool funcanno = Equal(Getattr(n, "feature:python:annotations"), "c") ? true : false;
     String *params = NewString("");
     String *_params = make_autodocParmList(n, false, ((in_class || has_self_for_count)? 2 : 1), is_calling, funcanno);
 
@@ -2391,8 +2390,8 @@ public:
       if (ret)
 	ret = SwigType_str(ret, 0);
     }
-    return (ret && py3) ? NewStringf(" -> \"%s\"", ret)
-	: NewString("");
+    bool funcanno = Equal(Getattr(n, "feature:python:annotations"), "c") ? true : false;
+    return (ret && funcanno) ? NewStringf(" -> \"%s\"", ret) : NewString("");
   }
 
   /* ------------------------------------------------------------