From ba40c4a2562c935dd2846dd15cdfc55952c7b8f6 Mon Sep 17 00:00:00 2001 From: Alec Cooper Date: Thu, 28 Jan 2016 08:29:01 -0500 Subject: [PATCH] Documentation on Python Bytes/Unicode distinction --- CHANGES.current | 5 +++ Doc/Manual/Python.html | 85 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+) diff --git a/CHANGES.current b/CHANGES.current index 2a0017822..f99e70398 100644 --- a/CHANGES.current +++ b/CHANGES.current @@ -5,6 +5,11 @@ See the RELEASENOTES file for a summary of changes in each release. Version 3.0.9 (in progress) =========================== +2016-01-27: ahnolds + [Python] Added support for differentiating between Python Bytes + and Unicode objects using by defining SWIG_PYTHON_STRICT_BYTE_CHAR + and SWIG_PYTHON_STRICT_UNICODE_WCHAR. + 2016-01-27: steeve [Go] Ensure structs are properly packed between gc and GCC/clang. diff --git a/Doc/Manual/Python.html b/Doc/Manual/Python.html index c691d89cf..0cd656ff2 100644 --- a/Doc/Manual/Python.html +++ b/Doc/Manual/Python.html @@ -6165,6 +6165,84 @@ For more details about the surrogateescape error handler, please see PEP 383.

+

+In some cases, users may wish to instead handle all byte strings as bytes +objects in Python 3. This can be accomplished by adding +SWIG_PYTHON_STRICT_BYTE_CHAR to the generated code: +

+ +
+%module char_to_bytes
+%begin %{
+#define SWIG_PYTHON_STRICT_BYTE_CHAR
+%}
+
+char *charstring(char *s) {
+  return s;
+}
+
+ +

+This will modify the behavior so that only Python 3 bytes objects will be +accepted and converted to a C/C++ string, and any string returned from C/C++ +will be converted to a bytes object in Python 3: +

+ +
+>>> from char_to_bytes import *
+>>> charstring(b"hi") # Byte string
+b'hi'
+>>> charstring("hi")  # Unicode string
+Traceback (most recent call last):
+  File "<stdin>", line 1, in ?
+TypeError: in method 'charstring', argument 1 of type 'char *'
+
+ +

+Note that in Python 2, defining SWIG_PYTHON_STRICT_BYTE_CHAR has no +effect, since strings in Python 2 are equivalent to Python 3 bytes objects. +However, there is a similar capability to force unicode-only handling for +wide characters C/C++ strings (wchar_t * or std::wstring +types) in Python 2. By default, in Python 2 both strings and unicode strings +are converted to C/C++ wide strings, and returned wide strings are converted +to a Python unicode string. To instead only convert unicode strings to wide +strings, users can add SWIG_PYTHON_STRICT_UNICODE_WCHAR to the +generated code: +

+ +
+%module wchar_to_unicode
+%begin %{
+#define SWIG_PYTHON_STRICT_UNICODE_WCHAR
+%}
+
+wchar_t *wcharstring(wchar_t *s) {
+  return s;
+}
+
+ +

+This ensures that only unicode strings are accepted by wcharstring in both +Python 2 and Python 3: +

+ +
+>>> from wchar_to_unicode import *
+>>> wcharstring(u"hi") # Unicode string
+u'hi'
+>>> wcharstring(b"hi") # Byte string
+Traceback (most recent call last):
+  File "<stdin>", line 1, in ?
+TypeError: in method 'charstring', argument 1 of type 'wchar_t *'
+
+ +

+By defining both SWIG_PYTHON_STRICT_BYTE_CHAR and +SWIG_PYTHON_STRICT_UNICODE_WCHAR, Python wrapper code can support +overloads taking both std::string (as Python bytes) and std::wstring +(as Python unicode). +

+

36.12.5 Python 2 Unicode

@@ -6230,6 +6308,13 @@ but note that they are returned as a normal Python 2 string: >>> +

+Note that defining both SWIG_PYTHON_2_UNICODE and +SWIG_PYTHON_STRICT_BYTE_CHAR at the same time is not allowed, since +the first is allowing unicode conversion and the second is explicitly +prohibiting it. +

+