Skip to content

Commit

Permalink
python: Fix pickling support for Python 3.11
Browse files Browse the repository at this point in the history
The addition of object.__getstate__ in Python 3.11(*) broke logic in
the pickling support code that assumed that __getstate__ would only
exist on a wrapped class if it were explicitly provided. Under
3.11+, this code now ignores __getstate__ if it came from object
and not the class itself.

This maintains behavior prior to Python 3.11 when pickling instances
with attributes added directly to their __dict__. A new test case was
added to pickle1 to exercise this behavior. This also fixes other
failures in pickle1 and pickle4 where __reduce__ would otherwise
include an unexpected "None" entry in the tuple it returns.

(*) https://docs.python.org/3/whatsnew/3.11.html#other-language-changes

Fixes #3384

(Internal change: 2351569)
  • Loading branch information
sunyab authored and pixar-oss committed Dec 16, 2024
1 parent 08ce683 commit 45a2d33
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 0 deletions.
19 changes: 19 additions & 0 deletions pxr/external/boost/python/src/object/pickle_support.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,25 @@ namespace {
}
result.append(initargs);
object getstate = getattr(instance_obj, "__getstate__", none);

// Python 3.11 added a default implementation of __getstate__ to
// object, which we need to ignore to maintain previous behavior.
//
// For example, if a class had pickle support enabled but did not
// provide __getstate__, instances with items directly added to their
// __dict__ would skip the __getstate_manages_dict__ safeguard below
// prior to 3.11. After 3.11, the safeguard would be triggered.
#if PY_VERSION_HEX >= 0x030b00f0
if (!getstate.is_none()) {
object class_getstate = getattr(instance_class, "__getstate__", none);
handle<> obj_getstate(PyObject_GetAttrString(
(PyObject*)&PyBaseObject_Type, "__getstate__"));
if (class_getstate.ptr() == obj_getstate.get()) {
getstate = none;
}
}
#endif

object instance_dict = getattr(instance_obj, "__dict__", none);
long len_instance_dict = 0;
if (!instance_dict.is_none()) {
Expand Down
9 changes: 9 additions & 0 deletions pxr/external/boost/python/test/pickle1.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,15 @@
>>> print(wl.greet())
Hello from California!
>>> wd = pickle1_ext.world('Emeryville')
>>> wd.zipcode = 94608
>>> pstr = pickle.dumps(wd)
>>> wl = pickle.loads(pstr)
>>> print(wd.greet(), "({})".format(wd.zipcode))
Hello from Emeryville! (94608)
>>> print(wl.greet(), "({})".format(wl.zipcode))
Hello from Emeryville! (94608)
>>> noop = pickle1_ext.noop()
>>> try: pickle.dumps(noop)
... except RuntimeError as e: print(str(e)[:55])
Expand Down

0 comments on commit 45a2d33

Please sign in to comment.