Difference in behavior between pure Python pickle and C extension _pickle when unpickling instance of instance of metaclass

[jamesmurphy-mc]

Bug report

When pickling and unpickling an instance of an instance of a metaclass, _pickle reaches into the class object directly and calls tp_new from the C side, whereas pickle uses cls.__new__, which triggers a custom __getattribute__ if defined. Whether this is a bug or just an intentional optimization I'm not sure but at the very least one there is an observable difference in behavior in that unpickling from a distribution where _pickle is available does not call __getattribute__ but unpickling from a distribution where _pickle is not available does call __getattribute__.

MWE

(Works on all available CPythons on Compiler Explorer, 3.5 - 3.11)
https://godbolt.org/z/x76bs8dzv

To simulate different distributions having _pickle available or not, we can use a meta path entry to cause it to fail to import, causing pickle to fall back to the pure Python version.

import importlib.abc
import sys

class HideModuleFinder(importlib.abc.MetaPathFinder):
    def __init__(self, hidden):
        self.hidden = set(hidden)

    def find_spec(self, fullname, path, target=None):
        if fullname in self.hidden:
            raise ImportError("Module is hidden")
        return None  # let next finder try

def install(hidden):
    sys.meta_path.insert(0, HideModuleFinder(hidden))

hide_pickle = False # change me for testing different behavior

if hide_pickle:
    install({"_pickle"})

import pickle # must be done after meta path hook

class Meta(type):
    def __getattribute__(self, item):
        print("__getattribute__ called with", item)
        return type.__getattribute__(self, item)

MyClass = Meta("MyClass", (), {})

obj = MyClass()
print("PICKLING")
obj_str = pickle.dumps(obj)

print("UNPICKLING")
new_obj = pickle.loads(obj_str)

Output with hide_pickle = True

PICKLING
__getattribute__ called with __reduce__
__getattribute__ called with __dict__
__getattribute__ called with __slots__
__getattribute__ called with __new__
__getattribute__ called with __qualname__
__getattribute__ called with __module__
UNPICKLING
__getattribute__ called with __new__

Output with hide_pickle = False

PICKLING
__getattribute__ called with __reduce__
__getattribute__ called with __dict__
__getattribute__ called with __slots__
__getattribute__ called with __qualname__
__getattribute__ called with __module__
UNPICKLING