Use Py_T_OBJECT_EX instead of _Py_T_OBJECT

[serhiy-storchaka]

_Py_T_OBJECT is considered legacy PyMemberDef type. The difference between _Py_T_OBJECT and Py_T_OBJECT_EX is that the former returns None if read NULL, while the latter raises AttrributeError. _Py_T_OBJECT manifests itself in two effects:

1. The default value of the attribute is None, even if it was not initialized in the constructor. It is a desirable behavior in some cases.
2. After deleting an attribute its value is still None. You cannot truly delete it.

A Py_T_OBJECT_EX member behaves like a normal attribute in Python object, while a _Py_T_OBJECT member behaves like in the case when the corresponding class attribute was set to None:

class A:
    attr = None

x = A()
assert x.attr is None
x.attr = 5
assert x.attr == 5
del x.attr
assert x.attr is None

What if replace _Py_T_OBJECT with Py_T_OBJECT_EX? It turns out that you can replace it in 105 sites but 31 sites should keep _Py_T_OBJECT to make existing tests pass. This is not a very reliable result because the tests may not cover all cases. On the other hand, some tests are too picky and check the attributes of a newly created uninitialized object, even if they are normally initialized.

In any case, we can take these results and replace _Py_T_OBJECT with Py_T_OBJECT_EX on case by case basis.

@vstinner @encukou

Linked PRs

• gh-107253: Use Py_T_OBJECT_EX instead of _Py_T_OBJECT #107254

[vstinner]

Can we agree first if we rename the constant to Py_T_OBJECT? :-)

[serhiy-storchaka]

About 31 cases still need _Py_T_OBJECT. It can be fixed if add explicit code in constructors, maybe, but it increases the size of the code and spent the CPU time. On other hand, it may be much more than 31 cases if we look closer, and this behavior is not inherently wrong to get rid of it.

Note also that Py_T_STRING behaves somewhat close to _Py_T_OBJECT. Shall we add Py_T_STRING_EX?

[vstinner]

Note also that Py_T_STRING behaves somewhat close to _Py_T_OBJECT. Shall we add Py_T_STRING_EX?

If we rename constants before Python 3.12 final, I would prefer to have more explicit names:

• _Py_T_OBJECT_NONE which replaces T_OBJECT
• Py_T_OBJECT: new constant
• _Py_T_STRING_NONE which replaces T_STRING
• Py_T_STRING: new constant

It's more explicit that it returns None if the C value is NULL, and avoid the ugly "_EX" suffix for a new API :-)

[serhiy-storchaka]

In this case we do not need the underscore prefix. Just Py_T_OBJECT_NONE.

[serhiy-storchaka]

It can even be Py_T_OBJECT_NONE == Py_T_OBJECT|Py_T_NONE. But this new Py_T_NONE is different from old _Py_T_NONE.

[vstinner]

In this case we do not need the underscore prefix. Just Py_T_OBJECT_NONE.

I suppose @encukou chose to use an underscore to deprecate this old API, to discourage its usage.

It can even be Py_T_OBJECT_NONE == Py_T_OBJECT|Py_T_NONE.

I don't think that it's worth it if there are only two ..._NONE constants.

[encukou]

FWIW, I don't think it's worth it to change the behaviour. Does it help users in any way?

We found a better way to do things, but the old way is OK.

[encukou]

Oh, and: the public (but soft-deprecated) API for this is T_OBJECT, with #include "structmember.h". That didn't change.