Stable ABI doesn't work with PySide (Shiboken) on Python 3.12

[vstinner]

Bug report

PEP 683 "Immortal Objects, Using a Fixed Refcount" announces that the stable ABI is not affected: https://peps.python.org/pep-0683/#stable-abi

Well, in practice, it is affected. Shiboken used by PySide runs the following code at startup:

    PyObject *type = (PyObject*)&PyType_Type;
    PyObject *bases = PyObject_GetAttrString(type, "__bases__");
    ...
    Py_DECREF(bases);

PyObject_GetAttrString() leaves PyType_Type.tp_bases.ob_refcnt unchanged since it's an immortal tuple, and the code runs on Python 3.12 which is aware of immortal object.

Py_DECREF(bases) decrements PyType_Type.tp_bases.ob_refcnt since the code is built with Python 3.9 and Py_DECREF() is inlined.

The problem occurs at Python exit, when static types are cleared by _PyStaticType_Dealloc(). The following code is executed:

static inline void
clear_tp_bases(PyTypeObject *self)
{
    if (self->tp_flags & _Py_TPFLAGS_STATIC_BUILTIN) {
        ...
        assert(_Py_IsImmortal(self->tp_bases));
        _Py_ClearImmortal(self->tp_bases);
        ...
    }
    Py_CLEAR(self->tp_bases);
}

with:

/* _Py_ClearImmortal() should only be used during runtime finalization. */
static inline void _Py_ClearImmortal(PyObject *op)
{
    if (op) {
        assert(op->ob_refcnt == _Py_IMMORTAL_REFCNT);
        op->ob_refcnt = 1;
        Py_DECREF(op);
    }
}
#define _Py_ClearImmortal(op) \
    do { \
        _Py_ClearImmortal(_PyObject_CAST(op)); \
        op = NULL; \
    } while (0)

The condition assert(op->ob_refcnt == _Py_IMMORTAL_REFCNT); is false.

IMO _Py_ClearImmortal() should use _Py_IsImmortal() in the assertion instead, as clear_tp_bases() does.

Fedora downstream issue: https://bugzilla.redhat.com/show_bug.cgi?id=2279088

Linked PRs

• [3.12] gh-118997: Fix _Py_ClearImmortal() assertion #119001

[hroncok]

Does this not happen on 3.13+ as well?

[vstinner]

Does this not happen on 3.13+ as well?

3.13 uses _Py_IsImmortal() and so is not affected.

[ericsnowcurrently]

Thanks for fixing this, @vstinner.