Importing ssl After Reinitializing Crashes

[ericsnowcurrently]

Crash report

What happened?

Using 3.13b4 and main (5592399):

$ Programs/_testembed test_repeated_init_exec 'import ssl'
--- Loop #1 ---
--- Loop #2 ---
Segmentation fault (core dumped)

Using 3.12:

$ Programs/_testembed test_repeated_init_exec 'import ssl'
--- Loop #1 ---
--- Loop #2 ---
_testembed: Python/getargs.c:2052: parser_init: Assertion `parser->kwtuple != NULL' failed.
Aborted (core dumped)

The 3.12 output implies a problem related to argument clinic and the kwarg names tuple that gets cached.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Output from running 'python -VV' on the command line:

No response

Linked PRs

• gh-122334: Fix crash when importing ssl after re-initialization #122481
• [3.12] gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) #122495
• [3.13] gh-122334: Fix crash when importing ssl after re-initialization (GH-122481) #122614
• gh-122334: Fix test_embed failure when missing _ssl module #122630
• [3.13] gh-122334: Fix test_embed failure when missing _ssl module (GH-122630) #122647
• [3.12] gh-122334: Fix test_embed failure when missing _ssl module (GH-122630) #122648

[neonene]

The following command crashes as well:

_testembed test_repeated_init_exec "import _ssl; _ssl.txt2obj(txt='1.3')"

Related clinic code (actually unchanged since Python 3.5):

static PyObject *
_ssl_txt2obj(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
    ...
    static const char * const _keywords[] = {"txt", "name", NULL};
    static _PyArg_Parser _parser = {
        .keywords = _keywords,
        .fname = "txt2obj",
        .kwtuple = KWTUPLE,  // KWTUPLE: NULL
    };
    ...
    args = _PyArg_UnpackKeywords(..., &_parser, ...);  // segfault

Python/getargs.c:

PyObject * const *
_PyArg_UnpackKeywords(..., struct _PyArg_Parser *parser, ...)
{
    PyObject *kwtuple;
    ...
    if (parser_init(parser) < 0) {  // set `parser->kwtuple` only once
        return NULL;
    }
    kwtuple = parser->kwtuple;      // get NULL after restarting interp
    ...
    maxargs = posonly + (int)PyTuple_GET_SIZE(kwtuple);  // segfault

Currently, _PyArg_Fini() clears parser->kwtuple only:

void
_PyArg_Fini(void)
{
    struct _PyArg_Parser *tmp, *s = _PyRuntime.getargs.static_parsers;
    while (s) {
        tmp = s->next;
        s->next = NULL;
        parser_clear(s);
        s = tmp;
    }
    _PyRuntime.getargs.static_parsers = NULL;
}

// 3.13+
static void
parser_clear(struct _PyArg_Parser *parser)
{
    if (parser->is_kwtuple_owned) {
        Py_CLEAR(parser->kwtuple);
    }
}

// 3.12
static void
parser_clear(struct _PyArg_Parser *parser)
{
    if (parser->initialized == 1) {
        Py_CLEAR(parser->kwtuple);
    }
}

IIUC, clearing parser->once.v or parser->initialized there will allow parser_init() to run again, which would work around the issue.

[ericsnowcurrently]

Good sleuthing!

[neonene]

Some buildbots fail the test_embed without building _ssl module.
https://buildbot.python.org/#/builders/1366/builds/1792

[neonene]

I think there is nothing left to do here.

[encukou]

Indeed. Thank you for the fix!

[diegorusso]

This is the same issue of #121087 Thanks for fixing it!

I tested the code on that issue and it is passing. In that case the problem was that the once.v wasn't reset to 0. Is it worth adding that test in the test_embed?