gh-101819: Prepare to modernize the _io extension

[vstinner]

• Add references to static types to _PyIO_State:

  • PyBufferedIOBase_Type
  • PyBytesIOBuffer_Type
  • PyIncrementalNewlineDecoder_Type
  • PyRawIOBase_Type
  • PyTextIOBase_Type

• Add the defining class to methods:

  • _io.BytesIO.getbuffer()
  • _io.FileIO.close()

• Add get_io_state_by_cls() function.

• Add state parameter to _textiowrapper_decode()

• io_TextIOWrapper___init_() now sets self->state before calling _textiowrapper_set_decoder().

• Issue: Isolate the _io extension module #101819

[vstinner]

This PR is based on @erlend-aasland's PR #101948. Changes:

• Don't convert static types to heap types: it will be done on a separated PR
• _io_StringIO___init___impl() only calls find_io_state_by_def() once
• _io__WindowsConsoleIO___init___impl() doesn't declare a variable which is only used by an assertion, to avoid the risk of getting a compiler warning. If the line is too long, I can make the declaration conditional on the NDEBUG macro.
• _io_FileIO_close_impl() and _io__WindowsConsoleIO_close_impl(): move variable declaration (minor coding style change).
• Don't add PyIOBase_Type since such change would have no point for now.

I copied the clinic_state() macros.

Add the defining class to methods: (...)

I was not sure that it works on static types... but it seems like it does :-) I didn't get any crash.

[vstinner]

@corona10 @erlend-aasland: Would you mind to review it?

[corona10]

sure

[vstinner]

Oh, the Windows change didn't go well:

0:20:02 load avg: 8.57 [317/463/2] test_winconsoleio crashed (Exit code 3) -- (...)
Assertion failed: mod != NULL, file D:\a\cpython\cpython\Modules\_io\_iomodule.h, line 192
Fatal Python error: Aborted

Current thread 0x000004a0 (most recent call first):
  File "D:\a\cpython\cpython\Lib\test\test_winconsoleio.py", line 104 in test_conin_conout_names
(...)

[vstinner]

The failed assertion is:

assert(PyObject_TypeCheck(self, find_io_state_by_def(Py_TYPE(self))->PyWindowsConsoleIO_Type));

in _io__WindowsConsoleIO___init___impl(). The assertion fails because PyType_GetModuleByDef() returns NULL.

[vstinner]

On static types, PyType_GetModuleByDef() always returns NULL, since the function reads PyHeapTypeObject.ht_module which only exists in heap types:

        if(!_PyType_HasFeature((PyTypeObject *)super, Py_TPFLAGS_HEAPTYPE)) {
            // Static types in the MRO need to be skipped
            continue;
        }

[vstinner]

Using APIs to get the module state from a type works on heap types like _io.FileIO, but failed on _io.WindowsConsoleIO which remains a static type with my PR.

I fixed my PR by reverting changes related to WindowsConsoleIO: this type should be modified in a following PR.

[vstinner]

I reverted changes related to PyWindowsConsoleIO_Type and _io._WindowsConsoleIO.close().

[erlend-aasland]

I prefer doing Windows only changes in separate PRs. Everytime I touch Windows stuff, something weird happen :)

[vstinner]

Aha, now the CI pass. I would feel more comfortable with a review. @erlend-aasland: you can also review if you want, to check if i picked the right parts or tour code 😁

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @corona10 for commit dfc1e9e 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

[corona10]

lgtm, let's wait refleak bot.

[vstinner]

I was not sure that it works on static types... but it seems like it does :-) I didn't get any crash.

I missed something: Erlend already converted multiple _io types to heap types: see commit c00faf7. Types used in this PR are heap types, that's why it just works!

[vstinner]

let's wait refleak bot

Thanks for running these jobs. I ran refleaks locally on test_io, but i don't trust myself 🤣 So having reliable buildbot checks sound safer.

I will merge my PR once the Refleaks jobs pass, unless someome wants me to wait for their second review.

[erlend-aasland]

No refleaks; good bot 🤖 ✅