Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-108494: Argument Clinic: fix support of Limited C API #108536

Merged
merged 7 commits into from
Aug 28, 2023

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Aug 27, 2023

I tried to enable the use of the limited C API in Argument Clinic for all code. With this PR it generates working code, and all tests are passed, except the low-level vectorcall test in test_call and few tests in test_clinic (as expected). _struct.c and winreg.c use converters incompatible with the limited C API. I temporary blacklisted these files. I only tested on Linux, so there may be errors in Windows or macOS specific modules.

@serhiy-storchaka
Copy link
Member Author

Yet one issue: the TokenizerIter constructor is incompatible with PyArg_ParseTupleAndKeywords. I temporary worked it around by adding the default value for keyword-only extra_tokens parameter. Similar incompatibility was also a cause of failing two test_clinic tests. Resolving #74366 should fix this.

@@ -65,7 +65,8 @@ def test_varargs3(self):
self.assertRaisesRegex(TypeError, msg, int.from_bytes, b'a', 'little', False)

def test_varargs1min(self):
msg = r"get expected at least 1 argument, got 0"
msg = (r"get\(\) takes at least 1 argument \(0 given\)|"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyArg_ParseTuple() produces slightly different error message.

Modules/_sqlite/module.c Show resolved Hide resolved
Modules/_tracemalloc.c Show resolved Hide resolved
@vstinner
Copy link
Member

I'm very excited by this change! I will try to review it soon.

Would it be possible to write unit tests in _testclinic_limited and test_clinic, for unusual cases like deprecated parameters and (if possible) defining class?

In my other PR, I wrote tests for basic positional or keywords parameters.

@serhiy-storchaka
Copy link
Member Author

As for tests, I think that we should run all (or almost all) clinic tests in two modes. Instead of adding new tests for the limited API or duplicating test code we should generate and build different modules from the same source with different options. But this is a different PR.

@@ -37,8 +37,7 @@ my_int_func(PyObject *module, PyObject *arg_)
int arg;
int _return_value;

arg = PyLong_AsInt(arg_);
if (arg == -1 && PyErr_Occurred()) {
if (!PyArg_Parse(arg_, "i:my_int_func", &arg)) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like regression, but actually not all inlined parsing code is compatible with the limited C API. In following PR I will implement inlining correctly, only for converters compatible with the limited C API. It will be large but mostly boring change (adding new parameter and simple checks in tens of functions), so I did not include it here.

@serhiy-storchaka serhiy-storchaka requested a review from a team as a code owner August 28, 2023 11:11
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Copy link
Contributor

@erlend-aasland erlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may have missed something, but this looks good to me. Thanks, Serhiy!

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No complaints from me (though I'm not an expert on the limited C API)

@erlend-aasland erlend-aasland changed the title gh-108494: Argument Clinic: fix support of limited C API gh-108494: Argument Clinic: fix support of Limited C API Aug 28, 2023
@serhiy-storchaka serhiy-storchaka merged commit bc5356b into python:main Aug 28, 2023
23 of 24 checks passed
@serhiy-storchaka serhiy-storchaka deleted the clinic-limited-capi branch August 28, 2023 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants