bpo-43931: Export Python version as API data

[P403n1x87]

When Python is embedded in other applications, it is not easy to determine which version of Python is being used. This change exposes the Python version as part of the API data. Tools like Austin (https://github.com/P403n1x87/austin) can benefit from this data when targeting applications like uWSGI, as the Python version can then be inferred systematically by looking at the exported symbols rather than relying on unreliable pattern matching or other hacks (like remote code execution etc...).

https://bugs.python.org/issue43931

Automerge-Triggered-By: GH:pablogsal

[markshannon]

Wouldn't it make more sense to export the hex version number?
The format is fixed and won't need parsing.

[P403n1x87]

Wouldn't it make more sense to export the hex version number?
The format is fixed and won't need parsing.

Yes it makes a lot of sense. Indeed I have thought about exporting something like

(PY_MAJOR << 16) | (PY_MINOR << 8) | PY_PATCH

as an alternative, which would certainly be more efficient to read and parse, but then I realised that I could do

(gdb) print Py_Version
$1 = 0x2d788d "3.10.0a7+"

which is neat 🙂. Personally, as far as my use-cases go, I'm fine either way, so if you prefer that I make the change that's totally cool.

[pablogsal]

I concur with @markshannon, I think is better to export the hex string, because the format specifier can change in the future

[pablogsal]

Can you add a small test in the testcapi module to check the final format?

[bedevere-bot]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

And if you don't make the requested changes, you will be put in the comfy chair!

[pablogsal]

This needs also an entry in the What's new for 3.11

[P403n1x87]

I have made the requested changes; please review again

[bedevere-bot]

Thanks for making the requested changes!

@pablogsal: please review the changes made to this pull request.

[github-actions]

This PR is stale because it has been open for 30 days with no activity.

[vstinner]

I love this idea, but would it be possible to document this variable in Doc/c-api/ ?

[encukou]

But they don't always have the same value – one is retrieved at runtime, the other is compiled into an extension.

Suggested change

	This version is also available via the symbol :data:`Py_Version`.
	.. c:var:: const unsigned long Py_Version
	The Python version number encoded as the single constant integer, same value
	as the c:macro:`PY_VERSION_HEX` macro.
	.. versionadded:: 3.11
	.. c:var:: const unsigned long Py_Version
	The Python version number encoded as a single constant integer
	in the same format as the c:macro:`PY_VERSION_HEX` macro.
	This contains the Python version used at run time.
	.. versionadded:: 3.11

[vstinner]

"same format as the PY_VERSION_HEX macro" sounds like a good description. So we only have to document the format once, in PY_VERSION_HEX.

[P403n1x87]

Hmm, I'm not sure I understand this. The PY_VERSION_HEX is a C macro, which is then not available at runtime, and Py_Version is binary constant data, that is assigned the value of PY_VERSION_HEX at compile time. Loosely speaking, one can use sys.hexversion to access PY_VERSION_HEX from Python, and I can see why sys.hexversion might not be the same as Py_Version. But the documentation here is stating that PY_VERSION_HEX and Py_Version have the same value at the C API level, which should always be true?

[vstinner]

I guess that Petr is talking about a C extension built with Python 3.8 and run with Python 3.11.

Py_Version and sys.hexversion is the version of the running Python process. PY_VERSION_HEX can be different in a C extension built with an older Python version.

I understood that the documentation can be misleading in such corner case.

[encukou]

Yes.
This is what the note at the top of the document is about:

CPython exposes its version number in the following macros. Note that these correspond to the version code is built with, not necessarily the version used at run time.

Feel free to clarify it if you find better wording :)

[P403n1x87]

Ah, I see now, thanks for clarifying!

[P403n1x87]

@pablogsal @encukou may I ask for another look at this PR, please? I would love to see this get in 3.11 if possible 🙏

[pablogsal]

@P403n1x87 Can you solve the merge conflicts? After that I can land it

[P403n1x87]

@pablogsal thanks, done!

[P403n1x87]

Ah looks like I need to regenerate some sources 🙁

[pablogsal]

LGTM

Thanks, @P403n1x87 for your contribution and patience. It's quite busy these days in core development so apologies for taking so long to review

[miss-islington]

@P403n1x87: Status check is done, and it's a success ❌ .

[miss-islington]

@P403n1x87: Status check is done, and it's a success ✅ .

[P403n1x87]

Thanks, @P403n1x87 for your contribution and patience. It's quite busy these days in core development so apologies for taking so long to review

🙌 Thank you, @pablogsal, for regenerating the sources and no worries, I understand this is quite an active repo with a lot going on all the time! 🙂

[vstinner]

Thank you @P403n1x87 for the documentation, IMO it was worth it to spend time on describing exactly what the variable contains and how it is intended to be used.