Implement PEP 696 (defaults for type params)

[zsol]

https://peps.python.org/pep-0696/#grammar-changes will ship with Python 3.13, so I'd like to get this into LibCST by the time the release candidates for 3.13 are out. Assign to yourself or comment if you start working on it, please.

Node changes

Seems like the most straightforward way is to extend statement.TypeParam to own the equals sign (which owns its surrounding whitespace), and the default value itself.

• The python version needs these extra fields. You can take inspiration from expression.Param.
• Also add validation to make sure the equals sign and the default value can only appear together (expression.Param also has an example for this)
• We should also add validation in statement.TypeParameters to ensure that a TypeParam without a default value never follows one with.
• You can exercise the Python code with tests here.
• The rust version needs extending in a similar way.
  • This part explains various naming conventions and helpers available in Rust for defining nodes.
  • There's no validation for the Rust nodes, and also no MaybeSentinel - we use Option<_> to signal optional values.
  • I think you won't need to store any actual tokens, so the Rust changes should look equivalent to Python.

Grammar changes

I recommend reading this readme to get acquainted with the rust bits at a high level.

• The grammar (starts here) needs to be changed following the PEP as closely as possible. This is the relevant part. You can also take a peek at the cpython grammar changes (note that the grammar in the PEP seems different than actual CPython grammar - the latter doesn't mention constraints - no idea what's the cause of this, but CPython is the source of truth).
  • The only significant syntactic difference between LibCST and CPython grammar is that LibCST grammar inlines the actions to be taken as a result of a successful parse between {} braces in the rule. I try to keep these short, preferably a single function call that's defined later in the file.
  • I think this change won't require any tricks, but of course, feel free to reach out if you get stuck.

Roundtrip tests

• this file hosts a bunch of type parameter related test cases that aim to exercise edge cases of the syntax.
  • The entire file is parsed into a Rust CST, then codegenned back to source code, to be compared with the original source byte-by-byte. Then the same thing is done using the Python CST (this also exercises validation).
  • Feel free to go crazy with the syntax for new test cases here. These are great ways of catching subtle bugs like whitespace being owned by two nodes simultaneously (thus duplicating it during codegen - make sure there's a case where every possible whitespace is different from the default), inconsistent default values between Python/Rust, too strict validation rules, etc.
• Run them with cargo test (the rust roundtrip) or pytest -k roundtrip (you might have to --ignore-glob '*fuzz*').

[JelleZijlstra]

• note that the grammar in the PEP seems different than actual CPython grammar - the latter doesn't mention constraints - no idea what's the cause of this, but CPython is the source of truth

The PEP grammar might have been derived from an early version of PEP 695. When I implemented PEP 695 in Python I put the bounds and constraint in the same place in the grammar; they're distinguished later on in the parsing process (if it's a Tuple node, use constraints; otherwise use bound).

[zsol]

3.13.0 beta 1 is expected to come out next week. @thereversiblewheel need help? :)

[zsol]

I'll try and get a new libcst release while on the pycon sprints now that #1141 has been merged