PEP 696: Proposed changes (python#3638)

peps/pep-0696.rst

@@ -30,7 +30,7 @@ and can be found in its
 Motivation
 ----------
 
-.. code-block:: py
+::
 
    T = TypeVar("T", default=int)  # This means that if no type is specified T = int
 
@@ -43,9 +43,7 @@ Motivation
 
 One place this `regularly comes
 up <https://github.com/python/typing/issues/975>`__ is ``Generator``. I
-propose changing the *stub definition* to something like:
-
-.. code-block:: py
+propose changing the *stub definition* to something like::
 
    YieldT = TypeVar("YieldT")
    SendT = TypeVar("SendT", default=None)
@@ -57,7 +55,7 @@ propose changing the *stub definition* to something like:
 
 This is also useful for a ``Generic`` that is commonly over one type.
 
-.. code-block:: py
+::
 
    class Bot: ...
 
@@ -76,14 +74,15 @@ also helps non-typing users who rely on auto-complete to speed up their
 development.
 
 This design pattern is common in projects like:
- - `discord.py <https://github.com/Rapptz/discord.py>`__ — where the
-   example above was taken from.
- - `NumPy <https://github.com/numpy/numpy>`__ — the default for types
-   like ``ndarray``'s ``dtype`` would be ``float64``. Currently it's
-   ``Unknown`` or ``Any``.
- - `TensorFlow <https://github.com/tensorflow/tensorflow>`__ — this
-   could be used for Tensor similarly to ``numpy.ndarray`` and would be
-   useful to simplify the definition of ``Layer``.
+
+- `discord.py <https://github.com/Rapptz/discord.py>`__ — where the
+  example above was taken from.
+- `NumPy <https://github.com/numpy/numpy>`__ — the default for types
+  like ``ndarray``'s ``dtype`` would be ``float64``. Currently it's
+  ``Unknown`` or ``Any``.
+- `TensorFlow <https://github.com/tensorflow/tensorflow>`__ — this
+  could be used for Tensor similarly to ``numpy.ndarray`` and would be
+  useful to simplify the definition of ``Layer``.
 
 
 Specification
@@ -96,9 +95,9 @@ The order for defaults should follow the standard function parameter
 rules, so a type parameter with no ``default`` cannot follow one with
 a ``default`` value. Doing so should ideally raise a ``TypeError`` in
 ``typing._GenericAlias``/``types.GenericAlias``, and a type checker
-should flag this an error.
+should flag this as an error.
 
-.. code-block:: py
+::
 
    DefaultStrT = TypeVar("DefaultStrT", default=str)
    DefaultIntT = TypeVar("DefaultIntT", default=int)
@@ -137,9 +136,14 @@ should flag this an error.
        AllTheDefaults[int, complex, str, int, bool]
    )  # All valid
 
-This cannot be enforced at runtime for functions, for now, but in the
-future, this might be possible (see `Interaction with PEP
-695 <#interaction-with-pep-695>`__).
+With the new Python 3.12 syntax for generics (introduced by :pep:`695`), this can
+be enforced at compile time::
+
+   type Alias[DefaultT = int, T] = tuple[DefaultT, T]  # SyntaxError: non-default TypeVars cannot follow ones with defaults
+
+   def generic_func[DefaultT = int, T](x: DefaultT, y: T) -> None: ...  # SyntaxError: non-default TypeVars cannot follow ones with defaults
+
+   class GenericClass[DefaultT = int, T]: ...  # SyntaxError: non-default TypeVars cannot follow ones with defaults
 
 ``ParamSpec`` Defaults
 ''''''''''''''''''''''
@@ -148,7 +152,7 @@ future, this might be possible (see `Interaction with PEP
 ``TypeVar`` \ s but use a ``list`` of types or an ellipsis
 literal "``...``" or another in-scope ``ParamSpec`` (see `Scoping Rules`_).
 
-.. code-block:: py
+::
 
    DefaultP = ParamSpec("DefaultP", default=[str, int])
 
@@ -165,7 +169,7 @@ literal "``...``" or another in-scope ``ParamSpec`` (see `Scoping Rules`_).
 ``TypeVar`` \ s but use an unpacked tuple of types instead of a single type
 or another in-scope ``TypeVarTuple`` (see `Scoping Rules`_).
 
-.. code-block:: py
+::
 
    DefaultTs = TypeVarTuple("DefaultTs", default=Unpack[tuple[str, int]])
 
@@ -190,7 +194,7 @@ a ``TypeVar``, etc.).
 where the ``start`` parameter should default to ``int``, ``stop``
 default to the type of ``start`` and step default to ``int | None``.
 
-.. code-block:: py
+::
 
    StartT = TypeVar("StartT", default=int)
    StopT = TypeVar("StopT", default=StartT)
@@ -220,19 +224,18 @@ Where ``T1`` is the default for ``T2`` the following rules apply.
 - `Scoping Rules`_ does not allow usage of type parameters
   from outer scopes.
 - Multiple ``TypeVarTuple``\s cannot appear in the type
-  parameter list for a single class, as specified in
+  parameter list for a single object, as specified in
   :pep:`646#multiple-type-variable-tuples-not-allowed`.
-- type parameter defaults in functions are not supported.
 
 These reasons leave no current valid location where a
-``TypeVarTuple`` could have a default.
+``TypeVarTuple`` could be used as the default of another ``TypeVarTuple``.
 
 Scoping Rules
 ~~~~~~~~~~~~~
 
 ``T1`` must be used before ``T2`` in the parameter list of the generic.
 
-.. code-block:: py
+::
 
    DefaultT = TypeVar("DefaultT", default=T)
 
@@ -252,7 +255,7 @@ Bound Rules
 
 ``T2``'s bound must be a subtype of ``T1``'s bound.
 
-.. code-block:: py
+::
 
    T = TypeVar("T", bound=float)
    TypeVar("Ok", default=T, bound=int)        # Valid
@@ -264,7 +267,7 @@ Constraint Rules
 
 The constraints of ``T2`` must be a superset of the constraints of ``T1``.
 
-.. code-block:: py
+::
 
    T1 = TypeVar("T1", bound=int)
    TypeVar("Invalid", float, str, default=T1)         # Invalid: upper bound int is incompatible with constraints float or str
@@ -281,7 +284,7 @@ Type parameters are valid as parameters to generics inside of a
 ``default`` when the first parameter is in scope as determined by the
 `previous section <scoping rules_>`_.
 
-.. code-block:: py
+::
 
    T = TypeVar("T")
    ListDefaultT = TypeVar("ListDefaultT", default=list[T])
@@ -312,7 +315,7 @@ that hasn't been overridden it should be treated like it was
 substituted into the ``TypeAlias``. However, it can be specialised
 further down the line.
 
-.. code-block:: py
+::
 
    class SomethingWithNoDefaults(Generic[T, T2]): ...
 
@@ -328,7 +331,7 @@ Subclassing
 Subclasses of ``Generic``\ s with type parameters that have defaults
 behave similarly to ``Generic`` ``TypeAlias``\ es.
 
-.. code-block:: py
+::
 
    class SubclassMe(Generic[T, DefaultStrT]):
        x: DefaultStrT
@@ -356,7 +359,7 @@ If both ``bound`` and ``default`` are passed ``default`` must be a
 subtype of ``bound``. Otherwise the type checker should generate an
 error.
 
-.. code-block:: py
+::
 
    TypeVar("Ok", bound=float, default=int)     # Valid
    TypeVar("Invalid", bound=str, default=int)  # Invalid: the bound and default are incompatible
@@ -368,7 +371,7 @@ For constrained ``TypeVar``\ s, the default needs to be one of the
 constraints. A type checker should generate an error even if it is a
 subtype of one of the constraints.
 
-.. code-block:: py
+::
 
    TypeVar("Ok", float, str, default=float)     # Valid
    TypeVar("Invalid", float, str, default=int)  # Invalid: expected one of float or str got int
@@ -378,17 +381,53 @@ subtype of one of the constraints.
 Function Defaults
 '''''''''''''''''
 
-Type parameters currently are not supported in the signatures of
-functions as ensuring the ``default`` is returned in every code path
-where the type parameter can go unsolved is too hard to implement.
+We leave the semantics of type parameter defaults in generic functions
+unspecified, as ensuring the ``default`` is returned in every code path
+where the type parameter can go unsolved may be too hard to implement.
+Type checkers are free to either disallow this case or experiment with
+implementing support.
+
+Defaults following ``TypeVarTuple``
+'''''''''''''''''''''''''''''''''''
+
+A ``TypeVar`` that immediately follows a ``TypeVarTuple`` is not allowed
+to have a default, because it would be ambiguous whether a type argument
+should be bound to the ``TypeVarTuple`` or the defaulted ``TypeVar``.
+
+::
+
+   Ts = TypeVarTuple("Ts")
+   T = TypeVar("T", default=bool)
+
+   class Foo(Generic[Ts, T]): ...  # Type checker error
+
+   # Could be reasonably interpreted as either Ts = (int, str, float), T = bool
+   # or Ts = (int, str), T = float
+   Foo[int, str, float]
+
+With the Python 3.12 built-in generic syntax, this case should raise a SyntaxError.
+
+However, it is allowed to have a ``ParamSpec`` with a default following a
+``TypeVarTuple`` with a default, as there can be no ambiguity between a type argument
+for the ``ParamSpec`` and one for the ``TypeVarTuple``.
+
+::
+
+   Ts = TypeVarTuple("Ts")
+   P = ParamSpec("P", default=[float, bool])
+
+   class Foo(Generic[Ts, P]): ...  # Valid
+
+   Foo[int, str]  # Ts = (int, str), P = [float, bool]
+   Foo[int, str, [bytes]]  # Ts = (int, str), P = [bytes]
 
 Binding rules
 -------------
 
 Type parameter defaults should be bound by attribute access
 (including call and subscript).
 
-.. code-block:: python
+::
 
    class Foo[T = int]:
        def meth(self) -> Self:
@@ -414,6 +453,9 @@ The following changes would be required to both ``GenericAlias``\ es:
 -  ideally, logic to determine if subscription (like
    ``Generic[T, DefaultT]``) would be valid.
 
+The grammar for type parameter lists would need to be updated to
+allow defaults; see below.
+
 A reference implementation of the runtime changes can be found at
 https://github.com/Gobot1234/cpython/tree/pep-696
 
@@ -423,14 +465,14 @@ https://github.com/Gobot1234/mypy/tree/TypeVar-defaults
 Pyright currently supports this functionality.
 
 
-Interaction with PEP 695
-------------------------
+Grammar changes
+'''''''''''''''
 
-The syntax proposed in :pep:`695` will be extended to introduce a way
+The syntax added in :pep:`695` will be extended to introduce a way
 to specify defaults for type parameters using the "=" operator inside
 of the square brackets like so:
 
-.. code-block:: py
+::
 
    # TypeVars
    class Foo[T = str]: ...
@@ -454,10 +496,7 @@ avoid the unnecessary usage of quotes around them.
 This functionality was included in the initial draft of :pep:`695` but
 was removed due to scope creep.
 
-Grammar Changes
-'''''''''''''''
-
-::
+The following changes would be made to the grammar::
 
     type_param:
         | a=NAME b=[type_param_bound] d=[type_param_default]
@@ -469,17 +508,17 @@ Grammar Changes
         | '=' e=expression
         | '=' e=starred_expression
 
-This would mean that type parameters with defaults proceeding those
-with non-defaults can be checked at compile time.
-
+The compiler would enforce that type parameters without defaults cannot
+follow type parameters with defaults and that ``TypeVar``\ s with defaults
+cannot immediately follow ``TypeVarTuple``\ s.
 
 Rejected Alternatives
 ---------------------
 
 Allowing the Type Parameters Defaults to Be Passed to ``type.__new__``'s ``**kwargs``
 '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 
-.. code-block:: py
+::
 
    T = TypeVar("T")
 
@@ -494,9 +533,7 @@ backwards compatible as ``T`` might already be passed to a
 metaclass/superclass or support classes that don't subclass ``Generic``
 at runtime.
 
-Ideally, if :pep:`637` wasn't rejected, the following would be acceptable:
-
-.. code-block:: py
+Ideally, if :pep:`637` wasn't rejected, the following would be acceptable::
 
    T = TypeVar("T")
 
@@ -507,7 +544,7 @@ Ideally, if :pep:`637` wasn't rejected, the following would be acceptable:
 Allowing Non-defaults to Follow Defaults
 ''''''''''''''''''''''''''''''''''''''''
 
-.. code-block:: py
+::
 
    YieldT = TypeVar("YieldT", default=Any)
    SendT = TypeVar("SendT", default=Any)
@@ -525,7 +562,7 @@ above two forms were valid. Changing the argument order now would also
 break a lot of codebases. This is also solvable in most cases using a
 ``TypeAlias``.
 
-.. code-block:: py
+::
 
    Coro: TypeAlias = Coroutine[Any, Any, T]
    Coro[int] == Coroutine[Any, Any, int]
@@ -538,7 +575,7 @@ to ``bound`` if no value was passed for ``default``. This while
 convenient, could have a type parameter with no default follow a
 type parameter with a default. Consider:
 
-.. code-block:: py
+::
 
    T = TypeVar("T", bound=int)  # default is implicitly int
    U = TypeVar("U")