Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PEP 695] Allow Self return types with contravariance #17786

Merged
merged 3 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions mypy/subtypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -2024,6 +2024,16 @@ def infer_variance(info: TypeInfo, i: int) -> bool:

typ = find_member(member, self_type, self_type)
if typ:
# It's okay for a method in a generic class with a contravariant type
# variable to return a generic instance of the class, if it doesn't involve
# variance (i.e. values of type variables are propagated). Our normal rules
# would disallow this. Replace such return types with 'Any' to allow this.
#
# This could probably be more lenient (e.g. allow self type be nested, don't
# require all type arguments to be identical to self_type), but this will
# hopefully cover the vast majority of such cases, including Self.
typ = erase_return_self_types(typ, self_type)

typ2 = expand_type(typ, {tvar.id: object_type})
if not is_subtype(typ, typ2):
co = False
Expand Down Expand Up @@ -2066,3 +2076,20 @@ def infer_class_variances(info: TypeInfo) -> bool:
if not infer_variance(info, i):
success = False
return success


def erase_return_self_types(typ: Type, self_type: Instance) -> Type:
"""If a typ is function-like and returns self_type, replace return type with Any."""
proper_type = get_proper_type(typ)
if isinstance(proper_type, CallableType):
ret = get_proper_type(proper_type.ret_type)
if isinstance(ret, Instance) and ret == self_type:
return proper_type.copy_modified(ret_type=AnyType(TypeOfAny.implementation_artifact))
elif isinstance(proper_type, Overloaded):
return Overloaded(
[
cast(CallableType, erase_return_self_types(it, self_type))
for it in proper_type.items
]
)
return typ
58 changes: 57 additions & 1 deletion test-data/unit/check-python312.test
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ b: Invariant[int]
if int():
a = b # E: Incompatible types in assignment (expression has type "Invariant[int]", variable has type "Invariant[object]")
if int():
b = a # E: Incompatible types in assignment (expression has type "Invariant[object]", variable has type "Invariant[int]")
b = a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is not right. It happens because with current implementation we check return type after bind_self(), so we can't distinguish situation with an explicit Self from situation where return type just happens to (accidentally) match current class. I understand that Self-types are inherently (slightly) unsafe, but I don't want this unsafety to "leak" to code that doesn't use Self.

I think it makes sense to limit the return type erasure only to situations where the original callable type contained an explicit Self type (or the "old style" self type).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw when looking at this I found a bug in variance inference that affects protocols as well. In particular, all three examples above should be inferred as covariant (or even bivariant, but we don't have that).

Also to clarify, the unsafety I meant above relates to examples like

class C[T]:
    def meth(self, x: T, y: C[T]) -> C[T]: ...

that should not be contravariant, while it seems to me with the proposed PR it will be. But after playing with this more, inference doesn't behave as it should on master either. This needs some cleanup. @JukkaL we can discuss this in our meeting on Tuesday.


c: Covariant[object]
d: Covariant[int]
Expand Down Expand Up @@ -393,6 +393,62 @@ inv3_1: Invariant3[float] = Invariant3[int](1) # E: Incompatible types in assig
inv3_2: Invariant3[int] = Invariant3[float](1) # E: Incompatible types in assignment (expression has type "Invariant3[float]", variable has type "Invariant3[int]")
[builtins fixtures/property.pyi]

[case testPEP695InferVarianceWithInheritedSelf]
from typing import overload, Self, TypeVar, Generic

T = TypeVar("T")
S = TypeVar("S")

class C(Generic[T]):
def f(self, x: T) -> Self: ...
def g(self) -> T: ...

class D[T1, T2](C[T1]):
def m(self, x: T2) -> None: ...

a1: D[int, int] = D[int, object]()
a2: D[int, object] = D[int, int]() # E: Incompatible types in assignment (expression has type "D[int, int]", variable has type "D[int, object]")
a3: D[int, int] = D[object, object]() # E: Incompatible types in assignment (expression has type "D[object, object]", variable has type "D[int, int]")
a4: D[object, int] = D[int, object]() # E: Incompatible types in assignment (expression has type "D[int, object]", variable has type "D[object, int]")

[case testPEP695InferVarianceWithReturnSelf]
from typing import Self, overload

class Cov[T]:
def f(self) -> Self: ...

a1: Cov[int] = Cov[float]() # E: Incompatible types in assignment (expression has type "Cov[float]", variable has type "Cov[int]")
a2: Cov[float] = Cov[int]()

class Contra[T]:
def f(self) -> Self: ...
def g(self, x: T) -> None: ...

b1: Contra[int] = Contra[float]()
b2: Contra[float] = Contra[int]() # E: Incompatible types in assignment (expression has type "Contra[int]", variable has type "Contra[float]")

class Cov2[T]:
@overload
def f(self, x): ...
@overload
def f(self) -> Self: ...
def f(self, x=None): ...

c1: Cov2[int] = Cov2[float]() # E: Incompatible types in assignment (expression has type "Cov2[float]", variable has type "Cov2[int]")
c2: Cov2[float] = Cov2[int]()

class Contra2[T]:
@overload
def f(self, x): ...
@overload
def f(self) -> Self: ...
def f(self, x=None): ...

def g(self, x: T) -> None: ...

d1: Contra2[int] = Contra2[float]()
d2: Contra2[float] = Contra2[int]() # E: Incompatible types in assignment (expression has type "Contra2[int]", variable has type "Contra2[float]")

[case testPEP695InheritInvariant]
class Invariant[T]:
x: T
Expand Down
20 changes: 20 additions & 0 deletions test-data/unit/pythoneval.test
Original file line number Diff line number Diff line change
Expand Up @@ -2196,3 +2196,23 @@ type K4 = None | B[int]

type L1 = Never
type L2 = list[Never]

[case testPEP695VarianceInferenceSpecialCaseWithTypeshed]
# flags: --python-version=3.12
class C1[T1, T2](list[T1]):
def m(self, a: T2) -> None: ...

def func1(p: C1[int, object]):
x: C1[int, int] = p

class C2[T1, T2, T3](dict[T2, T3]):
def m(self, a: T1) -> None: ...

def func2(p: C2[object, int, int]):
x: C2[int, int, int] = p

class C3[T1, T2](tuple[T1, ...]):
def m(self, a: T2) -> None: ...

def func3(p: C3[int, object]):
x: C3[int, int] = p
Loading