Order-dependent overload resolution failure due to memorised Protocol conformance check on generic class

[cmp0xff]

Related to #10607, now stand-alone example.

Describe the bug

When an operation evaluates whether a generic class matches an @overload conditioned on a Protocol parameter and fails (e.g. Series[A] does not implement Supports_ProtoAdd[B, A], see the example below), pyright appears to memorise or cache a negative conformance result for the generic class against that subsequent expressions on valid specializations (such as Series[bool] with True, where Series[bool] correctly implements Supports_ProtoAdd[bool, bool]) erroneously bypass the protocol overload and fall through to fallback overloads (e.g., matching Series[bool] + int because bool is a subclass of int), returning the wrong type.

If the prior failing operation is removed or moved after the valid operation, both expressions type-check correctly with 0 errors.

Code or Screenshots

https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAqAmgAoCiA%2BgMIASZVA0gJIByA4gDRRsCmKPIJAGMuJcDDBCwAGy4FEPAGoBDEF2UBnDQJgV4CHlzAA3AdLDKAJgChrBClJQwQyqAF5CClSAAUAIntHZ2U-LiCXY1UkZSc3AhAAVx4ASjt3TwNvfwI-VIBldPlM1X883Os8hzAnF0KvEr9K8JCw6uDIwRiYOMSU2yFpTQ0oMmkeCD4YAHkEDQBZJAAPVB9efkEhAG08gF1kgC5rKGOoAAETMwsbE6hLHmAoCgRxMAorSx9taWB9qD9R8aTGbzJaoTYAIzAMh2oSgYBgAAsBL9ITIuAB6ZJQAC0AD4-gCJk5gQtligIVDpDDfgA6OlHE7nUwgcxWBnHO4PJ4vN6WD5fH4EsZE6azUlg1AwGFGRHIzBODFYvFCwHEsWg8mS6lQOk09m3e6PZ7w17vT48b4ypEgX5gcEAKx4Qhgipx%2BLtjudhxuNxAPBgCRAKCgAv6gy0UDyCQQODwGgoYhNAEE%2BT5ExIpNJNoE2i45HtvSdOUaeWaBVa5TmaspXcrCUD1WTsztafTrAMhpGBEgeBofPW1SCm7suGtu1tdgd9UzLmybsWKLzLIvzd9flGY7gYPH02AU5ZtlVq1wR3DZTbI0fgrX8Y1u73ttrddOLiyrvqF0uVwLfnfBA-USpWF4WtX5JRvP48nvDRNi1PxWz1edDUXd5vwtYAKwvD0nRdKBMTdOEHRwwsfSgP0AyDEN0LDTskwQ9tw2GAAhejrAAYkjGAeAQKAAEZfjISJpASZQuKgCAkA0CBRKEJFLDhAwXBgJBqjhYMoP-GCkx2KAAGooCY9iu00zZtNuMBeygFgpgITBsGFSYSxNJcoGAXADL1eNlHXaDTJ0jwNJ7PtUnBX4mPSJifFSJAHmIchqDoRhWDYEjHnSLy9KgcFjg4-QeDAtAUFwHhNjARTRNwHSOIQOBBDQBEYAKoq-U2P1Y1FAQKpAJgtCSDg2q3ABVFAAGsioAdxQbxonBMYih4KqoBqv1gGkOAmuKzYEhQDRo3anhLGxMrOuU6pFvgDaWu23bNzwA6jvKiQQB2WwOLyLieIAJl%2BFgwHGqAeCEkSxKEpB5OOpSVPU3zAJ0-T4iSax40AnyTNh9JAt7KLXpGAANcgqAIMgABFUaCiloSgHxjGiSM9q3HcXn3CnZCyyk9iMpNCcGpMABlfmOTGYK1Y4fA3drGeTPkoHGzQoCEZRZIOqA5dQUHLC4YBlGkaRUAwcFFZG2AwGM8n0f0yUsqdZQEm0NmZEwYZXF28E8rhB5JVSP1hg8ZHKUyhGeGsIYdD0BQfG9k8YfZrEoA46o1qW2qkHq-AXEkyzrSDoA

from typing import TYPE_CHECKING, Generic, Protocol, TypeVar, assert_type, overload

T_contra = TypeVar("T_contra", contravariant=True)
T = TypeVar("T")
S = TypeVar("S")
S_contra = TypeVar("S_contra", contravariant=True)

class ElementOpsMixin(Generic[S]):
    @overload
    def _proto_add(self: "ElementOpsMixin[bool]", other: bool, /) -> "ElementOpsMixin[bool]": ...
    @overload
    def _proto_add(self: "ElementOpsMixin[int]", other: int, /) -> "ElementOpsMixin[int]": ...
    def _proto_add(self, other: object, /) -> object:
        return self

class Supports_ProtoAdd(Protocol[T_contra, T]):
    def _proto_add(self, other: T_contra, /) -> ElementOpsMixin[T]: ...

class Series(ElementOpsMixin[S], Generic[S]):
    @overload
    def __add__(self: Supports_ProtoAdd[S_contra, S], other: S_contra, /) -> "Series[S]": ...
    @overload
    def __add__(self: "Series[bool]", other: int, /) -> "Series[int]": ...
    def __add__(self, other: object, /) -> object:
        return self

class A: ...
class B: ...

# Step 1: Evaluate mismatched operation on Series[A] + B
# Series[A] does NOT implement _proto_add for B.
s_a: Series[A] = Series()
b: B = B()
if TYPE_CHECKING:
    _ = s_a + b  # type: ignore[operator] # pyright: ignore[reportOperatorIssue,reportUnknownVariableType] # pyrefly: ignore[unsupported-operation] # ty: ignore[unsupported-operator]

# Step 2: Now evaluate valid operation Series[bool] + True
s_bool: Series[bool] = Series()

# EXPECTED: Series[bool] (via Supports_ProtoAdd[bool, bool])
# ACTUAL:   Series[int]  (Supports_ProtoAdd was cached as invalid, falling back to Series[bool] + int because bool is a subtype of int)
res = s_bool + True
assert_type(res, Series[bool])  # only pyright raises here

If your code relies on symbols that are imported from a third-party library, include the associated import statements and specify which versions of those libraries you have installed.

VS code or command line

playground

Co-authored by Gemini 3.7 Flash

[cmp0xff]

Hi Rich Chiodo (@rchiodo), thank you for your #11649! I see you've reopened the issue. Is it fixed?

[rchiodo]

It should be fixed when we ship the next release. That's why I added the 'addressed in next version' label.

[cmp0xff]

Thank you! I know the publishing of pyright to pypi is managed by a third party, but apparently they are inactive recently (e.g. RobertCraigie/pyright-python#365), so users have troubles obtaining new versions. It would be great if there could be some work arounds or other forms of help.

[rchiodo]

I believe the Pyright PyPI package has flags that lets you specify the version. I did this:

set PYRIGHT_PYTHON_FORCE_VERSION=latest

And it used 1.1.413