[ty] Unify polarity-aware relation construction - #27707
Merged
Merged
Conversation
Typing conformance resultsNo changes detected ✅Current numbersThe percentage of diagnostics emitted that were expected errors held steady at 96.98%. The percentage of expected errors that received a diagnostic held steady at 93.62%. The number of fully passing files held steady at 106/133. |
Memory usage reportSummary
Significant changesClick to expand detailed breakdownprefect
sphinx
trio
flake8
|
|
charliermarsh
approved these changes
Aug 13, 2026
Member
|
The change makes sense to me conceptually though not super clear to me from the summary how much we're changing behavior here vs. a pure refactor. (My read is that this is fixing some limitations, it's not a pure refactor.) |
carljm
marked this pull request as draft
August 13, 2026 01:56
Contributor
Author
|
Yes, this is not a refactor, it's fixing the fact that we were incorrectly always treating several kinds of relations in SpecializationBuilder as if they were covariant relations (where the actual argument must be assignable to the formal parameter), and ignoring the actual polarity. So e.g. for protocols nested inside Callable arguments or whatever, the direction should be flipped and we weren't doing that. This PR fixes that. I'll make the description clearer, thanks. |
carljm
marked this pull request as ready for review
August 13, 2026 05:23
George-Ogden
pushed a commit
to George-Ogden/ruff
that referenced
this pull request
Aug 16, 2026
## Summary Part of astral-sh/ty#3557 and the second step in rebooting astral-sh#26712 as smaller focused changes. On `main`, generic inference does not consistently propagate the variance of an enclosing generic class into structural-protocol and callable comparisons. Those comparisons instead construct the forward `actual <= formal` relation even when the surrounding position is contravariant or invariant. This produces incorrect bounds and inferred types: ```py from typing import Callable class Middle: ... class Derived(Middle): ... class Sink[T]: def put(self, value: T) -> None: ... def infer[T](sink: Sink[Callable[[], T]], value: T) -> T: return value reveal_type(infer(Sink[Callable[[], Middle]](), Derived())) # main: Middle # this PR: Derived ``` This PR centralizes relation construction so inference consistently follows the surrounding polarity: covariance requires `actual <= formal`, contravariance requires `formal <= actual`, invariance requires both, and bivariance adds no constraints. The fix applies to nested generic protocols, callable signatures, callback protocols, TypedDict/protocol relations, and materialized protocols while preserving existing nominal inference, callable-union alternatives, TypedDict-union optimizations, and cycle-safe handling of recursive protocols. ## Limitation The new constraint solver does not yet support `ParamSpec` or `TypeVarTuple`. Whenever an inference context includes either variadic, all affected relations must continue through the old, forward-only inference path, including comparisons involving ordinary type variables alongside an unrelated variadic. This deliberately avoids regressions in parameter inference, structural protocols, descriptor signatures, and downstream argument diagnostics, but it also means the polarity fixes in this PR do not apply whenever variadics are involved. Focused TODO tests document cases that still incorrectly infer `Middle` instead of `Derived`; fixing those cases requires adding variadic support to the new solver. ## Test plan Added mdtests cover nested nominal classes, generic protocols, callable signatures, and callback protocols under covariance, contravariance, and invariance; double contravariance, callable-union alternatives, and invariant rejection; finite, recursive, and recursive-only materialized protocols; invariant, contravariant, and covariant `ParamSpec` inference through `Concatenate`; bounded and constrained prefixes, descriptor receivers, structural protocol unions, nominal callable objects, higher-order callbacks, and mixed ordinary/variadic inference; analogous `TypeVarTuple` callable, protocol, structural-union, and nominal-implementation cases; and explicit TODO coverage for unrelated variadics suppressing otherwise-correct polarity. Additional mdtests cover repeated comparisons of the same protocol or callable under opposite polarities, plus overloaded callable acceptance and rejection across covariant, contravariant, and invariant wrappers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of astral-sh/ty#3557 and the second step in rebooting #26712 as smaller focused changes.
On
main, generic inference does not consistently propagate the variance of an enclosing generic class into structural-protocol and callable comparisons. Those comparisons instead construct the forwardactual <= formalrelation even when the surrounding position is contravariant or invariant. This produces incorrect bounds and inferred types:This PR centralizes relation construction so inference consistently follows the surrounding polarity: covariance requires
actual <= formal, contravariance requiresformal <= actual, invariance requires both, and bivariance adds no constraints. The fix applies to nested generic protocols, callable signatures, callback protocols, TypedDict/protocol relations, and materialized protocols while preserving existing nominal inference, callable-union alternatives, TypedDict-union optimizations, and cycle-safe handling of recursive protocols.Limitation
The new constraint solver does not yet support
ParamSpecorTypeVarTuple. Whenever an inference context includes either variadic, all affected relations must continue through the old, forward-only inference path, including comparisons involving ordinary type variables alongside an unrelated variadic. This deliberately avoids regressions in parameter inference, structural protocols, descriptor signatures, and downstream argument diagnostics, but it also means the polarity fixes in this PR do not apply whenever variadics are involved. Focused TODO tests document cases that still incorrectly inferMiddleinstead ofDerived; fixing those cases requires adding variadic support to the new solver.Test plan
Added mdtests cover nested nominal classes, generic protocols, callable signatures, and callback protocols under covariance, contravariance, and invariance; double contravariance, callable-union alternatives, and invariant rejection; finite, recursive, and recursive-only materialized protocols; invariant, contravariant, and covariant
ParamSpecinference throughConcatenate; bounded and constrained prefixes, descriptor receivers, structural protocol unions, nominal callable objects, higher-order callbacks, and mixed ordinary/variadic inference; analogousTypeVarTuplecallable, protocol, structural-union, and nominal-implementation cases; and explicit TODO coverage for unrelated variadics suppressing otherwise-correct polarity.Additional mdtests cover repeated comparisons of the same protocol or callable under opposite polarities, plus overloaded callable acceptance and rejection across covariant, contravariant, and invariant wrappers.