Skip to content

Conversation

@dcreager
Copy link
Member

@dcreager dcreager commented Oct 24, 2025

It's possible for a constraint to mention two typevars. For instance, in the body of

def f[S: int, T: S](): ...

the baseline constraint set would be (T ≤ S) ∧ (S ≤ int). That is, S must specialize to some subtype of int, and T must specialize to a subtype of the type that S specializes to.

This PR updates the new "constraint implication" relationship from #21010 to work on these kinds of constraint sets. For instance, in the example above, we should be able to see that T ≤ int must always hold:

def f[S, T]():
    constraints = ConstraintSet.range(Never, S, int) & ConstraintSet.range(Never, T, S)
    static_assert(constraints.implies_subtype_of(T, int))  # now succeeds!

This did not require major changes to the implementation of implies_subtype_of. That method already relies on how our simplify and domain methods expand a constraint set to include the transitive closure of the constraints that it mentions, and to mark certain combinations of constraints as impossible. Previously, that transitive closure logic only looked at pairs of constraints that constrain the same typevar. (For instance, to notice that (T ≤ bool) ∧ ¬(T ≤ int) is impossible.)

Now we also look at pairs of constraints that constraint different typevars, if one of the constraints is bound by the other — that is, pairs of the form T ≤ S and S ≤ something, or S ≤ T and something ≤ S. In those cases, transitivity lets us add a new derived constraint that T ≤ something or something ≤ T, respectively. Having done that, our existing implies_subtype_of logic finds and takes into account that derived constraint.

@dcreager dcreager added internal An internal refactor or improvement ty Multi-file analysis & type inference labels Oct 24, 2025
@dcreager dcreager force-pushed the dcreager/subtype-given-typevars branch from fdea956 to c15f58c Compare October 24, 2025 18:20
@dcreager dcreager changed the base branch from main to dcreager/new-relation October 24, 2025 18:20
@dcreager dcreager force-pushed the dcreager/subtype-given-typevars branch from c15f58c to eed87d1 Compare October 24, 2025 20:15
@github-actions
Copy link
Contributor

github-actions bot commented Oct 24, 2025

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@github-actions
Copy link
Contributor

github-actions bot commented Oct 24, 2025

mypy_primer results

No ecosystem changes detected ✅
No memory usage changes detected ✅

Base automatically changed from dcreager/new-relation to main October 28, 2025 02:01
@dcreager dcreager force-pushed the dcreager/subtype-given-typevars branch 2 times, most recently from 5e98232 to e8fd554 Compare October 28, 2025 14:30
@dcreager dcreager changed the base branch from main to dcreager/constraint-set-api October 28, 2025 14:31
@dcreager dcreager marked this pull request as ready for review October 28, 2025 16:55
Base automatically changed from dcreager/constraint-set-api to main October 28, 2025 18:32
@dcreager dcreager requested a review from MichaReiser as a code owner October 28, 2025 18:32
@dcreager dcreager force-pushed the dcreager/subtype-given-typevars branch from 7227293 to 64402d1 Compare October 28, 2025 19:00
@codspeed-hq
Copy link

codspeed-hq bot commented Oct 29, 2025

CodSpeed Performance Report

Merging #21068 will not alter performance

Comparing dcreager/subtype-given-typevars (c054447) with main (d0aebaa)

Summary

✅ 22 untouched
⏩ 30 skipped1

Footnotes

  1. 30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

static_assert(not given_int.implies_subtype_of(T, str))

# If [T ≤ U ∧ U ≤ int], then [T ≤ int] must be true as well.
given_int = ConstraintSet.range(Never, T, U) & ConstraintSet.range(Never, U, int)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More an understand question for me. What's the difference between Never and passing U?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

range(Never, T, U) is Never ≤ T ≤ U, which simplifies to T ≤ U since every type is a supertype of Never.

range(U, T, U) is U ≤ T ≤ U, which simplifies to T = U.

So the first one is "T must be a subtype of U", and the second one is "T must be exactly the same type as U"

Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me

@dcreager dcreager merged commit c0b04d4 into main Oct 30, 2025
41 checks passed
@dcreager dcreager deleted the dcreager/subtype-given-typevars branch October 30, 2025 20:11
dcreager added a commit that referenced this pull request Oct 30, 2025
* origin/main: (21 commits)
  [ty] Update "constraint implication" relation to work on constraints between two typevars (#21068)
  [`flake8-type-checking`] Fix `TC003` false positive with `future-annotations` (#21125)
  [ty] Fix lookup of `__new__` on instances (#21147)
  Fix syntax error false positive on nested alternative patterns (#21104)
  [`pyupgrade`] Fix false positive for `TypeVar` with default on Python <3.13 (`UP046`,`UP047`) (#21045)
  [ty] Reachability and narrowing for enum methods (#21130)
  [ty] Use `range` instead of custom `IntIterable` (#21138)
  [`ruff`] Add support for additional eager conversion patterns (`RUF065`) (#20657)
  [`ruff-ecosystem`] Fix CLI crash on Python 3.14 (#21092)
  [ty] Infer type of `self` for decorated methods and properties (#21123)
  [`flake8-bandit`] Fix correct example for `S308` (#21128)
  [ty] Dont provide goto definition for definitions which are not reexported in builtins (#21127)
  [`airflow`] warning `airflow....DAG.create_dagrun` has been removed (`AIR301`) (#21093)
  [ty] follow the breaking API changes made in salsa-rs/salsa#1015 (#21117)
  [ty] Rename `Type::into_nominal_instance` (#21124)
  [ty] Filter out "unimported" from the current module
  [ty] Add evaluation test for auto-import including symbols in current module
  [ty] Refactor `ty_ide` completion tests
  [ty] Render `import <...>` in completions when "label details" isn't supported
  [`refurb`] Preserve digit separators in `Decimal` constructor (`FURB157`) (#20588)
  ...
dcreager added a commit that referenced this pull request Nov 18, 2025
)

This saga began with a regression in how we handle constraint sets where
a typevar is constrained by another typevar, which #21068 first added
support for:

```py
def mutually_constrained[T, U]():
    # If [T = U ∧ U ≤ int], then [T ≤ int] must be true as well.
    given_int = ConstraintSet.range(U, T, U) & ConstraintSet.range(Never, U, int)
    static_assert(given_int.implies_subtype_of(T, int))
```

While working on #21414, I saw a regression in this test, which was
strange, since that PR has nothing to do with this logic! The issue is
that something in that PR made us instantiate the typevars `T` and `U`
in a different order, giving them differently ordered salsa IDs. And
importantly, we use these salsa IDs to define the variable ordering that
is used in our constraint set BDDs. This showed that our "mutually
constrained" logic only worked for one of the two possible orderings.
(We can — and now do — test this in a brute-force way by copy/pasting
the test with both typevar orderings.)

The underlying bug was in our `ConstraintSet::simplify_and_domain`
method. It would correctly detect `(U ≤ T ≤ U) ∧ (U ≤ int)`, because
those two constraints affect different typevars, and from that, infer `T
≤ int`. But it wouldn't detect the equivalent pattern in `(T ≤ U ≤ T) ∧
(U ≤ int)`, since those constraints affect the same typevar. At first I
tried adding that as yet more pattern-match logic in the ever-growing
`simplify_and_domain` method. But doing so caused other tests to start
failing.

At that point, I realized that `simplify_and_domain` had gotten to the
point where it was trying to do too much, and for conflicting consumers.
It was first written as part of our display logic, where the goal is to
remove redundant information from a BDD to make its string rendering
simpler. But we also started using it to add "derived facts" to a BDD. A
derived fact is a constraint that doesn't appear in the BDD directly,
but which we can still infer to be true. Our failing test relies on
derived facts — being able to infer that `T ≤ int` even though that
particular constraint doesn't appear in the original BDD. Before,
`simplify_and_domain` would trace through all of the constraints in a
BDD, figure out the full set of derived facts, and _add those derived
facts_ to the BDD structure. This is brittle, because those derived
facts are not universally true! In our example, `T ≤ int` only holds
along the BDD paths where both `T = U` and `U ≤ int`. Other paths will
test the negations of those constraints, and on those, we _shouldn't_
infer `T ≤ int`. In theory it's possible (and we were trying) to use BDD
operators to express that dependency...but that runs afoul of how we
were simultaneously trying to _remove_ information to make our displays
simpler.

So, I ripped off the band-aid. `simplify_and_domain` is now _only_ used
for display purposes. I have not touched it at all, except to remove
some logic that is definitely not used by our `Display` impl. Otherwise,
I did not want to touch that house of cards for now, since the display
logic is not load-bearing for any type inference logic.

For all non-display callers, we have a new **_sequent map_** data type,
which tracks exactly the same derived information. But it does so (a)
without trying to remove anything from the BDD, and (b) lazily, without
updating the BDD structure.

So the end result is that all of the tests (including the new
regressions) pass, via a more efficient (and hopefully better
structured/documented) implementation, at the cost of hanging onto a
pile of display-related tech debt that we'll want to clean up at some
point.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal An internal refactor or improvement ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants