Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mypy doesn't escape typevars from generics in self-type #18400

Open
A5rocks opened this issue Jan 1, 2025 · 3 comments · May be fixed by #18401
Open

mypy doesn't escape typevars from generics in self-type #18400

A5rocks opened this issue Jan 1, 2025 · 3 comments · May be fixed by #18401
Labels
bug mypy got something wrong

Comments

@A5rocks
Copy link
Contributor

A5rocks commented Jan 1, 2025

Bug Report

Passing a callable to another which matches the typevar with another typevar doesn't allow passing them into the return type. (if that makes no sense, the reproducer below is simple)

To Reproduce

from typing import Generic, TypeVar, Callable

T = TypeVar("T")
V = TypeVar("V")

class X(Generic[T]):
    def f(self: X[Callable[[V], None]]) -> Callable[[V], V]:
        def inner_f(v: V) -> V:
            return v

        return inner_f

reveal_type(X[Callable[[T], None]]().f())  # N: Revealed type is "def (Never) -> Never"
# ^ this should keep the generic

Expected Behavior

I don't expect a Never.

Your Environment

Checked on mypy playground.

  • Mypy version used: v1.14
  • Mypy command-line flags: --strict
  • Mypy configuration options from mypy.ini (and other config files): N/A
  • Python version used: 3.12
@A5rocks A5rocks added the bug mypy got something wrong label Jan 1, 2025
@A5rocks A5rocks changed the title mypy doesn't escaped stored typevartuples in a self-type mypy doesn't escape stored typevartuples in a self-type Jan 1, 2025
@A5rocks
Copy link
Contributor Author

A5rocks commented Jan 1, 2025

Turns out this applies to TypeVars too -- I'll update this issue for that (since it's simpler).

@A5rocks A5rocks changed the title mypy doesn't escape stored typevartuples in a self-type mypy doesn't escape stored typevars in a self-type Jan 1, 2025
@A5rocks A5rocks changed the title mypy doesn't escape stored typevars in a self-type mypy doesn't escape typevars from a passed in function Jan 1, 2025
@A5rocks A5rocks changed the title mypy doesn't escape typevars from a passed in function mypy doesn't escape typevars that partially match Jan 1, 2025
@A5rocks A5rocks changed the title mypy doesn't escape typevars that partially match mypy doesn't escape typevars from generics in self-type Jan 1, 2025
@sterliakov
Copy link
Collaborator

Hm. Are you sure this is supposed to work at all? There is no precedent of bare typevar at top level working like this, otherwise we'd have some fancy features like dict[T, Callable[[T], None]] top-level type map to object handlers which, I believe, have never been supported (and maybe shouldn't be - what does the spec say?).

As a rule of thumb, there's no way to spell your repro using horrible PEP695 syntax, so the typevar is used in a wrong scope. Mypy should produce an error right there ("Type variable T is unbound"), and absence of that diagnostic is indeed a bug.

If I fix your example to put the typevar in scope, mypy does what you expected:

from typing import Generic, TypeVar, Callable

T = TypeVar("T")
V = TypeVar("V")

class X(Generic[T]):
    def f(self: X[Callable[[V], None]]) -> Callable[[V], V]:
        def inner_f(v: V) -> V:
            return v

        return inner_f


def fn(_: T) -> None:
    reveal_type(X[Callable[[T], None]]().f())  # N: Revealed type is "def (T`-1) -> T`-1"

plaground

@A5rocks
Copy link
Contributor Author

A5rocks commented Jan 10, 2025

Hm. Are you sure this is supposed to work at all? There is no precedent of bare typevar at top level working like this, otherwise we'd have some fancy features like dict[T, Callable[[T], None]] top-level type map to object handlers which, I believe, have never been supported (and maybe shouldn't be - what does the spec say?).

This should be valid because callables capture their generic. I'm also just doing this for a more simple reproducer. Here's the less minimized version of my failing example:

from __future__ import annotations

from dataclasses import dataclass
from typing import TypeVarTuple, Unpack, Generic, TypeVar, Any, overload
from collections.abc import Callable

# TODO: hide this in an internal file so that I can have descriptive names without polluting namespace
Args = TypeVarTuple("Args")

class Build(Generic[Unpack[Args]]):
    pass

class Built(Generic[Unpack[Args]]):
    pass

From = TypeVarTuple("From")
To = TypeVarTuple("To")
InitialCallable = TypeVar("InitialCallable", bound=Callable[[Build[Unpack[tuple[Any, ...]]]], Build[Unpack[tuple[Any, ...]]]])

@dataclass
class Builder(Generic[InitialCallable]):
    initial_callable: InitialCallable

    def __call__(self: Builder[Callable[[Build[Unpack[From]]], Build[Unpack[To]]]]) -> Callable[[Callable[[object, Unpack[To]], None]], Built[Unpack[From]]]:
        def build_up(in_progress: Callable[[object, Unpack[To]], None]) -> Built[Unpack[From]]:
            ...

        return build_up

def make_builder() -> Callable[[Callable[[Build[Unpack[From]]], Build[Unpack[To]]]], Builder[Callable[[Build[Unpack[From]]], Build[Unpack[To]]]]]:
    def inner_make_builder(command: Callable[[Build[Unpack[From]]], Build[Unpack[To]]]) -> Builder[Callable[[Build[Unpack[From]]], Build[Unpack[To]]]]:
        return Builder(command)

    return inner_make_builder

from typing import Unpack, TypeVarTuple

#Args = TypeVarTuple("Args")

def test_basic_command() -> None:

    @make_builder()
    def command_builder(b: Build[Unpack[Args]]) -> Build[Unpack[Args]]:
        return b
    
    reveal_type(command_builder())  # N: Revealed type is "def (def (builtins.object, *Never)) -> __main__.Built[Unpack[builtins.tuple[Never, ...]]]"
    # ^ I expect this to work

To prove this should work, I've turned Builder.__call__ into an explicit call to Builder_call and removed use of a self-type:

from __future__ import annotations

from dataclasses import dataclass
from typing import TypeVarTuple, Unpack, Generic, TypeVar, Any, overload
from collections.abc import Callable

# TODO: hide this in an internal file so that I can have descriptive names without polluting namespace
Args = TypeVarTuple("Args")

class Build(Generic[Unpack[Args]]):
    pass

class Built(Generic[Unpack[Args]]):
    pass

From = TypeVarTuple("From")
To = TypeVarTuple("To")
InitialCallable = TypeVar("InitialCallable", bound=Callable[[Build[Unpack[tuple[Any, ...]]]], Build[Unpack[tuple[Any, ...]]]])

@dataclass
class Builder(Generic[InitialCallable]):
    initial_callable: InitialCallable

def Builder_call(self: Builder[Callable[[Build[Unpack[From]]], Build[Unpack[To]]]]) -> Callable[[Callable[[object, Unpack[To]], None]], Built[Unpack[From]]]:
    def build_up(in_progress: Callable[[object, Unpack[To]], None]) -> Built[Unpack[From]]:
        ...

    return build_up

def make_builder() -> Callable[[Callable[[Build[Unpack[From]]], Build[Unpack[To]]]], Builder[Callable[[Build[Unpack[From]]], Build[Unpack[To]]]]]:
    def inner_make_builder(command: Callable[[Build[Unpack[From]]], Build[Unpack[To]]]) -> Builder[Callable[[Build[Unpack[From]]], Build[Unpack[To]]]]:
        return Builder(command)

    return inner_make_builder

from typing import Unpack, TypeVarTuple

#Args = TypeVarTuple("Args")

def test_basic_command() -> None:

    @make_builder()
    def command_builder(b: Build[Unpack[Args]]) -> Build[Unpack[Args]]:
        return b
    
    reveal_type(Builder_call(command_builder))  # N: Revealed type is "def [From] (def (builtins.object, *Unpack[From`330])) -> __main__.Built[Unpack[From`330]]"
    # ^ now it works!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants