-
Notifications
You must be signed in to change notification settings - Fork 28
feat: mostly support Generic #272
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
nstarman
wants to merge
31
commits into
beartype:master
Choose a base branch
from
nstarman:generics
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 23 commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
2d0db17
feat: limited support for generics
nstarman a6ac22c
feat: honor __orig_class__ for custom generic dispatch
nstarman 08ddce6
Add A[Any] explicit fallback for custom Generic dispatch
nstarman 4b57bb2
Extract generic timing into standalone script + {include}
nstarman 9a9d284
feat: add @plum.generic decorator for auto-inferring __orig_class__
nstarman e22b0d1
docs: remove % skip from bare-class-hint example, add live doctests
nstarman d9339b8
fix: type hints
nstarman 5f4640c
fix: address Copilot PR review comments on generics support
nstarman cff0abe
tests: use dispatch fixture, add parametrize; clean up comments
nstarman 56b9ac5
Use beartype.typing.Protocol instead of typing.Protocol
nstarman dff94e1
Fix is_generic_hint to exclude Annotated via __metadata__
nstarman 7ee254c
Trust __orig_class__ by convention in is_bearable_with_orig and _arg_key
nstarman 8f8c757
perf: build TypeHintWrapper pairs once in Signature.__le__
nstarman cecebad
perf: hoist type-tuple computation before needs_generic check
nstarman d11a3e8
perf: short-circuit Signature.__eq__ on cheap scalar checks
nstarman c44885f
perf: override is_comparable on Signature to call __le__ at most twice
nstarman 3ee50f3
perf: eliminate intermediate list allocation in _resolve_from
nstarman d0217c3
perf: use weakref.WeakSet for Function._instances to prevent memory l…
nstarman 6a6d318
fix: resolve_for_type falls back to full resolve on NotFoundLookupError
nstarman 19d6476
fix: fall back to full resolve when prefiltered bucket misses Any/Uni…
nstarman 0ebf1aa
test: document AmbiguousLookupError behavior with non-bucket Any fall…
nstarman b0eec5e
fix: mark test_resolve_from_does_not_materialise_filter_list as incom…
nstarman 3c3452f
perf: update Resolver.register metadata incrementally instead of full…
nstarman 23296dd
perf: replace layer-peeling sort with Kahn's topological sort
nstarman 0590343
simplify: condense tests and source code
nstarman 36ff88e
docs: add generics guide
nstarman 0bed947
fix: @generic now correctly handles subscripted construction on froze…
nstarman 7c294d1
fix: enforce @classmethod on __infer_type_parameter__ at decoration time
nstarman 8fd538f
fix: deduplicate generic cache by hint_tuple, not impl identity
nstarman ebf98b6
Apply suggestions from code review
nstarman f6f0f95
Update generics.md for clarity on type inference
nstarman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| | Call | Scenario | µs / call | vs faithful | | ||
| | :--- | :--- | ---: | ---: | | ||
| | `f(B(1))` | faithful — bare `B` overload | 4.01 | 1.0× | | ||
| | `f(A(1))` | generic — `A[Any]` fallback | 5.81 | 1.4× | | ||
| | `f(A[int](1))` | generic — `A[int]` overload | 6.28 | 1.6× | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| """Generate the generic-dispatch timing table for docs/generics.md. | ||
|
|
||
| Run directly:: | ||
|
|
||
| uv run python docs/_scripts/time_generics.py | ||
|
|
||
| or via nox (which also builds the docs):: | ||
|
|
||
| uv run nox -s docs | ||
|
|
||
| Output is written to docs/_generated/generics_timing.md, which is | ||
| included verbatim by docs/generics.md via the MyST {include} directive. | ||
| """ | ||
|
|
||
| import dataclasses | ||
| import pathlib | ||
| import timeit | ||
| import typing | ||
|
|
||
| import plum | ||
|
|
||
| T = typing.TypeVar("T") | ||
|
|
||
|
|
||
| # ── Faithful baseline ────────────────────────────────────────────────────── | ||
| # A bare `B` overload with no parameterized overloads. Plum uses the fast | ||
| # faithful-cache path (cache key = type(arg), no __orig_class__ handling). | ||
|
|
||
|
|
||
| @dataclasses.dataclass | ||
| class B(typing.Generic[T]): | ||
| x: T | ||
|
|
||
|
|
||
| d_faithful = plum.Dispatcher() | ||
|
|
||
|
|
||
| @d_faithful | ||
| def f_faithful(b: B) -> str: | ||
| return "B" | ||
|
|
||
|
|
||
| # ── Generic dispatch ─────────────────────────────────────────────────────── | ||
| # Three overloads on A: A[Any] (bare-instance fallback), A[int], A[str]. | ||
| # Plum uses the two-tier generic cache, keyed on __orig_class__ when present. | ||
|
|
||
|
|
||
| @dataclasses.dataclass | ||
| class A(typing.Generic[T]): | ||
| x: T | ||
|
|
||
|
|
||
| d_generic = plum.Dispatcher() | ||
|
|
||
|
|
||
| @d_generic | ||
| def f_generic(a: A[typing.Any]) -> str: | ||
| return "A[Any]" | ||
|
|
||
|
|
||
| @d_generic | ||
| def f_generic(a: A[int]) -> str: | ||
| return "A[int]" | ||
|
|
||
|
|
||
| @d_generic | ||
| def f_generic(a: A[str]) -> str: | ||
| return "A[str]" | ||
|
|
||
|
|
||
| # ── Measure ──────────────────────────────────────────────────────────────── | ||
|
|
||
| # Warm up all cache paths before measuring. | ||
| f_faithful(B(1)) | ||
| f_generic(A(1)) | ||
| f_generic(A[int](1)) | ||
| f_generic(A[str]("x")) | ||
|
|
||
| N = 50_000 | ||
|
|
||
| t_faithful = timeit.timeit(lambda: f_faithful(B(1)), number=N) / N * 1e6 | ||
| t_any = timeit.timeit(lambda: f_generic(A(1)), number=N) / N * 1e6 | ||
| t_int = timeit.timeit(lambda: f_generic(A[int](1)), number=N) / N * 1e6 | ||
|
|
||
| rows = [ | ||
| ("f(B(1))", "faithful — bare `B` overload", t_faithful, 1.0), | ||
| ("f(A(1))", "generic — `A[Any]` fallback", t_any, t_any / t_faithful), | ||
| ("f(A[int](1))", "generic — `A[int]` overload", t_int, t_int / t_faithful), | ||
| ] | ||
|
|
||
| # ── Write markdown table ─────────────────────────────────────────────────── | ||
|
|
||
| lines = [ | ||
| "| Call | Scenario | µs / call | vs faithful |", | ||
| "| :--- | :--- | ---: | ---: |", | ||
| ] | ||
| for call, scenario, us, rel in rows: | ||
| lines.append(f"| `{call}` | {scenario} | {us:.2f} | {rel:.1f}× |") | ||
|
|
||
| out = pathlib.Path(__file__).parent.parent / "_generated" / "generics_timing.md" | ||
| out.parent.mkdir(exist_ok=True) | ||
| out.write_text("\n".join(lines) + "\n") | ||
| print(f"Written {out.relative_to(pathlib.Path(__file__).parent.parent.parent)}") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,231 @@ | ||
| # Custom Generic Types | ||
|
|
||
| Plum has **partial support** for dispatching on user-defined [generic classes](https://docs.python.org/3/library/typing.html#generics) — that is, classes you define yourself using `typing.Generic[T]`. This page explains exactly what works, what doesn't, and the recommended pattern for writing fallback overloads. | ||
|
|
||
| ```{note} | ||
| Built-in generic containers like `list[int]` and `dict[str, int]` are fully supported and inspected by Beartype directly. This page is about *your own* generic classes such as `class Box(Generic[T])`. | ||
| ``` | ||
|
|
||
| ## What works | ||
|
|
||
| Subscripted instances dispatch correctly: | ||
|
|
||
| ```python | ||
| from typing import Any, Generic, TypeVar | ||
| from plum import dispatch | ||
|
|
||
| T = TypeVar("T") | ||
|
|
||
|
|
||
| class Box(Generic[T]): | ||
| def __init__(self, value: T) -> None: | ||
| self.value = value | ||
|
|
||
|
|
||
| @dispatch | ||
| def unwrap(b: Box[int]) -> str: | ||
| return "int box" | ||
|
|
||
|
|
||
| @dispatch | ||
| def unwrap(b: Box[str]) -> str: | ||
| return "str box" | ||
| ``` | ||
|
|
||
| ```python | ||
| >>> unwrap(Box[int](1)) | ||
| 'int box' | ||
|
|
||
| >>> unwrap(Box[str]("hello")) | ||
| 'str box' | ||
| ``` | ||
|
|
||
| This works because Python sets `instance.__orig_class__ = Box[int]` after `Box[int](1)` finishes constructing the instance. Plum reads that attribute during dispatch and uses Beartype's type-hint subtype ordering (`TypeHint(Box[int]) <= TypeHint(Box[str])`) to choose the most specific overload. | ||
|
|
||
| (generics-bare-instance-limitation)= | ||
| ## The "bare instance" limitation | ||
|
|
||
| Python only sets `__orig_class__` when you instantiate through the subscripted form `Box[int](...)`. When you write `Box(1)` directly there is **no** parameterization information attached to the instance — neither Plum nor Beartype can recover `T`. | ||
|
|
||
| This means a bare `Box(1)` cannot be told apart from `Box[anything](1)`. To keep dispatch deterministic, Plum treats parameterized custom-generic hints as **non-matching** for instances that lack `__orig_class__`. With only the two overloads above, calling `unwrap(Box(1))` raises `NotFoundLookupError`: | ||
|
|
||
| ```python | ||
| >>> from plum import NotFoundLookupError | ||
| >>> try: | ||
| ... unwrap(Box(1)) | ||
| ... except NotFoundLookupError: | ||
| ... print("no match") | ||
| no match | ||
| ``` | ||
|
|
||
| ## The `A[Any]` fallback pattern | ||
|
|
||
| The recommended way to handle bare instances is to register an explicit **`Any`-parameterized fallback overload**. Extending the same `unwrap` above: | ||
|
|
||
| ```python | ||
| @dispatch | ||
| def unwrap(b: Box[Any]) -> str: | ||
| return "unknown parameterization" | ||
| ``` | ||
|
|
||
| Now all three call shapes resolve cleanly: | ||
|
|
||
| ```python | ||
| >>> unwrap(Box(1)) # no __orig_class__ → falls through to Box[Any] | ||
| 'unknown parameterization' | ||
|
|
||
| >>> unwrap(Box[int](1)) # __orig_class__ = Box[int]; most specific wins | ||
| 'int box' | ||
|
|
||
| >>> unwrap(Box[str]("hello")) # __orig_class__ = Box[str]; most specific wins | ||
| 'str box' | ||
| ``` | ||
|
|
||
| `Box[Any]` is treated as a strict supertype of every other `Box[X]`, so: | ||
|
|
||
| - **Subscripted instances** still pick the most specific overload — `Box[int](1)` prefers `Box[int]` over `Box[Any]`. | ||
| - **Bare instances** match only `Box[Any]` (the other overloads are excluded because there is no parameterization to verify), so dispatch is unambiguous. | ||
|
|
||
| ```{tip} | ||
| Think of `A[Any]` as the explicit way to say *"this overload is the fallback for any `A` instance whose parameter is unknown at runtime"*. Without it, bare instances will raise `NotFoundLookupError` when only parameterized overloads are registered. | ||
| ``` | ||
|
|
||
| ## Bare class hints (`Box` without brackets) | ||
|
|
||
| You can also write a fallback against the **bare** (unsubscripted) class: | ||
|
|
||
| ```python | ||
| @dispatch | ||
| def g(b: Box) -> str: | ||
| return "bare Box" | ||
|
|
||
|
|
||
| @dispatch | ||
| def g(b: Box[int]) -> str: | ||
| return "int" | ||
| ``` | ||
|
|
||
| A bare hint like `Box` behaves the same way as `Box[Any]` for dispatch purposes — every `Box` instance matches it, and parameterized overloads still win for subscripted instances when their parameter agrees. `Box` and `Box[Any]` are interchangeable as fallback overloads; pick whichever reads more clearly in your codebase. | ||
|
|
||
| ```python | ||
| >>> g(Box(1)) # bare instance — no __orig_class__ → Box fallback | ||
| 'bare Box' | ||
|
|
||
| >>> g(Box[int](1)) # __orig_class__ = Box[int] → specific overload wins | ||
| 'int' | ||
|
|
||
| >>> g(Box[str]("hi")) # __orig_class__ = Box[str] — no str overload → Box fallback | ||
| 'bare Box' | ||
| ``` | ||
|
|
||
| ## Auto-inferring types with `@plum.generic` | ||
|
|
||
| If your class can always infer `T` from the constructor arguments — for | ||
| example, `Box(1)` should behave exactly like `Box[int](1)` — you can opt into | ||
| automatic inference with `@plum.generic`: | ||
|
nstarman marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```python | ||
| from typing import Generic, TypeVar | ||
| from plum import dispatch, generic | ||
|
|
||
| T = TypeVar("T") | ||
|
|
||
|
|
||
| @generic | ||
| class Box(Generic[T]): | ||
| def __init__(self, value) -> None: | ||
| self.value = value | ||
|
|
||
| @classmethod | ||
| def __infer_type_parameter__(cls, instance): | ||
| return type(instance.value) | ||
|
|
||
|
|
||
| @dispatch | ||
| def unwrap(b: Box[int]) -> str: | ||
| return "int box" | ||
|
|
||
|
|
||
| @dispatch | ||
| def unwrap(b: Box[str]) -> str: | ||
| return "str box" | ||
| ``` | ||
|
|
||
| Bare instances now dispatch correctly **without** an `A[Any]` fallback: | ||
|
|
||
| ```python | ||
| >>> unwrap(Box(1)) | ||
| 'int box' | ||
|
|
||
| >>> unwrap(Box("hello")) | ||
| 'str box' | ||
|
|
||
| >>> unwrap(Box[str](1)) # explicit subscription still wins | ||
| 'str box' | ||
| ``` | ||
|
|
||
| The decorator wraps `__init__` so that after construction it sets `instance.__orig_class__ = Box[inferred_T]`. When you use the subscripted form `Box[str](1)`, Python's own machinery overwrites that attribute after `__init__` returns, so explicit parameterisation always takes precedence. | ||
|
|
||
| **Inference rule**: define `__infer_type_parameter__(cls, instance)` as a classmethod that inspects the freshly-constructed instance and returns the type parameter. For multi-parameter generics return a tuple: | ||
|
|
||
| ```python | ||
| from typing import Generic, TypeVar | ||
| from plum import generic | ||
|
|
||
| T = TypeVar("T") | ||
| S = TypeVar("S") | ||
|
|
||
|
|
||
| @generic | ||
| class Pair(Generic[T, S]): | ||
| def __init__(self, x, y) -> None: | ||
| self.x, self.y = x, y | ||
|
|
||
| @classmethod | ||
| def __infer_type_parameter__(cls, instance): | ||
| return (type(instance.x), type(instance.y)) | ||
| ``` | ||
|
|
||
| `@generic` raises `TypeError` at decoration time if `__infer_type_parameter__` is not defined on the class or any of its ancestors. | ||
|
|
||
| ```{note} | ||
| `@plum.generic` is a **lightweight** opt-in that simply sets `__orig_class__`. For richer parametric machinery — covariant subclassing, custom type-parameter validation, and multi-parameter ordering — see [Parametric Classes](parametric.md). | ||
|
nstarman marked this conversation as resolved.
Outdated
|
||
| ``` | ||
|
|
||
| ## Why this isn't fully automatic | ||
|
|
||
| There are two fundamental limits at play: | ||
|
|
||
| 1. **Python only attaches `__orig_class__` on subscripted construction.** An instance created via `Box(1)` simply does not carry information about `T`, so no runtime introspection can recover it. | ||
| 1. **Beartype validates type parameters by inspecting elements**, which works for containers (`list`, `dict`, etc.) but not for user-defined generic classes whose type-variable usage is opaque to the runtime. | ||
|
|
||
| Rather than guess or silently pick an arbitrary overload, Plum requires you to state your intent explicitly via the `A[Any]` (or bare `A`) fallback overload. | ||
|
|
||
| ## Summary | ||
|
|
||
| | Call | Without `@generic` | With `@generic` | | ||
| | ------------------------- | ---------------------------------------------------------------- | --------------------------------------------------- | | ||
| | `f(Box[int](1))` | `Box[int]` (or `Box[Any]` / `Box` if `Box[int]` not present) | `Box[int]` (same; explicit subscription always wins)| | ||
| | `f(Box[str]("x"))` | `Box[str]` (or `Box[Any]` / `Box` if `Box[str]` not present) | `Box[str]` (same) | | ||
| | `f(Box(1))` | `Box[Any]` or bare `Box` only; `NotFoundLookupError` if absent | `Box[int]` (inferred from `type(1)`) | | ||
|
|
||
| For more advanced parametric-class machinery (covariance, custom type-parameter inference, etc.), see [Parametric Classes](parametric.md). | ||
|
|
||
| (generics-performance)= | ||
| ## Performance | ||
|
|
||
| Generic dispatch carries a small overhead compared to a regular faithful type. The table below is generated each time the documentation is built so the numbers reflect your actual hardware and Python version. | ||
|
|
||
| Two scenarios are measured: | ||
|
|
||
| - **Faithful** — only a bare `B` overload (no type-parameter overloads). Plum uses the fast faithful-cache path. | ||
| - **Generic** — `A[Any]`, `A[int]`, and `A[str]` overloads. Plum uses the two-tier generic cache, keyed on `__orig_class__` when present. | ||
|
|
||
| ```{include} _generated/generics_timing.md | ||
| ``` | ||
|
|
||
| The faithful path is the fastest because Plum caches by `type(arg)` and checks membership with a single `issubclass` call. The generic path must additionally read `__orig_class__` (or detect its absence), build the two-tier cache key, and run the TypeHint subtype check on a cache miss. | ||
|
nstarman marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```{tip} | ||
| If your code only dispatches on the *class* `A` and never needs to distinguish `A[int]` from `A[str]`, declare a bare `A` (or `B` in the example above) overload and omit the parameterized ones. You get the faithful-cache speed with no change to the calling code. | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.