Skip to content

[JuliaLowering] Implement static parameter capture - #62374

Merged
topolarity merged 10 commits into
JuliaLang:masterfrom
mlechu:jl-sparam-capture
Jul 15, 2026
Merged

[JuliaLowering] Implement static parameter capture#62374
topolarity merged 10 commits into
JuliaLang:masterfrom
mlechu:jl-sparam-capture

Conversation

@mlechu

@mlechu mlechu commented Jul 13, 2026

Copy link
Copy Markdown
Member

Static parameter capture is (as far as I know) the last non-trivial, non-performance piece of
JuliaLowering that doesn't match flisp yet. I kind of understand why it was
left unimplemented. In flisp lowering:

  • Scope resolution for static parameters in method signatures is more-or-less
    done in desugaring
  • We need two new variables for every sparam: the sparam usable in the lambda
    body, and the method-signature typevar at top level
  • Capturing an sparam into an inner method signature means we need to add
    somthing to the method definition's typevar list
  • Capturing an sparam into an inner lambda body means we need to add to both the
    inner lambda's sparams as well as the inner method definition's typevar list
  • Defining and instantiating the closure type also requires knowing what static
    parameters to feed into the apply_type call
  • Captures are only known after variable analysis but need to change
    desugaring's decisions, so:
    • The typevar list is constructed in desugaring, then the core svec
      expression is re-constructed to contain sparam captures in closure
      conversion
    • analyze-vars-lambda mutates the lambda's sparam list

Static param capture into the lambda body in JuliaLowering happens to work today
because it's treated the same as a never-boxed local, so it's just a struct
field. I assumed this might cause inference issues, but admittedly didn't check.
The main issue is that capture into signatures doesn't work
(JuliaLang/JuliaLowering.jl#140).

This PR:

  • Don't do the flisp renaming-sparams-early step, since it just makes the hard
    parts of this harder.
  • Change the method form out of desugaring to be (method argtype_svec lambda) and fully construct the svec after its contents are known
  • Change the method_defs form to be (method_defs id typevar_list body),
    because typevar_list should always precede lifted stuff in closure conversion,
    and because special resolution needs to happen when each (typevar a b) gets
    resolved (resolve b first, then declare a).
    • This is kind of a hack, but I didn't want deeply-nested scope blocks. These
      were previously declared as locals, which had the wrong scope resolution
      (they could all see each other), and this was a pkgeval problem.
  • Add new binding kind :typevar. This behaves like a local up until
    before linearization, and more like an ssavalue after that.
  • In scope resolution, we collect the mapping from static parameter to typevar,
    so that inner method signatures may resolve successfully (and even re-use the
    original typevar)

This change also implements "underscore sparams are readable" #60626 and ports
#57928 (sparam capture in opaque closure is more like capture of locals). It
does not fix the pkgeval bug @topolarity reported to me with same-named
functions that require different closure types.

I've also added a cleaned-up Claude fix for typevars that depend on each other
in their bounds. Not sure if it's the best way of doing this. There's an easy
way out if this approach doesn't work (flisp just creates typevars <:Any in
closures).

The real change isn't as big as the diff implies, it just changes the closure
type creation near the beginning of the IR in many tests, so ends up changing a
lot of IR numbering.

@mlechu
mlechu requested a review from topolarity July 13, 2026 23:54
@mlechu
mlechu force-pushed the jl-sparam-capture branch 2 times, most recently from af68d22 to a0a065e Compare July 14, 2026 00:04
@topolarity

Copy link
Copy Markdown
Member

Counter-examples, courtesy of you-know-who:

# 1. Bound references the loop variable → UndefVarError at load time
for x in (Int, Float64)
    global f
    f(y::T) where {T<:x} = T
end
# flisp: works, defines both methods.  This PR: UndefVarError(:x)

# 2. Bound references a global reassigned in the loop → silently stale bound
global b = Int
for i in 1:2
    global g, b
    g(y::T) where {T<:b} = (i, T)
    b = Float64
end
g(1.5)   # flisp: (2, Float64).  This PR: MethodError (bound stuck at Int)

seems that TypeVars can end up hoisted even when the Method def they accompany is not

@topolarity topolarity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few corners to solve, but largely this change looks quite good to me.

Nice that we were able to recover quite a bit of parity with flisp AFAICT, despite the difference in implementation.

Comment thread JuliaLowering/src/scope_analysis.jl Outdated
Comment thread JuliaLowering/src/compat.jl
Comment thread JuliaLowering/src/kinds.jl
Comment thread JuliaLowering/src/scope_analysis.jl
b = get_binding(ctx, bid)
if b.kind === :typevar # only visible to lifted scopes
ctx.scopes[ctx.scope_stack[end]].is_lifted || continue
end

@topolarity topolarity Jul 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

semi-exotic case:

global TE = Int
fE(x, y=((z::TE)->z)) where TE = y
fE(1.)(2)

prints 2 in flisp

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to match flisp, though I think this behaviour probably isn't what we want. Left a comment above the test case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree w.r.t. questionable-ness

@mlechu
mlechu force-pushed the jl-sparam-capture branch from a0a065e to 27e0cca Compare July 15, 2026 01:55
Comment thread JuliaLowering/src/runtime.jl Outdated

@topolarity topolarity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work as usual @mlechu

@topolarity
topolarity merged commit 8399119 into JuliaLang:master Jul 15, 2026
9 checks passed
@mlechu mlechu linked an issue Jul 15, 2026 that may be closed by this pull request
aviatesk added a commit to aviatesk/JETLS.jl that referenced this pull request Jul 17, 2026
JuliaLowering’s static-parameter capture changes (JuliaLang/julia#62374)
method-definition lowering and introduces distinct `:typevar` bindings.
Without matching updates, closure rewriting assumes the old method
shape, diagnostics use obsolete context fields, and binding-based
language features can miss or misclassify method type parameters.

Update JuliaSyntax and JuliaLowering to `8399119aa7`, adapt
`Closure2Opaque` to the separated typevar setup and method body, and
use `sp_typevars` and `tv_deps` for static-parameter diagnostics.
Handle `:typevar` bindings across cursor selection, completion, hover,
semantic tokens, and occurrence analysis, preferring the typevar binding
at a shared `where` binder source.

Generic and generated local methods remain on the conservative
synthetic-closure path. TypeAnnotation now covers outer static-parameter
capture at closure call sites, while signature-view body inference still
cannot recover static-parameter bounds from thunk method instances.

Co-authored-by: GPT-5.6 Sol <noreply@openai.com>
topolarity added a commit to topolarity/julia that referenced this pull request Jul 17, 2026
Upstream's static-parameter capture redesign (JuliaLang#62374)
collapsed the two top-level sequencing notions in closure conversion
into a single flag that remains set inside loop bodies. Closure type
definitions and method statements consequently stayed inside top-level
loops, so every iteration re-ran `Core._structtype` and rebound the
closure type name to a fresh DataType: any downstream code specialized
on the previous iteration's type was invalidated and recompiled, an
O(iterations) recompilation storm. ReverseDiff, whose array-derivative
tables evaluate comprehension closures inside a ~200-iteration
top-level `@eval` loop, paid 9.7x on precompile (126s vs flisp's 13s).

Restore the distinction: statement-sequence constructs (if, elseif,
block, try) preserve top-level sequencing, while loops reset it, so
closure types and their methods hoist to before the loop -- defined
once per site, exactly as flisp lowers them -- and only instance
construction remains per-iteration. The typevar-setup hoist is gated to
closures: a global method defined in a loop with a loop-dependent
typevar bound must keep its signature evaluation in place.

ReverseDiff returns to parity (14.4s, 1.06x flisp); per-iteration
capture semantics are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parts of static param capture unimplemented

2 participants