Evaluator streaming - #2339
Merged
Merged
Conversation
Negabinary
marked this pull request as ready for review
June 12, 2026 14:11
Contributor
Author
|
Guide for reviewing this pr Helpful Background for Reviewing A key change we've made here is to the protocol for discussing back and forth between the webworker and the main thread. You can read these protocol specifications in the web worker server file. The new protocol is as follows:
WorkerServer.re is mostly GPT-generated but should give you an overview of how this works. Because this changes the way the webworker works, I've tried my hardest but wouldn't be surprised if there are weird browser states you can get into here. A functional review where you try ot break it might be good. In addition I've done some focussed human cleanup on EvaluatorState.re and Evaluator.re, these should be significantly more readable now, and I would encourage you to have a read through and see if they match your understanding of how the Evaluator should work. I tried to keep my commits clean so reading through them should also give you a sense for what I've added. |
Negabinary
force-pushed
the
eval-streaming
branch
from
June 12, 2026 15:36
96cfdae to
98312d9
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #2339 +/- ##
==========================================
- Coverage 52.54% 50.90% -1.65%
==========================================
Files 323 315 -8
Lines 47996 41889 -6107
==========================================
- Hits 25219 21322 -3897
+ Misses 22777 20567 -2210
🚀 New features to boost your workflow:
|
Open
disconcision
added a commit
that referenced
this pull request
Jul 2, 2026
fn_def_id is only observable at the Ap step; state is the channel to the Sample minted later. Probe targets only, wiped pre-postMessage; becomes an emitted event under evaluator streaming (#2339). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Negabinary
force-pushed
the
eval-streaming
branch
3 times, most recently
from
July 29, 2026 14:15
4afcff9 to
8398930
Compare
Negabinary
force-pushed
the
eval-streaming
branch
from
July 29, 2026 15:12
8398930 to
01fd8b0
Compare
Negabinary
force-pushed
the
eval-streaming
branch
2 times, most recently
from
July 29, 2026 16:54
c0b74d4 to
7b8c62e
Compare
… Segment fold_right/incomplete_tiles_deep) Displaying a large result (e.g. range(1,5000): ~17k pieces) overflowed the JS stack in stdlib fold_right/concat recursion during reassemble (Aba.split via split_by_matching), regrout_affix, and CachedSyntax's global_missing_shards (incomplete_tiles_deep). Rewrite them on fold_left over reversed lists (order-preserving); recursion now only descends into tile children, bounded by nesting depth.
yielding_step_count counts trampoline transitions (Bind/Next/Done), not evaluator steps; 1M transitions is ~2s of sliced evaluation, so the runaway backstop fired long before the 20s client wall-clock timer and moderate programs (range(1,5000)) died with a spurious 'Evaluation timed out'. Raise the backstop to 100M transitions so it is strictly a last-resort guard beyond the client timer.
Negabinary
force-pushed
the
eval-streaming
branch
from
July 29, 2026 18:02
7b8c62e to
9da202b
Compare
disconcision
added a commit
that referenced
this pull request
Jul 31, 2026
Conflict resolution notes: - EvaluatorState: took dev's appendable-log structure (append/rebase replaces StateSlice capture/replay; StateSlice deleted). iv's app_args-to-app_data rename + stack_frame threading ported into dev's CallStack module: entries are now (call_stack, elided_arg, frame) and lookup_app returns the frame so probe samples on Aps carry the invoked fn's fn_def_id. - CallStack.state.app_data is a shared ref (not threaded): an Ap id re-evaluated via cast distribution (iv's rewrap) must see recordings made during sub-evaluation, which dev's threaded stack discards on the way back up. Restores iv's global-visibility semantics; CallStack.empty is now a function so each run gets a fresh ref. - Evaluator: dev's layered eval_0..5 taken wholesale; frame lookup ported into eval_3_record_probe_sample. - Transition cast-Ap: dev's generated() (temp ids when no probes) combined with iv's rewrap on the inner Ap (id preservation through casts). - EvalResult: dev's streaming rework taken; iv's Sample.Map.finalize applied in dynamics_of_state (covers streaming + final paths). - Action.StepInto now carries CallStack.frame (iv) with dev's type names. - Tests: ported iv-side tests to the new API (EvalInfo.of_targets and of_info_map, eval_incr_with_plan/has_reuse, EvaluatorState.incr_eval). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pr adds streaming to the evaluator. This means results come through and are displayed to the user as the web worker is running.
Screen.Recording.2026-06-12.at.10.06.04.AM.mov
I've also done a lot of work cleaning up the evaluator - this may create some conflicts, but I'm hoping it'll be a lot easier to work with.
This pr commits to making evaluator state write-only (module steps), I think that's fine because we're a pure language.
We also switch the web worker to being always alive, instead of the current regime where we spin up a new web worker for each evaluation.