Skip to content

Commit 984849e

Browse files
committed
Add multi-signal convergence boost, config file penalty, keyword-derived directory globs, and live GLM plan evaluation script
Multi-signal convergence boost (fast-context.ts): - Track 3 independent signal categories: planGlobMatchedSet, planGrepFileSet, suppGlobMatchedSet - +3 per signal beyond the first, added to contentScore BEFORE type multiplier - Rewards files matching plan-glob + plan-grep + supp-glob (convergent evidence) - Fixes conversation-context can1357#7->#1, MCP #2->#1, git-status #2->#1, worker-host #2->#1 - MRR 0.9444->0.9475 (deterministic, 27 queries). 25/27 at #1 (was 21/22) Config/data file penalty (fast-context.ts): - .json/.yaml/.toml/.csv/.svg files get 0.7x type multiplier - Prevents porcelain.json from outranking git.ts via convergence boost Keyword-derived directory globs (fast-context.ts): - Query keywords >=6 chars generate **/${kw}/**/* directory globs - Catches GT files in named directories (identity/classify.ts, session/session-context.ts) Hint-system prompt refinement (fast-context-hint-system.md): - Added glob specificity guidance with GOOD/BAD examples - Trimmed to reduce token overhead (avoids GLM parse failures from longer prompt) Live GLM plan evaluation script (bench-fast-context-live-glm.ts): - Calls real GLM (zai/glm-5-turbo) to generate search plans - Scores plan quality: parse rate, glob hit rate, grep hit rate, keyword coverage - Feeds real plans through ranking pipeline for end-to-end MRR with real model plans - Key finding: MRR drops 0.94->0.80 with real GLM plans — plan quality is the gap - This is the ONLY benchmark that exercises the hint-system prompt (deterministic benchmark mocks the plan) - Separated from autoresearch.sh to preserve deterministic contract
1 parent 90c2f56 commit 984849e

3 files changed

Lines changed: 466 additions & 14 deletions

File tree

0 commit comments

Comments
 (0)