Commit 984849e
committed
Add multi-signal convergence boost, config file penalty, keyword-derived directory globs, and live GLM plan evaluation script
Multi-signal convergence boost (fast-context.ts):
- Track 3 independent signal categories: planGlobMatchedSet, planGrepFileSet, suppGlobMatchedSet
- +3 per signal beyond the first, added to contentScore BEFORE type multiplier
- Rewards files matching plan-glob + plan-grep + supp-glob (convergent evidence)
- Fixes conversation-context can1357#7->#1, MCP #2->#1, git-status #2->#1, worker-host #2->#1
- MRR 0.9444->0.9475 (deterministic, 27 queries). 25/27 at #1 (was 21/22)
Config/data file penalty (fast-context.ts):
- .json/.yaml/.toml/.csv/.svg files get 0.7x type multiplier
- Prevents porcelain.json from outranking git.ts via convergence boost
Keyword-derived directory globs (fast-context.ts):
- Query keywords >=6 chars generate **/${kw}/**/* directory globs
- Catches GT files in named directories (identity/classify.ts, session/session-context.ts)
Hint-system prompt refinement (fast-context-hint-system.md):
- Added glob specificity guidance with GOOD/BAD examples
- Trimmed to reduce token overhead (avoids GLM parse failures from longer prompt)
Live GLM plan evaluation script (bench-fast-context-live-glm.ts):
- Calls real GLM (zai/glm-5-turbo) to generate search plans
- Scores plan quality: parse rate, glob hit rate, grep hit rate, keyword coverage
- Feeds real plans through ranking pipeline for end-to-end MRR with real model plans
- Key finding: MRR drops 0.94->0.80 with real GLM plans — plan quality is the gap
- This is the ONLY benchmark that exercises the hint-system prompt (deterministic benchmark mocks the plan)
- Separated from autoresearch.sh to preserve deterministic contract1 parent 90c2f56 commit 984849e
3 files changed
Lines changed: 466 additions & 14 deletions
File tree
- packages/coding-agent
- scripts
- src
- prompts/tools
- tools
0 commit comments