Skip to content

Commit 85ca8b4

Browse files
committed
Add background claim investigation actions
1 parent 0ae1017 commit 85ca8b4

34 files changed

Lines changed: 1996 additions & 141 deletions

docs/plans/general-page-reader.md

Lines changed: 45 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ session-only multi-page switching are implemented on this branch. The
88
live-DOM review has been summarized in
99
`general-page-reader-quality-findings-2026-07-03-live-dom.md`. Merge-readiness evidence is indexed in
1010
`general-page-reader-merge-readiness.md`.
11-
Last updated: 2026-07-14
11+
Last updated: 2026-07-16
1212

1313
## Decision
1414

@@ -714,21 +714,37 @@ next candidate gate closed; no fresh holdout should be created yet.
714714

715715
### Phase 4: Session-only Claim Investigation
716716

717-
- Status: implementation and fail-closed contract completed on the feature
718-
branch, but candidates v1 and v2 did not clear their private holdout gates and
719-
the v3 development probe did not clear the coverage/runtime-stability boundary.
717+
- Status: the session-only prepared-action vertical slice and its fail-closed
718+
contract are implemented on the feature branch. Candidates v1 and v2 did not
719+
clear their private holdout gates and the v3 development probe did not clear
720+
the coverage/runtime-stability boundary, so this remains unreleased.
720721
- A grounded `claims.q` is preferred; a bounded natural-question fallback from
721722
`claim.c + claim.need` is used only when the model question is missing or
722723
locally rejected. URLs, domains, search-engine instructions, vague references,
723724
and likely compound claims fail closed instead of bypassing the guard.
724-
- `查核選項` only expands a bounded, session-only intent in the current Page or
725-
Focus scope. It does not start the future Truly Agent, open a tab, send
726-
another model request, persist history, or assign a verdict.
727-
- The intent is compiled into two distinct external payloads: concise claim and
728-
source keywords for standard Google Search, and a natural-language evidence
729-
request for Google AI Mode. Copy and original-source actions remain explicit.
730-
Page navigation, reread, a new analysis key, and a new Focus target clear
731-
stale task state.
725+
- When a completed Page or Focus reading contains a candidate claim, the
726+
service worker schedules one lower-priority `derived` adapter request. The
727+
claim stays visible with a compact preparing status, then changes directly to
728+
the prepared question and actions. Abstention, malformed output, a stale
729+
analysis key, or an unavailable side panel quietly falls back to the original
730+
claim. This preparation is ephemeral and never creates durable history.
731+
- Model work shares one resource-aware scheduler: explicit user work is
732+
`user_blocking`, current reading is `foreground`, prepared actions are
733+
`derived`, and speculative work is `prefetch`. Each model resource executes
734+
one request at a time; deduplication, supersession, and a bounded foreground
735+
burst keep Page preparation from starving Feed work without increasing the
736+
number of model calls.
737+
- A prepared intent exposes three explicit actions: standard Google Search,
738+
Google AI Mode, and copy. Standard Search receives concise claim/source
739+
keywords only. AI Mode receives a natural-language evidence request and may
740+
receive the current HTTP(S) URL as metadata; the URL is never treated as
741+
evidence or copied into model output. The former original-source action is
742+
intentionally absent because it duplicated the page the user is already on.
743+
- The adapter requires an exact `sourceQuote` grounding span, preserves source
744+
language for the atomic claim/question, tolerates harmless schema-version and
745+
optional-attribution drift, and applies the existing deterministic eligibility
746+
guard after model output. Page navigation, reread, a new analysis key, and a
747+
new Focus target clear stale task state; Page and Focus keep separate slots.
732748
- The future Truly Agent uses a separate non-runtime semantic Case draft. The
733749
model selects document families, source roles, authority hints, and numbered
734750
question coverage; local code owns IDs, question linkage, verification
@@ -738,14 +754,23 @@ next candidate gate closed; no fresh holdout should be created yet.
738754

739755
### Phase 5: Runtime and UX Gate
740756

741-
- Status: implementation verification completed; product-quality holdout gates
742-
failed for candidates v1 and v2, while v3 remains a development-only probe,
743-
so the investigation action remains unreleased.
744-
- Focused unit coverage validates query sanitization, deterministic fallback,
745-
fail-closed eligibility, Page/Focus state isolation, and the two-step UI.
746-
- The CDP UI audit validates that preparing a task opens no browser target and
747-
captures the expanded card at 430px alongside Page, Focus, screenshot
748-
recovery, loading, and ready states.
757+
- Status: implementation and live UX verification completed; product-quality
758+
holdout gates failed for candidates v1 and v2, while v3 remains a
759+
development-only probe, so the investigation action remains unreleased.
760+
- Focused unit coverage validates scheduler priority/fairness, adapter parsing
761+
and grounding, query sanitization, deterministic fallback, fail-closed
762+
eligibility, Page/Focus race isolation, and the automatic
763+
preparing-to-ready/fallback transitions. The 180 ms height/fade transition is
764+
skipped for reduced motion and never delays the underlying session update.
765+
- A 2026-07-16 no-focus CDP check used dev build
766+
`1784200373031-0ae1017-dirty` on a real Financial Times page. It observed an
767+
automatic `preparing -> ready` transition with no manual click, no redundant
768+
label, and Google Search / Gemini / Copy actions at 430 px. A separate live
769+
run exercised `preparing -> fallback`, confirming that unavailable model
770+
output clears the loading state and restores the original claim.
771+
- Review screenshots remain local-only:
772+
`/private/tmp/truly-auto-investigation-preparing-430-2026-07-16.png` and
773+
`/private/tmp/truly-auto-investigation-ready-430-2026-07-16.png`.
749774
- Real-content paired audit artifacts remain private under `tmp/`; only
750775
anonymized aggregate findings may be copied into tracked documentation.
751776

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@
9595
"audit:general-page-model-integration": "vitest run tests/audit/general-page-model-integration-audit.test.ts",
9696
"audit:release-bundle": "node scripts/audit-release-bundle.mjs --dist dist",
9797
"test:contract:public": "vitest run tests/contract/current-region-targeting-contract.test.ts tests/contract/general-page-analysis-contract.test.ts tests/contract/general-page-extraction-contract.test.ts tests/contract/general-page-model-context-contract.test.ts tests/contract/general-page-parser-advisor-contract.test.ts tests/contract/model-response-contract.test.ts tests/contract/post-context-boundary-contract.test.ts tests/contract/reading-action-contract.test.ts tests/contract/zhtw-segmentation-contract.test.ts",
98-
"test:gpr": "vitest run tests/audit/general-page-model-integration-audit.test.ts tests/contract/current-region-targeting-contract.test.ts tests/contract/general-page-analysis-contract.test.ts tests/contract/general-page-extraction-contract.test.ts tests/contract/general-page-model-context-contract.test.ts tests/contract/general-page-parser-advisor-contract.test.ts tests/contract/model-response-contract.test.ts tests/contract/reading-action-contract.test.ts tests/unit/cdp-client.test.mjs tests/unit/web-focus-continuity-scenario.test.mjs tests/unit/meaningful-navigation-scenario.test.mjs tests/unit/dev-build-freshness.test.mjs tests/unit/general-page-audit-runtime-reload.test.mjs tests/unit/general-page-host-permission.test.ts tests/unit/general-page-real-world-sanitizer.test.mjs tests/unit/private-general-page-eval.test.mjs tests/unit/i18n.test.ts tests/unit/i18n-hardcoded-strings.test.ts tests/unit/page-reader-content-script.test.ts tests/unit/page-reader-tab-transport.test.ts tests/unit/page-reading-analysis-coordinator.test.ts tests/unit/page-reading-export.test.ts tests/unit/page-reading-presentation.test.ts tests/unit/page-reading-runtime.test.ts tests/unit/page-reading-session.test.ts tests/unit/page-readability.test.ts tests/unit/page-url-identity.test.ts tests/unit/reading-brief-question-list.test.ts tests/unit/reading-command-envelope.test.ts tests/unit/runtime-message.test.ts tests/unit/screenshot-data-url.test.ts",
98+
"test:gpr": "vitest run tests/audit/general-page-model-integration-audit.test.ts tests/contract/current-region-targeting-contract.test.ts tests/contract/general-page-analysis-contract.test.ts tests/contract/general-page-extraction-contract.test.ts tests/contract/general-page-model-context-contract.test.ts tests/contract/general-page-parser-advisor-contract.test.ts tests/contract/model-response-contract.test.ts tests/contract/reading-action-contract.test.ts tests/unit/cdp-client.test.mjs tests/unit/web-focus-continuity-scenario.test.mjs tests/unit/meaningful-navigation-scenario.test.mjs tests/unit/dev-build-freshness.test.mjs tests/unit/general-page-audit-runtime-reload.test.mjs tests/unit/general-page-host-permission.test.ts tests/unit/general-page-investigation-adapter.test.ts tests/unit/general-page-investigation-background.test.ts tests/unit/general-page-real-world-sanitizer.test.mjs tests/unit/model-work-scheduler.test.ts tests/unit/private-general-page-eval.test.mjs tests/unit/i18n.test.ts tests/unit/i18n-hardcoded-strings.test.ts tests/unit/page-claim-investigation.test.ts tests/unit/page-reader-content-script.test.ts tests/unit/page-reader-tab-transport.test.ts tests/unit/page-reading-analysis-coordinator.test.ts tests/unit/page-reading-export.test.ts tests/unit/page-reading-presentation.test.ts tests/unit/page-reading-runtime.test.ts tests/unit/page-reading-session.test.ts tests/unit/page-readability.test.ts tests/unit/page-url-identity.test.ts tests/unit/reading-brief-question-list.test.ts tests/unit/reading-command-envelope.test.ts tests/unit/runtime-message-router.test.ts tests/unit/runtime-message.test.ts tests/unit/screenshot-data-url.test.ts",
9999
"test:unit:public": "vitest run tests/unit/cdp-client.test.mjs tests/unit/web-focus-continuity-scenario.test.mjs tests/unit/meaningful-navigation-scenario.test.mjs tests/unit/cdp-page-source.test.mjs tests/unit/dev-build-freshness.test.mjs tests/unit/feed-boundary.test.ts tests/unit/general-page-audit-runtime-reload.test.mjs tests/unit/general-page-host-permission.test.ts tests/unit/general-page-real-world-sanitizer.test.mjs tests/unit/private-general-page-eval.test.mjs tests/unit/heads-up-chip-policy.test.ts tests/unit/heads-up-panel-status.test.ts tests/unit/i18n.test.ts tests/unit/i18n-hardcoded-strings.test.ts tests/unit/investigation-actions-renderer.test.ts tests/unit/logging-policy.test.ts tests/unit/model-display.test.ts tests/unit/model-source-config.test.ts tests/unit/ollama-cloud-capabilities.test.ts tests/unit/openai-api-key-mock.test.ts tests/unit/page-reader-content-script.test.ts tests/unit/page-reader-tab-transport.test.ts tests/unit/page-reading-analysis-coordinator.test.ts tests/unit/page-reading-export.test.ts tests/unit/page-reading-presentation.test.ts tests/unit/page-reading-runtime.test.ts tests/unit/page-reading-session.test.ts tests/unit/page-readability.test.ts tests/unit/page-url-identity.test.ts tests/unit/provider-capabilities.test.ts tests/unit/reading-brief-question-list.test.ts tests/unit/reading-command-envelope.test.ts tests/unit/reading-question-policy.test.ts tests/unit/request-auth.test.ts tests/unit/runtime-message.test.ts tests/unit/screenshot-data-url.test.ts tests/unit/snapshot-redaction.test.ts tests/unit/sponsored-memory.test.ts tests/unit/ssr-cache-bridge.test.ts tests/unit/tabs.test.ts tests/unit/theme-mode.test.ts tests/unit/tier-b-vision-probe.test.ts tests/unit/trusted-model-runtime.test.ts",
100100
"test:unit:watch": "vitest",
101101
"check:public": "npm run check:public-boundary && npm run check:release-metadata && npm run check:general-page && npm run check:type && npm run test:contract:public && npm run test:unit:public && npm run test:gpr:investigation && npm run build && npm run audit:release-bundle",

scripts/audit-general-page-reader.mjs

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,8 @@ async function startMockOpenAiEndpoint() {
328328
const hasImageUrl = JSON.stringify(userContent).includes('"image_url"');
329329
const kind = /parser recovery classifier/i.test(systemText)
330330
? "parser-advisor"
331+
: /prepare one candidate fact-check action/i.test(systemText)
332+
? "investigation-adapter"
331333
: hasImageUrl
332334
? "screenshot-brief"
333335
: /dominant color/i.test(systemText)
@@ -355,6 +357,20 @@ async function startMockOpenAiEndpoint() {
355357
riskTags: ["needs_visual_grounding"],
356358
rationale: "The synthetic fixture needs visible screenshot grounding.",
357359
});
360+
} else if (kind === "investigation-adapter") {
361+
content = JSON.stringify({
362+
schemaVersion: 1,
363+
decision: "prepared",
364+
reason: "actionable",
365+
claim: {
366+
c: "The analyzed content is synthetic.",
367+
why: "The UI check must not depend on live page content.",
368+
need: "Confirm the expected scope.",
369+
q: "Is the analyzed content synthetic?",
370+
atom: { s: "The analyzed content", p: "is", o: "synthetic" },
371+
policy: { claimKind: "fact", consequence: "public_interest" },
372+
},
373+
});
358374
} else {
359375
const targetKind = /targetKind:\s*selection/i.test(userText)
360376
? "selection"
@@ -1458,6 +1474,12 @@ async function auditSuccessfulRead(extensionId, allowedBase) {
14581474
})()`);
14591475

14601476
const pageBrief = await observePageBrief(side, "page-analysis-ready.png");
1477+
await waitFor(
1478+
side,
1479+
`Boolean(document.querySelector('#page-pane .page-claim-start'))`,
1480+
5000,
1481+
"background claim investigation preparation",
1482+
);
14611483
const claimInvestigation = await observeClaimInvestigation(side);
14621484
const initialLoadTimeline = await side.evaluateJson(`(() => {
14631485
const timeline = globalThis.__trulyPagePaneTimeline;
@@ -3068,7 +3090,10 @@ function claimActionPayloadContract(result) {
30683090
aiModeUrl.searchParams.get("udm") === "50" &&
30693091
standardQuery && aiModePrompt && standardQuery !== aiModePrompt &&
30703092
!/Please verify this claim|/u.test(standardQuery) &&
3071-
/Evidence needed|/u.test(aiModePrompt)
3093+
!/https?:\/\//u.test(standardQuery) &&
3094+
/Evidence needed|/u.test(aiModePrompt) &&
3095+
/Source URL \(metadata\)|metadata/u.test(aiModePrompt) &&
3096+
/127\.0\.0\.1/u.test(aiModePrompt)
30723097
),
30733098
standardQueryLength: standardQuery.length,
30743099
aiModePromptLength: aiModePrompt.length,

scripts/lib/private-general-page-eval.mjs

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@ import fs from "node:fs";
22
import path from "node:path";
33

44
export const PRIVATE_EVAL_SURFACES = ["facebook", "news"];
5+
const SOURCE_CONTEXT_LIMITS = {
6+
title: 100,
7+
sourceName: 60,
8+
publishedAt: 32,
9+
};
10+
const SOURCE_CONTEXT_ARTIFACT_RE = /https?:\/\/|\[[^\]]+\]\([^\)]+\)|(?:^|\s)(?:curl|wget|npm|pnpm|brew|git)\s/i;
511

612
export function parsePrivateEvalJsonl(text) {
713
return String(text)
@@ -45,6 +51,31 @@ export function privateEvalInputErrors(rows, expectedCount, declaredCategories)
4551
if (!['zh-TW', 'en'].includes(row.language)) errors.push(`${label}: invalid language`);
4652
if (typeof row.text !== "string" || row.text.trim().length < 80 || row.text.length > 12000) errors.push(`${label}: text must be 80-12000 characters`);
4753
if (typeof row.sourceSha256 !== "string" || !/^[a-f0-9]{64}$/.test(row.sourceSha256)) errors.push(`${label}: invalid sourceSha256`);
54+
if (row.sourceContext !== undefined) {
55+
if (!row.sourceContext || typeof row.sourceContext !== "object" || Array.isArray(row.sourceContext)) {
56+
errors.push(`${label}: sourceContext must be an object`);
57+
} else {
58+
for (const [key, limit] of Object.entries(SOURCE_CONTEXT_LIMITS)) {
59+
const value = row.sourceContext[key];
60+
if (value === undefined) continue;
61+
if (typeof value !== "string" || !value.trim() || value.length > limit || SOURCE_CONTEXT_ARTIFACT_RE.test(value)) {
62+
errors.push(`${label}: invalid sourceContext.${key}`);
63+
}
64+
}
65+
if (row.sourceContext.url !== undefined) {
66+
try {
67+
const url = new URL(row.sourceContext.url);
68+
if (!/^https?:$/.test(url.protocol) || url.username || url.password || row.sourceContext.url.length > 320) {
69+
errors.push(`${label}: invalid sourceContext.url`);
70+
}
71+
} catch {
72+
errors.push(`${label}: invalid sourceContext.url`);
73+
}
74+
}
75+
const unknownKeys = Object.keys(row.sourceContext).filter((key) => !(key in SOURCE_CONTEXT_LIMITS) && key !== "url");
76+
if (unknownKeys.length > 0) errors.push(`${label}: unsupported sourceContext fields`);
77+
}
78+
}
4879
}
4980
if ([...actual].some((category) => !declared.has(category)) || [...declared].some((category) => !actual.has(category))) {
5081
errors.push(`data categories mismatch: declared ${[...declared].sort().join(",")}; actual ${[...actual].sort().join(",")}`);

scripts/private-general-page-eval-entry.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,12 @@ interface InputRow {
2727
language: "zh-TW" | "en";
2828
sourceSha256: string;
2929
text: string;
30+
sourceContext?: {
31+
title?: string;
32+
sourceName?: string;
33+
publishedAt?: string;
34+
url?: string;
35+
};
3036
}
3137

3238
function option(name: string, fallback?: string): string | undefined {
@@ -133,6 +139,7 @@ async function evaluateRow(row: InputRow) {
133139
claimIndex: 0,
134140
claim,
135141
groundingText: row.text,
142+
source: row.sourceContext,
136143
}) : undefined;
137144
return {
138145
schemaVersion: 1,
@@ -149,6 +156,8 @@ async function evaluateRow(row: InputRow) {
149156
eligibilityReason: eligibility && !eligibility.ok ? eligibility.reason : undefined,
150157
questionSource: task ? (modelQuestion ? "model" : "deterministic_fallback") : "none",
151158
question: task?.intent.question,
159+
googleKeywords: task?.googleKeywords,
160+
aiModePrompt: task?.aiModePrompt,
152161
},
153162
raw: response.raw,
154163
};

0 commit comments

Comments
 (0)