You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(agent-forge): update reviewer guide and final proof pack with latest VM proof details
- Enhanced `AGENTFORGE-REVIEWER-GUIDE.md` to include the latest VM proof results from 2026-05-03, detailing the status of local and Tier 2/Tier 4 evaluations.
- Revised `FINAL-PROOF-PACK.md` to document the latest VM artifacts and clarify the distinction between checked-in and VM-only results, emphasizing the need to attach these artifacts to the final submission.
- Added instructions for running commands within the OpenEMR container to ensure proper environment configuration.
These updates improve the clarity and completeness of the documentation, ensuring accurate representation of the latest evaluation results and submission requirements.
Copy file name to clipboardExpand all lines: AGENTFORGE-REVIEWER-GUIDE.md
+11-1Lines changed: 11 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -77,7 +77,15 @@ Current checked-in proof snapshot:
77
77
| Deployed latency trace | VM artifact `/root/repos/openemr/agent-forge/eval-results/deployed-latency-trace-20260503-190443.json`; `agent-forge/docs/operations/LATENCY-RESULTS.md`| A1c 20/20, p95 `3212 ms`; visit briefing 20/20, p95 `8309 ms`; both under `10000 ms` demo budget |
78
78
| Deployed browser proof pack |`agent-forge/docs/submission/browser-proof/`| Four attached browser proof screenshots for A1c trend, visit briefing, missing microalbumin, and clinical-advice refusal |
79
79
80
-
Additional later VM proof was captured after the checked-in Tier 2/Tier 4 summaries, but the corresponding JSON files are not present in this checkout. The checked-in summaries above are the local artifacts a reviewer can inspect directly from the repository.
80
+
Latest VM proof supplied on 2026-05-03 after the checked-in summaries:
81
+
82
+
| Check | Latest VM artifact or result | Status |
83
+
| --- | --- | --- |
84
+
| Local AgentForge gate from a normal VM shell |`agent-forge/scripts/check-local.sh`; deterministic eval artifact `/root/repos/openemr/agent-forge/eval-results/eval-results-20260503-201548.json`| PASS; PHP syntax, shell syntax, isolated PHPUnit `298 tests / 1547 assertions`, deterministic evals `32 passed, 0 failed`, PHPStan `161/161`, PHPCS no changed AgentForge PHP files |
| Tier 4 deployed HTTP/session/audit smoke from the VM |`/root/repos/openemr/agent-forge/eval-results/deployed-smoke-20260503-201547.json`| 5 passed, 0 failed; aggregate latency `14734 ms`; audit assertions enabled; includes `tier4_visit_briefing_live_verified`; code version `6769aa908887`|
87
+
88
+
The checked-in summaries above are the local artifacts a reviewer can inspect directly from the repository. The later VM artifacts should be attached to the final submission packet if possible.
81
89
82
90
To reproduce the deployed smoke proof from the VM host:
If the current shell is already inside `agent-forge/`, run `php scripts/run-deployed-smoke.php` instead.
102
+
93
103
Browser proof request ids from the later deployed UI proof pass: A1c `7cf183f7-5607-403e-9559-e2689a0769aa`, visit briefing `bbbddd92-df71-4835-951b-f14279abe18c`, dosing refusal `ee2fe6c2-56cc-47ac-8731-a3fd885ad9e3`, missing microalbumin `e4ca6da4-9cd9-4222-a9c3-06651098fb49`, and cross-patient refusal `7489b25d-2af1-42d8-9c04-ec7ee3166dbc`.
94
104
95
105
Browser proof files are stored under `agent-forge/docs/submission/browser-proof/`, with request ids listed in `agent-forge/docs/submission/FINAL-PROOF-PACK.md` and `agent-forge/docs/submission/browser-proof/MANIFEST.md`. The stale/cross-patient conversation boundary is covered by the deployed smoke runner and should return HTTP 403 with `tools_called=[]` and `verifier_result=not_run`.
Copy file name to clipboardExpand all lines: agent-forge/docs/submission/FINAL-PROOF-PACK.md
+20-1Lines changed: 20 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,12 +27,24 @@ The proof rows below now distinguish checked-in repository artifacts from later
27
27
28
28
Additional later VM-only Tier 2 and Tier 4 runs were captured after the checked-in summaries, but their JSON artifacts are not present in this checkout. The table above is the synchronized repository-local proof set.
29
29
30
+
Latest VM proof supplied on 2026-05-03:
31
+
32
+
| Check | Artifact | Result |
33
+
| --- | --- | --- |
34
+
| Local AgentForge gate from a normal VM shell |`agent-forge/scripts/check-local.sh`; deterministic eval artifact `/root/repos/openemr/agent-forge/eval-results/eval-results-20260503-201548.json`| PASS; PHP syntax, shell syntax, isolated PHPUnit `298 tests / 1547 assertions`, deterministic evals `32 passed, 0 failed`, PHPStan `161/161`, PHPCS no changed AgentForge PHP files |
35
+
| Tier 2 live model from OpenEMR container |`/var/www/localhost/htdocs/openemr/agent-forge/eval-results/tier2-live-20260503-202550.json`| 14 passed, 0 failed; tokens in/out `5943/2476`; estimated cost `$0.015599`; provider `openai/gpt-5.4-mini`|
36
+
| Tier 4 deployed smoke from VM |`/root/repos/openemr/agent-forge/eval-results/deployed-smoke-20260503-201547.json`| 5 passed, 0 failed; aggregate latency `14734 ms`; audit assertions enabled; includes `tier4_visit_briefing_live_verified`; code version `6769aa908887`|
37
+
38
+
Attach these VM/container JSON artifacts to the final submission packet if possible; they are referenced here because they are not present in this local checkout.
39
+
30
40
The checked-in Tier 4 smoke command was run against the deployed app on 2026-05-03:
31
41
32
42
```sh
33
43
php agent-forge/scripts/run-deployed-smoke.php
34
44
```
35
45
46
+
If the current shell is already inside `agent-forge/`, run `php scripts/run-deployed-smoke.php` instead.
47
+
36
48
## Deployed Browser Proof
37
49
38
50
Manual browser screenshots are attached under `agent-forge/docs/submission/browser-proof/` for the four deployed UI scenarios below. They were captured on 2026-05-03 against fake patient `900001 / AF-DEMO-900001` only.
@@ -70,7 +82,12 @@ Live model tier:
70
82
php agent-forge/scripts/run-tier2-evals.php
71
83
```
72
84
73
-
This requires provider credentials configured out of band.
85
+
This requires provider credentials configured out of band. If host PHP reports that Tier 2 is using the fixture provider, run the command inside the OpenEMR container so Docker Compose supplies the configured provider environment:
0 commit comments