Skip to content

[aw] Failure Investigator (6h) #497

[aw] Failure Investigator (6h)

[aw] Failure Investigator (6h) #497

Triggered via schedule August 18, 2026 13:03
Status Success
Total duration 8m 59s
Artifacts 10
Fit to window
Zoom out
Zoom in

Annotations

3 errors
agent
`, timeout, or auth message anywhere in the captured tail.\n\n### Probable root cause\n\nA silent/deferred failure — likely a non-zero exit code or a downstream safe-outputs validation check that fires *after* the visible Copilot CLI stream ends, since housekeeping still completes normally. Because this hit 3 independent workflows same-day, treat as infra-level (shared Copilot CLI wrapper/harness), not workflow-specific logic.\n\n### Proposed remediation\n\n1. Instrument the Copilot CLI wrapper to emit an explicit exit-code/error line on termination — a non-zero exit with a clean-looking log is currently indistinguishable from success.\n2. Audit the safe-outputs post-processing step that runs after the visible Copilot CLI stream ends for a swallowed exception or silent validation failure.\n3. Prioritize the shared invocation code path over per-workflow logic, given 3 unrelated workflows failed identically the same day.\n\n### Success criteria\n\n`Execute GitHub Copilot CLI` failures surface a specific, non-empty error/exit-code in logs; the three affected workflows either pass or fail with a diagnosable cause, and their traffic-volume profile matches known-good runs.\n\n**Parent:** #53129\nRelated to #53129\n\n\n\u003c!-- gh-aw-tracker-id: aw-failure-investigator --\u003e\n\n\u003e Generated by [🔍 [aw] Failure Investigator (6h)](https://github.com/github/gh-aw/actions/runs/32109525808) · agent · 251.3 AIC · ⌖ 6.96 AIC · ⊞ 5.9K · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Faw-failure-investigator%22\u0026type=issues)\n\u003e - [x] expires \u003c!-- gh-aw-expires: 2026-08-25T07:21:03.164Z --\u003e on Aug 24, 2026, 11:21 PM UTC-08:00\n\n\u003c!-- gh-aw-agentic-workflow: [aw] Failure Investigator (6h), gh-aw-tracker-id: aw-failure-investigator, engine: claude, model: agent, id: 32109525808, workflow_id: aw-failure-investigator, run: https://github.com/github/gh-aw/actions/runs/32109525808 --\u003e\n\n\u003c!-- gh-aw-workflow-id: aw-failure-investigator --\u003e\n\u003c!-- gh-aw-workflow-call-id: github/gh-aw/aw-failure-investigator --\u003e","id":"I_kwDOPc1QR88AAAABNLRnpg","number":53620,"title":"[aw-failures] [P1] Execute GitHub Copilot CLI silently exits early — 3 unrelated workflows same day","updatedAt":"2026-08-18T11:49:03Z"}}}}}
agent
`, timeout, or auth message anywhere in the captured tail.\\n\\n### Probable root cause\\n\\nA silent/deferred failure — likely a non-zero exit code or a downstream safe-outputs validation check that fires *after* the visible Copilot CLI stream ends, since housekeeping still completes normally. Because this hit 3 independent workflows same-day, treat as infra-level (shared Copilot CLI wrapper/harness), not workflow-specific logic.\\n\\n### Proposed remediation\\n\\n1. Instrument the Copilot CLI wrapper to emit an explicit exit-code/error line on termination — a non-zero exit with a clean-looking log is currently indistinguishable from success.\\n2. Audit the safe-outputs post-processing step that runs after the visible Copilot CLI stream ends for a swallowed exception or silent validation failure.\\n3. Prioritize the shared invocation code path over per-workflow logic, given 3 unrelated workflows failed identically the same day.\\n\\n### Success criteria\\n\\n`Execute GitHub Copilot CLI` failures surface a specific, non-empty error/exit-code in logs; the three affected workflows either pass or fail with a diagnosable cause, and their traffic-volume profile matches known-good runs.\\n\\n**Parent:** #53129\\nRelated to #53129\\n\\n\\n<!-- gh-aw-tracker-id: aw-failure-investigator -->\\n\\n> Generated by [🔍 [aw] Failure Investigator (6h)](https://github.com/github/gh-aw/actions/runs/32109525808) · agent · 251.3 AIC · ⌖ 6.96 AIC · ⊞ 5.9K · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Faw-failure-investigator%22&type=issues)\\n> - [x] expires <!-- gh-aw-expires: 2026-08-25T07:21:03.164Z --> on Aug 24, 2026, 11:21 PM UTC-08:00\\n\\n<!-- gh-aw-agentic-workflow: [aw] Failure Investigator (6h), gh-aw-tracker-id: aw-failure-investigator, engine: claude, model: agent, id: 32109525808, workflow_id: aw-failure-investigator, run: https://github.com/github/gh-aw/actions/runs/32109525808 -->\\n\\n<!-- gh-aw-workflow-id: aw-failure-investigator -->\\n<!-- gh-aw-workflow-call-id: github/gh-aw/aw-failure-investigator -->\",\"number\":53620,\"title\":\"[aw-failures] [P1] Execute GitHub Copilot CLI silently exits early — 3 unrelated workflows same day\",\"updatedAt\":\"2026-08-18T11:49:03Z\"}","is_error":false}]},"parent_tool_use_id":null,"session_id":"f8f276ed-dc38-40bf-a1bf-3866de5e1e24","uuid":"d6b60c09-8410-48f6-ab1e-80cc18598a74","timestamp":"2026-08-18T13:08:23.750Z","tool_use_result":{"stdout":"{\"body\":\"### Fix: `Execute GitHub Copilot CLI` step silently exits early across 3 unrelated workflows\\n\\nFix the Copilot CLI execution wrapper — it silently truncates most of its egress traffic and exits early with no visible error, hitting 3 unrelated workflows the same day.\\n\\n**Affected workflows:**\\n- Daily Sub-Agent Model Resolution Audit — [§32103509594](https://github.com/github/gh-aw/actions/runs/32103509594)\\n- Daily GitHub Docs SEO Optimizer — [§32100895296](https://github.com/github/gh-aw/actions/runs/32100895296)\\n- Metrics Collector — [§32092825920](https://github.com/github/gh-aw/actions/runs/32092825920)\\n\\n### Evidence\\n\\n`audit-diff` against the nearest successful run of the same Model Resolution Audit workflow ([§31998521926](https://github.com/github/gh-aw/actions/runs/31998521926)) shows:\\n\\n<details>\\n<summary>Traffic volume comparison (failed run vs. known-good run)</summary>\\n\\n| Domain | Failed run | Success run | Change |\\n|---|---|---|---|\\n| `api.githubcopilot.com` | 6 calls | 20 calls | -70% |\\n| `o205451.ingest.us.sentry.io` | 3 calls | 7 calls | -57% |\\n| `otlp-gateway-prod-eu-west-2.grafana.net` | 1 call | 5 calls | -80% |\\n| GitHub REST core rate-limit consumed | 56 | 609 | -91% |\\n\\nNo blocked requests, no new/removed domains — ruling out firewall/egress policy as the cause. The failed run simply did far less work before stopping.\\n</details>\\n\\nIn the clearest case (Metrics Collector, 32092825920), the step's own artifact/log post-processing (`rpc-messages.jsonl`, `gateway.md`, `token-
evals
`, timeout, or auth message anywhere in the captured tail.\n\n### Probable root cause\n\nA silent/deferred failure — likely a non-zero exit code or a downstream safe-outputs validation check that fires *after* the visible Copilot CLI stream ends, since housekeeping still completes normally. Because this hit 3 independent workflows same-day, treat as infra-level (shared Copilot CLI wrapper/harness), not workflow-specific logic.\n\n### Proposed remediation\n\n1. Instrument the Copilot CLI wrapper to emit an explicit exit-code/error line on termination — a non-zero exit with a clean-looking log is currently indistinguishable from success.\n2. Audit the safe-outputs post-processing step that runs after the visible Copilot CLI stream ends for a swallowed exception or silent validation failure.\n3. Prioritize the shared invocation code path over per-workflow logic, given 3 unrelated workflows failed identically the same day.\n\n### Success criteria\n\n`Execute GitHub Copilot CLI` failures surface a specific, non-empty error/exit-code in logs; the three affected workflows either pass or fail with a diagnosable cause, and their traffic-volume profile matches known-good runs.\n\n**Parent:** #53129\nRelated to #53129\n\n\n\u003c!-- gh-aw-tracker-id: aw-failure-investigator --\u003e\n\n\u003e Generated by [🔍 [aw] Failure Investigator (6h)](https://github.com/github/gh-aw/actions/runs/32109525808) · agent · 251.3 AIC · ⌖ 6.96 AIC · ⊞ 5.9K · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Faw-failure-investigator%22\u0026type=issues)\n\u003e - [x] expires \u003c!-- gh-aw-expires: 2026-08-25T07:21:03.164Z --\u003e on Aug 24, 2026, 11:21 PM UTC-08:00\n\n\u003c!-- gh-aw-agentic-workflow: [aw] Failure Investigator (6h), gh-aw-tracker-id: aw-failure-investigator, engine: claude, model: agent, id: 32109525808, workflow_id: aw-failure-investigator, run: https://github.com/github/gh-aw/actions/runs/32109525808 --\u003e\n\n\u003c!-- gh-aw-workflow-id: aw-failure-investigator --\u003e\n\u003c!-- gh-aw-workflow-call-id: github/gh-aw/aw-failure-investigator --\u003e","id":"I_kwDOPc1QR88AAAABNLRnpg","number":53620,"title":"[aw-failures] [P1] Execute GitHub Copilot CLI silently exits early — 3 unrelated workflows same day","updatedAt":"2026-08-18T11:49:03Z"}}}}}

Artifacts

Produced during runtime
Name Size Digest
activation Expired
9.09 MB
sha256:1596678f36c57b8aaeb32f87ea1a84760067dd0a1e54696e7571d2387c287f22
agent
163 KB
sha256:6c92319912f54a49de7e91344cbefcb2037944ca78bddc7fb3b34d6557192dc6
agent-output-fallback
1.03 KB
sha256:4335a26f6d0456eb4bca0d5d3bfcb3a0240a513ea6d0b0e46fbf997359c9c900
aic-usage-cache
494 Bytes
sha256:aa1da052affa29b803674480d30e7d6b3c90e9bebc8a6b3e4ecd752edf23fae7
awfailureinvestigator-experiment
5.37 KB
sha256:13b5b3c9e33fd7aee3d60ba002a1c57b15c53f8cfb4aa75d1d1beecc6fc49d43
detection
233 Bytes
sha256:a4aee374539f200738969c574f050b1b26dfd7f42941100bc9b0f4a88f0cefbc
evals
389 Bytes
sha256:45a969f349c0378a8af167c9f0046286ffe2538d1fcc335c546fa36057b4b801
github~gh-aw~SQR0LE.dockerbuild
24.6 KB
sha256:d5b708e11de87d54ecf4d6ac988d66da0ed661a2ae89518d30db4f7f8a65de46
safe-outputs-items
301 Bytes
sha256:a5073ddc17893d19917919639092131cabd6d992dd480eb5ed3d8ee6bfc5959c
usage
2.92 KB
sha256:5ba8c722d14a48071b439c9cc8456e93cd1a86f39972214fb0f570809ee6068e