Skip to content

[aw] Failure Investigator (6h) #3

[aw] Failure Investigator (6h)

[aw] Failure Investigator (6h) #3

Triggered via schedule April 17, 2026 07:13
Status Success
Total duration 8m 15s
Artifacts 5
Fit to window
Zoom out
Zoom in

Annotations

1 error, 11 warnings, and 1 notice
agent
Failed to add reaction: Resource not accessible by integration\\n##[error]ERR_API: Failed to add reaction: Resource not accessible by integration\\n```\\n\\n**Impact:** Activation fails → agent never runs → entire workflow skipped. These smoke tests provide no signal while this PR is open.\\n\\n### Cluster B — Codex Agent Read-only Filesystem\\n\\nAI Moderator run [§24528406902](https://github.com/github/gh-aw/actions/runs/24528406902) triggered on `issues` event. Pre-activation and activation succeeded; the agent job failed in 50 s.\\n\\n**Error (from `agent-stdio.log`):**\\n```\\nWARNING: proceeding, even though we could not update PATH: Read-only file system (os error 30)\\nError: Read-only file system (os error 30)\\n```\\nCodex v0.118.0 attempts to write PATH updates to the filesystem inside the chroot, but the filesystem is mounted read-only. A subsequent successful AI Moderator run ([§24528802957](https://github.com/github/gh-aw/actions/runs/24528802957)) skipped activation entirely (triggered by `issue_comment`, not relevant), so there is no clean baseline comparison.\\n\\n**audit-diff ([§24528802957](https://github.com/github/gh-aw/actions/runs/24528802957) vs [§24528406902](https://github.com/github/gh-aw/actions/runs/24528406902)):**\\n- Duration delta: +9m47s (failure ran much longer before crashing)\\n- No firewall or domain anomalies\\n\\n**Tracked in:** #26707\\n\\n### Cluster C — Copilot API Server Error (Scheduled Workflows)\\n\\nTwo scheduled workflows at 13:41–13:42 UTC both terminated with `exit code 2`:\\n\\n**Affected runs:**\\n- [§24513598667](https://github.com/github/gh-aw/actions/runs/24513598667) — CLI Consistency Checker (21.7 min, 10 turns, 450K tokens)\\n- [§24513607305](https://github.com/github/gh-aw/actions/runs/24513607305) — CI Optimization Coach (3.5 min, 0 turns)\\n\\n**Evidence from `agent-stdio.log` (CLI Consistency Checker):**\\n```\\nResponse was interrupted due to a server error. Retrying...\\n````\\nCLI Consistency Checker ran 10 turns successfully before the API interrupted the stream at the final turn. CI Optimization Coach exited with 0 turns (vs 8 in yesterday's successful baseline), suggesting it failed immediately at API startup.\\n\\n**audit baseline comparison (CI Optimization Coach):**\\n- Baseline run 24457879701 (yesterday): 8 turns, success\\n- This run: 0 turns, failure → classified as \\\"changed (turns_decrease)\\\"\\n\\nBoth runs use Copilot v1.0.21, claude-sonnet-4.6, triggered at the same scheduled time — consistent with a transient API hiccup at that moment.\\n\\n### Existing Issue Correlation\\n\\n| Issue | Status | Relation |\\n|-------|--------|----------|\\n| #26707 [aw] AI Moderator failed | Open | **Directly tracks** Cluster B (run 24528406902) |\\n| #26351 [aw] Smoke Gemini failed | Open | Different failure (Gemini `API key not valid`, Apr 15) — not related to Cluster A |\\n| #26405 [aw] Daily Fact About gh-aw failed | Open | Apr 15 — different window, not analyzed |\\n| #25214 [aw] No-Op Runs | Open | Different issue class |\\n\\n### Proposed Fix Roadmap\\n\\n| Priority | Cluster | Fix |\\n|----------|---------|-----|\\n| **P0** | A (Activation) | Add `continue-on-error: true` to the reaction step, or gate reaction on token scope check |\\n| **P1** | C (Copilot API) | Add retry/resilience logic at agent startup; surface `exit code 2` as retryable |\\n| **P2** | B (Codex FS) | Investigate Codex v0.118.0 PATH-write on read-only chroot; pin to earlier version or fix mount |\\n\\n### Sub-issues\\n\\n- #26710 — Activation reaction step fails on Copilot-authored PR branches\\n- #26711 — Copilot API server interruption exits scheduled workflows with code 2\\n- Cluster B tracked in existing #26707\\n\\n**References:**\\n- [§24528647283](https://github.com/github/gh-aw/actions/runs/24528647283) — Smoke Copilot (Cluster A representative)\\n- [§24528406902](https://github.com/github/gh-aw/actions/runs/24528406902) — AI Moderator (Cluster B)\\n- [§24513598667](https://github.com/github/gh-aw/actions/runs/24513598667) — CLI
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078521/workflow-logs/pre_activation: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078521/workflow-logs/pre_activation'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078521/workflow-logs/detection: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078521/workflow-logs/detection'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078521/workflow-logs/activation: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078521/workflow-logs/activation'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078519/workflow-logs/safe_outputs: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078519/workflow-logs/safe_outputs'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078519/workflow-logs/agent: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078519/workflow-logs/agent'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078519/workflow-logs/activation: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078519/workflow-logs/activation'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078512/workflow-logs/unlock: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078512/workflow-logs/unlock'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078512/workflow-logs/pre_activation: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078512/workflow-logs/pre_activation'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078512/workflow-logs/conclusion: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078512/workflow-logs/conclusion'
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-24546078512/workflow-logs/activation: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-24546078512/workflow-logs/activation'
agent
GitHub MCP guard policy automatically applied for public repository. min-integrity='approved' and repos='all' ensure only approved-integrity content is accessible.
agent
Safe Outputs MCP Server Startup Log

Artifacts

Produced during runtime
Name Size Digest
activation Expired
22.9 KB
sha256:7b741e9c61eb6c90462e3c404207c40eb42eeaee3b6be056d7b12416948e5aab
agent Expired
420 KB
sha256:3864d0b7a43e62b8caa1d8cdfe5d6ec39cd5021ba720306ccf9e4e919774693d
detection Expired
18.5 KB
sha256:adb7ac4c47c9b70a05db3c58524ea08cfe519ec37e0deeaf5787d2a0eb05aed5
github~gh-aw~TKEQQT.dockerbuild Expired
23.6 KB
sha256:040c76341563f3ffe42469c8a6fb640a419d8e88c6b2500d6d35a720fbe90e6f
safe-outputs-items Expired
301 Bytes
sha256:fdb259dfe3494fefe2913f02cbd72e1bdfe68c3a42b1f80f5a39bf37d00a5ccb