[aw] Failure Investigator (6h) #506
aw-failure-investigator.lock.yml
on: schedule
Annotations
14 errors and 4 warnings
|
agent
Process completed with exit code 2.` This blocks the entire Smoke Copilot fleet — no smoke test in any variant can execute past the build step.\n\n### Affected workflows / run IDs\n\n- Smoke Copilot — [§32375443008](https://github.com/github/gh-aw/actions/runs/32375443008)\n- Smoke Copilot - AOAI apikey — [§32375445846](https://github.com/github/gh-aw/actions/runs/32375445846)\n- Smoke Copilot - AOAI Entra — [§32375448656](https://github.com/github/gh-aw/actions/runs/32375448656)\n\nAll 3 failed within a 6-minute window (13:38–13:44 UTC) with the identical signature, confirming a single shared root cause rather than 3 independent flakes.\n\n### Probable root cause\n\nThe Go toolchain directive (`go.mod` `toolchain` line, or an equivalent version pin referenced by `Makefile:42`) still specifies `go1.26.5`. The CI runner image now ships `go1.26.6`. `go tool`'s strict version check treats this as a mismatch and refuses to compile, rather than silently upgrading.\n\n### Proposed remediation\n\n- Bump the `toolchain`/version pin in `go.mod` to `go1.26.6` (or to whatever version the runner image guarantees), OR relax `Makefile:42`'s build invocation to tolerate a compatible patch-version drift.\n- Add a fast-fail check earlier in CI (before the smoke fleet) that surfaces toolchain-version drift with a clearer message, so this doesn't have to be root-caused via raw compile output next time.\n\n### Success criteria\n\n- All 3 Smoke Copilot variants pass the build step on the next scheduled run.\n- No recurrence of the `does not match go tool version` error across a full 24h window.","labels":["agentic-workflows","bug"],"parent":"failure-report-2026-08-20","title":"Smoke Copilot fleet: Go toolchain version mismatch breaks make build (go1.26.6 vs go1.26.5)"}}}
|
|
agent
Agent execution exited with code 1\\n##[error]Process completed with exit code 1.\",\n \"impact\": \"Workflow did not complete successfully and may need intervention\"\n },\n {\n \"category\": \"network\",\n \"severity\": \"medium\",\n \"title\": \"Blocked Network Requests\",\n \"description\": \"1 network request(s) were blocked by firewall\",\n \"impact\": \"Blocked requests may indicate missing network permissions or unexpected behavior\"\n },\n {\n \"category\": \"performance\",\n \"severity\": \"high\",\n \"title\": \"Resource Heavy For Domain\",\n \"description\": \"This Code Fix run consumed a heavy execution profile for its task shape.\",\n \"impact\": \"Higher cost and latency than a comparable well-behaved run\"\n }\n ],\n \"recommendations\": [\n {\n \"priority\": \"high\",\n \"action\": \"Review error logs to identify root cause of failure\",\n \"reason\": \"Understanding failure causes helps prevent recurrence\",\n \"example\": \"Check the errors field for specific error messages, or inspect the log files in logs_path\"\n },\n {\n \"priority\": \"medium\",\n \"action\": \"Add blocked domains to the workflow network allow-list\",\n \"reason\": \"Firewall-blocked domains prevent the agent from completing its tasks\",\n \"example\": \"Add allowed domains to network configuration or review firewall rules\"\n },\n {\n \"priority\": \"high\",\n \"action\": \"Compare this run to similar successful runs and trim unnecessary turns, tools, or write actions.\",\n \"reason\": \"This Code Fix run consumed a heavy execution profile for its task shape.\"\n }\n ],\n \"observability_insights\": [\n {\n \"category\": \"execution\",\n \"severity\": \"medium\",\n \"title\": \"Exploratory execution path\",\n \"summary\": \"The agent used 2 turns across 56 tool types, which indicates adaptive planning instead of a strictly linear path.\",\n \"evidence\": \"turns=2 tool_types=56\"\n },\n {\n \"category\": \"actuation\",\n \"severity\": \"info\",\n \"title\": \"Read-only posture observed\",\n \"summary\": \"The workflow stayed in an analysis posture and did not emit any GitHub write actions.\",\n \"evidence\": \"created_items=0 safe_items=0\"\n },\n {\n \"category\": \"network\",\n \"severity\": \"high\",\n \"title\": \"Network friction detected\",\n \"summary\": \"The firewall observed 1 request(s) with 1 blocked, for a 100% block rate.\",\n \"evidence\": \"blocked=1 total=1\"\n },\n {\n \"category\": \"execution\",\n \"severity\": \"info\",\n \"title\": \"Log template patterns mined\",\n \"summary\": \"Analysis identified 3 distinct event templates across 6 pipeline stages from 58 events.\",\n \"evidence\": \"plan=1 tool_call=1 finish=1\"\n },\n {\n \"category\": \"reliability\",\n \"severity\": \"medium\",\n \"title\": \"2 anomalous event pattern(s) detected\",\n \"summary\": \"Anomaly detection flagged 2 event(s) as unusual based on template similarity and cluster rarity.\",\n \"evidence\": \"stage=plan score=0.65: new log template discovered; rare cluster (few observations); stage=tool_call score=0.65: new log template discovered; rare cluster (few observations)\"\n },\n {\n \"category\": \"execution\",\n \"severity\": \"info\",\n \"title\": \"Agent stage sequence\",\n \"summary\": \"The observed pipeline stage sequence for this run.\",\n \"evidence\": \"plan tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call to
|
|
agent
KVM kernel module is not loaded. docker-sbx requires a KVM-capable runner with nested virtualisation enabled.\n8:1272:2026-08-20T14:24:06.9246680Z ##[error]Process completed with exit code 1."}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da70ff788","uuid":"574440c5-608b-4c71-9f0f-3a7fb9a2026a","timestamp":"2026-08-20T19:10:01.147Z","tool_use_result":{"mode":"content","numFiles":0,"filenames":[],"content":"7:1271:2026-08-20T14:24:06.9238490Z ##[error]KVM kernel module is not loaded. docker-sbx requires a KVM-capable runner with nested virtualisation enabled.\n8:1272:2026-08-20T14:24:06.9246680Z ##[error]Process completed with exit code 1.","numLines":2,"totalLines":2}}
|
|
agent
2 safe output(s) failed:\n1003:2026-08-20T16:10:50.6190295Z Set output 'create_discussion_error_count'\n=== run 32381373575 (PR Sous Chef, safe_outputs job) ===\n369:2026-08-20T14:48:49.4115796Z digest-mismatch: error\n496:2026-08-20T14:48:51.0569896Z digest-mismatch: error\n784:2026-08-20T14:48:55.7368414Z ##[error]1 safe output(s) failed:\n1016:2026-08-20T14:49:00.3323449Z Set output 'create_discussion_error_count'","is_error":false}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da70ff788","uuid":"6479a208-3e02-48ae-82ac-95df397bc86b","timestamp":"2026-08-20T19:09:38.837Z","tool_use_result":{"stdout":"=== run 32389691755 (PR Sous Chef, safe_outputs job) ===\n368:2026-08-20T16:10:41.3437375Z digest-mismatch: error\n493:2026-08-20T16:10:42.6666595Z digest-mismatch: error\n772:2026-08-20T16:10:45.9463939Z ##[error]2 safe output(s) failed:\n1003:2026-08-20T16:10:50.6190295Z Set output 'create_discussion_error_count'\n=== run 32381373575 (PR Sous Chef, safe_outputs job) ===\n369:2026-08-20T14:48:49.4115796Z digest-mismatch: error\n496:2026-08-20T14:48:51.0569896Z digest-mismatch: error\n784:2026-08-20T14:48:55.7368414Z ##[error]1 safe output(s) failed:\n1016:2026-08-20T14:49:00.3323449Z Set output 'create_discussion_error_count'","stderr":"","interrupted":false,"isImage":false,"noOutputExpected":false}}
|
|
agent
Process completed with exit code 2.\n2026-08-20T13:44:33.6716745Z ##[group]Run bash \"${RUNNER_TEMP}/gh-aw/actions/stop_cli_proxy.sh\"\n2026-08-20T13:44:33.6717293Z \u001b[36;1mbash \"${RUNNER_TEMP}/gh-aw/actions/stop_cli_proxy.sh\"\u001b[0m\n2026-08-20T13:44:33.6754588Z shell: /usr/bin/bash -e {0}","is_error":false}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da70ff788","uuid":"c9abbaf1-71cd-45a7-9fd0-fc8051cc7137","timestamp":"2026-08-20T19:09:28.138Z","tool_use_result":{"stdout":"2026-08-20T13:44:33.4478887Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4482584Z # crypto/internal/fips140/alias\n2026-08-20T13:44:33.4483531Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4506057Z # crypto/internal/constanttime\n2026-08-20T13:44:33.4506964Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4536670Z # charm.land/huh/v2/internal/selector\n2026-08-20T13:44:33.4537507Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4567121Z # github.com/cli/go-gh/v2/internal/set\n2026-08-20T13:44:33.4567971Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4595476Z # vendor/golang.org/x/crypto/cryptobyte/asn1\n2026-08-20T13:44:33.4596467Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4626703Z # vendor/golang.org/x/crypto/internal/alias\n2026-08-20T13:44:33.4627763Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4657850Z # internal/nettrace\n2026-08-20T13:44:33.4658794Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4688803Z # log/slog/internal\n2026-08-20T13:44:33.4689891Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4720631Z # github.com/github/gh-aw/pkg/setutil\n2026-08-20T13:44:33.4721449Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4753824Z # github.com/segmentio/asm/cpu/cpuid\n2026-08-20T13:44:33.4754789Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4785373Z # github.com/segmentio/asm/internal/unsafebytes\n2026-08-20T13:44:33.4786415Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.4815140Z # golang.org/x/crypto/internal/alias\n2026-08-20T13:44:33.4815825Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.6553060Z # crypto/internal/boring/sig\n2026-08-20T13:44:33.6553931Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.6554386Z # sync/atomic\n2026-08-20T13:44:33.6555119Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.6555869Z # internal/cpu\n2026-08-20T13:44:33.6556629Z compile: version \"go1.26.6\" does not match go tool version \"go1.26.5\"\n2026-08-20T13:44:33.6592289Z make: *** [Makefile:42: build] Error 1\n2026-08-20T13:44:33.6609153Z ##[error]Process completed with exit code 2.\n2026-08-20T13:44:33.6716745Z ##[group]Run bash \"${RUNNER_TEMP}/gh-aw/actions/stop_cli_proxy.sh\"\n2026-08-20T13:44:33.6717293Z \u001b[36;1mbash \"${RUNNER_TEMP}/gh-aw/actions/stop_cli_proxy.sh\"\u001b[0m\n2026-08-20T13:44:33.6754588Z shell: /usr/bin/bash -e {0}","stderr":"","interrupted":false,"isImage":false,"noOutputExpected":false}}
|
|
agent
Process completed with exit code 2.\n2844:2026-08-20T13:43:23.5956228Z ##[group]Run node \"${RUNNER_TEMP}/gh-aw/actions/detect_agent_errors.cjs\"\n2845:2026-08-20T13:43:23.5956795Z \u001b[36;1mnode \"${RUNNER_TEMP}/gh-aw/actions/detect_agent_errors.cjs\"\u001b[0m\n2873:2026-08-20T13:43:23.6317840Z [detect-agent-errors] Log file not found: /tmp/gh-aw/agent-stdio.log\n3250:2026-08-20T13:43:25.4456564Z [ERROR] No log sources found. Run awf with a command first to generate logs.\n3609:/tmp/gh-aw/otlp-export-errors.jsonl\n3736:2026-08-20T13:43:35.2585838Z Set output 'ai_credits_rate_limit_error'\n3740:2026-08-20T13:43:35.2601318Z Set output 'http_400_response_error'\n3741:2026-08-20T13:43:35.2601804Z Set output 'inference_access_error'\n3744:2026-08-20T13:43:35.2603495Z Set output 'mcp_policy_error'\n3745:2026-08-20T13:43:35.2603938Z Set output 'missing_model_pricing_error'\n3747:2026-08-20T13:43:35.2604810Z Set output 'model_not_supported_error'\n=== run 32375445846 ===\n1693:2026-08-20T13:41:58.9593580Z digest-mismatch: error\n1827:2026-08-20T13:42:01.8308421Z fail-on-cache-miss: false\n2337:2026-08-20T13:42:26.0485563Z drwxr-xr-x 2 runner runner 4096 Aug 20 13:42 error-messages\n2338:2026-08-20T13:42:26.0486010Z drwxr-xr-x 2 runner runner 4096 Aug 20 13:42 error-pattern-safety\n2339:2026-08-20T13:42:26.0486496Z drwxr-xr-x 2 runner runner 4096 Aug 20 13:42 error-recovery-patterns\n2664:2026-08-20T13:42:58.0446743Z make: *** [Makefile:42: build] Error 1\n2665:2026-08-20T13:42:58.0465267Z ##[error]Process completed with exit code 2.\n2691:2026-08-20T13:42:58.0846986Z ##[group]Run node \"${RUNNER_TEMP}/gh-aw/actions/detect_agent_errors.cjs\"\n2692:2026-08-20T13:42:58.0847567Z \u001b[36;1mnode \"${RUNNER_TEMP}/gh-aw/actions/detect_agent_errors.cjs\"\u001b[0m\n2718:2026-08-20T13:42:58.1205447Z [detect-agent-errors] Log file not found: /tmp/gh-aw/agent-stdio.log\n3076:2026-08-20T13:43:00.1226105Z [ERROR] No log sources found. Run awf with a command first to generate logs.\n3417:/tmp/gh-aw/otlp-export-errors.jsonl\n3542:2026-08-20T13:43:09.9979688Z Set output 'ai_credits_rate_limit_error'\n3546:2026-08-20T13:43:09.9995240Z Set output 'http_400_response_error'\n3547:2026-08-20T13:43:09.9995759Z Set output 'inference_access_error'\n3550:2026-08-20T13:43:09.9997225Z Set output 'mcp_policy_error'\n3551:2026-08-20T13:43:09.9997666Z Set output 'missing_model_pricing_error'\n3553:2026-08-20T13:43:09.9998520Z Set output 'model_not_supported_error'\n=== run 32375443008 ===\n1732:2026-08-20T13:43:44.4937971Z digest-mismatch: error\n1866:2026-08-20T13:43:47.1930697Z fail-on-cache-miss: false\n2404:2026-08-20T13:44:05.9463722Z drwxr-xr-x 2 runner runner 4096 Aug 20 13:44 error-messages\n2405:2026-08-20T13:44:05.9464390Z drwxr-xr-x 2 runner runner 4096 Aug 20 13:44 error-pattern-safety\n2406:2026-08-20T13:44:05.9465072Z drwxr-xr-x 2 runner runner 4096 Aug 20 13:44 error-recovery-patterns\n2731:2026-08-20T13:44:33.6592289Z make: *** [Makefile:42: build] Error 1\n2732:2026-08-20T13:44:33.6609153Z ##[error]Process completed with exit code 2.\n2758:2026-08-20T13:44:33.7018055Z ##[group]Run node \"${RUNNER_TEMP}/gh-aw/actions/detect_agent_errors.cjs\"\n2759:2026-08-20T13:44:33.7018687Z \u001b[36;1mnode \"${RUNNER_TEMP}/gh-aw/actions/detect_agent_errors.cjs\"\u001b[0m\n2785:2026-08-20T13:44:33.7411246Z [detect-agent-errors] Log file not found: /tmp/gh-aw/agent-stdio.log\n3142:2026-08-20T13:44:35.6748748Z [ERROR] No log sources found. Run awf with a command first to generate logs.\n3483:/tmp/gh-aw/otlp-export-errors.jsonl\n3608:2026-08-20T13:44:45.4436781Z Set output 'ai_credits_rate_limit_error'\n3612:2026-08-20T13:44:45.4439083Z Set output 'http_400_response_error'\n3613:2026-08-20T13:44:45.4439845Z Set output 'inference_access_error'\n3616:2026-08-20T13:44:45.4441190Z Set output 'mcp_policy_error'\n3617:2026-08-20T13:44:45.4441618Z Set output 'missing_model_pricing_error'\n3619:2026-08-20T13:44:45.4442435Z Set output 'model_not_supported_error'","is_error":false}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da
|
|
agent
Agent execution exited with code 1\n2131:2026-08-20T17:05:26.7088570Z ##[error]Process completed with exit code 1."}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da70ff788","uuid":"167a358d-7d4d-4ce3-8f1d-c1bc11dd87c5","timestamp":"2026-08-20T19:08:48.456Z","tool_use_result":{"mode":"content","numFiles":0,"filenames":[],"content":"2128:2026-08-20T17:05:26.6979117Z [WARN] Command completed with exit code: 1\n2130:2026-08-20T17:05:26.7079839Z ##[error]Agent execution exited with code 1\n2131:2026-08-20T17:05:26.7088570Z ##[error]Process completed with exit code 1.","numLines":3,"totalLines":3}}
|
|
agent
Agent execution exited with code 139\n19:2262:2026-08-20T18:25:09.0030150Z ##[error]Process completed with exit code 139.\n21:2702:2026-08-20T18:25:11.5668401Z [2026-08-20T18:24:11.598Z] [DEBUG] [tracing:provider] resolveEndpoint: endpoint=\"***\", signalPath=\"/v1/traces\"\n24:[Omitted long matching line]\n26:[Omitted long matching line]\n28:[Omitted long matching line]\n29:[Omitted long matching line]"}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da70ff788","uuid":"e08d58d3-e434-46ef-9316-89193a0c87ef","timestamp":"2026-08-20T19:08:37.877Z","tool_use_result":{"mode":"content","numFiles":0,"filenames":[],"content":"16:2229:2026-08-20T18:25:04.3201500Z 111 Segmentation fault | pi --print --mode json --no-session --model aw-gateway/gpt-5.4 --extension \"${RUNNER_TEMP}/gh-aw/actions/pi_provider.cjs\" --extension \"${RUNNER_TEMP}/gh-aw/actions/pi_steering_extension.cjs\" 2>&1\n17:2259:2026-08-20T18:25:08.9953227Z [WARN] Command completed with exit code: 139\n18:2261:2026-08-20T18:25:09.0023530Z ##[error]Agent execution exited with code 139\n19:2262:2026-08-20T18:25:09.0030150Z ##[error]Process completed with exit code 139.\n21:2702:2026-08-20T18:25:11.5668401Z [2026-08-20T18:24:11.598Z] [DEBUG] [tracing:provider] resolveEndpoint: endpoint=\"***\", signalPath=\"/v1/traces\"\n24:[Omitted long matching line]\n26:[Omitted long matching line]\n28:[Omitted long matching line]\n29:[Omitted long matching line]","numLines":9,"totalLines":9}}
|
|
agent
Agent execution exited with code 139\n2262:2026-08-20T18:25:09.0030150Z ##[error]Process completed with exit code 139.\n2689:2026-08-20T18:25:11.5660509Z [2026-08-20T18:24:11.590Z] [DEBUG] [config:validation_env] Validation complete: valid=true, errors=0, warnings=0\n2702:2026-08-20T18:25:11.5668401Z [2026-08-20T18:24:11.598Z] [DEBUG] [tracing:provider] resolveEndpoint: endpoint=\"***\", signalPath=\"/v1/traces\"\n2732:2026-08-20T18:25:11.5685519Z [2026-08-20T18:24:11.619Z] [DEBUG] [launcher:pool] Creating new session connection pool: idleTimeout=6h0m0s, cleanupInterval=5m0s, maxErrors=10\n2872:2026-08-20T18:25:11.5794202Z [2026-08-20T18:24:12.448Z] [DEBUG] [mcp:connection_logging] Inbound RPC response: serverID=safeoutputs, payloadBytes=9354, hasError=false\n2876:2026-08-20T18:25:11.5823451Z [2026-08-20T18:24:12.460Z] [DEBUG] [rpc] safeoutputs←resp 9354b {\"jsonrpc\":\"2.0\",\"id\":1,\"result\":{\"ttlMs\":0,\"cacheScope\":\"\",\"tools\":[{\"description\":\"Create a GitHub discussion for announcements, Q\\u0026A, reports, status updates, or community conversations. Use this for content that benefits from threaded replies, doesn't require task tracking, or serves as documentation. For actionable work items that need assignment and status tracking, use create_issue instead. Arguments must be flat tool arguments (title, body), not nested under create_discussion. CONSTRAINTS: Maximum 1 discussion(s) can be created. Title will be prefixed with \\\"[Auto-Triage] \\\". Discussions will be created in category \\\"audits\\\".\",\"inputSchema\":{\"additionalProperties\":false,\"properties\":{\"body\":{\"description\":\"Discussion content in Markdown. Do NOT repeat the title as a heading since it already appears as the discussion's h1. Include all relevant context, findings, or questions.\",\"maxLength\":65536,\"type\":\"string\"},\"category\":{\"description\":\"Discussion category by name (e.g., 'General'), slug (e.g., 'general'), or ID. If omitted, uses the first available category. Category must exist in the repository. NOTE: the field is named category (not categoryId or category_id as in the GitHub GraphQL API).\",\"type\":\"string\"},\"integrity\":{\"description\":\"Trustworthiness level of the message source (e.g., \\\"low\\\", \\\"medium\\\", \\\"high\\\").\",\"type\":\"string\"},\"secrecy\":{\"description\":\"Confidentiality level of the message content (e.g., \\\"public\\\", \\\"internal\\\", \\\"private\\\").\",\"type\":\"string\"},\"title\":{\"description\":\"Concise discussion title summarizing the topic. The title appears as the main heading, so keep it brief and descriptive.\",\"type\":\"string\"}},\"required\":[\"title\",\"body\"],\"type\":\"object\"},\"name\":\"create_discussion\"},{\"description\":\"Add labels to an existing GitHub issue or pull request for categorization and filtering. Labels must already exist in the repository. For creating new issues with labels, use create_issue with the labels property instead. CONSTRAINTS: Maximum 10 label(s) can be added. INTENT ENCOURAGED: Include rationale (string, max 280 chars) and confidence (exactly one of: LOW, MEDIUM, HIGH) with each call. These fields are optional but strongly encouraged — they improve transparency and should normally be included. Use suggest: true alongside rationale and confidence to route for human review.\",\"inputSchema\":{\"additionalProperties\":false,\"properties\":{\"integrity\":{\"description\":\"Trustworthiness level of the message source (e.g., \\\"low\\\", \\\"medium\\\", \\\"high\\\").\",\"type\":\"string\"},\"item_number\":{\"description\":\"Issue or PR number to add labels to. This is the numeric ID from the GitHub URL (e.g., 456 in github.com/owner/repo/issues/456). Can also be a temporary_id from a previously created issue in the same workflow run — use the '#aw_abc123' form; the bare 'aw_abc123' form is also accepted and normalised to '#aw_abc123'. If omitted, adds labels to the issue or PR that triggered this","stderr":"","interrupted":false,"isImage":false,"noOutputExpected":false,"persistedOutputPath":"/home/runner/.cla
|
|
agent
shared audit types and state used across the audit_*.go files\\\\n+// (command wiring, run pipeline, analysis fan-out, summary building, and\\\\n+// report rendering).\\\\n \\\\n-\\\\t\\\\\\\"github.com/github/gh-aw/pkg/console\\\\\\\"\\\\n-\\\\t\\\\\\\"github.com/github/gh-aw/pkg/constants\\\\\\\"\\\\n-\\\\t\\\\\\\"github.com/github/gh-aw/pkg/errorutil\\\\\\\"\\\\n-\\\\t\\\\\\\"github.com/github/gh-aw/pkg/fileutil\\\\\\\"\\\\n+import (\\\\n \\\\t\\\\\\\"github.com/github/gh-aw/pkg/logger\\\\\\\"\\\\n-\\\\t\\\\\\\"github.com/github/gh-aw/pkg/parser\\\\\\\"\\\\n-\\\\t\\\\\\\"github.com/spf13/cobra\\\\\\\"\\\\n )\\\\n \\\\n var auditLog = logger.New(\\\\\\\"cli:audit\\\\\\\")\\\\n@@ -41,281 +29,6 @@ type AuditOptions struct {\\\\n \\\\tEvalsOnly bool\\\\n }\\\\n \\\\n-var auditCommandLong = `Audit one or more workflow runs by downloading artifacts and logs, detecting errors,\\\\n-analyzing MCP tool usage, and generating a concise report suitable for AI agents.\\\\n-\\\\n-When a single run is provided, generates a detailed Markdown report for that run.\\\\n-When two or more runs are provided, the first is used as the base (reference) and the\\\\n-remaining runs are compared against it, producing a diff report.\\\\n-\\\\n-Each argument accepts:\\\\n-- A numeric run ID (e.g., 1234567890)\\\\n-- A GitHub Actions run URL (e.g., https://github.com/owner/repo/actions/runs/1234567890)\\\\n-- A GitHub Actions job URL (e.g., https://github.com/owner/repo/actions/runs/1234567890/job/9876543210)\\\\n-- A GitHub Actions job URL with step (e.g., https://github.com/owner/repo/actions/runs/1234567890/job/9876543210#step:7:1)\\\\n-- A GitHub workflow run URL (e.g., https://github.com/owner/repo/runs/1234567890)\\\\n-- GitHub Enterprise URLs (e.g., https://github.example.com/owner/repo/actions/runs/1234567890)\\\\n-\\\\n-When a job URL is provided (single-run mode only):\\\\n-- If a step number is included (#step:7:1), extracts that specific step's output\\\\n-- If no step number, finds and extracts the first failing step's output\\\\n-- Saves job logs to the output directory`\\\\n-\\\\n-var auditCommandExample = ` ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 --repo owner/repo # Audit with bare run ID (--repo required)\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/actions/runs/1234567890 # Audit from run URL\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/actions/runs/1234567890/job/9876543210 # Audit job and extract first failing step\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/actions/runs/1234567890/job/9876543210#step:7:1 # Extract step 7 output\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/runs/1234567890 # Audit from workflow run URL\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit https://github.example.com/owner/repo/actions/runs/1234567890 # Audit from GitHub Enterprise\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 -o ./audit-reports # Custom output directory\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 -v # Verbose output\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 --parse # Parse agent logs and firewall logs, generating log.md and firewall.md\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 --repo owner/repo # Audit run from a specific repository\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 1234567891 # Diff two runs (base vs comparison)\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 1234567891 1234567892 # Diff base against multiple runs\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 1234567891 --format markdown # Markdown diff output for PR comments\\\\n- ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 --runtime gvisor # Skip run unless sandbox agent runtime matches`\\\\n-\\\\n-type auditCommandOptions struct {\\
|
|
agent
Agent execution exited with code 1\\n##[error]Process completed with exit code 1.\",\n \"file\": \"agent\"\n }\n ],\n \"warnings\": null,\n \"firewall_analysis\": {\n \"total_requests\": 1,\n \"allowed_requests\": 0,\n \"blocked_requests\": 1,\n \"requests_by_domain\": {\n \"(unknown)\": {\n \"allowed\": 0,\n \"blocked\": 1\n }\n }\n },\n \"mcp_failures\": null,\n \"missing_tools\": null\n}","is_error":false}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da70ff788","uuid":"bbf99f33-d9af-4f3c-85ce-097b8ee89001","timestamp":"2026-08-20T19:06:59.392Z","tool_use_result":{"stdout":"{\n \"errors\": [\n {\n \"type\": \"step_failure\",\n \"message\": \"##[error]Agent execution exited with code 1\\n##[error]Process completed with exit code 1.\",\n \"file\": \"agent\"\n }\n ],\n \"warnings\": null,\n \"firewall_analysis\": {\n \"total_requests\": 1,\n \"allowed_requests\": 0,\n \"blocked_requests\": 1,\n \"requests_by_domain\": {\n \"(unknown)\": {\n \"allowed\": 0,\n \"blocked\": 1\n }\n }\n },\n \"mcp_failures\": null,\n \"missing_tools\": null\n}","stderr":"","interrupted":false,"isImage":false,"noOutputExpected":false}}
|
|
agent
Agent execution exited with code 1\\n##[error]Process completed with exit code 1.\",\n \"impact\": \"Workflow did not complete successfully and may need intervention\"\n },\n {\n \"category\": \"network\",\n \"severity\": \"medium\",\n \"title\": \"Blocked Network Requests\",\n \"description\": \"1 network request(s) were blocked by firewall\",\n \"impact\": \"Blocked requests may indicate missing network permissions or unexpected behavior\"\n },\n {\n \"category\": \"performance\",\n \"severity\": \"high\",\n \"title\": \"Resource Heavy For Domain\",\n \"description\": \"This Code Fix run consumed a heavy execution profile for its task shape.\",\n \"impact\": \"Higher cost and latency than a comparable well-behaved run\"\n }\n ],\n \"recommendations\": [\n {\n \"priority\": \"high\",\n \"action\": \"Review error logs to identify root cause of failure\",\n \"reason\": \"Understanding failure causes helps prevent recurrence\",\n \"example\": \"Check the errors field for specific error messages, or inspect the log files in logs_path\"\n },\n {\n \"priority\": \"medium\",\n \"action\": \"Add blocked domains to the workflow network allow-list\",\n \"reason\": \"Firewall-blocked domains prevent the agent from completing its tasks\",\n \"example\": \"Add allowed domains to network configuration or review firewall rules\"\n },\n {\n \"priority\": \"high\",\n \"action\": \"Compare this run to similar successful runs and trim unnecessary turns, tools, or write actions.\",\n \"reason\": \"This Code Fix run consumed a heavy execution profile for its task shape.\"\n }\n ],\n \"observability_insights\": [\n {\n \"category\": \"execution\",\n \"severity\": \"medium\",\n \"title\": \"Exploratory execution path\",\n \"summary\": \"The agent used 2 turns across 56 tool types, which indicates adaptive planning instead of a strictly linear path.\",\n \"evidence\": \"turns=2 tool_types=56\"","is_error":false}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da70ff788","uuid":"ec240cc6-160b-4618-a896-112f8bcbe68a","timestamp":"2026-08-20T19:06:54.115Z","tool_use_result":{"stdout":"[info] [agenticworkflows] Bridge invoked: argumentCount=1\n[info] [agenticworkflows] Stdin captured: 52 bytes\n[info] [agenticworkflows] Calling tool 'audit' (52 argument bytes)\n[info] [agenticworkflows] MCP initialize: POST http://awmg-mcpg:8080/mcp/agenticworkflows\n[info] [agenticworkflows] MCP initialize: status=200, sessionId=IL25IK2L..., elapsed=30ms\n[info] [agenticworkflows] MCP notifications/initialized\n[info] [agenticworkflows] MCP notifications/initialized: done (6ms)\n[info] [agenticworkflows] MCP keepalive started (interval=10000ms)\n[info] [agenticworkflows] MCP tools/call: tool=audit, argumentBytes=52\n[info] [agenticworkflows] MCP keepalive ping: id=1000, elapsed=11ms\n[info] [agenticworkflows] MCP tools/call: status=200, elapsed=13019ms\n[info] [agenticworkflows] MCP keepalive stopped\n[info] [agenticworkflows] Tool call complete: total=13058ms\n{\n \"overview\": {\n \"run_id\": 32395535034,\n \"workflow_name\": \"Ponytail Reviewer\",\n \"status\": \"completed\",\n \"conclusion\": \"failure\",\n \"created_at\": \"2026-08-20T17:04:09Z\",\n \"started_at\": \"2026-08-20T17:04:09Z\",\n \"updated_at\": \"2026-08-20T17:10:05Z\",\n \"duration\": \"5.9m\",\n \"event\": \"pull_request\",\n \"branch\": \"gh-aw/pre-created/32393155021-1\",\n \"url\": \"https://github.com/github/gh-aw/actions/runs/32395535034\",\n \"logs_path\": \"/tmp/gh-aw/aw-mcp/logs/run-32395535034\"\n },\n \"comparison\": {\n \"baseline_found\": true,\n \"baseline\": {\n \"run_id\": 32387174411,\n \"workflow_name\": \"Ponytail Reviewer\",\n \"conclusion\": \"success\",\n \"created_at\": \"2026-08-20T15:36:48Z\",\n \"selection\": \"cohort_match\",\n \"matched_on\": [\n \"event\",\n \"task_domain\",\n \
|
|
evals
Process completed with exit code 2.` This blocks the entire Smoke Copilot fleet — no smoke test in any variant can execute past the build step.\n\n### Affected workflows / run IDs\n\n- Smoke Copilot — [§32375443008](https://github.com/github/gh-aw/actions/runs/32375443008)\n- Smoke Copilot - AOAI apikey — [§32375445846](https://github.com/github/gh-aw/actions/runs/32375445846)\n- Smoke Copilot - AOAI Entra — [§32375448656](https://github.com/github/gh-aw/actions/runs/32375448656)\n\nAll 3 failed within a 6-minute window (13:38–13:44 UTC) with the identical signature, confirming a single shared root cause rather than 3 independent flakes.\n\n### Probable root cause\n\nThe Go toolchain directive (`go.mod` `toolchain` line, or an equivalent version pin referenced by `Makefile:42`) still specifies `go1.26.5`. The CI runner image now ships `go1.26.6`. `go tool`'s strict version check treats this as a mismatch and refuses to compile, rather than silently upgrading.\n\n### Proposed remediation\n\n- Bump the `toolchain`/version pin in `go.mod` to `go1.26.6` (or to whatever version the runner image guarantees), OR relax `Makefile:42`'s build invocation to tolerate a compatible patch-version drift.\n- Add a fast-fail check earlier in CI (before the smoke fleet) that surfaces toolchain-version drift with a clearer message, so this doesn't have to be root-caused via raw compile output next time.\n\n### Success criteria\n\n- All 3 Smoke Copilot variants pass the build step on the next scheduled run.\n- No recurrence of the `does not match go tool version` error across a full 24h window.","labels":["agentic-workflows","bug"],"parent":"failure-report-2026-08-20","title":"Smoke Copilot fleet: Go toolchain version mismatch breaks make build (go1.26.6 vs go1.26.5)"}}}
|
|
evals
Agent execution exited with code 1\\n##[error]Process completed with exit code 1.\",\n \"impact\": \"Workflow did not complete successfully and may need intervention\"\n },\n {\n \"category\": \"network\",\n \"severity\": \"medium\",\n \"title\": \"Blocked Network Requests\",\n \"description\": \"1 network request(s) were blocked by firewall\",\n \"impact\": \"Blocked requests may indicate missing network permissions or unexpected behavior\"\n },\n {\n \"category\": \"performance\",\n \"severity\": \"high\",\n \"title\": \"Resource Heavy For Domain\",\n \"description\": \"This Code Fix run consumed a heavy execution profile for its task shape.\",\n \"impact\": \"Higher cost and latency than a comparable well-behaved run\"\n }\n ],\n \"recommendations\": [\n {\n \"priority\": \"high\",\n \"action\": \"Review error logs to identify root cause of failure\",\n \"reason\": \"Understanding failure causes helps prevent recurrence\",\n \"example\": \"Check the errors field for specific error messages, or inspect the log files in logs_path\"\n },\n {\n \"priority\": \"medium\",\n \"action\": \"Add blocked domains to the workflow network allow-list\",\n \"reason\": \"Firewall-blocked domains prevent the agent from completing its tasks\",\n \"example\": \"Add allowed domains to network configuration or review firewall rules\"\n },\n {\n \"priority\": \"high\",\n \"action\": \"Compare this run to similar successful runs and trim unnecessary turns, tools, or write actions.\",\n \"reason\": \"This Code Fix run consumed a heavy execution profile for its task shape.\"\n }\n ],\n \"observability_insights\": [\n {\n \"category\": \"execution\",\n \"severity\": \"medium\",\n \"title\": \"Exploratory execution path\",\n \"summary\": \"The agent used 2 turns across 56 tool types, which indicates adaptive planning instead of a strictly linear path.\",\n \"evidence\": \"turns=2 tool_types=56\"\n },\n {\n \"category\": \"actuation\",\n \"severity\": \"info\",\n \"title\": \"Read-only posture observed\",\n \"summary\": \"The workflow stayed in an analysis posture and did not emit any GitHub write actions.\",\n \"evidence\": \"created_items=0 safe_items=0\"\n },\n {\n \"category\": \"network\",\n \"severity\": \"high\",\n \"title\": \"Network friction detected\",\n \"summary\": \"The firewall observed 1 request(s) with 1 blocked, for a 100% block rate.\",\n \"evidence\": \"blocked=1 total=1\"\n },\n {\n \"category\": \"execution\",\n \"severity\": \"info\",\n \"title\": \"Log template patterns mined\",\n \"summary\": \"Analysis identified 3 distinct event templates across 6 pipeline stages from 58 events.\",\n \"evidence\": \"plan=1 tool_call=1 finish=1\"\n },\n {\n \"category\": \"reliability\",\n \"severity\": \"medium\",\n \"title\": \"2 anomalous event pattern(s) detected\",\n \"summary\": \"Anomaly detection flagged 2 event(s) as unusual based on template similarity and cluster rarity.\",\n \"evidence\": \"stage=plan score=0.65: new log template discovered; rare cluster (few observations); stage=tool_call score=0.65: new log template discovered; rare cluster (few observations)\"\n },\n {\n \"category\": \"execution\",\n \"severity\": \"info\",\n \"title\": \"Agent stage sequence\",\n \"summary\": \"The observed pipeline stage sequence for this run.\",\n \"evidence\": \"plan tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call tool_call to
|
|
agent
Failed to scan directory /tmp/gh-aw/aw-mcp/logs/run-32395535034/usage: EACCES: permission denied, scandir '/tmp/gh-aw/aw-mcp/logs/run-32395535034/usage'
|
|
agent
Skipped (no handler): 2\n2026-08-20T16:10:45.9397100Z Types: approve_workflow_run\n2026-08-20T16:10:45.9399154Z Temporary IDs registered: 0\n2026-08-20T16:10:45.9401810Z Synthetic updates: 0\n2026-08-20T16:10:45.9412703Z ##[warning]2 message(s) failed to process\n2026-08-20T16:10:45.9431808Z ##[warning]2 message(s) were skipped because no handler was loaded. Check your workflow's safe-outputs configuration.\n2026-08-20T16:10:45.9443677Z Exported temporary ID map with 0 mapping(s)\n2026-08-20T16:10:45.9454897Z Wrote temporary ID map to file for artifact upload\n2026-08-20T16:10:45.9463939Z ##[error]2 safe output(s) failed:\n - approve_workflow_run: No handler loaded for type 'approve_workflow_run'\n - approve_workflow_run: No handler loaded for type 'approve_workflow_run'\n2026-08-20T16:10:45.9742373Z ##[group]Run actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a\n2026-08-20T16:10:45.9744903Z with:\n2026-08-20T16:10:45.9746397Z name: safe-outputs-items\n2026-08-20T16:10:45.9748008Z path: /tmp/gh-aw/safe-output-items.jsonl\n/tmp/gh-aw/temporary-id-map.json\n\n---second---\n2026-08-20T14:48:55.7248937Z Total messages: 3\n2026-08-20T14:48:55.7250377Z Status: partial_success\n2026-08-20T14:48:55.7251778Z Successful: 2\n2026-08-20T14:48:55.7253167Z Failed: 1\n2026-08-20T14:48:55.7256105Z ##[warning]Skipped (no handler): 1\n2026-08-20T14:48:55.7258777Z Types: approve_workflow_run\n2026-08-20T14:48:55.7260057Z Temporary IDs registered: 0\n2026-08-20T14:48:55.7261240Z Synthetic updates: 0\n2026-08-20T14:48:55.7299247Z ##[warning]1 message(s) failed to process\n2026-08-20T14:48:55.7305177Z ##[warning]1 message(s) were skipped because no handler was loaded. Check your workflow's safe-outputs configuration.\n2026-08-20T14:48:55.7314060Z Exported temporary ID map with 0 mapping(s)\n2026-08-20T14:48:55.7319058Z Wrote temporary ID map to file for artifact upload\n2026-08-20T14:48:55.7357181Z Exported comment_id: 5357538772\n2026-08-20T14:48:55.7360726Z Exported comment_url: https://github.com/github/gh-aw/pull/54068#issuecomment-5357538772\n2026-08-20T14:48:55.7368414Z ##[error]1 safe output(s) failed:\n - approve_workflow_run: No handler loaded for type 'approve_workflow_run'\n2026-08-20T14:48:55.7583141Z ##[group]Run actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a\n2026-08-20T14:48:55.7583827Z with:\n2026-08-20T14:48:55.7584612Z name: safe-outputs-items\n2026-08-20T14:48:55.7585248Z path: /tmp/gh-aw/safe-output-items.jsonl\n/tmp/gh-aw/temporary-id-map.json","is_error":false}]},"parent_tool_use_id":null,"session_id":"77239985-04b2-42ad-9967-335da70ff788","uuid":"5acc55be-ff8b-4a29-8fa7-ca7eaddb8977","timestamp":"2026-08-20T19:09:44.591Z","tool_use_result":{"stdout":"2026-08-20T16:10:45.9372849Z Total messages: 3\n2026-08-20T16:10:45.9376515Z Status: partial_success\n2026-08-20T16:10:45.9379986Z Successful: 1\n2026-08-20T16:10:45.9384178Z Failed: 2\n2026-08-20T16:10:45.9391510Z ##[warning]Skipped (no handler): 2\n2026-08-20T16:10:45.9397100Z Types: approve_workflow_run\n2026-08-20T16:10:45.9399154Z Temporary IDs registered: 0\n2026-08-20T16:10:45.9401810Z Synthetic updates: 0\n2026-08-20T16:10:45.9412703Z ##[warning]2 message(s) failed to process\n2026-08-20T16:10:45.9431808Z ##[warning]2 message(s) were skipped because no handler was loaded. Check your workflow's safe-outputs configuration.\n2026-08-20T16:10:45.9443677Z Exported temporary ID map with 0 mapping(s)\n2026-08-20T16:10:45.9454897Z Wrote temporary ID map to file for artifact upload\n2026-08-20T16:10:45.9463939Z ##[error]2 safe output(s) failed:\n - approve_workflow_run: No handler loaded for type 'approve_workflow_run'\n - approve_workflow_run: No handler loaded for type 'approve_workflow_run'\n2026-08-20T16:10:45.9742373Z ##[group]Run actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a\n2026-08-20T16:10:45.9744903Z with:\n2026-08-20T16:10:45.9746397Z name: safe-outputs-items\n2026-08-20T16:10:45.9748008Z path: /tmp/gh-aw/safe-output-items.jsonl\n/tmp/gh-aw/temporary-id-map.json\n\n---second---\n2026-08-20T
|
|
safe_outputs
Invalid parent value: failure-report-2026-08-20. Expected either a valid temporary ID (format: aw_XXXXXXXXXXXX where X is a hex digit) or a numeric issue number.
|
|
safe_outputs
Invalid temporary_id format: 'failure-report-2026-08-20'. Temporary IDs must be in format 'aw_' followed by 3 to 12 alphanumeric or underscore characters (A-Za-z0-9_). Example: 'aw_abc' or 'aw_pr_fix'. Using auto-generated ID: 'aw_NH9DJk52'
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
activation
Expired
|
9.07 MB |
sha256:98401e831c96c58570a545ee8dbc3cff300b8ecb73319d507b6746dd806b34b8
|
|
|
agent
|
1.43 MB |
sha256:f66228616fac1b1c4f7cbf5afbd6f6b0ec23f71c24ae132e12d467838eeeefff
|
|
|
agent-output-fallback
|
9.71 KB |
sha256:c2d77d4cfd69a88c3ecbd1ce10aa1ec80de6ea8f3898271cc99533983cf0bb22
|
|
|
aic-usage-cache
|
467 Bytes |
sha256:3772f2ebc440f4d5af333638adfb890fefab7dc92d62e3570aa084860b4c9b55
|
|
|
awfailureinvestigator-experiment
|
5.51 KB |
sha256:4a497dc544f8f08ba0a6dda05f7b0ff51040454c655df0accb7be330bf8a384e
|
|
|
detection
|
233 Bytes |
sha256:0b58f8911b9f44d50f05699f0cdc3eb74498630f89857aefe805a602c4fda782
|
|
|
evals
|
383 Bytes |
sha256:336cb535411d4eb111b481b496b98e0099616ffef20fd953860aafd816a0609c
|
|
|
github~gh-aw~ZQXG36.dockerbuild
|
24.7 KB |
sha256:08ff468484d28dd66292a8aed3668c2ac2073b48785bc9ed0b3a56a7a0dd0043
|
|
|
safe-outputs-items
|
1.52 KB |
sha256:107cb3ff5cc6f1b70df9808836660ee9b0c772a05dcbf0c247ad3509402b5e39
|
|
|
usage
|
6.4 KB |
sha256:43f1701f51f5c08f8c32ef53dad1caac61966265d137040ea1651717f06f65a2
|
|