Fix Metrics Collector full-window coverage (#54356)

* Initial plan

* Fix metrics collector full-window collection

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

* Delete pkg/cli/metrics_collector_workflow_contract_test.go

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: Peli de Halleux <pelikhan@users.noreply.github.com>

Author: Copilot

.github/workflows/metrics-collector.lock.yml

@@ -26,7 +26,7 @@ tools:
     branch-name: memory/meta-orchestrators
     file-glob: "metrics/**"
     max-patch-size: 131072 # 128KB - handles large daily metrics snapshots without patch-size gate failures
-timeout-minutes: 15
+timeout-minutes: 30
 safe-outputs:
   noop:
 
@@ -72,20 +72,25 @@ If you see any `.md` files at the root (e.g. `agent-performance-latest.md`, `sha
 
 **Workflow Status and Runs**:
 - Use the `status` tool to get a list of all workflows in the repository
-- Use the `logs` tool to download workflow run data from the last 24 hours, **in small
-  paginated batches** to avoid the tool timing out on large windows:
+- Use the `logs` tool to download workflow run data from the last 24 hours, **in bounded
+  paginated batches** to cover the full window without a single long-running request:
   ```
   Parameters (first call):
   - start_date: "-1d" (last 24 hours)
-  - count: 20
+  - count: 80
+  - timeout: 3
   - Include all workflows (no workflow_name filter)
   ```
 - **Pagination loop (required)**: the `logs` tool returns a `continuation` field when it stops
   early (timeout or count limit). While a `continuation` field is present in the returned data,
   issue another `logs` call using the parameters it provides (notably `before_run_id`, plus the
-  original `start_date`) with `count: 20`, and accumulate the runs from every batch. Stop when
-  either no `continuation` field is returned or you have made 10 batch calls (~200 runs), so the
-  overall step stays within the workflow timeout.
+  original `start_date`) with `count: 80` and `timeout: 3`, and accumulate the runs from every
+  batch. Stop only when there is no `continuation` field **and** the oldest collected run is at or
+  before the 24h window start. Do not stop after a fixed number of batches if the oldest collected
+  run is still newer than the window start; that produces a partial ~10h snapshot.
+- If the logs tool continues returning `continuation` after the oldest collected run reaches the
+  24h window start, stop paginating and ignore the remaining older cursor because the requested
+  window is complete.
 - Never issue a single `logs` call with `count >= 100` for the full `-1d` window: that request
   has repeatedly exceeded the 60s tool timeout and yields truncated data.
 - From the logs data, extract for each workflow:
@@ -103,9 +108,16 @@ If you see any `.md` files at the root (e.g. `agent-performance-latest.md`, `sha
   - Comments added by workflows
   - Discussions created by workflows
 - Extract and count these for each workflow
-- **If the accumulated log batches still do not cover the full 24h window** (i.e. the last batch
-  returned a `continuation` field or the oldest run collected is newer than 24h ago), do **not**
-  report `safe_outputs` as zero. Instead compute safe-output counts from the GitHub API fallback
+- Extract typed safe-output counts as well:
+  - `safe_outputs_by_type`: aggregate usage activity `safe_outputs.items_by_type` from each run
+    (for example `create_issue`, `create_pull_request`, `add_comment`, `create_discussion`)
+  - `safe_output_outcomes`: aggregate outcome fields from logs summaries and outcome reports:
+    `accepted`, `rejected`, `ignored`, `pending`, `lifecycle`, and `lifecycle_close`
+  - Never leave these fields absent; use empty objects/zero counts only when the full data source
+    was checked and no matching items exist.
+- **If the accumulated log batches still do not cover the full 24h window** (i.e. the oldest run
+  collected is newer than 24h ago), do **not** report `safe_outputs`, `safe_outputs_by_type`, or
+  `safe_output_outcomes` as zero. Instead compute safe-output counts from the GitHub API fallback
   described below and mark the source accordingly.
 
 **Safe Outputs Fallback (GitHub API)**:
@@ -119,9 +131,11 @@ API instead of zeroing them:
 - Comments added: `list_issues` / issue comment listing for issues updated in the window,
   counting comments authored by the agent app
 - Discussions created: `list_discussions` filtered by creation date in the window
-Attribute each result to a workflow via its `gh-aw-workflow-call-id` footer marker, aggregate the
-counts per workflow, and set `"safe_outputs_source": "github_api_fallback"` alongside a
-`collection_note` explaining the truncation. The same fallback applies to `engagement` fields.
+Attribute each result to a workflow and type via its `gh-aw-workflow-call-id` footer marker and
+item shape, aggregate the counts per workflow, and set `"safe_outputs_source":
+"github_api_fallback"` alongside a `collection_note` explaining the truncation. When API fallback
+cannot determine outcome status, set those items to `"pending"` rather than omitting the
+`safe_output_outcomes` breakdown. The same fallback applies to `engagement` fields.
 
 **Additional Metrics via GitHub API**:
 - Use GitHub MCP server (default toolset) to supplement with:
@@ -142,6 +156,13 @@ Create a JSON object following this schema:
 {
   "timestamp": "2024-12-24T00:00:00Z",
   "period": "daily",
+  "collection_status": "complete",
+  "collection_window": {
+    "start": "2024-12-23T00:00:00Z",
+    "end": "2024-12-24T00:00:00Z",
+    "coverage_hours": 24,
+    "logs_batches": 6
+  },
   "collection_duration_seconds": 45,
   "workflows": {
     "workflow-name": {
@@ -151,6 +172,20 @@ Create a JSON object following this schema:
         "comments_added": 10,
         "discussions_created": 1
       },
+      "safe_outputs_by_type": {
+        "create_issue": 5,
+        "create_pull_request": 2,
+        "add_comment": 10,
+        "create_discussion": 1
+      },
+      "safe_output_outcomes": {
+        "accepted": 9,
+        "rejected": 1,
+        "ignored": 2,
+        "pending": 6,
+        "lifecycle": 0,
+        "lifecycle_close": 0
+      },
       "workflow_runs": {
         "total": 7,
         "successful": 6,
@@ -176,6 +211,20 @@ Create a JSON object following this schema:
     "total_workflows": 120,
     "active_workflows": 85,
     "total_safe_outputs": 45,
+    "safe_outputs_by_type": {
+      "create_issue": 20,
+      "create_pull_request": 8,
+      "add_comment": 15,
+      "create_discussion": 2
+    },
+    "safe_output_outcomes": {
+      "accepted": 24,
+      "rejected": 5,
+      "ignored": 6,
+      "pending": 10,
+      "lifecycle": 0,
+      "lifecycle_close": 0
+    },
     "overall_success_rate": 0.892,
     "total_tokens": 1250000,
     "total_cost_usd": 12.50
@@ -258,9 +307,11 @@ find /tmp/gh-aw/repo-memory/default/metrics/daily/ -name "*.json" -mtime +30 -de
 
 **Primary data source**: Use the agentic-workflows tool for all workflow run metrics:
 1. Start with `status` tool to get workflow inventory
-2. Use `logs` tool with `start_date: "-1d"` and `count: 20`, then follow the `continuation`
-   field (using its `before_run_id`) for up to 10 batches to cover the full window
-3. Extract metrics from the accumulated log data (success/failure, tokens, costs, safe outputs)
+2. Use `logs` tool with `start_date: "-1d"`, `count: 80`, and `timeout: 3`, then follow the
+   `continuation` field (using its `before_run_id`) until the oldest collected run reaches the
+   24h window start
+3. Extract metrics from the accumulated log data (success/failure, tokens, costs, safe outputs,
+   typed safe-output counts, and outcome breakdowns)
 
 **Secondary data source**: Use GitHub MCP server for engagement metrics only:
 - Reactions on issues/PRs created by workflows
@@ -278,6 +329,10 @@ find /tmp/gh-aw/repo-memory/default/metrics/daily/ -name "*.json" -mtime +30 -de
 - If token/cost data is unavailable, omit or set to null
 - Always include workflows in the metrics even if they have no activity (helps detect stalled workflows)
 - **If the agentic-workflows `logs` tool is unavailable**, collect what you can from the GitHub API directly (workflow runs via `list_workflow_runs`) and set `"data_source": "github_api_fallback"` in the JSON
+- Set `"collection_status": "complete"` only when workflow run counts and safe-output breakdowns
+  cover the full 24h window through logs data and/or the GitHub API fallback. Set
+  `"collection_status": "partial"` only when both logs pagination and fallback collection fail to
+  cover the window, and include a precise `collection_note` plus `collection_window.coverage_hours`.
 - **NEVER write a partial stub file like `{"date": "...", "status": "no-data"}`** — if you can't collect data, write a minimal valid metrics JSON with zeros instead:
   ```json
   {