Implementing AIC usage in otel#38163
Conversation
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
|
✅ PR Code Quality Reviewer completed the code quality review. |
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. No ADR enforcement needed: PR #38163 does not have the 'implementation' label (has_implementation_label=false) and has 0 new lines of code in business logic directories (≤100 threshold, requires_adr_by_default_volume=false). |
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ |
|
🧪 Test Quality Sentinel completed test quality analysis. No test files were added or modified in this PR. All 98 changed files are .lock.yml workflow files (version updates). Test Quality Sentinel skipped. |
There was a problem hiding this comment.
Pull request overview
Updates gh-aw’s OTEL observability specification to report AI Credits Consumed (AIC) and regenerates many compiled workflow lockfiles to tighten safe-outputs validation for issue creation.
Changes:
- Replace the conclusion-span attribute
gh-aw.effective_tokenswithgh-aw.aicin the OTEL observability spec. - Regenerate
.github/workflows/*.lock.ymlfiles to addminLength: 20validation forcreate_issue.body.
Show a summary per file
| File | Description |
|---|---|
| specs/otel-observability-spec.md | Swap conclusion-span token attribute from effective tokens to AIC. |
| .github/workflows/workflow-skill-extractor.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/workflow-normalizer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/workflow-health-manager.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/video-analyzer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/test-workflow.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/super-linter.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/step-name-alignment.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/static-analysis-report.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/stale-repo-identifier.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/spec-librarian.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-workflow-call-with-inputs.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-update-cross-repo-pr.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-temporary-id.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-project.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-pi.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-otel-backends.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-opencode.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-gemini.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-crush.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-create-cross-repo-pr.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-copilot.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-copilot-sdk.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-copilot-arm.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-copilot-aoai-apikey.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-codex.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-claude.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-ci.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/smoke-antigravity.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/sergo.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/semantic-function-refactor.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/security-compliance.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/ruflo-backed-task.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/refactoring-cadence.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/pr-triage-agent.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/poem-bot.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/plan.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/outcome-collector.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/otlp-data-quality-validator.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/objective-impact-report.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/metrics-collector.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/lint-monster.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/issue-arborist.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/hippo-embed.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/gpclean.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/go-pattern-detector.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/go-fan.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/firewall.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/example-permissions-warning.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/duplicate-code-detector.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/discussion-task-miner.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/dev.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/designer-drift-audit.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/deployment-incident-monitor.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/dependabot-go-checker.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/dependabot-burner.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/delight.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/deep-report.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-windows-terminal-integration-builder.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-token-consumption-report.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-testify-uber-super-expert.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-team-status.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-syntax-error-quality.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-spdd-spec-planner.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-skill-optimizer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-security-red-team.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-safeoutputs-git-simulator.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-safe-outputs-conformance.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-safe-output-optimizer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-reliability-review.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-otel-instrumentation-advisor.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-multi-device-docs-tester.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-model-inventory.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-mcp-concurrency-analysis.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-max-ai-credits-test.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-grafana-otel-instrumentation-advisor.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-function-namer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-formal-spec-verifier.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-file-diet.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-doc-healer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-credit-limit-test.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-community-attribution.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-cli-tools-tester.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-cli-performance.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-cache-strategy-analyzer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-byok-ollama-test.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-awf-spec-compiler-surfacing.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-aw-cross-repo-compile-check.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-architecture-diagram.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-ambient-context-optimizer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/daily-agentrx-trace-optimizer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/copilot-opt.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/contribution-check.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/codex-github-remote-mcp-test.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/cli-version-checker.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/cli-consistency-checker.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/ci-doctor.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/breaking-change-checker.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/bot-detection.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/aw-failure-investigator.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/architecture-guardian.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/agentic-token-trend-audit.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/agentic-token-optimizer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/agentic-token-audit.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/agent-persona-explorer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/agent-performance-analyzer.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/ace-editor.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
| .github/workflows/ab-testing-advisor.lock.yml | Add minLength: 20 for create_issue.body validation in embedded JSON config. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 108/108 changed files
- Comments generated: 2
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /grill-with-docs, /zoom-out, and /tdd — requesting changes on spec completeness and test coverage.
📋 Key Themes & Findings
Key Themes
- Breaking spec rename without change log —
gh-aw.effective_tokens→gh-aw.aicis a type change (int → double) and attribute rename; downstream OTel consumers (dashboards, Grafana, alerting) will silently break without a migration note in §16 - Spec/code gap — the spec says
gh-aw.aicis a conditional attribute (MUST when available), but the implementation skips it when AIC = 0; the condition is never defined - Two unrelated concerns bundled — the OTel spec alignment (1 line) and 108 lock file regenerations (minLength catch-up) should be separate PRs
- No minLength tests — the
minLength: 20constraint ships across 108 generated files with no unit test for the boundary
Positive Highlights
- ✅ The implementation in
send_otlp_span.cjsand tests insend_otlp_span.test.cjsalready correctly emitgh-aw.aicand assertgh-aw.effective_tokensis absent — solid prior work - ✅ The spec change accurately reflects the existing implementation (spec-follows-code is fine for a working draft)
- ✅
minLength: 20aligns the compiled artifacts with the already-definedMinIssueBodyLengthconstant — no new surprise behavior
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 978.7 AIC · ⌖ 13.6 AIC
There was a problem hiding this comment.
Non-blocking observations on two distinct changes bundled here.
🔍 What was reviewed
Changes analyzed
-
specs/otel-observability-spec.md— single-line swap:gh-aw.effective_tokens(int) →gh-aw.aic(double). Verified againstactions/setup/js/send_otlp_span.cjs, which already emitsbuildAttr("gh-aw.aic", aiCredits)— so the implementation was already correct; only the spec was wrong. -
100+
.lock.ymlfiles — identical one-line addition:"minLength": 20oncreate_issue.body. Source of truth ispkg/workflow/safe_outputs_validation_config.go:MinIssueBodyLength = 20, which predates this PR. This regeneration silently activates the constraint in the safeoutputs server for all active workflows.
No blocking issues found
- No logic errors or panics in the changed lines.
- No security-adjacent concerns.
- No race conditions introduced.
- The spec change is a correct retroactive doc fix; the
minLengthfloor is intentional and the threshold is sensibly low.
Two comments flag documentation and process gaps worth addressing before or shortly after merge.
🔎 Code quality review by PR Code Quality Reviewer · ⌖ 13.5 AIC
Pull request created by AI Agent