Skip to content

Add objective mapping constants and tests#38293

Merged
mnkiefer merged 8 commits into
mainfrom
objective-mapping-spec
Jun 10, 2026
Merged

Add objective mapping constants and tests#38293
mnkiefer merged 8 commits into
mainfrom
objective-mapping-spec

Conversation

@mnkiefer

@mnkiefer mnkiefer commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Add objective mapping and outcomes history subcommand

Summary

Introduces a label-driven objective-value scoring system that maps GitHub issue/PR labels to numeric impact scores, threads that mapping through the existing outcome-evaluation pipeline, adds a new gh aw outcomes history subcommand for scoring recent closed issues and merged PRs, and updates the objective-impact-report workflow to use the new config-backed scoring in place of hardcoded priority tiers.

A companion spec document (specs/objective-mapping-portfolio-reporting.md) defines the full architecture; Phase 1 (this PR) is fully implemented.


Changes

New: pkg/github — Objective mapping foundation

File Description
label_objective_mapping.go New ObjectiveMapping type. Computes numeric objective values from a slice of GitHub labels using configurable sum / max / first multi-label logic. Loads config from env var, .github/objective-mapping.json, or built-in defaults in that precedence order.
label_objective_mapping_constants.go Exported Go constants for all objective label names, their numeric values, and the three multi-label mode strings (max, sum, first).
label_objective_mapping_test.go Unit tests covering all three logic modes, edge cases (no labels, no matches, case-insensitivity, whitespace trimming), and the full LoadObjectiveMappingFromConfig precedence chain.

New: .github/objective-mapping.json

Checked-in JSON configuration mapping GitHub labels to numeric objective values, specifying max as the multi-label aggregation strategy, and providing a ranked priority-labels list used by the impact-efficiency workflow.

Modified: pkg/cli — Outcome evaluation pipeline

File Description
outcome_eval.go Breaking. EvaluateOutcomes and ComputeOutcomeSummary each gain a required *github.ObjectiveMapping parameter. OutcomeReport gains ObjectiveValue, ObjectiveLabels, and TracedRootURL. OutcomeSummary gains TotalObjectiveValue, AcceptedObjectiveValue, ObjectiveEfficiency, and DomainBreakdowns. Adds a GraphQL-backed PR root-tracing pipeline (ghAPIGraphQL, tracePullRequestRoot, enrichOutcomeWithObjectiveValue) to resolve labels from closing issues.
outcome_domain_breakdown.go New file. ComputeDomainBreakdowns aggregates per-label metrics and sorts by total objective value. AnalyzeDomainPerformance classifies domain efficiency into excellent / good / fair / poor / new tiers with human-readable messages and improvement suggestions.
audit_report.go Updated buildAuditData to call github.LoadObjectiveMappingFromConfig and forward the result to the updated EvaluateOutcomes and ComputeOutcomeSummary.
outcomes_command.go Registers the new outcomes history subcommand and threads the loaded ObjectiveMapping through both EvaluateOutcomes and ComputeOutcomeSummary call sites.
outcome_eval_test.go Updated all existing call sites to pass github.DefaultObjectiveMapping(). Added TestEnrichOutcomeWithObjectiveValue_TracesPullRequestToRootIssue and TestEnrichOutcomeWithObjectiveValue_FallsBackToDirectLabels with injectable API stubs.

New: pkg/cli/outcomes_history.go + outcomes_history_test.go

Implements the gh aw outcomes history subcommand. Fetches recent closed issues and merged PRs from GitHub, scores each against the objective label mapping, and renders a bucketed summary with representative items to stderr (or JSON to stdout with --json). Tests cover subcommand registration, historical report building, JSON output mode, and verbose flag inheritance.

Modified: .github/workflows/objective-impact-report.md

Rewrites the workflow to consume objective-mapping.json-based scoring, removing all hardcoded P0–P3 tiers and milestone/project bonuses. Adds a close-issue safe-output permission so stale same-day reports are replaced before a new one is created. Formalises a deterministic AIC source-of-truth ordering that prefers precomputed gh aw logs --json data over live MCP retrieval.

New: specs/objective-mapping-portfolio-reporting.md

Specification covering implementation scope (Phase 1 complete, Phase 2+ as future extensions), configuration format, root-tracing architecture, campaign alignment, and end-to-end data flow.

Modified: pkg/cli/README.md

Added documentation row for gh aw outcomes history (NewOutcomesHistorySubcommand).


Breaking changes

These changes alter public function signatures in pkg/cli. Any caller of the affected functions must be updated.

Symbol Change
EvaluateOutcomes(...) New required parameter: objectiveMapping *github.ObjectiveMapping
ComputeOutcomeSummary(...) New required parameter: objectiveMapping *github.ObjectiveMapping

All in-tree call sites (audit_report.go, outcomes_command.go, outcome_eval_test.go) have been updated.


Testing

  • pkg/github/label_objective_mapping_test.go — unit tests for all scoring logic modes and config loading precedence
  • pkg/cli/outcome_eval_test.go — updated existing tests; two new tests for PR root-tracing and direct-label fallback using injectable stubs
  • pkg/cli/outcomes_history_test.go — four new tests for the outcomes history subcommand (registration, report building, JSON mode, verbose inheritance)

Generated by PR Description Updater for issue #38293 · 506 AIC · ⌖ 13.4 AIC · ⊞ 19.8K ·

@mnkiefer mnkiefer self-assigned this Jun 10, 2026
Copilot AI review requested due to automatic review settings June 10, 2026 06:36
@mnkiefer

Copy link
Copy Markdown
Collaborator Author

@copilot Resolve merge conflicts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an objective-mapping layer (label → numeric value) and threads it through outcome evaluation so gh aw outcomes (and related reporting) can compute objective value/efficiency from traced “root” GitHub objects (e.g., PR → closing issue labels). It also adds a new gh aw outcomes history subcommand plus repository configuration and workflow-spec updates to support impact/efficiency reporting.

Changes:

  • Add ObjectiveMapping loading (env/config/default), label scoring logic, and unit tests.
  • Enrich outcome evaluation with objective_value, objective_labels, traced_root_url, plus summary/domain breakdown aggregation.
  • Add outcomes history CLI subcommand and repo/workflow artifacts (.github/objective-mapping.json, updated impact-report workflow spec/lock, and a detailed design spec).
Show a summary per file
File Description
specs/objective-mapping-portfolio-reporting.md New end-to-end specification for objective mapping, root tracing, and future portfolio reporting.
pkg/github/label_objective_mapping.go Implements mapping model, scoring logic, and config/env/default loading.
pkg/github/label_objective_mapping_test.go Adds unit tests for mapping scoring and config loading behavior.
pkg/github/label_objective_mapping_constants.go Adds label/value constants intended to describe default objective labels/values.
pkg/cli/README.md Documents the new gh aw outcomes history subcommand.
pkg/cli/outcomes_history.go Implements historical scoring of recent closed issues/merged PRs via objective mapping.
pkg/cli/outcomes_history_test.go Adds tests for the history report builder and JSON output path.
pkg/cli/outcomes_command.go Wires objective mapping into the outcomes command execution flow and registers the history subcommand.
pkg/cli/outcome_eval.go Adds objective enrichment + PR→closing-issue tracing and extends OutcomeReport/OutcomeSummary.
pkg/cli/outcome_eval_test.go Updates tests for new function signatures and adds tests for objective enrichment/root tracing.
pkg/cli/outcome_domain_breakdown.go New domain/objective breakdown aggregation + insight helper.
pkg/cli/audit_report.go Wires objective mapping into audit report outcome evaluation/summary computation.
.github/workflows/objective-impact-report.md Updates the workflow spec to use objective mapping + adds safe-output close-issue behavior.
.github/workflows/objective-impact-report.lock.yml Regenerates the compiled lock workflow to reflect the spec changes (adds close_issue).
.github/objective-mapping.json Adds repo-local objective mapping configuration used by the CLI.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 15/15 changed files
  • Comments generated: 9

Comment thread pkg/cli/outcomes_history_test.go
Comment thread pkg/cli/outcomes_history_test.go
Comment thread pkg/cli/outcomes_history_test.go
Comment thread pkg/github/label_objective_mapping.go
Comment thread pkg/github/label_objective_mapping_constants.go
Comment thread pkg/github/label_objective_mapping_constants.go Outdated
Comment thread pkg/cli/outcomes_history.go
Comment thread pkg/cli/outcome_eval.go
Comment thread pkg/cli/outcome_eval.go
mnkiefer and others added 6 commits June 10, 2026 08:45
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@mnkiefer

Copy link
Copy Markdown
Collaborator Author

@copilot Fix remaining review comments by github-actions or copilot and reolve merge conflicts.

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>

Copilot AI commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

@copilot Resolve merge conflicts

Merged main into the branch in commit 981c0b2. Resolved the conflict in .github/workflows/objective-impact-report.lock.yml (updated firewall version to 0.27.0 via make recompile), fixed malformed struct literals in the test file, and addressed lint issues (assert.NotEmpty, assert.Contains, strconv.Itoa, errors.New).

@mnkiefer mnkiefer merged commit 703c5df into main Jun 10, 2026
1 check failed
@mnkiefer mnkiefer deleted the objective-mapping-spec branch June 10, 2026 06:59
Copilot stopped work on behalf of mnkiefer due to an error June 10, 2026 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants