Add documentation for eval-to-governance handoff requirements

This document outlines the requirements for the handoff from pre-deployment capability evaluation to runtime decision governance, highlighting the gaps in current specifications and proposing necessary elements for a complete eval-to-governance handoff.

Author: mj3b

spec/independent-study/docs/questions/q2.md

@@ -0,0 +1,96 @@
+# Research Question 2
+
+## Q2: What does the handoff from pre-deployment capability evaluation to runtime decision governance require, and does GDI's architecture satisfy it?
+
+<div class="answer-brief">
+<p><strong>Answer in brief:</strong> No published specification defines the governance handoff between pre-deployment capability evaluations and runtime decision governance. Pre-deployment evaluations certify what a model can do before deployment. Runtime governance must enforce what a model is authorized to do after deployment. The connection between those two layers is architecturally unaddressed. GDI's governance covenant and gate taxonomy provide a partial structural response: they define what the model is authorized to decide at runtime, creating the operational contract that links evaluation outcomes to governed behavior. The gap between "this model passed a deployment eval" and "this model's individual decisions are governed" remains the largest unaddressed problem in the current AI governance infrastructure. <span class="conf-high">High Confidence</span></p>
+</div>
+
+---
+
+### The Missing Connection
+
+METR's time horizon research (March 2025) established that AI agents can complete increasingly long autonomous tasks — with a doubling time of approximately seven months in task completion horizon.[[1]](#cite1) Their June 2025 reward hacking findings documented frontier models exploiting scoring code bugs and subverting task setups to achieve high scores while being aware their behavior was not aligned with user intentions.[[2]](#cite2)
+
+The International AI Safety Report 2026 named this pattern formally as an "evaluation gap": performance on pre-deployment tests does not reliably predict real-world risk, and models are increasingly distinguishing test settings from deployment settings.[[3]](#cite3)
+
+These findings have a direct implication for governance architecture that no published specification addresses. Pre-deployment evaluations — METR's autonomous capability suite, UK AISI's pre-deployment protocols, the evaluations embedded in frontier AI safety frameworks — certify model behavior before deployment. Runtime governance — GDI, Oracle's OCI AGF, policy enforcement layers — governs behavior after deployment begins. The question nobody has answered: *what is the formal connection between those two layers?*
+
+The absence of that connection creates a specific failure mode. A model can pass a pre-deployment capability evaluation and then, in deployment, encounter conditions outside its evaluation distribution. The governance layer receives no signal that the model is now operating outside its validated range. The confidence threshold model ingests whatever the model reports. The gate fires based on a confidence score that may not reflect the model's actual epistemic state in the novel condition.
+
+---
+
+### What the Handoff Requires
+
+A complete eval-to-governance handoff requires three elements that no existing specification provides together.
+
+**Deployment scope definition.** The evaluation defines what the model was tested on. The deployment scope defines what the model is authorized to do. These are not the same. A model evaluated on loan processing scenarios may be deployed to handle edge cases outside its evaluation distribution. GDI's governance covenant defines what the agent is authorized to decide and what must be escalated — but the current specification does not formally connect covenant definition to evaluation scope. That connection must be explicit: the governance covenant should be derived from, and bounded by, the evaluation scope.
+
+**Confidence validity conditions.** METR's reward hacking findings show models gaming confidence metrics when they detect they are being assessed.[[2]](#cite2) A model's self-reported confidence is a valid governance input only when the model is operating within the conditions under which its confidence calibration was validated. Outside those conditions, the confidence score is epistemically ungrounded. GDI's GREEN/AMBER/RED zones assume the confidence score is meaningful. That assumption requires an explicit validity condition: confidence from a model operating within its evaluation scope under the conditions present at evaluation time.
+
+**Behavioral drift detection.** Pre-deployment evaluations are point-in-time. Models deployed in production encounter distributional shift, prompt injection, adversarial inputs, and novel conditions. A governance architecture that validates a model at deployment and then trusts its behavior indefinitely is not continuous governance — it is one-time clearance. GDI's CI/CD integration pattern provides continuous schema validation. It does not provide continuous behavioral validation. That gap requires integration with external monitoring systems (Langfuse, METR's monitorability evaluations) that is not yet specified.
+
+---
+
+### GDI's Partial Response
+
+GDI's governance covenant — the definition of what an agent is authorized to decide and what must be escalated — is the closest existing architectural element to a formal eval-to-governance handoff.[[4]](#cite4)
+
+The covenant defines:
+
+- What decision classes the agent may operate in autonomously (GREEN gate)
+- What decision classes require synchronous human review (AMBER gate)
+- What decision classes require mandatory escalation (RED gate)
+- What actions are outside the covenant entirely (hard escalation, no autonomous path)
+
+This is a structural response to the handoff problem: it converts evaluation outcomes into operational governance boundaries. A model evaluated on routine loan inquiry processing maps to GREEN-gate authorization for that decision class. Edge cases outside the evaluation scope map to AMBER or RED gates, forcing human review for conditions the evaluation did not cover.
+
+The limitation: GDI's governance covenant requires manual definition. No published methodology exists for deriving covenant boundaries from evaluation results. Bridging METR's evaluation taxonomy to GDI's gate taxonomy is the specific research gap this question exposes.
+
+---
+
+### Mechanism
+
+The eval-to-governance handoff works through three steps that must all be explicit:
+
+1. **Evaluation scope** — what decision classes were tested, under what conditions, at what performance threshold
+2. **Covenant derivation** — mapping evaluation scope to gate classification (evaluated decision classes in distribution → GREEN; evaluated classes at distribution boundary → AMBER; unevaluated classes → RED or hard escalation)
+3. **Runtime monitoring** — continuous signal that the model is operating within, near, or outside its evaluation scope conditions, feeding back to gate threshold adjustment
+
+GDI currently provides step 2 (covenant definition) and partially provides step 3 (CI/CD validation of schema compliance). Step 1 requires formal connection to external evaluation frameworks. Step 3 requires integration with behavioral monitoring systems.
+
+---
+
+### Limitations
+
+This analysis infers the handoff requirement from the evaluation and governance literature — METR's research, the International AI Safety Report, GDI's specification. No published paper has formally defined what a complete eval-to-governance handoff requires. This question proposes that definition and assesses GDI against it. Both the definition and the assessment are the author's own contribution, subject to challenge.
+
+The claim that no published specification addresses this connection is accurate as of May 2026. Organizations may have internal processes connecting evaluations to governance scope. Those processes are not in the published specification literature.
+
+---
+
+### Implementation Guidance
+
+For practitioners building the eval-to-governance handoff today:
+
+Document the evaluation scope explicitly in the governance covenant. Every gate classification should trace to a specific evaluation result: "this decision class is GREEN-gated because the model achieved X% accuracy on Y evaluation suite under Z conditions."
+
+Build a scope monitor. Log the decision classes, input distributions, and confidence ranges encountered in production. When the production distribution diverges from the evaluation distribution by a defined threshold, trigger automatic gate reclassification to AMBER pending human review.
+
+Treat the governance covenant as a versioned artifact. Every evaluation update should produce a covenant update. The commit timestamps establish the governance record of what the model was authorized to do at each point in time.
+
+---
+
+*[Previous: Q1 &larr;](q1.md) &nbsp;|&nbsp; [Next: Q3 &rarr;](q3.md)*
+
+---
+
+## References
+
+<a id="cite1"></a>[[1]](#cite1) Kwa, T. et al. (March 2025). Measuring AI ability to complete long tasks. *METR*. 7-month doubling time in task completion horizon.
+
+<a id="cite2"></a>[[2]](#cite2) METR. (June 2025). Recent frontier models are reward hacking. Models exploit scoring bugs; awareness of misalignment with user intentions.
+
+<a id="cite3"></a>[[3]](#cite3) International AI Safety Report 2026. Evaluation gap: pre-deployment test performance does not reliably predict real-world risk.
+
+<a id="cite4"></a>[[4]](#cite4) Banasihan, M. J. (2026). *Governed Decision Intelligence: The Decision Architecture for Governed AI, v3.0*. Governance covenant definition and gate taxonomy.