|
| 1 | +# Research Question 4 |
| 2 | + |
| 3 | +## Q4: Under what conditions is named human authorization itself insufficient for meaningful accountability? |
| 4 | + |
| 5 | +<div class="answer-brief"> |
| 6 | +<p><strong>Answer in brief:</strong> Named human authorization is insufficient for meaningful accountability when the authorizing human cannot evaluate the reasoning chain behind what they are authorizing. GDI's accountability chain satisfies the structural requirement for accountability — a named human is recorded. It does not satisfy the philosophical requirement — that human's understanding of what they are authorizing meets the threshold Santoni de Sio and van den Hoven's tracing condition implies. This is an open problem GDI exposes but does not resolve. <span class="conf-med">Medium Confidence</span> <span class="orig">Original Contribution</span></p> |
| 7 | +</div> |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +### The Problem |
| 12 | + |
| 13 | +GDI's accountability chain requires a named human owner, reviewer, and approver for every governed decision record. This satisfies the structural requirement for accountability: a named agent holds responsibility for the decision. |
| 14 | + |
| 15 | +Santoni de Sio and van den Hoven (2018) proposed that meaningful human control over autonomous systems requires two conditions: a tracking condition (the system responds to relevant human moral reasons) and a tracing condition (behavior is traceable to proper human understanding).[[1]](#cite1) The tracing condition is the hard one. It requires not merely that a human's name appears in a record, but that the human's understanding of what they authorized is sufficient to make that authorization meaningful. |
| 16 | + |
| 17 | +The Right-to-Act paper (arxiv 2604.24153) proved a related point: a high-confidence, policy-compliant decision can still be illegitimate if a required condition is missing. Compensatory scoring systems cannot guarantee legitimacy when required constraints must hold individually.[[2]](#cite2) The comprehension threshold problem extends this: a human who authorized a decision without understanding the model's reasoning chain has satisfied a compensatory accountability requirement — a name is present — while a required condition is missing — their understanding was insufficient. |
| 18 | + |
| 19 | +--- |
| 20 | + |
| 21 | +### The Mechanism of Failure |
| 22 | + |
| 23 | +The failure mode has a specific causal path. A human is presented with a GDR containing a confidence score, an AI prediction, and a recommended action. The human approves the record. The name is logged. The accountability chain is technically complete. The model's reasoning chain — which may involve inference patterns the human cannot evaluate — was not legible to the approving human. If the decision produces a harmful outcome, the governance record shows authorization. The authorization did not reflect comprehension. |
| 24 | + |
| 25 | +This is distinct from rubber-stamping — a human who deliberately approves without engaging. It includes good-faith authorization by a human whose domain expertise does not include the model's reasoning methodology. |
| 26 | + |
| 27 | +The problem appears in high-stakes AI deployments in medicine (clinicians approving AI diagnostic recommendations), finance (loan officers approving AI credit decisions), and pharma (regulatory affairs specialists approving AI-generated submission sections). In each case, domain expertise does not confer competence to evaluate model reasoning. |
| 28 | + |
| 29 | +--- |
| 30 | + |
| 31 | +### GDI's Partial Response |
| 32 | + |
| 33 | +GDI's AI Assistance Disclosure Protocol requires structured disclosure when AI was used: tool identity, purpose, confidence metric, human reviewer identity, override log, and review tier.[[3]](#cite3) The review tier field is the partial response to the comprehension threshold problem. A human reviewer who acknowledges "I reviewed this AI output" at a specific review tier creates a governance record that includes the scope of their review. |
| 34 | + |
| 35 | +This is incomplete. The review tier documents what the reviewer claimed to review. It does not verify that the reviewer understood what they reviewed. The gap between documented review and actual comprehension is the comprehension threshold problem. |
| 36 | + |
| 37 | +--- |
| 38 | + |
| 39 | +### What a More Complete Response Requires |
| 40 | + |
| 41 | +Three elements, none of which GDI currently specifies: |
| 42 | + |
| 43 | +**Comprehension verification.** Some form of active engagement requirement that provides weak evidence the reviewer engaged with the model's reasoning, not just its output. In aviation, crew resource management requires verbal confirmation of key decision parameters — not because it proves understanding, but because it creates a moment of active engagement that passive approval does not.[[4]](#cite4) |
| 44 | + |
| 45 | +**Scope-matched review tier.** The review tier should be defined relative to the decision class: a GREEN-gate routine loan inquiry requires only output review; an AMBER-gate edge case requires evidence review; a RED-gate novel condition requires methodology review. The review tier field should be schema-constrained to the gate classification. |
| 46 | + |
| 47 | +**Comprehension threshold definition.** What minimum understanding satisfies the tracing condition for each decision class? This is a normative question that governance frameworks have not answered. GDI exposes it. Answering it requires collaboration between AI governance practitioners, domain experts, and philosophers of mind. |
| 48 | + |
| 49 | +--- |
| 50 | + |
| 51 | +### Limitations |
| 52 | + |
| 53 | +This question is stated as an open problem GDI exposes, not a problem GDI resolves. The comprehension threshold problem has no published solution. The Right-to-Act paper named a related problem (pre-action legitimacy) and proved non-equivalence results about compensatory scoring systems. It did not address the comprehension dimension. |
| 54 | + |
| 55 | +The honest position: GDI makes the comprehension threshold problem visible and provides partial governance infrastructure around it. Full resolution requires theoretical work outside the scope of this specification. |
| 56 | + |
| 57 | +--- |
| 58 | + |
| 59 | +*[Previous: Q3 ←](q3.md) | [Next: Q5 →](q5.md)* |
| 60 | + |
| 61 | +--- |
| 62 | + |
| 63 | +## References |
| 64 | + |
| 65 | +<a id="cite1"></a>[[1]](#cite1) Santoni de Sio, F., & van den Hoven, J. (2018). Meaningful human control over autonomous systems. *Frontiers in Robotics and AI, 5*, Article 15. |
| 66 | + |
| 67 | +<a id="cite2"></a>[[2]](#cite2) Right-to-Act: A Pre-Execution Non-Compensatory Decision Protocol for AI Systems. arXiv:2604.24153. (2026). |
| 68 | + |
| 69 | +<a id="cite3"></a>[[3]](#cite3) Banasihan, M. J. (2026). *GDI v3.0*. AI Assistance Disclosure Protocol; review tier field definition. |
| 70 | + |
| 71 | +<a id="cite4"></a>[[4]](#cite4) Helmreich, R. L., Merritt, A. C., & Wilhelm, J. A. (1999). The evolution of Crew Resource Management training in commercial aviation. *International Journal of Aviation Psychology, 9*(1), 19–32. |
0 commit comments