Skip to content

Commit 3708a41

Browse files
riccajasonclaude
andcommitted
docs/adr: ADR-034 Phase C — global-slot gap resolved (5258da6)
The global task-slot allocator landed in 5258da6, closing the global-slot-uniqueness gap the Phase B adversarial review surfaced. Per the ADR-edit protocol the body is updated to current truth and the prior Residual- Risk wording moves to the Divergence appendix. User approved all five edits. What: - §3: the slot-reuse-protection clause now states the actual mechanism — the slot's global TASK_SLOT_BITMAP bit is released *after* the generation bump, so a later claimer observes the new generation. (Pre-allocator, "under the SCHEDULER lock so no create can grab it" only held same-CPU; a different CPU's local scan could grab the index regardless. Now true by construction.) - §4: adds a "Slot selection is globally arbitrated" bullet — the lock-free TASK_SLOT_BITMAP (claim_task_slot / release_task_slot) CAS-claims each slot before the per-CPU insert; create_isr_task takes the pre-claimed slot. - Phasing: Phase C — DONE (5258da6): the allocator, create_task #[cfg(test)] gating, and the 6-reviewer adversarial pass that hardened the loader to fail safe on the occupied-slot case (ScheduleError::SlotOccupied). - Residual Risks: the "Task slots are not a globally-arbitrated namespace" active risk → a "*(Resolved 5258da6 — Phase C, recorded for history …)*" note (the same in-line resolved-note style the section already uses for the cross-CPU foreign-SCHEDULER item; no strikethrough ghost). - Divergence: the 2026-06-04 "assumed, not enforced" entry gains a "→ Closed (Phase C 5258da6)" clause and preserves the full prior Residual-Risk wording verbatim (moved from the body, per the body=current-truth / appendix=history protocol). Why: ADR-034's body should read as current truth top-to-bottom — the global-slot gap is no longer residual. Superseded wording lives in the Divergence appendix. Verification: - make check-adrs: 36 ADRs scanned, cross-refs valid, INDEX regenerated (unchanged — status/title untouched). - make check-deferrals: pass (485/485, 0 new) after the baseline regen; the one ADR-034 "deferral" token drifted 151→153 (line-drift only, no new token). Staged files: docs/adr/034-deferred-task-resource-reclamation.md tools/check-deferrals-baseline.txt Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 5258da6 commit 3708a41

2 files changed

Lines changed: 6 additions & 4 deletions

File tree

docs/adr/034-deferred-task-resource-reclamation.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ The predicate **"free iff (logically-dead ∧ zero-occupancy)"** does not change
5555

5656
The death path reclaims caps/channels/clusters/VMAs/heap **inline, unchanged**. For the self-referential set it instead **captures, by value, at hand-off time** `{ root_phys, kernel_stack region, (cpu, slot) }` — never by re-resolving `desc.cr3` or `task.process_id` later, since the inline ProcessId generation bump may have invalidated/reused those ids — marks the task Terminated, enqueues, and yields.
5757

58-
**Per-CPU reapers.** A **per-CPU idle-loop reaper** (`reaper::drain_local`, run from each CPU's idle loop — *the vehicle diverged from the originally-specified dedicated task; see Divergence*) drains its CPU's bounded pending-reclaim queue in **normal context** (interrupts enabled, `SCHEDULER` not held). Per-CPU is deliberate: a dying task's slot lives in *its own* CPU's `PER_CPU_SCHEDULER`, so a same-CPU reaper frees it **locally** — no foreign-CPU `SCHEDULER` acquisition, which dissolves the cross-CPU-scheduler deadlock surface entirely. Frame frees go to the global `FRAME_ALLOCATOR`. The reaper frees, **by the values captured at hand-off**: the root + its intermediate frames (`reclaim_process_page_tables`, now operating on a non-active root), the kernel stack, and the local `(cpu, slot)` TCB. The three resources have **distinct lifetimes** and are freed independently: the root once occupancy is 0; the stack once no CPU's RSP/SP is within it; the slot as soon as it is drained (`Scheduler::reap_slot`) — its exit status was latched into the `TaskExitRing` at exit time (§4) so it is never pinned by a waiter; `reap_slot` frees it only if the slot still holds that exact `(slot, generation)` Terminated task (guarding double-drain or a slot already reused) and bumps the slot's generation under the same `SCHEDULER` lock so no `create_*_task` can grab it with the stale generation. *(Global-reaper-with-ascending-CPU-index is the fallback if per-CPU reapers prove too heavy; it reintroduces the foreign-`SCHEDULER` surface and must then follow the existing `migrate_task` ascending-index protocol.)*
58+
**Per-CPU reapers.** A **per-CPU idle-loop reaper** (`reaper::drain_local`, run from each CPU's idle loop — *the vehicle diverged from the originally-specified dedicated task; see Divergence*) drains its CPU's bounded pending-reclaim queue in **normal context** (interrupts enabled, `SCHEDULER` not held). Per-CPU is deliberate: a dying task's slot lives in *its own* CPU's `PER_CPU_SCHEDULER`, so a same-CPU reaper frees it **locally** — no foreign-CPU `SCHEDULER` acquisition, which dissolves the cross-CPU-scheduler deadlock surface entirely. Frame frees go to the global `FRAME_ALLOCATOR`. The reaper frees, **by the values captured at hand-off**: the root + its intermediate frames (`reclaim_process_page_tables`, now operating on a non-active root), the kernel stack, and the local `(cpu, slot)` TCB. The three resources have **distinct lifetimes** and are freed independently: the root once occupancy is 0; the stack once no CPU's RSP/SP is within it; the slot as soon as it is drained (`Scheduler::reap_slot`) — its exit status was latched into the `TaskExitRing` at exit time (§4) so it is never pinned by a waiter; `reap_slot` frees it only if the slot still holds that exact `(slot, generation)` Terminated task (guarding double-drain or a slot already reused) and bumps the slot's generation under the same `SCHEDULER` lock; the slot's global `TASK_SLOT_BITMAP` bit (§4) is then released *after* the bump, so a later claimer of the slot is guaranteed to observe the new generation — no `create_*_task` can reuse it with the stale one. *(Global-reaper-with-ascending-CPU-index is the fallback if per-CPU reapers prove too heavy; it reintroduces the foreign-`SCHEDULER` surface and must then follow the existing `migrate_task` ascending-index protocol.)*
5959

6060
**Enqueue-once.** Only the context that wins the `Running → Terminated` transition (published under `SCHEDULER(1)`) enqueues the self-referential set. A second enqueue — double-exit, or an [ADR-019](019-process-fault-reaping-and-peer-generation.md) fault racing a clean exit — would double-free the root/stack. The frame allocator's `is_set` pre-pass turns most double-frees into a typed `DoubleFree` (no corruption), but a free-then-realloc-then-free window is real, so single-enqueue is gated on the state transition, not left to the allocator to catch.
6161

@@ -77,6 +77,7 @@ Closing the slot leak unmasks a UAF: `handle_wait_task` reads child `exit_code`
7777
- Latch `(slot, generation, exit_code)` into a bounded **`TaskExitRing`** under the same `SCHEDULER(1)` section that sets Terminated, so the reaper frees the TCB unconditionally (pinning until the parent reads reintroduces the leak for fork-and-forget / parent-exits-first — the shell+games common case). `WaitTask` reads from the ring keyed by `(TaskId, generation)`, returning a typed not-found on age-out (the `RecentExitsRing` / `SYS_GET_PROCESS_PRINCIPAL`-after-exit pattern, applied to task exit codes).
7878
- **Co-fix H5:** `handle_wait_task` sets `Blocked` without cli-before-block ([dispatcher.rs:2503](../../src/syscalls/dispatcher.rs#L2503)); a prompt reaper widens the lost-wake window into "blocks forever holding an unreapable slot." Route through `block_task()` / disable interrupts before the Blocked write through the yield.
7979
- The reaper frees the slot **locally** — the dying task enqueued to its own CPU's queue and a Terminated+purged task cannot migrate, so the slot lives on the reaper's CPU (no foreign-`SCHEDULER` acquisition, consistent with §3; *this corrects the originally-specified cross-CPU slot free — see Divergence*). `WaitTask` **does** resolve the child cross-CPU, but only for a *live* child the load balancer may have moved: `task_parent_and_state` reads `(parent, state)` via `PER_CPU_SCHEDULER[child_cpu]` through `TASK_CPU_MAP` (with a bounded retry across one in-flight migration) to authorize and decide whether to block. The exit *code* itself is read from the CPU-agnostic global `TaskExitRing` keyed by `(slot, generation)`, not from a per-CPU slot, so a child that exited on any CPU — including after its slot was reaped — is collected correctly.
80+
- **Slot *selection* is globally arbitrated** (Phase C). Because `(slot, generation)` is a global identity, slot allocation is global too: a lock-free `TASK_SLOT_BITMAP` (`claim_task_slot` / `release_task_slot`) CAS-claims each slot before the per-CPU insert (`create_isr_task` takes the pre-claimed slot; it no longer scans), and the reaper releases the bit only after the local free + generation bump. Two CPUs running `SYS_SPAWN` concurrently can therefore never produce a duplicate `(slot, generation)` — the per-CPU-local free-slot scan that could is gone.
8081
- Preserve the exit-time record ordering (Principal → `RecentExitsRing` *before* the generation bump) so [ADR-019](019-process-fault-reaping-and-peer-generation.md) peer-generation and `SYS_GET_PROCESS_PRINCIPAL`-after-exit stay correct. The reaper writes **no** records, only frees captured resources.
8182

8283
### 5. Bounded queue — overflow is structurally impossible
@@ -117,6 +118,7 @@ Size each `PER_CPU_RECLAIM_QUEUE` for the per-CPU worst case (concurrent Termina
117118
- **Phase 0 — DONE** (committed): riscv C1 walk-guard (`reclaim` walks 0..256) + riscv H1 canonical-satp wiring. Surgical, standalone; removes the catastrophic global-HHDM corruption and the satp UAF. The common active-root self-free remains (Phase A).
118119
- **Phase A — DONE** (06eddd8): per-CPU reaper drain (idle-loop, *not a dedicated task — see Divergence*) + `PER_CPU_RECLAIM_QUEUE`; defer `{ root + intermediates + kernel stack }` (not the slot). Closes the x86_64 + riscv64 triple-fault and the kernel-stack leak. Adds the **release-build active-root guard** (§3), the **enqueue-once** state gate, and the **peer-walk `FRAME_ALLOCATOR` audit** (occupancy class 2). Uses the lock-isolated per-CPU queue (placement signed off).
119120
- **Phase B — DONE** (B.1 3cea74e, B.2-B.4 b4d10e0): TaskId generation + `TaskExitRing` + `WaitTask` rewrite + H5 co-fix + **local** slot reclaim (*not cross-CPU — see Divergence*) + selective `ChildWait(slot)` wake. Extends the Phase A reaper; no tear-down. A 7-reviewer adversarial pass before landing fixed a HIGH (ring re-check on the resolve's not-found arm) + two LOWs and surfaced the global-slot-uniqueness gap below.
121+
- **Phase C — DONE** (5258da6): global task-slot allocator. A lock-free `TASK_SLOT_BITMAP` arbitrates the slot namespace (`claim_task_slot` before the per-CPU insert; `release_task_slot` in the reaper after free+bump), closing the global-slot-uniqueness gap the Phase B review surfaced (Residual Risks / Divergence). `create_isr_task` takes a pre-claimed slot; the test-only `create_task` local scan is `#[cfg(test)]`-gated. A 6-reviewer adversarial pass hardened the loader to fail safe on the (unreachable) occupied-slot case (`ScheduleError::SlotOccupied`). Extends the Phase A/B reaper; no tear-down.
120122

121123
## Open Problems (Convention 9 triggers)
122124

@@ -133,7 +135,7 @@ Size each `PER_CPU_RECLAIM_QUEUE` for the per-CPU worst case (concurrent Termina
133135
- TaskId generation width / wraparound horizon must be Convention-8 tagged (Phase B): mirror `ProcessId`'s `u32`, name the horizon (2³² slot reuses, far beyond v1) in ASSUMPTIONS.md.
134136
- `TaskExitRing` age-out is best-effort: a waiter slower than the ring's wrap misses the exit code (typed not-found) — the same contract as `RecentExitsRing` for Principal-after-exit. Size the ring; document the semantics so callers don't assume guaranteed delivery.
135137
- [ADR-019](019-process-fault-reaping-and-peer-generation.md) fault-path capture compatibility: a faulting task's context differs from a clean `SYS_EXIT` (partial operations, fault frame); the substrate must capture `{ root, stack, slot }` correctly from the fault path. Tracked as an ADR-019 integration item.
136-
- **Task slots are not a globally-arbitrated namespace** (surfaced by the Phase B adversarial review). Each per-CPU scheduler searches its *own* `tasks[]` for a free slot, so two CPUs running `SYS_SPAWN` concurrently can claim the same slot index — yielding two live tasks with an identical `(slot, generation)`, which this phase's reaper, exit ring, and generation guard all assume cannot happen. **Pre-existing** — the bare-`u32` `TaskId` already collided (mis-routed cross-CPU wakes via `TASK_CPU_MAP` last-writer-wins; `migrate_task_between` drops a task on accept-failure); Phase B inherits the "slots are global" assumption that `TASK_CPU_MAP` and migration already make, **without worsening** the danger. Not triggered by the current single-shell spawner; reachable on 2+ CPUs with concurrent cross-CPU spawns. **Fix:** a global task-slot allocator (an atomic free-list claimed before the per-CPU insert), landing as a focused follow-up; the deferral is recorded at `create_isr_task`. **Revisit when** a second concurrent cross-CPU `SYS_SPAWN` call site exists.
138+
- *(Resolved 5258da6 — Phase C, recorded for history: task slots were **not** a globally-arbitrated namespace — each per-CPU scheduler scanned its own `tasks[]`, so concurrent cross-CPU `SYS_SPAWN` could claim the same index and duplicate `(slot, generation)`. Pre-existing and not worsened by Phase B; unreachable under the single-shell spawner. The lock-free `TASK_SLOT_BITMAP` allocator now CAS-claims each slot before the per-CPU insert, so the namespace is globally arbitrated by construction. Full prior wording in the Divergence appendix.)*
137139
- *(Resolved, recorded for history: the cross-CPU foreign-`SCHEDULER` deadlock surface is dissolved by per-CPU reapers (§3) — not a residual risk under that topology.)*
138140

139141
## Verification hook (CLAIMS.md)
@@ -148,4 +150,4 @@ The body above reflects the implemented design. The original wordings, with the
148150

149151
- **Slot reclaim is local, not cross-CPU** (2026-06-04, Phase B b4d10e0). With the per-CPU idle-loop reaper, the slot always lives on the reaper's own CPU (the dying task enqueued to its own CPU's queue; a Terminated+purged task cannot migrate), so the slot free is **local** — the same foreign-`SCHEDULER` dissolution §3 already describes. The cross-CPU resolution that *did* land is in `WaitTask`'s pre-block authorize/state check (`task_parent_and_state`), for a *live* child the load balancer may have moved; the exit *code* is read from the CPU-agnostic global `TaskExitRing`. *Prior wording (§4): "The reaper frees the slot **cross-CPU** (the child may have migrated) via `PER_CPU_SCHEDULER[child_cpu]` resolved through `TASK_CPU_MAP`; `WaitTask` must likewise resolve the child cross-CPU for both its pre-block check and post-yield read."*
150152

151-
- **Global task-slot uniqueness is assumed, not enforced** (2026-06-04, Phase B b4d10e0). §1/§4 treat the task slot as a global namespace (as do `TASK_CPU_MAP` and migration). The per-CPU-local allocator does not enforce it — recorded as a new Residual Risk and a Convention-9 deferral at `create_isr_task`. This is the gap between the ADR's assumption and the as-built allocator; the global task-slot allocator that closes it is the next focused change.
153+
- **Global task-slot uniqueness is assumed, not enforced** (2026-06-04, Phase B b4d10e0) → **Closed** (2026-06-04, Phase C 5258da6). §1/§4 treat the task slot as a global namespace (as do `TASK_CPU_MAP` and migration); the Phase-B per-CPU-local allocator did not enforce it — recorded then as a new Residual Risk and a Convention-9 deferral at `create_isr_task`. Phase C closes the gap: the lock-free `TASK_SLOT_BITMAP` allocator (`claim_task_slot` / `release_task_slot`) now arbitrates the slot namespace, so `(slot, generation)` is globally unique by construction and the as-built design matches §1/§4 again. *Prior Residual-Risk wording (now resolved, moved here from the body): "**Task slots are not a globally-arbitrated namespace** (surfaced by the Phase B adversarial review). Each per-CPU scheduler searches its own `tasks[]` for a free slot, so two CPUs running `SYS_SPAWN` concurrently can claim the same slot index — yielding two live tasks with an identical `(slot, generation)`, which this phase's reaper, exit ring, and generation guard all assume cannot happen. Pre-existing — the bare-`u32` `TaskId` already collided (mis-routed cross-CPU wakes via `TASK_CPU_MAP` last-writer-wins; `migrate_task_between` drops a task on accept-failure); Phase B inherits the 'slots are global' assumption that `TASK_CPU_MAP` and migration already make, without worsening the danger. Not triggered by the current single-shell spawner; reachable on 2+ CPUs with concurrent cross-CPU spawns. Fix: a global task-slot allocator (an atomic free-list claimed before the per-CPU insert), landing as a focused follow-up; the deferral is recorded at `create_isr_task`. Revisit when a second concurrent cross-CPU `SYS_SPAWN` call site exists."*

tools/check-deferrals-baseline.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ docs/adr/031-unified-storage-substrate.md:24:deferred
6262
docs/adr/033-multi-principal-vault.md:149:deferred
6363
docs/adr/034-deferred-task-resource-reclamation.md:11:deferred
6464
docs/adr/034-deferred-task-resource-reclamation.md:16:deferred
65-
docs/adr/034-deferred-task-resource-reclamation.md:151:deferred
65+
docs/adr/034-deferred-task-resource-reclamation.md:153:deferred
6666
docs/adr/035-translation-layer-compatibility.md:54:deferred
6767
docs/identity-synthesis.md:124:deferred
6868
docs/manuals/03-signature-chain.md:67:placeholder

0 commit comments

Comments
 (0)