You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs/adr: ADR-034 Phase C — global-slot gap resolved (5258da6)
The global task-slot allocator landed in 5258da6, closing the
global-slot-uniqueness gap the Phase B adversarial review surfaced. Per the
ADR-edit protocol the body is updated to current truth and the prior Residual-
Risk wording moves to the Divergence appendix. User approved all five edits.
What:
- §3: the slot-reuse-protection clause now states the actual mechanism — the
slot's global TASK_SLOT_BITMAP bit is released *after* the generation bump,
so a later claimer observes the new generation. (Pre-allocator, "under the
SCHEDULER lock so no create can grab it" only held same-CPU; a different
CPU's local scan could grab the index regardless. Now true by construction.)
- §4: adds a "Slot selection is globally arbitrated" bullet — the lock-free
TASK_SLOT_BITMAP (claim_task_slot / release_task_slot) CAS-claims each slot
before the per-CPU insert; create_isr_task takes the pre-claimed slot.
- Phasing: Phase C — DONE (5258da6): the allocator, create_task #[cfg(test)]
gating, and the 6-reviewer adversarial pass that hardened the loader to fail
safe on the occupied-slot case (ScheduleError::SlotOccupied).
- Residual Risks: the "Task slots are not a globally-arbitrated namespace"
active risk → a "*(Resolved 5258da6 — Phase C, recorded for history …)*"
note (the same in-line resolved-note style the section already uses for the
cross-CPU foreign-SCHEDULER item; no strikethrough ghost).
- Divergence: the 2026-06-04 "assumed, not enforced" entry gains a "→ Closed
(Phase C 5258da6)" clause and preserves the full prior Residual-Risk wording
verbatim (moved from the body, per the body=current-truth / appendix=history
protocol).
Why: ADR-034's body should read as current truth top-to-bottom — the global-slot
gap is no longer residual. Superseded wording lives in the Divergence appendix.
Verification:
- make check-adrs: 36 ADRs scanned, cross-refs valid, INDEX regenerated
(unchanged — status/title untouched).
- make check-deferrals: pass (485/485, 0 new) after the baseline regen; the
one ADR-034 "deferral" token drifted 151→153 (line-drift only, no new token).
Staged files:
docs/adr/034-deferred-task-resource-reclamation.md
tools/check-deferrals-baseline.txt
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/adr/034-deferred-task-resource-reclamation.md
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -55,7 +55,7 @@ The predicate **"free iff (logically-dead ∧ zero-occupancy)"** does not change
55
55
56
56
The death path reclaims caps/channels/clusters/VMAs/heap **inline, unchanged**. For the self-referential set it instead **captures, by value, at hand-off time**`{ root_phys, kernel_stack region, (cpu, slot) }` — never by re-resolving `desc.cr3` or `task.process_id` later, since the inline ProcessId generation bump may have invalidated/reused those ids — marks the task Terminated, enqueues, and yields.
57
57
58
-
**Per-CPU reapers.** A **per-CPU idle-loop reaper** (`reaper::drain_local`, run from each CPU's idle loop — *the vehicle diverged from the originally-specified dedicated task; see Divergence*) drains its CPU's bounded pending-reclaim queue in **normal context** (interrupts enabled, `SCHEDULER` not held). Per-CPU is deliberate: a dying task's slot lives in *its own* CPU's `PER_CPU_SCHEDULER`, so a same-CPU reaper frees it **locally** — no foreign-CPU `SCHEDULER` acquisition, which dissolves the cross-CPU-scheduler deadlock surface entirely. Frame frees go to the global `FRAME_ALLOCATOR`. The reaper frees, **by the values captured at hand-off**: the root + its intermediate frames (`reclaim_process_page_tables`, now operating on a non-active root), the kernel stack, and the local `(cpu, slot)` TCB. The three resources have **distinct lifetimes** and are freed independently: the root once occupancy is 0; the stack once no CPU's RSP/SP is within it; the slot as soon as it is drained (`Scheduler::reap_slot`) — its exit status was latched into the `TaskExitRing` at exit time (§4) so it is never pinned by a waiter; `reap_slot` frees it only if the slot still holds that exact `(slot, generation)` Terminated task (guarding double-drain or a slot already reused) and bumps the slot's generation under the same `SCHEDULER` lock so no `create_*_task` can grab it with the stale generation. *(Global-reaper-with-ascending-CPU-index is the fallback if per-CPU reapers prove too heavy; it reintroduces the foreign-`SCHEDULER` surface and must then follow the existing `migrate_task` ascending-index protocol.)*
58
+
**Per-CPU reapers.** A **per-CPU idle-loop reaper** (`reaper::drain_local`, run from each CPU's idle loop — *the vehicle diverged from the originally-specified dedicated task; see Divergence*) drains its CPU's bounded pending-reclaim queue in **normal context** (interrupts enabled, `SCHEDULER` not held). Per-CPU is deliberate: a dying task's slot lives in *its own* CPU's `PER_CPU_SCHEDULER`, so a same-CPU reaper frees it **locally** — no foreign-CPU `SCHEDULER` acquisition, which dissolves the cross-CPU-scheduler deadlock surface entirely. Frame frees go to the global `FRAME_ALLOCATOR`. The reaper frees, **by the values captured at hand-off**: the root + its intermediate frames (`reclaim_process_page_tables`, now operating on a non-active root), the kernel stack, and the local `(cpu, slot)` TCB. The three resources have **distinct lifetimes** and are freed independently: the root once occupancy is 0; the stack once no CPU's RSP/SP is within it; the slot as soon as it is drained (`Scheduler::reap_slot`) — its exit status was latched into the `TaskExitRing` at exit time (§4) so it is never pinned by a waiter; `reap_slot` frees it only if the slot still holds that exact `(slot, generation)` Terminated task (guarding double-drain or a slot already reused) and bumps the slot's generation under the same `SCHEDULER` lock; the slot's global `TASK_SLOT_BITMAP` bit (§4) is then released *after* the bump, so a later claimer of the slot is guaranteed to observe the new generation — no `create_*_task` can reuse it with the stale one. *(Global-reaper-with-ascending-CPU-index is the fallback if per-CPU reapers prove too heavy; it reintroduces the foreign-`SCHEDULER` surface and must then follow the existing `migrate_task` ascending-index protocol.)*
59
59
60
60
**Enqueue-once.** Only the context that wins the `Running → Terminated` transition (published under `SCHEDULER(1)`) enqueues the self-referential set. A second enqueue — double-exit, or an [ADR-019](019-process-fault-reaping-and-peer-generation.md) fault racing a clean exit — would double-free the root/stack. The frame allocator's `is_set` pre-pass turns most double-frees into a typed `DoubleFree` (no corruption), but a free-then-realloc-then-free window is real, so single-enqueue is gated on the state transition, not left to the allocator to catch.
61
61
@@ -77,6 +77,7 @@ Closing the slot leak unmasks a UAF: `handle_wait_task` reads child `exit_code`
77
77
- Latch `(slot, generation, exit_code)` into a bounded **`TaskExitRing`** under the same `SCHEDULER(1)` section that sets Terminated, so the reaper frees the TCB unconditionally (pinning until the parent reads reintroduces the leak for fork-and-forget / parent-exits-first — the shell+games common case). `WaitTask` reads from the ring keyed by `(TaskId, generation)`, returning a typed not-found on age-out (the `RecentExitsRing` / `SYS_GET_PROCESS_PRINCIPAL`-after-exit pattern, applied to task exit codes).
78
78
-**Co-fix H5:**`handle_wait_task` sets `Blocked` without cli-before-block ([dispatcher.rs:2503](../../src/syscalls/dispatcher.rs#L2503)); a prompt reaper widens the lost-wake window into "blocks forever holding an unreapable slot." Route through `block_task()` / disable interrupts before the Blocked write through the yield.
79
79
- The reaper frees the slot **locally** — the dying task enqueued to its own CPU's queue and a Terminated+purged task cannot migrate, so the slot lives on the reaper's CPU (no foreign-`SCHEDULER` acquisition, consistent with §3; *this corrects the originally-specified cross-CPU slot free — see Divergence*). `WaitTask`**does** resolve the child cross-CPU, but only for a *live* child the load balancer may have moved: `task_parent_and_state` reads `(parent, state)` via `PER_CPU_SCHEDULER[child_cpu]` through `TASK_CPU_MAP` (with a bounded retry across one in-flight migration) to authorize and decide whether to block. The exit *code* itself is read from the CPU-agnostic global `TaskExitRing` keyed by `(slot, generation)`, not from a per-CPU slot, so a child that exited on any CPU — including after its slot was reaped — is collected correctly.
80
+
-**Slot *selection* is globally arbitrated** (Phase C). Because `(slot, generation)` is a global identity, slot allocation is global too: a lock-free `TASK_SLOT_BITMAP` (`claim_task_slot` / `release_task_slot`) CAS-claims each slot before the per-CPU insert (`create_isr_task` takes the pre-claimed slot; it no longer scans), and the reaper releases the bit only after the local free + generation bump. Two CPUs running `SYS_SPAWN` concurrently can therefore never produce a duplicate `(slot, generation)` — the per-CPU-local free-slot scan that could is gone.
80
81
- Preserve the exit-time record ordering (Principal → `RecentExitsRing`*before* the generation bump) so [ADR-019](019-process-fault-reaping-and-peer-generation.md) peer-generation and `SYS_GET_PROCESS_PRINCIPAL`-after-exit stay correct. The reaper writes **no** records, only frees captured resources.
81
82
82
83
### 5. Bounded queue — overflow is structurally impossible
@@ -117,6 +118,7 @@ Size each `PER_CPU_RECLAIM_QUEUE` for the per-CPU worst case (concurrent Termina
117
118
-**Phase 0 — DONE** (committed): riscv C1 walk-guard (`reclaim` walks 0..256) + riscv H1 canonical-satp wiring. Surgical, standalone; removes the catastrophic global-HHDM corruption and the satp UAF. The common active-root self-free remains (Phase A).
118
119
-**Phase A — DONE** (06eddd8): per-CPU reaper drain (idle-loop, *not a dedicated task — see Divergence*) + `PER_CPU_RECLAIM_QUEUE`; defer `{ root + intermediates + kernel stack }` (not the slot). Closes the x86_64 + riscv64 triple-fault and the kernel-stack leak. Adds the **release-build active-root guard** (§3), the **enqueue-once** state gate, and the **peer-walk `FRAME_ALLOCATOR` audit** (occupancy class 2). Uses the lock-isolated per-CPU queue (placement signed off).
119
120
-**Phase B — DONE** (B.1 3cea74e, B.2-B.4 b4d10e0): TaskId generation + `TaskExitRing` + `WaitTask` rewrite + H5 co-fix + **local** slot reclaim (*not cross-CPU — see Divergence*) + selective `ChildWait(slot)` wake. Extends the Phase A reaper; no tear-down. A 7-reviewer adversarial pass before landing fixed a HIGH (ring re-check on the resolve's not-found arm) + two LOWs and surfaced the global-slot-uniqueness gap below.
121
+
-**Phase C — DONE** (5258da6): global task-slot allocator. A lock-free `TASK_SLOT_BITMAP` arbitrates the slot namespace (`claim_task_slot` before the per-CPU insert; `release_task_slot` in the reaper after free+bump), closing the global-slot-uniqueness gap the Phase B review surfaced (Residual Risks / Divergence). `create_isr_task` takes a pre-claimed slot; the test-only `create_task` local scan is `#[cfg(test)]`-gated. A 6-reviewer adversarial pass hardened the loader to fail safe on the (unreachable) occupied-slot case (`ScheduleError::SlotOccupied`). Extends the Phase A/B reaper; no tear-down.
120
122
121
123
## Open Problems (Convention 9 triggers)
122
124
@@ -133,7 +135,7 @@ Size each `PER_CPU_RECLAIM_QUEUE` for the per-CPU worst case (concurrent Termina
133
135
- TaskId generation width / wraparound horizon must be Convention-8 tagged (Phase B): mirror `ProcessId`'s `u32`, name the horizon (2³² slot reuses, far beyond v1) in ASSUMPTIONS.md.
134
136
-`TaskExitRing` age-out is best-effort: a waiter slower than the ring's wrap misses the exit code (typed not-found) — the same contract as `RecentExitsRing` for Principal-after-exit. Size the ring; document the semantics so callers don't assume guaranteed delivery.
135
137
-[ADR-019](019-process-fault-reaping-and-peer-generation.md) fault-path capture compatibility: a faulting task's context differs from a clean `SYS_EXIT` (partial operations, fault frame); the substrate must capture `{ root, stack, slot }` correctly from the fault path. Tracked as an ADR-019 integration item.
136
-
- **Task slots are not a globally-arbitrated namespace** (surfaced by the Phase B adversarial review). Each per-CPU scheduler searches its *own* `tasks[]` for a free slot, so two CPUs running `SYS_SPAWN` concurrently can claim the same slot index — yielding two live tasks with an identical `(slot, generation)`, which this phase's reaper, exit ring, and generation guard all assume cannot happen. **Pre-existing** — the bare-`u32` `TaskId` already collided (mis-routed cross-CPU wakes via `TASK_CPU_MAP` last-writer-wins; `migrate_task_between` drops a task on accept-failure); Phase B inherits the "slots are global" assumption that `TASK_CPU_MAP` and migration already make, **without worsening** the danger. Not triggered by the current single-shell spawner; reachable on 2+ CPUs with concurrent cross-CPU spawns. **Fix:** a global task-slot allocator (an atomic free-list claimed before the per-CPU insert), landing as a focused follow-up; the deferral is recorded at `create_isr_task`. **Revisit when** a second concurrent cross-CPU `SYS_SPAWN` call site exists.
138
+
-*(Resolved 5258da6 — Phase C, recorded for history: task slots were **not** a globally-arbitrated namespace — each per-CPU scheduler scanned its own `tasks[]`, so concurrent cross-CPU `SYS_SPAWN` could claim the same index and duplicate `(slot, generation)`. Pre-existing and not worsened by Phase B; unreachable under the single-shell spawner. The lock-free `TASK_SLOT_BITMAP` allocator now CAS-claims each slot before the per-CPU insert, so the namespace is globally arbitrated by construction. Full prior wording in the Divergence appendix.)*
137
139
-*(Resolved, recorded for history: the cross-CPU foreign-`SCHEDULER` deadlock surface is dissolved by per-CPU reapers (§3) — not a residual risk under that topology.)*
138
140
139
141
## Verification hook (CLAIMS.md)
@@ -148,4 +150,4 @@ The body above reflects the implemented design. The original wordings, with the
148
150
149
151
-**Slot reclaim is local, not cross-CPU** (2026-06-04, Phase B b4d10e0). With the per-CPU idle-loop reaper, the slot always lives on the reaper's own CPU (the dying task enqueued to its own CPU's queue; a Terminated+purged task cannot migrate), so the slot free is **local** — the same foreign-`SCHEDULER` dissolution §3 already describes. The cross-CPU resolution that *did* land is in `WaitTask`'s pre-block authorize/state check (`task_parent_and_state`), for a *live* child the load balancer may have moved; the exit *code* is read from the CPU-agnostic global `TaskExitRing`. *Prior wording (§4): "The reaper frees the slot **cross-CPU** (the child may have migrated) via `PER_CPU_SCHEDULER[child_cpu]` resolved through `TASK_CPU_MAP`; `WaitTask` must likewise resolve the child cross-CPU for both its pre-block check and post-yield read."*
150
152
151
-
-**Global task-slot uniqueness is assumed, not enforced** (2026-06-04, Phase B b4d10e0). §1/§4 treat the task slot as a global namespace (as do `TASK_CPU_MAP` and migration). The per-CPU-local allocator does not enforce it — recorded as a new Residual Risk and a Convention-9 deferral at `create_isr_task`. This is the gap between the ADR's assumption and the as-built allocator; the global task-slot allocator that closes it is the next focused change.
153
+
- **Global task-slot uniqueness is assumed, not enforced** (2026-06-04, Phase B b4d10e0) → **Closed** (2026-06-04, Phase C 5258da6). §1/§4 treat the task slot as a global namespace (as do `TASK_CPU_MAP` and migration); the Phase-B per-CPU-local allocator did not enforce it — recorded then as a new Residual Risk and a Convention-9 deferral at `create_isr_task`. Phase C closes the gap: the lock-free `TASK_SLOT_BITMAP` allocator (`claim_task_slot` / `release_task_slot`) now arbitrates the slot namespace, so `(slot, generation)` is globally unique by construction and the as-built design matches §1/§4 again. *Prior Residual-Risk wording (now resolved, moved here from the body): "**Task slots are not a globally-arbitrated namespace** (surfaced by the Phase B adversarial review). Each per-CPU scheduler searches its own `tasks[]` for a free slot, so two CPUs running `SYS_SPAWN` concurrently can claim the same slot index — yielding two live tasks with an identical `(slot, generation)`, which this phase's reaper, exit ring, and generation guard all assume cannot happen. Pre-existing — the bare-`u32` `TaskId` already collided (mis-routed cross-CPU wakes via `TASK_CPU_MAP` last-writer-wins; `migrate_task_between` drops a task on accept-failure); Phase B inherits the 'slots are global' assumption that `TASK_CPU_MAP` and migration already make, without worsening the danger. Not triggered by the current single-shell spawner; reachable on 2+ CPUs with concurrent cross-CPU spawns. Fix: a global task-slot allocator (an atomic free-list claimed before the per-CPU insert), landing as a focused follow-up; the deferral is recorded at `create_isr_task`. Revisit when a second concurrent cross-CPU `SYS_SPAWN` call site exists."*
0 commit comments