Skip to content

Timeouts do not stop execution; pool can wedge; executionTimeout unenforced #42

Description

@zklapow

Why

Timeouts don't actually stop Python execution, and the consequences compound:

  • On timeout, runOnWasmThread does future.cancel(true) + poison() (PythonExecutorFactory.java:154-161), but interruption is only observed inside host-function callbacks (HostBridge.java:131-136). CPU-bound Python that never calls a host function — exactly what an AI-generated script can produce — keeps running on its pool thread indefinitely.
  • The pool is hardcoded at 10 threads with an unbounded queue (PythonExecutorFactory.java:440-447). Ten runaway scripts deadlock the whole factory with no diagnostics.
  • The README suggests reset() + reuse after poisoning, but reset() clears privatePages (CopyOnWriteMemory.java:546-551) while the abandoned WASM call may still be writing to the same CopyOnWriteMemory from the stuck thread — a data race.
  • ResourceLimits.executionTimeout (ResourceLimits.java:7) is accepted, stored, and returned but never enforced anywhere. Callers reasonably assume it works.

What

  • Wire up a hard-stop mechanism (Chicory epoch interruption / fuel, or a guest-side instruction budget) so timed-out executions actually terminate.
  • Enforce ResourceLimits.executionTimeout or remove it.
  • Make the executor pool size configurable with a bounded queue and rejection diagnostics.
  • Resolve the reset()-while-running race (refuse reset while a call is in flight, or document that poisoned instances must be discarded).

Related cleanup to fold in:

  • PythonInstance.close() only flips a flag (PythonInstance.java:390-394); the per-instance and per-factory WasiPreview1 instances are never closed.
  • PythonExecutorFactory has no close()/shutdown() — the executor and the multi-hundred-MB golden snapshot are pinned forever.
  • While touching this code: add unit tests for CopyOnWriteMemory (most intricate correctness-critical class in the repo; core/ currently has zero unit tests).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions