Skip to content

Crash-loop: stack overflow (SIGBUS) in CmuxTopProcessSnapshot.processTreeNode when a pane's process tree is ~590+ generations deep #7848

Description

@llamallamaredpajama

Summary

cmux crashes with EXC_BAD_ACCESS (SIGBUS) — "Thread stack size exceeded due to excessive recursion" when any pane's process tree becomes very deep (~590+ generations). Because the deep process chain lives outside cmux, the app then crash-loops: every relaunch dies ~16 seconds in, as soon as the first system-top request runs. Only killing the deep chain (or rebooting) recovers.

Environment

  • cmux 0.64.17 (build 97), latest release at time of report
  • macOS 27.0 (26A5368g), Apple Silicon (M4 Max)
  • Reproduced 5× in one day (same signature every time, recursion depth 593 in each report)

Crash signature

Faulting thread (a spawnClientHandler socket worker handling v2SystemTop):

Thread 61 Crashed:
0  ___chkstk_darwin
1  specialized MutableCollection<>._insertionSort(within:sortedEnd:by:)
2  specialized MutableCollection<>.sort(by:)
3  CmuxTopProcessSnapshot.processTreeNode(pid:allowedPIDs:rootPIDs:visited:)
4  CmuxTopProcessSnapshot.processTreeNode(pid:allowedPIDs:rootPIDs:visited:)
   -------- RECURSION LEVEL 593 ... ELIDED 587 LEVELS ...
597 CmuxTopProcessSnapshot.processTreePayload(for:rootPIDs:)
598 TerminalController.v2AnnotateTopSurface(_:processSnapshot:browserPIDOccurrences:includeProcesses:)
...
602 TerminalController.v2SystemTop(params:)
603 TerminalController.socketWorkerV2Response(_:)
607 TerminalController.handleClient(_:peerPid:)

VM region info confirms the fault address is the thread's stack guard page (secondary pthread stack, 544K).

Root cause

processTreeNode in Sources/CmuxTopSnapshot.swift recurses once per generation of the process tree. The visited cycle guard is correct — the problem is purely depth. On a default 512KB secondary-thread stack, each frame costs ~900 bytes (the inline sort(by:) of children makes frames heavy), so any ancestor chain deeper than ~590 processes overflows the guard page deterministically.

Deep chains are a real-world occurrence with agentic workloads: in my case an overnight multi-agent run repeatedly launched codex exec child agents in the foreground of the previous agent's shell, accumulating a live parent→child chain of 590+ processes in a single pane over several hours. cmux walked it and died — and kept dying on every relaunch until the chain was cleared.

Reproduction

# chain.sh — build an N-deep live process chain in a cmux pane
#!/bin/bash
n=$1
if [ "$n" -gt 0 ]; then bash "$0" $((n-1)); else sleep 600; fi
  1. In a cmux pane: bash chain.sh 700
  2. Trigger the system-top path (task manager / resource annotation, or a v2SystemTop socket request).
  3. App crashes with the signature above; relaunching crashes again until the chain exits.

Suggested fix

Any of these (first is cleanest):

  • Rewrite processTreeNode as an iterative traversal with an explicit stack — depth becomes a heap concern instead of a stack concern.
  • Or cap recursion depth (e.g. 256) and emit a truncated node beyond it.
  • Defensively, run the walk on a thread with a larger explicit stack size — mitigates but doesn't bound.

processTreePayloadprocessTreeNode is the only recursive walk on this path; the sibling loops are already iterative.

Happy to provide the full .ips crash reports privately if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions