Docker-in-Docker guide repro has no Docker socket in local Sandbox

[ericclemmons]

Docker-in-Docker guide repro: docker:dind-rootless Sandbox has no Docker socket locally

Summary

I created a minimal Worker + Sandbox repro following the Docker-in-Docker guide:

https://developers.cloudflare.com/sandbox/guides/docker-in-docker/#create-a-docker-enabled-image

The Sandbox container starts and sandbox.exec(...) works, but Docker is not available inside the Sandbox. docker version cannot connect to /var/run/docker.sock. Listing Sandbox background processes shows no startup process from the Dockerfile CMD. Manually starting the documented boot script produces Docker/rootful mount errors and still does not create a usable Docker socket.

Repro

Repository:

https://github.com/ericclemmons/sandbox-dind-repro

Run locally:

pnpm install
pnpm dev

In another terminal:

curl -sS http://localhost:8789/docker-version
curl -sS http://localhost:8789/processes
curl -sS http://localhost:8789/start-docker
curl -sS http://localhost:8789/docker-version

Expected Behavior

Because the Dockerfile matches the guide, the Sandbox container CMD should start Docker and keep it running:

ENTRYPOINT ["/sandbox"]
CMD ["/home/rootless/boot-docker-for-dind.sh"]

Then sandbox.exec("docker version") should show both Docker client and server info.

Actual Behavior

/docker-version:

{
  "success": false,
  "exitCode": 1,
  "stdout": "Client:\n Version:           29.4.1\n API version:       1.54\n Go version:        go1.26.2\n Git commit:        055a478\n Built:             Mon Apr 20 16:31:59 2026\n OS/Arch:           linux/amd64\n Context:           default",
  "stderr": "failed to connect to the docker API at unix:///var/run/docker.sock; check if the path is correct and if the daemon is running: dial unix /var/run/docker.sock: connect: no such file or directory"
}

/processes before manual startup:

{"processes":[]}

/start-docker manually starts /home/rootless/boot-docker-for-dind.sh, but logs include:

Device "nf_tables" does not exist.
modprobe: can't change directory to '/lib/modules': No such file or directory
Device "ip_tables" does not exist.
modprobe: can't change directory to '/lib/modules': No such file or directory
Device "ip6_tables" does not exist.
modprobe: can't change directory to '/lib/modules': No such file or directory
mount: permission denied (are you root?)
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied (are you root?)

After manual startup, /docker-version still fails with missing /var/run/docker.sock.

Environment

• macOS / local Wrangler dev
• @cloudflare/sandbox@0.9.2
• wrangler@4.86.0
• Docker image copies cloudflare/sandbox:0.9.2-musl into docker:dind-rootless

Question

Is the Docker-in-Docker guide expected to work in local Wrangler dev, or only after deploying to Cloudflare Containers? If it should work locally, is there an additional container capability/security setting required for docker:dind-rootless to create the Docker socket?

[ericclemmons]

Update after deploying the minimal repro to production:

Repro repo:
https://github.com/ericclemmons/sandbox-dind-repro

Production Worker URL:
https://sandbox-dind-repro.ericclemmons.workers.dev/

Container instance URL from dashboard:
https://dash.cloudflare.com/?to=/:account/workers/containers/sandbox-dind-repro-sandbox

Local Wrangler behavior

Local repro worker was running on port 8789.

GET /docker-version fails consistently:

failed to connect to the docker API at unix:///var/run/docker.sock; check if the path is correct and if the daemon is running: dial unix /var/run/docker.sock: connect: no such file or directory

GET /processes returns no startup process:

{"processes":[]}

Manually starting the guide's boot script via GET /start-docker starts a background process, but Docker still does not become usable. Logs include variants of:

Device "nf_tables" does not exist.
modprobe: can't change directory to '/lib/modules': No such file or directory
Device "ip_tables" does not exist.
modprobe: can't change directory to '/lib/modules': No such file or directory
Device "ip6_tables" does not exist.
modprobe: can't change directory to '/lib/modules': No such file or directory
mount: permission denied (are you root?)
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied (are you root?)

When forcing the script down the rootless path during earlier debugging, local Docker/rootlesskit failed with:

[rootlesskit:parent] error: failed to start the child: fork/exec /proc/self/exe: operation not permitted

So locally, the Docker daemon/socket never becomes available.

Production behavior

After deploying the same repro, production behaves differently.

A clean sequence was:

curl -sS https://sandbox-dind-repro.ericclemmons.workers.dev/destroy
curl -sS https://sandbox-dind-repro.ericclemmons.workers.dev/docker-version
curl -sS https://sandbox-dind-repro.ericclemmons.workers.dev/docker-run

/destroy succeeds.

An immediate /docker-version right after destroy/cold start can still fail because Docker is not ready yet:

failed to connect to the docker API at unix:///var/run/docker.sock; check if the path is correct and if the daemon is running: dial unix /var/run/docker.sock: connect: no such file or directory

However, production container logs show the guide's CMD is being spawned, and subsequent Docker commands prove Docker does come up:

Starting sandbox entrypoint
Container server started
Container started
command.exec success docker version
command.exec success docker build --network=host -t sandbox-dind-smoke /workspace
command.exec success docker run --network=host --rm sandbox-dind-smoke

On a later clean production run, /docker-run showed:

• docker build --network=host -t sandbox-dind-smoke /workspace succeeded.
• docker run --network=host --rm sandbox-dind-smoke failed with a cgroup/systemd runtime error:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration: unable to start unit "docker-ec439ac3bfb71b9c1a08d373d93e30b21a1f93588d96be466219e3ab37efaac6.scope" (properties [{Name:Description Value:"libcontainer container ec439ac3bfb71b9c1a08d373d93e30b21a1f93588d96be466219e3ab37efaac6"} {Name:Slice Value:"system.slice"} {Name:Delegate Value:true} {Name:PIDs Value:@au [277]} {Name:MemoryAccounting Value:true} {Name:CPUAccounting Value:true} {Name:IOAccounting Value:true} {Name:TasksAccounting Value:true} {Name:DefaultDependencies Value:false}]): Failed to set unit properties: No such process

Refined summary

This appears to be two related issues:

1. Local Wrangler/Sandbox does not appear to support the documented Docker-in-Docker setup well enough to create /var/run/docker.sock at all.
2. Production gets further: Docker can start and docker build --network=host can succeed, but docker run --network=host can fail with a cgroup/systemd-related runtime error.

There may also be a readiness race in production: immediately calling docker version after cold start can fail before the Docker daemon is ready, while later Docker commands can succeed.

[ericclemmons]

CC @mikenomitch from cloudflare/cloudflare-docs#28337

[ericclemmons]

Update from the repro: Eric asked me to keep this issue and the repro repository updated as we worked through this, based on his persistence in getting Docker-in-Docker figured out locally and in production.

The repro has been updated and pushed here:
ericclemmons/sandbox-dind-repro@7d3b85f

Verified local workaround for Wrangler:

• Start a local Docker API Unix socket proxy from the repro's pnpm dev script.
• The proxy listens at .wrangler/docker-privileged.sock.
• The proxy forwards to DOCKER_SOCKET, the active Docker context socket, or /var/run/docker.sock.
• It injects HostConfig.Privileged=true into Docker POST /containers/create requests.
• pnpm dev sets DOCKER_HOST=unix://$PWD/.wrangler/docker-privileged.sock before running wrangler dev --port 8787.
• This avoids committing an absolute machine-specific dev.container_engine path while still using Wrangler/workerd's supported Docker socket override behavior.

Validated locally after this update:

pnpm check
pnpm dev
curl --max-time 120 -sS -i http://localhost:8787/docker-version
curl --max-time 180 -sS -i http://localhost:8787/docker-run

Both /docker-version and /docker-run returned 200 OK. /docker-run successfully ran:

docker build --network=host -t sandbox-dind-smoke /workspace
docker run --network=host --rm sandbox-dind-smoke

and returned Hello from Docker!.

This confirms local Wrangler can run this DinD repro if the local container create call is made privileged. Production still needs the dockerd cgroup driver fix noted earlier:

--exec-opt native.cgroupdriver=cgroupfs

[ericclemmons]

One more classification note based on the repro work:

I do not think this is primarily a Sandbox SDK API bug. Once the container has Docker available, sandbox.exec() / execStream() can run Docker and act successfully.

The issue seems split across docs and local container runtime behavior:

1. Production: likely a docs bug.

Production Docker-in-Docker works for this repro once dockerd is started with:

--exec-opt native.cgroupdriver=cgroupfs

The DinD guide should probably mention that flag. In this setup, inner Docker commands also need --network=host.

2. Local Wrangler/workerd: likely a local Containers feature gap or parity bug.

Local wrangler dev supports selecting the Docker socket via dev.container_engine / DOCKER_HOST, but I could not find a supported way to set Docker create options like:

{
  "HostConfig": {
    "Privileged": true
  }
}

The local repro works when a Docker socket proxy injects HostConfig.Privileged=true into POST /containers/create. Without that, local DinD fails before Docker becomes usable inside the Sandbox container.

So my current read is:

• Docs bug: missing production cgroup driver guidance and local limitation/workaround.
• Wrangler/workerd local Containers gap: no supported local privileged container option.
• Local Containers parity issue: documented DinD image can work in production, but not in local dev without an unsupported Docker socket proxy.
• Sandbox SDK: probably not the root cause.

[thesiti92]

this is blocking me from adopting it for ci too

[aron-cf]

This isn't something the Sandbox SDK can work around currently. I'm looking to understand if something changed in the containers config. Otherwise I'll fix by removing the docker-in-docker docs for now.