[Bug]: @qvac/diffusion-cpp still DT_NEEDED-links libvulkan.so.1 on linux-arm64; eager addon load blocks all CPU inference (follow-up to #1823)

[hevans714]

🐛 Bug Description

On a headless, CPU-only linux-arm64 host, no model can be loaded with @qvac/sdk@0.17.0. The bare worker aborts during startup and the server reports only RPC initialization timed out after 30000ms.

The cause is libvulkan.so.1, which is the same class of problem as #1823 ("device: cpu should not be affected by GPU dependencies"). That issue was closed on 2026-08-07 on the basis of an artifact audit of @qvac/llm-llamacpp and @qvac/decoder-audio → bare-ffmpeg.

I can confirm both of those are now clean. The remaining offender is a third addon that the audit did not cover: @qvac/diffusion-cpp, which still DT_NEEDED-links libvulkan.so.1 and which the default worker loads eagerly at startup — even when the config declares only an LLM and image generation is never called.

readelf -d on the linux-arm64 prebuilds from a clean install of @qvac/cli (SDK 0.17.0):

addon	version	DT_NEEDED vulkan/va/drm
@qvac/llm-llamacpp	0.39.4	none ✅ (fix from #1823 confirmed landed)
@qvac/embed-llamacpp	0.30.1	none ✅
@qvac/diffusion-cpp	0.17.0	libvulkan.so.1 ❌

So the lazy-backend work done for llm-llamacpp achieves nothing for a CPU-only host as long as the worker eagerly requires diffusion-cpp: one unused modality engine takes down all inference, including plain LLM completion.

🔄 Steps to Reproduce

Minimal, from a clean image with no GPU stack (no Vulkan loader, no ICD):

FROM node:22-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
        python3 build-essential ca-certificates curl \
    && rm -rf /var/lib/apt/lists/*
RUN npm install -g @qvac/cli
WORKDIR /app
CMD ["qvac", "serve", "openai", "--host", "0.0.0.0", "--port", "11434"]

qvac.config.json declares a single LLM and nothing else:

{
  "serve": {
    "models": {
      "default": {
        "model": "QWEN3_600M_INST_Q4",
        "default": true,
        "config": { "ctx_size": 8192 }
      }
    }
  }
}

1. Build and run on linux/arm64.
2. The server starts and serves routes, but GET /v1/models returns {"object":"list","data":[]} and every completion returns 503.

✅ Expected Behavior

A config that declares only an LLM should not load the image-generation engine, and a host with no Vulkan loader should still run CPU inference — the same outcome #1823 established for llm-llamacpp. Either load diffusion-cpp lazily when an image request first arrives, or dlopen its Vulkan backend as a separate module the way llm-llamacpp now does.

❌ Actual Behavior

The worker aborts before it can answer the RPC handshake, so no model ever loads.

Server-side, the only visible symptom is:

Preloading 1 model(s): default
Loading model "default" from registry://hf/unsloth/Qwen3-0.6B-GGUF/blob/.../Qwen3-0.6B-Q4_0.gguf...
Failed to preload "default": RPC initialization timed out after 30000ms — the worker process may have failed to start

📜 Stack Trace / Error Output

The real error is only visible by running the worker entry directly — node-rpc-client.js spawns it with stdio: ['inherit','inherit','pipe'], so the worker's stderr never reaches the user:

bare .../@qvac/sdk/dist/server/worker.js /root '{"QVAC_IPC_SOCKET_PATH":"/tmp/x.sock","HOME_DIR":"/root"}'

Uncaught AddonError: CANNOT_LOAD: Cannot load addon 'file:///usr/local/lib/node_modules/@qvac/cli/node_modules/@qvac/diffusion-cpp/prebuilds/linux-arm64/qvac__diffusion-cpp.bare'
    at new Addon (bare:/bare.js:2613:24)
    at Addon.load (bare:/bare.js:2652:31)
    at require.addon (bare:/bare.bundle/node_modules/bare-module/index.js:822:30)
    at file:///usr/local/lib/node_modules/@qvac/cli/node_modules/@qvac/diffusion-cpp/binding.js:1:26
    at Module._evaluate (bare:/bare.bundle/node_modules/bare-module/index.js:215:7)
    at Module.load (bare:/bare.bundle/node_modules/bare-module/index.js:477:21)
    at require (bare:/bare.bundle/node_modules/bare-module/index.js:807:27)
    at file:///usr/local/lib/node_modules/@qvac/cli/node_modules/@qvac/diffusion-cpp/addonLogging.js:4:17 {
  [cause]: Error: libvulkan.so.1: cannot open shared object file: No such file or directory,
  code: 'CANNOT_LOAD'
}
Aborted

🩹 Workaround

Install the Vulkan loader only — apt-get install -y libvulkan1. With no ICD present Vulkan enumerates zero devices and the CPU path proceeds normally. This is the same shape as the workaround reported in #1823 by the embedded-ARM evaluation.

After that single change, on the same host: model loads, and qvac_benchmark-style throughput on Qwen3-0.6B-Q4_0 is ~15.8 tok/s warm median (~16s for a 250-token generation), which is a perfectly reasonable CPU result. The engine works fine — it was only the eager addon load standing in the way.

⚠️ Secondary: qvac doctor reports this as non-fatal

Worth fixing alongside, because it actively points people away from the cause. On the broken host, doctor prints:

⚠️  GPU acceleration — Vulkan ICD not found
    ... Without a Vulkan ICD, GPU acceleration is unavailable and inference runs on
    CPU, which is significantly slower.
...
✅ All required checks passed.

Both statements are wrong in this configuration: inference does not run on CPU, it does not run at all, and the required checks have not passed. Something like a DT_NEEDED check across the addons the default worker eagerly loads would turn this into an accurate, fatal error.

I also spent a while chasing ⚠️ @qvac/sdk resolvable from project — not found, which doctor reports when the CLI is installed globally with an empty WORKDIR. That one is accurate but is not the cause of the failure — making it resolvable changes nothing. Only mentioning it because it is the natural first suspect when the worker won't start.

Happy to test a patched build on this host if that is useful — it is a reproducible CPU-only arm64 environment.

💻 Platform / OS

Linux (arm64) — node:22-slim container on Apple Silicon (Docker, linux/arm64)

⚙️ Runtime Environment

Node.js (@qvac/cli, qvac serve openai)

📦 Runtime Version

Node.js v22.23.2 · bare v1.31.0 · bare-runtime-linux-arm64 present

🏷️ SDK Version

0.17.0

🔁 Frequency

Always (100%)

🔥 Severity

Critical - Complete blocker, no workaround (before the loader is installed)

[victorchimakanu]

Confirmed in source. The default worker entry, packages/sdk/server/worker.ts, registers every modality plugin, and the diffusion plugin at packages/sdk/server/bare/plugins/sdcpp-generation/plugin.ts imports @qvac/diffusion-cpp and its addonLogging at module top level. The native addon loads the moment the worker starts, before any config is read, so a config that never mentions image generation still dies on the diffusion prebuild. The stack trace in the report matches that chain exactly, addonLogging.js:4 into binding.js:1.

The #1823 close was scoped to @qvac/llm-llamacpp and the @qvac/decoder-audio to bare-ffmpeg path, so @qvac/diffusion-cpp sat outside that audit. On the build side, the diffusion addon's CMakeLists.txt notes that Vulkan is auto enabled through default features of the ggml and stable-diffusion-cpp registry ports, so unlike llm-llamacpp there is no separately loaded Vulkan backend module here yet.

I also checked 0.17.1, published yesterday. The changelog is Python wheel packaging, a Windows build fix, and a bare-rpc stream destroy fix. Nothing touches addon loading, so the current release behaves the same way.

One point worth adding to the framing: the worker eagerly loads eleven plugin addons (packages/bare-sdk/scripts/plugin-addons.mjs is the canonical list), so any one of them acquiring an unmet DT_NEEDED on some platform takes down all inference on that platform. That is how this class of failure came back after the llm-llamacpp fix, and it can recur each time a new modality ships, like audiogen in 0.17.0. Lazy plugin import in the worker, or a CI check that walks DT_NEEDED across the prebuilds the default worker loads, would close the class instead of the instance. The qvac doctor change you describe is the runtime half of that same check.

I will raise this with the SDK team so it gets tracked internally. Until a fix lands, installing the libvulkan1 loader is the right workaround, same shape as the embedded evaluation in #1823. A reproducible CPU-only arm64 environment to test a patched build on will be useful once a fix is up.

[hevans714]

Thanks — the eleven-addon list is the part I didn't have, and it reframes the report. I filed
this as a diffusion-cpp instance; plugin-addons.mjs makes it a property of the default worker,
where any prebuild acquiring an unmet DT_NEEDED on any platform takes all inference down with
it. Agreed that lazy plugin import or a DT_NEEDED walk across the loaded prebuilds closes the
class rather than the instance. Good to know 0.17.1 doesn't change the behaviour — saves us a
test cycle.

Happy to help however is useful. On the environment you asked for:

Both architectures, not just arm64. The original report was arm64 — Docker on Apple Silicon,
node:22-slim, @qvac/sdk 0.17.0. But this also runs on our production box, which is
x86_64, Ubuntu 24.04.1, 2 vCPU / 7GB RAM, no GPU and no graphics stack at all. Same failure,
same fix, so it isn't arm-specific. Both are plain Dockerfiles — I can share either or both,
whichever is more useful, or push them somewhere you can pull.

For testing a patched build, the matrix that actually matters to us is:

	libvulkan1 absent	libvulkan1 installed
arm64 (CPU)	fails today — the case to fix	works (current workaround)
x86_64 (CPU)	fails today	works (current workaround)

The left column is the real test: does a patched worker start on a host that has never had a
graphics stack installed? That is the normal state of a server, and it is trivial for us to
reproduce on demand in both arches since it is just a container without the loader. Give me a
branch, tag or prebuild and I'll run it and report back with the worker output.

Worth noting for whoever picks this up: the RPC path swallows worker stderr, so the only symptom
upstream is RPC initialization timed out after 30000ms. Running the worker by hand is what
surfaced the actual AddonError CANNOT_LOAD:

bare .../@qvac/sdk/dist/server/worker.js /root \
  '{"QVAC_IPC_SOCKET_PATH":"/tmp/x.sock","HOME_DIR":"/root"}'

If a fix lands, that timeout message would be worth improving too — it names neither the addon
nor the missing library, which is most of why this took a day to find rather than a minute.

I can also A/B the qvac doctor output before and after, if the runtime check gets changed.

Context on why we care, since it affects how much use this environment will get. We're a small
commercial development shop, and this is a production deployment on a platform we're building
out — not an evaluation spike. QVAC is merged and the Bergamot NMT path is live, handling
cross-language messaging at 29ms per string against ~6.3s for the same string through a hosted
frontier model on the CPU-only box above.

To be precise about the rest: the chat provider is wired in as first in our AI provider chain
with fallback to hosted models, but it's still flag-disabled while we settle a quality question.
Qwen3-0.6B locally is ~7x faster than the hosted model for us and, more interestingly, more
truthful — asked to describe a real sports venue, the hosted frontier model invented plausible
attributes ("full-size", "well-maintained") that were nowhere in the prompt, while the 0.6B wrote
duller sentences that were all true. For a platform meant to be a trustworthy record of real
places that isn't a close call, and it's pushing us toward more local inference rather than less
as we grow.

Which is also why the CPU path matters to us specifically. Our whole reason for choosing QVAC
over a hosted API is that user data stays on hardware we control, and that argument only holds
if local inference is dependable on ordinary servers with no graphics stack. A failure mode that
takes down all inference because of an image-generation addon we never call is the kind of thing
that decides whether a team commits to the SDK or routes around it — so it's worth us putting
real time into helping close it.

One separate thing I ran into that isn't this bug and that I'd be glad to file on its own if
it's useful: the OpenAI-compatible server exposes chat, completions, embeddings, images and
audio, but no text-to-text translation route — the NMT engine appears reachable only from the JS
SDK. We worked around it with a small Node sidecar in front of qvac.translate, which is fine,
but it does mean a non-JS consumer can't reach the capability QVAC is best known for over the
documented HTTP surface. Say the word and I'll write it up properly with the details.

[victorchimakanu]

File the translation route as its own issue. It is verified on main: the serve router has route modules for chat, completions, embeddings, files, images, models, responses, vector stores, videos and audio, and nothing else. The only translation on the HTTP surface is /v1/audio/translations, which is Whisper speech to English text and requires an audio translation model. The Bergamot text to text engine sits behind the SDK's translate call with no route in front of it, so a consumer outside JS cannot reach it. Your sidecar is the right workaround today, and describing it in the issue will show the shape a real route should take.

One framing point for the issue body. Every route serve exposes today mirrors an endpoint in OpenAI's own API, and OpenAI has no text to text translation endpoint. So this is not a compatibility gap, it would be the server's first extension route. That is a real design decision for the team, and the strongest argument for it is the number you posted here: 29ms per string for the capability this stack is best known for, on a box with no GPU.

Your test matrix and the offer to run a patched build on both architectures are now on the thread for whoever picks up the fix. Agreed that the empty libvulkan1 column is the definition of done, a worker that starts on a host that has never had a graphics stack. The timeout message that names neither the addon nor the missing library belongs in the same fix, and your report is the evidence for why.

[hevans714]

Two things: a correction to a number I posted, and a fresh reproduction on 0.17.1.

First, the correction. I quoted 29ms per string for Bergamot EN→ES. That figure is real but I
attributed it loosely, and since you have picked it up as the argument for a route, it should be
exact. The 29ms is my development machine — Apple Silicon, in Docker. On the box I actually
described in this thread, a 2 vCPU / 7GB x86_64 server with no GPU, the same measurement is
55ms per string, 18ms batched.

machine (both CPU-only, no GPU)	per string	batched
2 vCPU / 7GB, x86_64 (the production box)	55ms	18ms
Apple Silicon, Docker (dev)	~29ms (median 27, n=15 warm)	not measured

Against 6,310ms for the same string through a hosted frontier model. So the comparison is ~115×
rather than ~215×. The argument does not need the faster number and I should not have used it
without saying which machine it came from.

Second, 0.17.1 still fails, and I have run it rather than read the changelog.

I stripped the Vulkan loader from an otherwise-working image and started the worker:

The base is node:22-slim + npm install -g @qvac/cli@0.10.0, which resolves @qvac/sdk
0.17.1 and @qvac/diffusion-cpp 0.17.0, built with libvulkan1 present so it works.
Then one layer on top:

RUN dpkg --force-depends --remove libvulkan1 && ldconfig

The prebuild still carries the link:

$ readelf -d .../@qvac/diffusion-cpp/prebuilds/linux-arm64/qvac__diffusion-cpp.bare | grep NEEDED
 (NEEDED)  Shared library: [libm.so.6]
 (NEEDED)  Shared library: [libvulkan.so.1]     <-- still here on 0.17.0 / sdk 0.17.1
 (NEEDED)  Shared library: [libgcc_s.so.1]
 (NEEDED)  Shared library: [libc.so.6]
 (NEEDED)  Shared library: [ld-linux-aarch64.so.1]

And the runtime failure, which I think is the most useful part — this run was loading a
translation model and never asks for image generation at all:

Loading BERGAMOT_EN_ES …
RPC_INIT_TIMEOUT: RPC initialization timed out after 30000ms — the worker process may have failed to start
Caused by: Error: Worker process exited with code null, signal SIGABRT before IPC connection was established

Worker stderr:
Uncaught AddonError: CANNOT_LOAD: Cannot load addon
  'file:///.../@qvac/diffusion-cpp/prebuilds/linux-arm64/qvac__diffusion-cpp.bare'
    at new Addon (bare:/bare.js:2613:24)
    at Addon.load (bare:/bare.js:2652:31)
    at require.addon (bare:/bare.bundle/node_modules/bare-module/index.js:822:30)
    at file:///.../@qvac/diffusion-cpp/binding.js:1:26

That is the chain you traced from source, confirmed at runtime on the current release: a
translation-only workload dies on the diffusion prebuild before any config is read.

On the fix being useful to test: stripping the loader from a warm image takes about a second,
so I can run the matrix on a patched build the same day rather than rebuilding the toolchain.
Both columns of the empty-libvulkan1 case are covered — arm64 here, x86_64 on the production
box.

Two small things for whoever picks it up, both visible above:

• The user-facing error still names neither the addon nor the missing library. RPC initialization timed out after 30000ms is what an operator sees; everything diagnostic is in
  the worker stderr underneath it.
• qvac serve starts and prints its full route table fine without the loader — the HTTP surface
  comes up healthy and only inference is dead. Anything that health-checks the port will report
  the service as up.

[hevans714]

A second unmet DT_NEEDED turned up in 0.18.1, published today. Posting it here rather than as
a new issue because it is the same failure mode — and because it widens the class described
above rather than repeating it.

A clean node:22-slim with @qvac/cli@0.12.0 (SDK 0.18.1, nmtcpp 0.10.1) and libvulkan1
already installed, so the workaround from this thread is in place, still cannot start the
worker:

[cause]: Error: libatomic.so.1: cannot open shared object file: No such file or directory
    at require.addon (bare-module/index.js:822:30)
    at rocksdb-native/binding.js:3:26
  code: 'CANNOT_LOAD',
  href: 'file:///usr/local/.../rocksdb-native/prebuilds/linux-arm64/rocksdb-native.bare'
...
code: 50204

apt-get install -y libatomic1 fixes it, the same shape as libvulkan1.

It is not one of the eleven plugin addons. rocksdb-native 3.17.4 arrives through
hyperdb → hypercore-storage, in the worker's startup path rather than as a modality plugin.
That matters for the fix: a check scoped to plugin-addons.mjs would not have caught this. The
property this issue identified — one eagerly-loaded native addon with an unmet DT_NEEDED
takes the whole worker down, and with it every modality — turns out to hold for the worker's
infrastructure dependencies too, not only its plugins.

Both linux prebuilds need it, so it is not arch-specific. Walking DT_NEEDED over the
installed package tree:

prebuild	NEEDED
linux-arm64/rocksdb-native.bare	libatomic.so.1, libstdc++.so.6, libm.so.6, libgcc_s.so.1, libc.so.6
linux-x64/rocksdb-native.bare	libatomic.so.1, libstdc++.so.6, libm.so.6, libgcc_s.so.1, libc.so.6, ld-linux-x86-64.so.2

libatomic.so.1 is absent from node:22-slim, debian:12-slim and ubuntu:24.04 — checked,
not assumed — so a server install hits this before running any inference.

Two things worth drawing out.

The blast radius is the point of this issue and it held. RocksDB has nothing to do with
translation, yet the missing library takes down all inference. We hit it while trying to
measure NMT cancel behaviour for #3889, on a container built for that and nothing else.

The symptom is still opaque. The top-level failure is a worker startup error with
code: 50204; the missing library appears only in the nested [cause]. As with the Vulkan
case, running the worker by hand was the fastest route to the real message.

The DT_NEEDED walk above took about a minute. Run in CI across every native addon the default
worker loads — plugins and infrastructure both — it would have caught this before release, and
it does not require anyone to guess which library is next.

Happy to test a patched build, lazy plugin import or otherwise, on x86_64 and arm64 against the
matrix earlier in this thread.