Add a preflight check for addon/fabric compatibility before mobile E2E

[localhost41]

Problem

The SDK can assemble addon packages whose embedded qvac-fabric/GGML dependencies are not compatible under a shared mobile backend name.

This surfaced during #3579: the VLA addon update introduced a backend BuildId that was incompatible with the older embedding addon. The Android matrix then failed after the initial model load, with the consumer crashing during model-load-embedding.

The coordinated addon update in #3588 fixes the immediate version-alignment problem, but there is not currently a preflight check that prevents a future mixed addon set from reaching the mobile device matrix.

Proposed contribution

I would like to add a preflight check that runs against the published addon set selected by the SDK and verifies that the addons agree on the qvac-fabric/GGML compatibility identity they depend on.

The check would:

• resolve the addon versions selected by the SDK;
• identify the relevant qvac-fabric/GGML compatibility information for each addon;
• detect a mixed or incompatible addon set before device tests start;
• report the conflicting packages and versions clearly; and
• fail early with an actionable message explaining what needs to be aligned.

The preflight should run before the mobile E2E matrix. It would complement the device tests, not replace them.

Initial scope

I would start with the SDK addon set and the published-package path used by the mobile E2E workflow. I would avoid changing addon APIs or runtime behavior unless the team identifies a compatibility signal that is not currently exposed.

Before implementing, I would like to confirm where the team wants the authoritative compatibility metadata to come from. Once that is agreed, I can add the check, a deliberately mismatched-package regression case, and coverage showing that the aligned versions from #3588 pass.

Acceptance criteria

• A deliberately mismatched addon set is rejected before device tests start.
• The aligned versions from QVAC-23051 chore: bump addon versions to use same fabric #3588 pass the preflight.
• A failure identifies the conflicting addons and compatibility versions.
• The check works against published packages, not only local workspace builds.
• Existing desktop, iOS, and Android test coverage remains in place, with the preflight acting as an earlier validation step.

Related: #3579 and #3588.

[victorchimakanu]

The piece this check needs is currently missing from the published packages, so here is the state of the compatibility metadata as it stands, verified against the repo and the npm artifacts.

The compatibility identity lives in each addon's vcpkg manifest. A vcpkg manifest is the file that declares a package's native dependencies. Every qvac-fabric consumer pins fabric there as a version floor in packages//vcpkg.json. CI validates that in the repo with .github/actions/verify-qvac-fabric-lockstep/action.yml, which today checks three hardcoded manifests (llm-llamacpp, embed-llamacpp, translation-nmtcpp). #3665, opened this morning, replaces the hardcoded list with auto-discovery across all consumers. That covers the source side of the problem you are describing.

There is no equivalent on the published package side, which is the side your preflight targets. The manifest does not travel with the artifact. The files list of the published packages does not include vcpkg.json. For example @qvac/vla-ggml 0.17.0 ships addon.js, binding.js, index.js, lib, prebuilds, LICENSE and NOTICE, and nothing that states the fabric version it was built against. So a preflight that resolves the published addon set has no machine readable compatibility identity to read today.

That splits your open question in two. The authoritative source already exists and is the vcpkg manifest. What is missing is a publish step that stamps that identity into the artifact, either by shipping vcpkg.json in the files list or by writing the fabric version into package.json or a small metadata file at pack time. That publishing change is the part to agree with the team first, because it touches every qvac-fabric consumer package. Whether the backend build identity can instead be read out of the prebuilt binaries without loading them on a device is a question for the fabric maintainers.

Once the identity ships in the artifact, the check itself becomes a read and compare across the resolved set, and the deliberately mismatched regression case you propose is straightforward to construct.

[localhost41]

Thanks, this clarifies the boundary really well. Since #3665 now covers source-side lockstep discovery, I think the next decision is how we want the compatibility identity to travel with published packages: either ship the relevant vcpkg metadata, or stamp the fabric version into package.json or a small dedicated metadata file during publish.

I’m happy to shape the preflight around whichever representation the team prefers. Once that is settled, I can add the aligned-package and deliberately mismatched-package cases against the published-package path.

[localhost41]

Quick status check: #3665 has now handled source-side fabric consumer discovery, while #3827 improves fabric-stack CI but does not appear to expose a compatibility identity in the published addon artifacts. That means the published-package preflight proposed here is still blocked on the metadata representation.

Would the team prefer the fabric identity in package.json, a dedicated metadata file, or should we park this until the publishing contract is revisited? I’m happy to implement the preflight once that direction is settled.

[gianni-cor]

Hi @localhost41 , we are migrating to all the addons to depend on fabric npm pkg, instead of vcpkg port. right now classification-ggml depends on it already and other addons are wip. so you will have fabric version exposed on package.json. I think this is aligned with your proposal