Skip to content

QVAC-21751 chore[bc]: use @qvac/inference for SDK - #3595

Merged
opaninakuffo merged 21 commits into
mainfrom
arun-mani-j/useInference
Aug 24, 2026
Merged

QVAC-21751 chore[bc]: use @qvac/inference for SDK#3595
opaninakuffo merged 21 commits into
mainfrom
arun-mani-j/useInference

Conversation

@arun-mani-j

@arun-mani-j arun-mani-j commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

🎯 What problem does this PR solve?

  • The inference engine (model loading, native-addon hosting, plugin registry, dispatch, schemas, model registry) lives inside @qvac/sdk, coupling the engine to the SDK and blocking standalone use of the engine.
  • @qvac/bare-sdk carries a copy of the @qvac/sdk sources and a mirrored dependency set, kept aligned by parity/leak scripts.

📝 How does it solve it?

  • @qvac/inference is a standalone package holding the engine: dispatch, validation, handlers, plugins, registry, runtime, schemas, model registry, and the engine-level server/utils/* helpers (completion normalizer, tool parsing, path-security, archive, gguf/shard, config-transform, …).
  • @qvac/sdk is a thin client over @qvac/inference. It keeps its client, transports, and worker orchestration, and re-exports the engine surface, so consumer imports (loadModel, completion, plugins, plugin-utils, logging ids, model constants) are unchanged.
  • The Bare worker initializes the @qvac/inference engine; the Node/Electron/Expo transports drive that worker over RPC.
  • Plugin registration routes through the single shared @qvac/inference registry — no duplicate registry in the SDK.
  • Built-in plugin ids stay under the public @qvac/sdk/* name (PluginId, SDK_DEFAULT_PLUGINS, PLUGIN_*) and resolve to the inference plugins.
  • SDK tests and build tooling target @qvac/inference: engine unit tests live only in inference, the SDK's Bare/security test tooling is retired (surface is test:unit + e2e), and the Node contract is generated from the @qvac/inference surface.
  • @qvac/bare-sdk is a thin re-export of @qvac/inference (index / logging / models / plugins / plugin-utils and each built-in plugin, one ./plugins export), with no addon dependencies and no bundled _sdk/ tree. ./onnx-tts/plugin is kept as a one-line alias.
  • The sibling-SDK bundling machinery and its checks are removed: build-sibling-sdk.mjs, bundle-from-sdk.mjs, check-deps-vs-sdk.mjs, check-no-addon-deps.mjs, check-no-addon-leaks.mjs, plugin-addons.mjs, and the bare-sdk dep-parity step in pr-checks-sdk-pod.yml.

🧪 How was it tested?

  • @qvac/sdk: unit, lint, build, and contract checks pass; the regenerated contract matches a fresh generation from the @qvac/inference surface.
  • @qvac/inference: format, lint, typecheck, build, and test:bare pass.
  • Desktop (Node) E2E locally: 467/472 passed, 3 skipped, 0 infrastructure failures. The 2 reds are non-deterministic tiny-model (LLAMA_3_2_1B) content assertions in completion, unrelated to the split; every structural category (model, plugin, kv-cache, tools, ocr, diffusion, vla, tts, delegated-inference, lifecycle, registry, system-resources, …) passed 100%.
  • The cross-platform E2E matrix — desktop (macOS, Linux, Windows), Electron consumer (Linux, Windows), mobile (Android, iOS) — runs in CI on this PR.

💥 Breaking Changes

The public @qvac/sdk API (loadModel, completion, plugins, plugin-utils, logging, model constants, plugin ids) is unchanged. Two consumer-visible subpath categories, both cushioned by deprecation shims:

@qvac/sdk worker-orchestration subpaths — the lifecycle subpath is renamed and the raw worker file sits under src/worker/. The old subpath and file still resolve through deprecated aliases that warn and forward.

BEFORE:

import {  } from '@qvac/sdk/worker-core'
const worker = '<sdk_root>/dist/server/worker.js'

AFTER:

import {  } from '@qvac/sdk/worker-lifecycle'
const worker = '<sdk_root>/dist/src/worker/index.js'

@qvac/bare-sdk is a thin @qvac/inference re-export — ./commands and ./worker-core are gone; importing either throws with migration guidance. ./onnx-tts/plugin is retained as an alias.

BEFORE:

import {  } from '@qvac/bare-sdk/commands'
import {  } from '@qvac/bare-sdk/worker-core'
import {  } from '@qvac/bare-sdk/onnx-tts/plugin'

AFTER:

// bundling lives in @qvac/sdk; register plugins via @qvac/inference directly
import {  } from '@qvac/bare-sdk/onnx-tts/plugin' // alias → @qvac/inference/tts-ggml/plugin

@arun-mani-j
arun-mani-j requested review from a team as code owners August 3, 2026 08:39
@arun-mani-j
arun-mani-j marked this pull request as draft August 3, 2026 08:39
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

License compliance — findings detected (warn-only)

Critical: 0 · High: 4 · Medium: 0

Dependency License Scope Severity Outcome
actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd (none detected) runtime High blocks
actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f (none detected) runtime High blocks
actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f (none detected) runtime High blocks
oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 (none detected) runtime High blocks

How to resolve a blocking finding:

  • Remove or replace the disallowed dependency, or
  • If the license is genuinely acceptable, run the compliance SKILL and record the decision in .github/license-allowlist.yml (CODEOWNERS-reviewed), or
  • For a one-off, a maintainer can apply the license-override label (High findings only; Critical cannot be overridden).

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

  • ./.github/actions/release-merge-guard
  • ./docs/website
  • ./packages/ggml-coload-smoke
  • ./packages/fabric/test/integration
  • ./packages/inference-addon-cpp/mobile
  • ./packages/sdk/e2e
  • ./packages/llm-llamacpp/benchmarks/performance
  • ./packages/llm-llamacpp/benchmarks/server
  • ./packages/vla-ggml/sim/server
  • ./packages/embed-llamacpp/benchmarks/performance
  • ./packages/embed-llamacpp/benchmarks/server
  • ./packages/asr-ggml/benchmarks/server

@arun-mani-j
arun-mani-j force-pushed the arun-mani-j/useInference branch 2 times, most recently from ecd2bd0 to b69b8e0 Compare August 3, 2026 08:58
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review Status

Current Status: ✅ APPROVED
Approvals so far: Team Lead: 1, Member: 2

@arun-mani-j
arun-mani-j force-pushed the arun-mani-j/useInference branch from b69b8e0 to 4129c6d Compare August 3, 2026 11:45
@arun-mani-j arun-mani-j added test-e2e-full Triggers full e2e test suite [Currently SDK-only] test-e2e-smoke Triggers smoke e2e test suite [Currently SDK-only] and removed test-e2e-full Triggers full e2e test suite [Currently SDK-only] labels Aug 3, 2026
Comment thread packages/bare-sdk/package.json Outdated
Comment thread packages/sdk/package.json Outdated
Comment thread packages/sdk/examples/config-reload.ts

@lauripiisang lauripiisang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few things to still catch up on, mainly the python SDK lookup breaking (it wasn't easily findable by the full path).

The link workspace having an unlink counterpart is a nice-to-have.

Comment thread packages/sdk/package.json
Comment thread packages/sdk/contract/schema.json Outdated
Comment thread .github/actions/sdk-e2e-prepare-inference/prepare.mjs Outdated
Comment thread packages/bare-sdk/scripts/link-workspace-inference.mjs
Comment thread packages/bare-sdk/README.md
Comment thread .github/actions/sdk-e2e-prepare-inference/prepare.mjs
Comment thread packages/sdk/src/commands/bundle/plugins.ts
arun-mani-j and others added 18 commits August 21, 2026 19:43
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
…istry

Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
…hrough the SDK

Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Resolve inference once before platform fan-out so SDK e2e runs use the same branch build or published version everywhere. Normalize Windows branch tarball paths for Bun compatibility.
…dk layout

Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Private GitHub Packages access is unavailable under the CI token policy, so keep branch builds, npm versions, and manifest dependencies as the supported SDK E2E sources.
Signed-off-by: Arun Mani J <j.arunmani@proton.me>
…gration guidance

Signed-off-by: Arun Mani J <j.arunmani@proton.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test-e2e-smoke Triggers smoke e2e test suite [Currently SDK-only]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants