Skip to content

QVAC-23575 chore[bc]: add @qvac/test-suite as a monorepo package - #4082

Merged
Victor-Rodzko merged 3 commits into
mainfrom
feature-test-suite-fold
Aug 26, 2026
Merged

QVAC-23575 chore[bc]: add @qvac/test-suite as a monorepo package#4082
Victor-Rodzko merged 3 commits into
mainfrom
feature-test-suite-fold

Conversation

@Victor-Rodzko

@Victor-Rodzko Victor-Rodzko commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

🎯 What problem does this solve?

@qvac/qvac-test-suite lived in a separate repository (tetherto/qvac-test-suite) while its only consumer is packages/sdk/e2e here. Every framework change needed a publish → bump → install round-trip, and CI had no way to test an SDK change against an unreleased framework change.

Part 1 of 2 for QVAC-23575. This PR only adds the package; migrating the SDK e2e consumer onto it is #4083, which is stacked on this branch.

📝 How is it solved?

Folded in as a normal SDK-pod package. framework/ is flattened into packages/test-suite/ and registered in sdk-pod-checks.json, the release-guard pod list, pr-validation-sdk-pod paths, teams/sdk.json, the SDK-pod Cursor rules and pr-test-sdk.mjs. It now publishes exactly like @qvac/rag and @qvac/logging.

Renamed to @qvac/test-suite, dropping the duplicated scope word (GPR: @tetherto/test-suite-mono). The directory name now equals the release slug, so scripts/sdk/package-paths.cjs resolves it with no override.

Publishing uses the prebuilt-dist pattern introduced for rag/logging/error in #4055. This package ships only compiled output and deliberately has no prepare script, so dist is built once in a dedicated build job and every publish job downloads it and publishes with NPM_CONFIG_IGNORE_SCRIPTS=true. This also matches how the standalone repo published before the fold.

Nothing consumes the package yet. packages/sdk/e2e keeps its published @qvac/qvac-test-suite dependency, so this PR is additive and cannot affect the SDK e2e suite.

🧪 How was this verified?

  • npm run check / typecheck / build clean in packages/test-suite.
  • npm pack --dry-run ships dist/ + templates/ + LICENSE + NOTICE + README.md — 104 entries, no stale build output.
  • A real tarball installed via file:// into a clean consumer, and qvac-test --version works from it.
  • actionlint structural gate clean; 97/97 CI policy tests pass.

Because the gate reads its package config from the base commit, the new package's own SDK Pod Checks do not run on this PR (documented first-add exemption). They were run locally instead. Once this lands, every later PR touching the package is gated.

💥 Breaking Changes

The npm package is renamed @qvac/qvac-test-suite@qvac/test-suite; the GPR dev build is renamed @tetherto/qvac-test-suite@tetherto/test-suite-mono.

BEFORE:

{ "dependencies": { "@qvac/qvac-test-suite": "^0.10.3" } }
import type { TestDefinition } from '@qvac/qvac-test-suite'

AFTER:

{ "dependencies": { "@qvac/test-suite": "^0.11.0" } }
import type { TestDefinition } from '@qvac/test-suite'

No consumer moves in this PR — packages/sdk/e2e keeps its published @qvac/qvac-test-suite dependency, and the migration above lands in #4083.

The old package stays installable and will be deprecated, not unpublished, so SDK release branches pinned to a released 0.10.x keep resolving. The framework still recognises all four names when resolving the installed package for mobile scaffolding and when externalising consumer test definitions, so a mixed setup during migration resolves correctly.

📦 Package Surface Changes

New package @qvac/test-suite (bin: qvac-test, exports . and ./mobile). The runtime API is unchanged from @qvac/qvac-test-suite@0.10.2 — only the package name moved. publishConfig.access: public is set because the new name is a brand-new scoped package and would otherwise default to restricted on first publish.

🚀 Workflow / Publish Changes

New .github/workflows/trigger-reusable-lib-test-suite.yml: release-merge-guard, GPR dev/feature/tmp publishing, npm release publishing and the release tag. Copied from trigger-reusable-lib-rag.yml post-#4055, with npm instead of bun for the build job.

Merging this to main publishes a GPR dev build. Pushes to this feature-* branch publish a GPR feature build, which is a useful early smoke test of the new publish pipeline before the real release.

🔄 Migration Notes

Before cutting release-test-suite-0.11.0: an npm Trusted Publisher must exist for @qvac/test-suite on tetherto/qvac, workflow trigger-reusable-lib-test-suite.yml, environment npm. No code can do this, and the first publish fails without it.

📚 Docs Changes

packages/test-suite/README.md, CHANGELOG.md, docs/repository-layout.md, docs/architecture/ARCHITECTURE.md, docs/ci/TEAMS.md.

.cursor/rules/framework/*.mdc are ported to .cursor/rules/sdk/test-suite/*.mdc, rewritten for monorepo context, with two documented inaccuracies corrected: api-exports omitted testReloadSchema / startNodeMemoryPoller / startDesktopMemoryPoller, and architecture described a {repo}-{branch}-{commit}-{timestamp} runId the code never implements (the runIdStrategy config field is declared but never read — flagged in the rule, tracked as a separate task).

The qts-* skills are dropped in favour of the existing qv-sdk-pr-create / qv-pr-review / qv-skill-list.

🤖 Generated with Claude Code

@Victor-Rodzko
Victor-Rodzko requested review from a team as code owners August 26, 2026 11:36
@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Review Status

Current Status: ✅ APPROVED
Approvals so far: Team Lead: 2, Member: 2

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Workflow security (shadow mode)

zizmor found 1074 finding(s) in .github/ (highest severity: high). This check is warn-only and does not block the merge.

Findings are annotated inline on the changed files and listed in the job summary.

Reproduce locally:

pipx run zizmor==1.27.0 --offline .github/

@github-actions

Copy link
Copy Markdown
Contributor

License compliance — findings detected (warn-only)

Critical: 0 · High: 5 · Medium: 0

Dependency License Scope Severity Outcome
actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd (none detected) runtime High blocks
actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f (none detected) runtime High blocks
actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f (none detected) runtime High blocks
tetherto/qvac-actions/npm-dist-tag-determination@096b1cfbd1d95399ae64d37f55cc16fd7a43314e (none detected) runtime High blocks
tetherto/qvac-actions/publish-library-to-npm@096b1cfbd1d95399ae64d37f55cc16fd7a43314e (none detected) runtime High blocks

How to resolve a blocking finding:

  • Remove or replace the disallowed dependency, or
  • If the license is genuinely acceptable, run the compliance SKILL and record the decision in .github/license-allowlist.yml (CODEOWNERS-reviewed), or
  • For a one-off, a maintainer can apply the license-override label (High findings only; Critical cannot be overridden).

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

  • ./.github/actions/release-merge-guard
  • ./docs/website
  • ./packages/ggml-coload-smoke
  • ./packages/fabric/test/integration
  • ./packages/inference-addon-cpp/mobile
  • ./packages/sdk/e2e
  • ./packages/llm-llamacpp/benchmarks/performance
  • ./packages/llm-llamacpp/benchmarks/server
  • ./packages/vla-ggml/sim/server
  • ./packages/embed-llamacpp/benchmarks/performance
  • ./packages/embed-llamacpp/benchmarks/server
  • ./packages/asr-ggml/benchmarks/server

@socket-security

socket-security Bot commented Aug 26, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​eslint-config-prettier@​10.1.810010010083100

View full report

Comment thread packages/test-suite/src/cli/commands/build-consumer-mobile.ts Fixed
Comment thread packages/test-suite/src/utils/mobile-env-baker.ts Fixed
Comment thread packages/test-suite/src/utils/mqtt-connection.ts Fixed
Moves the test-orchestration framework out of tetherto/qvac-test-suite into
packages/test-suite, flattening the former framework/ directory, and
registers it as a first-class SDK-pod package: SDK Pod Checks, the release
guard, GPR-dev / npm-release publishing and the changelog tooling now cover
it like @qvac/rag and @qvac/logging.

Renames the package from @qvac/qvac-test-suite to @qvac/test-suite,
dropping the duplicated scope word (GPR: @tetherto/test-suite-mono). The
directory name now equals the release slug, so package-paths.cjs resolves
it with no override. All four names stay recognised at runtime, so a
consumer pinned to a released 0.10.x keeps resolving.

Publishing follows the prebuilt-dist pattern used by rag/logging/error:
the package ships only compiled output and deliberately has no `prepare`
script, so dist is compiled once in a dedicated build job and every
publish job downloads it and publishes with
NPM_CONFIG_IGNORE_SCRIPTS=true. This matches how the standalone repo
published before the fold.

Nothing consumes the package yet — packages/sdk/e2e keeps its published
@qvac/qvac-test-suite dependency. Migrating that consumer, and the
test-sdk.yml source selector that goes with it, is a follow-up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Victor-Rodzko
Victor-Rodzko merged commit 0359b82 into main Aug 26, 2026
38 checks passed
@Victor-Rodzko
Victor-Rodzko deleted the feature-test-suite-fold branch August 26, 2026 14:36
Victor-Rodzko added a commit that referenced this pull request Aug 28, 2026
Switches packages/sdk/e2e from @qvac/qvac-test-suite to @qvac/test-suite
^0.11.0 — the same framework, published from this monorepo since the fold
in #4082 — and renames the import specifier across the e2e suite.

Adds a test-suite-source input (manifest | branch | npm) plus
test-suite-version to test-sdk.yml, mirroring the existing
inference-source selector. A new prepare-test-suite job resolves the
framework once per run and .github/actions/sdk-e2e-prepare-test-suite
applies it before each consumer install, so an SDK change can be tested
against an unreleased framework change without publishing. PRs touching
packages/test-suite run their e2e suite in branch mode.

The new action is added to all five sparse-checkout blocks across
test-node-sdk.yml, test-android-sdk.yml and test-ios-sdk.yml;
packages/test-suite itself is only checked out by the prepare-test-suite
job, which is the sole place that packs it. The report actions gain a
test-suite-provenance input so run summaries state which framework build
was used.

Callers that invoke the platform workflows directly without the new inputs
— coload-smoke-mobile.yml calls test-android-sdk.yml — resolve as
`manifest`, which now installs the published range as-is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Victor-Rodzko added a commit that referenced this pull request Aug 28, 2026
Switches packages/sdk/e2e from @qvac/qvac-test-suite to @qvac/test-suite
^0.11.0 — the same framework, published from this monorepo since the fold
in #4082 — and renames the import specifier across the e2e suite.

Adds a test-suite-source input (manifest | branch | npm) plus
test-suite-version to test-sdk.yml, mirroring the existing
inference-source selector. A new prepare-test-suite job resolves the
framework once per run and .github/actions/sdk-e2e-prepare-test-suite
applies it before each consumer install, so an SDK change can be tested
against an unreleased framework change without publishing. PRs touching
packages/test-suite run their e2e suite in branch mode.

The new action is added to all five sparse-checkout blocks across
test-node-sdk.yml, test-android-sdk.yml and test-ios-sdk.yml;
packages/test-suite itself is only checked out by the prepare-test-suite
job, which is the sole place that packs it. The report actions gain a
test-suite-provenance input so run summaries state which framework build
was used.

Callers that invoke the platform workflows directly without the new inputs
— coload-smoke-mobile.yml calls test-android-sdk.yml — resolve as
`manifest`, which now installs the published range as-is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants