QVAC-23799 feat[asr-ggml]: add opt-in CUDA GPU backend for whisper and parakeet - #4059
Closed
Zbig9000 wants to merge 1 commit into
Closed
QVAC-23799 feat[asr-ggml]: add opt-in CUDA GPU backend for whisper and parakeet#4059Zbig9000 wants to merge 1 commit into
Zbig9000 wants to merge 1 commit into
Conversation
…d parakeet Both engines already resolved and reported CUDA at runtime (BackendId.CUDA, 2) and the GPU integration tests already had cuda branches, but no build ever compiled the backend in: the speech-cpp dependencies only ever requested vulkan, metal and opencl. Add a cuda manifest feature forwarding to speech-cpp[cuda] -> ggml-speech[cuda] (GGML_CUDA=ON), selected by a new ASR_CUDA CMake option and the build:cuda / build:native:cuda scripts. CUDA is opt-in rather than default because it needs nvcc on the build host, which the prebuild runners do not carry; only the NVIDIA driver is needed at runtime. It is compiled alongside Vulkan and ggml registers CUDA first, so a use_gpu / useGPU request prefers CUDA and falls back to Vulkan when no supported device is present. Apple and Android are excluded. The cuda feature carries its own speech-cpp floor (2026-08-24#1, the version that introduced the feature). Feature dependencies only apply when the feature is selected, so default builds keep resolving at 2026-08-18.
Contributor
Review StatusCurrent Status: ❌ PENDING Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member. |
mexxik
approved these changes
Aug 25, 2026
GustavoA1604
left a comment
Contributor
There was a problem hiding this comment.
Do not open from fork
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 What problem does this PR solve?
asr-ggmlcould never run on CUDA. Both engines already resolved and reported CUDA at runtime —WhisperModel.cppandParakeetModel.cppmap ggml'sCUDA*backend name toBackendId.CUDA(2), andgpu.test.js/parakeet-gpu-smoke.test.jsalready hadcudabranches — but no build ever compiled the backend in.vcpkg.jsonrequestedspeech-cppwithvulkan(desktop/Android),metal(Apple) andopencl(Android), nevercuda. On NVIDIA hosts ause_gpu/useGPUrequest therefore always landed on Vulkan.📝 How does it solve it?
cudafeature invcpkg.jsonforwarding tospeech-cpp[cuda]→ggml-speech[cuda](GGML_CUDA=ON, nvcc resolved by the port).ASR_CUDACMake option appending"cuda"toVCPKG_MANIFEST_FEATURES, following theSD_CUDAprecedent indiffusion-cpp.build:cuda/build:native:cudascripts (bare-make generate -D ASR_CUDA=ON).nvccon the build host, which the prebuild runners do not carry. Only the NVIDIA driver is needed at runtime."supports": "!(osx | ios | android)".cudafeature carries its ownspeech-cppfloor,2026-08-24#1— the version that introduced the feature. Feature dependencies only apply when the feature is selected, so default builds keep resolving at2026-08-18and are unaffected.No companion PRs needed
speech-cpp[cuda]→ggml-speech[cuda]already shipped intetherto/qvac-registry-vcpkg@cc90801(QVAC-23802, speech-cpp2026-08-24#1). Nothing to add there.vcpkg-configuration.jsonbaseline: unchanged. The pinned baselinedca20a94carries only speech-cpp2026-08-10/2026-08-07, yetmainalready requires>= 2026-08-18and builds green — version constraints resolve against fetched registry HEAD, not the baseline commit.2026-08-24#1resolves the same way.Unreleasedentry without touchingversion, and bumps land in dedicated release PRs (e.g.chore[mod]: release asr-ggml 0.3.1, ...). This PR adds to## [Unreleased].🧪 How was it tested?
scripts/__tests__/build-backends.test.js(6 tests) pins the wiring: the option defaults toOFF,build:nativedoes not enable it,ASR_CUDAmaps to thecudamanifest feature, the feature forwards tospeech-cpp[cuda]withdefault-features: false, the platform guards match, and the CUDA floor is not below the base floor.node --test "scripts/__tests__/*.test.js"→ 46/46 pass.ON) — each correctly fails, so the assertions are not vacuous.vcpkg format-manifestaccepts the manifest. It also wants to canonically reorder the pre-existing dependency blocks, which the committed file does not follow, so that churn was left out to keep the diff reviewable.Not covered: an actual
-D ASR_CUDA=ONcompile and a CUDA-vs-Vulkan runtime check. Both need a host with the CUDA toolkit; by design this PR adds no CI job, so the first real CUDA build is manual. The claim that ggml prefers CUDA over Vulkan is documented but unmeasured.🔌 API Changes
No API changes.
BackendId.CUDA(2) was already exported and already returned bygetBackendInfo(); this PR only makes it reachable.