Is your feature request related to a problem? Please describe.
Summary
The ggml port builds with GGML_NATIVE=ON by default (upstream ggml's own CMake default) and does not expose any way to override this or the individual GGML_AVX/GGML_AVX2/GGML_AVX512/GGML_FMA/GGML_F16C options as a vcpkg feature. This causes a portability problem specific to vcpkg's binary caching model.
Problem
GGML_NATIVE=ON makes the compiler target whatever CPU is running the build (-march=native), so the resulting binary only runs correctly on CPUs with an equal-or-greater instruction set to the build host.
vcpkg's binary caching computes its cache key from port version, triplet, and compiler config not from the specific CPU that performed the build.
So in any environment where build hosts vary between cache-populating
runs and cache-consuming runs (e.g. GitHub Actions hosted runners, whose
underlying hardware generation is not guaranteed consistent across jobs),
a binary compiled on a host with e.g. AVX-512 gets cached and later served
to a host without AVX-512, resulting in a SIGILL (illegal instruction)
at runtime.
This is not an upstream ggml/llama.cpp bug, GGML_NATIVE and the individual ISA flags work as documented. It's a vcpkg packaging gap: there's no supported way for a consumer to request a portable, fixed-ISA build via
vcpkg.json.
Proposed solution
Expose CPU-targeting as a vcpkg feature on the ggml port, following the same pattern already used for blas, cuda, metal, opencl, openmp, and vulkan in ports/ggml/vcpkg.json. For example:
- A
portable (or no-native) feature that sets GGML_NATIVE=OFF and pins a conservative baseline (e.g. x86-64-v2-equivalent: SSE4.2, no AVX/AVX2/AVX512/FMA), for consumers who need cache-safe, host-independent binaries.
- Optionally, exposing
GGML_CPU_ALL_VARIANTS + GGML_BACKEND_DL as a separate feature (e.g. runtime-dispatch) for consumers who want per-host optimal ISA selection at the cost of longer build times and larger artifacts (this is what upstream's own release pipeline uses: https://github.com/ggml-org/llama.cpp/blob/master/.github/workflows/release.yml).
Describe alternatives you've considered
Meanwhile, Docwire [https://github.com/docwire/docwire] is considering maintain an overlay port but this means every consumer hitting this issue has to independently discover and maintain the same overlay.
Additional context
Context
This surfaced from intermittent CI SIGILL crashes in our project (DocWire, github.com/docwire/docwire) using llama-cpp/ggml via vcpkg on GitHub Actions, traced to exactly this build-host/run-host ISA mismatch via vcpkg's binary cache.
Is your feature request related to a problem? Please describe.
Summary
The
ggmlport builds withGGML_NATIVE=ONby default (upstream ggml's own CMake default) and does not expose any way to override this or the individualGGML_AVX/GGML_AVX2/GGML_AVX512/GGML_FMA/GGML_F16Coptions as a vcpkg feature. This causes a portability problem specific to vcpkg's binary caching model.Problem
GGML_NATIVE=ONmakes the compiler target whatever CPU is running the build (-march=native), so the resulting binary only runs correctly on CPUs with an equal-or-greater instruction set to the build host.vcpkg's binary caching computes its cache key from port version, triplet, and compiler config not from the specific CPU that performed the build.
So in any environment where build hosts vary between cache-populating
runs and cache-consuming runs (e.g. GitHub Actions hosted runners, whose
underlying hardware generation is not guaranteed consistent across jobs),
a binary compiled on a host with e.g. AVX-512 gets cached and later served
to a host without AVX-512, resulting in a
SIGILL(illegal instruction)at runtime.
This is not an upstream ggml/llama.cpp bug,
GGML_NATIVEand the individual ISA flags work as documented. It's a vcpkg packaging gap: there's no supported way for a consumer to request a portable, fixed-ISA build viavcpkg.json.Proposed solution
Expose CPU-targeting as a vcpkg feature on the
ggmlport, following the same pattern already used forblas,cuda,metal,opencl,openmp, andvulkaninports/ggml/vcpkg.json. For example:portable(orno-native) feature that setsGGML_NATIVE=OFFand pins a conservative baseline (e.g.x86-64-v2-equivalent: SSE4.2, no AVX/AVX2/AVX512/FMA), for consumers who need cache-safe, host-independent binaries.GGML_CPU_ALL_VARIANTS+GGML_BACKEND_DLas a separate feature (e.g.runtime-dispatch) for consumers who want per-host optimal ISA selection at the cost of longer build times and larger artifacts (this is what upstream's own release pipeline uses: https://github.com/ggml-org/llama.cpp/blob/master/.github/workflows/release.yml).Describe alternatives you've considered
Meanwhile, Docwire [https://github.com/docwire/docwire] is considering maintain an overlay port but this means every consumer hitting this issue has to independently discover and maintain the same overlay.
Additional context
Context
This surfaced from intermittent CI SIGILL crashes in our project (DocWire, github.com/docwire/docwire) using
llama-cpp/ggmlvia vcpkg on GitHub Actions, traced to exactly this build-host/run-host ISA mismatch via vcpkg's binary cache.