Skip to content

2026-07-06-candidate-repo-survey.md

Latest commit

 

History

History
362 lines (243 loc) · 10.7 KB

2026-07-06-candidate-repo-survey.md

File metadata and controls

362 lines (243 loc) · 10.7 KB
Raw

Candidate Repo Survey

Date: 2026-07-06. Agent: Codex. Scope: shallow local survey only. No builds or benchmarks were run.

Local Hardware Facts Captured

  • Host: gigul2.
  • GPU target: gfx1100, RX 7900 XTX class, device id 0x744c.
  • ROCm SMI driver: 6.16.6.
  • hipcc: /opt/rocm-7.1.1/bin/hipcc.
  • HIP version: 7.1.52802-26aae437f6.
  • Installed ROCm trees: /opt/rocm-7.1.1, /opt/rocm-7.2.0, /opt/rocm-7.2.1.

Current Ranking

Rank Repo Current read
1 hipfire/ Baseline and most relevant immediate worktree
2 lucebox-hub/ Strong gfx1100 DFlash lessons; likely source of config ideas
3 buun-llama-cpp/ TCQ KV-cache idea is directly about long context; ROCm path claims gfx1100 support
4 rocmfp4-llama/ and ROCmFPX/ AMD llama.cpp format experiments; useful but need RDNA3 validation
5 llamacpp-rocm/ Packaged ROCm llama.cpp baseline; useful for smoke and portability
6 vllm/ and amd-r9700-vllm-toolboxes/ Operational/server baseline; less likely to beat Hipfire on this exact workload
7 tinygrad/ Compiler/runtime research source, not a direct serving candidate
8 QuixiCore-ROCm/ and QuixiCore-CUDA/ Kernel idea libraries; ROCm branch is CDNA-focused, CUDA branch is translation-only

Hipfire

Path: hipfire/. Remote: https://github.com/Kaden-Schutt/hipfire.git. Size observed: about 50 GB.

What it is:

  • Rust + HIP LLM inference engine for AMD RDNA GPUs.
  • Current baseline for gigul2.
  • Supports Qwen 3.5/3.6, LFM, DeepSeek V4 Flash, MiniMax-M2, and more.
  • README reports 7900 XTX headline numbers including Qwen 3.5 27B at 47 tok/s decode and strong DFlash speculative decode.

Local baseline evidence:

  • hipfire/HIPFIRE_QWEN36_27B_GIGUL2.md records Qwen3.6-27B MQ4 on RX 7900 XTX at about 40-42 tok/s decode through very long contexts, including 37.2 tok/s at 260K single-shot context.
  • Best recorded config uses kv_cache asym2, max_seq 262144, and dflash_mode auto.

Useful local gates:

  • hipfire/docs/methodology/perf-benchmarking.md
  • hipfire/scripts/coherence-gate.sh
  • hipfire/scripts/coherence-gate-dflash.sh
  • hipfire/scripts/qwen36_bench.sh
  • hipfire/scripts/bench_qwen36_ar_dflash.sh
  • hipfire/cli/e8_longctx_smoke.ts
  • hipfire/cli/e8_coherence_smoke.ts

First next action:

  • Reproduce the current Hipfire baseline under the new gold speed/correctness protocol before comparing anything else.

tinygrad

Path: tinygrad/. Remote: git@github.com:ljubomirj/tinygrad.git. Size observed: about 225 MB.

What it is:

  • End-to-end tensor/compiler/runtime stack.
  • Supports AMD via tinygrad/runtime/ops_amd.py and HIP via tinygrad/runtime/ops_hip.py.
  • Useful for understanding AMD codegen, scheduling, and high-utilization experiments.

Why it matters:

  • LJ noted tinygrad previously drove the 7900 XTX hard enough to expose thermal limits.
  • It may contain lessons about occupancy, compiler search, or AMD runtime usage.

Caveat:

  • Not a drop-in Qwen3.6-27B serving replacement for this project.

First next action:

  • If this path is pursued, run a bounded AMD matmul/bandwidth smoke and record GPU utilization, not an LLM serving benchmark.

QuixiCore-ROCm

Path: QuixiCore-ROCm/. Remote: https://github.com/QuixiAI/QuixiCore-ROCm. Size observed: about 309 MB.

What it is:

  • Native ROCm QuixiCore backend, forked from HipKittens.
  • README describes CDNA2/CDNA3/CDNA4 accelerator focus.
  • Common entrypoints include scripts/configure, scripts/build, scripts/test, and scripts/bench.

Useful files:

  • QuixiCore-ROCm/perf/README.md
  • QuixiCore-ROCm/docs/profiling/profile_pmc_counters.sh
  • QuixiCore-ROCm/.quixicore/kernels.yaml

Caveat:

  • README setup is MI300/MI350 oriented. gigul2 is RDNA3/gfx1100, so this is not a direct build target until proven.

First next action:

  • Inspect kernel coverage and try only a tiny build/test if the code clearly supports gfx1100. Otherwise keep as a CDNA kernel-design reference.

QuixiCore-CUDA

Path: QuixiCore-CUDA/. Remote: https://github.com/QuixiAI/QuixiCore-CUDA.git. Size observed: about 516 MB.

What it is:

  • CUDA implementation of QuixiCore, with extensive SM86 quant, serving, MoE, sampler, sparse attention, and MetalForge-derived ports.
  • README contains many RTX 3090 numbers and algorithm descriptions.

Why it matters:

  • It may contain algorithmic patterns transferable to HIP/RDNA3.
  • It is not directly runnable on gigul2's AMD GPU.

First next action:

  • Use only as a source-reading reference for a specific Hipfire or llama.cpp hypothesis. Do not start by porting broad subsystems.

ROCmFPX

Path: ROCmFPX/. Remote: https://github.com/charlie12345/ROCmFPX.git. Size observed: about 190 MB.

What it is:

  • Experimental AMD-focused ROCmFP3/4/6/8 quantization formats for llama.cpp.
  • Provides build scripts for RDNA2, RDNA3/gfx1100, RDNA4, and Strix.
  • Includes HIP/ROCm and Vulkan coverage plus many check scripts.

Useful files:

  • ROCmFPX/README.md
  • ROCmFPX/ROCMFP4-UPSTREAM-INTEGRATION.md
  • ROCmFPX/scripts/check-rocmfpx-*.sh
  • ROCmFPX/scripts/build-rdna3.sh

Caveat:

  • Experimental research build. Some docs and helpers default to Strix paths or internal artifact locations.

First next action:

  • Run only build/quant/runtime checks that explicitly target RDNA3/gfx1100, and keep results below Hipfire until silver correctness passes.

buun-llama-cpp

Path: buun-llama-cpp/. Remote: https://github.com/spiritbuun/buun-llama-cpp.git. Size observed: about 976 MB.

What it is:

  • Experimental llama.cpp fork with Trellis-Coded Quantization for KV cache.
  • README claims ROCm/HIP build tested on ROCm 7.2 + RDNA3/gfx1100.
  • Goal is 2-3x more context in the same VRAM with similar quality.

Useful files:

  • buun-llama-cpp/README.md
  • buun-llama-cpp/SD-080-benchmark-notes.txt
  • buun-llama-cpp/scripts/compare-llama-bench.py

Potential value:

  • TCQ long-context KV compression is directly relevant to the Hipfire baseline.

Caveat:

  • Existing local Hipfire report says a prior buun TCQ attempt failed to compile because CUDA-only kernels had no ROCm path. The current README claims ROCm, so this needs a fresh, bounded check.

First next action:

  • Attempt a no-modification RDNA3 build in a separate build dir. If it fails, document the exact missing CUDA/HIP path rather than patching immediately.

rocmfp4-llama

Path: rocmfp4-llama/. Remote: https://github.com/charlie12345/rocmfp4-llama. Size observed: about 485 MB.

What it is:

  • llama.cpp fork for AMD-focused ROCmFP4 quantization and backend work.
  • Validated strongest local results in its README are Strix Halo, not 7900 XTX.
  • Provides an RDNA3 build script.

Useful files:

  • rocmfp4-llama/README.md
  • rocmfp4-llama/docs/BUILD-AMD-ARCHITECTURES.md
  • rocmfp4-llama/scripts/build-rdna3.sh
  • rocmfp4-llama/scripts/check-rocmfp4-*.sh

Caveat:

  • Do not apply Strix defaults blindly to gfx1100.

First next action:

  • Run RDNA3 build and the smallest ROCmFP4 runtime regression guard if model files are available.

lucebox-hub

Path: lucebox-hub/. Remote: https://github.com/Luce-Org/lucebox-hub.git. Size observed: about 120 MB.

What it is:

  • Local LLM inference server and optimization collection.
  • Includes DFlash, PFlash, speculative decode, and HIP support.
  • README explicitly lists RX 7900 XTX/gfx1100 as supported for HIP DFlash.

Most important finding:

  • lucebox-hub/server/docs/HIP_PERF_PLAN.md says on gfx1100, lowering --ddtree-budget from 22 to 8 improved Lucebox HIP decode from 49.81 tok/s to 76.02 tok/s in that stack.
  • The same doc says this is arch-specific and regresses on gfx1151 and gfx1201.

Why it matters:

  • This is a concrete gfx1100 lesson. Even if Lucebox does not replace Hipfire, the shape of the result may guide Hipfire DFlash/DDTree tuning.

First next action:

  • Check whether Hipfire has an analogous DFlash/DDTree budget or verification batch shape. If yes, benchmark the config-only change before touching kernels.

amd-r9700-vllm-toolboxes

Path: amd-r9700-vllm-toolboxes/. Remote: https://github.com/kyuz0/amd-r9700-vllm-toolboxes. Size observed: about 18 MB.

What it is:

  • Container/toolbox setup for vLLM on AMD Radeon R9700/gfx1201.
  • Built around TheRock ROCm nightlies and vLLM.
  • Includes benchmark automation.

Relevance:

  • Operational reference for ROCm vLLM containers and benchmark setup.

Caveat:

  • Primary target is R9700/gfx1201, not RX 7900 XTX/gfx1100.
  • Tuning doc is for R9700 undervolting and power limit, not this card.

First next action:

  • Use only as a vLLM benchmark/container reference unless adapting to gfx1100 is explicitly in scope.

vLLM

Path: vllm/. Remote: https://github.com/vllm-project/vllm.git. Size observed: about 296 MB.

What it is:

  • Upstream vLLM source tree.
  • Supports AMD GPUs and many serving features.

Local context:

  • Root LEARNINGS.md records prior vLLM ROCm setup on this machine.
  • Stable local vLLM serving needed constraints such as Python 3.12, TORCHDYNAMO_DISABLE=1, text-only mode, small context, and eager mode in some cases.

Relevance:

  • Useful OpenAI-compatible serving baseline and model support testbed.
  • Less likely to beat Hipfire's current Qwen3.6-27B long-context decode on the 24 GB 7900 XTX without substantial tuning.

First next action:

  • If revisited, start from the documented local venv and eager-safe launch profile, then compare as silver correctness/speed only.

llamacpp-rocm

Path: llamacpp-rocm/. Remote: https://github.com/lemonade-sdk/llamacpp-rocm.git. Size observed: about 566 MB.

What it is:

  • Packaged llama.cpp ROCm build/release project using TheRock ROCm 7.
  • Supports gfx110X, gfx115x, gfx120X, and gfx103X.
  • Local tree contains a prebuilt Ubuntu gfx110X release zip.

Useful files:

  • llamacpp-rocm/README.md
  • llamacpp-rocm/docs/manual_instructions.md
  • llamacpp-rocm/releases/llama-b1261-ubuntu-rocm-gfx110X-x64.zip

Relevance:

  • Quick smoke baseline for standard llama.cpp ROCm behavior on gfx1100.
  • Useful for "does standard llama.cpp run this GGUF here?" checks.

Caveat:

  • A generic packaged baseline is unlikely to beat Hipfire on the specific Qwen3.6-27B long-context target, but it can be a clean control.

First next action:

  • Extract/use the gfx110X package only for a bronze/silver llama.cpp baseline, not for code modification.

Global Next Steps

  1. Reproduce Hipfire current baseline under the new protocol.
  2. Inspect Hipfire's DFlash/DDTree config surface for the Lucebox gfx1100 budget lesson.
  3. Try buun TCQ RDNA3 build as a bounded compile-only trial.
  4. Try ROCmFP4/ROCmFPX RDNA3 smoke only if model/build prerequisites are cheap.
  5. Keep QuixiCore and tinygrad as reference paths until a specific kernel hypothesis needs them.