@@ -182,296 +182,6 @@ the generator instead. Prose outside the markers is preserved. -->
182182| ` whispercpp_args ` | ` --whispercpp-args ` | ARGS | "" | Custom arguments to pass to whisper-server |
183183<!-- END GENERATED: backend-options -->
184184
185- ## Models
186-
187- <!-- BEGIN GENERATED: backend-models -->
188- #### ` acestep ` — ACE-Step (1 models)
189-
190- | Model | Size (GB) | Labels |
191- | -------| -----------| --------|
192- | ` ACE-Step-Music ` | 10.5 | audio-generation |
193-
194- #### ` collection.omni ` — collection.omni (5 models)
195-
196- | Model | Size (GB) | Labels |
197- | -------| -----------| --------|
198- | ` LMX-Omni-5.5B-Lite ` | 9.3 | — |
199- | ` LMX-Omni-52B-Halo ` | 44.77 | — |
200- | ` Lite Collection ` | | — |
201- | ` RPG-HaloTales-V1 ` | 39.77 | — |
202- | ` Ultra Collection ` | | — |
203-
204- #### ` kokoro ` — Kokoro (1 models)
205-
206- | Model | Size (GB) | Labels |
207- | -------| -----------| --------|
208- | ` kokoro-v1 ` | 0.354 | tts |
209-
210- #### ` llamacpp ` — Llama.cpp GPU (88 models)
211-
212- | Model | Size (GB) | Labels |
213- | -------| -----------| --------|
214- | ` Bonsai-1.7B-gguf ` | 0.25 | llamacpp |
215- | ` Bonsai-4B-gguf ` | 0.572 | llamacpp |
216- | ` Bonsai-8B-gguf ` | 1.16 | llamacpp |
217- | ` Cogito-v2-llama-109B-MoE-GGUF ` | 65.4 | vision |
218- | ` DeepSeek-Qwen3-8B-GGUF ` | 5.25 | reasoning |
219- | ` Devstral-Small-2507-GGUF ` | 14.3 | coding, tool-calling |
220- | ` GLM-4.5-Air-UD-Q4K-XL-GGUF ` | 67.7 | reasoning |
221- | ` GLM-4.7-Flash-GGUF ` | 17.5 | tool-calling |
222- | ` Gemma-3-4b-it-GGUF ` | 3.34 | vision |
223- | ` Gemma-4-12B-it-GGUF ` | 7.29 | tool-calling, vision, llamacpp |
224- | ` Gemma-4-12B-it-MTP-GGUF ` | 7.75 | tool-calling, llamacpp, vision, mtp |
225- | ` Gemma-4-26B-A4B-it-GGUF ` | 18.1 | hot, tool-calling, vision, llamacpp |
226- | ` Gemma-4-26B-A4B-it-MTP-GGUF ` | 18.5 | hot, tool-calling, vision, llamacpp, mtp |
227- | ` Gemma-4-31B-it-GGUF ` | 19.5 | hot, tool-calling, vision, llamacpp |
228- | ` Gemma-4-31B-it-MTP-GGUF ` | 20.0 | hot, tool-calling, vision, llamacpp, mtp |
229- | ` Gemma-4-E2B-it-GGUF ` | 4.09 | tool-calling, vision, llamacpp |
230- | ` Gemma-4-E4B-it-GGUF ` | 5.97 | tool-calling, vision, llamacpp |
231- | ` Jan-nano-128k-GGUF ` | 2.5 | — |
232- | ` Jan-v1-4B-GGUF ` | 2.5 | — |
233- | ` LFM2-1.2B-GGUF ` | 0.731 | — |
234- | ` LFM2-24B-A2B-GGUF ` | 14.4 | — |
235- | ` LFM2-8B-A1B-GGUF ` | 5.04 | — |
236- | ` LFM2.5-1.2B-Instruct-GGUF ` | 0.731 | — |
237- | ` LFM2.5-8B-A1B ` | 5.16 | — |
238- | ` Llama-3.2-1B-Instruct-GGUF ` | 0.834 | — |
239- | ` Llama-3.2-3B-Instruct-GGUF ` | 2.06 | — |
240- | ` Llama-4-Scout-17B-16E-Instruct-GGUF ` | 63.2 | vision |
241- | ` MiniCPM-V-4-GGUF ` | 3.15 | vision |
242- | ` MiniCPM-V-4.5-GGUF ` | 6.12 | vision |
243- | ` MiniCPM-V-4.6-GGUF ` | 1.64 | hot, vision |
244- | ` MiniCPM-V-4.6-Thinking-GGUF ` | 1.64 | reasoning, vision |
245- | ` MiniCPM-o-2.6-Vision-GGUF ` | 5.73 | vision |
246- | ` MiniCPM-o-4.5-Vision-GGUF ` | 6.12 | vision |
247- | ` MiniCPM3-4B-GGUF ` | 2.47 | — |
248- | ` MiniCPM4-8B-GGUF ` | 4.97 | — |
249- | ` MiniCPM4.1-8B-GGUF ` | 4.97 | reasoning |
250- | ` MiniCPM5-1B-GGUF ` | 0.69 | hot, reasoning |
251- | ` Ministral-3-3B-Instruct-2512-GGUF ` | 2.99 | vision |
252- | ` Muse-Glimmer-30B-GGUF ` | 19.9 | hot, tool-calling, vision, reasoning, llamacpp, dflash |
253- | ` Nemotron-3-Nano-30B-A3B-GGUF ` | 22.8 | — |
254- | ` Phi-4-mini-instruct-GGUF ` | 2.49 | — |
255- | ` Playable1-GGUF ` | 4.68 | coding |
256- | ` PromptBridge-0.6b-Alpha-GGUF ` | 0.397 | — |
257- | ` Qwen2.5-Coder-32B-Instruct-GGUF ` | 19.9 | coding |
258- | ` Qwen2.5-Omni-3B-GGUF ` | 4.73 | vision, chat-transcription |
259- | ` Qwen2.5-Omni-7B-GGUF ` | 7.33 | vision, chat-transcription |
260- | ` Qwen2.5-VL-3B-Instruct-GGUF ` | 3.27 | vision |
261- | ` Qwen2.5-VL-7B-Instruct-GGUF ` | 6.04 | vision |
262- | ` Qwen3-0.6B-GGUF ` | 0.38 | reasoning |
263- | ` Qwen3-1.7B-GGUF ` | 1.06 | reasoning |
264- | ` Qwen3-14B-GGUF ` | 8.54 | reasoning |
265- | ` Qwen3-30B-A3B-GGUF ` | 17.4 | reasoning |
266- | ` Qwen3-30B-A3B-Instruct-2507-GGUF ` | 17.4 | tool-calling |
267- | ` Qwen3-4B-GGUF ` | 2.38 | reasoning |
268- | ` Qwen3-4B-Instruct-2507-GGUF ` | 2.5 | tool-calling |
269- | ` Qwen3-8B-GGUF ` | 5.25 | reasoning |
270- | ` Qwen3-Coder-30B-A3B-Instruct-GGUF ` | 18.6 | coding, tool-calling, hot |
271- | ` Qwen3-Coder-Next-GGUF ` | 48.0 | coding, tool-calling, hot |
272- | ` Qwen3-Embedding-0.6B-GGUF ` | 0.64 | embeddings |
273- | ` Qwen3-Embedding-4B-GGUF ` | 4.28 | embeddings |
274- | ` Qwen3-Embedding-8B-GGUF ` | 8.05 | embeddings |
275- | ` Qwen3-Next-80B-A3B-Instruct-GGUF ` | 46.1 | tool-calling |
276- | ` Qwen3-VL-4B-Instruct-GGUF ` | 3.33 | vision |
277- | ` Qwen3-VL-8B-Instruct-GGUF ` | 6.19 | vision |
278- | ` Qwen3.5-0.8B-GGUF ` | 0.764 | vision, tool-calling |
279- | ` Qwen3.5-122B-A10B-GGUF ` | 77.9 | vision, tool-calling |
280- | ` Qwen3.5-122B-A10B-MTP-GGUF ` | 79.6 | vision, tool-calling, mtp |
281- | ` Qwen3.5-27B-GGUF ` | 18.5 | vision, tool-calling |
282- | ` Qwen3.5-2B-GGUF ` | 2.01 | vision, tool-calling |
283- | ` Qwen3.5-35B-A3B-GGUF ` | 23.1 | vision, tool-calling |
284- | ` Qwen3.5-4B-GGUF ` | 3.58 | vision, tool-calling, hot |
285- | ` Qwen3.5-4B-MTP-GGUF ` | 3.66 | vision, tool-calling, mtp |
286- | ` Qwen3.5-9B-GGUF ` | 6.88 | vision, tool-calling |
287- | ` Qwen3.6-27B-GGUF ` | 18.5 | vision, tool-calling |
288- | ` Qwen3.6-27B-MTP-GGUF ` | 18.8 | vision, tool-calling, mtp, hot |
289- | ` Qwen3.6-35B-A3B-GGUF ` | 23.3 | vision, tool-calling, hot |
290- | ` Qwen3.6-35B-A3B-MTP-GGUF ` | 23.8 | vision, tool-calling, mtp |
291- | ` SmolLM3-3B-GGUF ` | 1.94 | — |
292- | ` Tiny-Test-Model-GGUF ` | 0.18 | — |
293- | ` bge-reranker-v2-m3-GGUF ` | 0.636 | reranking |
294- | ` gpt-oss-120b-GGUF ` | 62.8 | reasoning, tool-calling |
295- | ` gpt-oss-120b-mxfp-GGUF ` | 63.4 | hot, reasoning, tool-calling |
296- | ` gpt-oss-20b-GGUF ` | 11.6 | reasoning, tool-calling |
297- | ` gpt-oss-20b-mxfp4-GGUF ` | 12.1 | hot, reasoning, tool-calling |
298- | ` granite-4.0-h-tiny-GGUF ` | 4.25 | tool-calling |
299- | ` jina-reranker-v1-tiny-en-GGUF ` | 0.0367 | reranking |
300- | ` nomic-embed-text-v1-GGUF ` | 0.0781 | embeddings |
301- | ` nomic-embed-text-v2-moe-GGUF ` | 0.51 | embeddings |
302-
303- #### ` moonshine ` — Moonshine (3 models)
304-
305- | Model | Size (GB) | Labels |
306- | -------| -----------| --------|
307- | ` Moonshine-Medium-Streaming ` | 1.08 | transcription, realtime-transcription, hot |
308- | ` Moonshine-Small-Streaming ` | 0.431 | transcription, realtime-transcription |
309- | ` Moonshine-Tiny-Streaming ` | 0.202 | transcription, realtime-transcription |
310-
311- #### ` onnxruntime ` — ONNX Runtime (2 models)
312-
313- | Model | Size (GB) | Labels |
314- | -------| -----------| --------|
315- | ` Bert-Phishing-ONNX ` | 1.34 | classification |
316- | ` Phishing-Email-Detection-ONNX ` | 0.27 | classification |
317-
318- #### ` openmoss ` — OpenMOSS TTS (2 models)
319-
320- | Model | Size (GB) | Labels |
321- | -------| -----------| --------|
322- | ` MOSS-VoiceGen ` | 7.3 | tts, voice-design |
323- | ` OpenMOSS-TTS ` | 12.5 | tts |
324-
325- #### ` ryzenai-llm ` — Ryzen AI LLM (79 models)
326-
327- | Model | Size (GB) | Labels |
328- | -------| -----------| --------|
329- | ` AMD-OLMo-1B-SFT-DPO-Hybrid ` | 1.48 | — |
330- | ` CodeLlama-7b-Instruct-hf-Hybrid ` | 7.24 | coding |
331- | ` CodeLlama-7b-Instruct-hf-NPU ` | 7.54 | coding |
332- | ` DeepSeek-R1-Distill-Llama-8B-CPU ` | 6.2 | reasoning |
333- | ` DeepSeek-R1-Distill-Llama-8B-Hybrid ` | 9.09 | reasoning |
334- | ` DeepSeek-R1-Distill-Llama-8B-NPU ` | 9.3 | reasoning |
335- | ` DeepSeek-R1-Distill-Qwen-1.5B-Hybrid ` | 2.19 | reasoning |
336- | ` DeepSeek-R1-Distill-Qwen-1.5B-NPU ` | 2.3 | reasoning |
337- | ` DeepSeek-R1-Distill-Qwen-7B-CPU ` | 6.2 | reasoning |
338- | ` DeepSeek-R1-Distill-Qwen-7B-Hybrid ` | 8.67 | reasoning |
339- | ` DeepSeek-R1-Distill-Qwen-7B-NPU ` | 8.87 | reasoning |
340- | ` Gemma-3-4b-it-mm-NPU ` | 6.68 | vision |
341- | ` Llama-2-7b-chat-hf-Hybrid ` | 7.31 | — |
342- | ` Llama-2-7b-chat-hf-NPU ` | 7.47 | — |
343- | ` Llama-2-7b-hf-Hybrid ` | 7.31 | — |
344- | ` Llama-2-7b-hf-NPU ` | 7.47 | — |
345- | ` Llama-3.1-8B-Hybrid ` | 9.09 | — |
346- | ` Llama-3.1-8B-NPU ` | 9.3 | — |
347- | ` Llama-3.2-1B-Hybrid ` | 1.89 | — |
348- | ` Llama-3.2-1B-Instruct-CPU ` | 1.76 | — |
349- | ` Llama-3.2-1B-Instruct-Hybrid ` | 1.89 | — |
350- | ` Llama-3.2-1B-Instruct-NPU ` | 1.96 | — |
351- | ` Llama-3.2-1B-NPU ` | 1.96 | — |
352- | ` Llama-3.2-3B-Hybrid ` | 4.28 | — |
353- | ` Llama-3.2-3B-Instruct-CPU ` | 3.38 | — |
354- | ` Llama-3.2-3B-Instruct-Hybrid ` | 4.28 | — |
355- | ` Meta-Llama-3-8B-Hybrid ` | 9.06 | — |
356- | ` Meta-Llama-3-8B-NPU ` | 9.23 | — |
357- | ` Meta-Llama-3.1-8B-Instruct-Hybrid ` | 9.09 | — |
358- | ` Meta-Llama-3.1-8B-Instruct-NPU ` | 9.3 | — |
359- | ` Mistral-7B-Instruct-v0.1-Hybrid ` | 7.34 | — |
360- | ` Mistral-7B-Instruct-v0.1-NPU ` | 8.01 | — |
361- | ` Mistral-7B-Instruct-v0.2-Hybrid ` | 7.34 | — |
362- | ` Mistral-7B-Instruct-v0.2-NPU ` | 8.01 | — |
363- | ` Mistral-7B-Instruct-v0.3-Hybrid ` | 7.35 | — |
364- | ` Mistral-7B-Instruct-v0.3-NPU ` | 8.09 | — |
365- | ` Mistral-7B-v0.3-Hybrid ` | 7.35 | — |
366- | ` Mistral-7B-v0.3-NPU ` | 8.09 | — |
367- | ` Phi-3-Mini-Instruct-CPU ` | 2.39 | — |
368- | ` Phi-3-mini-128k-instruct-Hybrid ` | 4.21 | — |
369- | ` Phi-3-mini-128k-instruct-NPU ` | 4.35 | — |
370- | ` Phi-3-mini-4k-instruct-Hybrid ` | 4.19 | — |
371- | ` Phi-3-mini-4k-instruct-NPU ` | 4.3 | — |
372- | ` Phi-3.5-mini-instruct-Hybrid ` | 4.21 | — |
373- | ` Phi-3.5-mini-instruct-NPU ` | 4.35 | — |
374- | ` Phi-4-mini-instruct-Hybrid ` | 5.47 | — |
375- | ` Phi-4-mini-instruct-NPU ` | 5.59 | — |
376- | ` Phi-4-mini-reasoning-Hybrid ` | 5.47 | reasoning |
377- | ` Qwen-1.5-7B-Chat-CPU ` | 6.32 | — |
378- | ` Qwen-2.5-1.5B-Instruct-Hybrid ` | 2.17 | — |
379- | ` Qwen-2.5-1.5B-Instruct-NPU ` | 2.25 | — |
380- | ` Qwen1.5-7B-Chat-Hybrid ` | 8.83 | — |
381- | ` Qwen1.5-7B-Chat-NPU ` | 9.02 | — |
382- | ` Qwen2-1.5B-Hybrid ` | 2.19 | — |
383- | ` Qwen2-1.5B-NPU ` | 2.3 | — |
384- | ` Qwen2-7B-Hybrid ` | 8.68 | — |
385- | ` Qwen2-7B-NPU ` | 8.88 | — |
386- | ` Qwen2.5-0.5B-Instruct-CPU ` | 0.834 | — |
387- | ` Qwen2.5-0.5B-Instruct-Hybrid ` | 0.828 | — |
388- | ` Qwen2.5-14B-instruct-Hybrid ` | 16.5 | — |
389- | ` Qwen2.5-3B-Instruct-Hybrid ` | 3.97 | — |
390- | ` Qwen2.5-3B-Instruct-NPU ` | 4.1 | — |
391- | ` Qwen2.5-7B-Instruct-Hybrid ` | 8.65 | — |
392- | ` Qwen2.5-7B-Instruct-NPU ` | 8.83 | — |
393- | ` Qwen2.5-Coder-0.5B-Instruct-Hybrid ` | 0.828 | coding |
394- | ` Qwen2.5-Coder-1.5B-Instruct-Hybrid ` | 2.17 | coding |
395- | ` Qwen2.5-Coder-1.5B-Instruct-NPU ` | 2.25 | coding |
396- | ` Qwen2.5-Coder-7B-Instruct-Hybrid ` | 8.65 | coding |
397- | ` Qwen2.5-Coder-7B-Instruct-NPU ` | 8.83 | coding |
398- | ` Qwen3-1.7B-Hybrid ` | 2.55 | reasoning |
399- | ` Qwen3-14B-Hybrid ` | 16.5 | reasoning |
400- | ` Qwen3-4B-Hybrid ` | 5.17 | reasoning |
401- | ` Qwen3-8B-Hybrid ` | 9.42 | reasoning |
402- | ` SmolLM-135M-Instruct-Hybrid ` | 0.232 | — |
403- | ` SmolLM2-135M-Instruct-Hybrid ` | 0.233 | — |
404- | ` chatglm3-6b-Hybrid ` | 6.9 | — |
405- | ` chatglm3-6b-NPU ` | 7.04 | — |
406- | ` gemma-2-2b-Hybrid ` | 4.04 | — |
407- | ` gpt-oss-20b-NPU ` | 13.4 | — |
408-
409- #### ` sd-cpp ` — StableDiffusion.cpp (12 models)
410-
411- | Model | Size (GB) | Labels |
412- | -------| -----------| --------|
413- | ` Flux-2-Klein-4B ` | 16.1 | image, edit |
414- | ` Flux-2-Klein-9B-GGUF ` | 19.0 | image, edit |
415- | ` Qwen-Image-2512-GGUF ` | 19.4 | image |
416- | ` Qwen-Image-GGUF ` | 18.2 | image |
417- | ` RealESRGAN-x4plus ` | 0.064 | upscaling, image |
418- | ` RealESRGAN-x4plus-anime ` | 0.017 | upscaling, image |
419- | ` SD-1.5 ` | 7.7 | image |
420- | ` SD-Turbo ` | 5.21 | image |
421- | ` SD-Turbo-GGUF ` | 2.02 | image |
422- | ` SDXL-Base-1.0 ` | 6.94 | image |
423- | ` SDXL-Turbo ` | 6.94 | image |
424- | ` Z-Image-Turbo ` | 20.7 | image |
425-
426- #### ` thenoise ` — TheNoise ROCm (experimental) (4 models)
427-
428- | Model | Size (GB) | Labels |
429- | -------| -----------| --------|
430- | ` Anima-Aesthetic ` | 5.4 | image |
431- | ` Anima-Base ` | 5.4 | image |
432- | ` Anima-Turbo ` | 5.4 | image |
433- | ` Krea-2-Turbo ` | 35 | image |
434-
435- #### ` thinksound ` — ThinkSound (1 models)
436-
437- | Model | Size (GB) | Labels |
438- | -------| -----------| --------|
439- | ` ThinkSound-SFX ` | 6.4 | audio-generation |
440-
441- #### ` trellis ` — TRELLIS.2 (1 models)
442-
443- | Model | Size (GB) | Labels |
444- | -------| -----------| --------|
445- | ` TRELLIS-3D ` | 15.4 | 3d |
446-
447- #### ` vllm ` — vLLM ROCm (experimental) (11 models)
448-
449- | Model | Size (GB) | Labels |
450- | -------| -----------| --------|
451- | ` GLM-4.7-Flash-FP16-vLLM ` | 62.47 | reasoning, tool-calling |
452- | ` Qwen3.5-0.8B-FP16-vLLM ` | 1.77 | reasoning |
453- | ` Qwen3.5-2B-FP16-vLLM ` | 4.57 | reasoning, tool-calling |
454- | ` Qwen3.5-4B-FP16-vLLM ` | 9.34 | reasoning, hot, tool-calling |
455- | ` Qwen3.5-9B-FP16-vLLM ` | 19.3 | reasoning, tool-calling |
456- | ` Qwen3.6-27B-FP16-vLLM ` | 55.59 | reasoning, tool-calling, vision |
457- | ` Qwen3.6-27B-FP8-vLLM-highconc ` | 27.8 | reasoning, tool-calling, vision, mtp |
458- | ` Qwen3.6-27B-FP8-vLLM-lowconc ` | 27.8 | reasoning, tool-calling, vision, mtp |
459- | ` Qwen3.6-35B-A3B-FP16-vLLM ` | 71.93 | reasoning, tool-calling, vision |
460- | ` Qwen3.6-35B-A3B-FP8-vLLM-highconc ` | 36.0 | reasoning, tool-calling, vision, mtp |
461- | ` Qwen3.6-35B-A3B-FP8-vLLM-lowconc ` | 36.0 | reasoning, tool-calling, vision, mtp |
462-
463- #### ` whispercpp ` — Whisper.cpp (6 models)
464-
465- | Model | Size (GB) | Labels |
466- | -------| -----------| --------|
467- | ` Whisper-Base ` | 0.148 | transcription, realtime-transcription |
468- | ` Whisper-Large-v3 ` | 3.1 | transcription, realtime-transcription |
469- | ` Whisper-Large-v3-Turbo ` | 1.62 | transcription, realtime-transcription, hot |
470- | ` Whisper-Medium ` | 1.53 | transcription, realtime-transcription |
471- | ` Whisper-Small ` | 0.488 | transcription, realtime-transcription |
472- | ` Whisper-Tiny ` | 0.075 | transcription, realtime-transcription |
473- <!-- END GENERATED: backend-models -->
474-
475185## Implementation notes
476186
477187### ACE-Step (` acestep ` )
0 commit comments