Studio: add reasoning budget controls

[alkinun]

Summary

• add first-class reasoning_budget and reasoning_budget_message GGUF load settings
• expose both controls in Studio per-model advanced settings and persist them across reloads, model switches, presets, and API auto-switches
• preserve existing llama-server passthrough and LLAMA_ARG_THINK_BUDGET* environment behavior with exact effective-state reporting
• report effective values through load/status responses and retain them across llama-server respawns
• reject NUL and messages over 8 KiB UTF-8 before teardown while preserving verbatim whitespace in passthrough values
• probe custom/older llama-server binaries, omit unsupported/default flags, and reject explicitly configured unsupported settings during /validate before the resident model is unloaded
• parse-probe positive budgets so b6000-era binaries that advertise but reject them fail before teardown
• emit reset tombstones only for an explicit UI transition to defaults, removing matching legacy flags without touching hidden passthrough settings during unrelated saves

Closes #7524.

Test plan

• Reasoning/autoswitch suite (874 passed)
• Mike review regressions: four backend route tests and three source contracts (7 passed)
• Model-picker and deep-research source contracts (122 passed)
• Combined capability, env, validation, and test-isolation suite (608 passed)
• cd studio/frontend && npm run typecheck
• cd studio/frontend && npm test (303 passed)
• Python compile checks and git diff --check

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

[oobabooga]

The normal flow works, the settings are wired through persistence, and they survive llama-server respawn. I found two issues that should be fixed before merging:

1. reasoning_budget_message is limited by character count rather than its encoded size, and embedded NUL is also accepted. A UI-reachable value containing 32,768 emoji passes validation but fails process creation with E2BIG. An embedded NUL also passes request parsing and then fails in Popen. Process creation happens after the active llama-server has been stopped, so a direct load failure can leave no model running. Please validate this before teardown, reject NUL, and enforce a conservative encoded-size limit across direct loads, stored overrides, presets, auto-switching, and passthrough arguments.

2. Passthrough values are not reported exactly. For example, " PAD " is reported by /load and /status as "PAD", but the raw padded value is appended last to the llama-server command and is the value actually used. Since passthrough arguments are documented as verbatim, please preserve the raw value in effective state and use a trimmed copy only to reject empty input.

Please add regression coverage for multibyte overflow, embedded NUL, and whitespace in the last-wins passthrough value, including rejection before the active process is stopped. The existing focused checks passed: 570 backend tests and 161 frontend tests.

[oobabooga]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 02fc4ba967

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[alkinun]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f712f0c727

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[alkinun]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 120c9ad373

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[shimmyshimmer]

Setup

Fresh Studio pair in unsloth_studio_000054 (nothing reused):

	tree	Studio
pre-PR (main @ 06a0d60)	pre/repo	http://127.0.0.1:8901
PR #7701 (pr7701 @ 120c9ad)	pr/repo	http://127.0.0.1:8902

Both installed with their own UNSLOTH_STUDIO_HOME, both serving 200.

Before the PR

--reasoning-budget / --reasoning-budget-message are real llama.cpp server flags (confirmed in common/arg.cpp on master: -1 unrestricted, 0 immediate end, N>0 token budget; env LLAMA_ARG_THINK_BUDGET). The bundled llama-server in this install advertises both.

On main, Studio has no first-class control for either. Confirmed empirically: /openapi.json on 8901 has 0 occurrences of reasoning_budget; LoadRequest has no such field; nothing in studio/ references the flags. Your only option was dropping them into llama_extra_args passthrough, where they are unvalidated, invisible in the UI, not echoed by /status, not part of the reload-match comparison (so a change wouldn't trigger a respawn), and not preserved as a per-model setting. Studio only exposed a reasoning on/off toggle — exactly what issue #7524 says.

After the PR

• reasoning_budget + reasoning_budget_message on LoadRequest, ValidateModelRequest, ModelOverridePayload, chat presets; echoed back in /load and /status (10 occurrences in the 8902 OpenAPI schema, verified).
• Two inputs in the GGUF advanced settings pane, persisted per-model (localStorage schema v1→v2, forward-normalized so old blobs survive), replayed across reloads, model switches, presets and OpenAI auto-switch.
• Flags are only emitted if the binary's --help advertises them; explicit settings against a binary that doesn't (or can't be probed) are rejected during /validate, before the resident model is unloaded.
• --reasoning-budget-message is validated (no NUL, ≤8 KiB UTF-8, whitespace preserved verbatim).

The issue is real, and the PR solves it. Not a fake/cosmetic fix.

Merge risk

It does not add a hardware path — it's a launch-flag feature gated behind a capability probe, so the CPU/CUDA/Metal/Vulkan paths are untouched. But it is not safe to merge as-is: it breaks pre-existing tests.

7 tests that pass on main fail on the PR branch (same machine, same env, reproduced in isolation, not just in the full run — main: 3 unrelated env failures; PR: those 3 + 7 more):

4 backend route tests — real code-path change. The new preflight at inference.py:6164 calls backend.reasoning_budget_settings_requested(...) on every GGUF load, even when nothing is configured, and it runs ahead of the GPU/hub/training guards. Where those tests stub the backend, it raises and the load returns 500 "…object has no attribute 'reasoning_budget_settings_requested'" instead of the expected 400/409. Production uses the real class so it won't crash there, but the error-precedence reordering is genuine — a request with both a bad GPU pin and a reasoning setting now fails on the reasoning check first.

3 source-contract tests — formatting churn. The PR reformats large stretches of untouched frontend code (80-col rewrapping in chat-runtime-store.ts, use-chat-model-runtime.ts, apply-inference-status-to-store.ts). Contract tests assert exact source strings, so e.g. approvedRemoteCodeFingerprints.get(previousCheckpoint) ?? null, split across two lines fails test_a_pinned_cached_row_loads_from_the_id_the_backend_pinned. Same for the slot-baseline and deep-research contracts. Harmless at runtime, red CI. (PR CI was still pending when I checked.)

Four more things worth flagging, none of which break old pathways but all of which change behavior:

1. Every GGUF load on a capable binary now gets --reasoning-budget -1 appended, and LLAMA_ARG_THINK_BUDGET / _MESSAGE are unconditionally popped from the child env (llama_cpp.py:9386). -1 is llama.cpp's own default so the model behaves identically — but anyone who was setting those env vars loses them silently.
2. The probe checks flag presence, not accepted range. llama.cpp b6000-era builds have --reasoning-budget but throw invalid value for anything other than 0/-1 (verified against tag b6000). On such a build, setting 512 passes the gate, then llama-server fails to spawn — after teardown.
3. Fail-closed on an unprobeable binary: a custom llama-server whose --help doesn't exit 0 makes any configured reasoning setting a hard 400, whereas the existing MTP probe fails open. Also, setting a budget on a GGUF that isn't downloaded yet is a 400 telling you to load it once first.
4. Legacy passthrough gets stripped silently. The UI has no control for llama_extra_args, yet a normal per-model save now sends reasoning_budget: -1 as a reset tombstone, and settings.py strips a matching --reasoning-budget 512 out of your stored extras. Partially mitigated by /status seeding the field after a load, but someone who set it via CLI/API and never loaded first loses it.

The pre-existing capability detection itself is safe: I ran the real probe against the same bundled binary on both trees and every pre-existing key is byte-identical (mtp_token: draft-mtp, supports_mtp, ngram_mod_flavor, supports_metrics, …). The help-parsing rewrite (column-0 → minimum-indent declarations) is a no-op on real llama.cpp output, where flags sit at column 0 and descriptions at column 40.

Bottom line: real issue, correct approach, old pathways (non-GGUF, diffusion, plain GGUF loads, extras passthrough) are structurally preserved and gated — but 7 green-on-main tests go red, so it needs the preflight made defensive/late-bound and the formatting churn reverted before merging.

## Setup

Fresh Studio pair in unsloth_studio_000054 (nothing reused):

	tree	Studio
pre-PR (main @ 06a0d6004)	pre/repo	http://127.0.0.1:8901
PR #7701 (pr7701 @ 120c9ad37)	pr/repo	http://127.0.0.1:8902

Both installed with their own UNSLOTH_STUDIO_HOME, both serving 200.

Before the PR

--reasoning-budget / --reasoning-budget-message are real llama.cpp server flags (confirmed in common/arg.cpp on master: -1 unrestricted, 0 immediate end, N>0 token budget; env LLAMA_ARG_THINK_BUDGET). The bundled llama-server in this install advertises both.

On main, Studio has no first-class control for either. Confirmed empirically: /openapi.json on 8901 has 0 occurrences of reasoning_budget; LoadRequest has no such field; nothing in studio/ references the flags. Your only option was dropping them into llama_extra_args passthrough, where they are unvalidated, invisible in the UI, not echoed by /status, not part of the reload-match comparison (so a change wouldn't trigger a respawn), and not preserved as a per-model setting. Studio only exposed a reasoning on/off toggle — exactly what issue #7524 says.

After the PR

• reasoning_budget + reasoning_budget_message on LoadRequest, ValidateModelRequest, ModelOverridePayload, chat presets; echoed back in /load and /status (10 occurrences in the 8902 OpenAPI schema, verified).
• Two inputs in the GGUF advanced settings pane, persisted per-model (localStorage schema v1→v2, forward-normalized so old blobs survive), replayed across reloads, model switches, presets and OpenAI auto-switch.
• Flags are only emitted if the binary's --help advertises them; explicit settings against a binary that doesn't (or can't be probed) are rejected during /validate, before the resident model is unloaded.
• --reasoning-budget-message is validated (no NUL, ≤8 KiB UTF-8, whitespace preserved verbatim).

The issue is real, and the PR solves it. Not a fake/cosmetic fix.

Merge risk

It does not add a hardware path — it's a launch-flag feature gated behind a capability probe, so the CPU/CUDA/Metal/Vulkan paths are untouched. But it is not safe to merge as-is: it breaks pre-existing tests.

7 tests that pass on main fail on the PR branch (same machine, same env, reproduced in isolation, not just in the full run — main: 3 unrelated env failures; PR: those 3 + 7 more):

4 backend route tests — real code-path change. The new preflight at [inference.py:6164](unsloth_studio_000054/pr/repo/studio/backend/routes/inference.py:6164) calls backend.reasoning_budget_settings_requested(...) on every GGUF load, even when nothing is configured, and it runs ahead of the GPU/hub/training guards. Where those tests stub the backend, it raises and the load returns 500 "…object has no attribute 'reasoning_budget_settings_requested'" instead of the expected 400/409. Production uses the real class so it won't crash there, but the error-precedence reordering is genuine — a request with both a bad GPU pin and a reasoning setting now fails on the reasoning check first.

3 source-contract tests — formatting churn. The PR reformats large stretches of untouched frontend code (80-col rewrapping in chat-runtime-store.ts, use-chat-model-runtime.ts, apply-inference-status-to-store.ts). Contract tests assert exact source strings, so e.g. approvedRemoteCodeFingerprints.get(previousCheckpoint) ?? null, split across two lines fails test_a_pinned_cached_row_loads_from_the_id_the_backend_pinned. Same for the slot-baseline and deep-research contracts. Harmless at runtime, red CI. (PR CI was still pending when I checked.)

Four more things worth flagging, none of which break old pathways but all of which change behavior:

1. Every GGUF load on a capable binary now gets --reasoning-budget -1 appended, and LLAMA_ARG_THINK_BUDGET / _MESSAGE are unconditionally popped from the child env ([llama_cpp.py:9386](unsloth_studio_000054/pr/repo/studio/backend/core/inference/llama_cpp.py:9386)). -1 is llama.cpp's own default so the model behaves identically — but anyone who was setting those env vars loses them silently.
2. The probe checks flag presence, not accepted range. llama.cpp b6000-era builds have --reasoning-budget but throw invalid value for anything other than 0/-1 (verified against tag b6000). On such a build, setting 512 passes the gate, then llama-server fails to spawn — after teardown.
3. Fail-closed on an unprobeable binary: a custom llama-server whose --help doesn't exit 0 makes any configured reasoning setting a hard 400, whereas the existing MTP probe fails open. Also, setting a budget on a GGUF that isn't downloaded yet is a 400 telling you to load it once first.
4. Legacy passthrough gets stripped silently. The UI has no control for llama_extra_args, yet a normal per-model save now sends reasoning_budget: -1 as a reset tombstone, and [settings.py](unsloth_studio_000054/pr/repo/studio/backend/routes/settings.py) strips a matching --reasoning-budget 512 out of your stored extras. Partially mitigated by /status seeding the field after a load, but someone who set it via CLI/API and never loaded first loses it.

The pre-existing capability detection itself is safe: I ran the real probe against the same bundled binary on both trees and every pre-existing key is byte-identical (mtp_token: draft-mtp, supports_mtp, ngram_mod_flavor, supports_metrics, …). The help-parsing rewrite (column-0 → minimum-indent declarations) is a no-op on real llama.cpp output, where flags sit at column 0 and descriptions at column 40.

Bottom line: real issue, correct approach, old pathways (non-GGUF, diffusion, plain GGUF loads, extras passthrough) are structurally preserved and gated — but 7 green-on-main tests go red, so it needs the preflight made defensive/late-bound and the formatting churn reverted before merging.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f28a4eac9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Track reasoning fields in preset dirty state

With reasoning budget/message now captured in capturePresetLoadConfig(), the Preset section’s hasUnsavedPresetChanges and currentLoadSummary memos in chat-settings-sheet.tsx still only subscribe/depend on the older load settings through nParallel. If the preset sheet is already open and the user changes only Reasoning Budget or Reasoning Budget Message, the sheet does not re-render/recompute, so the active summary and Update button stay stale until some unrelated setting changes; add these runtime fields to the sheet selectors/dependency lists wherever capturePresetLoadConfig() is memoized.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Honor explicit reasoning resets over environment

When the Studio process has LLAMA_ARG_THINK_BUDGET or LLAMA_ARG_THINK_BUDGET_MESSAGE set, resetting the first-class controls to -1/empty still emits no CLI override here, so the child llama-server inherits the environment and the subsequent effective-state resolver reports the old budget/message again. This only affects users with those environment defaults configured, but it makes the new reset controls unable to clear them; distinguish omitted defaults from explicit resets and either emit supported reset flags or scrub the inherited env for that load.

Useful? React with 👍 / 👎.

[shimmyshimmer]

Re-ran my review against the current head (4f28a4eac) rather than the commits the earlier rounds were written on, in a fresh Studio pair. Most of the open feedback is already fixed, two things were still live, and I have pushed both to this branch as b53774ae7.

Already fixed since the earlier reviews

• Encoded-size and NUL validation on reasoning_budget_message: validate_reasoning_budget_message is wired into all six persistence boundaries (LoadRequest/ValidateModelRequest, chat presets, model overrides, OpenAI auto-switch, and the extras parser), so an emoji-heavy or NUL-bearing value is rejected before teardown.
• Verbatim passthrough reporting: preserve_raw = True keeps " PAD " intact in effective state, with .strip() used only to reject empty input.
• Unconditional --reasoning-budget -1: gone. The flag is only emitted when the value is non-default and the probe confirms support.
• /validate now carries both fields and runs the same capability gate before the resident model is unloaded.
• Reset tombstones are gated on an actual default transition, so an unrelated per-model save no longer strips stored llama_extra_args.
• Positive-budget range probe is in, so a b6000-era binary fails before teardown.

My earlier "7 red tests" note is stale

I ran the full backend suite on this branch and on main in the same environment. The failure lists are byte-identical: 49 pre-existing failures, all macOS/ROCm install and API-key cases unrelated to this PR. tests/studio is clean apart from the same install failures. The route preflight now sits after the training, hub and sidecar guards, and the frontend formatting churn was reverted. That objection is resolved.

Two that were still live

1. An inherited LLAMA_ARG_THINK_BUDGET counted as a user request.

_validate_reasoning_budget_preflight env-resolved before testing whether anything was configured, while LlamaCppBackend's own load-time gate did not. With only the env var set and nothing set in Studio, I reproduced:

• /validate returning 400 for any GGUF whose header cannot be read yet (the tri-state None case, so every not-yet-downloaded repo)
• /validate returning 400 for every DiffusionGemma
• validate_reasoning_budget_capabilities rejecting the load on any binary whose --help cannot be probed

All three load fine on main. The error tells the user to clear the Reasoning Budget settings, but the UI already shows them at their defaults, so there is no way out of it. Meanwhile reasoning_budget_settings_requested(extra_args=None, reasoning_budget=-1, reasoning_budget_message="") correctly returns (False, False), so the route and the backend disagreed about the same inputs.

Both now gate on explicit configuration only: the first-class fields plus passthrough extras. The environment stays a reporting input, so effective state still reports the inherited value exactly and llama-server keeps validating its own env. This is the same fail-open posture as the existing MTP probe.

I updated test_reasoning_budget_capability_gate_rejects_legacy_positive_range, which asserted the old env behaviour, so it now covers env-fails-open while keeping the passthrough-flag case gating.

2. The preset dirty state does not react to a reasoning budget change.

capturePresetLoadConfig() captures reasoningBudget and reasoningBudgetMessage through getState(), but chat-settings-sheet.tsx neither subscribed to them nor listed them in the hasUnsavedPresetChanges or currentLoadSummary dependency arrays. With the preset sheet open, changing only the reasoning budget left the Update button and the current-settings summary stale until some unrelated setting happened to change. Both are now subscribed and listed.

Verification

Each fix has a regression test that I confirmed fails without the fix and passes with it.

• Backend suite failure list identical to the main baseline (no new failures)
• tests/studio: only the pre-existing macOS install failures
• Reasoning, autoswitch, args, reload-inheritance and validation suites: 866 passed
• npm run typecheck clean, npm test 303 passed
• Frontend rebuilt, Studio serves 200, 10 reasoning_budget occurrences in the OpenAPI schema
• git diff --check clean, Python compile checks clean

One thing I left alone

Resetting the control to -1 still cannot clear an LLAMA_ARG_THINK_BUDGET default, since -1 means "no override" and there is no way to express an explicit unrestricted value without a tri-state. Effective state reports the inherited value exactly, and changing this would undo the env preservation this PR promises. Worth a follow-up, not a blocker.

Looks good to me now.

[shimmyshimmer]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b53774ae78

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Include the budget message in preset summaries

When a preset sets only reasoningBudgetMessage while leaving the numeric budget at -1, hasPresetLoadConfig() correctly treats it as a non-default load configuration, but this formatter adds no summary part and returns null. Consequently, chat-settings-sheet.tsx suppresses both the “Active now” and “Saved in preset” summaries even though applying the preset changes llama-server behavior; include a summary marker for a non-empty message.

Useful? React with 👍 / 👎.

[shimmyshimmer]

Went through the latest review round. Only one of the five inline comments is new against b53774ae7; the other four are the earlier ones re-anchored to the new head. That one is correct and I have pushed the fix as 18de51401.

The new one: budget message missing from the preset summary

preset-load-config.ts line 278, "Include the budget message in preset summaries". Confirmed:

• perModelConfigsEqual compares reasoningBudgetMessage (apply-per-model-config.ts:139), so hasPresetLoadConfig({...EMPTY, reasoningBudgetMessage: "..."}) is true. The preset is saved, applied, and does change llama-server behaviour.
• formatPresetLoadConfigSummary had a part for reasoningBudget but none for the message, so with only the message set parts stays empty and the function returns null.
• chat-settings-sheet.tsx:918-930 renders both "Active now" and "Saved in preset" only when the summary is non-null, so the Preset section showed no load settings at all for that preset.

Fixed by pushing a Budget msg marker, not the text itself, since the message is free prose up to 8 KiB and the other parts are all short.

I checked main while I was in there: maxSeqLength and nCpuMoe have exactly the same gap, both can make a config non-default with no summary part. Both predate this branch, so I left them alone rather than widen this PR. Worth a separate issue.

The other four

Re-verified each against the current head rather than the commit it was written on.

• "Validate reasoning controls before unloading the resident model": fixed. validateModel sends reasoning_budget and reasoning_budget_message (chat-api.ts:245-246), the reload path fills both before the unload (use-chat-model-runtime.ts:954), and /validate runs the same capability gate.
• "Clear passthrough flags when resetting reasoning controls": fixed. resetReasoningBudget and resetReasoningBudgetMessage markers exist on PutModelOverrideOptions and are gated on an actual default transition, so an unrelated save no longer strips stored extras.
• "Track reasoning fields in preset dirty state": fixed in b53774ae7. The sheet subscribes to both and lists both in the hasUnsavedPresetChanges and currentLoadSummary dependency arrays.
• "Honor explicit reasoning resets over environment": still open, still deliberate. -1 means "no override", so there is no way to express an explicit unrestricted value without a tri-state, and forcing one would undo the env preservation this PR promises. Effective state reports the inherited value exactly. Follow-up, not a blocker.

Verification

The new test fails without the fix and passes with it.

• Backend suite failure list identical to the main baseline (49 pre-existing macOS/ROCm/API-key failures, no new ones)
• tests/studio: only those same pre-existing install failures
• npm run typecheck clean, npm test 303 passed
• biome check on the touched file: 3 findings, all pre-existing in normalizePresetLoadConfig, capturePresetLoadConfig and coalesceDefaultLoadKnobs, byte-identical before and after my change
• Frontend rebuilt, Studio serves 200 with a clean log
• git diff --check clean

Still good from my side.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 18de51401e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep diffusion models in GGUF context capture

When a DiffusionGemma model is using automatic context sizing (snapshot.customContextLength is null), this new condition classifies it as non-GGUF even though store.ggufContextLength contains its effective context. Saving a preset therefore records no customContextLength, whereas the previous code captured the resolved GGUF context, so applying that preset later can load with a different context and change memory use or experiment behavior. Keep the GGUF test independent of diffusion and suppress only the unsupported reasoning fields for diffusion models.

Useful? React with 👍 / 👎.

[shimmyshimmer]

One new inline comment since 18de51401, and it is correct. Fix pushed as 6ad56463b.

Diffusion GGUFs were dropped from the preset context capture

preset-load-config.ts line 173, "Keep diffusion models in GGUF context capture".

capturePresetLoadConfig folded !store.loadedIsDiffusion into isGguf, but isGguf gates two separate things: whether the reasoning fields capture, and whether effectiveContextLength falls back to store.ggufContextLength. Only the first wanted the diffusion check.

Verified the premise rather than taking it at face value:

• A loaded DiffusionGemma reports is_gguf = True and is_diffusion = llama_backend.is_diffusion from the same /status branch (routes/inference.py:7946,7950).
• apply-inference-status-to-store.ts:186 sets ggufContextLength from status.context_length whenever is_gguf, and line 360 sets loadedIsDiffusion from is_diffusion, so both are populated together for a diffusion GGUF.
• On main the same expression is a plain GGUF test with no diffusion term, so it captured the resolved context.

So with automatic sizing (snapshot.customContextLength null) the branch captured null where main captured the model's resolved context. That context is usually the only non-default knob in that situation, so coalesceDefaultLoadKnobs left the config empty, hasPresetLoadConfig went false, and capturePresetLoadConfig returned undefined. The preset then stored no loadConfig at all, not merely a different context.

isGguf is back to the plain GGUF test, with a separate capturesReasoning = isGguf && !store.loadedIsDiffusion suppressing only the reasoning fields. I also updated test_preset_load_config_carries_reasoning_budget, which pinned the old expression.

Verification

The new test fails without the fix and passes with it.

• Backend suite failure list identical to the main baseline (49 pre-existing macOS/ROCm/API-key failures, no new ones)
• tests/studio: 24 failures, all the same pre-existing install ones, 0 others
• npm run typecheck clean, npm test 303 passed
• biome check on the touched file: same three rules as before the change, line-number shift only, all pre-existing in coalesceDefaultLoadKnobs and the normalizers
• Frontend rebuilt, Studio serves 200 with a clean log
• git diff --check clean

For the record on the four older inline comments still anchored to the head: three are fixed and re-verified against the current source, and "Honor explicit reasoning resets over environment" stays open by choice, since -1 means "no override" and there is no way to express an explicit unrestricted value without a tri-state.

[shimmyshimmer]

No new review comments since 6ad56463b. Two things I did find while checking, one pushed and one that needs you.

Pushed: 97c4cfbef, ruff-format on the tests I added

Two call sites in the tests from my earlier commits fit on one line under the repo's ruff-format-with-kwargs hook. Committed the join so pre-commit.ci does not have to push an autofix. Cosmetic only, both files still pass.

For the record, biome is not in .pre-commit-config.yaml, and chat-settings-sheet.tsx was already not biome-format clean at 4f28a4eac, before any of my commits, so I left that alone.

Needs you: the branch now conflicts with main

mergeable: CONFLICTING, mergeStateStatus: DIRTY, and the pre-commit.ci - pr failure is just "error during mergeable check" falling out of that, not a hook failure.

Main has moved 7 commits past the merge base at 3044401c2, and two of them land on the same files this branch does:

• 957b99429 Studio: simplify the GGUF loading backend (Studio: simplify the GGUF loading backend #7663)
• 2419ff199 Support audio input in Studio chat on Apple Silicon (Support audio input in Studio chat on Apple Silicon #7699)

Four files conflict: core/inference/llama_cpp.py (4 hunks), routes/inference.py (6), models/inference.py (1), tests/test_gguf_reload_inheritance.py (1).

I checked whether my commits caused any of it. They did not. git merge-tree against origin/main produces the identical four conflicts from 4f28a4eac, the branch state before I pushed anything.

I have not resolved it, deliberately. These are not textual collisions. #7663 rewrote the GGUF loading path (+666/-1149 across those four files), models/inference.py shows main deleting the whole status-field block this branch edits, and test_gguf_reload_inheritance.py now imports GgufLoadIntent where the branch still imports InferenceStatusResponse/LoadRequest/LoadResponse. Re-plumbing the reasoning-budget feature onto the new loading structure is a design call on your feature, not a merge I should make for you.

Everything else still checks out at this head: backend suite failure list identical to the main baseline, tests/studio clean apart from the pre-existing macOS install failures, npm run typecheck clean, npm test 303 passed, ruff check clean on every file I touched.

Happy to take the merge if you would rather I did, just say so.