Skip to content

fix: forward falsy logits processor in mlxlm format_output_type - #1995

Open
serhiizghama wants to merge 2 commits into
dottxt-ai:mainfrom
serhiizghama:fix/mlxlm-format-output-type-identity-check
Open

fix: forward falsy logits processor in mlxlm format_output_type#1995
serhiizghama wants to merge 2 commits into
dottxt-ai:mainfrom
serhiizghama:fix/mlxlm-format-output-type-identity-check

Conversation

@serhiizghama

Copy link
Copy Markdown

Closes #1980. MLXLMTypeAdapter.format_output_type used if not output_type to decide whether a logits processor was supplied. That's a truthiness check, so any processor whose __bool__ (or __len__) evaluates to False gets silently dropped and generation runs unconstrained — no error, just wrong output. Only None should mean "no constraint".

Switched to the identity check if output_type is not None, matching transformers.py, llamacpp.py and the other backends.

Added two tests that don't need Apple Silicon (they build the adapter with a mock tokenizer): one for the None case and one for a falsy processor. The falsy-processor test fails on the old code and passes on the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mlxlm: MLXLMTypeAdapter.format_output_type uses falsy check instead of identity-None check

1 participant