Skip to content

fix(mlxlm): guard generate_batch output_type with 'is not None' - #1999

Open
eeshsaxena wants to merge 1 commit into
dottxt-ai:mainfrom
eeshsaxena:fix/mlxlm-batch-output-type-guard
Open

fix(mlxlm): guard generate_batch output_type with 'is not None'#1999
eeshsaxena wants to merge 1 commit into
dottxt-ai:mainfrom
eeshsaxena:fix/mlxlm-batch-output-type-guard

Conversation

@eeshsaxena

Copy link
Copy Markdown

MLXLMModel.generate_batch guards against structured generation (which mlx-lm can't do with batching) using a truthy check:

if output_type:
    raise NotImplementedError(
        "mlx-lm does not support constrained generation with batching."
        ...
    )

Every other place in the codebase that inspects output_type uses an identity check (is None / is not None); this is the only if output_type: in the repo. The difference matters here: if an output_type is provided that happens to be falsy, for example an object whose __bool__ returns False, the guard is skipped and batch_generate runs with no constraint. The caller asked for structured output and silently gets unconstrained text instead of the NotImplementedError they should get.

Switching to is not None makes the guard fire for any provided output type and matches the convention used everywhere else.

The mlxlm test module is gated behind @pytest.mark.skipif(not HAS_MLX, ...) and needs a real model, so there is no CI-runnable spot to add a case; this is a one-line guard correction that can't change the None (default) or normal-object paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant