Skip to content

fix(models): validate multimodal text prompts consistently - #1994

Open
weivwang wants to merge 1 commit into
dottxt-ai:mainfrom
weivwang:fix/validate-multimodal-input
Open

fix(models): validate multimodal text prompts consistently#1994
weivwang wants to merge 1 commit into
dottxt-ai:mainfrom
weivwang:fix/validate-multimodal-input

Conversation

@weivwang

Copy link
Copy Markdown

Summary

  • validate the shared [text, *assets] multimodal input contract before provider-specific formatting
  • return actionable ValueErrors for empty lists and non-string first items instead of leaking IndexError or passing invalid prompts to SDKs
  • apply the same boundary to OpenAI, Anthropic, Gemini, Ollama, Mistral, LM Studio, and Transformers adapters
  • add focused helper and OpenAI entry-point regressions, while retaining Mistral's existing behavior through the shared helper

Root cause

Most model adapters indexed content[0] directly and validated only the remaining assets. An empty list therefore crashed with IndexError; a list starting with an image or another object passed that value through as the provider's text prompt. Mistral already performed both validations independently, so the rules also drifted between adapters.

This change centralizes the documented multimodal input boundary in split_multimodal_input, keeping provider-specific asset conversion separate.

Validation

  • locked project environment: 48 adapter/unit tests passed
  • core environment: 36 adapter/unit tests passed
  • Ruff 0.9.1 on all changed files
  • mypy (--allow-redefinition) on all changed source files
  • git diff --check

The pre-commit wrapper could not finish cloning its hook environments over the local Git transport; its Ruff and mypy commands were run directly at the repository-pinned versions instead.

Checklist

  • The title describes the change
  • This PR contains one logical bug fix
  • Naming and NumPy-style docstrings follow the project conventions
  • Tests cover the changed behavior
  • No documentation change is needed; the accepted input contract is unchanged

@weivwang
weivwang marked this pull request as ready for review August 13, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant