Fix post-compact 400 on SGLang/vLLM: merge leading system messages - #860
Merged
mpfaffenberger merged 1 commit intoAug 24, 2026
Merged
Conversation
Fixes a 400 'System message must be at the beginning.' from SGLang /
vLLM-style endpoints when resuming after auto-compact. The wire format
ends up with two leading system messages -- the compaction summary
(SystemPromptPart produced by pydantic-ai-harness) plus the agent's
per-turn instruction_parts -- and strict backends accept only one.
pydantic-ai already ships _merge_leading_system_messages for exactly
this case, but it only fires when the model profile sets
openai_chat_supports_multiple_system_messages=False, which none of our
OpenAI-compatible model types did.
Add _strict_openai_profile() in model_factory.py:
- defaults openai_chat_supports_multiple_system_messages=False so
leading system messages are merged (joined with \n\n); harmless on
endpoints that accept multiples since content is identical
- merges any thinking_tags profile and an optional extra= profile
- per-model opt-out via "supports_multiple_system_messages": true
(JSON boolean; non-bool values raise TypeError rather than silently
inverting intent, since bool("false") is True)
Applied to custom_openai, openrouter, cerebras (via extra= alongside
its strict-tool-def override), and Zhipu zai_coding/zai_api. Routing
cerebras through the shared helper also picks up thinking_tags, which
the old bare profile skipped.
Tests: per-model-type profile flags, config override + fail-fast
guard, thinking-tags x extra x merge triple-merge, and an end-to-end
_map_messages wire-format test proving two leading system parts
collapse to exactly one merged system message. README documents the
default and the opt-out.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resuming a session after auto-compact on SGLang / vLLM-style OpenAI-compatible endpoints fails with:
Root cause
After
SummarizingCompactionruns, the compacted history opens with a summarySystemPromptPart(produced bypydantic-ai-harness). On the next request, pydantic-ai'sOpenAIChatModel._map_messagesalso inserts the agent's per-turninstruction_partsright after leading system messages — so the wire format carries two leadingsystemmessages. Strict backends accept only one.pydantic-ai already ships
_merge_leading_system_messagesfor exactly this case, but it only fires when the model profile setsopenai_chat_supports_multiple_system_messages=False— and none of our OpenAI-compatible model types did.Fix
New
_strict_openai_profile()inmodel_factory.py:openai_chat_supports_multiple_system_messages=False, so leading system messages are merged (joined with\n\n). Harmless for endpoints that accept multiples — content is identical, just in one message.thinking_tagsprofile and an optionalextra=profile (preserves the Cerebras strict-tool-def override)."supports_multiple_system_messages": true(JSON boolean). A non-bool value raisesTypeErrorinstead of silently inverting intent —bool("false")isTruein Python, so coercion can't save a string config.custom_openai,openrouter,cerebras,zai_coding, andzai_api. Routing Cerebras through the shared helper also picks upthinking_tags, which the old bare profile skipped.openai/azure_openaiare intentionally untouched: real OpenAI accepts multiple system messages, and merging would forfeit inter-system-message prompt-cache breakpoints.README documents the default behavior, the SGLang symptom, and the opt-out.
Testing
custom_openai(default + opt-out),openrouter,cerebras(both flags),zai_coding,zai_api"false"raisesTypeErrorinstruction_partspassed through_map_messagesyields exactly one merged system messageruff format+ruff checkclean