[pull] main from danny-avila:main#126
Merged
Merged
Conversation
* fix: fail fast on tool calls truncated by the output token limit When a model turn is cut off by the max output token limit while it is still streaming a tool call, the arguments are necessarily incomplete (the tool-use block is the last thing the model emits). Providers surface this differently: Bedrock Converse buries it in response_metadata.messageStop.stopReason, Anthropic uses stop_reason, OpenAI finish_reason='length', Google finishReason='MAX_TOKENS'. The partial args parse into a well-formed-but-empty tool call (e.g. create_file missing its content), so the agent loops on a malformed request until it hits the recursion limit. Detect the truncation stop reason across providers and, when the turn carries a tool call, throw an actionable OutputTruncationError from the single invoke funnel instead of executing/looping on the incomplete call. Adds unit coverage for the cross-provider detector and a gated live Bedrock reproduction (RUN_BEDROCK_TRUNCATION_LIVE=1). * fix: skip truncation guard for atomic-tool-call providers (Google/Vertex) Google/Vertex (GenAI) deliver function calls as complete objects sealed on arrival, not streamed argument deltas, and treat MAX_TOKENS as a final chunk. A Gemini response with a valid functionCall ending in finishReason MAX_TOKENS therefore has complete args, so the provider-blind guard wrongly threw on it. Make assertNotTruncatedToolCall provider-aware: skip providers that deliver tool calls atomically. Streaming-arg providers (Anthropic, Bedrock, OpenAI) still fail fast on truncated tool calls. * fix: detect truncation for Anthropic streaming and OpenAI Responses Two provider shapes the detector missed: - Anthropic streaming stores the terminal stop_reason on AIMessageChunk.additional_kwargs (response_metadata only gets model_provider/context_management), so a streamed Claude tool call cut off at max_tokens slipped past the guard. - OpenAI Responses API signals truncation via response_metadata.status 'incomplete' + incomplete_details.reason 'max_output_tokens', not a stop/finish field. Read additional_kwargs.stop_reason and incomplete_details.reason in getTruncationStopReason, and normalize max_output_tokens. Non-token incomplete reasons (e.g. content_filter) still return null.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )