fix(gemini): exclude tool-execution tokens from output_tokens - #2406
Open
iluv7 wants to merge 2 commits into
Open
fix(gemini): exclude tool-execution tokens from output_tokens#2406iluv7 wants to merge 2 commits into
iluv7 wants to merge 2 commits into
Conversation
…cope-ai#2405) Gemini's total_token_count covers prompt, candidates, tool-use prompt, and thinking tokens. Deriving output_tokens as total - prompt counted the tool-execution results fed back to the model as output, inflating usage reported for server-side tools like Google Search grounding and budget accounting in ReplyBudgetControlMiddleware. Count only the tokens the model generated: candidates_token_count plus thoughts_token_count, falling back to total - prompt on older SDKs. Co-Authored-By: Claude <noreply@anthropic.com>
iluv7
changed the title
iluv7
marked this pull request as draft
August 24, 2026 02:17
Member
|
@iluv7 Is this pr ready for review? |
iluv7
marked this pull request as ready for review
August 24, 2026 08:08
Contributor
Author
Yep. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AgentScope Version
2.0.6
Description
Gemini reports prompt, candidate, server-side tool-use, and thinking tokens separately in
usage_metadata.GeminiChatModelpreviously derivedoutput_tokensastotal_token_count - prompt_token_count, which incorrectly classifiedtool_use_prompt_token_countas model output. These tokens come from server-side tool results (for example, Google Search grounding or code execution) that are fed back to the model as input.This caused incorrect input/output usage reporting and distorted weighted cost or budget accounting such as
ReplyBudgetControlMiddleware.Fixes #2405
Changes:
input_tokens = prompt_token_count + tool_use_prompt_token_countoutput_tokens = candidates_token_count + thoughts_token_countcandidates_token_countis unavailable, fall back tototal_token_count - input_tokens, so tool-use tokens are still excluded from outputtests/model_gemini_test.pyFor example, given
prompt=500,candidates=120,tool_use=300,thoughts=10, andtotal=930, the model now reports:The two values preserve the API-reported total while correctly classifying tool-use results as input rather than output.
How to test:
Checklist