Skip to content

fix(google): count Gemini thinking tokens in usage.output_tokens 馃馃馃馃 - #2653

Open
davidberenstein1957 wants to merge 3 commits into
mainfrom
chore/format-pr-2622
Open

fix(google): count Gemini thinking tokens in usage.output_tokens 馃馃馃馃#2653
davidberenstein1957 wants to merge 3 commits into
mainfrom
chore/format-pr-2622

Conversation

@davidberenstein1957

@davidberenstein1957 davidberenstein1957 commented Jul 29, 2026

Copy link
Copy Markdown
Member

Summary

Formatted follow-up of #2622 (@ebarkhordar).

Same change as #2622, plus ruff format/lint fixes so CI make check can get past format/import sorting.

Why

#2622 was blocked on CI lint (typically check-format on README markdown fences; for some PRs also ruff check import sorting).

Supersedes

Please close #2622 in favor of this PR once CI is green.

Test plan

ebarkhordar and others added 3 commits July 19, 2026 00:21
@ebarkhordar
fix(google): count Gemini thinking tokens in usage.output_tokens
1bd9772 
GoogleChatTranslator.from_google mapped output_tokens to
candidates_token_count only, but google-genai defines total_token_count as
prompt + candidates + tool_use_prompt + thoughts. When the model thinks,
thoughts_token_count is generated (and billed) output that the mapping
dropped, so input_tokens + output_tokens != total_tokens on the same Usage
object.

Fold thoughts_token_count into output_tokens and tool_use_prompt_token_count
into input_tokens so the sum matches total_tokens, per Google's own
definition. Add a regression test covering a thinking response.
@davidberenstein1957 @gemini-code-assist
Apply suggestion from @gemini-code-assist[bot]
faf4465 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@davidberenstein1957
style: reformat markdown for ruff 0.16 format check
7032af1 
CI installs latest ruff via uv tool install, which formats Python
fenced blocks in Markdown. Align docs with that so make check passes.
@davidberenstein1957 davidberenstein1957 mentioned this pull request Jul 29, 2026
Closed
6 tasks
@ebarkhordar

ebarkhordar commented Aug 3, 2026

Copy link
Copy Markdown

#2686 and #2674 both landed on main since this branch was cut, and between them they cover the formatting work here. Four of the five markdown files (libs/giskard-agents/README.md, libs/giskard-checks/README.md, libs/giskard-llm/README.md, libs/giskard-llm/docs/design.md) are now byte-identical between this branch and main, so they no longer carry a change. The conflict is README.md alone, which #2674 rewrote a few hours ago.

The two functional files, libs/giskard-llm/src/giskard/llm/translators/google_chat.py and libs/giskard-llm/tests/translators/test_google_chat_return.py, are untouched on main since this PR's base (8aafe58), so the token-counting change still applies as is. Checking out main's README.md onto the branch should be enough to clear the conflict.

@BILLKISHORE BILLKISHORE mentioned this pull request Aug 10, 2026
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kevinmessiaen kevinmessiaen Awaiting requested review from kevinmessiaen

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

safe for build Scope: Agents Scope: Checks Scope: LLM

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@davidberenstein1957 @ebarkhordar