Skip to content

fix(google): map Gemini finish_reason via enum value, not str()#2595

Merged
kevinmessiaen merged 2 commits into
Giskard-AI:mainfrom
WatchTree-19:fix-gemini-finish-reason-enum
Jul 8, 2026
Merged

fix(google): map Gemini finish_reason via enum value, not str()#2595
kevinmessiaen merged 2 commits into
Giskard-AI:mainfrom
WatchTree-19:fix-gemini-finish-reason-enum

Conversation

@WatchTree-19

Copy link
Copy Markdown
Contributor

problem

in the Gemini -> giskard translator (llm/translators/google_chat.py), the finish reason is looked up with str(candidate.finish_reason):

finish_reason = FINISH_REASON_MAP.get(str(candidate.finish_reason), "stop")

but candidate.finish_reason is a google.genai FinishReason enum, and str(FinishReason.MAX_TOKENS) is "FinishReason.MAX_TOKENS", not "MAX_TOKENS". FINISH_REASON_MAP is keyed on the bare names ("MAX_TOKENS", "SAFETY", ...), so every non-STOP reason misses the map and falls back to "stop". STOP only "works" by accident because the fallback is also "stop".

net effect: Gemini truncations are reported as normal completions (length lost) and, more importantly for a safety tool, SAFETY blocks / refusals are never surfaced as refusals.

repro (real google-genai SDK)

from google.genai.types import FinishReason
str(FinishReason.MAX_TOKENS)          # 'FinishReason.MAX_TOKENS'  (not 'MAX_TOKENS')
FINISH_REASON_MAP.get(str(FinishReason.MAX_TOKENS), "stop")  # 'stop'  -> should be 'length'

fix

look up via the enum's .value (getattr(finish_reason, "value", finish_reason), robust to both the real enum and the plain-string form used in dict-validated inputs). also fixed the refusal_out line so refusal text is a clean reason string rather than "FinishReason.SAFETY".

test

added test_from_google_max_tokens_maps_to_length and test_from_google_safety_maps_to_refusal mirroring the file's existing tests. before: 2 failed (assert 'stop' == 'length', assert 'stop' == 'refusal'); after: pass. the existing translator tests only covered STOP, which is indistinguishable from the fallback, so the bug went unnoticed. ruff clean.

found by reading the translator.

Signed-off-by: WatchTree-19 <119982314+WatchTree-19@users.noreply.github.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes an issue in the Google Chat translator where the candidate.finish_reason enum was not being correctly mapped because its string representation included the enum class name (e.g., FinishReason.SAFETY). The fix extracts the underlying string value using getattr before performing the lookup. Additionally, regression tests have been added to verify that finish reasons like MAX_TOKENS and SAFETY map correctly. I have no further feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@kevinmessiaen kevinmessiaen self-requested a review July 8, 2026 04:23
…message

Follow-up on the finish_reason enum fix from code review:

- Extract the finish reason via an explicit ``FinishReason.value`` ternary
  instead of ``str(getattr(candidate.finish_reason, "value", ...))``. The SDK
  is an optional dependency imported by the caller, so ``FinishReason`` is
  resolved with a lazy local import (never at module load). This drops the
  redundant ``str()`` on the enum path while staying type-clean.
- Remove the dead ``"SAFETY": "content_filter"`` map entry: it is
  unconditionally overridden to ``"refusal"`` by the REFUSAL_REASONS merge.
- Add a regression test covering the ``finish_message`` branch of the refusal
  text (previously only the reason-code fallback was exercised).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E5chPh9nQey5A8ztMeqjQf

@kevinmessiaen kevinmessiaen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @WatchTree-19

@kevinmessiaen kevinmessiaen merged commit 10b426a into Giskard-AI:main Jul 8, 2026
29 of 50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants