Skip to content

fix(google): map content blocks to refusal - #2715

Open
BILLKISHORE wants to merge 1 commit into
Giskard-AI:mainfrom
BILLKISHORE:fix/google-image-safety-recitation-refusals
Open

fix(google): map content blocks to refusal#2715
BILLKISHORE wants to merge 1 commit into
Giskard-AI:mainfrom
BILLKISHORE:fix/google-image-safety-recitation-refusals

Conversation

@BILLKISHORE

Copy link
Copy Markdown
Contributor

What

FINISH_REASON_MAP in translators/google_chat.py covers 7 of the 17 finish reasons the google-genai SDK declares. Everything else falls through to the "stop" default, including three cases where Gemini blocked the content:

SAFETY                    -> refusal
PROHIBITED_CONTENT        -> refusal
IMAGE_PROHIBITED_CONTENT  -> refusal
IMAGE_SAFETY              -> stop      <-- blocked, reported as a normal completion
RECITATION                -> stop      <-- blocked
IMAGE_RECITATION          -> stop      <-- blocked

IMAGE_PROHIBITED_CONTENT is in REFUSAL_REASONS but its sibling IMAGE_SAFETY is not, which looks like an oversight rather than a decision. REFUSAL_REASONS hasn't changed since #2329 introduced it.

This matters for scan results: when Gemini blocks a generation, the choice comes back as finish_reason="stop" with no refusal set, so a scenario reads as though the agent answered normally. #2595 fixed the enum lookup that collapsed every non-STOP reason to "stop", but the map itself stayed incomplete.

RECITATION is the one I'd flag most: it fires when output reproduces memorised or copyrighted material, and AdversarialScenarioGenerator ships a "Copyright Violations" category. Right now that block is invisible to the scan.

Fix

Add the three content-block reasons to REFUSAL_REASONS:

         "IMAGE_PROHIBITED_CONTENT",
+        "IMAGE_SAFETY",
+        "RECITATION",
+        "IMAGE_RECITATION",

Three lines. They flow into FINISH_REASON_MAP through the existing comprehension, and the refusal text path already handles them.

What I left alone

Seven reasons are still unmapped and I did not touch them, because none is clearly a refusal and there is no obvious target value:

LANGUAGE, OTHER, MALFORMED_FUNCTION_CALL, UNEXPECTED_TOOL_CALL, NO_IMAGE, IMAGE_OTHER, FINISH_REASON_UNSPECIFIED

MALFORMED_FUNCTION_CALL and UNEXPECTED_TOOL_CALL in particular look like they deserve something other than "stop", but that seemed like your call rather than mine. Happy to follow up if you want a direction.

Testing

Added test_from_google_content_blocks_map_to_refusal, parametrised over the three reasons, next to the existing SAFETY test. All three fail before the change and pass after. Nothing covered the map previously.

libs/giskard-llm/tests/translators/test_google_chat_return.py: 13 passed. make check exits 0. make test-unit: 41 / 218 / 129 / 805 / 168 passed across core, llm, agents, checks and scan.

xref: the one open PR touching this file (#2653) changes thinking-token usage only, not the finish-reason map.

Type of Change

  • 馃敡 Bug fix (non-breaking change which fixes an issue)

Disclosure: drafted with AI assistance (Claude Code). I diffed the map against the SDK enum, confirmed the tests fail before the fix and pass after, and ran the checks myself before submitting.

IMAGE_SAFETY, RECITATION and IMAGE_RECITATION were missing from
REFUSAL_REASONS, so a blocked generation fell through to the default and was
reported as a normal completion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

1 participant