Skip to content

Commit b5ecb1b

Browse files
committed
fix(multimodal): collect all surrounding text as context instead of only the nearest segment
Previously, `get_image_context` only picked the single text item immediately adjacent to the image (via `break`). When users placed the image at the end of the `response` list (e.g. after a task description + source + target), `context_above` would contain only the last string (often a single word like "Off" or "Custom") while the task description was silently dropped. This caused `ImageCoherenceGrader` and `ImageHelpfulnessGrader` to receive near-empty context, resulting in unstable and severely low scores (1–2) for the same content that scored 5 when the image was placed first. The fix concatenates all text segments before/after the image, ensuring task instructions and surrounding content are always fully passed to the model regardless of image position. Made-with: Cursor
1 parent 0b5669d commit b5ecb1b

1 file changed

Lines changed: 19 additions & 18 deletions

File tree

openjudge/graders/multimodal/_internal/context_utils.py

Lines changed: 19 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -58,23 +58,24 @@ def get_image_context(
5858
... max_context_size=500
5959
... )
6060
"""
61-
context_above = None
62-
context_below = None
63-
64-
# Find context above (last text before image)
65-
for i in range(image_index - 1, -1, -1):
66-
if isinstance(content_list[i], str):
67-
context_above = content_list[i]
68-
if max_context_size and len(context_above) > max_context_size:
69-
context_above = context_above[-max_context_size:]
70-
break
71-
72-
# Find context below (first text after image)
73-
for i in range(image_index + 1, len(content_list)):
74-
if isinstance(content_list[i], str):
75-
context_below = content_list[i]
76-
if max_context_size and len(context_below) > max_context_size:
77-
context_below = context_below[:max_context_size]
78-
break
61+
# Collect all text segments above the image (in order)
62+
above_parts = [
63+
content_list[i]
64+
for i in range(image_index)
65+
if isinstance(content_list[i], str)
66+
]
67+
context_above = "\n".join(above_parts) if above_parts else None
68+
if context_above and max_context_size and len(context_above) > max_context_size:
69+
context_above = context_above[-max_context_size:]
70+
71+
# Collect all text segments below the image (in order)
72+
below_parts = [
73+
content_list[i]
74+
for i in range(image_index + 1, len(content_list))
75+
if isinstance(content_list[i], str)
76+
]
77+
context_below = "\n".join(below_parts) if below_parts else None
78+
if context_below and max_context_size and len(context_below) > max_context_size:
79+
context_below = context_below[:max_context_size]
7980

8081
return context_above, context_below

0 commit comments

Comments
 (0)