Skip to content

Add readability check#2555

Open
josediegorobles wants to merge 3 commits into
Giskard-AI:mainfrom
josediegorobles:codex/add-readability-check
Open

Add readability check#2555
josediegorobles wants to merge 3 commits into
Giskard-AI:mainfrom
josediegorobles:codex/add-readability-check

Conversation

@josediegorobles

Copy link
Copy Markdown

Closes #2349

Adds a built-in Readability check for model outputs. The check supports flesch_reading_ease, flesch_kincaid_grade, and gunning_fog through textstat, with optional min_score and max_score thresholds.

The score is included as a Metric, textstat is exposed through the nlp extra, and unit tests cover thresholds, JSONPath extraction, metric variants, serialization, and the missing dependency error.

Tests:

  • make check
  • make test-unit PACKAGE=giskard-checks

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new Readability check to evaluate NLP readability metrics (such as Flesch Reading Ease, Flesch-Kincaid Grade, and Gunning Fog) using the textstat library. It includes the necessary dependency updates, exports, and unit tests. The feedback suggests explicitly handling empty or whitespace-only strings to prevent undefined behavior in textstat, and broadening the exception handling in run from RuntimeError to Exception to catch any unexpected errors during score computation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread libs/giskard-checks/src/giskard/checks/builtin/nlp_metrics.py
Comment thread libs/giskard-checks/src/giskard/checks/builtin/nlp_metrics.py
@josediegorobles

Copy link
Copy Markdown
Author

Heads-up — I opened this before noticing #2412, which already adds a readability check and is further along (it's already been through review rounds). Apologies for the duplicate. I'm happy to close this in favor of #2412; if it's useful I can help review/test there instead. Let me know what you'd prefer.

@josediegorobles josediegorobles force-pushed the codex/add-readability-check branch from f1348e5 to c33d221 Compare June 24, 2026 05:19
@josediegorobles josediegorobles mentioned this pull request Jun 24, 2026
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

Add readability check

1 participant