Add readability check#2555
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new Readability check to evaluate NLP readability metrics (such as Flesch Reading Ease, Flesch-Kincaid Grade, and Gunning Fog) using the textstat library. It includes the necessary dependency updates, exports, and unit tests. The feedback suggests explicitly handling empty or whitespace-only strings to prevent undefined behavior in textstat, and broadening the exception handling in run from RuntimeError to Exception to catch any unexpected errors during score computation.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
Heads-up — I opened this before noticing #2412, which already adds a readability check and is further along (it's already been through review rounds). Apologies for the duplicate. I'm happy to close this in favor of #2412; if it's useful I can help review/test there instead. Let me know what you'd prefer. |
f1348e5 to
c33d221
Compare
Closes #2349
Adds a built-in
Readabilitycheck for model outputs. The check supportsflesch_reading_ease,flesch_kincaid_grade, andgunning_fogthroughtextstat, with optionalmin_scoreandmax_scorethresholds.The score is included as a
Metric,textstatis exposed through thenlpextra, and unit tests cover thresholds, JSONPath extraction, metric variants, serialization, and the missing dependency error.Tests:
make checkmake test-unit PACKAGE=giskard-checks