feat(checks): opt-in progress logging for scenario/check execution (#… - #2607
feat(checks): opt-in progress logging for scenario/check execution (#…#2607demoiverrakada wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces opt-in progress logging at the INFO level for the scenario runner, allowing callers to track step-by-step execution and check status. It also adds a comprehensive test suite to verify this behavior and ensure backward compatibility. The feedback suggests wrapping the logging calls and string formatting inside if logger.isEnabledFor(logging.INFO): checks to prevent unnecessary overhead when logging is disabled.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
@demoiverrakada, thank for the contribution! Would you be able to resolve the conflict? |
davidberenstein1957
left a comment
There was a problem hiding this comment.
Reviewed. Ready to merge pending CI.
…2421)
Description
Adds opt-in, per-step progress logging to the scenario runner so long
Suite.run()executions surface which scenario and checks are running — useful in CI, scripts, and
notebooks where the rich progress bar doesn't render.
Suite.run()already exposes averboseflag +_SuiteProgressbar for scenario-levelprogress; this closes the check-level gap, where
ScenarioRunner._run_oncepreviouslyemitted nothing.
Changes
scenarios/runner.py: module logger; INFO logs at step start (scenario name + stepindex + check identities) and completion (pass/fail).
check.namefalls back tocheck.kindwhen unset.tests/scenarios/test_runner_logging.py: covers logs-at-INFO and quiet-by-default.Design
logging.getLogger("giskard").setLevel(logging.INFO). No new API, backward-compatible(uses the Python-logging option suggested in the issue).
parallel=Truestays readable.Testing — full
giskard-checkssuite (754 passed), whole monorepo (1,268 passed,0 failed), serial +
parallel=Trueverified manually,make checkclean.Developed with AI assistance (human-in-the-loop; I reviewed and verified all changes).
Related Issue
Type of Change
Coding agents
Autonomous agents with no human in the loop must read AUTONOMOUS.md before opening a PR.
PR title: agent-opened PRs must end the title with
🤖🤖🤖🤖(exactly four robot emojis). Do not omit — that suffix is how the expedited agent PR workflow picks up the PR.Checklist
CODE_OF_CONDUCT.mddocument.CONTRIBUTING.mdguide.uv.lockrunninguv lock(only applicable whenpyproject.tomlhas beenmodified)