feat(checks): add pydantic-settings for default model and config

[kevinmessiaen]

Summary

Adds pydantic-settings to giskard-checks so library configuration can be driven by environment variables (or a .env file), with set_default_generator() retained as a runtime override.

Settings

New GiskardChecksSettings class (GISKARD_CHECKS_* prefix):

Env var	Purpose	Default
GISKARD_CHECKS_DEFAULT_MODEL	Default LLM for checks/scan	openai/gpt-4o-mini
GISKARD_CHECKS_DEFAULT_EMBEDDING_MODEL	Default embedding model	text-embedding-3-small
GISKARD_CHECKS_MAX_REPORTED_FAILURES	Suite report failure cap	unlimited
GISKARD_CHECKS_DISABLE_RICH_PRETTY	Disable rich REPL pretty-print	false

Precedence: set_default_generator() / set_default_embedding_model() runtime overrides beat environment settings.

Impact on scan

giskard-scan already calls get_default_generator() for recommendations — it automatically picks up GISKARD_CHECKS_DEFAULT_MODEL with no scan code changes.

CI / tests

• Integration workflow for checks now sets GISKARD_CHECKS_DEFAULT_MODEL and GISKARD_CHECKS_DEFAULT_EMBEDDING_MODEL instead of TEST_MODEL.
• Integration test fixtures use get_default_generator() instead of hardcoded model strings.
• New unit tests cover settings resolution, validation, and override precedence.

Verification

make check          # passed
make test-unit PACKAGE=giskard-checks  # 727 passed, 4 skipped
make test-unit PACKAGE=giskard-scan    # 153 passed

  

[gemini-code-assist]

Code Review

This pull request introduces pydantic-settings to manage configuration via environment variables and .env files, replacing direct os.getenv calls with a structured GiskardChecksSettings class. Feedback focuses on avoiding eager settings instantiation at import time in __init__.py, removing unnecessary caching via @lru_cache to keep settings dynamic, and refining the max_reported_failures validator to prevent booleans from being incorrectly treated as integers.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[davidberenstein1957]

I had a minor remark about reasoning mode for model settings.

[davidberenstein1957]

should we also give control over reasonign modes?

[kevinmessiaen]

For now we don't really set reasoning mode globally. Each call side can set a value.

But long term we will probably have something like this:

EASY_TASK_MODEL = {"model": ..., "reasoning_effort": "low"}
HARD_TASK_MODEL = {"model": ..., "reasoning_effort": "high"}