TrustyAI Service

The TrustyAI Service is a REST API for Responsible AI workflows: drift detection, fairness monitoring, and model explainability. Built on FastAPI + Hypercorn, it consumes inference data from KServe, stores it, and computes metrics on a schedule via Prometheus.

Part of Red Hat OpenShift AI and Open Data Hub.

Documentation · API Reference

---

Key Features

• Real-time drift detection on live inference streams
• Automatic Prometheus metric publishing on a configurable schedule
• KServe-native — consumes inference payloads directly via CloudEvents
• Dual storage backends — PVC (HDF5) or MariaDB
• Runs anywhere — locally, in Jupyter, or on Kubernetes

---

Metrics

Drift Detection

• Compare Means (Welch's t-test)
• Kolmogorov–Smirnov Test (scipy.stats.ks_2samp)
• Streaming Kolmogorov–Smirnov Test (Lall 2015) using the Greenwald–Khanna quantile sketch (Greenwald & Khanna 2001)
• Jensen–Shannon Divergence (scipy.spatial.distance)
• Maximum Mean Discrepancy (Domingo-Enrich et al. 2023) with CTT, RFF, and ACTT methods via goodpoints

Fairness

• Statistical Parity Difference (SPD)
• Disparate Impact Ratio (DIR)

---

Quickstart

# Install
uv sync --all-groups

# Run
uv run python -m src.main

Once running, the API is available at http://localhost:8080. Interactive OpenAPI documentation is at http://localhost:8080/docs.

---

Container

# Minimal (PVC storage only)
podman build -t trustyai:latest .

# With MariaDB support
podman build -t trustyai:latest --build-arg EXTRAS="mariadb" .

# Run
podman run -p 8080:8080 trustyai:latest

---

Configuration

Environment Variable	Default	Description
SERVICE_STORAGE_FORMAT	PVC	Storage backend (PVC or MARIA)
SERVICE_METRICS_SCHEDULE	30	Seconds between scheduled computations
HTTP_PORT	8080	HTTP listener port
SSL_PORT	4443	HTTPS listener port
TLS_CERT_FILE	/etc/tls/internal/tls.crt	TLS certificate path
TLS_KEY_FILE	/etc/tls/internal/tls.key	TLS private key path
DATABASE_HOST	—	MariaDB hostname
DATABASE_PORT	3306	MariaDB port
DATABASE_USERNAME	—	MariaDB username
DATABASE_PASSWORD	—	MariaDB password
DATABASE_DATABASE	—	MariaDB database name

TLS is enabled automatically when both the certificate and key files are present.

---

Testing

uv run pytest tests/ -v
uv run pytest tests/ -v --cov=src --cov-report=xml  # with coverage

---

Contributing

See CONTRIBUTING.md for development setup, coding standards, and the pull request process.

Please report security vulnerabilities via GitHub Security Advisories, not public issues. See SECURITY.md for details.

---

License

Apache License 2.0