Skip to content

Commit 563f6a5

Browse files
committed
git commit -m "Release v0.1.2 with Double-X trust reporting and CLI formatter"
1 parent 9ce3bb0 commit 563f6a5

25 files changed

Lines changed: 1052 additions & 866 deletions

CHANGELOG.md

Lines changed: 59 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,67 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8-
## [0.1.1] - 2026-03-28
8+
## [0.1.2]
9+
10+
### Added
11+
12+
- **CLI trust report** - `dokis audit sample_audit.json` now renders a
13+
screenshot-friendly terminal report that highlights:
14+
- allowed vs blocked sources
15+
- human-readable blocked-source reasons
16+
- supported vs unsupported claim verdicts
17+
- `policy_issues`
18+
- `enforcement_mode` and `enforcement_verdict`
19+
- final compliance status
20+
21+
- **Structured trust result** - Dokis now returns a richer per-response
22+
provenance report aimed at runtime trust enforcement, not only a compliance
23+
score. `ProvenanceResult` now includes:
24+
- `blocked_source_details` with structured `BlockedSource` entries
25+
- `claim_verdicts` with compact per-claim reporting
26+
- `policy_issues` summarising whether blocked sources and/or unsupported
27+
claims were present
28+
- `has_blocked_sources` and `has_unsupported_claims`
29+
- `enforcement_mode`, `enforcement_verdict`, and `raised_on_violation`
30+
31+
- **Explicit enforcement modes** - `Config` now supports
32+
`enforcement_mode="audit" | "guardrail" | "enforce"`.
33+
- `audit` always returns a result
34+
- `guardrail` returns a result and marks trust failure in the result
35+
- `enforce` fails closed by raising `ComplianceViolation`
36+
37+
- **Structured blocked-source classification** - `DomainEnforcer.inspect()`
38+
returns `BlockedSource` records with a `reason` field:
39+
- `domain_not_allowlisted`
40+
- `malformed_source_url`
41+
- `missing_source_url`
42+
43+
- **Top-level exports for new report models** - `dokis.BlockedSource` and
44+
`dokis.ClaimVerdict` are now part of the public package surface.
45+
46+
47+
### Changed
48+
49+
- **Legacy fail-closed config remains supported** - `fail_on_violation` is now
50+
treated as a backwards-compatible alias for `enforcement_mode="enforce"`.
51+
If both are provided, `enforcement_mode` wins.
52+
53+
- **Async wrappers are documented honestly** - `aaudit()` and `afilter()`
54+
remain awaitable wrappers around the synchronous pipeline. They are
55+
convenient for async call sites, but they do not offload matching work from
56+
the event loop.
57+
58+
- **ComplianceViolation messaging** now includes both the final
59+
`enforcement_verdict` and the compact `policy_issues` summary.
60+
61+
- **README and example config** now reflect the modern `enforcement_mode`
62+
interface and the richer result object.
963

1064
### Fixed
11-
- Use absolute asset URLs for PyPI README rendering
12-
- Force Python badge cache refresh
65+
66+
- **BM25 progress noise suppressed** - BM25 tokenization, indexing, and
67+
retrieval now pass `show_progress=False` so runtime audits and tests stay
68+
quiet by default.
1369

1470
## [0.1.0] - 2026-03-25
1571

README.md

Lines changed: 55 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -24,19 +24,19 @@ Existing tools don't solve this at runtime:
2424
- **LLM guardrails** handle safety and policy enforcement well - toxicity, jailbreaks, off-topic content. Their provenance validators strip unsupported sentences but don't return a structured claim→URL map, a compliance rate, or a source allowlist.
2525
- **Prompt engineering** reduces the problem. It doesn't eliminate it.
2626

27-
Dokis sits inline - between your retriever and your LLM response going out - and enforces provenance in real time.
27+
Dokis sits inline between retrieval and response delivery and returns a runtime trust report for the exact answer your system is about to ship. It acts as a provenance and enforcement boundary in real time.
2828

2929
---
3030

3131
## How it works
3232

33-
Dokis does exactly two things:
33+
Dokis does two things in one deterministic runtime pass:
3434

3535
**1. Pre-retrieval enforcement.** Strip chunks whose source URL is not on your allowlist before they enter the prompt.
3636

37-
**2. Post-generation auditing.** Split the response into atomic claim sentences. Match each claim to the chunk it came from using BM25 lexical scoring. Build a `claim → chunk → URL` provenance map. Compute a compliance rate. Flag anything below your threshold.
37+
**2. Post-generation auditing.** Split the response into atomic claim sentences. Match each claim to the best supporting chunk using BM25 lexical scoring by default. Build a `claim → chunk → URL` provenance map. Compute a compliance rate. Return blocked-source details, claim-level verdicts, policy issues, and a final enforcement verdict.
3838

39-
No LLM call. No API key. No network request after startup. Deterministic output.
39+
No LLM call or API key is required for the default BM25 path. Output is deterministic for identical inputs and config.
4040

4141
<div align="center">
4242

@@ -51,7 +51,7 @@ No LLM call. No API key. No network request after startup. Deterministic output.
5151

5252
<div align="center">
5353

54-
![Dokis CLI demo](https://raw.githubusercontent.com/Vbj1808/dokis/main/assets/demo.gif)
54+
![Dokis CLI demo](https://raw.githubusercontent.com/Vbj1808/dokis/main/assets/demo.png)
5555

5656
</div>
5757

@@ -70,8 +70,26 @@ print(result.compliance_rate) # 0.91
7070
print(result.passed) # True
7171
print(result.provenance_map) # {"Aspirin inhibits...": "https://pubmed.com/1"}
7272
print(result.violations) # claims with no source
73+
print(result.claim_verdicts) # explicit supported / unsupported report
74+
print(result.policy_issues) # [] | ["blocked_sources"] | ...
75+
print(result.enforcement_mode) # "guardrail"
76+
print(result.enforcement_verdict) # "passed"
7377
```
7478

79+
### CLI trust report
80+
81+
```bash
82+
dokis audit sample_audit.json
83+
```
84+
85+
The CLI reads a JSON file containing `query`, `chunks`, and `response`. If a
86+
`provenance.toml` file is present in the current directory or beside the input
87+
file, Dokis loads it automatically so the report reflects your real allowlist,
88+
threshold, matcher, and enforcement mode. Use `--config path/to/file.toml` to
89+
override that discovery. Use `--no-color` for plain output. Exit code is `0`
90+
when the audit passes, `1` when it fails policy/compliance checks, and `2`
91+
for CLI/input errors.
92+
7593
### With config
7694

7795
```python
@@ -81,6 +99,7 @@ config = dokis.Config(
8199
allowed_domains = ["pubmed.ncbi.nlm.nih.gov", "cochrane.org"],
82100
min_citation_rate = 0.85,
83101
claim_threshold = 0.3,
102+
enforcement_mode = "guardrail",
84103
)
85104

86105
clean_chunks = dokis.filter(raw_chunks, config)
@@ -122,9 +141,10 @@ result = response.metadata["provenance"]
122141
```bash
123142
dokis audit input.json
124143
dokis audit input.json --config provenance.toml
125-
cat input.json | dokis audit -
126144
```
127145

146+
`dokis audit` currently expects a file path. Stdin piping is not supported.
147+
128148
### Reusable middleware (production pattern)
129149

130150
```python
@@ -135,6 +155,7 @@ mw = ProvenanceMiddleware(Config(
135155
min_citation_rate = 0.85,
136156
matcher = "bm25",
137157
claim_threshold = 0.3,
158+
enforcement_mode = "guardrail",
138159
))
139160

140161
result = mw.audit(query, chunks, response)
@@ -170,11 +191,15 @@ dokis.Config(
170191
extractor = "regex", # "regex" | "nltk" | "llm"
171192
matcher = "bm25", # "bm25" | "semantic"
172193
model = "all-MiniLM-L6-v2",
173-
fail_on_violation = False,
194+
enforcement_mode = "guardrail", # "audit" | "guardrail" | "enforce"
174195
domain = None,
175196
)
176197
```
177198

199+
`fail_on_violation` still works as a backwards-compatible alias for
200+
`enforcement_mode="enforce"`, but `enforcement_mode` is the recommended
201+
interface for new configs and examples.
202+
178203
**`claim_threshold` by matcher:**
179204
- `matcher="bm25"`: normalised per-query BM25 score. Recommended: `0.3–0.5`.
180205
- `matcher="semantic"`: cosine similarity. Recommended: `0.65–0.85`.
@@ -193,18 +218,36 @@ config = dokis.Config.from_yaml("provenance.toml")
193218
```python
194219
result.compliance_rate # float
195220
result.passed # bool
196-
result.violations # list[Claim]
197-
result.provenance_map # dict[claim_text, source_url]
198-
result.blocked_sources # list[str]
199-
result.claims # list[Claim]
221+
result.violations # list[Claim] (derived unsupported claims)
222+
result.provenance_map # dict[claim_text, source_url] (derived supported claims)
223+
result.blocked_sources # list[str] (backwards-compatible)
224+
result.blocked_source_details # list[BlockedSource]
225+
result.claim_verdicts # list[ClaimVerdict]
226+
result.policy_issues # ["blocked_sources", "unsupported_claims"]
227+
result.has_blocked_sources # bool
228+
result.has_unsupported_claims # bool
229+
result.enforcement_mode # "audit" | "guardrail" | "enforce"
230+
result.enforcement_verdict # "passed" | "..._failed" | "enforce_raised"
231+
result.raised_on_violation # bool
232+
result.claims # list[Claim] (full per-claim audit records)
200233

201234
claim.text # str
202235
claim.supported # bool
203236
claim.confidence # float - always set, even when False
204237
claim.source_url # str | None
205238
claim.source_chunk # Chunk | None
206239

207-
record = result.model_dump_json() # fully JSON-serialisable
240+
blocked.url # str
241+
blocked.domain # str | None
242+
blocked.reason # "domain_not_allowlisted" | "malformed_source_url" | "missing_source_url"
243+
244+
verdict.claim_text # str
245+
verdict.verdict # "supported" | "unsupported"
246+
verdict.confidence # float
247+
verdict.supporting_url # str | None
248+
verdict.note # str | None
249+
250+
record = result.model_dump_json() # fully JSON-serialisable trust report
208251
```
209252

210253
---

assets/demo.png

133 KB
Loading

dokis/__init__.py

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
result = dokis.audit(query, chunks, response)
1212
1313
# With config
14-
config = dokis.Config.from_yaml("provenance.yaml")
14+
config = dokis.Config.from_yaml("provenance.toml")
1515
clean_chunks = dokis.filter(raw_chunks, config)
1616
result = dokis.audit(query, clean_chunks, response, config=config)
1717
@@ -26,7 +26,13 @@
2626
from dokis.config import Config
2727
from dokis.exceptions import ComplianceViolation, DomainViolation
2828
from dokis.middleware import ProvenanceMiddleware
29-
from dokis.models import Chunk, Claim, ProvenanceResult
29+
from dokis.models import (
30+
BlockedSource,
31+
Chunk,
32+
Claim,
33+
ClaimVerdict,
34+
ProvenanceResult,
35+
)
3036

3137
# Module-level default configuration set via configure().
3238
_default_config: Config | None = None
@@ -160,11 +166,13 @@ def audit(
160166
"Config",
161167
"Chunk",
162168
"Claim",
169+
"BlockedSource",
170+
"ClaimVerdict",
163171
"ProvenanceResult",
164172
"ProvenanceMiddleware",
165173
"ComplianceViolation",
166174
"DomainViolation",
167175
]
168176

169177
# Version
170-
__version__ = "0.1.1"
178+
__version__ = "0.1.2"

dokis/__main__.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
"""Module entrypoint for ``python -m dokis``."""
2+
3+
from __future__ import annotations
4+
5+
from dokis.cli import main
6+
7+
if __name__ == "__main__":
8+
raise SystemExit(main())

dokis/adapters/langchain.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from __future__ import annotations
44

55
import logging
6-
from typing import Any
6+
from typing import Any, cast
77

88
try:
99
from langchain_core.callbacks import CallbackManagerForRetrieverRun
@@ -20,8 +20,8 @@
2020

2121
logger = logging.getLogger(__name__)
2222

23-
24-
class ProvenanceRetriever(BaseRetriever): # type: ignore[misc]
23+
24+
class ProvenanceRetriever(BaseRetriever):
2525
"""LangChain retriever that enforces source provenance via DomainEnforcer.
2626
2727
Wraps any existing LangChain ``BaseRetriever`` and filters its results
@@ -60,7 +60,8 @@ def __init__(
6060
url_metadata_key: str = "source",
6161
**kwargs: Any,
6262
) -> None:
63-
super().__init__(
63+
super_init = cast(Any, super().__init__)
64+
super_init(
6465
base_retriever=base_retriever,
6566
config=config or Config(),
6667
url_metadata_key=url_metadata_key,

dokis/adapters/llamaindex.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
from __future__ import annotations
44

55
import logging
6-
from typing import Any
76

87
try:
98
from llama_index.core.base.base_query_engine import BaseQueryEngine
@@ -81,7 +80,7 @@ def query(self, query: str) -> RESPONSE_TYPE:
8180
``metadata["provenance"]`` populated as a
8281
:class:`~dokis.models.ProvenanceResult`.
8382
"""
84-
response: Any = self._base_engine.query(query)
83+
response = self._base_engine.query(query)
8584
result = self._middleware.audit(
8685
query=query,
8786
chunks=self._chunks,

0 commit comments

Comments
 (0)