I maintain PRML — an open standard that locks the success criteria of an AI evaluation to a SHA-256 hash before the run, so a claim about a model can be checked by a third party afterwards. Reference implementation and public registry at falsify.dev.
It proves the bar was locked. Never the result.
The claim below is a real PRML v0.1 manifest — manifest.yaml in this repository. It commits this profile to something checkable: at least 4 reference implementations, byte-identical across the spec's 13 conformance vectors.
version: prml/0.1
claim_id: 0198f2c4-7a1e-7000-8000-4d5e6f7a8b9c
created_at: "2026-08-20T21:30:00Z"
metric: byte_equivalent_reference_implementations
comparator: ">="
threshold: 4
dataset:
id: spec.falsify.dev/v0.1/test-vectors
hash: cd7d22ee5bda1b5834c74f955fe3452b49e5357d5a7c9933805d0d0d99aa2a8e
seed: 0
producer:
id: github.com/sk8ordie84Don't take my word for any of it:
# 1 — the file is what this page says it is (no install needed)
curl -s https://raw.githubusercontent.com/sk8ordie84/sk8ordie84/main/manifest.yaml | shasum -a 256
# → 960b0483f7e503e87f4d14cf1421d7385e67954712cf445d931acd7a622fa37e
# 2 — the PRML canonical hash, from the reference implementation
pip install falsify
curl -sO https://raw.githubusercontent.com/sk8ordie84/sk8ordie84/main/manifest.yaml
falsify hash manifest.yaml
# → 4f1ebcc8ddc249ff656d7d040367fcef64ced224e2a90941a68b18d8fd9261e7
# 3 — the claim itself: 4 byte-equivalent implementations clear the locked bar
falsify verify manifest.yaml \
--expected-hash 4f1ebcc8ddc249ff656d7d040367fcef64ced224e2a90941a68b18d8fd9261e7 \
--observed 4
# → PASS (edit any field first and it prints TAMPERED instead — try it)The dataset.hash binds the 13 conformance vectors the claim is measured against, and CI re-runs the byte-equivalence check on every push.
| Claim | Check it |
|---|---|
| Openly specified, CC BY 4.0, archived | DOI 10.5281/zenodo.20177839 |
| 4 reference implementations — Python, JS, Go, Rust — byte-equivalent | conformance CI · falsify-js |
| JSON Schema in the SchemaStore catalogue | catalog.json → "PRML manifest" |
| Proposed as an in-toto attestation predicate | in-toto/attestation#587 |
| Public registry: Ed25519-signed receipts, RFC 3161 countersigned, mirrored to Rekor | registry.falsify.dev · /pubkey |
| Works inside real eval harnesses | examples: lm-eval-harness, Inspect, promptfoo, Langfuse, Opik, Moonshot, more |
Falsify OÜ · Tallinn · hello@falsify.dev — the discipline is optional; the hash is not.