Test gate 6/6 · Enforce the gate — branch protection, flakiness, speed

[lucaosti]

Sub-issue of #146 — layer 6 of 6. The step that turns the suite from advisory into a gate.

Goal

Make the suite genuinely block merges to master, and keep it trustworthy enough that nobody wants to bypass it.

Why this is separate

Layers 1–5 build the tests. This makes them matter. A comprehensive suite that people merge past is theatre — and the project has already lived through the failure mode: CI triggered on branches: [main, rag], neither being the default branch, so nothing ran on master for weeks and nobody noticed (fixed in #124).

Branch protection

• Enable branch protection on master
• Require the suite as a status check before merge
• Require at least one approving review, per AGENTS.md
• Require branches to be up to date before merge, so the suite runs against what will actually land
• Prevent force-push and deletion of master
• Decide explicitly whether administrators can bypass, and document the decision

Verify the gate actually blocks

• Open a deliberately failing PR and confirm it cannot be merged
• Confirm the required check is the right one — a check that always passes satisfies branch protection while protecting nothing
• Confirm a PR from a fork, if applicable, still runs the required checks

Reliability

A flaky gate teaches people to re-run until green, which is worse than no gate — it produces the habit of ignoring red.

• Run the full suite repeatedly on an unchanged commit; any test that is not deterministic is flaky
• Quarantine each flaky test with its own issue; do not leave it in the required set
• Zero tolerance for retry-until-green as a workaround
• No silent skips: a skipped test is reported and justified

Speed

• Measure total runtime
• If too slow to respect, split: fast checks on every push, full suite required before merge
• Cache dependencies and models in CI where possible — model downloads dominate otherwise

Documentation

• Update AGENTS.md with exactly what must be green, how to run each layer locally, and what to do when a check fails legitimately
• Document known coverage gaps honestly, from the Test gate 1/6 · Coverage audit and gap report #154 gap report

Acceptance criteria

• A PR breaking any layer cannot merge into master — verified by trying
• Branch protection configured and documented
• Zero flaky tests in the required set
• Runtime acceptable, or staged
• AGENTS.md states the gate precisely

Closing #146

When this lands, the parent issue can close: the suite exists, it is comprehensive, and it is enforced.

[lucaosti]

Verified the current state of branch protection on `master` via the API:

```
required_status_checks: null
required_approving_review_count: 1
enforce_admins: false
```

So today: a review is required, but no status check is. A pull request with completely red CI can be merged as soon as one person approves it — nothing mechanically prevents it. That is the gap this issue closes.

With #166 the Quality gate job now exists and aggregates backend, frontend, QVAC, compose validation and the Docker builds. Making that one job a required check is the remaining step:

```bash
gh api -X PATCH repos/BitPolito/bitcoin-academy/branches/master/protection/required_status_checks
-f "strict=true" -f "contexts[]=Quality gate"
```

strict=true also requires the branch to be up to date before merging, so the suite runs against what will actually land.

Two decisions still to make, both noted in the issue body:

• whether enforce_admins should be turned on (currently administrators bypass protection entirely)
• whether to require the branch to be up to date, given it forces a rebase when master moves

Not applying this yet — it changes repository settings and needs an explicit decision.