Skip to content

tdd.md

Latest commit

 

History

History
62 lines (44 loc) · 3.27 KB

tdd.md

File metadata and controls

62 lines (44 loc) · 3.27 KB
Raw

Test-Driven Development

First, the distinction that matters

This is a test package, so "test" is overloaded. Keep two meanings separate:

Term What it is Where TDD?
Certification scenario the product — checks an AP against a spec (D-01, TG-06, AC-01) src/suites/ it is a test of the AP, but authored from the plan (see below)
Runner test checks the runner software itself (does the gateway frame JSON-RPC right? does the oracle predict the right verdict?) tests/ yes — red/green/refactor

TDD here means: we build the runner test-first. The certification scenarios are then a thin, well-tested translation of the test plans.

The loop

  1. Red — write a failing tests/ test for the next small behavior. For a component, mock its collaborators (DUT, network, clock). For a scenario, assert against a fake/reference peer.
  2. Green — minimal code to pass.
  3. Refactor — clean up; update the package's contract README.md if the interface moved.

Nothing lands in src/ without a test that failed first.

Test layers

  • Unit (tests/unit/…, mirrors src/) — one module, collaborators mocked. Fast, the bulk. e.g. gateway frame encode/decode; cert-profile loading; oracle verdict from a fixed config; coverage node counting; timeline event merge.
  • Component (tests/component/…) — one service against a fake counterpart: gateway stub ↔ a scripted client; oracle ↔ golden verdicts; engine ↔ a mock DutController.
  • E2E (tests/e2e/…) — a real certification scenario against a real AP on the bench (the lab / Pi appliance). There is no virtual or emulated AP — the DUT is always a supplied, physical device. Runs on hardware, not plain CI; slow, run before release.

The unit/component "scripted client" and "mock DutController" are test doubles that replay protocol frames to exercise the runner — they are not a virtual AP and never stand in for a certifiable device. Certification always targets a real AP (E2E).

Authoring a certification scenario (also test-first)

Driven from the spec, not guessed:

  1. Start from a REQ and its plan row (e.g. REQ-PKI-19 → TG-06).
  2. Write the scenario's check() first, asserting the expected observable, against a fake peer that reproduces the expected behavior → it passes; against one that doesn't → it fails. This proves the assertion has teeth before it ever meets a DUT.
  3. Wire the setup/stimulus. Record the REQ id on the scenario so traceability stays live.

Golden fixtures (the trust anchors)

  • Oracle golden-verdicts — a pinned set of (config → {error, rejected}) the oracle MUST reproduce. This is how we keep the oracle honest (see the review's oracle-trust concern).
  • Gateway frame goldens — canonical JSON-RPC frames the stub MUST emit/accept, tied to the REQ-UC-37 dispatch set.

Both live under tests/fixtures/ and are pinned to the manifest schema SHA.

Running

  • ruff check . && mypy src && pytest tests/unit tests/component — every change (fast).
  • pytest tests/e2e — on the bench against a real AP (hardware-in-the-loop), before release.
  • Coverage target for runner code: high (it judges certifications). Report per package.