Skip to content

SMOKE.md

Latest commit

 

History

History
238 lines (162 loc) · 7.01 KB

SMOKE.md

File metadata and controls

238 lines (162 loc) · 7.01 KB
Raw

Smoke Test — Verify the End-to-End Loop

A scripted, interactive verification that the SDK → API → SPA loop actually works. Run this once before declaring a release. ~10 minutes start to finish.

The pieces have all been built and unit-tested, but only a human at a terminal can confirm Supabase + Vercel + the SDK actually talk to each other.

Prereqs

  • Docker running (Supabase local uses it)
  • Node 20+, Python 3.13+, openssl
  • Supabase CLI: brew install supabase/tap/supabase
  • Vercel CLI: npm install -g vercel
  • Repo cloned, git status clean

Pass criteria

The test passes if every numbered step below produces the expected output. Fail = file an issue with the step number and the actual output.

0. Clean slate

cd /path/to/monora-sdk
rm -rf node_modules monora-node/node_modules platform/app/node_modules
rm -rf monora-node/dist platform/app/dist
rm -rf examples/agent_causal_proof/wal examples/agent_causal_proof/output
rm -f .env.local platform/app/.env.local
supabase stop --no-backup 2>/dev/null || true

1. Fresh install

npm ci
npm --prefix monora-node ci && npm --prefix monora-node run build
npm --prefix platform/app ci --include=dev
python3 -m venv .venv && .venv/bin/pip install -e .
.venv/bin/pip install requests pyyaml cryptography

Expected: No npm ERR!, no pip resolution errors. monora-node/dist/causal_proof.js exists.

2. Unit tests (fast feedback before the real boot)

.venv/bin/python -m pytest tests/test_agent.py tests/test_causal_proof.py tests/test_hasher.py tests/test_signing.py tests/test_wal.py tests/test_verification.py --no-header
cd monora-node && npx jest src/agent.test.ts src/causal_proof.test.ts src/pdf_report.test.ts && cd ..

Expected: 144+ passed (Python), 12 passed (Node).

3. Local-WAL demo (sanity — proves the SDK works without any backend)

node examples/agent_causal_proof/node/run.js

Expected (tail):

Verified: YES ✓
Root:     LLM call: classify_intent
Chain:
  1. llm_call:classify_intent              … conf=100%
  2. tool_call:crm.getOpportunities        … conf=100%
  3. decision:correlate_signals            … conf=100%
  4. outcome:renewal_at_risk               … conf=100%

examples/agent_causal_proof/output/proof.json and proof.html exist.

4. Boot the local stack

In terminal A:

./scripts/dev.sh

Expected:

  • ▶ Booting Supabase… then a table of local Supabase URLs.
  • ✓ Wrote .env.local and platform/app/.env.local
  • ▶ Booting vercel dev… followed by > Ready! Available at http://localhost:3000.

Leave this terminal running.

5. Sign in (creates owner account)

Open http://localhost:3000 in a browser.

  1. Click Sign Up.
  2. Email: anything (e.g. you@example.com), password ≥ 8 chars.
  3. Expected: redirected to /traces with an empty trace inbox. Top-right shows your email + Sign Out.

6. Mint an API key + sample project

In terminal B:

./scripts/seed.sh you@example.com

Expected:

✓  Seeded.
  project_id:        <uuid>
  api_key (prefix):  mnr_…

.env.local now has MONORA_API_KEY=mnr_… and MONORA_DEMO_PROJECT_ID=<uuid>.

7. (Optional) Enable signing

./scripts/keygen.sh

Expected: ✓ Wrote MONORA_SIGNING_KEY to .env.local (HMAC-SHA256, 256-bit).

8. Run the agent against the hosted backend

node examples/agent_causal_proof/hosted/run.js

Expected (tail):

▶  Shipping events to http://127.0.0.1:54321/functions/v1/ingest-events
▶  Signing: hmac-sha256 (MONORA_SIGNING_KEY set)     ← if you ran step 7; else "disabled"
▶  Running 4-step agent on trace trc_hosted_<…>
✓  Agent emitted 4 events; outcome evt_…

Done. Open the SPA to see the trace:
  http://localhost:3000/traces

9. Verify the trace in the SPA inbox

Refresh http://localhost:3000/traces.

Expected:

  • A new row appears matching the Trace ID printed in step 8.
  • Integrity badge: ✓ verified.
  • Event count: 4.

10. Open the proof viewer

Click into the trace.

Expected:

  • 4 numbered chain steps render: llm_call:classify_intent, tool_call:crm.getOpportunities, decision:correlate_signals, outcome:renewal_at_risk.
  • Green ✓ HASH CHAIN VERIFIED banner at top.
  • Each step shows inputs/outputs/latency/confidence.

11. Download the PDF

Click Download PDF.

Expected:

  • ~2-3 second delay (Puppeteer cold start), then the PDF downloads.
  • PDF opens cleanly. Header: "Causal Proof of Record". Same chain rendered with the integrity banner.

12. Create a share link

Click Create share link.

Expected:

  • URL copied to clipboard or shown. Form: http://localhost:3000/share/share_<random>.
  • Open it in an incognito window.
  • Proof renders without a sign-in wall.

13. RBAC sanity check

In Supabase Studio (http://127.0.0.1:54323):

  1. Open the SQL editor.
  2. Run: update public.organization_members set role = 'auditor' where user_id = (select id from auth.users limit 1);
  3. Refresh the SPA — note nothing breaks immediately because cached JWT still has owner-level access. Sign out and back in.
  4. Try API Keys page → "New" button.
  5. Expected: the request 403s with Your role (auditor) is not authorized for this action. (read-only paths like Trace Inbox + Proof Viewer + Share Link creation still work.)
  6. Reset: update public.organization_members set role = 'owner' where user_id = (select id from auth.users limit 1);, sign out, sign back in.

14. Tamper test (the load-bearing demo)

In Supabase Studio SQL editor:

update public.proof_events
set body = body || '{"outputs": "TAMPERED"}'::jsonb
where event_type = 'decision'
  and trace_id = '<trace id from step 8>';

Reload the Proof Viewer page.

Expected:

  • Red banner: ✗ INTEGRITY CHECK FAILED — LOG MAY BE COMPROMISED.
  • The failing step is highlighted with a red border.
  • Confidence drops to 0.0% on the tampered step.
  • Re-downloading the PDF carries the failure forward.

15. Shutdown

Terminal A: Ctrl-C.

Expected: ▶ Stopping Supabase… then a clean exit.

What this verifies

  • Fresh install works (Phase 1 of the workspace + SDK build pipeline)
  • Unit tests pass (regression detection)
  • Local-WAL demo works (SDK in isolation)
  • Local Supabase + Vercel boot cleanly (scripts/dev.sh)
  • Auth + RLS round-trip (signup → owner → first signup gets role)
  • API key minting end-to-end (scripts/seed.sh)
  • HTTPS sink ships events to the Edge Function
  • Edge Function writes proof_events + upserts trace_summaries
  • GET /api/traces lists the trace
  • GET /api/proofs/.../... builds the proof from proof_events
  • POST /api/proofs/.../pdf renders Puppeteer PDF
  • POST /api/share-links mints + GET /api/share/:token resolves
  • RBAC denies cross-role writes
  • Tamper detection works against live DB

If a step fails

Take a screenshot, attach terminal A's logs and the browser console, and file an issue with the step number.