Skip to content

book: cache top-kDepthLevels Level* per side to cut publish_depth cost - #29

Merged
MustafaNazeer merged 7 commits into
mainfrom
v1.1.1-depth-cache
May 13, 2026
Merged

book: cache top-kDepthLevels Level* per side to cut publish_depth cost#29
MustafaNazeer merged 7 commits into
mainfrom
v1.1.1-depth-cache

Conversation

@MustafaNazeer

Copy link
Copy Markdown
Owner

Summary

Measurement-driven follow-up to v1.1. With observability=true, the bench drops from 4.79 to 2.79 M evt/s on the desktop reference, a 142 ns/event tax. A per-piece bisection (toggle each side-effect, re-bench) attributes 90 ns of that to Book::publish_depth: an 8-step std::map iteration over the bid and ask price maps on every accepted event, plus a ~400 byte DepthSnapshot copy through the seqlock.

This PR makes Book carry a top-kDepthLevels Level* cache per side, updated by Book::add when a new Level is emplaced and by Book::remove_by_id / Book::erase_empty_level when a Level is erased. Book::publish_depth walks the cache (cache-warm objects the matching thread just touched) instead of the map. No wire format change; no thread-safety change; the existing depth seqlock still anchors reader correctness.

Measured paired runs on the desktop reference (same seed, same 1M events, same 100k warmup):

Mode Pre-cache Post-cache Delta
observability=false (matching only) 4.79 M evt/s 4.60 M evt/s -4%
observability=true (depth + trades + histogram) 2.79 M evt/s 3.39 M evt/s +21%

Remaining observability overhead drops from 142 ns/event to 78 ns/event (about 45 percent reduction). p50 latency on the instrumented path drops from 288 to 230 ns. The observability=false regression is the cache maintenance cost on the Book::add / Book::remove_by_id hot path (linear scan plus shift, at most kDepthLevels operations) and remains comfortably above the 4.10 M evt/s CI baseline floor.

Cache correctness is exercised by seven new unit tests in tests/unit/test_book_depth_cache.cpp, including a 2000-step random-sequence test that audits the cache after every mutation via a new Book::audit_depth_cache_for_test() accessor (walks the maps and asserts cached pointers match the first kDepthLevels entries of each side). All 221 C++ tests pass on release build; existing property invariants P11 and P12, the byte-for-byte Differential test against the Python reference, and the SeqlockDepth concurrency test all continue to pass against the new publish_depth path.

Test plan

  • Full C++ test suite green: ctest --test-dir build-release 221/221.
  • Bench regression check vs bench/baseline.json: throughput -1.2%, p50 +9.4%, p99 +9.9%, p99.9 -41% (all within tolerance).
  • Paired observability=on/off measurement captured in docs/perf/budget.md.
  • CI green on both clang and gcc matrix jobs.
  • Manual smoke test of the live demo after merge: Ladder, Tape, DepthChart, PerfPanel histogram continue to render real data.

The cache buys ~60 ns/event on the observability=true path, but its
maintenance on Book::add and Book::remove_by_id cost roughly 9 ns/event
on the matching loop, which surfaced as a +12% p99 latency regression
against bench/baseline.json on the CI runner. Threading an observability
flag through BookRegistry into Book lets the bench skip the cache
update entirely, matching the existing MatchingEngine observability
flag. Live demo (meridian-server) keeps the default true and the cache
stays live for publish_depth.
@MustafaNazeer
MustafaNazeer merged commit b592323 into main May 13, 2026
3 checks passed
@MustafaNazeer
MustafaNazeer deleted the v1.1.1-depth-cache branch May 13, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant