diff --git a/docs/developer/adr/0002-controller-syncer-architecture.md b/docs/developer/adr/0002-controller-syncer-architecture.md index 46ab623e3a..689a544704 100644 --- a/docs/developer/adr/0002-controller-syncer-architecture.md +++ b/docs/developer/adr/0002-controller-syncer-architecture.md @@ -1,5 +1,17 @@ # ADR 0002: Controller-Syncer-State Architecture +## Status + +- **Status:** Superseded +- **Date:** 2025-12-25 + +## Relationships + +- **Amends:** None. +- **Supersedes:** None. +- **Amended by:** None. +- **Superseded by:** [ADR 0015](0015-sql-shared-chain-ownership.md). + ## 1. Context The legacy `btcwallet` architecture tightly coupled lifecycle management, synchronization logic, and state tracking within a single `Wallet` struct. This monolithic design led to several issues: @@ -45,7 +57,3 @@ Instead of a single status enum, we track three separate dimensions: ### Cons * **Complexity:** Increases the number of distinct types and files. * **Indirection:** Calls to sync functionality now go through a channel-based request mechanism rather than direct method calls. - -## 4. Status - -Accepted and Implemented. diff --git a/docs/developer/adr/0015-sql-shared-chain-ownership.md b/docs/developer/adr/0015-sql-shared-chain-ownership.md new file mode 100644 index 0000000000..3a22f1fb84 --- /dev/null +++ b/docs/developer/adr/0015-sql-shared-chain-ownership.md @@ -0,0 +1,227 @@ +# ADR 0015: SQL Shared-Chain Ownership + +## Status + +- **Status:** Accepted +- **Date:** 2026-08-13 + +## Relationships + +- **Amends:** None. +- **Supersedes:** [ADR 0002](0002-controller-syncer-architecture.md). +- **Amended by:** None. +- **Superseded by:** None. + +## 1. Problem + +ADR 0002 gives each Wallet Syncer ownership of chain synchronization and its +database mutations. That model does not fit a SQL Manager, where multiple +Wallets share one SQLite or PostgreSQL database, block history, and rollback +operation. + +SQL therefore needs one owner for its authoritative chain source, canonical +state, event ordering, and reorgs. Wallets must still own their matching and +persistence policy. This decision also retains the modern-kvdb per-Wallet +chain-source path as a temporary exception until that backend is retired. + +## 2. Context + +Wallet accounts, addresses, transactions, UTXOs, scan horizons, birthday +state, and synchronization tips are wallet-scoped. Canonical block identity +and rollback are database-wide. Allowing independent Wallet Syncers to mutate +both kinds of state would give shared facts multiple authorities. + +Chain backends also differ in notification shape, buffering, and watch/filter +capabilities. Those differences must be normalized before they affect shared +state. Modern kvdb, by contrast, has one Wallet and chain source per database, +so it does not have the shared SQL ownership problem. + +### Constraints + +- SQLite and PostgreSQL must expose the same ownership and ordering behavior. +- The supported runtime has one live SQL Manager per database; multi-process + leases and fencing are out of scope. +- Historical Wallet scans must not advance database-wide canonical state. +- Source ingress and Wallet delivery must be bounded, ordered, and lossless + after admission. +- Reorgs and indeterminate database outcomes must halt forward progress until + durable state is proven. +- Attachment, removal, and shutdown must not create event gaps or let chain + work outlive the Store. + +## 3. Decision + +### 3.1 Ownership and lifecycle + +Each SQL Manager owns one authoritative chain source supplied through +`ManagerConfig`. A Wallet created or loaded by that Manager cannot nominate or +replace the source through its per-Wallet `Config`. + +One Manager-owned coordinator consumes the notification stream and owns: + +- source lifecycle, alignment, and the source-wide watch/filter union; +- notification normalization, admission, and bounded delivery; +- canonical extension and reorg detection; and +- database-wide rollback admission. + +Wallet Controllers and processors continue to own API validation, address and +account matching, transaction ownership, signing, and wallet-scoped +persistence. + +Before admitting events, the coordinator installs the initial watch union, +starts the source, waits for readiness, and aligns it with durable canonical +state. Source adapters must provide bounded or producer-flow-controlled +ingress. Shutdown joins the source and all accepted chain work before closing +the Store. + +### 3.2 Canonical and Wallet state + +The database-wide source of truth is a durable canonical frontier height and +hash plus the block identity leading to it. A stored historical block does not +by itself prove canonical membership or frontier position. + +The first verified candidate initializes an absent frontier atomically. Later +candidates extend only the exact expected height and hash with the expected +parent and next height. Equal replay is idempotent. A stale expectation, +conflict, gap, failure, or indeterminate result does not authorize Wallet +delivery. + +Wallets retain their scan horizons, relevant transactions and UTXOs, +synchronization tips, birthday state, and targeted-rescan progress. Historical +or birthday verification may update those fields without creating or moving +the canonical frontier. SQLite and PostgreSQL must observe the same contract. + +### 3.3 Admission and Wallet delivery + +The coordinator normalizes connected blocks, filtered blocks, relevant +transactions, and disconnects. Duplicate representations form one canonical +candidate. The candidate is proven before its wallet-scoped persistence, and +unconfirmed transactions never move the frontier. A disconnect closes forward +admission and begins coordinated rollback. + +An event is accepted when it crosses the bounded admission boundary with a +stable target-Wallet snapshot. It cannot then be dropped or reordered for a +targeted Wallet. Each live Wallet has at most one long-lived delivery +processor; queue pressure stops later admission instead of skipping a Wallet +or spawning around capacity. A processor acknowledges only a determinate +durable result. Failure or ambiguity halts admission until durable state is +reconciled. + +A new or restarted Wallet catches up to an exact canonical boundary before an +atomic handoff makes it eligible for later live snapshots. Watch expansion +uses the same boundary: register before exposing new material, or pin and +reconcile the pre-registration interval. An overinclusive stale backend watch +may remain until restart but cannot authorize Wallet persistence. Source +optimization hints use the earliest required history and never define the +frontier or authorize a rewind. + +Targeted rescans request verified historical data through the coordinator and +apply matches through the Wallet processor. They do not consume the live +stream, extend the frontier, or request database-wide rollback. Durable rescan +jobs remain outside this decision. + +### 3.4 Reorg, removal, and shutdown + +The coordinator is the sole reorg detector and rollback owner. It closes +forward admission, quiesces affected processors in stable Wallet-ID order, +applies one atomic rollback across shared and wallet-scoped consequences, +reconciles durable results, and resumes only from proven state. If commit +success is ambiguous, it rereads durable state rather than retrying an inferred +failure. + +Removal excludes a Wallet from new snapshots before joining its accepted work +and processor. Manager shutdown rejects new chain work, joins the source, +reconciles accepted deliveries, joins Wallet processors and lifecycle +teardown, and closes the Store last. Unreconciled accepted work keeps the +Store open. + +### 3.5 Boundaries + +The maintained modern-kvdb backend is an explicit temporary exception under +this decision. It keeps its per-Wallet chain-source path only until +modern-kvdb is retired. It does not join the SQL coordinator and must not +expand into a second shared-chain implementation. + +This decision also excludes durable replay frameworks, generic operation +registries or database write locks, lifetime database leases, multi-Manager +coordination, cross-process fencing, and modern-kvdb migration. + +## 4. Rationale + +Canonical state and rollback are database-wide facts, so the database-owning +Manager must be their single authority. Keeping matching and persistence in +Wallet processors preserves the Controller/processor separation without +allowing Syncers to race over shared state. + +Normalization, a stable target snapshot, and an atomic catch-up handoff create +one boundary between backend transport, historical work, and live delivery. +Bounded queues turn overload into backpressure rather than data loss. + +Halting on reorgs and indeterminate outcomes favors proven durable state over +optimistic continuation, where one mistaken retry could corrupt several +Wallets. + +## 5. Alternatives Considered + +### Independent SQL Wallet Syncers + +They preserve the existing runtime shape but give one source, frontier, and +rollback multiple owners. Adding coordination between them recreates the +chosen Manager boundary indirectly. + +### Put all Wallet policy in the coordinator + +This provides one mutation path but collapses the Controller/processor +separation and centralizes unrelated Wallet policy. The coordinator should own +only shared-chain decisions. + +### Use database locks as the coordination model + +Locks can serialize mutations but do not define source ownership, event order, +target snapshots, catch-up handoff, or shutdown. They are an implementation +mechanism, not the ownership contract. + +### Include modern kvdb + +A uniform coordinator would broaden the decision to a backend that does not +share the multi-Wallet database premise and would duplicate behavior without +solving the stated problem. + +## 6. Consequences + +### Positive + +- Shared SQL chain state has one authority. +- Historical Wallet scans cannot advance canonical state. +- Bounded delivery and deterministic quiescence make overload, reorgs, + removal, and shutdown explicit. +- Wallet processors retain wallet-scoped policy. + +### Negative and Risks + +- Startup, attachment, watch registration, rollback, and shutdown require + coordinator barriers. +- A slow or failed Wallet can backpressure source admission until reconciled + or safely removed. +- Backends with unbounded notification queues need bounded adapters. +- Cross-process database ownership remains unresolved. + +## 7. Implementation Overview + +The SQL Manager will construct one database-scoped coordinator and attach +wallet-scoped processors through lifecycle barriers. The Store will represent +canonical frontier state separately from historical block identity and expose +determinate extension and rollback outcomes. + +Source adapters will normalize notifications and provide bounded ingress. +Validation will cover both SQL backends, duplicate notifications, historical +scans, catch-up handoff, backpressure, reorg reconciliation, indeterminate +outcomes, removal, and shutdown ordering. + +## 8. References + +- [ADR 0001](0001-multi-wallet-architecture.md): multi-Wallet Manager scope. +- [ADR 0002](0002-controller-syncer-architecture.md): historical + Controller/Syncer design superseded by this decision. +- [ADR 0004](0004-targeted-rescan-vs-rewind.md): targeted scan boundaries. +- [ADR 0006](0006-wtxmgr-sql-schema.md): SQL transaction and block schema. diff --git a/docs/developer/adr/README.md b/docs/developer/adr/README.md index b0d6630fbb..28adabd098 100644 --- a/docs/developer/adr/README.md +++ b/docs/developer/adr/README.md @@ -35,7 +35,9 @@ relationship metadata, but do not rewrite its historical decision body. ## Existing ADRs - [ADR 0001: Multi-Wallet Architecture](./0001-multi-wallet-architecture.md) - Decides on the architecture for managing multiple distinct wallets and networks within a single daemon instance. -- [ADR 0002: Controller-Syncer-State Architecture](./0002-controller-syncer-architecture.md) - Decouples lifecycle management, synchronization logic, and state tracking from the monolithic `Wallet` struct. +- [ADR 0002: Controller-Syncer-State + Architecture](./0002-controller-syncer-architecture.md) + Superseded by ADR 0015; retained as historical context. - [ADR 0003: Optimistic CFilter Batch Scanning](./0003-optimistic-cfilter-batching.md) - Optimizes BIP 157/158 Compact Filter synchronization using optimistic batch scanning. - [ADR 0004: Targeted Rescan vs. Global Rewind](./0004-targeted-rescan-vs-rewind.md) - Introduces "Targeted Rescans" to replace global "Rewinds" for more efficient transaction discovery. - [ADR 0005: Explicit Rescan on Import](./0005-no-auto-rescan-on-import.md) - Disables automatic blockchain scanning during import operations, requiring explicit user initiation. @@ -50,3 +52,8 @@ relationship metadata, but do not rewrite its historical decision body. - [ADR 0014: Durable SQL Database Identity](./0014-sql-database-identity.md) - Defines the durable role-wallet identity and identity-first initialization order for SQLite and PostgreSQL. +- [ADR 0015: SQL Shared-Chain Ownership](./0015-sql-shared-chain-ownership.md) + Makes one SQL Manager own the authoritative chain source, canonical frontier + admission, bounded Wallet delivery, coordinated rollback, and chain-work + shutdown while retaining the per-Wallet modern-kvdb path as a temporary + exception until that backend is retired.