Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions docs/developer/adr/0002-controller-syncer-architecture.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# ADR 0002: Controller-Syncer-State Architecture

## Status

- **Status:** Superseded
- **Date:** 2025-12-25

## Relationships

- **Amends:** None.
- **Supersedes:** None.
- **Amended by:** None.
- **Superseded by:** [ADR 0015](0015-sql-shared-chain-ownership.md).

## 1. Context

The legacy `btcwallet` architecture tightly coupled lifecycle management, synchronization logic, and state tracking within a single `Wallet` struct. This monolithic design led to several issues:
Expand Down Expand Up @@ -45,7 +57,3 @@ Instead of a single status enum, we track three separate dimensions:
### Cons
* **Complexity:** Increases the number of distinct types and files.
* **Indirection:** Calls to sync functionality now go through a channel-based request mechanism rather than direct method calls.

## 4. Status

Accepted and Implemented.
227 changes: 227 additions & 0 deletions docs/developer/adr/0015-sql-shared-chain-ownership.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
# ADR 0015: SQL Shared-Chain Ownership

## Status

- **Status:** Accepted
- **Date:** 2026-08-13

## Relationships

- **Amends:** None.
- **Supersedes:** [ADR 0002](0002-controller-syncer-architecture.md).
- **Amended by:** None.
- **Superseded by:** None.

## 1. Problem

ADR 0002 gives each Wallet Syncer ownership of chain synchronization and its
database mutations. That model does not fit a SQL Manager, where multiple
Wallets share one SQLite or PostgreSQL database, block history, and rollback
operation.

SQL therefore needs one owner for its authoritative chain source, canonical
state, event ordering, and reorgs. Wallets must still own their matching and
persistence policy. This decision also retains the modern-kvdb per-Wallet
chain-source path as a temporary exception until that backend is retired.

## 2. Context

Wallet accounts, addresses, transactions, UTXOs, scan horizons, birthday
state, and synchronization tips are wallet-scoped. Canonical block identity
and rollback are database-wide. Allowing independent Wallet Syncers to mutate
both kinds of state would give shared facts multiple authorities.

Chain backends also differ in notification shape, buffering, and watch/filter
capabilities. Those differences must be normalized before they affect shared
state. Modern kvdb, by contrast, has one Wallet and chain source per database,
so it does not have the shared SQL ownership problem.

### Constraints

- SQLite and PostgreSQL must expose the same ownership and ordering behavior.
- The supported runtime has one live SQL Manager per database; multi-process
leases and fencing are out of scope.
- Historical Wallet scans must not advance database-wide canonical state.
- Source ingress and Wallet delivery must be bounded, ordered, and lossless
after admission.
- Reorgs and indeterminate database outcomes must halt forward progress until
durable state is proven.
- Attachment, removal, and shutdown must not create event gaps or let chain
work outlive the Store.

## 3. Decision

### 3.1 Ownership and lifecycle

Each SQL Manager owns one authoritative chain source supplied through
`ManagerConfig`. A Wallet created or loaded by that Manager cannot nominate or
replace the source through its per-Wallet `Config`.

One Manager-owned coordinator consumes the notification stream and owns:

- source lifecycle, alignment, and the source-wide watch/filter union;
- notification normalization, admission, and bounded delivery;
- canonical extension and reorg detection; and
- database-wide rollback admission.

Wallet Controllers and processors continue to own API validation, address and
account matching, transaction ownership, signing, and wallet-scoped
persistence.

Before admitting events, the coordinator installs the initial watch union,
starts the source, waits for readiness, and aligns it with durable canonical
state. Source adapters must provide bounded or producer-flow-controlled
ingress. Shutdown joins the source and all accepted chain work before closing
the Store.

### 3.2 Canonical and Wallet state

The database-wide source of truth is a durable canonical frontier height and
hash plus the block identity leading to it. A stored historical block does not
by itself prove canonical membership or frontier position.

The first verified candidate initializes an absent frontier atomically. Later
candidates extend only the exact expected height and hash with the expected
parent and next height. Equal replay is idempotent. A stale expectation,
conflict, gap, failure, or indeterminate result does not authorize Wallet
delivery.

Wallets retain their scan horizons, relevant transactions and UTXOs,
synchronization tips, birthday state, and targeted-rescan progress. Historical
or birthday verification may update those fields without creating or moving
the canonical frontier. SQLite and PostgreSQL must observe the same contract.

### 3.3 Admission and Wallet delivery

The coordinator normalizes connected blocks, filtered blocks, relevant
transactions, and disconnects. Duplicate representations form one canonical
candidate. The candidate is proven before its wallet-scoped persistence, and
unconfirmed transactions never move the frontier. A disconnect closes forward
admission and begins coordinated rollback.

An event is accepted when it crosses the bounded admission boundary with a
stable target-Wallet snapshot. It cannot then be dropped or reordered for a
targeted Wallet. Each live Wallet has at most one long-lived delivery
processor; queue pressure stops later admission instead of skipping a Wallet
or spawning around capacity. A processor acknowledges only a determinate
durable result. Failure or ambiguity halts admission until durable state is
reconciled.

A new or restarted Wallet catches up to an exact canonical boundary before an
atomic handoff makes it eligible for later live snapshots. Watch expansion
uses the same boundary: register before exposing new material, or pin and
reconcile the pre-registration interval. An overinclusive stale backend watch
may remain until restart but cannot authorize Wallet persistence. Source
optimization hints use the earliest required history and never define the
frontier or authorize a rewind.

Targeted rescans request verified historical data through the coordinator and
apply matches through the Wallet processor. They do not consume the live
stream, extend the frontier, or request database-wide rollback. Durable rescan
jobs remain outside this decision.

### 3.4 Reorg, removal, and shutdown

The coordinator is the sole reorg detector and rollback owner. It closes
forward admission, quiesces affected processors in stable Wallet-ID order,
applies one atomic rollback across shared and wallet-scoped consequences,
reconciles durable results, and resumes only from proven state. If commit
success is ambiguous, it rereads durable state rather than retrying an inferred
failure.

Removal excludes a Wallet from new snapshots before joining its accepted work
and processor. Manager shutdown rejects new chain work, joins the source,
reconciles accepted deliveries, joins Wallet processors and lifecycle
teardown, and closes the Store last. Unreconciled accepted work keeps the
Store open.

### 3.5 Boundaries

The maintained modern-kvdb backend is an explicit temporary exception under
this decision. It keeps its per-Wallet chain-source path only until
modern-kvdb is retired. It does not join the SQL coordinator and must not
expand into a second shared-chain implementation.

This decision also excludes durable replay frameworks, generic operation
registries or database write locks, lifetime database leases, multi-Manager
coordination, cross-process fencing, and modern-kvdb migration.

## 4. Rationale

Canonical state and rollback are database-wide facts, so the database-owning
Manager must be their single authority. Keeping matching and persistence in
Wallet processors preserves the Controller/processor separation without
allowing Syncers to race over shared state.

Normalization, a stable target snapshot, and an atomic catch-up handoff create
one boundary between backend transport, historical work, and live delivery.
Bounded queues turn overload into backpressure rather than data loss.

Halting on reorgs and indeterminate outcomes favors proven durable state over
optimistic continuation, where one mistaken retry could corrupt several
Wallets.

## 5. Alternatives Considered

### Independent SQL Wallet Syncers

They preserve the existing runtime shape but give one source, frontier, and
rollback multiple owners. Adding coordination between them recreates the
chosen Manager boundary indirectly.

### Put all Wallet policy in the coordinator

This provides one mutation path but collapses the Controller/processor
separation and centralizes unrelated Wallet policy. The coordinator should own
only shared-chain decisions.

### Use database locks as the coordination model

Locks can serialize mutations but do not define source ownership, event order,
target snapshots, catch-up handoff, or shutdown. They are an implementation
mechanism, not the ownership contract.

### Include modern kvdb

A uniform coordinator would broaden the decision to a backend that does not
share the multi-Wallet database premise and would duplicate behavior without
solving the stated problem.

## 6. Consequences

### Positive

- Shared SQL chain state has one authority.
- Historical Wallet scans cannot advance canonical state.
- Bounded delivery and deterministic quiescence make overload, reorgs,
removal, and shutdown explicit.
- Wallet processors retain wallet-scoped policy.

### Negative and Risks

- Startup, attachment, watch registration, rollback, and shutdown require
coordinator barriers.
- A slow or failed Wallet can backpressure source admission until reconciled
or safely removed.
- Backends with unbounded notification queues need bounded adapters.
- Cross-process database ownership remains unresolved.

## 7. Implementation Overview

The SQL Manager will construct one database-scoped coordinator and attach
wallet-scoped processors through lifecycle barriers. The Store will represent
canonical frontier state separately from historical block identity and expose
determinate extension and rollback outcomes.

Source adapters will normalize notifications and provide bounded ingress.
Validation will cover both SQL backends, duplicate notifications, historical
scans, catch-up handoff, backpressure, reorg reconciliation, indeterminate
outcomes, removal, and shutdown ordering.

## 8. References

- [ADR 0001](0001-multi-wallet-architecture.md): multi-Wallet Manager scope.
- [ADR 0002](0002-controller-syncer-architecture.md): historical
Controller/Syncer design superseded by this decision.
- [ADR 0004](0004-targeted-rescan-vs-rewind.md): targeted scan boundaries.
- [ADR 0006](0006-wtxmgr-sql-schema.md): SQL transaction and block schema.
9 changes: 8 additions & 1 deletion docs/developer/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@ relationship metadata, but do not rewrite its historical decision body.
## Existing ADRs

- [ADR 0001: Multi-Wallet Architecture](./0001-multi-wallet-architecture.md) - Decides on the architecture for managing multiple distinct wallets and networks within a single daemon instance.
- [ADR 0002: Controller-Syncer-State Architecture](./0002-controller-syncer-architecture.md) - Decouples lifecycle management, synchronization logic, and state tracking from the monolithic `Wallet` struct.
- [ADR 0002: Controller-Syncer-State
Architecture](./0002-controller-syncer-architecture.md)
Superseded by ADR 0015; retained as historical context.
- [ADR 0003: Optimistic CFilter Batch Scanning](./0003-optimistic-cfilter-batching.md) - Optimizes BIP 157/158 Compact Filter synchronization using optimistic batch scanning.
- [ADR 0004: Targeted Rescan vs. Global Rewind](./0004-targeted-rescan-vs-rewind.md) - Introduces "Targeted Rescans" to replace global "Rewinds" for more efficient transaction discovery.
- [ADR 0005: Explicit Rescan on Import](./0005-no-auto-rescan-on-import.md) - Disables automatic blockchain scanning during import operations, requiring explicit user initiation.
Expand All @@ -50,3 +52,8 @@ relationship metadata, but do not rewrite its historical decision body.
- [ADR 0014: Durable SQL Database Identity](./0014-sql-database-identity.md) -
Defines the durable role-wallet identity and identity-first initialization
order for SQLite and PostgreSQL.
- [ADR 0015: SQL Shared-Chain Ownership](./0015-sql-shared-chain-ownership.md)
Makes one SQL Manager own the authoritative chain source, canonical frontier
admission, bounded Wallet delivery, coordinated rollback, and chain-work
shutdown while retaining the per-Wallet modern-kvdb path as a temporary
exception until that backend is retired.
Loading