Skip to content

wallet: serialize Manager wallet assembly - #1336

Open
yyforyongyu wants to merge 2 commits into
btcsuite:sql-walletfrom
yyforyongyu:task-manager-wallet-publication
Open

wallet: serialize Manager wallet assembly#1336
yyforyongyu wants to merge 2 commits into
btcsuite:sql-walletfrom
yyforyongyu:task-manager-wallet-publication

Conversation

@yyforyongyu

@yyforyongyu yyforyongyu commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Change Description

Implements roadmap Task 453.

Serialize every Manager Create and cold Load operation through one
Manager-wide lock. The lock spans durable creation, or the Load cache
decision, through backend assembly and cache installation. Concurrent
same-name Loads therefore return one exact Wallet pointer, and Create cannot
be overtaken between durable commit and runtime installation.

The cache remains map[string]*Wallet and contains only fully assembled
Wallets. Different wallet names intentionally assemble sequentially; there
are no per-name entries, channels, transient nil values, or result latches.

Steps to Test

  1. Run: make unit pkg=wallet
  2. Run the concurrent Manager Load integration test for each backend:
    • make itest chain=btcd db=kvdb icase="manager load concurrent"
    • make itest chain=btcd db=sqlite icase="manager load concurrent"
    • make itest chain=btcd db=postgres icase="manager load concurrent"
  3. Run: make lint-check

Pull Request Checklist

Testing

  • Your PR passes all CI checks.
  • Tests covering the positive and negative (error paths) are included.
  • Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

📝 Please see our Contribution Guidelines for further guidance.

Comment thread itest/manager_test.go Outdated
Comment thread itest/manager_test.go Outdated
Comment thread itest/manager_test.go
Comment thread itest/manager_test.go Outdated
Comment thread wallet/manager.go Outdated
Comment thread wallet/manager.go Outdated
Comment thread wallet/manager.go Outdated
Comment thread wallet/manager.go Outdated
Hold the Manager lock from durable Create or the Load cache
decision through runtime assembly and cache installation.

Keep only fully assembled Wallet pointers in the cache.
@yyforyongyu
yyforyongyu force-pushed the task-manager-wallet-publication branch from 1b1f436 to 27bc79f Compare August 18, 2026 17:31
@yyforyongyu yyforyongyu changed the title wallet: serialize runtime publication wallet: serialize Manager wallet assembly Aug 18, 2026
@saubyk saubyk added this to lnd v0.22 Aug 19, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in lnd v0.22 Aug 19, 2026
Comment thread wallet/manager_assembly_test.go Outdated
Comment thread wallet/manager_assembly_test.go Outdated
Comment thread itest/list_on_test.go Outdated
@yyforyongyu
yyforyongyu force-pushed the task-manager-wallet-publication branch from 27bc79f to cb61f0c Compare August 19, 2026 14:54
Create durable state with a setup Manager, then contend cold Loads
through a fresh Manager. Require every backend to return one exact
runtime pointer.
@yyforyongyu
yyforyongyu force-pushed the task-manager-wallet-publication branch from cb61f0c to 629af93 Compare August 19, 2026 15:11
@yyforyongyu
yyforyongyu marked this pull request as ready for review August 19, 2026 16:21

@Lrifton92 Lrifton92 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The race is real and the fix closes it. Worth saying that testManagerLoadConcurrent is genuinely discriminating rather than decorative: if two callers both assembled, each would return the runtime it built, so require.Same against first would fail — the assertion actually proves single assembly rather than just a consistent final state. Three things on the approach.

The lock is global, the documented race was per-name

The TODO this removes reads "concurrent Load calls for the same name are not serialized". The fix serializes every name: Load now holds the exclusive Manager lock across m.backend.load(...) (wallet/manager.go:381) and Create holds it across m.backend.create(...) (line 213). Both of those open stores, so wallet assembly across the whole Manager is now strictly one at a time, and a slow or hanging store open for one wallet blocks Load, Create and String for every other.

What makes this worth raising rather than shrugging at is where it lands. NewManager's own comment says "the legacy kvdb backend remains single-wallet until it is removed; SQL stores distinguish wallets by ID" — so the coarse lock is free on kvdb, which only ever holds one wallet, and costs exactly on the SQL backends that multi-wallet support exists for. A per-name guard (an in-flight map keyed by cfg.Name, or singleflight) would close the same race without the global chokepoint.

If the global lock is a deliberate simplification for now, that is a defensible call — but the new comment on wallets says "The Manager lock serializes Create and Load through assembly and cache installation" without noting it serializes across all names, and that is the part a future reader needs.

Create and Load now have different lock discipline

Load takes m.Lock(); defer m.Unlock(). Create takes m.Lock() at line 213 and unlocks manually on two paths — line 219 on the backend error, line 226 after installation. Both are correct today. But Create keeps going after the unlock (the ModeShell import), so it cannot simply defer, and that asymmetry is the fragile kind: any early return added between 213 and 226 later leaks the lock and wedges the Manager permanently, with no test that would catch it. Extracting the locked span into a small helper that can defer would make it structurally safe instead of safe-by-inspection.

Does Create need to consult the cache before installing?

Create never reads m.wallets[cfg.Name] before m.wallets[cfg.Name] = w. Under the new lock two concurrent Create calls for one name no longer interleave, but the second still overwrites the first's entry, orphaning a runtime the first caller is holding and may already have started — and the same applies to a Create following a Load of that name.

I could not settle from here whether the backends make that unreachable by failing the second create on a uniqueness constraint; if they do, this is a non-issue and worth a line in the comment saying so. If they do not, then "installation" clobbering a live entry seems in scope for a PR about making installation safe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants