multi: add HashiCorp Vault KV v2 secret support#92
Open
Roasbeef wants to merge 2 commits into
Open
Conversation
In this commit, we add `vault` as a third secret source and target alongside the existing `file` and `k8s` backends, wired into the `init-wallet`, `store-secret` and `load-secret` commands. This lets an lnd deployment keep its wallet seed and password in HashiCorp Vault directly, so the secrets never need to be materialized as a Kubernetes Secret in etcd. The implementation targets the KV v2 secrets engine using the official `vault/api` KV v2 helper: reads and writes go through `client.KVv2(mount)`, which handles the nested `data/` path shaping that KV v2 requires. Writes are read-modify-write merges so that storing one entry (e.g. the seed) never clobbers a sibling entry (e.g. the password) in the same secret. The `errTargetExists` sentinel is preserved on the write path so the existing idempotency contract (`--error-on-existing`) keeps working across restarts, which is important to make sure we never silently rotate a seed out from under an initialized wallet. Authentication uses the Kubernetes auth method via the official `vault/api/auth/kubernetes` login helper. The pod's projected ServiceAccount token is exchanged for a Vault token bound to a role and policy. The auth mount, role and token path are all configurable; the Vault server address is taken from the `VAULT_ADDR` environment variable or the `--vault.addr` flag. A companion `example-init-wallet-vault.sh` and a README section round out the change.
In this commit, we add the design doc that motivates the KV v2 Vault variant: how our production Vault in lightning-infra is actually set up (KV v2, Kubernetes auth, VSO), the gap versus the earlier KV v1 approach, the locked design decisions, and the companion infra wiring needed to roll it out.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR, we add
vaultas a third secret source and target alongside the existingfileandk8sbackends, wired into theinit-wallet,store-secretandload-secretcommands. This lets an lnd deployment keep its wallet seed and password in HashiCorp Vault directly, so the secrets never need to be materialized as a Kubernetes Secret in etcd.This supersedes #62, which was a straight port of an older
lnd-fork branch written against a KV v1 Vault with a hand-rolled Kubernetes login. Our production Vault targets the KV v2 secrets engine, so that version wouldn't work as-is: a KV v2 read comes back with the values nested underdata, which the KV v1Logical().Readpath doesn't unwrap.What's here
client.KVv2(mount), which handles the nesteddata/path shaping KV v2 requires. Writes are read-modify-write merges, so storing one entry (e.g. the seed) never clobbers a sibling entry (e.g. the password) in the same secret. UsingPuton a merged map (rather thanPatch) keeps the required policy to justcreate/read/update.errTargetExistssentinel is kept on the write path, so--error-on-existingkeeps working across restarts. This matters: we never want to silently rotate a seed out from under an already-initialized wallet.vault/api/auth/kuberneteslogin helper. The pod's projected ServiceAccount token is exchanged for a Vault token bound to a role and policy. The auth mount, role and token path are configurable; the server address comes fromVAULT_ADDRor--vault.addr.example-init-wallet-vault.shstartup script, and avault_test.gosuite (a mock KV v2 + k8s-auth server covering round-trip, no-clobber merge, the overwrite guard, newline trimming, and the not-found/missing-key/empty-value error paths).The
docs/vault-integration-design.mdfile captures how our production Vault inlightning-infrais set up (KV v2, Kubernetes auth, VSO), the gap versus the KV v1 approach, and the companion infra wiring (an lnd policy scoped tosecret/data/lnd/*, a k8s auth role, and the lnd chart init-script) that will follow in a separate lightning-infra PR.