[LFXV2-2190, LFXV2-2193] fix: lazy-bind weekly-brief KV buckets and fix query-service version param - #167
Conversation
Pods that started before the group-weekly-briefs buckets were provisioned held a nil handle permanently (one-shot bind at startup, no retry), causing GET /weekly-briefs/current to 503 for the pod's entire lifetime. Replace the best-effort startup loop with a GetOrBindKVStore method that binds on first request access under a RWMutex, so any pod self-heals the moment the bucket exists without requiring a rolling restart. Generated with [Claude Code](https://claude.ai/code) Signed-off-by: Andres Tobon <andrest2455@gmail.com>
WalkthroughThe NATS client now synchronizes lazy KV-store binding. Weekly-brief storage uses this resolver for reads and writes. Mailing-list and meeting queries add ChangesNATS KV binding
M2M API versioning
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant WeeklyBriefStorage
participant NATSClient
participant JetStream
WeeklyBriefStorage->>NATSClient: GetOrBindKVStore(bucketName)
NATSClient->>NATSClient: Check cached bucket under read lock
NATSClient->>JetStream: Bind missing bucket
JetStream-->>NATSClient: Return KeyValue bucket
NATSClient-->>WeeklyBriefStorage: Return bucket or ServiceUnavailable
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Pull request overview
Adds lazy binding for weekly-brief NATS KV buckets so pods recover without restarting.
Changes:
- Adds mutex-protected first-access KV binding.
- Routes weekly-brief storage operations through the new binding method.
- Removes best-effort startup binding.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
internal/infrastructure/nats/client.go |
Implements lazy KV binding and removes startup binding. |
internal/infrastructure/nats/group_weekly_brief_storage.go |
Uses lazy binding for weekly-brief buckets. |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/infrastructure/nats/client.go`:
- Around line 88-117: Update GetOrBindKVStore so the potentially blocking
js.KeyValue call runs without holding c.kvMu. Recheck the cache before binding,
perform the bind outside the global lock, then reacquire the lock to store the
handle while preserving an already-cached handle if another request won the
race; use per-bucket coordination only if duplicate binds must be prevented.
- Around line 91-115: Synchronize every access to c.kvStore consistently across
GetOrBindKVStore, KeyValueStore, and the storage methods that read it directly.
Protect all lookups and writes with kvMu, or route them through a shared
synchronized accessor, while preserving the existing caching and binding
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7c43db14-e434-4689-9a43-8a750bac04ad
📒 Files selected for processing (2)
internal/infrastructure/nats/client.gointernal/infrastructure/nats/group_weekly_brief_storage.go
All three query-service source clients (meetings, mailing lists, votes) were omitting the required version=1 query parameter, causing the query-service to return HTTP 400 on every request. With MaxDeliver:3, all three NATS delivery attempts would exhaust and the weekly brief would be permanently orphaned in the "generating" state. Generated with [Claude Code](https://claude.ai/code) Signed-off-by: Andres Tobon <andrest2455@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Suppressed comments (1)
internal/infrastructure/nats/client.go:115
- This mutates
kvStorewhile most storage paths still read that same map directly withoutkvMu(for example,storage.go:40andstorage.go:114). A first weekly-brief bind concurrent with any ordinary committee request can therefore race and triggerconcurrent map read and map write. Keep lazy handles in a separate mutex-protected map, or route every access tokvStorethrough the same lock before writing it at runtime.
c.kvStore[bucketName] = kv
- Add kvStoreLazy field: lazy weekly-brief handles are kept separate from the startup-populated kvStore, which is written only during NewClient (single-threaded). The 50+ direct kvStore reads in storage.go, link_storage.go, document_storage.go etc. remain race-free; no data race between GetOrBindKVStore writes and those reads can occur. - Move js.KeyValue outside kvMu: the NATS bind call is a network op; holding the write lock while it ran blocked concurrent cache-hit RLock lookups. The bind now runs unlocked; the lock is reacquired only to store the result, with a double-check to discard a duplicate bind if another goroutine won the race. LFXV2-2190 Generated with [Claude Code](https://claude.ai/code) Signed-off-by: Andres Tobon <andrest2455@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (3)
internal/infrastructure/m2m/meeting_source.go:100
- This required parameter is missing from
MeetingSource’s documented request at lines 40–41, which still shows/query/resources?type=.... That stale example documents the same invalid request shape this change fixes; please addversion=1to it so the adapter contract matches the request sent here.
q.Set("version", "1")
internal/infrastructure/m2m/mailing_list_source.go:87
- This required parameter is missing from
MailingListSource’s documented request at lines 36–37, which still shows/query/resources?type=.... That stale example documents the same invalid request shape this change fixes; please addversion=1so the adapter contract matches the request sent here.
q.Set("version", "1")
internal/infrastructure/m2m/vote_source.go:87
- This required parameter is missing from
VoteSource’s documented request at lines 36–37, which still shows/query/resources?type=.... That stale example documents the same invalid request shape this change fixes; please addversion=1so the adapter contract matches the request sent here.
q.Set("version", "1")
The query-service Goa design uses dsl.Param("version:v"), mapping the
Go attribute "version" to the HTTP query parameter "v". The generated
server reads qp.Get("v"), so sending version=1 was silently ignored and
the 400 error persisted. Changed all three m2m source clients to send
v=1 instead.
LFXV2-2193
Generated with [Claude Code](https://claude.ai/code)
Signed-off-by: Andres Tobon <andrest2455@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (4)
internal/infrastructure/nats/client.go:98
- The lazy-bind and synchronization path has no regression coverage; the only weekly-brief NATS test covers key formatting. Add a co-located test using an in-process JetStream server that verifies a failed lookup succeeds after the bucket is created and that concurrent first accesses are race-free. Otherwise the startup-race fix and its cache locking can regress while the suite remains green.
func (c *NATSClient) GetOrBindKVStore(ctx context.Context, bucketName string) (jetstream.KeyValue, error) {
internal/infrastructure/m2m/meeting_source.go:100
- The existing
TestMeetingSource_M2MTokenUsed_NotCallerBearerexercises this request but never asserts the required API-version query parameter. Extend itshttptesthandler (or add a table case) to requirer.URL.Query().Get("v") == "1"; without that assertion, removing this line recreates the reported HTTP 400 failure while tests still pass.
q.Set("v", "1")
internal/infrastructure/m2m/vote_source.go:87
- No test exercises the vote source request, so the newly required
v=1contract is unprotected. Add a co-locatedhttptest-based method test that rejects requests unlessr.URL.Query().Get("v") == "1"; otherwise this exact HTTP 400 regression can return unnoticed.
q.Set("v", "1")
internal/infrastructure/m2m/mailing_list_source.go:87
- No test exercises the mailing-list source request, so the newly required
v=1contract is unprotected. Add a co-locatedhttptest-based method test that rejects requests unlessr.URL.Query().Get("v") == "1"; otherwise this exact HTTP 400 regression can return unnoticed.
q.Set("v", "1")
Summary
GetOrBindKVStoremethod that binds on first request access under aRWMutexversion=1query parameter to all three m2m source clients (meetings, mailing lists, votes); the query-service requires this param and was returning HTTP 400, causing NATS to exhaustMaxDeliver:3and orphan every brief permanently ingeneratingstateHow it works
GetOrBindKVStoreuses a double-checked locking pattern: read-lock fast path for the common case (bucket already bound), write-lock slow path for first-access binding. Once bound, subsequent requests take the fast path. The throttle bucket inGetGroupWeeklyBriefForWindowretains its best-effort semantics — a bind failure there is still silently skipped since throttle data is advisory.Tickets
LFXV2-2190 — committee-service: GET weekly-briefs/current 503s "bucket not initialized" though bucket exists + writes persist (read-path bug)
LFXV2-2193 — committee-service: weekly brief stuck in "generating" state; all source fetches returning 400 due to missing version=1 query param
🤖 Generated with Claude Code