docs(send): R2 staging → B2 replication design & feasibility (#959)#961
Draft
aatchison wants to merge 3 commits into
Draft
docs(send): R2 staging → B2 replication design & feasibility (#959)#961aatchison wants to merge 3 commits into
aatchison wants to merge 3 commits into
Conversation
…tion (#959) Two design-first deliverables for issue #959, no production code changes: - R2StagingFeasibility.md: evaluates the four feasibility caveats (job infrastructure, replication byte-path cost, client rollout, storage-layer refactor), prices three coherent architecture bundles at 1/10/50 TB-uploaded/month, and recommends the Cloudflare Worker replication path. All pricing verified against official vendor sources on 2026-07-06. - R2StagingReplicationDesign.md: full implementation blueprint for the selected design - copier Worker pipeline, event/queue plumbing, backend integration points (file:line), IaC/CI/secrets plan, failure-mode table with testable data-loss invariants, risk register, sequenced delivery plan gated on two Phase 0 prototypes, and the open decisions requiring human sign-off. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review fixes: transient-vs-terminal split at the flip endpoint (no transient poisoning of 'failed'), B2-aware aged-404/'gone' paths (no permanent B2 orphans, no sweep livelock), aws4fetch retries:0, encrypted-vs-plaintext size correction (calculateEncryptedSize), CREATE INDEX CONCURRENTLY, COPIER_HMAC_SECRET blast radius, region canary prototype (P0), rollout success metrics, corrected evidence ledger rows, GDPR scoping (Cloudflare as subprocessor, EU jurisdiction = at-rest only), redacted live-vulnerability framing (tracked privately), and readability fixes for cold readers. Maintainer feedback (#959 comments): B2-first with R2 fallback after retry exhaustion (UPLOADS_R2_MODE=off|fallback|all), grace delete 6h→1h (minutes-level Cloudflare residency), P2 gains throughput + billing assertions, success metric reframed as fallback rescue rate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ards Lead both notes with the goal: make Send uploads succeed when a direct B2 upload stalls/fails (UPLOAD_FAILED, #959). Add a Goal + Non-goals block to the design note clarifying that R2 is an upload-rescue endpoint (B2-first, R2 only on failure), B2 remains the sole serving tier, and download-from-R2 is an optional pre-replication stopgap, not a goal. Corrects the earlier Background wording that implied all uploads route through R2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Design-first deliverable for #959 — no production code changes. Two notes in
packages/send/docs/(following theResumableUploadDesign.mdprecedent):R2StagingFeasibility.md— evaluates the four feasibility caveats (background-job infra, replication byte-path cost, client rollout, storage-layer refactor), prices three architecture bundles at 1/10/50 TB-uploaded/month, and recommends the Cloudflare Worker replication path. Core finding: any AWS-resident replicator pays$0.09/GB egress out of eu-central-1 ($922/mo at 10 TB), while a Worker moves the same bytes for ~$5–15/mo flat — R2 egress is free and Cloudflare bills storage on peak-per-day, so minutes-level staging residency costs near-zero.R2StagingReplicationDesign.md— the implementation blueprint: B2-first upload with R2 fallback, the copier-Worker pipeline, R2-event→Queues plumbing, backend integration points down tofile:line, IaC/CI/secrets, a failure-mode table with testable data-loss invariants (I1–I10), a delivery plan (14 workstreams) gated on three Phase 0 prototypes, and the open decisions needing human sign-off.All platform/pricing claims were verified against official vendor docs (2026-07-06); claims unverifiable on paper are marked ⚠ and gated on the Phase 0 prototypes (design note §7).
Incorporates maintainer feedback from #959
UPLOADS_R2_MODE=off|fallback|all): clients PUT to B2 as today and only fall back to R2 after the retry budget exhausts. This makes every fallback a same-file/same-client A/B proof that R2 resolves B2 failures, and collapses R2 cost/volume to the failing fraction.Review notes
This went through a multi-lens review before publishing; two substantive fixes are already folded in: a transient-vs-terminal split at the flip endpoint (a momentary B2 blip could otherwise poison a fully-copied file into a lifecycle-deletable
failedstate), and B2-aware orphan handling (no permanent B2 orphan of a user-deleted file; no reconciliation livelock).Suggested review focus:
r2 | b2 | failedpointer states.A few pre-existing, unrelated hardening items surfaced during the investigation (telemetry hygiene, upload-size enforcement, rate limiting) are tracked separately in the private issue tracker rather than here.
Informs #959; closes nothing.
🤖 Generated with Claude Code