CI: decouple multi-arch image publishing into a standalone GHCR pipeline#1087
Open
aatchison wants to merge 7 commits into
Open
CI: decouple multi-arch image publishing into a standalone GHCR pipeline#1087aatchison wants to merge 7 commits into
aatchison wants to merge 7 commits into
Conversation
Replace the QEMU-emulated `docker buildx --platform amd64,arm64` build in the deploy-stage workflow with a per-arch native-runner strategy (adapted from sredevopsorg/multi-arch-docker-github-workflow): - New reusable workflow build-multiarch-image.yml builds one arch per native runner (ubuntu-latest for amd64, ubuntu-24.04-arm for arm64), pushes each to ECR by digest, then stitches them into a manifest list with `docker buildx imagetools create`. Emitted tag is unchanged. - merge.yml now calls it once for accounts and once for keycloak, gated on the same detect-changes outputs, and the deploy job consumes the resulting manifest tags instead of building inline (QEMU/buildx/ECR-login steps dropped from deploy). Native builds run in parallel and skip emulation overhead. The published tags stay multi-arch manifest lists, so release.yml's retag step and the ECS (amd64) / EKS Graviton (arm64) runtimes are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lets this branch exercise the split-arch build (and stage deploy). Revert before merging. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
For collecting evidence that the native-runner arm64/amd64 builds work, before wiring up ECR/deploy: - merge.yml: trigger on every push; deploy job disabled via an always-false condition (cascades to skip create-release + e2e); drop `secrets: inherit` from the build caller jobs. - build-multiarch-image.yml: build each arch natively but do NOT push; AWS/ECR steps, digest export/upload, and the imagetools merge are commented out. Merge job neutralized to a no-op. This also resolves the ci-cd-trust-boundary finding for the branch trigger: with no secrets:inherit and no role assumption on the build path, and deploy disabled, no secret-bearing job runs on branch pushes. All changes are marked TEMPORARY and reverted before merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reworks PR to separate image publishing from deployment: - merge.yml: revert the deploy-stage build to single-arch amd64 -> ECR (removes the QEMU/Buildx multi-arch build added in #1027/#974). Deploy behavior is otherwise unchanged; change detection stays its own job. - publish-images.yml (new): standalone pipeline that, on merge to main, detects which image changed (separate detect-changes job), reads the version, and builds accounts + keycloak as native multi-arch manifest lists pushed to GHCR. No cloud creds, no Pulumi, no deploy. - build-multiarch-image.yml: reusable native split-arch builder, now targeting GHCR via GITHUB_TOKEN. Builds amd64 (ubuntu-latest) and arm64 (ubuntu-24.04-arm) on native runners, pushes by digest, merges into one manifest list tagged :<sha>, :v<version>, and :latest. Separate GHCR packages per image. - docs/ci/image-publish-pipeline.md: mermaid architecture diagram of both pipelines and why they're decoupled. Decoupling keeps cloud credentials out of the image-publish path (resolves the earlier ci-cd-trust-boundary concern) and moves arm64 off QEMU onto native Graviton runners. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
aatchison
changed the title
publish-images.yml (change detection): - accounts-changed now includes keycloak/themes/** -- the accounts frontend imports the shared keycloak theme via the @kc vite alias, so `npm run build` bakes it into the accounts image; theme edits must rebuild accounts too. - fix vite.config.mjs -> vite.config.mts typo; add tsconfig.json, env.d.ts (all COPYd by the Dockerfile) so they trigger rebuilds. - broaden scripts/entry.sh -> scripts/** (Dockerfile COPYs the whole dir). - drop README.md as a trigger (docs-only, no runtime effect) to avoid a full multi-arch republish + :latest churn on doc changes. - concurrency cancel-in-progress: false so a merged commit's :<sha> image is never dropped and a run is never aborted mid-push. build-multiarch-image.yml (build): - --provenance=false so each per-arch push is a plain single manifest (clean 2-entry merged manifest list, no attestation index). - add GitHub Actions layer cache (type=gha) scoped per image+arch; ephemeral runners otherwise rebuild every layer on every merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Same change-detection gaps the GHCR pipeline review surfaced, applied to the deploy pipeline's src-changed filter (which gates the accounts stage deploy): - fix vite.config.mjs -> vite.config.mts typo; add tsconfig.json + env.d.ts (all COPYd by the Dockerfile). - add scripts/** (Dockerfile COPYs the whole dir; was only scripts/entry.sh). - add keycloak/themes/** -- the accounts bundle imports the shared keycloak theme via the @kc vite alias, so a theme-only change previously rebuilt the keycloak image but NOT accounts, deploying a stale accounts frontend. Additive only (never under-builds); no change to deploy behavior otherwise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update and SHA-pin every third-party action in the two new workflows (build-multiarch-image.yml, publish-images.yml). No existing workflow touched. actions/checkout v4 -> v7.0.0 docker/login-action v3 -> v4.4.0 docker/setup-buildx-action v3 -> v4.2.0 actions/upload-artifact v4 -> v7.0.1 actions/download-artifact v4 -> v8.0.1 (v8 pairs with upload v7) dorny/paths-filter v3 -> v4.0.2 SebRollen/toml-action v1.2.0 (pinned; already latest) Each is pinned to the commit SHA the release tag resolves to, with the version in a trailing comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Splits container-image publishing out of the deploy pipeline into a standalone GHCR pipeline, and moves the multi-arch (amd64 + arm64) build onto native runners (no QEMU) using the split-worker strategy from sredevopsorg/multi-arch-docker-github-workflow.
Two independent pipelines
merge.ymlpublish-images.yml+build-multiarch-image.ymlSee
docs/ci/image-publish-pipeline.mdfor a mermaid diagram of both.Changes
merge.yml— reverts the deploy build to single-arch amd64 → ECR (undoing the QEMU/Buildx multi-arch added in ci: build accounts image multi-arch (amd64+arm64) for Graviton EKS #1027 / ci(keycloak): build the keycloak image multi-arch (amd64 + arm64) #974). Deploy, artifacts, anddetect-changes(its own job) are otherwise unchanged.publish-images.yml(new) — on merge tomain(orworkflow_dispatch): a separatedetect-changesjob decides which image changed and reads the version, then buildsaccountsandkeycloakto GHCR. No cloud credentials, no Pulumi.build-multiarch-image.yml— reusable native split-arch builder targeting GHCR viaGITHUB_TOKEN: amd64 onubuntu-latest, arm64 onubuntu-24.04-arm, pushed by digest, merged into one manifest list tagged:<sha>,:v<version>,:latest. Separate GHCR packages per image (.../thunderbird-accounts,.../thunderbird-accounts-keycloak).docs/ci/image-publish-pipeline.md— architecture diagram + rationale.Verified
An earlier revision of this branch ran the native builds green (all four amd64/arm64 build jobs succeeded; arm64 on native Graviton runners ~165s for accounts).
actionlintpasses on all three workflows.Reverting
merge.ymlto amd64-only means ECR no longer gets an arm64 image. The Graviton/arm64 EKS need that #1027/#974 addressed is now served only from GHCR. This is intentional per the decoupling, but anything currently pulling the arm64 variant from ECR must be repointed to GHCR (or ECR multi-arch restored) before those consumers rely on it. Flagging so it's a conscious decision.Action versions (reviewer note)
All third-party actions in the two new workflows are pinned to a commit SHA (with a
# vX.Y.Zcomment); no existing workflow was touched. Getting to "latest" meant some notable major bumps that differ from thev3/v4tags the rest of the repo pins:v4v3v3v4v4v3v1.2.0Low-risk for our simple usage, and
upload-artifact@v7↔download-artifact@v8are the intended compatible pair (v8 supports v7's direct-upload feature). Flagging in case reviewers prefer version consistency with the rest of the repo — any single one is easy to dial back.Follow-up
Deploy-side companion (pull the accounts image from GHCR on the Graviton EKS clusters): thunderbird/accounts-deploy#25 (draft) — blocked on this PR merging + the GHCR images publishing.
Notes
release.ymlis left as-is: itsimagetoolsretag (from ci: build accounts image multi-arch (amd64+arm64) for Graviton EKS #1027) works fine on a single-arch amd64 image, so no revert needed there.🤖 Generated with Claude Code