feat(devcontainer): derive the base-image pin instead of bumping it

.claude/CLAUDE.md

@@ -116,12 +116,13 @@ the `Renovate helper`. Load-bearing facts:
   make it a silent no-op.
 - **`.bazelversion` invalidates the lock too.** `MODULE.bazel.lock`'s `lockFileVersion` (and the
   shape of its recorded extensions) tracks the bazel release, so a Renovate bazel bump leaves the
-  committed lock stale. CI hides it — `--lockfile_mode=update` rewrites in memory and stays green —
-  but the `bazel mod tidy` pre-commit hook rewrites it on disk, so the staleness surfaces as a
-  blocked local commit on any `go.mod`/`MODULE.bazel` change. The workflow therefore triggers on
-  `.bazelversion` and folds it into the same `bazel` classification as `MODULE.bazel`: one output,
-  one `bazel mod deps` refresh. bazelisk reads the checked-out `.bazelversion`, so the regenerated
-  lock is in the bumped version's format.
+  committed lock stale. Builds don't notice — `--lockfile_mode=update` rewrites in memory and stays
+  green — so the staleness surfaces either as a blocked local commit (the `bazel mod tidy`
+  pre-commit hook rewrites it on disk on any `go.mod` change) or as ci.yml's `MODULE.bazel.lock
+  freshness` job, which is the backstop for commits that ran no hook. The workflow therefore
+  triggers on `.bazelversion` and folds it into the same `bazel` classification as `MODULE.bazel`:
+  one output, one `bazel mod deps` refresh. bazelisk reads the checked-out `.bazelversion`, so the
+  regenerated lock is in the bumped version's format.
 - **Go tidy/sync rides the same commit.** Renovate's `go get` bumps `go.mod`/`go.sum` but never
   runs `go mod tidy` (opt-in) or `go work sync` (Renovate does it only when vendoring, which this
   repo doesn't) — so the indirect block and `go.work.sum` are left stale. The workflow runs
@@ -134,6 +135,12 @@ the `Renovate helper`. Load-bearing facts:
   uv path, there's no "conflict" review: a bump `go mod tidy` can't settle just fails the job.
   MODULE.bazel.lock is *not* in this set — gazelle's `go_deps` extension is reproducible and absent
   from the lockfile, and `use_repo` tracks only direct imports (unchanged by a version bump).
+- **The devcontainer base-image pin rides it too.** `.devcontainer/Dockerfile`'s `FROM` digest
+  is derived from `//meta/devcontainer-base:image`, which is assembled over the
+  `devcontainers_base_debian` pull — so a `MODULE.bazel` bump restales it. The workflow rebuilds
+  the image and rewrites the pin (`meta/scripts/sync_base_image_pin.py`) *after* `bazel mod deps`,
+  never before: that step needs a cold output base, and the build would warm it. Runs
+  `--config=local`, since this job carries no BuildBuddy key.
 - **Devcontainer feature lock rides it too.** `devcontainer upgrade` reruns when
   `devcontainer.json` moves. Like Go, it is *independent* of the uv→Bazel ordering and shares the
   job only so a grouped PR settles in one `expectedHeadOid` mutation. Needs no Docker (OCI metadata
@@ -213,6 +220,15 @@ What is local to this repo:
   top. The `BASE_IMAGE` override and the exact three-line shape it needs are under "Consuming the
   image" in that README; `.devcontainer/test_devcontainer_config.py` asserts the couplings, because
   every wrong shape fails at container-build time or not at all.
+- **That pin is a derived file, not a dependency.** The digest is reproducible from source, so the
+  PR that changes the image carries the new pin. Three callers, one per source of change: the
+  `base-image-pin` pre-commit hook for our edits, this workflow for Renovate's, and
+  `//.devcontainer:test_base_image_pin` as the check under both — hooks are bypassable and the
+  workflow only fires for Renovate's own PRs. Renovate is configured to ignore the dep. The cost
+  is that a base-editing branch pins an image the registry doesn't have yet; set
+  `DEVCONTAINER_BASE_IMAGE` to the published `:latest` to keep working. Don't reach for
+  `bazel run :load` locally — it needs a Docker daemon the devcontainer doesn't have, which is why
+  that path is CI's.
 - **`.devcontainer/initialize.sh` is the host stub** — the read-and-drop half the image cannot
   carry, since it runs on the host before any container exists. It writes `.git-plumbing/` and the
   `.host-*` symlinks `devcontainer.json` binds.

.devcontainer/BUILD.bazel

@@ -25,4 +25,22 @@ py_test(
         "//:.github/workflows/devcontainer.yml",
     ],
     main = "test_devcontainer_config.py",
+    deps = ["//meta/scripts:sync_base_image_pin_lib"],
+)
+
+# The pin is a derived file (see meta/scripts/sync_base_image_pin.py). The `base-image-pin`
+# pre-commit hook keeps it fresh; this is the half that cannot be bypassed — hooks don't run for
+# `--no-verify`, a web edit, or the helper app's API commits, and don't re-run on a rebase.
+# Asserting it needs only the assembled image, which Bazel produces without a daemon, so it rides
+# `bazel test //...` instead of costing a CI job of its own.
+py_test(
+    name = "test_base_image_pin",
+    size = "small",
+    srcs = ["test_base_image_pin.py"],
+    data = [
+        "Dockerfile",
+        "//meta/devcontainer-base:image",
+    ],
+    main = "test_base_image_pin.py",
+    deps = ["//meta/scripts:sync_base_image_pin_lib"],
 )

.devcontainer/Dockerfile

@@ -3,7 +3,7 @@
 # canonical home for the mechanism, this three-line override shape, and why the alias is
 # load-bearing — do not collapse it without reading "Consuming the image" there.
 ARG BASE_IMAGE=pinned-base
-FROM ghcr.io/syndic/unnatural_designs-devcontainer-base:latest@sha256:99e1405fcb245513ab0e033c51b190ee9b27377d0787f89f73dcea6a405d94cb AS pinned-base
+FROM ghcr.io/syndic/unnatural_designs-devcontainer-base:latest@sha256:0f31f38a212f69b1831010aad682bda8f99a6f8a6b2f975aef5b9715331d1376 AS pinned-base
 FROM ${BASE_IMAGE}
 
 # renovate: datasource=github-releases depName=bazelbuild/bazelisk

.devcontainer/test_base_image_pin.py

@@ -0,0 +1,45 @@
+"""Asserts the Dockerfile pins the base image this tree actually builds.
+
+The pin is derived, not depended on: the digest is a pure function of `meta/devcontainer-base/`
+plus the upstream base in `MODULE.bazel`, so a change to either restales it in the same commit.
+The `base-image-pin` pre-commit hook rewrites it, exactly as `bazel mod tidy` does for
+`MODULE.bazel.lock`. This is the backstop under that hook: hooks don't run for `--no-verify`, a
+web edit, or the helper app's API commits, and don't re-run when a branch rebases onto someone
+else's base change — two individually-fresh pins can be jointly stale. It can be a plain test
+rather than a CI job because the digest falls out of an already-built artifact and needs no
+daemon.
+
+Without it, a stale pin is invisible: the devcontainer still builds, from the previous image, and
+the difference only shows up as plumbing that mysteriously predates your change.
+"""
+
+import sys
+import unittest
+from pathlib import Path
+
+from meta.scripts.sync_base_image_pin import index_digest, pinned_digest
+
+# Not .resolve(): the image is a generated cross-package data dep, so it lives in the runfiles
+# tree beside this file rather than in the source tree a resolved symlink leads back to.
+_HERE = Path(__file__).parent
+_DOCKERFILE = _HERE / "Dockerfile"
+_LAYOUT_INDEX = _HERE.parent / "meta" / "devcontainer-base" / "image" / "index.json"
+
+
+class TestBaseImagePinIsFresh(unittest.TestCase):
+    def test_pin_matches_the_built_image(self):
+        built = index_digest(_LAYOUT_INDEX.read_text(encoding="utf-8"))
+        current = pinned_digest(_DOCKERFILE.read_text(encoding="utf-8"))
+        self.assertEqual(
+            current,
+            built,
+            "The devcontainer's base-image pin is stale. Run:\n"
+            "  bazel build //meta/devcontainer-base:image && "
+            "python3 meta/scripts/sync_base_image_pin.py\n"
+            "and commit .devcontainer/Dockerfile. The digest is reproducible, so this is the "
+            "one a merge will publish.",
+        )
+
+
+if __name__ == "__main__":
+    sys.exit(0 if unittest.main(exit=False).result.wasSuccessful() else 1)

.devcontainer/test_devcontainer_config.py

@@ -22,6 +22,8 @@
 import unittest
 from pathlib import Path
 
+from meta.scripts.sync_base_image_pin import pinned_digest
+
 # Not .resolve(): devcontainer.yml is a cross-package data dep and lives in the runfiles tree,
 # which a resolved symlink would lead back out of. The rest read fine either way.
 _HERE = Path(__file__).parent
@@ -257,11 +259,15 @@ def test_sentinel_is_a_stage_alias_on_the_pinned_base(self):
     def test_pinned_base_carries_both_a_tag_and_a_digest(self):
         # Digest for reproducibility, tag for Renovate to have something to compare against —
         # the same pairing MODULE.bazel's oci.pull uses for this image's own base.
+        #
+        # The digest half comes from sync_base_image_pin, which owns the pin and rewrites it;
+        # a second regex here would be the copy that silently stops agreeing with the writer.
         _, sentinel = self._global_arg_default()
         image = next(img for _, img, alias in self.froms if alias == sentinel)
-        reference, _, digest = image.partition("@")
-        self.assertRegex(digest, r"\Asha256:[0-9a-f]{64}\Z")
+        reference, _, _ = image.partition("@")
         self.assertRegex(reference, rf"\A{re.escape(_BASE_REPOSITORY)}:[\w][\w.-]*\Z")
+        # Raises if the Dockerfile carries no single well-formed pinned FROM.
+        pinned_digest(_DOCKERFILE.read_text(encoding="utf-8"))
 
     def test_alias_precedes_the_consuming_from(self):
         # Renovate's stage-name check only knows aliases declared above the line it is looking

.github/workflows/ci.yml

@@ -25,6 +25,32 @@ jobs:
           buildbuddy-api-key: ${{ secrets.BUILDBUDDY_API_KEY }}
       - run: bazel run --config=ci //:gazelle -- -mode=diff
 
+  # MODULE.bazel.lock is the one derived file with no CI backstop of its own: builds run with
+  # `--lockfile_mode=update`, which rewrites the lock in memory and stays green, so a `bazel mod
+  # tidy` hook that never ran is invisible until it blocks someone's commit. Fail-only on
+  # purpose — the regenerated lock belongs in the authoring commit, not in a CI push.
+  #
+  # `bazel mod tidy` also rewrites MODULE.bazel's `use_repo` lines, so both files are diffed.
+  bazel-lock-check:
+    name: MODULE.bazel.lock freshness
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: ./.github/actions/setup-bazel-remote
+        with:
+          buildbuddy-api-key: ${{ secrets.BUILDBUDDY_API_KEY }}
+      - run: |
+          set -euo pipefail
+          # `--config=ci` for consistency with gazelle-check next door. It changes nothing
+          # here — `mod tidy` executes no actions, so BES and remote execution are moot.
+          bazel mod tidy --config=ci
+          if ! git diff --exit-code -- MODULE.bazel MODULE.bazel.lock; then
+            MSG="stale — run 'bazel mod tidy' and commit the result."
+            echo "::error file=MODULE.bazel.lock::${MSG}"
+            exit 1
+          fi
+          echo "MODULE.bazel and MODULE.bazel.lock are what 'bazel mod tidy' produces."
+
   # The repo's `meta/scripts/*.py` target Python 3.14 (matches the rules_python
   # toolchain pin in MODULE.bazel and the `target-version` in pyproject.toml's
   # [tool.ruff]). Ubuntu-24.04 still ships 3.12 as `python3`, so the language-

.github/workflows/devcontainer.yml

@@ -57,6 +57,19 @@ jobs:
           # `changed` too so the automerged bump is gated by a real consumer build, which is
           # what `publish` needs.
           #
+          # `.bazelversion` is in both for a narrower reason: renovate-derived-files.yml
+          # classifies it as a bazel change and re-derives the pin from it, so leaving it out
+          # here would let a bump write a pin that no job ever publishes — a red PR with no way
+          # to green it. The two classifications have to agree on what feeds the image.
+          #
+          # This one is the expensive direction, deliberately, and it cuts against the
+          # false-positive argument below: the bazel version cannot enter `pkg_tar` layers over
+          # a pulled base, so every bazelisk bump buys a publish of a byte-identical index. The
+          # cheap alternative — drop it from the re-derivation instead and let
+          # //.devcontainer:test_base_image_pin fail if the digest ever does move — leaves the
+          # recovery path needing a config change at the moment someone is already confused.
+          # A recurring known cost beats a rare unrecoverable one.
+          #
           # Deliberately the whole file, not a grep for the pull's name. MODULE.bazel moves on
           # roughly 1 in 6 commits here and most of those cannot touch the image, so the false
           # positives are real and not cheap: two base-image runners, plus a consumer build
@@ -68,8 +81,8 @@ jobs:
           # build when both are true.
           python3 meta/scripts/classify_changed_paths.py \
             --base "$base" \
-            --rule 'changed=^\.devcontainer/|^meta/devcontainer-base/|^MODULE\.bazel$|^\.github/workflows/devcontainer\.yml$' \
-            --rule 'base=^meta/devcontainer-base/|^MODULE\.bazel$'
+            --rule 'changed=^\.devcontainer/|^meta/devcontainer-base/|^MODULE\.bazel$|^\.bazelversion$|^\.github/workflows/devcontainer\.yml$' \
+            --rule 'base=^meta/devcontainer-base/|^MODULE\.bazel$|^\.bazelversion$'
 
   # ── Shared base image ───────────────────────────────────────────────────────────────
   # meta/devcontainer-base/ is published for other Syndic repos to FROM. Bazel assembles it and
@@ -368,3 +381,53 @@ jobs:
           set -euo pipefail
           bazel run --config=ci //meta/devcontainer-base:push -- \
             --tag latest --tag "sha-${GITHUB_SHA}"
+
+      # The one loose end neither the pre-commit hook nor renovate-derived-files.yml can see:
+      # both run before the merge, and this is the step that can fail after it. `crane push`
+      # then one `crane tag` per tag is not atomic, so a failure in between leaves main pinned
+      # to a digest the registry does not serve — silent until someone rebuilds a devcontainer.
+      #
+      # Reads the pin with sync_base_image_pin.py rather than a grep of its own, so the digest
+      # has one parser and this cannot degrade into a check that always passes.
+      #
+      # Retried because GHCR does not document tag resolution as read-your-writes and this runs
+      # seconds after `crane tag`. A transient miss would turn main red for exactly the reason
+      # the step exists to rule out, and tell the reader to re-run a job that was fine.
+      - name: Verify the registry serves what main pins
+        run: |
+          set -euo pipefail
+          repo=ghcr.io/syndic/unnatural_designs-devcontainer-base
+          pinned="$(python3 meta/scripts/sync_base_image_pin.py --print-pinned)"
+          err="${RUNNER_TEMP}/imagetools-inspect.err"
+          attempts=3
+
+          # `@digest` is what a consumer's FROM resolves; the tags are what Renovate and any
+          # non-deriving consumer read, and they are the half `crane tag` can leave behind.
+          # One tool for all three: `imagetools inspect` takes a digest reference too, and
+          # `docker manifest` is still nominally an experimental CLI command.
+          for ref in "@${pinned}" ":latest" ":sha-${GITHUB_SHA}"; do
+            resolved=""
+            rc=0
+            for attempt in $(seq "$attempts"); do
+              # stderr and the exit status are both kept: propagation lag, an auth failure and
+              # an unreachable registry otherwise look alike, and a `timeout` kill writes no
+              # stderr at all, so the status is the only thing that names that one.
+              rc=0
+              resolved="$(timeout 60 docker buildx imagetools inspect \
+                --format '{{.Manifest.Digest}}' "${repo}${ref}" 2>"$err")" || rc=$?
+              [ "$resolved" = "$pinned" ] && break
+              # `||` rather than `&&`: a failing `&&` guard returns 1, which set -e traps.
+              [ "$attempt" -eq "$attempts" ] || sleep $((attempt * 5))
+            done
+            if [ "$resolved" != "$pinned" ]; then
+              echo "::error::${repo}${ref} resolved to '${resolved:-nothing}' after" \
+                   "${attempts} attempts, but main pins ${pinned}." \
+                   "Re-run this job — crane tags non-atomically."
+              hint=""
+              [ "$rc" -ne 124 ] || hint=" (timed out)"
+              echo "Last attempt exited ${rc}${hint}; its stderr follows:"
+              cat "$err"
+              exit 1
+            fi
+          done
+          echo "${repo}@${pinned} is published and both tags point at it."

.github/workflows/renovate-derived-files.yml

@@ -199,6 +199,26 @@ jobs:
         if: (steps.changed.outputs.bazel == 'true' || steps.lock.outputs.moved == 'true') && steps.ratify.outputs.conflicts == ''
         run: bazel mod deps --lockfile_mode=update
 
+      # ── Devcontainer base image: re-derive the consumer's pin ─────────────────────
+      # The shared base image is assembled from MODULE.bazel's `devcontainers_base_debian` pull,
+      # so a bump there changes the digest .devcontainer/Dockerfile pins. That digest is knowable
+      # here rather than after publishing: the build is reproducible, and `oci_push` publishes the
+      # exact index Bazel built. //.devcontainer:test_base_image_pin is the matching check.
+      #
+      # AFTER the lock refresh above, never before: `bazel mod deps` only rewrites the pip `facts`
+      # on a cold output base, and this build would warm it — turning that step into the silent
+      # no-op its own comment warns about.
+      #
+      # `--config=local` because this job deliberately carries no BuildBuddy key (see the
+      # setup-bazel note above), and the default config points every action at a cache it cannot
+      # authenticate to. The image is two shell scripts over a pulled base; building it cold is
+      # cheaper than the round trip.
+      - name: Re-derive the devcontainer base-image pin
+        if: steps.changed.outputs.bazel == 'true' && steps.ratify.outputs.conflicts == ''
+        run: |
+          bazel build --config=local //meta/devcontainer-base:image
+          python3 meta/scripts/sync_base_image_pin.py
+
       # ── Go: tidy each module and sync the workspace ───────────────────────────────
       # Renovate's gomod manager runs `go get`, which bumps go.mod/go.sum but never
       # `go mod tidy` (opt-in) or `go work sync` (vendor-only in Renovate) — so the indirect
@@ -330,6 +350,7 @@ jobs:
             uv.lock
             requirements_lock.txt
             MODULE.bazel.lock
+            .devcontainer/Dockerfile
             .devcontainer/devcontainer-lock.json
             ${{ steps.gotidy.outputs.files }}
           commit-message: "chore(deps): re-derive lock files"