Skip to content

Commit 17c41e8

Browse files
Syndicclaude
andcommitted
ci: share the module check between CI and Security via a composite action
ci.yml and security.yml both need `check_modules.py` — CI gates golangci-lint's module matrix on it, security.yml gates govulncheck's and pip-audit's — and jobs cannot be depended on across workflows, so both must run it. Each carried its own copy of the steps, restating the Python pin and two action SHAs with nothing holding the two together: a diverged copy would have run the same check under a different interpreter in each workflow, silently. Extract the steps to `.github/actions/check-modules`. This shares the definition, not the execution — the check still runs twice per PR, in parallel, ~10s each. A reusable workflow would not have changed that either; GitHub instantiates a called workflow's jobs into each caller's run and dedupes nothing. The pin moves out of `.github/workflows/` with the steps, so renovate.json's marker-driven customManager widens to `.github/actions/*/action.yml`. Verified the widened pattern claims the new site and adds nothing else: the two existing action.yml files carry no `# renovate:` markers. `test_check_modules_action` guards both couplings, each of which fails silently — a caller that re-inlines the steps, and a pattern that stops reaching the pin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 309d1da commit 17c41e8

9 files changed

Lines changed: 225 additions & 34 deletions

File tree

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: Module completeness check
2+
description: Installs uv and the pinned Python, then runs the polyglot module completeness check.
3+
4+
# Shared by ci.yml and security.yml, which each need the check for different reasons: CI gates
5+
# golangci-lint's module matrix on it, security.yml gates govulncheck's and pip-audit's. Jobs
6+
# cannot be depended on across workflows, so both run it — this action shares the *definition*,
7+
# not the execution. That is the whole point of it: the two copies restated the Python pin and
8+
# two action SHAs, and nothing failed if they diverged, so the check could silently run under a
9+
# different interpreter in each workflow.
10+
#
11+
# Callers check out the repo themselves, matching setup-bazel-remote next door; the script reads
12+
# the worktree, so a caller without a checkout is the error case rather than a supported one.
13+
runs:
14+
using: composite
15+
steps:
16+
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
17+
# The `# renovate:` marker is claimed by a renovate.json customManager whose file pattern
18+
# covers `.github/actions/*/action.yml` for this pin; see README's "Marker-driven".
19+
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
20+
with:
21+
# renovate: datasource=python-version depName=python
22+
python-version: "3.14"
23+
- run: python3 meta/scripts/check_modules.py
24+
shell: bash

.github/workflows/ci.yml

Lines changed: 11 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -51,31 +51,26 @@ jobs:
5151
fi
5252
echo "MODULE.bazel and MODULE.bazel.lock are what 'bazel mod tidy' produces."
5353
54-
# The repo's `meta/scripts/*.py` target Python 3.14 (matches the rules_python
55-
# toolchain pin in MODULE.bazel and the `target-version` in pyproject.toml's
56-
# [tool.ruff]). Ubuntu-24.04 still ships 3.12 as `python3`, so the language-
57-
# toolchain pin needs to be applied explicitly on every job that runs a script
58-
# outside Bazel. The `# renovate:` comment above each `python-version` is
59-
# picked up by the regex matcher in renovate.json and grouped with the
60-
# MODULE.bazel python_version under "Language toolchain SDKs".
61-
6254
# Polyglot module/project completeness check. Per language (Go, Python):
6355
# config-file reachability + workflow matrix consistency. Python-only:
6456
# root pyproject.toml has the required tool sections, every per-project
6557
# pyproject is covered by [tool.uv.workspace].members, and
66-
# requirements_lock.txt is fresh relative to uv.lock. uv is needed for the
67-
# last check — setup-uv brings both uv and a 3.14 Python interpreter.
58+
# requirements_lock.txt is fresh relative to uv.lock. The steps are shared
59+
# with security.yml, which gates its own per-module jobs on the same check.
6860
modules-check:
6961
name: Module completeness check
7062
runs-on: ubuntu-latest
7163
steps:
7264
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
73-
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
74-
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
75-
with:
76-
# renovate: datasource=python-version depName=python
77-
python-version: "3.14"
78-
- run: python3 meta/scripts/check_modules.py
65+
- uses: ./.github/actions/check-modules
66+
67+
# The repo's `meta/scripts/*.py` target Python 3.14 (matches the rules_python
68+
# toolchain pin in MODULE.bazel and the `target-version` in pyproject.toml's
69+
# [tool.ruff]). Ubuntu-24.04 still ships 3.12 as `python3`, so the language-
70+
# toolchain pin needs to be applied explicitly on every job that runs a script
71+
# outside Bazel. The `# renovate:` comment above each `python-version` is
72+
# picked up by the regex matcher in renovate.json and grouped with the
73+
# MODULE.bazel python_version under "Language toolchain SDKs".
7974

8075
go-work-check:
8176
name: go.work consistency/completeness check

.github/workflows/security.yml

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -21,19 +21,14 @@ jobs:
2121
# ── Monorepo Structure Maintenance ────────────────────────────────────────────
2222
# Checks that per-language tooling is fully wired up before per-module jobs run. Covers Go
2323
# (module matrix completeness, .golangci.yml presence) and Python (workspace tool config,
24-
# member glob coverage, requirements_lock.txt freshness). uv is needed for the lock check;
25-
# setup-uv brings it alongside the 3.14 Python interpreter the script targets.
24+
# member glob coverage, requirements_lock.txt freshness). The steps are shared with ci.yml,
25+
# which runs the same check as its own gate.
2626
modules-check:
2727
name: Module completeness check
2828
runs-on: ubuntu-latest
2929
steps:
3030
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
31-
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
32-
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
33-
with:
34-
# renovate: datasource=python-version depName=python
35-
python-version: "3.14"
36-
- run: python3 meta/scripts/check_modules.py
31+
- uses: ./.github/actions/check-modules
3732

3833
# ── Static Application Security Testing (SAST) ───────────────────────────────
3934
# Analyses source code for security vulnerabilities — injection flaws, insecure API usage,

BUILD.bazel

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,24 @@ load("@gazelle//:def.bzl", "gazelle")
33
# gazelle:prefix github.com/Syndic/unnatural_designs
44
gazelle(name = "gazelle")
55

6-
# Workflows read as *test data*, not built: three tests assert couplings that hold across files —
6+
# Workflows read as *test data*, not built: four tests assert couplings that hold across files —
77
# the classification rules these workflows pass to classify_changed_paths.py, the env var
8-
# devcontainer.yml sets for devcontainer.json's BASE_IMAGE override, and the Go toolchain step
9-
# security.yml's CodeQL job needs ahead of extraction. Each stays green while drifting, so the
8+
# devcontainer.yml sets for devcontainer.json's BASE_IMAGE override, the Go toolchain step
9+
# security.yml's CodeQL job needs ahead of extraction, and the shared action ci.yml and
10+
# security.yml both delegate their module check to. Each stays green while drifting, so the
1011
# tests read the real file.
1112
exports_files([
13+
".github/actions/check-modules/action.yml",
1214
".github/workflows/ci.yml",
1315
".github/workflows/devcontainer.yml",
1416
".github/workflows/renovate-derived-files.yml",
1517
".github/workflows/security.yml",
1618
])
1719

20+
# The custom-manager patterns are read as test data too: a marker the patterns stop claiming is
21+
# a pin that silently freezes — see //meta/scripts:test_check_modules_action.
22+
exports_files(["renovate.json"])
23+
1824
# Same posture for prose that hand-copies something the tests can check it against: the hook table
1925
# against the config's ids (//meta/scripts:test_precommit_docs), and both docs against the required
2026
# check name security.yml's fan-in carries (//meta/scripts:test_codeql_toolchain).

README.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ Three GitHub Actions workflows run on every push and pull request to `main`.
185185
| ---------------------------- | -------------------------------------------------------------------------------------------------- |
186186
| Gazelle check | Always - verifies BUILD files match source |
187187
| MODULE.bazel.lock freshness | Always - verifies `bazel mod tidy` leaves MODULE.bazel and its lock unchanged |
188-
| Module completeness check | Always - verifies Go module matrix/config and Python workspace/lock invariants |
188+
| Module completeness check | Always - verifies Go module matrix/config and Python workspace/lock invariants (shared action) |
189189
| go.work check | Always - verifies all Go modules are registered in `go.work` |
190190
| Secrets check | Always - verifies the `secrets/` directory contains no committed files |
191191
| No-cgo policy check | Always - rejects `import "C"` and transitive deps that compile C/C++/cgo/SWIG |
@@ -200,7 +200,7 @@ Three GitHub Actions workflows run on every push and pull request to `main`.
200200
201201
| Job | Purpose |
202202
| --------------------------------- | ------------------------------------------------------------------------------- |
203-
| Module completeness check | Gate for the per-module security jobs below |
203+
| Module completeness check | Gate for the per-module security jobs below - same shared action as CI |
204204
| Semgrep | SAST - scans for injection flaws, insecure API usage, and hardcoded secrets |
205205
| `CodeQL Analysis (<language>)` | SAST - one job per language: actions, Go, Python |
206206
| `CodeQL Analysis (all languages)` | Fan-in over the per-language jobs - the name to require in the ruleset |
@@ -316,11 +316,12 @@ Versions that live in a plain string rather than a manifest Renovate understands
316316
`customManagers` regexes in [`renovate.json`](renovate.json). They come in two flavours, and the
317317
difference matters when adding one:
318318
319-
- **Marker-driven** — a Dockerfile `ARG`, a shell assignment in `post-create.sh`, a workflow
320-
`with:`/`env:` value. Each expects a `# renovate: datasource=<ds> depName=<name>` comment on the
321-
line **immediately above** the value, and the value must be quoted in the workflow case
319+
- **Marker-driven** — a Dockerfile `ARG`, a shell assignment in `post-create.sh`, a workflow or
320+
composite-action `with:`/`env:` value. Each expects a `# renovate: datasource=<ds> depName=<name>`
321+
comment on the line **immediately above** the value, and the value must be quoted in the YAML case
322322
(`key: "1.2.3"`). The key name is not constrained, so both `version:` and `TY_VERSION:` are
323-
tracked.
323+
tracked. The YAML pattern claims `.github/workflows/*.yml` and `.github/actions/*/action.yml`
324+
both, so a pin keeps its coverage when steps move into a shared action.
324325
- **Structural** — the two `MODULE.bazel` patterns, which carry `datasourceTemplate`/`depNameTemplate`
325326
in the config and match the pin site directly (`go_sdk.download(… version = "…")`, and any
326327
`python_version = "…"`). There is no marker comment to grep for, so these are easy to forget when

meta/scripts/BUILD.bazel

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,21 @@ py_test(
115115
main = "test_codeql_toolchain.py",
116116
)
117117

118+
# Same shape again: the assertion is the whole gate. Guards the action ci.yml and security.yml
119+
# share against a caller re-inlining it, and against the Renovate pattern that reaches its pin.
120+
py_test(
121+
name = "test_check_modules_action",
122+
size = "small",
123+
srcs = ["test_check_modules_action.py"],
124+
data = [
125+
"//:.github/actions/check-modules/action.yml",
126+
"//:.github/workflows/ci.yml",
127+
"//:.github/workflows/security.yml",
128+
"//:renovate.json",
129+
],
130+
main = "test_check_modules_action.py",
131+
)
132+
118133
py_library(
119134
name = "classify_changed_paths_lib",
120135
srcs = ["classify_changed_paths.py"],

meta/scripts/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,14 @@ branch protection requires. Same reason it rides `bazel test //...`: the couplin
3636
checked-in files, and nothing fails while they drift — not until `go.work` outruns the runner
3737
image's Go, or a green required check turns out to have been skipped.
3838

39+
`test_check_modules_action.py` is the third of that shape. `ci.yml` and `security.yml` both run
40+
`check_modules.py` — jobs can't be depended on across workflows — and both reach it through
41+
`.github/actions/check-modules`, so the two callers share one definition of the setup rather than a
42+
copy each. It asserts that neither caller has re-inlined the steps, and that a `renovate.json`
43+
customManager still claims the Python pin now that the pin lives under `.github/actions/`. Both
44+
failures are silent: a re-inlined copy runs the same check under whatever it pins, and an unclaimed
45+
marker simply stops moving.
46+
3947
`smoke_py/` is a transient `py_test` that proves the end-to-end Python plumbing chain
4048
(`pyproject.toml``uv.lock``requirements_lock.txt``pip.parse``@unnatural_designs_pypi//...`) by importing
4149
`requests` and asserting it loads. Slated for deletion once gazelle_python is wired (see
Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
"""Holds the shared module-check action to the two couplings that fail silently.
2+
3+
`ci.yml` and `security.yml` both need the module completeness check — CI gates golangci-lint's
4+
module matrix on it, security.yml gates govulncheck's and pip-audit's — and jobs cannot be
5+
depended on across workflows, so both run it. `.github/actions/check-modules` exists so the two
6+
callers share one definition of *how*: before it, each restated the Python pin and two action
7+
SHAs, and a diverged copy would have run the same check under a different interpreter in each
8+
workflow with nothing to say so.
9+
10+
That is the first coupling here: a caller that re-inlines the steps gets its own copy back, and
11+
CI stays green while it drifts.
12+
13+
The second is the Renovate pattern. The pin moved out of `.github/workflows/` when the steps did,
14+
and the customManager that claims `# renovate:` markers is scoped by file path — so the pattern
15+
had to widen to `.github/actions/*/action.yml` in the same change. README's "Marker-driven"
16+
records that this class of failure is silent and has already happened twice: the pin simply stops
17+
moving and Renovate says nothing.
18+
"""
19+
20+
import json
21+
import re
22+
import unittest
23+
from pathlib import Path
24+
25+
# Not .resolve(): every file read here is a cross-package data dep, so each lives in the runfiles
26+
# tree beside this one rather than at the source path a resolved symlink would lead back to.
27+
_ROOT = Path(__file__).parent.parent.parent
28+
# Repo-relative, because that is what Renovate matches its file patterns against — passing a
29+
# bare filename here would miss every anchored pattern and quietly claim nothing.
30+
_ACTION_PATH = ".github/actions/check-modules/action.yml"
31+
_ACTION = _ROOT / _ACTION_PATH
32+
_RENOVATE = _ROOT / "renovate.json"
33+
_CALLERS = (
34+
_ROOT / ".github" / "workflows" / "ci.yml",
35+
_ROOT / ".github" / "workflows" / "security.yml",
36+
)
37+
38+
# The path a workflow `uses:` to reach the action, and the script that action is the wrapper for.
39+
_USES = "uses: ./.github/actions/check-modules"
40+
_SCRIPT = "meta/scripts/check_modules.py"
41+
42+
43+
def job_block(text: str, job: str) -> str:
44+
"""One job's lines: its key through the line before the next job key."""
45+
start = re.search(rf"^ {re.escape(job)}:$", text, re.M)
46+
if start is None:
47+
raise AssertionError(f"no `{job}:` job")
48+
rest = text[start.end() :]
49+
end = re.search(r"^ [a-zA-Z_][\w-]*:$", rest, re.M)
50+
return rest[: end.start()] if end else rest
51+
52+
53+
class DelegationTest(unittest.TestCase):
54+
"""Both callers reach the check through the action rather than through steps of their own."""
55+
56+
def test_every_caller_uses_the_action(self):
57+
for caller in _CALLERS:
58+
with self.subTest(workflow=caller.name):
59+
block = job_block(caller.read_text(encoding="utf-8"), "modules-check")
60+
self.assertIn(
61+
_USES,
62+
block,
63+
"this job runs the module check without the shared action, so its setup "
64+
"steps are a second copy that nothing holds to the first",
65+
)
66+
67+
def test_no_caller_reinlines_the_script(self):
68+
"""The tell that a copy came back: the workflow invoking the script directly again."""
69+
for caller in _CALLERS:
70+
with self.subTest(workflow=caller.name):
71+
block = job_block(caller.read_text(encoding="utf-8"), "modules-check")
72+
self.assertNotIn(_SCRIPT, block, "call it through the action, not beside it")
73+
74+
def test_the_action_runs_the_script(self):
75+
"""Non-vacuity guard: the assertions above mean nothing if the action does not run it."""
76+
self.assertIn(_SCRIPT, _ACTION.read_text(encoding="utf-8"))
77+
78+
79+
class RenovateCoverageTest(unittest.TestCase):
80+
"""The action's pin is only tracked while a customManager's file pattern reaches it."""
81+
82+
def setUp(self):
83+
self.action = _ACTION.read_text(encoding="utf-8")
84+
config = json.loads(_RENOVATE.read_text(encoding="utf-8"))
85+
self.managers = [m for m in config["customManagers"] if m["customType"] == "regex"]
86+
87+
def claims(self, path: str, text: str) -> list[tuple[str | None, str]]:
88+
"""Every (depName, currentValue) a manager whose pattern reaches `path` finds in `text`.
89+
90+
depName is None for the managers that name the dep in `depNameTemplate` instead of a
91+
capture group, so callers filter on it rather than assuming it is there.
92+
"""
93+
found = []
94+
for manager in self.managers:
95+
patterns = [re.compile(p.strip("/")) for p in manager["managerFilePatterns"]]
96+
if not any(p.search(path) for p in patterns):
97+
continue
98+
for match in manager["matchStrings"]:
99+
# Renovate's regexes are JS-flavoured; only the named-group spelling differs.
100+
for hit in re.finditer(match.replace("(?<", "(?P<"), text):
101+
groups = hit.groupdict()
102+
found.append((groups.get("depName"), groups["currentValue"]))
103+
return found
104+
105+
def test_the_action_carries_a_python_pin(self):
106+
"""Non-vacuity guard: a pin that moved back out would pass the coverage test trivially."""
107+
self.assertRegex(
108+
self.action,
109+
r'python-version: "\d+\.\d+"',
110+
"the action no longer pins Python; the coverage assertion below has nothing to hold",
111+
)
112+
113+
def test_renovate_claims_the_actions_python_pin(self):
114+
claimed = self.claims(_ACTION_PATH, self.action)
115+
self.assertIn(
116+
"python",
117+
[dep for dep, _ in claimed],
118+
"no customManager pattern reaches `.github/actions/*/action.yml`, so this pin is "
119+
"frozen — Renovate reports nothing when a marker stops being claimed",
120+
)
121+
122+
def test_the_pin_matches_the_workflows_that_still_carry_one(self):
123+
"""Divergent copies are two dependencies to Renovate, and they drift apart separately."""
124+
versions = {
125+
value for dep, value in self.claims(_ACTION_PATH, self.action) if dep == "python"
126+
}
127+
self.assertTrue(versions, "the action's own pin was not read; the comparison is vacuous")
128+
for caller in _CALLERS:
129+
text = caller.read_text(encoding="utf-8")
130+
versions |= {
131+
value
132+
for dep, value in self.claims(f".github/workflows/{caller.name}", text)
133+
if dep == "python"
134+
}
135+
self.assertEqual(
136+
len(versions),
137+
1,
138+
f"the Python pin is spelled {sorted(versions)} across the action and its callers; "
139+
"keep duplicate pins byte-identical or Renovate tracks them as separate updates",
140+
)
141+
142+
143+
if __name__ == "__main__":
144+
unittest.main()

renovate.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,10 @@
4949
},
5050
{
5151
"customType": "regex",
52-
"managerFilePatterns": [ "/^\\.github/workflows/.*\\.ya?ml$/" ],
52+
"managerFilePatterns": [
53+
"/^\\.github/workflows/.*\\.ya?ml$/",
54+
"/^\\.github/actions/.*/action\\.ya?ml$/"
55+
],
5356
"matchStrings": [
5457
"#\\s*renovate:\\s*datasource=(?<datasource>[a-z-]+?)\\s+depName=(?<depName>\\S+?)\\s*\\n\\s*[A-Za-z0-9_-]+:\\s*\"(?<currentValue>[^\"]+)\""
5558
]

0 commit comments

Comments
 (0)