Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/actions/check-modules/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Module completeness check
description: Installs uv and the pinned Python, then runs the polyglot module completeness check.

# Shared by ci.yml and security.yml, which each need the check for different reasons: CI gates
# golangci-lint's module matrix on it, security.yml gates govulncheck's and pip-audit's. Jobs
# cannot be depended on across workflows, so both run it — this action shares the *definition*,
# not the execution. That is the whole point of it: the two copies restated the Python pin and
# two action SHAs, and nothing failed if they diverged, so the check could silently run under a
# different interpreter in each workflow.
#
# Callers check out the repo themselves, matching setup-bazel-remote next door; the script reads
# the worktree, so a caller without a checkout is the error case rather than a supported one.
runs:
using: composite
steps:
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
# The `# renovate:` marker is claimed by a renovate.json customManager whose file pattern
# covers `.github/actions/*/action.yml` for this pin; see README's "Marker-driven".
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
# renovate: datasource=python-version depName=python
python-version: "3.14"
- run: python3 meta/scripts/check_modules.py
shell: bash
27 changes: 11 additions & 16 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,31 +51,26 @@ jobs:
fi
echo "MODULE.bazel and MODULE.bazel.lock are what 'bazel mod tidy' produces."

# The repo's `meta/scripts/*.py` target Python 3.14 (matches the rules_python
# toolchain pin in MODULE.bazel and the `target-version` in pyproject.toml's
# [tool.ruff]). Ubuntu-24.04 still ships 3.12 as `python3`, so the language-
# toolchain pin needs to be applied explicitly on every job that runs a script
# outside Bazel. The `# renovate:` comment above each `python-version` is
# picked up by the regex matcher in renovate.json and grouped with the
# MODULE.bazel python_version under "Language toolchain SDKs".

# Polyglot module/project completeness check. Per language (Go, Python):
# config-file reachability + workflow matrix consistency. Python-only:
# root pyproject.toml has the required tool sections, every per-project
# pyproject is covered by [tool.uv.workspace].members, and
# requirements_lock.txt is fresh relative to uv.lock. uv is needed for the
# last check — setup-uv brings both uv and a 3.14 Python interpreter.
# requirements_lock.txt is fresh relative to uv.lock. The steps are shared
# with security.yml, which gates its own per-module jobs on the same check.
modules-check:
name: Module completeness check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
# renovate: datasource=python-version depName=python
python-version: "3.14"
- run: python3 meta/scripts/check_modules.py
- uses: ./.github/actions/check-modules

# The repo's `meta/scripts/*.py` target Python 3.14 (matches the rules_python
# toolchain pin in MODULE.bazel and the `target-version` in pyproject.toml's
# [tool.ruff]). Ubuntu-24.04 still ships 3.12 as `python3`, so the language-
# toolchain pin needs to be applied explicitly on every job that runs a script
# outside Bazel. The `# renovate:` comment above each `python-version` is
# picked up by the regex matcher in renovate.json and grouped with the
# MODULE.bazel python_version under "Language toolchain SDKs".

go-work-check:
name: go.work consistency/completeness check
Expand Down
11 changes: 3 additions & 8 deletions .github/workflows/security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,14 @@ jobs:
# ── Monorepo Structure Maintenance ────────────────────────────────────────────
# Checks that per-language tooling is fully wired up before per-module jobs run. Covers Go
# (module matrix completeness, .golangci.yml presence) and Python (workspace tool config,
# member glob coverage, requirements_lock.txt freshness). uv is needed for the lock check;
# setup-uv brings it alongside the 3.14 Python interpreter the script targets.
# member glob coverage, requirements_lock.txt freshness). The steps are shared with ci.yml,
# which runs the same check as its own gate.
modules-check:
name: Module completeness check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
# renovate: datasource=python-version depName=python
python-version: "3.14"
- run: python3 meta/scripts/check_modules.py
- uses: ./.github/actions/check-modules

# ── Static Application Security Testing (SAST) ───────────────────────────────
# Analyses source code for security vulnerabilities — injection flaws, insecure API usage,
Expand Down
12 changes: 9 additions & 3 deletions BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,24 @@ load("@gazelle//:def.bzl", "gazelle")
# gazelle:prefix github.com/Syndic/unnatural_designs
gazelle(name = "gazelle")

# Workflows read as *test data*, not built: three tests assert couplings that hold across files —
# Workflows read as *test data*, not built: four tests assert couplings that hold across files —
# the classification rules these workflows pass to classify_changed_paths.py, the env var
# devcontainer.yml sets for devcontainer.json's BASE_IMAGE override, and the Go toolchain step
# security.yml's CodeQL job needs ahead of extraction. Each stays green while drifting, so the
# devcontainer.yml sets for devcontainer.json's BASE_IMAGE override, the Go toolchain step
# security.yml's CodeQL job needs ahead of extraction, and the shared action ci.yml and
# security.yml both delegate their module check to. Each stays green while drifting, so the
# tests read the real file.
exports_files([
".github/actions/check-modules/action.yml",
".github/workflows/ci.yml",
".github/workflows/devcontainer.yml",
".github/workflows/renovate-derived-files.yml",
".github/workflows/security.yml",
])

# The custom-manager patterns are read as test data too: a marker the patterns stop claiming is
# a pin that silently freezes — see //meta/scripts:test_check_modules_action.
exports_files(["renovate.json"])

# Same posture for prose that hand-copies something the tests can check it against: the hook table
# against the config's ids (//meta/scripts:test_precommit_docs), and both docs against the required
# check name security.yml's fan-in carries (//meta/scripts:test_codeql_toolchain).
Expand Down
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ Three GitHub Actions workflows run on every push and pull request to `main`.
| ---------------------------- | -------------------------------------------------------------------------------------------------- |
| Gazelle check | Always - verifies BUILD files match source |
| MODULE.bazel.lock freshness | Always - verifies `bazel mod tidy` leaves MODULE.bazel and its lock unchanged |
| Module completeness check | Always - verifies Go module matrix/config and Python workspace/lock invariants |
| Module completeness check | Always - verifies Go module matrix/config and Python workspace/lock invariants (shared action) |
| go.work check | Always - verifies all Go modules are registered in `go.work` |
| Secrets check | Always - verifies the `secrets/` directory contains no committed files |
| No-cgo policy check | Always - rejects `import "C"` and transitive deps that compile C/C++/cgo/SWIG |
Expand All @@ -200,7 +200,7 @@ Three GitHub Actions workflows run on every push and pull request to `main`.

| Job | Purpose |
| --------------------------------- | ------------------------------------------------------------------------------- |
| Module completeness check | Gate for the per-module security jobs below |
| Module completeness check | Gate for the per-module security jobs below - same shared action as CI |
| Semgrep | SAST - scans for injection flaws, insecure API usage, and hardcoded secrets |
| `CodeQL Analysis (<language>)` | SAST - one job per language: actions, Go, Python |
| `CodeQL Analysis (all languages)` | Fan-in over the per-language jobs - the name to require in the ruleset |
Expand Down Expand Up @@ -316,11 +316,12 @@ Versions that live in a plain string rather than a manifest Renovate understands
`customManagers` regexes in [`renovate.json`](renovate.json). They come in two flavours, and the
difference matters when adding one:

- **Marker-driven** — a Dockerfile `ARG`, a shell assignment in `post-create.sh`, a workflow
`with:`/`env:` value. Each expects a `# renovate: datasource=<ds> depName=<name>` comment on the
line **immediately above** the value, and the value must be quoted in the workflow case
- **Marker-driven** — a Dockerfile `ARG`, a shell assignment in `post-create.sh`, a workflow or
composite-action `with:`/`env:` value. Each expects a `# renovate: datasource=<ds> depName=<name>`
comment on the line **immediately above** the value, and the value must be quoted in the YAML case
(`key: "1.2.3"`). The key name is not constrained, so both `version:` and `TY_VERSION:` are
tracked.
tracked. The YAML pattern claims `.github/workflows/*.yml` and `.github/actions/*/action.yml`
both, so a pin keeps its coverage when steps move into a shared action.
- **Structural** — the two `MODULE.bazel` patterns, which carry `datasourceTemplate`/`depNameTemplate`
in the config and match the pin site directly (`go_sdk.download(… version = "…")`, and any
`python_version = "…"`). There is no marker comment to grep for, so these are easy to forget when
Expand Down
15 changes: 15 additions & 0 deletions meta/scripts/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,21 @@ py_test(
main = "test_codeql_toolchain.py",
)

# Same shape again: the assertion is the whole gate. Guards the action ci.yml and security.yml
# share against a caller re-inlining it, and against the Renovate pattern that reaches its pin.
py_test(
name = "test_check_modules_action",
size = "small",
srcs = ["test_check_modules_action.py"],
data = [
"//:.github/actions/check-modules/action.yml",
"//:.github/workflows/ci.yml",
"//:.github/workflows/security.yml",
"//:renovate.json",
],
main = "test_check_modules_action.py",
)

py_library(
name = "classify_changed_paths_lib",
srcs = ["classify_changed_paths.py"],
Expand Down
8 changes: 8 additions & 0 deletions meta/scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ branch protection requires. Same reason it rides `bazel test //...`: the couplin
checked-in files, and nothing fails while they drift — not until `go.work` outruns the runner
image's Go, or a green required check turns out to have been skipped.

`test_check_modules_action.py` is the third of that shape. `ci.yml` and `security.yml` both run
`check_modules.py` — jobs can't be depended on across workflows — and both reach it through
`.github/actions/check-modules`, so the two callers share one definition of the setup rather than a
copy each. It asserts that neither caller has re-inlined the steps, and that a `renovate.json`
customManager still claims the Python pin now that the pin lives under `.github/actions/`. Both
failures are silent: a re-inlined copy runs the same check under whatever it pins, and an unclaimed
marker simply stops moving.

`smoke_py/` is a transient `py_test` that proves the end-to-end Python plumbing chain
(`pyproject.toml` → `uv.lock` → `requirements_lock.txt` → `pip.parse` → `@unnatural_designs_pypi//...`) by importing
`requests` and asserting it loads. Slated for deletion once gazelle_python is wired (see
Expand Down
144 changes: 144 additions & 0 deletions meta/scripts/test_check_modules_action.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
"""Holds the shared module-check action to the two couplings that fail silently.

`ci.yml` and `security.yml` both need the module completeness check — CI gates golangci-lint's
module matrix on it, security.yml gates govulncheck's and pip-audit's — and jobs cannot be
depended on across workflows, so both run it. `.github/actions/check-modules` exists so the two
callers share one definition of *how*: before it, each restated the Python pin and two action
SHAs, and a diverged copy would have run the same check under a different interpreter in each
workflow with nothing to say so.

That is the first coupling here: a caller that re-inlines the steps gets its own copy back, and
CI stays green while it drifts.

The second is the Renovate pattern. The pin moved out of `.github/workflows/` when the steps did,
and the customManager that claims `# renovate:` markers is scoped by file path — so the pattern
had to widen to `.github/actions/*/action.yml` in the same change. README's "Marker-driven"
records that this class of failure is silent and has already happened twice: the pin simply stops
moving and Renovate says nothing.
"""

import json
import re
import unittest
from pathlib import Path

# Not .resolve(): every file read here is a cross-package data dep, so each lives in the runfiles
# tree beside this one rather than at the source path a resolved symlink would lead back to.
_ROOT = Path(__file__).parent.parent.parent
# Repo-relative, because that is what Renovate matches its file patterns against — passing a
# bare filename here would miss every anchored pattern and quietly claim nothing.
_ACTION_PATH = ".github/actions/check-modules/action.yml"
_ACTION = _ROOT / _ACTION_PATH
_RENOVATE = _ROOT / "renovate.json"
_CALLERS = (
_ROOT / ".github" / "workflows" / "ci.yml",
_ROOT / ".github" / "workflows" / "security.yml",
)

# The path a workflow `uses:` to reach the action, and the script that action is the wrapper for.
_USES = "uses: ./.github/actions/check-modules"
_SCRIPT = "meta/scripts/check_modules.py"


def job_block(text: str, job: str) -> str:
"""One job's lines: its key through the line before the next job key."""
start = re.search(rf"^ {re.escape(job)}:$", text, re.M)
if start is None:
raise AssertionError(f"no `{job}:` job")
rest = text[start.end() :]
end = re.search(r"^ [a-zA-Z_][\w-]*:$", rest, re.M)
return rest[: end.start()] if end else rest


class DelegationTest(unittest.TestCase):
"""Both callers reach the check through the action rather than through steps of their own."""

def test_every_caller_uses_the_action(self):
for caller in _CALLERS:
with self.subTest(workflow=caller.name):
block = job_block(caller.read_text(encoding="utf-8"), "modules-check")
self.assertIn(
_USES,
block,
"this job runs the module check without the shared action, so its setup "
"steps are a second copy that nothing holds to the first",
)

def test_no_caller_reinlines_the_script(self):
"""The tell that a copy came back: the workflow invoking the script directly again."""
for caller in _CALLERS:
with self.subTest(workflow=caller.name):
block = job_block(caller.read_text(encoding="utf-8"), "modules-check")
self.assertNotIn(_SCRIPT, block, "call it through the action, not beside it")

def test_the_action_runs_the_script(self):
"""Non-vacuity guard: the assertions above mean nothing if the action does not run it."""
self.assertIn(_SCRIPT, _ACTION.read_text(encoding="utf-8"))


class RenovateCoverageTest(unittest.TestCase):
"""The action's pin is only tracked while a customManager's file pattern reaches it."""

def setUp(self):
self.action = _ACTION.read_text(encoding="utf-8")
config = json.loads(_RENOVATE.read_text(encoding="utf-8"))
self.managers = [m for m in config["customManagers"] if m["customType"] == "regex"]

def claims(self, path: str, text: str) -> list[tuple[str | None, str]]:
"""Every (depName, currentValue) a manager whose pattern reaches `path` finds in `text`.

depName is None for the managers that name the dep in `depNameTemplate` instead of a
capture group, so callers filter on it rather than assuming it is there.
"""
found = []
for manager in self.managers:
patterns = [re.compile(p.strip("/")) for p in manager["managerFilePatterns"]]
if not any(p.search(path) for p in patterns):
continue
for match in manager["matchStrings"]:
# Renovate's regexes are JS-flavoured; only the named-group spelling differs.
for hit in re.finditer(match.replace("(?<", "(?P<"), text):
groups = hit.groupdict()
found.append((groups.get("depName"), groups["currentValue"]))
return found

def test_the_action_carries_a_python_pin(self):
"""Non-vacuity guard: a pin that moved back out would pass the coverage test trivially."""
self.assertRegex(
self.action,
r'python-version: "\d+\.\d+"',
"the action no longer pins Python; the coverage assertion below has nothing to hold",
)

def test_renovate_claims_the_actions_python_pin(self):
claimed = self.claims(_ACTION_PATH, self.action)
self.assertIn(
"python",
[dep for dep, _ in claimed],
"no customManager pattern reaches `.github/actions/*/action.yml`, so this pin is "
"frozen — Renovate reports nothing when a marker stops being claimed",
)

def test_the_pin_matches_the_workflows_that_still_carry_one(self):
"""Divergent copies are two dependencies to Renovate, and they drift apart separately."""
versions = {
value for dep, value in self.claims(_ACTION_PATH, self.action) if dep == "python"
}
self.assertTrue(versions, "the action's own pin was not read; the comparison is vacuous")
for caller in _CALLERS:
text = caller.read_text(encoding="utf-8")
versions |= {
value
for dep, value in self.claims(f".github/workflows/{caller.name}", text)
if dep == "python"
}
self.assertEqual(
len(versions),
1,
f"the Python pin is spelled {sorted(versions)} across the action and its callers; "
"keep duplicate pins byte-identical or Renovate tracks them as separate updates",
)


if __name__ == "__main__":
unittest.main()
5 changes: 4 additions & 1 deletion renovate.json
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,10 @@
},
{
"customType": "regex",
"managerFilePatterns": [ "/^\\.github/workflows/.*\\.ya?ml$/" ],
"managerFilePatterns": [
"/^\\.github/workflows/.*\\.ya?ml$/",
"/^\\.github/actions/.*/action\\.ya?ml$/"
],
"matchStrings": [
"#\\s*renovate:\\s*datasource=(?<datasource>[a-z-]+?)\\s+depName=(?<depName>\\S+?)\\s*\\n\\s*[A-Za-z0-9_-]+:\\s*\"(?<currentValue>[^\"]+)\""
]
Expand Down
Loading