Skip to content

ci: cut runner minutes by about two thirds - #388

Merged
nk-o merged 2 commits into
masterfrom
claude/ci-optimization-main-projects-8e300e
Aug 11, 2026
Merged

ci: cut runner minutes by about two thirds#388
nk-o merged 2 commits into
masterfrom
claude/ci-optimization-main-projects-8e300e

Conversation

@nk-o

@nk-o nk-o commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Why

Blacksmith meters the free tier in 2 vCPU minutes and bills per vCPU — the dashboard spells it out: "Free minutes are 2 vCPU Linux runner minutes. Larger instances or other OS will use free credits faster."

Every job here ran on blacksmith-4vcpu-ubuntu-2404, so each wall-clock minute drew two credits. August's 3,467 runner-minutes consumed 6,934 against a 3,000 allowance; the free tier ran out roughly 1,500 minutes in, and $27.74 was billed across about three active days.

Nothing in these pipelines uses four cores. PHPUnit is single-threaded, Playwright runs with workers: 1, and the rest of each job waits on Docker and the network. The extra two cores bought no wall-clock and cost exactly double.

What changed

Before After
Runner blacksmith-4vcpu blacksmith-2vcpu
PHP matrix, per PR 4 legs 3 corner legs
PHP matrix, per push to master 4 legs + lint + phpcs 1 leg
Playwright browsers chromium + firefox + webkit chromium
Playwright browser cache none added, with restore-keys
Previous-WP lookup a job blocking the whole matrix a step, only on legs that need it
apt-get install subversion every PHP leg removed
Debug dumps 3 steps on every green run 1 step behind if: failure()
Composer resolution against the runner's default PHP against matrix.php
Job timeouts missing on the e2e job set on every job

On the matrix

Adjacent PHP minors were running identical code paths through the same suite. What's kept is the corners — oldest and newest supported PHP on current WordPress, plus oldest PHP on the previous WordPress. The static compatibility guarantee across the full supported range comes from the PHPCompatibility sniffs in the phpcs job (testVersion 7.2-), not from the runtime matrix.

On the master run

A push to master re-tests the tree its pull request validated minutes earlier — the history here is linear squash-merges. It drops to a single leg whose only job is catching a bad squash. lint and phpcs are skipped entirely on push: they are deterministic, so a green PR cannot go red on a squash of that same tree.

There is no branch protection on this repo, so the master run is the only backstop against two PRs merging into an untested combination. That is why it is reduced rather than removed. This repo is public, so enabling branch protection with "require branches to be up to date" would make the PR run authoritative and let the master run go away entirely — worth doing as a follow-up.

Composer ordering (correctness, not cost)

Set up PHP now runs before the Node setup. npm ci triggers postinstall, which runs composer install — so PHP dependencies were being resolved against whatever PHP the runner image ships rather than against matrix.php. Every leg was quietly testing one dependency set.

Projected effect

Applied to the same 60 days of real runs at the same per-step durations, plus a 12% allowance for the slower runner: ~252 → ~90 runner-minutes/month for this repo. Across all six plugin repos, credits go from 5,665 to ~1,562 a month — back inside the free tier with roughly 2x headroom.

Verification

This PR is its own test: pull_request runs the workflow from the branch, so the checks below are the new pipeline. Worth eyeballing before merge:

  • 3 PHP legs, not 4, and all on blacksmith-2vcpu
  • the previous WP leg resolves and prints the version it picked
  • Playwright installs only chromium and still passes

Also fixed in passing: the matrix include set wordpress: 'previous major version' as a literal string, which rendered job names like "PHP 8.4 (WP previous major version)"; the '**/node_modules' cache glob replaced with an explicit path.


Separately, Docker container caching should be turned on in the Blacksmith dashboard (currently off, and free — it is exempt from sticky-disk billing). It targets wp-env start, which is 53-88s in every test job and the largest fixed cost left after this change.

nk-o added 2 commits August 11, 2026 10:05
Blacksmith meters the free tier in 2 vCPU minutes and bills per vCPU, so
every job on blacksmith-4vcpu burned credits at double rate. Nothing here
saturated four cores: PHPUnit is single-threaded and Playwright runs with
`workers: 1`. Dropping to 2 vCPU halves the rate on its own.

On top of that:

- Matrix down to corners — oldest and newest supported PHP on current
  WordPress, oldest on previous. Adjacent minors were re-proving the same
  code paths; PHPCompatibility sniffs in phpcs carry the static guarantee
  across the full supported range.
- A push to master re-tests the tree its pull request just validated, so
  it drops to one leg and skips lint and phpcs, which cannot go red on a
  squash of a green branch.
- Playwright installs chromium only. The shared preset gates webkit and
  firefox behind @WebKit / @Firefox tags and no spec carries either, so
  both were downloaded on every run and never launched.
- `compute-previous-wordpress-version` spent a whole runner on one curl
  and blocked the matrix behind it; it is now a step, and only the legs
  that need it pay for it.
- `apt-get install subversion` is gone. Nothing in the test path uses
  svn — only the deploy workflows, which install it themselves.
- Debug dumps run behind `if: failure()` instead of on every green run.
- Set up PHP moved ahead of the Node setup. `npm ci` triggers
  `postinstall`, which runs `composer install`, so PHP dependencies were
  being resolved against the runner's default PHP rather than matrix.php.
The original comment claimed `workers: 1` left three cores idle. Measuring
the switch disproves it: `playwright test` runs 40-70% slower on 2 vCPU,
because the browser and the wp-env stack use cores the Playwright worker
count says nothing about.

The change still pays -- every job came out cheaper in credits, since
halving the rate more than covers the extra wall time -- but the stated
reason was wrong, and the trade-off it hides (slower pull-request
feedback) is the thing worth knowing when revisiting this.
@nk-o
nk-o merged commit fdae281 into master Aug 11, 2026
7 checks passed
@nk-o
nk-o deleted the claude/ci-optimization-main-projects-8e300e branch August 11, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant