ci: cut runner minutes by about two thirds - #388
Merged
Conversation
Blacksmith meters the free tier in 2 vCPU minutes and bills per vCPU, so every job on blacksmith-4vcpu burned credits at double rate. Nothing here saturated four cores: PHPUnit is single-threaded and Playwright runs with `workers: 1`. Dropping to 2 vCPU halves the rate on its own. On top of that: - Matrix down to corners — oldest and newest supported PHP on current WordPress, oldest on previous. Adjacent minors were re-proving the same code paths; PHPCompatibility sniffs in phpcs carry the static guarantee across the full supported range. - A push to master re-tests the tree its pull request just validated, so it drops to one leg and skips lint and phpcs, which cannot go red on a squash of a green branch. - Playwright installs chromium only. The shared preset gates webkit and firefox behind @WebKit / @Firefox tags and no spec carries either, so both were downloaded on every run and never launched. - `compute-previous-wordpress-version` spent a whole runner on one curl and blocked the matrix behind it; it is now a step, and only the legs that need it pay for it. - `apt-get install subversion` is gone. Nothing in the test path uses svn — only the deploy workflows, which install it themselves. - Debug dumps run behind `if: failure()` instead of on every green run. - Set up PHP moved ahead of the Node setup. `npm ci` triggers `postinstall`, which runs `composer install`, so PHP dependencies were being resolved against the runner's default PHP rather than matrix.php.
The original comment claimed `workers: 1` left three cores idle. Measuring the switch disproves it: `playwright test` runs 40-70% slower on 2 vCPU, because the browser and the wp-env stack use cores the Playwright worker count says nothing about. The change still pays -- every job came out cheaper in credits, since halving the rate more than covers the extra wall time -- but the stated reason was wrong, and the trade-off it hides (slower pull-request feedback) is the thing worth knowing when revisiting this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Blacksmith meters the free tier in 2 vCPU minutes and bills per vCPU — the dashboard spells it out: "Free minutes are 2 vCPU Linux runner minutes. Larger instances or other OS will use free credits faster."
Every job here ran on
blacksmith-4vcpu-ubuntu-2404, so each wall-clock minute drew two credits. August's 3,467 runner-minutes consumed 6,934 against a 3,000 allowance; the free tier ran out roughly 1,500 minutes in, and $27.74 was billed across about three active days.Nothing in these pipelines uses four cores. PHPUnit is single-threaded, Playwright runs with
workers: 1, and the rest of each job waits on Docker and the network. The extra two cores bought no wall-clock and cost exactly double.What changed
blacksmith-4vcpublacksmith-2vcpurestore-keysapt-get install subversionif: failure()matrix.phpOn the matrix
Adjacent PHP minors were running identical code paths through the same suite. What's kept is the corners — oldest and newest supported PHP on current WordPress, plus oldest PHP on the previous WordPress. The static compatibility guarantee across the full supported range comes from the PHPCompatibility sniffs in the
phpcsjob (testVersion 7.2-), not from the runtime matrix.On the master run
A push to master re-tests the tree its pull request validated minutes earlier — the history here is linear squash-merges. It drops to a single leg whose only job is catching a bad squash.
lintandphpcsare skipped entirely on push: they are deterministic, so a green PR cannot go red on a squash of that same tree.There is no branch protection on this repo, so the master run is the only backstop against two PRs merging into an untested combination. That is why it is reduced rather than removed. This repo is public, so enabling branch protection with "require branches to be up to date" would make the PR run authoritative and let the master run go away entirely — worth doing as a follow-up.
Composer ordering (correctness, not cost)
Set up PHPnow runs before the Node setup.npm citriggerspostinstall, which runscomposer install— so PHP dependencies were being resolved against whatever PHP the runner image ships rather than againstmatrix.php. Every leg was quietly testing one dependency set.Projected effect
Applied to the same 60 days of real runs at the same per-step durations, plus a 12% allowance for the slower runner: ~252 → ~90 runner-minutes/month for this repo. Across all six plugin repos, credits go from 5,665 to ~1,562 a month — back inside the free tier with roughly 2x headroom.
Verification
This PR is its own test:
pull_requestruns the workflow from the branch, so the checks below are the new pipeline. Worth eyeballing before merge:blacksmith-2vcpuprevious WPleg resolves and prints the version it pickedAlso fixed in passing: the matrix
includesetwordpress: 'previous major version'as a literal string, which rendered job names like "PHP 8.4 (WP previous major version)"; the'**/node_modules'cache glob replaced with an explicit path.Separately, Docker container caching should be turned on in the Blacksmith dashboard (currently off, and free — it is exempt from sticky-disk billing). It targets
wp-env start, which is 53-88s in every test job and the largest fixed cost left after this change.