ci(smoke): keep the compositor alive between hardware cases #68
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: smoke | |
| # Launch the real Waybar binary (built with AddressSanitizer) inside a headless | |
| # compositor and exercise it: render, per-module matrix, pointer interaction, | |
| # accessibility-tree assertions, and a layout matrix. Complements the unit | |
| # tests and build jobs, which never actually run the bar. | |
| on: | |
| push: | |
| branches: [master] | |
| pull_request: | |
| workflow_dispatch: | |
| inputs: | |
| bless: | |
| description: "Regenerate and commit the golden reference screenshot" | |
| type: boolean | |
| default: false | |
| permissions: | |
| contents: write | |
| concurrency: | |
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} | |
| cancel-in-progress: true | |
| env: | |
| WAYBAR_BIN: ${{ github.workspace }}/build/waybar | |
| jobs: | |
| smoke: | |
| runs-on: ubuntu-latest | |
| # Safety net: the whole smoke job is minutes of work. Cap it so a hung tier | |
| # (e.g. a daemon left running under a bare `wait`) fails fast instead of | |
| # burning the default 6h runner budget. | |
| timeout-minutes: 20 | |
| steps: | |
| - uses: actions/checkout@v6 | |
| - name: Install dependencies | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install --no-install-recommends -y \ | |
| sway labwc grim imagemagick fonts-dejavu-core jq \ | |
| at-spi2-core python3-pyatspi dbus-x11 \ | |
| mpd mpc pulseaudio pulseaudio-utils umockdev \ | |
| meson ninja-build g++ pkg-config scdoc wayland-protocols \ | |
| gobject-introspection libdbusmenu-gtk3-dev libevdev-dev libfmt-dev \ | |
| libgirepository1.0-dev libgtk-3-dev libgtkmm-3.0-dev libinput-dev \ | |
| libjsoncpp-dev libmpdclient-dev libnl-3-dev libnl-genl-3-dev \ | |
| libpulse-dev libsigc++-2.0-dev libspdlog-dev libwayland-dev \ | |
| upower libxkbregistry-dev libxkbcommon-dev | |
| # Cheap static gate (source-only): every Factory module must be classified | |
| # for smoke coverage. Runs before the build so it fails fast. | |
| - name: Module coverage tripwire | |
| run: ./test/smoke/coverage.sh | |
| - name: Build Waybar with AddressSanitizer | |
| run: | | |
| # _GLIBCXX_ASSERTIONS turns libstdc++ container bounds/precondition | |
| # violations (e.g. out-of-range vector access) into aborts under ASan. | |
| meson setup build -Dman-pages=disabled -Db_sanitize=address \ | |
| -Dcpp_args=-D_GLIBCXX_ASSERTIONS | |
| ninja -C build | |
| # --- Tier 0 + 2: render + sanitizer/critical gate --- | |
| - name: Render deterministic config | |
| run: ./test/smoke/run.sh test/smoke/config.jsonc test/smoke/style.css bar.png | |
| - name: Check the bar is not blank | |
| run: | | |
| sd=$(convert bar.png -crop 1920x30+0+0 +repage -colorspace Gray \ | |
| -format '%[fx:standard_deviation]' info:) | |
| echo "bar strip stddev = $sd" | |
| awk "BEGIN{ exit !($sd > 0.01) }" || { echo "::error::bar looks blank"; exit 1; } | |
| # --- Manual re-bless of the reference image --- | |
| - name: Bless reference screenshot | |
| if: github.event_name == 'workflow_dispatch' && inputs.bless | |
| run: | | |
| cp bar.png test/smoke/reference.png | |
| git config user.name "github-actions[bot]" | |
| git config user.email "41898282+github-actions[bot]@users.noreply.github.com" | |
| git add test/smoke/reference.png | |
| git commit -m "ci(smoke): re-bless reference screenshot [skip ci]" | |
| git push | |
| - name: Golden comparison | |
| run: | | |
| if [ -f test/smoke/reference.png ]; then | |
| ae=$(compare -metric AE -fuzz 8% test/smoke/reference.png bar.png diff.png 2>&1 || true) | |
| ae=$(printf '%s' "$ae" | grep -oE '^[0-9]+' || echo 0) | |
| echo "differing pixels (fuzz 8%): $ae" | |
| [ "$ae" -gt 800 ] && { echo "::error::visual regression: $ae px (see diff.png artifact)"; exit 1; } || echo "✓ within tolerance" | |
| else | |
| echo "::warning::No reference image; run this workflow via 'Run workflow' with bless=true to create it." | |
| fi | |
| # --- Tier 1: per-module matrix (standalone + grouped) --- | |
| - name: Per-module render matrix | |
| run: ./test/smoke/modules.sh module-shots | |
| # --- Tier 4: lifecycle / teardown (exit, output hotplug, reload) --- | |
| # Runs the C++ destructor path a plain kill never touches (#5182). | |
| - name: Lifecycle / teardown | |
| run: ./test/smoke/lifecycle.sh | |
| # --- Tier 4: custom-backend fuzz (pathological exec output) --- | |
| - name: Custom-backend fuzz | |
| run: ./test/smoke/fuzz.sh | |
| # --- Tier 5: state transitions with real daemons (mpd/pulseaudio) --- | |
| # Best-effort: daemon setup varies by runner; keep non-blocking for now. | |
| - name: State transitions (mpd / pulseaudio) | |
| continue-on-error: true | |
| run: ./test/smoke/state.sh | |
| # --- Tier 6: fake hardware via umockdev (backlight/slider, battery) --- | |
| # Best-effort: the mock device descriptions may need per-distro tuning. | |
| - name: Fake hardware (umockdev) | |
| continue-on-error: true | |
| run: ./test/smoke/hardware.sh | |
| # --- Leak check (report-only) — we fixed several leaks recently --- | |
| - name: Leak check (LSan, report-only) | |
| continue-on-error: true | |
| run: ./test/smoke/leakcheck.sh | |
| # --- Tier 2: pointer interaction (best-effort) --- | |
| # Headless seats may not expose a pointer capability, so injected clicks | |
| # can be dropped; keep this non-blocking while it is hardened. | |
| - name: Interaction (click + format-alt) | |
| continue-on-error: true | |
| run: ./test/smoke/interact.sh interaction-shots | |
| # --- Tier 2: accessibility assertions --- | |
| - name: Accessibility (AT-SPI) assertions | |
| run: dbus-run-session -- ./test/smoke/a11y.sh | |
| # --- Tier 3: layout matrix (positions + HiDPI) --- | |
| - name: Layout matrix (top/bottom/left + scale 2) | |
| run: ./test/smoke/positions.sh layout-shots | |
| # --- Tier 3: second compositor --- | |
| - name: Launch under labwc | |
| run: COMPOSITOR=labwc ./test/smoke/run.sh test/smoke/config.jsonc test/smoke/style.css labwc.png | |
| - name: Upload screenshots & diffs | |
| uses: actions/upload-artifact@v4 | |
| if: always() | |
| with: | |
| name: waybar-smoke | |
| path: | | |
| bar.png | |
| diff.png | |
| labwc.png | |
| module-shots/ | |
| interaction-shots/ | |
| layout-shots/ | |
| if-no-files-found: warn |