Skip to content
Rekor Monitor

Rekor Monitor #1156

Sign in to view logs

Rekor Monitor

Rekor Monitor #1156

Workflow file for this run

.github/workflows/rekor-monitor.yaml at 709b179
# Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Transparency-log monitoring for AICR's release supply chain, on Rekor v2.
#
# Hourly, this runs our own monitor (tools/rekor-monitor) to do both checks in a
# single job:
# - Consistency: prove the Rekor v2 log stays append-only between runs (Merkle
# consistency from the last checkpoint to the current tree head). O(log n),
# finishes in seconds. The checkpoint (cursor) is persisted as the
# `rekor-v2-checkpoint` artifact between runs.
# - Identity: scan entries added since the last checkpoint for AICR's release
# signing identity. An entry under that identity that a release did not
# produce signals OIDC/key compromise. The scan is resumable: a large backlog
# window (from a multi-hour outage or a held cursor) is scanned in bounded
# slices across several runs, persisting progress in a checkpoint companion,
# rather than re-scanning (and timing out on) the whole window every pass. The
# signed checkpoint advances only once the scan has caught up to head.
# The monitor exits non-zero on any failure and prints a CLASSIFICATION line, and
# this workflow branches notifications on it so infra flakiness never pages like a
# security event:
# - tamper / identity (a real consistency break or an unexplained identity hit):
# open a security tracking issue (mentioning the maintainers group) and post a
# Slack alert (via the SLACK_SERVICE webhook).
# - operational (Sigstore/Rekor/TUF/GitHub-API trouble, or a crash before any
# classification) or degraded (the identity catch-up is not converging: the
# log is outpacing the bounded per-run scan, so `remaining` is not shrinking):
# no page. Only after three consecutive failed scheduled runs does it open a
# low-urgency "degraded" issue; a single blip stays quiet.
# The job still goes red on every failure. A later clean run closes both the
# security and degraded issues. The tags a real release actually signed are fetched
# first from the RELEASE_WORKFLOW_FILE (on-tag.yaml) run history and passed to the
# tool (--known-tags-file), so an entry under the release identity that a real
# release produced is not flagged. Run history is used rather than the repo's
# current tags/releases because those are deleted for ephemeral release candidates
# (NVIDIA/aicr#1902) while run history persists, and runs count in ANY lifecycle
# state so a release that is still signing is not briefly invisible to the scan
# (NVIDIA/aicr#2153). Note: a workflow that fails to *start* (startup_failure)
# cannot alert on itself, so an external liveness check (dead-man's-switch) is
# tracked as a follow-up.
#
# Why our own tool, not upstream sigstore/rekor-monitor (NVIDIA/aicr#1623).
# Identity monitoring is a linear scan of every entry added since the last
# checkpoint (Rekor's index cannot be queried by certificate SAN, and our keyless
# release identity has no email or fixed key). On the Rekor **v1** firehose that
# scan runs ~50x slower than the log grows, so it can never keep up in a bounded
# CI job. Rekor **v2** is tile-based: bulk 256-entry reads make a single-worker
# scan outpace the log, so the whole thing is one cheap job. This rides on
# release signing having moved to v2 in NVIDIA/aicr#1650.
#
# The catch: the upstream reusable workflow selects its Rekor version and shards
# from Sigstore's *default* signing config (`signing_config.v0.2.json`), which is
# v1-only and stays that way "for the foreseeable future". AICR opted into v2
# early via the separate `signing_config_rekor_v2.v0.2.json` TUF target, which
# upstream never reads and cannot be pointed at. So we run tools/rekor-monitor:
# it reads the v2 signing config AICR actually signs against (pkg/trust) and
# reuses the upstream rekor-monitor *library* packages for the verification. When
# Sigstore makes v2 the ecosystem default, upstream can monitor v2 directly and
# this tool can be retired. See docs/contributor/maintaining.md and the tool's
# package doc for the full rationale.
#
# The monitored identity is AICR's release signer (see .goreleaser.yaml and
# .github/workflows/on-tag.yaml): the GitHub Actions OIDC SAN for on-tag.yaml,
# issued by token.actions.githubusercontent.com.
#
# Triage, by classification:
# - Identity hit (security issue + page): known release tags are already
# excluded via --known-tags-file, so a hit means an entry under the release
# identity for a tag with *no corresponding release* (or an entry that failed
# verification). Cross-check the entry's log index and timestamp against known
# release runs; if unrecognized, treat as potential OIDC/key compromise and
# begin incident response. Residual gap: an attacker re-signing an *existing*
# release tag would be suppressed (a per-tag entry-count/provenance check is a
# follow-up).
# - Tamper / consistency failure (security issue + page): the Merkle consistency
# proof failed. Re-run once to rule out a corrupt cached proof; a persistent
# break is a tamper signal: escalate to Sigstore.
# - Operational (no page): a red hourly job with no issue is a transient infra
# blip (Sigstore/Rekor/TUF/GitHub-API) that self-heals; the "degraded" issue
# appears only after three consecutive failed runs and needs no action unless
# it persists.
# - Degraded (no page, but real): the same issue title also covers a catch-up
# that is NOT converging (the log outpacing the bounded per-run scan). Here the
# monitor completed every pass; it will NOT self-heal. The issue body says so
# and gives the remediation (more scan budget per run, or triage a held finding
# pinning the cursor).
name: Rekor Monitor
on:
schedule:
- cron: "17 * * * *" # hourly, offset off the top of the hour
workflow_dispatch: {}
permissions: {}
concurrency:
group: rekor-monitor
cancel-in-progress: false
env:
CHECKPOINT_FILE: checkpoint_v2.txt
# Cross-run cursor. A fresh name (not the old v1 `checkpoint`) means the stale
# v1 checkpoint is simply ignored; no migration needed.
ARTIFACT_NAME: rekor-v2-checkpoint
ALERT_TITLE: "Rekor v2 monitor: release identity / log consistency alert"
DEGRADED_TITLE: "Rekor v2 monitor: degraded"
# Maintainers group notified on an alert (GitHub team; issues cannot be
# assigned to a team, so it is @-mentioned in the issue body instead).
MAINTAINERS_TEAM: "@nvidia/aicr-maintainer"
# AICR release signer identity (see .goreleaser.yaml and on-tag.yaml). Kept in
# the workflow so it is the auditable source of truth for what is monitored.
CERT_SUBJECT: '^https://github\.com/NVIDIA/aicr/\.github/workflows/on-tag\.yaml@refs/tags/.*$'
CERT_ISSUER: '^https://token\.actions\.githubusercontent\.com$'
# The release signing workflow whose run history is the correlation source for
# "which tags a real release actually signed" (see the "Fetch signed release
# tags" step). MUST name the same workflow as CERT_SUBJECT above: the cert SAN
# is <this-workflow>@refs/tags/<tag>, so its run history is the authoritative,
# deletion-proof record of legitimate signings. Change both together.
RELEASE_WORKFLOW_FILE: on-tag.yaml
jobs:
monitor:
name: AICR release identity + log consistency (Rekor v2)
runs-on: ubuntu-latest
# Backstop the per-pass context timeout in the tool (defaultTimeout, 45m):
# the job is killed if a network stall somehow outlives it. Sized so the
# tool's own 45m deadline (which classifies as operational) fires first, and
# so the post-monitor notification steps still have room, rather than a hard
# job kill. Paranoid worst case: three gh-api-retry fetches each exhausting
# 3 x 180s attempts (~27m), the 45m monitor pass, checkout/setup-go (~3m), and
# the post-monitor issue/Slack/degraded steps (~a few min of gh/curl). Note
# the fetch-outage (~27m) and backlog-scan (45m) worst cases do not actually
# stack (an outage that slows fetches also makes the monitor fail fast), so
# 90m leaves generous headroom for the realistic large-backlog case.
timeout-minutes: 90
permissions:
contents: read # checkout to build tools/rekor-monitor
actions: read # read the prior checkpoint artifact via the Actions API
issues: write # open/close the alert issue
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: go.mod
- name: Fetch previous checkpoint artifact
# Cross-run artifact fetch via the API (download-artifact only sees the
# current run). A missing artifact is the expected first-run state, and
# leaves checkpoint.zip absent so the monitor baselines. The monitor
# reads the zip natively (--restore-zip); no unzip here.
#
# SECURITY: the repo-wide `artifacts?name=` list also returns same-named
# artifacts uploaded by *any* run, including a fork pull_request run whose
# GITHUB_TOKEN can upload under any name. A poisoned checkpoint (a genuine
# one copied from the live log head) would pass the consistency proof
# while collapsing the identity-scan window to empty, silently skipping
# entries. We therefore accept only artifacts whose producing run was in
# THIS repository on `main`:
# - head_repository_id == repository_id rejects every fork run (a fork's
# head repo differs), even one whose branch is named "main";
# - head_branch == "main" limits to the scheduled/dispatched main runs.
# The whole artifact history is paginated (not just recent runs) so an
# older valid cursor survives a long stretch of artifact-less runs
# (startup failures, outages); a failed API call aborts the step (set -e /
# pipefail on the assignment) rather than falling through to a re-baseline.
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
# .github/scripts/gh-api-retry.sh writes each attempt to OUTFILE.part and
# publishes it only on success (recovers 5xx/429, round-trips the binary
# zip, never leaks a failed attempt's error body). A total failure aborts
# the step (operational, never a security signal).
.github/scripts/gh-api-retry.sh artifacts.json --paginate \
"repos/${GITHUB_REPOSITORY}/actions/artifacts?name=${ARTIFACT_NAME}&per_page=100"
id="$(jq -rs '[.[].artifacts[]
| select(.expired == false)
| select(.workflow_run.head_branch == "main")
| select(.workflow_run.head_repository_id == .workflow_run.repository_id)]
| sort_by(.created_at) | last | .id // empty' artifacts.json)"
if [ -z "${id}" ]; then
echo "No prior checkpoint artifact from this repository on main; treating this as the first run."
exit 0
fi
.github/scripts/gh-api-retry.sh checkpoint.zip "repos/${GITHUB_REPOSITORY}/actions/artifacts/${id}/zip"
echo "Fetched checkpoint artifact ${id}."
- name: Fetch signed release tags
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
# The correlation source is the RELEASE_WORKFLOW_FILE run history, NOT
# the repo's current tags. The identity SAN is <workflow>@refs/tags/<tag>,
# so a tag-push run of that workflow is the authoritative proof that a
# real release signed <tag>. Unlike /tags or /releases, run history
# PERSISTS after a tag/release is deleted, so an ephemeral release
# candidate (e.g. vX.Y.Z-rc1, whose tag+release are cleaned up once the
# final ships) is still recognised and does not trip the identity alarm
# (NVIDIA/aicr#1902). head_branch is the tag for a tag-push run.
#
# Runs count in ANY lifecycle state and with any conclusion; the query is
# deliberately unfiltered on `status`. GitHub mints the SAN only while a
# run of this workflow is executing at refs/tags/<tag>, so the run's
# existence at that ref IS the proof; its lifecycle state is not part of
# the identity. Filtering on status=completed hid a release for the ~32
# minutes it takes to run while its signing jobs were already writing
# entries ~20 minutes in, which is exactly where the hourly scan lands
# (NVIDIA/aicr#2153). It also let a re-run of a job that signs nothing
# (a flaky post-publish deploy) revoke a tag's allowlist entry by
# flipping the run back to in_progress. Gating on conclusion=success is
# likewise not viable: signing happens mid-run, so a real release whose
# later step flakes concludes "failure" while still having produced a
# legitimate entry (e.g. v0.18.0 itself), and gating would re-create the
# #1902 false positive. Widening to in-flight runs does not weaken the
# control: under the old gate, anyone able to push a tag could already
# allowlist it by waiting for the run to finish. Per-signing-step
# correlation would be tighter but needs per-run job inspection; see the
# maintaining.md residual note.
# De-duplicate. The reusable retry is already factored into the shared
# gh-api-retry.sh; this fetch+jq is specific to this monitor's
# correlation, so it stays inline (a composite action for one call site
# would add indirection without reuse).
.github/scripts/gh-api-retry.sh runs.json --paginate \
"repos/${GITHUB_REPOSITORY}/actions/workflows/${RELEASE_WORKFLOW_FILE}/runs?event=push&per_page=100"
jq -rs '[.[].workflow_runs[] | select(.head_branch != null) | .head_branch] | unique | .[]' \
runs.json > known-tags.txt
echo "Fetched $(wc -l < known-tags.txt) signed release tags from ${RELEASE_WORKFLOW_FILE} run history."
- name: Monitor Rekor v2 (consistency + identity)
id: monitor
run: |
set -uo pipefail
set +e
GOFLAGS="-mod=vendor" go run ./tools/rekor-monitor \
--file "${CHECKPOINT_FILE}" \
--restore-zip checkpoint.zip \
--cert-subject "${CERT_SUBJECT}" \
--cert-issuer "${CERT_ISSUER}" \
--known-tags-file known-tags.txt | tee monitor.out
code=${PIPESTATUS[0]}
class="$(grep -oE '^CLASSIFICATION=[a-z]+' monitor.out | tail -n1 | cut -d= -f2)"
# A crash before any CLASSIFICATION line (build error, OOM) is treated
# as operational: infra trouble must never masquerade as a security hit.
[ -n "${class}" ] || class="operational"
echo "classification=${class}" >> "$GITHUB_OUTPUT"
exit "${code}"
- name: Persist checkpoint
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ env.ARTIFACT_NAME }}
# The signed checkpoint plus its two companions (<file>.scan and
# <file>.stall): the tool advances the checkpoint only once the identity
# scan has caught up to head, persists partial progress in .scan so a
# large backlog is scanned across several runs, and tracks catch-up
# convergence in .stall so a diverging (non-converging) catch-up can page.
# All must travel in the artifact so the next run resumes and the trend
# continues. Every advance path writes .scan (advanceCheckpoint resets it
# to 0) and clears .stall, so the files are absent only when the run fails
# before it can advance the cursor or save a chunk (e.g. a first run whose
# consistency check errors); if-no-files-found: ignore covers that so a
# pre-advance failure does not fail the upload.
path: |
${{ env.CHECKPOINT_FILE }}
${{ env.CHECKPOINT_FILE }}.scan
${{ env.CHECKPOINT_FILE }}.stall
# Must exceed the hourly cadence so the cursor never expires between
# runs; wide enough to survive a brief scheduling pause.
retention-days: 30
if-no-files-found: ignore
- name: Open security alert issue
id: alert_issue
if: ${{ failure() && (steps.monitor.outputs.classification == 'tamper' || steps.monitor.outputs.classification == 'identity') }}
env:
GH_TOKEN: ${{ github.token }}
CLASSIFICATION: ${{ steps.monitor.outputs.classification }}
run: |
set -euo pipefail
existing="$(gh issue list --repo "${GITHUB_REPOSITORY}" --state open \
--search "in:title \"${ALERT_TITLE}\"" --json title,url \
| jq -r --arg t "${ALERT_TITLE}" '[.[] | select(.title == $t)] | .[0].url // empty')"
if [ -n "${existing}" ]; then
echo "Alert issue already open: ${existing}"
echo "url=${existing}" >> "$GITHUB_OUTPUT"
exit 0
fi
run_url="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
if [ "${CLASSIFICATION}" = "tamper" ]; then
detail="the Rekor v2 **consistency proof failed** (the log's Merkle root did not verify as append-only)."
detail="${detail} This is a tamper signal: **escalate to Sigstore** and do not advance the checkpoint."
detail="${detail} Re-run once to rule out a corrupt cached proof; a persistent failure is a genuine break."
else
detail="an entry was found under **AICR's release signing identity for a tag with no corresponding release** (or an entry that failed verification)."
detail="${detail} Known release tags are already excluded, so treat this as potential **OIDC/key compromise** and begin incident response:"
detail="${detail} cross-check the entry's log index and timestamp in the run logs against known release runs."
fi
{
echo "${MAINTAINERS_TEAM}: the Rekor v2 monitor flagged a security signal on [run ${GITHUB_RUN_ID}](${run_url})."
echo
echo "${detail}"
echo
echo "See the run logs and the triage notes in the workflow header (\`.github/workflows/rekor-monitor.yaml\`)."
echo
echo "This issue auto-closes on the next clean run."
} > alert-body.md
url="$(gh issue create --repo "${GITHUB_REPOSITORY}" \
--title "${ALERT_TITLE}" \
--label "area/security,theme/supply-chain" \
--body-file alert-body.md)"
echo "Opened alert issue: ${url}"
echo "url=${url}" >> "$GITHUB_OUTPUT"
- name: Post Slack alert on security finding
if: ${{ failure() && (steps.monitor.outputs.classification == 'tamper' || steps.monitor.outputs.classification == 'identity') }}
env:
# Slack incoming-webhook path suffix; same secret the release and
# vuln-scan workflows use. Unset in forks, so this no-ops there.
SLACK_SERVICE: ${{ secrets.SLACK_SERVICE }}
ISSUE_URL: ${{ steps.alert_issue.outputs.url }}
CLASSIFICATION: ${{ steps.monitor.outputs.classification }}
run: |
set -euo pipefail
if [ -z "${SLACK_SERVICE:-}" ]; then
echo "::warning::SLACK_SERVICE not set; skipping Slack notification"
exit 0
fi
run_url="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
# To ping a Slack usergroup, prepend "<!subteam^GROUP_ID> " below.
text=":rotating_light: *Rekor v2 monitor: ${CLASSIFICATION} signal*: needs maintainer triage."
text="${text} <${run_url}|View run>"
if [ -n "${ISSUE_URL:-}" ]; then
text="${text} · <${ISSUE_URL}|tracking issue>"
fi
jq -n --arg t "${text}" '{text: $t}' > slack-payload.json
curl -sSf -X POST -H 'Content-type: application/json' \
--data @slack-payload.json \
"https://hooks.slack.com/services/${SLACK_SERVICE}"
echo "Posted Slack alert."
- name: Open degraded issue if a non-security failure is persistent
# Inverse of the security allowlist: any failure that is NOT a confirmed
# tamper/identity signal is operational. This deliberately also covers an
# EMPTY classification, i.e. a step BEFORE the monitor failed (tags fetch,
# checkpoint fetch) so the monitor never ran and printed no CLASSIFICATION.
# Those GitHub-API/upstream outages are exactly what the degraded path is
# for; without this they would keep the job red hourly yet never surface a
# calm tracking issue. The two gates stay mutually exclusive and together
# partition every failure() case.
if: ${{ failure() && steps.monitor.outputs.classification != 'tamper' && steps.monitor.outputs.classification != 'identity' }}
env:
GH_TOKEN: ${{ github.token }}
CLASSIFICATION: ${{ steps.monitor.outputs.classification }}
run: |
set -euo pipefail
# Persistence via run history: only escalate once this run plus the two
# prior scheduled runs all concluded failure. A single blip stays quiet
# (job is red, but no issue/Slack) and self-heals next hour. The artifact
# API is one of the flaky deps we are hardening, so we intentionally do
# NOT use a state-file counter here.
# The API returns completed runs most-recent-first (this in-progress run
# is excluded), so index("success") is the count of consecutive leading
# failures; `// length` handles the all-failures window. This is a true
# consecutive streak, not a plain failure count: [fail,success,fail]
# yields 1 (stays quiet), not 2.
streak="$(gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/rekor-monitor.yaml/runs?event=schedule&status=completed&per_page=3" \
| jq -r '[.workflow_runs[].conclusion] | (index("success") // length)')"
# This run is not yet "completed" in the API, so 2 consecutive prior
# failures + this one = 3 consecutive = persistent.
if [ "${streak}" -lt 2 ]; then
echo "Operational failure but not yet persistent (prior failures: ${streak}); staying quiet."
exit 0
fi
existing="$(gh issue list --repo "${GITHUB_REPOSITORY}" --state open \
--search "in:title \"${DEGRADED_TITLE}\"" --json title,url \
| jq -r --arg t "${DEGRADED_TITLE}" '[.[] | select(.title == $t)] | .[0].url // empty')"
if [ -n "${existing}" ]; then
echo "Degraded issue already open: ${existing}"
exit 0
fi
run_url="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
# The degraded sink covers two very different states; write the body for
# the one that actually happened. A catch-up stall is NOT an upstream blip
# that self-heals -- the monitor is completing every pass -- so its
# remediation is concrete, not "wait for upstream".
if [ "${CLASSIFICATION}" = "degraded" ]; then
opening="The Rekor v2 monitor's identity catch-up is **not converging** (latest: [run ${GITHUB_RUN_ID}](${run_url}))."
opening="${opening} The transparency log is growing faster than the bounded per-run scan, so \`remaining\` is not shrinking;"
opening="${opening} the monitor **is** completing each pass and persisting progress."
detail="This is **not** a security finding (no tamper or identity signal). It will **not** self-heal on its own."
detail="${detail} Remediate by giving the scan more budget per run (raise the pass \`--timeout\` or lower \`scanBudgetHeadroom\`),"
detail="${detail} or by triaging a held identity finding that is pinning the cursor. See the catch-up notes in \`docs/contributor/maintaining.md\`."
else
opening="The Rekor v2 monitor has been **unable to complete** for 3+ consecutive hourly runs (latest: [run ${GITHUB_RUN_ID}](${run_url}))."
detail="This is an **operational** problem (Sigstore/Rekor/TUF reachability or the GitHub API), **not** a security finding: no tamper or identity signal was detected."
detail="${detail} No maintainer action is required unless it persists; the check will resume automatically when upstream recovers."
fi
{
echo "${opening}"
echo
echo "${detail}"
echo
echo "This issue auto-closes on the next clean run."
} > degraded-body.md
gh issue create --repo "${GITHUB_REPOSITORY}" \
--title "${DEGRADED_TITLE}" \
--label "area/ci" \
--body-file degraded-body.md
- name: Close alert issue on success
if: ${{ success() }}
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
# Exact-title filter: title search is phrase-based, so never close an
# issue whose title merely contains the alert text.
gh issue list --repo "${GITHUB_REPOSITORY}" --state open \
--search "in:title \"${ALERT_TITLE}\"" --json number,title \
| jq -r --arg t "${ALERT_TITLE}" '.[] | select(.title == $t) | .number' \
| while read -r n; do
[ -n "$n" ] || continue
# Tolerate a per-issue close failure (transient API error, rate
# limit, already-closed race): it must not abort the loop or fail
# this step and flip a clean monitor run to "failed".
gh issue close "$n" --repo "${GITHUB_REPOSITORY}" \
--comment "Rekor v2 monitor completed cleanly on run ${GITHUB_RUN_ID}; auto-closing." \
|| echo "::warning::failed to close alert issue #${n}"
done
# Same for the degraded (operational) issue: a clean run means the
# monitor completed, so any open degraded tracker is stale.
gh issue list --repo "${GITHUB_REPOSITORY}" --state open \
--search "in:title \"${DEGRADED_TITLE}\"" --json number,title \
| jq -r --arg t "${DEGRADED_TITLE}" '.[] | select(.title == $t) | .number' \
| while read -r n; do
[ -n "$n" ] || continue
# Tolerate a per-issue close failure (transient API error, rate
# limit, already-closed race): it must not abort the loop or fail
# this step and flip a clean monitor run to "failed".
gh issue close "$n" --repo "${GITHUB_REPOSITORY}" \
--comment "Rekor v2 monitor completed cleanly on run ${GITHUB_RUN_ID}; auto-closing." \
|| echo "::warning::failed to close degraded issue #${n}"
done