docs(ai): canbench results are regenerated manually, not by automation#13511
Open
AntonioVentilii wants to merge 2 commits into
Open
docs(ai): canbench results are regenerated manually, not by automation#13511AntonioVentilii wants to merge 2 commits into
AntonioVentilii wants to merge 2 commits into
Conversation
docs/ai/governance.md listed canbench_results.yml among 'tracked artifacts updated by automation', but no workflow runs canbench — the file is only updated when a developer runs npm run benchmark and commits the result, as docs/ai/backend/testing.md correctly states. Split it into its own row with the accurate regeneration instructions so agents don't wait for an automation that doesn't exist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GiZ8mmts9aZCCANKQfQ1Xi
|
✅ No security or compliance issues detected. Reviewed everything up to 52482fc. Security Overview
Detected Code Changes
|
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the AI governance documentation to accurately reflect how canbench_results.yml is maintained, aligning the “protected paths” table with the repo’s actual benchmark workflow (manual regeneration via npm run benchmark, not CI automation).
Changes:
- Removes
canbench_results.ymlfrom the “Tracked artifacts updated by automation” entry. - Adds a dedicated table row clarifying
canbench_results.ymlis manually regenerated vianpm run benchmark. - Links the governance entry to the canonical canbench documentation section in
docs/ai/backend/testing.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
An audit of the benchmark/performance tooling found a documentation inconsistency:
docs/ai/governance.mdlistscanbench_results.ymlamong "tracked artifacts updated by automation", but no workflow anywhere in.github/workflows/runs canbench. The file only changes when a developer runsnpm run benchmarkand commits the result — which is whatdocs/ai/backend/testing.md("active but lightly maintained — not gated by CI yet") anddocs/ai/pr-and-ci.mdcorrectly describe. The governance table is the entry point agents use to decide whether a file self-updates, so the wrong claim can make them (and humans) assume benchmark drift is handled when it isn't.Changes
canbench_results.ymlout of the "updated by automation" row indocs/ai/governance.mdinto its own row stating it is manually regenerated vianpm run benchmark, with a link to the canbench section ofbackend/testing.md.Tests
Docs-only change;
prettier --checkpasses. (Separately worth considering, not in this PR: actually gating canbench in CI, since results currently drift silently between manual runs.)🤖 Generated with Claude Code
https://claude.ai/code/session_01GiZ8mmts9aZCCANKQfQ1Xi
Generated by Claude Code