Skip to content

detect_tag_changes.py: normalize RISC-V macros in the change report (#245)#247

Merged
Bill Traynor (wmat) merged 1 commit into
mainfrom
245-normalizer-riscv-macros
Jul 17, 2026
Merged

detect_tag_changes.py: normalize RISC-V macros in the change report (#245)#247
Bill Traynor (wmat) merged 1 commit into
mainfrom
245-normalizer-riscv-macros

Conversation

@wmat

Copy link
Copy Markdown
Collaborator

Part of #245 (item 2 of 3) — the highest-value item.

The tags backend stores raw AsciiDoc (#169), so the reviewer-facing change report compared markup-migrated text as different: mip vs csr:mip[], 11b vs 0b11, -- vs {endash}. On the real riscv-isa-manual reference regeneration (riscv/riscv-isa-manual#3249) this made 199 of 216 "Modified" tags (92%) markup-only false positives, and the tool exited non-zero — which consumers treat as a breaking normative change.

Change

Teach _strip_asciidoc_formatting the semantic macros so a pure markup migration reports as no change:

form normalizes to
csr:NAME[] NAME
csr:NAME[FIELD] NAME.FIELD (field upper-cased)
csr:[FIELD] / csr::[FIELD] FIELD
ext:NAME[] NAME (capitalized: F, D, Sscofpmf, Zicsr)
insn:NAME[] NAME

Also normalizes binary renotation (11b0b11) and en dash (--{endash}).

This is applied only to the review comparison, never as a freshness check — erasing formatting is correct for "did the normative text change" and wrong for "is this file byte up to date", the exact conflation that caused ref/ to drift. --strict still compares prose byte-for-byte and is unaffected.

Validation

Ran the old vs new normalizer against the two committed riscv-isa-manual reference revisions spanning the #3249 macro migration:

The remaining 32 are genuine normative edits (added clauses, table restructures) plus a few markup forms outside this migration's scope (insn: with operands, single-hyphen ranges) that are appropriate for a human to review. make test passes.

Note: the issue's lone ext:f[]F example was misleading — multi-letter extension names render capitalize-first (Sscofpmf, not SSCOFPMF); handled accordingly.

🤖 Generated with Claude Code

The tags backend stores raw AsciiDoc (see #169), so
the reviewer-facing change report compared markup-migrated text as
different: `mip` vs `csr:mip[]`, `11b` vs `0b11`, `--` vs `{endash}`.
On the real riscv-isa-manual reference regeneration (#3249) this made
199 of 216 "Modified" tags (92%) markup-only false positives, and the
tool exited non-zero, which consumers treat as a breaking normative
change.

Teach `_strip_asciidoc_formatting` the semantic macros so a pure markup
migration reports as no change:

    csr:NAME[]        -> NAME
    csr:NAME[FIELD]   -> NAME.FIELD   (field upper-cased, as prose renders)
    csr:[FIELD]       -> FIELD
    csr::[FIELD]      -> FIELD
    ext:NAME[]        -> NAME         (capitalized: F, D, Sscofpmf, Zicsr)
    insn:NAME[]       -> NAME

Also normalize binary renotation (11b -> 0b11) and en dash
(-- -> {endash}), the other two forms in that migration.

This is applied ONLY to the review comparison, never as a freshness
check: erasing formatting is correct for "did the normative text change"
and wrong for "is this file byte up to date" -- the exact conflation
that caused ref/ to drift. Strict mode (--strict) still compares prose
byte-for-byte and is unaffected.

Verified end-to-end against the two committed riscv-isa-manual reference
revisions spanning the #3249 macro migration: the report drops from
Modified 216 to Modified 32, and the ~32 that remain are genuine
normative edits (added clauses, table restructures) plus a few markup
forms outside this migration's scope (insn: with operands, single-hyphen
ranges) that are appropriate for a human to review.

Add isolated fixtures (macros-plain/macros-markup) and make targets
test-tag-changes-macros / test-tag-changes-strict-macros.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Bill Traynor <wmat@riscv.org>
@wmat

Copy link
Copy Markdown
Collaborator Author

Validated against riscv-isa-manual via the manual's exact detect_tag_changes.py invocation on the real #3249 macro migration: the change report drops from Modified 216 to Modified 32. Full evidence: #245.

@wmat Bill Traynor (wmat) added the enhancement New feature or request label Jul 16, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this might be tricky to maintain. It would be better to parse this from the macro definition file or something like that. Fine for now though and a clear improvement even as is.

@rpsene Rafael Sene (rpsene) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that normalization stays confined to the review report while --strict remains byte-exact. +1 to Jordan on eventually sourcing the macro list from the definitions rather than hardcoded regex.

@wmat
Bill Traynor (wmat) merged commit b8dab26 into main Jul 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants