Community AI RMF profile examples in XML (rework of #333)#338
Open
eeee2345 wants to merge 1 commit into
Open
Conversation
Four worked-example profiles (baseline, tier 1/2/3) authored as XML in src/examples/profile/xml/, reworked from usnistgov#333 per the guidance in that thread and the example checklist in usnistgov#337. JSON/YAML left for the CI/CD pipeline. Each profile imports a placeholder catalog path (../../catalog/xml/ai-rmf_catalog.xml) pending the official NIST AI RMF OSCAL catalog. CC0 1.0; not endorsed by NIST.
eeee2345
force-pushed
the
contrib/ai-rmf-profile-xml
branch
from
475dfcd to
530f145
Compare
Author
|
Hi @iMichaela. Reworked per your 5/21 guidance, now ready for review. The three points you raised:
The four profiles are one baseline (include-all 72 subcategories) plus three tier examples. Metadata marks them as community examples, not NIST-produced. One open question on direction: since @selenaxiao-nist's official AI RMF catalog is still in progress, these profiles currently reference the community catalog. Would you prefer they stay pointed there for now, or hold until the official catalog lands so they can reference it directly? Happy to go either way, and to adjust placement or structure if this is not the pattern you had in mind. |
Contributor
|
Thank you for your contribution! Please hold until the official AI RMF catalog. |
eeee2345
added a commit
to Agent-Threat-Rule/agent-threat-rules
that referenced
this pull request
Jun 14, 2026
The ecosystem entry pointed to the wrong repo (usnistgov/OSCAL root). The live, in-review submission is usnistgov/oscal-content#338 ("Community AI RMF profile examples in XML, rework of #333"), which is what every page already cites. Now consistent across stats.ts and all pages. Still framed as "submission in review, not a NIST endorsement."
eeee2345
added a commit
to Agent-Threat-Rule/agent-threat-rules
that referenced
this pull request
Jun 14, 2026
…metrics) (#159) * docs(website): sync every page to current state (651 rules, v3.4.0, honest metrics) Full-site audit + sync after the npm 3.4.0 / 651-rule publish. Every page checked against the verified current state; honesty boundaries preserved. Facts corrected: - 'largest malware campaign / 751' reconciled everywhere to the honest split: 1,302 flagged across 96,096 scanned, 552 confirmed after manual review (home, research, red-team, mega-scan-report.json) — matches layout metadata. - npm download card 23K -> 2.3K (real last-30-day = 2,290; the 23K was a ~10x overstatement that would undercut trust with adopters). - hades blog: 464 -> 651 detection rules; install pinned to v3.4.0. - quality-standard: PINT precision 99.6% -> 99.7%; npm pins v3.4.0; Cisco line de-stale'd. about / nist-ai-rmf: v3.3.x -> v3.4.0. - stats.ts Cisco detail: dropped the stale '(314 at time of PR #99)' snapshot. Disclosed (new, verified): npm 3.4.0 live with 651 rules; the red-team + CVE flywheels now running toward daily updates; auto-crystallization 462 -> 651. Deliberately untouched (correct/honest): spec doc version 3.0.0-alpha.1 (distinct from the rule package version); version-pinned 3.0.0 benchmark numbers; NIST OSCAL 'submission in review' (never adoption/endorsement); Microsoft/Cisco framed as USE not endorsement; dynamic ruleCount/benchmarks. Verified: next build PASS (1372/1372 static pages, exit 0). * docs(website): correct ecosystem integration statuses against live GitHub Live-verified all 30 ecosystem/red-team integrations via GitHub API: - PyRIT #1715 + ProjectRecon #17: open -> merged (now actually merged) - Remove 4 closed PRs that never landed: corca-ai/awesome-llm-security #117, cisco-ai-defense/mcp-scanner #151, Portkey-AI/gateway #1652, IBM/mcp-context-forge #4109 - Fix NIST OSCAL broken URL (#2234 was 404 -> repo root); keep honest "submission in review, not a NIST endorsement" framing - Net: 13 merged / 26 total, every status verified live. Counts render dynamically so home/wall/integrate auto-update. * docs(website): fix NIST OSCAL reference to live oscal-content#338 The ecosystem entry pointed to the wrong repo (usnistgov/OSCAL root). The live, in-review submission is usnistgov/oscal-content#338 ("Community AI RMF profile examples in XML, rework of #333"), which is what every page already cites. Now consistent across stats.ts and all pages. Still framed as "submission in review, not a NIST endorsement." * docs(website): surface 2026-06 red-team mega-scan on /red-team The evidence table showed only the 2026-05-12 five-corpus sprint while the narrative already referenced the mega-scan flywheel. Add the recent 8-corpus mega-scan as its own block with verified provenance: - 8 corpora ingested (LLMail-Inject, TensorTrust, InjecAgent, AgentDojo, PoisonedRAG, ToolEmu, MCPSecBench, AgentPoison) - 6 produced rules; 29 distinct detection rules (de-duplicated across overlapping corpus attribution) - 2 (TensorTrust, PoisonedRAG) yielded no novel misses beyond existing coverage - Honest footnote on shared attribution + mixed maturity; no fabricated per-corpus recall (no canonical measurement file exists for those) --------- Co-authored-by: eeee2345 <eeee2345@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #333, reworked to match the repo conventions (XML in src/) noted in that thread and in the example checklist added in #337.
Four AI RMF community profiles, now authored as XML in src/examples/profile/xml/ (OSCAL 1.2.2), each validated against oscal_complete_schema.xsd:
JSON/YAML are left for the CI/CD pipeline. The catalog JSON, the workflow, and the python tests from the earlier PR are removed. All four are community worked examples (CC0 1.0), marked "Not produced by, endorsed by, or affiliated with NIST".
One point I'd like your guidance on: each profile imports a catalog, and the AI RMF catalog is not in the repo yet. For now the imports use a placeholder path (../../catalog/xml/ai-rmf_catalog.xml). Once an official AI RMF catalog is available here, I will repoint the imports to it.
Into the feature-ai-rmf-profile-examples branch you set up.