feat(skills): reorganize skills/ into domain suites

.gitignore

@@ -40,6 +40,11 @@ local/
 tests/plt_*
 tests/graders/skills/*.json
 
+# Skill functional-test fixtures — tracked despite the *.jsonl rule above;
+# only the fixtures themselves, not generated results/ (ignored below).
+!skills/**/tests/*_test_cases.jsonl
+skills/**/tests/results/
+
 # Security Files
 *.pem
 *.key

skills/README.md

@@ -0,0 +1,100 @@
+# Skills
+
+This directory holds Agent Skills (Anthropic Agent Skill protocol — YAML
+frontmatter + Markdown body) for OpenJudge. Every `SKILL.md`, wherever it
+lives, is **independently installable**: it carries everything it needs
+inline, so a single `<name>/SKILL.md` folder can be pulled out and installed
+on its own in Claude Code, Cursor, Codex, Hermes, OpenClaw, or any other
+client that speaks the same protocol — you never need this README or a
+suite's own README to make one skill work.
+
+Grouping below **is folder structure only** — there is no code-level registry
+(no `SUITE_REGISTRY`, no `DomainSuite` class). A "domain suite" is just a
+directory of related `<NN-name>/SKILL.md` workflows plus a `README.md` that
+explains how they relate.
+
+## Domain suites (multi-skill, folder-grouped)
+
+| Suite | Skills | Dimension | What it covers |
+|---|---|---|---|
+| [`eval_pipeline/`](eval_pipeline/) | 9 (`00-meta-eval` … `08-bootstrap`) | Horizontal methodology | How to build an evaluation system from scratch — dataset design, metric/grader selection, human-alignment calibration, reporting, plus RAG/prompt-regression/redteam/bootstrap scenarios. Router at `00-meta-eval`. |
+| [`academic-eval/`](academic-eval/) | 4 (`00-academic-router`, `01-paper-review`, `02-bib-verify`, `03-ref-hallucination-arena`) | Vertical domain | Academic paper review, BibTeX verification, and citation-hallucination benchmarking. |
+| [`arena-eval/`](arena-eval/) | 3 (`00-arena-router`, `01-auto-arena`, `02-ref-hallucination-arena`) | Vertical domain | Arena-style model/agent comparison — generic win-rate ranking, and citation-hallucination-specific ranking. |
+| [`openjudge-core/`](openjudge-core/) | 2 (`01-graders-and-pipeline`, `02-rl-reward`) | Horizontal methodology | Direct OpenJudge core-library API: graders/`GradingRunner`/aggregators, and RL reward-signal construction. No router — see the suite's own README for why. |
+
+`eval_pipeline` is the range-of-motion paradigm all the other suites above
+were reshaped to match (three-layer structure, directory-as-grouping, router
+only where sub-skills are genuinely ambiguous, every `<NN-name>/SKILL.md`
+independently installable). See
+[`docs/superpowers/specs/2026-07-09-skills-domain-suite-proposal.md`](../docs/superpowers/specs/2026-07-09-skills-domain-suite-proposal.md)
+for the full design rationale.
+
+### Functional test coverage
+
+`eval_pipeline`, `academic-eval`, and `arena-eval` each have an actor+judge
+functional test harness under their own `tests/` folder — a *runner*
+(shared, parametrized: `eval_pipeline/tests/run_eval_pipeline_skill_tests.py
+--skill-root <suite> --cases <suite>/tests/<suite>_test_cases.jsonl`), a
+suite-specific JSONL fixture, and a testing guide. `openjudge-core` and the
+isolated skills don't have this yet (see each suite's `README.md` under
+"Validating the skills themselves" for the ones that do).
+
+### `ref-hallucination-arena` — intentional duplication, not a bug
+
+`academic-eval/03-ref-hallucination-arena/SKILL.md` and
+`arena-eval/02-ref-hallucination-arena/SKILL.md` start as full copies of the
+same content and are **independently maintained** — no shared `references/`
+directory, no symlink, no sync script. This is a direct consequence of the
+independent-installability constraint: a skill can't rely on content that
+lives outside its own folder. The two copies are free to diverge (the
+academic copy can lean into "how trustworthy are this paper's citations",
+the arena copy into "how does this model compare to others"); duplicated
+content is an accepted cost, not a problem to solve.
+
+## Isolated skills (single-skill, no suite directory)
+
+These have no other skill in this repo referencing them, so they stay at
+`skills/<name>/SKILL.md` with no suite folder — the simplest possible
+grouping (1 skill = 1 domain).
+
+| Skill | What it covers |
+|---|---|
+| [`claude-authenticity/`](claude-authenticity/) | Detect whether an API endpoint is genuinely backed by Claude (vs. a wrapper/proxy/impersonator); extract injected system prompts. Zero OpenJudge dependency. |
+| [`mmx-cli/`](mmx-cli/) | Generate text/image/video/speech/music via the MiniMax AI platform. Third-party CLI wrapper, zero OpenJudge dependency. |
+| [`find-skills-combo/`](find-skills-combo/) | Discover and recommend combinations of skills from the **external** open agent-skills ecosystem to cover a multi-part task. Unrelated to this repo's own suite grouping. |
+
+**Future rule**: if an isolated skill above ever becomes something a domain
+suite depends on or wraps, move the whole folder into that suite's directory
+(same copy-don't-link approach as `ref-hallucination-arena` above, or a
+rename+move if a single owner is clear) rather than keeping it independent
+and cross-linking from two places.
+
+## Breaking change: path migration (this reorg)
+
+This reorg moved 6 previously-published skills to new install paths. There
+are no redirect stubs — installing the old path will 404; use the new path.
+
+| Old path | New path |
+|---|---|
+| `skills/paper-review/` | `skills/academic-eval/01-paper-review/` |
+| `skills/bib-verify/` | `skills/academic-eval/02-bib-verify/` |
+| `skills/ref-hallucination-arena/` | `skills/academic-eval/03-ref-hallucination-arena/` **and** `skills/arena-eval/02-ref-hallucination-arena/` (two independent copies — see above) |
+| `skills/auto-arena/` | `skills/arena-eval/01-auto-arena/` |
+| `skills/openjudge/` | `skills/openjudge-core/01-graders-and-pipeline/` |
+| `skills/rl-reward/` | `skills/openjudge-core/02-rl-reward/` |
+
+`skills/eval_pipeline/`, `skills/claude-authenticity/`, `skills/mmx-cli/`,
+and `skills/find-skills-combo/` are unaffected.
+
+## Adding a new skill
+
+- **Fits an existing suite's methodology or domain?** Add it as
+  `<suite>/<NN-name>/SKILL.md`, update that suite's `README.md`, and add a
+  router entry only if it creates real selection ambiguity with a sibling
+  (see each suite's README for its router-inclusion rationale).
+- **Genuinely stands alone?** Add it as `skills/<name>/SKILL.md` — no suite
+  folder needed.
+- **Either way**, the skill's `SKILL.md` must be installable on its own: only
+  `name` + `description` in frontmatter, no links outside its own folder (or
+  its own suite folder), and any non-`openjudge`/`rl-reward` pip dependency
+  listed explicitly under Prerequisites.

skills/academic-eval/00-academic-router/SKILL.md

@@ -0,0 +1,76 @@
+---
+name: academic-router
+description: >
+  Use when the user wants help with academic papers or citations but it's unclear
+  which specific workflow fits — reviewing a paper, checking a BibTeX file for fake
+  references, or benchmarking multiple LLMs on reference-recommendation accuracy.
+  Also use when the user mentions paper review, peer review, BibTeX verification,
+  citation checking, reference hallucination, or academic literature accuracy and
+  hasn't specified which of those three tasks they mean. This skill is the entry
+  router for the academic-eval suite: it asks one diagnostic question then routes
+  to the right sub-skill.
+---
+
+# Academic Eval Router
+
+Entry router for the `academic-eval` suite. You diagnose what the user actually
+wants and route them to one of three sub-skills. You don't review papers, verify
+BibTeX files, or run arena benchmarks yourself — you're the triage desk.
+
+Each sub-skill is self-contained: it carries inline everything it needs, so it can
+be installed and used on its own.
+
+## Diagnostic Question
+
+Ask (unless the user's request already makes the answer obvious):
+
+```
+To route you correctly, which of these matches what you want?
+
+a) Review a single paper (PDF or LaTeX source) for correctness/quality/novelty
+   — optionally also check its bibliography
+b) Check a standalone .bib file for fabricated or mismatched references
+   (no paper review needed)
+c) Benchmark/compare multiple LLMs on how often they hallucinate references
+   when asked to recommend citations (arena-style, many queries)
+```
+
+**Shortcut rule**: if the user already said "review my paper", "check this PDF",
+"verify this .bib file", or "compare models on reference hallucination", skip the
+question — the routing is already clear from their phrasing.
+
+## Triage Table
+
+| User says / has | Use workflow | What it does |
+|---|---|---|
+| "Review this paper" (PDF or `.tar.gz`/`.zip` TeX source) | `01-paper-review` | Multi-stage review: safety, correctness, quality/novelty score, criticality — optionally + BibTeX check |
+| "Review this paper AND check its references" | `01-paper-review` | Same pipeline with `--bib` set — one run covers both |
+| "Just check this .bib file, no paper" | `02-bib-verify` | Cross-checks every entry against CrossRef/arXiv/DBLP, flags `verified`/`suspect`/`not_found` |
+| "Compare N models on how often they cite fake papers" / "benchmark reference hallucination rate" | `03-ref-hallucination-arena` | Runs many recommendation queries per model, verifies every returned reference, ranks models by hallucination rate |
+| "Compare models on general quality/response, not specifically citations" | — | Not this suite — see the `arena-eval` suite's `01-auto-arena` instead |
+
+## Key distinctions
+
+- **`01-paper-review` vs `02-bib-verify`**: both use the same underlying
+  `cookbooks.paper_review` pipeline. Use `01-paper-review` whenever a paper file
+  exists (even if the *only* thing the user cares about is the bibliography —
+  `--bib_only` mode is documented there). Use `02-bib-verify` only when there is
+  **no paper**, just a loose `.bib` file to sanity-check.
+- **`01-paper-review`/`02-bib-verify` vs `03-ref-hallucination-arena`**: the first
+  two evaluate *one document's* existing references after the fact. The third
+  evaluates *model behavior* — how often a model invents fake citations when
+  asked to recommend some, across a benchmark of queries and models. If the user
+  wants a leaderboard/ranking of models, not a report on one document, route to
+  `03-ref-hallucination-arena`.
+
+## Output
+
+```
+Recommended workflow: `[skill-name]`
+
+Why: [one sentence tying the user's request to the triage table row]
+```
+
+Recommend exactly one workflow. If the request spans two (e.g., "review this
+paper, and separately benchmark 3 models on citation accuracy"), say so
+explicitly and give both, in the order the user would naturally do them.

skills/bib-verify/SKILL.md → skills/academic-eval/02-bib-verify/SKILL.md

@@ -73,5 +73,5 @@ Each reference entry is assigned one of three statuses:
 
 ## Additional resources
 
-- Full pipeline options: [../paper-review/reference.md](../paper-review/reference.md)
-- Combined PDF review + BibTeX verification: [../paper-review/SKILL.md](../paper-review/SKILL.md)
+- Full pipeline options: [../01-paper-review/reference.md](../01-paper-review/reference.md)
+- Combined PDF review + BibTeX verification: [../01-paper-review/SKILL.md](../01-paper-review/SKILL.md)

skills/ref-hallucination-arena/SKILL.md → skills/academic-eval/03-ref-hallucination-arena/SKILL.md

@@ -254,7 +254,7 @@ evaluation_results/ref_hallucination_arena/
 
 ## Additional resources
 
-- Full config examples: [cookbooks/ref_hallucination_arena/examples/](../../cookbooks/ref_hallucination_arena/examples/)
-- Documentation: [docs/validating_graders/ref_hallucination_arena.md](../../docs/validating_graders/ref_hallucination_arena.md)
+- Full config examples: [cookbooks/ref_hallucination_arena/examples/](../../../cookbooks/ref_hallucination_arena/examples/)
+- Documentation: [docs/validating_graders/ref_hallucination_arena.md](../../../docs/validating_graders/ref_hallucination_arena.md)
 - Official dataset: [HuggingFace](https://huggingface.co/datasets/OpenJudge/ref-hallucination-arena)
 - Leaderboard: [openjudge.me/leaderboard](https://openjudge.me/leaderboard)

skills/academic-eval/README.md

@@ -0,0 +1,63 @@
+# Academic Eval — paper review & citation accuracy
+
+A set of skills for academic-paper workflows built on OpenJudge: reviewing a
+paper end-to-end, spot-checking a bibliography for fabricated references, or
+benchmarking how often LLMs hallucinate citations at scale.
+
+Each skill is a self-contained workflow in `<NN-name>/SKILL.md`. Start at
+`00-academic-router` if you're not sure which one you need.
+
+## The workflows
+
+| # | Skill | Use it when |
+|---|---|---|
+| 00 | `00-academic-router` | You're not sure whether you want a paper review, a BibTeX check, or an arena-style benchmark. |
+| 01 | `01-paper-review` | You have a paper (PDF or LaTeX source) and want a multi-stage review — safety, correctness, quality/novelty, criticality, optionally + BibTeX. |
+| 02 | `02-bib-verify` | You have a standalone `.bib` file (no paper) and want to check it for fabricated/mismatched references. |
+| 03 | `03-ref-hallucination-arena` | You want to benchmark/compare multiple LLMs on how often they invent fake citations, across many queries. |
+
+## Relationship between the workflows
+
+`01-paper-review` and `02-bib-verify` both run on top of the same
+`cookbooks.paper_review` pipeline — `01` is the full document review (with an
+optional `--bib` flag to also verify references), `02` is the BibTeX-only mode
+for when there's no paper to review, just a bibliography to sanity-check.
+
+`03-ref-hallucination-arena` is a different axis entirely: instead of
+evaluating one document's existing references, it evaluates *model behavior* —
+how often a model fabricates references when asked to recommend citations,
+scored across a benchmark of queries and ranked across models. It shares
+`ref-hallucination-arena`'s content with the `arena-eval` suite's
+`02-ref-hallucination-arena` — the two copies are independently maintained
+(see the root [`skills/README.md`](../README.md) for why).
+
+## Dependencies
+
+| Skill | Cookbook |
+|---|---|
+| `01-paper-review` | `cookbooks/paper_review/` |
+| `02-bib-verify` | `cookbooks/paper_review/` (BibTeX-only mode) |
+| `03-ref-hallucination-arena` | `cookbooks/ref_hallucination_arena/` |
+
+```bash
+pip install py-openjudge litellm
+pip install matplotlib   # only needed by 03-ref-hallucination-arena (charts)
+```
+
+## Self-contained skills
+
+Each `<NN-name>/SKILL.md` is self-contained per the Anthropic Agent Skill
+protocol — it can be installed and used on its own without this README or the
+rest of the suite. Cross-references between skills in this suite use relative
+links scoped to this directory (e.g. `02-bib-verify` links to
+`../01-paper-review/`); there are no links out to other suites.
+
+## Validating the skills themselves
+
+This suite has an actor+judge functional test harness, same pattern as
+`eval_pipeline`: an *actor* model follows a skill to answer a realistic user
+request, and a separate *judge* model grades the answer against the case's
+acceptance criteria. See [`tests/academic_eval_testing_guide.md`](tests/academic_eval_testing_guide.md)
+for how to run it — it reuses `eval_pipeline`'s runner (parametrized, not
+duplicated) against this suite's own 12 test cases in
+[`tests/academic_eval_test_cases.jsonl`](tests/academic_eval_test_cases.jsonl).

skills/academic-eval/tests/academic_eval_skill_audit.md

@@ -0,0 +1,73 @@
+# Academic Eval Skill Audit
+
+Initial functional-test audit of the `academic-eval` suite, run right after the
+suite was created (folder-migrated from `paper-review`/`bib-verify`/
+`ref-hallucination-arena` + new `00-academic-router`).
+
+## Run 1 — full set, `--repeat 1`
+
+```bash
+python skills/eval_pipeline/tests/run_eval_pipeline_skill_tests.py \
+  --skill-root skills/academic-eval \
+  --cases skills/academic-eval/tests/academic_eval_test_cases.jsonl \
+  --out-dir skills/academic-eval/tests/results \
+  --report-prefix academic_eval
+```
+
+Actor: `qwen3.6-plus` · Judge: `qwen3-max`
+
+**Result: 12/12 pass, 0 partial, 0 fail.**
+
+| Case | Skill | Verdict | Score |
+|---|---|---:|---:|
+| `academic_router_001_route_paper_review` | `00-academic-router` | pass | 1.00 |
+| `academic_router_002_bib_only_no_paper` | `00-academic-router` | pass | 1.00 |
+| `academic_router_003_paper_plus_bib` | `00-academic-router` | pass | 1.00 |
+| `academic_router_004_arena_benchmark_request` | `00-academic-router` | pass | 1.00 |
+| `academic_router_005_general_quality_not_citations` | `00-academic-router` | pass | 1.00 |
+| `paper_review_001_model_fallback_no_explicit_model` | `01-paper-review` | pass | 1.00 |
+| `paper_review_002_bad_request_base_url_diagnosis` | `01-paper-review` | pass | 1.00 |
+| `bib_verify_001_standalone_bib_zh_report` | `02-bib-verify` | pass | 1.00 |
+| `bib_verify_002_interpret_suspect_entry` | `02-bib-verify` | pass | 1.00 |
+| `ref_arena_001_dataset_format_check` | `03-ref-hallucination-arena` | pass | 1.00 |
+| `ref_arena_002_interpret_low_accuracy` | `03-ref-hallucination-arena` | pass | 1.00 |
+| `academic_eval_001_end_to_end_route_and_execute` | `academic_eval_collection` | pass | 1.00 |
+
+## Run 2 — variance check, `--repeat 3` on the two most ambiguity-prone router cases
+
+```bash
+python skills/eval_pipeline/tests/run_eval_pipeline_skill_tests.py \
+  --skill-root skills/academic-eval \
+  --cases skills/academic-eval/tests/academic_eval_test_cases.jsonl \
+  --out-dir /tmp/academic_eval_repeat_check --report-prefix academic_eval \
+  --case-id academic_router_003_paper_plus_bib \
+  --case-id academic_router_005_general_quality_not_citations \
+  --repeat 3
+```
+
+- `academic_router_003_paper_plus_bib`: pass / pass / pass — score 1.00 (n=3)
+- `academic_router_005_general_quality_not_citations`: pass / pass / pass — score 1.00 (n=3)
+
+No variance observed across 3 runs on either boundary case.
+
+## Reading
+
+The suite's 4 skills (router + 3 migrated originals) pass their functional
+tests immediately after the folder migration and router rewrite — no
+follow-up SKILL.md patch was needed. The router's triage-table wording for
+the two trickiest rows (combined paper+bib request; out-of-scope redirect to
+`arena-eval`) held up under repeat sampling.
+
+**Caveat**: 21/21 clean passes across two suites on a first run is a stronger
+result than typical for a brand-new skill collection (`eval_pipeline`'s own
+first audit — see `../../eval_pipeline/tests/eval_pipeline_skill_audit.md` —
+needed several fix rounds). Two contributing factors, not just "this suite
+is more polished": (1) most of the SKILL.md content is unchanged domain
+documentation carried over from the pre-migration skills, which had already
+been used in practice; (2) all 12 test cases here were authored by the same
+person who wrote the router content in the same sitting, so criteria and
+skill text may share blind spots that an independently-written test set or a
+different actor/judge model pairing would catch. Re-running with `--repeat
+5`, a different judge model, or independently authored adversarial cases is
+the natural next step before treating this as a strong reliability claim
+rather than an initial smoke signal.