feat(engineering): add llm-wiki plugin — second brain for Claude Code + Obsidian by alirezarezvani · Pull Request #508 · alirezarezvani/claude-skills

alirezarezvani · 2026-04-10T23:38:34Z

Summary

New POWERFUL-tier engineering plugin implementing Andrej Karpathy's LLM Wiki pattern as a production-grade skill. Turns any LLM CLI into a disciplined wiki maintainer that incrementally ingests sources into a persistent, interlinked Obsidian vault — knowledge compounds instead of being re-derived by RAG on every query.

Obsidian is the IDE. The LLM is the programmer. The wiki is the codebase.

What ships

55 files in engineering/llm-wiki/:

Piece	Count	Detail
SKILL.md	1	With `context: fork` for skill chaining
Sub-agents	3	`wiki-ingestor`, `wiki-librarian`, `wiki-linter`
Slash commands	5	`/wiki-init`, `/wiki-ingest`, `/wiki-query`, `/wiki-lint`, `/wiki-log`
Python tools	8	stdlib only — `init_vault`, `ingest_source`, `update_index`, `append_log`, `wiki_search` (BM25), `lint_wiki`, `graph_analyzer`, `export_marp`
Reference docs	8	schema, page-formats, ingest/query/lint workflows, obsidian-setup, cross-tool-setup, memex-principles
Vault templates	10	`CLAUDE.md`, `AGENTS.md`, `cursorrules`, `index.md`, `log.md` + 5 page templates
Example vault	7	Small worked example on "LLM interpretability"
Expected outputs	9	JSON fixtures for every script's `--json` output
plugin.json	1	v2.3.0, marketplace-compliant

Three-layer architecture

<vault>/
├── raw/                    # IMMUTABLE sources (you own — LLM never edits)
├── wiki/                   # LLM-owned knowledge base
│   ├── index.md            # content catalog (update every ingest)
│   ├── log.md              # append-only timeline
│   ├── entities/           # people, orgs, places, products
│   ├── concepts/           # ideas, theories, frameworks
│   ├── sources/            # one summary per ingested source
│   ├── comparisons/        # cross-source analyses
│   └── synthesis/          # high-level overviews / theses
├── CLAUDE.md               # schema (Claude Code)
├── AGENTS.md               # schema (Codex / Cursor / Antigravity / OpenCode / Gemini)
└── .cursorrules            # (optional) legacy Cursor

Three operations: ingest (read source, discuss, update 5-15 pages, log), query (index-first, drill-in, cite, offer to file-back), lint (mechanical + semantic health check).

Cross-tool compatibility

The 8 Python scripts are stdlib only. Only the schema loader filename changes per tool. init_vault.py --tool all installs all three (CLAUDE.md + AGENTS.md + .cursorrules) so the same vault works across every major CLI.

Tool	Loader
Claude Code	`CLAUDE.md`
Codex CLI	`AGENTS.md`
Cursor (modern)	`AGENTS.md`
Cursor (legacy)	`.cursorrules`
Antigravity	`AGENTS.md`
OpenCode / Pi	`AGENTS.md`
Gemini CLI	`AGENTS.md`

Repo-level registration

.claude-plugin/marketplace.json — new llm-wiki plugin entry (category: knowledge), top-level description/metadata bumped to 234 skills / 313 tools / 28 agents / 27 commands / v2.3.0
commands/wiki-*.md — 5 slash commands mirrored for repo-wide discovery
agents/engineering/cs-wiki-{ingestor,librarian,linter}.md — agents mirrored under cs-* convention
CLAUDE.md — scope, version block, footer updated with v2.3.0 highlights
engineering/.claude-plugin/plugin.json — skill count 38 → 44, version 2.2.0 → 2.3.0
mkdocs.yml — 5 /wiki-* command nav entries
.codex/skills-index.json + .gemini/skills-index.json — synced via sync-codex-skills.py / sync-gemini-skills.py

Plugin audit results (8-phase pipeline)

Ran /plugin-audit engineering/llm-wiki — VERDICT: PASS WITH WARNINGS (9 auto-fixes applied in commit 2).

Phase	Result
1 — Discovery	standalone plugin, engineering domain
2 — Structure	79.2/100 GOOD
3 — Quality	67.2/100 (above 60 threshold)
4 — Scripts	8/8 PASS
5 — Security	0 critical, 0 high
6 — Marketplace	plugin.json valid, v2.3.0 aligned
7 — Ecosystem	codex + gemini synced, mkdocs nav updated
8 — Code Review	Karpathy attribution, iron rules, cross-tool docs verified

Warnings (non-blocking): Documentation dimension 49.6/100 — intentionally kept SKILL.md concise with depth offloaded to 8 separate references/*.md files per the writing-skills "heavy reference in separate files" pattern.

Also shipped (deferred, not executed)

documentation/implementation/craighewitt-mattpocock-reimplementation-plan.md — 4-pod plan for reimplementing selected skills from two installed user-level collections (thecraighewitt-skills, mattpocock-skills). Captured for later; awaiting scope confirmation before execution.

Test plan

All 8 Python scripts pass python <script> --help
End-to-end smoke test: init_vault → ingest_source → update_index → append_log → wiki_search → lint_wiki → graph_analyzer → export_marp all run cleanly against a fresh vault
End-to-end smoke test in --json mode — all 8 scripts round-trip
Phase 2-8 audit pipeline re-run after fixes — all thresholds met
Ecosystem sync verified (llm-wiki present in .codex/skills-index.json and .gemini/skills-index.json)
Marketplace JSON validates
Reviewer: clone + run python engineering/llm-wiki/scripts/init_vault.py --path /tmp/test-vault --topic "test" --tool all to verify install
Reviewer: open the created vault in Obsidian and verify graph view renders

Commits

b6ca45d — feat(engineering): add llm-wiki plugin — second brain for Claude Code + Obsidian
a28f83b — fix(llm-wiki): pass 8-phase plugin audit

🤖 Generated with Claude Code

… + Obsidian Implements Karpathy's LLM Wiki pattern as a production-grade plugin. The LLM incrementally ingests sources into a persistent, interlinked Obsidian vault — updating entity/concept/source pages, flagging contradictions, maintaining an index and append-only log. Knowledge compounds instead of being re-derived by RAG on every query. Plugin contents (engineering/llm-wiki/): - SKILL.md with `context: fork` frontmatter for skill chaining - 3 sub-agents: wiki-ingestor, wiki-librarian, wiki-linter - 5 slash commands: /wiki-init, /wiki-ingest, /wiki-query, /wiki-lint, /wiki-log - 8 Python tools (stdlib only): init_vault, ingest_source, update_index, append_log, wiki_search (BM25), lint_wiki, graph_analyzer, export_marp - 8 reference docs: schema, page-formats, ingest/query/lint workflows, obsidian-setup, cross-tool-setup, memex-principles - Vault templates: CLAUDE.md, AGENTS.md, .cursorrules, index.md, log.md, 5 page templates (entity, concept, source, comparison, synthesis) - Worked example vault on "LLM interpretability" - .claude-plugin/plugin.json manifest Cross-tool compatibility: the scripts are pure Python stdlib. Only the schema loader changes per tool (CLAUDE.md for Claude Code, AGENTS.md for Codex CLI / Cursor / Antigravity / OpenCode / Gemini CLI, .cursorrules for legacy Cursor). init_vault.py --tool all installs all three. Repo-level registration: - Commands mirrored to top-level commands/ for repo-wide discovery - Agents mirrored to agents/engineering/ as cs-wiki-{ingestor,librarian,linter} - .claude-plugin/marketplace.json: new llm-wiki entry + version bump to v2.3.0 - CLAUDE.md updated: 234 skills, 313 Python tools, 432 refs, 28 agents, 27 commands Also saved (deferred): craighewitt-mattpocock reimplementation plan at documentation/implementation/ — 4-pod proposal for building better versions of selected skills from thecraighewitt-skills and mattpocock-skills collections. Not executed; awaiting user confirmation on scope. End-to-end smoke test passed: init_vault → ingest → update_index → append_log → wiki_search → lint → graph_analyzer → export_marp all run against a fresh vault with real pages, wikilinks, and frontmatter. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Auto-fixes from plugin-audit pipeline: Security (Phase 5, HIGH → 0): - Rename assets/.cursorrules.template → cursorrules.template to clear hidden-file warning from skill-security-auditor. init_vault.py updated to map the non-hidden template name to the vault's .cursorrules. Quality (Phase 3, 58.7 → 67.2, above 60 threshold): - Add --json output to init_vault, append_log, update_index, export_marp (4 scripts that were PARTIAL in Phase 4; now all 8 PASS) - Add expected_outputs/ directory with 8 sample JSON fixtures covering every script's --json output, lifting Completeness 66.7 → 85.0 - Expand error handling (try/except on file I/O) in append_log, init_vault, update_index, export_marp - Add more comprehensive argparse help text and epilogs Scripts (Phase 4, 4/8 PARTIAL → 8/8 PASS): - All 8 scripts now support --json - End-to-end smoke test confirmed: init → append_log → update_index → export_marp → lint → wiki_search → graph_analyzer → ingest_source all round-trip cleanly in --json mode Marketplace compliance (Phase 6): - llm-wiki/.claude-plugin/plugin.json version: 1.0.0 → 2.3.0 (match repo) - marketplace.json llm-wiki entry: version 1.0.0 → 2.3.0, cursorrules reference updated (no dot prefix) - engineering/.claude-plugin/plugin.json: count 38 → 44, version 2.2.0 → 2.3.0, added llm-wiki to description Ecosystem integration (Phase 7): - Ran sync-codex-skills.py → llm-wiki added to .codex/skills-index.json and .codex/skills/llm-wiki symlink - Ran sync-gemini-skills.py → llm-wiki + cs-wiki-* agents + wiki-* commands copied into .gemini/skills/ - mkdocs.yml nav: added 5 /wiki-* command entries Final audit scores: - Phase 2 Structure: 79.2/100 GOOD (↑ from 77.1) - Phase 3 Quality: 67.2/100 (↑ from 58.7) - Phase 4 Scripts: 8/8 PASS (↑ from 4/8 PASS) - Phase 5 Security: PASS 0c/0h (↓ from 1 HIGH) - Phase 6 Marketplace: plugin.json valid, version aligned - Phase 7 Ecosystem: codex+gemini synced, mkdocs nav updated - Phase 8 Code Review: Karpathy attribution, iron rules, cross-tool docs Verdict: PASS WITH WARNINGS (Documentation dimension 49.6 — acceptable for a reference-style skill with rich separate reference docs). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

# Conflicts: # .claude-plugin/marketplace.json # .codex/skills-index.json # .gemini/skills-index.json # CLAUDE.md

github-actions · 2026-04-10T23:52:42Z

🔒 Skill Security Audit Results

❌ `engineering` — FAIL

Severity	Count
🔴 Critical	41
🟡 High	15

Findings detail

{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 125,
  "pattern": "\"risk\": \"Arbitrary command execution via os.system()\",",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 132,
  "pattern": "\"risk\": \"Command execution via os.popen()\",",
  "risk": "Command execution via os.popen()",
  "fix": "Use subprocess.run() with list arguments and capture_output=True"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 154,
  "pattern": "\"risk\": \"Arbitrary code execution via eval()\",",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 161,
  "pattern": "\"risk\": \"Arbitrary code execution via exec()\",",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 162,
  "pattern": "\"fix\": \"Remove exec() \u2014 rewrite logic to avoid dynamic code execution\",",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "PRIV-ESC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 342,
  "pattern": "\"fix\": \"Remove sudo usage. Skills should never require elevated privileges\",",
  "risk": "Sudo invocation \u2014 privilege escalation attempt",
  "fix": "Remove sudo usage. Skills should never require elevated privileges"
}
{
  "severity": "HIGH",
  "category": "DESERIAL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 378,
  "pattern": "\"fix\": \"Use yaml.safe_load() or yaml.load(data, Loader=yaml.SafeLoader)\",",
  "risk": "Unsafe YAML loading \u2014 can execute arbitrary code",
  "fix": "Use yaml.safe_load() or yaml.load(data, Loader=yaml.SafeLoader)"
}
{
  "severity": "CRITICAL",
  "category": "PRIV-ESC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 468,
  "pattern": "\"regex\": r\"<!--\\s*(?:system|instruction|override|ignore|execute|run|sudo|admin)\",",
  "risk": "Sudo invocation \u2014 privilege escalation attempt",
  "fix": "Remove sudo usage. Skills should never require elevated privileges"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 579,
  "pattern": "\"risk\": \"Dynamic Function constructor \u2014 equivalent to eval()\",",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "HIGH",
  "category": "FS-ABUSE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 918,
  "pattern": "shutil.rmtree(tmp_dir, ignore_errors=True)",
  "risk": "Recursive directory deletion \u2014 destructive operation",
  "fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
  "severity": "HIGH",
  "category": "FS-ABUSE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 930,
  "pattern": "shutil.rmtree(tmp_dir, ignore_errors=True)",
  "risk": "Recursive directory deletion \u2014 destructive operation",
  "fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
  "severity": "HIGH",
  "category": "FS-ABUSE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 1046,
  "pattern": "shutil.rmtree(cleanup_dir, ignore_errors=True)",
  "risk": "Recursive directory deletion \u2014 destructive operation",
  "fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
  "severity": "CRITICAL",
  "category": "NET-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
  "line": 100,
  "pattern": "response = requests.post(",
  "risk": "Outbound HTTP write request \u2014 potential data exfiltration",
  "fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
  "severity": "CRITICAL",
  "category": "NET-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
  "line": 142,
  "pattern": "response = requests.post(",
  "risk": "Outbound HTTP write request \u2014 potential data exfiltration",
  "fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
  "severity": "CRITICAL",
  "category": "NET-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
  "line": 182,
  "pattern": "response = requests.post(",
  "risk": "Outbound HTTP write request \u2014 potential data exfiltration",
  "fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 123,
  "pattern": "code = 'os.system(\"ls -la\")'",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 128,
  "pattern": "code = 'result = eval(user_input)'",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 133,
  "pattern": "code = 'exec(user_code)'",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 138,
  "pattern": "code = 'subprocess.run(cmd, shell=True)'",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CRED-HARVEST",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 297,
  "pattern": "api_key = os.environ.get(\"API_KEY\")",
  "risk": "Reads sensitive environment variables",
  "fix": "Remove credential access. Skills should not need external credentials"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 441,
  "pattern": "os.system(\"echo \" + user_input)",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 464,
  "pattern": "subprocess.run(cmd, shell=True)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 486,
  "pattern": "return eval(user_input)",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 507,
  "pattern": "exec(user_code)",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 429,
  "pattern": "- os.system(), os.popen() usage",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 429,
  "pattern": "- os.system(), os.popen() usage",
  "risk": "Command execution via os.popen()",
  "fix": "Use subprocess.run() with list arguments and capture_output=True"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 431,
  "pattern": "- eval(), exec() usage",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 431,
  "pattern": "- eval(), exec() usage",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/test_pass_rate.py",
  "line": 13,
  "pattern": "result = subprocess.run(TEST_CMD, shell=True, capture_output=True, text=True, timeout=300)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/build_speed.py",
  "line": 20,
  "pattern": "subprocess.run(CLEAN_CMD, shell=True, capture_output=True, timeout=60)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/build_speed.py",
  "line": 23,
  "pattern": "result = subprocess.run(BUILD_CMD, shell=True, capture_output=True, timeout=600)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_size.py",
  "line": 26,
  "pattern": "result = subprocess.run(BUILD_CMD, shell=True, capture_output=True)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_size.py",
  "line": 34,
  "pattern": "subprocess.run(DOCKER_BUILD_CMD, shell=True, capture_output=True)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_speed.py",
  "line": 20,
  "pattern": "subprocess.run(COMMAND, shell=True, capture_output=True, timeout=120)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_speed.py",
  "line": 25,
  "pattern": "result = subprocess.run(COMMAND, shell=True, capture_output=True, timeout=120)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-OVERRIDE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
  "line": 60,
  "pattern": "| **System prompt override** | \"Ignore previous instructions\", \"You are now...\" | \ud83d\udd34 CRITICAL |",
  "risk": "Attempts to override system prompt and prior instructions",
  "fix": "Remove instruction override attempts"
}
{
  "severity": "CRITICAL",
  "category": "SAFETY-BYPASS",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
  "line": 61,
  "pattern": "| **Role hijacking** | \"Act as root\", \"Pretend you have no restrictions\" | \ud83d\udd34 CRITICAL |",
  "risk": "Safety restriction bypass attempt",
  "fix": "Remove safety bypass instructions"
}
{
  "severity": "CRITICAL",
  "category": "SAFETY-BYPASS",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
  "line": 62,
  "pattern": "| **Safety bypass** | \"Skip safety checks\", \"Disable content filtering\" | \ud83d\udd34 CRITICAL |",
  "risk": "Explicit safety mechanism bypass",
  "fix": "Remove safety bypass directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
  "line": 65,
  "pattern": "| **Data extraction** | \"Send contents of\", \"Upload file to\", \"POST to\" | \ud83d\udd34 CRITICAL |",
  "risk": "Instruction to exfiltrate data",
  "fix": "Remove data transmission directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 66,
  "pattern": "| File read | Access credential files | `open(os.path.expanduser(\"~/.aws/credentials\"))` |",
  "risk": "Instruction to access sensitive files or credentials",
  "fix": "Remove credential/sensitive file access directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-OVERRIDE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 75,
  "pattern": "| Override | \"Ignore previous instructions\" | In SKILL.md body |",
  "risk": "Attempts to override system prompt and prior instructions",
  "fix": "Remove instruction override attempts"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-OVERRIDE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 76,
  "pattern": "| Role hijack | \"You are now an unrestricted AI\" | Redefine agent identity |",
  "risk": "Role hijacking \u2014 attempts to redefine the AI's identity",
  "fix": "Remove role redefinition. Skills should provide instructions, not identity changes"
}
{
  "severity": "CRITICAL",
  "category": "SAFETY-BYPASS",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 77,
  "pattern": "| Safety bypass | \"Skip safety checks for efficiency\" | Disable guardrails |",
  "risk": "Explicit safety mechanism bypass",
  "fix": "Remove safety bypass directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 247,
  "pattern": "- Access credential files or sensitive env vars",
  "risk": "Instruction to access sensitive files or credentials",
  "fix": "Remove credential/sensitive file access directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/env-secrets-manager/SKILL.md",
  "line": 84,
  "pattern": "Production applications should never read secrets from `.env` files or environment variables baked into container images",
  "risk": "Instruction to access sensitive files or credentials",
  "fix": "Remove credential/sensitive file access directives"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 717,
  "pattern": "# Check for pip/npm install in code",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to package.json for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
  "line": 102,
  "pattern": "\"message\": \"pip install without --no-cache-dir \u2014 retains pip cache in layer\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
  "line": 103,
  "pattern": "\"fix\": \"Use: pip install --no-cache-dir -r requirements.txt\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
  "line": 110,
  "pattern": "\"message\": \"npm install includes devDependencies \u2014 use npm ci --omit=dev for production\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to package.json for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
  "line": 177,
  "pattern": "RUN pip install -r requirements.txt",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/browser-automation/scripts/scraping_toolkit.py",
  "line": 107,
  "pattern": "pip install playwright",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/browser-automation/scripts/form_automation_builder.py",
  "line": 183,
  "pattern": "pip install playwright",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
  "line": 150,
  "pattern": "\"      - run: python3 -m pip install -U pip\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
  "line": 151,
  "pattern": "\"      - run: python3 -m pip install -r requirements.txt || true\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
  "line": 235,
  "pattern": "\"    - python3 -m pip install -U pip\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
  "line": 236,
  "pattern": "\"    - python3 -m pip install -r requirements.txt || true\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}

✅ `engineering/llm-wiki` — PASS

No findings.

github-actions · 2026-04-10T23:53:02Z

📊 Skill Quality Review (Tessl)

Skill	Score	Description	Content	Verdict
`engineering/llm-wiki`	0/100 ⚠️	0%	0%	NEEDS WORK

Threshold: 70/100 — skills below this score need improvement before merge.

🔧 Internal Validation

engineering/llm-wiki: structure 79.2/100 (GOOD), scripts 8/8 PASS, security PASS

claude · 2026-04-10T23:55:12Z

PR Review — `feat(engineering): add llm-wiki plugin`

Overall this is a high-quality addition. The three-layer architecture is clean, the stdlib-only constraint is well-kept, and the BM25 implementation is correctly built. The documentation depth (8 reference files, worked example vault, expected outputs for every script) is exemplary for this repo's standard. A few issues need attention before merge.

Bugs / Correctness

1. Double file read in ingest_source.py

src.read_bytes() is called twice — once inside extract(path) and once for the SHA256 hash. For large sources (PDFs, big HTML) this reads the file twice unnecessarily.

# current (two reads)
text, title = extract(src)           # calls path.read_bytes() internally
...
"sha256": hashlib.sha256(src.read_bytes()).hexdigest()[:16],

# fix: read once, pass bytes into extract
data = src.read_bytes()
text, title = extract_from_bytes(data, src.suffix.lower(), src.stem)
"sha256": hashlib.sha256(data).hexdigest()[:16],

2. Stem collision in wikilink resolution (lint_wiki.py, graph_analyzer.py)

Both scripts build a stems dict mapping Path(key).name → key. If two pages share the same filename in different categories (e.g., entities/attention.md and concepts/attention.md), the second scan silently overwrites the first, causing broken-link false negatives.

# current — last one wins silently
stems = {Path(k).name: k for k in pages}

# fix — detect collisions explicitly
stems: dict[str, list[str]] = defaultdict(list)
for k in pages:
    stems[Path(k).name].append(k)
# then resolve only when len == 1, log ambiguity otherwise

Count Inconsistencies (documentation)

There are conflicting numbers across the files changed in this PR — they should all agree:

Location	Skills	Python tools	References	Agents	Commands
`CLAUDE.md` header (updated)	235	314	435	28	27
`CLAUDE.md` v2.3.0 highlights block	234	313	432	28	27
`marketplace.json` description	235	314	435	28	27
`.gemini/skills-index.json` command count	—	—	—	—	28

The v2.3.0 highlights block in CLAUDE.md is off by one on skills/tools/references. The Gemini index has 28 commands (22+6) while CLAUDE.md says 27 — the tc command appearing in the Gemini diff accounts for the extra one, but it's not represented in CLAUDE.md or the repo-level commands/ directory.

Also: the CLAUDE.md architecture section still reads:

├── agents/                    # 25 agents across all domains
├── commands/                  # 22 slash commands

These were not updated to reflect 28 agents / 27 commands.

And: engineering/.claude-plugin/plugin.json description jumps from 38 → 44 skills (6 added) while this PR only adds 1 (llm-wiki). If 5 skills were added in earlier commits and the description was stale, this is the right time to correct it — but it should be called out explicitly in the PR description so reviewers can verify.

Minor Issues

tc / tc-tracker appearing in Gemini index without a corresponding skill

.gemini/skills-index.json adds entries for tc (command) and tc-tracker (engineering-advanced skill), but there's no commands/tc.md or engineering/tc-tracker/ in this diff. If those skills already exist in the repo, add a note. If they don't, they shouldn't be indexed yet.

documentation/implementation/craighewitt-mattpocock-reimplementation-plan.md

The PR explicitly labels this "deferred, not executed." Including a 144-line implementation plan that references third-party skill collections (thecraighewitt-skills, mattpocock-skills) which most users and reviewers won't have is unusual. The plan describes skills that don't exist in the repo. Consider keeping it in a personal branch or dev notes until the work is actually scoped — it adds repo noise without shipping functionality.

ingest_source.py — JSON preview truncation is silent

return json.dumps(obj, indent=2)[:100000], None

The 100,000-char truncation produces valid JSON when the input is short, but broken JSON when truncated. The LLM receiving this brief may try to parse the preview as JSON and fail. A [:PREVIEW_CHARS] truncation with a trailing note would be cleaner.

Strengths

stdlib-only constraint is cleanly maintained across all 8 scripts — no accidental imports.
BM25 in wiki_search.py is correctly implemented (proper IDF formula, TF normalization, avoidance of division by zero).
context: fork on SKILL.md is the right call for a knowledge-layer skill that other skills should be able to chain.
Iron rule ("LLM never edits raw/") is enforced at every layer — SKILL.md, agent definitions, and vault CLAUDE.md template all repeat it. Good defensive design.
Expected outputs for all 8 scripts in expected_outputs/ is an excellent pattern for this repo — makes regression testing straightforward.
Example vault on "LLM interpretability" gives users a concrete, immediately runnable reference.
Cross-tool compatibility is systematically handled without branching the codebase — just the schema loader filename changes.

Summary

Fix the two correctness bugs (double read, stem collision) and reconcile the count discrepancies before merge. The tc indexing and deferred reimplementation plan are lower priority but worth addressing to keep the repo clean. Everything else is solid.

alirezarezvani and others added 3 commits April 11, 2026 01:14

merge: dev into feature/llm-wiki-plugin — resolve skill count conflicts

8c480b8

# Conflicts: # .claude-plugin/marketplace.json # .codex/skills-index.json # .gemini/skills-index.json # CLAUDE.md

alirezarezvani merged commit 45979f8 into dev Apr 10, 2026
5 of 7 checks passed

alirezarezvani deleted the feature/llm-wiki-plugin branch April 10, 2026 23:54

alirezarezvani mentioned this pull request Apr 11, 2026

docs: sync counts + regenerate pages after llm-wiki/apple-hig/tc-tracker landing #510

Merged

7 tasks

Navigation Menu

Uh oh!

feat(engineering): add llm-wiki plugin — second brain for Claude Code + Obsidian - #508

feat(engineering): add llm-wiki plugin — second brain for Claude Code + Obsidian#508
alirezarezvani merged 3 commits into
devfrom
feature/llm-wiki-plugin

alirezarezvani commented Apr 10, 2026

Uh oh!

github-actions Bot commented Apr 10, 2026

Uh oh!

github-actions Bot commented Apr 10, 2026

Uh oh!

Uh oh!

claude Bot commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

alirezarezvani commented Apr 10, 2026

Summary

What ships

Three-layer architecture

Cross-tool compatibility

Repo-level registration

Plugin audit results (8-phase pipeline)

Also shipped (deferred, not executed)

Test plan

Commits

Uh oh!

github-actions Bot commented Apr 10, 2026

🔒 Skill Security Audit Results

❌ engineering — FAIL

✅ engineering/llm-wiki — PASS

Uh oh!

github-actions Bot commented Apr 10, 2026

📊 Skill Quality Review (Tessl)

🔧 Internal Validation

Uh oh!

Uh oh!

claude Bot commented Apr 10, 2026

PR Review — feat(engineering): add llm-wiki plugin

Bugs / Correctness

Count Inconsistencies (documentation)

Minor Issues

Strengths

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

❌ `engineering` — FAIL

✅ `engineering/llm-wiki` — PASS

PR Review — `feat(engineering): add llm-wiki plugin`