feat(engineering): add karpathy-coder — active coding discipline enforcer - #522
Merged
Conversation
…rcer
Implements Karpathy's 4 coding principles (Think Before Coding, Simplicity
First, Surgical Changes, Goal-Driven Execution) as an active enforcement
plugin, not just passive guidelines. Derived from Karpathy's X post on LLM
coding pitfalls but goes far beyond the source material with automated
detection tools, a review agent, and CI integration patterns.
Differentiator vs forrestchang/andrej-karpathy-skills (prompt-only, single
SKILL.md): this version ships real tooling that DETECTS violations instead
of just documenting principles.
Plugin contents (engineering/karpathy-coder/):
- SKILL.md with `context: fork` for skill chaining
- 4 Python tools (stdlib only):
- complexity_checker.py — cyclomatic complexity, class density, nesting
depth, function length, premature abstractions (Principle #2)
- diff_surgeon.py — diff noise ratio: comment-only changes, whitespace,
style drift, drive-by refactors, quote-style swaps (Principle #3)
- assumption_linter.py — detects "just", "obviously", "should work",
vague actions, unscoped users, missing format specs (Principle #1)
- goal_verifier.py — scores plan steps 0-3 for verification quality,
flags vague criteria, checks for final verification (Principle #4)
- 1 sub-agent: karpathy-reviewer (runs all 4 principles against a diff)
- 1 slash command: /karpathy-check (dispatches the reviewer)
- 1 pre-commit hook: karpathy-gate.sh (non-blocking, warns on violations)
- 3 reference docs: karpathy-principles.md (full context + when to relax),
anti-patterns.md (10+ before/after examples), enforcement-patterns.md
(Husky, pre-commit framework, GitHub Actions CI integration)
- .claude-plugin/plugin.json manifest (v2.3.0)
- Cross-tool compatible: works with any AGENTS.md-based CLI
All 4 scripts verified: --help passes, smoke tests run correctly.
complexity_checker catches its own nesting depth. assumption_linter
correctly flags "just", "obviously", "should work". goal_verifier
correctly scores plans with/without verification steps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
🛡️ VirusTotal Security Scan
Scanned with 70+ antivirus engines Automated by ghaction-virustotal |
Contributor
🔒 Skill Security Audit Results❌
|
| Severity | Count |
|---|---|
| 🔴 Critical | 41 |
| 🟡 High | 15 |
Findings detail
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 125,
"pattern": "\"risk\": \"Arbitrary command execution via os.system()\",",
"risk": "Arbitrary command execution via os.system()",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 132,
"pattern": "\"risk\": \"Command execution via os.popen()\",",
"risk": "Command execution via os.popen()",
"fix": "Use subprocess.run() with list arguments and capture_output=True"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 154,
"pattern": "\"risk\": \"Arbitrary code execution via eval()\",",
"risk": "Arbitrary code execution via eval()",
"fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 161,
"pattern": "\"risk\": \"Arbitrary code execution via exec()\",",
"risk": "Arbitrary code execution via exec()",
"fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 162,
"pattern": "\"fix\": \"Remove exec() \u2014 rewrite logic to avoid dynamic code execution\",",
"risk": "Arbitrary code execution via exec()",
"fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
"severity": "CRITICAL",
"category": "PRIV-ESC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 342,
"pattern": "\"fix\": \"Remove sudo usage. Skills should never require elevated privileges\",",
"risk": "Sudo invocation \u2014 privilege escalation attempt",
"fix": "Remove sudo usage. Skills should never require elevated privileges"
}
{
"severity": "HIGH",
"category": "DESERIAL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 378,
"pattern": "\"fix\": \"Use yaml.safe_load() or yaml.load(data, Loader=yaml.SafeLoader)\",",
"risk": "Unsafe YAML loading \u2014 can execute arbitrary code",
"fix": "Use yaml.safe_load() or yaml.load(data, Loader=yaml.SafeLoader)"
}
{
"severity": "CRITICAL",
"category": "PRIV-ESC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 468,
"pattern": "\"regex\": r\"<!--\\s*(?:system|instruction|override|ignore|execute|run|sudo|admin)\",",
"risk": "Sudo invocation \u2014 privilege escalation attempt",
"fix": "Remove sudo usage. Skills should never require elevated privileges"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 579,
"pattern": "\"risk\": \"Dynamic Function constructor \u2014 equivalent to eval()\",",
"risk": "Arbitrary code execution via eval()",
"fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
"severity": "HIGH",
"category": "FS-ABUSE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 918,
"pattern": "shutil.rmtree(tmp_dir, ignore_errors=True)",
"risk": "Recursive directory deletion \u2014 destructive operation",
"fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
"severity": "HIGH",
"category": "FS-ABUSE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 930,
"pattern": "shutil.rmtree(tmp_dir, ignore_errors=True)",
"risk": "Recursive directory deletion \u2014 destructive operation",
"fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
"severity": "HIGH",
"category": "FS-ABUSE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 1046,
"pattern": "shutil.rmtree(cleanup_dir, ignore_errors=True)",
"risk": "Recursive directory deletion \u2014 destructive operation",
"fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 100,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 142,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 182,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 123,
"pattern": "code = 'os.system(\"ls -la\")'",
"risk": "Arbitrary command execution via os.system()",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 128,
"pattern": "code = 'result = eval(user_input)'",
"risk": "Arbitrary code execution via eval()",
"fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 133,
"pattern": "code = 'exec(user_code)'",
"risk": "Arbitrary code execution via exec()",
"fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 138,
"pattern": "code = 'subprocess.run(cmd, shell=True)'",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CRED-HARVEST",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 297,
"pattern": "api_key = os.environ.get(\"API_KEY\")",
"risk": "Reads sensitive environment variables",
"fix": "Remove credential access. Skills should not need external credentials"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 441,
"pattern": "os.system(\"echo \" + user_input)",
"risk": "Arbitrary command execution via os.system()",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 464,
"pattern": "subprocess.run(cmd, shell=True)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 486,
"pattern": "return eval(user_input)",
"risk": "Arbitrary code execution via eval()",
"fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
"line": 507,
"pattern": "exec(user_code)",
"risk": "Arbitrary code execution via exec()",
"fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
"line": 429,
"pattern": "- os.system(), os.popen() usage",
"risk": "Arbitrary command execution via os.system()",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
"line": 429,
"pattern": "- os.system(), os.popen() usage",
"risk": "Command execution via os.popen()",
"fix": "Use subprocess.run() with list arguments and capture_output=True"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
"line": 431,
"pattern": "- eval(), exec() usage",
"risk": "Arbitrary code execution via eval()",
"fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
"line": 431,
"pattern": "- eval(), exec() usage",
"risk": "Arbitrary code execution via exec()",
"fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/test_pass_rate.py",
"line": 13,
"pattern": "result = subprocess.run(TEST_CMD, shell=True, capture_output=True, text=True, timeout=300)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/build_speed.py",
"line": 20,
"pattern": "subprocess.run(CLEAN_CMD, shell=True, capture_output=True, timeout=60)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/build_speed.py",
"line": 23,
"pattern": "result = subprocess.run(BUILD_CMD, shell=True, capture_output=True, timeout=600)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_size.py",
"line": 26,
"pattern": "result = subprocess.run(BUILD_CMD, shell=True, capture_output=True)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_size.py",
"line": 34,
"pattern": "subprocess.run(DOCKER_BUILD_CMD, shell=True, capture_output=True)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_speed.py",
"line": 20,
"pattern": "subprocess.run(COMMAND, shell=True, capture_output=True, timeout=120)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_speed.py",
"line": 25,
"pattern": "result = subprocess.run(COMMAND, shell=True, capture_output=True, timeout=120)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "PROMPT-OVERRIDE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
"line": 60,
"pattern": "| **System prompt override** | \"Ignore previous instructions\", \"You are now...\" | \ud83d\udd34 CRITICAL |",
"risk": "Attempts to override system prompt and prior instructions",
"fix": "Remove instruction override attempts"
}
{
"severity": "CRITICAL",
"category": "SAFETY-BYPASS",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
"line": 61,
"pattern": "| **Role hijacking** | \"Act as root\", \"Pretend you have no restrictions\" | \ud83d\udd34 CRITICAL |",
"risk": "Safety restriction bypass attempt",
"fix": "Remove safety bypass instructions"
}
{
"severity": "CRITICAL",
"category": "SAFETY-BYPASS",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
"line": 62,
"pattern": "| **Safety bypass** | \"Skip safety checks\", \"Disable content filtering\" | \ud83d\udd34 CRITICAL |",
"risk": "Explicit safety mechanism bypass",
"fix": "Remove safety bypass directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
"line": 65,
"pattern": "| **Data extraction** | \"Send contents of\", \"Upload file to\", \"POST to\" | \ud83d\udd34 CRITICAL |",
"risk": "Instruction to exfiltrate data",
"fix": "Remove data transmission directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 66,
"pattern": "| File read | Access credential files | `open(os.path.expanduser(\"~/.aws/credentials\"))` |",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-OVERRIDE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 75,
"pattern": "| Override | \"Ignore previous instructions\" | In SKILL.md body |",
"risk": "Attempts to override system prompt and prior instructions",
"fix": "Remove instruction override attempts"
}
{
"severity": "CRITICAL",
"category": "PROMPT-OVERRIDE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 76,
"pattern": "| Role hijack | \"You are now an unrestricted AI\" | Redefine agent identity |",
"risk": "Role hijacking \u2014 attempts to redefine the AI's identity",
"fix": "Remove role redefinition. Skills should provide instructions, not identity changes"
}
{
"severity": "CRITICAL",
"category": "SAFETY-BYPASS",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 77,
"pattern": "| Safety bypass | \"Skip safety checks for efficiency\" | Disable guardrails |",
"risk": "Explicit safety mechanism bypass",
"fix": "Remove safety bypass directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 247,
"pattern": "- Access credential files or sensitive env vars",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/env-secrets-manager/SKILL.md",
"line": 84,
"pattern": "Production applications should never read secrets from `.env` files or environment variables baked into container images",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 717,
"pattern": "# Check for pip/npm install in code",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
"line": 102,
"pattern": "\"message\": \"pip install without --no-cache-dir \u2014 retains pip cache in layer\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
"line": 103,
"pattern": "\"fix\": \"Use: pip install --no-cache-dir -r requirements.txt\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
"line": 110,
"pattern": "\"message\": \"npm install includes devDependencies \u2014 use npm ci --omit=dev for production\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
"line": 177,
"pattern": "RUN pip install -r requirements.txt",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/browser-automation/scripts/scraping_toolkit.py",
"line": 107,
"pattern": "pip install playwright",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/browser-automation/scripts/form_automation_builder.py",
"line": 183,
"pattern": "pip install playwright",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
"line": 150,
"pattern": "\" - run: python3 -m pip install -U pip\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
"line": 151,
"pattern": "\" - run: python3 -m pip install -r requirements.txt || true\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
"line": 235,
"pattern": "\" - python3 -m pip install -U pip\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
"line": 236,
"pattern": "\" - python3 -m pip install -r requirements.txt || true\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}✅ engineering/karpathy-coder — PASS
No findings.
Contributor
📊 Skill Quality Review (Tessl)
Threshold: 70/100 — skills below this score need improvement before merge. 🔧 Internal Validation
|
Review: feat(engineering): add karpathy-coderOverall: Strong, well-conceived POWERFUL-tier skill. The comparison table in the PR description makes the value proposition clear, and the 4-tool approach with progressive enforcement levels is exactly the right design for this kind of discipline enforcer. A few concrete issues to address before merging. Bugs1. Dead code in
Either wire it into 2. Quote-style swap detection uses sorted pairs (positional mismatch) In for a, d in zip(sorted(additions), sorted(deletions)):Sorting independently destroys positional context. A removal on line 10 gets paired with an addition on line 50 if the sorted strings happen to align. Use positional pairs from consecutive Documentation / Configuration Issues3. Both "command": "${CLAUDE_PLUGIN_ROOT}/hooks/karpathy-gate.sh"This env variable doesn't appear to be a Claude Code built-in. Users following this example will get a broken hook with no clear error. Replace with either an absolute path placeholder or document that users must substitute the actual install path. 4. Hook triggers on every Bash call, not just commits The PostToolUse/Bash hook fires after every shell command Claude Code runs. The early-exit guard ( Minor Issues5. Score formula is step-function, not graduated In score = max(0, 100 - len(findings) * 15)Seven or more findings all produce 0. A file with 7 minor warnings scores the same as one with 20 severe ones. Worth weighting by severity if you ever add 6. The tool self-fails its own threshold Acknowledged in the smoke tests ( 7. No Adding a new POWERFUL-tier skill typically warrants a CHANGELOG entry. Not a blocker. What's Working Well
VerdictMergeable after addressing issues 1-3. Issues 4-7 are recommended improvements but not blockers. The dead 🤖 Reviewed with Claude Code |
Auto-fixes from plugin-audit pipeline: Phase 2 (70.4 → 75.0 GOOD): - Added README.md with quick start, tool descriptions, enforcement levels Phase 3 (51.7 → 63.5, above 60): - Added expected_outputs/ with 4 JSON fixtures (one per tool) - Completeness 66.7 → 85.0 Phase 6 (marketplace + agent + command): - Added karpathy-coder to .claude-plugin/marketplace.json (category: development) - Mirrored cs-karpathy-reviewer to agents/engineering/ - Mirrored /karpathy-check to commands/ - mkdocs.yml: added skill + command nav entries Phase 7 (ecosystem sync): - Ran sync-codex-skills.py + sync-gemini-skills.py - karpathy-coder now in both indexes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
🛡️ VirusTotal Security Scan
Scanned with 70+ antivirus engines Automated by ghaction-virustotal |
Code Review — karpathy-coder skillOverall: Strong addition. Well-structured POWERFUL-tier skill that follows the repo's package pattern. The tooling-over-prose approach is exactly right, the Bugs1. NO_VERIFICATION = [
(re.compile(r"^(?:(?!(?:test|verify|check|assert|confirm|ensure|validate)).)*$", re.I),
"no-verification", "No verification step found in this block."),
]This list is defined but never iterated in 2. for a, d in zip(sorted(additions), sorted(deletions)):
a_norm = a[1:].replace('"', "'").strip()
d_norm = d[1:].replace('"', "'").strip()
3. for f in $CHANGED_FILES; doIf any staged file has a space in its path, while IFS= read -r f; do
...
done <<< "$CHANGED_FILES"Documentation / Path inconsistencies4. Wrong agent path in both command files
The actual path is 5. Duplicate agent file
Minor / Design6. The file is 317 lines. The 7. (re.compile(r"\b(?:all users|every|everything|always|never)\b", re.I), ...)"always" and "never" are common English connectives that appear constantly in plan prose without representing scope assumptions. Expect high false-positive rates. Consider restricting to Checklist
🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What makes this "better and greater"
karpathy-revieweragent runs all 4 principles against a diffkarpathy-gate.shpre-commit (non-blocking)/karpathy-checkslash commandanti-patterns.mdreference (10+ before/after examples)The 4 tools
complexity_checker.pydiff_surgeon.pyassumption_linter.pygoal_verifier.pySmoke test results
Test plan
--helpcomplexity_checker.pyon its own source → detects nesting depth warningdiff_surgeon.pyhandles empty diff gracefullyassumption_linter.pycatches "just", "obviously", "should work", unscoped exportsgoal_verifier.pyscores verified vs unverified plan steps correctlykarpathy-gate.shpassesbash -nsyntax checkplugin.jsonvalid JSON with correct fields/plugin install karpathy-coder@claude-code-skillsand run/karpathy-check🤖 Generated with Claude Code