fix(engineering): improve 7 low-scoring skills + 3 security fixes - #331
Conversation
- payment_processor.py: add disclaimer header + replace realistic-looking keys with EXAMPLE_NOT_REAL - project_scaffolder.py: add SCAFFOLDING PLACEHOLDER comments to generated secrets - pipeline_orchestrator.py: no change needed (compile() used for syntax validation only)
…xtract references
🔒 Skill Security Audit Results❌
|
| Severity | Count |
|---|---|
| 🔴 Critical | 15 |
| 🟡 High | 19 |
Findings detail
{
"severity": "HIGH",
"category": "FS-HIDDEN",
"file": ".claude-plugin",
"line": 0,
"pattern": ".claude-plugin",
"risk": "Hidden file '.claude-plugin' \u2014 may contain secrets or hidden config",
"fix": "Remove hidden files from skill distribution"
}
{
"severity": "HIGH",
"category": "FS-HIDDEN",
"file": ".codex",
"line": 0,
"pattern": ".codex",
"risk": "Hidden file '.codex' \u2014 may contain secrets or hidden config",
"fix": "Remove hidden files from skill distribution"
}
{
"severity": "HIGH",
"category": "FS-HIDDEN",
"file": "playwright-pro/.claude-plugin",
"line": 0,
"pattern": ".claude-plugin",
"risk": "Hidden file '.claude-plugin' \u2014 may contain secrets or hidden config",
"fix": "Remove hidden files from skill distribution"
}
{
"severity": "HIGH",
"category": "FS-HIDDEN",
"file": "playwright-pro/.mcp.json",
"line": 0,
"pattern": ".mcp.json",
"risk": "Hidden file '.mcp.json' \u2014 may contain secrets or hidden config",
"fix": "Remove hidden files from skill distribution"
}
{
"severity": "HIGH",
"category": "FS-HIDDEN",
"file": "google-workspace-cli/.claude-plugin",
"line": 0,
"pattern": ".claude-plugin",
"risk": "Hidden file '.claude-plugin' \u2014 may contain secrets or hidden config",
"fix": "Remove hidden files from skill distribution"
}
{
"severity": "HIGH",
"category": "FS-HIDDEN",
"file": "self-improving-agent/.claude-plugin",
"line": 0,
"pattern": ".claude-plugin",
"risk": "Hidden file '.claude-plugin' \u2014 may contain secrets or hidden config",
"fix": "Remove hidden files from skill distribution"
}
{
"severity": "HIGH",
"category": "FS-ABUSE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-computer-vision/scripts/dataset_pipeline_builder.py",
"line": 812,
"pattern": "shutil.rmtree(temp_coco, ignore_errors=True)",
"risk": "Recursive directory deletion \u2014 destructive operation",
"fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
"severity": "HIGH",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/playwright-pro/hooks/validate-test.sh",
"line": 10,
"pattern": "FILE_PATH=$(echo \"$INPUT\" | python3 -c \"",
"risk": "Inline code execution in shell script",
"fix": "Move code to a separate, inspectable script file"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/google-workspace-cli/scripts/gws_recipe_runner.py",
"line": 321,
"pattern": "result = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=30)",
"risk": "Shell injection via subprocess with shell=True",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CRED-HARVEST",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 524,
"pattern": "SECRET_KEY = os.environ.get(\"DJANGO_SECRET_KEY\", \"change-me\")",
"risk": "Reads sensitive environment variables",
"fix": "Remove credential access. Skills should not need external credentials"
}
{
"severity": "CRITICAL",
"category": "CRED-HARVEST",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 558,
"pattern": "\"PASSWORD\": os.environ.get(\"DB_PASSWORD\", \"password\"),",
"risk": "Reads sensitive environment variables",
"fix": "Remove credential access. Skills should not need external credentials"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-secops/scripts/security_scanner.py",
"line": 118,
"pattern": "'exec() with potential user input'),",
"risk": "Arbitrary code execution via exec()",
"fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-secops/scripts/security_scanner.py",
"line": 120,
"pattern": "'eval() with potential user input'),",
"risk": "Arbitrary code execution via eval()",
"fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-data-engineer/scripts/pipeline_orchestrator.py",
"line": 339,
"pattern": "compile(code, '<string>', 'exec')",
"risk": "Dynamic code compilation for execution",
"fix": "Remove compile() with exec mode \u2014 use explicit logic instead"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-data-engineer/scripts/pipeline_orchestrator.py",
"line": 452,
"pattern": "compile(code, '<string>', 'exec')",
"risk": "Dynamic code compilation for execution",
"fix": "Remove compile() with exec mode \u2014 use explicit logic instead"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-data-engineer/scripts/pipeline_orchestrator.py",
"line": 579,
"pattern": "compile(code, '<string>', 'exec')",
"risk": "Dynamic code compilation for execution",
"fix": "Remove compile() with exec mode \u2014 use explicit logic instead"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-security/scripts/secret_scanner.py",
"line": 384,
"pattern": "\"scan_date\": __import__('datetime').datetime.now().isoformat(),",
"risk": "Dynamic module import \u2014 can load arbitrary code",
"fix": "Use explicit import statements"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-security/scripts/threat_modeler.py",
"line": 468,
"pattern": "\"analysis_date\": __import__('datetime').datetime.now().isoformat(),",
"risk": "Dynamic module import \u2014 can load arbitrary code",
"fix": "Use explicit import statements"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-architect/references/tech_decision_guide.md",
"line": 240,
"pattern": "| Access token | 15-60 minutes |",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/playwright-pro/templates/api/auth-headers.md",
"line": 55,
"pattern": "// Happy path: refresh token obtains new access token",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/references/architecture_patterns.md",
"line": 454,
"pattern": "1. User logs in \u2192 Server returns access token (15min) + refresh token (7d)",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/references/architecture_patterns.md",
"line": 456,
"pattern": "3. Access token expires \u2192 Client uses refresh token to get new pair",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-security/references/security-architecture-patterns.md",
"line": 241,
"pattern": "\"\"\"Create short-lived access token.\"\"\"",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-computer-vision/scripts/inference_optimizer.py",
"line": 431,
"pattern": "'onnxruntime': f'pip install onnxruntime-gpu',",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-frontend/scripts/frontend_scaffolder.py",
"line": 890,
"pattern": "\"npm install\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 661,
"pattern": "RUN npm install",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 673,
"pattern": "npm install",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 773,
"pattern": "\"nextjs\": [f\"cd {name}\", \"npm install\", \"cp .env.example .env.local\", \"npm run dev\"],",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 777,
"pattern": "\"cd backend && pip install -r requirements.txt && uvicorn app.main:app --reload\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 778,
"pattern": "\"cd frontend && npm install && npm run dev\"",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 783,
"pattern": "\"cd server && npm install && npm run dev\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 784,
"pattern": "\"cd client && npm install && npm run dev\"",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 789,
"pattern": "\"cd backend && pip install -r requirements.txt && python manage.py migrate && python manage.py runserver\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering-team/senior-fullstack/scripts/project_scaffolder.py",
"line": 790,
"pattern": "\"cd frontend && npm install && npm run dev\"",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}❌ engineering — FAIL
| Severity | Count |
|---|---|
| 🔴 Critical | 22 |
| 🟡 High | 11 |
Findings detail
{
"severity": "HIGH",
"category": "FS-HIDDEN",
"file": ".claude-plugin",
"line": 0,
"pattern": ".claude-plugin",
"risk": "Hidden file '.claude-plugin' \u2014 may contain secrets or hidden config",
"fix": "Remove hidden files from skill distribution"
}
{
"severity": "HIGH",
"category": "FS-HIDDEN",
"file": ".codex",
"line": 0,
"pattern": ".codex",
"risk": "Hidden file '.codex' \u2014 may contain secrets or hidden config",
"fix": "Remove hidden files from skill distribution"
}
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 100,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 142,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 182,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 125,
"pattern": "\"risk\": \"Arbitrary command execution via os.system()\",",
"risk": "Arbitrary command execution via os.system()",
"fix": "Use subprocess.run() with list arguments and shell=False"
}
{
"severity": "CRITICAL",
"category": "CMD-INJECT",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 132,
"pattern": "\"risk\": \"Command execution via os.popen()\",",
"risk": "Command execution via os.popen()",
"fix": "Use subprocess.run() with list arguments and capture_output=True"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 154,
"pattern": "\"risk\": \"Arbitrary code execution via eval()\",",
"risk": "Arbitrary code execution via eval()",
"fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 161,
"pattern": "\"risk\": \"Arbitrary code execution via exec()\",",
"risk": "Arbitrary code execution via exec()",
"fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 162,
"pattern": "\"fix\": \"Remove exec() \u2014 rewrite logic to avoid dynamic code execution\",",
"risk": "Arbitrary code execution via exec()",
"fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
"severity": "CRITICAL",
"category": "PRIV-ESC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 342,
"pattern": "\"fix\": \"Remove sudo usage. Skills should never require elevated privileges\",",
"risk": "Sudo invocation \u2014 privilege escalation attempt",
"fix": "Remove sudo usage. Skills should never require elevated privileges"
}
{
"severity": "HIGH",
"category": "DESERIAL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 378,
"pattern": "\"fix\": \"Use yaml.safe_load() or yaml.load(data, Loader=yaml.SafeLoader)\",",
"risk": "Unsafe YAML loading \u2014 can execute arbitrary code",
"fix": "Use yaml.safe_load() or yaml.load(data, Loader=yaml.SafeLoader)"
}
{
"severity": "CRITICAL",
"category": "PRIV-ESC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 468,
"pattern": "\"regex\": r\"<!--\\s*(?:system|instruction|override|ignore|execute|run|sudo|admin)\",",
"risk": "Sudo invocation \u2014 privilege escalation attempt",
"fix": "Remove sudo usage. Skills should never require elevated privileges"
}
{
"severity": "CRITICAL",
"category": "CODE-EXEC",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 579,
"pattern": "\"risk\": \"Dynamic Function constructor \u2014 equivalent to eval()\",",
"risk": "Arbitrary code execution via eval()",
"fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
"severity": "HIGH",
"category": "FS-ABUSE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 917,
"pattern": "shutil.rmtree(tmp_dir, ignore_errors=True)",
"risk": "Recursive directory deletion \u2014 destructive operation",
"fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
"severity": "HIGH",
"category": "FS-ABUSE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 929,
"pattern": "shutil.rmtree(tmp_dir, ignore_errors=True)",
"risk": "Recursive directory deletion \u2014 destructive operation",
"fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
"severity": "HIGH",
"category": "FS-ABUSE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 1045,
"pattern": "shutil.rmtree(cleanup_dir, ignore_errors=True)",
"risk": "Recursive directory deletion \u2014 destructive operation",
"fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/env-secrets-manager/references/secret-patterns.md",
"line": 8,
"pattern": "- GitHub personal access tokens (`ghp_...`)",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/env-secrets-manager/references/validation-detection-rotation.md",
"line": 304,
"pattern": "- **GitHub PAT**: Settings \u2192 Developer Settings \u2192 Personal access tokens \u2192 Revoke \u2192 Create new",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-OVERRIDE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
"line": 60,
"pattern": "| **System prompt override** | \"Ignore previous instructions\", \"You are now...\" | \ud83d\udd34 CRITICAL |",
"risk": "Attempts to override system prompt and prior instructions",
"fix": "Remove instruction override attempts"
}
{
"severity": "CRITICAL",
"category": "SAFETY-BYPASS",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
"line": 61,
"pattern": "| **Role hijacking** | \"Act as root\", \"Pretend you have no restrictions\" | \ud83d\udd34 CRITICAL |",
"risk": "Safety restriction bypass attempt",
"fix": "Remove safety bypass instructions"
}
{
"severity": "CRITICAL",
"category": "SAFETY-BYPASS",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
"line": 62,
"pattern": "| **Safety bypass** | \"Skip safety checks\", \"Disable content filtering\" | \ud83d\udd34 CRITICAL |",
"risk": "Explicit safety mechanism bypass",
"fix": "Remove safety bypass directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
"line": 65,
"pattern": "| **Data extraction** | \"Send contents of\", \"Upload file to\", \"POST to\" | \ud83d\udd34 CRITICAL |",
"risk": "Instruction to exfiltrate data",
"fix": "Remove data transmission directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 66,
"pattern": "| File read | Access credential files | `open(os.path.expanduser(\"~/.aws/credentials\"))` |",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-OVERRIDE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 75,
"pattern": "| Override | \"Ignore previous instructions\" | In SKILL.md body |",
"risk": "Attempts to override system prompt and prior instructions",
"fix": "Remove instruction override attempts"
}
{
"severity": "CRITICAL",
"category": "PROMPT-OVERRIDE",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 76,
"pattern": "| Role hijack | \"You are now an unrestricted AI\" | Redefine agent identity |",
"risk": "Role hijacking \u2014 attempts to redefine the AI's identity",
"fix": "Remove role redefinition. Skills should provide instructions, not identity changes"
}
{
"severity": "CRITICAL",
"category": "SAFETY-BYPASS",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 77,
"pattern": "| Safety bypass | \"Skip safety checks for efficiency\" | Disable guardrails |",
"risk": "Explicit safety mechanism bypass",
"fix": "Remove safety bypass directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
"line": 247,
"pattern": "- Access credential files or sensitive env vars",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
"line": 150,
"pattern": "\" - run: python3 -m pip install -U pip\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
"line": 151,
"pattern": "\" - run: python3 -m pip install -r requirements.txt || true\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
"line": 235,
"pattern": "\" - python3 -m pip install -U pip\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
"line": 236,
"pattern": "\" - python3 -m pip install -r requirements.txt || true\",",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to requirements.txt for pre-install review"
}
{
"severity": "HIGH",
"category": "DEPS-RUNTIME",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
"line": 717,
"pattern": "# Check for pip/npm install in code",
"risk": "Runtime package installation \u2014 may install untrusted code",
"fix": "Move dependencies to package.json for pre-install review"
}✅ engineering/agent-workflow-designer — PASS
No findings.
✅ engineering/codebase-onboarding — PASS
No findings.
❌ engineering/env-secrets-manager — FAIL
| Severity | Count |
|---|---|
| 🔴 Critical | 2 |
Findings detail
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/env-secrets-manager/references/secret-patterns.md",
"line": 8,
"pattern": "- GitHub personal access tokens (`ghp_...`)",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}
{
"severity": "CRITICAL",
"category": "PROMPT-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/env-secrets-manager/references/validation-detection-rotation.md",
"line": 304,
"pattern": "- **GitHub PAT**: Settings \u2192 Developer Settings \u2192 Personal access tokens \u2192 Revoke \u2192 Create new",
"risk": "Instruction to access sensitive files or credentials",
"fix": "Remove credential/sensitive file access directives"
}✅ engineering/interview-system-designer — PASS
No findings.
✅ engineering/monorepo-navigator — PASS
No findings.
✅ engineering/performance-profiler — PASS
No findings.
✅ engineering/runbook-generator — PASS
No findings.
❌ engineering/tech-debt-tracker — FAIL
| Severity | Count |
|---|---|
| 🔴 Critical | 3 |
Findings detail
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 100,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 142,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
"severity": "CRITICAL",
"category": "NET-EXFIL",
"file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
"line": 182,
"pattern": "response = requests.post(",
"risk": "Outbound HTTP write request \u2014 potential data exfiltration",
"fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
|
test |
PR Review - Part 1: Overview and Security FixesOverall this is a well-structured batch improvement. The security fixes are correct and the new Python scripts are clean and stdlib-only. A few issues worth addressing before merge. payment_processor.py - Good call. Replacing sk_test_1234567890 with sk_test_EXAMPLE_NOT_REAL removes the realistic-looking test key pattern that secret scanners (including the new env_auditor.py) would flag. The disclaimer block is clear. project_scaffolder.py - Inline SCAFFOLDING PLACEHOLDER comments on both SECRET_KEY usages are helpful. The .env.example comment is inside a Python string (generated file content), so it will appear literally in generated files, which is intentional and correct. |
PR Review - Part 2: New Scriptsworkflow_scaffolder.py - Clean, minimal, correct. Pattern dispatch via dict is idiomatic. main() returning int and raise SystemExit(main()) follows best practices. codebase_analyzer.py - Solid. path.is_file() guard in iter_files is good. Minor: visible_files[:10] silently truncates files per directory, worth a comment or note in output. monorepo_analyzer.py - Good YAML parsing without a YAML dependency. The pnpm-workspace.yaml line parser handles comments and blank lines correctly. runbook_generator.py - Simple and correct. f-string template is appropriate for generating Markdown for human editing. interview_planner.py - Well-structured. Level-keyed BASE_ROUNDS dict is easy to extend. performance_profiler.py - Good heuristic. EXT_WEIGHTS for bundle weight estimation is a reasonable proxy where no build tooling is available. |
PR Review - Part 3: Issues to AddressIssue 1: env_auditor.py - High False Positive Rate on generic_secret_assignment The pattern for generic_secret_assignment will fire on many legitimate lines: test fixtures, config validation lines, inline comments, and even the scanner's own PATTERNS list would flag itself. Suggested fixes: bump the minimum value length from 8 to 16+ characters; require the value to be quoted (preceded by a quote character); skip comment lines (first non-whitespace char is the hash symbol). Issue 2: .env.example Files Are Scanned is_candidate() returns True for any file starting with .env, including .env.example. These files are designed to have placeholder key names with empty values, so scanning them produces noise. An exclusion for files ending in .example or .sample should be added. Issue 3: agent-workflow-designer - Python Examples Discarded Rather Than Relocated The SKILL.md went from 443 to 83 lines, meeting the 200-line target. However the original contained working Python implementations for all 5 patterns (sequential pipeline, parallel fan-out/fan-in, hierarchical delegation, error recovery, context budgeting), the AgentHandoff dataclass template, and a cost comparison table. The new references/workflow-patterns.md only has JSON skeleton configs, which are much less actionable than the removed Python code. A user who wants to implement a parallel fan-out pattern now gets a 6-line JSON blob instead of a working asyncio implementation. Suggestion: move the Python examples into references/implementation-examples.md rather than deleting them. The 200-line limit applies to SKILL.md, not to references files. Issue 4: env_auditor.py - Same-Line Multi-Pattern Duplicates A single line matching multiple patterns generates multiple findings for the same line, inflating finding counts. Worth noting in --help or deduplicating by (file, line) in the summary. |
PR Review - Part 4: Minor Notes and SummaryMinor Notes codebase_analyzer.py top_level_structure: dirnames cleared after max depth prevents recursion but also silences file listings at that depth. Not a bug, but a comment would help. performance_profiler.py: A comment explaining the .map weight of 2.0 (source maps add parse overhead) would help users understand the heuristic. Summary
The security fixes and new scripts are ready to merge. I would recommend addressing the env_auditor.py false-positive issues (1 and 2) before this lands, since a noisy scanner that fires on comments and .env.example files will lose user trust quickly. Reviewed by Claude Sonnet 4.6 |
- performance-profiler: add Quick Start section with script usage examples - interview-system-designer: fix references to match actual filenames
PR Review — fix(engineering): improve 7 low-scoring skills + 3 security fixesGood overall direction — leaner SKILL.md files, runnable scripts, and the security fixes are appropriate. Here are my findings: Security Fixes ✅payment_processor.py — Replacing project_scaffolder.py — Caveat: the inline comment added to -SECRET_KEY="your-secret-here"
+SECRET_KEY="your-secret-here" # ⚠️ SCAFFOLDING PLACEHOLDER — replace before deploymentMost dotenv parsers (Python The Script QualityThe new scripts are clean and follow the repo's conventions (stdlib-only, performance_profiler.py / codebase_analyzer.py / monorepo_analyzer.py — All use unbounded env_auditor.py — Reads all files in a directory looking for secret patterns. This is appropriate for its purpose, but the runbook_generator.py — Clean and self-contained. The f-string template at line 35 uses workflow_scaffolder.py — The generated output is abstract JSON config (not runnable code). This is a significant change from the old SKILL.md which had working Python implementations. The new approach is more pattern-agnostic, which is fine, but users who want runnable code now have nothing to start from. Consider adding a note in SKILL.md's Quick Start clarifying these are scaffolding configs, not executable code. Content Tradeoff: SKILL.md CondensationThe condensation approach is correct for the Tessl scoring criteria (under 200 lines, frontmatter preserved, scripts added). However, some skills lost significant immediately-usable content:
This is a deliberate tradeoff for scoring purposes, but it's worth flagging: if Tessl scores are the primary target, this works. If actual user utility is the metric, some of the removed content (especially the concrete bash commands and working Python snippets) was genuinely valuable. Minor Issues
Summary
The |
| ("critical", "aws_access_key_id", re.compile(r"\bAKIA[0-9A-Z]{16}\b")), | ||
| ("high", "slack_token", re.compile(r"\bxox[baprs]-[A-Za-z0-9-]{10,}\b")), | ||
| ("high", "private_key_block", re.compile(r"-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----")), | ||
| ("high", "generic_secret_assignment", re.compile(r"(?i)\b(secret|token|password|passwd|api[_-]?key)\b\s*[:=]\s*['\"]?[A-Za-z0-9_\-\/.+=]{8,}")), |
There was a problem hiding this comment.
🔴 Secret scanner regex \b word boundary prevents matching common variable names like SECRET_KEY, DB_PASSWORD, JWT_SECRET
The generic_secret_assignment regex pattern in env_auditor.py:47 uses \b (word boundary) around keywords like secret, token, password. Because _ is a word character in regex, \b does NOT fire between _ and an adjacent letter. This means compound variable names — SECRET_KEY, DB_PASSWORD, JWT_SECRET, AUTH_TOKEN, STRIPE_SECRET_KEY — will never match the pattern, even when they are assigned real credential values. These are the most common naming conventions for secrets in .env files, Python configs, and JavaScript/TypeScript configs.
The tool's own reference documentation at engineering/env-secrets-manager/references/secret-patterns.md:15 explicitly lists "Hardcoded assignments to secret, token, password, api_key" as High severity findings. Only the bare standalone words (e.g. password = "value") will trigger, missing the vast majority of real-world secret assignments. For a security auditing tool, false negatives are dangerous — users will trust a clean report when secrets are actually exposed.
| ("high", "generic_secret_assignment", re.compile(r"(?i)\b(secret|token|password|passwd|api[_-]?key)\b\s*[:=]\s*['\"]?[A-Za-z0-9_\-\/.+=]{8,}")), | |
| ("high", "generic_secret_assignment", re.compile(r"(?i)\b\w*(secret|token|password|passwd|api[_-]?key)\w*\b\s*[:=]\s*['\"\]?[A-Za-z0-9_\-\/.+=]{8,}")), |
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
What changed
7 low-scoring skills improved (all were <50% in Tessl audit)
performance_profiler.pycodebase_analyzer.pymonorepo_analyzer.pyworkflow_scaffolder.pyenv_auditor.pyrunbook_generator.pyinterview_planner.py3 security findings addressed
payment_processor.py: disclaimer + replaced realistic-looking keysproject_scaffolder.py: placeholder comments on generated secretspipeline_orchestrator.py: no change needed (syntax validation only)Verification
py_compile) and--helpworks