Skip to content

feat(skill-tester): add plugin.json, improve description, sync indexes - #459

Closed
alirezarezvani wants to merge 2 commits into
devfrom
fix/skill-tester-integration
Closed

feat(skill-tester): add plugin.json, improve description, sync indexes#459
alirezarezvani wants to merge 2 commits into
devfrom
fix/skill-tester-integration

Conversation

@alirezarezvani

@alirezarezvani alirezarezvani commented Mar 31, 2026

Copy link
Copy Markdown
Owner

Summary

Post-merge integration for PR #430 (security dimension by xingzihai):

  • plugin.json created for standalone installation of skill-tester
  • SKILL.md description improved from bare "Skill Tester" to detailed trigger-oriented description
  • Codex/Gemini indexes synced after the security_scorer.py addition
  • 4 scripts verified: skill_validator.py, quality_scorer.py, security_scorer.py, script_tester.py — all pass --help
  • --include-security flag confirmed working (opt-in 5×20% mode)

Test plan

  • All 4 scripts pass --help
  • plugin.json valid (author object, version 2.2.0)
  • --include-security flag present in quality_scorer.py
  • Codex/Gemini symlinks synced

🤖 Generated with Claude Code

@alirezarezvani @claude
feat(skill-tester): add plugin.json, improve description, sync indexes
7ddedf7 
- Created .claude-plugin/plugin.json for standalone installation
- Improved SKILL.md description from bare "Skill Tester" to detailed
  trigger-oriented description mentioning all 4 tools and --include-security
- Synced Codex/Gemini indexes after PR #430 merge (security_scorer.py)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Mar 31, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🔒 Skill Security Audit Results

❌ engineering — FAIL

Severity Count
🔴 Critical 41
🟡 High 15
Findings detail 
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_size.py",
  "line": 26,
  "pattern": "result = subprocess.run(BUILD_CMD, shell=True, capture_output=True)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_size.py",
  "line": 34,
  "pattern": "subprocess.run(DOCKER_BUILD_CMD, shell=True, capture_output=True)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/test_pass_rate.py",
  "line": 13,
  "pattern": "result = subprocess.run(TEST_CMD, shell=True, capture_output=True, text=True, timeout=300)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/build_speed.py",
  "line": 20,
  "pattern": "subprocess.run(CLEAN_CMD, shell=True, capture_output=True, timeout=60)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/build_speed.py",
  "line": 23,
  "pattern": "result = subprocess.run(BUILD_CMD, shell=True, capture_output=True, timeout=600)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_speed.py",
  "line": 20,
  "pattern": "subprocess.run(COMMAND, shell=True, capture_output=True, timeout=120)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/autoresearch-agent/evaluators/benchmark_speed.py",
  "line": 25,
  "pattern": "result = subprocess.run(COMMAND, shell=True, capture_output=True, timeout=120)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 125,
  "pattern": "\"risk\": \"Arbitrary command execution via os.system()\",",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 132,
  "pattern": "\"risk\": \"Command execution via os.popen()\",",
  "risk": "Command execution via os.popen()",
  "fix": "Use subprocess.run() with list arguments and capture_output=True"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 154,
  "pattern": "\"risk\": \"Arbitrary code execution via eval()\",",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 161,
  "pattern": "\"risk\": \"Arbitrary code execution via exec()\",",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 162,
  "pattern": "\"fix\": \"Remove exec() \u2014 rewrite logic to avoid dynamic code execution\",",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "PRIV-ESC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 342,
  "pattern": "\"fix\": \"Remove sudo usage. Skills should never require elevated privileges\",",
  "risk": "Sudo invocation \u2014 privilege escalation attempt",
  "fix": "Remove sudo usage. Skills should never require elevated privileges"
}
{
  "severity": "HIGH",
  "category": "DESERIAL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 378,
  "pattern": "\"fix\": \"Use yaml.safe_load() or yaml.load(data, Loader=yaml.SafeLoader)\",",
  "risk": "Unsafe YAML loading \u2014 can execute arbitrary code",
  "fix": "Use yaml.safe_load() or yaml.load(data, Loader=yaml.SafeLoader)"
}
{
  "severity": "CRITICAL",
  "category": "PRIV-ESC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 468,
  "pattern": "\"regex\": r\"<!--\\s*(?:system|instruction|override|ignore|execute|run|sudo|admin)\",",
  "risk": "Sudo invocation \u2014 privilege escalation attempt",
  "fix": "Remove sudo usage. Skills should never require elevated privileges"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 579,
  "pattern": "\"risk\": \"Dynamic Function constructor \u2014 equivalent to eval()\",",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "HIGH",
  "category": "FS-ABUSE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 918,
  "pattern": "shutil.rmtree(tmp_dir, ignore_errors=True)",
  "risk": "Recursive directory deletion \u2014 destructive operation",
  "fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
  "severity": "HIGH",
  "category": "FS-ABUSE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 930,
  "pattern": "shutil.rmtree(tmp_dir, ignore_errors=True)",
  "risk": "Recursive directory deletion \u2014 destructive operation",
  "fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
  "severity": "HIGH",
  "category": "FS-ABUSE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 1046,
  "pattern": "shutil.rmtree(cleanup_dir, ignore_errors=True)",
  "risk": "Recursive directory deletion \u2014 destructive operation",
  "fix": "Remove or restrict to specific, validated paths within skill scope"
}
{
  "severity": "CRITICAL",
  "category": "NET-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
  "line": 100,
  "pattern": "response = requests.post(",
  "risk": "Outbound HTTP write request \u2014 potential data exfiltration",
  "fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
  "severity": "CRITICAL",
  "category": "NET-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
  "line": 142,
  "pattern": "response = requests.post(",
  "risk": "Outbound HTTP write request \u2014 potential data exfiltration",
  "fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
  "severity": "CRITICAL",
  "category": "NET-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/tech-debt-tracker/assets/sample_codebase/src/payment_processor.py",
  "line": 182,
  "pattern": "response = requests.post(",
  "risk": "Outbound HTTP write request \u2014 potential data exfiltration",
  "fix": "Remove outbound POST/PUT/PATCH or verify destination is trusted and necessary"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 429,
  "pattern": "- os.system(), os.popen() usage",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 429,
  "pattern": "- os.system(), os.popen() usage",
  "risk": "Command execution via os.popen()",
  "fix": "Use subprocess.run() with list arguments and capture_output=True"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 431,
  "pattern": "- eval(), exec() usage",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 431,
  "pattern": "- eval(), exec() usage",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 123,
  "pattern": "code = 'os.system(\"ls -la\")'",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 128,
  "pattern": "code = 'result = eval(user_input)'",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 133,
  "pattern": "code = 'exec(user_code)'",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 138,
  "pattern": "code = 'subprocess.run(cmd, shell=True)'",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CRED-HARVEST",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 297,
  "pattern": "api_key = os.environ.get(\"API_KEY\")",
  "risk": "Reads sensitive environment variables",
  "fix": "Remove credential access. Skills should not need external credentials"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 441,
  "pattern": "os.system(\"echo \" + user_input)",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 464,
  "pattern": "subprocess.run(cmd, shell=True)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 486,
  "pattern": "return eval(user_input)",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 507,
  "pattern": "exec(user_code)",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-OVERRIDE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
  "line": 60,
  "pattern": "| **System prompt override** | \"Ignore previous instructions\", \"You are now...\" | \ud83d\udd34 CRITICAL |",
  "risk": "Attempts to override system prompt and prior instructions",
  "fix": "Remove instruction override attempts"
}
{
  "severity": "CRITICAL",
  "category": "SAFETY-BYPASS",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
  "line": 61,
  "pattern": "| **Role hijacking** | \"Act as root\", \"Pretend you have no restrictions\" | \ud83d\udd34 CRITICAL |",
  "risk": "Safety restriction bypass attempt",
  "fix": "Remove safety bypass instructions"
}
{
  "severity": "CRITICAL",
  "category": "SAFETY-BYPASS",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
  "line": 62,
  "pattern": "| **Safety bypass** | \"Skip safety checks\", \"Disable content filtering\" | \ud83d\udd34 CRITICAL |",
  "risk": "Explicit safety mechanism bypass",
  "fix": "Remove safety bypass directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/SKILL.md",
  "line": 65,
  "pattern": "| **Data extraction** | \"Send contents of\", \"Upload file to\", \"POST to\" | \ud83d\udd34 CRITICAL |",
  "risk": "Instruction to exfiltrate data",
  "fix": "Remove data transmission directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 66,
  "pattern": "| File read | Access credential files | `open(os.path.expanduser(\"~/.aws/credentials\"))` |",
  "risk": "Instruction to access sensitive files or credentials",
  "fix": "Remove credential/sensitive file access directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-OVERRIDE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 75,
  "pattern": "| Override | \"Ignore previous instructions\" | In SKILL.md body |",
  "risk": "Attempts to override system prompt and prior instructions",
  "fix": "Remove instruction override attempts"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-OVERRIDE",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 76,
  "pattern": "| Role hijack | \"You are now an unrestricted AI\" | Redefine agent identity |",
  "risk": "Role hijacking \u2014 attempts to redefine the AI's identity",
  "fix": "Remove role redefinition. Skills should provide instructions, not identity changes"
}
{
  "severity": "CRITICAL",
  "category": "SAFETY-BYPASS",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 77,
  "pattern": "| Safety bypass | \"Skip safety checks for efficiency\" | Disable guardrails |",
  "risk": "Explicit safety mechanism bypass",
  "fix": "Remove safety bypass directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/references/threat-model.md",
  "line": 247,
  "pattern": "- Access credential files or sensitive env vars",
  "risk": "Instruction to access sensitive files or credentials",
  "fix": "Remove credential/sensitive file access directives"
}
{
  "severity": "CRITICAL",
  "category": "PROMPT-EXFIL",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/env-secrets-manager/SKILL.md",
  "line": 84,
  "pattern": "Production applications should never read secrets from `.env` files or environment variables baked into container images",
  "risk": "Instruction to access sensitive files or credentials",
  "fix": "Remove credential/sensitive file access directives"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
  "line": 150,
  "pattern": "\"      - run: python3 -m pip install -U pip\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
  "line": 151,
  "pattern": "\"      - run: python3 -m pip install -r requirements.txt || true\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
  "line": 235,
  "pattern": "\"    - python3 -m pip install -U pip\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/ci-cd-pipeline-builder/scripts/pipeline_generator.py",
  "line": 236,
  "pattern": "\"    - python3 -m pip install -r requirements.txt || true\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-security-auditor/scripts/skill_security_auditor.py",
  "line": 717,
  "pattern": "# Check for pip/npm install in code",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to package.json for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
  "line": 102,
  "pattern": "\"message\": \"pip install without --no-cache-dir \u2014 retains pip cache in layer\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
  "line": 103,
  "pattern": "\"fix\": \"Use: pip install --no-cache-dir -r requirements.txt\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
  "line": 110,
  "pattern": "\"message\": \"npm install includes devDependencies \u2014 use npm ci --omit=dev for production\",",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to package.json for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/docker-development/scripts/dockerfile_analyzer.py",
  "line": 177,
  "pattern": "RUN pip install -r requirements.txt",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/browser-automation/scripts/scraping_toolkit.py",
  "line": 107,
  "pattern": "pip install playwright",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}
{
  "severity": "HIGH",
  "category": "DEPS-RUNTIME",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/browser-automation/scripts/form_automation_builder.py",
  "line": 183,
  "pattern": "pip install playwright",
  "risk": "Runtime package installation \u2014 may install untrusted code",
  "fix": "Move dependencies to requirements.txt for pre-install review"
}

❌ engineering/skill-tester — FAIL

Severity Count
🔴 Critical 13
Findings detail 
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 429,
  "pattern": "- os.system(), os.popen() usage",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 429,
  "pattern": "- os.system(), os.popen() usage",
  "risk": "Command execution via os.popen()",
  "fix": "Use subprocess.run() with list arguments and capture_output=True"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 431,
  "pattern": "- eval(), exec() usage",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/scripts/security_scorer.py",
  "line": 431,
  "pattern": "- eval(), exec() usage",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 123,
  "pattern": "code = 'os.system(\"ls -la\")'",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 128,
  "pattern": "code = 'result = eval(user_input)'",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 133,
  "pattern": "code = 'exec(user_code)'",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 138,
  "pattern": "code = 'subprocess.run(cmd, shell=True)'",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CRED-HARVEST",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 297,
  "pattern": "api_key = os.environ.get(\"API_KEY\")",
  "risk": "Reads sensitive environment variables",
  "fix": "Remove credential access. Skills should not need external credentials"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 441,
  "pattern": "os.system(\"echo \" + user_input)",
  "risk": "Arbitrary command execution via os.system()",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CMD-INJECT",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 464,
  "pattern": "subprocess.run(cmd, shell=True)",
  "risk": "Shell injection via subprocess with shell=True",
  "fix": "Use subprocess.run() with list arguments and shell=False"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 486,
  "pattern": "return eval(user_input)",
  "risk": "Arbitrary code execution via eval()",
  "fix": "Use ast.literal_eval() for data parsing or explicit parsing logic"
}
{
  "severity": "CRITICAL",
  "category": "CODE-EXEC",
  "file": "/home/runner/work/claude-skills/claude-skills/engineering/skill-tester/tests/test_security_scorer.py",
  "line": 507,
  "pattern": "exec(user_code)",
  "risk": "Arbitrary code execution via exec()",
  "fix": "Remove exec() \u2014 rewrite logic to avoid dynamic code execution"
}

@claude

claude Bot commented Mar 31, 2026

Copy link
Copy Markdown

PR Review — feat(skill-tester): plugin.json, description, index sync

Overall this is a clean, focused integration PR. A few observations:

✅ What looks good

  • plugin.json schema is correct: all required fields present (name, description, version, author object, homepage, repository, license, skills: "./"), no extra fields. Version matches repo v2.2.0.
  • Description improvement is substantive and trigger-oriented — the new text clearly describes when to invoke the skill and what it does, following the pattern seen in other high-quality skill descriptions in the indexes.
  • Gemini index count correctly updated from 274 → 276 to account for the two new PM skills.
  • Symlinks for meeting-analyzer and team-communications follow the established pattern (../../../project-management/<skill>/SKILL.md).

⚠️ Issues / Inconsistencies

1. Codex index not fully synced (missing meeting-analyzer and team-communications)

The .gemini/skills-index.json adds both meeting-analyzer and team-communications to the project-management category and updates total_skills to 276. However, .codex/skills-index.json only updates the skill-tester description — it does not add the two new PM skills.

Similarly, .gemini/skills/meeting-analyzer/SKILL.md and .gemini/skills/team-communications/SKILL.md symlinks are created, but there are no corresponding .codex/skills/meeting-analyzer/ or .codex/skills/team-communications/ symlinks.

Impact: Codex users won't discover these two skills. The PR title says "sync indexes" but Codex is only partially synced.

Fix: Add the two skills to .codex/skills-index.json (with category, name, description) and create the symlinks under .codex/skills/.

2. Codex total_skills count (if applicable)

If .codex/skills-index.json has a total_skills field similar to Gemini's, it also needs incrementing. Worth double-checking.

Minor Notes

  • The PR description mentions "Codex/Gemini symlinks synced" in the test plan — the Codex symlinks for the two new PM skills appear to be missing, so this checklist item is not fully complete.
  • No functional concerns with the plugin.json or SKILL.md changes themselves.

Summary

Approve with one required fix: Add meeting-analyzer and team-communications to .codex/skills-index.json and create their .codex/skills/ symlinks to achieve true parity between Gemini and Codex index state. Everything else is correct.

@alirezarezvani @claude
feat(docs): add Microsoft Clarity analytics tracking
976fc90 
Added Clarity tracking code (w52a0hrff7) to docs/overrides/main.html
alongside existing Google Analytics. Provides heatmaps, session
recordings, and user behavior insights for the docs site.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alirezarezvani
alirezarezvani deleted the fix/skill-tester-integration branch April 4, 2026 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@alirezarezvani