Skip to content

Latest commit

Β 

History

History
552 lines (426 loc) Β· 17.3 KB

File metadata and controls

552 lines (426 loc) Β· 17.3 KB

Agentic Agile-V OpenHands Integration - Implementation Summary

Date: 2026-06-08
Status: MVP Implemented (Phases 0-4 Complete)
Implementation Plan: /Users/chris/Downloads/agentic_agile_v_openhands_implementation_plan.md


Executive Summary

Successfully implemented the MVP scope of the OpenHands integration for Agentic Agile-V, establishing OpenHands as an execution backend while preserving Agile-V as the control, evidence, and verification layer.

What Was Implemented

βœ… Phase 0: Integration contract and ADR documentation
βœ… Phase 1: OpenHands CLI namespace (agilev openhands ...)
βœ… Phase 2: Five skills for progressive disclosure
βœ… Phase 3: Seven lifecycle hooks with mechanical enforcement
βœ… Phase 4: Task context resolution from multiple sources

What Remains

⏳ Phases 5-12: Evidence schema extensions, evidence adapter, scope enforcement, builder/verifier pattern, GitHub Actions, event ledger, reports, and examples.


Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚      Agentic Agile-V Control Plane      β”‚
β”‚  - Task briefs & risk classification    β”‚
β”‚  - Evidence requirements & validation   β”‚
β”‚  - Scope control & approval policy      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              β”‚ skills + hooks
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚       OpenHands Execution Plane         β”‚
β”‚  - Repository inspection & analysis     β”‚
β”‚  - Code implementation                  β”‚
β”‚  - Test execution                       β”‚
β”‚  - Pull request creation                β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              β”‚ evidence
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     Evidence Bundle + Verification      β”‚
β”‚  - Changed files & test results         β”‚
β”‚  - Tool usage logs                      β”‚
β”‚  - Verifier reports (L2+)               β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Implementation Details

Phase 0: Integration Contract and ADR βœ…

Created:

  • docs/adr/ADR-0001-openhands-execution-backend.md
  • docs/integrations/openhands.md
  • config/openhands.yaml

Key Decisions:

  • OpenHands owns execution, Agile-V owns acceptance
  • Skills teach, hooks enforce
  • Evidence validation is independent of agent claims
  • L3/L4 changes cannot be self-approved by agents

Files:

  • Integration responsibilities documented
  • Non-goals explicit (no self-approval, no bypassing briefs)
  • Risk mitigation strategies defined

Phase 1: OpenHands CLI Namespace βœ…

Created:

  • src/agilev/openhands/__init__.py
  • src/agilev/openhands/scaffold.py (1400+ lines)
  • Extended src/agilev/cli.py with openhands subcommand group

Commands:

agilev openhands init      # Generate integration files
agilev openhands doctor    # Validate setup
agilev openhands scaffold  # Regenerate files
agilev openhands validate  # Validate session for task
agilev openhands handoff   # Show handoff report

Features:

  • Idempotent initialization (--force to overwrite)
  • Comprehensive validation (21 checks across skills, hooks, policies, config)
  • Force regeneration support
  • Task-specific validation

Phase 2: OpenHands Skills βœ…

Created Skills:

  1. agile-v-core (.agents/skills/agile-v-core/SKILL.md)

    • Core workflow rules
    • Task brief requirement before implementation
    • Evidence-controlled acceptance
    • Risk-level guidance (L0-L4)
    • Never: remove tests, weaken security, self-approve L3/L4
  2. agile-v-builder (.agents/skills/agile-v-builder/SKILL.md)

    • Implementation workflow (inspect β†’ plan β†’ edit β†’ test β†’ document)
    • Scope minimization
    • Test-first mindset
    • Evidence collection guidance
    • Implementation summary requirements
  3. agile-v-verifier (.agents/skills/agile-v-verifier/SKILL.md)

    • Independent verification from fresh context
    • Acceptance criteria mapping
    • Edge case detection
    • Scope creep detection
    • Verification report requirements
    • Cannot self-approve L3/L4
  4. agile-v-evidence (.agents/skills/agile-v-evidence/SKILL.md)

    • Evidence bundle maintenance
    • Never fabricate evidence
    • Git/CI as source of truth
    • OpenHands metadata extensions
  5. agile-v-risk-classifier (.agents/skills/agile-v-risk-classifier/SKILL.md)

    • L0-L4 classification guidance
    • Evidence requirements by level
    • Examples for each level
    • Escalation rules (when uncertain, escalate)

Design:

  • Concise enough to load efficiently
  • Reference Agile-V artifacts (don't duplicate methodology)
  • Progressive disclosure (load based on mode: builder/verifier)

Phase 3: OpenHands Hooks βœ…

Created Hooks:

Hook Lifecycle Blocking Purpose
enforce_task_brief.sh user_prompt_submit Yes Require task ID/brief before implementation
block_unsafe_commands.sh pre_tool_use (terminal) Yes Block destructive commands
validate_scope.sh pre_tool_use (all) Yes Check scope (MVP: allow + warn)
log_tool_usage.sh post_tool_use No Append to tool log (JSONL)
collect_session_metadata.sh session_start No Record session metadata
validate_evidence_on_stop.sh stop Yes Block until evidence passes
generate_handoff_on_session_end.sh session_end No Generate handoff summary

Hook Configuration:

  • .openhands/hooks.json - Hook registry with matchers and timeouts
  • Hooks return JSON: {"decision": "allow|deny", "reason": "..."}
  • Blocking hooks exit with code 2 on deny
  • All hooks are executable (chmod +x)

Dangerous Command Patterns Blocked:

  • rm -rf /
  • dd if=
  • mkfs
  • Fork bombs
  • chmod 777
  • curl ... | sudo bash
  • And more...

Setup Script:

  • .openhands/setup.sh - Validates environment, makes hooks executable, checks for agilev CLI

Phase 4: Task Context Resolution βœ…

Created:

  • src/agilev/task_context.py - TaskContextResolver class

Resolution Order:

  1. Explicit CLI option (--task AAV-001)
  2. Environment variable (AGILEV_TASK_ID=AAV-001)
  3. Git branch name (aav-001-* pattern)
  4. GitHub metadata (GITHUB_PR_TITLE, GITHUB_ISSUE_TITLE)
  5. Latest modified task (if unambiguous within 24 hours)
  6. Fail with clear error if ambiguous

Features:

  • Task ID normalization (AAV-001 β†’ AAV-0001)
  • Fail-closed on ambiguity (prevents writing to wrong task)
  • Branch pattern matching (case-insensitive)
  • Validation of task existence

Policy Files Created βœ…

Policy Configuration:

  • config/policies/openhands_dangerous_commands.yaml - Blocked command patterns
  • config/policies/scope_policy.yaml - Default scope behavior, security-sensitive paths
  • config/policies/approval_policy.yaml - Human approval requirements by risk level
  • config/policies/evidence_policy.yaml - Evidence requirements by risk level
  • config/policies/risk_level_policy.yaml - Risk classification guidance and auto-classification

Highlights:

  • L0: Documentation (no approval)
  • L1: Tests or rationale (no approval)
  • L2: Passing tests + verifier (reviewer approval)
  • L3: L2 + rollback path (domain owner approval)
  • L4: L3 + simulation/HIL/formal + traceability (formal approval)

File Structure Created

agentic_agile_v/
β”œβ”€β”€ docs/
β”‚   β”œβ”€β”€ adr/
β”‚   β”‚   └── ADR-0001-openhands-execution-backend.md
β”‚   └── integrations/
β”‚       └── openhands.md
β”œβ”€β”€ config/
β”‚   β”œβ”€β”€ openhands.yaml
β”‚   └── policies/
β”‚       β”œβ”€β”€ openhands_dangerous_commands.yaml
β”‚       β”œβ”€β”€ scope_policy.yaml
β”‚       β”œβ”€β”€ approval_policy.yaml
β”‚       β”œβ”€β”€ evidence_policy.yaml
β”‚       └── risk_level_policy.yaml
β”œβ”€β”€ .agents/
β”‚   └── skills/
β”‚       β”œβ”€β”€ agile-v-core/SKILL.md
β”‚       β”œβ”€β”€ agile-v-builder/SKILL.md
β”‚       β”œβ”€β”€ agile-v-verifier/SKILL.md
β”‚       β”œβ”€β”€ agile-v-evidence/SKILL.md
β”‚       └── agile-v-risk-classifier/SKILL.md
β”œβ”€β”€ .openhands/
β”‚   β”œβ”€β”€ setup.sh
β”‚   β”œβ”€β”€ hooks.json
β”‚   β”œβ”€β”€ hooks/
β”‚   β”‚   β”œβ”€β”€ enforce_task_brief.sh
β”‚   β”‚   β”œβ”€β”€ block_unsafe_commands.sh
β”‚   β”‚   β”œβ”€β”€ validate_scope.sh
β”‚   β”‚   β”œβ”€β”€ log_tool_usage.sh
β”‚   β”‚   β”œβ”€β”€ collect_session_metadata.sh
β”‚   β”‚   β”œβ”€β”€ validate_evidence_on_stop.sh
β”‚   β”‚   └── generate_handoff_on_session_end.sh
β”‚   └── logs/
└── src/agilev/
    β”œβ”€β”€ cli.py (extended with openhands commands)
    β”œβ”€β”€ task_context.py
    β”œβ”€β”€ openhands/
    β”‚   β”œβ”€β”€ __init__.py
    β”‚   └── scaffold.py
    β”œβ”€β”€ policies/
    β”‚   └── __init__.py
    └── ledger/
        └── __init__.py

Testing and Validation

Test Results

Test Script: test_openhands_integration.py

βœ… OpenHandsScaffold - All components created
βœ… Doctor checks - 21/21 passed after init
βœ… TaskContextResolver - Explicit and normalized task IDs
βœ… Skills - All 5 skills created with proper YAML frontmatter
βœ… Hooks - All 7 hooks created and executable
βœ… Policies - All 5 policy files created

Manual Validation

# Check skills exist
$ ls .agents/skills/
agile-v-builder  agile-v-core  agile-v-evidence  
agile-v-risk-classifier  agile-v-verifier

# Check hooks are executable
$ ls -la .openhands/hooks/*.sh
-rwxr-xr-x  block_unsafe_commands.sh
-rwxr-xr-x  collect_session_metadata.sh
-rwxr-xr-x  enforce_task_brief.sh
-rwxr-xr-x  generate_handoff_on_session_end.sh
-rwxr-xr-x  log_tool_usage.sh
-rwxr-xr-x  validate_evidence_on_stop.sh
-rwxr-xr-x  validate_scope.sh

# Check doctor validation
$ python3 -c "..."
Total checks: 21
Passed: 21
Failed: 0

Usage Guide

Initialize OpenHands Integration

cd your-agentic-agile-v-repo
agilev openhands init

Output:

πŸ”§ Initializing OpenHands integration...

βœ… OpenHands integration initialized

Created 19 files:

πŸ“š Skills:
  βœ“ .agents/skills/agile-v-core/SKILL.md
  βœ“ .agents/skills/agile-v-builder/SKILL.md
  βœ“ .agents/skills/agile-v-verifier/SKILL.md
  βœ“ .agents/skills/agile-v-evidence/SKILL.md
  βœ“ .agents/skills/agile-v-risk-classifier/SKILL.md

πŸͺ Hooks:
  βœ“ .openhands/hooks/enforce_task_brief.sh
  βœ“ .openhands/hooks/block_unsafe_commands.sh
  βœ“ .openhands/hooks/validate_scope.sh
  ...

πŸ“‹ Policies:
  βœ“ config/policies/openhands_dangerous_commands.yaml
  ...

Validate Setup

agilev openhands doctor

Output:

πŸ” Checking OpenHands integration setup...

Core Files:
  βœ… agents_md
  βœ… setup_script
  βœ… hooks_config
  βœ… openhands_config

Skills:
  βœ… skill_agile-v-core
  βœ… skill_agile-v-builder
  ...

Results: 21 passed, 0 failed

βœ… OpenHands integration ready

Create a Task

agilev new --title "Add retry handling" --risk L2

Run OpenHands

Currently: Launch OpenHands manually and point to the repository. Skills and hooks activate automatically.

Future: agilev openhands run --task AAV-0001 --mode builder

Validate Session

agilev openhands validate --task AAV-0001

View Handoff

agilev openhands handoff --task AAV-0001

Next Steps (Remaining Phases)

Phase 5: Extend Evidence Bundle Schema

  • Add agent_execution section (engine, mode, session_id, tool_log_path)
  • Add scope_control section (allowed_paths, blocked_paths, violations)
  • Add verification section (builder_summary, verifier_report, result)
  • Maintain backward compatibility with existing evidence bundles

Phase 6: Implement Evidence Adapter

  • agilev openhands evidence collect --task AAV-XXXX
  • Map OpenHands tool log β†’ evidence bundle
  • Map Git diff β†’ changed_files (truth over agent claims)
  • Map test outputs β†’ tests.results
  • Map CI results β†’ checks
  • Never fabricate passed tests

Phase 7: Add Scope Policy Enforcement

  • Parse task brief YAML frontmatter for allowed/blocked paths
  • Compare changed files (from Git) against allowed paths
  • Block or warn on out-of-scope changes
  • Detect dependency changes (Python, Node, Rust, Go, Java)
  • Flag public API changes
  • Full implementation of validate_scope.sh hook

Phase 8: Add Builder/Verifier Workflow

  • agilev openhands run --task AAV-XXXX --mode builder
  • agilev openhands verify --task AAV-XXXX --fresh-context
  • Separate builder and verifier sessions
  • Verifier read-only by default
  • Verifier report schema and validation
  • L2+ requires verifier

Phase 9: Add GitHub Actions Workflows

  • .github/workflows/agilev-openhands-builder.yml
  • .github/workflows/agilev-openhands-verifier.yml
  • .github/workflows/agilev-gates.yml
  • Label-driven automation
  • PR comment with evidence summary
  • Merge blocking on failed gates

Phase 10: Add Event Ledger

  • Append-only event log (events.jsonl)
  • Hash chain for tamper detection
  • Event types: TaskBriefCreated, ToolUsed, EvidenceUpdated, etc.
  • Ledger validation command

Phase 11: Add Reports and Handoff

  • agilev report --task AAV-XXXX
  • Deterministic from files/evidence (not agent memory)
  • Suitable for PR comments and human review
  • Handoff includes: objective, changed files, tests, risks, next action

Phase 12: Documentation and Examples

  • Quickstart guide
  • Integration patterns (builder + verifier)
  • Example task packages (L0-L4)
  • Troubleshooting guide
  • Video walkthrough

Metrics

Code Statistics

Component Files Created Lines of Code
Documentation 2 ~1,500
Configuration 6 ~500
Python Modules 4 ~1,600
Skills 5 ~400
Hooks 8 ~200
Total 25 ~4,200

Test Coverage

  • βœ… OpenHandsScaffold initialization
  • βœ… Doctor validation (21 checks)
  • βœ… Task context resolution
  • βœ… File creation and permissions
  • ⏳ Hook execution (not yet tested)
  • ⏳ Evidence adapter (not yet implemented)
  • ⏳ Scope validation (not yet implemented)

Risks and Mitigations

Risk Mitigation Status
Agent bypasses task brief enforce_task_brief.sh hook blocks βœ… Implemented
Agent removes tests Skill forbids; evidence tracks delta βœ… Skill documented
Agent expands scope validate_scope.sh hook (MVP: warn) ⏳ Partial (warns only)
Agent self-approves L3/L4 Policy enforces human approval; CI blocks βœ… Policy defined
Agent fabricates evidence Evidence adapter uses Git/CI truth ⏳ Not yet implemented
Hook or skill ignored Stop hook blocks; CI fails βœ… Stop hook implemented

Lessons Learned

What Worked Well

  1. Skill-based progressive disclosure: Skills are concise, loadable, and extensible
  2. Hook-based enforcement: Mechanical gates that don't rely on agent compliance
  3. Policy-driven design: YAML policies are human-readable and version-controllable
  4. Task context resolution: Flexible resolution order reduces friction
  5. Idempotent scaffolding: Safe to re-run, respects existing files

Challenges

  1. Python environment issues: Testing required PYTHONPATH workarounds
  2. Scope validation complexity: Full implementation deferred to Phase 7
  3. Evidence schema extension: Backward compatibility requires careful design
  4. GitHub Actions integration: Requires OpenHands SDK or wrapper scripts

Design Trade-offs

  1. MVP scope validation: Allow + warn vs. strict blocking (chose allow for MVP)
  2. Hook language: Bash vs. Python (chose Bash for simplicity and no dependencies)
  3. Evidence source: Agent claims vs. Git/CI (chose Git/CI as source of truth)
  4. Skill size: Comprehensive vs. concise (chose concise for loadability)

Conclusion

Successfully implemented the MVP scope of the OpenHands integration for Agentic Agile-V. The foundation is solid:

βœ… Control separation: Agile-V controls acceptance, OpenHands executes
βœ… Skills teach: Five skills for progressive disclosure
βœ… Hooks enforce: Seven lifecycle hooks with mechanical gates
βœ… Policies guide: Risk-based evidence requirements
βœ… Context resolves: Flexible task ID resolution

The integration is additive (doesn't break existing workflows), safe (fail-closed on ambiguity), and extensible (ready for Phases 5-12).

Next Priority: Phase 5 (Evidence schema extension) and Phase 6 (Evidence adapter) to close the loop between OpenHands execution and Agile-V validation.


References

  • Implementation Plan: /Users/chris/Downloads/agentic_agile_v_openhands_implementation_plan.md
  • ADR: docs/adr/ADR-0001-openhands-execution-backend.md
  • Integration Guide: docs/integrations/openhands.md
  • Test Script: test_openhands_integration.py
  • Repository: /Users/chris/Dev/agile-v/agentic_agile_v

Implemented by: OpenCode Agent
Date: 2026-06-08
Phases Completed: 0-4 of 12
Status: MVP Ready for Testing