Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/aw/create-agentic-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@ Rules:
- always restrict `create-pull-request.allowed-files`
- prefer the dedicated safe output instead of shelling out to `gh` for the same mutation
- include `noop` guidance in the prompt so successful no-op runs are explicit
- when using `create-issue`, instruct the agent to provide a meaningful body (20-65000 characters; avoid placeholder-only text)

### 7. Decide who can trigger the workflow

Expand Down
2 changes: 1 addition & 1 deletion .github/aw/experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,5 +292,5 @@ experiments:
- ❌ **Interpreting early results** (<~20 runs/variant) — chance variation dominates.
- ❌ **Experiments as feature flags** — use `features:` for deterministic switches.
- ❌ **Engine experiments in one file** — `engine:` cannot switch mid-run; use two parallel files.
- ❌ **Nesting `{{#if experiments.<name> }}` inside `{{#runtime-import? }}`** — evaluation order not guaranteed across import boundaries.
- ❌ **Nesting `{{#if experiments.<name> }}` inside `{{#runtime-import? }}`** — evaluation order is brittle across import boundaries. Prefer explicit branching in the main workflow prompt or separate workflow files per variant.
- ❌ **Writing the internal env-var form** `__GH_AW_EXPERIMENTS__*` — implementation detail, may change.
4 changes: 4 additions & 0 deletions .github/aw/safe-outputs-content.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ description: Safe-output reference for issue, discussion, comment, and pull requ
allowed-repos: [owner/other] # Optional: additional repos agent can target (agent uses `repo` field in output)
```

`create_issue` output validation requires:
- `body` minimum length: **20** characters
- `body` maximum length: **65000** characters

**Auto-Expiration**: The `expires` field auto-closes issues after a time period. Supports integers (days) or relative formats (2h, 7d, 2w, 1m, 1y). Generates `agentics-maintenance.yml` workflow that runs at minimum required frequency based on shortest expiration time: 1 day or less → every 2 hours, 2 days → every 6 hours, 3-4 days → every 12 hours, 5+ days → daily.
**Deduplication for Scheduled Workflows**: When a `schedule:` trigger is combined with `create-issue`, use `skip-if-match:` in the `on:` block to prevent opening a duplicate issue on every run. Pair with `expires:` so stale issues are cleaned up automatically:

Expand Down
5 changes: 4 additions & 1 deletion .github/aw/syntax-agentic.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ description: Agentic workflow specific frontmatter fields for GitHub Agentic Wor
- `cli-proxy: true` - Enable AWF CLI proxy sidecar for secure read-only `gh` CLI access without exposing `GITHUB_TOKEN` (requires AWF v0.26.0+). Prerequisite for `integrity-reactions`; the compiler enables it automatically when `integrity-reactions: true` is set.
- `integrity-reactions: true` - Enable reaction-based integrity promotion/demotion. Maintainers can use 👍/❤️ reactions to promote content to `approved` and 👎/😕 to demote it to `none`. Compiler automatically enables `cli-proxy`. Requires `tools.github.min-integrity` to be set and MCPG >= v0.2.18. Defaults: endorsement reactions THUMBS_UP/HEART, disapproval reactions THUMBS_DOWN/CONFUSED, endorser-min-integrity: approved, disapproval-integrity: none.
- `mcp-cli: true` - Deprecated. This flag has been removed; MCP CLI mounting is now always enabled when `tools.cli-proxy: true` is set.
- `dangerously-disable-sandbox-agent: "<justification>"` - Required when `sandbox.agent: false` is set. Must be a plain string justification (minimum 20 characters; expressions are not allowed) that explains why disabling the sandbox is safe for this workflow.

- **`experiments:`** - A/B testing experiments for balanced variant selection (object)
- Maps experiment names to variant lists (bare array) or full config objects
Expand Down Expand Up @@ -343,9 +344,11 @@ description: Agentic workflow specific frontmatter fields for GitHub Agentic Wor
model-fallback: false # Optional: disable model fallback (default true); set false for BYOK Azure OpenAI to prevent deployment-name rewriting
```

- To disable the agent firewall while keeping MCP gateway enabled (not allowed in strict mode):
- To disable the agent firewall while keeping MCP gateway enabled, you must provide the dangerous-disable justification feature:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] Wording inconsistency: "the dangerous-disable justification feature" does not match the actual feature key dangerously-disable-sandbox-agent. A reader skimming this doc will not know what key to write.

Suggest replacing with: "you must provide the dangerously-disable-sandbox-agent justification:"


```yaml
features:
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
```
Expand Down
8 changes: 7 additions & 1 deletion .github/aw/upgrade-agentic-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,13 @@ Before attempting to compile, apply automatic codemods:

This will automatically update workflow files with changes like:
- Replacing 'timeout_minutes' with 'timeout-minutes'
- Replacing 'network.firewall' with 'sandbox.agent: false'
- Replacing `network.firewall: false` with:
```yaml
features:
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
```
- Removing deprecated 'mcp-scripts.mode' field

2. **Review the Changes**
Expand Down
2 changes: 2 additions & 0 deletions docs/src/content/docs/reference/safe-outputs.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,8 @@ See [Cross-Repository Operations](/gh-aw/reference/cross-repository/) for compre

#### `create_issue` tool field schema (`fields`)

`create_issue.body` must be between **20** and **65000** characters.

| Parameter | Type | Required | Description | Example |
|-----------|------|----------|-------------|---------|
| `fields` | `array<object>` | No | Optional issue field updates to apply immediately after issue creation. | `[{"name":"Priority","value":"P1"}]` |
Expand Down
77 changes: 77 additions & 0 deletions pkg/cli/codemod_network_firewall.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package cli

import (
"fmt"
"strconv"
"strings"

Expand All @@ -9,6 +10,8 @@ import (

var networkFirewallCodemodLog = logger.New("cli:codemod_network_firewall")

const migratedSandboxDisableJustification = "migrated from deprecated network.firewall disable setting"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The migratedSandboxDisableJustification constant is auto-injected into migrated workflows, which are then validated by getSandboxDisableJustification() (≥20 chars, no expressions). There is no test asserting the constant itself satisfies those rules. A future edit that shortens the string below 20 characters would silently produce codemod output that fails validation.

💡 Suggested cross-validation test
func TestMigratedJustificationPassesValidation(t *testing.T) {
    data := &workflow.WorkflowData{
        Features: map[string]any{
            "dangerously-disable-sandbox-agent": migratedSandboxDisableJustification,
        },
        SandboxConfig: &workflow.SandboxConfig{
            Agent: &workflow.AgentSandboxConfig{Disabled: true},
        },
    }
    // This would require exporting getSandboxDisableJustification or
    // testing via the full compiler path:
    require.NoError(t, compiler.CompileWorkflow(workflowWithMigratedJustification))
}

Alternatively, a simple length assertion in a package test is enough to lock the contract.


// getNetworkFirewallCodemod creates a codemod for migrating network.firewall to sandbox.agent
func getNetworkFirewallCodemod() Codemod {
return newFieldRemovalCodemod(fieldRemovalCodemodConfig{
Expand All @@ -21,26 +24,44 @@ func getNetworkFirewallCodemod() Codemod {
LogMsg: "Applied network.firewall migration (firewall now always enabled via sandbox.agent: awf default)",
Log: networkFirewallCodemodLog,
PostTransform: func(lines []string, frontmatter map[string]any, fieldValue any) []string {
requiresDisableFlag := requiresSandboxDisableFeatureFlag(fieldValue)
_, hasSandbox := frontmatter["sandbox"]

if !hasSandbox {
sandboxLines := sandboxAgentLinesFromFirewall(fieldValue)
if len(sandboxLines) > 0 {
lines = insertSandboxAfterNetworkBlock(lines, sandboxLines)
if requiresDisableFlag {
lines = ensureSandboxDisableFeatureFlag(lines)
}
networkFirewallCodemodLog.Print("Converted deprecated network.firewall to sandbox.agent")
}
return lines
}

lines, merged := mergeFirewallIntoExistingSandbox(lines, fieldValue)
if merged {
if requiresDisableFlag {
lines = ensureSandboxDisableFeatureFlag(lines)
}
networkFirewallCodemodLog.Print("Merged deprecated network.firewall into existing sandbox.agent")
}
return lines
},
})
}

func requiresSandboxDisableFeatureFlag(fieldValue any) bool {
switch value := fieldValue.(type) {
case bool:
return !value
case string:
return strings.EqualFold(strings.TrimSpace(value), "disable")
default:
return false
}
}

func sandboxAgentLinesFromFirewall(fieldValue any) []string {
switch value := fieldValue.(type) {
case bool:
Expand Down Expand Up @@ -260,3 +281,59 @@ func indentLines(lines []string, indent string) []string {
}
return indented
}

func ensureSandboxDisableFeatureFlag(lines []string) []string {
featureKey := "dangerously-disable-sandbox-agent:"

featuresIdx := -1
featuresIndent := ""
featuresEnd := len(lines)
for i, line := range lines {
trimmed := strings.TrimSpace(line)
if isTopLevelKey(line) && strings.HasPrefix(trimmed, "features:") {
featuresIdx = i
featuresIndent = getIndentation(line)
for j := i + 1; j < len(lines); j++ {
if isTopLevelKey(lines[j]) {
featuresEnd = j
break
}
}
break
}
}

if featuresIdx >= 0 {
featureIndent := featuresIndent + " "
for i := featuresIdx + 1; i < featuresEnd; i++ {
if strings.HasPrefix(strings.TrimSpace(lines[i]), featureKey) {
return lines
}
Comment on lines +308 to +311
}
featureLine := fmt.Sprintf("%s%s %s", featureIndent, featureKey, strconv.Quote(migratedSandboxDisableJustification))
newLines := make([]string, 0, len(lines)+1)
newLines = append(newLines, lines[:featuresEnd]...)
newLines = append(newLines, featureLine)
newLines = append(newLines, lines[featuresEnd:]...)
return newLines
}

insertIndex := len(lines)
for i, line := range lines {
if isTopLevelKey(line) && strings.HasPrefix(strings.TrimSpace(line), "sandbox:") {
insertIndex = i
break
}
}

featureLines := []string{
"features:",
" " + featureKey + " " + strconv.Quote(migratedSandboxDisableJustification),
}

newLines := make([]string, 0, len(lines)+len(featureLines))
newLines = append(newLines, lines[:insertIndex]...)
newLines = append(newLines, featureLines...)
newLines = append(newLines, lines[insertIndex:]...)
return newLines
}
38 changes: 38 additions & 0 deletions pkg/cli/codemod_network_firewall_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ permissions:
assert.NotContains(t, result, "firewall:", "Should remove firewall field")
assert.Contains(t, result, "sandbox:", "Should add sandbox block")
assert.Contains(t, result, "agent: false", "Should convert firewall false to sandbox.agent: false")
assert.Contains(t, result, `dangerously-disable-sandbox-agent: "migrated from deprecated network.firewall disable setting"`, "Should add required sandbox-disable justification feature")
}

func TestNetworkFirewallCodemod_NoNetworkField(t *testing.T) {
Expand Down Expand Up @@ -208,6 +209,43 @@ sandbox:
assert.Contains(t, result, "sandbox:", "Should preserve existing sandbox block")
assert.Contains(t, result, "mcp: true", "Should preserve existing sandbox settings")
assert.Contains(t, result, "agent: false", "Should migrate firewall false to sandbox.agent: false")
assert.Contains(t, result, `dangerously-disable-sandbox-agent: "migrated from deprecated network.firewall disable setting"`, "Should add required sandbox-disable justification feature")
}

func TestNetworkFirewallCodemod_PreservesExistingSandboxDisableJustification(t *testing.T) {
codemod := getNetworkFirewallCodemod()

content := `---
on: workflow_dispatch
network:
firewall: false
features:
dangerously-disable-sandbox-agent: "already documented justification string with enough detail"
sandbox:
mcp: true
---

# Test Workflow`

frontmatter := map[string]any{
"on": "workflow_dispatch",
"network": map[string]any{
"firewall": false,
},
"features": map[string]any{
"dangerously-disable-sandbox-agent": "already documented justification string with enough detail",
},
"sandbox": map[string]any{
"mcp": true,
},
}

result, applied, err := codemod.Apply(content, frontmatter)

require.NoError(t, err)
assert.True(t, applied)
assert.Contains(t, result, `dangerously-disable-sandbox-agent: "already documented justification string with enough detail"`)
assert.NotContains(t, result, migratedSandboxDisableJustification, "Should not overwrite existing justification")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] There is no test for when features: already exists with other keys (not dangerously-disable-sandbox-agent). The if featuresIdx >= 0 branch in ensureSandboxDisableFeatureFlag injects the key into the existing block — this path is currently untested.

💡 Suggested test scenario

Add a test where the content includes:

features:
  some-other-flag: true

but not dangerously-disable-sandbox-agent. After the codemod runs, assert:

  • dangerously-disable-sandbox-agent appears inside the features: block
  • some-other-flag: true is preserved
  • No duplicate features: top-level key is added

Without this, a line-index off-by-one in ensureSandboxDisableFeatureFlag could corrupt the features: block without any test catching it.

}

func TestNetworkFirewallCodemod_MigratesFirewallVersionIntoExistingSandbox(t *testing.T) {
Expand Down
2 changes: 1 addition & 1 deletion pkg/constants/feature_constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,6 @@ const (
// Workflow frontmatter usage:
//
// features:
// dangerously-disable-sandbox-agent: true
// dangerously-disable-sandbox-agent: "controlled environment with no internet access"
DangerouslyDisableSandboxAgentFeatureFlag FeatureFlag = "dangerously-disable-sandbox-agent"
)
2 changes: 1 addition & 1 deletion pkg/workflow/compiler_validators_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@ func TestValidateToolConfiguration_EmitsSandboxWarningBeforeThreatDetectionError
workflowData := &WorkflowData{
Name: "Test",
Features: map[string]any{
"dangerously-disable-sandbox-agent": true,
"dangerously-disable-sandbox-agent": "controlled environment with no internet access",
},
SandboxConfig: &SandboxConfig{
Agent: &AgentSandboxConfig{Disabled: true},
Expand Down
6 changes: 3 additions & 3 deletions pkg/workflow/importable_tools_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -802,7 +802,7 @@ permissions:
tools:
bash: true
features:
dangerously-disable-sandbox-agent: true
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
imports:
Expand Down Expand Up @@ -893,7 +893,7 @@ permissions:
contents: read
issues: read
features:
dangerously-disable-sandbox-agent: true
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
imports:
Expand Down Expand Up @@ -967,7 +967,7 @@ permissions:
tools:
github: false
features:
dangerously-disable-sandbox-agent: true
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
imports:
Expand Down
63 changes: 63 additions & 0 deletions pkg/workflow/imports_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,69 @@ This is a test workflow with multiple imports.
}
}

func TestCompileWorkflowWithConditionalImport(t *testing.T) {
tempDir := testutil.TempDir(t, "test-*")

sharedPath := filepath.Join(tempDir, "shared-conditional.md")
sharedContent := `---
steps:
- name: Conditional Imported Step
run: echo "from import"
---

Imported conditional instructions.
`
if err := os.WriteFile(sharedPath, []byte(sharedContent), 0644); err != nil {
t.Fatalf("Failed to write shared conditional file: %v", err)
}

workflowPath := filepath.Join(tempDir, "test-workflow.md")
workflowContent := `---
on: issues
permissions:
contents: read
issues: read
pull-requests: read
engine: copilot
experiments:
strategy: [eager, lazy]
imports:
- path: shared-conditional.md
if: "experiments.strategy == 'eager'"
---

# Test Workflow

Main workflow body.
`
if err := os.WriteFile(workflowPath, []byte(workflowContent), 0644); err != nil {
t.Fatalf("Failed to write workflow file: %v", err)
}

compiler := workflow.NewCompiler()
if err := compiler.CompileWorkflow(workflowPath); err != nil {
t.Fatalf("CompileWorkflow failed: %v", err)
}

lockFilePath := stringutil.MarkdownToLockFile(workflowPath)
lockFileContent, err := os.ReadFile(lockFilePath)
if err != nil {
t.Fatalf("Failed to read lock file: %v", err)
}

compiled := string(lockFileContent)
assertions := []string{
`{{#if experiments.strategy == "eager"}}`,
"{{/if}}",
"needs.activation.outputs.strategy == 'eager'",
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] These assertions verify that the template guard syntax is present in the compiled output, but not that the imported step content is wrapped inside the {{#if}} block. The step echo "from import" could appear outside the guard and all three assertions would still pass.

💡 Stronger structural assertion
ifIdx := strings.Index(compiled, `{{#if experiments.strategy == "eager"}}`)
endifIdx := strings.Index(compiled, "{{/if}}")
stepIdx := strings.Index(compiled, `echo "from import"`)

require.Greater(t, ifIdx, -1, "{{#if}} guard should be present")
require.Greater(t, endifIdx, -1, "{{/if}} guard should be present")
require.Greater(t, stepIdx, ifIdx, "imported step should appear after {{#if}} guard")
require.Less(t, stepIdx, endifIdx, "imported step should appear before {{/if}} guard")

This turns a structural smoke test into a real gating assertion.

for _, expected := range assertions {
if !strings.Contains(compiled, expected) {
t.Errorf("Expected compiled workflow to contain %q", expected)
}
}
}

func TestCompileWorkflowWithMCPServersImport(t *testing.T) {
// Create a temporary directory for test files
tempDir := testutil.TempDir(t, "test-*")
Expand Down
8 changes: 4 additions & 4 deletions pkg/workflow/pull_request_target_validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ on:
tools:
github: false
features:
dangerously-disable-sandbox-agent: true
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
checkout: false
Expand All @@ -61,7 +61,7 @@ on:
tools:
github: false
features:
dangerously-disable-sandbox-agent: true
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
---
Expand Down Expand Up @@ -108,7 +108,7 @@ on:
tools:
github: false
features:
dangerously-disable-sandbox-agent: true
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
---
Expand All @@ -131,7 +131,7 @@ on:
tools:
github: false
features:
dangerously-disable-sandbox-agent: true
dangerously-disable-sandbox-agent: "controlled environment with no internet access"
sandbox:
agent: false
---
Expand Down
Loading
Loading