Skip to content

CLI gen: detect webhook-shaped sub-objects and emit composite --webhook-* builder #348

Description

@HavenDV

Part of #338. Depends on gap #1 (per-parameter flags).

Today

Many APIs have a webhook config nested inside the request body:

webhook:
  type: object
  properties:
    url: { type: string, format: uri }
    headers: { type: object, additionalProperties: { type: string } }
    metadata: { type: object }
    events: { type: array, items: { type: string, enum: [...] } }

Per-property flattening (gap #1) skips nested objects today. Firecrawl hand-writes BuildCrawlWebhookAsync / BuildBatchWebhookAsync to compose flags like --webhook-url, --webhook-header k=v, --webhook-metadata k=v, --webhook-event ... into a single webhook struct. ~150 lines, repeated twice with cosmetic differences.

Target

firecrawl crawl https://example.com \
  --webhook-url https://my.app/hook \
  --webhook-header authorization=Bearer\\ xyz \
  --webhook-header x-request-id=abc \
  --webhook-metadata customer-id=42 \
  --webhook-event completed --webhook-event errored

Proposed approach

  1. Detection heuristic: a nested object property is webhook-shaped when it has:
    • a property of string/format: uri (any name; url/endpoint/callback/webhook preferred)
    • AT LEAST ONE of: a string-map (headers), a generic-object (metadata), an enum-array (events)
  2. Generate --<prefix>-* flag set where <prefix> is the property's kebab-cased name (webhook, notification-url, etc.):
    • URL → single --<prefix>-url (string)
    • headers map → repeatable --<prefix>-header KEY=VALUE (Option<Dictionary<string,string>> with custom parser)
    • metadata map → repeatable --<prefix>-metadata KEY=VALUE
    • events array → repeatable --<prefix>-event with FromAmong(...) for enum constraint
  3. Composite builder in generated invocation: when --<prefix>-url is present, construct the nested object from all --<prefix>-* flags and assign to the request body.
  4. Add helpers to CliRuntime: ParseKeyValueAsync(IEnumerable<string> tokens) -> Dictionary<string,string>.
  5. Vendor extension x-cli-webhook: false to opt out (fall back to nested --<prop>-json).

Acceptance criteria

  • Firecrawl crawl and batch-scrape expose --webhook-url, --webhook-header, --webhook-metadata, --webhook-event after regen.
  • Repeated flags accumulate (--webhook-header a=1 --webhook-header b=2).
  • Without any --webhook-* flag, the webhook field stays unset in the request.
  • Snapshot test covers an operation with a webhook-shaped property.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions