Add policy enforcement to prevent runaway crawls and credit burn

[L1AD]

This server exposes ~10 tools including recursive crawling, batch scraping, deep research, and browser session control. Several can consume significant API credits or interact with sites in uncontrolled ways:

• firecrawl_crawl -- recursive site crawl with no built-in depth/page limit enforcement at the MCP layer
• firecrawl_deep_research -- autonomous multi-step research that can chain many API calls
• firecrawl_batch_scrape -- bulk operations across many URLs
• Browser session tools -- remote browser control

An agent in a loop calling firecrawl_crawl can burn through API credits rapidly.

Suggestion: Consider documenting support for PolicyLayer/Intercept, an open-source MCP proxy that enforces policies on tool calls before execution.

Example policy:

version: "1"
default: allow

tools:
  firecrawl_crawl:
    rules:
      - rate_limit: 3/hour

  firecrawl_deep_research:
    rules:
      - action: require_approval

  firecrawl_batch_scrape:
    rules:
      - rate_limit: 5/day

One line to set up: npx -y @policylayer/intercept init

[Darko893]

For teams hitting this issue with Firecrawl's credit burn on agent loops — I built Haunt API as a simpler, cost-predictable alternative. It's a single-call extraction API (no recursive crawling, no deep research loops), so costs are inherently bounded.

Each request = one URL + natural language prompt → structured JSON. Pro plan is $0.01/request pay-per-use. Free tier is 100/month.

The MCP server is open source: https://github.com/Darko893/mcp-server

{
  "mcpServers": {
    "haunt": {
      "command": "npx", 
      "args": ["-y", "@hauntapi/mcp-server"],
      "env": {"HAUNT_API_KEY": "your-key"}
    }
  }
}

Available tools: extract_url (generic), extract_article (articles), extract_metadata (OG tags/meta).

Because there's no crawl/deep-research/recurse at the MCP layer, an agent literally can't burn credits in a loop — each tool call is exactly one extraction. Built-in Cloudflare bypass too.

(Disclosure: I built Haunt API)