Firecrawl

Integrates with the Firecrawl API to enable web scraping and intelligent content searching for extracting structured data and performing customizable searches.

Installation Instructions

README: https://github.com/Msparihar/mcp-server-firecrawl

Firecrawl MCP Server

A Model Context Protocol (MCP) server for web scraping, content searching, site crawling, and data extraction using the Firecrawl API.

Features

• Web Scraping: Extract content from any webpage with customizable options

  • Mobile device emulation
  • Ad and popup blocking
  • Content filtering
  • Structured data extraction
  • Multiple output formats

• Content Search: Intelligent search capabilities

  • Multi-language support
  • Location-based results
  • Customizable result limits
  • Structured output formats

• Site Crawling: Advanced web crawling functionality

  • Depth control
  • Path filtering
  • Rate limiting
  • Progress tracking
  • Sitemap integration

• Site Mapping: Generate site structure maps

  • Subdomain support
  • Search filtering
  • Link analysis
  • Visual hierarchy

• Data Extraction: Extract structured data from multiple URLs

  • Schema validation
  • Batch processing
  • Web search enrichment
  • Custom extraction prompts

Installation

# Global installation
npm install -g @modelcontextprotocol/mcp-server-firecrawl

# Local project installation
npm install @modelcontextprotocol/mcp-server-firecrawl

Quick Start

1. Get your Firecrawl API key from the developer portal

2. Set your API key:

   Unix/Linux/macOS (bash/zsh):

   export FIRECRAWL_API_KEY=your-api-key
   

   Windows (Command Prompt):

   set FIRECRAWL_API_KEY=your-api-key
   

   Windows (PowerShell):

   $env:FIRECRAWL_API_KEY = "your-api-key"
   

   Alternative: Using .env file (recommended for development):

   # Install dotenv
   npm install dotenv
   
   # Create .env file
   echo "FIRECRAWL_API_KEY=your-api-key" > .env
   

   Then in your code:

   import dotenv from 'dotenv';
   dotenv.config();
   

3. Run the server:

   mcp-server-firecrawl
   

Integration

Claude Desktop App

Add to your MCP settings:

{
  "firecrawl": {
    "command": "mcp-server-firecrawl",
    "env": {
      "FIRECRAWL_API_KEY": "your-api-key"
    }
  }
}

Claude VSCode Extension

Add to your MCP configuration:

{
  "mcpServers": {
    "firecrawl": {
      "command": "mcp-server-firecrawl",
      "env": {
        "FIRECRAWL_API_KEY": "your-api-key"
      }
    }
  }
}

Usage Examples

Web Scraping

// Basic scraping
{
  name: "scrape_url",
  arguments: {
    url: "https://example.com",
    formats: ["markdown"],
    onlyMainContent: true
  }
}

// Advanced extraction
{
  name: "scrape_url",
  arguments: {
    url: "https://example.com/blog",
    jsonOptions: {
      prompt: "Extract article content",
      schema: {
        title: "string",
        content: "string"
      }
    },
    mobile: true,
    blockAds: true
  }
}

Site Crawling

// Basic crawling
{
  name: "crawl",
  arguments: {
    url: "https://example.com",
    maxDepth: 2,
    limit: 100
  }
}

// Advanced crawling
{
  name: "crawl",
  arguments: {
    url: "https://example.com",
    maxDepth: 3,
    includePaths: ["/blog", "/products"],
    excludePaths: ["/admin"],
    ignoreQueryParameters: true
  }
}

Site Mapping

// Generate site map
{
  name: "map",
  arguments: {
    url: "https://example.com",
    includeSubdomains: true,
    limit: 1000
  }
}

Data Extraction

// Extract structured data
{
  name: "extract",
  arguments: {
    urls: ["https://example.com/product1", "https://example.com/product2"],
    prompt: "Extract product details",
    schema: {
      name: "string",
      price: "number",
      description: "string"
    }
  }
}

Configuration

See configuration guide for detailed setup options.

API Documentation

See API documentation for detailed endpoint specifications.

Development

# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

# Start in development mode
npm run dev

Examples

Check the examples directory for more usage examples:

• Basic scraping: scrape.ts
• Crawling and mapping: crawl-and-map.ts

Error Handling

The server implements robust error handling:

• Rate limiting with exponential backoff
• Automatic retries
• Detailed error messages
• Debug logging

Security

• API key protection
• Request validation
• Domain allowlisting
• Rate limiting
• Safe error messages

Contributing

See CONTRIBUTING.md for contribution guidelines.

License

MIT License - see LICENSE for details.