ScraperAPI MCP server

The ScraperAPI MCP server enables LLM clients to retrieve and process web scraping requests using the ScraperAPI services.

This is the self-hosted (local) server. A hosted (remote) version is also available.

Table of Contents

• Features
• Architecture
• Installation
• API Reference
• Configuration
  • Settings
  • Client Setup
• Development

Features

• Full implementation of the Model Context Protocol specification
• Seamless integration with ScraperAPI for web scraping
• Simple setup with Python or Docker

Architecture

          ┌───────────────┐     ┌───────────────────────┐     ┌───────────────┐
          │  LLM Client   │────▶│  Scraper MCP Server   │────▶│    AI Model   │
          └───────────────┘     └───────────────────────┘     └───────────────┘
                                            │
                                            ▼
                                  ┌──────────────────┐
                                  │  ScraperAPI API  │
                                  └──────────────────┘

Installation

The ScraperAPI MCP Server is designed to run as a local server on your machine, your LLM client will launch it automatically when configured.

Prerequisites

• Python 3.11+
• Docker (optional)

Using Python

Install the package:

pip install scraperapi-mcp-server

Add this to your client configuration file:

{
  "mcpServers": {
    "ScraperAPI": {
      "command": "python",
      "args": ["-m", "scraperapi_mcp_server"],
      "env": {
        "API_KEY": "<YOUR_SCRAPERAPI_API_KEY>"
      }
    }
  }
}

Using Docker

Add this to your client configuration file:

{
  "mcpServers": {
    "ScraperAPI": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "-e",
        "API_KEY=${API_KEY}",
        "--rm",
        "scraperapi-mcp-server"]
    }
  }
}

[!TIP]

If your command is not working (for example, you see a package not found error when trying to start the server), double-check the path you are using. To find the correct path, activate your virtual environment first, then run:

which <YOUR_COMMAND>

Available tools

API

Structured Data Endpoints (SDEs) return pre-parsed JSON (or CSV) instead of raw HTML. Every SDE tool also accepts output_format (json by default, or csv), tld, and country_code, shown in each tool's table.

scrape — Scrape any web page or image, bypassing anti-bot protection

Retrieve the content of a web page, or download an image, from a URL.

Parameter	Type	Description	Required
url	string	Target URL to scrape	Yes
render	boolean	Enable JavaScript rendering for dynamic pages (default: false)	No
country_code	string	ISO 2-letter country code for geo-targeting	No
premium	boolean	Use premium residential/mobile proxies (default: false)	No
ultra_premium	boolean	Advanced anti-bot bypass; incompatible with premium (default: false)	No
device_type	string	mobile or desktop user agent	No
output_format	string	markdown (default), text, csv, or json	No
autoparse	boolean	Auto-parse supported sites into structured data (default: false)	No

Returns: the scraped content as a string, or image data for image URLs.

SDEs

Google

google_search — Parsed Google Search (SERP) results

Parameter	Type	Description	Required
query	string	Search query, as typed into Google	Yes
num	integer	Number of results to return on the page	No
start	integer	Zero-based result offset for pagination	No
hl	string	Interface/host language code (e.g. en, es)	No
gl	string	Country edition to search (2-letter code)	No
uule	string	Google uule geolocation string (advanced)	No
date_range_start	string	Start of a custom date range, MM/DD/YYYY	No
date_range_end	string	End of a custom date range, MM/DD/YYYY	No
time_period	string	Recent window: 1H, 1D, 1W, 1M, or 1Y	No
include_html	boolean	Include raw HTML alongside parsed data (default: false)	No
tbs	string	Raw Google tbs filter parameter (advanced)	No
output_format	string	json (default) or csv	No
tld	string	Top-level domain (e.g. com, co.uk)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

google_news — Parsed Google News results

Parameter	Type	Description	Required
query	string	Search query	Yes
num	integer	Number of results to return on the page	No
start	integer	Zero-based result offset for pagination	No
hl	string	Interface/host language code	No
gl	string	Country edition to search (2-letter code)	No
uule	string	Google uule geolocation string (advanced)	No
date_range_start	string	Start of a custom date range, MM/DD/YYYY	No
date_range_end	string	End of a custom date range, MM/DD/YYYY	No
time_period	string	Recent window: 1H, 1D, 1W, 1M, or 1Y	No
output_format	string	json (default) or csv	No
tld	string	Top-level domain (e.g. com, co.uk)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

google_jobs — Parsed Google Jobs listings

Parameter	Type	Description	Required
query	string	Search query	Yes
num	integer	Number of results to return on the page	No
start	integer	Zero-based result offset for pagination	No
hl	string	Interface/host language code	No
gl	string	Country edition to search (2-letter code)	No
uule	string	Google uule geolocation string (advanced)	No
output_format	string	json (default) or csv	No
tld	string	Top-level domain (e.g. com, co.uk)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

google_shopping — Parsed Google Shopping product results

Parameter	Type	Description	Required
query	string	Search query	Yes
num	integer	Number of results to return on the page	No
start	integer	Zero-based result offset for pagination	No
hl	string	Interface/host language code	No
gl	string	Country edition to search (2-letter code)	No
uule	string	Google uule geolocation string (advanced)	No
include_html	boolean	Include raw HTML alongside parsed data (default: false)	No
output_format	string	json (default) or csv	No
tld	string	Top-level domain (e.g. com, co.uk)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

google_maps_search — Parsed Google Maps place/business results

Parameter	Type	Description	Required
query	string	Search query (e.g. coffee shops in Austin)	Yes
latitude	number	Latitude of the map center, in decimal degrees	Yes
longitude	number	Longitude of the map center, in decimal degrees	Yes
zoom	integer	Map zoom level (roughly 3=country, 10=city, 15=street)	No
include_html	boolean	Include raw HTML alongside parsed data (default: false)	No
output_format	string	json (default) or csv	No
tld	string	Top-level domain (e.g. com, co.uk)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

Amazon

amazon_product — Parsed Amazon product details by ASIN

Parameter	Type	Description	Required
asin	string	10-character Amazon product identifier (e.g. B08N5WRWNW)	Yes
language	string	Language code for localized content (e.g. en_US)	No
include_html	boolean	Include raw HTML alongside parsed data (default: false)	No
output_format	string	json (default) or csv	No
tld	string	Amazon TLD (e.g. com, co.uk, de)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

amazon_search — Parsed Amazon product search results

Parameter	Type	Description	Required
query	string	Search query, as typed into Amazon	Yes
page	integer	1-based results page number	No
sort_by	string	Sort order (e.g. price-asc-rank, review-rank)	No
department	string	Restrict search to a department/category (e.g. electronics)	No
ref	string	Amazon ref referral/context token (advanced)	No
language	string	Language code for localized content	No
output_format	string	json (default) or csv	No
tld	string	Amazon TLD (e.g. com, co.uk, de)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

amazon_offers — Parsed seller offers for an Amazon product

Parameter	Type	Description	Required
asin	string	10-character Amazon product identifier	Yes
condition	string	Comma-separated condition filters (e.g. f_new,f_usedlikenew,f_usedverygood,f_usedgood,f_usedacceptable)	No
f_new	boolean	Include only New-condition offers	No
f_used_like_new	boolean	Include Used - Like New offers	No
f_used_very_good	boolean	Include Used - Very Good offers	No
f_used_good	boolean	Include Used - Good offers	No
f_used_acceptable	boolean	Include Used - Acceptable offers	No
language	string	Language code for localized content	No
output_format	string	json (default) or csv	No
tld	string	Amazon TLD (e.g. com, co.uk, de)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

Walmart

walmart_search — Parsed Walmart product search results

Parameter	Type	Description	Required
query	string	Product search query, as typed into Walmart	Yes
page	integer	1-based results page number	No
output_format	string	json (default) or csv	No
tld	string	Walmart TLD (e.g. com, ca, com.mx)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

walmart_product — Parsed Walmart product details by product ID

Parameter	Type	Description	Required
product_id	string	Walmart product ID	Yes
output_format	string	json (default) or csv	No
tld	string	Walmart TLD (e.g. com, ca, com.mx)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

walmart_category — Parsed products within a Walmart category

Parameter	Type	Description	Required
category	string	Walmart category ID	Yes
page	integer	1-based results page number	No
output_format	string	json (default) or csv	No
tld	string	Walmart TLD (e.g. com, ca, com.mx)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

walmart_review — Parsed customer reviews for a Walmart product

Parameter	Type	Description	Required
product_id	string	Walmart product ID	Yes
page	integer	1-based results page number	No
sort	string	Sort order (e.g. helpful, recent)	No
ratings	string	Comma-separated star ratings to filter by (e.g. 4,5)	No
verified_purchase	boolean	Include only reviews with a "Verified Purchase" badge (default: false)	No
output_format	string	json (default) or csv	No
tld	string	Walmart TLD (e.g. com, ca, com.mx)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

eBay

ebay_search — Parsed eBay product search results

Parameter	Type	Description	Required
query	string	Search keywords	Yes
page	integer	1-based results page number	No
items_per_page	integer	Number of items per page	No
seller_id	string	Restrict results to a specific seller	No
condition	string	Comma-separated: new, used, open_box, refurbished, for_parts, not_working	No
buying_format	string	buy_it_now, auction, or accepts_offers	No
show_only	string	Comma-separated: returns_accepted, authorized_seller, completed_items, sold_items, sale_items, listed_as_lots, search_in_description, benefits_charity, authenticity_guarantee	No
sort_by	string	best_match, ending_soonest, newly_listed, price_lowest, price_highest, or distance_nearest	No
output_format	string	json (default) or csv	No
tld	string	eBay TLD (e.g. com, co.uk, de)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

ebay_product — Parsed eBay listing details by item ID

Parameter	Type	Description	Required
product_id	string	eBay item ID	Yes
output_format	string	json (default) or csv	No
tld	string	eBay TLD (e.g. com, co.uk, de)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

Redfin

Every Redfin tool takes a full Redfin URL matching the tool (property, search, or agent page).

redfin_for_sale — Parsed Redfin listing data for a home for sale

Parameter	Type	Description	Required
url	string	Full Redfin for-sale property URL	Yes
raw	boolean	Return raw extracted JSON instead of parsed data (default: false)	No
output_format	string	json (default) or csv	No
tld	string	Redfin TLD (com, ca)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

redfin_for_rent — Parsed Redfin listing data for a rental property

Parameter	Type	Description	Required
url	string	Full Redfin rental property URL	Yes
raw	boolean	Return raw extracted JSON instead of parsed data (default: false)	No
output_format	string	json (default) or csv	No
tld	string	Redfin TLD (com, ca)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

redfin_search — Parsed Redfin search results

Parameter	Type	Description	Required
url	string	Full Redfin search-results URL (with filters)	Yes
output_format	string	json (default) or csv	No
tld	string	Redfin TLD (com, ca)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

redfin_agent — Parsed Redfin real-estate agent profile data

Parameter	Type	Description	Required
url	string	Full Redfin agent profile URL	Yes
output_format	string	json (default) or csv	No
tld	string	Redfin TLD (com, ca)	No
country_code	string	ISO 2-letter country code for geo-targeting	No

Crawler

The crawler is asynchronous: crawler_job_start returns a job id immediately, then you poll crawler_job_status until the crawl finishes. Per-page results can also be pushed to a callback_url webhook.

crawler_job_start — Start an async crawl job from a starting URL

Parameter	Type	Description	Required
start_url	string	The URL where crawling begins (depth 0)	Yes
url_regexp_include	string	Regex selecting which links to follow. Use .* to crawl all links. Advanced: named groups (?<full_url>...) / (?<relative_url>...) target absolute vs relative URLs	Yes
max_depth	integer	Maximum crawl depth (start URL is depth 0). Provide max_depth or crawl_budget	No*
crawl_budget	integer	Maximum ScraperAPI credits the crawl may consume. Provide max_depth or crawl_budget	No*
url_regexp_exclude	string	Regex for URLs to exclude from crawling	No
api_params	object	Per-scrape controls applied to each page (e.g. render, country_code, premium, device_type, output_format)	No
callback_url	string	Webhook URL to receive per-page results and the final summary	No
additional_data	object	Arbitrary metadata to attach to the job	No
schedule	object	Recurring schedule: { name, interval (once/hourly/daily/weekly/monthly), cron }	No
enabled	boolean	For scheduled projects, whether the schedule is enabled (default: true)	No

* Provide either max_depth or crawl_budget.

Returns: JSON with the job id and initial status (e.g. {"status": "initiated", "jobId": "..."}).

crawler_job_status — Get the status of a crawl job

Parameter	Type	Description	Required
job_id	string	The job id returned by crawler_job_start	Yes

Returns: JSON with the job's page counts (done/failed/active).

crawler_job_delete — Cancel and delete a crawl job

Parameter	Type	Description	Required
job_id	string	The job id returned by crawler_job_start	Yes

AI Parser

Build a reusable parser from a few example URLs, then apply it to any similar page for structured extraction. Two-phase: ai_parser_create returns a parser id immediately and generation runs in the background — poll ai_parser_get_details until its status is FINISHED, then call ai_parser_parse_url.

ai_parser_create — Create a reusable AI parser from example URLs

Parameter	Type	Description	Required
name	string	A name for the parser	Yes
urls	array<string>	1–3 example URLs of the same page type (same structure)	Yes
scraper_params	object	ScraperAPI fetch options for the example pages: render, country_code, premium, session_number, keep_headers, device_type, ultra_premium, follow_redirect, retry_404	No
fields	array	Pre-declared fields to extract: [{ name, description, type?, selector? }] (type is string/number/array). If omitted, fields are inferred	No

Returns: JSON with the new parser's id and version. Generation is asynchronous — poll ai_parser_get_details until status is FINISHED.

ai_parser_get_details — Get a parser's details and generation status

Parameter	Type	Description	Required
parser_id	string	The parser id returned by ai_parser_create	Yes
version	integer	Specific parser version (default: latest)	No

Returns: JSON with the parser's status (GENERATING/FINISHED/FAILED), fields, and example results.

ai_parser_parse_url — Parse a URL with an existing parser

Parameter	Type	Description	Required
parser_id	string	The parser id to run	Yes
url	string	The URL to scrape and parse	Yes
version	integer	Specific parser version (default: latest)	No

Returns: JSON {"parser": ..., "version": ..., "result": {...}}. Costs 1 credit per call.

ai_parser_list — List the parsers on your account

No parameters. Returns a JSON array of parser summaries (id, name, status, version, generation time).

ai_parser_delete — Delete a parser

Parameter	Type	Description	Required
parser_id	string	The parser id to delete	Yes

ai_parser_update — Edit a parser's fields (creates a new version)

Parameter	Type	Description	Required
parser_id	string	The parser id to update	Yes
version	integer	Version to base the update on (default: latest)	No
add_fields	array	Fields to add: [{ name, description, type?, selector? }] (triggers async regeneration)	No
modify_fields	array	Fields to modify (triggers async regeneration)	No
rename_fields	array	Renames: [{ name, new_name }] (applied immediately)	No
remove_fields	array<string>	Field names to remove (applied immediately)	No

Prompt templates

• Please scrape this URL <URL>. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures.
• Can you scrape URL <URL> to extract <SPECIFIC_DATA>? If the request returns missing/incomplete<SPECIFIC_DATA>, set render=true to enable JS Rendering.

Configuration

Settings

Configure the server through environment variables. Only API_KEY is required.

Variable	Default	Description
API_KEY	—	Required. Your ScraperAPI API key.
API_URL	https://api.scraperapi.com	Base URL for the ScraperAPI API and structured-data endpoints.
SCRAPER_SDK	mcp-server	Client identifier sent to ScraperAPI on every request.
API_TIMEOUT_SECONDS	70	Per-request timeout, in seconds.
RATE_LIMIT_MAX_CALLS	10	Maximum tool calls allowed per rate-limit window.
RATE_LIMIT_WINDOW_SECONDS	60	Length of the rate-limit window, in seconds.
IMAGE_SIZE_LIMIT_BYTES	700000	Maximum size of an image the scrape tool returns inline.

Client Setup

Use the JSON configuration file from the Installation section. Below are steps for common clients.

Claude Desktop & Claude Code

Claude Desktop:

1. Open Claude Desktop and click the settings icon
2. Select the "Developer" tab
3. Click "Edit Config" and paste the JSON configuration file

Claude Code:

1. Add the server manually to your .claude/settings.json with the JSON configuration file, or run:

   claude mcp add scraperapi -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> -- python -m scraperapi_mcp_server
   

Cursor Editor

1. Open Cursor
2. Access the Settings Menu
3. Open Cursor Settings
4. Go to Tools & Integrations section
5. Click '+ Add MCP Server'
6. Choose Manual and paste the JSON configuration file

More here

Windsurf Editor

1. Open Windsurf
2. Access the Settings Menu
3. Click on the Cascade settings
4. Click on the MCP server section
5. Click on the gear icon, the mcp_config.json file will open
6. Paste the JSON configuration file

More here

Cline (VS Code extension)

1. Open VS Code and click the Cline icon in the activity bar to open the Cline panel
2. Click the MCP Servers icon in the top navigation bar of the Cline pane
3. Select the "Configure" tab
4. Click "Configure MCP Servers" at the bottom of the pane — this opens cline_mcp_settings.json
5. Paste the JSON configuration file

More here

Development

Local setup

1. Clone the repository:

   git clone https://github.com/scraperapi/scraperapi-mcp
   cd scraperapi-mcp
   

2. Install dependencies:

   • Using Poetry:

     poetry install
     

   • Using pip:

     # Create virtual environment and activate it
     python -m venv .venv
     source .venv/bin/activate # MacOS/Linux
     # OR
     .venv/Scripts/activate # Windows
     
     # Install the local package in editable mode
     pip install -e .
     

   • Using Docker:

     # Build the Docker image locally
     docker build -t scraperapi-mcp-server .
     

Run the server

• Using Python:

  python -m scraperapi_mcp_server
  

• Using Docker:

  # Run the Docker container with your API key
  docker run -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> scraperapi-mcp-server
  

Debug

python3 -m scraperapi_mcp_server --debug

Testing

This project uses pytest for testing.

Install Test Dependencies

• Using Poetry:

  poetry install --with dev
  

• Using pip:

  pip install -e .
  pip install pytest pytest-mock pytest-asyncio
  

Running Tests

# Run All Tests
pytest

# Run Specific Test
pytest <TEST_FILE_PATH>