Web access for AI agents, including the sites that block bots
Agents get stuck on the same pages scrapers do: bot checks, pages that only load in a browser, and search results that are more than ten links. ScrapeUnblocker gives your agent rendered pages, real Google results and ready-made site data, through MCP, an SDK or your workflow tool, on your own API key.
500 free requests on signup · no card needed
Where agents get stuck
Much of the web is hard for an agent to read
A challenge page instead of content
Cloudflare, DataDome, PerimeterX and Akamai answer automated requests with a 403, a captcha or a challenge loop. The agent reads the block page and moves on with nothing.
Pages that are empty without a browser
Prices, listings and reviews are often loaded by JavaScript. A plain HTTP fetch returns the page shell without them.
Search results without the full picture
What a person sees on Google includes ads and an AI Overview, and it changes by country. A short list of links leaves most of that out.
What your agent gets
Built for the pages agents fail on
Protected sites
Requests go through anti-bot handling for Cloudflare, DataDome, PerimeterX (HUMAN) and Akamai, with JavaScript rendering included in every request. Our published benchmark measured 97.0% content-verified success across 14 protected targets. It is our own measurement, not a head-to-head test.
See the benchmark →Real search results
serpApi returns Google organic results plus the top and bottom ads and the AI Overview as JSON, searched from the country you choose. Bing, DuckDuckGo, Yahoo and Brave have endpoints with the same response shape.
Site plugins as tools
Ready-made structured JSON for Amazon, Walmart, eBay, Google Maps, Google Shopping, YouTube transcripts, Zillow, Indeed, Trustpilot, Booking.com and 100+ more. The agent gets fields, not HTML to parse.
Browse the scrapers →Live pages
Every call fetches the page live. Nothing is served from a cache, and proxy_country lets the agent see the page from a specific country.
You pay for delivered pages
Blocks (403), timeouts, an unreachable origin (523), concurrency limits (429) and errors on our side are not billed. A delivered page is one request, JavaScript rendering included, and that counts a site's own 404 or error page too.
A short tool list
The MCP server exposes a few core tools, the most-used site plugins as tools of their own and one run_plugin tool for everything else. fetch_markdown reads a page as Markdown, usually 2-25% the size of the HTML, and search_and_read searches Google and reads the top results in one call. The list stays short, so it does not crowd the agent's context.
Connect
Use it where your agent already runs
One account and one key everywhere. Replace YOUR_API_KEY with the key from your dashboard.
Hosted MCP server
For Claude on the web, desktop and mobile, Claude Code, Cursor, VS Code and other MCP clients. Sign in with your ScrapeUnblocker account (OAuth) or put your key in the URL. In ChatGPT, add it under Settings → Plugins → Developer mode, with the key in the URL.
OAuth: https://mcp.scrapeunblocker.com/mcp Key: https://mcp.scrapeunblocker.com/mcp?key=YOUR_API_KEY
Claude Code plugin
Installs the MCP server, asks for your key once and keeps it in your OS keychain. Adds a /scrape-url command and reference skills.
/plugin marketplace add ScrapeUnblocker/claude-code-plugin /plugin install scrapeunblocker@scrapeunblocker
Local MCP server
Runs on your machine with Node.js 18+, so the key stays local. This config works in Cursor and Claude Desktop.
{
"mcpServers": {
"scrapeunblocker": {
"command": "npx",
"args": ["-y", "scrapeunblocker-mcp"],
"env": { "SCRAPEUNBLOCKER_KEY": "YOUR_API_KEY" }
}
}
} SDKs
For agents and pipelines you write yourself: Python, Node.js, Ruby and PHP clients with a method for every endpoint and site plugin.
pip install scrapeunblocker npm install scrapeunblocker gem install scrapeunblocker composer require scrapeunblocker/client
n8n
The ScrapeUnblocker node for n8n workflows. Search for ScrapeUnblocker in the nodes panel, or install the package under Settings → Community nodes.
n8n-nodes-scrapeunblocker
Haystack
A fetcher component for Haystack pipelines, for RAG and agents that read live pages.
pip install scrapeunblocker-haystack
Setting it up in Claude? The MCP server page walks through claude.ai, Claude Code and Claude Desktop step by step.
In practice
Tasks agents run with it
FAQ
Common questions
Which agents can use it?
Anything that speaks the Model Context Protocol: Claude on the web, desktop and mobile, Claude Code, ChatGPT in developer mode, Cursor, VS Code and other MCP clients. Agents you build yourself call the API through the Python, Node.js, Ruby or PHP SDK, and workflows through the n8n node or the Haystack component.
Do I need an API key?
On the hosted MCP server you can sign in with your ScrapeUnblocker account instead of pasting a key. Everywhere else you use your own key from the dashboard. Every account starts with 500 free requests, no card needed.
What does an agent task cost?
One request per page fetched or per search results page. Site plugins that list results bill one request per result page they load. search_and_read bills its search plus one request per page it reads. Blocks, timeouts, an unreachable origin, concurrency limits and errors on our side are not billed.
Can my agent click, type or scroll?
Yes. browse_page runs browser steps (click, type, select, press a key, scroll, wait) that run in a real browser before the page is captured, and list_elements shows the selectors to use. Sign-in and payment pages are not fetched, and steps do not fill password or card fields.
When should my agent use this instead of its built-in fetch?
When the built-in fetch comes back blocked, empty or from the wrong country, when you need Google results with their ads and AI Overview, or when a site plugin can return the data as structured fields.
Give your agent the pages it keeps failing on
Create a free account, connect it in about a minute, and pay only for pages that arrive.