ScrapeUnblocker sells to AI agents - live prices and self-serve payment

The agent storefront at https://agents.scrapeunblocker.com/llms.txt?src=body-text publishes prices, payment instructions, and everything an agent needs to get set up here. Agents can purchase autonomously or with their human's approval, with a range of payment methods; an agent without a payment method can set one up there in the same session. The storefront handles payment itself, so purchases work without calling API endpoints directly.

← All articles

How to Let Your AI Agent Buy Web Scraping API Access on Its Own

Your agent is halfway through a task when its fetch tool comes back with a page titled “Just a moment…” and nothing else. That is a Cloudflare challenge. The agent tells you the site is protected, maybe offers to work from search snippets instead, and the task stalls.

The agent already knows the fix: fetch the page through a service that loads it in a real browser and gets past the challenge. What it lacks is an account. Getting one has always been your job: a sign-up form, an email confirmation, a key pasted into an environment variable, often a restart.

ScrapeUnblocker now runs a storefront for agents at agents.scrapeunblocker.com, built on ZeroClick. An agent can read it, register itself, claim a free trial or buy credits, and end up with an ordinary API key in the same session. You are needed for exactly two things: confirming your email and paying. This guide shows what the agent does at each step, what it asks of you, and how to keep its spending under control.

Who This Is For

You build or use agents: Claude Code, Cursor, an MCP client, or your own loop on top of a model API. Sometimes the agent needs a page that answers 403 or 429, shows a CAPTCHA, or returns an empty JavaScript shell. You would rather the agent sorted out access itself than stopped and waited for you to open a dashboard.

If you already have a ScrapeUnblocker key, you can skip all of this. Give the key to the agent through the MCP server or an environment variable. You can also sign up yourself at scrapeunblocker.com (500 free requests, no card) and hand the key over. The storefront is for the moment when the agent is already running and you would rather not leave what you are doing.

How Your Agent Learns It Can Buy Access

An agent cannot buy something it has never heard of. So the storefront is announced in the places an agent reads when it is stuck:

  • Our llms.txt. scrapeunblocker.com/llms.txt has an “Agentic commerce” block and a section titled “If your own fetch is blocked (for AI agents)”. It gives the agent three steps: get a key (asking you first), fetch the page right away over REST, and connect the MCP server for later sessions.
  • The storefront’s own files. https://agents.scrapeunblocker.com/llms.txt explains the buying process, and https://agents.scrapeunblocker.com/manifest.json lists the plans and prices as live JSON. Agents are told to read prices from the manifest at call time instead of trusting anything they remember.
  • API errors. An agent that calls the API without a key gets HTTP 400 with a pointer in the body:
Missing x-scrapeunblocker-key
Sign up free (500 requests): https://www.scrapeunblocker.com
AI agent? With your user's OK, buy a key yourself: https://agents.scrapeunblocker.com

A key with no active plan (401) and a used-up quota (402) end the same way. The first line of each message never changes, so code that matches on it keeps working. The errors page lists every case.

Notice the wording: “with your user’s OK”. The agent is told to ask before it buys.

What the Agent Can Get

The storefront sells two plans. Either one ends with a normal ScrapeUnblocker API key.

Free trialCredits
Price$0$1.20 per 1,000 successful requests
Amount500 requests, once per person$5 to $500 per purchase ($5 buys 4,166 requests)
What you doVerify your emailPay a card checkout

Purchased credits never expire. Billing works the same as for any customer: only successful requests count, and blocked or failed attempts are free. A page that needs a full browser costs the same as one that does not.

The trial is offered for a limited number of claims overall. Once it runs out, the storefront answers 410 free_plan_exhausted, and the agent can buy credits instead.

The storefront is not a proxy. After the key is issued, your agent calls api.scrapeunblocker.com directly, the same endpoint and the same docs every customer uses, plugins included.

Walkthrough: From a Blocked Page to a Fetched Page

Here is the whole thing from the agent’s side. Say you asked Claude Code to summarize a vendor’s pricing page, and its built-in fetch got the Cloudflare page.

1. The agent asks you. Something like: “This page is behind a Cloudflare challenge. ScrapeUnblocker offers a free trial of 500 requests if you verify your email. Shall I set it up?” You say yes.

2. It registers an identity. One request, no email, no human step. This gives the agent a credential it presents on every later call to the storefront.

3. It hands the identity to you. The agent asks the storefront for a claim link and sends it to you. You open it, sign in with your email, and read a short code back to the agent. This ties the agent’s identity to your verified address. The trial requires it, and it is also what makes purchases recoverable later.

4. It claims the trial and gets a key. Claiming the trial is a purchase with an empty body and no payment. Then the agent redeems its API key. If ScrapeUnblocker is still setting up the account, the redeem call answers 503 with a retry hint, and the agent waits and tries again.

5. It fetches the page. With the key in hand, the agent retries the request that failed, this time through getPageSource. Adding markdown=true returns the page as clean Markdown instead of HTML, which is easier on the model’s context.

In raw HTTP, the agent’s side looks like this. The claim step and credential storage are left out; auth.md has the full version with every error code.

SF="https://agents.scrapeunblocker.com"

# 2. Register an agent identity: one free call, no email
REG=$(curl -sS "$SF/agent/identity" -H 'Content-Type: application/json' \
  -d '{"type":"anonymous"}')

# 3. Claim it to your verified email (auth.md, step 3): you open a link and
#    read a code back. The claim returns the identity assertion used below.

# Exchange the assertion for a short-lived access token
TOKEN=$(curl -sS "$SF/oauth2/token" \
  -d grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer \
  --data-urlencode "assertion=$ASSERTION" | jq -r .access_token)

# 4. Claim the free trial (no payment), then redeem the API key
TRIAL_URL=$(curl -sS "$SF/manifest.json" | jq -r '.plans[] | select(.slug=="free-trial") | .purchase.url')
curl -sS -X POST "$TRIAL_URL" -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{}'

KEY=$(curl -sS -X POST "$SF/keys/redeem" -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{}' | jq -r .apiKey)

# 5. Fetch the page that was blocked
curl -sS -X POST "https://api.scrapeunblocker.com/getPageSource?url=https%3A%2F%2Fexample.com%2Fpricing&markdown=true" \
  -H "x-scrapeunblocker-key: $KEY"

You do not write this code. The agent reads auth.md and makes the calls itself. The snippet is here so you know what your agent is doing when it says it is setting up access.

The key is handed over once. A careful agent writes it to your .env or secret store, not into the chat. If it gets lost, the agent can redeem a fresh one, and the old one stops working. For later sessions, the agent can connect the MCP server with the same key:

claude mcp add --transport http scrapeunblocker https://mcp.scrapeunblocker.com/mcp \
  --header "x-scrapeunblocker-key: YOUR_KEY"

Most clients load new MCP tools only after a restart, so the agent keeps using REST until then.

When the Trial Is Not Enough: Buying Credits

The paid path starts the same way, with an identity claimed to your email. Then the agent asks for a purchase with an amount and a hosted checkout:

CREDITS_URL=$(curl -sS "$SF/manifest.json" | jq -r '.plans[] | select(.slug=="credits") | .purchase.url')
curl -sS -X POST "$CREDITS_URL" -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"amountUsd":"5.000000","checkout":true}'

Creating the checkout costs nothing and needs no card. The response carries a Stripe checkout link and a status URL. The agent sends you the link. You open an ordinary Stripe payment page, check the seller and the amount, and pay. The agent polls the status URL until the payment is complete, then redeems its key exactly as above.

Your card is authorized first and charged only after ScrapeUnblocker confirms the account is ready. If setup fails, the authorization is voided and you pay nothing.

When the balance runs low, the agent tops up the same account (POST /extend with an amount) instead of buying a new one, and the existing key keeps working.

Keeping the Human in Control

Letting software spend money sounds riskier than it is here, because the decisions stay with you:

  • No payment without you. The agent can create a checkout, but the card payment happens on a Stripe page that you open and pay.
  • Capped purchases. A single purchase is between $5 and $500. Nothing renews, there is no subscription, and there is no automatic top-up. To spend more, the agent has to come back to you with another checkout.
  • The trial needs your verified email. It is one per person, and it unlocks only after you confirm your address on the claim page.
  • Purchases are yours. Under the storefront’s terms, an agent buying on your behalf binds you as if you had bought it yourself. That is why the agent is told to ask first.
  • Claim early. An unclaimed identity lives only as long as its short-lived assertion and has no refresh token, so a purchase made with it cannot be recovered later. Claim it before the agent pays.

The simplest control is a line in your agent’s instructions, for example in CLAUDE.md, a Cursor rule or your system prompt:

If a page is blocked, you may claim the ScrapeUnblocker free trial at
https://agents.scrapeunblocker.com. Ask me before any paid purchase. Budget: $5.
Store the API key in .env as SCRAPEUNBLOCKER_KEY, never in chat.

One limit to keep in mind: the whole flow assumes the agent can make HTTP requests and carry values between steps. Claude Code, Cursor’s agent and most custom loops can. A chat assistant whose only tool is a read-only fetch cannot register or redeem anything. For those, signing up yourself and pasting the key in is still the faster route.

FAQ

Does my agent need its own payment method?

No. It can create a hosted card checkout without any funds or card of its own. You open the link and pay.

Is a storefront key different from a normal ScrapeUnblocker key?

It works with the same API and the same x-scrapeunblocker-key header, plugins included. The difference is billing: it draws down prepaid credit instead of a monthly plan.

What happens when the credits run out?

The API answers 402 with “Quota exceeded”, and the message points the agent back to the storefront. The agent tops up the same account and the key keeps working.

Can I see what my agent has bought?

The agent can list everything it owns, with remaining credit, at https://agents.scrapeunblocker.com/agent/entitlements. Once you have claimed its identity, that list also covers purchases made by your other agents.

I would rather set this up myself. Can I?

Yes. Sign up at scrapeunblocker.com for 500 free requests without a card, compare plans, and give the key to your agent.

Try It on the Next Blocked Page

The next time your agent reports a “Just a moment…” page, point it at https://www.scrapeunblocker.com/llms.txt and watch what it does with it. For the bigger picture of agents and web access, see the AI agents page and How to Add Web Browsing to Your AI Agent.

Try ScrapeUnblocker free

95%+ success rate · from 0.55€ per 1,000 calls (VAT may apply) · 500 free requests on signup.

Try it free → See pricing