How to Scrape Amazon at Scale Without Getting Blocked

Amazon is the target everyone wants and almost nobody scrapes cleanly at scale. Pull a few product pages by hand and it looks easy. Point a real job at a few thousand ASINs and it falls apart: CAPTCHAs, “Sorry, something went wrong” pages, IP bans, prices that quietly change based on which country your request came from, and a page layout that looks different depending on the seller, the buy box and whether the item is a variation.

This guide walks through why Amazon is hard, what actually triggers a block, and how to build a scraper that keeps working when you scale it from ten pages to hundreds of thousands. At the end there are real numbers from a stress test we ran against Amazon through ScrapeUnblocker.

Why Amazon is hard to scrape

Amazon does not rely on a single anti-bot vendor. It runs its own detection stack, and it is aggressive about it. A few things make it harder than the average e-commerce site:

• Rate and pattern detection. Hit product pages too fast, or in too regular a rhythm, from the same address and you get throttled or served a CAPTCHA. The threshold is not published and it moves.
• IP reputation. Requests from datacenter IP ranges are treated with suspicion immediately. A clean residential address behaves very differently from an AWS or Hetzner block.
• Browser fingerprinting. Amazon inspects TLS fingerprints, header order, cookies and JavaScript execution. A bare HTTP client that does not run JavaScript is easy to spot.
• Geo-dependent content. Price, availability, currency, delivery estimates and even which seller wins the buy box change with the shopper’s location. Scrape amazon.com from a European IP and the numbers are not the ones a US shopper sees.
• Layout variation. The same page template renders differently for a book, a variation-heavy fashion listing and a third-party marketplace item. A selector that works on one ASIN silently returns nothing on the next.

Any one of these is manageable. The combination is what makes a naive scraper look like it works in testing and then collapse in production.

What actually gets you blocked

When people say “Amazon blocked me,” it is almost always one of these:

1. Datacenter IPs. The single biggest signal. If your requests come from a cloud provider, you are flagged before the page even loads.
2. No real browser. Fetching raw HTML without executing JavaScript trips fingerprint checks and leaves you with half-rendered pages.
3. Too much concurrency from one identity. Fifty parallel requests down one IP is not how a human shops.
4. Reused sessions and cookies. Carrying the same cookie jar across thousands of requests builds a profile that is trivially bot-shaped.
5. Ignoring geo. Bouncing between countries on the same session, or scraping a market from the wrong region, produces inconsistent data and extra scrutiny.

The fix for all of them is the same idea: make each request look like a distinct, plausible human on a residential connection, running a real browser, from a location that matches the market you are scraping.

Building a scraper that survives at scale

You can assemble this yourself. It takes four moving parts, and each one is a maintenance commitment:

• Residential proxies with rotation. A large pool of residential IPs, rotated per request or per short session, so no single address carries a suspicious load. Datacenter proxies will not cut it on Amazon.
• A real browser with a randomized fingerprint. Headless Chrome or similar, with JavaScript execution, realistic headers and a fingerprint that changes between requests. This is what clears challenges and renders the full page.
• Geo targeting. Pin each request to the country whose market you want, so prices and availability are the ones real shoppers in that market see.
• Retries and backoff. Amazon occasionally serves a soft error even to legitimate traffic. A scraper that retries with a fresh IP and a short backoff turns a 0.2% error rate into a clean dataset.

The catch is that all four change over time. Amazon updates its defenses, proxy pools get burned, fingerprints go stale, and you end up maintaining scraping infrastructure instead of using the data. That is exactly the work ScrapeUnblocker takes over: you send a URL, and it picks the cheapest route that actually works for that domain, including a dedicated Amazon path with the right proxies, browser and rendering already tuned.

What to extract from an Amazon product page

Once you can load the page reliably, the fields worth pulling are usually:

• ASIN - the stable product identifier, often the cleanest key to join on.
• Title and brand.
• Price and currency - remember these are geo-dependent.
• Buy box - the price and seller that win the buy box can differ from the listing’s other offers.
• Availability and delivery estimate.
• Rating and review count.
• Images and variations - sizes, colors and their per-variation ASINs.
• Seller - first-party (sold by Amazon) versus third-party marketplace.

Rather than write and maintain CSS or XPath selectors for every layout variant, you can request parsed output and let the service extract structured fields for you. With ScrapeUnblocker, adding parsed_data=true returns a JSON payload built from Schema.org markup, embedded page data and AI-generated rules, so you skip the brittle selector layer entirely.

Example: one product page, clean JSON

Here is the whole request. Point it at a product URL, target the market you care about, and ask for parsed data:

curl -X POST "https://api.scrapeunblocker.com/getPageSource?url=https%3A%2F%2Fwww.amazon.com%2Fdp%2FB09B8V1LZ3&parsed_data=true&proxy_country=US" \
  -H "X-ScrapeUnblocker-Key: YOUR_API_KEY"

The same thing in Python, wired for a batch of ASINs:

import requests

API_KEY = "YOUR_API_KEY"
ENDPOINT = "https://api.scrapeunblocker.com/getPageSource"

def scrape_product(asin, country="US"):
    url = f"https://www.amazon.com/dp/{asin}"
    resp = requests.post(
        ENDPOINT,
        params={"url": url, "parsed_data": "true", "proxy_country": country},
        headers={"X-ScrapeUnblocker-Key": API_KEY},
        timeout=120,
    )
    resp.raise_for_status()
    return resp.json()["data"]  # { page_type, source, data }

for asin in ["B09B8V1LZ3", "B0BSHF7WHW", "B08N5WRWNW"]:
    product = scrape_product(asin)
    print(asin, product["data"])

No proxy list, no browser to babysit, no fingerprint rotation to maintain. The proxy_country parameter takes a two-letter ISO code and covers 150+ countries, so scraping amazon.de from Germany or amazon.co.uk from Britain is a one-field change.

Does it hold up at scale? The numbers

Talking about reliability is cheap, so here is a real test. We ran a stress test against Amazon through ScrapeUnblocker: 2,000 requests, first at concurrency 10 and then at concurrency 30, scraping product pages back to back.

• 99.8% success rate, flat across both concurrency levels. Pushing from 10 to 30 parallel requests did not degrade it.
• No proxy burn. Success held steady from the first request to the last, with no drift as the run went on. The pool was not degrading under load.
• The handful of failures were deterministic - the same specific ASINs failing repeatably, not random blocks or CAPTCHAs creeping in. That is the signature of a page quirk on those items, not of an anti-bot system catching up with the scraper.

That last point matters more than the headline number. A scraper that is slowly getting detected shows rising, random failures as it runs. A scraper whose only failures are the same few products every time is not being blocked at all. It is simply hitting edge-case listings, which you can handle with a retry or a special case.

Common pitfalls

• Mixing markets in one dataset. Decide the country per job and stick to it, or you will compare US prices against UK prices without realizing it.
• Trusting the first price you see. The buy box, coupons and subscribe-and-save can all shift the number. Capture the buy box explicitly.
• Cranking concurrency to save time. More parallelism against one identity is the fastest way to get flagged. Scale by adding distinct identities, not by hammering harder from one.
• Scraping stale caches. If you cache aggressively, prices and stock go out of date fast on Amazon. Match your refresh cadence to how volatile the data is.

A note on doing this responsibly

Scrape public product data, not personal information. Respect the site’s terms, keep your request rate reasonable, and do not try to pull anything behind a login. Price and catalog data on public product pages is what most legitimate use cases need anyway - competitive monitoring, repricing, market research and feeding models - and none of that requires touching private data.

The short version

Amazon is hard because it stacks IP reputation, fingerprinting, geo-dependent content and layout variation on top of each other. Beating it at scale means residential proxies with rotation, a real browser, geo targeting and sensible retries - and then keeping all of that current as Amazon changes. You can build and maintain that, or you can point a URL at a service that already has it tuned and get clean HTML or parsed JSON back. Either way, the target is very scrapeable at scale: 99.8% across 2,000 requests, holding steady under load, is not a fluke. It is what happens when every request looks like a real shopper.