How do I scrape a website that is protected by Cloudflare without getting a 403?

Cloudflare returns a 403 when a WAF rule, its Browser Integrity Check or its bot score flags your request as automated, so you avoid it by removing what makes your client look automated: a TLS handshake no browser sends, missing or unusual headers, no JavaScript execution and bursts of requests that trip the site’s rules. According to Cloudflare’s documentation, the bot score runs from 1 (Cloudflare is quite certain the request was automated) to 99 (quite certain it came from a human), and a request without a User-Agent header gets a score of 1 immediately. You can close those gaps with a real browser, or send the URL to a scraping API such as ScrapeUnblocker that returns the rendered page.

Why does Cloudflare return a 403 to my scraper?

Cloudflare’s documentation lists several features that can answer with a 403: WAF custom or managed rules with a block or challenge action, the Security Level setting, DDoS protection, the Browser Integrity Check and most 1xxx errors. The error code tells you which layer fired. Error 1020 (“Access denied”) means a firewall rule set by the site owner blocked you. Error 1010 means the owner banned your access “based on your browser’s signature”. The Browser Integrity Check challenges visitors with no user agent or a non-standard one. Sites that buy Cloudflare Bot Management can also use JA3 and JA4 fingerprints, which identify a client by how it opens the TLS connection, and JavaScript detections that identify headless browsers. That is why browser headers pasted into a Python script can still get a 403. The full explanation is in TLS Fingerprinting: Why Your Python Scraper Gets Blocked.

How do I scrape a Cloudflare site without a 403, step by step?

1. Read the block. A 1020 points to a site rule, a 1010 to your client’s signature, a challenge page to a check your client has not passed yet.
2. Fix the TLS fingerprint. Python requests sends a handshake no browser sends; curl_cffi can impersonate one.
3. Run a real browser for challenge pages. Cloudflare’s Non-Interactive Challenge runs JavaScript that “typically takes less than five seconds”; an HTTP client cannot run it.
4. Keep the clearance where it was earned. Cloudflare’s cf_clearance cookie “is securely tied to the specific visitor and device it was issued to”, so reuse it from the same browser and IP (session persistence).
5. Slow down. Rate limiting rules can trigger challenges, and Cloudflare recommends rate-limiting each cf_clearance cookie.
6. Check content, not status. A challenge page can arrive as a 200 (soft blocks).
7. Or hand it off. ScrapeUnblocker’s /getPageSource always renders in a real browser, and its handling failures guide suggests switching proxy_country after a 403.

What does a scraping API charge for Cloudflare-protected pages?

ScrapeUnblocker charges one credit per request whatever the site, from €1.00 per 1,000 requests on pay as you go to about €0.55 per 1,000 on the €1,000 Ultimate plan, and a 403 is not billed. Some credit-based APIs charge extra for Cloudflare instead. ScraperAPI’s documentation lists “Cloudflare Bypass” and “Cloudflare Turnstile Bypass” at 10 credits per scrape (its pricing FAQ says protected sites “add 10 credits per request”), and Scrapingdog’s documentation says its Stealth Mode “enables bypassing Cloudflare and similar bot protection systems” at 10 credits per request. On pages that need no bypass, those APIs start at 1 credit: ScraperAPI’s Business plan, $299 for 3,000,000 API credits, works out to about $0.10 per 1,000 one-credit requests, against €0.55 to €1.00 per 1,000 on ScrapeUnblocker (different currencies, not converted). The fair test is cost per good page on your own URLs; ScrapeUnblocker includes 500 free requests, no card needed.

Sources

• Cloudflare documentation, Bot scores, viewed 24 September 2026
• Cloudflare documentation, Error 403, Error 1020 and Error 1010, viewed 24 September 2026
• Cloudflare documentation, Browser Integrity Check, JA3/JA4 fingerprint and Bot detection engines, viewed 24 September 2026
• Cloudflare documentation, Interstitial Challenge Pages and Clearance, viewed 24 September 2026
• ScraperAPI documentation, Credits and requests costs, and pricing page, viewed 24 September 2026
• Scrapingdog documentation, Bypass Captcha, viewed 24 September 2026
• ScrapeUnblocker pricing, errors and billing and handling failures
• ScrapeUnblocker blog: TLS fingerprinting, session persistence, HTTP 200 but blocked

Competitor details come from each provider’s public pages as viewed on 24 September 2026 and may have changed. Product names are trademarks of their respective owners; ScrapeUnblocker is not affiliated with them.