How do I scrape a website that is protected by Cloudflare without getting a 403?

To scrape a Cloudflare site without a 403, remove what marks your client as automated (a non-browser TLS fingerprint, missing headers, no JavaScript execution and bursty traffic), or send the URL to an unblocking API such as ScrapeUnblocker, Bright Data Web Unlocker, Zenrows, ScrapingBee or ScraperAPI. On 9 October 2026, a plain curl request to glassdoor.com’s reviews page got HTTP 403 with Cloudflare’s cf-mitigated: challenge header. ScrapeUnblocker’s scraper pages publish 100% success at 6.6 seconds over 1,000 Glassdoor page fetches, and 99.6% at 3.8 seconds over 1,100 calls on its Glassdoor Reviews plugin, which the page lists as Cloudflare-protected. These are ScrapeUnblocker’s own measurements, not a head-to-head test.

Why does Cloudflare return a 403 to my scraper?

Cloudflare’s documentation lists several features that can answer with a 403: WAF custom or managed rules with a block or challenge action, the Security Level setting, DDoS protection, the Browser Integrity Check and most 1xxx errors. Error 1020 means a firewall rule set by the site owner blocked you; error 1010 means the owner banned your access based on your browser’s signature. Cloudflare’s bot score runs from 1 (quite certain the request was automated) to 99, and a request without a User-Agent header gets 1 immediately. Sites on Bot Management can also use JA3 and JA4 TLS fingerprints and JavaScript detections that identify headless browsers, which is why browser headers pasted into a Python script can still get a 403.

How do I get past it, step by step?

1. Read the block. Cloudflare marks challenge pages with cf-mitigated: challenge; a 1020 means a site rule, a 1010 your client’s signature.

2. Fix the TLS fingerprint. The open-source curl_cffi library can impersonate browsers’ TLS and HTTP/2 fingerprints; Python requests cannot (details).

3. Run a real browser for challenges. Cloudflare says the challenge JavaScript “typically takes less than five seconds”; HTTP clients cannot run it.

4. Keep the clearance, and slow down. cf_clearance “is securely tied to the specific visitor and device it was issued to”, and rate limiting rules can trigger challenges.

5. Check content, not status. A challenge page can arrive as a 200 (soft blocks).

6. Or hand it off. ScrapeUnblocker’s /getPageSource always renders in a real browser:

   curl -X POST "https://api.scrapeunblocker.com/getPageSource?url=https%3A%2F%2Fwww.glassdoor.com%2FReviews%2Findex.htm" \
     -H "x-scrapeunblocker-key: YOUR_API_KEY"

   A 403 from it is not billed; retry with another proxy_country, as the handling failures guide suggests.

What do scraping APIs charge for Cloudflare-protected pages?

Credit-based APIs charge 10 to 75 credits for a protected page, while ScrapeUnblocker and Bright Data charge a flat price per request. Figures come from each provider’s own pages:

Provider	Published cost for a protected page	Entry plan
ScrapeUnblocker	1 request; a 403 is not billed	€1.00 per 1,000 pay as you go
Bright Data Web Unlocker	$1.50 per 1,000 requests, pay only for success	Pay as you go
ScraperAPI	10 credits for “Cloudflare Bypass” or “Cloudflare Turnstile Bypass”	Hobby $49, 100,000 credits
Scrapingdog	10 credits with Stealth Mode	Lite $40, 200,000 credits
Zenrows	25 credits with rendering and premium proxies	Build $19, 45,000 credits
ScrapingBee	25 credits with premium proxy and rendering; 75 with stealth proxy	Freelance $49, 250,000 credits

Credit APIs charge 1 credit for pages that need no bypass, so on lightly protected sites they can cost less per page than ScrapeUnblocker. The fair test is cost per good page on your own URLs; ScrapeUnblocker includes 500 free requests, no card needed.

Sources

• Cloudflare docs: Error 403, Error 1020, Error 1010, Bot scores, JA3/JA4, detection engines, challenge pages, clearance, viewed 24 September 2026; detect a challenge response, viewed 9 October 2026
• curl_cffi on GitHub, viewed 24 September 2026
• ScrapeUnblocker: Glassdoor and Glassdoor Reviews scraper pages, pricing, errors, handling failures
• Bright Data Web Unlocker pricing, viewed 9 October 2026
• ScraperAPI pricing, viewed 9 October 2026; credit costs, viewed 24 September 2026
• Scrapingdog pricing, viewed 9 October 2026; Stealth Mode, viewed 24 September 2026
• Zenrows pricing docs, ScrapingBee pricing and documentation, viewed 9 October 2026

Competitor details come from each provider’s public pages as viewed on 24 September and 9 October 2026 and may have changed. Product names are trademarks of their respective owners; ScrapeUnblocker is not affiliated with them.