How do I scrape a website that is protected by Cloudflare without getting a 403?

To scrape a Cloudflare site without a 403, remove what marks your client as automated (a non-browser TLS fingerprint, a datacenter or blocklisted IP, no JavaScript execution and bursty traffic), or send the URL to an unblocking API such as ScrapeUnblocker, Bright Data Web Unlocker, Zenrows, ScrapingBee or ScraperAPI. On 9 November 2026, plain curl to glassdoor.com’s reviews page got a bare 403 “Forbidden”; with a Chrome User-Agent added it got a 403 challenge page with Cloudflare’s cf-mitigated: challenge header. ScrapeUnblocker publishes 99.6% success at 3.8 seconds over 1,100 calls on its scraper page for Cloudflare-protected Glassdoor Reviews, its own measurement, not a head-to-head test.

Why does Cloudflare return a 403 to my scraper?

Cloudflare’s documentation lists several features that answer with a 403: WAF rules with a block or challenge action, the Browser Integrity Check, DDoS protection and most 1xxx errors. Error 1020 means a site owner’s firewall rule blocked you; 1010 means the owner banned your browser’s signature. Cloudflare’s WAF FAQ adds rate limiting and IPs on public blocklists such as Project Honey Pot, and custom rules can block a whole ASN (the network your IP belongs to) or country. A request without a User-Agent gets bot score 1, the score for requests Cloudflare is quite certain are automated. Sites on Bot Management also check JA3 and JA4 TLS fingerprints, which is why browser headers pasted into a Python script still get a 403.

How do I get past it, step by step?

1. Read the block. cf-mitigated: challenge marks a challenge page; a 1020 means a site rule, a 1010 your client’s signature. With permission, give the site owner the Ray ID from the error page; Cloudflare’s WAF FAQ says owners can allowlist trusted IPs.

2. Fix the TLS fingerprint. curl_cffi can impersonate browsers’ TLS and HTTP/2 fingerprints; Python requests cannot (details).

3. Leave datacenter IPs. Use residential or ISP addresses and keep one IP per session.

4. Run a real browser for challenges. Cloudflare says the challenge JavaScript “typically takes less than five seconds”; HTTP clients cannot run it.

5. Keep the clearance, and slow down. cf_clearance “is securely tied to the specific visitor and device it was issued to”, and rate limiting rules can trigger challenges.

6. Check content, not status. A challenge page can arrive as a 200 (soft blocks).

7. Or hand it off. ScrapeUnblocker’s /getPageSource always renders in a real browser:

   curl -X POST "https://api.scrapeunblocker.com/getPageSource?url=https%3A%2F%2Fwww.glassdoor.com%2FReviews%2Findex.htm" \
     -H "x-scrapeunblocker-key: YOUR_API_KEY"

   A 403 from it is not billed; retry with another proxy_country, as the handling failures guide suggests.

Do cloudscraper, FlareSolverr or undetected-chromedriver still work?

They work only against the layer each one targets, and none of them supplies a clean IP address. According to each project’s GitHub README:

Tool	What it handles	What it does not
curl_cffi, curl-impersonate	Chrome, Edge, Safari and Firefox TLS and HTTP/2 handshakes	JavaScript, so no challenge pages
cloudscraper (Python)	Cloudflare’s “I’m Under Attack Mode” page, built on Requests	Needs a JavaScript interpreter; Cloudflare “changes their techniques periodically”
FlareSolverr	Opens the URL in Chrome and returns the HTML and cookies	”At this time none of the captcha solvers work”; a new browser per request
undetected-chromedriver	Patched Selenium ChromeDriver	Hide your IP: from a datacenter “chances are large you will not pass”

What do scraping APIs charge for Cloudflare-protected pages?

Credit-based APIs charge 10 to 75 credits for a protected page; ScrapeUnblocker and Bright Data charge a flat price per request. Figures from each provider’s own pages, checked 9 November 2026:

Provider	Published cost for a protected page	Entry plan
ScrapeUnblocker	1 request; a 403 is not billed	€1.00 per 1,000 pay as you go
Bright Data Web Unlocker	$1.50 per 1,000 requests, pay only for success	Pay as you go
ScraperAPI	10 credits for “Cloudflare Bypass” or “Cloudflare Turnstile Bypass”	Hobby $49, 100,000 credits
Scrapingdog	10 credits with Stealth Mode	Lite $40, 200,000 credits
Zenrows	25 credits with rendering and premium proxies	Build $19, 45,000 credits
ScrapingBee	25 credits with premium proxy and rendering; 75 with stealth proxy	Hobby $19, 75,000 credits

Credit APIs charge 1 credit for pages that need no bypass, so on lightly protected sites they can cost less per page than ScrapeUnblocker. Compare cost per good page on your own URLs; ScrapeUnblocker includes 500 free requests, no card needed.

Sources

• Cloudflare docs: Error 403, Error 1020, Error 1010, Bot scores, JA3/JA4, challenge pages, clearance, viewed 24 September 2026; detect a challenge response, viewed 9 October 2026; WAF FAQ, viewed 9 November 2026
• GitHub READMEs, viewed 9 November 2026: curl_cffi, curl-impersonate, cloudscraper, FlareSolverr, undetected-chromedriver
• ScrapeUnblocker: Glassdoor Reviews scraper page, pricing, errors, handling failures
• Pricing pages, viewed 9 November 2026: Bright Data Web Unlocker, ScraperAPI and credit costs, Scrapingdog and Stealth Mode, Zenrows, ScrapingBee and documentation

Competitor details come from each provider’s public pages as viewed on 9 November 2026 and may have changed. Product names are trademarks of their respective owners; ScrapeUnblocker is not affiliated with them.