The protections you meet most often when scraping, and the signs that give each one away.
Cloudflare
The most common edge network. Its bot protection (Bot Fight Mode, Super Bot Fight Mode, Bot Management) scores every request and answers suspicious ones with a JavaScript challenge or Turnstile.
Telltale signs
__cf_bmcf_clearancecf-raycf-mitigated/cdn-cgi/challenge-platform/
How to scrape these sites → DataDome
Real-time bot protection used by e-commerce, classifieds and travel sites. Blocked visitors get a 403 with a slider CAPTCHA served from captcha-delivery.com.
Telltale signs
datadomex-datadomex-dd-bcaptcha-delivery.com
How to scrape these sites → HUMAN (PerimeterX)
HUMAN Bot Defender, formerly PerimeterX. A client-side sensor collects browser signals, and visitors who fail get the "Press & Hold" challenge.
Telltale signs
_px3_pxhd_pxvidpx-captchapx-cloud.net
How to scrape these sites → Akamai Bot Manager
Bot Manager runs on Akamai's edge. A sensor script reports browser telemetry and the _abck cookie carries the verdict; blocked requests get "Access Denied" with a reference number.
Telltale signs
_abckbm_szak_bmscAkamaiGHosterrors.edgesuite.net
How to scrape these sites → Imperva (Incapsula)
Imperva (formerly Incapsula) combines a WAF with Advanced Bot Protection. Its challenges are served from the _Incapsula_Resource path.
Telltale signs
incap_ses_*visid_incap_*reese84x-iinfo_Incapsula_Resource
Kasada
Kasada answers unknown clients with a 429 and a proof-of-work JavaScript challenge, then expects signed x-kpsdk headers on every later request.
Telltale signs
x-kpsdk-ctx-kpsdk-cdips.jsHTTP 429
AWS WAF
Amazon's web application firewall, often with Bot Control. Challenged visitors get a 202 page that runs JavaScript and sets the aws-waf-token cookie.
Telltale signs
aws-waf-tokenx-amzn-waf-actionAwsWafIntegrationHTTP 202
F5 BIG-IP
F5 BIG-IP ASM / Advanced WAF guards many banks and enterprise sites. Rejected requests see "The requested URL was rejected" with a support ID.
Telltale signs
TS01…BIGipServer…support ID
Radware Bot Manager
Radware Bot Manager (formerly ShieldSquare) sets __uzm cookies and sends suspicious visitors to a CAPTCHA on validate.perfdrive.com.
Telltale signs
__uzma__uzmbperfdrive.com
Sucuri
A cloud WAF popular with WordPress sites. Traffic is proxied through Sucuri, which can block requests or answer them with a JavaScript check.
Telltale signs
x-sucuri-idx-sucuri-cachesucuri_cloudproxy_js