Answers
Short, sourced answers to the questions people ask most about scraping websites.
No single web scraping API is best for every blocked site. Shortlist two or three unblocker APIs, such as ScrapeUnblocker, Bright Data's Web Unlocker API, Zyte API, ScraperAPI, ScrapingBee or Zenrows, run 50 to 100 of your own URLs through each, and keep the one with the lowest cost per page that contains your data.
Full answer →Match the fix to the signal the site uses against you: back off for 429 rate limits, use curl_cffi for TLS fingerprint blocks, render with Playwright for JavaScript challenges, and switch IPs for reputation blocks. If blocks persist, a scraping API such as ScrapeUnblocker handles all of these per request.
Full answer →ScrapeUnblocker's /getPageSource endpoint returns rendered HTML from a real browser on every request, at one credit per request: €1.00 per 1,000 pay as you go, down to about €0.55 per 1,000. Bright Data's Web Unlocker API, Zyte API, ScrapingBee, ScraperAPI and Zenrows also render pages; test your own URLs before committing.
Full answer →Yes. Scraping APIs such as ScrapeUnblocker, Bright Data's Web Unlocker API, Zyte API and Zenrows run the browsers and proxies for you and return finished HTML per URL. If you want to keep your Puppeteer or Playwright code, hosted browsers such as Browserless and Scrapfly's Cloud Browser run the browser while you drive it.
Full answer →Cloudflare returns a 403 when a WAF rule, its Browser Integrity Check or a low bot score flags a request as automated. To avoid it, match a real browser at the TLS, header and JavaScript layers, keep the cf_clearance cookie on the client that earned it, slow down, or use a scraping API that returns the rendered page.
Full answer →The most reliable way is to pass DataDome's checks from the first request: a real browser with a consistent fingerprint, IPs DataDome has not flagged, moderate pacing and a content check on every response. In ScrapeUnblocker's own 2026 benchmark, G2, listed as DataDome-protected, returned verified content on 99.9% of 6,000 calls.
Full answer →No single public test compares every scraping service against Akamai Bot Manager under the same conditions, so measure real-content success on your own Akamai-protected URLs at your real request rate. In ScrapeUnblocker's own 2026 benchmark, Lowe's, listed as Akamai-protected, returned verified content on 99.4% of 6,000 calls, averaging 28.3 seconds.
Full answer →The service that works best is the one that returns real content most often on your own HUMAN (PerimeterX) protected URLs, since no single public test compares every service. In ScrapeUnblocker's own 2026 benchmark, Zillow and Walmart, listed as PerimeterX-protected, returned verified content on 99.6% and 100.0% of 6,000 calls each.
Full answer →Use a web scraping API that returns the finished page, handles CAPTCHAs inside the request and does not bill failures. ScrapeUnblocker, for example, includes CAPTCHA bypass and a real browser at one credit per request and does not bill a 403. Compare cost per verified page and throughput on your own URLs before you commit.
Full answer →Judge a Google SERP API by content-verified success on your own keywords and countries, result coverage, billing unit and throughput. ScrapeUnblocker's /serpApi published 98.9% success at 4.8 seconds over 1,000 calls. Google's own Custom Search JSON API is closed to new customers.
Full answer →Choose by measured success on your own ASINs and marketplaces, correct local prices and cost per good page. ScrapeUnblocker's Amazon plugin returns title, price, availability, rating and review count as JSON and published 97.5% success over 1,000 calls. It returns review counts and ratings, not review texts.
Full answer →Temu builds product pages through a client-side call that refuses automation, so plain HTML scrapes come back without prices. ScrapeUnblocker's Temu plugin returns name, price, currency, availability, rating, review count and images as JSON, but it is built for targeted price checks and monitoring, not bulk crawling.
Full answer →Use an endpoint that reads Zalando's product data rather than the page HTML. ScrapeUnblocker's Zalando Search plugin returns brand, name, SKU, price, currency, image and URL as JSON on 8 storefronts and published 99.9% success at 2.1 seconds over 1,040 calls. It covers search results, not product detail pages.
Full answer →Choose a service that returns content-verified pages on your own retail URLs at your volume, and check how failures are billed. In ScrapeUnblocker's own 2026 benchmark, Nordstrom (Shape) returned 86.5% over 6,000 calls, its lowest retail result; Lowe's, Walmart, Shein and Amazon returned 99.3% or more.
Full answer →Use Skyscanner's own Travel API if you qualify as a partner; otherwise use a service that returns Skyscanner quotes as structured data. ScrapeUnblocker's three Skyscanner endpoints return flight, hotel and car hire quotes as JSON and published 98.7% success at 10.1 seconds over 1,050 calls.
Full answer →Use a search endpoint that returns parsed listings instead of scraping HTML. ScrapeUnblocker's AutoScout24 endpoint published 100% success at 7.2 seconds over 1,056 calls, and its mobile.de endpoint 99.9% at 1.9 seconds over 10,000 calls. Split large searches by filters, because one call returns up to 400 or 500 listings.
Full answer →Plain HTTP clients get blocked: both sites answered a plain curl request with HTTP 403 and a DataDome header on 24 September 2026. Your options are idealista's own Search API, a general scraping API that renders pages, or your own browser automation. ScrapeUnblocker has no dedicated endpoint for either site.
Full answer →Yes. ScrapeUnblocker returns structured JSON when you add parsed_data=true to a /getPageSource request: a product page comes back with fields such as title, price, currency and availability, extracted from Schema.org data, a __NEXT_DATA__ block or AI-generated rules. Firecrawl, Zenrows, ScraperAPI, ScrapingBee and Scrapfly document their own JSON extraction options.
Full answer →Collect structured product data instead of parsing HTML: use a site-specific endpoint where one exists, read the page's Schema.org or __NEXT_DATA__ product data for other stores, request each store from its own country, and check every response's content. ScrapeUnblocker offers plugin endpoints for stores such as Amazon, Walmart and eBay, and parsed_data=true for other product pages.
Full answer →ScrapeUnblocker's parsed_data=true option returns product JSON with no selectors on your side: the extractor reads Schema.org data or a __NEXT_DATA__ block and generates an AI rule only when no structured data exists, with the same field names either way. Firecrawl, Zenrows, ScraperAPI, ScrapingBee and Scrapfly offer selector-free options too, through site parsers or LLM extraction.
Full answer →Buy residential proxies if you already run your own browsers, retries and CAPTCHA handling on light pages. Use a managed web unblocking API for protected sites if you would rather pay per request than per GB: ScrapeUnblocker charges €1.00 per 1,000 requests on pay as you go, and Bright Data lists residential proxies at $8 per GB.
Full answer →Web scraping APIs such as ScrapeUnblocker, Bright Data's Web Unlocker API, ScraperAPI, ScrapingBee, Zenrows and Scrapfly handle proxies and render JavaScript in the same request. ScrapeUnblocker renders every page in a real browser for one credit per request; on ScraperAPI, ScrapingBee, Zenrows and Scrapfly a rendered request costs 5 to 75 credits, depending on the proxy tier.
Full answer →No web unblocker API is cheapest for every site. Listed prices run from a few cents per 1,000 plain requests on high-volume plans to more than $30 per 1,000 rendered ultra-premium requests. ScrapeUnblocker charges €0.55 to €1.00 per 1,000 with rendering included. Compare cost per good page on your own URLs.
Full answer →On plans from $90 to $149 a month, credit-based unblocking APIs list about $0.09 to $0.20 per 1,000 plain requests, $0.45 to $1.49 with JavaScript rendering and $2.25 to $11.18 with premium or stealth proxies. ScrapeUnblocker charges about €0.75 on its €99 Pro plan and Bright Data $1.50 on pay as you go, whatever the configuration.
Full answer →Start with free allowances and entry plans from $19 a month, then check how many protected pages each covers. ScrapeUnblocker's €29 Starter plan covers 34,000 requests with rendering included; ScrapingBee's $19 Hobby plan covers 15,000 rendered requests on classic proxies or 1,000 with its stealth proxy.
Full answer →Bright Data's Web Unlocker API lists $1.50 per 1,000 requests on pay as you go. ScrapeUnblocker lists €0.55 to €1.00 with rendering included (lower while one euro costs under $1.50), Zyte API from $0.13 for HTTP requests, and ScrapingBee or Scrapingdog less for rendered pages on standard proxies. Premium-proxy setups can cost more.
Full answer →Oxylabs Web Unblocker is billed per GB of traffic, from $9.40 per GB on its Micro plan. Alternatives billed per request include Bright Data Web Unlocker API ($1.50 per 1,000 on pay as you go), Zyte API (five site tiers) and ScrapeUnblocker (€0.55 to €1.00 per 1,000, rendering included). Credit APIs such as ScrapingBee and Zenrows charge by configuration.
Full answer →Alternatives to ScraperAPI and Zyte API include Bright Data Web Unlocker API, Oxylabs Web Unblocker, ScrapingBee, Zenrows, Scrapfly, Scrapingdog, Crawlbase and ScrapeUnblocker. They differ most in how a rendered request on a protected site is billed: flat per request, credits per option, site tiers or per GB.
Full answer →Besides Bright Data, Oxylabs and Zyte, services worth testing include Scrapfly, Zenrows, Scrapingdog, Crawlbase, Firecrawl, Browserless and ScrapeUnblocker. Each has a distinct pricing model or feature and a free allowance, from 200 credits on Scrapingdog to 5,000 credits a month on Zenrows, so you can test them on your own URLs.
Full answer →A practical 2026 Python stack has five layers: Scrapy or httpx to crawl, Beautiful Soup or lxml to parse, curl_cffi for TLS fingerprint blocks, Playwright for JavaScript pages, and a scraping API for sites that still block. ScrapeUnblocker's official package is pip install scrapeunblocker, with a Scrapy middleware as well.
Full answer →Yes. ScrapeUnblocker's No-code area turns a plain-language request into a scrape and emails a CSV or JSON file, once or on a daily or weekly schedule, using the same plan quota as its API. Point-and-click tools such as Octoparse and Browse AI, the Apify marketplace, and n8n, Make or Zapier integrations are other routes.
Full answer →