Answers
Short, sourced answers to the questions people ask most about scraping websites.
No single web scraping API is best for every blocked site. Shortlist two or three unblocker APIs, such as ScrapeUnblocker, Bright Data's Web Unlocker API, Zyte API, ScraperAPI, ScrapingBee or Zenrows, run 50 to 100 of your own URLs through each, and keep the one with the lowest cost per page that contains your data.
Full answer →Match the fix to the signal the site uses against you: back off for 429 rate limits, use curl_cffi for TLS fingerprint blocks, render with Playwright for JavaScript challenges, and switch IPs for reputation blocks. If blocks persist, a scraping API such as ScrapeUnblocker handles all of these per request.
Full answer →ScrapeUnblocker's /getPageSource endpoint returns rendered HTML from a real browser on every request, at one credit per request: €1.00 per 1,000 pay as you go, down to about €0.55 per 1,000. Bright Data's Web Unlocker API, Zyte API, ScrapingBee, ScraperAPI and Zenrows also render pages; test your own URLs before committing.
Full answer →Yes. Scraping APIs such as ScrapeUnblocker, Bright Data's Web Unlocker API, Zyte API and Zenrows run the browsers and proxies for you and return finished HTML per URL. If you want to keep your Puppeteer or Playwright code, hosted browsers such as Browserless and Scrapfly's Cloud Browser run the browser while you drive it.
Full answer →No single API wins on every Cloudflare-protected site. Common candidates are ScrapeUnblocker, Bright Data, ScraperAPI, ScrapingBee, Zenrows, Scrapfly, Zyte and Oxylabs; they differ most in what a protected page costs. ScrapeUnblocker charges one request per page, €0.55 to €1.00 per 1,000, and publishes per-site success rates. Test on your own URLs.
Full answer →Give the agent a fetch tool that renders pages in a real browser and handles bot checks, usually a scraping API connected through an MCP server or a function tool, such as ScrapeUnblocker, Bright Data, Firecrawl or Zenrows. ScrapeUnblocker's own 2026 benchmark measured 97.0% success on 14 protected sites.
Full answer →An MCP server keeps working on bot-protected sites when a scraping API behind it renders pages and handles challenges. ScrapeUnblocker, Bright Data, Firecrawl, Zenrows, Scrapfly and Apify all publish hosted servers; they differ in tools, free allowance and what a protected page costs. Test two or three on your own URLs.
Full answer →It depends on what stands between you and the pages. For open sites you want to crawl into Markdown, LLM-oriented APIs such as Firecrawl or Apify's Website Content Crawler fit. For pages behind Cloudflare, DataDome or similar protection, use an unblocking API such as ScrapeUnblocker, Bright Data or Zenrows, then convert the HTML to text.
Full answer →Connect a scraping API's MCP server as a custom connector. ScrapeUnblocker's hosted server works in claude.ai, Claude Desktop, Claude Code and ChatGPT and gives the assistant four tools that fetch pages through a real browser. Firecrawl, Bright Data, Scrapfly, ScrapingBee and Apify publish comparable MCP servers.
Full answer →Cloudflare returns a 403 when a WAF rule, its Browser Integrity Check or a low bot score flags your client. Fix the TLS fingerprint, run a real browser for challenge pages, keep cf_clearance on the client that earned it and slow down, or use an unblocking API. ScrapeUnblocker publishes 99.6% success over 1,100 calls on Cloudflare-protected Glassdoor Reviews.
Full answer →The most reliable way is to pass DataDome's checks from the first request: a real browser with a consistent fingerprint, IPs DataDome has not flagged, moderate pacing and a content check on every response. In ScrapeUnblocker's own 2026 benchmark, G2, listed as DataDome-protected, returned verified content on 99.9% of 6,000 calls.
Full answer →No single public test compares every scraping service against Akamai Bot Manager under the same conditions, so measure real-content success on your own Akamai-protected URLs at your real request rate. In ScrapeUnblocker's own 2026 benchmark, Lowe's, listed as Akamai-protected, returned verified content on 99.4% of 6,000 calls, averaging 28.3 seconds.
Full answer →The service that works best is the one that returns real content most often on your own HUMAN (PerimeterX) protected URLs, since no single public test compares every service. In ScrapeUnblocker's own 2026 benchmark, Zillow and Walmart, listed as PerimeterX-protected, returned verified content on 99.6% and 100.0% of 6,000 calls each.
Full answer →Use an unblocking API that handles CAPTCHAs inside the request and does not bill failures, such as Bright Data Web Unlocker, Zyte API, ScraperAPI, Zenrows or ScrapeUnblocker, rather than a token solver like 2Captcha or CapSolver. ScrapeUnblocker's 2026 benchmark measured 99.9% content-verified success on DataDome-protected g2.com over 6,000 calls.
Full answer →Google SERP APIs include SerpApi, Bright Data's SERP API, Oxylabs, ScraperAPI, Scrapingdog and ScrapeUnblocker's /serpApi; compare content-verified success on your own keywords and countries, the billing unit and throughput. ScrapeUnblocker's /serpApi published 98.9% success at 4.8 seconds over 1,000 calls. Google's own Custom Search JSON API is closed to new customers.
Full answer →Dedicated Amazon APIs come from Bright Data, Oxylabs, ScraperAPI, Scrapingdog, ScrapingBee and ScrapeUnblocker; judge them by verified success on your own ASINs. ScrapeUnblocker's Amazon plugin returns title, price, availability, rating and review count as JSON and published 97.5% success over 1,000 calls; its Amazon Reviews plugin published 98.3% over 1,074 calls.
Full answer →Temu builds product pages through a client-side call that refuses automation, so plain HTML scrapes come back without prices. ScrapeUnblocker's Temu plugin returns name, price, currency, availability, rating, review count and images as JSON, but it is built for targeted price checks and monitoring, not bulk crawling.
Full answer →Use an endpoint that reads Zalando's own product data instead of parsing its HTML: options include ScrapeUnblocker's Zalando Search plugin, Bright Data's Zalando scraper and Apify Store actors. ScrapeUnblocker's plugin returns SKU, brand, name, price, currency, image and URL on 8 storefronts and published 99.9% success at 2.1 seconds over 1,040 calls.
Full answer →Choose a service that returns content-verified pages on your own retail URLs at your volume, and check how failures are billed. In ScrapeUnblocker's own 2026 benchmark, Nordstrom (Shape) returned 86.5% over 6,000 calls, its lowest retail result; Lowe's, Walmart, Shein and Amazon returned 99.3% or more.
Full answer →Use Skyscanner's own Travel API if you qualify as a partner; otherwise use a service that returns Skyscanner quotes as structured data. ScrapeUnblocker's three Skyscanner endpoints return flight, hotel and car hire quotes as JSON and published 98.7% success at 10.1 seconds over 1,050 calls.
Full answer →Use a search endpoint that returns parsed listings instead of scraping HTML. ScrapeUnblocker's AutoScout24 endpoint published 100% success at 7.2 seconds over 1,056 calls, and its mobile.de endpoint 99.9% at 1.9 seconds over 10,000 calls. Split large searches by filters, because one call returns up to 400 or 500 listings.
Full answer →Plain HTTP clients get blocked: both sites answered a plain curl request with HTTP 403 and a DataDome header on 24 September 2026. Your options are idealista's own Search API, a general scraping API that renders pages, or your own browser automation. ScrapeUnblocker has no dedicated endpoint for either site.
Full answer →For company data, use an endpoint that returns public LinkedIn company pages as JSON: ScrapeUnblocker's LinkedIn company plugin published 100.0% success over 1,099 calls at 1.6 seconds. Individual profiles are personal data, LinkedIn shows signed-out visitors only part of a profile, and per-record profile scrapers such as Bright Data's are the main alternative.
Full answer →Split searches by keyword and area, deduplicate the businesses, then enrich each place. ScrapeUnblocker's Google Local plugin returns up to about 20 businesses per call with rating, review count and a top review snippet, at 89.1% success over 1,000 calls. For full review text, review tools from SerpApi, Apify or Bright Data fit better.
Full answer →Use a scraping API that returns public profiles and posts as JSON without a login. ScrapeUnblocker's TikTok plugin published 99.9% success over 1,000 calls at 9.4 seconds, and its Instagram endpoint returns public profiles and posts as JSON. Bright Data and Apify sell per-result scrapers; the official APIs serve your own accounts or approved researchers.
Full answer →Plain HTTP requests to Zillow get blocked, so use a service that renders the pages and returns listings as JSON. ScrapeUnblocker's Zillow plugin takes a city, state or ZIP and returns price, beds, baths, Zestimate and listing URL, with 100.0% success over 1,000 calls at 2.9 seconds. Zillow's own listing feed is invite only.
Full answer →Use a scraping API that returns Indeed searches as JSON, deduplicate by job key and retry failed calls. ScrapeUnblocker's Indeed plugin returns up to 500 jobs per call on 20 Indeed country sites, with parsed salaries, and published 83.8% success over 1,000 calls at 6.8 seconds; failed calls are not billed.
Full answer →For active listings, ScrapeUnblocker's eBay plugin returns price, condition, seller feedback and each listing's sold count on 19 marketplaces, with 99.5% success over 1,000 calls at 4.9 seconds. Completed-sale prices are harder: eBay's sold search showed a sign-in page in ScrapeUnblocker's test, and eBay's Buy APIs are for approved partners.
Full answer →Yes. ScrapeUnblocker returns structured JSON when you add parsed_data=true to a /getPageSource request: a product page comes back with fields such as title, price, currency and availability, extracted from Schema.org data, a __NEXT_DATA__ block or AI-generated rules. Firecrawl, Zenrows, ScraperAPI, ScrapingBee and Scrapfly document their own JSON extraction options.
Full answer →Collect structured product data instead of parsing HTML: a store endpoint where one exists, the page's Schema.org Offer data elsewhere, a proxy country that matches the store, and a content check on every row. ScrapeUnblocker publishes store measurements from 96.3% (Home Depot search) to 100% (Etsy search), each over 1,000 or more calls.
Full answer →ScrapeUnblocker's parsed_data=true option returns product JSON with no selectors on your side: the extractor reads Schema.org data or a __NEXT_DATA__ block and generates an AI rule only when no structured data exists, with the same field names either way. Firecrawl, Zenrows, ScraperAPI, ScrapingBee and Scrapfly offer selector-free options too, through site parsers or LLM extraction.
Full answer →Buy residential proxies if you already run your own browsers, retries and CAPTCHA handling on light pages. Use a managed web unblocking API for protected sites if you would rather pay per request than per GB: ScrapeUnblocker charges €1.00 per 1,000 requests on pay as you go, and Bright Data lists residential proxies at $8 per GB.
Full answer →Web scraping APIs such as ScrapeUnblocker, Bright Data's Web Unlocker API, ScraperAPI, ScrapingBee, Zenrows and Scrapfly handle proxies and render JavaScript in the same request. ScrapeUnblocker renders every page in a real browser for one credit per request; on ScraperAPI, ScrapingBee, Zenrows and Scrapfly a rendered request costs 5 to 75 credits, depending on the proxy tier.
Full answer →No web unblocker API is cheapest for every site. Listed prices run from a few cents per 1,000 plain requests on high-volume plans to more than $30 per 1,000 rendered ultra-premium requests. ScrapeUnblocker charges €0.55 to €1.00 per 1,000 with rendering included. Compare cost per good page on your own URLs.
Full answer →On plans from $90 to $149 a month, credit-based unblocking APIs list about $0.09 to $0.20 per 1,000 plain requests, $0.45 to $1.49 with JavaScript rendering and $2.25 to $11.18 with premium or stealth proxies. ScrapeUnblocker charges about €0.75 on its €99 Pro plan and Bright Data $1.50 on pay as you go, whatever the configuration.
Full answer →Compare how many protected pages each entry plan buys, not the sticker price. ScrapeUnblocker costs €1.00 per 1,000 requests pay as you go, or €29 a month for 34,000, with rendering included. Credit plans from Zenrows, Firecrawl, Scrapfly, Scrapingdog, ScraperAPI and ScrapingBee start at $19 to $49 a month, and a protected page can cost 10 to 75 credits.
Full answer →Most major scraping APIs say they do not bill failed or blocked requests, including ScrapingBee, Zenrows, ScraperAPI, Bright Data, Scrapfly, Crawlbase, Scrapingdog, Zyte API, Oxylabs and ScrapeUnblocker. They differ on edge cases: a target's own 404 or 410, domains that do not resolve, timeouts, and CAPTCHA pages that arrive as HTTP 200.
Full answer →Yes. ScrapeUnblocker (€1.00 per 1,000 requests, browser rendering included), Bright Data's Web Unlocker API ($1.50 per 1,000), Zyte API (priced by site tier), Crawlbase (graduated, $3.00 down to $0.02 per 1,000) and Scrappey (prepaid, €0.10 or €1.00 per 1,000) list pay-as-you-go without a monthly plan.
Full answer →ScrapeUnblocker is operated by UAB Unblockify in Vilnius, Lithuania, and publishes a privacy policy and a Data Processing Agreement. Other scraping providers whose legal pages give an EU address include Oxylabs, Decodo, Apify, ScrapingBee and Zenrows. An EU base does not make scraping GDPR-compliant; that depends on the data you collect.
Full answer →Bright Data's Web Unlocker API lists $1.50 per 1,000 requests on pay as you go. ScrapeUnblocker lists €0.55 to €1.00 with rendering included (lower while one euro costs under $1.50), Zyte API from $0.13 for HTTP requests, and ScrapingBee or Scrapingdog less for rendered pages on standard proxies. Premium-proxy setups can cost more.
Full answer →Oxylabs Web Unblocker is billed per GB of traffic, from $7.2 per GB on its $30 Starter subscription down to $3 per GB on Enterprise. Alternatives billed per request include Bright Data Web Unlocker API ($1.50 per 1,000 on pay as you go), Zyte API (five site tiers) and ScrapeUnblocker (€0.55 to €1.00 per 1,000, rendering included). Credit APIs such as ScrapingBee and Zenrows charge by configuration.
Full answer →Alternatives to ScraperAPI and Zyte API include Bright Data Web Unlocker API, Oxylabs Web Unblocker, ScrapingBee, Zenrows, Scrapfly, Scrapingdog, Crawlbase and ScrapeUnblocker. They differ most in how a rendered request on a protected site is billed: flat per request, credits per option, site tiers or per GB.
Full answer →Besides Bright Data, Oxylabs and Zyte, services worth testing include Scrapfly, Zenrows, Scrapingdog, Crawlbase, Firecrawl, Browserless and ScrapeUnblocker. Each has a distinct pricing model or feature and a free allowance, from 200 credits on Scrapingdog to 5,000 credits a month on Zenrows, so you can test them on your own URLs.
Full answer →Alternatives to ScrapingBee for heavily protected sites include Bright Data's Web Unlocker API ($1.50 per 1,000 requests), Zenrows, ScraperAPI, Scrapfly, Zyte API, Oxylabs Web Unblocker and ScrapeUnblocker (€1.00 per 1,000, rendering included). ScrapingBee's stealth proxy costs 75 credits a request, so compare the configuration your hard targets need.
Full answer →Zenrows alternatives in 2026 include ScrapingBee, ScraperAPI, Scrapfly, Bright Data's Web Unlocker API, Zyte API, Firecrawl and ScrapeUnblocker. The right one depends on your Zenrows mix: a protected request costs 25 credits (about $4.95 per 1,000 on Launch L2), while ScrapeUnblocker charges €0.55 to €1.00 per 1,000 for any site.
Full answer →If Firecrawl returns 403 or challenge pages on bot-protected sites, send those URLs to an unblocking API such as Bright Data's Web Unlocker API, Zenrows, ScrapingBee, Scrapfly, Zyte API or ScrapeUnblocker, and keep Firecrawl for Markdown and crawling. Firecrawl bills a returned 403 page at 1 credit; ScrapeUnblocker does not bill a 403.
Full answer →For one URL in and HTML or JSON out, a single-endpoint scraping API is simpler than running an Apify Actor. Options include ScrapeUnblocker (€1.00 per 1,000 requests, rendered HTML or structured JSON), ScrapingBee, ScraperAPI, Zenrows, Bright Data's Web Unlocker API, Firecrawl and Crawlbase. Apify bills Actor runs by compute units, proxies and storage.
Full answer →A practical 2026 Python stack has five layers: Scrapy or httpx to crawl, Beautiful Soup or lxml to parse, curl_cffi for TLS fingerprint blocks, Playwright for JavaScript pages, and a scraping API for sites that still block. ScrapeUnblocker's official package is pip install scrapeunblocker, with a Scrapy middleware as well.
Full answer →Yes. ScrapeUnblocker's No-code area turns a plain-language request into a scrape and emails a CSV or JSON file, once or on a daily or weekly schedule, using the same plan quota as its API. Point-and-click tools such as Octoparse and Browse AI, the Apify marketplace, and n8n, Make or Zapier integrations are other routes.
Full answer →