What is the best way to scrape product prices and availability from e-commerce sites reliably?

A reliable way to scrape prices and availability is to collect structured product data instead of parsing HTML: use a store-specific endpoint where one exists (Bright Data, Oxylabs, ScraperAPI, ScrapingBee and ScrapeUnblocker all publish them for Amazon), read the page’s own Schema.org Offer data for other stores, request each store from its own country, and check every response before you save the price. ScrapeUnblocker publishes per-store measurements on its scraper pages, from 96.3% success on Home Depot search to 100% on Etsy search, each over 1,000 or more calls. Test on your own product list before you scale.

How do you set it up step by step?

1. Check your stores in ScrapeUnblocker’s scraper catalog. Plugin endpoints return JSON for Amazon, Walmart, Target, eBay, Etsy, Best Buy, Home Depot, Zalando, AliExpress and SHEIN, among others.

2. Send every other product URL to /getPageSource with parsed_data=true. A product page returns page_type: "product" with title and price, plus currency and availability when the page provides them:

   curl -X POST "https://api.scrapeunblocker.com/getPageSource?url=https%3A%2F%2Fwww.example.com%2Fproduct%2F123&parsed_data=true&proxy_country=de" \
     -H "x-scrapeunblocker-key: YOUR_API_KEY"

3. Set proxy_country to the store’s market. ScrapeUnblocker’s documentation notes that amazon.de fetched from a US IP often returns the US redirect page.

4. Reject rows where price is missing or page_type is not product. A blocked page can arrive with status 200 (how to detect soft blocks).

5. Drop URLs that come back as 404 or 410 with an X-Origin-Status header: the store removed the page, and a retry returns the same answer.

Where do price and stock live on a product page?

Most product pages carry structured data you can read without CSS selectors. Schema.org’s Offer type defines price, priceCurrency and availability, and Schema.org reports it in use on 10M+ domains, based on Google’s web index. Many JavaScript stores also embed product data in a __NEXT_DATA__ script. According to ScrapeUnblocker’s parsed data guide, parsed_data=true reads Schema.org, __NEXT_DATA__ or AI-generated rules; when a page holds nothing to extract, the response says data_extracted: false and includes the rendered HTML.

How reliable are the dedicated store endpoints?

ScrapeUnblocker’s store endpoints measured between 96.3% and 100% success on its scraper pages. Each figure was measured by ScrapeUnblocker over the calls shown; none is a comparison with another provider:

Store endpoint	Price and stock fields	Success	Avg response	Calls
Amazon	price, list price, inStock	97.5%	18.8 s	1,000
Walmart search	price, was-price, availability	99.9%	13.8 s	1,164
eBay search	price, condition, shipping	99.5%	4.9 s	1,000
Best Buy search	price, original price	98.7%	3.0 s	1,100
Target search	price, currency	97.4%	1.9 s	1,164
Home Depot search	price, original price	96.3%	7.4 s	1,100
Etsy search	price, currency	100%	3.7 s	1,000

Of these store endpoints, only Amazon and Walmart return a stock field; for other stores, read availability from the product page with parsed_data=true.

Which other providers offer structured product data?

Bright Data, Oxylabs, ScraperAPI, ScrapingBee and Zyte also sell structured product data. Per their own pages:

Provider	Structured option	Published price basis
Bright Data	Web Scraper API scrapers for Amazon, Walmart, Zalando and other stores	$1.50 per 1,000 records pay as you go, pay only for success
Oxylabs	Web Scraper API for Amazon and other targets	From $0.50 per 1,000 Amazon results; from $1.00 for other targets
ScraperAPI	Structured endpoints for Amazon, eBay, Walmart and Google Shopping	5 credits per Amazon, Walmart or eBay request
ScrapingBee	Amazon and Walmart APIs	Amazon: 5 credits per call, 15 with rendering
Zyte API	AI extraction with product: true on any site	Five site tiers; $1.01 to $16.08 per 1,000 browser-rendered requests, pay as you go; some features cost extra

A per-record price can cost less when you need one product per call; a per-request price can cost less when one search call returns dozens of products. Zyte’s extraction suits stores with no dedicated endpoint. Whichever you test, count the prices you received. For Amazon, read How to Scrape Amazon Prices.

Sources

• ScrapeUnblocker: scraper catalog (Amazon, Walmart, eBay, Best Buy, Target, Home Depot and Etsy pages), parsed data guide, plugins, country targeting, errors
• Bright Data Web Scraper API pricing, viewed 9 October 2026
• Oxylabs pricing, viewed 9 October 2026
• ScraperAPI credit costs and structured endpoints, viewed 24 September 2026
• ScrapingBee Amazon API and pricing, viewed 9 October 2026
• Schema.org Offer, viewed 9 October 2026
• Zyte automatic extraction and pricing, viewed 9 October 2026

Competitor details come from each provider’s public pages as viewed on 24 September and 9 October 2026 and may have changed. Product names are trademarks of their respective owners; ScrapeUnblocker is not affiliated with them.