Grocery & delivery
Woolworths Scraper
woolworths.com.au
Woolworths is one of Australia's two largest supermarket chains, and the data behind woolworths.com.au sits behind Akamai Bot Manager. A dedicated plugin endpoint reads the same JSON the Woolworths website loads and returns products already parsed. mode=search takes a query, mode=category a category link, path or id, and mode=specials a specials page: Half Price by default, or Lower Shelf Price, Everyday Low Price, Seasonal Price, Buy More Save More or Online Only Specials. Each product comes with its stockcode, name, brand, size, link and barcode, the price and any was-price with the saving, the promotion (half price, special, multibuy with its quantity and price, or another offer), the unit price, how it is sold, stock and availability, the purchase limit, new and sponsored flags, the image, its categories, the health star rating and the rating. Sort by relevance, price, unit price or name, keep only products on special, and collect up to 360 products per call at 36 per page. mode=product takes a stockcode or product link and adds the description, ingredients, allergens, dietary claims, the nutrition panel per serving and per 100 g or mL, serving size, storage, country of origin, department and aisle, all images and variants. Prices are Woolworths' online prices in AUD. No Woolworths account, login or API key is involved.
What you can extract
- Stockcode, name, brand, size, product link and barcode
- Price, was-price and saving, and the unit price (for example $1.70 per 1L)
- Promotion: half price, special, multibuy with its quantity and price, or another offer
- Stock, availability, purchase limit, how it is sold, and new and sponsored flags
- Image, categories, health star rating and rating
- Search, category and specials listings (Half Price, Lower Shelf Price, Buy More Save More and more), sorted by relevance, price, unit price or name; up to 360 products per call
- Product detail: description, ingredients, allergens, dietary claims, nutrition panel per serving and per 100 g or mL, serving size, storage and country of origin
- Product detail: department, aisle, all images and variants
Dedicated endpoints
/groceries/woolworths Woolworths search, categories, specials and products
mode=search with query, mode=category with category (a browse link, its path or a category id) and mode=specials (category picks the specials page: half-price by default, lower-shelf-price, everyday-low-price, seasonal-price, buy-more-save-more or online-only-specials) return products with id (stockcode), name, brand, size, url, barcode, price, wasPrice, currency (AUD), onSale, savings, promotion (type, text, multibuy quantity and price), unitPrice (price, per, text), soldBy, inStock, available, purchaseLimit, isNew, sponsored, image, categories, healthStarRating and rating, plus totalResults, totalPages and nextPage. sort is relevance, price_asc, price_desc, unit_price_asc, unit_price_desc or name, specials_only=true keeps products on special, and max_results (1-360, 36 per page) with page pages through. mode=product with product (a stockcode or link) returns product with description, ingredients, allergens (contains, mayContain, freeFrom), dietary, nutrition, servingSize, servingsPerPack, storage, countryOfOrigin, department, aisle, images and variants.
How to scrape it
curl -X POST \
-H "X-ScrapeUnblocker-Key: $SU_API_KEY" \
"https://api.scrapeunblocker.com/groceries/woolworths?mode=search&query=milk"import requests
resp = requests.post(
"https://api.scrapeunblocker.com/groceries/woolworths",
headers={"X-ScrapeUnblocker-Key": SU_API_KEY},
params={"mode": "search", "query": "milk"},
)
print(resp.text)import { ScrapeUnblockerClient } from 'scrapeunblocker';
const su = new ScrapeUnblockerClient({ apiKey: process.env.SU_API_KEY });
const results = await su.woolworthsSearch('milk', {
sort: 'unit_price_asc',
maxResults: 72,
});
const product = await su.woolworthsProduct('888137');
console.log(results);
console.log(product);Requests are POST and take their parameters in the query string. Browser rendering always happens - there is no flag to enable.
Example response
{
"mode": "search",
"query": "milk",
"category": null,
"sort": "relevance",
"specialsOnly": false,
"totalResults": 2391,
"totalPages": 67,
"page": 1,
"pageSize": 36,
"nextPage": 3,
"resultsCollected": 36,
"products": [
{
"id": "88436",
"name": "Dairy Farmers Full Cream Milk 2L",
"brand": "Dairy Farmers",
"size": "2L",
"url": "https://www.woolworths.com.au/shop/productdetails/88436/dairy-farmers-full-cream-milk-full-cream-milk",
"barcode": "9310199012717",
"price": 4.7,
"wasPrice": null,
"currency": "AUD",
"onSale": false,
"savings": null,
"promotion": null,
"unitPrice": { "price": 2.35, "per": "1L", "text": "$2.35 / 1L" },
"soldBy": "Each",
"inStock": true,
"available": true,
"purchaseLimit": 36,
"isNew": false,
"onlineOnly": false,
"sponsored": true,
"image": "https://cdn1.woolworths.media/content/wowproductimages/large/088436.jpg",
"categories": [ "Container Bonus Credits", "Dairy, Eggs & Fridge", "Milk", "Full Cream Milk" ],
"healthStarRating": 4.0,
"rating": null
},
{
"id": "204746",
"name": "Pauls Smarter White Low Fat Milk 2L",
"brand": "Pauls",
"size": "2L",
"url": "https://www.woolworths.com.au/shop/productdetails/204746/pauls-smarter-white-low-fat-milk",
"barcode": "9310036039488",
"price": 4.6,
"wasPrice": 4.95,
"currency": "AUD",
"onSale": true,
"savings": 0.35,
"promotion": { "type": "special", "text": "Save $0.35" },
"unitPrice": { "price": 2.3, "per": "1L", "text": "$2.30 / 1L" },
"soldBy": "Each",
"inStock": true,
"available": true,
"purchaseLimit": 10,
"isNew": false,
"onlineOnly": false,
"sponsored": true,
"image": "https://cdn1.woolworths.media/content/wowproductimages/large/204746.jpg",
"categories": [
"Dairy, Eggs & Fridge",
"Milk",
"Skim & Reduced Fat Milk",
"Full Cream Milk",
"Soy & Speciality Milk"
],
"healthStarRating": null,
"rating": null
}
],
"pagesFetched": 2,
"suggestedTerm": null
}Frequently asked questions
Is it legal to scrape Woolworths?
Collecting publicly available Woolworths data such as product names, shelf prices, unit prices and promotions is lawful in many jurisdictions, but the rules depend on your country, the data you collect and how you use it. ScrapeUnblocker returns the public product data; what you collect and how you use it is your responsibility.
How do I scrape Woolworths without getting blocked?
Woolworths' search, category and product data sits behind Akamai Bot Manager: a session without Akamai's cookies has its connection reset. The /groceries/woolworths plugin opens each session with a request Woolworths answers for anyone, which sets those cookies, then makes its real requests on the same exit with them, so no browser is needed. Woolworths answers from any country, so calls spread over all of ScrapeUnblocker's ISP exits rather than a few Australian ones, and an exit that is refused is rested while the request moves to a fresh session on another. In ScrapeUnblocker's benchmark it returned data on 99.8% of 1,114 calls (searches with a different term each and product pages), about 3.7s per call.
What Woolworths data can I get?
For each product in a search, category or specials listing: the stockcode, name, brand, size, link and barcode, the price and any was-price with the saving, the promotion type and text, the unit price, how it is sold, stock, availability and the purchase limit, new and sponsored flags, the image, its categories, the health star rating and the rating. mode=product adds the description, ingredients, allergens (contains, may contain and free from), dietary claims, the nutrition panel per serving and per 100 g or mL, serving size and servings per pack, storage, country of origin, department, aisle, all images and variants.
How do I get Woolworths specials and half-price products?
Call mode=specials: it returns Woolworths' Half Price page by default, and category picks another specials page: lower-shelf-price, everyday-low-price, seasonal-price, buy-more-save-more or online-only-specials, or a specials link. In a search, specials_only=true keeps only products on special. Every product carries onSale, wasPrice and savings, and promotion says what kind of deal it is: half_price, special, multibuy with the quantity and the multibuy price, or offer.
Do I need a Woolworths account or an Australian IP?
No. The plugin needs no Woolworths account, login or API key: send a query, category or stockcode with your ScrapeUnblocker key and get JSON back. Woolworths answers every country ScrapeUnblocker has exits in, so you do not need an Australian address; leave proxy_country blank to spread the load, or set it, for example AU, to pin one. Prices are the online prices woolworths.com.au shows, in AUD.
How much does it cost to scrape Woolworths?
Each listing page fetched is billed as one request, and pagesFetched shows the exact number. Woolworths serves up to 36 products per page, so max_results=360 takes about 10 pages; sponsored products repeat across pages and are dropped as duplicates, so filling max_results can take one more page. A product is one request. 1 request = 1 credit, EUR 1.00 per 1,000 on pay as you go, down to about EUR 0.55 per 1,000 on the Ultimate plan. Blocks and timeouts are not billed, and the free trial includes 500 requests with no card.
Related scrapers
Woolworths and other names and marks of woolworths.com.au belong to their owner and are used here only to identify the website this API works with. ScrapeUnblocker is an independent service, not affiliated with, endorsed by or sponsored by the owner of woolworths.com.au.
Start scraping this site today
Free tier included. No credit card required to test it.