Grocery & delivery
Coles Scraper
coles.com.au
Coles is one of Australia's two largest supermarket chains, and coles.com.au sits behind Imperva, which serves the site only to some countries. A dedicated plugin endpoint reads the data each Coles page is built from and returns products already parsed. mode=search takes a query, mode=category a browse link or path, and mode=specials lists the products on special: all of them, or only half-price, multibuy or online-only deals. Each product comes with its id, name, brand, size and link, the price and any was-price with the saving, the promotion (half price, special, multibuy with its quantity and price, or another offer), the unit price, stock and availability with the quantity available, the purchase and promotional limits, online-only and sponsored flags, the image, its categories and the minimum shelf life. Sort by relevance, price or unit price, and collect up to 480 products per call at 48 per page. mode=product takes a product id or link and adds the barcode, description, ingredients, allergens, dietary claims, storage, the nutrition panel per serving and per 100 g or mL, serving size, country of origin, the image gallery and variants. Prices are Coles' online prices in AUD. No Coles account, login or API key is involved.
What you can extract
- Product id, name, brand, size and product link
- Price, was-price and saving, and the unit price (for example $1.65 per 1L)
- Promotion: half price, special, multibuy with its quantity and price, or another offer
- Stock, availability and the quantity available, purchase and promotional limits, online-only and sponsored flags
- Image, categories and minimum shelf life
- Search, category and specials listings, filtered to half-price, multibuy or online-only deals and sorted by relevance, price or unit price; up to 480 products per call
- Product detail: barcode, description, ingredients, allergens, dietary claims, storage, nutrition panel per serving and per 100 g or mL, serving size and country of origin
- Product detail: image gallery and variants
Dedicated endpoints
/groceries/coles Coles search, categories, specials and products
mode=search with query, mode=category with category (a browse link or its path) and mode=specials return products with id, name, brand, size, url, price, wasPrice, currency (AUD), onSale, savings, promotion (type, text, multibuy quantity and price), unitPrice (price, per, text), inStock, available, availableQuantity, purchaseLimit, promotionalLimit, onlineOnly, sponsored, image, categories and minimumShelfLife, plus totalResults, totalPages and nextPage. special (all, half-price, multibuy or online-only) narrows the specials listing or filters a search, sort is relevance, price_asc, price_desc or unit_price_asc, and max_results (1-480, 48 per page) with page pages through. mode=product with product (a product id or link) returns product with barcode, description, ingredients, allergens, dietary, storage, nutrition, servingSize, servingsPerPack, countryOfOrigin, images and variants. proxy_country takes AU, NZ, US or CA.
How to scrape it
curl -X POST \
-H "X-ScrapeUnblocker-Key: $SU_API_KEY" \
"https://api.scrapeunblocker.com/groceries/coles?mode=search&query=milk"import requests
resp = requests.post(
"https://api.scrapeunblocker.com/groceries/coles",
headers={"X-ScrapeUnblocker-Key": SU_API_KEY},
params={"mode": "search", "query": "milk"},
)
print(resp.text)import { ScrapeUnblockerClient } from 'scrapeunblocker';
const su = new ScrapeUnblockerClient({ apiKey: process.env.SU_API_KEY });
const results = await su.colesSearch('milk', {
special: 'half-price',
maxResults: 96,
});
const product = await su.colesProduct('8150288');
console.log(results);
console.log(product);Requests are POST and take their parameters in the query string. Browser rendering always happens - there is no flag to enable.
Example response
{
"mode": "search",
"query": "milk",
"category": null,
"special": null,
"sort": "relevance",
"totalResults": 326,
"totalPages": 7,
"page": 1,
"pageSize": 48,
"nextPage": 2,
"resultsCollected": 48,
"products": [
{
"id": "8150288",
"name": "Full Cream Milk",
"brand": "Coles",
"size": "3L",
"url": "https://www.coles.com.au/product/coles-full-cream-milk-3l-8150288",
"barcode": null,
"price": 4.95,
"wasPrice": null,
"currency": "AUD",
"onSale": false,
"savings": null,
"promotion": null,
"unitPrice": { "price": 1.65, "per": "1l", "text": "$1.65/ 1L" },
"priceDescription": null,
"inStock": true,
"available": true,
"availableQuantity": 2749,
"purchaseLimit": 20,
"promotionalLimit": 12,
"onlineOnly": false,
"sponsored": false,
"image": "https://cdn.productimages.coles.com.au/productimages/8/8150288.jpg",
"categories": [ "Dairy, Eggs & Fridge", "Milk", "Full Cream Milk" ],
"minimumShelfLife": "Life 7 days min"
},
{
"id": "5477930",
"name": "Full Cream Milk",
"brand": "Pura",
"size": "2L",
"url": "https://www.coles.com.au/product/pura-full-cream-milk-2l-5477930",
"barcode": null,
"price": 4.1,
"wasPrice": 4.65,
"currency": "AUD",
"onSale": true,
"savings": 0.55,
"promotion": { "type": "special", "text": "save $0.55", "specialType": null },
"unitPrice": { "price": 2.05, "per": "1l", "text": "$2.05/ 1L" },
"priceDescription": null,
"inStock": true,
"available": true,
"availableQuantity": 374,
"purchaseLimit": 20,
"promotionalLimit": 12,
"onlineOnly": false,
"sponsored": true,
"image": "https://cdn.productimages.coles.com.au/productimages/5/5477930.jpg",
"categories": [ "Dairy, Eggs & Fridge", "Milk", "Full Cream Milk" ],
"minimumShelfLife": "Life 7 days min"
}
],
"pagesFetched": 1,
"didYouMean": null
}Frequently asked questions
Is it legal to scrape Coles?
Collecting publicly available Coles data such as product names, shelf prices, unit prices and promotions is lawful in many jurisdictions, but the rules depend on your country, the data you collect and how you use it. ScrapeUnblocker returns the public product data; what you collect and how you use it is your responsibility.
How do I scrape Coles without getting blocked?
Coles sits behind Imperva (Incapsula), which serves coles.com.au only to some countries: Australian, New Zealand, US and Canadian addresses get the pages, while European and Asian ones get a challenge or a 403 whatever the client. The /groceries/coles plugin sends every call from an ISP exit in one of those four countries, spread over all of them rather than a few Australian IPs, reads the data the Coles page is built from instead of the full HTML, and moves to a fresh session on another exit when Imperva refuses one. When no exit gets an answer, the page comes through the full chain behind /getPageSource. In ScrapeUnblocker's benchmark it returned data on 100% of 1,112 calls (searches with a different term each and product pages), about 1.8s per call.
What Coles data can I get?
For each product in a search, category or specials listing: the id, name, brand, size and link, the price and any was-price with the saving, the promotion type and text, the unit price, stock, availability and the quantity available, the purchase and promotional limits, online-only and sponsored flags, the image, its categories and the minimum shelf life. mode=product adds the barcode, description, ingredients, allergens, dietary claims, storage, the nutrition panel per serving and per 100 g or mL, serving size and servings per pack, country of origin, the image gallery and variants.
How do I get Coles specials and half-price products?
Call mode=specials to list every product on special, or add special to narrow it: half-price, multibuy or online-only. The same special filter works on a search, so mode=search with query=milk and special=half-price returns only half-price milk. Every product carries onSale, wasPrice and savings, and promotion says what kind of deal it is: half_price, special, multibuy with the quantity and the multibuy price, or offer.
Do I need a Coles account or an Australian IP?
No account is needed, and no Australian address of your own: Coles serves its site only to Australian, New Zealand, US and Canadian addresses, and the plugin picks an exit in one of them for you. Pass proxy_country as AU, NZ, US or CA to pin one; any other country is rejected, because Coles would not answer it. Prices are the online prices coles.com.au shows, in AUD.
How much does it cost to scrape Coles?
Each listing page fetched is billed as one request: Coles serves 48 products per page, so the default search is 1 request and max_results=480 up to 10, and pagesFetched shows the exact number. A product is one request. 1 request = 1 credit, EUR 1.00 per 1,000 on pay as you go, down to about EUR 0.55 per 1,000 on the Ultimate plan. Blocks and timeouts are not billed, and the free trial includes 500 requests with no card.
Related scrapers
Coles and other names and marks of coles.com.au belong to their owner and are used here only to identify the website this API works with. ScrapeUnblocker is an independent service, not affiliated with, endorsed by or sponsored by the owner of coles.com.au.
Start scraping this site today
Free tier included. No credit card required to test it.