Business data
Semrush Scraper
semrush.com
Semrush publishes a free website page for the sites it tracks, about 200,000 of the most visited on the web, with its own estimates of their traffic and search performance. A dedicated plugin endpoint takes a domain, any URL on the site or a Semrush website page URL and returns that page already parsed: the Authority Score, global, country and category rank with their changes, monthly visits with the change and their monthly history split into desktop and mobile, pages per visit, time on site and bounce rate, organic and paid search traffic with their history, backlinks and referring domains with their history and Semrush's Google penalty risk, the top organic keywords with position, search volume, CPC, traffic share and intent, top countries with their desktop and mobile share, competitors, incoming and outgoing traffic sources, and the visits the site gets from AI assistants such as ChatGPT and Claude. With report=competitors it returns the site's competitor list instead, each competitor with its Authority Score, ranks, visits and engagement. The figures are Semrush's own estimates, and dataDate names the month they describe. A domain Semrush has no public page for comes back with found: false. It reads Semrush's public pages: no Semrush account, login or API key is involved.
What you can extract
- Authority Score, global rank, country rank and category rank, each with its change
- Monthly visits with the change, and the visit history split into desktop and mobile
- Pages per visit, time on site and bounce rate
- Organic and paid search traffic with their change and history
- Backlinks and referring domains with their change and history, and the Google penalty risk
- Top organic keywords with position, search volume, CPC, traffic share and intent
- Top countries by visits with their desktop and mobile share, and competitors with relevance and visits
- Incoming and outgoing traffic sources, and visits from AI assistants such as ChatGPT and Claude
- Competitor list: each competitor's Authority Score, ranks, visits, pages per visit, time on site and bounce rate
Dedicated endpoints
/seo/semrush Semrush website overview and competitors
Semrush's public website data for domain (a bare domain, any URL on the site or a Semrush website page URL). report=overview (default): authorityScore, globalRank, countryRank, categoryRank, visits with visitsChangePercent and visitsHistory (desktopVisits, mobileVisits), pagesPerVisit, timeOnSiteSeconds, bounceRate, searchTraffic (organic and paid with history), backlinks (backlinks, referringDomains, googlePenaltyRisk, history), topKeywords (position, searchVolume, cpc, trafficShare, intents), topCountries, competitors, trafficSources (incoming and outgoing) and aiTraffic. report=competitors: competitors, each with authorityScore, globalRank, countryRank, categoryRank, visits, visitsChangePercent, pagesPerVisit, timeOnSiteSeconds and bounceRate. found is false for a domain Semrush has no public page for.
How to scrape it
curl -X POST \
-H "X-ScrapeUnblocker-Key: $SU_API_KEY" \
"https://api.scrapeunblocker.com/seo/semrush?domain=github.com"import requests
resp = requests.post(
"https://api.scrapeunblocker.com/seo/semrush",
headers={"X-ScrapeUnblocker-Key": SU_API_KEY},
params={"domain": "github.com"},
)
print(resp.text)import { ScrapeUnblockerClient } from 'scrapeunblocker';
const su = new ScrapeUnblockerClient({ apiKey: process.env.SU_API_KEY });
const overview = await su.semrushOverview('github.com');
const rivals = await su.semrushOverview('github.com', { report: 'competitors' });
console.log(overview);
console.log(rivals);Requests are POST and take their parameters in the query string. Browser rendering always happens - there is no flag to enable.
Example response
{
"domain": "github.com",
"found": true,
"report": "overview",
"source": "https://www.semrush.com/website/github.com/overview/",
"dataDate": "2026-09-15",
"updatedAt": "2026-10-11",
"brandName": "GitHub",
"authorityScore": 100,
"globalRank": 24,
"globalRankChange": 1,
"countryRank": { "rank": 9, "country": "BD", "countryName": "Bangladesh", "rankChange": 0 },
"categoryRank": { "rank": 1, "category": "Computer Software and Development", "rankChange": 0 },
"visits": 1436619552,
"visitsChangePercent": null,
"categories": [ "Computer Software and Development", "Online Services" ],
"pagesPerVisit": 6.075,
"timeOnSiteSeconds": 719,
"bounceRate": 0.4011,
"visitsHistory": [
{
"date": "2026-09-01",
"visits": 1436619552,
"desktopVisits": 1166423897,
"mobileVisits": 270195655
},
{
"date": "2026-08-01",
"visits": 1212551945,
"desktopVisits": 928213741,
"mobileVisits": 284338204
}
],
"searchTraffic": {
"date": "2026-09-15",
"organic": 72449659,
"organicChangePercent": 2.49,
"paid": 21088,
"paidChangePercent": -13.47,
"history": [
{ "date": "2026-09-15", "organic": 72449659, "paid": 21088, "total": 72470747 },
{ "date": "2026-08-15", "organic": 70689927, "paid": 24372, "total": 70714299 }
]
},
"backlinks": {
"date": "2026-09-01",
"backlinks": 5088322474,
"backlinksChangePercent": -4.63,
"referringDomains": 4217125,
"referringDomainsChangePercent": 2.48,
"googlePenaltyRisk": "Low",
"history": [
{ "date": "2026-09-01", "backlinks": 5088322474, "referringDomains": 4217125 },
{ "date": "2026-08-01", "backlinks": 5335220083, "referringDomains": 4115198 }
]
},
"topKeywords": {
"date": "2026-09-15",
"country": "BD",
"keywords": [
{
"keyword": "github",
"position": 1,
"searchVolume": 135000,
"cpc": 0.62,
"trafficShare": 0.2785,
"intents": [ "navigational" ]
},
{
"keyword": "zefoy",
"position": 4,
"searchVolume": 1000000,
"cpc": 0,
"trafficShare": 0.0361,
"intents": [ "navigational" ]
}
]
},
"topCountries": [
{
"country": "BD",
"countryName": "Bangladesh",
"visits": 350209633,
"trafficShare": 0.2438,
"desktopShare": 0.9886,
"mobileShare": 0.0114
},
{
"country": "US",
"countryName": "United States",
"visits": 181575163,
"trafficShare": 0.1264,
"desktopShare": 0.7269,
"mobileShare": 0.2731
}
],
"competitors": [
{ "domain": "stackoverflow.com", "relevance": 0.23, "visits": 25915762 },
{ "domain": "sourceforge.net", "relevance": 0.14, "visits": 38202451 }
],
"trafficSources": {
"date": "2026-09-15",
"incoming": [
{ "source": "Direct", "share": 0.63535549, "visits": 741093818, "previousVisits": 574779767 },
{
"source": "google.com - Google organic",
"share": 0.16852929,
"visits": 196576582,
"previousVisits": 157886998
}
],
"outgoing": [
{
"destination": "release-assets.githubusercontent.com",
"share": 0.09621352,
"visits": 17832949,
"previousVisits": 12893263
}
]
},
"aiTraffic": [
{
"source": "chatgpt.com",
"name": "ChatGPT",
"visits": 11744546,
"share": 0.00873,
"history": [
{ "date": "2026-09-01", "visits": 11744546 },
{ "date": "2026-08-01", "visits": 7665006 }
]
},
{
"source": "claude.ai",
"name": "Claude",
"visits": 4149332,
"share": 0.00309,
"history": [
{ "date": "2026-09-01", "visits": 4149332 },
{ "date": "2026-08-01", "visits": 1952240 }
]
}
]
}Frequently asked questions
Is it legal to scrape Semrush?
Collecting the estimates Semrush shows on its public website pages is lawful in many jurisdictions, but the rules depend on your country, the data you collect and how you use it. The figures are Semrush's estimates, not the site's own analytics, so present them as estimates. ScrapeUnblocker returns the public page; what you collect and how you use it is your decision.
How do I scrape Semrush without getting blocked?
Semrush's public website pages are rendered on the server, so the /seo/semrush plugin asks for them with an HTTP client that presents a browser's TLS fingerprint, from a fresh ISP exit each time, and reads the data the page is built from. An exit Semrush refuses with a 403 or 429 is rested and the next try uses another one; only when no exit returns the page does the call go through the full chain that serves /getPageSource. No browser is involved in the usual case. In ScrapeUnblocker's benchmark it returned data on 100% of 1,096 calls, with a different domain on every call, about 0.9s per call.
What data do I get for a website?
The Authority Score, the global, country and category rank with their changes, monthly visits with the change and the monthly history split into desktop and mobile, pages per visit, time on site and bounce rate, organic and paid search traffic with their history, backlinks and referring domains with their history and the Google penalty risk, the top organic keywords with position, search volume, CPC, traffic share and intent, the top countries with their visits and desktop and mobile share, competitors, incoming and outgoing traffic sources, and visits from AI assistants such as ChatGPT, Claude and others with their history. report=competitors returns the competitor list, each with its Authority Score, ranks, visits and engagement.
Which websites does Semrush have public pages for?
Semrush publishes these public pages only for the sites it tracks, about 200,000 of the most visited. A domain without a page comes back with found: false and the data keys empty, so you can tell it apart from a failure, which returns an error status. A leading www. is dropped and other subdomains are kept, and dataDate with updatedAt says how fresh the figures are.
Do I need a Semrush account or API key?
No. Semrush's API needs a Semrush subscription and API units; the plugin needs neither, and no login: send a domain with your ScrapeUnblocker key and get the overview back as JSON. You get the same estimates Semrush shows signed-out visitors on its public website page.
How much does it cost to scrape Semrush?
Each call is one request: one domain, one page, the overview or the competitor list. 1 request = 1 credit, EUR 1.00 per 1,000 on pay as you go, down to about EUR 0.55 per 1,000 on the Ultimate plan. Blocks and timeouts are not billed, and the free trial includes 500 requests with no card.
Related scrapers
Semrush and other names and marks of semrush.com belong to their owner and are used here only to identify the website this API works with. ScrapeUnblocker is an independent service, not affiliated with, endorsed by or sponsored by the owner of semrush.com.
Start scraping this site today
Free tier included. No credit card required to test it.