Call Any ScrapeUnblocker Plugin From Python, Node.js, PHP or Ruby

Plugins are the fastest way to get structured data from a specific site. You send a keyword or an ID and get clean JSON back, with no selectors to maintain. The question has always been how much glue code sits between your project and that JSON.

As of SDK 0.11.0, very little. The Python, Node.js, PHP and Ruby clients have a method for every public plugin, plus a generic call for any plugin endpoint. This guide shows how the methods work, walks through a small multi-retailer script, and covers the parts that matter once you run it on a schedule: concurrency and error handling.

What Changed

Three things shipped together:

• A method for every plugin in all four SDKs. The plugins overview now lists more than 60 plugins in 14 categories: e-commerce, groceries, travel, Google, other search engines, real estate, vehicles, jobs, reviews, local businesses, apps, video and music, and SEO and ads. Each one has a matching method.
• A documentation page per plugin. Every page lists the fields, shows a sample response and has the call in cURL, Python, Node.js, PHP and Ruby side by side.
• Plugins in the MCP server. The MCP server and the hosted connector gained two tools, list_plugins and run_plugin, so an AI assistant can find and run the same plugins.

Install the SDK

Pick your language:

pip install scrapeunblocker              # Python 3.8+
npm install scrapeunblocker              # Node.js 18+
composer require scrapeunblocker/client  # PHP 8.1+
gem install scrapeunblocker              # Ruby 2.7+

Then put your API key in an environment variable. All four clients read it automatically:

export SCRAPEUNBLOCKER_KEY="YOUR_API_KEY"

You get the key from the dashboard. If you already have an SDK installed, upgrade to 0.11.0 or later to get every plugin method.

How the Plugin Methods Work

The pattern is the same in every language, so once you have used one plugin you can use all of them:

• The method is named after the plugin. Walmart Search is walmart_search in Python and Ruby, and walmartSearch in Node.js and PHP.
• The main input goes first. A search keyword, a product ID or a domain is the first argument.
• Everything else is an option. Keyword arguments in Python and Ruby, an options object in Node.js (camelCase keys), an array keyed by the API’s parameter names in PHP.
• Options you leave out are not sent. The API’s own defaults apply, so you only pass what you want to change.
• Lists are handled for you. Pass a list where a plugin accepts several values and the SDK sends it comma-separated.

Here is the same Best Buy search in all four languages:

# Python
result = su.bestbuy_search("laptop", sort="price_low", max_results=48)

// Node.js
const result = await su.bestbuySearch("laptop", { sort: "price_low", maxResults: 48 });

// PHP
$result = $su->bestbuySearch('laptop', ['sort' => 'price_low', 'max_results' => 48]);

# Ruby
result = su.bestbuy_search("laptop", sort: "price_low", max_results: 48)

In each case su is the client: Client() in Python, new ScrapeUnblockerClient() in Node.js, new Client() in PHP and ScrapeUnblocker::Client.new in Ruby.

Example: Compare Prices Across Three Retailers

A good first project is a price check across stores. Walmart, Target and Best Buy each have a search plugin, and all three accept the same sort="price_low" and max_results options. Their responses are not identical, though. Each plugin returns the fields the site itself uses:

• Walmart calls the product name name, its ID id and the old price wasPrice.
• Target uses title and its own product ID tcin.
• Best Buy uses title, sku and originalPrice, and adds a model number.

A field with no value is left out of that product entirely. So the first job is to map the three shapes into one:

from scrapeunblocker import Client

su = Client()  # reads SCRAPEUNBLOCKER_KEY
QUERY = "wireless headphones"

def normalize(store, item):
    return {
        "store": store,
        "id": item.get("id") or item.get("tcin") or item.get("sku"),
        "title": item.get("name") or item.get("title"),
        "brand": item.get("brand") or "",
        "price": item.get("price"),
        "regular_price": item.get("wasPrice") or item.get("originalPrice"),
        "url": item.get("url"),
    }

stores = {
    "walmart": su.walmart_search,
    "target": su.target_search,
    "bestbuy": su.bestbuy_search,
}

rows = []
for store, search in stores.items():
    result = search(QUERY, sort="price_low", max_results=48)
    rows += [normalize(store, p) for p in result.get("products", [])]

sony = [r for r in rows if r["brand"].lower() == "sony" and r["price"]]
for r in sorted(sony, key=lambda r: r["price"])[:10]:
    print(f'{r["store"]:8} {r["price"]:>8.2f}  {r["title"][:60]}')

Three calls, one table, every row with a store, a price and a link. From here you can write the rows to a database once a day and you have a price history.

One honest caveat: matching the same product across stores is the hard part, not the scraping. Titles differ from store to store. Brand plus model number is a much better key than the title, and Best Buy gives you model directly. Plan for some manual matching on your most important products.

Run Calls in Parallel Without Hitting 429

Once you track more than a few keywords, sequential calls get slow. Every method has an async twin on AsyncClient in Python, and every method returns a promise in Node.js. Cap the number of calls in flight with a semaphore:

import asyncio
from scrapeunblocker import AsyncClient

QUERIES = ["wireless headphones", "air fryer", "robot vacuum", "4k tv"]
STORES = ["walmart", "target", "bestbuy"]
CONCURRENCY = 10  # keep this below your plan's concurrency limit

async def search(su, sem, store, query):
    async with sem:
        method = getattr(su, f"{store}_search")
        return store, query, await method(query, max_results=48)

async def main():
    sem = asyncio.Semaphore(CONCURRENCY)
    async with AsyncClient() as su:
        jobs = [search(su, sem, s, q) for s in STORES for q in QUERIES]
        return await asyncio.gather(*jobs, return_exceptions=True)

results = asyncio.run(main())

Why the cap matters: plans are limited by concurrent requests, not requests per second. The limits are 20 on pay as you go, 25 on Starter, 100 on Pro, 200 on Business, 300 on Scale and 400 on Ultimate. A request over the limit gets a 429 right away and is not billed. The SDK retries a 429 with backoff on its own, but staying under the limit keeps your run fast and predictable. See concurrency limits for details.

Handle Errors Like a Pipeline, Not a Script

A script you run once can crash on an error. A job that runs every night should not. The SDKs raise typed errors, all subclasses of ScrapeUnblockerError, and they already retry the transient ones (429, 502, 503, 504 and network errors) with exponential backoff. A plugin call that runs out of time now returns 502 and is not billed, so the built-in retry covers it too.

What is left for you to handle:

from scrapeunblocker import (
    Client,
    BudgetExceededError,
    ScrapeUnblockerError,
    ValidationError,
)

su = Client()

try:
    deals = su.bestbuy_search("oled tv", sort="discount", max_results=96)
except BudgetExceededError:
    # 402: this month's spend reached the limit you set. Stop, do not retry.
    raise SystemExit("Monthly budget reached, stopping the run")
except ValidationError as e:
    # 422: a parameter is missing or has the wrong type
    print("fix the request:", e.body)
except ScrapeUnblockerError as e:
    # anything that survived the built-in retries
    print("skipping this query:", e)

The BudgetExceededError case is new. You can now set a monthly budget limit in your profile. Once your spend for the billing period reaches it, calls return 402 until the next period or until you raise the limit. For scheduled jobs this is a simple safety net: a runaway loop cannot surprise you on the invoice. The SDKs never retry a 402, because it only clears when the billing state changes.

The full list of status codes is in the errors reference.

The plugin() Escape Hatch

New plugins are added over time, and your installed SDK version may not have a method for the newest one yet. For that case every client has a generic call that takes the endpoint path and its parameters:

data = su.plugin("/marketplace/target-search", query="tv", max_results=24)

const data = await su.plugin("/marketplace/target-search", { query: "tv", max_results: 24 });

Two details to remember. The path is the one shown at the top of each plugin’s docs page. And parameters go in under the names the API uses, so in Node.js you write max_results here, not the camelCase maxResults that the named methods take.

The Same Plugins Inside Your AI Assistant

If you use Claude, ChatGPT or another MCP client, you do not need code at all. With the new list_plugins and run_plugin tools, the assistant can find the right plugin and run it from a plain request such as “Find coffee makers on Target under $50 and compare them with Walmart’s.” Listing plugins is free. Each run is billed like an API request.

The setup is covered in Give Claude Web Access With the ScrapeUnblocker MCP Server and in the MCP docs.

FAQ

Do I need an SDK to use plugins? No. Every plugin is a plain HTTP endpoint: a POST with the parameters in the query string and your key in the x-scrapeunblocker-key header. The SDKs add typed errors, retries and method names, but any HTTP client works.

How are plugin calls billed? A plugin run draws from the same plan quota as any other API request. There is no separate plugin billing. Calls rejected with 429, plugin timeouts returned as 502, and blocks returned as 403 are not billed.

Why is a field missing from some products? Plugins leave out fields that have no value on the source page. A product without a markdown has no wasPrice, and a search tile without a brand has no brand. Read fields with a default, like .get() in Python, instead of assuming they exist.

Can I try a plugin before writing code? Yes. Open the plugins in the dashboard, fill in the fields and run it. You get the JSON and a copy-paste snippet you can move into your project.

Where to Start

Pick one site you already collect data from by hand, find its plugin in the plugins overview or the scrapers catalogue, and replace your scraper with a single method call. Then add the concurrency cap and the error handling above before you put it on a schedule.

ScrapeUnblocker handles the blocking, rendering and parsing behind each plugin, so your code stays a few lines long no matter which site is on the other end. If you do not have a key yet, you can start on the pricing page.