TLS Fingerprinting: Why Your Python Scraper Gets Blocked

You copy the headers straight out of your browser’s network tab. User-Agent, Accept, Accept-Language, all of it. You paste them into a Python script, send the request, and still get a 403. Open the same URL in a real browser and it loads instantly.

If that has happened to you, the block is almost certainly not about your headers. It happens before a single header is read. The site is reading your TLS handshake, and your HTTP client looks nothing like a browser at that layer.

This post explains what a TLS fingerprint is, why standard HTTP clients stand out, how to inspect your own, and how to blend in using open-source tools.

What a TLS Fingerprint Is

Every HTTPS connection starts with a TLS handshake. The very first thing your client sends is a ClientHello message. Long before any URL, cookie, or header is exchanged, that message already reveals a lot:

• The TLS version you support
• The list of cipher suites you offer, in a specific order
• The extensions you advertise (SNI, ALPN, supported groups, signature algorithms, and more)
• The elliptic curves and point formats you accept

None of this is secret, and all of it is deterministic per client. A TLS library assembles the ClientHello the same way every time. So you can hash those fields into a short string and get a stable identifier for “what kind of client is this.” That identifier is a JA3 fingerprint (and its newer, more robust successor, JA4).

The important part: the fingerprint is a property of the TLS library, not of your code. Python’s requests sits on top of urllib3, which uses the system OpenSSL. Chrome uses BoringSSL. Firefox uses NSS. Each one builds the ClientHello differently - different cipher order, different extension set, different padding. So a server that records JA3 fingerprints can tell “this is OpenSSL-based tooling” apart from “this is Chrome” without decrypting anything.

Why requests Stands Out

Here is the problem in one sentence: real browsers make up the overwhelming majority of TLS handshakes on the public web, and Python’s default stack produces a handshake that no browser produces.

A few concrete reasons your client looks unusual:

• Cipher suite order. OpenSSL’s default preference list is not the list Chrome ships, and order is part of the fingerprint.
• Extensions. Browsers send extensions like application_settings, GREASE values, and specific supported_versions that a bare OpenSSL client does not.
• GREASE. Chrome deliberately injects random “GREASE” values into its cipher and extension lists to keep servers tolerant of unknown values. Their presence (and pattern) is itself a signal.

Add it up and the JA3 of a default requests install matches a small, well-known set of values shared by scraping scripts and bots. Anti-bot vendors keep lists of those values. When your handshake matches one, you can be blocked, throttled, or fed a fake page - all before your carefully copied headers are ever parsed.

This is also why swapping in a browser User-Agent string does nothing on its own. You changed a header. The fingerprint that gave you away lives one layer down.

See Your Own Fingerprint

You can inspect exactly what your client sends. Several public services echo your TLS details back to you. Compare a plain client against a browser:

import requests

r = requests.get("https://tls.peet.ws/api/all")
data = r.json()
print(data["tls"]["ja3"])
print(data["tls"]["ja3_hash"])

Run that, then open https://tls.peet.ws/api/all in Chrome and look at the ja3_hash field. They will not match. The browser’s hash belongs to a value shared by millions of real users; the requests hash belongs to a value shared mostly by other scripts.

That single comparison is usually the “aha” moment. Your headers were fine all along.

How to Blend In: curl_cffi

The cleanest fix in Python is curl_cffi, a binding to curl-impersonate. It ships TLS profiles that reproduce real browsers’ ClientHello byte for byte, including cipher order, extensions, and GREASE.

from curl_cffi import requests

# Impersonate a recent Chrome build - TLS + HTTP/2 fingerprint included
r = requests.get(
    "https://example.com/api/products",
    impersonate="chrome",
    timeout=30,
)
print(r.status_code)
print(r.json())

The API is intentionally close to requests, so migrating existing code is mostly a matter of the import and the impersonate argument. It also handles HTTP/2, which matters (more on that below). You can pin a specific target like impersonate="chrome124" or impersonate="firefox" when a site is picky about a particular version.

Verify it worked by hitting the echo service again through curl_cffi - the JA3 hash should now match a real browser.

Other Options

curl_cffi is not the only route. Pick based on your stack and how strict the target is.

• tls-client - a Python wrapper around a Go TLS implementation with a large library of browser profiles. A good alternative if a curl_cffi profile does not match a specific site.
• A real browser - Playwright or Selenium drive an actual browser engine, so the TLS fingerprint is genuine by definition. The cost is speed and memory: a browser is far heavier than an HTTP client, so reserve it for pages that truly need JavaScript rendering.
• Node.js - the ecosystem is thinner, but wrappers around curl-impersonate exist. Undici lets you tune some TLS options, though matching a browser exactly is harder than in Python.

A practical pattern is to warm a session in a real browser once, capture the cookies, and then replay cheap requests with a fingerprint-matching HTTP client. You get browser-grade trust without paying browser cost on every call.

The Fingerprint Is Only Half the Story

Matching your JA3 gets you past the TLS layer, but modern anti-bot systems check more than one thing. If you fix the handshake and still get blocked, look at:

• Header order and casing. HTTP/2 sends headers in a specific order and lowercases them. If your client sends browser headers in the wrong order, that is its own fingerprint (sometimes hashed as an “HTTP/2 fingerprint” or Akamai fingerprint). This is a big reason HTTP/2 support matters - curl_cffi handles it, plain requests does not speak HTTP/2 at all.
• TCP/IP signals. Some systems fingerprint the OS from TCP window sizes and options. You cannot easily fake this from user space, which is where proxies with clean, residential-grade IPs come in.
• Behavior. Request timing, navigation patterns, and whether you load the page’s sub-resources all feed reputation scoring.

The point is to think in layers. A blocked request is rarely one problem; it is the sum of every layer where your client differs from a human’s browser.

When to Stop Fine-Tuning

Matching TLS and HTTP/2 fingerprints yourself is very doable for a handful of targets. It gets tedious at scale, because profiles drift: browsers update, anti-bot vendors adjust, and a profile that worked last month starts failing. Maintaining a fleet of fingerprints, proxies, and retry logic becomes its own project.

If the fingerprinting arms race is not the thing you want to own, a scraping API handles the whole stack for you - a browser-accurate TLS and HTTP/2 fingerprint, IP rotation, and retries - behind one HTTP call. ScrapeUnblocker does exactly that: you send a URL, it returns the real page, and the handshake looks like a real browser because the request is made by real browser infrastructure. It is a clean fallback for the sites where hand-tuned fingerprints stop being worth the upkeep. You can read the API docs to see how it fits into an existing scraper.

FAQ

Can I change my TLS fingerprint by setting a browser User-Agent? No. The User-Agent is an HTTP header sent after the handshake. Your JA3 is decided by the TLS library building the ClientHello. Changing the header does not touch the fingerprint that gave you away.

Is JA3 the same as JA4? They serve the same purpose - identifying a client from its ClientHello - but JA4 is newer and more resistant to trivial evasion. Many systems now log both. Tools like curl_cffi match real browsers closely enough to satisfy both.

Does using a proxy fix TLS fingerprinting? No. A proxy changes your IP address, not your handshake. You can have a perfect residential IP and still be blocked if your JA3 screams “Python script.” You usually need both a clean IP and a browser-like fingerprint.

Will a browser-matching fingerprint make me undetectable? No single fix does. It removes one strong signal. Header order, HTTP/2 frames, IP reputation, and behavior all still matter. Treat the fingerprint as necessary, not sufficient.

Wrapping Up

When a request works in your browser but not in your script, resist the urge to keep tweaking headers. Check the layer underneath. Compare your JA3 against a real browser with a public echo service, and if they differ - they will - reach for curl_cffi or a real browser engine to close the gap.

Fix the handshake first. Then worry about headers, HTTP/2, IPs, and behavior, in that order. And when maintaining all of that stops being a good use of your time, a scraping API like ScrapeUnblocker exists to absorb the whole problem so you can get back to using the data.