- Data analytics
- Proxy infrastructure
Defeating JA3 and JA4 TLS fingerprinting in scrapers
Learn how JA3 and JA4 TLS fingerprinting expose your web scrapers, and why rotating proxy IPs alone cannot bypass modern bot detection.
Table of contents
You route your traffic through pristine residential IPs and still get hit with HTTP 403 errors. The problem is not your proxy provider. Modern anti-bot systems analyze your TLS connection before they even look at your headers. This guide explains exactly how JA3 and JA4 fingerprinting works and what you need to do to bypass it.
The mechanics of the ClientHello packet
When your scraper connects to a target server via HTTPS, it initiates a TLS handshake. The very first step of this handshake is the ClientHello packet. This packet contains specific parameters that tell the server how the client wishes to communicate. It includes the supported TLS versions, a list of cipher suites the client can use, elliptic curve details, and various TLS extensions.
Because different operating systems and applications use different underlying TLS libraries, the exact contents and ordering of these parameters vary wildly. A Chrome browser on Windows 11 constructs its ClientHello differently than Firefox on macOS. More importantly, Chrome constructs it entirely differently than a standard Python script using the OpenSSL library.
Anti-bot vendors like Cloudflare, Akamai, and DataDome intercept this packet before any HTTP request is made. They parse the parameters, generate a signature, and compare it against a database of known browser signatures. If your signature points to a programmatic script instead of a human-operated browser, the server drops the connection.
Understanding the JA3 fingerprint
JA3 is an open-source methodology for profiling SSL/TLS clients. It extracts five specific fields from the ClientHello packet: the SSL/TLS version, accepted cipher suites, list of extensions, elliptic curves, and elliptic curve formats.
The algorithm takes these values, concatenates them into a comma-separated string, and hashes them using MD5. The resulting 32-character string is the JA3 fingerprint. For example, a standard Python requests call will consistently produce a specific MD5 hash because it relies on the default configuration of the system's OpenSSL installation.
A typical JA3 string for a bot looks like 771,4865-4866-4867-49195,0-23-65281-10-11,29-23-24,0 before it is hashed.
Security systems use JA3 fingerprints to instantly identify malicious actors or automated tools. If the target server logs a request with a User-Agent claiming to be Google Chrome, but the JA3 hash matches a known Node.js TLS configuration, the server flags the mismatch as an anomaly and blocks the request.
The evolution to JA4 TLS
While JA3 is highly effective, it has a core limitation. Because it relies on a strict MD5 hash, changing a single parameter alters the entire hash. If a browser update adds a new cipher suite or slightly changes the order of extensions, the JA3 hash breaks.
JA4 was developed to solve this fragility. Instead of a single irreversible hash, a JA4 TLS signature segments the fingerprint into readable components. It captures the protocol version, the number of cipher suites, and the number of extensions in the first block. The second block contains a truncated hash of the cipher suites, and the third block contains a truncated hash of the extensions.
Consider the JA4 string t13d1516h2_8daaf6152771_b154aaeea456. The first block tells the server the connection is TCP (t), uses TLS 1.3 (13), includes a Server Name Indication domain (d), supports 15 ciphers and 16 extensions, and requests HTTP/2 (h2).
This structure allows anti-bot systems to perform fuzzy matching. If an engineer attempts a basic TLS fingerprinting bypass by merely shuffling the order of cipher suites, a JA3 hash will change, but the JA4 parser will recognize that the underlying capabilities remain identical to a known bot tool. Defeating JA4 requires comprehensive mimicry of the entire network stack.
Why your proxy network cannot hide your stack
Many data engineers assume that routing traffic through high-quality proxies resolves detection issues. This reflects a misunderstanding of the OSI model and how modern web filtering operates.
A standard proxy server operates as a tunnel. It forwards your TCP packets to the destination server. The actual TLS handshake occurs end-to-end between your local machine running the scraper and the target application server. The target server sees the IP address of the proxy, but it reads the TLS fingerprint generated by your local code.
Even traffic sent through a high-trust residential proxy pool containing IPs from real consumer devices gets blocked at the network layer when your TLS handshake announces that the client is an outdated Go script. Your HTTP headers are never even evaluated, and the pristine residential IP gets burned unnecessarily.
The HTTP/2 and ALPN discrepancy
TLS fingerprinting extends beyond cipher suites. Modern browsers rely heavily on HTTP/2 for performance. During the TLS handshake, clients use ALPN (Application-Layer Protocol Negotiation) to tell the server whether they support HTTP/2 or HTTP/1.1.
Browsers default to negotiating HTTP/2. However, many legacy scraping frameworks and standard HTTP libraries still default to HTTP/1.1 or fail to implement multiplexing correctly. If your scraper negotiates HTTP/1.1 via ALPN while masquerading as a modern desktop browser, you trigger another high-confidence bot signal.
Furthermore, browsers utilize GREASE (Generate Random Extensions And Sustain Extensibility). GREASE introduces random, reserved values into the ClientHello packet to ensure servers correctly handle unrecognized parameters. Standard programming libraries do not generate GREASE values. The absence of these random extensions in a scraper TLS fingerprint is a definitive indicator of non-browser traffic.
Strategies for a TLS fingerprinting bypass
Fixing detection rates requires matching your network layer signature to your application layer headers. There are a few engineering paths to achieve this.
Headless browsers like Playwright or Puppeteer are the most straightforward approach. Because these tools launch actual instances of Chromium or WebKit, their network stack perfectly matches a real user. The tradeoff is severe infrastructure overhead. Rendering the DOM, managing memory, and executing JavaScript consumes massive compute resources, making headless browsers cost-prohibitive for high-volume data extraction.
For scaled operations, developers turn to TLS impersonation libraries. In the Go ecosystem, the utls library replaces the standard crypto/tls package. It allows developers to construct custom ClientHello packets that mimic specific browser versions down to the exact cipher suite ordering and GREASE values.
In Python and Node.js environments, impersonation wrappers like curl-impersonate are the standard solution. These tools compile a custom version of cURL linked against a modified BoringSSL library, matching the exact network behavior of major browsers. When combined with headless HTTP requests, you achieve browser-level TLS trust without the compute overhead of a rendered DOM.
Aligning network signatures with IP reputation
Successful data collection pipelines require strict alignment between your IP type, your TLS signature, and your HTTP headers. Anti-bot algorithms score these data points collectively to calculate a risk profile.
Once your scraper successfully spoofs a mobile Safari TLS fingerprint, route those requests through mobile proxies. Sending a mobile device fingerprint from a known datacenter ASN creates an architectural anomaly. The target system knows that a real iPhone connecting via 5G does not operate from an AWS data center in Virginia.
Operating at scale means managing hundreds of concurrent connections while rotating both IPs and TLS fingerprints to match. Standard open-source tools often require heavy modification to maintain high success rates without leaking memory over long sessions. If you are building infrastructure for millions of requests per day, scaling effectively often requires migrating to custom proxy infrastructure that handles IP rotation and session management at the network edge.
Audit your ClientHello packets first
Rotating IPs and randomizing user agents alone is not enough to scrape high-security targets. A scraper TLS fingerprint tells the target server exactly what software is executing the request before the server even processes your request payload. You must control your network stack just as strictly as you control your application code.
Start by auditing your current scraping stack. Inspect your outgoing ClientHello packets using network analyzers like Wireshark. Verify the JA3 hashes generated by your code against public browser databases. If your client parameters do not perfectly mirror the browser environment you claim to be, you are wasting proxy bandwidth on doomed requests.
If you need help sizing the right proxy stack for your use case, talk to our team.
