Peakhour.IO - Browser Fingerprinting

Fingerprints Are Evidence, Not Identity

2026-06-19T00:00:00+10:00

The word "fingerprint" can create the wrong expectation.

In security, a browser or network fingerprint is not the same as a human fingerprint. It does not prove who a person is. It does not remove uncertainty. It should not be treated as a permanent identity for a customer.

A fingerprint is evidence. Sometimes it is strong evidence. Sometimes it is weak, common, stale, or deliberately manipulated. Its value comes from how it is combined with route, behaviour, account state, network context, credential risk, and the action being requested.

That distinction is more than wording. It affects how security teams design controls, explain decisions, and avoid overblocking legitimate users.

What Network Fingerprints Can Tell You

Network fingerprinting compares connection and protocol evidence. TCP behaviour, TLS handshakes, JA3 or JA4-style representations, HTTP/2 settings, header shape, MTU, proxy indicators, ASN, and path characteristics can all help classify the client or infrastructure behind a request.

That can be useful during credential stuffing, scraping, scanning, API abuse, or Layer 7 attack pressure. Attackers may rotate IP addresses, but parts of the client stack or automation framework can remain consistent. Grouping requests by network evidence can make rate limiting, bot detection, and investigation more precise than IP-only rules.

But the fingerprint is still not identity.

Common browsers can share similar network shapes. Mobile networks and carrier-grade NAT can make unrelated users appear close together. VPNs and residential proxies can distort source context. Browser and library updates can change fingerprints overnight. Hashing can make signals portable while hiding useful detail. Attack tools can also try to imitate normal clients.

The right conclusion from a suspicious network fingerprint is not "we know who this is". It is "this request deserves a different level of confidence".

What Browser Fingerprints Can Add

Browser fingerprinting adds evidence from the application layer and, where appropriate, browser-side checks. Headers, client hints, JavaScript-visible properties, rendering behaviour, storage behaviour, timezone, language, permissions, and API availability can help decide whether a request looks like the browser it claims to be.

This matters because many attacks try to borrow the appearance of ordinary browser traffic. Automation frameworks, emulators, headless browsers, anti-detect browsers, and scripted API clients can all present a user-agent string that looks plausible. Browser evidence helps compare the claim with the rest of the request.

Again, the useful output is confidence, not identity. A browser fingerprint might support a challenge. It might support a lower rate limit. It might explain why a session changing an email address needs step-up verification. It might help bot management separate obvious automation from normal traffic.

It should not become a claim that one technical pattern equals one person.

The Comparison Matters

Peakhour's page on browser fingerprinting vs network fingerprinting makes the operational split clear. Network fingerprints usually come from passive connection and protocol evidence. Browser fingerprints often involve request and browser-side evidence. They answer related but different questions.

A strong decision often needs both.

A request claiming to be a normal browser should look broadly consistent across TLS, HTTP/2, headers, JavaScript-visible browser properties, proxy context, route behaviour, and account history. If the browser looks normal but the network path resembles a known automation cluster, that is useful. If the network path looks ordinary but the browser evidence is inconsistent or missing on a sensitive route, that is useful too.

The mismatch is the signal. The response still depends on consequence.

A suspicious request to a public asset route might only need logging. The same evidence on login, password reset, stored-card checkout, account email change, admin access, or an expensive API route may justify a challenge, tighter limit, temporary hold, or review.

How to Use Fingerprints Responsibly

Fingerprints work best when they are attached to an explainable decision. A security event should show the route, account or token context where relevant, source network evidence, browser evidence, policy action, response code, and review outcome. That gives operators a way to understand and correct decisions.

Peakhour's guide to network fingerprint signals and security decisions frames the choices properly: allow, log, challenge, rate limit, block, or review. A fingerprint should help choose among those actions. It should not replace judgement.

Responsible use also means accepting uncertainty. Fingerprints collide. They drift. They can be spoofed. Some privacy tools intentionally reduce or alter browser signals. Some legitimate users have unusual configurations. Some high-risk requests have only partial evidence.

That uncertainty does not make fingerprints useless. It means they should be one layer in a wider control set.

For account and API security, the practical question is not "can this fingerprint identify a person?" It is "does this evidence change the confidence we should place in this request?"

If the answer is yes, use it carefully. Increase scrutiny on sensitive actions. Reduce friction where evidence is clean. Preserve enough context for review. Avoid pretending that a technical fingerprint is a human identity.

That is the more accurate model, and it leads to better security decisions.

For a protocol-level example, see A network fingerprint is a cohort, not a client. It compares the information retained by JA3, JA4 and Cisco Mercury and explains why a matching TLS value can still cover many applications and users.

The Invisibility Cloak

2025-09-01T00:00:00+10:00

Every time you connect to a website, you leave behind a "digital fingerprint." This is not a physical fingerprint, but a set of signals from your device and browser. Security tools analyse this fingerprint—which includes your IP address, browser type, operating system, supported fonts, and even subtle characteristics of your network connection (TLS fingerprinting)—to distinguish legitimate users from malicious bots.

For years, this was a reliable way to spot automated threats. Bots often had clumsy, inconsistent fingerprints that made them easier to identify. Today, attackers can combine tools that mimic real users closely enough to weaken many traditional defences. The two most important components of this modern "invisibility cloak" are residential proxies and anti-detect browsers.

What Are Residential Proxies?

A residential proxy is an intermediary server that uses an IP address assigned by an Internet Service Provider (ISP) to a real home internet connection. When a bot routes its traffic through a residential proxy, its requests appear to originate from a genuine home user, not a data centre.

These proxy networks are large, often containing millions of IP addresses sourced from around the globe. How are these IPs obtained? Often through questionable means:

Malware and Botnets: Unsuspecting users' devices are infected with malware that turns them into proxy endpoints.
SDKs in Free Apps: Some free applications (often VPNs or mobile apps) include code that enrols the user's device into a proxy network in exchange for using the app, often without the user's full knowledge or consent.

By rotating through this large pool of legitimate-looking IPs, attackers can launch large-scale attacks that are difficult to separate from normal traffic. To a website's security system, a distributed attack from a residential proxy network looks like thousands of individual customers from different locations.

What Are Anti-Detect Browsers?

While residential proxies mask the attacker's network location, anti-detect browsers are designed to spoof the rest of the digital fingerprint. These specialised browsers allow an attacker to create and manage thousands of unique browser profiles, each with a customised and consistent fingerprint.

An anti-detect browser can control and randomise every detail a website uses for identification, including:

Browser type and version (e.g., Chrome, Firefox, Safari)
Operating system (Windows, macOS, iOS, Android)
Screen resolution, fonts, and plugins
Time zone and language settings
Subtle browser characteristics like Canvas and WebGL rendering

With a few clicks, an attacker can make a single machine in one country appear as thousands of unique users on different devices and operating systems from all over the world.

The Combined Threat: A Perfect Storm for Attacks

When attackers combine residential proxies with anti-detect browsers, they cover both the network and browser layers that many controls rely on. The residential proxy provides a legitimate IP address, and the anti-detect browser provides a consistent, human-looking browser fingerprint.

This combination makes attacks like large-scale credential stuffing, content scraping, and inventory scalping much harder to distinguish from legitimate user traffic. Each malicious request appears to be from a unique person on a standard device, using a normal home internet connection.

Why Traditional Defenses Fail and What to Do About It

This level of sophistication weakens traditional security measures:

IP Blocklists and Reputation Services: These struggle when attackers are using a constantly rotating pool of millions of legitimate residential IP addresses. Our own research shows that even the best IP intelligence services fail to detect the vast majority of residential proxy traffic.
Basic Browser Fingerprinting: Anti-detect browsers are specifically designed to defeat these checks by providing a consistent and realistic fingerprint.

To combat this combined threat, organisations need a modern approach to bot detection that looks beyond the surface:

Advanced Network Fingerprinting: Instead of just looking at the IP address, modern solutions analyse the underlying characteristics of the network connection itself (like the TLS/JA3 fingerprint). These signatures can often identify the underlying automation tool or proxy network, even when the IP address appears legitimate.
Behavioural Analysis: Advanced systems model normal user behaviour—such as mouse movements, typing speed, and page navigation—to identify the subtle, non-human patterns of automation that even sophisticated bots can't perfectly mimic.
Hardware and Rendering Fingerprinting: While anti-detect browsers can spoof software-level details, faking the underlying hardware is far more difficult. Advanced techniques, such as those used in Google's Picasso, analyse how a device renders graphics (e.g., Canvas and WebGL), processes audio, and performs CPU-intensive tasks. This creates a hardware fingerprint based on the unique characteristics of the GPU, audio stack, and CPU clock speed. This fingerprint can reveal inconsistencies between the claimed browser profile and the actual hardware being used. When combined with network fingerprinting and residential proxy detection, this becomes a strong signal for identifying a single machine attempting to impersonate many different users.
Dedicated Residential Proxy Detection: Specialised techniques are required to identify traffic coming from residential proxy networks. This is a critical signal, as very few legitimate users have a reason to route their traffic this way.

Attackers using residential proxies and anti-detect browsers are harder to identify, but they still leave signals. Network characteristics, hardware fingerprints, and the behavioural tells of automation give security teams a better chance of separating the bot from the user it is trying to resemble.

Anti-Detect Browsers

2025-01-15T10:00:00+11:00

Anti-detect browsers matter to defenders because they attack the assumptions behind browser trust. Many bot and fraud controls look for consistency between the browser, network, session, and behaviour. Anti-detect tooling is designed to make automated or repeated activity look more like separate ordinary browser sessions.

This article is not a guide to using those tools. The defensive question is simpler: when a browser tries to look ordinary, what can still be observed safely, and how should that evidence affect a request decision?

Why They Create Risk

Anti-detect browsers are often discussed as a browser-fingerprinting problem, but the risk is wider than that. The same traffic may also involve residential proxies, credential lists, automation, and API requests that never run browser-side checks. A login attempt, account creation flow, product scrape, checkout request, or mobile API call may look valid at the protocol level while still being part of an automated campaign.

The hard part is that some signals are genuinely ambiguous. A changed browser, a new device, a shared network, or a privacy tool does not prove abuse. A defensive system has to treat those observations as risk inputs, not as standalone verdicts.

Browser Consistency Is Evidence, Not Identity

Anti-detect tooling tries to make browser-reported attributes look internally consistent. That weakens simple checks that only ask whether the browser appears plausible. Defenders need a broader view: does the claimed browser line up with the network stack, TLS and HTTP behaviour, session history, cookie continuity, route sequence, response-code pattern, and recent account behaviour?

That does not mean a fingerprint identifies a person. Fingerprints classify software, client behaviour, and connection characteristics. They can help separate likely automation from ordinary traffic, but they need to be combined with route, account, proxy, and behavioural context. The result should be a risk classification with evidence attached, not an unexplained block.

Residential Proxies Change the Decision

Residential proxies are a common companion signal because they make requests appear to come from consumer networks. That creates a false-positive problem. Real customers also use shared residential, mobile, office, carrier-grade NAT, and public Wi-Fi networks. Blocking every suspicious or shared source would damage legitimate traffic.

The safer approach is to use residential proxy detection as one input in the decision. A proxy signal on a public content page may be logged. The same signal on repeated login failures, account creation, checkout abuse, or sensitive APIs may justify a challenge, rate limit, or block. Context changes the action.

The API Gap

Browser-side checks are weakest where the browser is not present. Mobile apps, partner integrations, token routes, and direct API clients may not expose the same JavaScript or browser evidence that a web page does. Attackers do not need a convincing browser if the target workflow accepts valid-looking API requests.

That is why anti-detect risk belongs in the wider application security model. API routes need method, schema, authentication, token, request cadence, response-code, account, and bot context. If the only signal available is an IP address, the decision will be too blunt.

Observable Signals Defenders Can Use

The useful evidence is usually the mismatch between what the request claims to be and how it behaves over time. A browser may look plausible on one request, but the wider pattern can still show automation: repeated attempts across accounts, route sequences that normal users do not follow, cache-miss pressure on expensive pages, unusual response-code loops, or browser and network characteristics that drift in ways ordinary clients rarely do.

Bot Management works best when it combines these signals rather than chasing a single magic detector. IP intelligence, proxy classification, network and browser fingerprints, route-aware rates, API state, WAF findings, and behaviour should all feed the same action vocabulary: allow, challenge, rate limit, block, log, or review.

Safer Defensive Response

The defensive response should be proportionate. High-confidence exploit traffic can be blocked quickly. Uncertain browser or proxy evidence may be better challenged, rate limited, or logged until the pattern is clearer. Sensitive routes should have tighter policy than public content. Account-impacting actions should preserve enough evidence for review.

This is especially important for support teams. If a real customer is challenged or blocked, operators need to see which signal drove the action and which route was involved. Without that record, anti-bot policy becomes a black box.

Final Thoughts

Anti-detect browsers are a practical problem because they reduce the value of simple browser checks. They do not make traffic invisible. They leave request-path evidence in network behaviour, route sequences, account activity, API usage, proxy signals, and response patterns.

The right goal is not to identify a person from a fingerprint or to block every unusual browser. The goal is to classify risk with enough context to choose a safe action at the edge, then keep the evidence available for tuning and review.

JA4 and JA4+ Network Fingerprinting

2023-10-25T13:00:00+11:00

JA4+ is the name FoxIO uses for a family of network fingerprinting methods. JA4 itself is the TLS ClientHello method. It builds on lessons from JA3, but the wider family also contains separate methods for servers, HTTP, certificates, TCP, SSH and other observations.

JA4 and JA4+

JA4 produces an a_b_c value. Its readable a section records selected connection properties and counts. The b and c sections are truncated SHA-256 values derived from normalised ClientHello fields. Analysts can compare selected components, such as JA4_ac, when the complete fingerprint is too narrow for the question being asked. Other JA4+ methods have their own inputs and specifications; they should not be treated as extra fields inside core JA4.

JA4+ consists of various components:

JA4: TLS Client
JA4S: TLS Server Response
JA4H: HTTP Client
JA4L: Light Distance/Location
JA4X: X509 TLS Certificate
JA4SSH: SSH Traffic

For a more thorough breakdown, the JA4 blog provides the announcement and describes the fingerprints.

JA4+ brings useful improvements, but a few aspects and quirks deserve closer attention.

What sorting changes

JA4 sorts cipher identifiers and most extension identifiers before hashing them. This was especially useful after Chrome began permuting TLS extension order. Sorting puts those permutations back into one cohort. It also discards the order as evidence. That is the trade-off: a more stable identifier retains less information about how the ClientHello was serialised.

Where investigation matters, retain the raw JA4 form as well as the compact value. JA4_r exposes the normalised cipher, extension and signature-algorithm lists, which makes a difference easier to inspect.

The overview of TLS fingerprinting provides a more in-depth explanation of how a TLS signature is formed.

Chrome's change was intended to stop servers and middleboxes from depending on one fixed extension order. In our extension-randomisation analysis, the number of order-sensitive TLS fingerprints rose sharply after the rollout. Sorting reduced that artificial fragmentation. It did not make the resulting value a client identity, and it did not preserve every distinction in the original handshake.

JA3 and Mercury took different paths

Before digging further into JA4+'s features and limitations, it helps to separate two related lineages. The original JA3 established a portable TLS fingerprint that was easy to share and match. Cisco Mercury developed a richer protocol representation and a separate destination-context classification system. Mercury is not a predecessor in the JA3-to-JA4 naming line. Our history of the two lineages explains where their work overlaps and where it does not.

Implementation differences still matter

While sharing signatures through SHA is appealing, it has limits, most notably potential compatibility issues. As Fastly noted, differences in the implementation can be hidden behind the SHA hash, causing issues when searching for and correlating signatures between different services. Record the implementation and version that generated a value; a shared format name does not prove that two sensors handled every field identically.

Check the method, implementation and licence

The official JA4+ repository contains the current specifications and implementations. Check the licence for the individual method before adopting it: core JA4 is BSD-3-Clause, while most other JA4+ methods use the FoxIO Licence and place additional conditions on commercial use.

For a field-level example rather than a format summary, our same-ClientHello lab records JA3, JA4, JA4_r and Mercury NPF output from one packet and pins the implementations that generated them.

Web scraping another Business' website

2023-10-11T13:00:00+11:00

As businesses continue to build their presence online, screen scraping is becoming more prevalent. Screen scraping is the use of software or code to take data from another website. For example, popular platforms like Skyscanner or booking.com usually take price data on flights and accommodation and display it on their websites. However, Australian copyright laws or the website owner’s terms and conditions may forbid you from screen scraping. This article explains the legal aspects of scraping data from another business’ website and the precautions you should take.

Am I Violating the Law by Screen Scraping?

Australian copyright law safeguards ‘original creative works’, including:

written works;
visual images;
music; and
moving images.

Copyright can also protect documents such as government reports and legal forms. When determining whether copyright protects a creative work, the work does not need to be intricate or of high quality. It only needs to demonstrate originality and not be copied from another source.

Is Data an ‘Original Work’?

Data is usually fact-based and primarily consists of statistics or numbers. As a result, copyright usually does not protect data.

Examples of such data include:

the consumer price index for a particular quarter;
monthly house price increases in a city;
the number of students in a class; or
the count of films released in a year.

Generally, the law does not consider this an original work because it merely represents real-world information.

What Data is an ‘Original Work’?

However, data can be an original work in some circumstances. For example, if you organise data in a unique manner that reflects someone's creativity, the law might consider that data an ‘original work’.

Examples of organised data that copyright protects include;

accounting forms;
sequences of numbers or letters for a bingo game; or
a car parts catalogue.

Consequently, screen scraping data from a website is unlikely to infringe copyright unless it involves protected, creatively organised data. Infringing someone’s copyright means using their copyright-protected material without their permission.

Are There Exceptions to Copyright Law?

In the rare event that your screen scraping infringes copyright, your use could fall under an exception to copyright infringement. Australian copyright law refers to these exceptions as 'fair dealing.'

The four ‘fair dealing’ exceptions include using copyright-protected materials for:

research or study;
review or critique;
parody or satire; and
reporting the news.

For instance, a journalist scraping original data sets to report potential price-gouging among airlines could potentially rely on the exception for reporting the news. However, if you are scraping data for business purposes, the fair dealing exceptions may not apply.

What if a Website Explicitly Bans Screen Scraping?

Even if screen scraping is not always illegal under Australian copyright law, website owners can use their terms of use to prohibit data scraping. These terms of use often appear as website pop-ups. The pop-ups typically state that by continuing to use the website, you accept the terms of use.

These terms can explicitly forbid:

data scraping;
copying;
hacking; or
any form of data extraction.

Violating these terms would result in you breaching the website’s terms of use. As a result, the website owner may take legal action against you. If the data on the website qualifies as original work, copyright infringement claims may also arise.

Therefore, it is advisable not to screen scrape from websites with explicit terms of use against that activity. If you do engage in screen scraping, ensure you only extract factual information.

Key Takeaways

Screen scraping is generally lawful if you extract strictly factual information from other websites. However, if a website's terms of use prohibit screen scraping, even for factual data, it is advisable to avoid data scraping. Otherwise, you could face potential breach of contract and copyright infringement claims.

For assistance with your legal obligations, LegalVision’s experienced IT lawyers can assist as part of their membership. For a low monthly fee, you will have unlimited access to lawyers who can answer your questions and draft and review your documents. Call LegalVision today on 1800 296 912 or visit their membership page.

Interaction to Next Paint (INP)

2023-09-11T13:00:00+10:00

Google has announced that Interaction to Next Paint (INP) will replace First Input Delay (FID) as a Core Web Vital as of March 2024. Introduced as a metric in 2022, INP covers gaps in FID by measuring more of what happens after a user interacts with a page.

To help site owners prepare for its introduction as a Core Web Vital, INP is already included in the Chrome User Experience Report (CrUX). By analysing the CrUX data, website owners can see their current INP performance and make targeted optimisations ahead of the March 2024 change.

A better metric than First Input Delay

First Input Delay, as its name suggests, only measures the delay between an input, such as a keypress or mouse click, and the point where the browser begins to handle that event. It does not include the time spent processing the input. It only measures how long the browser was blocked before it could start handling it.

That leaves two issues: it only considers the FIRST event, and it does not measure how long it takes for the user to see the result of their input.

INP is designed to cover both issues. It measures the latency of ALL 'interactions' through to the visual response for that interaction. As explained by Google, an interaction like a tap on a touch screen device can consist of several input events.

"An interaction's latency consists of the single longest duration of a group of event handlers that drives the interaction, from the time the user begins the interaction to the moment the next frame is presented with visual feedback."

After measuring all interactions, the final INP score is the longest interaction observed, ignoring any outliers.

What Constitutes a Good Score

INP is measured in milliseconds (ms), with lower scores indicating better performance:

Good: < 200 ms
Needs Improvement: 200-500 ms
Poor: > 500 ms

How to ensure you have a good INP score

Minimise Main-Thread Work

Long-running JavaScript can block the main thread and increase INP times. Break these tasks into smaller parts and run them asynchronously to reduce delays.

Efficiently Use Browser APIs

APIs that trigger layout recalculations can be expensive. Use them sparingly and look for alternatives that put less pressure on the browser.

Defer Non-Essential CSS and Scripts

Postpone the loading of non-critical CSS and JavaScript. Use techniques like asynchronous loading to improve INP scores.

Monitor Third-Party Scripts

Heavy third-party scripts can degrade INP performance. Use asynchronous or deferred loading for these scripts to limit their impact.

Conclusion

Google estimates that 90% of a user's time on a page is after it has finished loading. FID focused on first impressions, with the assumption that a fast start meant the page would stay responsive. Interaction to Next Paint addresses that gap and gives a more accurate view of user experience. If you want to know your current INP score, you can use our free website speed comparison tool to view it alongside your other Web Vitals, and see how your website compares to your competitors.

Down But Not Out - JXL Will Return on Safari

2023-06-04T00:00:00+10:00

Just as we were coming to terms with the controversial decision by Google to drop support for JPEG-XL (JXL) in Chrome, Apple announced support for JXL during the WWDC June 5th livestream. That is a meaningful shift. JXL was down, but not out.

Google's decision to stop JXL support in Chrome surprised us at Peakhour, along with plenty of others who care about web performance and image delivery. Google Chrome, as the most used browser globally, often sets the course for web standards. In deciding to drop JXL, Google appeared to be exercising its dominance over those standards, and the decision drew real debate in the web community.

Apple's announcement changes the picture. Apple has long pushed high dynamic colour and high-resolution features, and Safari support is a useful signal for image delivery. By bringing JXL support to Safari, Apple is giving this promising image format a fair go.

This move also hints at wider JXL support across the entire Apple ecosystem, which includes iPad, iPhone, Mac, and Apple TV. While there are still some limitations - embedded colour profiles and animations are not yet supported in the current MacOS Sonoma beta - we hope these gaps are fixed soon.

At Peakhour, this is good news. We look forward to welcoming Apple users to our websites, where they will be able to see the quality benefits of JXL images as soon as their operating systems support it.

This turn of events gives JXL a much-needed boost. It does not undo Google's Chrome decision, but it keeps the format in play and makes the future of web image formats less settled than it looked a short while ago.

Chrome's TLS Extension Randomisation Experiment

2023-02-02T13:00:00+11:00

Transport Layer Security (TLS) fingerprinting is a commonly used technique for identifying client processes. To reduce the risk of server and middlebox fingerprinting of Chrome's current ClientHello and to make the TLS ecosystem more resilient to changes, Google Chrome ran an experiment to randomise a portion of the TLS fingerprint. This experiment was included in Chrome version 108, which was released on December 8, 2022. You can read the status of the current experiment on the chrome status site

The aim of this experiment was to make it more difficult for server implementers to fingerprint Chrome and assume specific implementation behaviour from a fixed extension order. By randomly ordering extensions (subject to the pre_shared_key constraint in the RFC), Chrome hoped to reduce the risk of server and middlebox fixating on details of its current ClientHello.

The above graph correlates to the Chrome experiment and subsequent release of the feature. The number of unique TLS signatures dramatically increased.

From Peakhour data, we can see a large number of unique fingerprints appearing since the date of the experiment, making it very difficult to identify the Chrome network stack by a TLS fingerprint alone. However, an analysis by David McGrew, a Cisco Fellow, cast doubt on the effectiveness of this experiment. In his article, McGrew proposed a lexicographical sorting of TLS extensions and found that 98.8% of signatures were unique after sorting. He argues that the canonical ordering of the TLS extensions in the TLS fingerprint can achieve nearly the same level of entropy as randomising them and still be effective at client identification. Furthermore, he claims that the RFC should be amended to state that extensions SHOULD be sent in an ordered fashion in the ClientHello packet. McGrew also highlights the potential dangers of allowing unordered extension lists, as it could create a \"subliminal channel\" that could be used for tracking or transmitting information. Let's now graph, over the same period, the number of TLS signatures with TLS extension sorting.

The above graph correlates to David's assertion that sorting TLS extensions has minimal impact on TLS fingerprinting.

It appears that David's assertion is correct: sorting TLS extensions has minimal impact on the number of unique TLS fingerprints. Let's now look at in-the-wild Chrome 109 data:

Hashed sorted TLS Fingerprint	Unique unsorted TLS fingerprints	Browser	Version	% of clients	% of hits
3241796329	2	Chrome Mobile WebView	109	6.14	1.01
3313291307	8566	Chrome	109	1.83	1.54
1537819294	26587	Chrome Mobile	109	6.05	4.64
3944870384	1	Chrome Mobile iOS	109	4.31	5.35
3241796329	51594	Chrome Mobile	109	16.9	14.43
1537819294	121346	Chrome	109	15.66	20.7
3241796329	156405	Chrome	109	35.52	37.79

It's interesting that the experiment does not run on WebView.

Final Thoughts

While Chrome's experiment may have reduced the risk of server and middlebox fingerprinting of Chrome's current ClientHello, it seems that randomising TLS extensions alone is not enough to prevent TLS fingerprinting, and may be a useful indicator that it is The Real Chrome.

This experiment became one of the reasons newer formats normalise extension order. Our JA3, JA4 and Mercury lab shows exactly where each format keeps, sorts or discards ClientHello detail. The accompanying history of the two TLS fingerprinting lineages explains why Cisco Mercury and JA4 made related but different design choices.

The remaining research question is whether discarded order ever helps distinguish an imitator or evasive client. Does TLS fingerprint canonicalisation hide useful attacker variation? defines the labelled corpus and holdout study needed to answer it without confusing uniqueness with detection accuracy.

TLS Fingerprinting

2023-02-02T13:00:00+11:00

What is Fingerprinting?

Fingerprinting is a technique that may be used to identify the specific device, web browser, and operating system making a request, regardless of what the client says in its user-agent header. By helping organisations identify and characterise the attributes of a client's connection, fingerprinting can improve network security and help protect against malicious traffic.

Fingerprinting can also refer to techniques for following or uniquely identifying individual users across the web. That is a separate set of techniques and is not discussed in this article.

Transport Layer Security (TLS) Fingerprinting determines the specific characteristics of a client's TLS implementation by examining the initial TLS handshake packet, known as the "Client Hello." This packet contains fields and parameters such as supported cipher suites, extensions, and the client's preferred order of those parameters, which can be used to create a unique "fingerprint" of the client's TLS implementation.

Why is it used?

Fingerprinting has several uses, including bot protection, DDoS protection, and client identification. By identifying and characterising the attributes of a client's connection, fingerprinting can improve network security and help protect against malicious traffic.

How does TLS Fingerprinting work?

TLS Fingerprinting examines the initial TLS handshake packet, known as the "Client Hello". The Client Hello packet is sent by the client during the initial phase of the TLS handshake, which establishes a secure connection between the client and the server. It contains information about the client's preferred encryption methods, extensions, and parameters, including:

Protocol Version: The version of the TLS protocol desired by the client.
Random: A 32-byte random value generated by the client, used in key generation and derivation.
Session ID: An optional session identifier for resuming a previous session.
Cipher Suites: A list of supported encryption algorithms, ordered by preference.
Compression Methods: A list of supported compression algorithms, ordered by preference.
Extensions: Optional extensions that can negotiate additional parameters, such as Server Name Indication (SNI) and Elliptic Curve Supported (ECS).

The Client Hello packet is central to the operation and security of the TLS connection because it provides information the server uses to select encryption algorithms and parameters. The packet also enables the client and server to negotiate an appropriate encryption method for their communication. The Client Hello's variable content, based on the TLS version, library, cipher suites, extensions, and settings supported by the client, makes it a strong candidate for fingerprinting.

Common components used to create a TLS fingerprint include:

Cipher Suites: The order of cipher suites supported by the client.
Extensions: Supported extensions included in the Client Hello packet, such as SNI and ECS.
TLS Point Formats: Encoding of cryptographic parameters in a format that can be transmitted as part of the TLS protocol, used in elliptic curve cryptography (ECC).
TLS Curves: The specific elliptic curves used in ECC, a type of public-key cryptography used in the TLS protocol.

TLS fingerprinting has been a topic of research for several years, with a number of tools and techniques developed from that work. Notable examples include JA3, developed by John Althouse, Jeff Atkinson, and Josh Atkins of Salesforce, which uses a hash of the client's SSL/TLS parameters as a unique identifier for tracking and analysing SSL/TLS traffic. Another tool, Mercury by David McGrew and Blake Anderson, can be used to fingerprint client connections and identify the device, operating system, and application making the connection.

TLS fingerprinting has a variety of uses, including bot protection, DDoS protection, malware identification and client identification. By enabling organisations to identify and characterise the attributes of a client's TLS implementation, TLS fingerprinting can improve network security and help protect against malicious traffic.

In production, TLS fingerprints are most useful when combined with IP intelligence and residential proxy detection, rather than treated as a standalone verdict.

Representation of a TLS Fingerprint

A TLS fingerprint is commonly represented as a string or hash that summarises the important components of the Client Hello packet. The most common components used to create a TLS fingerprint include the supported cipher suites, extensions, and TLS point formats. The cipher suites are represented as a list of hexadecimal values in the order they are presented by the client, while extensions and point formats are represented as a list of hexadecimal values or a unique identifier.

Raw JA3 signatures are represented by the following fields, which are then hashed with MD5:

SSLVersion, Cipher, SSLExtension, EllipticCurve, EllipticCurvePointFormat

An example raw signature is:

 771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-34-51-43-13-45-28-21,29-23-24-25-256-257,0

An MD5 hash is then applied, resulting in the final signature.

579ccef312d18482fc42e2b822ca2430

Mercury signatures are represented by:

"tls/1" (TLS_Version) (TLS_Ciphersuite) [ Extension* ]

An example signature is:

tls/1/
(0303)
(130213031301c02cc030009fcca9cca8ccaac02bc02f009ec024c028006bc023c0270067c00ac0140039c009c0130033009d009c003d003c0035002f00ff)
[
   (0000)
   (000a000c000a001d0017001e00190018)
   (000b000403000102)
   (000d0030002e040305030603080708080809080a080b080408050806040105010601030302030301020103020202040205020602)
   (0016)
   (0017)
   (0023)
   (002b0009080304030303020301)
   (002d00020101)
   (0033)
]

Hash Functions for Representing TLS Fingerprints

Hashing algorithms, such as MD5, are commonly used to create a unique representation of a TLS fingerprint. These hash functions take the client's TLS parameters as input and produce a fixed-length output, which serves as a unique identifier for the client. The hash value can be compared against a database of known TLS fingerprints to help determine the identity of the client.

Other techniques for representing TLS fingerprints include base64 encoding of the client's TLS parameters, such as in the Mercury fingerprint.

Challenges with TLS fingerprinting

TLS fingerprinting is not a foolproof method for identifying clients and their attributes. It has several limitations that need to be considered.

False Positives: TLS fingerprinting relies on the assumption that the client's Client Hello packet uniquely identifies a connecting process by its TLS implementation. However, it is possible for a client to alter the Client Hello packet by customising TLS parameters, which affects the Client Hello packet and can result in a false positive identification. This makes it important to use multiple methods for identifying clients. For example, Mercury takes into account destination ports to add additional context.
False Negatives: While TLS fingerprinting can identify many different clients and their attributes, it is not capable of identifying all clients. Some clients may have a unique or unusual TLS implementation that cannot be accurately fingerprinted. Additionally, some clients may actively attempt to evade fingerprinting by customising TLS parameters or using tools to anonymise their connections.
Forging of TLS Fingerprints: It is possible for attackers to deliberately forge or modify the information contained in their Client Hello packet to appear as a different client. This makes it difficult for fingerprinting tools to accurately identify the true identity of a client and can be used for malicious purposes, such as evading security measures or disguising the origin of an attack.
Incomplete Data: TLS fingerprinting is limited by the information contained in the Client Hello packet, which may not contain all of the necessary data to accurately identify a client. For example, a client may not send a full list of supported cipher suites or extensions, may use a modified version of the TLS protocol that is not recognised by the fingerprinting tool, or the fingerprint may not be present in available databases.

Different fingerprinting implementations can result in different hashes for the same TLS connection, even though the underlying SSL/TLS protocol remains unchanged. This happens due to the various algorithms, parameters, and representations used by different fingerprinting tools.

For instance, implementation differences when generating the TLS fingerprint may cause hashes found in public databases to be inconsistent with a locally generated hash.

Final Thoughts

Be aware of the limitations and differences between fingerprinting implementations, and choose the right tool and representation for your specific use case. Standardising the representation of fingerprints and using common hash algorithms can help avoid confusion and improve interoperability between databases.

What is the Chrome UX Report (CrUX), and why should you care?

2021-02-26T13:00:00+11:00

A faster website is better for clients: they buy more, and they engage more with your content. However there's someone else that rewards fast websites: Google. Fast websites rank higher in organic search results than slower websites. They will also achieve higher quality scores in Google Ads, resulting in lower ad spend.

We've previously written about Google's Web Vitals. The practical question is where the real-world data comes from. It is not gathered by Googlebot. Google uses field data from the Chrome User Experience Report, usually shortened to CrUX, to show how eligible Chrome users actually experienced a page or origin.

That matters because a fast lab score is not the same thing as a fast customer experience. WebPageTest and Lighthouse help diagnose a controlled test run. CrUX shows the field data behind PageSpeed Insights, Search Console's Core Web Vitals report, and Peakhour's website speed comparison tool.

Introducing the Chrome UX Report (CRuX)

The CrUX report is a public data set of real-user measurements (RUM) of website performance across millions of sites. The report has been around since 2017 and is updated regularly, but the value is still often missed: it shows what real users experienced, not what a synthetic test predicted.

The data is collected from real Chrome browser users who have opted in to send browsing information back to Google. This opt-in requires that the user has:

Opted in to syncing browser history
Not set up a sync passphrase
Usage statistic reporting enabled

Despite these conditions, millions of Chrome users still report statistics back to Google. A given website still needs to be fairly busy before there are useful statistics in the report.

Gathered Metrics

The current Core Web Vitals are:

Largest Contentful Paint (LCP): how quickly the main content appears.
Interaction to Next Paint (INP): how responsive the page is to real user interactions.
Cumulative Layout Shift (CLS): how visually stable the page is while it loads.

CrUX also exposes supporting performance metrics and dimensions that help explain those headline scores. Older metrics such as First Input Delay (FID) still appear in older reports and tools, but INP is now the responsiveness metric to watch.

Dimensions

Because performance can vary widely, the metrics are divided into the following dimensions to help segment and understand the user experience.

Country
Device Type: Tablet, Phone, Desktop
Connection Speed: slow 2g, 2g, 3g, 4g, or offline

Viewing data in the report

There are several ways to see how your website performs in the report. These include:

Pagespeed insights

Google's website analysis tool provides summary CRuX data for the analysed URL and, if data is available, for the entire site.

Page Speed Insights Field Data" style="max-width: 100%;margin-bottom: 20px"/>

Google BigQuery

The most flexible option is to access it directly via BigQuery. You query it with SQL (database query language).

The downside is that you need to understand SQL and have a Google account.

Google's Search Console (formerly Webmaster Tools)

The search console now has a section 'Core Web Vitals' that shows whether URLs pass the Core Web Vitals, as well as a historical graph of performance for both mobile and desktop.

Looker Studio

Looker Studio, formerly Google Data Studio, can be used to build dashboards on top of CrUX data and other sources. It lets you visualise the performance of your website, or a competitor's website, over time.

Third party tools

Like our own website speed comparison tool. It uses the Chrome UX API to retrieve the information.

CrUX vs lab tools vs RUM

CrUX is field data. It is useful because it reflects real Chrome users, but it only reports where there is enough eligible traffic. It can also be slower to reveal the cause of a problem because it is aggregated.

Lab tools such as WebPageTest are better for diagnosis. They show waterfalls, redirects, blocked resources, caching issues, image weight, and third-party requests. Your own real user monitoring can go further again, because it can include business context that CrUX does not know: customer type, template, campaign, cache state, bot pressure, origin load, and release timing.

Conclusion - Why you should care

The data in the Chrome UX Report is one of the clearest public views of how Google sees the performance of your website. It is also a free source of real-world user measurements that helps you understand how visitors experience your pages.

Use CrUX to see whether your site is passing Core Web Vitals, use lab tools to find the technical cause, and use traffic visibility to understand whether bots, crawlers, bursts, cache misses, or expensive requests are putting the experience under pressure. If the first question is "how do we compare?", start with the website speed comparison tool. If the question is "what is slowing us down?", start with Traffic Management.

Test Your Website Performance With Google Lighthouse

2020-09-14T13:00:00+10:00

Today we're introducing our other favourite tool for testing website performance: Google Lighthouse (from here we'll just call it Lighthouse). Lighthouse measures page experience across accessibility, performance, SEO, and Progressive Web Apps for desktop and mobile devices.

Lighthouse is the engine behind web.dev/measure and PageSpeed Insights. It is also available in Chrome DevTools, via npm, or as a browser extension in Chrome and Firefox. At time of writing Lighthouse is up to version 6, which introduced Web Vitals as the basis for scoring. If you are unsure which version of the tool you are using, then scroll right to the bottom of the report it generates where it will state the version.

Lighthouse generates its report by simulating a specific device and network speed, rather than running at the full speed of your computer. That matters because speed issues are more noticeable on slower devices, and users are not all on newer devices or fast internet connections. You should test for a good load experience across that range. The current simulated mobile device is a Moto G4 on a ~1.5 megabit connection. For reference, it would take over 5s to download 1mb of data at this speed. If your page weight is typical, ie over 2.5mb, you should not expect a strong score.

How to use Google Lighthouse

There are two ways you can run a Google Lighthouse report:

In your local browser;
Online via web.dev/measure or PageSpeed Insights.

We recommend running Google Lighthouse in your local browser because the online versions operate out of the US. If your website and customers are elsewhere, that extra network latency can pull the score down.

Here we'll focus on running Lighthouse from within Chrome DevTools. To do this, click on the three vertical dots in the top right-hand corner, then select 'More Tools', then 'Developer Tools'.

The developer tools will then be displayed. Along the top of the tools window are a number of tabs. Select the 'Lighthouse' tab.

Since we're only interested in performance, make sure only the Performance category is ticked. You also want to make sure 'Clear storage' is ticked (in the top left) to simulate a first load of your site. Finally, choose the device you want to report on, mobile or desktop, and click 'Generate report'.

While the report is being generated, avoid doing anything else on your computer, and don't leave it busy with background tasks. Otherwise, the score can be affected.

Understanding the Score

Once the report has finished you'll see a performance summary, like this mobile one we ran on Peakhour.io while we were developing the website:

Scoring in version 6 is based on Google's web vitals, which are metrics that indicate a good user experience, and Webpagetest's speed index, which measures visual loading performance. Each metric is colour coded as good, ok, or bad. If the measurement is good you get a green circle to the left, if it's ok you get an orange square, if it is bad you get a red triangle.

Each raw metric (the number listed in the report) is compared to real website performance data sourced from the HTTP archive and converted into a score out of 100. This is done by grading the reference data on a curve, so if your website performs in the top 8% of websites, it gets a score of 90. Similarly, if it scores in the top 25%, it gets a score of 50. If you are interested in the technical details, Google has in-depth explanations of the scoring at web.dev

Each metric is assigned a weight, and the scores are tallied into one overall number based on the weighting. Here is a breakdown of the test we just ran (in the screen shot above), obtained by clicking on the 'See Calculator' link between the Metrics section and the screen shots:

Score Variability

79 is a good result for a mobile device. However, we ran it several times and obtained scores between 60 on the low end and 85 on the high end. Scores can fluctuate widely, even when testing on the same device repeatedly. Reasons for this include:

Small differences in internet performance.
Your computer CPU load when performing the test.
Web server variability.

Again, Google has in-depth documentation around what might be causing this.

Opportunities

If your site loads slowly then Lighthouse will list addressable reasons in the opportunities section of the report.

This is the part we generally find most useful. It identifies items that slow the initial load and items that affect the rendering of a website once it is downloaded. A page can be downloaded very quickly, but the end user still sees a slow site because CSS and Javascript are blocking rendering. This is a common problem in Wordpress and Magento themes. These themes include large amounts of third party code that ultimately never gets used for a particular site, but which the browser still has to download and parse before it can display anything.

In this case, the main bottleneck appears to be a font loaded from Google Fonts. This is delaying the rendering of our page by 1.3s.

Diagnostics

The diagnostics section provides additional information you can use to improve load times.

Here we have a few small problems, mainly associated with the development status of our site.

web.dev and PageSpeed Insights

If you do choose to run your report online, we recommend using PageSpeed Insights.

In addition to the generated report, PageSpeed Insights also shows you Field Data for the page you are testing, and an Origin Summary for all pages on the website.

This is real-world data gathered from Chrome users who have opted in to allowing Google to gather their data. If your site isn't very busy then Google might not have any data to share. It is not truly representative, but it is useful information and can often tell a very different story to your Lighthouse score.

Summary

Regular measurement matters because performance issues are often specific and easy to miss. Lighthouse is useful for identifying issues that affect website performance, and when used in conjunction with Webpagetest.org, you'll be in a better position to provide a good experience for your users. Next we'll cover common issues that can impact site speed, so read on.