<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"><title>Peakhour.IO - TLS Fingerprinting</title><link href="https://www.peakhour.io/" rel="alternate"></link><link href="https://www.peakhour.io/feeds/tag/tls-fingerprinting.atom.xml" rel="self"></link><id>https://www.peakhour.io/</id><updated>2026-09-06T09:00:00+10:00</updated><entry><title>What an Open Network Fingerprint Database Should Publish</title><link href="https://www.peakhour.io/blog/open-network-fingerprint-database-schema/" rel="alternate"></link><published>2026-09-06T09:00:00+10:00</published><updated>2026-09-06T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-09-06:/blog/open-network-fingerprint-database-schema/</id><summary type="html">&lt;p&gt;A useful open fingerprint database needs provenance, competing labels, raw evidence, format versions and licences—not another unexplained hash list.&lt;/p&gt;</summary><content type="html">&lt;p&gt;An open fingerprint database should let another researcher disagree with it.&lt;/p&gt;
&lt;p&gt;That requires more than a hash and an application name. The row needs to show which bytes produced the fingerprint, how the label was established, where the traffic was observed, which software generated the value and what competing explanations remain plausible.&lt;/p&gt;
&lt;p&gt;Most public databases were built for a narrower purpose. Historical JA3 lists made values easy to share. Threat feeds made malware observations easy to match. Nmap and p0f made signature rules executable. TLS observatories count what appears at their vantage points. Those are sensible designs for their jobs.&lt;/p&gt;
&lt;p&gt;They do not yet add up to a reusable open ground-truth corpus.&lt;/p&gt;
&lt;h2&gt;Start with observations, not verdicts&lt;/h2&gt;
&lt;p&gt;The fundamental database object should be an observation:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;observation_id&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;observed_at&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2026-07-12T00:00:00Z&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;capture&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;position&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;client-facing edge&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;collector&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;mercury&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;collector_revision&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;3172786...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;source&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;controlled-client-run&amp;quot;&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;fingerprints&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[],&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;labels&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[],&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;evidence&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[],&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;licence&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The observation can then carry several fingerprint methods and several candidate labels. This is safer than making the hash the primary truth and forcing one application name into the same row.&lt;/p&gt;
&lt;h2&gt;Store the reversible material&lt;/h2&gt;
&lt;p&gt;Each fingerprint entry should contain the published identifier and the material used to derive it:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;method&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;ja4&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;implementation&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;FoxIO Python&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;implementation_revision&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;value&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;t12d2709h2_..._...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;raw_value&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;t12d2709h2_002f,...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;source_message_digest&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;sha256:...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;truncated&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;For JA3, retain the five-part source string beside the MD5. For JA4, retain &lt;code&gt;JA4_r&lt;/code&gt; where collection policy permits it. For Mercury, retain the complete versioned NPF string rather than only its hash nickname. For HASSH, retain the algorithm lists. For a matcher rule, retain the response bytes or sanitised fixture that satisfied it.&lt;/p&gt;
&lt;p&gt;This permits field-level comparison, implementation testing and migration when a definition changes. It also exposes incompatible values hidden behind a common field name.&lt;/p&gt;
&lt;h2&gt;Make labels many-to-many&lt;/h2&gt;
&lt;p&gt;Labels should be separate evidence-backed assertions:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;type&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;process&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;value&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;example-client&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;version&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;1.2.3&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;source&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;endpoint-telemetry-join&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;review_state&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;reviewed&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;confidence&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.94&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;first_seen&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;last_seen&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;One fingerprint can have several process labels because applications share libraries. One process can have several fingerprints because versions, platforms and configurations differ. The schema should preserve counts and conflict instead of selecting one winner during ingestion.&lt;/p&gt;
&lt;p&gt;Cisco Mercury's public &lt;a href="https://github.com/cisco/mercury/blob/main/doc/resources.md"&gt;resource schema&lt;/a&gt; points in this direction: fingerprint entries contain candidate processes, observation counts, operating systems and destinations. The production database is private, but the many-to-many model is the right starting point.&lt;/p&gt;
&lt;h2&gt;Describe how ground truth was produced&lt;/h2&gt;
&lt;p&gt;Use a controlled vocabulary for evidence sources:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;controlled_capture
endpoint_process_join
malware_sandbox
reviewed_pcap
active_probe_match
passive_observation
community_submission
derived_from_external_database
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Each type needs its own metadata. An endpoint join should record the join window and ambiguity rules. A controlled capture should name the client build, operating system and generation script. A malware sandbox label should identify the sample and distinguish the sandbox process from the malware process. A community submission should identify what a reviewer actually checked.&lt;/p&gt;
&lt;p&gt;Derived labels should never masquerade as independent evidence. If a mapping was imported from an older JA3 list, cite that row and keep its original uncertainty.&lt;/p&gt;
&lt;h2&gt;Separate prevalence from classification&lt;/h2&gt;
&lt;p&gt;Prevalence belongs in observation aggregates:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;source&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;university-vantage-a&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;first_seen&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;last_seen&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;count&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;184233&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;It should not silently increase application-label confidence. A common fingerprint is not necessarily well-labelled; a rare fingerprint is not necessarily suspicious.&lt;/p&gt;
&lt;p&gt;The &lt;a href="https://tlsfingerprint.io/"&gt;TLS Fingerprint Observatory&lt;/a&gt; demonstrates the value of publishing counts even when labels are missing. Its large unlabelled TLS corpus is more useful for prevalence research than a catalogue filled with guesses.&lt;/p&gt;
&lt;h2&gt;Represent threat observations explicitly&lt;/h2&gt;
&lt;p&gt;A malware feed should attach an observation, not overwrite the fingerprint's identity:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;type&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;malware-sandbox-observation&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;family&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;example-family&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;sample_sha256&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;observed_at&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;...&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;known_good_evaluation&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;not-performed&amp;quot;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;This retains the warning published by sources such as &lt;a href="https://sslbl.abuse.ch/blacklist/"&gt;SSLBL&lt;/a&gt;. It allows a consumer to ask whether a fingerprint was seen in malware without treating every matching benign process as infected.&lt;/p&gt;
&lt;h2&gt;Publish validation sets and negative evidence&lt;/h2&gt;
&lt;p&gt;A useful database should include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;controlled positive captures;&lt;/li&gt;
&lt;li&gt;known-good observations that share supposedly malicious fingerprints;&lt;/li&gt;
&lt;li&gt;deliberately conflicting labels;&lt;/li&gt;
&lt;li&gt;client upgrades that changed fingerprints;&lt;/li&gt;
&lt;li&gt;malformed and truncated handshakes;&lt;/li&gt;
&lt;li&gt;captures before and after a TLS-terminating proxy;&lt;/li&gt;
&lt;li&gt;implementation conformance cases.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Negative and conflicting evidence is not database dirt. It tells consumers where a label stops working.&lt;/p&gt;
&lt;h2&gt;Version the data and the interpretation&lt;/h2&gt;
&lt;p&gt;Every release should state:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;schema version;&lt;/li&gt;
&lt;li&gt;database snapshot identifier;&lt;/li&gt;
&lt;li&gt;fingerprint implementation revisions;&lt;/li&gt;
&lt;li&gt;collection time range;&lt;/li&gt;
&lt;li&gt;label additions, removals and merges;&lt;/li&gt;
&lt;li&gt;retired mappings;&lt;/li&gt;
&lt;li&gt;licence changes;&lt;/li&gt;
&lt;li&gt;reproducible validation results.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Consumers should be able to pin a snapshot and explain which database caused a decision. A live service may remain convenient, but an incident review needs the state that existed when the event was classified.&lt;/p&gt;
&lt;h2&gt;Make licensing part of the schema&lt;/h2&gt;
&lt;p&gt;Fingerprint method, implementation and dataset rights are separate.&lt;/p&gt;
&lt;p&gt;Core JA4 is BSD-licensed. Other JA4+ methods use different FoxIO terms. Nmap's data files use the Nmap Public Source License. A combined historical repository may contain rows imported under different licences. An API may permit lookup but prohibit bulk redistribution.&lt;/p&gt;
&lt;p&gt;Store a licence or source-rights reference for each imported dataset and, where necessary, each record. If the rights are unclear, publish a pointer and transformation recipe rather than republishing the row.&lt;/p&gt;
&lt;h2&gt;Provide a confidence model that can be audited&lt;/h2&gt;
&lt;p&gt;A score without calibration is decoration. For each label type, document:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;what the score estimates;&lt;/li&gt;
&lt;li&gt;the labelled validation set;&lt;/li&gt;
&lt;li&gt;true-positive and false-positive definitions;&lt;/li&gt;
&lt;li&gt;treatment of unseen applications;&lt;/li&gt;
&lt;li&gt;time and environment holdouts;&lt;/li&gt;
&lt;li&gt;how conflicts and stale observations affect the score.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Where calibration is unavailable, use review states such as &lt;code&gt;submitted&lt;/code&gt;, &lt;code&gt;reproduced&lt;/code&gt;, &lt;code&gt;reviewed&lt;/code&gt; and &lt;code&gt;contested&lt;/code&gt; instead of inventing numeric precision.&lt;/p&gt;
&lt;h2&gt;The minimum viable open record&lt;/h2&gt;
&lt;p&gt;If full packet evidence cannot be published, a useful minimum is:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;method and rule version
implementation revision
compact and raw fingerprint
candidate label, including version/platform
ground-truth method
capture position
first and last seen
observation count
confidence or review state
evidence reference
licence
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;That is more expensive than a CSV containing &lt;code&gt;hash,name&lt;/code&gt;. It is also what makes the mapping useful outside the environment and assumptions of its original collector.&lt;/p&gt;
&lt;p&gt;The public ecosystem already contains most of the parts: JA4DB's cross-method model, the TLS observatory's prevalence data, Mercury's candidate-process and destination schema, FingerprinTLS's retained ClientHello fields, and executable fixtures in matcher projects such as Rapid7 Recog. The missing step is to combine those strengths without erasing provenance.&lt;/p&gt;
&lt;p&gt;For the current resource landscape, see &lt;a href="/learning/fingerprinting/public-network-fingerprint-databases/"&gt;Public Network Fingerprint Databases and What They Cover&lt;/a&gt;. For the import checklist, see &lt;a href="/learning/fingerprinting/how-to-evaluate-a-fingerprint-database/"&gt;How to Evaluate a Network Fingerprint Database&lt;/a&gt;.&lt;/p&gt;</content><category term="Security"></category><category term="Network Fingerprinting"></category><category term="TLS Fingerprinting"></category><category term="JA4"></category><category term="Cisco Mercury"></category><category term="Security Research"></category></entry><entry><title>Public Fingerprint Databases Are Not Ground Truth</title><link href="https://www.peakhour.io/blog/public-fingerprint-databases-are-not-ground-truth/" rel="alternate"></link><published>2026-08-30T09:00:00+10:00</published><updated>2026-08-30T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-08-30:/blog/public-fingerprint-databases-are-not-ground-truth/</id><summary type="html">&lt;p&gt;We reviewed the main public fingerprint resources. The formats are open, but current application labels and auditable ground truth remain scarce.&lt;/p&gt;</summary><content type="html">&lt;p&gt;Network fingerprint formats are public. Reliable labels are not.&lt;/p&gt;
&lt;p&gt;We reviewed the main public JA3, JA4, TLS, TCP, HTTP, SSH, JARM, DHCP, operating-system and service-fingerprint resources. The result was not one large ecosystem of interchangeable databases. It was a set of much narrower things: small mapping samples, old signature lists, active-probe rules, malware feeds, account services and a large observatory whose TLS records are almost entirely unlabelled.&lt;/p&gt;
&lt;p&gt;That is not a criticism of the maintainers. It is a warning about what happens after a fingerprint field reaches a dashboard. A precise-looking value invites a precise-looking name. The evidence behind that name may be a controlled capture, a best guess from 2018, a community submission or a malware sandbox observation that was never compared with benign traffic.&lt;/p&gt;
&lt;p&gt;Those labels should not produce the same decision.&lt;/p&gt;
&lt;h2&gt;The largest public TLS database has almost no TLS labels&lt;/h2&gt;
&lt;p&gt;The relaunched &lt;a href="https://tlsfingerprint.io/"&gt;TLS Fingerprint Observatory&lt;/a&gt; is an unusually valuable public resource. It exposes more than a million distinct TLS fingerprints and billions of passive observations from the University of Colorado Boulder and Merit Network. Records can include first and last seen, counts by source, cipher suites, extensions, groups, signature algorithms, ALPN and other parsed fields.&lt;/p&gt;
&lt;p&gt;At the time of our review, essentially none of the TLS fingerprints had implementation labels.&lt;/p&gt;
&lt;p&gt;That fact makes the observatory more credible, not less. The collection can answer prevalence and protocol-evolution questions without pretending it knows which application created every handshake. Its smaller QUIC corpus has several hundred controlled labels, often tied to generated capture files.&lt;/p&gt;
&lt;p&gt;The limitation is equally clear. A prevalence database cannot become a browser or malware classifier merely because its records are detailed. The labels require another source of truth.&lt;/p&gt;
&lt;h2&gt;The public JA4 database is a sample, not JA4DB&lt;/h2&gt;
&lt;p&gt;FoxIO maintains &lt;a href="https://ja4db.foxio.io/"&gt;JA4DB&lt;/a&gt;, a hosted service covering several JA4-family methods, applications and detection guidance. It is the closest current service to a multi-surface mapping database.&lt;/p&gt;
&lt;p&gt;The public file most people can inspect is different. FoxIO's &lt;a href="https://github.com/FoxIO-LLC/ja4/blob/main/ja4plus-mapping.csv"&gt;&lt;code&gt;ja4plus-mapping.csv&lt;/code&gt;&lt;/a&gt; contains 66 data rows. Only 35 carry a core JA4 value; the other JA4+ columns are sparser.&lt;/p&gt;
&lt;p&gt;That file is useful documentation. It shows how an application, library, device, operating system and several network observations can sit in one row. It is not broad enough to serve as a general browser, bot or malware catalogue.&lt;/p&gt;
&lt;p&gt;The hosted service is now account-oriented. Its bulk-data licence and per-record provenance are not publicly clear enough to call it an open database. Core JA4's BSD licence does not automatically cover the hosted labels, and it does not cover all other JA4+ methods under the same terms.&lt;/p&gt;
&lt;h2&gt;The open JA3 mappings are mostly historical&lt;/h2&gt;
&lt;p&gt;Salesforce's archived &lt;a href="https://github.com/salesforce/ja3/tree/master/lists"&gt;JA3 lists&lt;/a&gt; contain roughly 159 application mappings for macOS and Linux. The repository describes them as example or best-guess material. They have no per-row capture date, client version, source PCAP or confidence.&lt;/p&gt;
&lt;p&gt;Trisul's &lt;a href="https://github.com/trisulnsm/ja3prints"&gt;ja3prints&lt;/a&gt; is larger: 626 JSONL mappings assembled from Salesforce examples, malware-traffic-analysis captures, FingerprinTLS and browser additions. Its last update was in 2018, and the combined dataset does not have a clear repository-wide licence.&lt;/p&gt;
&lt;p&gt;The historical &lt;a href="https://github.com/LeeBrotherston/tls-fingerprinting"&gt;FingerprinTLS&lt;/a&gt; database preserves richer ClientHello fields than JA3, which makes it valuable for research. It is now archived, and many records lack consistent capture dates, operating-system evidence and confidence.&lt;/p&gt;
&lt;p&gt;These sources still matter. They document the lineage and can help with an older incident. They should not silently become current application ground truth.&lt;/p&gt;
&lt;h2&gt;Rules are not observations&lt;/h2&gt;
&lt;p&gt;Some of the strongest public databases are actually matcher corpora.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://github.com/nmap/nmap/blob/master/nmap-os-db"&gt;Nmap's OS database&lt;/a&gt; contains actively maintained probe-response signatures. &lt;a href="https://github.com/nmap/nmap/blob/master/nmap-service-probes"&gt;Nmap service probes&lt;/a&gt; combine test payloads with regular expressions that extract products and versions. &lt;a href="https://github.com/rapid7/recog"&gt;Rapid7 Recog&lt;/a&gt; maintains XML signatures for SSH, HTTP, SNMP, favicons, JARM and other surfaces. p0f's &lt;code&gt;p0f.fp&lt;/code&gt; contains passive TCP and HTTP traits, although its upstream corpus is now dated.&lt;/p&gt;
&lt;p&gt;These resources can be excellent at their stated job. A rule match means that a response satisfied the signature. It does not establish population prevalence, exclusivity or a particular process behind the connection.&lt;/p&gt;
&lt;p&gt;Flattening a matcher result into the same table as an observed application mapping discards that distinction.&lt;/p&gt;
&lt;h2&gt;Malware observation is not malware identity&lt;/h2&gt;
&lt;p&gt;SSLBL publishes a &lt;a href="https://sslbl.abuse.ch/blacklist/"&gt;JA3 blacklist&lt;/a&gt; under CC0. It records fingerprints observed while analysing more than 25 million malware PCAPs and offers both CSV and Suricata rules.&lt;/p&gt;
&lt;p&gt;SSLBL also says the values were not tested against known-good traffic and may cause substantial false positives.&lt;/p&gt;
&lt;p&gt;That warning is part of the data. A row supports this statement:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;malware samples assigned this family label produced this JA3
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;It does not support this statement:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;every connection with this JA3 is that malware
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Shared libraries make the second statement unsafe. Cisco's &lt;a href="https://arxiv.org/abs/2009.01939"&gt;destination-context research&lt;/a&gt; examined public malware JA3 indicators and found that many were more strongly associated with benign processes in its enterprise observations.&lt;/p&gt;
&lt;p&gt;VirusTotal's behavioural pivots have the same boundary. Files sharing a JA4 may be related malware, come from one developer or merely use the same TLS library. The pivot begins an investigation; it does not finish one.&lt;/p&gt;
&lt;h2&gt;Mercury publishes the schema, not the labels&lt;/h2&gt;
&lt;p&gt;Cisco Mercury offers one of the clearest public designs for a fingerprint knowledge base. Its &lt;a href="https://github.com/cisco/mercury/blob/main/doc/resources.md"&gt;resource documentation&lt;/a&gt; describes mappings from fingerprints to candidate processes, process counts, operating-system observations and destinations. The classifier can use destination address, port and server name to rank those candidates.&lt;/p&gt;
&lt;p&gt;The current Cisco-labelled resource database is not in the public repository.&lt;/p&gt;
&lt;p&gt;This is an honest architectural boundary. The NPF format, collector and database contract are inspectable. The production labels depend on continuously collected endpoint, network and malware-analysis data that Cisco does not publish as an open corpus.&lt;/p&gt;
&lt;h2&gt;What is actually missing&lt;/h2&gt;
&lt;p&gt;The public ecosystem does not mainly need another hash list. It needs evidence attached to each mapping:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the raw or reversible fingerprint;&lt;/li&gt;
&lt;li&gt;the method and implementation version;&lt;/li&gt;
&lt;li&gt;an independent label source;&lt;/li&gt;
&lt;li&gt;capture position and environment;&lt;/li&gt;
&lt;li&gt;first and last seen;&lt;/li&gt;
&lt;li&gt;observation count;&lt;/li&gt;
&lt;li&gt;competing labels;&lt;/li&gt;
&lt;li&gt;confidence and review state;&lt;/li&gt;
&lt;li&gt;a usable dataset licence.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Almost none of the public resources supplies all of these. Some optimise for scale, some for current labels, some for inspectability and some for rules that can run in an existing scanner.&lt;/p&gt;
&lt;p&gt;The practical response is to preserve those categories. Use an observatory for prevalence, a matcher corpus for response signatures, a threat feed for malware observations, and a mapping database for candidate labels. Then validate the result against local evidence before it reaches enforcement.&lt;/p&gt;
&lt;p&gt;The full directory is in &lt;a href="/learning/fingerprinting/public-network-fingerprint-databases/"&gt;Public Network Fingerprint Databases and What They Cover&lt;/a&gt;. The evaluation checklist is in &lt;a href="/learning/fingerprinting/how-to-evaluate-a-fingerprint-database/"&gt;How to Evaluate a Network Fingerprint Database&lt;/a&gt;.&lt;/p&gt;</content><category term="Security"></category><category term="Network Fingerprinting"></category><category term="TLS Fingerprinting"></category><category term="JA3"></category><category term="JA4"></category><category term="Threat Intelligence"></category></entry><entry><title>Using Network Fingerprints in Bot and Rate-Limit Decisions</title><link href="https://www.peakhour.io/blog/using-network-fingerprints-in-bot-and-rate-limit-decisions/" rel="alternate"></link><published>2026-08-23T09:00:00+10:00</published><updated>2026-08-23T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-08-23:/blog/using-network-fingerprints-in-bot-and-rate-limit-decisions/</id><summary type="html">&lt;p&gt;A practical way to use network fingerprints for bot and rate-limit decisions without mistaking a shared client cohort for identity.&lt;/p&gt;</summary><content type="html">&lt;p&gt;A network fingerprint is a useful rate-limit key. It is a poor verdict.&lt;/p&gt;
&lt;p&gt;That sounds like a small distinction, but it changes how a control behaves. A limit keyed only by source IP can miss a distributed scraper moving through thousands of addresses. Add a TLS or HTTP fingerprint and the requests may fall into a recognisable cohort. The operator can count and compare them even while the IPs rotate.&lt;/p&gt;
&lt;p&gt;The same key can also group thousands of ordinary users running the same browser release. If the control reads “this fingerprint is a bot” and blocks it everywhere, the grouping feature has become a false-positive multiplier.&lt;/p&gt;
&lt;p&gt;The practical job is to use the fingerprint where grouping helps, then make the decision from the route, behaviour and consequence. This article sets out one way to do that.&lt;/p&gt;
&lt;h2&gt;Start with the route, not the fingerprint&lt;/h2&gt;
&lt;p&gt;A request for a cached image and a request to submit a password do not deserve the same policy. Neither do a search request and an export that starts an expensive database job.&lt;/p&gt;
&lt;p&gt;Before adding fingerprint data, classify the routes that matter:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;what the request can change or disclose;&lt;/li&gt;
&lt;li&gt;whether it consumes scarce application or origin capacity;&lt;/li&gt;
&lt;li&gt;whether a legitimate user can retry safely;&lt;/li&gt;
&lt;li&gt;whether the client can complete a challenge;&lt;/li&gt;
&lt;li&gt;which identity is available, such as an account, session, API key or no identity at all;&lt;/li&gt;
&lt;li&gt;what an incorrect block would cost.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This route map sets the response. An unfamiliar cohort fetching public content might only be worth observing. The same cohort attempting passwords across many accounts may justify a tight shared limit. A fingerprint mismatch on an authenticated payment action may call for step-up verification rather than a network block.&lt;/p&gt;
&lt;p&gt;Route sensitivity also prevents a common emergency mistake: applying a site-wide rule because an attack was visible on one endpoint. Narrow rules are easier to explain, test and remove.&lt;/p&gt;
&lt;h2&gt;Use the fingerprint as a cohort key&lt;/h2&gt;
&lt;p&gt;JA3, JA4 and Cisco Mercury do not identify a person or device. Each method selects and normalises protocol fields, then groups connections that look equivalent under those rules. Our &lt;a href="/blog/one-clienthello-ja3-ja4-mercury-lab/"&gt;same-ClientHello lab&lt;/a&gt; shows how three methods produce three different representations from one handshake.&lt;/p&gt;
&lt;p&gt;That makes the fingerprint useful for questions such as:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Are login failures arriving from many IP addresses but one narrow set of client stacks?&lt;/li&gt;
&lt;li&gt;Did an expensive API route suddenly attract a new protocol cohort?&lt;/li&gt;
&lt;li&gt;Does traffic claiming to be a browser have a consistent TLS, HTTP and header shape?&lt;/li&gt;
&lt;li&gt;Did a cohort disappear after an abusive tool changed version, or did the fingerprinting rules change underneath us?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The key should be composite. A workable rate counter might resemble:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;route_class + fingerprint_method + fingerprint_version + fingerprint_value
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Depending on the abuse case, add an account, tenant, API credential, ASN or country. Do not throw every field into every key. An over-specific key divides a distributed campaign into buckets too small to see; an over-broad key combines unrelated users.&lt;/p&gt;
&lt;p&gt;Run a few candidate keys in observation mode and measure their bucket sizes. Look at both tails. A key that places half the site's users in one bucket is too blunt for direct enforcement, however suspicious that bucket looked during one incident.&lt;/p&gt;
&lt;h2&gt;Record where the value came from&lt;/h2&gt;
&lt;p&gt;Fingerprint provenance is operational data, not documentation trivia.&lt;/p&gt;
&lt;p&gt;At an origin behind a CDN or reverse proxy, the visible TLS connection may belong to that intermediary. A fingerprint forwarded in a header might describe the original client-facing connection, or it might describe a later hop. The receiving service cannot tell from the hash alone.&lt;/p&gt;
&lt;p&gt;For each event, keep at least:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;method:       ja4
version:      pinned implementation or rule version
value:        t13d..._..._...
capture:      client-facing edge
source:       named collector or trusted forwarding hop
observed_at:  timestamp
policy:       policy identifier and revision
action:       observe, challenge, rate-limit or block
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Where the format permits it, retaining the selected raw fields or unhashed representation makes later comparison much easier. Cisco Mercury's full Network Protocol Fingerprint is structured and versioned; its optional hash is a compact nickname that discards that structure. JA4 publishes both a canonical hashed form and a raw &lt;code&gt;JA4_r&lt;/code&gt; form. The details are in the &lt;a href="https://github.com/cisco/mercury/blob/main/doc/npf.md"&gt;Mercury NPF specification&lt;/a&gt; and &lt;a href="https://github.com/FoxIO-LLC/ja4/blob/main/technical_details/JA4.md"&gt;JA4 technical specification&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Trust the forwarding path as deliberately as any other security metadata. Strip client-supplied copies at the boundary, set the authoritative value there, and record the hop that produced it. Otherwise an attacker may be able to choose the value used by the policy.&lt;/p&gt;
&lt;h2&gt;Use an action ladder&lt;/h2&gt;
&lt;p&gt;Bot controls work better with space between “allow” and “block”. A simple ladder is observe, challenge, rate limit, then block. It is not a mandatory sequence for every request; it is a way to match disruption to confidence.&lt;/p&gt;
&lt;h3&gt;Observe&lt;/h3&gt;
&lt;p&gt;Observation is the right first action for a new cohort, a changed implementation or a weak hypothesis. Record volume, routes, account outcomes, response codes, request cost and how the cohort overlaps with known legitimate clients.&lt;/p&gt;
&lt;p&gt;Set an end date before starting. “Log this for seven days and review on Monday” produces a decision. “Log this” often produces another permanent stream nobody owns.&lt;/p&gt;
&lt;h3&gt;Challenge&lt;/h3&gt;
&lt;p&gt;A challenge asks the client for more evidence. It can separate some interactive browsers from simple automation, but it is not a universal test of humanity. Browser automation can complete challenges; privacy software, accessibility tools and broken JavaScript can make legitimate users fail them.&lt;/p&gt;
&lt;p&gt;Use challenges where the client can reasonably complete one and where failure has a safe recovery path. They are a poor fit for machine-to-machine APIs unless the protocol already defines an authentication or proof step.&lt;/p&gt;
&lt;h3&gt;Rate limit&lt;/h3&gt;
&lt;p&gt;Rate limiting fits repeated or costly behaviour. Count a cohort against the route it is affecting, then layer in the identities that make sense for that route. For example:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;login:  fingerprint + account + failed outcome
search: fingerprint + route + request cost
API:    fingerprint + tenant or credential + operation
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;A limit should say what it protects. Requests per minute is sometimes enough. Expensive operations may need a cost-weighted budget. Login protection may count failed attempts while allowing successful account recovery to proceed. Distributed abuse may need a cohort-wide ceiling plus per-account and per-IP limits so that no single key carries the whole policy.&lt;/p&gt;
&lt;p&gt;Return an explicit response and keep the retry behaviour predictable. The HTTP &lt;code&gt;429 Too Many Requests&lt;/code&gt; status was defined for rate limiting in &lt;a href="https://www.rfc-editor.org/rfc/rfc6585.html#section-4"&gt;RFC 6585&lt;/a&gt;, including the option to send &lt;code&gt;Retry-After&lt;/code&gt;. Clients still need sensible backoff, and operators need to watch whether retries are making origin pressure worse.&lt;/p&gt;
&lt;h3&gt;Block&lt;/h3&gt;
&lt;p&gt;Block when the evidence is strong, the harm is current, and a less disruptive action cannot protect the route. Good candidates include a confirmed exploit tool hitting the vulnerable route, a cohort causing active availability loss, or repeated abuse that has failed narrower controls.&lt;/p&gt;
&lt;p&gt;Keep the scope bounded: fingerprint method and version, affected route, relevant behaviour, start time, owner and expiry. A bare fingerprint deny-list with no incident context is hard to audit and tends to outlive its evidence.&lt;/p&gt;
&lt;p&gt;For a broader mapping of actions to evidence, see &lt;a href="/learning/fingerprinting/network-fingerprint-signals-and-security-decisions/"&gt;Network Fingerprint Signals and Security Decisions&lt;/a&gt; and &lt;a href="/learning/fingerprinting/network-fingerprinting-for-rate-limiting/"&gt;Network Fingerprinting for Rate Limiting&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Combine evidence before increasing friction&lt;/h2&gt;
&lt;p&gt;The most useful policy does not ask whether a fingerprint is good or bad. It asks whether several observations agree.&lt;/p&gt;
&lt;p&gt;Consider a login campaign. The source addresses rotate through residential networks. The TLS cohort is common because the tool drives a real browser. The fingerprint alone is weak. But the same traffic may attempt many accounts, reuse a route sequence, fail at a high rate, omit normal session history and arrive at a cadence unlike ordinary users. Together those observations justify a shared limit or challenge.&lt;/p&gt;
&lt;p&gt;Now reverse the example. A rare TLS cohort signs in successfully to one account from its usual region, follows a normal route sequence and maintains an established session. Rarity does not add much. Blocking it would mostly punish an unusual client.&lt;/p&gt;
&lt;p&gt;This is also the line between a format and a detection system. JA4 defines a fingerprint. Mercury defines fingerprint representations and includes separate analysis components. Any claim such as “this is Chrome”, “this is a scraper” or “this is malicious” comes from surrounding data and inference, not from the fingerprint characters themselves. &lt;a href="/blog/fingerprint-is-a-cohort-not-a-client/"&gt;A Network Fingerprint Is a Cohort, Not a Client&lt;/a&gt; explains that boundary in more detail. &lt;a href="/blog/fingerprints-are-evidence-not-identity/"&gt;Fingerprints Are Evidence, Not Identity&lt;/a&gt; covers the same problem across browser, network and behavioural fingerprints.&lt;/p&gt;
&lt;h2&gt;Expect versions and populations to drift&lt;/h2&gt;
&lt;p&gt;Browsers ship, TLS libraries change defaults, mobile applications update unevenly and attack tools copy popular handshakes. Fingerprint implementations also change their parsing rules. Store the method and implementation version so client drift can be separated from detector drift.&lt;/p&gt;
&lt;p&gt;Before changing an implementation, calculate old and new values side by side on a sample. Compare cohort sizes and known-good traffic by route. A partner API, mobile application and public website have different expected populations; a global label may be irrelevant to the local control.&lt;/p&gt;
&lt;h2&gt;Put expiry and rollback in the policy&lt;/h2&gt;
&lt;p&gt;Every enforcement rule needs an owner, reason, expiry and removal condition. Before enabling it, record request volume, successful transactions, challenges, &lt;code&gt;429&lt;/code&gt; responses and origin health. Compare the same measures afterwards. A falling attack rate is not success if legitimate completion falls with it.&lt;/p&gt;
&lt;p&gt;When a false positive appears, retain the fingerprint version, capture point, contributing evidence, route, identity, policy revision and rollback result. That record shows whether to change a threshold, split a route class, add an exception, demote the action or remove the fingerprint from the key.&lt;/p&gt;
&lt;h2&gt;A deployable first policy&lt;/h2&gt;
&lt;p&gt;For a first production use, choose one abused route. Generate a versioned fingerprint at a trusted capture point, log it beside the route and outcome, and observe cohort sizes long enough to include normal variation. Then choose one reversible action with success measures, rollback and an expiry.&lt;/p&gt;
&lt;p&gt;The fingerprint has done its job when it helps a team see and control a group of requests that an IP-only rule would miss. It has exceeded its job when the group is treated as proof of a client, person or intent.&lt;/p&gt;</content><category term="Security"></category><category term="Network Fingerprinting"></category><category term="TLS Fingerprinting"></category><category term="JA4"></category><category term="Bot Management"></category><category term="Rate Limiting"></category></entry><entry><title>Does TLS Fingerprint Canonicalisation Hide Attacker Variation? How to Test It</title><link href="https://www.peakhour.io/blog/tls-fingerprint-canonicalisation-attacker-variation/" rel="alternate"></link><published>2026-08-16T09:00:00+10:00</published><updated>2026-08-16T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-08-16:/blog/tls-fingerprint-canonicalisation-attacker-variation/</id><summary type="html">&lt;p&gt;Sorting makes TLS fingerprints more stable, but it also removes ordering evidence. Here is how to test whether the discarded variation matters.&lt;/p&gt;</summary><content type="html">&lt;p&gt;Canonicalisation solves a real problem in TLS fingerprinting. If two ClientHello messages differ only because a client shuffled its extensions, treating them as unrelated fingerprints creates noise. Sorting those extensions puts the messages back into one cohort.&lt;/p&gt;
&lt;p&gt;It also destroys the original order.&lt;/p&gt;
&lt;p&gt;That is not automatically a mistake. A fingerprint is useful partly because it ignores variation that does not help with the job at hand. The unanswered question is whether some of the discarded variation separates ordinary client behaviour from automation, scanners or deliberate evasion.&lt;/p&gt;
&lt;p&gt;Our &lt;a href="/blog/one-clienthello-ja3-ja4-mercury-lab/"&gt;one-ClientHello lab&lt;/a&gt; cannot answer that. It proves that three pinned implementations produce recorded representations from the same bytes. It says nothing about a population of clients or attackers. Answering the canonicalisation question needs a corpus and a labelled experiment.&lt;/p&gt;
&lt;h2&gt;What gets collapsed?&lt;/h2&gt;
&lt;p&gt;JA3 removes GREASE values but otherwise preserves the order of the selected cipher, extension, supported-group and point-format lists. Change one of those ordered inputs and the MD5 digest changes.&lt;/p&gt;
&lt;p&gt;JA4 deliberately defines a broader equivalence class. Its canonical &lt;code&gt;b&lt;/code&gt; section hashes sorted cipher identifiers. Its &lt;code&gt;c&lt;/code&gt; section hashes sorted extension identifiers followed by signature algorithms in their advertised order. SNI and ALPN extension codes are omitted from that list because related information is represented in the readable &lt;code&gt;a&lt;/code&gt; section. FoxIO's &lt;a href="https://github.com/FoxIO-LLC/ja4/blob/main/technical_details/JA4.md"&gt;JA4 specification&lt;/a&gt; documents those choices.&lt;/p&gt;
&lt;p&gt;Cisco Mercury makes the rule version visible. In the current &lt;a href="https://github.com/cisco/mercury/blob/main/doc/npf.md"&gt;draft NPF specification&lt;/a&gt;, the older unversioned TLS format retains extension order, &lt;code&gt;tls/1&lt;/code&gt; sorts all represented extensions, and &lt;code&gt;tls/2&lt;/code&gt; sorts selected extensions while applying more specific inclusion and normalisation rules.&lt;/p&gt;
&lt;p&gt;These methods do not merely encode the same fingerprint differently. They define different ideas of “the same”.&lt;/p&gt;
&lt;h2&gt;Why Chrome forced the issue&lt;/h2&gt;
&lt;p&gt;Chrome's extension permutation rollout showed why order-sensitive identifiers can become operationally brittle. Peakhour's &lt;a href="/blog/tls-extension-randomisation/"&gt;2023 analysis&lt;/a&gt; recorded a sharp rise in unique order-sensitive signatures after the change. The browser family had not suddenly split into thousands of independent TLS implementations. Much of the new variation came from ordering.&lt;/p&gt;
&lt;p&gt;Sorting is an effective response if the goal is to recover the implementation cohort. It is also consistent with &lt;a href="https://chromestatus.com/feature/5124606246518784"&gt;Chrome's stated reason for making the change&lt;/a&gt;: servers and middleboxes should not depend on one fixed extension order.&lt;/p&gt;
&lt;p&gt;But an analyst may have another question. Does a tool permute extensions using the same mechanism and constraints as the browser it imitates? Does a scanner generate an ordering distribution that differs from Chrome's? Does malware preserve the static order supplied by its TLS library while claiming a Chrome user agent?&lt;/p&gt;
&lt;p&gt;A canonical JA4 can group those handshakes even when their ordering behaviour differs. That is expected. JA4 answered the cohort question, not every possible behavioural question.&lt;/p&gt;
&lt;h2&gt;The wrong experiment&lt;/h2&gt;
&lt;p&gt;Counting how many unique raw fingerprints map to one canonical fingerprint is a useful descriptive statistic. It is not, by itself, evidence that canonicalisation weakened detection.&lt;/p&gt;
&lt;p&gt;A common browser with extension permutation should produce many raw orders. A large raw-to-canonical ratio may therefore be evidence of normal deployment scale. Calling every collapsed value a loss of “fidelity” assumes that all variation was useful before the test has measured its relationship with any outcome.&lt;/p&gt;
&lt;p&gt;The reverse shortcut is also wrong. Stable canonical values do not prove that sorting is harmless for every detector. A field can be poor for application identification but useful for distinguishing one implementation path, library version or evasion technique.&lt;/p&gt;
&lt;p&gt;The experiment needs labels and a defined decision.&lt;/p&gt;
&lt;h2&gt;A testable study design&lt;/h2&gt;
&lt;p&gt;We would structure the study around observations, transformations and outcomes.&lt;/p&gt;
&lt;h3&gt;1. Preserve the original ClientHello&lt;/h3&gt;
&lt;p&gt;Store the permitted raw handshake metadata or a reversible representation alongside derived identifiers. Record:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;capture point and TLS termination path;&lt;/li&gt;
&lt;li&gt;sensor implementation and revision;&lt;/li&gt;
&lt;li&gt;timestamp and software-release period;&lt;/li&gt;
&lt;li&gt;JA3 source string and digest;&lt;/li&gt;
&lt;li&gt;JA4, &lt;code&gt;JA4_r&lt;/code&gt;, &lt;code&gt;JA4_o&lt;/code&gt; and &lt;code&gt;JA4_ro&lt;/code&gt; where the implementation provides them;&lt;/li&gt;
&lt;li&gt;a full, versioned Mercury NPF string;&lt;/li&gt;
&lt;li&gt;HTTP and browser claims kept separate from the TLS representation.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Without the raw or reversible material, a later study cannot recover the ordering that canonicalisation removed.&lt;/p&gt;
&lt;h3&gt;2. Define labels that do not come from the fingerprint&lt;/h3&gt;
&lt;p&gt;Labels need an independent source. Depending on the environment, that could include controlled browser runs, endpoint process telemetry, sandbox execution, signed test clients or reviewed incident cases.&lt;/p&gt;
&lt;p&gt;Do not label traffic as Chrome because its JA4 resembles Chrome and then report that JA4 identifies Chrome. That is circular evaluation.&lt;/p&gt;
&lt;p&gt;Cisco's destination-context research used joined endpoint and network observations to build process labels. The paper also discusses how sandbox and environment choices affect the resulting knowledge base. &lt;a href="https://arxiv.org/abs/2009.01939"&gt;Accurate TLS Fingerprinting Using Destination Context and Knowledge Bases&lt;/a&gt; is useful here because it treats ground truth as a system component rather than a list of famous hashes.&lt;/p&gt;
&lt;h3&gt;3. Compare representations at the same grouping level&lt;/h3&gt;
&lt;p&gt;Measure at least:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;raw ordered representation;&lt;/li&gt;
&lt;li&gt;canonical JA4;&lt;/li&gt;
&lt;li&gt;useful JA4 component combinations such as &lt;code&gt;JA4_ac&lt;/code&gt;;&lt;/li&gt;
&lt;li&gt;Mercury rule versions that preserve or sort different structures;&lt;/li&gt;
&lt;li&gt;raw ordering features added beside the canonical value.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The comparison should use the same captures, time split and labels. Otherwise a newer fingerprint method can appear better simply because it was evaluated on newer or cleaner data.&lt;/p&gt;
&lt;h3&gt;4. Use time and environment holdouts&lt;/h3&gt;
&lt;p&gt;Randomly splitting individual connections leaks near-duplicates between training and test data. Prefer a forward time split and, where possible, a separate network or capture environment.&lt;/p&gt;
&lt;p&gt;That exposes two operational questions: does the result survive a browser or library update, and does it survive outside the environment where the labels were collected?&lt;/p&gt;
&lt;h3&gt;5. Measure decisions, not just uniqueness&lt;/h3&gt;
&lt;p&gt;Useful measurements include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;collision and fragmentation rates by independently labelled client;&lt;/li&gt;
&lt;li&gt;precision and recall for a stated classification or detection task;&lt;/li&gt;
&lt;li&gt;false-positive rates on high-volume legitimate cohorts;&lt;/li&gt;
&lt;li&gt;stability across software releases;&lt;/li&gt;
&lt;li&gt;the incremental value of raw order after canonical identifiers and context are already present;&lt;/li&gt;
&lt;li&gt;review volume at an actual alert or policy threshold.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If adding original order improves a classifier by a tiny amount but creates millions of unstable keys, the operational cost may outweigh the gain. If it separates a specific impersonation technique with few false positives, keeping it as a secondary feature may be worthwhile.&lt;/p&gt;
&lt;h2&gt;Keep both when the questions differ&lt;/h2&gt;
&lt;p&gt;The design choice does not have to be raw or canonical.&lt;/p&gt;
&lt;p&gt;A compact canonical identifier is useful for grouping, counters, joins and rules. A raw or reversible representation is useful for investigation, feature research and migration when the canonical rules change. Storage policy can keep the compact value broadly and retain detailed material for a bounded sample, selected security events or an approved research window.&lt;/p&gt;
&lt;p&gt;That split also makes detector claims easier to audit. The rule can say it grouped on JA4 while the event retains enough source material to explain which handshake produced the value.&lt;/p&gt;
&lt;h2&gt;What we can say now&lt;/h2&gt;
&lt;p&gt;Sorting removes ordering information. It reduces fragmentation caused by clients that permute their lists. Both statements follow from the format definitions and can be demonstrated with controlled captures.&lt;/p&gt;
&lt;p&gt;Whether the removed order contains useful attacker variation is an empirical question tied to a dataset, capture point, label source and decision. Until that study is run, the honest position is to preserve the evidence needed to test it and avoid turning either uniqueness or stability into a claim of detection accuracy.&lt;/p&gt;
&lt;p&gt;For the wider format comparison, see &lt;a href="/learning/fingerprinting/mercury-vs-ja4-vs-ja3/"&gt;Mercury vs JA4 vs JA3&lt;/a&gt;. For the identity boundary that applies to every result, read &lt;a href="/blog/fingerprint-is-a-cohort-not-a-client/"&gt;A network fingerprint is a cohort, not a client&lt;/a&gt;.&lt;/p&gt;</content><category term="Security"></category><category term="TLS Fingerprinting"></category><category term="JA4"></category><category term="Cisco Mercury"></category><category term="Network Fingerprinting"></category><category term="Security Research"></category></entry><entry><title>A Network Fingerprint Is a Cohort, Not a Client</title><link href="https://www.peakhour.io/blog/fingerprint-is-a-cohort-not-a-client/" rel="alternate"></link><published>2026-08-09T09:00:00+10:00</published><updated>2026-08-09T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-08-09:/blog/fingerprint-is-a-cohort-not-a-client/</id><summary type="html">&lt;p&gt;TLS fingerprints group similar protocol implementations. They do not prove which application, device or person made a request.&lt;/p&gt;</summary><content type="html">&lt;p&gt;A TLS fingerprint usually tells you that two connections look alike under a particular set of rules. That is useful. It is not the same as showing that they came from the same client.&lt;/p&gt;
&lt;p&gt;The distinction becomes obvious when a common TLS library sits underneath many programs. Those programs can offer the same protocol version, cipher suites, extensions and signature algorithms. A fingerprint built from those fields groups them together even though their purpose, owner and risk are different.&lt;/p&gt;
&lt;p&gt;The reverse also happens. One application can generate several fingerprints after a browser update, operating-system change, feature rollout or configuration difference. A fixed application name does not imply a fixed ClientHello.&lt;/p&gt;
&lt;p&gt;The safest mental model is a cohort: traffic that looks the same after a fingerprint method has selected and normalised its inputs.&lt;/p&gt;
&lt;h2&gt;The method defines the cohort&lt;/h2&gt;
&lt;p&gt;JA3 preserves the order of its selected ClientHello feature lists. Change the order and the MD5 value changes. JA4 sorts selected identifiers before calculating two of its components, so permutations that split a JA3 cohort may remain grouped under JA4.&lt;/p&gt;
&lt;p&gt;Cisco Mercury can preserve more packet-derived structure in its full Network Protocol Fingerprint. Different Mercury rule versions also make different normalisation choices. A &lt;code&gt;tls/2&lt;/code&gt; value is therefore not just a longer spelling of JA4. It is the result of another feature-selection contract.&lt;/p&gt;
&lt;p&gt;This means there is no format-independent "real fingerprint" hiding underneath the tools. Each method answers its own equivalence question: which differences count, and which should be ignored?&lt;/p&gt;
&lt;p&gt;Our &lt;a href="/blog/one-clienthello-ja3-ja4-mercury-lab/"&gt;same-PCAP lab&lt;/a&gt; demonstrates that point without relying on a hypothetical browser. The three tools inspect the same ClientHello and produce representations with different retained detail.&lt;/p&gt;
&lt;h2&gt;Common does not mean safe&lt;/h2&gt;
&lt;p&gt;A popular browser fingerprint will naturally appear in a great deal of legitimate traffic. An attacker can also use a browser, drive one through automation, or imitate its TLS stack. Matching a common browser value therefore says little about intent by itself.&lt;/p&gt;
&lt;p&gt;The opposite shortcut is just as risky. An uncommon fingerprint is not proof of malware. Internal tools, older mobile applications, embedded devices, accessibility software and regional client variants can all be rare in one dataset.&lt;/p&gt;
&lt;p&gt;Rarity is relative to the observation point. A fingerprint that is common across a public content site may be unusual on an administrative API. A value common in one country, network or month may be rare in another.&lt;/p&gt;
&lt;h2&gt;Capture point changes what you see&lt;/h2&gt;
&lt;p&gt;TLS fingerprinting only works where the relevant handshake is visible. At an origin behind a CDN or reverse proxy, the TLS connection may have been terminated and replaced upstream. The origin can then observe the proxy's connection rather than the end user's ClientHello unless the edge explicitly forwards a derived fingerprint.&lt;/p&gt;
&lt;p&gt;That forwarded value also needs provenance. Operators should know which implementation and format version produced it, whether it came from the client-facing connection, and whether middleware transformed or sampled the traffic.&lt;/p&gt;
&lt;p&gt;Without that information, two values that look compatible may have been generated under different rules. Fastly has documented how implementation differences can undermine the portability promised by a shared hash in &lt;a href="https://www.fastly.com/blog/the-state-of-tls-fingerprinting-whats-working-what-isnt-and-whats-next"&gt;The State of TLS Fingerprinting&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Context can improve an inference&lt;/h2&gt;
&lt;p&gt;Cisco's Mercury research is helpful because it does not hide the ambiguity. The 2020 destination-context paper reports that one TLS fingerprint often maps to tens or hundreds of process names. Its classifier adds destination IP address, port and server name, backed by a labelled knowledge base, to rank the possible processes.&lt;/p&gt;
&lt;p&gt;That is stronger than treating a bare fingerprint as an application name. It is still conditional. Change the environment, the age of the knowledge base, the available destination fields or the software population and the probabilities can change. The paper's reported accuracy belongs to its datasets and experimental design, not to every network that runs Mercury.&lt;/p&gt;
&lt;p&gt;JA4 deployments often add context too. A security platform might combine the JA4 value with request rate, geography, path, account state or observations from other customers. Those extra fields are not secretly part of JA4. They are features in the surrounding detection system.&lt;/p&gt;
&lt;h2&gt;Use fingerprints where grouping helps&lt;/h2&gt;
&lt;p&gt;Fingerprints earn their place when grouping similar connections improves an investigation or control. Examples include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;counting login failures across rotating IP addresses;&lt;/li&gt;
&lt;li&gt;finding a TLS stack that appeared at the start of an incident;&lt;/li&gt;
&lt;li&gt;comparing a claimed browser with HTTP and browser-side evidence;&lt;/li&gt;
&lt;li&gt;monitoring drift after a client or library release;&lt;/li&gt;
&lt;li&gt;selecting traffic for review before writing a narrower rule.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For the decision and enforcement consequences, see &lt;a href="/blog/fingerprints-are-evidence-not-identity/"&gt;Fingerprints are evidence, not identity&lt;/a&gt;. The protocol-specific conclusion here is narrower: a matching TLS fingerprint places connections in a cohort defined by one method. It cannot tell you, on its own, who is on the other end.&lt;/p&gt;
&lt;p&gt;The practitioner follow-up, &lt;a href="/blog/using-network-fingerprints-in-bot-and-rate-limit-decisions/"&gt;Using network fingerprints in bot and rate-limit decisions&lt;/a&gt;, turns that boundary into a route-scoped policy and rollback model.&lt;/p&gt;</content><category term="Security"></category><category term="TLS Fingerprinting"></category><category term="JA3"></category><category term="JA4"></category><category term="Cisco Mercury"></category><category term="Network Fingerprinting"></category><category term="Bot Management"></category></entry><entry><title>From Joy to Mercury and EVE: Following Cisco's Network Fingerprinting Work</title><link href="https://www.peakhour.io/blog/from-joy-to-mercury-and-eve/" rel="alternate"></link><published>2026-08-02T09:00:00+10:00</published><updated>2026-08-02T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-08-02:/blog/from-joy-to-mercury-and-eve/</id><summary type="html">&lt;p&gt;Cisco's open-source collectors, fingerprinting research and Encrypted Visibility Engine form a clear lineage, but they are not interchangeable parts of one public system.&lt;/p&gt;</summary><content type="html">&lt;p&gt;Cisco's network fingerprinting work is often compressed into a neat product story: Joy became Mercury, and Mercury became the Encrypted Visibility Engine. There is a real lineage here, but that sentence hides more than it explains.&lt;/p&gt;
&lt;p&gt;Joy and Mercury are public software projects. Network Protocol Fingerprinting is a representation. The 2020 destination-context paper describes a classification system and the data needed to support it. EVE is a Cisco Secure Firewall capability backed by Cisco's operational data and machine-learning systems.&lt;/p&gt;
&lt;p&gt;Those pieces share ideas, authors and engineering history. They are not four names for the same thing.&lt;/p&gt;
&lt;h2&gt;Joy started with flow records, then looked inside them&lt;/h2&gt;
&lt;p&gt;Cisco released &lt;a href="https://github.com/cisco/joy"&gt;Joy&lt;/a&gt; as a BSD-licensed tool for collecting network flow and &lt;em&gt;intraflow&lt;/em&gt; data from live traffic or PCAP files. Its basic unit was recognisable to anyone who had worked with NetFlow or IPFIX: a flow with addresses, ports and counters. Joy then added observations from within that flow.&lt;/p&gt;
&lt;p&gt;Those observations included packet lengths and arrival times, byte distributions, TLS record sequences, visible TLS handshake fields, DNS data and selected HTTP fields. It wrote the results as JSON so researchers could feed them into analysis tools without building a packet parser first.&lt;/p&gt;
&lt;p&gt;That breadth matters. Joy was not originally just a ClientHello hash generator. It was a network-research instrument for asking what could still be learned from traffic when the application payload was encrypted.&lt;/p&gt;
&lt;p&gt;The research around Joy shows how Cisco used it. The 2016 paper &lt;a href="https://arxiv.org/abs/1607.01639"&gt;Deciphering Malware's Use of TLS (without Decryption)&lt;/a&gt; combined conventional flow measurements, packet-length and timing sequences, byte distributions and visible TLS handshake features. Its authors were interested in malware detection and family attribution, but they were also explicit about dataset bias. A sandbox operating system or its default TLS library could become an accidental shortcut for the classifier.&lt;/p&gt;
&lt;p&gt;In 2017, &lt;a href="https://arxiv.org/abs/1706.08003"&gt;OS Fingerprinting: New Techniques and a Study of Information Gain and Obfuscation&lt;/a&gt; used an extended Joy collector to record features from TCP SYN packets, TLS ClientHello messages and HTTP requests. The study combined evidence across protocols and sessions rather than expecting one handshake to provide a definitive operating-system label.&lt;/p&gt;
&lt;p&gt;The enduring idea was broader than any one model: retain useful network observations, join them to trustworthy labels where possible, and test what the combined evidence can support.&lt;/p&gt;
&lt;h2&gt;Mercury narrowed and hardened the collection path&lt;/h2&gt;
&lt;p&gt;The Joy repository now directs readers to Mercury for Cisco's more recent fingerprinting tools and data. &lt;a href="https://github.com/cisco/mercury"&gt;Mercury&lt;/a&gt; carries forward the packet-metadata and JSON-output model, but it is a separate implementation with a stronger focus on fast capture, protocol fingerprints and online analysis.&lt;/p&gt;
&lt;p&gt;The public repository contains a C++ application and library, along with the portable Python implementation &lt;code&gt;pmercury&lt;/code&gt;. Mercury can read PCAPs or live traffic, identify supported protocols and emit selected metadata. On Linux, its native collector uses the kernel's &lt;code&gt;AF_PACKET&lt;/code&gt; &lt;code&gt;TPACKETv3&lt;/code&gt; path and multiple workers. The repository describes use in some production applications, while still asking users to treat the open software as beta.&lt;/p&gt;
&lt;p&gt;Mercury also makes a useful boundary visible in its JSON:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;packet
  -&amp;gt; protocol metadata
  -&amp;gt; fingerprint object
  -&amp;gt; optional destination-context analysis
  -&amp;gt; analysis object
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The fingerprint is an observation. The analysis is an inference made from that observation and other data. They should not be collapsed into one label.&lt;/p&gt;
&lt;h2&gt;NPF is the representation, not the classifier&lt;/h2&gt;
&lt;p&gt;Mercury's fingerprints use Cisco's Network Protocol Fingerprinting format. NPF selects characteristic fields from an initial protocol message, normalises values that should not distinguish implementations, and writes the retained structure as a tree of hexadecimal byte strings.&lt;/p&gt;
&lt;p&gt;A full TLS fingerprint can therefore be inspected. Its &lt;code&gt;tls/2&lt;/code&gt; prefix identifies the protocol and rule version; parentheses and brackets preserve the selected tree structure and sorting decisions. Mercury can also use a compact hash nickname when fixed-length storage is more useful than inspection.&lt;/p&gt;
&lt;p&gt;The &lt;a href="https://github.com/cisco/mercury/blob/main/doc/npf.md"&gt;draft NPF specification&lt;/a&gt; covers more than TLS. It documents rules for protocols including QUIC, TCP, HTTP, SSH, STUN and OpenVPN. This is one point at which the Joy heritage remains visible: the project treats protocol evidence as a network-wide problem, not solely as a way to name TLS clients.&lt;/p&gt;
&lt;p&gt;Our guide to &lt;a href="/learning/fingerprinting/what-is-cisco-mercury-fingerprinting/"&gt;Cisco Mercury fingerprinting&lt;/a&gt; separates the collector, NPF representation, knowledge base and classifier in more detail. &lt;a href="/learning/fingerprinting/inside-mercury-npf-fingerprint/"&gt;Inside a Mercury NPF fingerprint&lt;/a&gt; works through the full notation.&lt;/p&gt;
&lt;h2&gt;The 2020 paper made the attribution problem explicit&lt;/h2&gt;
&lt;p&gt;A fingerprint is useful for grouping similar protocol implementations. It is often a poor application name.&lt;/p&gt;
&lt;p&gt;Blake Anderson and David McGrew quantified that problem in &lt;a href="https://arxiv.org/abs/2009.01939"&gt;Accurate TLS Fingerprinting Using Destination Context and Knowledge Bases&lt;/a&gt;. TLS libraries are shared. Common fingerprints can map to tens or hundreds of processes. A dictionary that assigns one process name to one fingerprint will therefore produce convincing-looking false positives.&lt;/p&gt;
&lt;p&gt;The paper added three destination features to the TLS fingerprint: destination address, destination port and the TLS server name, when present. A weighted naive Bayes classifier used those features and prevalence counts from a labelled knowledge base to rank candidate processes.&lt;/p&gt;
&lt;p&gt;The knowledge base was the difficult part. The researchers joined network observations to endpoint process data, generated fresh knowledge bases daily and merged them into an operational view. They also used malware-sandbox observations. The paper describes billions of connections, a changing population of fingerprints and destinations, and the need to discard stale data.&lt;/p&gt;
&lt;p&gt;Its reported results were strong, including a process-family F1 score above 0.99 and high precision and recall for the malware task. Those figures belong to the paper's datasets and evaluation. They are not an accuracy guarantee for a Mercury installation downloaded from GitHub.&lt;/p&gt;
&lt;p&gt;The authors also documented important limits: their endpoint data was dominated by desktop Windows and macOS systems, mobile and IoT coverage was absent, and most data came from one enterprise. They wrote that a site could build a custom knowledge base if it had suitable endpoint ground truth and network monitoring, but acknowledged the significant initial investment.&lt;/p&gt;
&lt;p&gt;That is the operational lesson. A classifier is not made current by having a good fingerprint format. It stays current through labelled collection, joining, curation, expiry and evaluation.&lt;/p&gt;
&lt;p&gt;For a closer reading, see &lt;a href="/learning/fingerprinting/destination-context-tls-attribution/"&gt;how destination context changes TLS attribution&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;EVE puts the research into a firewall product&lt;/h2&gt;
&lt;p&gt;Cisco describes its Encrypted Visibility Engine as work based on that earlier destination-context research. EVE first appeared in &lt;a href="https://www.cisco.com/c/en/us/td/docs/security/firepower/710/relnotes/firepower-release-notes-710/features.html"&gt;Secure Firewall 7.1&lt;/a&gt; as a disabled-by-default experimental beta for visibility only; it did not enforce actions and Cisco warned that it could produce false positives. The &lt;a href="https://www.cisco.com/c/en/us/td/docs/security/secure-firewall/release-notes/threat-defense/720/threat-defense-release-notes-72.html"&gt;7.2 release notes&lt;/a&gt; say EVE began working with QUIC and allowed high-confidence process assignments to feed application policy. &lt;a href="https://www.cisco.com/c/en/us/td/docs/security/secure-firewall/release-notes/threat-defense/730/threat-defense-release-notes-73.html"&gt;Version 7.3&lt;/a&gt; specifically added HTTP/3 and SMB-over-QUIC detection, along with indication-of-compromise events for unsafe applications.&lt;/p&gt;
&lt;p&gt;In Cisco's account of &lt;a href="https://blogs.cisco.com/security/how-eve-detects-malicious-uses-of-trustworthy-cloud-services"&gt;how EVE detects malicious use of trustworthy cloud services&lt;/a&gt;, the runtime inputs have the same recognisable shape:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;an NPF representing client-side protocol characteristics; and&lt;/li&gt;
&lt;li&gt;server context such as address, port and domain name.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;EVE uses machine learning over Cisco's labelled data to estimate the client process and detect suspicious encrypted traffic without decrypting its payload. Cisco says its training data is refreshed daily with network samples joined to endpoint ground truth, with additional malicious observations from Cisco Secure Malware Analytics.&lt;/p&gt;
&lt;p&gt;This is the clearest public link from NPF and the 2020 paper to the commercial system. It is also where careful wording matters most.&lt;/p&gt;
&lt;p&gt;The public Mercury repository is not a source release of the complete EVE service. Building Mercury does not provide Cisco's continuously collected production dataset, its current models, Secure Firewall integration, Talos context or product policy behaviour. Conversely, the fact that EVE uses NPF does not mean every detail of its current implementation is present in the open repository or frozen at the method described in the 2020 paper.&lt;/p&gt;
&lt;p&gt;Open code, published research and a maintained security product have different release cycles and different evidence behind them.&lt;/p&gt;
&lt;h2&gt;What the lineage actually establishes&lt;/h2&gt;
&lt;p&gt;There is a coherent progression:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;Joy
  broad flow and intraflow metadata for research

Mercury + NPF
  faster collection and versioned, multi-protocol fingerprints

2020 destination-context system
  fingerprints joined to destinations and endpoint labels

EVE
  Cisco-maintained classification in Secure Firewall
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;This is a lineage of ideas and systems, not a promise that each box is a drop-in edition of the next one.&lt;/p&gt;
&lt;p&gt;Joy established a practical way to collect visible evidence around encrypted flows. Mercury made protocol fingerprinting a more explicit and efficient part of that collection. NPF gave those fingerprints a structured, versioned form. The destination-context work showed why labelled, continuously refreshed context was necessary for process attribution. EVE operationalised those ideas inside a commercial firewall environment with data and integrations that do not ship with the public collector.&lt;/p&gt;
&lt;p&gt;That history also explains why Cisco's branch of fingerprinting developed differently from JA3 and JA4. A portable hash is useful for logging and exchange. Cisco's work kept returning to a harder question: what additional evidence, labels and maintenance are required before a network observation can support a process assessment?&lt;/p&gt;
&lt;p&gt;Neither approach turns a handshake into identity. Our &lt;a href="/blog/two-lineages-tls-fingerprinting/"&gt;two lineages of TLS fingerprinting&lt;/a&gt; article places Cisco's work alongside JA3 and JA4. The reproducible &lt;a href="/blog/one-clienthello-ja3-ja4-mercury-lab/"&gt;one ClientHello, three fingerprints lab&lt;/a&gt; shows the format differences without relying on any vendor classification database.&lt;/p&gt;</content><category term="Security"></category><category term="Cisco Joy"></category><category term="Cisco Mercury"></category><category term="Encrypted Visibility Engine"></category><category term="TLS Fingerprinting"></category><category term="Network Fingerprinting"></category><category term="Encrypted Traffic"></category></entry><entry><title>Two Lineages of TLS Fingerprinting: JA3, JA4 and Cisco Mercury</title><link href="https://www.peakhour.io/blog/two-lineages-tls-fingerprinting/" rel="alternate"></link><published>2026-07-26T09:00:00+10:00</published><updated>2026-07-26T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-07-26:/blog/two-lineages-tls-fingerprinting/</id><summary type="html">&lt;p&gt;JA4 did not descend from Cisco Mercury. The two projects come from different strands of TLS fingerprinting research and solve different operational problems.&lt;/p&gt;</summary><content type="html">&lt;p&gt;It is tempting to draw the history of TLS fingerprinting as a single line: JA3, then JA4, with Cisco Mercury somewhere nearby. That version is tidy. It is also wrong.&lt;/p&gt;
&lt;p&gt;Two strands of work developed around the same observation: a TLS ClientHello exposes enough information to say something useful about the software that created it. One strand concentrated on portable identifiers that could be logged and exchanged. The other concentrated on retaining protocol structure and combining it with evidence that could improve classification.&lt;/p&gt;
&lt;p&gt;JA3 and JA4 belong mainly to the first strand. Cisco Mercury belongs mainly to the second. For the technical work that preceded JA3, see &lt;a href="/blog/before-ja3-tls-fingerprinting-history/"&gt;how TLS handshakes became fingerprints&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Before JA3&lt;/h2&gt;
&lt;p&gt;Passive fingerprinting predates TLS. Tools such as p0f identified operating-system characteristics from TCP/IP behaviour without sending probes to the target. Researchers later applied the same instinct to fields exposed during SSL and TLS negotiation.&lt;/p&gt;
&lt;p&gt;In 2009, Ivan Ristić published an &lt;a href="https://blog.ivanristic.com/2009/06/http-client-fingerprinting-using-ssl-handshake-analysis.html"&gt;SSL handshake fingerprinting experiment&lt;/a&gt; that compared ClientHello messages from web clients. Marek Majkowski followed with a &lt;a href="https://idea.popcount.org/2012-06-17-ssl-fingerprinting-for-p0f/"&gt;TLS fingerprinting patch for p0f&lt;/a&gt; in 2012. Lee Brotherston's &lt;a href="https://github.com/LeeBrotherston/tls-fingerprinting"&gt;FingerprinTLS&lt;/a&gt; later provided tools and a database for creating and matching TLS fingerprints.&lt;/p&gt;
&lt;p&gt;Salesforce's JA3 project drew directly on that work. JA3 serialised five ordered ClientHello feature groups, removed GREASE values and calculated an MD5 digest. The result was compact enough to put in a log, share in threat intelligence or match in a rule. The &lt;a href="https://github.com/salesforce/ja3"&gt;archived JA3 repository&lt;/a&gt; documents both the format and its debt to FingerprinTLS.&lt;/p&gt;
&lt;p&gt;JA3's compactness came with a cost. A digest does not explain why two clients differ. Ordered inputs also meant that harmless permutation could produce a different value. Most importantly, a matching digest did not prove that the traffic came from one application. Programs built on a shared TLS library could produce the same ClientHello.&lt;/p&gt;
&lt;h2&gt;The JA4 branch&lt;/h2&gt;
&lt;p&gt;FoxIO introduced JA4 in 2023 after Chrome began permuting TLS extension order. Peakhour saw the practical effect of that change in our &lt;a href="/blog/tls-extension-randomisation/"&gt;Chrome extension-randomisation analysis&lt;/a&gt;: a representation that preserved extension order split one common browser family into a large number of values.&lt;/p&gt;
&lt;p&gt;JA4 canonicalises selected ClientHello features before hashing them. Its &lt;code&gt;a_b_c&lt;/code&gt; structure keeps a readable summary in the first section, a digest of sorted cipher identifiers in the second, and a digest derived from extensions and signature algorithms in the third. This makes the components useful independently. An analyst can group on part of a JA4 value without pretending every field is identical.&lt;/p&gt;
&lt;p&gt;That is deliberate lossy compression. JA4 is useful because it throws away distinctions its designers judged unstable or unhelpful for this job. It is not a reversible rendering of the ClientHello, and its truncated SHA-256 sections do not provide a measure of semantic distance. The exact format is set out in the &lt;a href="https://github.com/FoxIO-LLC/ja4/blob/main/technical_details/JA4.md"&gt;FoxIO JA4 technical specification&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;JA4 is one method. JA4+ is the name used for a wider family that includes server, HTTP, TCP, SSH, certificate and other fingerprints. Those methods do not all share JA4's licence, which matters if the fingerprints will be built into a commercial service.&lt;/p&gt;
&lt;h2&gt;The Cisco research branch&lt;/h2&gt;
&lt;p&gt;Cisco's work took a different route. In 2016, Blake Anderson, Subharthi Paul and David McGrew studied how observable TLS features could help distinguish malware from enterprise traffic without decrypting it. Their paper, &lt;a href="https://arxiv.org/abs/1607.01639"&gt;Deciphering Malware's Use of TLS&lt;/a&gt;, also dealt with an awkward issue that still matters: malware-sandbox data can bias a classifier.&lt;/p&gt;
&lt;p&gt;Anderson and McGrew's 2017 &lt;a href="https://arxiv.org/abs/1706.08003"&gt;operating-system fingerprinting research&lt;/a&gt; combined evidence from TCP/IP, TLS and HTTP across multiple sessions. The point was not to mint a universally portable hash. It was to ask whether several kinds of passive evidence, accumulated over time, reduced uncertainty about the endpoint.&lt;/p&gt;
&lt;p&gt;The same multi-protocol approach appears in Cisco's Joy and Mercury projects. Mercury's Network Protocol Fingerprinting format represents selected protocol features as a tree of hexadecimal byte strings. The full form retains structure. Its naming can state the protocol and fingerprint rule version. An optional compact hash can be used where a fixed-length value is more practical. Cisco's current &lt;a href="https://github.com/cisco/mercury/blob/main/doc/npf.md"&gt;draft NPF specification&lt;/a&gt; defines fingerprints for TLS, QUIC, TCP, HTTP, SSH and other protocols.&lt;/p&gt;
&lt;p&gt;Mercury also keeps fingerprint generation separate from process classification. That distinction is easy to miss.&lt;/p&gt;
&lt;h2&gt;A fingerprint and a label are different things&lt;/h2&gt;
&lt;p&gt;In Cisco's 2020 paper, &lt;a href="https://arxiv.org/abs/2009.01939"&gt;Accurate TLS Fingerprinting Using Destination Context and Knowledge Bases&lt;/a&gt;, the authors found that common TLS fingerprints mapped to many processes. For the 100 most prevalent fingerprints in their May 2020 data, the median was 24.5 process names per fingerprint.&lt;/p&gt;
&lt;p&gt;Their response was not a longer hash. They combined the fingerprint with destination address, port and server name, then used a weighted naïve Bayes classifier backed by a continually updated knowledge base.&lt;/p&gt;
&lt;p&gt;That produces an inference, not a property embedded in the fingerprint string. The result depends on labelled observations, their age, the monitored environment and the destination evidence available for the connection. The open Mercury repository can generate fingerprints without possessing Cisco's production knowledge base.&lt;/p&gt;
&lt;p&gt;This is the clearest difference between the two lineages:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;JA3 and JA4 define portable representations for selected TLS observations.&lt;/li&gt;
&lt;li&gt;Mercury NPF retains a richer, versioned representation that can be fed into a separate analysis system.&lt;/li&gt;
&lt;li&gt;Mercury's destination-context classifier is another layer again.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;None of these layers proves who made a request.&lt;/p&gt;
&lt;h2&gt;Where the lineages meet&lt;/h2&gt;
&lt;p&gt;The projects respond to many of the same protocol changes. Both JA4 and recent Mercury formats sort selected TLS fields to reduce instability caused by permutation. Both deal explicitly with GREASE. Both recognise that operators need compact values for logs as well as enough detail to investigate differences.&lt;/p&gt;
&lt;p&gt;They make different trade-offs. JA4 is convenient for grouping and interchange. Mercury's full NPF form is better suited to inspection and to analysis that benefits from retained structure. JA4's wider family adds fingerprints for other observations, while Mercury is also a packet metadata collector and protocol-analysis library. Comparing only the length of their hashes misses most of the design.&lt;/p&gt;
&lt;p&gt;The lab article makes that concrete. In &lt;a href="/blog/one-clienthello-ja3-ja4-mercury-lab/"&gt;one ClientHello, three fingerprints&lt;/a&gt;, we run JA3, JA4 and Mercury against the same packet capture, record the exact tool versions and compare what each output preserves.&lt;/p&gt;</content><category term="Security"></category><category term="TLS Fingerprinting"></category><category term="JA3"></category><category term="JA4"></category><category term="Cisco Mercury"></category><category term="Network Fingerprinting"></category><category term="Threat Detection"></category></entry><entry><title>Before JA3: How TLS Handshakes Became Fingerprints</title><link href="https://www.peakhour.io/blog/before-ja3-tls-fingerprinting-history/" rel="alternate"></link><published>2026-07-19T09:00:00+10:00</published><updated>2026-07-19T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-07-19:/blog/before-ja3-tls-fingerprinting-history/</id><summary type="html">&lt;p&gt;JA3 made TLS fingerprints easy to log and share, but the technical ideas behind it had already been tested in SSL Labs experiments, a p0f patch and FingerprinTLS.&lt;/p&gt;</summary><content type="html">&lt;p&gt;JA3 is often treated as the beginning of TLS fingerprinting. It was not. Its real contribution was narrower and, operationally, just as important: JA3 took a set of ideas that had been explored for years and turned them into a small identifier that ordinary security tools could carry.&lt;/p&gt;
&lt;p&gt;The path to that format runs through an SSL Labs experiment in 2009, an experimental p0f extension in 2012 and Lee Brotherston's FingerprinTLS work in 2015. Each step answered a different question. What can the cleartext handshake reveal? Which details survive often enough to identify a client? How do you turn those details into something an analyst can match? And, finally, how do you make the result portable?&lt;/p&gt;
&lt;p&gt;This is a documented lineage where the authors themselves cite the earlier work. The claim that JA3's decisive move was simplification is our interpretation of those sources, not a claim that every project shared one design plan.&lt;/p&gt;
&lt;h2&gt;2009: the cipher list as a client signature&lt;/h2&gt;
&lt;p&gt;In June 2009, Ivan Ristić described an experiment in &lt;a href="https://blog.ivanristic.com/2009/06/http-client-fingerprinting-using-ssl-handshake-analysis.html"&gt;HTTP client fingerprinting using SSL handshake analysis&lt;/a&gt;. He was working on SSL Labs and noticed a useful property of the initial handshake: clients sent different lists of supported cipher suites, and those lists were visible before encryption began.&lt;/p&gt;
&lt;p&gt;The important observation was not that any one cipher identified a browser. It was the combination of ciphers a client offered. Ristić recorded the entire list as a signature and compared it with the HTTP User-Agent seen after the connection was established. He then published &lt;a href="https://blog.ivanristic.com/2009/07/examples-of-the-information-collected-from-ssl-handshakes.html"&gt;examples collected from real SSL handshakes&lt;/a&gt;, showing that the approach could separate a range of browsers, command-line clients and crawlers.&lt;/p&gt;
&lt;p&gt;This early method was deliberately modest. It concentrated on the cipher-suite list. It did not define a general-purpose fingerprint containing every useful ClientHello feature, and it did not claim that a signature proved the identity of a process.&lt;/p&gt;
&lt;p&gt;Even so, the core technical idea was in place:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;unencrypted ClientHello
  -&amp;gt; implementation-dependent choices
  -&amp;gt; repeatable signature
  -&amp;gt; comparison with previously observed clients
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The handshake was no longer just cryptographic setup. It was passive metadata about the software constructing it.&lt;/p&gt;
&lt;h2&gt;2012: p0f adds order, extensions and matching rules&lt;/h2&gt;
&lt;p&gt;Marek Majkowski pushed the idea further in his 2012 &lt;a href="https://idea.popcount.org/2012-06-17-ssl-fingerprinting-for-p0f/"&gt;SSL fingerprinting patch for p0f&lt;/a&gt;. p0f was already known for passive operating-system fingerprinting at lower layers. Majkowski applied a similar signature-and-database model to SSL and TLS ClientHello messages.&lt;/p&gt;
&lt;p&gt;His post explicitly credits Ristić's 2009 work, then points to two details he believed deserved more attention: ordering and TLS extensions. The patch represented a fingerprint as four fields:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;requested version : ordered ciphers : ordered extensions : flags
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;That is a meaningful technical step. Cipher suites are normally sent in preference order, and extension order can differ between implementations. Retaining those sequences gives the signature more discriminatory power than an unordered inventory. The flags recorded other behaviours, such as compression support or an unusual relationship between record and handshake versions.&lt;/p&gt;
&lt;p&gt;The implementation did more than print a string. It matched the result against a database of signatures that could contain wildcards and return a browser family, possible versions and sometimes a platform. The &lt;a href="https://gist.github.com/majek/2721464"&gt;original p0f patch and notes&lt;/a&gt; are still useful because they expose the boundary between observation and label: one part generates the raw signature; another compares it with knowledge gathered elsewhere.&lt;/p&gt;
&lt;p&gt;Majkowski also wrote with appropriate caution. His notes say that a ClientHello can sometimes identify the underlying SSL library and, for software with a custom build or distinctive feature set, may narrow the application version. "Sometimes" matters. Two applications using the same TLS stack can look alike, while one application can change its handshake when its library, configuration or build changes.&lt;/p&gt;
&lt;p&gt;The p0f work did not become the universal exchange format for TLS fingerprints. It did, however, demonstrate most of the ingredients that later systems would reuse: selected fields, preserved order, a serialised signature and a separate matching database.&lt;/p&gt;
&lt;h2&gt;2015: FingerprinTLS turns a method into a toolset&lt;/h2&gt;
&lt;p&gt;Lee Brotherston's 2015 work expanded the practical surface again. His DerbyCon and SecTor presentation, &lt;a href="https://archives.sector.ca/presentations15/BrotherstonTLS%20Fingerprinting%20SecTor.pdf"&gt;Stealthier Attacks and Smarter Defending with TLS Fingerprinting&lt;/a&gt;, examined TLS fingerprinting from both sides: defenders could recognise unexpected software, while an operator could alter a client's handshake to blend in or evade a simplistic rule.&lt;/p&gt;
&lt;p&gt;The associated &lt;a href="https://github.com/LeeBrotherston/tls-fingerprinting"&gt;FingerprinTLS repository&lt;/a&gt; packaged the approach into several working parts:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;FingerprinTLS&lt;/code&gt; detected TLS sessions on a live interface or in a PCAP, created fingerprints and matched them;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;fingerprints.json&lt;/code&gt; stored the known fingerprint database;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;Fingerprintout&lt;/code&gt; exported observations into other forms, including Snort and Suricata rules.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This was broader than calculating a digest. It was a workflow for capturing a handshake, retaining many of its characteristics, associating that observation with known software and moving the result into operational tools.&lt;/p&gt;
&lt;p&gt;The project also made an awkward truth visible: rich fingerprints are not especially convenient interchange formats. FingerprinTLS could inspect more detail, but exporting that detail into a rule language could lose accuracy. Its README warned that Snort and Suricata exports might require tuning because their rule syntax could not express the full matching logic.&lt;/p&gt;
&lt;p&gt;That trade-off set the stage for JA3. A detailed signature helps an analyst explain why two handshakes differ. A compact value is easier to add to a connection log, compare across sensors and share with another team. It is difficult to optimise one representation for both jobs.&lt;/p&gt;
&lt;h2&gt;2017: JA3 chooses portability&lt;/h2&gt;
&lt;p&gt;Salesforce open-sourced JA3 in 2017. John Althouse's original &lt;a href="https://engineering.salesforce.com/open-sourcing-ja3-92c9e53c3c41/"&gt;JA3 announcement&lt;/a&gt; directly cites Ristić's 2009 post and Brotherston's 2015 research. It also states the team's design requirement plainly: the result had to work with existing monitoring systems and load balancers, be independent of the destination, and be easy for other tools to consume.&lt;/p&gt;
&lt;p&gt;JA3 selected five ordered ClientHello feature groups:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;TLS version,
cipher suites,
extension types,
supported groups,
elliptic-curve point formats
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;It rendered their numeric values into a comma-separated string, removed GREASE values, then calculated an MD5 digest. A long handshake description became a 32-character identifier.&lt;/p&gt;
&lt;p&gt;MD5 was not being used here to protect a password or prove integrity. It was a compact naming function. The security weakness of MD5 still means a JA3 value should not be treated as proof, but changing to a stronger digest would not solve the more common identification problem: unrelated software can naturally produce the same selected features, and software can deliberately copy another client's ClientHello.&lt;/p&gt;
&lt;p&gt;The simplification was substantial. Compared with FingerprinTLS, JA3 retained fewer fields and discarded the explanatory structure once the string was hashed. Compared with the p0f patch, it did not carry matching wildcards or classification rules in the fingerprint. What it gained was a common unit that could fit almost anywhere an operator could put a string.&lt;/p&gt;
&lt;p&gt;That was why JA3 travelled. A sensor could calculate the value, a SIEM could index it, an intelligence report could publish it and a rule could match it without every participant adopting the same fingerprint database or packet parser.&lt;/p&gt;
&lt;h2&gt;What JA3 inherited, and what it left behind&lt;/h2&gt;
&lt;p&gt;The documented history supports a few specific claims:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Ristić showed in 2009 that an unencrypted SSL handshake, particularly its cipher list, could distinguish HTTP clients.&lt;/li&gt;
&lt;li&gt;Majkowski's 2012 p0f work explicitly built on that experiment and added ordered extensions, behavioural flags and database matching.&lt;/li&gt;
&lt;li&gt;Brotherston's 2015 research and FingerprinTLS made detailed capture, matching, creation and export available as a standalone toolset.&lt;/li&gt;
&lt;li&gt;Salesforce cited the earlier work when it released JA3 and designed a smaller representation for existing operational systems.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;It does not support treating a fingerprint as an application identity. Every stage depended on a set of observed features and, when a software name was returned, knowledge collected outside the handshake itself. The label could be useful without being certain.&lt;/p&gt;
&lt;p&gt;That distinction is easier to see when the formats are run side by side. Our reproducible lab feeds &lt;a href="/blog/one-clienthello-ja3-ja4-mercury-lab/"&gt;one ClientHello to JA3, JA4 and Cisco Mercury&lt;/a&gt; and records both the compact outputs and the detail each format preserves.&lt;/p&gt;
&lt;p&gt;JA3 was not the final step either. JA4 later changed the normalisation and output structure to cope with modern sources of instability, while Cisco's research followed a more structured, context-aware path. Those are separate branches, not one straight succession. We trace them in &lt;a href="/blog/two-lineages-tls-fingerprinting/"&gt;Two Lineages of TLS Fingerprinting&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The useful lesson from the early history is not who coined the first fingerprint. It is that the representation determines the work you can do with it. Rich detail helps investigation. Compact identifiers help distribution. Neither turns a handshake into an identity document.&lt;/p&gt;</content><category term="Security"></category><category term="TLS Fingerprinting"></category><category term="JA3"></category><category term="FingerprinTLS"></category><category term="p0f"></category><category term="Network Fingerprinting"></category><category term="Security Research"></category></entry><entry><title>One ClientHello, Three Fingerprints: JA3, JA4 and Mercury</title><link href="https://www.peakhour.io/blog/one-clienthello-ja3-ja4-mercury-lab/" rel="alternate"></link><published>2026-07-12T09:00:00+10:00</published><updated>2026-07-12T09:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2026-07-12:/blog/one-clienthello-ja3-ja4-mercury-lab/</id><summary type="html">&lt;p&gt;A reproducible lab runs JA3, JA4 and Cisco Mercury against the same TLS ClientHello and compares what each fingerprint preserves.&lt;/p&gt;</summary><content type="html">&lt;p&gt;The easiest way to misunderstand network fingerprints is to compare example strings taken from different clients. We wanted a cleaner test: one packet capture, one TLS ClientHello and three fingerprint formats.&lt;/p&gt;
&lt;p&gt;The complete lab is checked into this site's source under &lt;code&gt;labs/network-fingerprinting/&lt;/code&gt;, and the &lt;a href="/static/downloads/network-fingerprinting-lab.tar.gz"&gt;publication bundle is available here&lt;/a&gt;. It pins the tool revisions, reconstructs the fixture, verifies its checksum, runs the tools and checks that their outputs refer to the same connection. Nothing in the comparison depends on a vendor database or an application label.&lt;/p&gt;
&lt;h2&gt;The input&lt;/h2&gt;
&lt;p&gt;The fixture is a 329-byte Peakhour-generated capture containing a local OpenSSL 3.5.6 ClientHello wrapped in one synthetic Ethernet/IPv4/TCP packet:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;10.1.1.1:40000 -&amp;gt; 10.2.2.2:443
SNI: lab.peakhour.test
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The lab stores the small fixture as base64 under an adjacent BSD 3-Clause licence and verifies the decoded PCAP with SHA-256 before using it. The reserved SNI and private addresses did not cross a network. We select packet 1 and TCP stream 0. That selection matters: saying that several tools read the same PCAP is weaker than proving that their output describes the same flow and ClientHello.&lt;/p&gt;
&lt;p&gt;The pinned revisions for this run are:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;Cisco Mercury  3172786645f70e1a8347d8cf020b736e185651e5
FoxIO JA4      0e54bc8371de34df94a35f2442c05bda2e8b2034
Salesforce JA3 502cc6395811c54743b0561419d61900a6df3ff7
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;These pins are part of the result. Fingerprint implementations and specifications change. A value without its method and version is harder to reproduce than it first appears.&lt;/p&gt;
&lt;h2&gt;Running the lab&lt;/h2&gt;
&lt;p&gt;From the repository root:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;./labs/network-fingerprinting/run.sh
python&lt;span class="w"&gt; &lt;/span&gt;labs/network-fingerprinting/verify.py
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The runner fetches the pinned source archives, builds or invokes the implementations in a temporary work directory, and writes evidence to &lt;code&gt;labs/network-fingerprinting/results/&lt;/code&gt;. The verifier checks the fixture and output checksums, connection tuple, SNI and output shape. The pinned source URLs are enforced by the runner rather than inferred by the verifier.&lt;/p&gt;
&lt;p&gt;This is a fingerprint-format lab, not a speed test. Build time, runtime and memory use depend heavily on language, wrapper and capture path, so we do not compare them here.&lt;/p&gt;
&lt;h2&gt;JA3: a portable exact-match digest&lt;/h2&gt;
&lt;p&gt;For this ClientHello, the canonical JA3 feature string is:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;771,49196-49200-159-52393-52392-52394-49195-49199-158-49188-49192-107-49187-49191-103-49162-49172-57-49161-49171-51-157-156-61-60-53-47,65281-0-11-10-35-16-22-23-13,29-23-30-24-25,0-1-2
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Its MD5 digest is:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;e1934f32e97b0bd52227953ca7d30118
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The digest is convenient for logs and exact lookup. On its own it does not show which cipher, extension or group changed. The pre-hash string retains enough information to investigate that difference, which is why throwing it away too early can make later analysis harder.&lt;/p&gt;
&lt;p&gt;JA3 removes GREASE values but otherwise retains the order of its selected lists. A client that permutes extension order can therefore generate a new JA3 digest without changing its effective TLS capabilities. The &lt;a href="https://github.com/salesforce/ja3"&gt;archived Salesforce JA3 repository&lt;/a&gt; defines the input fields and GREASE handling.&lt;/p&gt;
&lt;h2&gt;JA4: canonicalised components&lt;/h2&gt;
&lt;p&gt;The same ClientHello produces this JA4:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;t12d2709h2_a2460661a67a_36cef8aed422
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Its first section is readable:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;t&lt;/code&gt; means TLS over TCP;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;12&lt;/code&gt; is the highest supported TLS version after ignoring GREASE;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;d&lt;/code&gt; says a domain was present in SNI;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;27&lt;/code&gt; and &lt;code&gt;09&lt;/code&gt; are the cipher and extension counts after the format's exclusions;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;h2&lt;/code&gt; summarises the first ALPN value.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The second section is the first 12 hexadecimal characters of SHA-256 over sorted cipher identifiers. The third is a truncated SHA-256 value derived from sorted extension identifiers and the signature algorithms in their original order. The canonical &lt;a href="https://github.com/FoxIO-LLC/ja4/blob/main/technical_details/JA4.md"&gt;JA4 technical specification&lt;/a&gt; defines the exact exclusions and encodings.&lt;/p&gt;
&lt;p&gt;The lab also records &lt;code&gt;JA4_r&lt;/code&gt;, the raw form used by the FoxIO tooling:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;t12d2709h2_002f,0033,0035,0039,003c,003d,0067,006b,009c,009d,009e,009f,c009,c00a,c013,c014,c023,c024,c027,c028,c02b,c02c,c02f,c030,cca8,cca9,ccaa_000a,000b,000d,0016,0017,0023,ff01_0403,0503,0603,0807,0808,0809,080a,080b,0804,0805,0806,0401,0501,0601,0303,0301,0302,0402,0502,0602
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;That makes the normalisation visible. It also shows why JA4 is not fuzzy hashing: sorting makes selected permutations equivalent, while the hashes still support equality matching rather than semantic distance.&lt;/p&gt;
&lt;h2&gt;Mercury NPF: a retained protocol tree&lt;/h2&gt;
&lt;p&gt;Cisco Mercury 2.18 emits this &lt;code&gt;tls/2&lt;/code&gt; fingerprint for the same ClientHello:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;tls/2/(0303)(c02cc030009fcca9cca8ccaac02bc02f009ec024c028006bc023c0270067c00ac0140039c009c0130033009d009c003d003c0035002f)[(0000)(000a000c000a001d0017001e00180019)(000b000403000102)(000d002a0028040305030603080708080809080a080b080408050806040105010601030303010302040205020602)(0010000e000c02683208687474702f312e31)(0016)(0017)(ff01)]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The value is longer because it is doing another job. Parentheses and square brackets describe an ordered tree of selected byte strings. In the draft NPF notation, square brackets mark a lexicographically sorted list. The &lt;code&gt;tls/2&lt;/code&gt; prefix names the protocol and fingerprint rule version.&lt;/p&gt;
&lt;p&gt;An analyst can inspect the retained values rather than relying only on a digest. Mercury also defines a compact hash nickname when a fixed-length index is needed, but that nickname loses the structure used for inspection, prefix comparison or approximate matching. Cisco's &lt;a href="https://github.com/cisco/mercury/blob/main/doc/npf.md"&gt;draft NPF specification&lt;/a&gt; documents both representations.&lt;/p&gt;
&lt;p&gt;The Mercury JSON includes the same source and destination tuple and the same SNI as the JA3 and JA4 records. It does not identify the client application in this lab because we did not run a labelled fingerprint knowledge base or the destination-context classifier. A packet-derived NPF value and a process assessment are separate outputs.&lt;/p&gt;
&lt;h2&gt;What the comparison establishes&lt;/h2&gt;
&lt;p&gt;All three methods observe the same ClientHello, but they define similarity differently.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Question&lt;/th&gt;
&lt;th&gt;JA3&lt;/th&gt;
&lt;th&gt;JA4&lt;/th&gt;
&lt;th&gt;Mercury NPF &lt;code&gt;tls/2&lt;/code&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Compact default&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;No; optional hash available&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Inspectable selected inputs&lt;/td&gt;
&lt;td&gt;Only if the pre-hash string is retained&lt;/td&gt;
&lt;td&gt;Partly in &lt;code&gt;a&lt;/code&gt;; fully in the recorded raw form&lt;/td&gt;
&lt;td&gt;Yes in the full tree&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Selected list sorting&lt;/td&gt;
&lt;td&gt;No, after GREASE removal&lt;/td&gt;
&lt;td&gt;Ciphers and most extensions&lt;/td&gt;
&lt;td&gt;Rule-specific selected extensions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Explicit format version in value&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;Encoded field semantics, but no separate rule number&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Semantic or approximate comparison&lt;/td&gt;
&lt;td&gt;Not from the digest&lt;/td&gt;
&lt;td&gt;Component grouping, not hash distance&lt;/td&gt;
&lt;td&gt;Full structure can support richer matching&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Application attribution in the format&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The table does not produce a universal winner. JA3 remains useful where historical compatibility matters. JA4 is compact and handles selected permutations cleanly. Mercury retains more material for inspection and for analysis systems that need structured features.&lt;/p&gt;
&lt;p&gt;It also shows what none of the values can establish. The capture does not prove which person, device or application created the connection. Shared libraries, browser impersonation and software updates all complicate that inference. See &lt;a href="/blog/fingerprint-is-a-cohort-not-a-client/"&gt;A network fingerprint is a cohort, not a client&lt;/a&gt; for the operational consequences.&lt;/p&gt;
&lt;p&gt;For the history behind these design choices, read &lt;a href="/blog/two-lineages-tls-fingerprinting/"&gt;Two lineages of TLS fingerprinting&lt;/a&gt;. The durable format comparison is in &lt;a href="/learning/fingerprinting/mercury-vs-ja4-vs-ja3/"&gt;Mercury vs JA4 vs JA3&lt;/a&gt;.&lt;/p&gt;</content><category term="Security"></category><category term="TLS Fingerprinting"></category><category term="JA3"></category><category term="JA4"></category><category term="Cisco Mercury"></category><category term="Network Fingerprinting"></category><category term="Security Research"></category></entry><entry><title>The Invisibility Cloak</title><link href="https://www.peakhour.io/blog/bots-residential-proxies-anti-detect-browsers/" rel="alternate"></link><published>2025-09-01T00:00:00+10:00</published><updated>2025-09-01T00:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2025-09-01:/blog/bots-residential-proxies-anti-detect-browsers/</id><summary type="html">&lt;p&gt;Learn how attackers combine residential proxies and anti-detect browsers to evade detection and how modern security tools can fight back.&lt;/p&gt;</summary><content type="html">&lt;p&gt;Every time you connect to a website, you leave behind a "digital fingerprint." This is not a physical fingerprint, but a set of signals from your device and browser. Security tools analyse this fingerprint—which includes your IP address, browser type, operating system, supported fonts, and even subtle characteristics of your network connection (&lt;a href="/blog/tls-fingerprinting/"&gt;TLS fingerprinting&lt;/a&gt;)—to distinguish legitimate users from malicious bots.&lt;/p&gt;
&lt;p&gt;For years, this was a reliable way to spot automated threats. Bots often had clumsy, inconsistent fingerprints that made them easier to identify. Today, attackers can combine tools that mimic real users closely enough to weaken many traditional defences. The two most important components of this modern "invisibility cloak" are &lt;a href="/products/residential-proxy-detection/"&gt;residential proxies&lt;/a&gt; and anti-detect browsers.&lt;/p&gt;
&lt;h2&gt;What Are Residential Proxies?&lt;/h2&gt;
&lt;p&gt;A residential proxy is an intermediary server that uses an IP address assigned by an Internet Service Provider (ISP) to a real home internet connection. When a bot routes its traffic through a residential proxy, its requests appear to originate from a genuine home user, not a data centre.&lt;/p&gt;
&lt;p&gt;These proxy networks are large, often containing millions of IP addresses sourced from around the globe. How are these IPs obtained? Often through questionable means:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Malware and Botnets&lt;/strong&gt;: Unsuspecting users' devices are infected with malware that turns them into proxy endpoints.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;SDKs in Free Apps&lt;/strong&gt;: Some free applications (often VPNs or mobile apps) include code that enrols the user's device into a proxy network in exchange for using the app, often without the user's full knowledge or consent.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;By rotating through this large pool of legitimate-looking IPs, attackers can launch large-scale attacks that are difficult to separate from normal traffic. To a website's security system, a distributed attack from a residential proxy network looks like thousands of individual customers from different locations.&lt;/p&gt;
&lt;h2&gt;What Are Anti-Detect Browsers?&lt;/h2&gt;
&lt;p&gt;While residential proxies mask the attacker's network location, anti-detect browsers are designed to spoof the rest of the digital fingerprint. These specialised browsers allow an attacker to create and manage thousands of unique browser profiles, each with a customised and consistent fingerprint.&lt;/p&gt;
&lt;p&gt;An anti-detect browser can control and randomise every detail a website uses for identification, including:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Browser type and version (e.g., Chrome, Firefox, Safari)&lt;/li&gt;
&lt;li&gt;Operating system (Windows, macOS, iOS, Android)&lt;/li&gt;
&lt;li&gt;Screen resolution, fonts, and plugins&lt;/li&gt;
&lt;li&gt;Time zone and language settings&lt;/li&gt;
&lt;li&gt;Subtle browser characteristics like Canvas and WebGL rendering&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;With a few clicks, an attacker can make a single machine in one country appear as thousands of unique users on different devices and operating systems from all over the world.&lt;/p&gt;
&lt;h2&gt;The Combined Threat: A Perfect Storm for Attacks&lt;/h2&gt;
&lt;p&gt;When attackers combine residential proxies with anti-detect browsers, they cover both the network and browser layers that many controls rely on. The residential proxy provides a legitimate IP address, and the anti-detect browser provides a consistent, human-looking browser fingerprint.&lt;/p&gt;
&lt;p&gt;This combination makes attacks like large-scale credential stuffing, content scraping, and inventory scalping much harder to distinguish from legitimate user traffic. Each malicious request appears to be from a unique person on a standard device, using a normal home internet connection.&lt;/p&gt;
&lt;h2&gt;Why Traditional Defenses Fail and What to Do About It&lt;/h2&gt;
&lt;p&gt;This level of sophistication weakens traditional security measures:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;IP Blocklists and Reputation Services&lt;/strong&gt;: These struggle when attackers are using a constantly rotating pool of millions of legitimate residential IP addresses. Our own research shows that even the best IP intelligence services &lt;a href="/blog/anti-fraud-residential-proxy-detection/"&gt;fail to detect the vast majority of residential proxy traffic&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Basic Browser Fingerprinting&lt;/strong&gt;: Anti-detect browsers are specifically designed to defeat these checks by providing a consistent and realistic fingerprint.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;To combat this combined threat, organisations need a modern approach to bot detection that looks beyond the surface:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Advanced Network Fingerprinting&lt;/strong&gt;: Instead of just looking at the IP address, modern solutions analyse the underlying characteristics of the network connection itself (like the TLS/JA3 fingerprint). These signatures can often identify the underlying automation tool or proxy network, even when the IP address appears legitimate.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Behavioural Analysis&lt;/strong&gt;: Advanced systems model normal user behaviour—such as mouse movements, typing speed, and page navigation—to identify the subtle, non-human patterns of automation that even sophisticated bots can't perfectly mimic.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Hardware and Rendering Fingerprinting&lt;/strong&gt;: While anti-detect browsers can spoof software-level details, faking the underlying hardware is far more difficult. Advanced techniques, such as those used in &lt;a href="/learning/fingerprinting/what-is-google-picasso/"&gt;Google's Picasso&lt;/a&gt;, analyse how a device renders graphics (e.g., Canvas and WebGL), processes audio, and performs CPU-intensive tasks. This creates a hardware fingerprint based on the unique characteristics of the GPU, audio stack, and CPU clock speed. This fingerprint can reveal inconsistencies between the claimed browser profile and the actual hardware being used. When combined with network fingerprinting and residential proxy detection, this becomes a strong signal for identifying a single machine attempting to impersonate many different users.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Dedicated Residential Proxy Detection&lt;/strong&gt;: Specialised techniques are required to identify traffic coming from residential proxy networks. This is a critical signal, as very few legitimate users have a reason to route their traffic this way.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Attackers using residential proxies and anti-detect browsers are harder to identify, but they still leave signals. Network characteristics, hardware fingerprints, and the behavioural tells of automation give security teams a better chance of separating the bot from the user it is trying to resemble.&lt;/p&gt;</content><category term="Security"></category><category term="Browser Fingerprinting"></category><category term="Fingerprinting"></category><category term="Residential Proxies"></category><category term="Bot Management"></category><category term="TLS Fingerprinting"></category><category term="Credential Stuffing"></category></entry><entry><title>How MTU Fingerprinting Identifies VPNs and Mobile Users</title><link href="https://www.peakhour.io/blog/mtu-fingerprinting-vpn-mobile-detection/" rel="alternate"></link><published>2025-01-15T14:00:00+11:00</published><updated>2025-01-15T14:00:00+11:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2025-01-15:/blog/mtu-fingerprinting-vpn-mobile-detection/</id><summary type="html">&lt;p&gt;Learn how MTU fingerprinting reveals VPN usage, mobile connections, and network technologies through TCP handshake analysis. Discover practical SQL techniques for dynamic network intelligence.&lt;/p&gt;</summary><content type="html">&lt;p&gt;For traffic analysis, it helps to know how a user reached the service. Are they on a home network, a mobile connection, or a VPN? Deep packet inspection is invasive, but TCP handshake metadata can still carry useful context about the Maximum Transmission Unit (MTU) a connection appears to be using. By analysing those inferred MTU values, we can build "fingerprints" that point to the underlying network technology carrying the connection.&lt;/p&gt;
&lt;p&gt;This article looks at how common technologies affect MTU values and shows how a SQL query can turn that data into useful network labels.&lt;/p&gt;
&lt;h2&gt;What is MTU and Why Does it Change?&lt;/h2&gt;
&lt;p&gt;The Maximum Transmission Unit (MTU) is the largest data packet, or frame, that a network-connected device can transmit. On standard Ethernet networks, this value is typically 1500 bytes. Larger payloads have to be split into chunks that fit that limit.&lt;/p&gt;
&lt;h3&gt;Encapsulation and Tunneling&lt;/h3&gt;
&lt;p&gt;The value starts to shift when tunnelling protocols are involved, including those used by VPNs and mobile networks. These protocols wrap the original data packet inside another packet, a process called encapsulation. The outer packet has its own headers for routing and management.&lt;/p&gt;
&lt;p&gt;This encapsulation "steals" space from the original 1500 bytes available on the physical network. If a tunnelling protocol adds 60 bytes of headers, for example, the maximum size for the &lt;em&gt;original&lt;/em&gt; data packet is now 1440 bytes (&lt;code&gt;1500 - 60&lt;/code&gt;).&lt;/p&gt;
&lt;h3&gt;The Problem with Fragmentation&lt;/h3&gt;
&lt;p&gt;What happens if a device tries to send a 1500-byte packet through this 1440-byte tunnel? The packet has to be broken into smaller pieces, a process called fragmentation. It works, but it is inefficient. Fragmentation consumes CPU resources on the router performing it, adds header overhead to each fragment, and requires the receiving device to reassemble the pieces. The result is lower speed and higher latency.&lt;/p&gt;
&lt;p&gt;To avoid that penalty, operating systems and network devices reduce the MTU of the connection to account for the tunnel's overhead. The amount of the reduction follows from the tunnelling protocol in use. That predictable drop is the basis for MTU fingerprinting.&lt;/p&gt;
&lt;h2&gt;A Guide to Common MTU Values&lt;/h2&gt;
&lt;p&gt;Different technologies add different overheads, which produces distinct MTU values.&lt;/p&gt;
&lt;h3&gt;WireGuard&lt;/h3&gt;
&lt;p&gt;WireGuard is a modern VPN known for its efficiency, but it still adds overhead.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;IPv4 Overhead&lt;/strong&gt;: 60 bytes (20-byte IPv4 header + 8-byte UDP header + 32-byte WireGuard header).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;IPv6 Overhead&lt;/strong&gt;: 80 bytes (40-byte IPv6 header + 8-byte UDP header + 32-byte WireGuard header).&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;On a standard 1500-byte network, that produces predictable MTU values:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;1500 - 60 = 1440 bytes&lt;/code&gt; (WireGuard over IPv4)&lt;/li&gt;
&lt;li&gt;&lt;code&gt;1500 - 80 = 1420 bytes&lt;/code&gt; (WireGuard over IPv6)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;There is a special case with ISPs that use DS-Lite (Dual-Stack Lite) to carry IPv4 traffic over an IPv6 network. This adds another 40-byte IPv6 header, reducing the MTU further.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;1420 - 40 = 1380 bytes&lt;/code&gt; (WireGuard over DS-Lite)&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;OpenVPN&lt;/h3&gt;
&lt;p&gt;OpenVPN is another common VPN solution, but its fingerprint is less tidy. Instead of setting a static interface MTU, OpenVPN often uses a feature called &lt;code&gt;mssfix&lt;/code&gt;. This dynamically adjusts the Maximum Segment Size (MSS) value within the TCP headers of encapsulated packets to prevent fragmentation.&lt;/p&gt;
&lt;p&gt;The MSS is the MTU minus the IP and TCP header sizes (typically 40 bytes for IPv4). The exact MSS value, and therefore the effective MTU, depends on OpenVPN's configuration, including the transport protocol (UDP or TCP), cipher, MAC algorithm, and compression. As noted by security researcher ValdikSS, these unique MSS values can be used to fingerprint a connection with high precision. For example, a common configuration might result in an MSS of 1369, which corresponds to an effective MTU of 1409 (&lt;code&gt;1369 + 40&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;For general analysis, connections with an MTU around &lt;strong&gt;1400&lt;/strong&gt; or &lt;strong&gt;1380&lt;/strong&gt; bytes often indicate OpenVPN or other VPN usage, especially when seen with other factors.&lt;/p&gt;
&lt;h3&gt;Mobile Networks (LTE &amp;amp; 5G)&lt;/h3&gt;
&lt;p&gt;Mobile networks also modify MTU values. When your phone connects to the internet, its data is tunnelled through the carrier's network using the GPRS Tunnelling Protocol (GTP). This encapsulation adds its own layer of headers.&lt;/p&gt;
&lt;p&gt;As detailed by Nick vs Networking, the typical overhead for GTP traffic over an Ethernet transport network is &lt;strong&gt;50 bytes&lt;/strong&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;14 bytes for the Ethernet header&lt;/li&gt;
&lt;li&gt;20 bytes for the outer IPv4 header&lt;/li&gt;
&lt;li&gt;8 bytes for the UDP header&lt;/li&gt;
&lt;li&gt;8 bytes for the GTP header&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For a mobile carrier using a standard 1500-byte MTU on its transport network, the maximum MTU available to the user's device is &lt;strong&gt;1450 bytes&lt;/strong&gt; (&lt;code&gt;1500 - 50&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;Mobile devices don't guess this value; they are explicitly told what MTU to use by the network during the connection setup process (via Protocol Configuration Options). Mobile operators have two choices to avoid fragmentation:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Increase Transport MTU&lt;/strong&gt;: Enable jumbo frames (for example, 1600 bytes or more) on their internal network to accommodate the 50-byte overhead and still provide a full 1500-byte MTU to the user.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Lower Advertised MTU&lt;/strong&gt;: Advertise a lower MTU to the user's device. This is why values such as &lt;strong&gt;1450&lt;/strong&gt; are common. Some operators may configure a more conservative MTU, such as &lt;strong&gt;1300 bytes&lt;/strong&gt;, to maintain stability across all parts of their network.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;Other Common Values&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Standard Ethernet&lt;/strong&gt;: The baseline is &lt;strong&gt;1500 bytes&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;PPPoE&lt;/strong&gt;: Common for DSL connections, adds 8 bytes of overhead, resulting in an MTU of &lt;strong&gt;1492 bytes&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;IPv6 Minimum&lt;/strong&gt;: The IPv6 specification mandates a minimum MTU of &lt;strong&gt;1280 bytes&lt;/strong&gt;, so this value is also a significant marker.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Analysis with SQL&lt;/h2&gt;
&lt;p&gt;With this context, we can analyse network logs to classify user connections. The following SQL query buckets and attributes MTU values from a large dataset, turning raw numbers into meaningful labels.&lt;/p&gt;
&lt;p&gt;The query works in several stages:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Extract Data&lt;/strong&gt;: It parses the MTU from a fingerprint string in the logs.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Bucket MTUs&lt;/strong&gt;: It uses a &lt;code&gt;CASE&lt;/code&gt; statement to group MTUs. Specific known values, such as 1500, 1440, 1420, and 1380, go into their own buckets. Jumbo frames (&amp;gt;1500) are grouped into 100-byte buckets, and everything else is grouped into 20-byte buckets.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Attribute Buckets&lt;/strong&gt;: In the final &lt;code&gt;SELECT&lt;/code&gt;, another &lt;code&gt;CASE&lt;/code&gt; statement translates those numeric buckets into human-readable descriptions based on the fingerprints we've identified.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;The Query&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Bucketing logic and attribution informed by research from:&lt;/span&gt;
&lt;span class="c1"&gt;-- https://ripx80.de/posts/06-wg-mtu/ (WireGuard)&lt;/span&gt;
&lt;span class="c1"&gt;-- https://medium.com/@ValdikSS/detecting-vpn-and-its-configuration-and-proxy-users-on-the-server-side-1bcc59742413 (OpenVPN)&lt;/span&gt;
&lt;span class="c1"&gt;-- https://nickvsnetworking.com/mtu-in-lte-5g-transmission-networks-part-1/ (Mobile Networks)&lt;/span&gt;
&lt;span class="k"&gt;WITH&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;base_data&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;toInt32OrNull&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;splitByChar&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;:&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;splitByChar&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;,&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;synner_fingerprint&lt;/span&gt;&lt;span class="p"&gt;)[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;])[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;toInt32OrNull&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;splitByChar&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;:&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;splitByChar&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;,&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;synner_fingerprint&lt;/span&gt;&lt;span class="p"&gt;)[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;])[&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;wsize&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;toInt32OrNull&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;splitByChar&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;:&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;splitByChar&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;,&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;synner_fingerprint&lt;/span&gt;&lt;span class="p"&gt;)[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;])[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;scale&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tls&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;handshake_rtt_us&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;min_rtt_us&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;65000&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;is_high_latency&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;logs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;client_logs&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;WHERE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;2025-07-01&amp;#39;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AND&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;shielded&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;
&lt;span class="p"&gt;),&lt;/span&gt;
&lt;span class="n"&gt;main_aggs&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;CASE&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1420&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1420&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1380&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1380&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1501&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;intDiv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1501&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;ELSE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;intDiv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;END&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;countIf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;is_high_latency&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;high_latency_count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;countIf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;not&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;is_high_latency&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;normal_latency_count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;round&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;avg&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;wsize&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;pow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;scale&lt;/span&gt;&lt;span class="p"&gt;)))&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;avg_real_wsize&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;base_data&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;WHERE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AND&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;wsize&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AND&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;scale&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;GROUP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;
&lt;span class="p"&gt;),&lt;/span&gt;
&lt;span class="n"&gt;top_wsizes&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;groupArray&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;wsize&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;cnt&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;top_wsizes&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;FROM&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;CASE&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1420&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1420&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1380&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1380&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1501&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;intDiv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1501&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;ELSE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;intDiv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;END&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="n"&gt;wsize&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;count&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;cnt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="n"&gt;row_number&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;OVER&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;PARTITION&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;ORDER&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;cnt&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;DESC&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;rn&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;base_data&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHERE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AND&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;wsize&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AND&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;scale&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;GROUP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;wsize&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;WHERE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;rn&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;GROUP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;
&lt;span class="p"&gt;),&lt;/span&gt;
&lt;span class="n"&gt;top_scales&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="n"&gt;groupArray&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="k"&gt;scale&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;cnt&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;top_scales&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;FROM&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;CASE&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1420&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1420&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1380&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1380&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1501&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;intDiv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1501&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;
&lt;span class="w"&gt;                &lt;/span&gt;&lt;span class="k"&gt;ELSE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;intDiv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;END&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;scale&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;count&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;cnt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="n"&gt;row_number&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;OVER&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;PARTITION&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;ORDER&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;cnt&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;DESC&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;rn&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;base_data&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHERE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AND&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;wsize&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AND&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;scale&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NOT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;NULL&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;GROUP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;scale&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;WHERE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;rn&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;GROUP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;CASE&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;IN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1420&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1380&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;concat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;-&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;99&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;ELSE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;concat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;-&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;19&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;END&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_range&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;CASE&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Standard Ethernet&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1480&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Likely PPPoE (e.g., 1492)&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1460&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Likely DS-Lite/GRE Tunnel&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Likely Mobile LTE/5G (e.g., 1450) / WireGuard over IPv4&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1420&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;WireGuard over IPv6&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1400&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Likely OpenVPN / Mobile&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1380&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Likely OpenVPN / WireGuard over DS-Lite / Mobile&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1300&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Likely Mobile LTE/5G configured&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1280&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;IPv6 Minimum&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;WHEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;THEN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Jumbo Frame&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;ELSE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;Other&amp;#39;&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;END&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_attribution&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;high_latency_count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;normal_latency_count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;round&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;high_latency_count&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;high_latency_count&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;normal_latency_count&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;AS&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;high_latency_ratio&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;top_wsizes&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;top_scales&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;avg_real_wsize&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;main_aggs&lt;/span&gt;
&lt;span class="k"&gt;LEFT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;JOIN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;top_wsizes&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;USING&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;LEFT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;JOIN&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;top_scales&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;USING&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;high_latency_count&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;normal_latency_count&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;10000&lt;/span&gt;
&lt;span class="k"&gt;ORDER&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;BY&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;mtu_bucket&lt;/span&gt;
&lt;span class="k"&gt;LIMIT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;FORMAT&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Vertical&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h2&gt;Why Jumbo Frames Matter&lt;/h2&gt;
&lt;p&gt;Jumbo frames (MTU values greater than 1500 bytes) are a useful edge case in MTU fingerprinting. These frames, typically ranging from 9000-9216 bytes, are primarily used in high-performance computing environments, data centres, and enterprise networks where throughput optimisation is important.&lt;/p&gt;
&lt;p&gt;When we detect jumbo frame MTUs in our analysis, they often indicate:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Enterprise Users&lt;/strong&gt;: Corporate networks frequently enable jumbo frames for internal communications&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Data Centre Traffic&lt;/strong&gt;: Cloud services and CDNs often use jumbo frames between their infrastructure&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;High-Performance Applications&lt;/strong&gt;: Video streaming, large file transfers, and backup operations can benefit from larger frame sizes&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Network Misconfiguration&lt;/strong&gt;: Jumbo frames sometimes appear because of network equipment misconfiguration&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The presence of jumbo frames can help distinguish consumer and enterprise traffic, adding useful context for traffic classification and security analysis.&lt;/p&gt;
&lt;h2&gt;Practical Use Cases and Applications&lt;/h2&gt;
&lt;p&gt;MTU fingerprinting is useful across several security and operational domains:&lt;/p&gt;
&lt;h3&gt;Security Applications&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;VPN Detection for Compliance&lt;/strong&gt;: Organisations can identify employees bypassing corporate network policies with personal VPNs, supporting compliance with data governance requirements.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Bot Traffic Classification&lt;/strong&gt;: Automated traffic from residential proxy networks often shows consistent MTU patterns that differ from genuine residential users, improving bot detection.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Threat Intelligence Enhancement&lt;/strong&gt;: Correlating MTU patterns with other indicators helps build broader threat profiles and improves attack attribution.&lt;/p&gt;
&lt;h3&gt;Network Operations&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;Performance Optimisation&lt;/strong&gt;: Understanding the MTU distribution of your user base helps optimise content delivery and reduce fragmentation-related performance issues.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Infrastructure Planning&lt;/strong&gt;: MTU analysis reveals the underlying network technologies your users employ, informing CDN placement and capacity planning decisions.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Quality of Service&lt;/strong&gt;: Different MTU patterns correlate with connection quality, enabling proactive support for users on constrained networks.&lt;/p&gt;
&lt;h3&gt;Business Intelligence&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;Market Analysis&lt;/strong&gt;: Geographic and demographic patterns in MTU distribution reveal technology adoption trends and market characteristics.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;User Experience Optimisation&lt;/strong&gt;: Identifying users on mobile or constrained networks enables adaptive content delivery and interface optimisation.&lt;/p&gt;
&lt;h2&gt;Dynamic Analysis vs Static IP Databases&lt;/h2&gt;
&lt;p&gt;MTU fingerprinting is a dynamic signal, which makes it useful alongside static IP reputation databases. It has several practical advantages:&lt;/p&gt;
&lt;h3&gt;Real-Time Adaptation&lt;/h3&gt;
&lt;p&gt;Static IP databases go stale. A residential IP address might be flagged as malicious based on historical activity, but MTU fingerprinting analyses the current network configuration. This dynamic approach captures the infrastructure being used at the moment of connection, providing more accurate and timely intelligence.&lt;/p&gt;
&lt;h3&gt;Circumvention Resistance&lt;/h3&gt;
&lt;p&gt;Attackers can rotate IP addresses or use clean residential proxies to bypass static blacklists. It is harder to manipulate the network characteristics that influence MTU values, because MTU is determined by the underlying network infrastructure.&lt;/p&gt;
&lt;h3&gt;Granular Classification&lt;/h3&gt;
&lt;p&gt;Where IP databases provide binary classifications (malicious/benign), MTU fingerprinting offers more detail on the specific technologies and configurations in use. This granularity enables more sophisticated risk assessment and response strategies.&lt;/p&gt;
&lt;h3&gt;Reduced False Positives&lt;/h3&gt;
&lt;p&gt;Static databases often flag legitimate users sharing IP addresses with malicious actors, which is common with residential ISPs and mobile carriers. MTU fingerprinting focuses on network behaviour rather than IP reputation, reducing false positive rates while maintaining security effectiveness.&lt;/p&gt;
&lt;h3&gt;Infrastructure Transparency&lt;/h3&gt;
&lt;p&gt;MTU analysis reveals the network path and technologies involved in a connection, providing transparency that static IP databases cannot match. This visibility enables more informed security decisions and a better understanding of threat actor capabilities.&lt;/p&gt;
&lt;h2&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;MTU fingerprinting turns network metadata into useful context about the infrastructure behind a connection. Unlike static databases that rely on historical reputation, this dynamic analysis technique provides real-time insight into network technologies, user behaviours, and potential security threats.&lt;/p&gt;
&lt;p&gt;By understanding MTU patterns, security teams can identify VPN usage, classify mobile traffic, detect residential proxy abuse, and optimise network performance. Its resistance to circumvention and low false-positive rates make it a useful addition to modern security architectures.&lt;/p&gt;
&lt;p&gt;As network technologies continue to evolve, MTU fingerprinting provides a stable way to understand and classify traffic based on fundamental network characteristics rather than short-lived indicators. That makes it a practical signal for network security and operations.&lt;/p&gt;</content><category term="Bots"></category><category term="Threat Detection"></category><category term="Fingerprinting"></category><category term="Networking"></category><category term="Residential Proxies"></category><category term="TLS Fingerprinting"></category><category term="DDoS"></category></entry><entry><title>RFC 9460</title><link href="https://www.peakhour.io/blog/rfc-9460-dns-evolution/" rel="alternate"></link><published>2023-11-16T00:00:00+11:00</published><updated>2023-11-16T00:00:00+11:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2023-11-16:/blog/rfc-9460-dns-evolution/</id><summary type="html">&lt;p&gt;Introducing SVCB and HTTPS records in DNS and their impact on web connectivity.&lt;/p&gt;</summary><content type="html">&lt;p&gt;RFC 9460 introduces two DNS record types: "SVCB" (Service Binding) and "HTTPS". They let browsers learn more connection details during DNS lookup, before redirects and TLS negotiation add extra steps. The result is cleaner connection setup, with practical improvements in speed, security, and efficiency.&lt;/p&gt;
&lt;h2&gt;Understanding the Current Process&lt;/h2&gt;
&lt;p&gt;Traditionally, when a browser connects to a website, it follows a sequence:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Browser requests site via HTTP.&lt;/li&gt;
&lt;li&gt;Server redirects request to HTTPS.&lt;/li&gt;
&lt;li&gt;Browser receives ALPN (Application-Layer Protocol Negotiation) during the HTTPS handshake.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The model is secure, but it is not optimal. It involves multiple round trips, which affects Time to First Byte (TTFB) and the overall user experience. Load balancing and failover are also less direct than they could be. RFC 9460 changes this by allowing DNS to provide the necessary connection details earlier. That reduces the steps involved in establishing a secure connection, lowering TTFB.&lt;/p&gt;
&lt;h2&gt;The Impact of SVCB and HTTPS Records&lt;/h2&gt;
&lt;p&gt;SVCB and HTTPS records move useful connection hints into DNS. They speed up the time-to-first-packet by incorporating the Alt-Svc HTTP header and ALPN TLS extension into DNS, which shortens connection setup. These records also enable redirection at the zone apex, a task not possible with CNAMEs. They simplify DNS load distribution and failover, making web services more resilient. They also remove the need for HSTS preloading and support Encrypted Client Hello (ECH), formerly ESNI, for better privacy.&lt;/p&gt;
&lt;h2&gt;Adoption and Industry Response&lt;/h2&gt;
&lt;p&gt;Adoption started before the RFC was finalised. Firefox has been conducting HTTPS lookups since May 2020, limited to DNS over HTTPS (DoH). Apple's iOS, Safari, and macOS have followed suit since September 2020. Chrome introduced partial support in December 2020 and has recently enabled ECH by default. Various DNS service providers have also started supporting HTTPS and SVCB records.&lt;/p&gt;
&lt;p&gt;As reported on &lt;a href="https://netmeister.org/blog/https-rrs.html"&gt;Netmeister&lt;/a&gt;, adoption is still early but not insignificant. As of October 2023, about 10 million domains have implemented an HTTPS record for their 'www' service names, roughly 4.4% of domains. Around 9.1 million domains, or about 4.0%, use the record on their bare second-level domain name. Among the top 1 million domains, approximately 22.5K (25.5%) use HTTPS records for 'www' service names, and nearly 24K (25.6%) use them on bare domains.&lt;/p&gt;
&lt;p&gt;&lt;img alt="October 2023 Usage" src="/static/images/blog/https-records-oct-2023.png"&gt;&lt;/p&gt;
&lt;h2&gt;What the Records Look Like&lt;/h2&gt;
&lt;p&gt;A typical &lt;a href="/learning/service-binding-record/"&gt;SVCB record&lt;/a&gt; might look like this:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;_example.com. 7200 IN SVCB 1 svc4.example.net. (alpn=&amp;quot;h2,h3&amp;quot; port=&amp;quot;8004&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;This record indicates that the service at &lt;code&gt;_example.com&lt;/code&gt; can be accessed at &lt;code&gt;svc4.example.net&lt;/code&gt; using either HTTP/2 or HTTP/3 on port 8004.&lt;/p&gt;
&lt;p&gt;An &lt;a href="/learning/https-record/"&gt;HTTPS record&lt;/a&gt; could be:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;example.com. 3600 IN HTTPS 0 svc.example.net.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;This record suggests that &lt;code&gt;example.com&lt;/code&gt; should be accessed securely through &lt;code&gt;svc.example.net&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;Apex Domains and the Importance of SVCB/HTTPS Records&lt;/h2&gt;
&lt;p&gt;One long-running DNS limitation is the inability to use CNAME records at the apex (root level) of a domain due to conflicts with other necessary records like NS and SOA. RFC 9460's SVCB/HTTPS records address this by enabling apex domain aliasing without those conflicts. This matters for efficient content delivery networks (CDNs) and load balancing strategies.&lt;/p&gt;
&lt;h2&gt;These records enhance capability&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;1. Load Balancing:&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Consider a website that needs to distribute traffic across multiple servers. SVCB records can indicate different server endpoints with varying priorities.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;example.com. IN SVCB 10 server1.example.com. (alpn=&amp;quot;h2,h3&amp;quot;)
example.com. IN SVCB 20 server2.example.com. (alpn=&amp;quot;h2&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;In this example, &lt;code&gt;server1.example.com&lt;/code&gt; is the preferred endpoint (lower priority number), offering both HTTP/2 and HTTP/3 protocols. If it's unavailable, traffic automatically shifts to &lt;code&gt;server2.example.com&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. Failover Mechanism:&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;For a service that requires high availability, SVCB records can express failover directly:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;critical-service.example.com. IN SVCB 1 primary-service.example.com. (alpn=&amp;quot;h2,h3&amp;quot;)
critical-service.example.com. IN SVCB 2 backup-service.example.com. (alpn=&amp;quot;h2&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Here, &lt;code&gt;primary-service.example.com&lt;/code&gt; is the primary endpoint. If it fails, the system automatically falls back to &lt;code&gt;backup-service.example.com&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;3. Apex Domain Usage:&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;A practical advantage of SVCB/HTTPS records is their ability to handle apex domains, where CNAME records are not feasible. This is important for root domain aliasing to different service providers.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;example.com. IN HTTPS 0 cdn-provider.example.net.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;This record indicates that the apex domain &lt;code&gt;example.com&lt;/code&gt; is to be served through &lt;code&gt;cdn-provider.example.net&lt;/code&gt;, overcoming traditional DNS limitations.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;4. Encrypted ClientHello Support:&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Future enhancements of SVCB could include keys for Encrypted ClientHello, which improves privacy and security during the initial TLS handshake.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;secure.example.com. IN SVCB 1 tls-service.example.net. (ech=&amp;quot;base64-encoded-key&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;This record can be used to initiate a TLS connection with &lt;code&gt;tls-service.example.net&lt;/code&gt; using the provided Encrypted ClientHello key.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;5. Directing Traffic to Specific Protocols:&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;For services that need to direct clients to newer or more efficient protocols, SVCB records can specify the exact protocols to use.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;api.example.com. IN SVCB 1 api-server.example.com. (alpn=&amp;quot;h3&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Clients that understand HTTP/3 can connect directly using this protocol, bypassing the usual HTTP/1.1 or HTTP/2 protocols.&lt;/p&gt;
&lt;h2&gt;The Long Wait&lt;/h2&gt;
&lt;p&gt;HTTPS has been around for a while, so RFC 9460 raises an obvious question: why did this take so long? Apex records have had their share of problems, including not being able to use CNAMEs and having to resort to custom records like ALIAS or Cloudflare's 'cname flattening'.&lt;/p&gt;
&lt;p&gt;It is a fair question. We've had some bizarre records hanging around for ages along with a wide range of solutions to the "CNAME at the zone apex"
problem.&lt;/p&gt;
&lt;p&gt;Credit to the creators of RFC 9460 for getting this through and obtaining browser support:
   - B. Schwartz from Meta Platforms, Inc.
   - M. Bishop from Akamai Technologies
   - E. Nygren from Akamai Technologies&lt;/p&gt;
&lt;h2&gt;Final Thoughts&lt;/h2&gt;
&lt;p&gt;RFC 9460 gives DNS a more useful role in HTTPS connection setup. SVCB and HTTPS records let operators publish endpoint, protocol, failover, and privacy information before the browser starts negotiating the connection. That gives service providers more precise control over how clients reach web services, with practical benefits for performance, reliability, and security.&lt;/p&gt;</content><category term="Interest"></category><category term="HTTP"></category><category term="Web Performance"></category><category term="Rate Limiting"></category><category term="TLS Fingerprinting"></category><category term="CDN"></category><category term="DDoS"></category></entry><entry><title>JA4 and JA4+ Network Fingerprinting</title><link href="https://www.peakhour.io/blog/overview-of-ja4-network-fingerprinting/" rel="alternate"></link><published>2023-10-25T13:00:00+11:00</published><updated>2023-10-25T13:00:00+11:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2023-10-25:/blog/overview-of-ja4-network-fingerprinting/</id><summary type="html">&lt;p&gt;How JA4 constructs a TLS client fingerprint, what JA4+ names, and which details sorting and hashing discard.&lt;/p&gt;</summary><content type="html">&lt;p&gt;JA4+ is the name FoxIO uses for a family of network fingerprinting methods. JA4 itself is the TLS ClientHello method. It
builds on lessons from JA3, but the wider family also contains separate methods for servers, HTTP, certificates, TCP,
SSH and other observations.&lt;/p&gt;
&lt;h2&gt;JA4 and JA4+&lt;/h2&gt;
&lt;p&gt;JA4 produces an &lt;code&gt;a_b_c&lt;/code&gt; value. Its readable &lt;code&gt;a&lt;/code&gt; section records selected connection properties and counts. The &lt;code&gt;b&lt;/code&gt; and
&lt;code&gt;c&lt;/code&gt; sections are truncated SHA-256 values derived from normalised ClientHello fields. Analysts can compare selected
components, such as &lt;code&gt;JA4_ac&lt;/code&gt;, when the complete fingerprint is too narrow for the question being asked. Other JA4+
methods have their own inputs and specifications; they should not be treated as extra fields inside core JA4.&lt;/p&gt;
&lt;p&gt;JA4+ consists of various components:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;JA4&lt;/strong&gt;: TLS Client&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;JA4S&lt;/strong&gt;: TLS Server Response&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;JA4H&lt;/strong&gt;: HTTP Client&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;JA4L&lt;/strong&gt;: Light Distance/Location&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;JA4X&lt;/strong&gt;: X509 TLS Certificate&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;JA4SSH&lt;/strong&gt;: SSH Traffic&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For a more thorough breakdown, the &lt;a href="https://blog.foxio.io/ja4-network-fingerprinting-9376fe9ca637"&gt;JA4 blog&lt;/a&gt; provides
the announcement and describes the fingerprints.&lt;/p&gt;
&lt;p&gt;JA4+ brings useful improvements, but a few aspects and quirks deserve closer attention.&lt;/p&gt;
&lt;h2&gt;What sorting changes&lt;/h2&gt;
&lt;p&gt;JA4 sorts cipher identifiers and most extension identifiers before hashing them. This was especially useful after
Chrome began permuting TLS extension order. Sorting puts those permutations back into one cohort. It also discards the
order as evidence. That is the trade-off: a more stable identifier retains less information about how the ClientHello
was serialised.&lt;/p&gt;
&lt;p&gt;Where investigation matters, retain the raw JA4 form as well as the compact value. &lt;code&gt;JA4_r&lt;/code&gt; exposes the normalised
cipher, extension and signature-algorithm lists, which makes a difference easier to inspect.&lt;/p&gt;
&lt;p&gt;The &lt;a href="https://www.peakhour.io/blog/tls-fingerprinting/"&gt;overview of TLS fingerprinting&lt;/a&gt; provides a more in-depth explanation of how a TLS signature is formed.&lt;/p&gt;
&lt;p&gt;Chrome's change was intended to stop servers and middleboxes from depending on one fixed extension order. In our
&lt;a href="/blog/tls-extension-randomisation/"&gt;extension-randomisation analysis&lt;/a&gt;, the number of order-sensitive TLS fingerprints
rose sharply after the rollout. Sorting reduced that artificial fragmentation. It did not make the resulting value a
client identity, and it did not preserve every distinction in the original handshake.&lt;/p&gt;
&lt;h2&gt;JA3 and Mercury took different paths&lt;/h2&gt;
&lt;p&gt;Before digging further into JA4+'s features and limitations, it helps to separate two related lineages. The
&lt;a href="https://github.com/salesforce/ja3"&gt;original JA3&lt;/a&gt; established a portable TLS fingerprint that was easy to share and
match. Cisco Mercury developed a richer protocol representation and a separate destination-context classification
system. Mercury is not a predecessor in the JA3-to-JA4 naming line. Our &lt;a href="/blog/two-lineages-tls-fingerprinting/"&gt;history of the two lineages&lt;/a&gt;
explains where their work overlaps and where it does not.&lt;/p&gt;
&lt;h2&gt;Implementation differences still matter&lt;/h2&gt;
&lt;p&gt;While sharing signatures through SHA is appealing, it has limits, most notably potential compatibility issues. As Fastly
&lt;a href="https://www.fastly.com/blog/the-state-of-tls-fingerprinting-whats-working-what-isnt-and-whats-next"&gt;noted&lt;/a&gt;, differences
in the implementation can be hidden behind the SHA hash, causing issues when searching for and correlating signatures
between different services. Record the implementation and version that generated a value; a shared format name does not
prove that two sensors handled every field identically.&lt;/p&gt;
&lt;h2&gt;Check the method, implementation and licence&lt;/h2&gt;
&lt;p&gt;The &lt;a href="https://github.com/FoxIO-LLC/ja4"&gt;official JA4+ repository&lt;/a&gt; contains the current specifications and implementations.
Check the licence for the individual method before adopting it: core JA4 is BSD-3-Clause, while most other JA4+ methods
use the FoxIO Licence and place additional conditions on commercial use.&lt;/p&gt;
&lt;p&gt;For a field-level example rather than a format summary, our &lt;a href="/blog/one-clienthello-ja3-ja4-mercury-lab/"&gt;same-ClientHello lab&lt;/a&gt;
records JA3, JA4, &lt;code&gt;JA4_r&lt;/code&gt; and Mercury NPF output from one packet and pins the implementations that generated them.&lt;/p&gt;</content><category term="Security"></category><category term="TLS Fingerprinting"></category><category term="Fingerprinting"></category><category term="Browser Fingerprinting"></category><category term="TLS"></category><category term="SOC 2"></category><category term="Threat Detection"></category></entry><entry><title>Efficiently Generating and Printing All IPv4 Addresses in a Random Order</title><link href="https://www.peakhour.io/blog/linear-congruential-generator/" rel="alternate"></link><published>2023-05-15T13:00:00+10:00</published><updated>2023-05-15T13:00:00+10:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2023-05-15:/blog/linear-congruential-generator/</id><summary type="html">&lt;p&gt;Explains how to efficiently generate all IPv4 addresses in a random order using a Linear Congruential Generator (LCG), a memory-efficient method for tasks like security testing and network simulation.&lt;/p&gt;</summary><content type="html">&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;In&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;this&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;article&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;we&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;explored&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;an&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;efficient&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;way&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;generate&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;and&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;print&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;possible&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;IPv4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;addresses&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;random&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;using&lt;/span&gt;
&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Linear&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Congruential&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Generator&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;LCG&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;The&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;LCG&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;pseudorandom&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;number&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;generator&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;helps&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;generate&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;the&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;full&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;of&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;IP&lt;/span&gt;
&lt;span class="n"&gt;addresses&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;without&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;consuming&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;vast&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;amounts&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;of&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;memory&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;making&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;this&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;approach&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;suitable&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;for&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;systems&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;with&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;memory&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;constraints&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;
&lt;span class="n"&gt;We&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;also&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;provided&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Python&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;script&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;demonstrating&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;the&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;concept&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;along&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;with&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;test&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;case&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;verify&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;its&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;correctness&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;

&lt;span class="n"&gt;We&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;then&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;delved&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;into&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;the&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;importance&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;of&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;randomising&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;IP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;addresses&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;highlighting&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;its&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;critical&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;role&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;areas&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;like&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;security&lt;/span&gt;
&lt;span class="n"&gt;testing&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;load&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;balancing&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;enhancing&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;privacy&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;and&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;web&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;scraping&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;However&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;while&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;using&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;this&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;technique&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;it&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;s essential to&lt;/span&gt;
&lt;span class="n"&gt;respect&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;privacy&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;and&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;legality&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;as&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;misuse&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;can&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;lead&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;legal&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;repercussions&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;

&lt;span class="n"&gt;In&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;summary&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;the&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;ability&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;generate&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;and&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;print&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;IPv4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;addresses&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;random&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;is&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;powerful&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;tool&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;especially&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;the&lt;/span&gt;
&lt;span class="n"&gt;realm&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;of&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;networking&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;and&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;cybersecurity&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;and&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;can&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;be&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;achieved&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;efficiently&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;using&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;the&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;LCG&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;approach&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;In networking, some tasks require generating and printing every possible IPv4 address. Doing that in random order
without a large memory footprint is less straightforward. The IPv4 address space contains 2^32, or 4,294,967,296,
values. Storing all of them in memory at once is not feasible for most systems.&lt;/p&gt;
&lt;p&gt;This article uses a Linear Congruential Generator (LCG) to generate the full range without holding it in memory.&lt;/p&gt;
&lt;h2&gt;Linear Congruential Generator&lt;/h2&gt;
&lt;p&gt;A Linear Congruential Generator is a type of pseudorandom number generator that can run without storing the whole
sequence. It generates each next value from a linear equation based on the previous value. The basic form of the LCG is:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;X_(n+1) = (a*X_n + c) mod m
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Here, &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;c&lt;/code&gt;, and &lt;code&gt;m&lt;/code&gt; are constants, and &lt;code&gt;X_n&lt;/code&gt; is the nth number in the sequence. The initial seed or starting point
of the sequence is X_0.&lt;/p&gt;
&lt;p&gt;If we choose parameters such that the period of the LCG is maximum (equal to the modulus), and the modulus equals the
range of numbers we're generating (the number of possible IPv4 addresses in this case), then the LCG should generate
each number in the range exactly once before repeating.&lt;/p&gt;
&lt;p&gt;Here is that idea in Python:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;ipaddress&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;lcg&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;modulus&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;seed&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="sd"&gt;&amp;quot;&amp;quot;&amp;quot;Linear congruential generator.&amp;quot;&amp;quot;&amp;quot;&lt;/span&gt;
    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;seed&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;seed&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;%&lt;/span&gt; &lt;span class="n"&gt;modulus&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;seed&lt;/span&gt;

&lt;span class="n"&gt;start_ip_str&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;0.0.0.0&amp;#39;&lt;/span&gt;
&lt;span class="n"&gt;end_ip_str&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;255.255.255.255&amp;#39;&lt;/span&gt;

&lt;span class="n"&gt;start_ip&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ipaddress&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;IPv4Address&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;start_ip_str&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="n"&gt;end_ip&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ipaddress&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;IPv4Address&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;end_ip_str&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

&lt;span class="n"&gt;modulus&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;end_ip&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;start_ip&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
&lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1664525&lt;/span&gt;
&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1013904223&lt;/span&gt;
&lt;span class="n"&gt;seed&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;  &lt;span class="c1"&gt;# Arbitrary seed&lt;/span&gt;

&lt;span class="n"&gt;generator&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;lcg&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;modulus&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;seed&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;modulus&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;ip_int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;start_ip&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nb"&gt;next&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;generator&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;ip&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ipaddress&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;IPv4Address&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ip_int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nb"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The script first defines the parameters of the LCG. &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;c&lt;/code&gt;, and &lt;code&gt;seed&lt;/code&gt; are set to values used in Numerical Recipes'
LCG, a well-known and widely used LCG. The modulus is set to the total number of possible IPv4 addresses.&lt;/p&gt;
&lt;p&gt;The function &lt;code&gt;lcg()&lt;/code&gt; is implemented as a Python generator, yielding the next number in the sequence each time it is
called.&lt;/p&gt;
&lt;p&gt;The loop then generates and prints each IP address. It adds the output of the LCG to the start IP address, converts it
back to an IP address string, and prints it.&lt;/p&gt;
&lt;p&gt;This script generates and prints each IP address in random (more precisely, pseudorandom) order using very little
memory. Each IP address is printed exactly once, assuming the period of the LCG is maximum.&lt;/p&gt;
&lt;p&gt;The point is that a small pseudorandom number generator can walk a large range without materialising the whole list. The
code can still be tweaked and optimised for specific requirements and constraints.&lt;/p&gt;
&lt;h2&gt;The Importance of Randomising IP Addresses&lt;/h2&gt;
&lt;p&gt;Randomising IP addresses has practical uses in several networking workflows:&lt;/p&gt;
&lt;h3&gt;1. Security Testing and Penetration Testing&lt;/h3&gt;
&lt;p&gt;In cybersecurity, randomising IP addresses can help simulate attacks on a network from various sources. By using a range
of IP addresses in no particular order, penetration testers can mimic the unpredictable nature of real-world cyber
threats and build more robust test scenarios.&lt;/p&gt;
&lt;h3&gt;2. Load Balancing and Network Traffic Simulation&lt;/h3&gt;
&lt;p&gt;Randomising IP addresses is also useful in network traffic simulations. Network engineers and administrators can use
this approach to test network resilience and capacity. By sending requests to servers from randomised IP addresses, they
can evaluate how well their load balancing strategies are functioning and whether the network can handle high traffic
loads from various sources.&lt;/p&gt;
&lt;h3&gt;3. Anonymity and Privacy&lt;/h3&gt;
&lt;p&gt;In some cases, randomising IP addresses can help with privacy and anonymity. While it is not a foolproof method, using a
different IP address for each request can make it more challenging for online trackers to monitor user activity. It is a
common practice among privacy-focused internet users and is also used in some VPN (Virtual Private Network) services.&lt;/p&gt;
&lt;h3&gt;4. Web Scraping&lt;/h3&gt;
&lt;p&gt;Web scraping is another area where randomising IP addresses is useful. To prevent being blocked by anti-bot measures,
web scrapers often need to rotate their IP addresses. By using a different IP address for each request, they can avoid
being detected and blocked by the sites they are scraping.&lt;/p&gt;
&lt;p&gt;Randomising IP addresses can be useful in these cases, but privacy and legality still matter. Unauthorised network
scanning, privacy breaches, and cyberattacks are illegal and punishable under law.&lt;/p&gt;
&lt;p&gt;Generating and printing every possible IPv4 address in a random order is a valuable technique with various applications,
from network testing to privacy enhancement. With the Linear Congruential Generator approach, we can do it efficiently.&lt;/p&gt;</content><category term="Technical"></category><category term="Residential Proxies"></category><category term="Networking"></category><category term="DDoS"></category><category term="TLS Fingerprinting"></category></entry><entry><title>Chrome's TLS Extension Randomisation Experiment</title><link href="https://www.peakhour.io/blog/tls-extension-randomisation/" rel="alternate"></link><published>2023-02-02T13:00:00+11:00</published><updated>2023-02-02T13:00:00+11:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2023-02-02:/blog/tls-extension-randomisation/</id><summary type="html">&lt;p&gt;Does TLS extension randomisation assist in hiding Chrome?&lt;/p&gt;</summary><content type="html">&lt;p&gt;&lt;a href="/blog/tls-fingerprinting/"&gt;Transport Layer Security (TLS) fingerprinting&lt;/a&gt; is a commonly used
technique for identifying client processes. To reduce the
risk of server and middlebox fingerprinting of Chrome's current
ClientHello and to make the TLS ecosystem more resilient to changes,
Google Chrome ran an experiment to randomise a portion of
the TLS fingerprint. This experiment was included in Chrome version 108,
which was released on December 8, 2022. You can read the status of the
current experiment on the &lt;a href="https://chromestatus.com/feature/5124606246518784"&gt;chrome status site&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;The aim of this experiment was to make it more difficult for server
implementers to fingerprint Chrome and assume specific implementation
behaviour from a fixed extension order. By randomly ordering
extensions (subject to the pre_shared_key constraint in the RFC),
Chrome hoped to reduce the risk of server and middlebox fixating on
details of its current ClientHello.&lt;/p&gt;
&lt;p&gt;&lt;img alt="unique-tls-fingerprints-over-time" src="/static/images/blog/tls-unsorted-extensions.png"&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;The above graph correlates to the Chrome experiment and subsequent
release of the feature. The number of unique TLS signatures dramatically
increased.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;From Peakhour data, we can see a large number of unique
fingerprints appearing since the date of the experiment, making it
very difficult to identify the Chrome network stack by a TLS
fingerprint alone. However, &lt;a href="https://hnull.org/2022/12/01/sorting-out-randomized-tls-fingerprints/"&gt;an analysis&lt;/a&gt; by
David McGrew,
a Cisco Fellow, cast doubt on the effectiveness of this experiment. In his
article, McGrew proposed a lexicographical sorting of TLS extensions and
found that 98.8% of signatures were unique after sorting. He argues that
the canonical ordering of the TLS extensions in the TLS fingerprint can
achieve nearly the same level of entropy as randomising them and still
be effective at client identification. Furthermore, he claims that the
RFC should be amended to state that extensions SHOULD be sent in an
ordered fashion in the ClientHello packet. McGrew also highlights the
potential dangers of allowing unordered extension lists, as it could
create a \"subliminal channel\" that could be used for tracking or
transmitting information. Let's now graph, over the same period, the number
of TLS signatures with TLS extension sorting.&lt;/p&gt;
&lt;p&gt;&lt;img alt="unique-tls-fingerprints-sorted-extensions-over-time" src="/static/images/blog/tls-sorted-extensions.png"&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;The above graph correlates to David's assertion that sorting TLS
extensions has minimal impact on TLS fingerprinting.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;It appears that David's assertion is correct: sorting TLS extensions has
minimal impact on the number of unique TLS fingerprints. Let's now look
at in-the-wild Chrome 109 data:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;strong&gt;Hashed sorted TLS Fingerprint&lt;/strong&gt;&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;Unique unsorted TLS fingerprints&lt;/strong&gt;&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;Browser&lt;/strong&gt;&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;Version&lt;/strong&gt;&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;% of clients&lt;/strong&gt;&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;% of hits&lt;/strong&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;3241796329&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;Chrome Mobile WebView&lt;/td&gt;
&lt;td&gt;109&lt;/td&gt;
&lt;td&gt;6.14&lt;/td&gt;
&lt;td&gt;1.01&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3313291307&lt;/td&gt;
&lt;td&gt;8566&lt;/td&gt;
&lt;td&gt;Chrome&lt;/td&gt;
&lt;td&gt;109&lt;/td&gt;
&lt;td&gt;1.83&lt;/td&gt;
&lt;td&gt;1.54&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1537819294&lt;/td&gt;
&lt;td&gt;26587&lt;/td&gt;
&lt;td&gt;Chrome Mobile&lt;/td&gt;
&lt;td&gt;109&lt;/td&gt;
&lt;td&gt;6.05&lt;/td&gt;
&lt;td&gt;4.64&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3944870384&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;Chrome Mobile iOS&lt;/td&gt;
&lt;td&gt;109&lt;/td&gt;
&lt;td&gt;4.31&lt;/td&gt;
&lt;td&gt;5.35&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3241796329&lt;/td&gt;
&lt;td&gt;51594&lt;/td&gt;
&lt;td&gt;Chrome Mobile&lt;/td&gt;
&lt;td&gt;109&lt;/td&gt;
&lt;td&gt;16.9&lt;/td&gt;
&lt;td&gt;14.43&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1537819294&lt;/td&gt;
&lt;td&gt;121346&lt;/td&gt;
&lt;td&gt;Chrome&lt;/td&gt;
&lt;td&gt;109&lt;/td&gt;
&lt;td&gt;15.66&lt;/td&gt;
&lt;td&gt;20.7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3241796329&lt;/td&gt;
&lt;td&gt;156405&lt;/td&gt;
&lt;td&gt;Chrome&lt;/td&gt;
&lt;td&gt;109&lt;/td&gt;
&lt;td&gt;35.52&lt;/td&gt;
&lt;td&gt;37.79&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;em&gt;It's interesting that the experiment does not run on WebView.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;Final Thoughts&lt;/h2&gt;
&lt;p&gt;While Chrome's experiment may have reduced the risk of
server and middlebox fingerprinting of Chrome's current ClientHello, it
seems that randomising TLS extensions alone is not enough to
prevent TLS fingerprinting, and may be a useful indicator that
it is The Real Chrome.&lt;/p&gt;
&lt;p&gt;This experiment became one of the reasons newer formats normalise extension order. Our &lt;a href="/blog/one-clienthello-ja3-ja4-mercury-lab/"&gt;JA3, JA4 and Mercury lab&lt;/a&gt; shows exactly where each format keeps, sorts or discards ClientHello detail. The accompanying &lt;a href="/blog/two-lineages-tls-fingerprinting/"&gt;history of the two TLS fingerprinting lineages&lt;/a&gt; explains why Cisco Mercury and JA4 made related but different design choices.&lt;/p&gt;
&lt;p&gt;The remaining research question is whether discarded order ever helps distinguish an imitator or evasive client. &lt;a href="/blog/tls-fingerprint-canonicalisation-attacker-variation/"&gt;Does TLS fingerprint canonicalisation hide useful attacker variation?&lt;/a&gt; defines the labelled corpus and holdout study needed to answer it without confusing uniqueness with detection accuracy.&lt;/p&gt;</content><category term="Security"></category><category term="TLS Fingerprinting"></category><category term="TLS"></category><category term="Browser Fingerprinting"></category><category term="Fingerprinting"></category><category term="API Security"></category></entry><entry><title>TLS Fingerprinting</title><link href="https://www.peakhour.io/blog/tls-fingerprinting/" rel="alternate"></link><published>2023-02-02T13:00:00+11:00</published><updated>2023-02-02T13:00:00+11:00</updated><author><name>AC</name></author><id>tag:www.peakhour.io,2023-02-02:/blog/tls-fingerprinting/</id><summary type="html">&lt;p&gt;What is fingerprinting, and in particular TLS fingerprinting?&lt;/p&gt;</summary><content type="html">&lt;h2&gt;What is Fingerprinting?&lt;/h2&gt;
&lt;p&gt;Fingerprinting is a technique that may be used to identify the specific device, web browser,
and operating system making a request, regardless of what the client says in its user-agent header.
By helping organisations identify and characterise the attributes of a client's connection,
fingerprinting can improve network security and help protect against malicious traffic.&lt;/p&gt;
&lt;p&gt;Fingerprinting can also refer to techniques for following or uniquely identifying individual users across the web.
That is a separate set of techniques and is not discussed in this article.&lt;/p&gt;
&lt;p&gt;&lt;a href="/learning/fingerprinting/what-is-tls-fingerprinting/"&gt;Transport Layer Security (TLS) Fingerprinting&lt;/a&gt; determines the specific characteristics of a client's TLS
implementation by examining the initial TLS handshake packet, known as the "Client Hello." This packet
contains fields and parameters such as supported cipher suites, extensions, and the client's preferred order of
those parameters, which can be used to create a unique "fingerprint" of the client's TLS implementation.&lt;/p&gt;
&lt;h2&gt;Why is it used?&lt;/h2&gt;
&lt;p&gt;Fingerprinting has several uses, including &lt;a href="/products/bot-management/"&gt;bot protection&lt;/a&gt;, DDoS protection, and client
identification. By identifying and characterising the attributes of a client's connection,
fingerprinting can improve network security and help protect against malicious traffic.&lt;/p&gt;
&lt;h2&gt;How does TLS Fingerprinting work?&lt;/h2&gt;
&lt;p&gt;TLS Fingerprinting examines the initial TLS handshake packet, known as the "Client Hello".
The Client Hello packet is sent by the client during the initial phase of the TLS handshake, which establishes a secure
connection between the client and the server. It contains information about the client's preferred encryption methods,
extensions, and parameters, including:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Protocol Version: The version of the TLS protocol desired by the client.&lt;/li&gt;
&lt;li&gt;Random: A 32-byte random value generated by the client, used in key generation and derivation.&lt;/li&gt;
&lt;li&gt;Session ID: An optional session identifier for resuming a previous session.&lt;/li&gt;
&lt;li&gt;Cipher Suites: A list of supported encryption algorithms, ordered by preference.&lt;/li&gt;
&lt;li&gt;Compression Methods: A list of supported compression algorithms, ordered by preference.&lt;/li&gt;
&lt;li&gt;Extensions: Optional extensions that can negotiate additional parameters, such as Server Name Indication (SNI) and
   Elliptic Curve Supported (ECS).&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The Client Hello packet is central to the operation and security of the TLS connection because it provides information
the server uses to select encryption algorithms and parameters. The packet also enables the client and server to
negotiate an appropriate encryption method for their communication. The Client Hello's variable
content, based on the TLS version, library, cipher suites, extensions, and settings supported by the client, makes it
a strong candidate for fingerprinting.&lt;/p&gt;
&lt;p&gt;Common components used to create a TLS fingerprint include:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Cipher Suites: The order of cipher suites supported by the client.&lt;/li&gt;
&lt;li&gt;Extensions: Supported extensions included in the Client Hello packet, such as SNI and ECS.&lt;/li&gt;
&lt;li&gt;TLS Point Formats: Encoding of cryptographic parameters in a format that can be transmitted as part of the TLS
   protocol, used in elliptic curve cryptography (ECC).&lt;/li&gt;
&lt;li&gt;TLS Curves: The specific elliptic curves used in ECC, a type of public-key cryptography used in the TLS protocol.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;TLS fingerprinting has been a topic of research for several years, with a number of tools and techniques developed
from that work. Notable examples include &lt;a href="/learning/fingerprinting/what-is-ja3-fingerprinting/"&gt;JA3&lt;/a&gt;, developed by John Althouse, Jeff Atkinson, and Josh Atkins of Salesforce,
which uses a hash of the client's SSL/TLS parameters as a unique identifier for tracking and analysing
SSL/TLS traffic. Another tool, Mercury by David McGrew and Blake Anderson, can be used to fingerprint client connections
and identify the device, operating system, and application making the connection.&lt;/p&gt;
&lt;p&gt;TLS fingerprinting has a variety of uses, including bot protection, DDoS protection, malware identification and
client identification. By enabling organisations to identify and characterise the attributes of a client's TLS
implementation, TLS fingerprinting can improve network security and help protect against malicious traffic.&lt;/p&gt;
&lt;p&gt;In production, TLS fingerprints are most useful when combined with &lt;a href="/products/ip-intelligence/"&gt;IP intelligence&lt;/a&gt; and &lt;a href="/products/residential-proxy-detection/"&gt;residential proxy detection&lt;/a&gt;, rather than treated as a standalone verdict.&lt;/p&gt;
&lt;h2&gt;Representation of a TLS Fingerprint&lt;/h2&gt;
&lt;p&gt;A TLS fingerprint is commonly represented as a string or hash that summarises the important components of the Client
Hello packet. The most common components used to create a TLS fingerprint include the supported cipher suites,
extensions, and TLS point formats. The cipher suites are represented as a list of hexadecimal values in the order
they are presented by the client, while extensions and point formats are represented as a list of hexadecimal values
or a unique identifier.&lt;/p&gt;
&lt;p&gt;Raw JA3 signatures are represented by the following fields, which are then hashed with MD5:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;SSLVersion, Cipher, SSLExtension, EllipticCurve, EllipticCurvePointFormat
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;An example raw signature is:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt; 771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-34-51-43-13-45-28-21,29-23-24-25-256-257,0
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;An MD5 hash is then applied, resulting in the final signature.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="mf"&gt;579&lt;/span&gt;&lt;span class="n"&gt;ccef312d18482fc42e2b822ca2430&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Mercury signatures are represented by:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&amp;quot;tls/1&amp;quot; (TLS_Version) (TLS_Ciphersuite) [ Extension* ]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;An example signature is:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;tls/1/
(0303)
(130213031301c02cc030009fcca9cca8ccaac02bc02f009ec024c028006bc023c0270067c00ac0140039c009c0130033009d009c003d003c0035002f00ff)
[
   (0000)
   (000a000c000a001d0017001e00190018)
   (000b000403000102)
   (000d0030002e040305030603080708080809080a080b080408050806040105010601030302030301020103020202040205020602)
   (0016)
   (0017)
   (0023)
   (002b0009080304030303020301)
   (002d00020101)
   (0033)
]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h2&gt;Hash Functions for Representing TLS Fingerprints&lt;/h2&gt;
&lt;p&gt;Hashing algorithms, such as MD5, are commonly used to create a unique representation of a TLS fingerprint.
These hash functions take the client's TLS parameters as input and produce a fixed-length output, which serves as
a unique identifier for the client. The hash value can be compared against a database of known TLS fingerprints to
help determine the identity of the client.&lt;/p&gt;
&lt;p&gt;Other techniques for representing TLS fingerprints include base64 encoding of the client's TLS parameters, such as in the
Mercury fingerprint.&lt;/p&gt;
&lt;h2&gt;Challenges with TLS fingerprinting&lt;/h2&gt;
&lt;p&gt;TLS fingerprinting is not a foolproof method for identifying clients and their attributes. It has several limitations
that need to be considered.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;False Positives: TLS fingerprinting relies on the assumption that the client's Client Hello packet uniquely
   identifies a connecting process by its TLS implementation. However, it is possible for a client to alter the Client
   Hello packet by customising TLS parameters, which affects the Client Hello packet and can result in a false
   positive
   identification. This makes it important to use multiple methods for identifying clients. For example, Mercury takes
   into account destination ports to add additional context.&lt;/li&gt;
&lt;li&gt;False Negatives: While TLS fingerprinting can identify many different clients and their attributes, it is not capable
   of identifying all clients. Some clients may have a unique or unusual TLS implementation that cannot be accurately
   fingerprinted. Additionally, some clients may actively attempt to evade fingerprinting by customising
   TLS parameters or using tools to anonymise their connections.&lt;/li&gt;
&lt;li&gt;Forging of TLS Fingerprints: It is possible for attackers to deliberately forge or modify the information contained
   in their Client Hello packet to appear as a different client. This makes it difficult for fingerprinting tools to
   accurately identify the true identity of a client and can be used for malicious purposes, such as evading security
   measures or disguising the origin of an attack.&lt;/li&gt;
&lt;li&gt;Incomplete Data: TLS fingerprinting is limited by the information contained in the Client Hello packet, which may not
   contain all of the necessary data to accurately identify a client. For example, a client may not send a full list of
   supported cipher suites or extensions, may use a modified version of the TLS protocol that is not recognised by
   the fingerprinting tool, or the fingerprint may not be present in available databases.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Different fingerprinting implementations can result in different hashes for the same TLS connection, even though the
underlying SSL/TLS protocol remains unchanged. This happens due to the various algorithms, parameters, and
representations used by different fingerprinting tools.&lt;/p&gt;
&lt;p&gt;For instance, implementation differences when generating the TLS fingerprint may cause hashes found in public databases
to be inconsistent with a locally generated hash.&lt;/p&gt;
&lt;h2&gt;Final Thoughts&lt;/h2&gt;
&lt;p&gt;Be aware of the limitations and differences between fingerprinting implementations, and choose the right tool and
representation for your specific use case. Standardising the representation of fingerprints and using common hash
algorithms can help avoid confusion and improve interoperability between databases.&lt;/p&gt;</content><category term="Security"></category><category term="TLS Fingerprinting"></category><category term="Browser Fingerprinting"></category><category term="Fingerprinting"></category><category term="TLS"></category><category term="HTTP"></category><category term="DDoS"></category></entry></feed>