Peakhour.IO - DDoS

Beyond the IP Address

2025-09-01T00:00:00+10:00

For years, rate limiting has been a standard control for protecting websites and APIs from abuse. The basic model is simple: limit the number of requests a single "user" can make in a given period. If a user exceeds the limit (e.g., 10 login attempts in a minute), they are temporarily blocked.

The hard part has always been identifying that "user". Traditionally, the answer was the IP address. The assumption was that one IP address equaled one user. In the early days of the internet, this was a reasonable approximation. Today, that assumption no longer holds, and it leaves systems exposed to modern attacks.

The IP address is no longer a reliable identifier for a single user or device. There are three common reasons:

Proxy Networks: Attackers don't use a single IP address. They use large residential proxy networks to rotate requests through thousands or even millions of different IP addresses, making each request look like it comes from a new user.
Shared IPs (CGNAT): At the same time, a single IP address can represent thousands of legitimate users. Mobile carriers use Carrier-Grade NAT (CGNAT) to make many mobile devices share the same public IP. Similarly, an entire office building or university campus might appear to the internet as a single IP.
Distributed Attacks: Modern automated attacks, like Layer 7 DDoS or credential stuffing, are inherently distributed. Attackers use botnets or proxy networks to spread their attack across a large number of IPs, so no single IP ever exceeds a traditional rate limit.

Blocking a shared IP because of one bad actor can cause collateral damage, denying access to thousands of legitimate users. On the other side, failing to see that thousands of IPs are part of a single coordinated attack means the attack succeeds. Traditional IP-based rate limiting is no longer enough.

The New Way: Advanced Rate Limiting

Advanced Rate Limiting addresses this by moving beyond the IP address. Instead of grouping requests by a single, unreliable identifier, it lets you count requests using more stable and meaningful characteristics of the connection or the software making it.

This approach groups requests using identifiers like:

TLS/HTTP2 Fingerprints: Every client application (like a browser or a script) has a unique "fingerprint" based on how it initiates a secure connection (TLS) or communicates over HTTP/2. This fingerprint remains consistent even as an attacker rotates through thousands of IP addresses. By rate limiting based on the TLS fingerprint, you can track and block the underlying automation tool itself, not just the IPs it uses.
Device Characteristics: A fingerprint can be constructed from a range of attributes, including the device's operating system, browser version, and more. This allows for the detection of repeated requests coming from the same class of device.
A Combination of Headers: For authenticated APIs, you can group requests by an Authorization header or API key, enforcing fair usage and preventing abuse by a single authenticated client.

Practical Use Cases

The value of advanced rate limiting is clearest when it is applied to real-world threats:

Mitigating Distributed Credential Stuffing: An attacker using a tool like OpenBullet launches a credential stuffing attack against your login page, rotating through thousands of residential proxy IPs. Traditional rate limiting is ineffective here. However, the OpenBullet software has a consistent TLS fingerprint. By setting a rule to limit failed login attempts per TLS fingerprint, you can detect and block the entire distributed attack, regardless of how many IPs are involved.
Protecting APIs from Abuse: A partner is abusing their API key, sending far too many requests and degrading service for other users. By rate limiting based on the Authorization header, you can enforce usage limits on a per-client basis, keeping access fair without affecting other users.
Stopping Content Scrapers: A scraper is hammering your e-commerce site to steal pricing data. They are using a botnet to distribute the requests across hundreds of IPs. However, the scraping script has a unique combination of a user-agent and a TLS fingerprint. Advanced rate limiting can count requests based on this combined signature and block the scraper, protecting your intellectual property.

When attackers are distributed, your defences need to see the single actor behind the many IPs. Advanced rate limiting provides that visibility and should be part of a modern application security strategy.

The Bot Spectrum

2025-09-01T00:00:00+10:00

The word "bot" is often used as shorthand for unwanted automation: scripts trying to break into accounts, scrape content, or overwhelm websites. A large share of internet traffic does come from bad bots, but automated traffic is not automatically harmful. Some bots are part of how the web is discovered, monitored, and kept usable.

Effective bot management is not about blocking every automated request. It depends on accurate classification: separating good bots from bad bots, and recognising the "grey" bots that sit between them. That classification lets you apply controls that reduce risk without cutting off traffic that helps your site operate.

Good Bots: The Essential Workers of the Web

Good bots are automated programs that perform useful or necessary tasks. They are usually clear about who they are and respect the rules you set in your robots.txt file. Blocking them can damage search visibility, monitoring, or other business workflows.

Examples of Good Bots:

Search Engine Crawlers: Bots like Googlebot and Bingbot are the best-known good bots. They crawl and index your website's content, which is how your pages appear in search engine results. Blocking them would make your site invisible on Google.
Performance Monitoring Bots: These bots are used by services to check your website's uptime and performance from different locations around the world, and to alert you if your site goes down.
Copyright Bots: These bots scan the web for plagiarised content, helping to protect your intellectual property.

Management Strategy: Good bots should be identified and allowed to access your site freely. Verification techniques, such as reverse DNS lookups, can be used to confirm that a bot claiming to be Googlebot is actually coming from Google.

Bad Bots: The Malicious Actors

Bad bots are designed for malicious activity. They are a major reason bot management exists as a security function. These bots are deceptive, often hiding their identity and purpose, and they can be responsible for a wide range of costly and damaging activity.

Examples of Bad Bots:

Credential Stuffers: These bots use stolen usernames and passwords to carry out account takeover attacks.
Content and Price Scrapers: These bots steal your valuable content, product listings, and pricing data, often for use by competitors.
Spam Bots: These bots flood comment sections, forums, and contact forms with unwanted ads or malicious links.
Denial of Service (DDoS) Bots: These bots are part of a botnet used to overwhelm a website with traffic, causing it to slow down or crash.
Inventory Hoarding Bots: Common in e-commerce, these bots automatically add limited-edition products to shopping carts to prevent legitimate customers from buying them, often for resale at a higher price (scalping).

Management Strategy: Bad bots need to be accurately identified and blocked as quickly as possible, ideally at the network edge before they consume your server resources.

Grey Bots: The Nuanced Category

Grey bots are not inherently malicious, but their behaviour can still cause problems. They often serve a legitimate purpose, but become an issue when they crawl too aggressively, consume excessive bandwidth or server resources, and slow the site down for real users.

Examples of Grey Bots:

Aggressive SEO Tools: Bots from marketing tools like Ahrefs, SEMrush, and Majestic crawl websites to gather data for backlink analysis and competitive research. They can be useful, but their crawling can also be heavy.
Partner and Aggregator Bots: These could be bots from partner companies or price comparison websites that need to access your data. The activity may be legitimate, but it still needs to be managed.
Feed Fetchers: Bots that collect data for news aggregators or other applications fall into this category.

Management Strategy: Grey bots require more than a simple allow or block rule. The best strategy is often to rate-limit or tarpit them.

Rate-Limiting: This allows the bot to continue accessing your site, but slows it to a manageable level so it does not overwhelm your servers.
Tarpitting: This intentionally slows the connection for a specific bot, increasing the cost and time required to crawl your site and discouraging overly aggressive behaviour.

By classifying incoming bot traffic and applying the right control for each category, organisations can block threats, manage resource consumption, and allow the useful automation the modern web depends on.

Protecting Against a Share Point Zero Day Vulnerability with Network Fingerprinting

2025-07-23T13:00:00+10:00

Why Network Fingerprinting is Your Strongest First Defense

A critical new remote code execution (RCE) vulnerability in on-premises Microsoft SharePoint Server, identified as CVE-2025-53770, is being actively exploited and presents a serious risk to organisations. This flaw allows an unauthenticated attacker to take complete control of a server over the network, so immediate and effective defence is a priority. Microsoft disclosed the flaw on 19 July.

Vendor patches are essential, but zero-day activity often starts before most organisations can patch. That gap is where proactive controls matter.

This post looks at the technical nature of this threat and how a strategy centred on network fingerprinting can block zero-day exploit activity before a formal patch is deployed.

Understanding the Threat: CVE-2025-53770

The SharePoint vulnerability is particularly dangerous as it allows for the deserialization of untrusted data, leading to remote code execution without any need for attacker authentication. This makes any unpatched, internet-facing on-premises SharePoint server a potential target. The U.S. Cybersecurity and Infrastructure Security Agency (CISA) has underlined the severity of this threat by adding it to its Known Exploited Vulnerabilities Catalog.

Exploitation can lead to a complete compromise of the SharePoint server, allowing attackers to steal data, execute arbitrary code, and potentially move laterally across the internal network.

The Race Against Scanners

When a zero-day vulnerability like this is discovered, a global, automated race begins. Malicious actors immediately deploy scanners to canvass the internet for vulnerable systems.

Our own analysis shows that the majority of malicious requests targeting our clients came from the DigitalOcean and Scaleway ASNs, with Amazon Web Services (AWS) EC2 and Microsoft Azure also being a prominent source. These networks are well-known for being used by malicious actors to launch scanning and attack campaigns quickly. Notably, scans were happening on 16 and 17 July, before the vulnerability was disclosed by Microsoft.

This initial scanning phase, however, creates an opportunity for defence. Instead of waiting to analyse the specific attack payload, we can identify and block the very tools the attackers are using.

Exploits attempts in the wild. Note attempts days before disclosure.

Why IP Reputation Isn't Enough

For years, a primary method of defence has been IP reputation—blocking traffic from IP addresses known to be malicious. While simple and somewhat effective against basic attacks, this approach is increasingly unreliable in the face of modern threats.

The rise of sophisticated proxy services has changed the model. Attackers now have easy access to vast networks of residential, mobile, and rotating data centre proxies. These services allow them to distribute their attack traffic across thousands or even millions of seemingly legitimate IP addresses, making it impossible to maintain an effective blocklist. An IP that sends a malicious request one moment could be used by a legitimate customer the next.

Furthermore, attackers leveraging cloud infrastructure use ephemeral IPs that exist for only a short time, rendering IP-based blocking a constant and losing game of cat and mouse. This approach also carries a high risk of "collateral damage", where legitimate users are blocked simply because they share an IP address with a bad actor, a common scenario with Carrier-Grade NAT (CGNAT) or public Wi-Fi. Relying solely on where a request comes from is no longer a viable strategy.

Unmasking the Attacker's Tools with Network Fingerprinting

This is where network fingerprinting becomes useful as a zero-day defence. Fingerprinting in cybersecurity refers to methods used to identify the unique characteristics of devices, software, or users. It allows for the identification and categorisation of operating systems and software based on their distinct signatures in network communications.

When attackers rush to exploit a new vulnerability, they don't use standard web browsers. They quickly code scanners using programming languages and libraries like Python, Go, or Java. These tools and libraries create network connections with distinct, non-browser-like fingerprints. By analysing these, we can block the scanner before it ever delivers its malicious payload.

Peakhour uses several passive fingerprinting techniques to do this:

TCP Fingerprinting

This method identifies a device's operating system by analysing how it implements the TCP protocol. By examining nuances in TCP packets—like window size, Time to Live (TTL), and how the device responds to non-standard packets—we can identify the underlying system that created the request.

TLS Fingerprinting

This technique analyses the "ClientHello" message sent by the client during the initial TLS handshake to establish a secure connection. The combination of TLS version, supported cipher suites, and extensions creates a unique fingerprint. This is a highly effective way of identifying the classes of connecting clients, such as those made by Go, Python, or Java libraries, which are commonly used for attack tooling. JA4 and JA3 are popular TLS fingerprint formats.

HTTP/2 Fingerprinting

This involves analysing how clients use the HTTP/2 protocol, including their patterns in sending HTTP/2 frames and negotiating connections. This makes it easier to differentiate between legitimate browsers, bots, and the custom applications used in an attack campaign.

After identifying these fingerprints, Peakhour's bot management service uses machine learning to classify them as either a legitimate browser or a bot. This provides a strong layer of defence against zero-day exploits. The scanners are identified and blocked based on their fundamental network characteristics, irrespective of the specific vulnerability or payload they carry.

Defense in Depth

No single security measure is a silver bullet. While network fingerprinting provides a powerful first line of defence against automated scanners, a multi-layered, defence-in-depth strategy matters.

Any request that manages to bypass the initial fingerprinting checks must face the next layer: our standard Web Application Firewall (WAF) with post-body scanning. A WAF inspects every request before it reaches the application. By enabling the inspection of the full request body, the WAF can identify and block malicious payloads, such as the specific code used in an exploit attempt, that may be hidden within the data sent to the server. Our WAF was updated with a virtual patch on 22 July at 5am AEST to add protection against this vulnerability.

Staying Ahead in a Zero-Day World

The SharePoint CVE-2025-53770 vulnerability shows why a reactive security posture is not enough. While patching is essential, the reality is that attackers move first.

By using proactive techniques like network fingerprinting, organisations can identify and neutralise the automated tools attackers rely on during the critical opening hours of a zero-day exploit's life. This approach, when combined with payload inspection from a WAF, gives critical assets another layer of practical protection.

A Complete Guide to SMS Pumping Fraud

2025-03-13T14:00:00+11:00

The Growth of SMS Fraud

SMS pumping fraud is a costly online abuse pattern, with global losses reaching an estimated $6.7 billion in 2021 alone. It targets companies that rely on SMS for verification or customer communications, leaving them to pay for traffic they did not request.

The scheme relies on malicious actors and dishonest telecom operators working together to generate and monetise large volumes of fraudulent text messages. For businesses caught in these schemes, the financial impact can be severe. Twitter (now X) reportedly lost $60 million to this type of fraud.

This guide explains how SMS pumping works, which businesses face the highest risk, and the controls your organisation can use to reduce exposure.

Understanding SMS Pumping Fraud

SMS pumping (also called SMS toll fraud, SMS spamming, or Artificially Inflated Traffic) involves manipulating mobile networks to inflate charges for text messages. The term "pumping" describes fraudsters forcing high SMS volume through a target's systems.

This fraud exploits how SMS messages travel and get billed across phone networks. Attackers target companies that use SMS codes to verify users. Each time a business sends a verification code, it pays a fee. Fraudsters trigger these systems to send thousands of messages to numbers they control.

These attacks create direct costs for businesses and revenue for the attackers. The fraud works through coordination between criminals and corrupt telecom operators, who charge premium rates for message delivery and share the proceeds.

The fraud has changed as more businesses have adopted SMS verification. Attackers keep developing new methods, and the phone industry has not removed the risk. Many companies still carry the financial exposure.

How SMS Pumping Works

SMS pumping attacks usually exploit message systems through these steps:

Finding Targets: Attackers look for websites or apps that send SMS codes for account verification or password resets.
Creating Fake Requests: Fraudsters use automation to send thousands of code requests to phone numbers they own or control.
Hiding Their Tracks: Attackers change their IP addresses and device information so requests appear to come from real users.
Sharing Profits: Fraudsters work with dishonest phone companies that charge high fees when messages pass through their networks. These companies then share the money with the attackers.
Using Complex Routes: Messages travel through many networks before reaching their destination, making the source of the fraud harder to trace.
Targeting Expensive Routes: Attackers focus on international numbers where sending messages costs more or where rules are weaker.

These attacks look legitimate because each message contains a real code sent to what appears to be a normal phone number. Companies like Twilio or Bird must pay fees to deliver these messages. Most businesses only find out about the fraud when a large bill arrives from their SMS service.

SMS pumping differs from basic spam because the profit-sharing between attackers and phone companies creates a direct cost for the target business.

Businesses at Risk

SMS pumping is most likely to affect these types of businesses:

Financial Institutions

Banks, investment platforms, and cryptocurrency exchanges use SMS codes to protect accounts. These firms send thousands of codes each day, which makes it hard to spot fake requests mixed with real ones.

E-commerce Platforms

Online shops use SMS messages when users create accounts, reset passwords, or make purchases. These businesses often run on small profit margins, so extra SMS costs can hurt their earnings. High volumes of new users make it easier for attackers to hide their activity.

Social Media Companies

Social networks use text messages to check user identity and stop fake accounts. These companies send millions of codes each day to users around the world. Twitter lost $60 million from this type of fraud, showing the scale these bills can reach.

Software-as-a-Service (SaaS) Providers

These companies often offer free trials that require SMS verification. They plan for a set cost to acquire each new user, but fraud can push these costs much higher than expected.

Telecommunications Companies

Phone companies face two problems: their own systems can be attacked, and parts of their network might help fraudsters. They need strong monitoring tools to find unusual patterns in message traffic.

Small Businesses and Startups

While smaller firms send fewer messages, they often lack security teams and fraud detection tools. This makes them easier targets. The cost of an attack can put these businesses at risk of closing down because they have less money in reserve.

Advanced Attack Methods

Attackers now combine SMS pumping with other techniques to avoid detection.

Credential Stuffing

Fraudsters use passwords stolen in data breaches to break into accounts. Once inside, they change phone numbers to ones they control and trigger verification messages. This makes fraud appear to come from real users.

Peakhour's breach database detection identifies when stolen credentials are used to access accounts. The system flags these attempts before phone numbers can be changed, stopping the attack chain.

Residential Proxy Networks

Unlike data centre proxies that security systems can often spot, residential proxies hide attack traffic behind home internet connections. This makes fraud look like it comes from regular users in different locations.

Peakhour specialises in residential proxy detection. Its technology identifies these masked connections and blocks them before verification requests can pass through. The system maps known proxy networks and detects signs of traffic passing through residential IPs.

When combined with device fingerprinting, these protections create a stronger defence. Fingerprinting tracks device characteristics that remain consistent even when attackers change IP addresses or accounts. Peakhour's fingerprinting technology works without cookies, making it effective against attackers who clear browser data.

These methods focus on the techniques fraudsters use to hide their identity. With Peakhour's protection, businesses can detect and block these attacks before they trigger costly SMS verification messages.

Historical Incidents

Reported SMS pumping incidents show how quickly costs can build:

Twitter's $60 Million Loss

In January 2023, Twitter owner Elon Musk said the platform lost more than $60 million to SMS pumping fraud. He named over 390 phone companies that took part in the scheme. While Twitter later questioned some claims, the case brought public attention to this type of fraud.

Industry-Wide Financial Impact

The Communications Fraud Control Association reports that SMS pumping caused global losses of $6.7 billion in 2021. Many companies do not share their fraud losses with the public.

Costs to Individual Businesses

Companies hit by these attacks pay between tens of thousands and millions of dollars each month in fake charges. These costs grow fast because each fake message costs much more than normal text rates.

Verification Policy Changes

Because of these threats, many large platforms have moved away from SMS codes. Twitter removed SMS verification for most users in March 2023, stating fraud as the reason.

Operational Disruptions

Beyond the cost of messages, businesses can face service problems during attacks. Real users may not get their codes on time. This can cause users to abandon transactions, contact support more often, and lose confidence in the company.

Rules and Enforcement

Rules to stop these attacks differ around the world. Some telecoms authorities have strict rules and fines for networks that allow fraud, but enforcement remains hard. Fraudsters use complex message routes that cross many countries to avoid getting caught.

Understanding the Stakeholders

SMS pumping involves these key groups:

Businesses

Companies use SMS to check user identity and send updates. They hire SMS gateway providers to handle their messages. When fraud happens, these businesses pay for the fake messages. Most find out about the attack only when they receive an unexpected bill.

SMS Gateway Providers

Companies like Twilio and MessageBird connect businesses to phone carriers. They give businesses tools to send text messages without working with phone networks directly. When fraud passes through their systems, these providers may try to stop it, but still charge businesses for the messages sent.

Mobile Network Operators (MNOs)

These companies run the networks that deliver messages to phones. Most work honestly, but SMS pumping schemes often include corrupt operators who charge extra fees for messages to numbers they control. These operators then split the money with the attackers who started the fraud.

Content Aggregators

These middlemen combine message traffic and work with many carriers to find the best routes. Most run honest operations, but their position in the message chain creates routing and oversight gaps that attackers can use.

Regulatory Bodies

Groups like the GSM Association create rules and standards for the industry. These rules are hard to enforce because phone networks cross many countries with different laws.

Financial Flow

The payment flow starts when businesses pay gateway providers to send messages. The gateway providers then pay fees to network operators based on where messages go. In fraud schemes, inflated fees go to corrupt operators who share the money with attackers. This creates a system where sending more fake messages makes more money for criminals while costing honest businesses more.

Effective Protection Strategies

Protecting your organisation usually requires several controls:

Basic Protections

Rate Limits: Restrict how many verification attempts a user can make in a set time period.
Traffic Pattern Checks: Track normal SMS message patterns and watch for changes that might indicate attacks.
Provider Protection: Services like Prelude's SMS Pumping Protection find and block messages to fake numbers.
Other Ways to Verify Users: Use app-based verification or push alerts instead of SMS codes.
Control by Country: Limit SMS verification to countries where you do business and add more checks for countries with higher fraud risk.
Work with Trusted Partners: Choose SMS service providers that focus on security and can help stop fraud quickly.

Advanced Protection Methods

Residential Proxy Detection: Find and block users who hide their true location behind home networks used as proxies.
Device Fingerprinting: Collect device signals to track users across sessions and spot when many verification requests come from the same device.
User Behaviour Tracking: Learn how real users act on your site and flag unusual actions that might be bots.
Machine Learning Systems: Use systems that learn from data to find hidden fraud patterns and adapt to new attack types.
Phone Number Checks: Use lists of known bad numbers to decide which phone numbers need more verification steps.
Verify in Multiple Ways: Ask users to prove who they are in different ways, such as email plus SMS, to make attacks harder.
Work with Other Companies: Share information about new attack methods and bad phone numbers with other businesses.
Watch Transactions as They Happen: Use systems that can pause message sending when they spot unusual patterns and learn from both legitimate and abusive traffic.

Fighting SMS Pumping Fraud

SMS pumping fraud costs businesses $6.7 billion worldwide each year. Companies like Twitter lost $60 million to these attacks, showing that scale alone does not remove the risk.

SMS pumping works through a network of fraudsters, network operators, and service providers who exploit the payment system for text messages. Fraudsters target authentication systems to generate large volumes of SMS, then collect revenue shares from the process.

Peakhour and Prelude offer combined protection against these threats. Peakhour provides device fingerprinting to identify suspicious devices attempting verification. Its residential proxy detection stops fraudsters who hide behind legitimate IP addresses. These tools block attackers before they access verification systems.

Prelude complements this protection with their multi-routing SMS verification platform. Its system uses real-time fraud detection across five messaging channels in 230 countries. When Prelude detects a potential attack, it automatically redirects traffic through secure routes.

Businesses need to understand the SMS delivery chain to protect themselves. Gateway providers, network operators, and content aggregators each introduce possible points of exploitation.

Prevention requires multiple security layers:

Rate limiting to restrict message volume
Device fingerprinting to track suspicious patterns
Residential proxy detection to unmask hidden attackers
Behavioural analytics to spot unusual activity
Machine learning to adapt to new attack methods
Continuous learning based on real user interactions

The continuous learning systems from both Peakhour and Prelude build protection that improves with each user interaction. Their platforms analyse legitimate traffic patterns to differentiate them from attacks, helping protection adapt over time.

While SMS verification remains common, Peakhour and Prelude help businesses implement more secure authentication methods. Together, they provide protection that adapts to evolving threats and reduces the cost of fraudulent verification traffic.

See how Peakhour's Application Security Platform helps protect against SMS pumping and other automated threats. Contact our team to secure your applications.

Why We Can't Trust IP Addresses

2025-03-11T14:00:00+11:00

Blocking bad traffic by checking an IP address used to be a reasonable starting point. It is not enough anymore. The rise of residential proxies, especially mobile proxies like those from Proxidize, has weakened one of the simpler assumptions in web security: that an IP address tells you much about who is behind a request.

Why is this a problem now?

Residential proxies route traffic through real household IP addresses, so requests look as if they come from normal homes rather than data centres. Companies like Proxidize have made mobile proxy setups accessible using Android phones or USB modems.

In my presentations at AISA and other security conferences, I've described these proxies as systems that "masquerade internet usage as originating from residential and office networks," because they sit outside the assumptions used by many security controls.

What has changed recently is access. Proxidize offers kits that let anyone set up a proxy farm - from 5-modem kits at $499 to 80-modem setups for around $6,000. They have turned proxy farming into a plug-and-play system where you can be up and running "in less than 60 seconds."

The scale is large. Proxidize users process an estimated 80 billion records combined every single day: 80B+ Records Scraped Daily.

The model is also being sold as a "passive income opportunity," where people can earn money by setting up proxy farms and selling access to others. In their recent webinar, they announced plans for a "Proxidize Grid" marketplace where users can sell their proxies with "a single click through an automated Marketplace."

The BYOD mobile proxy revolution

Companies like iProxy.online have taken this further with a Bring Your Own Device (BYOD) approach. Rather than requiring specialised hardware, they let customers turn any Android device into a mobile proxy.

As Sabir, the cofounder of iProxy.online, explained in a recent interview, "You can install iProxy app here and in the dashboard you have proxy access like Socks5, HTTP accesses, and traffic goes through your device."

This means anyone with an old Android phone and a SIM card can create their own mobile proxy, lowering the barrier to entry. For around $59 per month (based on Proxidize's pricing), users get access to what Sabir calls "precious" mobile IP addresses.

Why are mobile IPs so valuable? As Sabir explains: "If you have Barcelona, we are here in Barcelona and you have like 2 million people living there and you have like several thousands of IP addresses from your mobile providers. And one IP address is shared by many. By thousands of people... And if you have mobile IP address, this cannot be blocked by Facebook or Instagram or any other services because in this case, like innocent people, like thousands of them will be blocked."

This carrier-grade NAT (CGNAT) technology means mobile IP addresses are shared across thousands of users, making broad IP blocks difficult without affecting legitimate users.

What this enables attackers to do

With residential proxies, attackers can:

Hide behind legitimate IP addresses that security systems trust
Bypass geo-restrictions to attack from what appears to be a local source
Distribute attacks across thousands of residential IPs to avoid detection
Make malicious traffic look like it comes from normal users

In my work at Peakhour.IO, we've seen a rise in attacks originating from these residential proxies. The Chinese state-sponsored group Camaro Dragon showed the potential of the model when they developed custom firmware for TP-Link routers, turning them into residential proxies for their operations. This method let them bypass traditional defences like GeoIP blocking because the traffic appeared to come from normal homes.

The broader trend is commoditisation. You no longer need to be a nation-state actor to use them. Anyone with a few hundred dollars can set up a residential proxy farm or use services like iProxy.online to route their traffic through mobile networks.

How it enables data exfiltration

Data exfiltration is harder to detect when residential proxies are involved. State-sponsored actors like Volt Typhoon have used compromised network devices to "proxy all network traffic to targets through compromised SOHO network edge devices."

This means stolen data travels through home routers or office equipment before reaching the attacker, making it harder to trace. Since the traffic appears to come from thousands of different legitimate sources, traditional data loss prevention tools struggle to identify and block the exfiltration.

I've worked with organisations that have suffered breaches where data was exfiltrated through residential proxies. In these cases, the traffic blended in with normal home user traffic, making it difficult to detect. These weren't sophisticated nation-state attacks - they were conducted by ordinary cybercriminals using commercially available residential proxy services.

How it enables credential stuffing and other attacks

Credential stuffing attacks have hit Australian businesses hard, with companies like The Iconic, Guzman y Gomez, Dan Murphy's, and others falling victim. Residential proxies help these attacks work because attackers can distribute their login attempts across thousands of residential IP addresses.

When an attack comes through residential proxies, each login attempt appears to come from a different legitimate user. IP-based rate limiting fails because no single IP shows suspicious volume. Even when security teams try to block suspicious regions, proxies let attackers appear to be local customers.

According to our research at Peakhour.IO, traditional IP intelligence services are failing to detect these proxies. Tests we conducted showed that top providers like Maxmind detected 0% of residential proxies, while even the best performer, IP Quality Score, only identified 24%.

The traffic share can be significant. We've seen cases where up to 40% of traffic to Australian e-commerce sites consists of bots using residential proxies for credential stuffing, price scraping, and inventory checking. This puts customer accounts at risk, distorts analytics, and wastes marketing budgets on fake traffic.

The TCP/IP fingerprinting challenge

One aspect of mobile proxies that makes them even more effective is the ability to match TCP/IP fingerprints with the purported device. As Sabir from iProxy.online explains:

"In some cases, your fingerprint, TCP fingerprint should match to your user agent. For example, if you like pretending to be a Mac user or iOS user or Windows user, your TCP fingerprint should be matched with your browser fingerprint."

This means detection mechanisms that look for mismatches between TCP/IP fingerprints and browser types can also be bypassed.

Anybody can now set them up

The barrier to entry for setting up residential proxies has fallen sharply. Companies like Proxidize market their products as simple to use, with statements like "Start using Proxidize in less than 60 seconds."

There are YouTube videos showing how to earn "passive income" by setting up proxy farms. One video explains how hosts can earn "$200 a month minimum" by hosting Proxidize hardware in their homes.

With iProxy.online, it's even simpler—just install an app on an Android phone, and you have a mobile proxy. As Sabir explains, "Actually your expenses are like you pay like for the SIM card, you pay a small subscription fee to the service and you just... That's it. It requires like one minute of work just to download an app."

This accessibility means residential proxy use is no longer limited to nation-states and sophisticated cybercriminal organisations. It is now within reach of anyone with basic technical skills.

The solution: per-connection detection

The rise of residential proxies means IP reputation databases are not enough on their own. As I've been explaining in my talks, "Residential proxies pose a significant challenge to traditional defense mechanisms... making malicious traffic appear legitimate."

The practical answer is per-connection detection that looks at network behaviour patterns rather than just IP addresses. At Peakhour.IO, we stack detections across layers to identify and mitigate proxy traffic.

A useful technique is analysing protocol behaviour. When traffic passes through a residential proxy, there are often detectable differences between network signatures (which come from the proxy) and the application behaviour (which comes from the third-party application).

These techniques can identify proxy connections even when they come from legitimate residential IP addresses, giving defenders a way to respond without blocking whole residential or mobile networks.

A call to action for businesses

If you're a business, especially in e-commerce, financial services, or any industry that relies on user accounts, residential proxy traffic needs to be part of your security model.

Traditional security approaches based on IP reputation, geolocation, and rate limiting are no longer sufficient. You need to implement per-connection detection that can identify residential proxy usage regardless of the source IP address.

At Peakhour.IO, we've seen organisations fall victim to attacks that could have been prevented with the right detection mechanisms. Waiting until credential stuffing or data exfiltration becomes visible is the expensive way to learn this lesson.

IP addresses alone can no longer tell us who to trust. We need to look deeper at each connection to protect systems and data now that proxy networks are easy to rent or build.

How AI Agents Are Writing Custom Exploits

2025-02-17T14:00:00+11:00

The trend is clear enough: AI agents can now craft exploits by analysing security responses in real time. That puts static security rules and traditional Web Application Firewalls (WAFs) under direct pressure. Here is why.

Last week I examined an AI agent probing a test environment. It sent requests, observed the responses, then built bypasses for each security control in sequence. The agent identified pattern-based rules, learned their structure, and generated variations until it found gaps. It did this without human intervention.

This kind of automated exploit development changes the operating conditions for defenders. Traditional defences rely on known patterns: regex rules, signature matching, IP reputation. Those approaches assume threats follow recognisable templates. That assumption is becoming much weaker.

Consider a standard WAF rule blocking SQL injection through pattern matching. An AI agent examines the responses, determines the matching patterns, then generates unique variants designed to bypass those rules while maintaining the exploit's functionality. The variants evolve as the agent learns which approaches succeed.

The same pattern applies beyond SQL injection. AI agents can probe XSS filters, access controls, and input validation in the same systematic way. Each static rule becomes something the agent can test, infer, and work around.

By 2026, I estimate AI agents will drive over 50% of exploit attempts. The speed of this shift stems from three factors:

AI agents operate continuously, testing and learning 24/7
Successful exploits feed back into training data, improving future attempts
Agents share knowledge, building collective intelligence about bypass techniques

This is the practical limit of static security. Traditional WAFs that rely on fixed rules and signatures struggle to keep pace with AI-generated exploits. Each rule loses value as agents discover new bypasses.

The path forward requires a different security architecture. Organisations need context-aware systems that analyse intent, not just patterns. These systems use behavioural AI to distinguish between legitimate requests and exploit attempts, even when the request structure changes.

Key elements of this new approach include:

Intent analysis through deep inspection of request sequences
Behavioural modelling of normal vs malicious patterns
Real-time adaptation as new exploit techniques emerge
Proactive identification of potential vulnerabilities
Integration of threat intelligence across systems

The challenge intensifies when AI agents leverage residential proxies. These proxies route traffic through real consumer IP addresses, bypassing location-based blocks. An AI agent operating through residential proxies can probe defences while appearing to come from legitimate users worldwide.

This combination of AI-driven exploit generation and residential proxy networks makes traditional controls much less reliable. Organisations that continue to rely on static rules face a growing risk of compromise.

Security teams should respond now:

Audit existing WAF rules to identify pattern-based weaknesses
Deploy behavioural analysis capabilities to detect malicious intent
Implement adaptive security controls that evolve with threats
Monitor for AI-driven probing attempts
Build detection for residential proxy traffic

Teams that wait risk watching their defences get mapped and bypassed by automated agents. Static rules alone are not enough for this level of probing.

This also requires a shift in how we approach security. Rather than only blocking specific patterns, we need to understand and control the broader context of system interactions. The goal moves from "preventing known attacks" to "identifying and blocking malicious behaviour, regardless of its specific form."

Adaptive security systems need to reason about traffic in the same context-aware way as the agents probing them. Static rules still have a role, but they cannot be the centre of the defence.

Security strategy needs to account for this now, because AI-driven probing is no longer hypothetical.

The Reasoning Model Revolution

The emergence of open reasoning models like DeepSeek pushes this further. Unlike traditional AI that follows programmed patterns, reasoning models understand context and adapt strategies dynamically. That creates harder security problems.

Consider how a reasoning model approaches security testing. Rather than simply probing for weaknesses, it builds a conceptual model of the system's defences. It understands the purpose of security controls and reasons about potential bypasses. That allows it to generate novel attack strategies that were not present in training data.

DeepSeek demonstrates this shift. Within months of release, it showed capabilities matching established players at a fraction of the cost. This rapid progress comes from reasoning models' ability to understand and adapt, not just pattern match.

For security teams, that is a material challenge. Reasoning models do not just find gaps in rules. They infer why rules exist, deduce the logic behind security controls, and generate attacks that exploit underlying assumptions.

By 2027, I expect reasoning models to handle most security testing and exploit development. Their advantages prove too compelling:

They understand system architecture and security principles
They generate novel attack strategies through reasoning
They adapt in real-time based on system responses
They share and build upon successful approaches

This shift pushes traditional security approaches past their useful boundary faster than many teams expect. Pattern matching and rule-based systems cannot reliably counter an opponent that understands and reasons about their operating logic.

The combination of reasoning models with residential proxies is especially difficult to defend against. Reasoning models devise sophisticated attacks while proxies mask their origin. Each successful breach feeds back into the model's understanding, improving future attempts.

Security teams must embrace a new paradigm focused on:

Understanding attack narratives rather than patterns
Detecting anomalous reasoning rather than known signatures
Building systems that adapt to novel attack strategies
Implementing security that reasons about intent
Developing defences that evolve through adversarial learning

Security systems need to reason about threats as effectively as the AI agents probing them. Traditional approaches will fail against opponents that understand the logic behind security controls and devise creative bypasses.

The age of reasoning security has begun. Static rules and pattern matching are no longer enough on their own.

The question is how quickly security teams can move from fixed patterns to adaptive, intent-aware defence.

How MTU Fingerprinting Identifies VPNs and Mobile Users

2025-01-15T14:00:00+11:00

For traffic analysis, it helps to know how a user reached the service. Are they on a home network, a mobile connection, or a VPN? Deep packet inspection is invasive, but TCP handshake metadata can still carry useful context about the Maximum Transmission Unit (MTU) a connection appears to be using. By analysing those inferred MTU values, we can build "fingerprints" that point to the underlying network technology carrying the connection.

This article looks at how common technologies affect MTU values and shows how a SQL query can turn that data into useful network labels.

What is MTU and Why Does it Change?

The Maximum Transmission Unit (MTU) is the largest data packet, or frame, that a network-connected device can transmit. On standard Ethernet networks, this value is typically 1500 bytes. Larger payloads have to be split into chunks that fit that limit.

Encapsulation and Tunneling

The value starts to shift when tunnelling protocols are involved, including those used by VPNs and mobile networks. These protocols wrap the original data packet inside another packet, a process called encapsulation. The outer packet has its own headers for routing and management.

This encapsulation "steals" space from the original 1500 bytes available on the physical network. If a tunnelling protocol adds 60 bytes of headers, for example, the maximum size for the original data packet is now 1440 bytes (1500 - 60).

The Problem with Fragmentation

What happens if a device tries to send a 1500-byte packet through this 1440-byte tunnel? The packet has to be broken into smaller pieces, a process called fragmentation. It works, but it is inefficient. Fragmentation consumes CPU resources on the router performing it, adds header overhead to each fragment, and requires the receiving device to reassemble the pieces. The result is lower speed and higher latency.

To avoid that penalty, operating systems and network devices reduce the MTU of the connection to account for the tunnel's overhead. The amount of the reduction follows from the tunnelling protocol in use. That predictable drop is the basis for MTU fingerprinting.

A Guide to Common MTU Values

Different technologies add different overheads, which produces distinct MTU values.

WireGuard

WireGuard is a modern VPN known for its efficiency, but it still adds overhead.

IPv4 Overhead: 60 bytes (20-byte IPv4 header + 8-byte UDP header + 32-byte WireGuard header).
IPv6 Overhead: 80 bytes (40-byte IPv6 header + 8-byte UDP header + 32-byte WireGuard header).

On a standard 1500-byte network, that produces predictable MTU values:

1500 - 60 = 1440 bytes (WireGuard over IPv4)
1500 - 80 = 1420 bytes (WireGuard over IPv6)

There is a special case with ISPs that use DS-Lite (Dual-Stack Lite) to carry IPv4 traffic over an IPv6 network. This adds another 40-byte IPv6 header, reducing the MTU further.

1420 - 40 = 1380 bytes (WireGuard over DS-Lite)

OpenVPN

OpenVPN is another common VPN solution, but its fingerprint is less tidy. Instead of setting a static interface MTU, OpenVPN often uses a feature called mssfix. This dynamically adjusts the Maximum Segment Size (MSS) value within the TCP headers of encapsulated packets to prevent fragmentation.

The MSS is the MTU minus the IP and TCP header sizes (typically 40 bytes for IPv4). The exact MSS value, and therefore the effective MTU, depends on OpenVPN's configuration, including the transport protocol (UDP or TCP), cipher, MAC algorithm, and compression. As noted by security researcher ValdikSS, these unique MSS values can be used to fingerprint a connection with high precision. For example, a common configuration might result in an MSS of 1369, which corresponds to an effective MTU of 1409 (1369 + 40).

For general analysis, connections with an MTU around 1400 or 1380 bytes often indicate OpenVPN or other VPN usage, especially when seen with other factors.

Mobile Networks (LTE & 5G)

Mobile networks also modify MTU values. When your phone connects to the internet, its data is tunnelled through the carrier's network using the GPRS Tunnelling Protocol (GTP). This encapsulation adds its own layer of headers.

As detailed by Nick vs Networking, the typical overhead for GTP traffic over an Ethernet transport network is 50 bytes:

14 bytes for the Ethernet header
20 bytes for the outer IPv4 header
8 bytes for the UDP header
8 bytes for the GTP header

For a mobile carrier using a standard 1500-byte MTU on its transport network, the maximum MTU available to the user's device is 1450 bytes (1500 - 50).

Mobile devices don't guess this value; they are explicitly told what MTU to use by the network during the connection setup process (via Protocol Configuration Options). Mobile operators have two choices to avoid fragmentation:

Increase Transport MTU: Enable jumbo frames (for example, 1600 bytes or more) on their internal network to accommodate the 50-byte overhead and still provide a full 1500-byte MTU to the user.
Lower Advertised MTU: Advertise a lower MTU to the user's device. This is why values such as 1450 are common. Some operators may configure a more conservative MTU, such as 1300 bytes, to maintain stability across all parts of their network.

Other Common Values

Standard Ethernet: The baseline is 1500 bytes.
PPPoE: Common for DSL connections, adds 8 bytes of overhead, resulting in an MTU of 1492 bytes.
IPv6 Minimum: The IPv6 specification mandates a minimum MTU of 1280 bytes, so this value is also a significant marker.

Analysis with SQL

With this context, we can analyse network logs to classify user connections. The following SQL query buckets and attributes MTU values from a large dataset, turning raw numbers into meaningful labels.

The query works in several stages:

Extract Data: It parses the MTU from a fingerprint string in the logs.
Bucket MTUs: It uses a CASE statement to group MTUs. Specific known values, such as 1500, 1440, 1420, and 1380, go into their own buckets. Jumbo frames (>1500) are grouped into 100-byte buckets, and everything else is grouped into 20-byte buckets.
Attribute Buckets: In the final SELECT, another CASE statement translates those numeric buckets into human-readable descriptions based on the fingerprints we've identified.

The Query

-- Bucketing logic and attribution informed by research from:
-- https://ripx80.de/posts/06-wg-mtu/ (WireGuard)
-- https://medium.com/@ValdikSS/detecting-vpn-and-its-configuration-and-proxy-users-on-the-server-side-1bcc59742413 (OpenVPN)
-- https://nickvsnetworking.com/mtu-in-lte-5g-transmission-networks-part-1/ (Mobile Networks)
WITH base_data AS (
    SELECT
        toInt32OrNull(splitByChar(':', splitByChar(',', synner_fingerprint)[1])[4]) AS mtu,
        toInt32OrNull(splitByChar(':', splitByChar(',', synner_fingerprint)[1])[5]) AS wsize,
        toInt32OrNull(splitByChar(':', splitByChar(',', synner_fingerprint)[2])[1]) AS scale,
        (tls.handshake_rtt_us - tcp.min_rtt_us) >= 65000 AS is_high_latency
    FROM logs.client_logs
    WHERE time >= '2025-07-01' AND shielded = 0
),
main_aggs AS (
    SELECT
        CASE
            WHEN mtu = 1500 THEN 1500
            WHEN mtu = 1440 THEN 1440
            WHEN mtu = 1420 THEN 1420
            WHEN mtu = 1380 THEN 1380
            WHEN mtu > 1500 THEN 1501 + intDiv(mtu - 1501, 100) * 100
            ELSE intDiv(mtu, 20) * 20
        END AS mtu_bucket,
        countIf(is_high_latency) AS high_latency_count,
        countIf(not is_high_latency) AS normal_latency_count,
        round(avg(wsize * pow(2, scale))) AS avg_real_wsize
    FROM base_data
    WHERE mtu IS NOT NULL AND wsize IS NOT NULL AND scale IS NOT NULL
    GROUP BY mtu_bucket
),
top_wsizes AS (
    SELECT
        mtu_bucket,
        groupArray((wsize, cnt)) AS top_wsizes
    FROM
    (
        SELECT
            CASE
                WHEN mtu = 1500 THEN 1500
                WHEN mtu = 1440 THEN 1440
                WHEN mtu = 1420 THEN 1420
                WHEN mtu = 1380 THEN 1380
                WHEN mtu > 1500 THEN 1501 + intDiv(mtu - 1501, 100) * 100
                ELSE intDiv(mtu, 20) * 20
            END AS mtu_bucket,
            wsize,
            count() AS cnt,
            row_number() OVER (PARTITION BY mtu_bucket ORDER BY cnt DESC) AS rn
        FROM base_data
        WHERE mtu IS NOT NULL AND wsize IS NOT NULL AND scale IS NOT NULL
        GROUP BY mtu_bucket, wsize
    )
    WHERE rn <= 5
    GROUP BY mtu_bucket
),
top_scales AS (
    SELECT
        mtu_bucket,
        groupArray((scale, cnt)) AS top_scales
    FROM
    (
        SELECT
            CASE
                WHEN mtu = 1500 THEN 1500
                WHEN mtu = 1440 THEN 1440
                WHEN mtu = 1420 THEN 1420
                WHEN mtu = 1380 THEN 1380
                WHEN mtu > 1500 THEN 1501 + intDiv(mtu - 1501, 100) * 100
                ELSE intDiv(mtu, 20) * 20
            END AS mtu_bucket,
            scale,
            count() AS cnt,
            row_number() OVER (PARTITION BY mtu_bucket ORDER BY cnt DESC) AS rn
        FROM base_data
        WHERE mtu IS NOT NULL AND wsize IS NOT NULL AND scale IS NOT NULL
        GROUP BY mtu_bucket, scale
    )
    WHERE rn <= 5
    GROUP BY mtu_bucket
)
SELECT
    CASE
        WHEN mtu_bucket IN (1500, 1440, 1420, 1380) THEN toString(mtu_bucket)
        WHEN mtu_bucket > 1500 THEN concat(toString(mtu_bucket), '-', toString(mtu_bucket + 99))
        ELSE concat(toString(mtu_bucket), '-', toString(mtu_bucket + 19))
    END AS mtu_range,
    CASE
        WHEN mtu_bucket = 1500 THEN 'Standard Ethernet'
        WHEN mtu_bucket = 1480 THEN 'Likely PPPoE (e.g., 1492)'
        WHEN mtu_bucket = 1460 THEN 'Likely DS-Lite/GRE Tunnel'
        WHEN mtu_bucket = 1440 THEN 'Likely Mobile LTE/5G (e.g., 1450) / WireGuard over IPv4'
        WHEN mtu_bucket = 1420 THEN 'WireGuard over IPv6'
        WHEN mtu_bucket = 1400 THEN 'Likely OpenVPN / Mobile'
        WHEN mtu_bucket = 1380 THEN 'Likely OpenVPN / WireGuard over DS-Lite / Mobile'
        WHEN mtu_bucket = 1300 THEN 'Likely Mobile LTE/5G configured'
        WHEN mtu_bucket = 1280 THEN 'IPv6 Minimum'
        WHEN mtu_bucket > 1500 THEN 'Jumbo Frame'
        ELSE 'Other'
    END AS mtu_attribution,
    high_latency_count,
    normal_latency_count,
    round(high_latency_count / (high_latency_count + normal_latency_count), 2) AS high_latency_ratio,
    top_wsizes,
    top_scales,
    avg_real_wsize
FROM main_aggs
LEFT JOIN top_wsizes USING (mtu_bucket)
LEFT JOIN top_scales USING (mtu_bucket)
WHERE (high_latency_count + normal_latency_count) > 10000
ORDER BY mtu_bucket
LIMIT 50 FORMAT Vertical

Why Jumbo Frames Matter

Jumbo frames (MTU values greater than 1500 bytes) are a useful edge case in MTU fingerprinting. These frames, typically ranging from 9000-9216 bytes, are primarily used in high-performance computing environments, data centres, and enterprise networks where throughput optimisation is important.

When we detect jumbo frame MTUs in our analysis, they often indicate:

Enterprise Users: Corporate networks frequently enable jumbo frames for internal communications
Data Centre Traffic: Cloud services and CDNs often use jumbo frames between their infrastructure
High-Performance Applications: Video streaming, large file transfers, and backup operations can benefit from larger frame sizes
Network Misconfiguration: Jumbo frames sometimes appear because of network equipment misconfiguration

The presence of jumbo frames can help distinguish consumer and enterprise traffic, adding useful context for traffic classification and security analysis.

Practical Use Cases and Applications

MTU fingerprinting is useful across several security and operational domains:

Security Applications

VPN Detection for Compliance: Organisations can identify employees bypassing corporate network policies with personal VPNs, supporting compliance with data governance requirements.

Bot Traffic Classification: Automated traffic from residential proxy networks often shows consistent MTU patterns that differ from genuine residential users, improving bot detection.

Threat Intelligence Enhancement: Correlating MTU patterns with other indicators helps build broader threat profiles and improves attack attribution.

Network Operations

Performance Optimisation: Understanding the MTU distribution of your user base helps optimise content delivery and reduce fragmentation-related performance issues.

Infrastructure Planning: MTU analysis reveals the underlying network technologies your users employ, informing CDN placement and capacity planning decisions.

Quality of Service: Different MTU patterns correlate with connection quality, enabling proactive support for users on constrained networks.

Business Intelligence

Market Analysis: Geographic and demographic patterns in MTU distribution reveal technology adoption trends and market characteristics.

User Experience Optimisation: Identifying users on mobile or constrained networks enables adaptive content delivery and interface optimisation.

Dynamic Analysis vs Static IP Databases

MTU fingerprinting is a dynamic signal, which makes it useful alongside static IP reputation databases. It has several practical advantages:

Real-Time Adaptation

Static IP databases go stale. A residential IP address might be flagged as malicious based on historical activity, but MTU fingerprinting analyses the current network configuration. This dynamic approach captures the infrastructure being used at the moment of connection, providing more accurate and timely intelligence.

Circumvention Resistance

Attackers can rotate IP addresses or use clean residential proxies to bypass static blacklists. It is harder to manipulate the network characteristics that influence MTU values, because MTU is determined by the underlying network infrastructure.

Granular Classification

Where IP databases provide binary classifications (malicious/benign), MTU fingerprinting offers more detail on the specific technologies and configurations in use. This granularity enables more sophisticated risk assessment and response strategies.

Reduced False Positives

Static databases often flag legitimate users sharing IP addresses with malicious actors, which is common with residential ISPs and mobile carriers. MTU fingerprinting focuses on network behaviour rather than IP reputation, reducing false positive rates while maintaining security effectiveness.

Infrastructure Transparency

MTU analysis reveals the network path and technologies involved in a connection, providing transparency that static IP databases cannot match. This visibility enables more informed security decisions and a better understanding of threat actor capabilities.

Conclusion

MTU fingerprinting turns network metadata into useful context about the infrastructure behind a connection. Unlike static databases that rely on historical reputation, this dynamic analysis technique provides real-time insight into network technologies, user behaviours, and potential security threats.

By understanding MTU patterns, security teams can identify VPN usage, classify mobile traffic, detect residential proxy abuse, and optimise network performance. Its resistance to circumvention and low false-positive rates make it a useful addition to modern security architectures.

As network technologies continue to evolve, MTU fingerprinting provides a stable way to understand and classify traffic based on fundamental network characteristics rather than short-lived indicators. That makes it a practical signal for network security and operations.

Residential Proxies - The Growing Threat to Ad Campaigns

2024-12-30T00:00:00+11:00

Digital advertising fraud costs organisations $42 billion annually through fake clicks and fake impressions. The growth of residential proxy networks has changed how this fraud reaches campaigns: bot traffic can now hide behind legitimate residential IP addresses, putting it outside the reach of many traditional checks.

Hiding in the crowd

Residential proxies make bad traffic harder to separate from real visitors. Unlike data centre IPs that traditional tools can often detect, residential proxies hide behind real households' internet connections. This means the traffic appears to come from genuine users in your target market. When a residential proxy network operates from Sydney suburbs to attack an Australian campaign, existing protection systems can be fooled into treating it as authentic local traffic.

The impact extends beyond direct financial losses. Your analytics may show engagement from what appears to be your target demographic, while the activity is bot traffic masquerading as potential customers. This contaminated data can push marketing strategy in the wrong direction and waste retargeting spend. Competitors can also use fake clicks to drain your budget while gathering intelligence on your campaigns.

Bad data then compounds the spend problem. Once bots are counted as engaged prospects, reporting and optimisation start from the wrong signal. The result is not only wasted media spend, but poorer decisions built on traffic that should never have been treated as customer intent.

A growing threat

The residential proxy industry continues to expand. Services now offer millions of residential IPs with precise geographic targeting capabilities. They rotate IPs automatically and match real browser fingerprints. Without specialised detection methods, the traffic can become indistinguishable from genuine users.

This is a budget problem, not just a technical one. Each day without protection means 30-40% of your ad budget feeds bot networks instead of reaching customers. The corrupted analytics drive decisions that compound these losses. As residential proxy services grow more sophisticated, basic controls fall further behind.

Traditional IP reputation and rate limiting fail against this distributed threat because the IP addresses are not obviously suspicious. Protection requires advanced network fingerprinting that looks beyond IP addresses. Peakhour's Ad Fraud Protection analyses subtle patterns in how residential proxies connect and behave, and detects the signs of proxy traffic that other solutions miss.

Knowledge is power

Peakhour integrates this protection with your existing ad platforms to stop fraud before it affects your campaigns. Our customers have reduced wasted ad spend by 35% while improving campaign performance through cleaner analytics. The system adapts as threat techniques change, so detection keeps pace with new residential proxy methods.

Residential proxies have changed ad fraud because traffic that appears local and legitimate may mask sophisticated bot networks. Protecting your campaigns requires detection that goes beyond IP addresses and treats residential proxy behaviour as its own signal. Contact us to learn how we can help secure your ad spend against residential proxy networks.

Next-Generation Application Security Defence Strategies

2024-11-15T14:00:00+11:00

As I look at recent cyber threat activity, the pattern is clear enough: AI is no longer only a defensive tool. Attackers are using it to probe, adapt, and automate application-layer attacks.

One recent incident made that plain. Our threat detection systems identified a series of probes against a client's infrastructure. These were not the typical brute-force attempts we are used to blocking. The attack patterns changed in real time, adapted to our defences, and probed for weaknesses in a way that pointed to AI-driven automation.

The individual attempts were not the main concern. What mattered was how the attack system learned and adjusted its approach. When we blocked one vector, it shifted to another. When we implemented rate limiting, it distributed its attempts through residential proxies. The attack showed a common trait of AI systems: rapid iteration and learning from failure.

This change in attack methodology puts real pressure on the traditional security model. Static defences, including controls that looked strong only months ago, are easier to route around. They might stop obvious threats, but more capable AI-powered attacks can keep testing the edges until they find a path.

The threat landscape has shifted in three practical ways. First, AI enables attacks to adapt and evolve in real time. Second, residential proxies give attackers a distributed network of IP addresses that appear legitimate, making traffic origin verification much harder. Third, AI can analyse and mimic legitimate user behaviour patterns closely enough to bypass traditional bot detection.

These changes require a change in defence strategy. Identifying and blocking known attack patterns still matters, but it is no longer enough on its own. We need systems that can anticipate and adapt to new threats as quickly as they emerge.

In our security operations, we've begun implementing what we call "contextual defence dynamics." The approach moves beyond simple pattern matching to analyse the intent and behaviour behind each request. We examine not just what a request does, but how it fits into broader patterns of behaviour and what it might indicate about the attacker's objectives.

That approach has already proved useful. When we implemented contextual defence dynamics for a major e-commerce client, we identified and blocked an AI-powered credential stuffing attack that had evaded traditional detection methods for weeks. The attack used residential proxies to distribute its attempts and mimicked human behaviour patterns, but our system identified subtle anomalies in its timing and response patterns.

That case highlighted a useful point: while AI-powered attacks grow more sophisticated, they still exhibit patterns. Those patterns may not appear in individual actions, but they do appear in broader behaviour and objectives. By shifting our focus from blocking specific actions to understanding and responding to these broader patterns, we can maintain effective defences even against evolving threats.

This approach requires a different way of thinking about security. We must move from a model of static defences to one of dynamic response. Our security systems must learn and adapt as quickly as the threats they face. This means implementing machine learning systems that can identify new attack patterns, updating defence strategies in real time, and maintaining awareness of emerging threat vectors.

The implications extend beyond technical implementation. Organisations need to treat security budgets and strategies as ongoing commitments, not one-off purchases. The era of "set and forget" security solutions has ended. Continuous adaptation and review now sit at the centre of effective defence.

I expect this arms race to keep accelerating. AI will continue to enhance both attack and defence capabilities. The organisations that maintain strong security will be those that accept this dynamic and build their defences around continuous adaptation.

For security professionals, this means developing new skills and approaches. We must understand not just the technical aspects of security, but the patterns of attack and defence that emerge in AI-driven systems. We must build systems that can learn and adapt, and we must be prepared to change strategy as the threat landscape evolves.

The security arms race has entered a new phase. The advantage will not sit with the strongest static defences alone, but with teams that can adapt and evolve their protection strategies in real time. The focus must shift from building walls to creating intelligent, adaptive defence systems that can match the sophistication of AI-powered threats.

This shift in security thinking is practical, not theoretical. The threats we face are becoming more capable, and defensive tooling is improving as well. The important step is recognising the change and adapting how we build, operate, and review application security controls.

Quantifying The Residential Proxy Threat

2024-07-18T10:00:00+10:00

Our 2024 survey found that only 15% of Australian businesses use residential proxy detection. That leaves a measurable blind spot in many security programmes: traffic routed through real consumer connections is harder to separate from legitimate users. This article looks at why residential proxy detection is difficult and how to quantify the risk before choosing controls.

Understanding the Residential Proxy Threat Landscape

Residential proxies use IP addresses assigned to residential internet connections, so malicious traffic can look legitimate. This weakens controls built around IP reputation, GeoIP, and simple request thresholds, and creates a specific detection problem for security teams.

The effectiveness of residential proxies stems from their ability to:

Use legitimate IP addresses, often from unsuspecting users
Bypass IP-based rate limiting and traditional bot detection methods
Evade geolocation restrictions, making GeoIP filtering less reliable
Support large-scale attacks without triggering typical alarm thresholds
Mimic legitimate user behaviour, which makes detection more difficult

These capabilities make residential proxies useful infrastructure for credential stuffing, data scraping, and attempts to bypass fraud detection systems. Because the traffic is distributed across many residential connections, attacks can stay below the thresholds that conventional controls rely on.

Limitations of Conventional Security Approaches

Conventional controls have clear gaps when they are applied to residential proxy traffic:

IP-based detection misses constantly changing, legitimate-appearing IP addresses.
GeoIP filtering becomes less useful against globally distributed residential IPs.
User agent analysis struggles because proxies can mimic legitimate browsers.
Standard rate limiting falters when attacks appear to originate from many unique IPs.
Behavioural analysis based on known bot patterns may miss more careful proxy-based attacks.

These limitations point to a practical requirement: security teams need controls that assess context, not just static request attributes. Residential proxies make simple rule-based decisions less reliable, especially when attacks are distributed and deliberately low-noise.

Quantifying the Risk

To make a sensible decision about residential proxy controls, organisations need to quantify the risk. This involves:

Assessing the potential financial impact of successful attacks via residential proxies
Evaluating the likelihood of such attacks based on industry trends and organisational attractiveness to attackers
Determining the effectiveness of current security measures against this specific threat
Calculating the return on investment for implementing advanced detection and mitigation strategies

Risk quantification gives businesses a clearer basis for investing in residential proxy detection. It aligns security spending with actual threat levels and potential impacts, rather than broad concern or industry pressure alone.

Reframing Security

The challenge of residential proxy detection is less about one new control and more about how signals are combined. A useful approach includes:

Contextual Analysis: Analyse the full context of each request, not just its origin. This includes examining patterns of behaviour across multiple sessions and users.
Continuous Monitoring and Adaptation: Use real-time monitoring systems that can detect subtle patterns indicative of proxy use. These systems should continuously adapt to new attack vectors.
Risk-Based Authentication: Use dynamic authentication mechanisms that adjust based on the assessed risk of each session or transaction.
Holistic Data Analysis: Correlate data from multiple sources - including login attempts, transaction patterns, and user behaviour - to identify anomalies that may indicate proxy use.
Proactive Threat Hunting: Actively search for indicators of residential proxy use within your network and user base, rather than waiting for attacks to trigger alerts.

This approach moves beyond simple allow/block decisions and gives teams a better view of user and network behaviour.

Implementing Advanced Detection Strategies

Residential proxy threats need detection that looks beyond the source IP:

Machine Learning-Based Behavioural Analysis: Use AI and machine learning to identify patterns consistent with proxy use, even when individual actions appear legitimate.
Device Fingerprinting Beyond IP: Use advanced fingerprinting techniques that identify individual devices based on a combination of factors, making it harder for proxies to mimic legitimate users.
Network Traffic Analysis: Analyse network behaviour at a granular level to identify patterns consistent with proxy network traffic.
Adaptive Challenge Mechanisms: Deploy targeted challenges based on risk assessment, without disrupting legitimate user experiences.
Cross-Organisational Data Sharing: Participate in threat intelligence sharing networks to gain broader insights into residential proxy activities and emerging attack patterns.

When used as part of the broader security stack, these strategies improve defence against residential proxy threats.

Elevating Security Through Risk Quantification

Residential proxies are not only a technical detection problem. They change the risk model for web applications because attacker traffic can borrow the appearance of ordinary residential users. By adopting a risk quantification approach and implementing advanced detection strategies, organisations can:

Align security investments with actual threat levels
Improve detection of sophisticated, proxy-based attacks
Strengthen overall security posture against evolving threats
Make data-driven decisions about security priorities and resource allocation

Organisations that handle this well will be able to quantify their risk, adapt their security strategies, and implement intelligent detection mechanisms. The goal is practical: identify, analyse, and mitigate sophisticated threats before they cause material damage.

Effective protection starts with understanding the risk well enough to measure it.

Rate Limiting for API Security

2024-01-24T13:00:00+11:00

Rate limiting prevents servers from being overwhelmed by too many requests in a short period of time. Typically, rate limiting is configured using rules made up of a filter, for example a path like /login, and a limit on the number of requests a user can make in a given time, such as 10 requests in a minute. If a user exceeds this limit, they are usually blocked for a timeout period.

But how do you identify a user? Traditionally rate limiting has used the IP address for grouping requests, assuming that requests from the same IP address will be the same user. That assumption is now weak. IP addresses are rarely static and are often shared. For example, an office network might have hundreds of individual computers in it but present a single IP address for all those computers to the internet. Mobile operators commonly use carrier-grade network address translation (CGNAT) to share the same IP across thousands of devices or users. Bot networks, seeking to avoid security controls like rate limiting, will rotate their requests through thousands of different IP addresses. This makes rate limiting based on IP addresses a poor choice from both a functional and a security perspective.

Introducing Advanced Rate Limiting

Peakhour's Advanced Rate Limiting service lets you create filters using any HTTP request characteristic, for example URI, request method, headers, cookies, country, network fingerprints and more. You can also use response headers and response codes, so a rule can count failed login attempts, repeated 404s from a scraper, or traffic that crosses an API threshold.

For counting requests you can use the following fields for grouping:

IP Address
ASN
Country Code
HTTP/2 Fingerprint
TLS Fingerprint
Any combination of Request Headers

You can use one of those fields, or a combination of them, to identify users with more control than IP address alone.

You can also separate the filter and mitigation expression. For example excessive attempts to /login can be blocked on the entire site.

This matters because rate limiting is not just a request counter. In Peakhour it sits beside bot management, WAF, DDoS protection, traffic controls, and origin shielding on the same managed edge path. That gives operators a practical way to set different thresholds for verified crawlers, suspicious automation, authenticated API clients, and normal visitors without pushing every policy change into the application. It also gives them allowed, blocked, and threshold-hit evidence to tune the rule after it is deployed, whether Peakhour is the active edge or adding controls beside an existing CDN or cloud edge.

Putting it into action

Advanced Rate Limiting can help protect applications from attacks like Layer 7 DDoS, Account Takeovers, Credential Stuffing, and more. Here are some real world examples you can configure using our dashboard and API.

Protect against general site abuse

Our example website is a medium-sized ecommerce store that has page URLs ending in /. It serves Australian clients and typically sees around 100 page requests a minute from non-search-engine traffic during peak traffic times. With that baseline, we can set up rate limiting to prevent general site abuse and protect against layer 7 DDoS attacks.

Peakhour rate limiting starts with zones. You specify your request limits in these zones.

Here we've specified a maximum of 45 requests in 1 minute. We're going to apply this limit to page loads only. Since our typical maximum for all users on this website is 100 in a minute, it seems reasonable that a real user is not going to view 40 pages in 1 minute. We could also specify a value for error responses in a minute. An error could be a 404, which a scraper might typically get when looking for removed URLs.

Now let's define our filter and our counter. For our filter we mentioned that pages end in /, so we'll use that, but exclude verified bots to make sure they're not restricted when crawling the site. A verified bot is a crawler like Google or Bing, that Peakhour has verified as legitimate by using reverse DNS to confirm they are who they say they are.

Attackers, scrapers, and others looking to abuse a site will launch an attack using a particular piece of software. That piece of software will have a TLS fingerprint (like JA3) that remains the same, even as the attacker rotates their user-agent, IP address, and other characteristics, so we'll use the TLS fingerprint as our request counter.

Rate Limit authenticated API Users

It is common for APIs to require an Authorization header as part of the request to authenticate access. By grouping requests on the value of this header, we can rate limit a specific API client even if it uses multiple applications, or if its credentials are stolen.

Protecting from Account Takeovers

Account Takeover attacks have been in the news recently, with several high-profile websites being victims. Credential Stuffing and Brute Force attacks rely on attempting lots of logins to identify valid credentials. Along with lots of attempts come lots of failures. Attackers will rely on software like openbullet to carry out their attacks, using proxy networks to constantly rotate IP addresses and defeat traditional rate limiting.

The program the attacker is using will present a consistent TLS fingerprint. We can make a special rule for our login form that tracks failed login attempts by TLS Fingerprint, effectively tracking the attacker as they rotate IP address.

If the attack is low and slow, we can track failed attempts over a longer timeframe by using the response from the server when adding to our counting zone.

Final Thoughts

Advanced rate limiting is a practical response to the limits of IP-based controls. IP address rotation is the standard amongst attackers and scrapers, rendering the traditional approach obsolete. Useful protection now needs to identify the actor behind the requests, protect the origin before expensive application work is triggered, and give teams enough evidence to adjust the policy without guesswork. Counting requests against a combination of network fingerprints, request fields, response signals, and bot context is how you stop abuse from scrapers, SEO spiders, and layer 7 attackers without treating every visitor the same.

HTTP Security Headers

2023-11-28T14:00:00+11:00

Traditionally, web security has focused on the server side: protecting the application itself from attack. That work is necessary, but it often leaves the client side under-specified. Client-side attacks move the exposure point into the user's browser, where the business impact can be serious.

Magecart attacks are a clear example. Attackers inject skimming scripts into websites to steal sensitive customer information, such as credit card details, directly from the user's browser. Session hijacking and Cross-Site Scripting (XSS) attacks also exploit browser vulnerabilities, leading to unauthorised access and data breaches. These attacks don't just risk user data; they can erode trust, damage reputations, and result in significant financial and legal repercussions for businesses.

HTTP security headers are practical controls for these types of attacks. Properly implemented, they instruct browsers on how to handle website content and interactions safely.

Key HTTP Security Headers

Content-Security-Policy (CSP)

Purpose: CSP prevents Cross-Site Scripting (XSS) attacks by specifying which sources browsers should allow when loading scripts, images, and other resources. It can also prevent MageCart-style attacks by restricting the host names that an injected script can communicate with.

Content-Security-Policy: script-src 'self' https://apis.google.com;

This example allows scripts to load only from the site's own domain ('self') and https://apis.google.com.

X-Frame-Options

Purpose: This header protects against clickjacking attacks by controlling whether a browser allows a page to be rendered in a <frame>, <iframe>, <embed>, or <object>.

X-Frame-Options: DENY

This setting prevents any domain from framing the content. Another option is SAMEORIGIN, which only allows framing by the same site.

X-Content-Type-Options

Purpose: This header prevents MIME-sniffing, where a browser might incorrectly interpret the content type of a resource, leading to security vulnerabilities.

X-Content-Type-Options: nosniff

This instructs the browser to follow the content type declared in the HTTP headers.

X-XSS-Protection

Purpose: This enables the browser's inbuilt XSS protection features. However, this header is largely deprecated in favour of CSP.

X-XSS-Protection: 1; mode=block

This configuration enables the protection and tells the browser to block the page if an XSS attack is detected.

Strict-Transport-Security (HSTS)

Purpose: HSTS forces the browser to use HTTPS over HTTP, ensuring encrypted communication and protecting against man-in-the-middle attacks. Alternatively, you can automatically redirect all requests to HTTPS on your web server or at your EDGE provider. For example, Peakhour allows you to set up EDGE redirects to force all traffic to HTTPS.

Strict-Transport-Security: max-age=31536000; includeSubDomains

This example tells the browser to use HTTPS for all subdomains for one year.

Conclusion

Implementing the correct HTTP security headers is a straightforward way to improve web application security. These headers form part of the first line of defence against many common security vulnerabilities. As threats evolve, keeping security headers current and properly configured helps safeguard your users and your brand.

RFC 9460

2023-11-16T00:00:00+11:00

RFC 9460 introduces two DNS record types: "SVCB" (Service Binding) and "HTTPS". They let browsers learn more connection details during DNS lookup, before redirects and TLS negotiation add extra steps. The result is cleaner connection setup, with practical improvements in speed, security, and efficiency.

Understanding the Current Process

Traditionally, when a browser connects to a website, it follows a sequence:

Browser requests site via HTTP.
Server redirects request to HTTPS.
Browser receives ALPN (Application-Layer Protocol Negotiation) during the HTTPS handshake.

The model is secure, but it is not optimal. It involves multiple round trips, which affects Time to First Byte (TTFB) and the overall user experience. Load balancing and failover are also less direct than they could be. RFC 9460 changes this by allowing DNS to provide the necessary connection details earlier. That reduces the steps involved in establishing a secure connection, lowering TTFB.

The Impact of SVCB and HTTPS Records

SVCB and HTTPS records move useful connection hints into DNS. They speed up the time-to-first-packet by incorporating the Alt-Svc HTTP header and ALPN TLS extension into DNS, which shortens connection setup. These records also enable redirection at the zone apex, a task not possible with CNAMEs. They simplify DNS load distribution and failover, making web services more resilient. They also remove the need for HSTS preloading and support Encrypted Client Hello (ECH), formerly ESNI, for better privacy.

Adoption and Industry Response

Adoption started before the RFC was finalised. Firefox has been conducting HTTPS lookups since May 2020, limited to DNS over HTTPS (DoH). Apple's iOS, Safari, and macOS have followed suit since September 2020. Chrome introduced partial support in December 2020 and has recently enabled ECH by default. Various DNS service providers have also started supporting HTTPS and SVCB records.

As reported on Netmeister, adoption is still early but not insignificant. As of October 2023, about 10 million domains have implemented an HTTPS record for their 'www' service names, roughly 4.4% of domains. Around 9.1 million domains, or about 4.0%, use the record on their bare second-level domain name. Among the top 1 million domains, approximately 22.5K (25.5%) use HTTPS records for 'www' service names, and nearly 24K (25.6%) use them on bare domains.

What the Records Look Like

A typical SVCB record might look like this:

_example.com. 7200 IN SVCB 1 svc4.example.net. (alpn="h2,h3" port="8004")

This record indicates that the service at _example.com can be accessed at svc4.example.net using either HTTP/2 or HTTP/3 on port 8004.

An HTTPS record could be:

example.com. 3600 IN HTTPS 0 svc.example.net.

This record suggests that example.com should be accessed securely through svc.example.net.

Apex Domains and the Importance of SVCB/HTTPS Records

One long-running DNS limitation is the inability to use CNAME records at the apex (root level) of a domain due to conflicts with other necessary records like NS and SOA. RFC 9460's SVCB/HTTPS records address this by enabling apex domain aliasing without those conflicts. This matters for efficient content delivery networks (CDNs) and load balancing strategies.

These records enhance capability

1. Load Balancing:

Consider a website that needs to distribute traffic across multiple servers. SVCB records can indicate different server endpoints with varying priorities.

example.com. IN SVCB 10 server1.example.com. (alpn="h2,h3")
example.com. IN SVCB 20 server2.example.com. (alpn="h2")

In this example, server1.example.com is the preferred endpoint (lower priority number), offering both HTTP/2 and HTTP/3 protocols. If it's unavailable, traffic automatically shifts to server2.example.com.

2. Failover Mechanism:

For a service that requires high availability, SVCB records can express failover directly:

critical-service.example.com. IN SVCB 1 primary-service.example.com. (alpn="h2,h3")
critical-service.example.com. IN SVCB 2 backup-service.example.com. (alpn="h2")

Here, primary-service.example.com is the primary endpoint. If it fails, the system automatically falls back to backup-service.example.com.

3. Apex Domain Usage:

A practical advantage of SVCB/HTTPS records is their ability to handle apex domains, where CNAME records are not feasible. This is important for root domain aliasing to different service providers.

example.com. IN HTTPS 0 cdn-provider.example.net.

This record indicates that the apex domain example.com is to be served through cdn-provider.example.net, overcoming traditional DNS limitations.

4. Encrypted ClientHello Support:

Future enhancements of SVCB could include keys for Encrypted ClientHello, which improves privacy and security during the initial TLS handshake.

secure.example.com. IN SVCB 1 tls-service.example.net. (ech="base64-encoded-key")

This record can be used to initiate a TLS connection with tls-service.example.net using the provided Encrypted ClientHello key.

5. Directing Traffic to Specific Protocols:

For services that need to direct clients to newer or more efficient protocols, SVCB records can specify the exact protocols to use.

api.example.com. IN SVCB 1 api-server.example.com. (alpn="h3")

Clients that understand HTTP/3 can connect directly using this protocol, bypassing the usual HTTP/1.1 or HTTP/2 protocols.

The Long Wait

HTTPS has been around for a while, so RFC 9460 raises an obvious question: why did this take so long? Apex records have had their share of problems, including not being able to use CNAMEs and having to resort to custom records like ALIAS or Cloudflare's 'cname flattening'.

It is a fair question. We've had some bizarre records hanging around for ages along with a wide range of solutions to the "CNAME at the zone apex" problem.

Credit to the creators of RFC 9460 for getting this through and obtaining browser support: - B. Schwartz from Meta Platforms, Inc. - M. Bishop from Akamai Technologies - E. Nygren from Akamai Technologies

Final Thoughts

RFC 9460 gives DNS a more useful role in HTTPS connection setup. SVCB and HTTPS records let operators publish endpoint, protocol, failover, and privacy information before the browser starts negotiating the connection. That gives service providers more precise control over how clients reach web services, with practical benefits for performance, reliability, and security.

A Risk Based Approach To Vulnerability Scoring

2023-11-10T00:00:00+11:00

The Exploit Prediction Scoring System (EPSS) estimates the likelihood that a published CVE will be exploited in the wild. Its value is that it brings several signals into one risk score, instead of treating every vulnerability with the same CVSS severity as equally urgent. The main inputs are:

Data Sources of EPSS

MITRE’s CVE List: EPSS scores only vulnerabilities that are "published" on this list.
Text-based “Tags”: Extracted from CVE descriptions and related discussions.
Publication Duration: The time period since the CVE was published.
Reference Count: The number of references in the CVE entry.
Published Exploit Code: Code from platforms such as Metasploit, ExploitDB, or GitHub.
Security Scanners: Data from security tools such as Jaeles and Nuclei.
CVSS v3 Vectors: Based on the base score in the National Vulnerability Database (NVD).
CPE (vendor) Information: Details about the vendors of the products involved, also from NVD.
Ground Truth Data: Real-world exploitation data from sources such as AlienVault.

EPSS Model and Tools

The current EPSS model, version 2022.01.01, uses 1,164 variables and is based on Gradient Boosting, a machine learning technique. For a visual and interactive view of EPSS scores, the EPSScall tool is useful. It provides historical data and graphs that make score movement easier to inspect.

The Drivers of EPSS Scores

To understand EPSS, it helps to look at which inputs carry the most weight. The variable importance graph shows the strongest contributors to the EPSS score.

Vendor data plays an outsized role in the scoring process. The graph shows how much weight each component has when estimating whether a vulnerability is likely to be exploited.

Why Does This Matter?

EPSS uses these data sources to predict exploit likelihood more directly than severity-only methods. By considering factors from the age of the CVE to real-world exploit instances, EPSS gives defenders a clearer view of which vulnerabilities are more likely to matter operationally. That makes patching and mitigation decisions easier to prioritise when resources are limited.

Understanding the components of EPSS also makes the score easier to interpret. It is not a single severity metric; it is a blend of several data points, each with different weight. Tools like EPSScall make those inputs easier to inspect when tuning a vulnerability management process.

Final Thoughts

EPSS is useful because it shifts vulnerability triage away from severity alone and towards exploit likelihood. Its use of multiple data sources and machine learning makes it a practical tool for defenders who need to decide what to fix first. Prioritising vulnerabilities this way does not replace judgement, but it gives teams a stronger starting point than CVSS alone.

Navigating CDN Consolidation

2023-11-01T00:00:00+11:00

The CDN industry is moving quickly, with major providers such as Akamai and Cloudflare consolidating their positions. For businesses caught in that movement, changing CDN providers is rarely a simple swap. Your CDN sits in front of your website or application, so migration decisions touch performance, security, routing, caching, and operational risk.

Market Shifts in the CDN Industry

The CDN market is being reshaped by large providers and newer entrants. Akamai's acquisition of Linode is one example, expanding its cloud services and strengthening its position beyond CDN. Cloudflare is moving in a similar direction, adding cloud-based services around its CDN platform.

Akamai's Strategic Moves

Akamai has recently bought customer contracts from both Lumen and StackPath. This is likely to lift its 2024 revenue by tens of millions of dollars. The transferred customers will also benefit from Akamai’s wider cloud and security services.

Azure CDN Standard from Akamai, StackPath CDN, and Lumen CDN are all going offline soon. Clients have received only 2-3 months' notice to migrate, which is a tight window for a service that usually has routing, security, caching, and origin dependencies. Vendors should avoid putting customers in this position. A multi-CDN strategy can reduce that exposure.

What Happened to Section.io?

Section.io, once a CDN, shifted to edge computing before being sold to Webscale. That leaves approximately 300 Australian websites looking for new service providers. If you are one of them, now is the time to act.

These moves make the decision to switch or stay with a CDN provider more complex, especially for smaller businesses that need flexible and reliable local alternatives such as Peakhour. Switching your CDN is not as straightforward as changing a DNS record. Your CDN acts as the gateway to your website or application, so a move can involve reconfiguring a large part of the delivery stack.

Why Peakhour Is the Right Choice

Peakhour is a local, reliable alternative in an industry changing quickly. We offer the flexibility needed for customisation and a full suite of services.

If you are considering a CDN switch, treat it as a technical migration rather than a procurement task. Peakhour can help make that transition smoother.

Peakhour's Top 10 Things to Consider When Changing Providers

Switching CDNs? Work through these ten factors before you move:

Caching Rules: Use the migration to review and optimise your caching settings.
POP Distribution: Understand how the new CDN's points of presence may affect your traffic.
Security Gaps: Evaluate how the new CDN's security measures compare to your current provider.
Access Lists: Make sure IP whitelists and blacklists are carried over cleanly.
Origin Security: Update IP addresses to ensure your origin server recognises the new CDN.
SSL/TLS Certificates: Confirm the new CDN supports your existing SSL/TLS settings and can carry over the certificates you need.
API Compatibility: Ensure the new CDN offers APIs that match or exceed your current usage.
Analytics and Monitoring: Assess if the new CDN's analytics tools meet your needs.
Rate Limiting: Review the new CDN's rate limiting options, especially if your site experiences traffic bursts.
Costs: Account for migration work, potential downtime, and any hidden fees.

Additional Considerations for a Seamless Transition

Beyond the top ten, also consider:

Bot Protection: Evaluate how the new CDN manages automated traffic.
User Agent Validation: Make sure the new CDN effectively screens search engine bots.
IP Reputation Lists: Know how your new CDN updates and uses IP reputation lists.
API Protection: Confirm that the new CDN provides strong API security controls.
Dynamic Page Caching: Check how the new CDN handles caching for dynamic content.
Query String Handling: Understand how your new CDN treats query strings, as this can affect cache performance after migration.

Special Concerns for E-commerce Sites

For e-commerce, also think about:

Site Integrations: Does the new CDN support plugins for your platform, such as Magento?
Custom WAF Rules and Exceptions: Ensure these can be moved to the new CDN.
Image Optimisation: Update Image APIs if your CDN handles image transformations.

Advanced Configurations

Advanced setups need closer review:

Origin Mounting: Confirm your multiple origins will work as needed with the new CDN.
Request Routing: Make sure you can replicate your existing routing configurations with the new provider.
Edge Redirects: Ensure the new CDN can handle any redirects you’ve configured at the edge.

Google Chrome's "IP Protection" and Online Privacy

2023-10-24T13:00:00+11:00

Google plans to introduce an "IP Protection" feature in Chrome. The feature is intended to improve privacy by masking IP addresses through proxy servers. It may also affect ad tracking and who controls access to online platforms.

Understanding IP Addresses and Google's Strategy

IP addresses can let websites follow user activity across platforms. Over time, that can build detailed profiles and create real privacy concerns. Google's "IP Protection" is designed to reduce that signal by sending third-party traffic through proxies, hiding user IPs. The feature will start as optional, then focus on domains thought to track users.

At first, Google will use a dedicated proxy for its own domains. As testing continues, the system may change. Google is also considering a 2-hop proxy system for better privacy, with an outside CDN handling the second proxy.

Google wants to use proxy connection IPs to give users broad locations, not exact ones. It will test this on platforms like Gmail and AdServices, in Chrome versions 119 to 225.

VPN Growth and Other Browsers

The growth of VPN use points to demand for online privacy. VPNs, like Google's IP Protection, hide user IP addresses. Firefox and Opera have added VPN features to their browsers. Apple, known for user privacy, has worked with CDN companies on similar privacy improvements.

This change has trade-offs. Sending traffic through Google's, or others', servers can make it harder for security teams to handle threats. Google has suggested fixes like checking users with the proxy and rate-limiting to tackle these problems.

What It Means

Traditional safety tools like IP reputation and GeoIP methods are becoming less reliable. This change highlights the role of network-based fingerprinting now. For more on this, read our article on TLS fingerprinting.

While firms talk about hiding IP addresses, ad tracking is still common. These changes might also push users to certain platforms. Even if users think they're safe, big tech's tracking tools can still watch them. That can give users a false sense of safety. Real privacy still needs practical tools and clear public understanding.

ModSecurity’s End-of-Life

2023-10-16T13:00:00+11:00

The end-of-life of ModSecurity on 1 July 2024 marks a practical turning point for application security teams. For DevOps, SRE, and DevSecOps professionals, it reinforces a wider shift towards Application Security Platforms that go beyond traditional Web Application Firewall (WAF) capabilities.

Modern Application Security Platforms use Web Application and API Protection (WAAP) as a core part of edge security. Peakhour's Application Security Platform extends traditional WAF protection with bot management, API security, DDoS mitigation, and account protection, backed by Peakhour Edge delivery infrastructure.

The bedrock of a WAF lies in two main components:

WAF Engine: Inspects and assesses web traffic.
WAF Rules: Guidelines that tell the engine what to inspect and how to respond.

Peakhour's Application Security Platform has used ModSecurity as part of our WAAP solution, integrating it with threat detection, behavioural analysis, and the proven OWASP ModSecurity Core Rule Set (CRS) for application protection.

For two decades, ModSecurity has been a fixture in web security. Its acquisition by Trustwave led to a sunset announcement in 2021, with the EOL set for July 2024.

Deciphering the EOL for ModSecurity

With the EOL, Trustwave will cease commercial support and updates for ModSecurity. That does not make ModSecurity irrelevant. It has been in 'maintenance mode', with Trustwave channelling its efforts towards bug fixes and security patches.

Despite this change, ModSecurity still has active community support. Tutorials and discussions centred around ModSecurity and CRS continue to appear each month. Entities like Atomicorp have pledged to extend their support to ModSecurity beyond its EOL, helping maintain its presence in the market.

Other WAF engines are emerging as potential contenders. The Coraza WAF engine, written in Go, is gaining a place in the market. The public Azure repository hosts Microsoft's ModSecurity fork, while the Edg.IO repository highlights Waflz, showing its role in the WAF ecosystem.

Recent players, such as OpenAppSec by Checkpoint, are also entering the scene. Positioned as an open-source ML-based WAF, OpenAppSec has publicly advised businesses to start their migration strategies and views itself as a viable migration path.

Peakhour's Application Security Platform Evolution

The ModSecurity transition fits with Peakhour's move towards a broader Application Security Platform. Our approach covers:

Immediate Continuity

Operational Continuity: ModSecurity continues to function within our platform, supported by active community development
No Service Interruption: Customers experience no service interruption as we implement next-generation capabilities
Tighter Integration: Existing ModSecurity capabilities are strengthened through integration with our threat detection systems

Advanced Platform Development

Peakhour is implementing security technologies that extend beyond traditional WAF capabilities:

Machine Learning Integration: AI-powered threat detection that adapts to emerging attack patterns
Behavioural Analysis: Algorithms that identify sophisticated threats including residential proxy attacks and anti-detect browser usage
API-Native Security: Protection designed for modern API-first architectures
Real-Time Threat Intelligence: Dynamic rule updates based on global threat landscape analysis

Future-Ready Architecture

Our Application Security Platform roadmap includes:

Multi-Engine Approach: Evaluation of next-generation engines including Coraza, Waflz, and custom ML-based solutions
Request-Path Protection: Security processing at Peakhour Edge locations for performance
DevSecOps Integration: API-first architecture enabling integration with CI/CD pipelines and security automation
Comprehensive WAAP: Integration of WAF, API protection, bot management, and DDoS mitigation in a unified platform

The Future of Application Security

ModSecurity's end-of-life is more than a technical transition. It reflects the move from traditional point solutions to broader Application Security Platforms. For DevOps, SRE, and DevSecOps teams, this shift enables:

Enhanced Security Posture

Unified Threat Protection: Comprehensive WAAP capabilities that protect applications, APIs, and users through a single platform
Advanced Threat Detection: Machine learning and behavioural analysis that identify sophisticated attack vectors
Real-Time Adaptation: Dynamic security policies that evolve with the threat landscape

Operational Excellence

Performance Integration: Security processing at the edge provides protection without compromising application performance
DevSecOps Compatibility: API-first architecture supports security automation and CI/CD integration
Global Scalability: Distributed protection that scales with application growth and user distribution

Strategic Advantages

Long-Term Investment: Platform approach that evolves with emerging threats and technologies
Comprehensive Coverage: Single-pane-of-glass management for application security, performance, and availability
Compliance Alignment: Built-in reporting and monitoring capabilities that support regulatory requirements

The transition from ModSecurity gives organisations a clear point to review and modernise their application security posture. By adopting Application Security Platforms, teams can improve protection whilst maintaining the performance and scalability required for modern applications.

Peakhour's Application Security Platform protects web applications and APIs with WAAP capabilities, delivery performance, bot management, and real-time threat intelligence. Contact our security team to learn how we can support your application security posture whilst maintaining performance.

HTTP/2 Rapid Reset Attack Deepdive

2023-10-12T00:00:00+11:00

Distributed Denial of Service (DDoS) attack vectors keep changing. The recent spike in HTTP/2-based DDoS attacks has been notable for its volume, with some attacks surpassing 398 million requests per second. Peakhour observed these attacks and worked through how to understand and mitigate them. This article explains how they work and what operators can do to reduce exposure.

The Rise of HTTP/2 in DDoS Attacks

HTTP/2 was designed to make web traffic more efficient. The same features that improve performance for legitimate users can also be abused in DDoS traffic.

Much of HTTP/2's efficiency lies in "stream multiplexing." It allows multiple messages to be sent over a single TCP connection. While HTTP/1.1 processes each request serially, HTTP/2 can manage multiple concurrent streams on a single connection. This means a client can send multiple requests in a single round trip, increasing how much work each connection can drive.

The 'Rapid Reset' Attack Explained

The "Rapid Reset" attack is a specific DDoS technique built around HTTP/2. The attacker starts by opening multiple streams, much like in a standard HTTP/2 attack. However, instead of waiting for responses, they cancel each request immediately.

The client does this by sending a RST_STREAM frame, indicating that a previous stream should be cancelled. The rapid request-and-reset sequence means the server spends resources processing the request, only for it to be cancelled before a response is generated. This tactic amplifies the server's workload without the attacker needing to wait for responses, which increases the pressure each connection can place on the server.

Variants of the Rapid Reset Attack

Attackers also used variations of the Rapid Reset attack:

One variant involves delaying the reset action. The attacker opens multiple streams, waits, then cancels the streams and instantly opens new ones. This method can evade some rate-based defences.
Another variant avoids stream cancellations. Instead, the attacker tries to open more streams than the server allows. This aims to keep the server continually busy, processing a near-constant flow of requests.

Effective Mitigation Techniques

Mitigation is not as simple as blocking individual malicious requests. A more effective approach is to close the entire TCP connection when malicious activity is detected. The HTTP/2 protocol supports connection termination through the GOAWAY frame. This feature needs to be used aggressively to prevent Rapid Reset attacks, rather than relying on the more passive, standard implementation.

Deciding which connections to treat as malicious is a challenge. One potential strategy is to monitor connection statistics. If a connection exceeds a set threshold of cancelled requests, it might be deemed malicious. Responses to suspect activity could range from sending a GOAWAY frame to terminating the TCP connection.

For the non-cancelling variant, the best approach is to shut down connections that breach the concurrent stream limit, either immediately or after a few violations.

Broader Protocol Implications

These attack techniques are specific to HTTP/2, but the wider protocol lesson still matters. The HTTP/3 (QUIC) protocol isn't directly vulnerable in the same way. As a precaution, server implementations should consider limiting the work done by a single connection.

The Importance of Industry Collaboration

When the threat of the Rapid Reset attack became apparent, the industry collaborated to address the issue. The vulnerability was disclosed to key HTTP/2 implementers, helping to devise and distribute effective countermeasures. The vulnerability is logged against CVE-2023-44487.

HTTP/2 'Rapid Reset' DDoS attacks pose a serious risk to services using the protocol. To reduce exposure, service providers should promptly apply available software patches and updates.

Learn how Peakhour's Application Security Platform helps protect against Layer 7 DDoS attacks, including the HTTP/2 Rapid Reset vulnerability. Contact our team to secure your infrastructure.

Understanding the HTTP/2 Rapid Reset Attack

2023-10-11T00:00:00+11:00

The discovery of the HTTP/2 Rapid Reset flaw exposed a serious weakness in a widely used version of the HTTP protocol. When exploited, it can be used to generate large Distributed Denial of Service (DDoS) attacks against HTTP/2 services. This post explains how the attack works and what operators can do to strengthen their defences.

A Deep Dive into the HTTP/2 Rapid Reset Flaw

HTTP/2 is widely deployed, so a flaw in how implementations handle rapid stream resets can have a large operational impact. To take advantage of the issue, a malicious actor sends a request and immediately cancels it, then repeats that pattern over the same HTTP/2 connection. By scaling this "request, cancel" behaviour thousands of times, an attacker can overwhelm vulnerable HTTP/2 implementations. The result is DDoS attacks at the application layer, with potential downtime and disruption.

Major companies including Cloudflare and Google have dealt with this issue. Google, for example, mitigated a DDoS attack reaching a peak of 398 million requests per second that relied on this technique. For scale, this two-minute-long attack generated more requests than the total number of article views reported by Wikipedia in September 2023.

Mitigating the Threat

Large infrastructure providers have led much of the work to understand the attack mechanics and develop mitigations:

Patching Systems: Prompt patching is the primary control for the HTTP/2 Rapid Reset attack. Companies including Peakhour, Microsoft, and others have tested and patched their systems against this threat.
Rate Limiting: Advanced rate limiting has been a recommended action. It provides an extra layer of protection, minimising the risk of massive request inflows.
Collaborative Efforts: Google and Microsoft have both shared intelligence and collaborated with other cloud providers and software maintainers implementing the HTTP/2 protocol stack. This has resulted in patches and mitigation techniques now employed by numerous large infrastructure providers.

What's Next for Users and Enterprises?

If you serve an HTTP-based workload online, understand whether this attack affects your environment. Verify that servers supporting HTTP/2 are either not vulnerable or have applied the necessary patches. Stay informed and consider reaching out to your service providers or account representatives for configuration assistance and guidance.

The HTTP/2 Rapid Reset flaw is a serious application-layer DDoS risk, but it is manageable with the right mitigations in place. Apply the recommended patches and keep HTTP/2-facing services under active review.

Discover how Peakhour's Application Security Platform protects against Layer 7 DDoS attacks, including the HTTP/2 Rapid Reset vulnerability. Contact our team to secure your infrastructure.

ZDNS - scan the entire internet

2023-06-20T13:00:00+10:00

The lack of a free Reverse DNS (rDNS) lookup database has made large-scale DNS research harder. To address this, we used ZDNS, an open-source, high-performance DNS toolkit developed by Stanford University, to create our own rDNS database. To reduce UDP timeout issues during rDNS operations, we devised a scan-ordering approach that randomised the IP space and improved the efficiency of the scanning process.

Leveraging ZDNS for rDNS Lookups Across the Internet

Understanding rDNS is useful for internet operations and research. Active DNS measurement helps us inspect how providers advertise the use of their IP address space. One of the components of this ecosystem is Reverse DNS (rDNS), which serves an important role in IP database categorisation and ASN (Autonomous System Number) classification. However, running rDNS across the entire internet is not a trivial task.

Previously, Rapid7 provided a free database for rDNS lookups, but it has discontinued the offering. This situation has prompted the need to create our own database, calling for a robust, efficient, and scalable tool to accomplish the task. ZDNS was the right fit.

Introducing ZDNS

ZDNS, a part of the ZMap.io project, is a capable tool developed by Stanford University to support scalable and reproducible DNS research. ZDNS is an open-source DNS measurement framework specifically optimised for large-scale DNS research on the public internet. It can resolve 50 million domains in 10 minutes and query the PTR records of the complete public IPv4 address space in approximately 12 hours.

This high-performance toolkit offers a modular interface, enabling researchers to safely implement new functionalities. Its architecture is designed to expose DNS lookup chains by performing recursive resolution. ZDNS supports a command-line interface and outputs results in JSON, a machine-parsable format.

Enhancements by ZDNS

ZDNS's architecture and feature set are tailored to the challenges of extensive DNS research. Its guiding principles are that the DNS lookup chain is exposed, and that the tool is safe, easy to use, and extensible.

ZDNS's performance optimisations make it a suitable tool for DNS experiments that require querying a large number of names. Parallelism, UDP socket reuse, and selective caching are some of the critical performance optimisations that enable ZDNS to efficiently handle large volumes of DNS queries.

ZDNS's scalability, execution time, and success rate have been evaluated against several existing tools, showcasing its performance. For instance, when it comes to exposing the DNS lookup chain, ZDNS is 85 times faster than Dig. ZDNS also outperforms other higher-performance tools, achieving 2.6 to 3.6 times more successful queries per second and experiencing about 30% less packet drop than MassDNS.

Our rDNS Journey

When we started scanning the whole internet with rDNS, we hit a practical roadblock: UDP timeouts made the scans slow. The system spent too much time waiting for responses from parts of the internet that were either empty or broken.

We used two changes. Firstly, instead of scanning the internet's addresses in order, we mixed them up and scanned randomly. This spread out our requests and stopped the system from getting stuck on troublesome ranges. Secondly, we checked smaller sections of the internet first, so we did not waste time waiting for big chunks of the internet that weren't responding.

With these changes, we scanned the whole internet in 13 days, finding over a billion addresses. The main lesson was straightforward: scan order matters when timeout behaviour dominates runtime.

Wrapping Up

ZDNS has proven to be a valuable tool for DNS research, especially for substantial tasks like performing a reverse DNS scan of the entire internet. Our experience underscores the value of practical adjustments when dealing with large-scale challenges, like randomising the IP space to avoid delays caused by UDP timeouts.

As an open-source tool, ZDNS is available on Github. For more detail, read the award-winning paper presented at IMC 2022.

Our work with ZDNS shows its value in DNS research and the operational detail involved in large-scale DNS work. By randomising the scan order, we mitigated timeout issues and improved the efficiency of our scanning process.

Izhikevich, L., Akiwate, G., Berger, B., Drakontaidis, S., Ascheman, A., Pearce, P., Adrian, D., & Durumeric, Z. (2022). ZDNS: a fast DNS toolkit for internet measurement. In Proceedings of the 22nd ACM Internet Measurement Conference (pp. 33-43). https://doi.org/10.1145/3517745.3561434 ↩
ZMap Project. (n.d.). ZDNS. GitHub. Retrieved 2023-05-15 13:00, from https://github.com/zmap/zdns. ↩

Enterprise DDoS Protection

2023-06-19T00:00:00+10:00

Cyber threats continue to grow in complexity and volume, and Layer 7 attacks remain especially difficult to defend against¹. Each layer presents its own set of vulnerabilities for threat actors to exploit. The 7th layer, or application layer, handles application-specific communications. That makes it a useful target because modern applications are complex and varied.

Defending against Layer 7 attacks requires continuous tuning and adaptation¹. Microsoft highlighted the issue in June 2023, when it reported a traffic surge that temporarily affected the availability of some of its services¹.

Microsoft's Layer 7 DDoS Attacks

Microsoft's security team detected and tracked DDoS activity from a threat actor it called Storm-1359. The actor used a mix of resources, including multiple virtual private servers (VPS), rented cloud infrastructure, open proxies, and DDoS tools¹.

The activity did not target layers 3 or 4. It targeted layer 7, where requests can look like regular traffic and arrive from source IPs distributed around the world.

The Attack Methods

Storm-1359 used several attack types, including:

HTTP(S) Flood Attack: The attacker aimed to exhaust system resources with a high load of SSL/TLS handshakes and HTTP(S) request processing. This attack led the application backend to run out of compute resources such as CPU and memory¹.
Cache Bypass: The attacker attempted to overload the origin servers by bypassing the CDN layer¹.
Slowloris: In this case, the client opens a connection to a web server, requests a resource, such as an image, but fails to acknowledge the download or accepts it slowly. This causes the web server to keep the connection open and hold the requested resource in memory¹. Strengthening Layer 7 Protections

Microsoft mitigated the majority of disruptions by hardening its Layer 7 protections. It fine-tuned Azure Web Application Firewall (WAF) to better defend customers from the impact of similar DDoS attacks¹.

Azure Web Application Firewall, ModSecurity, and DDoS Attacks

Azure Web Application Firewall (WAF), part of Microsoft's security architecture, is built upon ModSecurity⁴, a well-established open-source Web Application Firewall (WAF) module¹. The DDoS attack Microsoft faced highlighted potential limitations in using ModSecurity, or any conventional WAF, as the primary defence mechanism against such threats.

ModSecurity's Limitations in DDoS Defence

ModSecurity is effective against a variety of web application threats, but it has limitations when dealing with DDoS attacks:

Lack of Scalability: ModSecurity is not inherently scalable. It can struggle to handle the enormous traffic volume associated with DDoS attacks.
Delayed Response: ModSecurity's rule-based approach can result in slower response times to evolving DDoS threats. While it can block threats based on established rules, it can take time to identify and create rules for new or uncommon attack patterns.
Operational Complexity: ModSecurity requires substantial expertise and constant fine-tuning to remain effective, potentially slowing down response times during a fast-paced DDoS attack.

These limitations were visible during the DDoS attack Microsoft experienced. Even though Microsoft utilised ModSecurity via Azure WAF, the time it took for Azure to respond underlines the challenge of using traditional WAFs for this class of attack¹.

The Role of Residential Proxy Networks in Layer 7 DDoS Attacks

Residential proxy networks create a specific problem in the defence against Layer 7 DDoS attacks³. These networks use IP addresses tied to physical locations, often originating from typical home or office internet connections. That makes it harder to separate legitimate traffic from malicious traffic.

Unlike traditional proxy or VPN networks, where traffic can be blocked or rate-limited based on their recognisable IP ranges, residential proxy networks blend in with legitimate users. That complicates identifying and blocking malicious requests, as any blocking or limiting measures could affect legitimate traffic from residential IPs.

A Potential Solution

In this context, fingerprinting can help distinguish between legitimate clients and malicious actors. Fingerprinting involves gathering data points from each client request, including user agent, IP address, headers, cookies, and more. The combination of these data points creates a unique 'fingerprint' for each client.

By analysing these fingerprints, it is possible to detect anomalous request patterns and potentially identify malicious clients hidden behind residential IPs. Fingerprinting can improve the accuracy of identifying malicious traffic, but it is not foolproof and should sit inside a broader, layered defence strategy.

Implementing effective fingerprinting also requires substantial technical expertise and resources. The measures need to avoid degrading user experience or breaching privacy regulations.

The Need for Specialised Rate Limiting Services

A specialised rate limiting service could have offered a faster and more effective response to the DDoS attack. Rate limiting restricts the number of requests that an IP address can make within a specific time period².

Such a service offers several advantages when defending against DDoS attacks:

Rapid Response: Rate limiting can provide a quick initial defence against a DDoS attack by immediately limiting traffic from suspicious IP addresses.
Flexibility: Rate limiting rules can be applied to factors such as IP addresses, URL, headers, response codes, and more, creating more granular defence mechanisms.
Reduced Load: By limiting the rate of requests, these services can reduce the load on the server, preserving resources for legitimate traffic.

Advanced Rate Limiting and Custom Keys

One way to defend against these attacks is through advanced rate limiting². Rate limiting restricts the number of requests an IP address, URL, or another custom key can make in a set time period. This can stop a single actor from flooding a network with traffic.

Criteria Used in Rate Limiting

Rate limits can be defined using different criteria:

IP Address
URL
Query String
Headers
Response Codes
GeoIP Information: ASN or Country Code
Parsed User Agent Information: Different rules for search engines vs. generic 'bots'
Fingerprints: TCP, TLS or H2 fingerprints can uniquely identify the connecting software
Meta Information: From bot protection service²

This allows rate limiting to 'bucket' requests using different criteria, effectively rate limiting a larger group of connections.

The Role of Anomaly Detection

Anomaly detection is another useful tool against these attacks. It identifies patterns or events that deviate from the norm and may indicate suspicious activity. Detecting those anomalies quickly can help teams respond faster, identify a suitable rate limit key and stop the potential attack.

Caching as a Mitigation Strategy

Caching is an effective mitigation strategy for Layer 7 attacks. It stores static responses to requests, reducing load on the server by serving those responses instead of processing each request individually. In a DDoS scenario, where a flood of requests is sent to the server, caching can help maintain availability. Ignoring client-provided 'Cache Control' headers such as 'max-age=0' or 'no-cache' can be effective because these headers are typically used to bypass a CDN.

Recommendations for Defence Against Layer 7 Attacks

Use anomaly detection to identify an active attack.
Use Layer 7 protection services, including rate limiting, with past 99th percentile hit rates as a starting point.
Apply bot mitigation techniques, as most Layer 7 attacks originate from bots.
Use IP reputation as an early warning sign, as many IPs have been involved in attacks before.
Block, limit, or redirect traffic from outside a defined geographic region.
Rate limit or block requests from data centre and hosting ASNs.
Create custom WAF rules to automatically block and rate limit HTTP or HTTPS attacks with known signatures.
Use effective CDN caching and ignore client-presented Cache-Control headers.

Defending against Layer 7 attacks requires several controls working together. Rate limiting, anomaly detection, and effective caching all have a role.

Peakhour's advanced rate limiting and DDoS mitigation strategies help protect applications from sophisticated Layer 7 attacks. Contact our team to strengthen your defences.

The Rise of the Dragon

2023-05-17T13:00:00+10:00

Camaro Dragon, a Chinese state-sponsored group, has developed a custom firmware implant for TP-Link routers. Once installed, it can turn compromised routers into residential proxies. That weakens traditional cyber-defences, including GeoIP blocking, because traffic can appear to come from ordinary local connections. This article looks at how the malware works, why residential proxies matter for enterprise security, and where GeoIP security measures fall short.

Understanding the New Malware

Check Point's research describes Camaro Dragon's sophisticated attacks on European foreign affairs entities. The group uses a custom firmware implant, known as 'Horse Shell', designed specifically for TP-Link routers. The malware includes a backdoor that grants the attackers continuous access to compromised networks and allows them to build anonymous infrastructure.

'Horse Shell' can execute arbitrary commands on the infected router, transfer files, and relay communications using SOCKS tunnelling. Its design can be adapted to different vendors' firmware, suggesting the possibility of a wider spread.

The People and Intentions Behind The Malware

Investigations into the origin of the 'Horse Shell' malware by Check Point Research, Avast, and ESET point to a well-known cyber threat actor: Mustang Panda. This advanced persistent threat (APT) group, linked to the Chinese government, is known for complex attacks that often exploit Internet-facing network devices.

The primary function of 'Horse Shell' is to relay traffic between an infected device and the attackers' command and control servers. This method obscures the true source and destination of the communication, making it difficult to trace back to the attackers.

Importantly, Mustang Panda appears to choose router implant targets indiscriminately. The infection of a home router doesn't imply that the homeowner is a direct target. Instead, each infected router becomes a node in a broader chain that connects main infections with command and control operations.

Researchers identified this approach when they found the 'Horse Shell' implant during an investigation of targeted attacks against European foreign affairs entities. The implant allows the attackers to maintain ongoing access, establish anonymous infrastructure, and move laterally within compromised networks.

The Implications of Residential Proxies

Residential proxies serve as intermediaries, using real IP addresses issued by Internet Service Providers (ISPs). They are used across a range of applications, including business web scraping and anonymising user online activity.

Residential proxies become more serious when malware such as 'Horse Shell' is involved. This malware infects routers, turning them into a network of residential proxies that can then be used for malicious activity, including data breaches and distributed denial-of-service (DDoS) attacks.

Most importantly, this use of residential IP space can make an attack look as if it originates from a domestic source within the target's location. That undermines traditional cyber-defences.

GeoIP Security Measures and Their Limitations

GeoIP blocking, a traditional cyber security tool, works by limiting access from specific geographical regions or networks frequently associated with cyber threats. However, this method is becoming less effective against the rising use of residential proxies.

Residential proxies can disguise the actual origin of a cyber attack, giving the illusion that it's originating from a trusted, usually local, location. This capability allows them to effectively bypass GeoIP blocking measures. Consequently, malicious actors using residential proxies can carry out their activities with less obvious attribution and often go undetected.

The key operational issue is the exploitation of home routers by malware like 'Horse Shell,' which turns these devices into unwitting participants in cyber attacks. This manipulation means an attack could appear to originate from a seemingly trusted domestic source, which can render GeoIP blocking ineffective.

This threat shows why cyber security needs a more layered approach. Sole reliance on GeoIP blocking is no longer enough. As malware evolves to exploit residential proxies, detection and defence strategies need to adapt. Specifically, it's important to recognise that relying solely on GeoIP blocking, or trusting apparently local connections and deny-listing countries like Russia and China, can create a false sense of security.

Detecting Residential Proxies: The Role of Network Fingerprinting

The rise of residential proxy malware makes network fingerprinting important for identifying these threats. Five techniques can help detect residential proxies:

TCP Fingerprinting: Proxied requests may generate TCP fingerprints that don't match the expected device type. For example, a request from a residential IP address that bears the fingerprint of a server OS could be a strong signal of a proxy.
TLS and HTTP/2 Signatures: As with TCP fingerprints, unusual TLS and HTTP/2 signatures could reveal proxies. An incoming request using a version of TLS or HTTP/2 not commonly used in residential networks might indicate a proxy.
JavaScript-based Fingerprinting: This method identifies the specific browser in use. Discrepancies in JavaScript fingerprints, or the absence of a fingerprint, could suggest the presence of a residential proxy.
Timing Analysis: The timing of requests can also be a signal. Proxied requests might exhibit longer or inconsistent intervals between requests, indicating a residential proxy.
Port Scanning: This technique can detect open ports that could indicate the presence of SOCKS or other proxies, revealing possible exposure to threats.

While residential proxies have legitimate uses, such as web scraping, those applications sit beside a more serious risk: compromised trusted or local networks can be turned into proxy infrastructure at scale. Cyber threats like 'Horse Shell' use residential proxies to undermine traditional GeoIP security measures, which means defence strategies need to keep evolving.

In Part 1 of our series on residential proxies, we provide an overview of this topic and why it matters to security teams. From basic uses to their role in complicated cyber attacks, we cover the key points.

Learn how Peakhour's Application Security Platform protects against account takeovers and credential stuffing. Contact our team to secure your user accounts.

Cohen, I., Madej, R., & Threat Intelligence Team (2023). The Dragon Who Sold His Camaro: Analyzing Custom Router Implant. Check Point Research. Retrieved from https://research.checkpoint.com/2023/the-dragon-who-sold-his-camaro-analyzing-custom-router-implant/ ↩
Goodin, D. (2023, May 17). Malware turns home routers into proxies for Chinese state-sponsored hackers. Ars Technica. Retrieved from https://arstechnica.com/information-technology/2023/05/malware-turns-home-routers-into-proxies-for-chinese-state-sponsored-hackers/ ↩

Residential Proxies and MITRE Framework

2023-05-17T13:00:00+10:00

Residential proxies act as intermediaries, routing traffic through real-world IP addresses. That can mask user identity, bypass geographical restrictions, and improve privacy. The MITRE ATT&CK framework, a matrix of cyber adversary tactics and techniques, categorises proxy use under technique T1090. The classification helps explain how attackers use proxies to maintain command and control across target environments, including Linux, Windows, and macOS.

Residential proxies are useful and risky in equal measure. They support anonymity and data collection, but misuse creates ethical and security concerns, including credential stuffing and account takeovers. MITRE ATT&CK's treatment of proxy use gives security teams a clearer way to reason about those risks and plan mitigations.

Looking at residential proxies through the MITRE framework keeps the discussion grounded. It shows where proxy use fits into attacker tradecraft, and where defenders need practical controls rather than broad assumptions about intent.

From Credential Stuffing to Account Takeover and Data Exfiltration

Credential stuffing and account takeover incidents, including the Ubiquiti breach, show how exposed digital defences can be. Attackers use residential proxies to mask activity, which aligns with MITRE ATT&CK technique T1090. This technique describes proxy use for discreet command and control. In the Ubiquiti case, adversaries utilised proxies to test and apply stolen credentials across systems without revealing their true locations, a direct application of T1090's principles.

The Camaro Dragon malware demonstrates residential proxy exploitation for account takeovers. By infecting devices and incorporating them into a botnet, the malware facilitated remote control over victims' accounts, aligning with MITRE's T1090 for proxy-managed network communications. Camaro Dragon's operation reflects the tactic of maintaining anonymity while executing unauthorised access and control, a strategy documented within the MITRE framework.

Volt Typhoon's activities present a sophisticated use of residential proxies in data exfiltration. This group, known for targeting infrastructure, manipulated proxies to move data discreetly from compromised networks, a tactic that falls under MITRE's T1090. The operation shows how adversaries use residential proxies to obscure the digital footprint of data theft, complicating traceability and detection.

Viewed through MITRE ATT&CK, these examples show how residential proxies support credential stuffing, account takeovers, and data exfiltration. They also point to the need for integrated defence strategies that account for different forms of proxy misuse, rather than treating proxy traffic as a single problem.

The Role of Residential Proxies in Web Scraping

Residential proxies are common in web scraping because they let operators simulate requests from different geographic locations. That capability is especially useful when gathering data from websites with GeoIP restrictions or anti-scraping measures. In the MITRE ATT&CK framework, residential proxy use in web scraping aligns with several techniques that describe how adversaries gather information and evade detection.

Technique T1090, which details proxy use, illustrates how adversaries utilise residential proxies to disguise web scraping activity. By routing requests through proxies, they can avoid IP bans and rate limits, enabling the collection of large amounts of data without detection. This technique shows the practical advantage of residential proxies in bypassing network defences and aggregating targeted information discreetly.

Web scraping through residential proxies also intersects with the MITRE framework's emphasis on reconnaissance techniques. Adversaries use reconnaissance to gather valuable data about targets, and residential proxies help them do it discreetly. By presenting requests as coming from different residential IPs, attackers can compile detailed profiles on organisations, their operations, and vulnerabilities without revealing their intent or location.

For defenders, residential proxy use in web scraping creates a dual challenge. It can support legitimate data collection and market research, and it can also help adversaries gather intelligence before further attacks. That makes proxy handling a judgement problem as well as a blocking problem: organisations need to balance access to information with protection against unauthorised data extraction.

Understanding residential proxy use in web scraping through MITRE ATT&CK helps define the detection problem more precisely. Defenders need mechanisms that can distinguish legitimate proxy-backed activity from malicious use, and policies that can respond without over-blocking normal traffic.

Defending Against Proxy-Related Cyber Attacks Informed by MITRE

Defending against cyber attacks that use residential proxies requires layered controls informed by MITRE ATT&CK. Technique T1090, which focuses on proxy use for command and control activity, provides a useful base for designing those defences.

Network Monitoring and Analysis

A core defence is stronger network monitoring and analysis. By scrutinising network traffic, organisations can identify unusual patterns that may indicate malicious proxy use. This includes monitoring for excessive requests from varied geographic locations that do not align with normal user behaviour. The MITRE framework suggests network intrusion detection systems (NIDS) to detect suspicious activity, including anomalous residential proxy use.

Implementing Access Controls and Rate Limiting

To mitigate credential stuffing and account takeover through proxies, organisations need strict access controls and rate limiting. These measures can reduce automated attacks by limiting how many requests a user can make within a set timeframe, weakening distributed attempts to breach systems via residential proxies.

Application of Web Application Firewalls (WAFs)

Web Application Firewalls (WAFs) help defend against proxy-related attacks. When configured to recognise and block requests with patterns typical of proxy misuse, such as rapid request rates or known malicious IP addresses, WAFs provide a barrier against unauthorised data scraping and other proxy-facilitated intrusions.

Proxy Detection and Blocking

Advanced proxy detection tools help organisations identify and block traffic coming through known residential proxies. Techniques include analysing originating IP addresses for known proxies and using behaviour analysis to detect patterns indicative of proxy use. Once identified, these IP addresses can be blocked or subjected to additional scrutiny.

User Behavior Analytics (UBA)

User Behavior Analytics (UBA) helps detect anomalies that may signal a proxy-based attack. By establishing baselines of normal user activity, UBA systems can flag deviations that suggest malicious activity, such as multiple failed login attempts or unusual data access patterns, which are indicative of credential stuffing or data exfiltration attempts.

Educating Users on Security Hygiene

Educating users on security hygiene can help prevent inadvertent participation in malicious proxy networks. Users should understand the risks of downloading unverified software or browser extensions, which could turn their devices into nodes within a residential proxy network.

Informed by MITRE ATT&CK, these defence strategies give organisations a practical way to reduce exposure. Understanding the tactics and techniques used by adversaries helps teams strengthen controls against sophisticated residential proxy use in cyber attacks.

Detecting Malicious Use of Residential Proxies

Detecting malicious residential proxy use requires both technical controls and threat intelligence. The MITRE ATT&CK framework, particularly technique T1090, gives defenders a reference point for how adversaries use proxies and where detection mechanisms should focus.

Traffic Pattern Analysis

One primary method for detecting malicious residential proxy use is traffic pattern analysis. This includes monitoring for spikes in traffic from geographical locations that do not match the service's typical user profile. Anomalies in request rates or patterns that suggest automation, such as regular intervals between requests, can also indicate proxy abuse.

Behavioral Anomaly Detection

Behavioural anomaly detection systems identify actions that deviate from normal activity. These systems can flag unusual behaviour that might indicate malicious residential proxy use, such as repeated login attempts from different IP addresses in a short period, which could signify a credential stuffing attack.

IP Reputation and Proxy Lists

Utilising IP reputation databases and known proxy lists can help identify and block requests from suspicious sources quickly. These lists include IP addresses known to be part of residential proxy networks or previously implicated in malicious activity. Integrating this intelligence into security systems allows for real-time blocking or flagging of potentially harmful traffic.

Endpoint Detection and Response (EDR) Systems

Endpoint Detection and Response (EDR) systems help spot compromised devices within an organisation that could unknowingly be part of a residential proxy network. By monitoring endpoints for signs of malware or unexpected network traffic, organisations can detect and isolate infected devices before they are used in cyber attacks.

Advanced Machine Learning Models

Advanced machine learning models can be trained to recognise subtle signs of proxy misuse. By analysing large datasets of network traffic, these models can identify patterns that human analysts might miss. This includes detecting sophisticated attempts to mimic legitimate user behaviour through proxies, which could indicate reconnaissance or data exfiltration efforts.

Collaboration and Information Sharing

Collaboration and information sharing among organisations and cybersecurity entities can improve detection of malicious proxy use. Sharing indicators of compromise (IoCs) and tactics, techniques, and procedures (TTPs) associated with proxy misuse can help develop stronger detection strategies across the board.

Incorporating these detection methods, informed by MITRE ATT&CK, helps organisations identify and mitigate risks associated with malicious residential proxy use. The goal is not to label every proxy request as hostile, but to detect the patterns that matter when residential proxies are exploited for cyber attacks.

Residential Proxy Detection

2023-05-17T13:00:00+10:00

Residential proxies are under increasing scrutiny, both for how their IP addresses are obtained and for how those networks are used. They also expose how heavily many online services rely on GeoIP data, from content customisation to security controls.

That scrutiny reveals a complicated reality. Residential proxies can help businesses, researchers, and individuals preserve anonymity or work around GeoIP-dependent restrictions. The same properties also create ethical problems, particularly when the networks are misused.

This article explains what residential proxies are, how they work, where they are useful, and where the risk sits. The same properties that make them attractive for legitimate monitoring and research also make them useful for abuse.

Demystifying Residential Proxies

These proxies connect automated software to the internet through IP addresses tied to real-world residential locations. That lets the software look closer to ordinary internet usage, which can help it bypass geographical and network restrictions while adding a layer of anonymity.

Residential proxies need a clear legal and ethical distinction. Their use can be lawful, including for web scraping and data gathering, while still enabling activity that may breach the intended usage policies of some online services. This could include mass consumption of data intended for general use, such as scraping websites for machine learning datasets. These actions may not be strictly illegal, but they raise substantial ethical questions and are often unwelcome to the data providers.

Applications of Residential Proxies

The defining characteristic of residential proxies is that requests can appear to originate from local residential networks. That supports a wide range of use cases, including:

Concealing True IP Addresses: Residential proxies allow third parties to hide genuine IP addresses and location, making identity and origin harder to determine. By routing internet traffic through residential IP addresses, they can evade detection, bypass security rules, and access geo-restricted content.
Research and Monitoring: Residential proxies are often used by researchers, analysts, and market intelligence professionals to gather data and monitor online activity. By utilising residential IP addresses, they can emulate real user IP addresses and bypass restrictions.
Web Scraping and Data Gathering: Residential proxies are central to many web scraping and large-scale data collection workflows. With the capacity to rotate IP addresses and access a wide range of residential locations, third parties can scrape valuable data from websites without triggering anti-scraping measures. Residential proxies can make data scraping more discreet, with fewer access interruptions and cleaner collection results.
Ad Verification: Residential proxies are widely used for ad verification. Ad verification companies utilise residential IP addresses to confirm the accuracy and legitimacy of online advertisements. By mimicking genuine residential connections, they can check that ads are correctly displayed and monitor the performance and integrity of advertising campaigns.
Ad Fraud: Residential proxies can also be misused for ad fraud. Competitors or their agents may utilise residential IP addresses to falsely inflate the views of a rival's online advertisements. By using genuine residential connections, these entities can manipulate advertising metrics, compromising the accuracy and integrity of the ad's performance data. This abuse of residential proxies for ad fraud poses a significant concern for the online advertising industry.
Last Mile Monitoring: Last mile monitoring is another application for residential proxies, allowing companies to assess the user experience from a residential viewpoint. By using residential IP addresses, they can monitor website loading speeds, test service availability, and evaluate the performance of online platforms more accurately. This helps organisations pinpoint and rectify issues that may negatively affect user satisfaction.

Navigating the Risks and Concerns

Residential proxies create material risks, particularly when users are unaware that they are hosting one. Their use can introduce practical limits and security vulnerabilities that are easy to miss.

Despite their valid uses, residential proxies can be used for cybercriminal activity. Malicious actors may exploit them for account takeovers, fraud, or other targeted attacks.

Using residential proxies without the knowledge or consent of residential users creates serious security issues. These users, unaware of how their connections are being utilised, could face legal exposure, compromised privacy, and cyber threats. Their devices could unwittingly participate in malicious activity, leaving them exposed to legal consequences and reputational damage.

Exploring the Creation of Residential Proxies and their Implications

Residential proxy providers build their networks in several ways, some of which can have significant security implications.

Providers can obtain residential proxies through partnerships with Internet Service Providers (ISPs) or by leasing IP addresses from legitimate residential users. At the same time, some providers or private groups may use questionable practices to obtain residential proxies.

SDKs: Certain applications may include Software Development Kits (SDKs) that gather and sell user data, including their IP addresses. In some instances, these SDKs can be exploited by residential proxy providers to acquire residential IPs without the explicit consent or knowledge of the users.
Malware Exploitation: Malware, including botnets, can infiltrate the devices of unsuspecting residential users. Attackers may then exploit these infected devices as part of a broader residential proxy network, without user awareness. This unauthorised use of residential IPs poses significant security threats to both the affected users and the wider internet ecosystem.
Free VPN Services: Some free Virtual Private Network (VPN) services, which promise anonymity and privacy, may use users' connections as part of their residential proxy networks. Users unknowingly become exit nodes for other users' internet traffic, potentially exposing their connections to malicious activities.

Using residential proxies without the knowledge or consent of residential users raises serious security concerns. These users may not understand how their connections are being used, which can lead to legal consequences, compromised privacy, and exposure to cyber threats. Their devices might unknowingly participate in malicious activities, exposing them to potential legal consequences and reputational damage.

The Birth of 'Ethical' Proxies

An important part of the residential proxy discussion is the rise of providers claiming that their IP address pools are ethically sourced. These companies argue that they have obtained the consent of the original IP owners and provide transparency in how these connections are utilised. By positioning themselves as 'ethical' residential proxy providers, they aim to mitigate the associated risks and concerns.

Even where consent is obtained, the potential for misuse remains a significant issue. This is largely due to the inherent anonymity of residential proxies and the difficulty of tracing activity back to the original user. Despite claims of ethical sourcing, the complexity and opacity of the residential proxy environment mean that it remains a grey area, inviting scepticism and demanding further scrutiny.

The result is a nuanced market that consumers, providers, and regulators need to understand as the digital landscape continues to evolve.

From Hola VPN to the Camaro Dragon

Several publicised incidents show how residential proxies are formed and the impact they have had on the industry and users. These examples show the different ways residential proxies can be created and used, legitimately and otherwise.

Hola VPN is a well-known free VPN service that promises privacy, security, and access to blocked content. However, it fell under scrutiny when it was revealed that it was selling its users' bandwidth to its sister company, Luminati, which operates a residential proxy network. Users of Hola VPN unknowingly became part of a residential proxy network, with their connections being utilised by third parties. This raised significant ethical and security concerns, as users' devices could be implicated in illegal activities carried out using their IP addresses.
The residential proxy service known as 911 has been selling access to hundreds of thousands of Microsoft Windows computers for the past seven years. This service enables customers to route their internet traffic through these computers, allowing them to appear as if they are browsing from any country or city around the world. While 911 claims that its network comprises users who voluntarily install its "free VPN" software, recent research indicates that the proxy service has a history of obtaining installations through questionable "pay-per-install" affiliate marketing schemes, some of which were operated by 911 itself. The service primarily targets users in the United States but has a global user base. Residential proxy networks like 911 can serve legitimate business purposes, but they are often abused for cybercriminal activities due to the difficulty in tracing malicious traffic back to its source.
Cybercriminals are increasingly leveraging residential broadband and wireless data connections to anonymise their malicious traffic. One notable type of network, referred to as "bulletproof residential VPN services", has gained attention. These networks are constructed by acquiring discrete blocks of internet addresses from major internet service providers (ISPs) and mobile data providers. An investigation into one such company, Residential Networking Solutions LLC (also known as Resnet), unveiled that it had obtained a significant number of IP addresses, some of which were previously controlled by AT&T Mobility. Resnet leased these IP addresses, enabling it to resell data services for major providers such as AT&T, Verizon, and Comcast Cable. However, the precise nature of the relationship between Resnet and AT&T remains unclear, and the matter has been referred to law enforcement. Cases like this emphasise the potential abuse of IP addresses within residential proxy networks.
Infatica.io, a Singapore-based company, has developed a network of over 10 million web browsers that clients can rent to conceal their true internet addresses. The company achieved this by compensating browser extension developers to incorporate its code into their extensions. Many extension developers struggle to earn fair compensation for their work, making offers like these enticing. Infatica seeks extensions with at least 50,000 users and offers to pay developers between $15 and $45 per month for every 1,000 active users with the code included in their extensions. Infatica's code routes web traffic through the browsers of extension users, providing anonymity to the company's customers. The service's pricing depends on the volume of web traffic a customer wishes to anonymise. However, this approach raises concerns about privacy and the potential misuse of users' browsers for malicious activities. Developers, particularly those who author free software, can find the monetisation opportunity offered by residential proxies extremely tempting. The potential to earn revenue from their existing user base by incorporating such code into their extensions can present a persuasive proposition.
Camaro Dragon, a form of malware, provides a recent example of residential proxies being acquired through malicious means. This malware infects the devices of unsuspecting users, forming a botnet that can then be utilised as a residential proxy network. Infected devices can then be exploited for various cybercriminal activities without the knowledge or consent of the device owners. This example highlights the significant cybersecurity risks associated with residential proxies and emphasises the importance of robust protection measures.
Volt Typhoon is a state-sponsored actor based in China that typically focuses on espionage and information gathering. Volt Typhoon proxies all its network traffic to its targets through compromised SOHO network edge devices (including routers). Microsoft has confirmed that many of the devices, which include those manufactured by ASUS, Cisco, D-Link, NETGEAR, and Zyxel, allow the owner to expose HTTP or SSH management interfaces to the internet. Volt Typhoon has been active since mid-2021 and has targeted critical infrastructure organisations in Guam and elsewhere in the United States.

These examples illustrate the ethical, security, and legal issues surrounding residential proxies. They put transparency and consent at the centre of how proxy networks are acquired and used. The implications for users, the security industry, and the broader digital landscape are substantial, which is why regulation, user education, and responsible practices matter for protecting privacy, security, and the integrity of the internet.

Legal Consequences of Residential Proxies in Data Scraping Operations

Residential proxies are a concern because of their potential for misuse and their legal implications. Two notable cases, the Ticketmaster Case and the Meta vs Bright Data Case, have drawn attention to the challenges posed by the unauthorised use of residential proxies in commercial settings and data scraping operations. These cases show why the legal ramifications of residential proxy use need to be understood in real-world scenarios.

The Ticketmaster Case: In 2018, a major international case came to light when Ticketmaster sued Prestige Entertainment for using residential proxies to circumvent ticket-purchasing limits and scoop up large numbers of tickets for resale. This case underscores the potential misuse of residential proxies in commercial settings, and how they can be used to breach the terms of service of websites.
The Meta vs Bright Data Case: The legal case between Meta Platforms, Inc. (formerly Facebook) and Bright Data Ltd. demonstrates a contentious and potentially unlawful use of residential proxies in the real world. In this case, Meta accused Bright Data of operating a business designed to use automated software to scrape and sell data from various online platforms, including Facebook and Instagram. This scraping was allegedly facilitated using unauthorised tools and services that bypassed detection by Meta's security measures. Despite Meta's efforts to halt these activities, Bright Data purportedly continued its operations. The data involved included user profiles, follower counts, and shared posts. Bright Data was alleged to not only scrape this information but also advertised the sale of the scraped data. The scope of this operation was extensive, with the Instagram data set alone priced at $860,000.

These cases show how residential proxies are used in practice, the challenges they present, and why their use remains legally and commercially contested.

The Wider Implications for the Security Industry

The growth of residential proxies, and the way some networks are acquired, has broader implications for the security industry. It raises questions about transparency, ethical practices, and the responsibility of proxy providers.

Ethical and Regulatory Implications: The questionable practices some providers use to acquire residential proxies highlight the need for stronger regulation and industry standards. This would help ensure that residential proxies are obtained and used in a lawful and ethical manner, protecting users' privacy and the wider internet ecosystem. There is a clear demand for more transparency in how these services operate and procure their proxies.
Cybersecurity Implications: Residential proxies can enable malicious cyber activity, ranging from fraud to targeted attacks. This can increase the need for cybersecurity measures and protections, potentially reshaping strategies and priorities within the cybersecurity industry.
Legal and Reputational Implications: If individuals unknowingly become part of a proxy network, there could be legal repercussions for them if their connections are utilised for malicious activities. This could lead to greater scrutiny and liability for companies operating within this space.
State Actors and Residential Proxy Networks: State-sponsored actors have been known to establish their own residential proxy networks within foreign countries for various campaigns, including information warfare, disinformation campaigns, and surveillance, adds another layer of complexity to the issue. These activities pose significant geopolitical and security risks, requiring increased international cooperation and robust defence mechanisms.

The rise of residential proxies exposes a weakness in common security models: the assumption that residential and mobile IPs are inherently more trustworthy, and that GeoIP is a reliable reputation or security control. Widespread proxy use has shown how brittle that assumption can be.

Uncertain or unethical sourcing makes that trust problem harder. It can make online interactions less reliable and introduce security risks.

Residential proxies are not just tools; they highlight a deeper issue in how we approach digital access and security. Understanding what is already known, questioning current practices, and building better controls are practical steps towards using residential proxies responsibly and ethically. Recognising the false sense of security GeoIP restrictions can provide is part of that work.

Part 1 ends here. In Part 2: the Camaro Dragon malware, we look more closely at a specific case. This sophisticated malware uses residential proxies in a way that shows their potential for misuse. The next article covers how Camaro Dragon works, its impact on cybersecurity, and practical protection measures.

Mi, X., Tang, S., Li, Z., Liao, X., Qian, F., & Wang, X. (2021). Our Phone is My Proxy: Detecting and Understanding Mobile Proxy Networks. Retrieved from https://xianghang.me/files/ndss21_mobile_proxy.pdf ↩
Mi, X., Feng, X., Liao, X., Liu, B., Wang, X., Qian, F., Li, Z., Alrwais, S., Sun, L., & Liu, Y. (2019). Resident Evil: Understanding Residential IP Proxy as a Dark Service. Retrieved from https://www-users.cse.umn.edu/~fengqian/paper/rpaas_sp19.pdf ↩
Krebs, B. (2019, August 19). The Rise of "Bulletproof" Residential Networks. Retrieved from https://krebsonsecurity.com/2019/08/the-rise-of-bulletproof-residential-networks/ ↩
Krebs, B. (2022, July 18). A Deep Dive Into the Residential Proxy Service '911'. Retrieved from https://krebsonsecurity.com/2022/07/a-deep-dive-into-the-residential-proxy-service-911/ ↩
Krebs, B. (2021, March 1). Is Your Browser Extension a Botnet Backdoor? Retrieved from https://krebsonsecurity.com/2021/03/is-your-browser-extension-a-botnet-backdoor/ ↩
Meta Platforms, Inc. v. Bright Data Ltd. Retrieved from https://unicourt.com/case/pc-db5-meta-platforms-inc-v-bright-data-ltd-1374026 ↩
Volt Typhoon targets US critical infrastructure with living-off-the-land techniques. Retrieved from https://www.microsoft.com/en-us/security/blog/2023/05/24/volt-typhoon-targets-us-critical-infrastructure-with-living-off-the-land-techniques/ ↩

Advanced Anomaly Detection

2023-05-15T13:00:00+10:00

Modern Application Security Platforms need reliable anomaly detection to identify and respond to emerging threats in real-time. For DevOps, SRE, and DevSecOps teams, machine learning algorithms such as Robust Random Cut Forest (RRCF) provide the foundation for automated threat detection and response systems that can operate at the scale and speed contemporary applications require.

Strategic Importance of Anomaly Detection in Application Security

Real-time anomaly detection is a core Application Security Platform capability. It helps identify threats before attacks affect application performance or security posture:

Enterprise Threat Landscape

Modern applications face attack vectors that traditional signature-based detection cannot address:

Adaptive Bot Networks: AI-powered bots that modify behaviour based on defensive responses
Zero-Day Exploits: Previously unknown attack patterns that bypass traditional security rules
Volumetric Attacks: DDoS attacks that scale dynamically to evade rate limiting
Insider Threats: Subtle anomalies in user behaviour that indicate account compromise

Application Security Platform Requirements

Effective anomaly detection needs to integrate cleanly with broader security capabilities:

Real-Time Processing: Threat identification within milliseconds of detection
Scalable Architecture: Analysis of millions of requests without performance degradation
Context Awareness: Integration with application metadata and user behaviour profiles
Automated Response: Immediate threat mitigation through dynamic rule deployment

Advanced Machine Learning for Security

Robust Random Cut Forest provides anomaly detection capabilities designed for streaming data environments common in Application Security Platforms:

Algorithmic Advantages for Security Applications

Streaming Data Processing: Real-time analysis without historical data dependencies
Dimensionality Handling: Effective analysis of high-dimensional security feature vectors
Adaptive Learning: Continuous model updates based on evolving traffic patterns
Computational Efficiency: Linear scaling suitable for high-throughput security processing

Implementation in Application Security Platforms

RRCF enables threat detection across multiple security dimensions:

Traffic Pattern Analysis: Identification of unusual request volumes, frequencies, and distributions
Behavioural Anomalies: Detection of user actions that deviate from established profiles
Network Fingerprinting: Recognition of abnormal connection patterns and protocol usage
Content Analysis: Identification of malicious payloads and injection attempts

RRCF Advantages for Application Security Platforms

Traditional batch-processing anomaly detection systems are a poor fit for Application Security Platforms that must respond to threats in real-time. RRCF's streaming approach provides practical advantages:

Real-Time Threat Detection

Immediate Analysis: Process and analyse security events as they occur, without waiting for batch processing
Adaptive Baselines: Continuously update normal behaviour models based on current traffic patterns
Memory Efficiency: Maintain configurable rolling windows of security data for optimal performance
Scalable Processing: Handle millions of security events per second without degradation

Security-Optimised Implementation

RRCF's forest-based approach is useful for security applications:

Multi-Dimensional Analysis: Analyse request patterns, user behaviour, and network characteristics at the same time
Shape-Sensitive Detection: Identify subtle changes in attack patterns that signature-based systems miss
False Positive Reduction: Leverage ensemble methods to reduce noise in security alerting
Contextual Awareness: Understand normal application behaviour patterns for more accurate threat detection

Application Security Platform Integration

Enterprise Deployment Architecture

Peakhour's Application Security Platform implements RRCF through high-performance Rust-based processing:

Edge Processing Capabilities

Global Deployment: RRCF analysis deployed across CDN edge locations for minimal latency
Distributed Learning: Aggregated threat intelligence from multiple geographic regions
Local Response: Immediate threat mitigation at the edge without central processing delays
Bandwidth Optimisation: Process security events locally to reduce data transmission requirements

Platform Integration Benefits

Unified Threat Detection: RRCF analysis integrated with WAF/WAAP, bot management, and DDoS protection
Automated Response: Dynamic security rule generation based on anomaly detection results
DevSecOps Workflow: API-first architecture enabling integration with security automation tools
Compliance Reporting: Detailed anomaly detection logs for security audits and regulatory requirements

Advanced Security Use Cases

Credential Stuffing Detection

Behavioural Analysis: Identify unusual login patterns that indicate automated credential testing
Geographic Anomalies: Detect impossible travel scenarios and location-based attack patterns
Volume Analysis: Recognise subtle increases in authentication attempts that indicate coordinated attacks
Success Rate Monitoring: Identify campaigns through abnormal authentication success/failure ratios

API Threat Detection

Endpoint Anomalies: Detect unusual API usage patterns that indicate reconnaissance or exploitation
Rate Pattern Analysis: Identify sophisticated rate limiting evasion techniques
Response Time Analysis: Detect performance impacts from malicious API usage
Authentication Anomalies: Recognise token abuse and API key misuse patterns

Zero-Day Threat Identification

Traffic Pattern Deviations: Identify new attack vectors through unusual request characteristics
Response Pattern Analysis: Detect exploitation attempts through server response anomalies
Protocol Anomalies: Recognise malformed requests that indicate exploit attempts
Payload Analysis: Identify suspicious content patterns in request bodies and parameters

Operational Excellence Through Advanced Anomaly Detection

Performance and Security Integration

RRCF implementation delivers measurable improvements across security and performance metrics:

Threat Detection Speed: Sub-millisecond anomaly identification for real-time response
False Positive Reduction: Ensemble methods reduce security alert fatigue
System Performance: Efficient processing maintains CDN performance whilst enhancing security
Adaptive Learning: Continuous improvement in threat detection accuracy over time

DevSecOps Enablement

Modern Application Security Platforms provide APIs and automation capabilities:

Security Automation: Programmatic access to anomaly detection results for automated response
CI/CD Integration: Security testing and validation integrated into development workflows
Monitoring Integration: SIEM and SOC platform integration for security operations
Custom Rule Development: Framework for developing application-specific anomaly detection rules

Final Thoughts

Advanced anomaly detection through RRCF is a fundamental capability for modern Application Security Platforms. By implementing machine learning algorithms at the edge, organisations can achieve real-time threat detection that adapts to evolving attack patterns whilst maintaining application performance.

The integration of RRCF with security capabilities including WAAP, bot management, and DDoS protection creates a unified platform that addresses the security requirements of contemporary applications and APIs. For DevSecOps teams, this approach enables automated threat response whilst providing the visibility and control needed for effective security operations.

Double MAD?

2023-05-15T13:00:00+10:00

This article explores the use of Double Median Absolute Deviation (Double MAD) for [anomaly detection](/learning/threat-detection/what-is-anomaly-detection/) in time series
data, particularly in skewed or non-symmetric distributions. Double MAD, which calculates two median absolute
deviations — one for data below the median and one for data above — provides a more nuanced approach than traditional
MAD, allowing for accurate detection of anomalies even in skewed data distributions. We also delve into its application
in identifying slow abuse, like bots, by catching lower range anomalies. However, it's important to note Double MAD's
limitations such as not capturing seasonal data shape and trends over time. A comparison is also drawn with the Z-score
method, highlighting that the choice between the two depends on the nature of your data. The article provides insights
into the practical implementation of Double MAD and its potential to improve your data analysis toolkit.

Operational systems increasingly rely on time-series data for decisions. Anomaly detection is one practical use: by identifying patterns that deviate from the norm, businesses can investigate potential issues early or understand unexpected opportunities.

One useful technique for anomaly detection is the Median Absolute Deviation (MAD) and, more specifically, its extension, the Double MAD. This article explains where Double MAD fits in time-series anomaly detection and how it can help identify anomalous clients.

Understanding MAD and Double MAD

MAD, a robust measure of variability, is less susceptible to outliers than standard deviation. It calculates the median of absolute deviations from the data's median, often providing a better representation of 'normal' behaviour in datasets with skewed distributions or outliers.

Double MAD is an extension of MAD, where two MADs are calculated — one for the data below the median and another for the data above. This split gives the detection process a better fit for asymmetric data, which is common in real-world time series data.

Why Double MAD?

While MAD provides a robust way to understand the 'normal' range of a dataset, it assumes a symmetric distribution of data around the median, which may not always hold true. Double MAD is useful where that assumption breaks down, offering an improved anomaly detection process for skewed or asymmetric datasets.

In time-series analysis, especially with 24-hour cycles like web traffic or server usage, patterns can exhibit seasonality and trend components. These patterns can often be asymmetric, making Double MAD a valuable tool for capturing the variability in different parts of the data.

Using Double MAD in Anomaly Detection

The Double MAD implementation provided uses Rust, a system programming language known for speed and memory safety. The code calculates the lower and upper MAD values, along with their respective thresholds. Anomalies can then be detected by comparing each data point to these thresholds.

An anomaly is defined as a data point that deviates significantly from the expected range. If a data point falls below the lower MAD threshold or above the upper one, it can be flagged as an anomaly. This approach is especially effective when handling datasets with high variability or extreme values.

Double MAD for Anomalous Client Detection

Beyond time-series data, Double MAD can also be instrumental in identifying anomalous behaviour among clients. By comparing each client's behaviour against the Double MAD of the time-series data, teams can pinpoint clients that deviate from the norm.

For instance, in the context of web service usage, an anomalous client might be one that is sending an unusually high or low number of requests. By using Double MAD, you can flag such outliers and take appropriate action, such as investigating potential misuse or reaching out to understand and address any issues they may be facing.

Detecting Lower-Range Anomalies: A Case of Slow Abuse

An interesting application of Double MAD is in detecting lower-range anomalies, a pattern often associated with slow abuse such as bots or Distributed Denial of Service (DDoS) attacks. These abuses are characterised by an unusually low frequency of activity that is consistent over a prolonged period. This consistent, low-level activity can fly under the radar of typical anomaly detection systems.

By setting a lower MAD threshold, Double MAD can effectively detect these lower-range anomalies, providing early warning of slow abuse. Its ability to detect both high and low anomalies makes Double MAD a flexible tool for anomaly detection.

The Math Behind Double MAD

To illustrate the power of Double MAD, let's consider a dataset from a right-skewed distribution. Applying the conventional MAD approach might lead to false positives where normal data points are marked as outliers. This is because MAD uses a symmetric interval around the median, which doesn't account for the skewed nature of our data.

With Double MAD, we instead calculate two MADs — one for the data below the median (MAD-lower) and another for the data above (MAD-upper). Outlier thresholds are then defined using these two MADs. The lower threshold is calculated as the median minus a multiplier (k) times MAD-lower. The upper threshold is the median plus k times MAD-upper.

This approach takes into account the asymmetric nature of our data, providing more accurate anomaly detection. For example, in a right-skewed distribution, Double MAD would correctly identify only the extreme right tail values as outliers without incorrectly flagging data points on the left tail.

Wrapping Up

Accurate anomaly detection matters when teams rely on time-series data to operate and investigate systems. The Double MAD approach provides a robust method for this, allowing businesses to better understand their data, spot potential issues early, and make more informed decisions.

Whether you're monitoring web traffic, server usage, or client behaviour, leveraging Double MAD can offer valuable insights and help ensure your operations continue to run smoothly. The ability to detect both high and low anomalies makes it especially powerful, providing protection against potential threats like slow abuse.

Understanding and implementing Double MAD gives your data analysis toolkit a more complete view of asymmetric data and helps you detect potential anomalies earlier.

Efficiently Generating and Printing All IPv4 Addresses in a Random Order

2023-05-15T13:00:00+10:00

In this article, we explored an efficient way to generate and print all possible IPv4 addresses in a random order using
a Linear Congruential Generator (LCG). The LCG, a pseudorandom number generator, helps generate the full range of IP
addresses without consuming vast amounts of memory, making this approach suitable for systems with memory constraints.
We also provided a Python script demonstrating the concept, along with a test case to verify its correctness.

We then delved into the importance of randomising IP addresses, highlighting its critical role in areas like security
testing, load balancing, enhancing privacy, and web scraping. However, while using this technique, it's essential to
respect privacy and legality, as misuse can lead to legal repercussions.

In summary, the ability to generate and print all IPv4 addresses in a random order is a powerful tool, especially in the
realm of networking and cybersecurity, and can be achieved efficiently using the LCG approach.

In networking, some tasks require generating and printing every possible IPv4 address. Doing that in random order without a large memory footprint is less straightforward. The IPv4 address space contains 2^32, or 4,294,967,296, values. Storing all of them in memory at once is not feasible for most systems.

This article uses a Linear Congruential Generator (LCG) to generate the full range without holding it in memory.

Linear Congruential Generator

A Linear Congruential Generator is a type of pseudorandom number generator that can run without storing the whole sequence. It generates each next value from a linear equation based on the previous value. The basic form of the LCG is:

X_(n+1) = (a*X_n + c) mod m

Here, a, c, and m are constants, and X_n is the nth number in the sequence. The initial seed or starting point of the sequence is X_0.

If we choose parameters such that the period of the LCG is maximum (equal to the modulus), and the modulus equals the range of numbers we're generating (the number of possible IPv4 addresses in this case), then the LCG should generate each number in the range exactly once before repeating.

Here is that idea in Python:

import ipaddress

def lcg(modulus, a, c, seed):
    """Linear congruential generator."""
    while True:
        seed = (a * seed + c) % modulus
        yield seed

start_ip_str = '0.0.0.0'
end_ip_str = '255.255.255.255'

start_ip = int(ipaddress.IPv4Address(start_ip_str))
end_ip = int(ipaddress.IPv4Address(end_ip_str))

modulus = end_ip - start_ip + 1
a = 1664525
c = 1013904223
seed = 1  # Arbitrary seed

generator = lcg(modulus, a, c, seed)

for _ in range(modulus):
    ip_int = start_ip + next(generator)
    ip = ipaddress.IPv4Address(ip_int)
    print(ip)

The script first defines the parameters of the LCG. a, c, and seed are set to values used in Numerical Recipes' LCG, a well-known and widely used LCG. The modulus is set to the total number of possible IPv4 addresses.

The function lcg() is implemented as a Python generator, yielding the next number in the sequence each time it is called.

The loop then generates and prints each IP address. It adds the output of the LCG to the start IP address, converts it back to an IP address string, and prints it.

This script generates and prints each IP address in random (more precisely, pseudorandom) order using very little memory. Each IP address is printed exactly once, assuming the period of the LCG is maximum.

The point is that a small pseudorandom number generator can walk a large range without materialising the whole list. The code can still be tweaked and optimised for specific requirements and constraints.

The Importance of Randomising IP Addresses

Randomising IP addresses has practical uses in several networking workflows:

1. Security Testing and Penetration Testing

In cybersecurity, randomising IP addresses can help simulate attacks on a network from various sources. By using a range of IP addresses in no particular order, penetration testers can mimic the unpredictable nature of real-world cyber threats and build more robust test scenarios.

2. Load Balancing and Network Traffic Simulation

Randomising IP addresses is also useful in network traffic simulations. Network engineers and administrators can use this approach to test network resilience and capacity. By sending requests to servers from randomised IP addresses, they can evaluate how well their load balancing strategies are functioning and whether the network can handle high traffic loads from various sources.

3. Anonymity and Privacy

In some cases, randomising IP addresses can help with privacy and anonymity. While it is not a foolproof method, using a different IP address for each request can make it more challenging for online trackers to monitor user activity. It is a common practice among privacy-focused internet users and is also used in some VPN (Virtual Private Network) services.

4. Web Scraping

Web scraping is another area where randomising IP addresses is useful. To prevent being blocked by anti-bot measures, web scrapers often need to rotate their IP addresses. By using a different IP address for each request, they can avoid being detected and blocked by the sites they are scraping.

Randomising IP addresses can be useful in these cases, but privacy and legality still matter. Unauthorised network scanning, privacy breaches, and cyberattacks are illegal and punishable under law.

Generating and printing every possible IPv4 address in a random order is a valuable technique with various applications, from network testing to privacy enhancement. With the Linear Congruential Generator approach, we can do it efficiently.

Layer 7 DDoS Protection

2023-02-07T13:00:00+11:00

In previous blog posts we've covered the benefits that full page caching has on page load performance. We also covered how caching pages lowers origin server utilisation, so the site can handle more customers. A lesser-known side benefit is protection against Layer 7 denial of service (DoS) attacks.

What is a Denial of Service attack?

The goal of a DoS attack is to make a network resource, such as a website or networked service, unavailable to users by overwhelming it with excessive traffic or requests. DoS attacks can be launched from a single source or from multiple sources (known as a distributed denial of service, or DDoS attack).

Typically, an attacker floods the web server with HTTP or API requests to try to overwhelm it. An attacker might also launch a 'slow attack', especially if they find a weak point in the application that consumes a lot of server resources for a single request. One example is repeatedly using a site search function. By slowing down the rate at which requests are sent, the attacker can bypass common rate-limiting and traffic-shaping mechanisms that are designed to block high-volume traffic spikes.

DoS conditions can also be inadvertent. A CMS like Magento or WordPress on an underpowered server can be overwhelmed, or slowed to a crawl, by so-called 'grey' bots. Examples include Semrush, Ahrefs, dotbot, and MJ12Bot. These spiders can crawl a site aggressively enough to bring it to its knees. Even Bing or Google can negatively impact a site.

What is Layer 7?

In networking parlance, Layer 7 refers to the actual application running on a web server. It is part of the OSI (Open Systems Interconnection) model used to describe the various functions in transmitting data over a network. The model is as follows:

Layer 1 -> Physical Layer, responsible for transmitting raw bits of data, e.g., a wire or wireless link
Layer 2 -> Data Link, responsible for transmitting data frames between network devices and detecting transmission errors.
Layer 3 -> Network Layer, responsible for routing data packets between networks and determining the best path for transmission.
Layer 4 -> Transport Layer, responsible for end-to-end communication between applications, providing reliable data transmission and flow control.
Layer 5 -> Session Layer, responsible for establishing, maintaining, and terminating communication sessions between applications.
Layer 6 -> Presentation Layer, responsible for formatting data to be presented to the application layer and for converting data from the application layer into a standardised format for transmission.
Layer 7 -> Application Layer, responsible for providing services to the user, such as file transfer, email, and web access.

How Full Page Caching helps

Full page caching helps by reducing the number of dynamic requests to the server. Caching the entire page allows the server to serve pre-generated content to visitors, rather than generating it dynamically each time a page is requested. This reduces the server's processing load and helps prevent excessive requests from overloading the server or making it unavailable.

A real world example

As part of a series of Peakhour recommendations to 'harden' its website against layer 7 attacks, a Peakhour client (a government site) applied limited full page caching with a time to live of 10 minutes. This means that Peakhour will serve a page from its Edge cache for 10 minutes before hitting the origin again for a new version.

Not long after implementation, that change was tested: a DDoS originating from hundreds of IPs across multiple countries hammered a set of 5 pages. The attack was spread over several days with bursts of activity lasting about 15 minutes. With the first hit, the page was cached and every subsequent hit was served from our high-performance Edge cache. No slowdown was observed and the attackers gave up.

Real attack on a Peakhour client; the spikes formed in 15 minute bursts.

Real page load times measured in the client browser

But my site is dynamic...

Many websites have a small dynamic component on the page. For ecommerce sites this might be the mini cart in the top right showing the number of items in the cart, or it might be some personalisation for a user. Often these dynamic parts of the page can be rendered in the browser using Ajax or local storage, rather than rendered on the server. By moving the dynamic components to the browser, the full page becomes cacheable. Another option is to use Edge Side Includes (ESI), which enables the majority of the page to be cached in the CDN while the dynamic parts are fetched separately before serving the full page to the user.

Peakhour can help move dynamic page components from the server to the browser and cache more at the edge. We also have a range of CMS plugins that do just that. Contact us if you want some help.

Final Thoughts

Layer 7 DoS attacks have become more common than traditional DoS attacks, as they typically require far fewer resources from the attacker. Reducing dynamic requests to the origin using full page caching is a useful but underappreciated way to mitigate them. If you're concerned about website security, resilience, and performance, Peakhour helps you cache more so you can protect, accelerate, and scale your website.

TLS Fingerprinting

2023-02-02T13:00:00+11:00

What is Fingerprinting?

Fingerprinting is a technique that may be used to identify the specific device, web browser, and operating system making a request, regardless of what the client says in its user-agent header. By helping organisations identify and characterise the attributes of a client's connection, fingerprinting can improve network security and help protect against malicious traffic.

Fingerprinting can also refer to techniques for following or uniquely identifying individual users across the web. That is a separate set of techniques and is not discussed in this article.

Transport Layer Security (TLS) Fingerprinting determines the specific characteristics of a client's TLS implementation by examining the initial TLS handshake packet, known as the "Client Hello." This packet contains fields and parameters such as supported cipher suites, extensions, and the client's preferred order of those parameters, which can be used to create a unique "fingerprint" of the client's TLS implementation.

Why is it used?

Fingerprinting has several uses, including bot protection, DDoS protection, and client identification. By identifying and characterising the attributes of a client's connection, fingerprinting can improve network security and help protect against malicious traffic.

How does TLS Fingerprinting work?

TLS Fingerprinting examines the initial TLS handshake packet, known as the "Client Hello". The Client Hello packet is sent by the client during the initial phase of the TLS handshake, which establishes a secure connection between the client and the server. It contains information about the client's preferred encryption methods, extensions, and parameters, including:

Protocol Version: The version of the TLS protocol desired by the client.
Random: A 32-byte random value generated by the client, used in key generation and derivation.
Session ID: An optional session identifier for resuming a previous session.
Cipher Suites: A list of supported encryption algorithms, ordered by preference.
Compression Methods: A list of supported compression algorithms, ordered by preference.
Extensions: Optional extensions that can negotiate additional parameters, such as Server Name Indication (SNI) and Elliptic Curve Supported (ECS).

The Client Hello packet is central to the operation and security of the TLS connection because it provides information the server uses to select encryption algorithms and parameters. The packet also enables the client and server to negotiate an appropriate encryption method for their communication. The Client Hello's variable content, based on the TLS version, library, cipher suites, extensions, and settings supported by the client, makes it a strong candidate for fingerprinting.

Common components used to create a TLS fingerprint include:

Cipher Suites: The order of cipher suites supported by the client.
Extensions: Supported extensions included in the Client Hello packet, such as SNI and ECS.
TLS Point Formats: Encoding of cryptographic parameters in a format that can be transmitted as part of the TLS protocol, used in elliptic curve cryptography (ECC).
TLS Curves: The specific elliptic curves used in ECC, a type of public-key cryptography used in the TLS protocol.

TLS fingerprinting has been a topic of research for several years, with a number of tools and techniques developed from that work. Notable examples include JA3, developed by John Althouse, Jeff Atkinson, and Josh Atkins of Salesforce, which uses a hash of the client's SSL/TLS parameters as a unique identifier for tracking and analysing SSL/TLS traffic. Another tool, Mercury by David McGrew and Blake Anderson, can be used to fingerprint client connections and identify the device, operating system, and application making the connection.

TLS fingerprinting has a variety of uses, including bot protection, DDoS protection, malware identification and client identification. By enabling organisations to identify and characterise the attributes of a client's TLS implementation, TLS fingerprinting can improve network security and help protect against malicious traffic.

In production, TLS fingerprints are most useful when combined with IP intelligence and residential proxy detection, rather than treated as a standalone verdict.

Representation of a TLS Fingerprint

A TLS fingerprint is commonly represented as a string or hash that summarises the important components of the Client Hello packet. The most common components used to create a TLS fingerprint include the supported cipher suites, extensions, and TLS point formats. The cipher suites are represented as a list of hexadecimal values in the order they are presented by the client, while extensions and point formats are represented as a list of hexadecimal values or a unique identifier.

Raw JA3 signatures are represented by the following fields, which are then hashed with MD5:

SSLVersion, Cipher, SSLExtension, EllipticCurve, EllipticCurvePointFormat

An example raw signature is:

 771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-34-51-43-13-45-28-21,29-23-24-25-256-257,0

An MD5 hash is then applied, resulting in the final signature.

579ccef312d18482fc42e2b822ca2430

Mercury signatures are represented by:

"tls/1" (TLS_Version) (TLS_Ciphersuite) [ Extension* ]

An example signature is:

tls/1/
(0303)
(130213031301c02cc030009fcca9cca8ccaac02bc02f009ec024c028006bc023c0270067c00ac0140039c009c0130033009d009c003d003c0035002f00ff)
[
   (0000)
   (000a000c000a001d0017001e00190018)
   (000b000403000102)
   (000d0030002e040305030603080708080809080a080b080408050806040105010601030302030301020103020202040205020602)
   (0016)
   (0017)
   (0023)
   (002b0009080304030303020301)
   (002d00020101)
   (0033)
]

Hash Functions for Representing TLS Fingerprints

Hashing algorithms, such as MD5, are commonly used to create a unique representation of a TLS fingerprint. These hash functions take the client's TLS parameters as input and produce a fixed-length output, which serves as a unique identifier for the client. The hash value can be compared against a database of known TLS fingerprints to help determine the identity of the client.

Other techniques for representing TLS fingerprints include base64 encoding of the client's TLS parameters, such as in the Mercury fingerprint.

Challenges with TLS fingerprinting

TLS fingerprinting is not a foolproof method for identifying clients and their attributes. It has several limitations that need to be considered.

False Positives: TLS fingerprinting relies on the assumption that the client's Client Hello packet uniquely identifies a connecting process by its TLS implementation. However, it is possible for a client to alter the Client Hello packet by customising TLS parameters, which affects the Client Hello packet and can result in a false positive identification. This makes it important to use multiple methods for identifying clients. For example, Mercury takes into account destination ports to add additional context.
False Negatives: While TLS fingerprinting can identify many different clients and their attributes, it is not capable of identifying all clients. Some clients may have a unique or unusual TLS implementation that cannot be accurately fingerprinted. Additionally, some clients may actively attempt to evade fingerprinting by customising TLS parameters or using tools to anonymise their connections.
Forging of TLS Fingerprints: It is possible for attackers to deliberately forge or modify the information contained in their Client Hello packet to appear as a different client. This makes it difficult for fingerprinting tools to accurately identify the true identity of a client and can be used for malicious purposes, such as evading security measures or disguising the origin of an attack.
Incomplete Data: TLS fingerprinting is limited by the information contained in the Client Hello packet, which may not contain all of the necessary data to accurately identify a client. For example, a client may not send a full list of supported cipher suites or extensions, may use a modified version of the TLS protocol that is not recognised by the fingerprinting tool, or the fingerprint may not be present in available databases.

Different fingerprinting implementations can result in different hashes for the same TLS connection, even though the underlying SSL/TLS protocol remains unchanged. This happens due to the various algorithms, parameters, and representations used by different fingerprinting tools.

For instance, implementation differences when generating the TLS fingerprint may cause hashes found in public databases to be inconsistent with a locally generated hash.

Final Thoughts

Be aware of the limitations and differences between fingerprinting implementations, and choose the right tool and representation for your specific use case. Standardising the representation of fingerprints and using common hash algorithms can help avoid confusion and improve interoperability between databases.

IP Threat Intelligence

2022-07-15T13:00:00+10:00

Threat intelligence helps organisations make earlier decisions about cyber attacks. One of the most common forms of threat intelligence in cyber security is IP reputation lists. For example, a given IP address might have a poor reputation for spam, ddos attacks, malware, and several other categories. IP reputation lists often form a front line of defence in Web Application Firewalls and cyber security solutions.

How Peakhour uses IP threat intelligence

Peakhour supports threat intelligence across more than 20 categories, including:

Active DDoS attacks
Brute forcing
Active attackers
Computers infected with malware
Anonymous Proxies
Forum Spammers
TOR anonymous users
IPs with poor reputation
Unroutable and unassigned IPs
Robots and web scrapers
Datacenter
Hosting Providers
Crawlers
And more

Customers have access to all 10 lists, which can be enabled as blocklists or used as part of a custom firewall rule, rate limiting rule, or page rule. For example, you may want to disallow POSTs from forum spammers, rate limit proxies, and outright deny traffic from known brute-forcing IPs.

Creating a spammer can't post rule

How does Peakhour assemble these lists?

The IP reputation lists are sourced from third-party sources, including open source intelligence feeds (OSINT), commercial feeds, community feeds, and our own threat intelligence. IPs are categorised into our pre-defined lists and made available to the WAF and rules engine. Each list is re-evaluated and updated based on the data provider's update schedule; some are updated every minute.

Internally managed feeds include bot sources that are verified using reverse DNS lookups, PTR record lookups, and WHOIS verification (such as Facebook IPs). WAF hits across customers are consolidated and made available as the Active Attacker list, which is updated in near real time. Our Malware and C&C nodes lists are generated from various partnerships.

The Anonymous Proxies list contains known open proxies, services that relay traffic without authentication, whilst our targeted VPN list tracks known third-party VPN services.

IPs are fed back into our system for re-evaluation to help identify emerging behaviour within our customer data.

Data visualisation

Requests from IPs that match a blocklist are tagged with the lists they belong to. Firewall events are enriched with this information, providing visibility into security threats. This context helps you decide how to handle requests, whether they should be blocked, rate limited or observed.

Firewall events generated by reputation matches

Blocks generated by our reputation lists can also be viewed in our analytics section.

Firewall events generated by reputation matches

Future work

We are working on additional data sources to further refine and expand our lists. This includes further segregating our data centre lists and categorising IPs that appear on several lists. We are also introducing our threat research centre to discover possible threats and enrich data blocked only by our WAF.

IP threat intelligence adds another layer of security to a cyber defence system. Peakhour sources and maintains up-to-date threat intelligence, helping our clients better protect themselves against would-be attackers.

See how Peakhour's IP threat intelligence supports the first line of defence for your applications. Contact our team to discuss your security requirements.

Origin shield

2022-06-10T13:00:00+10:00

CDN providers often promote the size of their network, and how many Points of Presence (POPs) they have. Higher capacity, more resilient networks are useful from a security point of view (think DDoS attacks), but more POPs can also work against what the CDN was designed to do: take load off an origin and improve performance for end users.

The POP Problem

Modern CDNs are what's called 'Pull' CDNs. That means the CDN won't store content/resources until a user requests it. The first time a user requests a resource, it goes to the CDN POP, checks its local cache, gets a miss, and then passes the request through to origin. As the resource is returned, the CDN stores a copy for the next time someone wants it. If your CDN has 100 POPs, then this process has to be repeated 100 times to fully 'warm' the CDN for that specific resource. That's 100 requests to origin. The more POPs your CDN has, the more likely you are to get a miss and hit the origin.

When the caches at POPs are fully populated, the effect on your application can be minimal. During a cache MISS event, typically either due to resource expiration or a manual purge, many requests can be sent to the origin server concurrently while the individual POPs rebuild their caches. The more POPs, the longer the process takes.

This can be a problem, especially when caching dynamic pages that need to be server side rendered, large resources, or transformed resources. For example, take a busy ecommerce store running Magento during a sale, Magento will purge content when sales are made, forcing the cache to rebuild each time. During a busy period it can reduce your cache hit rate and degrade site performance.

Enter Origin Shield

CDN Origin Shield is a feature that lets you nominate the CDN Point of Presence closest to your server as a shield. All requests that hit other POPs and receive a cache miss will then go to the nominated shield before hitting the origin. The shield becomes a 'super cache' and can reduce the amount of requests to your origin in a cache miss.

Peakhour.IO implements origin shield as a simple dropdown on an origin pool where you can select the geographic location that should be used as a shield. Requests to your origin are now routed through this geographic location before reaching your origin in a cache miss scenario.

Clients who use multiple geographic origins can also benefit from Origin shield. Peakhour.IO allows the specification of an origin shield per origin. For geographic load balancing, you will need to contact support for setup.

Seeing is believing

The Peakhour.IO summary now includes your edge CHR, your shield CHR and your overall CHR so that you can see the effect in action.

Some of our clients have seen typical increases of 10-20% of their overall Cache Hit Rate, and greater than 40% when specifically looking at often flushed dynamic content.
Quicker cache convergence
Fewer hits to origin
Better end-user experience
Higher conversions

Final Thoughts

Origin Shield is an important feature for certain types of site, or when you're looking to maximise your cache hit rate. CMSs that offer built in full page caching, like Magento and Drupal, flush content often, and are susceptible to performance degradation as load increases. Minimising hits to the origin in these cases is vital.

If you are interested in getting more out of your CDN, need a bespoke CDN solution, or need a provider that offers performance, optimisation and security services, reach out to discuss the right setup.

Origin shield with request collapsing helps minimise origin hits, improve CHR and maintain user experience for your web application.

CVE-2022-26134

2022-06-02T00:00:00+10:00

On June 2, 2022, Volexity announced active exploitation of Atlassian Confluence. The issue is a Remote Code Execution vulnerability via OGNL injection, tracked as CVE-2022-26134, and impacts all Confluence Server and Data Center versions greater than 1.3.0.

Atlassian has released its security advisory with patches and mitigation instructions.

Peakhour WAF clients are already protected. Since the vulnerability was announced on June 2nd, we have observed a 200% increase in OGNL-based exploit attempts.

Peakhour's Web Application Firewall helps protect applications against zero-day exploitation attempts such as CVE-2022-26134. Contact our team to secure your applications.

Intelligent Rate Limiting

2022-05-16T13:00:00+10:00

Rate limits protect web applications from clients making excessive requests. Peakhour.IO supports rate limits with flexible controls for selecting which clients are limited and which type of limit applies.

What kinds of attacks are stopped by rate limiting

When an application is protected with rate limiting, the main attack patterns are:

Brute force and enumeration attacks
Denial of Service (DoS) and Distributed Denial of Service (DDoS)
Site scraping
Vulnerability scanners

What else can rate limiting protect

Public APIs and authenticated APIs can be abused or misused. Sensible rate limit policies on these endpoints can reduce attack traffic and help maintain service availability. Rate limiting can protect:

APIs
Overzealous 'good bots'

How does it work?

Rate limiting focuses on a connecting client and their IP address. The following measures can be used to track client requests for rate limiting:

Concurrent connections
Connections per interval
Hits per interval
HTTP 4xx responses per interval
HTTP 5xx responses per interval
Custom criteria

How granular can rate limiting be?

Using wirefilter rules, rate limiting can identify clients from both the HTTP request and response, allowing rate limits to be separated by endpoint or behaviour. For example, the URL /api can be rate limited separately from the /login endpoint. Rate limits can also be set on response codes; for example, the endpoint /search can be protected from scraping by rate limiting clients with excessive 4xx response codes.

What types of criteria can be used to define rate limits?

Rate limits can include any information defined in an HTTP request and response, including:

IP address
URL
Query string
Headers
Response codes
GeoIP information such as ASN or country code
Parsed user agent information allowing different rules for search engines vs generic 'bots'
Metadata we make available from our BOT protection service

Defining your rate limits

Picking sensible rate limits is difficult without adequate analytics on how the web application is typically used. The Peakhour dashboard includes rate-based analytics to help with setup.

Peakhour's Application Security Platform combines high-performance delivery and cache capabilities with security controls for applications and APIs. It maintains caching performance while applying advanced threat protection. Contact our team to discuss how rate limiting can improve application performance and security posture.

Rate limiting

2022-05-16T13:00:00+10:00

Rate limits are a useful control for protecting a web application from abuse. When setting them for a web application, the key elements to consider are:

What endpoints on the web application require protecting?
Do different endpoints require separate handling?
What is the normal request rate for the entire application over a time period?
How many concurrent connections are typically used by your clients?
What errors does your API endpoint return in response to requests?

Before setting those policies, it helps to understand how rate limits protect an application from abuse or misuse, the types of attacks they can reduce, and how the rate limiting algorithm makes decisions.

What kinds of attacks are stopped by rate limiting?

When protecting an application with rate limiting, common attack scenarios include:

Brute force and enumeration attacks
Denial of Service (DoS) and Distributed Denial of Service (DDoS)
Site scraping

What else can rate limiting protect?

Public APIs and authenticated APIs can be subject to both abuse and misuse. Sensible rate limit policies can be applied on these endpoints to help prevent attacks and maintain service availability. Rate limiting can help protect these endpoints.

How does rate limiting work with user logins?

A well designed web application should allow only a limited number of failed login attempts before locking an account and requiring a password reset. This is designed to protect against brute force attacks against an account. Bots commonly attempt to brute force logins to WordPress and other popular web applications. Determined attackers can also attempt to brute force API login endpoints.

Rate limiting on a login page can be applied to the IP address of a user attempting to log in. By rate limiting by IP address, you can limit both password brute force attacks and simpler username enumeration attempts.

Using Peakhour.IO rate limiting, responses to requests can be monitored and IPs blocked for administrator-defined periods. This saves origin server resources and stops repeated attempts before they reach the application.

How could API rate limiting work?

APIs are ubiquitous across the modern web. Single Page Applications (SPAs) can be built almost entirely on REST or GraphQL APIs, while legacy applications often use form submits. Even when browsing this blog, you have consumed a range of APIs.

Because APIs are often publicly available, rate limits are commonly used to reduce abuse. Rate limiting for APIs can protect against malicious attacks. An attacker could script a bot to perform many API calls and make the service unavailable for other users, causing unplanned downtime - a layer 7 DoS or DDoS attack.

APIs

Public and private APIs can be subject to abuse or misuse. Public APIs are discoverable by anyone and can be scripted for data mining or attacks. Rate limiting these endpoints based on fair use policies is commonplace. Keeping track of this within an endpoint can be expensive, so handling it through Peakhour can offload that work from developers.

Overzealous 'good bots'

Peakhour has seen websites where up to 65% of requests come from automated bots. These bots are typically indiscriminate when mining information, and they do not carry the operational cost when your site slows down or fails. Rate limiting good bots separately from your main users helps ensure these crawlers do not stop your site from generating revenue.

How is rate limiting implemented?

Rate limiting is typically implemented using several common methods:

Fixed window

Window-based rate limiting is the simplest to understand. Fixed window limits are easy to define, such as 5,000 requests per 60 minutes. Fixed window rate limiting is subject to spikes at the edges of the window. For example, 5,000 requests in the first 5 minutes of the window may overwhelm a service.

Sliding window

A sliding window keeps much of the simplicity of a fixed window, but uses a rolling window. This allows bursts to be smoothed.

Token bucket

A token bucket is an algorithm where tokens are placed into a fixed-capacity bucket. Tokens could be defined as bytes transferred or hits to an API. When a request is considered for rate limiting, tokens are removed from the bucket. If the bucket has a sufficient quantity of tokens, the request can proceed. If there are insufficient tokens, the request is considered to be non-conforming. Non-conforming requests are dropped.

Leaky bucket

Leaky buckets are a mirror image of token buckets. Instead of removing tokens from a bucket, tokens are added to a bucket. Tokens are removed from the bucket (leaks) at a fixed rate. When a request is considered for rate limiting, it is compared to the number of tokens in the bucket. If the bucket is full, the request is considered non-conforming and is dropped.

If rate limiting is something you need to do to protect and secure your website, reach out to see how we can help.

Learn how Peakhour's Application Security Platform can improve your application's performance and security. Contact our team to get started.

Fastly Outage

2021-06-09T13:00:00+10:00

You may have heard that Fastly, one of the world’s largest providers of CDN services, had an outage of about 1 hour on the 8th July. Some of the world's largest websites and services were down, including reddit, CNN, The Guardian, Shopify Stores, Stripe and Spotify, to name a few.

According to Fastly themselves, the outage was caused by a 'service misconfiguration' (Update: Bug triggered by a client changing their configuration), which propagated globally and took websites offline. When users tried to access a website using the Fastly service, they were presented with a Varnish 503 Guru Meditation error (for those of us old enough to remember, Guru Meditation is a geek reference to the Commodore Amiga computer of the late 80s!). This generally occurs when there is an issue contacting the server that the website is actually hosted on. There were also some reports on twitter saying 'unknown domain'.

Essentially, Fastly took down its own network with a bad software update. Similar problems have affected other online platforms in the recent past, including Google, Amazon, and Cloudflare.

Why wasn’t there a Plan B?

Fastly is an excellent service, with an enviable reliability record. There is a reason why they're trusted by some of the world's largest websites to improve reliability and load times. However, the vast majority of Fastly clients still had to sit tight and wait for Fastly to fix the issue. Luckily this was only an hour. It could have been much longer.

Just like death and taxes, software outages are a certainty. The real story is not that Fastly had an outage. It is why didn't these large websites have a contingency plan for a single point of failure. For sites at that scale, this is a major oversight in infrastructure planning.

How to handle a CDN failure

The simple solution is to have a backup CDN provider already configured and tested, ready to switch over to if your primary provider fails. You can then utilise short expiry of DNS records to redirect users when the failure happens. This needn't be very expensive or complicated, although individual circumstances vary.

A Quick Introduction To DNS (Domain Name System)

Modern CDNs, like Fastly, Cloudflare, and Peakhour, operate as ‘reverse proxies’. This means they sit between a website's end users and the website server itself. They achieve this through DNS configuration.

When someone types a domain url into a browser, eg fastly.com, a request is sent to a DNS server with the host name (eg fastly.com) to find the IP address of the server to retrieve the content from. CDNs, like Fastly, get website admins to list the address of the CDN on the DNS server. That means requests for a website go through the CDN first. The process is analogous to listing someone else’s number in the phone book so they take calls for you.

The DNS server has a TTL (Time To Live) associated with its records. This TTL tells whoever asked for an IP address, for a given hostname, to remember the answer and not ask again until after the TTL has passed. Typically DNS record TTLs will be 1 hour, but they can be shorter, eg 1 minute.

Switching providers in case of an outage

By keeping a short TTL in DNS, webmasters can switch the answer for a DNS request to that of another provider, meaning users can quickly be directed to an alternative Cloud Provider. Once service has resumed on the primary provider, DNS can be switched again so normal traffic is resumed. The key is that the alternative provider is configured, tested, and ready to go.

This switch can even be automated to minimise outages. Premium DNS services, like Amazon’s Route 53, have optional health checking of DNS answers. This allows a switch to happen nearly instantly. The only downtime would be for people already on the site who have to wait for the TTL to expire before being directed to the backup Cloud Provider. In fact this is exactly what Peakhour.io does. In the event of a catastrophic outage we use DNS to switch to backup infrastructure so our clients are minimally affected.

Backup provider options

Now we've shown how switching CDN providers can be done, let's compare the major players and how they might serve as a backup CDN for Fastly. The three things we'll look at are Cost, Features, Integration.

Simply route traffic to the origin

This would be the simplest and most cost effective option, Assuming your origin server can handle the increased load that removing its CDN would entail. It also assumes that it's ok to lose any features that you may have been relying on, eg load balancing, WAF, edge scripting, image optimisation etc.

Cloudflare

Many people use Fastly because it uses Varnish, a richly featured, programmable cache with several advanced features. If you rely on those features, eg cache tags, cache on cookie value, custom cache tags, then you have to be on Cloudflare's top plan, which is not cheap.

The other major drawback of Cloudflare is that, unless you are on the most expensive plans, you have to cede control of DNS to them by delegating your domain. Cloudflare DNS is a great service, however it has the major drawback of caching negative DNS requests for an hour. If you were switching from an A record to a CNAME record or vice versa, you could be down for an hour regardless. Not ideal.

Akamai

Akamai has a highly respected, fully featured, and very expensive product. Maintaining a backup option with them will run into the $1000s a month. Only you can decide whether it’s worth it.

Cloudfront

Amazon's CDN offering is the third of the big three alternatives. Since it uses volume based billing, it could be an attractive CDN option as a standby, as long as you don't mind missing out on cache by tag (sorry Magento and Drupal). It is also complicated to configure for dynamic content and could miss features that you need. In fact most people use Cloudfront for static content, eg images, CSS, etc and run a Varnish instance within AWS to provide easier to configure full page caching.

This is what the BBC did with the Fastly outage. They had their backup infrastructure on Cloudfront and, as of time of writing, hadn't switched back to Fastly.

Peakhour.io

Peakhour is also volume based billing with a minimum monthly charge of $20. We provide all the advanced caching features that Fastly does, as well as WAF and image optimisation as standard, all in the one service fee. We don't require you to cede control of DNS to us and we're Australian owned and based.

Final Thoughts

CDNs, no matter how big, can fail. If your website is important then it needs a Plan B. This is how that Plan B works, and it doesn't have to be expensive when using a provider like Peakhour.io.

The important part is having it configured and tested before you need it.

Why Manage Bots?

2020-11-30T13:00:00+11:00

Modern sophisticated bad bots often work around traditional security controls. They disrupt websites, mobile applications, and APIs. Malicious bot tactics include scraping user and pricing data, creating fake accounts, running advertising click fraud, exhausting online inventories, and taking websites offline with automated DDoS attacks.

About one-quarter of all website traffic in 2019 originated from bad bots, an increase of 18 percent over 2018. Advanced persistent bots (APBs) made up seventy-five percent of that bad bot traffic as they attempted to evade detection by cycling through random IP addresses, using anonymous/residential proxies, and changing their identities (user agent). The industries hit hardest by bad bots in 2019 included financial services, education, ecommerce, and government as well as media and airlines.

“Bot attack campaigns have become big business for threat actors, and major organizations are now fighting to support legitimate users and prospects while keeping attackers out of online applications and services,” says Paula Musich, Research Director, Enterprise Management Associates.

Bots have moved from simple scripts to distributed networks of automated agents that can mimic human interactions with machine learning techniques. They can avoid detection by network security technologies that have not kept pace with the way automated agents now operate.

Reducing the damage from bad bots means using security countermeasures that detect automated traffic and make attacks uneconomic, not just visible.

Bot Countermeasure Best Practices:

The following bad bot countermeasure practices cover network security, machine learning, and behavioural analysis. The aim is to reduce the economic harm that malicious bots inflict on businesses and end-users.

Web Application Firewalls

Web Application Firewalls (WAF) are a common first line of defence that filter out harmful Layer 7 web application (HTTP) traffic using rules or policies that protect organisations against Distributed Denial of Service (DDoS) bot attacks. WAFs also protect against cross-site forgery, cross-site-scripting (XSS), file inclusion, and SQL injection attacks. A WAF is considered a reverse proxy that protects servers and can be deployed as an appliance, server plug‑in, or filter, and customised by application type or use case. WAF rules can be updated or changed based on the type of bot attack.

IP Tracking and Reputation

Sophisticated bots can be detected with network forensics by inspecting web traffic and assessing whether requests come from actual users or bad bots. Requests can be analysed using data sources including Tor/proxy IPs, IP addresses, IP geo-location information, ISP information, and IP owners. Additional sources for real-time and near-time malicious IP threat data can come from network data, CERTs, MITRE and cooperating competitors.

Client/Device Fingerprinting

Fingerprinting attempts to identify devices, including PCs, Internet of Things (IoT) devices, mobile devices and servers, using data attributes that create real-time risk profiles to stop bot attacks. Using web page access data, a bot detection fingerprinting engine generates unique fingerprints for each end-user device and checks them against bad bots that use evasion techniques, including dynamic IP addresses and anonymous web proxies.

Machine Learning

Artificial Intelligence (AI) and machine learning algorithms are increasingly used to analyse malicious bot activity and make mitigation recommendations using data from sources such as user activity history, behavioural patterns and meta-data. Machine learning can use custom-tailored algorithms to target bots and iteratively process user data and identities to discern emerging bot attack patterns from very large amounts of real-time information.

Tarpitting

Tarpitting is a bot countermeasure that delays and slows down incoming malicious traffic from suspect connections. The technique is used to increase the financial and resource costs of bot attacks in an attempt to discourage malicious actors. Bad bot tar pits can delay bot request responses or take the bad bot IP address attack source offline completely. Innovative tarpitting techniques include requiring bad bots to solve computationally complex maths challenges to access resources or websites, thereby slowing down or stopping bot activity.

User Behavior Analysis

User interaction behaviour and identifying characteristics on a web page or mobile app differ from the behaviour of an automated malicious bot. Factors such as number of pages visited per session, time spent on each web page or within a mobile app and repeat visit frequency all help differentiate authentic users from bad bots. Defeating bad bots using Behavior Analysis involves creating a user model for individual sites with historical visitor data, then checking for anomalies that may indicate bad bot activity.

Intent-based Deep Behavior Analysis (IDBA)

Compared with Behavior Analysis, Intent-based Deep Behavior Analysis (IDBA) conducts behavioural analysis at the user intent level rather than the commonly used interaction-based behaviour analysis. IDBA consists of intent encoding, intent analysis, and adaptive learning. It also employs machine learning techniques to detect bad bots emulating on-site human behaviour interactions. Bad bot mitigation techniques include limiting attempts on login pages, web authentication pages and API call authentication pages.

Rate Limiting

Rate Limiting mitigates bad bots and DDoS attacks by restricting the amount of incoming traffic accepted by specific applications and API endpoints using pre-defined bandwidth limitation policies. Web applications, GET versus POST requests, APIs that receive queries, and login credentials can all be blocked if clients, IP addresses or IP and user-agent pairs violate Rate Limiting rules. Intellectual property scraping can also be protected by Rate Limiting policies that restrict repeated image or digital downloads.

Javascript Injection

JavaScript Injection techniques can help mitigate bad bot attacks in several ways. Scripts can be placed into web applications that “fingerprint” a user’s browser to distinguish humans versus bad bots emulating “human-like” mouse movements, keystrokes or clicks. Fingerprinting detection may also involve user agent identification, HTML5 canvas and audio fingerprinting, and protocol-level fingerprinting with TLS and HTTP2. JavaScript combined with browser cookies can also be used to identify anomalous behaviour from unwanted traffic or bad bots trending over time.

ANYCast DDoS Mitigation

Anycast is an IP addressing method that routes incoming traffic requests to the nearest location or “node.” Using ANYCast for selective routing enables network load resilience against DDoS attacks by routing high traffic across multiple servers and data centres. This prevents network resources from becoming overwhelmed with malicious or irrelevant traffic.

Alternative Content Serving

Serving Alternate and Cached Content when a bad bot is detected gives organisations a way to mislead bots without blocking them altogether. For instance, e-commerce sites may fool price scraping bots by serving alternative web pages that look like legitimate pages but with higher prices. Serving Cached Content when a bot is detected also minimises load on servers without affecting site performance.

Challenges

Requests from suspected bots can be redirected to Challenges or puzzles such as a CAPTCHA, also known as a Completely Automated Public Turing test, to help identify a bad bot versus a human. Online puzzles, such as letter matching, are easy for humans to solve but difficult for automated bots. reCAPTCHA, offered free from Google, is an advanced version of CAPTCHA puzzles that require users to identify text from real-world images such as street address signs, printed books or text from paper newspapers.

Final Thoughts

Bad bots hijack user accounts, create fake accounts, scrape websites for data and personal information, flood websites with traffic through automated distributed denial of service attacks and attack public-facing APIs using constantly changing techniques. They hide behind dynamic IP addresses, change their attack signatures, mimic human behaviours, and take over vast networks of hosts and IoT devices, creating zombie machines that distribute malware across the internet. Countermeasures ranging from Web Application Firewalls to sophisticated Machine Learning algorithms form an organisation's primary line of defence against bad bots.

Application Performance Optimisation

2020-09-12T13:00:00+10:00

Website performance testing is most useful when it explains where a request lost time. A single score can tell you that a page is slow, but it rarely tells the site team what to change next. The better starting point is the request path: where the visitor is, where the origin is, whether the response was cached, how much the browser had to download, and what work blocked rendering or interaction.

This article is a primer for that diagnosis. Start with a representative page, test it from a location that matches your users, then read the evidence in order.

Start With the First Request

The browser cannot render a page until it receives the main HTML document. That first request includes DNS lookup, TCP connection setup, TLS negotiation, any redirect, cache handling, origin processing, and the first byte coming back. When the origin is far away, latency compounds. Peakhour's performance material uses simple Australian examples: Sydney to Melbourne is about 5 ms one way, Sydney to Perth about 25 ms, and Sydney to San Francisco about 75 ms. Those numbers become larger when a page load needs several round trips.

This is why the main document matters in WebPageTest. In one Peakhour example, the main HTML document took 149 ms, which is fast. Many sites take 2 to 5 seconds before the browser receives that document. If the main response is already slow, the page has little chance of a good Largest Contentful Paint because the browser has not yet discovered the resources needed to paint the largest content.

Cache state is part of the same first request. A cache hit at the edge should look different from a miss that forwards to origin. Headers such as Cache-Status can show hit, miss, TTL, stored state, cache key, and collapsed request behaviour. Without that evidence, teams often guess whether a slow page is caused by the CDN, the origin, the application, or a cache-bypass rule.

Use WebPageTest for the Waterfall

WebPageTest is useful because it loads the page in a real browser and lets you choose test locations and connection profiles. That matters for Australian sites because a test from the wrong continent can make a local problem look worse or hide a regional problem from view.

The waterfall is the main working view. Read the first rows before looking at the rest of the page. A redirect on the first request adds delay before the useful page is even requested. Long DNS, connect, or TLS blocks point to connection setup or third-party domains. A long wait on the main document points to cache miss, origin processing, or backend work. Large downloads point to page weight. Red rows show broken requests, and the domain view shows how many external services the browser had to contact.

The filmstrip is just as important. It shows what the user saw while the waterfall was happening. If the HTML arrived quickly but the filmstrip stayed blank, the issue may be render-blocking CSS, JavaScript, fonts, or a hero image. If the page starts to render quickly but then jumps, you are looking at layout stability, not just network speed.

Use Lighthouse for Browser Work

Lighthouse is good at surfacing work inside the browser. It runs a controlled test, reports Core Web Vitals-related metrics, and lists opportunities such as eliminating render-blocking resources, reducing unused CSS, deferring JavaScript, compressing assets, and reducing main-thread work.

Treat the score as a prompt, not a verdict. Lighthouse scores can vary between runs because the local CPU, network conditions, and server response can vary. The useful part is the diagnostics. If Lighthouse points to a font, a theme stylesheet, or a third-party script that delays rendering, compare that with the WebPageTest waterfall and filmstrip. If both tools point to the same resource, you have a stronger case for change.

Connect Metrics to Causes

Core Web Vitals are easier to act on when each metric is tied to the part of the path it describes.

Symptom	Where to look	Likely next question
High TTFB	WebPageTest first row, cache headers, origin logs	Did the request hit cache, miss to origin, or wait on application work?
Slow FCP	Waterfall, filmstrip, render-blocking resources	Did HTML, CSS, or synchronous JavaScript stop the first paint?
Slow LCP	Main document timing, hero media, image weight	Was the largest element discovered and delivered early enough?
High CLS	Filmstrip, image dimensions, injected banners, fonts	Did content move after the visitor started reading?
Poor INP	Lighthouse main-thread diagnostics, third-party scripts	Is JavaScript delaying the next paint after interaction?
High page weight	WebPageTest content breakdown	Are images, JavaScript, fonts, or unused assets carrying the load?
Too many domains	WebPageTest domains view	Which third-party services are adding connection setup and blocking work?
Unclear cache behaviour	`Cache-Status`, CDN analytics, debug headers	Which paths are hits, misses, bypasses, stale responses, or collapsed misses?

This table is not a checklist to run forever. It is a triage map. Pick the symptom that matches the page, then follow the evidence to the next request-path decision.

Diagnose the Common Failures

Latency is the first failure to rule in or out. If the origin is in the United States and most customers are in Australia, the page can lose time before WordPress, Magento, Drupal, or a custom application has done any work. Edge caching helps when the response is safe to reuse, because the first byte can come from a location closer to the visitor.

Page weight is the next obvious drag. WebPageTest's content breakdown shows whether images, JavaScript, CSS, fonts, or other assets dominate the transfer. Unoptimised images are common, especially on CMS and ecommerce sites. Image variants in AVIF or WebP, responsive sizes, compression, and stable dimensions can improve both download time and LCP.

Render-blocking resources explain why a page can download quickly and still look blank. CSS and synchronous JavaScript in the head can stop the browser from painting. Large theme bundles and plugin scripts often include code that is not needed on the current page. The fix might be removing unused files, deferring non-critical scripts, splitting code by route, self-hosting critical third-party resources, or using preconnect only where the first view depends on a third-party domain.

Third-party domains need a sober review. Analytics, marketing tags, chat widgets, social embeds, payment scripts, and fonts can all be legitimate. They can also add DNS, TCP, TLS, download, parsing, and main-thread cost. If a third-party script is not needed for the first view, it should not block the first view.

Where Peakhour Evidence Fits

Peakhour performance work should show both user experience and origin relief. For caching, that means hit ratio, miss causes, Cache-Status, purge state, cache keys, shielded misses, collapsed requests, and origin fetch volume. For images, it means original size, transformed size, selected format, responsive variant, and cache hit state. For Core Web Vitals, it means LCP, CLS, INP, TTFB, page weight, and the same page tested before and after changes.

Security belongs in the performance review when it changes the request path. Bot filtering, WAF rules, rate limits, and login protection can reduce origin load by stopping abusive or noisy traffic before PHP, database, search, or API work begins. They can also create friction if rules are too broad. Measure the edge decision, the latency, the origin effect, and the false-positive risk. Do not treat "security enabled" as a generic performance story.

Good performance testing ends with a specific change to validate: cache this public route, purge it by tag, move this script later, replace this image variant, reduce these third-party domains, protect this login path, or investigate this slow origin query. Then run the same test again from the same location and compare the evidence.

Instant Alerts

2019-05-31T13:00:00+10:00

We've introduced Instant Alerts, an optional service that notifies you when there is a problem with your site. You can specify the email addresses and/or mobile numbers that should receive the alerts.

At the moment you can receive alerts when your origin server:

Times out
Cannot be reached at all
Returns an error

To avoid repeated messages, each alert has a cooldown limit you can set from 30 minutes to 24 hours.

If you're using a service like Pingdom to check that your site is up, you probably will want to enable Instant Alerts. Services like Pingdom might be served pages from our cache while your origin server is actually down.