Residential Proxy Detection

Wed 17 May 2023

Adam Cassar

Co-Founder

13 min read

Residential proxies are under increasing scrutiny, both for how their IP addresses are obtained and for how those networks are used. They also expose how heavily many online services rely on GeoIP data, from content customisation to security controls.

That scrutiny reveals a complicated reality. Residential proxies can help businesses, researchers, and individuals preserve anonymity or work around GeoIP-dependent restrictions. The same properties also create ethical problems, particularly when the networks are misused.

This article explains what residential proxies are, how they work, where they are useful, and where the risk sits. The same properties that make them attractive for legitimate monitoring and research also make them useful for abuse.

Demystifying Residential Proxies

These proxies connect automated software to the internet through IP addresses tied to real-world residential locations. That lets the software look closer to ordinary internet usage, which can help it bypass geographical and network restrictions while adding a layer of anonymity.

Residential proxies need a clear legal and ethical distinction. Their use can be lawful, including for web scraping and data gathering, while still enabling activity that may breach the intended usage policies of some online services. This could include mass consumption of data intended for general use, such as scraping websites for machine learning datasets. These actions may not be strictly illegal, but they raise substantial ethical questions and are often unwelcome to the data providers.

Applications of Residential Proxies

The defining characteristic of residential proxies is that requests can appear to originate from local residential networks. That supports a wide range of use cases, including:

1. Concealing True IP Addresses: Residential proxies allow third parties to hide genuine IP addresses and location, making identity and origin harder to determine. By routing internet traffic through residential IP addresses, they can evade detection, bypass security rules, and access geo-restricted content.

2. Research and Monitoring: Residential proxies are often used by researchers, analysts, and market intelligence professionals to gather data and monitor online activity. By utilising residential IP addresses, they can emulate real user IP addresses and bypass restrictions.

3. Web Scraping and Data Gathering: Residential proxies are central to many web scraping and large-scale data collection workflows. With the capacity to rotate IP addresses and access a wide range of residential locations, third parties can scrape valuable data from websites without triggering anti-scraping measures. Residential proxies can make data scraping more discreet, with fewer access interruptions and cleaner collection results.

4. Ad Verification: Residential proxies are widely used for ad verification. Ad verification companies utilise residential IP addresses to confirm the accuracy and legitimacy of online advertisements. By mimicking genuine residential connections, they can check that ads are correctly displayed and monitor the performance and integrity of advertising campaigns.

5. Ad Fraud: Residential proxies can also be misused for ad fraud. Competitors or their agents may utilise residential IP addresses to falsely inflate the views of a rival's online advertisements. By using genuine residential connections, these entities can manipulate advertising metrics, compromising the accuracy and integrity of the ad's performance data. This abuse of residential proxies for ad fraud poses a significant concern for the online advertising industry.

6. Last Mile Monitoring: Last mile monitoring is another application for residential proxies, allowing companies to assess the user experience from a residential viewpoint. By using residential IP addresses, they can monitor website loading speeds, test service availability, and evaluate the performance of online platforms more accurately. This helps organisations pinpoint and rectify issues that may negatively affect user satisfaction.

Navigating the Risks and Concerns

Residential proxies create material risks, particularly when users are unaware that they are hosting one. Their use can introduce practical limits and security vulnerabilities that are easy to miss.

Despite their valid uses, residential proxies can be used for cybercriminal activity. Malicious actors may exploit them for account takeovers, fraud, or other targeted attacks.

Using residential proxies without the knowledge or consent of residential users creates serious security issues. These users, unaware of how their connections are being utilised, could face legal exposure, compromised privacy, and cyber threats. Their devices could unwittingly participate in malicious activity, leaving them exposed to legal consequences and reputational damage.

Exploring the Creation of Residential Proxies and their Implications

Residential proxy providers build their networks in several ways, some of which can have significant security implications.

Providers can obtain residential proxies through partnerships with Internet Service Providers (ISPs) or by leasing IP addresses from legitimate residential users. At the same time, some providers or private groups may use questionable practices to obtain residential proxies.

1. SDKs: Certain applications may include Software Development Kits (SDKs) that gather and sell user data, including their IP addresses. In some instances, these SDKs can be exploited by residential proxy providers to acquire residential IPs without the explicit consent or knowledge of the users.

2. Malware Exploitation: Malware, including botnets, can infiltrate the devices of unsuspecting residential users. Attackers may then exploit these infected devices as part of a broader residential proxy network, without user awareness. This unauthorised use of residential IPs poses significant security threats to both the affected users and the wider internet ecosystem.

3. Free VPN Services: Some free Virtual Private Network (VPN) services, which promise anonymity and privacy, may use users' connections as part of their residential proxy networks. Users unknowingly become exit nodes for other users' internet traffic, potentially exposing their connections to malicious activities.

Using residential proxies without the knowledge or consent of residential users raises serious security concerns. These users may not understand how their connections are being used, which can lead to legal consequences, compromised privacy, and exposure to cyber threats. Their devices might unknowingly participate in malicious activities, exposing them to potential legal consequences and reputational damage.

The Birth of 'Ethical' Proxies

An important part of the residential proxy discussion is the rise of providers claiming that their IP address pools are ethically sourced. These companies argue that they have obtained the consent of the original IP owners and provide transparency in how these connections are utilised. By positioning themselves as 'ethical' residential proxy providers, they aim to mitigate the associated risks and concerns.

Even where consent is obtained, the potential for misuse remains a significant issue. This is largely due to the inherent anonymity of residential proxies and the difficulty of tracing activity back to the original user. Despite claims of ethical sourcing, the complexity and opacity of the residential proxy environment mean that it remains a grey area, inviting scepticism and demanding further scrutiny.

The result is a nuanced market that consumers, providers, and regulators need to understand as the digital landscape continues to evolve.

From Hola VPN to the Camaro Dragon

Several publicised incidents show how residential proxies are formed and the impact they have had on the industry and users. These examples show the different ways residential proxies can be created and used, legitimately and otherwise.

1. Hola VPN is a well-known free VPN service that promises privacy, security, and access to blocked content. However, it fell under scrutiny when it was revealed that it was selling its users' bandwidth to its sister company, Luminati, which operates a residential proxy network. Users of Hola VPN unknowingly became part of a residential proxy network, with their connections being utilised by third parties. This raised significant ethical and security concerns, as users' devices could be implicated in illegal activities carried out using their IP addresses.

2. The residential proxy service known as 911 has been selling access to hundreds of thousands of Microsoft Windows computers for the past seven years. This service enables customers to route their internet traffic through these computers, allowing them to appear as if they are browsing from any country or city around the world. While 911 claims that its network comprises users who voluntarily install its "free VPN" software, recent research indicates that the proxy service has a history of obtaining installations through questionable "pay-per-install" affiliate marketing schemes, some of which were operated by 911 itself. The service primarily targets users in the United States but has a global user base. Residential proxy networks like 911 can serve legitimate business purposes, but they are often abused for cybercriminal activities due to the difficulty in tracing malicious traffic back to its source.

3. Cybercriminals are increasingly leveraging residential broadband and wireless data connections to anonymise their malicious traffic. One notable type of network, referred to as "bulletproof residential VPN services", has gained attention. These networks are constructed by acquiring discrete blocks of internet addresses from major internet service providers (ISPs) and mobile data providers. An investigation into one such company, Residential Networking Solutions LLC (also known as Resnet), unveiled that it had obtained a significant number of IP addresses, some of which were previously controlled by AT&T Mobility. Resnet leased these IP addresses, enabling it to resell data services for major providers such as AT&T, Verizon, and Comcast Cable. However, the precise nature of the relationship between Resnet and AT&T remains unclear, and the matter has been referred to law enforcement. Cases like this emphasise the potential abuse of IP addresses within residential proxy networks.

4. Infatica.io, a Singapore-based company, has developed a network of over 10 million web browsers that clients can rent to conceal their true internet addresses. The company achieved this by compensating browser extension developers to incorporate its code into their extensions. Many extension developers struggle to earn fair compensation for their work, making offers like these enticing. Infatica seeks extensions with at least 50,000 users and offers to pay developers between $15 and $45 per month for every 1,000 active users with the code included in their extensions. Infatica's code routes web traffic through the browsers of extension users, providing anonymity to the company's customers. The service's pricing depends on the volume of web traffic a customer wishes to anonymise. However, this approach raises concerns about privacy and the potential misuse of users' browsers for malicious activities. Developers, particularly those who author free software, can find the monetisation opportunity offered by residential proxies extremely tempting. The potential to earn revenue from their existing user base by incorporating such code into their extensions can present a persuasive proposition.

5. Camaro Dragon, a form of malware, provides a recent example of residential proxies being acquired through malicious means. This malware infects the devices of unsuspecting users, forming a botnet that can then be utilised as a residential proxy network. Infected devices can then be exploited for various cybercriminal activities without the knowledge or consent of the device owners. This example highlights the significant cybersecurity risks associated with residential proxies and emphasises the importance of robust protection measures.

6. Volt Typhoon is a state-sponsored actor based in China that typically focuses on espionage and information gathering. Volt Typhoon proxies all its network traffic to its targets through compromised SOHO network edge devices (including routers). Microsoft has confirmed that many of the devices, which include those manufactured by ASUS, Cisco, D-Link, NETGEAR, and Zyxel, allow the owner to expose HTTP or SSH management interfaces to the internet. Volt Typhoon has been active since mid-2021 and has targeted critical infrastructure organisations in Guam and elsewhere in the United States.

These examples illustrate the ethical, security, and legal issues surrounding residential proxies. They put transparency and consent at the centre of how proxy networks are acquired and used. The implications for users, the security industry, and the broader digital landscape are substantial, which is why regulation, user education, and responsible practices matter for protecting privacy, security, and the integrity of the internet.

Legal Consequences of Residential Proxies in Data Scraping Operations

Residential proxies are a concern because of their potential for misuse and their legal implications. Two notable cases, the Ticketmaster Case and the Meta vs Bright Data Case, have drawn attention to the challenges posed by the unauthorised use of residential proxies in commercial settings and data scraping operations. These cases show why the legal ramifications of residential proxy use need to be understood in real-world scenarios.

1. The Ticketmaster Case: In 2018, a major international case came to light when Ticketmaster sued Prestige Entertainment for using residential proxies to circumvent ticket-purchasing limits and scoop up large numbers of tickets for resale. This case underscores the potential misuse of residential proxies in commercial settings, and how they can be used to breach the terms of service of websites.

2. The Meta vs Bright Data Case: The legal case between Meta Platforms, Inc. (formerly Facebook) and Bright Data Ltd. demonstrates a contentious and potentially unlawful use of residential proxies in the real world. In this case, Meta accused Bright Data of operating a business designed to use automated software to scrape and sell data from various online platforms, including Facebook and Instagram. This scraping was allegedly facilitated using unauthorised tools and services that bypassed detection by Meta's security measures. Despite Meta's efforts to halt these activities, Bright Data purportedly continued its operations. The data involved included user profiles, follower counts, and shared posts. Bright Data was alleged to not only scrape this information but also advertised the sale of the scraped data. The scope of this operation was extensive, with the Instagram data set alone priced at $860,000.

These cases show how residential proxies are used in practice, the challenges they present, and why their use remains legally and commercially contested.

The Wider Implications for the Security Industry

The growth of residential proxies, and the way some networks are acquired, has broader implications for the security industry. It raises questions about transparency, ethical practices, and the responsibility of proxy providers.

1. Ethical and Regulatory Implications: The questionable practices some providers use to acquire residential proxies highlight the need for stronger regulation and industry standards. This would help ensure that residential proxies are obtained and used in a lawful and ethical manner, protecting users' privacy and the wider internet ecosystem. There is a clear demand for more transparency in how these services operate and procure their proxies.

2. Cybersecurity Implications: Residential proxies can enable malicious cyber activity, ranging from fraud to targeted attacks. This can increase the need for cybersecurity measures and protections, potentially reshaping strategies and priorities within the cybersecurity industry.

3. Legal and Reputational Implications: If individuals unknowingly become part of a proxy network, there could be legal repercussions for them if their connections are utilised for malicious activities. This could lead to greater scrutiny and liability for companies operating within this space.

4. State Actors and Residential Proxy Networks: State-sponsored actors have been known to establish their own residential proxy networks within foreign countries for various campaigns, including information warfare, disinformation campaigns, and surveillance, adds another layer of complexity to the issue. These activities pose significant geopolitical and security risks, requiring increased international cooperation and robust defence mechanisms.

The rise of residential proxies exposes a weakness in common security models: the assumption that residential and mobile IPs are inherently more trustworthy, and that GeoIP is a reliable reputation or security control. Widespread proxy use has shown how brittle that assumption can be.

Uncertain or unethical sourcing makes that trust problem harder. It can make online interactions less reliable and introduce security risks.

Residential proxies are not just tools; they highlight a deeper issue in how we approach digital access and security. Understanding what is already known, questioning current practices, and building better controls are practical steps towards using residential proxies responsibly and ethically. Recognising the false sense of security GeoIP restrictions can provide is part of that work.

Part 1 ends here. In Part 2: the Camaro Dragon malware, we look more closely at a specific case. This sophisticated malware uses residential proxies in a way that shows their potential for misuse. The next article covers how Camaro Dragon works, its impact on cybersecurity, and practical protection measures.