Peakhour.IO - Web Performance

How To Exclude Query String Parameters from Search Engine crawling

2024-05-21T13:00:00+10:00

Last year we wrote about the problem of excessive crawling from search engine spiders. Search engines such as Google and Bing aim to index as much content as possible. For ecommerce sites, this often means indexing pages with query string parameters used for sorting, filtering, or pagination. Those parameters help users navigate the site, but they can cause a few predictable crawler problems:

Over-Crawling: Search engines may spend too much time crawling similar pages with different parameters, wasting crawl budget.
Duplicate Content: Pages with different parameters can be treated as duplicate content, weakening SEO performance.
Server Load: Excessive crawling can increase server load, slow down your site, and affect user experience. Search engines typically account for 30-50% of page requests to an ecommerce store. Managing their crawling effectively can have a material effect on site speed and server spend.

Another common cause of over crawling is internal searches being indexed.

In our previous article we mentioned using the webmaster tools provided by Google and Microsoft to manage crawler behaviour by adding ignored parameters. Since then, both tools have been updated and no longer allow you to add parameters to ignore during a crawl.

Differences in Crawling and Indexing

Search engines maintain an 'index' of web pages. Pages in this index are what appear in search results. To maintain the index, the search engine crawls a website to 'discover' new content and keep existing entries up to date. Webmasters can control what gets indexed with tags or headers in their web pages. These include:

Canonical Tags can be used to indicate the preferred version of a page. This helps consolidate link 'juice' and tell the search engine which URL to index.
Noindex tags can be used to prevent specific pages from being indexed. This is useful for thank you pages, admin pages or any content you don't want to appear in search results.
Nofollow links can be used to indicate to a search engine not to pass on SEO value to the linked page.

However, controlling what does or does not get indexed does not prevent content from being crawled. The only way to do that is via the robots.txt file. You may be familiar with the Disallow directive in the robots.txt file, but you can also use wildcards to prevent crawling of url parameters.

An example...

Consider an ecommerce store that has a category page which can then be customised with the following parameters:

    orderBy
    colors
    brands
    page
    results

These may appear in any order, and the combinations can result in 100s or even 1000s of variations of essentially the same page. Google is fairly smart when presented with this scenario, but Bing.... Bing can crawl very aggressively and it likes to try everything. In our example above, we may want to stop crawling everything except the page number, in which case an effective way to control crawler behaviour would be:

    User-agent: *
    Disallow: /*?*orderBy=*
    Disallow: /*?*colors=*
    Disallow: /*?*brands=*
    Disallow: /*?*results=*

We can't really do this in a single Disallow because the parameters might be in any order. By including the ? in the url we're ensuring that the parameter names are only in the query string, not in the main url path. This prevents crawlers from wasting crawl budget and putting unnecessary load on server resources.

Final Thoughts

Search engines can often make up 30-50% of the overall page requests to a website. Managing their behaviour helps maximise useful crawling and minimise server utilisation. Keep an eye on your access logs for unwanted behaviour, and use robots.txt where it gives you the right level of control.

RFC 9460

2023-11-16T00:00:00+11:00

RFC 9460 introduces two DNS record types: "SVCB" (Service Binding) and "HTTPS". They let browsers learn more connection details during DNS lookup, before redirects and TLS negotiation add extra steps. The result is cleaner connection setup, with practical improvements in speed, security, and efficiency.

Understanding the Current Process

Traditionally, when a browser connects to a website, it follows a sequence:

Browser requests site via HTTP.
Server redirects request to HTTPS.
Browser receives ALPN (Application-Layer Protocol Negotiation) during the HTTPS handshake.

The model is secure, but it is not optimal. It involves multiple round trips, which affects Time to First Byte (TTFB) and the overall user experience. Load balancing and failover are also less direct than they could be. RFC 9460 changes this by allowing DNS to provide the necessary connection details earlier. That reduces the steps involved in establishing a secure connection, lowering TTFB.

The Impact of SVCB and HTTPS Records

SVCB and HTTPS records move useful connection hints into DNS. They speed up the time-to-first-packet by incorporating the Alt-Svc HTTP header and ALPN TLS extension into DNS, which shortens connection setup. These records also enable redirection at the zone apex, a task not possible with CNAMEs. They simplify DNS load distribution and failover, making web services more resilient. They also remove the need for HSTS preloading and support Encrypted Client Hello (ECH), formerly ESNI, for better privacy.

Adoption and Industry Response

Adoption started before the RFC was finalised. Firefox has been conducting HTTPS lookups since May 2020, limited to DNS over HTTPS (DoH). Apple's iOS, Safari, and macOS have followed suit since September 2020. Chrome introduced partial support in December 2020 and has recently enabled ECH by default. Various DNS service providers have also started supporting HTTPS and SVCB records.

As reported on Netmeister, adoption is still early but not insignificant. As of October 2023, about 10 million domains have implemented an HTTPS record for their 'www' service names, roughly 4.4% of domains. Around 9.1 million domains, or about 4.0%, use the record on their bare second-level domain name. Among the top 1 million domains, approximately 22.5K (25.5%) use HTTPS records for 'www' service names, and nearly 24K (25.6%) use them on bare domains.

What the Records Look Like

A typical SVCB record might look like this:

_example.com. 7200 IN SVCB 1 svc4.example.net. (alpn="h2,h3" port="8004")

This record indicates that the service at _example.com can be accessed at svc4.example.net using either HTTP/2 or HTTP/3 on port 8004.

An HTTPS record could be:

example.com. 3600 IN HTTPS 0 svc.example.net.

This record suggests that example.com should be accessed securely through svc.example.net.

Apex Domains and the Importance of SVCB/HTTPS Records

One long-running DNS limitation is the inability to use CNAME records at the apex (root level) of a domain due to conflicts with other necessary records like NS and SOA. RFC 9460's SVCB/HTTPS records address this by enabling apex domain aliasing without those conflicts. This matters for efficient content delivery networks (CDNs) and load balancing strategies.

These records enhance capability

1. Load Balancing:

Consider a website that needs to distribute traffic across multiple servers. SVCB records can indicate different server endpoints with varying priorities.

example.com. IN SVCB 10 server1.example.com. (alpn="h2,h3")
example.com. IN SVCB 20 server2.example.com. (alpn="h2")

In this example, server1.example.com is the preferred endpoint (lower priority number), offering both HTTP/2 and HTTP/3 protocols. If it's unavailable, traffic automatically shifts to server2.example.com.

2. Failover Mechanism:

For a service that requires high availability, SVCB records can express failover directly:

critical-service.example.com. IN SVCB 1 primary-service.example.com. (alpn="h2,h3")
critical-service.example.com. IN SVCB 2 backup-service.example.com. (alpn="h2")

Here, primary-service.example.com is the primary endpoint. If it fails, the system automatically falls back to backup-service.example.com.

3. Apex Domain Usage:

A practical advantage of SVCB/HTTPS records is their ability to handle apex domains, where CNAME records are not feasible. This is important for root domain aliasing to different service providers.

example.com. IN HTTPS 0 cdn-provider.example.net.

This record indicates that the apex domain example.com is to be served through cdn-provider.example.net, overcoming traditional DNS limitations.

4. Encrypted ClientHello Support:

Future enhancements of SVCB could include keys for Encrypted ClientHello, which improves privacy and security during the initial TLS handshake.

secure.example.com. IN SVCB 1 tls-service.example.net. (ech="base64-encoded-key")

This record can be used to initiate a TLS connection with tls-service.example.net using the provided Encrypted ClientHello key.

5. Directing Traffic to Specific Protocols:

For services that need to direct clients to newer or more efficient protocols, SVCB records can specify the exact protocols to use.

api.example.com. IN SVCB 1 api-server.example.com. (alpn="h3")

Clients that understand HTTP/3 can connect directly using this protocol, bypassing the usual HTTP/1.1 or HTTP/2 protocols.

The Long Wait

HTTPS has been around for a while, so RFC 9460 raises an obvious question: why did this take so long? Apex records have had their share of problems, including not being able to use CNAMEs and having to resort to custom records like ALIAS or Cloudflare's 'cname flattening'.

It is a fair question. We've had some bizarre records hanging around for ages along with a wide range of solutions to the "CNAME at the zone apex" problem.

Credit to the creators of RFC 9460 for getting this through and obtaining browser support: - B. Schwartz from Meta Platforms, Inc. - M. Bishop from Akamai Technologies - E. Nygren from Akamai Technologies

Final Thoughts

RFC 9460 gives DNS a more useful role in HTTPS connection setup. SVCB and HTTPS records let operators publish endpoint, protocol, failover, and privacy information before the browser starts negotiating the connection. That gives service providers more precise control over how clients reach web services, with practical benefits for performance, reliability, and security.

Enterprise-Level Caching for All

2023-11-02T13:00:00+11:00

We've released our Magento 2 plugin for e-commerce stores using Magento. It brings Peakhour's caching features into Magento, including capabilities that other providers often reserve for enterprise plans. With Peakhour, 'Enterprise for Everyone' means making those features available to all customers, regardless of plan.

Why Cache Tags Matter

Cache tags solve a practical website management problem: keeping your cache current when content changes. In Magento 2, a single change, such as updating a product's price, can affect multiple pages. Cache tags ensure that only the relevant cached content is updated, maintaining cache efficiency and reducing server load. That matters for website speed and user experience, which directly affect sales and SEO rankings.

Enterprise for Everyone

While other providers offer cache tags only in expensive enterprise plans, Peakhour makes this feature available to everyone. Our infrastructure and caching algorithms make that possible. We also offer other enterprise-level features, including DDoS protection, real-time analytics, and custom caching rules, so 'Enterprise for Everyone' is reflected in the product rather than just the plan names.

Peakhour vs. Magento and Varnish Caching

Our plugin goes beyond Magento's built-in caching and Varnish cache in several ways:

Custom Cache Tags: Unlike Magento's built-in cache, we offer custom cache tags for more granular cache control.
Advanced Algorithms: Our caching algorithms go beyond Varnish, helping improve cache hit rates and lower server load.
Additional Features: With Peakhour, caching sits alongside real-time analytics, DDoS protection, and custom caching rules, features often missing in standard Magento or Varnish setups.

Expected Performance Improvements

By using Peakhour's Magento 2 plugin, you can expect performance improvements:

Faster Page Loads: Our caching can reduce page load times by up to 50%, giving users a smoother experience.
Reduced Server Load: Efficient caching means fewer requests to your origin server, reducing server load by as much as 70%.
Improved SEO: Faster websites are favoured by search engines, which can improve SEO rankings.
Higher Conversion Rates: A faster website gives users a better experience, which can lead to higher conversion rates and increased sales.

Installation Options

You can install the Magento 2 plugin through Magento Connect, Composer, or a ZIP file. For more detail, see our plugin page.

Useful tips to accelerate your Magento store

2023-11-01T13:00:00+11:00

Out of the box, Magento is not the fastest ecommerce platform. Magento 2 is built with Full Page Cache in mind, so repeat page requests do not always have to hit the application. A slow Magento store can frustrate customers, increase bounce rates, and cost sales. There are several practical ways to accelerate your Magento store and improve the user experience. These are good places to start.

Caching is King

Caching is usually the biggest performance lever for a Magento store. By storing pre-generated versions of pages, you reduce server response times and avoid asking Magento to rebuild the same page for every request.

Full Page Caching (FPC): Magento includes built-in FPC, but it can be extended. Varnish is a common choice for this role. Magento 2 has native support for Varnish, which acts as a web application accelerator.
Redis: Use Redis for session and cache storage. It is an in-memory data structure store that can speed up backend operations by reducing database load.
Edge Caching: Use Peakhour Edge to cache your dynamic pages close to users. This serves content from a nearby delivery path and reduces latency. Peakhour's Magento 1 and Magento 2 plugins make this straightforward to set up.

Optimise Your Images

Images often make up the bulk of a page's weight. Optimising them is one of the simplest ways to improve load times.

Compression: Use image compression tools to reduce file sizes without a noticeable loss in quality.
Next-Gen Formats: Serve images in modern formats like WebP or AVIF, which offer better compression. A CDN can often handle this conversion automatically.
Lazy Loading: Implement lazy loading for images that are "below the fold" (not immediately visible). This means they only load when they are about to enter the user's viewport.

Minify and Merge CSS/JavaScript

Magento has built-in features for merging and minifying CSS and JavaScript files.

Minification: Removes unnecessary characters (like whitespace and comments) from code to reduce file size.
Merging: Combines multiple CSS or JavaScript files into a single file to reduce the number of HTTP requests.

Note: Always test thoroughly after enabling merging, as it can sometimes cause issues with certain themes or extensions.

Keep Your Environment Updated

The environment your Magento store runs in has a direct effect on performance.

Latest PHP Version: Use the latest stable version of PHP supported by your Magento version. Each new release brings performance and security improvements.
Server Resources: Ensure your server has adequate RAM and CPU power to handle your traffic, especially during peak times.
Web Server: Use a high-performance web server like Nginx, which is known for its speed and efficiency.

Use Edge Caching and Delivery

An edge delivery layer is a practical requirement for many ecommerce stores. It caches your static assets (images, CSS, JavaScript) close to users.

Reduced Latency: Users receive content from the server geographically closest to them, which speeds up load times.
Reduced Origin Load: By serving cached content, an edge cache reduces the number of requests that hit your origin server, improving its performance and stability.
Enhanced Security: Peakhour also offers security features like a Web Application Firewall (WAF) and DDoS protection.

Database Optimisation

A slow database can slow the whole store.

Clean Logs: Regularly clean out Magento's log tables (e.g., log_customer, log_visitor). These can grow very large and slow down database queries.
Enable Flat Catalog: For Magento 1 and older versions of Magento 2, enabling the Flat Catalog for products and categories can improve performance by reducing the complexity of database queries.
Re-index Regularly: Keep your Magento indexes up to date. A cron job should be set up to handle this automatically.

Audit Third-Party Extensions

Poorly coded or unnecessary third-party extensions are a common cause of Magento performance issues.

Review Extensions: Audit your installed extensions regularly. If you're not using one, disable or uninstall it.
Use a Profiler: Use Magento's built-in profiler or a tool like New Relic to identify slow-running code, which can often be traced back to a specific extension.

These changes will not fix every Magento performance problem, but they cover the areas that usually matter first: cache behaviour, asset weight, the hosting environment, database maintenance, and extension overhead.

HTTP/2 Rapid Reset Attack Deepdive

2023-10-12T00:00:00+11:00

Distributed Denial of Service (DDoS) attack vectors keep changing. The recent spike in HTTP/2-based DDoS attacks has been notable for its volume, with some attacks surpassing 398 million requests per second. Peakhour observed these attacks and worked through how to understand and mitigate them. This article explains how they work and what operators can do to reduce exposure.

The Rise of HTTP/2 in DDoS Attacks

HTTP/2 was designed to make web traffic more efficient. The same features that improve performance for legitimate users can also be abused in DDoS traffic.

Much of HTTP/2's efficiency lies in "stream multiplexing." It allows multiple messages to be sent over a single TCP connection. While HTTP/1.1 processes each request serially, HTTP/2 can manage multiple concurrent streams on a single connection. This means a client can send multiple requests in a single round trip, increasing how much work each connection can drive.

The 'Rapid Reset' Attack Explained

The "Rapid Reset" attack is a specific DDoS technique built around HTTP/2. The attacker starts by opening multiple streams, much like in a standard HTTP/2 attack. However, instead of waiting for responses, they cancel each request immediately.

The client does this by sending a RST_STREAM frame, indicating that a previous stream should be cancelled. The rapid request-and-reset sequence means the server spends resources processing the request, only for it to be cancelled before a response is generated. This tactic amplifies the server's workload without the attacker needing to wait for responses, which increases the pressure each connection can place on the server.

Variants of the Rapid Reset Attack

Attackers also used variations of the Rapid Reset attack:

One variant involves delaying the reset action. The attacker opens multiple streams, waits, then cancels the streams and instantly opens new ones. This method can evade some rate-based defences.
Another variant avoids stream cancellations. Instead, the attacker tries to open more streams than the server allows. This aims to keep the server continually busy, processing a near-constant flow of requests.

Effective Mitigation Techniques

Mitigation is not as simple as blocking individual malicious requests. A more effective approach is to close the entire TCP connection when malicious activity is detected. The HTTP/2 protocol supports connection termination through the GOAWAY frame. This feature needs to be used aggressively to prevent Rapid Reset attacks, rather than relying on the more passive, standard implementation.

Deciding which connections to treat as malicious is a challenge. One potential strategy is to monitor connection statistics. If a connection exceeds a set threshold of cancelled requests, it might be deemed malicious. Responses to suspect activity could range from sending a GOAWAY frame to terminating the TCP connection.

For the non-cancelling variant, the best approach is to shut down connections that breach the concurrent stream limit, either immediately or after a few violations.

Broader Protocol Implications

These attack techniques are specific to HTTP/2, but the wider protocol lesson still matters. The HTTP/3 (QUIC) protocol isn't directly vulnerable in the same way. As a precaution, server implementations should consider limiting the work done by a single connection.

The Importance of Industry Collaboration

When the threat of the Rapid Reset attack became apparent, the industry collaborated to address the issue. The vulnerability was disclosed to key HTTP/2 implementers, helping to devise and distribute effective countermeasures. The vulnerability is logged against CVE-2023-44487.

HTTP/2 'Rapid Reset' DDoS attacks pose a serious risk to services using the protocol. To reduce exposure, service providers should promptly apply available software patches and updates.

Learn how Peakhour's Application Security Platform helps protect against Layer 7 DDoS attacks, including the HTTP/2 Rapid Reset vulnerability. Contact our team to secure your infrastructure.

Interaction to Next Paint (INP)

2023-09-11T13:00:00+10:00

Google has announced that Interaction to Next Paint (INP) will replace First Input Delay (FID) as a Core Web Vital as of March 2024. Introduced as a metric in 2022, INP covers gaps in FID by measuring more of what happens after a user interacts with a page.

To help site owners prepare for its introduction as a Core Web Vital, INP is already included in the Chrome User Experience Report (CrUX). By analysing the CrUX data, website owners can see their current INP performance and make targeted optimisations ahead of the March 2024 change.

A better metric than First Input Delay

First Input Delay, as its name suggests, only measures the delay between an input, such as a keypress or mouse click, and the point where the browser begins to handle that event. It does not include the time spent processing the input. It only measures how long the browser was blocked before it could start handling it.

That leaves two issues: it only considers the FIRST event, and it does not measure how long it takes for the user to see the result of their input.

INP is designed to cover both issues. It measures the latency of ALL 'interactions' through to the visual response for that interaction. As explained by Google, an interaction like a tap on a touch screen device can consist of several input events.

"An interaction's latency consists of the single longest duration of a group of event handlers that drives the interaction, from the time the user begins the interaction to the moment the next frame is presented with visual feedback."

After measuring all interactions, the final INP score is the longest interaction observed, ignoring any outliers.

What Constitutes a Good Score

INP is measured in milliseconds (ms), with lower scores indicating better performance:

Good: < 200 ms
Needs Improvement: 200-500 ms
Poor: > 500 ms

How to ensure you have a good INP score

Minimise Main-Thread Work

Long-running JavaScript can block the main thread and increase INP times. Break these tasks into smaller parts and run them asynchronously to reduce delays.

Efficiently Use Browser APIs

APIs that trigger layout recalculations can be expensive. Use them sparingly and look for alternatives that put less pressure on the browser.

Defer Non-Essential CSS and Scripts

Postpone the loading of non-critical CSS and JavaScript. Use techniques like asynchronous loading to improve INP scores.

Monitor Third-Party Scripts

Heavy third-party scripts can degrade INP performance. Use asynchronous or deferred loading for these scripts to limit their impact.

Conclusion

Google estimates that 90% of a user's time on a page is after it has finished loading. FID focused on first impressions, with the assumption that a fast start meant the page would stay responsive. Interaction to Next Paint addresses that gap and gives a more accurate view of user experience. If you want to know your current INP score, you can use our free website speed comparison tool to view it alongside your other Web Vitals, and see how your website compares to your competitors.

Cache Tags/Surrogate Keys

2023-06-28T13:00:00+10:00

Caches are a way of storing information so future requests for the same data can be served more quickly. CDNs, including Peakhour, run caches on each of their POPs (Points of Presence). At that level these caches are key/value stores, where the key can be a combination of several request details, as outlined in our previous blog post on cache keys.

An established way to improve website performance is full page caching, where a copy of a full page generated by a CMS is stored in a CDN. For a cache hit compared with a cache miss, this can typically cut 1-4s off a page load.

Main document load before caching: 2.07s

Main document load after caching: 82ms!!

A Simple Cache Example

When a website changes a resource, such as a page or image, it can instruct the CDN to flush the cache entry for that resource's key. For example, say we have a page:

/about-us/

If we're doing full page caching, it will be stored in the CDN with the key "/about-us/". If it changes, we can issue a flush using the key "/about-us/" and the CDN will fetch a fresh version.

A blog example

But consider a blog article with the URL "/caching/caching-explained/". A typical blog has categories, tags, and authors. A link and summary for the article can exist on many pages: the home page if it's recent, the category pages that the article belongs to, the author page, and so on.

When we update the article, flushing only the key "/caching/caching-explained/" is not enough. We also have to find the other pages it appears on and flush them too, because they may have changed. That means issuing database queries to find all the pages that our article appears on, gathering them into a list, and issuing a flush for each of them.

An eCommerce example

Another example is an ecommerce store with lots of products and product categories. A particular product might appear on 100s of pages with its price displayed. When that price changes, the cached site needs to reflect it. You have two choices: do a lot of work on the server to discover the pages the product is on and flush them, or flush everything. Neither option is good. The first can slow your website to a crawl with database queries; the second forces the cache to repopulate.

Enter Cache Tags

Cache tags, also known as surrogate keys, are a mechanism for adding another way to find content in a cache. Unlike the primary cache key, these tags are not unique.

A website utilises cache tags by returning them in an HTTP header with the response. For example, Magento 2 uses the header X-Magento-Tags:

x-magento-tags: cms_b_site_home_main_banner,store,cms_b,cms_b_site_homepage_bar,cms_p_47,cms_b_header_custom_notice,cms_b_porto_custom_block_for_header_home5,cms_b_site_header_social_links,cms_b_site_home_shopby_category,cms_b_site_home_shopby_brand,cat_c_p_2,cat_p_2508,cat_p,cat_p_2483,cat_p_2387,cat_p_2372,cat_p_1412,cat_p_1388,cat_p_2575,cat_p_2560,cat_p_2557,cat_p_2543,cat_p_2520,cat_p_1262,cat_p_2434,cat_p_2423,cat_p_1660,cat_p_1579,cat_p_1276,cat_p_1217,cms_b_site_footer_social_links,cms_b_site_footer_contact_us,cms_b_site_footer_popular_items,cms_b_site_footer_quick_links,cms_b_site_footer_information

Magento returns tags for page elements like the navigation, sidebar, and notice, as well as for product categories and products. Products have tags in the format cat_p_1234, where 1234 is the product id in the database.

When someone updates product 1234, a flush is issued for the tag cat_p_1234 and all pages that have that tag are flushed. Magento doesn't have to do any work trying to determine which page the product might be on. The cache can efficiently find those cached pages and invalidate them.

Cache tags in CMSs

As mentioned, Magento 2 uses a sophisticated cache tag strategy to maximise the performance of its full page cache. Other CMSs, including Drupal 8/9/10 and Typo3, also utilise cache tags. Peakhour adds cache tags to WordPress, Prestashop, Magento 1, and Opencart via our plugins to enable full page caching.

Cache Tag support amongst CDNs

If you're looking for maximum full page cache effectiveness for your website, especially if you're using a CMS with built-in cache tag support, cache tag support matters. The table below outlines support amongst major CDN providers.

CDN/Cache	Cache Tag Support	Custom Header
Peakhour
Cloudflare	Enterprise Plan Only
Fastly
Self Hosted Varnish
Cloudfront

Enterprise or essential?

In our view, cache tags are an essential feature for any CDN trying to maximise cache performance. They make cache invalidation targeted and efficient, which protects hit rates and reduces work on the origin server. They shouldn't be walled off in an Enterprise-level package.

Maximising Website Speed

2023-06-07T12:31:00+10:00

As businesses prepare for a global economic downturn, every source of friction matters. One of the most controllable is website speed.

For many customers, the website is where they first test whether a business is worth their time. They learn about the company, compare products, read content, and, if the experience holds up, buy. Loading time shapes that first impression, affects engagement, and can change whether a visitor becomes a customer.

This article looks at why speed deserves attention when trading conditions tighten. It covers search rankings, conversion impact, and published case studies where faster sites produced measurable gains.

The Need for Speed

Website speed is not an abstract technical score. It is how quickly users can see and interact with content. A delay measured in milliseconds can affect engagement, conversion rates, and customer retention.

Speed matters because user expectations are set by fast services and fast networks. When a page feels slow, people leave and are less likely to return.

Speed also affects how search engines, including Google, rank your website. For businesses trying to remain visible in a crowded market, especially during an economic downturn, performance is a practical lever.

Correlation with Search Rankings

The relationship between website speed and search rankings is supported by research and by statements from Google. A few years ago, Google announced that page speed would be a ranking factor. The change reflected Google's focus on relevant, usable pages.

Websites that meet all of Google's requirements receive a slight advantage, ranking one percentage point higher than the average. These requirements cover several areas, from content relevance and quality to mobile-friendliness and page speed.

By contrast, websites that fail to meet at least one of Google's requirements can sit at a measurable disadvantage, ranking 3.7 percentage points lower. That matters when search visibility is already under pressure.

Google's Core Web Vitals have also become a measurable factor in search rankings. These vitals measure aspects of page speed and user experience, showing how speed and SEO (Search Engine Optimisation) now overlap.

A study by Crystallize also found a correlation between speed and SEO. In their page speed score experiment, a page with a high score ranked #1 in Google with a featured snippet for the optimised item. Unoptimised pages with lower speed scores did not appear in search results.

The practical point is straightforward: website speed can improve search visibility. In an economic downturn, that extra visibility can matter.

Conversion Impact of Speed

Speed also affects conversion rates. Deloitte's 'Milliseconds Make Millions' report shows how small improvements in loading time can change commercial outcomes.

The study examined a 0.1 second decrease in loading time across different market sectors. In retail, a quicker page loading time led to an 8.4% rise in conversion rates and a 9.2% improvement in average shopping basket size. The travel sector saw a 10.1% increase in conversion rates and a 1.9% rise in average basket size. For luxury brands, faster loading times resulted in an 8.6% increase in page views per session and an 8.3% decrease in form bounce rates.

Peakhour clients have seen the same pattern. Pharmacy Direct reported a 30% increase in conversions and order value after reducing page load time by 90%. Kitchen Warehouse saw a 150% increase in revenue after decreasing page load times by 70%.

These numbers show that page speed is tied to business metrics, not just technical scores. The scale varies by site and sector, but the direction is consistent across the cited examples.

Real-Life Success Stories

The effects of website speed optimisation are visible in published case studies:

French linen brand Carré Blanc saw a 25% increase in conversion rates after improving web page loading speed.
Renault optimised the Largest Contentful Paint (LCP), leading to a 14 percentage point decrease in bounce rate and a 13% rise in conversions.
E-commerce platform eBay found that every 100ms improvement in search page loading time resulted in a 0.5% increase in additions to the shopping cart.
SnipesUSA.com doubled their average conversion rate from about 1% to about 2% by decreasing load times by 30%.
French toy retailer King Jouet enjoyed a 5% increase in conversion rates within a month through page speed optimisation.
AliExpress, a global online retail marketplace, experienced a 10.5% increase in orders and a 27% increase in conversions for new customers by reducing loading time by 36%.
Boutique designer brand Revelry saw 43% faster page loading, an 8% decrease in bounce rates, and a 30% increase in conversions after optimising images on their eCommerce site.
Zalando, an online fashion platform, reported a revenue increase of 0.7% per session by reducing web page loading time by 100ms.
Pinterest observed a 15% increase in platform registrations following an improvement in loading speed.
Telecommunications company Vodafone saw an 8% sales increase with a 31% improvement in Largest Contentful Paint ( LCP).
Mobile marketplace Swappie achieved a 42% increase in mobile revenue by focusing on Core Web Vitals.

These examples show how improving loading speed can lift conversion rates and revenue.

Optimising for Search Performance

Speed also affects search performance beyond organic ranking. Several examples point to paid search impact:

Lever Interactive Agency reported that one of their clients improved their Quality Score, resulting in a 17% decrease in Cost Per Click (CPC), a 31% decrease in Cost Per Acquisition (CPA), and a 20% increase in conversion rate on faster landing pages.
Crystallize Headless Commerce noted that scoring high in the Quality Score can lead to significant benefits, including up to a 50% discount on CPC prices. Conversely, a low Quality Score can result in paying up to 400% extra, severely impacting your marketing budget.

Core Web Vitals have also become a priority for eCommerce platform Shopify. The company continues to optimise speed performance to improve search rankings.

These cases show why performance work needs to be ongoing, especially where search traffic and paid acquisition costs are material to the business.

Enhancing Engagement

Engagement is not separate from speed. A fast, well-optimised site gives users less reason to leave and more opportunity to browse, compare, and interact. The data supports this in several ways.

Take eCommerce for instance. Customers are 10% more likely to recommend an eCommerce website when pages load in 10 seconds instead of 13 seconds. The likelihood of recommendation rises to 26% if loading time is reduced to 3 seconds. That shows how quickly performance changes user perception.

Other companies have also seen measurable effects from speed optimisation. Netflix implemented Gzip compression for resource optimisation, resulting in a 43% reduction in outbound traffic. Yahoo Japan News saw increases in both page views per session and session times (15% and 13% respectively), as well as a 1.72% decrease in bounce rate, by improving their Cumulative Layout Shift (CLS) by 0.2 points.

Google has also published data linking Core Web Vitals to engagement. Their data showed that favourable Core Web Vitals scores can reduce the likelihood of users abandoning a page before it loads by up to 24%. Meeting Core Web Vitals thresholds also led to an overall 43% improvement in bounce rate for The Economic Times.

The agriculture e-commerce platform, Agrofy, improved their Core Web Vitals scores by 70% for LCP and 72% for CLS, resulting in a 76% reduction in abandonment rate. Again, the useful lesson is not just that the site became faster. It is that users behaved differently once it did.

Key Speed Metrics

Website speed is about more than full-page load time. Several metrics help assess how fast and stable a page feels to a user. Google's Pagespeed Insights lists the following important metrics:

Largest Contentful Paint (LCP) measures the time taken to load the largest visible content on the page. The ideal target for this is less than 2.5 seconds. This metric matters because it provides a clear indicator of perceived load speed for the user.
Cumulative Layout Shift (CLS) evaluates the visual stability of a page during loading. The target here is less than 0.1. This helps limit content jumping or shifting while the page loads, providing a smoother user experience.
First Input Delay determines how quickly a page responds to user input, with the target being less than 0.1 seconds. This metric measures the interactivity and responsiveness of a website.

Together, these metrics show whether a website delivers a fast, smooth user experience.

User Expectations and Impact on Business

Users expect pages to respond quickly. When they do not, speed becomes a business issue rather than only an engineering issue.

According to Think with Google, slow-loading pages can affect user experience, resulting in higher bounce rates, negative brand perception, and an impact on conversions and revenue. When users have to wait too long for a webpage to load, they are likely to leave and look for a faster experience elsewhere.

Digital marketing expert Neil Patel highlights that a 1-second delay in page response can lead to a 7% reduction in conversions. To put that into perspective, if an e-commerce site is making $100,000 per day, a 1-second page delay could cost $2.5 million in lost sales every year.

Akamai also found that 53% of mobile site visitors will leave a page that takes longer than three seconds to load. This shows the standards modern users have for website performance and the revenue risk for businesses that fail to meet them.

Common Culprits

If your website is running slowly, a few common issues could be to blame. The usual causes are technical and operational:

Time to First Byte (TTFB) is the time it takes for the first byte of data to be received from the server. High TTFB can affect loading times and should be minimised.
Large Page Size and Resources can also contribute to slow loading times. This includes heavy content, such as images, videos, or large files. Optimising these resources can materially improve loading speed.
Third-Party Resources like ads, plugins, or widgets can require additional loading time. While these are often necessary, they need to be managed carefully to avoid excessive loading delays.
JavaScript can be a double-edged sword. While it enables advanced functionality, complex or poorly optimised JavaScript code can also hinder performance.
Single-Page Applications (SPAs) may experience slower initial loading due to their extensive scripting requirements, but they often offer faster navigation once loaded.
Busy Servers Handling Bot Traffic can also cause slowdowns. Bot traffic, in some instances, can account for over 40% of server load. Managing this effectively can help improve website speed.

Understanding which of these factors applies to your site helps you focus performance work where it will matter.

Continuous Monitoring and Performance Optimisation

Getting a site fast once is not enough. Speed can regress as content, third-party tags, releases, and traffic patterns change, so monitoring and performance optimisation need to be continuous.

Tools such as Google's Pagespeed Insights can help track website performance. Regular checks of key metrics can show which issues are slowing the site down and which changes need attention.

It is also important to test improvements on a staging website before deploying them to production. That reduces the risk of disrupting live performance or user experience. Regular diagnostic testing and iterative improvements help keep the site aligned with current performance expectations.

As SEO consulting company Moz highlights, focusing on continuous performance optimisation can have significant benefits. It can help maintain a fast, usable site and support higher search rankings, better engagement, and increased conversions and revenue.

Preparing for the Coming Recession

With an economic downturn on the horizon, a fast, well-optimised website becomes more important. Consumers are likely to be more selective with their spending, and businesses will need to compete harder for each sale.

A fast website can be a useful differentiator in this environment. It can boost your search rankings, making the site more visible to potential customers. It can improve engagement by giving visitors fewer reasons to leave. It can also increase conversion rates, which has a direct effect on sales.

In this context, website speed is not cosmetic. It is an operating requirement. The work is to measure the current experience, fix the main bottlenecks, and keep monitoring performance as the site changes.

The data and case studies point in the same direction: speed optimisation is a practical investment. It helps align the website with user expectations and makes the site a more effective part of the business.

Website speed is measurable, improvable, and commercially relevant. For businesses preparing for tighter conditions, it deserves active management rather than occasional clean-up.

Understanding HTTP Link Headers

2023-05-24T13:00:00+10:00

HTTP headers are part of browser requests and server responses. They carry information about the connecting client, the requested resource, the server, and other request context. An HTTP header has a case-insensitive name followed by a colon (:), then its value. Headers are used for authentication information, content negotiation, and related protocol behaviour. Here are some sample request headers sent by my browser when requesting a page on the Peakhour website:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-AU,en;q=0.9
Connection: keep-alive
Cookie: _ga_NRWSVE0PSC=GS1.1.1685943893.13.0.1685943893.0.0.0
Host: www.peakhour.io
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4.1 Safari/605.1.15

The HTTP Link header

An HTTP Link header lets a server send context about a document back to a client. It can identify related resources or the direct location of a specific asset. For page-load optimisation, Link headers can be an alternative to putting preload/preconnect/prefetch hints in the HTML.

History

HTTP Link headers were proposed as a standard in the late 1990s, around the same time the HTTP/1.1 protocol was defined. However, it wasn't until 2010 that HTTP Link headers were officially recognised by the Internet Engineering Task Force (IETF) in RFC 5988, which described their purpose and functionality.

Uses

HTTP Link headers have several uses in web development. Some common examples are:

Pagination: Say we have a blog site with hundreds of posts, and we display 10 posts per page. When a user requests a page, we can use Link headers to provide URLs for the next and previous pages. This helps navigation through the large list of posts. Here's an example of how it might look:
```
Link: </posts?page=2>; rel="next", </posts?page=4>; rel="prev"
```
Preloading: Suppose we have a heavy image or a large CSS file that we know will be required for a webpage. We can use a Link header to tell the browser to start downloading it early, improving the perceived page load speed. For instance:
```
Link: </images/big-picture.jpg>; rel=preload; as=image
```
Resource Hints: Link headers can give the browser hints about resources that might be needed in the future, so the browser can decide whether to fetch them ahead of time. For instance:
```
Link: </scripts/myscript.js>; rel=prefetch
```

Comparison to Link Tags and Why Headers Can Be Better

Now you might be wondering, "Why use Link headers when we can use HTML Link tags?" There are several reasons HTTP Link headers might be a better choice:

Faster Processing: Since HTTP Link headers are part of the HTTP response, they arrive before the HTML document. This allows browsers to start preloading or prefetching resources sooner, which can improve page load times.
Greater Flexibility: HTTP Link headers can be used in situations where HTML Link tags cannot. For instance, they can be used with file types that don't support HTML, like JSON or XML. They can also be added by a third party, e.g., an edge delivery layer such as Peakhour, without the need to parse and rewrite the HTML document.
Less Clutter: Link headers can make your HTML less cluttered, as you can avoid filling the HTML document with numerous Link tags.

Conclusion

HTTP Link headers are a small part of HTTP, but they are useful for performance and flexibility. An edge delivery layer can add Link headers without the overhead of parsing or manipulating the main document, which makes them useful for optimising website performance.

When Bots Break Bad

2023-05-16T13:00:00+10:00

Bots account for a large share of web traffic. Recent studies put automated traffic at nearly 50% of all internet requests. Some bots are useful, such as search engine crawlers that index your site. Some are clearly harmful, such as scrapers and sneaker bots. Others sit in a grey area, including backlink and marketing bots from services such as Ahrefs and SEMrush. Even useful bots can create problems when they crawl too hard. This article looks at the main bot types and how to manage them with robots.txt and bot management tools.

Understanding the Different Types of Bots

'Good Bots'

Good bots perform legitimate work. Search engine crawlers like Googlebot and Bingbot index webpages so search results can stay current and relevant. Other examples include uptime and performance monitoring bots.

'Bad Bots'

Bad bots harm websites, users, or both. Common examples include:

Scraping content, copying and repurposing data from websites.
Sneaker bots, automatically purchasing limited-edition products (like sneakers) before human users can.
Spam bots, posting unsolicited messages and advertisements in comment sections or forums.
Vulnerability Scanners, trying thousands of website URLs to find security vulnerabilities.
Account Takeover, attempting to gain access to existing user/admin accounts using either credential stuffing or brute-force attacks.

'Grey Bots'

Grey bots sit between good and bad. They often serve a useful purpose and may follow crawling directives in robots.txt, but they can still cause problems when they crawl too aggressively. Common examples include:

AhrefsBot: A backlink analysis bot used by Ahrefs, an SEO tool.
SEMrushBot: A bot used by SEMrush, another popular SEO and digital marketing tool.
MJ12bot: A bot used by Majestic, a service that provides backlink data and analysis.
ScreamingFrog: An SEO analyser run from a local desktop.

When Grey bots (and even Good Bots) go bad.

Left unattended, grey bots can create practical problems:

Slow page loading times, which affect user experience.
Strain on server resources, potentially causing crashes, downtime, and higher costs.
Distorted website analytics, when bot traffic is mistaken for human traffic.

Managing Grey Bots with Robots.txt

The robots.txt file is a simple text file that tells web crawlers which parts of your site they can or cannot access. You can use it to manage bot behaviour and protect your website from aggressive crawling. Useful controls include:

Disallowing specific bots: You can block specific bots from accessing your site by adding a "User-agent" and "Disallow" directive to your robots.txt file. For example:

User-agent: AhrefsBot
Disallow: /

Limiting crawl rate: You can ask bots to slow down their crawling by adding a "Crawl-delay" directive:

User-agent: SEMrushBot
Crawl-delay: 10

Not all bots will follow robots.txt. ScreamingFrog, for example, can be instructed to ignore robots.txt and crawl a site as quickly as possible. You would not want a competitor doing this to your site.

Bot Management Tools

In addition to robots.txt, bot management tools (like those provided by Peakhour) can protect your website from abusive bots. Good bot management tools automatically block most unwanted traffic using a combination of Threat Intelligence, Fingerprinting techniques, Reverse DNS verification, and Header Inspection.

Advanced techniques like rate limiting and machine learning can help identify more sophisticated bad bots.

Search Bots and Double Crawling

Search bots like Bingbot can sometimes blindly follow links and crawl the same page multiple times due to different URL parameters. This double, triple, or worse crawling can increase server load and make indexing less efficient. eCommerce sites are especially exposed because product catalogues often have several filtering paths. We've seen Bing go haywire on a number of sites. Most recently, it was issuing around 50,000 requests per day to the search function of a Magento 2 store while cycling through parameters. This dropped to 2-3k requests per day when fixed. On another store, Bing was responsible for nearly half of all page requests (40k page requests) on a busy OpenCart store. Configuring it to ignore parameters dropped this to around 4k per day.

Configuring Search Bots to Ignore Query Parameters

Note: Since publishing both Google and Bing have removed the ability to ignore parameters when crawling via their webmaster/search console tools. See using robots.txt to instruct search engines to ignore query string parameters

To help search bots crawl your site efficiently, you can configure them to ignore specific query parameters. Use these methods:

Configuring Bing Webmaster Tools

Bing Webmaster Tools provides an option to specify URL parameters that should be ignored during the crawling process. To configure this setting, follow these steps:

Sign in to your Bing Webmaster Tools account and select the website you want to manage.
Navigate to the "Configure My Site" section and click on "URL Parameters."
Click on "Add Parameter" and enter the parameter name you want Bingbot to ignore.
Select "Ignore this parameter" from the dropdown menu and click on "Save."
Configuring Bing Webmaster Tools this way helps stop Bingbot double crawling pages with specific URL parameters, reducing server load and improving indexing efficiency.

Managing Other Search Bots

For other search engines like Google, use the relevant webmaster tools to manage URL parameters. In Google Search Console, follow these steps:

Sign in to your Google Search Console account and select the property you want to manage.
Navigate to the "Crawl" section and click on "URL Parameters."
Click on "Add Parameter" and enter the parameter name you want Googlebot to ignore.
Choose "No URLs" from the "Does this parameter change page content seen by the user?" dropdown menu.
Click on "Save."
Specifying the parameters you want search bots to ignore can prevent double crawling and make indexing more efficient.

Final Thoughts

When good or grey bots crawl too aggressively, they can cause the same operational problems as malicious bots: overloaded servers, slower pages, and worse user experience. Monitor website traffic and server load, set clear robots.txt rules, and use the major search engines' webmaster tools to control inefficient crawling. Done properly, this improves website performance and can lower infrastructure costs.

Multi-Origin Load Balancing

2023-04-11T13:00:00+10:00

Websites and applications with global audiences often run into a problem that a standard CDN cannot always hide: not every request can be served from cache. Traditional Content Delivery Networks (CDNs) reduce latency by distributing content across multiple points of presence (POPs) around the world. But when content has to be fetched from a distant origin server, for example dynamic content that cannot be cached, users still wait on that round trip. This post explains the problem multi-origin load balancing solves and how Peakhour handles it.

Understanding the Problem

Web content usually starts from a single server, called the origin server. As a website's audience grows, that server takes more traffic, which can slow response times. CDNs reduce the pressure by caching and delivering content from servers distributed across multiple geographic locations. When the content is already cached, this reduces origin load and lowers latency for users accessing the content.

However, traditional CDNs still have limits when serving global audiences. If a user requests content that is not cached in the CDN, the request has to go back to the origin server. If that origin is far away from the user, latency increases and pages take longer to load.

Multi-origin load balancing addresses that remaining gap in CDN performance and further reduces latency.

Introducing Multi-Origin Load Balancing

Traditional load balancing distributes traffic evenly across two or more servers that are physically hosted in the same location.

Multi-origin load balancing extends that approach across origin servers in different geographical locations. The Peakhour EDGE can select the closest origin server to a user, or choose a different origin based on criteria such as device type, user preferences, or URL. For requests not stored in the CDN, this reduces the time spent crossing long network paths.

Real-Time Performance Monitoring: Peakhour continuously monitors its global network of servers in real time. This allows the system to detect potential issues or bottlenecks and adjust routing. If one origin server experiences high traffic or goes offline, Peakhour can reroute user requests to the next best server to keep the site responsive.

Adaptive Content Caching: Peakhour Edge uses adaptive content caching strategies, which dynamically cache both static and dynamic content based on user behaviour and request patterns. Frequently requested dynamic elements, such as personalised user data or search results, are cached on edge servers, reducing the need to fetch content from the origin servers and further minimising latency.

Load Balancing and Failover: Peakhour's multi-origin load balancing is complemented by load balancing and failover mechanisms. These features keep the system resilient and responsive during periods of high traffic or server outages. By distributing user requests evenly across origin servers and automatically redirecting traffic when a server fails, Peakhour maintains a stable and reliable content delivery experience for users worldwide.

Conclusion

Multi-origin load balancing addresses the limitations of traditional CDNs by optimising content delivery for a global audience. Peakhour Edge reduces latency and improves the experience for users who would otherwise wait on a distant origin server.

As web content and applications grow in complexity and reach, origin placement and routing become part of performance planning. Peakhour implements multi-origin load balancing with clients' origin servers so requests can be sent to the most suitable origin instead of treating a single distant server as the only fallback.

Automatic Image Optimisation

2022-12-06T13:00:00+11:00

Since releasing our automatic image optimisation capability in 2019, Peakhour optimises more than 35 million images each month. On average, clients reduce page download time by 35% and image sizes by 91%.

Automatic image optimisation offers a number of benefits over manual optimisation:

Image formats are dynamically chosen based on the content of the image.
Files are optimised based on client capabilities.
Designer time is saved by offloading image optimisation work.
Page load times can be reduced across an entire website.

What is Image Optimisation?

Image optimisation means delivering high quality images in the right format and resolution for the device viewing them. The aim is to minimise file size without compromising visual quality. Smaller image files generally load faster in a visitor's browser, which can improve website speed, reduce traffic and support higher page visits and conversions.

Why dynamically choose the image format?

Using a one size fits all optimisation process does not account for the different types of images used across a typical website.

Website images can include:

Photographs, such as product images, demonstration photos and title photos.
Logos, which are typically text and can include animation.
Illustrations that incorporate custom graphics to support the quick perception of information.
3D graphics from computer renders including buildings, interior and exterior designs and products.
Combinations of the above!

Any optimisation process needs to account for the end user's device type, supported file formats, resolution and the content of the image. For example, a photograph with text needs different optimisation parameters so the text remains crisp, while images with large areas of colour may need different formats to avoid colours being degraded by lossy compression.

Fine developer control

With the image optimisation API, your developers can enable reactive images by resizing images on the edge. Peakhour serves and caches these optimisations, reducing origin requests and traffic, cutting costs and making image changes faster to ship. You no longer need to wait for a designer to resize, crop or re-orient an image for a page.

How does image optimisation effect web application performance

Large portions of a website are typically built from images. Large images loading over a 4G network in a train can delay image rendering and create a poor user experience. Layout shifts can occur as the browser attempts to render the page, and the user's perceived page load time will be affected.

You can see the benefit of automatic image optimisation with our free tool. It will analyse any website, calculate its optimisation potential and let you download the optimised images.

See how it can help in a real life example below

security.png

Original

2362 x 1266px

PNG

190.2 kb

Optimised

605 x 324px

png

10.8 kb

jpeg

15.4 kb

[webp](/products/image-optimisation-and-transformation/)

10.7 kb

avif

8.2 kb

Benefits of Peakhour Automatic Image Optimisation

Peakhour image optimisation is integrated into your website so you can reduce traffic, speed up page loads and remove manual image preparation from the design workflow.

Transforming images on the Peakhour edge enables agility and offloads origin work.
User experience is improved through faster-loading pages, without a separate workflow for image optimisation.
Take advantage of new image formats, as supported by end users' browsers, to improve the page load experience.

Final Thoughts

Peakhour image optimisation runs on Peakhour Edge. Images are optimised as they pass through the Peakhour network, and optimised versions are cached for reuse. Peakhour image optimisation also works with Origin Shield, with the original and transformed images cached appropriately. This reduces origin hits and bandwidth while speeding up image delivery.

Prestashop 1.6/1.7 full page caching plugin released!

2022-08-04T13:00:00+10:00

Prestashop is a popular open source eCommerce platform written in PHP. We're releasing a Prestashop module that enables full page caching through Peakhour. Pages can stay cached at the Peakhour edge for longer, and the plugin refreshes them when the underlying content changes in Prestashop.

Full Page Caching reduces load times by removing the time it takes for a CMS to generate a dynamic page. That is typically around half a second for a quick page, and 3-5 seconds for a slow one. Here is a cache miss and hit from our client fatburnersonly.com.au:

A typical cache miss on a category page. The origin has to generate the page, taking 2.49 seconds

A cache hit: only 35.7 milliseconds to deliver to the client from our edge

Nearly 2.5 seconds removed from Time to First Byte and Largest Contentful Paint. That's without changing the site itself.

Tag Based Flushing

Like all our recent plugins, the Prestashop plugin adds a header with tag metadata for each cacheable page. This header is called X-Prestashop-Tag and contains the IDs of the relevant entities on the page, eg products, categories, brands, pages, etc. This metadata is stored alongside the cached page. When an entity changes in the Prestashop admin, eg a product price is updated, the plugin issues a flush-by-tag request to Peakhour. Peakhour finds the pages with the associated tag and invalidates them in the cache. The next request for the page passes through to origin and is then re-cached.

Headers returned by the [caching plugin](/blog/opencart/opencart-3-caching-plugin/).

Custom TTL

TTL (Time to live) is the time a resource stays in cache before the cache checks for a new version. You can control this within the plugin. The plugin then sets the peakhour-cdn-cache-control header, part of the new cdn-cache-control specification, so only Peakhour responds to the cache directive.

Ajax Mini Cart

Mini carts are dynamic sections of eCommerce websites, and they often stop pages from being cacheable. The Peakhour plugin loads them via Ajax so as many pages as possible remain cacheable.

The mini cart is in the top right; it is usually returned as part of the generated page.

Cache varying

The same page can show different information depending on factors such as whether a user is logged in, or whether a multicurrency store changes currency. The Peakhour Prestashop plugin handles this by changing a cookie value when those factors change, creating separate cache regions for the different possibilities.

The Results

The Peakhour Prestashop plugin can improve store performance measurably. Our client fatburnersonly.com.au improved their 'good' web vitals scores by 20% in the two months they've been using Peakhour.

Full page caching was enabled halfway through May. Note that a significant number of pages, eg checkout and admin, cannot be cached and are included in these stats

Grabbing the plugin

If you have a slow Prestashop store, see our plugin page or contact us for more information.

CDN Cache Keys

2022-05-16T13:00:00+10:00

Content Delivery Networks (CDNs) are common on business websites. CDNs speed up websites by storing website content and assets in caches on servers spread around the world. These caches are typically key/value stores. For example, let's say we are requesting an image found at:

https://www.example.com/someimage.jpg

Depending on how the CDN works internally, it may use /someimage.jpg (the key) to retrieve the image (the value) from its cache. Most CDNs are fairly rigid in what they use as keys, complicated to configure (eg requiring programming skills), or charge more for flexibility.

The Peakhour.io CDN cache keeps configuration flexible while still supporting fine-grained control of caching behaviour for your application.

The Peakhour.IO cache key consists of a primary key, sub key, and secondary key. Both the primary and secondary key must match to constitute a cache hit.

Primary key

The primary key is the key from the client request that uniquely identifies a resource. It consists of the scheme, host, path and query string from the request. Within our cache, the primary key may be augmented from other elements of the request, such as the presence of a particular header, or a header value such as a cookie value. At Peakhour.IO, we call these augmented keys cache subkey vars.

The following options can further manipulate the primary key by specifying how the query string is handled. These options include ignore query string (useful for defeating cache-busting techniques) or stripping certain tags (commonly UTM tags).

Secondary key

The secondary key describes parts of the request message that influenced the content of the response from the origin. This information is stored in the secondary key and the following headers are used:

Vary
Content-Encoding
User-Agent: (browser/mobile)
Accept-Language
Variant-06

Servers are commonly misconfigured and can send back headers that don't match the behaviour of the content. For example, it is very common for a server to send back a Vary header with Vary: user-agent. When you check the behaviour by sending different user agents, though, the content doesn't actually change. This can cause unnecessary cache fragmentation and lower hit rates. Peakhour allows you to override/ignore origin behaviour to correct this.

If you prefer a visual view, the diagram below shows all the elements of a request/response that could make up a cache key.

For more flexibility, Peakhour.io can use specific Cookie values in the secondary key. This is useful for Content Management Systems like Magento, which set a special cookie, X-Magento-Vary, to enable special caching behaviour that wouldn't normally be achievable; for example, whether the user is logged in, is part of a special group, or has chosen a different currency.

How can I see my keys?

Cache keys are generated on the fly based on the client's request and the origin's response. To see a cache key, enable Debug headers in the Peakhour.IO dashboard. As an example, for the given request/response pair:

GET / HTTP/1.1
Accept-Encoding: gzip, deflate, br

HTTP/1.1 200
Vary: accept-encoding
content-encoding: br

Peakhour.IO would generate the following cache-status header:

cache-status: peakhour.io; fwd=uri-miss; key="https://example.com/what-is-new.html"; secondary-key="encoding::br"; stored; ttl=31536000

Here, the key is the scheme://host/path and the secondary-key is the encoding served by the origin.

What's in it for me?

By understanding cache keys and how they are constructed, a web application can tailor its responses to better utilise a cache. The configuration options give you direct control for fine-tuning cache-key handling, improving user experience and cache hit rates.

Rate limiting

2022-05-16T13:00:00+10:00

Rate limits are a useful control for protecting a web application from abuse. When setting them for a web application, the key elements to consider are:

What endpoints on the web application require protecting?
Do different endpoints require separate handling?
What is the normal request rate for the entire application over a time period?
How many concurrent connections are typically used by your clients?
What errors does your API endpoint return in response to requests?

Before setting those policies, it helps to understand how rate limits protect an application from abuse or misuse, the types of attacks they can reduce, and how the rate limiting algorithm makes decisions.

What kinds of attacks are stopped by rate limiting?

When protecting an application with rate limiting, common attack scenarios include:

Brute force and enumeration attacks
Denial of Service (DoS) and Distributed Denial of Service (DDoS)
Site scraping

What else can rate limiting protect?

Public APIs and authenticated APIs can be subject to both abuse and misuse. Sensible rate limit policies can be applied on these endpoints to help prevent attacks and maintain service availability. Rate limiting can help protect these endpoints.

How does rate limiting work with user logins?

A well designed web application should allow only a limited number of failed login attempts before locking an account and requiring a password reset. This is designed to protect against brute force attacks against an account. Bots commonly attempt to brute force logins to WordPress and other popular web applications. Determined attackers can also attempt to brute force API login endpoints.

Rate limiting on a login page can be applied to the IP address of a user attempting to log in. By rate limiting by IP address, you can limit both password brute force attacks and simpler username enumeration attempts.

Using Peakhour.IO rate limiting, responses to requests can be monitored and IPs blocked for administrator-defined periods. This saves origin server resources and stops repeated attempts before they reach the application.

How could API rate limiting work?

APIs are ubiquitous across the modern web. Single Page Applications (SPAs) can be built almost entirely on REST or GraphQL APIs, while legacy applications often use form submits. Even when browsing this blog, you have consumed a range of APIs.

Because APIs are often publicly available, rate limits are commonly used to reduce abuse. Rate limiting for APIs can protect against malicious attacks. An attacker could script a bot to perform many API calls and make the service unavailable for other users, causing unplanned downtime - a layer 7 DoS or DDoS attack.

APIs

Public and private APIs can be subject to abuse or misuse. Public APIs are discoverable by anyone and can be scripted for data mining or attacks. Rate limiting these endpoints based on fair use policies is commonplace. Keeping track of this within an endpoint can be expensive, so handling it through Peakhour can offload that work from developers.

Overzealous 'good bots'

Peakhour has seen websites where up to 65% of requests come from automated bots. These bots are typically indiscriminate when mining information, and they do not carry the operational cost when your site slows down or fails. Rate limiting good bots separately from your main users helps ensure these crawlers do not stop your site from generating revenue.

How is rate limiting implemented?

Rate limiting is typically implemented using several common methods:

Fixed window

Window-based rate limiting is the simplest to understand. Fixed window limits are easy to define, such as 5,000 requests per 60 minutes. Fixed window rate limiting is subject to spikes at the edges of the window. For example, 5,000 requests in the first 5 minutes of the window may overwhelm a service.

Sliding window

A sliding window keeps much of the simplicity of a fixed window, but uses a rolling window. This allows bursts to be smoothed.

Token bucket

A token bucket is an algorithm where tokens are placed into a fixed-capacity bucket. Tokens could be defined as bytes transferred or hits to an API. When a request is considered for rate limiting, tokens are removed from the bucket. If the bucket has a sufficient quantity of tokens, the request can proceed. If there are insufficient tokens, the request is considered to be non-conforming. Non-conforming requests are dropped.

Leaky bucket

Leaky buckets are a mirror image of token buckets. Instead of removing tokens from a bucket, tokens are added to a bucket. Tokens are removed from the bucket (leaks) at a fixed rate. When a request is considered for rate limiting, it is compared to the number of tokens in the bucket. If the bucket is full, the request is considered non-conforming and is dropped.

If rate limiting is something you need to do to protect and secure your website, reach out to see how we can help.

Learn how Peakhour's Application Security Platform can improve your application's performance and security. Contact our team to get started.

Opencart 3 Full Page Caching Plugin Released

2022-03-25T13:00:00+11:00

Peakhour's Opencart plugin includes features that are usually reserved for enterprise plans with other providers. It follows our 'Enterprise for Everyone' approach in a practical way:

Tag-Based Flushing

Peakhour's plugin records metadata for each Opencart page in the cache. When you update a product or category, only the relevant pages are refreshed. That keeps cache flushing targeted instead of clearing more content than necessary.

Headers returned by caching plugin.

Custom TTL

Control how long a resource stays in the cache before it checks for a new version. This gives you a direct way to manage cache freshness.

Ajax Mini Cart and Wishlist

Dynamic sections like mini carts and wishlists usually prevent caching. The plugin loads these sections via Ajax, which makes more pages cacheable.

Cache Vary

The plugin adapts to different user states, currencies, and languages. It changes a cookie value to create separate cache regions for these variables.

Peakhour vs. LiteSpeed and Varnish Cache

LiteSpeed and Varnish Cache are good options, but Peakhour offers a more flexible and efficient caching solution for Opencart. The plugin makes Opencart as cache-friendly as Magento 2 or Drupal 8, if not more so.

The Results Speak for Themselves

Our client saw a clear improvement in their web vitals scores and website scalability. Full page caching was enabled in October, with the gains shown below.

Full page caching was enabled in October. Note a significant amount of pages, eg checkout, admin etc cannot be cached which affects these stats

For more information, visit our plugin page or contact us.

Cdn-Cache-Control

2022-02-28T13:00:00+11:00

Caching headers are easy to get wrong. Public resources may be cached in one layer, while the server configuration prevents an otherwise cacheable resource from being stored where you expect.

Controlling caching without a service like Peakhour usually means working directly with Cache-Control headers. This header has a wide range of fields and directives. Each one tells downstream clients (e.g., proxies, shared caches, and end browsers) how to handle caching for a particular resource.

Getting this right can be complicated, and it can become brittle when web server configuration also affects the response.

Enter CDN-Cache-Control

CDN-Cache-Control is a recently proposed header specified in https://datatracker.ietf.org/doc/html/draft-cdn-control-header-01

The draft RFC states:

This specification defines a convention for HTTP response header
fields that allow directives controlling caching to be targeted at
specific caches or classes of caches.  It also defines one such
header field, targeted at [Content Delivery Network](/learning/cdn/) (CDN) caches.

What does this mean in practice? Imagine a Magento application where pages should be cacheable by a shared cache but not by a browser. A Cache-Control header could look like:

Cache-Control: max-age=0, s-max-age=600

This tells browser caches not to reuse the response, while shared caches may cache it for 600 seconds.

What if the application does not want every shared cache to store the resource, because the application already sends purges to the cache when content changes?

Cache-Control does not give you that separation. The header above enables any shared cache to store the resource for the specified time. With CDN-Cache-Control, the application could instead send:

Cache-Control: max-age=0, s-max-age=60
CDN-Cache-Control: max-age=3600

This tells the browser not to cache the page, allows a shared cache to keep the page for 60s, and allows the CDN to keep it for one hour.

Targeted CDN-Cache-Control

What if you need to target a specific CDN only? Use the provider's targeted CDN-Cache-Control header.

For example, Peakhour-CDN-Cache-Control instructs Peakhour only to cache the resource. Other CDNs in the request path will not honour this header.

Header format

Examples

The following header fields instruct a CDN cache to consider the response fresh for 600 seconds, other shared caches for 120 seconds, and any remaining caches for 60 seconds:

Cache-Control: max-age=60, s-maxage=120
CDN-Cache-Control: max-age=600

These header fields instruct a CDN cache to consider the response fresh for 600 seconds, while all other caches are prevented from storing it:

Cache-Control: no-store
CDN-Cache-Control: max-age=600

Because CDN-Cache-Control is not present, this header field prevent all caches from storing the response:

Cache-Control: no-store

Whereas these prevent all caches except CDN caches from storing the response:

Cache-Control: no-store
CDN-Cache-Control: none

(note that 'none' is not a registered cache directive; it is used here to avoid sending a header field with an empty value, because such a header might not be preserved in all cases).

Cache-Status

2022-02-25T13:00:00+11:00

Diagnosing CDN caching can be complex, with multiple cache layers, Origin Shielding, and local caching. Understanding how these layers interact can mean spending weeks working through RFCs for the finer details of ETag, Cache-Control, and Last-Modified headers - before you even account for advanced controls that override them.

When you have a problem, or are trying to optimise caching, the first thing you need is visibility into how a request was handled.

This is where the new Cache-Status header helps. The new draft RFC, RFC draft-ietf-httpbis-cache-header-08, aims to:

  aide debugging by standardising the format of various non-standard debug headers
  used by major CDN providers. The semantics of these headers are often unclear and vary
  between implementations.

The draft RFC proposes a new header, Cache-Status, with a uniform format for showing how multiple caching providers interact and handle a request. The Cache-Status header forms a list. Each member of the list represents a cache that has handled the request, and the last member belongs to the cache that most recently served the user. The header is only applicable to responses directly generated by an origin server. Each member can add a parameter indicating how it handled the request.

A Cache-Status header looks like this:

  Cache-Status: OriginCache; hit; ttl=1100, "CDN Company Here"; fwd=uri-miss;

It is formatted and labelled, with each cache in the line separated in list format by a ','.

The format of the Cache-Status header is a list comprising of the following possible parameters:

hit = boolean
fwd = (bypass, method, uri-miss, vary-miss, request, stale, partial)
fwd-status = integer
ttl = integer
stored = boolean
collapsed = boolean
key = string
detail = token / string

What the parameters mean

Hit

The hit parameter signifies that a request was satisfied by the cache. It is a boolean parameter meaning that its presence indicates a cache hit.

For example:

  Cache-Status: Peakhour.IO; hit

Fwd

The Fwd parameter indicates when a response was forwarded to an origin server - and why. The Fwd parameter contains one of the following arguments:

bypass - The cache was configured to not handle this request
method - The request method's semantics require the request to be forwarded
uri-miss - The cache did not contain any responses that matched the request URI
vary-miss - The cache contained a response that matched the request URI, but could not select a response based upon this request's headers and stored Vary headers.
miss - The cache did not contain any responses that could be used to satisfy this request (to be used when an implementation cannot distinguish between uri-miss and vary-miss)
request - The cache was able to select a fresh response for the request, but the request's semantics (e.g., Cache-Control request directives) did not allow its use
stale - The cache was able to select a response for the request, but it was stale
partial - The cache was able to select a partial response for the request, but it did not contain all of the requested ranges (or the request was for the complete response)

For example:

  Cache-Status: Peakhour.IO; fwd=uri-miss

Fwd-status

The fwd-status parameter indicates the status code the next hop returned in response to the request. It is only meaningful when "fwd" is present. For example, a complete miss would look like fwd=uri-miss and the HTTP status code of the downstream response would be supplied in the fwd-status:

The following example shows that the cache did not satisfy the request and that the downstream server indicated an HTTP 304 response - HTTP Not Modified.

  Cache-Status: Peakhour.IO; fwd=uri-miss; fwd-status=304

ttl

Each cache item is associated with a lifetime, referred to as the Time To Live (TTL). A TTL is in seconds and indicates the remaining freshness of the resource. This value is calculated by the cache.

For example, a cache hit showing that the resource has a freshness of 376 seconds.

  Cache-Status: ExampleCache; hit; ttl=376

Another example shows a cache hit with negative freshness - a stale resource.

  Cache-Status: ExampleCache; hit; ttl=-412

stored

Indicates whether the received response was stored by the cache. This example shows a cache miss and the cache storing the response for the next hit.

  Cache-Status: ExampleCache; fwd=uri-miss; stored;

collapsed

Indicates whether the received response was collapsed with another request.

key

The key is the lookup index into a cache. Cache keys convey a representation of how the cache will look up the resource used for the response. The cache key is implementation-specific.

This example shows a cache hit, the cache key and secondary key, and the remaining TTL of the resource.

  cache-status: peakhour.io; hit; key="https://example.com/calendar.css"; secondary-key="encoding::gzip"; ttl=30674859

detail

Allows additional implementation-specific information not captured by other parameters.

Multiple layers of caching

The header allows multiple layers of caching. For example, a global CDN may sit in front of a local varnish server. Each cache appends its cache-status to the header, so the last item is the closest cache to the actual application and the first item is the closest cache to the accessing user.

  Cache-Status: OriginCache; hit; ttl=1100, "CDN Company Here"; hit; ttl=545

Security

There are security implications to consider with this header. Making it public gives an attacker insight into how the cache is configured, which may help them bypass a cache entirely and access an origin server directly. Cache header security is provider-specific, but could involve enabling it only when required, restricting access to specific IP addresses, or requiring a custom request header to trigger the cache-status header.

Peakhour.IO regularly sees automated scans sending other providers' headers to trigger a cache-status header response.

Conclusion

The Cache-Status header gives clear insight into how cache layers interact on a website. It provides a standardised format for showing how a cache handled a request and allows multiple caches to report their part of the request path.

Peakhour fully supports this new header, and it can be enabled/disabled and secured in the dashboard.

Website Optimisation

2021-05-17T13:00:00+10:00

We've previously covered ways to find resources that block rendering in a browser. Now we'll cover techniques for loading those resources without letting them block rendering. Before making changes, always run some website performance tests using your favourite testing tool to get a baseline so you can measure any improvements. We recommend testing with webpagetest.org.

Before we get into the techniques, it is worth defining the problem.

What is a Blocking Resource?

When a browser loads a page from a website it reads the HTML markup from top to bottom. If it encounters a CSS or JavaScript file while it's reading, the browser stops while it downloads and parses that file. While the browser is waiting it can’t process the rest of the HTML or display content to the user. Rendering is blocked.

Why is this a problem?

It would not be a problem if every page included only the information required to display it. Most websites do not work that way. They are typically built using prebuilt themes and third-party libraries that can contain lots of CSS and JavaScript. Only a tiny fraction of this code might be used on the current page, or it might only be needed for ‘below the fold’ content, or on another page of the site. Note: Below the fold means the content of a page that doesn't initially fit on the screen. Any unnecessary information keeps the browser blocked for longer than needed, slowing the load experience for the user.

CSS and JavaScript files are usually at the very top of the HTML document before any of the content. This means they have to be downloaded and parsed before the browser can display anything. Even a website that has extremely fast server response times can still seem very slow if it has blocking resources.

Take a simple example: a website that includes a live chat widget. The JavaScript for the widget is included at the very top of the page in the section. The browser has to stop, download, and parse the CSS and JavaScript for the widget (usually downloaded from a third-party domain) before any content is displayed to the user. The chat widget could appear after the rest of the content without affecting the user experience.

Things you should always be doing

Before targeting CSS- or JavaScript-specific techniques, cover the basics for both.

1. Minify all files

Minification is the removal of all non-essential characters in a file, e.g. irrelevant whitespace and code comments. It can make a substantial difference to the file size.

2. Ensure compression is enabled on your server

Make sure your server is configured to use either gzip or Brotli compression when serving CSS and JavaScript.

3. Self host files

If the CSS or JavaScript is on a third-party domain, i.e. not the domain name your website is on, you should strongly consider uploading the file to your website and serving it from your domain. This might not always be possible, but if it is it eliminates the overhead of the browser opening a connection to another domain, and it removes the possibility that the third party is not compressing, minifying, or using HTTP/2.

After the implementation of cache partitioning in the major browsers, there is no longer any potential advantage to using third-party CDNs to serve assets like JavaScript, fonts, or CSS. Make a copy on your server and host them on your domain.

4. If you can't self host then preconnect to third party domains

If external assets are on third-party domains, and you absolutely can't self host them, then you can improve loading by telling the browser to preconnect to the third-party domain.

Using preconnect tells the browser to establish an early connection to the domain before it has discovered the asset while reading the HTML. Use preconnect in the head of the HTML, e.g.:

Establishing early connections can shave 100-500ms off third-party load times, which is worth having. You should only use preconnect on critical third-party resources. For all the others you can use dns-prefetch, e.g.:

This tells the browser to perform DNS resolution of the domain early, which is similar to looking up a phone number in the phone book. Pre-resolving DNS can save 20-120ms. That sounds like small numbers, but remember, differences as small as 100ms can have large measurable impacts on conversion rates.

5. Select only what you need from frameworks

It is common for users of frameworks like Bootstrap or jQuery UI to only use a fraction of the functionality provided. If this is you, then the first step is to cut out what you're not using. Bootstrap and jQuery UI are modular, you can either download everything in one bundle, or you can break it up to take just the bits you are using.

These points will ensure that blocking resources are downloaded as quickly as possible. Now we can discuss some specific techniques that will help minimise blocking and get your page displaying quickly. Please be aware that this is not an exhaustive overview, but these techniques will probably get you 90% of the benefit with 10% of the work. Getting that last 10% requires deeper technical work. If you want to go there, Google’s web.dev website is a good resource.

Techniques for Optimising Blocking Resources

Javascript

There are a few different techniques for optimising the loading of JavaScript. First, let's look at the default loading behaviour. Credit for the images goes to Daniel Imms and his website Growing with the web. As stated earlier, the default behaviour is to stop parsing the HTML document when a script is encountered, download the script and execute it.

Now let's look at how we can change this behaviour.

1. Moving scripts to the very bottom of the page, right before the closing tag.

This is the original optimisation technique, before defer and async were introduced. It works by moving scripts to the very end of the HTML document, where they are downloaded and parsed after everything else.

In our example in the previous section, the chat widget would now load and pop up after the rest of the content has been displayed to the user.

2. Defer and Async

<script> tags that have a src attribute can be marked as either deferred defer, or asynchronous async, or both. For example:

<script src=”https://www.somedomain.com/somescript.js” defer async>

This tells the browser to change the script loading behaviour.

Defer

Defer was the original browser-based support for improved JavaScript loading. It tells the browser to download the script asynchronously while it keeps reading the HTML, and then execute it once it has finished reading the whole HTML document. Here's the loading behaviour with defer enabled.

As you can see the script no longer blocks the browser from doing anything else while downloading. Defer preserves script execution order. This can be very important if a script depends on one declared earlier in the page.

As noted earlier, you can only mark <script> tags that have a src attribute as deferred. Inline scripts, e.g.:

<script type="text/javascript">
    console.log("Hi I'm some inline script!");
</script>

Any attempt to defer that inline block will simply be ignored. This can cause problems if you have inline script scattered through your code and rely on a deferred third-party library, e.g. jQuery. There are two potential solutions if you can't simply move the code to the bottom of the page:

Move each inline code block into its own file and include it using the src attribute so you can defer it.
Or you can try declaring the block as a module. Test this carefully, as browser support may be limited and it also places the script into strict mode, which may break it.

Async

The async attribute tells the browser to download the script now and execute it as soon as it has finished. Like defer, the downloading is done asynchronously. Unlike defer, the script is executed as soon as it is downloaded. Here is the behaviour:

As you can see the browser doesn't stop while downloading, but does pause once downloading is finished so it can execute the script. async doesn't preserve script order, which can potentially cause issues when there are script dependencies.

All three methods have pros and cons. Defer and async result in faster page loads overall as the scripts are downloaded while the browser reads the HTML. Moving the script to the bottom of the page shifts the sequence around.

Your first option should be to defer all scripts. Then, if some above the fold content is taking too long because it has a JavaScript dependency, e.g. a carousel in the hero section, you can try making the necessary scripts async instead.

Optimising CSS

The key to fast CSS loading is to prioritise the CSS needed for the immediate above the fold content and then defer the rest. Many articles advocate extracting the precise CSS and then including it inline in the HTML document in a style block, i.e.:

<style type="text/css">
     .accordion-btn {background-color: #ADD8E6;color: #444;cursor: pointer;padding: 18px;width: 100%;border: none;text-align: left;outline: none;font-size: 15px;transition: 0.4s;}.container {padding: 0 18px;display: none;background-color: white;overflow: hidden;}h1 {word-spacing: 5px;color: blue;font-weight: bold;text-align: center;}
</style>

However, you have to do this on every possible landing page where the critical CSS might be different per page. Inlining CSS also adds to the download size for each page, and forces the browser to reparse the CSS rules for every load rather than being able to use a cached file for repeat views.

We advocate for putting the critical CSS into its own file and loading that normally, and then deferring any other non critical CSS. Identifying the critical CSS is the main task. The first thing to do is to look at the included CSS on your website and see if you can identify any non-core files. You can defer those files using this pattern:

<link as="style" href="/styles.css" onload="this.onload=null;this.rel='stylesheet'">
<noscript><link rel="stylesheet" href="/styles.css"></noscript>

The as="style" lets the browser download the file asynchronously. The onload changes the type to stylesheet, telling the browser to parse the file. Finally, we include a <noscript> to enable the CSS to load normally if JavaScript is turned off.

Once you've deferred non-core CSS files you can still check whether your core ones contain lots of unused rules. Google Chrome's coverage tool can show unused rules, but it doesn’t let you easily export used and unused rules. You have to go through it yourself, programmatically, or use a third-party tool to make two files: the critical CSS, and the non-critical CSS which can be deferred.

Conclusion

Eliminating, or at least minimising, blocking resources can be one of the biggest improvements to user experience you can make to your site. With Google Web Vitals being introduced soon as a search signal, it is important to make sure your website loads as fast as possible so your users stay longer and buy more from your site.

What is the Chrome UX Report (CrUX), and why should you care?

2021-02-26T13:00:00+11:00

A faster website is better for clients: they buy more, and they engage more with your content. However there's someone else that rewards fast websites: Google. Fast websites rank higher in organic search results than slower websites. They will also achieve higher quality scores in Google Ads, resulting in lower ad spend.

We've previously written about Google's Web Vitals. The practical question is where the real-world data comes from. It is not gathered by Googlebot. Google uses field data from the Chrome User Experience Report, usually shortened to CrUX, to show how eligible Chrome users actually experienced a page or origin.

That matters because a fast lab score is not the same thing as a fast customer experience. WebPageTest and Lighthouse help diagnose a controlled test run. CrUX shows the field data behind PageSpeed Insights, Search Console's Core Web Vitals report, and Peakhour's website speed comparison tool.

Introducing the Chrome UX Report (CRuX)

The CrUX report is a public data set of real-user measurements (RUM) of website performance across millions of sites. The report has been around since 2017 and is updated regularly, but the value is still often missed: it shows what real users experienced, not what a synthetic test predicted.

The data is collected from real Chrome browser users who have opted in to send browsing information back to Google. This opt-in requires that the user has:

Opted in to syncing browser history
Not set up a sync passphrase
Usage statistic reporting enabled

Despite these conditions, millions of Chrome users still report statistics back to Google. A given website still needs to be fairly busy before there are useful statistics in the report.

Gathered Metrics

The current Core Web Vitals are:

Largest Contentful Paint (LCP): how quickly the main content appears.
Interaction to Next Paint (INP): how responsive the page is to real user interactions.
Cumulative Layout Shift (CLS): how visually stable the page is while it loads.

CrUX also exposes supporting performance metrics and dimensions that help explain those headline scores. Older metrics such as First Input Delay (FID) still appear in older reports and tools, but INP is now the responsiveness metric to watch.

Dimensions

Because performance can vary widely, the metrics are divided into the following dimensions to help segment and understand the user experience.

Country
Device Type: Tablet, Phone, Desktop
Connection Speed: slow 2g, 2g, 3g, 4g, or offline

Viewing data in the report

There are several ways to see how your website performs in the report. These include:

Pagespeed insights

Google's website analysis tool provides summary CRuX data for the analysed URL and, if data is available, for the entire site.

Page Speed Insights Field Data" style="max-width: 100%;margin-bottom: 20px"/>

Google BigQuery

The most flexible option is to access it directly via BigQuery. You query it with SQL (database query language).

The downside is that you need to understand SQL and have a Google account.

Google's Search Console (formerly Webmaster Tools)

The search console now has a section 'Core Web Vitals' that shows whether URLs pass the Core Web Vitals, as well as a historical graph of performance for both mobile and desktop.

Looker Studio

Looker Studio, formerly Google Data Studio, can be used to build dashboards on top of CrUX data and other sources. It lets you visualise the performance of your website, or a competitor's website, over time.

Third party tools

Like our own website speed comparison tool. It uses the Chrome UX API to retrieve the information.

CrUX vs lab tools vs RUM

CrUX is field data. It is useful because it reflects real Chrome users, but it only reports where there is enough eligible traffic. It can also be slower to reveal the cause of a problem because it is aggregated.

Lab tools such as WebPageTest are better for diagnosis. They show waterfalls, redirects, blocked resources, caching issues, image weight, and third-party requests. Your own real user monitoring can go further again, because it can include business context that CrUX does not know: customer type, template, campaign, cache state, bot pressure, origin load, and release timing.

Conclusion - Why you should care

The data in the Chrome UX Report is one of the clearest public views of how Google sees the performance of your website. It is also a free source of real-world user measurements that helps you understand how visitors experience your pages.

Use CrUX to see whether your site is passing Core Web Vitals, use lab tools to find the technical cause, and use traffic visibility to understand whether bots, crawlers, bursts, cache misses, or expensive requests are putting the experience under pressure. If the first question is "how do we compare?", start with the website speed comparison tool. If the question is "what is slowing us down?", start with Traffic Management.

The CDN is Dead, Long Live the CDN!

2021-02-17T13:00:00+11:00

It has long been conventional wisdom that if your website uses a third-party library, such as jQuery or Bootstrap, you should load it from the library's high-performance CDN. For example, jQuery has

code.jquery.com

from which you can include any version of jQuery on your website, e.g.:

<script src="https://code.jquery.com/jquery-3.3.1.min.js"
          integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8="
          crossorigin="anonymous"></script>

This was meant to give you two performance benefits: the browser could load jQuery from a global CDN, AND if another site the visitor had already opened included the same jQuery file in the same way, the file might already be in their browser cache. In that case the browser could reuse the cached copy and the page would load faster.

In practice, website visitors rarely see much benefit. The browser still has to open a separate connection to the third-party CDN, which can cancel out any latency advantage the CDN provides. There are also so many versions of major libraries in use that the chance of two websites using the same version, AND both loading it from the CDN in the same way, is small.

Cache Partitioning in Chrome and Firefox

Until recently, Chrome and Firefox used a shared browser cache for all websites that a user visited. This means that if you visited a website and it loaded this resource:

https://www.somesite.com/foo.js

and you then visited a second website that included the same resource, the resource would be loaded from the shared cache rather than being downloaded from the internet a second time. Cookies set by these resources would also be shared.

As of Firefox v85 and Chrome v86, the browser cache will be partitioned. This means that the same resource included on two sites will have to be downloaded from the internet twice and stored separately.

Why are they doing this?

The main reason is privacy. Shared browser caches have been used by unscrupulous operators to track users across different websites without consent. They do this by utilising cache side-channel attacks.

What Are Side Channel Attacks?

In computer systems, a program or algorithm can be correct and secure in itself, while its interaction with the computer still leaves information that an attacker can exploit. A useful analogy from the spy world is using a laser beam to measure the vibrations on a pane of glass, so you can hear a conversation inside a room that you otherwise could not hear.

Side-channel attacks can be ingenious, utilising techniques such as:

observing power consumption/electromagnetic radiation;
timing data moving in and out of memory;
timing how long a CPU takes to execute an instruction;
measuring sounds emitted by a hard drive.

For the browser cache, a simple attack starts with the user opening a malicious website. That website then requests resources (e.g., an image) from another site. By timing how long the browser takes to load that image, it can determine whether it was fetched from the browser cache or had to be downloaded over the internet. According to Google, this technique can be used to:

Detect if a user has visited a specific site.
Detect if an arbitrary string is in the user's search results by checking for 'no search result' images used by particular sites.
Track users across sites using the cache.

GitHub user terjanq has published a cache side channel attack that can gather all sorts of information from Google services. He states that a regular Google user could have the following information leaked:

search history
videos watched
the exact URLs visited
time frames of the activities
private book collection
books read / purchased / bookmarked / favourite / etc.
private emails
tokens / credit card numbers / phone numbers / etc.
contacts (including email addresses, names, phone numbers)
bookmarked websites
and more.

Perhaps it was simpler to implement cache partitioning in Chrome than to keep patching these vulnerabilities...

What you should do

Google estimates that the changes in browser caching will have minimal impact, as little as a 0.3% difference in FCP (First Contentful Paint). However, since there will no longer be any shared-cache benefit from hosting on a third-party CDN, you should:

Host all third-party libraries on your own domain to eliminate the need for the browser to establish a second connection.
Use a delivery layer such as Peakhour Edge, which transparently caches and serves your website assets close to users.

Conclusion

Some third-party resources really are shared across millions of websites. Google Fonts is the obvious example. Under the new cache partitioning implementations, these fonts will have to be downloaded for every site that uses them. That has a cost for site speed and data usage. It also has the side effect of improving Google's ability to track users, because each font request tells Google another site the user has visited.

Safari has been partitioning the HTTP cache since 2013, leaving Microsoft's Edge as the last major browser with global HTTP caches. However, future versions will be based on Chromium (the open-source version of Chrome) so should get cache partitioning by default.

Secure Dynamic Content Caching

2021-02-09T13:00:00+11:00

Last time we covered how slow server performance can have a negative, sometimes very negative, effect on your website load times. The causes of slow server responses are many and varied, and not necessarily tied to the server specification. Diagnosing and dealing with them can be difficult, time-consuming, and costly. One practical way to reduce the impact is to look for opportunities to cache dynamic content.

What is Dynamic Content?

Content delivered by a web server is categorised as either static or dynamic. Static content is the same for every user and is delivered without being generated or processed by the server. Static content is fast to deliver and does not tax a server's resources.

Dynamic content is generated by the server for every request. That can involve querying a database several times and executing a large amount of code. Depending on the work required to generate the result, dynamic content can be resource-heavy.

Traditionally CDNs have only cached and served the static content of websites, usually the images, CSS files, Javascript files, etc. They required the webmaster to upload these resources to the CDN's servers and to modify their website source HTML to access the resources from the CDN. For example,

https://www.yourdomain.com/image1.jpg

would be changed to be something like

https://cdn.yourdomain.com/image1.jpg

Modern CDNs, like Peakhour, act as a reverse proxy{:target="learning"}, which means they sit between the end user and the website's origin server. This enables them to transparently cache a copy of anything cacheable being returned by the origin server. By transparently caching, we mean the caching happens without changes to the original website.

Full Page Caching

Full Page Caching (FPC) is where the actual HTML document of a web page is cached.

Most websites are built using a Content Management System (CMS). Widely used CMS platforms are Wordpress, Drupal, Magento, etc. By default, every time a page is viewed the CMS has to generate the content from its database. Most of the time this generation is unnecessary. The content is either the same for every user, changes very rarely, or only differs by a small amount of personalisation. In each case it is possible to perform Full Page Caching to improve page load times and to cut server load. Let's have a look at the difference full page caching makes to one of our clients, Magento 2 store savvysupporter.com.au.

Main document load before caching: 2.07s

Main document load after caching: 82ms!!

Caching the page has cut nearly 2 whole seconds from the download time. That matters: load differences as small as 100 milliseconds have measurable impacts on website conversion rates. With a Magento 2 website, it's possible to cache all full pages outside the checkout process and the customer/admin area. This can reduce load on the origin in the order of 60-70% and make the customer experience much better.

API (Application Programming Interface) Caching

Many API calls used by web applications are for the retrieval of information to be displayed to the end user. Examples include:

Pricing and product information
Form auto completion
Product catalogue searches

These are all strong candidates for caching, reducing load on your server and making websites faster.

Handling Stale Information

The one potential drawback of caching dynamic content is the possibility of returning out of date information to the end user. There are two strategies to deal with this risk.

Setting a short Time To Live (TTL) with the caching provider. By only keeping content cached for a short time, e.g. 5 minutes, the cache will never be without up to date information for very long. Cached content will expire and the new version fetched from the origin server. The drawback with this method is that, unless your site is very busy, cache hit rates can be low, users can frequently get slow loading pages, and stale content is still possible. However, for very busy sites this can be an effective, low-overhead strategy.
Flushing content when it changes. This strategy sets very long time to live in the cache, months or even years. When content is updated the cache is informed and the new version fetched. This notification of new content could happen manually or, in the case of some CMSs, automatically. For example, Magento 2 and Drupal 8 have a built in framework for integrating caching providers to handle flushing when content/stock changes. This strategy ensures very high hit rates, but unless the flushing is accurate and fast it can result in stale content.

Security-Performance Integration

Modern Application Security platforms like Peakhour combine caching with comprehensive security controls so performance optimisation does not weaken application protection:

Edge Security Processing

WAF/WAAP Integration: Security rules are processed at the edge before content is cached, ensuring malicious requests never reach your origin
Bot Management: Caching adapts based on traffic classification - legitimate users benefit from cached content whilst malicious bots are filtered out
API Protection: Secure caching of API responses with appropriate security headers and access controls

Cache Security Controls

Secure Purging: Authorised cache invalidation through secure API endpoints with proper authentication
Content Classification: Different caching policies for public, authenticated, and sensitive content
Header Security: Automatic injection of security headers (CSP, HSTS, etc.) into cached responses

Conclusion

Secure dynamic content caching works best when performance optimisation and application security are handled together. By implementing caching within an Application Security Platform, organisations can achieve:

Superior Performance: Dramatic improvements in load times and server capacity
Enhanced Security: Protection against threats at the edge before they impact cached content
Operational Efficiency: Reduced origin server load whilst maintaining security posture
Cost Optimisation: Lower infrastructure costs through intelligent caching and edge processing

For modern applications and APIs, secure dynamic caching should form part of the performance and security strategy, improving the user experience whilst maintaining threat protection.

Common Issues That Affect Website Performance

2020-11-30T13:00:00+11:00

Our last three website performance articles covered the why and how of testing website performance, and introduced our two favourite performance testing tools, WebPageTest.org, and Lighthouse. This article covers the common causes of slow loading times, and how to spot them using the same testing tools.

1. Latency

Latency is the time it takes for a request from a client's browser to traverse the internet to reach the website server. A number of factors affect latency, with physical distance usually the main one. Data on the internet travels at the speed of light, so distance may not sound like a major concern. In practice, small delays compound quickly. Here are some realistic examples of request latency:

Sydney to Melbourne: 5ms
Sydney to Perth: 25ms
Sydney to San Francisco: 75ms

NOTE: Common tools used to determine latency between servers are Ping and Traceroute, they will provide you with Request Round Trip (RTT) latency, ie the time it takes for a request to get to the server and back. So 2 times the numbers above.

Here is an example of latency during a webpage load. We'll ignore internet speed and any potential network congestion, and focus only on latency. We'll request a website hosted in San Francisco from a browser located in Sydney.

Browser establishes a TLS (Secure) connection with server, 6 * 75ms = 450ms
Browser sends HTTP Request for the main page = 75ms
Server sends HTTP Response containing page which specifies 10 assets = 75ms
Browser requests the 10 additional assets = 75ms
Server responds with 10 assets 10 * 75 = 750ms

In this example, a website with 10 assets, which is a fairly simple page, has nearly 1.5s added to the page load time through latency alone. If the same site was hosted in Melbourne, the time due to latency would be just 95ms. To solve this problem you would either move your website server closer to your customers, or use Peakhour Edge caching.

How do I check for latency problems?

If there is a long distance between your server and your customers, eg your website is hosted in the US and your customers are in Australia, page load times will be significantly affected by latency. A good tool for indicative RTT times between cities is provided by wonder network. Webpagetest.org also provides a Traceroute tool which can give you indicative RTT times between its testing locations and your website.

2. Old version of HTTP

Browsers communicate with websites using a protocol called HTTP. The protocol formalises the steps needed to connect a browser and server so a webpage can be downloaded. Since the introduction of the web, HTTP has gone through a number of revisions. The currently widely adopted version is HTTP/2.

Even though HTTP/2 was introduced over 5 years ago, over 50% of websites are still only served over HTTP/1.1. Without getting too technical, HTTP/2 has significant advantages over older versions because it reduces the number of connections between a browser and server, and transfers information more efficiently.

Serving your website over HTTP/2 can improve page load times by 10-15%.

How do I identify the version of HTTP my website is served over?

Run a performance test of your website using WebPageTest.org, once complete click on the 'Details' link in the report navigation. You will see the Waterfall View. Click on the first request, circled below in this image:

It will bring up the request details including the Protocol used, circled below:

3. Slow Server Performance

A slow initial response from the server is a common cause of poor page load performance. The majority of websites are built using some sort of Content Management System (CMS), eg Wordpress, Magento, Drupal, Shopify, to name a few. By default, a CMS has to construct a page each time a browser requests it, even when the content has not changed. That process can involve executing a lot of code and querying a database several times before returning the HTML that forms the page.

The specification of your server, the number of CMS plugins, the state of your database, and the number of simultaneous users can all affect the response time. A slow server may take 10s or more to respond, while even a fast server can still take over a second to generate a page. That is enough to pretty much make you fail the new Core Web Vitals guidelines before you even get started.

How do I check for slow server performance?

Server performance can be checked from the waterfall view in WebPageTest.org. The time taken to download the main document indicates whether server performance is affecting your page load. Here's an example:

The total time taken to download the main document here is over 1.3s. That is not too bad, but it has already used up over half the 2.5s required to achieve 'Good' for the Largest Contentful Paint (LCP) metric in web vitals

4. Page Weight

Even though internet speeds are getting faster, they are still a limiting factor to how fast a browser can download a page. If the requested page and the associated resources, eg images, javascript, CSS, are large files, it will take longer for the browser to download all the required information to display a page. Unoptimised images are a common culprit for inflating page weight. Unoptimised CMS themes and third party javascript libraries are another.

How do I check my page weight?

WebPageTest.org reports the page weight in the far right of its summary, circled here:

You should be aiming for 2mb or less. This particular website is more than a little obese, coming in at around 30mb... WebPageTest also has a section called 'Content Breakdown' which shows where the weight is, ie in images, javascript, etc.

Over 27mb in images is what's weighing down this page.

Peakhour also has a pageweight tool that you can run for a page on your website and receive a full optimisation report for the images on the page, along with downloadable optimised images.

5. Blocking Resources

Resources that can block a page include CSS and javascript. When the main HTML page is downloaded and parsed, the browser will not render anything to the screen until the CSS and javascript files that are referenced are downloaded and parsed. If your website includes a lot of CSS and javascript, which is not uncommon for pre built themes for CMS's like Wordpress and Magento, the downloading and parsing of these files can delay the browser from showing any content for several seconds.

How do I identify blocking resources?

The easiest way to check is to use Google Lighthouse) to identify them for you. After running the report, go to the opportunities section and expand the 'Eliminate render-blocking resources' section to see what it finds.

6. Third Party Resources

It is very common for websites to include third party resources. These are files that have to be fetched from a domain/url other than the one that the page is being requested on. Eg if you are requesting www.domain.com.au, it might include a resource from a third party, eg www.anotherdomain.com.au. Common third party resources might be analytics scripts (eg Google analytics), marketing tools (eg Mailchimp)

This forces the browser to open another connection to the third party. The time taken to do this, combined with the possibility that the third party might be slow to respond (see latency and server performance above), can often ruin load times. If the resource in question is also a blocking resource the problem is compounded.

Spotting a problem with third party resources

WebPageTest has a section 'Domains' which displays all the individual domains that the browser connected to when loading the page:

If your website is requesting resources from more than 10 separate domains then you should consider why that is happening and whether they are necessary. If you are using external CDNs to load javascript or CSS then you should probably move them onto your domain.

Social sharing plugins are notorious for pulling in a lot of resources from external domains. You should replace any share buttons using external scripts with simple static buttons. It is simple to do, and your website will be much smaller and faster. By using the javascript shares that social sites prescribe, you are slowing down your website and allowing third parties to track your clients across the internet.

Conclusion

There are many factors that can slow down your website, drive away customers, and cost you money. You have to regularly test your website to check for issues and address them when you find them. Next we'll cover what you can do to fix the issues we've mentioned here.

Are Australian Magento Stores Ready For Web Vitals?

2020-11-19T13:00:00+11:00

Google recently confirmed that the Core Web Vitals will be included as search signals from May 2021. This means that, all else being equal, sites that score well on the Core Web Vitals are likely to rank ahead of those that don't.

A quick refresher of Web Vitals

The Core Web Vitals consist of three metrics, chosen to measure the experience of browsing a website. Here they are, along with the current thresholds for a 'Good', 'Needs Improvement', or 'Poor' rating:

Web Vitals also defines several other metrics, including Time to First Byte (TTFB), and First Contentful Paint (FCP). While these aren't 'core' metrics, they are useful for diagnosing where performance problems come from. The target TTFB in the current version of Google Lighthouse is listed as 100ms, while poor is 600ms. FCP is good under 2s and poor over 4s.

How will Australian sites fare?

We asked a simple question: what percentage of Australian websites are ready for Web Vitals as a search signal, and what percentage could lose ground? To answer it, we ran them through our recently released Website Speed Comparison tool, which gathers Web Vitals metrics as part of its report.

Methodology

There are a lot of Australian websites, so we broke the analysis down by technology platform. We started with online stores running Magento.

We started with an initial list from BuiltWith of around 4000 domains. We then trimmed it down by removing development and demo sites, and sites returning an error, leaving a total of 2998. The list includes some of the largest retailers in Australia, including Harvey Norman, Sportsgirl, Philips and Dyson.

We then ran our competitor report for every one of them. The report was run from our Sydney office over a business-class internet connection. The test throttles the connection to simulate typical 4G mobile phone speeds, and uses a mobile phone user agent/screen size to view the mobile version of the site. We did not throttle CPU performance like Lighthouse does.

NOTE We're excluding First Input Delay in our results as Google defines First Input Delay (FID) as a real user measurement (RUM). FID measures the time taken for the website to respond to the first user interaction, such as clicking a link or button. Technically this interaction can happen any time after the first content appears in the browser; in practice, most people won't click something until after a page is visually complete, and this timing is highly variable. We do measure First Input Delay by simulating a click, but our interaction happens soon after the FCP, while content is still loading. That would cause more sites to fail the metric than would fail in real life, so we're excluding it from our calculations.

On to the results. We did not expect strong numbers, but the results were still worse than expected.

The results

The first check was for the number of websites that achieve a good rating in any of the Web Vitals metrics.

	TTFB (< 0.1s)	FCP (< 1s)	LCP (< 2.5s)	CLS (<.1)
Percentage of sites that are 'Good'
Result	99 (3.3%)	71 (2.37%)	254 (8.47%)	1074 (35.8%)

CLS was the strongest result, which isn't a surprise. The remaining results are not encouraging: only 8.5% pass LCP. Let's see how many need improvement.

	TTFB (< 0.6s)	FCP (< 2s)	LCP (< 4s)	CLS (< .25)
Percentage of sites that 'Needs Improvement'
Result	383 (12.7%)	728 (24.3%)	470 (15.7%)	625 (20.8%)

An additional 15.7% beat the 4s cut-off for LCP. That still means that 3/4 of Australian Magento stores take longer than 4s to visually load on a mobile device. Visualised, the numbers are not pretty.

Australian Magento sites are likely missing potential sales. Recent performance studies show:

The probability of a customer bouncing increases 90% if the page load time increases from 1s to 5s. (source Google)
A 100 millisecond delay in load time can hurt conversion rates by 7%. (source Akamai)

The experience of our own Magento 1 clients and Magento 2 clients backs this up: improving website speed affects conversions and revenue.

Sites that pass all criteria

Of our 2998 websites, we only found 163 that pass Core Web Vital 'good' criteria. That's only 5.5%.

If we again relax to include 'needs improvement' that list grows to 520, or 17.3% of sites tested.

How you can test your site

Google provides online analytics that you can query via BigQuery. If you want to reproduce the report this analysis is based on and compare your website to your competitors, you can use the Peakhour.IO Website Speed Comparison report. We automatically discover your competitors, run them through Web Vitals, and graph the results.

Conclusion

If nothing changes, quite a few Australian Magento stores could lose search visibility in May 2021. The majority of sites don't use Magento 2's ability to cache dynamic pages, and if they do, they're often not serving optimal images.

Core Web Vitals becomes a search signal next year. For Magento teams, it is time to get ready.

Test Your Website Performance With Google Lighthouse

2020-09-14T13:00:00+10:00

Today we're introducing our other favourite tool for testing website performance: Google Lighthouse (from here we'll just call it Lighthouse). Lighthouse measures page experience across accessibility, performance, SEO, and Progressive Web Apps for desktop and mobile devices.

Lighthouse is the engine behind web.dev/measure and PageSpeed Insights. It is also available in Chrome DevTools, via npm, or as a browser extension in Chrome and Firefox. At time of writing Lighthouse is up to version 6, which introduced Web Vitals as the basis for scoring. If you are unsure which version of the tool you are using, then scroll right to the bottom of the report it generates where it will state the version.

Lighthouse generates its report by simulating a specific device and network speed, rather than running at the full speed of your computer. That matters because speed issues are more noticeable on slower devices, and users are not all on newer devices or fast internet connections. You should test for a good load experience across that range. The current simulated mobile device is a Moto G4 on a ~1.5 megabit connection. For reference, it would take over 5s to download 1mb of data at this speed. If your page weight is typical, ie over 2.5mb, you should not expect a strong score.

How to use Google Lighthouse

There are two ways you can run a Google Lighthouse report:

In your local browser;
Online via web.dev/measure or PageSpeed Insights.

We recommend running Google Lighthouse in your local browser because the online versions operate out of the US. If your website and customers are elsewhere, that extra network latency can pull the score down.

Here we'll focus on running Lighthouse from within Chrome DevTools. To do this, click on the three vertical dots in the top right-hand corner, then select 'More Tools', then 'Developer Tools'.

The developer tools will then be displayed. Along the top of the tools window are a number of tabs. Select the 'Lighthouse' tab.

Since we're only interested in performance, make sure only the Performance category is ticked. You also want to make sure 'Clear storage' is ticked (in the top left) to simulate a first load of your site. Finally, choose the device you want to report on, mobile or desktop, and click 'Generate report'.

While the report is being generated, avoid doing anything else on your computer, and don't leave it busy with background tasks. Otherwise, the score can be affected.

Understanding the Score

Once the report has finished you'll see a performance summary, like this mobile one we ran on Peakhour.io while we were developing the website:

Scoring in version 6 is based on Google's web vitals, which are metrics that indicate a good user experience, and Webpagetest's speed index, which measures visual loading performance. Each metric is colour coded as good, ok, or bad. If the measurement is good you get a green circle to the left, if it's ok you get an orange square, if it is bad you get a red triangle.

Each raw metric (the number listed in the report) is compared to real website performance data sourced from the HTTP archive and converted into a score out of 100. This is done by grading the reference data on a curve, so if your website performs in the top 8% of websites, it gets a score of 90. Similarly, if it scores in the top 25%, it gets a score of 50. If you are interested in the technical details, Google has in-depth explanations of the scoring at web.dev

Each metric is assigned a weight, and the scores are tallied into one overall number based on the weighting. Here is a breakdown of the test we just ran (in the screen shot above), obtained by clicking on the 'See Calculator' link between the Metrics section and the screen shots:

Score Variability

79 is a good result for a mobile device. However, we ran it several times and obtained scores between 60 on the low end and 85 on the high end. Scores can fluctuate widely, even when testing on the same device repeatedly. Reasons for this include:

Small differences in internet performance.
Your computer CPU load when performing the test.
Web server variability.

Again, Google has in-depth documentation around what might be causing this.

Opportunities

If your site loads slowly then Lighthouse will list addressable reasons in the opportunities section of the report.

This is the part we generally find most useful. It identifies items that slow the initial load and items that affect the rendering of a website once it is downloaded. A page can be downloaded very quickly, but the end user still sees a slow site because CSS and Javascript are blocking rendering. This is a common problem in Wordpress and Magento themes. These themes include large amounts of third party code that ultimately never gets used for a particular site, but which the browser still has to download and parse before it can display anything.

In this case, the main bottleneck appears to be a font loaded from Google Fonts. This is delaying the rendering of our page by 1.3s.

Diagnostics

The diagnostics section provides additional information you can use to improve load times.

Here we have a few small problems, mainly associated with the development status of our site.

web.dev and PageSpeed Insights

If you do choose to run your report online, we recommend using PageSpeed Insights.

In addition to the generated report, PageSpeed Insights also shows you Field Data for the page you are testing, and an Origin Summary for all pages on the website.

This is real-world data gathered from Chrome users who have opted in to allowing Google to gather their data. If your site isn't very busy then Google might not have any data to share. It is not truly representative, but it is useful information and can often tell a very different story to your Lighthouse score.

Summary

Regular measurement matters because performance issues are often specific and easy to miss. Lighthouse is useful for identifying issues that affect website performance, and when used in conjunction with Webpagetest.org, you'll be in a better position to provide a good experience for your users. Next we'll cover common issues that can impact site speed, so read on.

Website Performance testing with WebPageTest.org

2020-09-13T13:00:00+10:00

WebPageTest is one of our favourite tools for measuring website performance: webpagetest.org

WebPageTest is a web page performance testing tool developed by AOL and open-sourced in 2008. It produces its metrics using real-world browsers to load the web page being tested. It's actively maintained by Google on GitHub, so you can download and install it on your own server if you prefer. It is also free.

The key advantage of WebPageTest is that tests can be run from locations around the world, using real browsers at actual connection speeds. That lets you test performance where your users are, and see real load times rather than arbitrary scores out of 100. You can run simple tests, advanced multi-step tests, video capture, content blocking, multi-site visual comparisons, and traceroute testing. While this is still a synthetic test, it is about as close to real-world performance measurement as you can get without using RUM.

Use WebPageTest when you need to understand why a page is slow. Use the Chrome UX Report when you need to know how real Chrome users experienced the page over time. Use Peakhour's website speed comparison tool when you want a fast comparison between your domain and competitor domains using CrUX field data.

Running a Simple Test

To run a simple test:

Visit webpagetest.org
Enter the URL of the page that you want to test in the 'Enter a Website URL' field. We usually enter only the domain name, with no www and no https:// at the front. This simulates someone typing your domain into a browser address bar and captures how much time any redirects add to the page load. Sometimes redirects can be very slow.
Select the location of the test from the 'Test Location' drop down. Choose locations that reflect where your users are. Note Australian locations are at the very bottom.
Choose the browser to run the test from. Chrome is the default.
Press 'Start Test'

Note there are advanced options but for simple testing you don't need to change these.

The test will perform three visits to the specified page, with the browser cache cleared in between each visit. This simulates someone visiting your site for the first time, including the parts of the page load that only happen on an uncached visit.

Interpreting the results.

A,B,C's

The results page has a summary section at the top giving you summary grades for several categories. Here's a sample report using our website, peakhour.io. Take note of the sections at the bottom, because we'll refer to them later.

While this section isn't representative of performance it can still give you actionable information. Here's a summary of the information (ignoring security since we're concerned with speed):

First Byte Time: How long it takes the server to respond to the browser request with the first byte of information. This is the same as the Web Vital Time To First Byte (TTFB).

Keep Alive Enabled: A server option that leaves the connection from the browser to the server open for a short time, usually a few seconds, after the server has finished transmitting a request. This allows the browser to reuse a connection and saves time because it does not need to reconnect as often.

Compress Transfer: When files are transferred from the server they are compressed, usually via gzip, to make sure transfer sizes are as small as possible. It looks like we have a problem here.

Compress Images: Images are usually the largest part of a web page by transfer size. Making sure they're well compressed is important for fast sites.

Cache Static Content: Checks that static files, e.g. JavaScript, CSS, and images, have appropriate cache headers so your browser doesn't re-fetch them every time it views a page. We're developing our site so have caching turned off at the moment, hence the bad score

Effective Use of CDN: Detects whether your website is using a CDN known by WebPageTest.

This report points to problems with transfer compression and static content caching.

Performance Metrics (The Important Stuff)

The next section summarises the key performance metrics of the page load. Google's web vitals are represented alongside lab metrics such as speed index, total blocking time, and page weight.

Despite some poor marks in the grading section, the peakhour.io website loads very quickly. Its Largest Contentful Paint (LCP) is less than 1s, well below Google's 2.5s target for a good result. We've already covered them in our web vitals, which also defines ideal values for each metric, so we won't cover them again here. One detail worth noting is that Total Blocking Time is still a useful lab signal for main-thread blocking. It is not the same as Interaction to Next Paint, because INP is measured from real user interactions in the field, but it helps identify JavaScript and third-party work that can harm responsiveness.

The Detail Section

Remember the sections immediately below the grade summaries? Now we're going to click on the 'Details' section. The part we want to highlight here is the Waterfall.

At the top of the waterfall chart is a colour key for reading the diagram. The key concepts are:

dns (Dark Green): This measures the time it takes for the browser to look up the location of your server.

connect (Orange): This measures the time taken to establish the TCP connection to download a resource. It should only appear on the first resource for a given host. Remember the keep-alives grade: if that is turned off, there will be more connections here.

ssl (Dark Purple): Any resources that are loading from a secure website will need to be processed as such – the purple will signify how long it is taking to connect to that SSL item.

The Waterfall View

The waterfall is an easy-to-read view of how your website loads, with all the resources listed in the order they're requested, along with the time taken to load each resource. You can click on any resource to view the request/response headers, file size, protocol and more.

In the example above the first line has a yellow background, which signifies a redirect. The three lines towards the bottom with red backgrounds signify 404 not found errors, which need to be fixed. The colourful vertical lines indicate where major load events, like first paint and document loaded, happen.

The other thing to look for is any resource that takes a long time to load. In our example the 2nd row, which is the main HTML document, only took 149ms, which is fast. A lot of websites take 2-5s to load the main document, putting the user experience under pressure before the rest of the page has started. The main concerns here are rows 27 and 28: two SVG images that took around three quarters of a second to load.

The Performance Review Section

The final section we're going to cover is the Performance Review section.

This section gives more detail for the performance grades at the top of the report. In the earlier example Peakhour.io scored poorly for Compress Transfer and now we can see why: we're not compressing SVG images, something that can save 234kb of file downloads.

Summary

WebPageTest gives you enough detail to see where a page is losing time, not just whether it passed a headline score. There is more in the tool than we've covered here, but this is enough to test a website, read the main report sections, and identify practical issues to fix. If you are starting with a market view rather than a single waterfall, compare your field data first with the website speed comparison tool, then come back to WebPageTest for the diagnostic pass.

Core Web Vitals Optimisation

2020-09-11T13:00:00+10:00

A good user experience matters for any website. It matters to users, and Google also measures aspects of the user experience when ranking your website in its organic results.

Google ranks websites and pages in its search results using 'search signals'. These signals include the quality of content on a page, the number of sites linking to that page, and the experience of the user browsing the site. Factors that make up a great experience include ease of use, accessibility, speed and responsiveness.

The set of search signals that Google uses to measure user experience beyond a page's content value is called 'Page experience'. Google updated the Page Experience signals with metrics called Web Vitals.

For years, performance testing used different proxies to determine whether a website was fast. Browser events like 'Page Load' or 'Dom Load' were used before it was realised that these do not necessarily reflect what an end user experiences. Testing then fragmented, with different teams focusing on the measures they considered important. Web Vitals is Google's performance testing initiative to provide "unified guidance for quality signals that they believe are essential to delivering a great user experience on the web."

The 'Core' Web Vitals join the existing search signals: mobile-friendliness, safe-browsing, HTTPS, and no intrusive interstitials as seen in the graphic below from Google.

Understanding the Core Web Vitals

Before diving in, it is worth noting that the metrics that make up Core Web Vitals are expected to evolve over time. Google states that "these signals are not perfect and future improvements or additions should be expected."

Largest Contentful Paint (LCP)

Largest Contentful Paint (LCP) measures when the "largest", or "main" piece of content has completely loaded and is visible, usually a hero image. Content refers to text, foreground images, background images, and elements. LCP complements First Contentful Paint (FCP), a metric that marks the initial web page loading experience. LCP calculates how quickly a user can see page content. Scores below 2.5 seconds are considered in the 'Good' range.

First Input Delay (FID) (Due to be replaced March 2024 by Interaction to Next Paint (INP))

First Input Delay (FID) measures page interactivity: how long it takes for a page to respond to input from the user, such as a key press or a mouse click. Low FID scores ensure pages are usable. FID is a real-world metric that cannot be measured in the lab. The Total Blocking Time (TBT) metric found in Lighthouse is lab-measurable and correlates with FID to simulate real-world interactivity.

Cumulative Layout Shift (CLS)

Cumulative Layout Shift (CLS) measures visual stability while the page is loading. This means that once displayed, a piece of content stays where it is; it does not jump around the screen as other content loads. Lower CLS scores mean that users are not experiencing unnecessary content shifts. CLS scores below 0.10 are 'Good'.

Understanding the rest of the Web Vitals

The Web Vitals metrics that are not part of the Core Web Vitals are Time to First Byte (TTFB), and First Contentful Paint (FCP). They provide additional ways to improve a web user’s experience, and help diagnose specific issues, either in the lab or in the field.

Time to First Byte (TTFB)

Time to First Byte is the time it takes for a user's browser to receive the first byte of page content.

First Contentful Paint (FCP)

First Contentful Paint (FCP) "measures the point from when a web page starts loading to when ANY content starts rendering on screen." The term 'Content' means text, images, <svg> elements, or non-white <canvas> elements. Just remember that FCP can be triggered very early in the page load, but may not necessarily deliver any visible content or information to the user.

Conclusion

Web Vitals are Google's attempt to unify the metrics webmasters use to measure the Page Experience of their site. With Core Vitals included in the ranking signals, site owners can see which performance measures Google treats as important. Measuring performance is the topic of our next few posts.

Is your website obese?

2020-03-19T13:00:00+11:00

Page weight has increased steadily since the early web, as connection speeds improved and sites became more interactive and visually rich. The HTTP Archive shows over a 300% increase over the last 10 years.

MEDIAN DESKTOP
1957.8 KB
▲318.6%

MEDIAN MOBILE
1791.9 KB
▲1137.5%

Why should I care?

Smaller page weight = Faster page load

Page weight directly affects page load times. The smaller the page, the faster your site can load. The slower the connection, the more obvious the effect. Faster page loads are associated with higher conversion rates, fewer bounces, and higher engagement with your site.

Smaller page weight = Less bandwidth = Savings

Your hosting provider or CDN may charge for data transfer. Reducing page size can reduce those costs.

How big is too big?

Google says that, to meet its best practice guidelines, you should aim for a page weight of 500 KB on mobile devices and a request count of fewer than 50.

What can I do to shrink my page weight?

Start with the bytes that browsers have to download, parse, and render.

1. Optimise Images

Images are often the biggest contributors to page weight. Careful optimisation can reduce their size without a noticeable loss in quality.

Compression: Use image compression tools to reduce file sizes. For JPEGs, a quality setting of 75-85% is often a practical balance.
Use Next-Gen Formats: Formats like WebP and AVIF compress better than older formats like JPEG and PNG. Modern browsers have wide support for them.
Responsive Images: Use the <picture> element or srcset attribute to serve different image sizes for different screen resolutions. There is no reason to send a large desktop image to a mobile phone.
Lazy Loading: Load images only when they are about to enter the viewport. This does not reduce total page weight, but it improves the initial load time.

2. Minify CSS, JavaScript, and HTML

Minification removes unnecessary characters from source code without changing its functionality. That includes white space, comments, and line breaks. Many build tools, including Webpack, and CMS plugins can automate this process.

3. Enable Server-Side Compression

Enabling compression like Gzip or Brotli on your web server can reduce the size of the HTML, CSS, and JavaScript files sent to the browser. Brotli offers better compression than Gzip and is supported by all modern browsers.

4. Leverage Browser Caching

Set appropriate Cache-Control and Expires headers so the browser can store files locally. On later visits, the user's browser can load those files from its local cache instead of downloading them again, making return visits much faster.

5. Remove Unused Code

Many websites, especially those built on complex themes or frameworks, load large CSS and JavaScript files with code that is never used. Use browser developer tools, such as the Coverage tab in Chrome DevTools, to identify and remove unused code. For JavaScript builds, this process is often called "tree-shaking," and it can significantly shrink your file sizes.

6. Use a Content Delivery Network (CDN)

A CDN stores copies of your assets on servers around the world. When a user visits your site, those assets are served from the server closest to them, reducing latency. Many CDNs also provide automatic optimisation features such as image compression and minification.

Final Thoughts

Reducing page weight is ongoing work. A lighter, faster website gives users less to wait for, improves search engine rankings and conversion rates, and reduces the amount of data you have to serve. Start with images, compression, caching, and unused code; those changes usually give you the clearest wins.

Instant Alerts

2019-05-31T13:00:00+10:00

We've introduced Instant Alerts, an optional service that notifies you when there is a problem with your site. You can specify the email addresses and/or mobile numbers that should receive the alerts.

At the moment you can receive alerts when your origin server:

Times out
Cannot be reached at all
Returns an error

To avoid repeated messages, each alert has a cooldown limit you can set from 30 minutes to 24 hours.

If you're using a service like Pingdom to check that your site is up, you probably will want to enable Instant Alerts. Services like Pingdom might be served pages from our cache while your origin server is actually down.

WordPress Performance Optimisation

2019-05-27T13:00:00+10:00

WordPress speed problems are usually request-path problems. A public article, product page, or campaign landing page should not make PHP, plugins, and the database rebuild the same HTML for every visitor. A login attempt, checkout session, admin request, or private API call should not be treated like public content just because the site is under load.

The job is to separate those paths clearly. Cache what can be reused, protect the routes that are expensive or abused, and keep enough evidence to prove that the change improved the visitor experience without hiding origin risk.

Start With Public Page Caching

Most WordPress sites have a large set of pages that are dynamic only because WordPress generated them. The content itself is public and often identical for many visitors: posts, pages, category archives, product listings, campaign pages, media assets, CSS, and JavaScript. These are the paths where full-page caching and edge delivery can change the result quickly.

When a public page is served from cache, the browser avoids the long trip to origin and the origin avoids running WordPress for a repeat response. That can move Time to First Byte and Largest Contentful Paint in the right direction before anyone touches the theme. Peakhour's older full-page caching examples showed the practical size of this change: a Magento main document fell from 2.07 seconds before caching to 82 ms after caching. WordPress sites have different internals, but the same pattern applies when anonymous pages are safe to reuse.

The cache policy should be route-aware, not blanket. A simple operating model looks like this:

WordPress path	Delivery stance	What to check
Posts, pages, category pages, campaign pages	Cache publicly with tags and purge controls	Confirm logged-out content is shared and fresh after publishing.
Featured images, media library files, theme assets	Cache and optimise variants	Track image weight, format, dimensions, and cache hit state.
WooCommerce product and category pages	Cache when no cart/session dependency changes the response	Keep stock, price, and promotion purges tied to content changes.
Cart, checkout, account, previews, admin	Bypass shared cache	Preserve session privacy and correctness.
`wp-login.php`, `/wp-admin/`, `xmlrpc.php`, sensitive plugin endpoints	Protect and rate-limit before origin	Keep noisy automation away from PHP workers and admin paths.

Publishing Should Not Clear the Whole Site

Caching WordPress is easy until someone edits content. Clearing the whole cache after every post update is safe in the narrow sense, but it damages hit ratio and pushes avoidable traffic back to origin. During a busy publishing period or campaign, that can make the site slower exactly when fresh content is being promoted.

Cache tags are a better fit. Tags label cached responses by the content or template they depend on, so a post update can purge the post, related archives, and affected modules without invalidating unrelated pages. The updated Peakhour WordPress plugin generates cache tags automatically and sends purge requests when content changes in the WordPress admin. That lets teams set longer cache lifetimes for public content while still publishing with confidence.

This is where advanced caching becomes operational rather than theoretical. The site team can review which tags were purged, which routes stayed cached, and whether the next request was a hit, miss, or stored response.

Images and Browser Work Still Matter

Page caching improves the start of the request path. It does not fix a 3 MB hero image, missing image dimensions, unused CSS, or a third-party script that blocks rendering. WordPress themes and plugin stacks often ship CSS and JavaScript for features that are not used on the current page. The browser still has to download and parse that code before it can paint useful content.

Use Lighthouse to find render-blocking resources and main-thread pressure. Use WebPageTest to see whether the main HTML arrived quickly but the filmstrip still stayed blank while CSS, fonts, scripts, or images loaded. Those are different failures, and they need different fixes.

For images, the best results usually come from serving the right variant rather than only compressing the original. Peakhour image optimisation can generate AVIF or WebP outputs, choose responsive sizes, and cache the resulting variants. That helps LCP when the largest visible element is an image, and it helps CLS when dimensions are kept stable.

For CSS and JavaScript, remove unused files where possible, defer non-critical scripts, self-host critical third-party assets when practical, and reserve preconnect for third-party domains that genuinely affect the first view. Moving every script later can break dependencies, so test the actual page type rather than applying a generic rule across the whole theme.

Protect Expensive WordPress Paths

Performance and security meet at origin capacity. Login floods, XML-RPC abuse, aggressive crawlers, scraper traffic, and noisy plugin endpoints can consume PHP workers and database connections that should be serving real visitors. If those requests are filtered only after WordPress has loaded, the site can look like it has a speed problem when it really has an unsorted traffic problem.

For WordPress, protection should be specific. wp-login.php, /wp-admin/, xmlrpc.php, the REST API, and plugin-specific endpoints should have their own bot, WAAP, and rate-limit policy. WooCommerce needs separate handling again: public catalogue pages can often be cached, but cart, checkout, account, and payment paths must stay dynamic and private.

This does not mean putting heavy checks in front of every visitor. It means making edge decisions before origin work: allow clean public page requests, serve cache hits, challenge or rate-limit suspicious login traffic, bypass cache for private sessions, and log what happened.

Measure the Outcome

The evidence should line up across tools. WebPageTest should show a faster main document on cache hits, fewer slow origin fetches, and a waterfall where critical resources are visible early. Lighthouse should show fewer render-blocking opportunities and less main-thread pressure. Core Web Vitals should move where the page had the relevant bottleneck: LCP for slow HTML or heavy hero media, CLS for unstable layout, and INP for JavaScript and interaction work.

Peakhour evidence should add the delivery side: cache hit ratio, miss causes, purge state, Cache-Status, image savings, shielded misses, blocked login or XML-RPC abuse, and origin request volume. That combination tells the site team whether WordPress is faster because visitors received lighter pages from the edge, because abusive traffic stopped draining origin, or because browser work was reduced.

Fast WordPress is not a plugin list. It is a set of clear route decisions backed by before-and-after measurements.

Boost Your Website Speed with Full Page Caching

2019-05-10T13:00:00+10:00

Full page caching is not a cosmetic optimisation. Users expect quick load times, and search engines reward fast websites with higher rankings. Full page caching supports both by storing fully rendered HTML pages, so the server does not have to rebuild the page from scratch for each visitor.

How Full Page Caching Enhances Performance and Capability

By caching full HTML pages, you create a snapshot of a page at a particular moment. When a user requests that page, the server can deliver the snapshot instead of generating the page again. This saves compute and database resources, allowing your server to handle more users simultaneously. The result is straightforward: faster load times for users and less strain on the server.

Real-World Impact

Our early adopters have reported clear improvements in website speed and server performance. One client saw a 40% reduction in server load and a 20% increase in page load speed. Those results point to a better user experience and potentially higher conversion rates.

Challenges with Magento 1 and How We Solved Them

Magento 1 presents specific challenges for full page caching, and our plugin handles them directly.

Mini Cart Issue

The mini cart needs to show real-time data, which makes it a barrier to full page caching. Our plugin uses targeted AJAX calls to fetch this data only when required, reducing unnecessary load on the server.

Form Key Problem

Magento 1 uses form keys to prevent CSRF attacks, but those keys make caching difficult. Our plugin replaces form keys with a strict referrer check, which is secure and cache-friendly.

Additional Features

Automatic Expires Headers: You do not need to manually set expiration times for your cache; our plugin does it for you.
Cache Tagging: This allows for precise cache management. When you update a product, only the relevant cached pages are flushed, keeping the process efficient.

The plugin is available for download, with the aim of improving performance and increasing server capability on Magento 1 stores.

Boost Your Website Speed with Peakhour's Full Page Caching

2019-04-02T13:00:00+11:00

Why Full Page Caching Matters

Site speed affects user experience, conversion, and server load. Slow pages can increase bounce rates and cost revenue. Full Page Caching improves response times by storing and serving static versions of dynamic pages. This reduces load on your server and gives visitors quicker page loads.

How Peakhour Optimises Your Site

Peakhour helps speed up and secure your website with a DNS change. Unlike basic caching solutions, Peakhour serves these static pages from its global ANYCast network. Your site's visitors download content from a server close to them, reducing latency and speeding up downloads.

Peakhour's Transparent Caching and Delivery

In addition to Full Page Caching, Peakhour can act as a transparent delivery and cache layer. It caches and optimises static assets like CSS, JavaScript, and images. You don't need to make any other changes to your site; a DNS change is all it takes.

Wordpress Plugin for Easy Integration

During our beta phase, many clients used early versions of our WordPress plugin. The plugin integrates with Peakhour's API to automate content flushing when you make edits in the WordPress admin panel. This simplifies publishing and allows you to set longer lifetimes for your dynamic content in our global cache.

Improved Performance Metrics

With Peakhour, sites can see faster download times, a higher cache hit rate, and reduced load on the origin server. Site owners get lower origin demand; visitors get faster pages.