When you look at your website’s performance, how do you measure it? We find that there are a variety of ways to think about performance and how to improve it.

Quantitative Measures

We have found that there are generally two categories of quantitative measures that you can use : Reachability analysis and Page Load Time analysis

There are a variety of tools that can measure what we call Reachability. These are often either IP or HTTP ping utilities that send a request to your web server, wait for a response and then report the round trip time.

These tools can be great for diagnosing network performance issues (e.g. how long does it take to reach my server from various points in the world) or monitoring uptime. But they are not a good measure of end user experience because they fail to take into account what a user actually sees when they load your site.

A better measure of end user experience is Page Load Time. This is accomplished by using a real or simulated browser that not only fetches a page from your website, but actually executes it. Executing the page includes fetching all of the included assets (images, javascripts, css, flash, video) and actually executing any active components in the page such as scripts of flash movies.

Page Load Time analysis will give you not just round trip time for each asset, but it will actually tell you how long the browser spent processing all of the various components of the page and how long it is before the user actually sees your site.

There tend to be two data points for page load monitoring, Time to Display (how long it is before the user’s browser displays something that “looks” like your site) and Time to Interactivity (how long it is before the user can actually scroll and click on active elements of the page.

Qualitative Measures

Ultimately, the most important measure of your web application performance is the perception of your users. Modern web applications can utilize techniques like Ajax to decouple long running pieces of the page from the page load time.  This means that you can have a page load time that appears to be very fast, but your users actually get a poor experience. The converse is true as well, proper use of Ajax can cause your sight to appear to the user very quickly even though there is still a lot of work going on in the background.

A good example of this is the Yahoo homepage. When that page loads, all of the boxes and widgets are loaded using Ajax calls. You will see that the basic frame of the page loads pretty quickly and the various boxes fill in over time. This is a great use of Ajax.

Qualitative measures can be difficult to capture because they often do not correspond to well defined events in the browser such as the onLoad event. A good qualitative measurement framework seeks feedback from real users and has them rate the performance of your site.

Comparative Measures

Our web applications do not exist in a vacuum. When our users interact with our site, they are comparing them not to a clock, but to their experience on other sites they frequent. Comparative measurement ranks your performance as compared to other sites on the web.

Comparative measures help normalize differences that are difficult to capture in other measurements.

The Internet is not just a series of tubes, it’s a turbulent series of tubes. There may be outages or congestion in the access, backbone or data center networks involved in transmitting packets from your website that affect end user performance. For example, your site may be slow today, but it may be because Layer 3 has an outage that is impacting 60% of the web. Comparative measurement will be more fair in these situations because your peers web sites will see performance impacts at the same time as you.

Also, there are great differences in the complexity of web applications. If you are building a 3D game that makes heavy use flash, javascript, video and other media elements, it’s probably not fair to compare your page load time to Google’s relatively spartan homepage. Comparative benchmarking allows you to compare against more meaningful user experiences.

Conclusions

If you worry about page performance or if you just want to make your site faster, you need to make sure you’re taking all of these different measures into account. If you’re measuring just one of these factors, you’re probably not seeing the real picture of your site’s performance.

Our users will thank us for every millisecond of performance we can eek out of our  web services.

The first step is knowing how fast is your website?