A page load SLA (Service Level Agreement) or site performance SLA is an agreement your organization develops internally to define an acceptable page load time.  Creating and enforcing a page load performance SLA guarantees a site experience that’s frictionless and valuable for your users.

Creating a strong page performance SLA isn’t as easy as it might seem, though.  In this post, we outline the steps to create a good one, as well as some key questions to ask during your SLA creation process.

How to Create a Strong Performance SLA

As defined in an earlier post on developing a web performance SLA by Jonathan Klein, a good page load performance SLA is as specific as possible.  It defines parameters that have clear failure states.  

Here’s an example of a good page load performance SLA:  “The homepage of our site will load in under 3 seconds measured at the 80^th percentile via synthetic tests running in New York, LA, Seattle, and Miami every 30 minutes.  We will measure this SLA at 8:00AM every morning and base it off the last 24 hours of data.”

It’s worth it to spend some time and effort researching and picking your initial parameters for your SLA.  Below, we outline some best practices to follow and key questions to consider when creating your page load performance SLA.

Step 1 – Establish Your Baseline

An SLA is only valuable if it’s reasonably achievable.  While it’s true that most site visitors will lose interest if a page fails to load within 3 seconds, setting 3 seconds as your SLA when most of your site pages currently take 8 seconds to load is unrealistic.

A performance SLA and a performance goal are different things.  It’s important to establish your current page load performance baseline before you create your SLA — otherwise, it will lose internal credibility right out of the gate.  

It’s easiest to use free site performance testing tools like Websitetest.com, Webpagetest.org, and Mobitest (for mobile) to find out how your site pages are currently performing. Below is an example of a baseline measurement:

Step 2 – Benchmark Against Competitors

Once you’ve established your own page performance baseline, the next step is to see how that baseline compares to your competitors.  While your goal might be to beat the page performance of your competitors, that may not be realistic for your initial SLA (again, performance goals and performance SLAs are different things).  You can use your competitor’s performance to help gauge what’s reasonable.

Above is a sample competitor performance comparison. You can use the same tools mentioned in Step 1 to do a rough benchmarking and create a similar report.  If you want deeper data, you can always request a free website benchmarking report.

Step 3 – Consider Key Questions

At this point, you should have a good sense of the parameters for a strong initial SLA for your organization.  It’s a good idea to vet it by asking and answering the following key questions:

• Which pages are you including in your SLA? Does it cover only your homepage, or other pages as well?  We recommend applying your SLA to your top 15 most-visited site pages, at the very least.  Alternatively, you could create different SLAs for different pages if performance expectations vary drastically between pages of interest.
  
• Which performance metrics do you want to include? There is a range of performance metrics you can include in your SLA.  DNS resolution time, Time to Last Byte, Time to Display, and Time to Interact are all common focuses.  (If you’re not sure which metrics are of most interest to you, our web performance metrics eBook offers some guidance.)
• What are acceptable thresholds for those metrics? For instance, you might define:
  • The average value of your metrics of interest (e.g., <4 seconds for Time to Interact)
  • The statistical distribution (e.g., >95% of samples are under 4 seconds for Time to Interact)
  • Overall uptime (e.g., for >99% of samples, the site is live)
• What locations should your SLA address?  It’s like certain locations or regions are of particular interest to your organization.  For instance, you might want page load time for your homepage to be under 3 seconds in North America, but aren’t worried if it’s over that in Tokyo.
• What are your user’s expectations? If you’re a video or games site, you might have looser SLA parameters than an e-commerce site.  Users expect different kinds of sites to behave differently, and while it’s always good to maximize performance and improve the experience for the end user, it’s reasonable to take the nature of your site content into account when creating your SLA.
• Which browsers should you include? Your web analytics should give you a sense of which browsers your site visitors use most.  It’s worth including parameters performance for specific browsers in your SLA, since your site visitors may use browsers not common among your development team (like IE).
• Is your primary audience mobile?  If a large percentage of your site visitors are mobile, you should consider creating a mobile-specific page load performance SLA.

Step 4 – Set Up Ongoing Monitoring & Revisit SLA Overtime

Once you have your SLA outlined, vetted, and agreed upon, it’s time to set up ongoing web performance monitoring to ensure you don’t violate it.  As you continue to make web performance improvements over time your frequency of meeting your SLA will tend toward 100%.  When you find that you’re continually meeting (and exceeding!) your SLA, then it’s time to revisit it and go through this process again.  Depending on how quickly your organization ships performance improvements, reviewing the SLA quarterly or biannually is usually appropriate.

What do you think?  In your experience, are there other key questions or steps to take when creating a web performance SLA?