SLAs, or service level agreements, are essential parts of any contract in the IT arena. If you manage or own an eCommerce business, you’ve undoubtedly come across SLAs as you or your IT leaders negotiated agreements for hosting services, data center space, shopping cart software, and any number of middleware services in your application stack.

Mostly, SLAs define minimum guarantees for things like “uptime,” response time for support, and disaster recovery. They specify how results are measured, what penalties are put into place for poor service, and more.  But as technology changes, the SLAs within tech relationships must change too. In particular, there’s a growing need for eCommerce businesses to address performance requirements and end-user experience in a codified fashion using the SLA format.

The Cloud and “EUE”

As cloud computing has brought shared services to the mainstream and even the enterprise, SLAs should (and already are) take on a new aspect: end-user experience (EUE). This is a welcome change, as we’re now living in the age of the customer and every business unit should be focusing on end-users. But as the industry moves away from physical hardware and singular provider-customer relationships to shared services and complex, multiparty technology stacks, it’s become harder to achieve solid, industry-standard SLAs. Providers are understandably hesitant to agree to promise minimum levels of end-user experience when they know apps are so complex that discovering and attributing EUE problems is a herculean task.  Not to mention that EUE in general can be a squishy and hard-to-define metric to begin with.

Despite these challenges, though, it’s important to demand and enforce agreements with EUE stipulations whenever and wherever possible.  If you run an eCommerce business, you’re likely all too aware that tiny degradations in performance and EUE can seriously impact the bottom line. So why let your trusted technology providers off without being held accountable for these crucial factors?

SLAs for eCommerce: How to Get Started

Define Success

In navigating the muddy waters of EUE, you can start by defining exactly what “success” means to everyone involved. You may know what top-line goals you have for the business, but finding out how those goals translate across your technology services is another task all its own. You can arrive at this by studying your end-user experience over time to set expectations.  That involves pinning down a measure of load time — whether it’s time to start rendering, display, interact, full page load, or a combination thereof — and tracking it over time using an APM tool (more on tools below).

Set a goal

Once you’ve arrived at a measure, you can turn it into a goal by tracing its effect on metrics like conversion rate, bounce rate, or revenue. You can do this by simply comparing the two data sets (eCommerce performance vs. EUE) matched over the time period.  A bolder and more definitive approach, though, is to conduct a split test wherein some traffic experiences an artificially delayed experience, and the eCommerce performance of both groups can be tracked in real-time. This may not be an easy sell, but it will very quickly yield clear data on exactly how much performance affects your bottom line.

However you choose to conduct this study, you can use your findings to set a goal. For example, if you determine that a 500-millisecond increase in display time means a 4% decrease in conversion rate, you can use this contextual information to set a goal that hinges on keeping average display time within a certain range.

Develop the SLA

Next, you’ll have to turn that goal into a functional SLA that’s workable for your team and your service providers.  That means making sure that it’s both reasonable and as streamlined as possible.

One way to accomplish this would be to express your goal as a percentile over a discrete time period, as is common with certain measures of technology performance. In keeping with the prior example, you might say that “X Application’s time to display will not go above 5 seconds at the 85th percentile over the course of a week.” In other words, as your application is tracked throughout the week, the display time experienced by your users will be under 5 seconds 85% of the time.

This approach is reasonable because it accommodates the isolated and difficult-to-track problems that occasionally send display time through the roof for a small subset of users, but don’t reflect the overall quality of experience. It focuses on the common goal of dependably maintaining EUE goals over time, rather than just serving as a “gotcha” lever to pull anytime anything goes wrong.

Clearly, the kinds of SLAs discussed here are more nuanced — and require more resources to manage — than simple figures like “99.999% uptime” or “20 seconds average speed to answer support calls.”  But then again, technology today is far more nuanced than ever before, so one must follow the other.

Other considerations and tips for your EUE-driven SLA

Now that you have a starting point, there are a few more tips to consider:

Establish a tool and method of record

Among the most important first steps is to agree upon a monitoring tool of record. This would most likely be your existing Application Performance Monitoring (APM) tool, but it could include a new tool that is agreed upon by all parties. This is important not only to establish credibility but also because different tools use different methods to track EUE. There are two major categories: real-user monitoring, or RUM, which tracks actual user sessions via a script, and synthetic monitoring, which creates different potential user contexts with user agent strings and tests performance from within the data center. There are pros and cons to each, and the various APM providers have different methods within the two categories. But what is most important is not the method itself but to land on a solution quickly so that it doesn’t become a bottleneck.

Getting Broad Buy-in

Include all internal stakeholders early on as you establish goals. There are likely considerations and points that will garner pushback. These SLAs, even if they’re created for third-party providers, should dovetail with your company’s inter-departmental “SLAs” and with any other extant efforts to foster communication and collaboration between IT, Operations, and business teams.

Building in Malleability from the Start

Goals can change and evolve over time, which is why your SLA should also be in a position to do so whenever necessary. As a result, one of the most important ways to maintain your SLA, in the long run, is to build in scheduled reviews or adjustments on an annual basis at a minimum. You may also need to perform an SLA review any time your business strategy changes, and certainly any time major changes are made to the application.

Key Takeaways

SLAs are more than just a collection of rules and regulations. They are more than just bargaining points in a contract. They’re documents that act as an outline for exactly what it is you hope to accomplish in both the short and long term. You need to always make sure that you’re devoting the appropriate amount of attention to SLAs that you establish and maintain for the best possible results overall.

• Defining success and being realistic are two of the keys to successful SLAs.
• You need to be prepared to change your SLA as your business does.

Sources: The Staffing Stream, Joomla Day