An outage at Google’s DoubleClick division had a significant impact on the retail industry yesterday. According to our data, the outage affected almost all eCommerce websites between 3pm and 6pm ET on Tuesday. Any website utilizing Google Ad services on their website for tracking or re-targeting experienced longer than usual page load times. These delays were caused by problems with Google tags connecting with the Google servers.  

Google Latency Costs Hundreds of Millions in Sales

We spoke to several eCommerce teams (not using Yottaa) about their experience, and they reported 20-30 second page load times during this three hour period. Industry research has demonstrated that visitors will leave an eCommerce site after waiting 3 seconds for a page to load, so this is no small problem. In fact, a study by the IR 500 retailer eBags revealed that conversion dropped 20% for every second that  a customer waited beyond 3 seconds. Given that consumers spent over $450 billion online in 2017, that 3 hour window of significant latency cost retailers over $150 million in eCommerce sales yesterday. And if that experience results in customers that never come back, it may have cost them billions. 

What Did Yottaa Customers Experience?

Here is a graph of the spike in last byte time that we detected on one of our clients.

The next graph shows a 4x change in the number of times when Google Ads services failed to load on a customer’s site.

Pages Failed to Load

Even websites using asynchronous loading for Google Ads would still have been hit with significant performance degradations. Fortunately, Yottaa was able to mitigate this issue for our customers automatically. In the graph below, the dark purple line shows the impact on Yottaa controlled pages, compared to the light purple line demonstrating the impact of pages not optimized and controlled by Yottaa. During the outage, non-optimized visitors saw a 3x greater delay relative to optimized users.

Behind Every Tag is an Application That Can Fail

We should all pause to take stock of what we can learn from this outage. First, go back to your decision to add 3rd party features to your site. A major selling point was probably how simple it is to add these tags to your website. “Just cut and paste this tag into your tag manager and ‘boom’ – start reaping the benefits” they will tell you. What’s not discussed is the complexity of the application behind this tag that is now operating as a piece of your website.

Once the tag is inserted onto your web page, every customer visit initiates a series of actions required for each feature to load correctly:

1. Browser loads the tag code

2. Browser fetches configuration file from 3rd party server

3. System manipulates the application UI in the browser

4. Sends data back to 3rd party servers

5. Repeats as necessary

Today’s eCommerce sites can have as many as 50-100 third party tags loading on their pages to deliver features and capabilities. And each tag requires an average of 15 requests to their 3rd party servers to successfully load. That represents hundreds of potential points of failure related to third party tags every time your page loads. You can see how serious a problem this can become if not monitored and managed correctly.

Controlling 3rd Parties Requires More than Point Optimization

This problem of complex 3rd party tags delaying page load times is common, and frequently solved by stripping tags from the site. But eventually new tags are inserted back onto the site, and each one is just as likely to suffer a failure as the faulty one you removed. If tag problems can impact Google’s Ad services or cause delays on Amazon.com during Prime Day, it can happen to anyone.  

Twitter was blowing up on Tuesday with eCommerce teams looking for “a non-blocking optimization” that will solve this problem. My response is simple:

1. Yottaa already solved this. We’ve spent almost 10 years building and refining 3rd party sequencing technology that allows eCommerce teams to optimize how they include 3rd parties on their website. The technology allows 3rd party tags to be removed from the critical rendering path, and in the case of DoubleClick it prevented 95% of our customers from seeing any impact on their application.

2. This is not a “one and done” optimization problem. One-off optimizations will only create one-off solutions specific to each 3rd party that will quickly degrade with time. Yottaa thinks about 3rd parties holistically and built a solution that governs how 3rd party applications are executed on your site. And you need a solution that has visibility and control over all third parties no matter how they are inserted – via tag manager, customer JavaScript, <script src=””></script>, etc.

Control Your Vulnerability to 3rd Party Problems

Many web teams will throw up their hands on days like this and say “It’s a Google problem. What can we do?” As if Google is so big that no one can be protected when their servers go down. But you can control this. Learn from yesterday’s latency and lost sales and be prepared the next time a major 3rd party application impacts the experience on your site. Take steps now to put in place technology and processes that can govern and control these 3rd party applications. Before the next 3 hour outage puts your business in the red.