The web was struggling on Tuesday election night. While television networks focused on the big race, millions took to Social Media to air their opinions and millions more visited news sites hoping to find up-to-the-second data on the results of local races and ballot questions. Big news sites and social media outlets surely prepared for an onslaught of traffic, but in many cases they did not do enough. Excessively long page load times and connection failures were common.

Figure 1: CBS News Page Load Screen Shot Film Stip during Election Night

Days before the election, we set up Yottaa Monitor to track the performance and availability of all major news sites and social media sites. The news sites include NBC, ABC News, FOX News, CNN, NewYork Times, Washington Post, CBS News, Politico, etc. The major social media sites monitored include Twitter and Facebook. These monitors run a real browser (IE 9) to capture the true end-user experience from a handful of North American locations (Chicago, Dallas, Miami, Portland, San Francisco, Washington DC). The monitors visited the selection of sites every five minutes and recorded the experience that a human visitor would have had to access the site at that moment.

We also configured these monitors to capture common website problems. This meant any time one of the monitored websites encountered an HTTP error, a page asset loading error, an excessively slow page load time, or a connection failure, that issue was captured and logged.

Having tracked major websites through big events like the Super Bowl, the Olympics, and Cyber Monday, we had an idea of what to expect. But we didn’t expect the web to come to a screeching halt!

50 Second Page Load Time?

Taking 50 seconds to load a web page? Yes. That was the average page load time for many major news sites during election night. The usual page load time for these major news sites is around or under 10 seconds. 10 seconds isn?t fast, but fairly acceptable for news sites due to the significant amount of assets on their pages. However, 50 seconds is an eternity.

Figure 2 shows the page load time (?Time to Interact?) averaged from nationwide measurements for 9 major news sites throughout Monday Nov 5^th at 8pm ET to Wednesday Nov 7th 2am ET (the time displayed in the figure are in GMT). Most sites experienced hours of significant performance degradation. The development clearly followed 4 different stages (click to enlarge):

Figure 2: Significant and sustained performance issues for major news sites on election night

1. Before Tuesday 1pm ET: the average sites load around 10 seconds with a very consistent performance
2. Tuesday 1pm ET to 5pm ET: Starting from 1pm ET, most of the news sites started to show significant performance degradation consistently, reaching as high as 30 seconds
3. Tuesday 5pm ET to 7PM ET: most of the sites resumed normal or close to normal performance. Dinner time? Commute time? The lifestyle of Americans is visible on the web.
4. Tuesday 7pm ET to Wednesday 2am ET: most news sites came to a halt. Some of them consistently took as long as 50 seconds to 60 seconds to load

After Wed Nov 7^th 2am ET, all sites resumed their normal performance. Social media sites had their own performance problems, as shown in Figure 3. Twitter and Facebook pages typically load under 5 seconds. But on election night, Barack Obama?s Facebook page slowed down to 25 seconds between 11pm ET and 1am ET. This is the period of time that Obama was projected as the winner. Is this the winner effect? (Click to enlarge).

Figure 3: Twitter and Facebook slowed down significantly election night

Errors Abound on Election Night

On Wednesday November 7^th, we woke up to an expected hard fought election outcome. We expected website errors as well. Figure 3 shows the number of errors we detected from all major news sites over the most recent five days including the Election Day:

Figure 4: Number of site errors per day detected for major news sites

The trend was even stronger than we’d expected. In the sample shown on this graph there are roughly 3800 total errors; almost 80% of them occurred on Tuesday, with only 1200 spread across the previous four days. The average increase in errors from Monday to Tuesday was over 500%, with Washington Post, New York Times, and Fox News each showing increases over 1500%.

Social media sites aren?t immune to problems either. In fact, they are the source for a lot of problems for other sites. Yottaa Site Monitor allows a user to monitor pages behind login wall. So we used this feature to monitor what an actual user of these social media sites would have seen. Figure 5 shows one example of captured page loading experience for Mitt Romney?s Twitter page:

Figure 5: Significant slowdown for Twitter during election night

The explosion of errors on Tuesday was seen on many of these social media pages as well. Figure 5 shows the number of errors detected for major Facebook and Twitter pages:

Figure 6: Number of errors by day from Twitter keyword search and Facebook pages

Social Media Causes Failure on Other Sites

Figure 6 in particular underscores a trend that is increasingly important: Social media sites should not only be concerned with delivering a good experience for their own users — they should also consider their effect on other sites. Many of the errors collected from news sites were a result of the failure of third-party social media assets present on their home pages.

On the graph above, for instance, “CBS_Social” is a duplicate of the monitor set for the CBS news website, but it was filtered to collect only errors reported from social media-related assets. As you can see, the increase in errors here mirrors the increase from the “CBS News” monitor from the first graph. The 255 errors collected on Tuesday by “CBS_Social” accounts for over 40% of the total errors on the primary CBS monitor.

Who Are The Winners?

Election night results showed the web is still too fragile. Most high end brand name websites, including technology darlings such as Facebook and Twitter, had significant performance and availability problems. However, there are winners. (Click to enlarge).

Figure 7: CSPAN and CNN held tough through the election

We applaud the consistency and reliability of CSPAN and CNN websites, especially considering the fact that neither is a technology company. Each site had its share of errors and performance degraded. However, compared with other industry-leading media sites on our list, these two remained remarkably consistent.

Lesson Learned

A few takeaways from the election night:

1. The web today is still way too fragile. Yes, US election is a big event, but US is far from being the most populous country. If US election can result in significant performance and availability problems for hours for major websites, there are too many situations these websites can have significant problems or even be brought down completely. The reality is that most of the websites are just too fragile.
2. Running a great site that performs, can scale and is robust is hard. It?s a testament to the challenges facing website owners today that the world’s most popular websites, some of which have been on the web for over 15 years, can’t avoid producing thousands of errors during a major event.
3. The Web is complex and highly intertwined. Social media is one of the top causes for web performance and availability problems. No fault on social media. It is due to how the web works. Every time you embed a social media asset onto your site, you are introducing a single point of failure.
4. Gaining visibility into website performance is not hard, but one should do it and learn how to do it. One has to measure the site from an end user perspective, from many different locations and must be using real browsers. Visibility is the key to tackle any problem.

These are some of the reasons that Yottaa built Site Optimizer, an infinitely scalable cloud service to help make any website fast, secure and scalable. To complement Site Optimizer, Yottaa built Site Monitor to provide deep, real-time visibility from an end-user perspective — the first step to great website performance.