At Yottaa we’re interested in the performance of every site on the web.  And an integral part of each website is the content management system (CMS) it’s built on – that much we know.  But is there a link between CMS and performance?  And if so, who’s fast and who’s slow?  We couldn’t find any recent, comprehensive data on the performance of sites across different CMSs, so we decided to investigate ourselves. 

Picking CMSs To Benchmark

When deciding which CMSs to include in our study, there were three obvious choices.  WordPress, Drupal, and Joomla! dominate the market in terms of volume.  Data from Builtwith.com shows that 51% of the top 1 million sites on the web are on WordPress, 8% are on Joomla!, and 7% are on Drupal. (The next largest share is only 2%.)  That means these three platforms were necessary for a decent portrayal of performance across CMSs.

In addition to the most popular platforms, we wanted to include data for CMSs that cater to the bigger sites – the ones that web users spend most of their time (and money) on. This, we thought, would help us understand the performance trends that affect the greatest number of people on a daily basis.  With this in mind we selected a handful of the top platforms used by the top 10,000 sites on the web: Demandware, Interwoven, Adobe CQ, and WordPress VIP.

We also added HubSpot to the list, since it’s the CMS we use here at Yottaa.

Gathering Data

We ran real-browser performance tests on blocks of thousands of sites running on each CMS.  We used a standard set of test criteria: Chrome browser from 3 locations (1 North America, 1 Europe, 1 Asia-Pacific).  The three real-browser samples were averaged to provide a single performance line for each URL.  In all, we collected a line of data for close to 15,000 sites across the 8 platforms.

What We Expected: Little Variation

We were interested to see the results, but we set our expectations low as far as juicy, groundbreaking findings. We know from experience that popular sites and enterprise business sites do not generally perform better than smaller sites, despite their deeper pockets.  We also know that the big-name CMSs offer a lot of flexibility in format and design – such that they can accommodate the creation of sites that perform awesomely, poorly, and everywhere in between.  In short, we expected to see a pretty tight grouping for performance metrics.

What We Found: Range!

We actually found a much wider array of performance than we’d expected.  Even taking out HubSpot, the clear winner, there’s a range of almost three seconds in Time to Interact between Demandware and WordPress.  That might not sound like much, but when we’re considering 15,000 sites, a 35% difference in median figures is not insignificant.  On the right site, a difference between 6 seconds and 9 seconds can mean to millions of dollars in revenue lost or gained.

The big loser isWordPress.  Its mainstream offering and – notably – its higher-end offering, are both at the bottom of the list for Time To Interact.  Even though their platform dominates the market, WordPress sites are less likely to be under the threshold of good performance (whatever you consider that to be) than any other on our list.

The three smaller, enterprise-focused platforms perform better, but among them there is little variation.  The medians for Demandware, Adobe CQ, and Interwoven are within .4 seconds of each other.

Beyond Page Load Time

You can only tell so much from the end results.  Let’s dig into a little into what?s driving performance, good and bad.

Common sense says there?s a general correlation between the weight (the number of bytes) of a web page and its total page load time.  More bytes = more work to do for the browser.  Indeed, among all 15k sites in this study we found a strong correlation (0.44) between total asset weight and Time to Interact.

With that correlation in mind, it?s interesting that the above graph isn’t purely an up-and-to-the-right affair.  Evidently Demandware sites are heavier than Adobe and Interwoven sites, yet they win on overall performance.  What’s behind Demandware sites that help them overcome the extra weight?

And once again, there’s a question about what’s going on with WordPress VIP.  The median site on that platform is over 1.5 MB, placing it in the bottom 20% of all sites on the web for heaviness.  The median!  That means half of all sites on WordPress VIP are in the heaviest quintile.  That definitely helps to explain why those sites are so much slower than those of other premium offerings.

For a quick example of what goes into that “weight” metric, here’s the median number of images on a site per CMS.  Notice that it matches up pretty well with the graph of overall weight.  Fewer images means less image weight, which means less weight overall.

It’s interesting that the number of images – something that is so easy to change for even a non-technical site owner – is so variable across CMSs.  Why do Adobe CQ site owners use fewer images than others?  Does the platform offer tools to optimize images (using sprites, for instance)?  Or are there templates that naturally accommodate less image-intensive sites?

Another angle to consider performance from is the backend.  Even though the main bottleneck for performance has shifted to content on the page (the “front end”), delivery of the assets is still a hurdle that must be dealt with by every site owner.  After all, nothing on the page can load until the assets have been delivered.

This is the first category where WordPress VIP comes out clearly on top of the standard WordPress offering.  That points us even more emphatically to the content on those VIP sites.  They enjoy much faster delivery than standard WordPress, and yet they end up almost as slow at the median.  The reason?  Heavy content, including tons of images.

Elsewhere, Joomla! sites slouch on delivery.  Joomla! was third-slowest on overall performance – perhaps consistently slower delivery is what has it in that position.  If Joomla! sites halved median delivery time, putting them roughly on par with Drupal, it might be enough to move them way up the depth chart on overall performance.

What About HubSpot?

We haven’t talked much about HubSpot, which wins almost every category.  That’s because it’s a bit different from the rest.  HubSpot’s CMS is fairly new; it doesn’t yet have a substantial share of the enterprise market, nor does it have enough adoption to register as a big player on the Internet-wide market view.  The platform also provides less flexibility than the others.  It caters to non-technical users who like a WYSIWYG editor for every piece of content on the site.  In this way, HubSpot is able to effectively save users from making decisions that will be bad for performance, since all assets are pre-packed and made not to be heavy.

That said, these qualifiers shouldn’t detract from the fact that HubSpot sites absolutely rock for performance.   No matter how it gets the job done, the fact remains that you’re more likely to have a fast site on HubSpot than any of the other CMSs we looked at.

Summary

• Sites on smaller, more expensive platforms generally perform better than those on the Big Three of Drupal, Joomla!, and WordPress
• WordPress (both its standard and its ?VIP? offering) are slowest of the pack
• HubSpot sites are by far the fastest and lightest
• Some platforms are able to overcome heavier pages (more bytes) better than others 
• There was no clearly defined leader in backend delivery, but Joomla! and WordPress were noticeably slower

What’s Next

In the next post, we’ll take a look deeper into the performance data.  We?ll look at how performance is distributed for each CMS and how those distributions compare in shape and breadth, seeking to answer the question of where YOUR site is most likely to end up, given your choice of platform.  But regardless which CMS you’re on right now, you can do more to speed up your site — check out our how-to guide below.  It caters to designers, but anyone can learn from WPO principles!