Customer Experience Shakeup In the Big Box Region
This is the third installment in our Retail Madness tournament. We’ve introduced the Customer Experience Index and covered the Luxury Region and the Tech Region, now it’s time for the big guys: the Big Box Region.
This region comprises mass merchants of all stripes: department stores, wholesalers, home shopping networks, national pharmacies, and online-only sellers.
On average, this region is…average. For the mean CXi score, it’s a close third behind the Tech region.
Retail Madness Results: Customer Experience Shakeup In the Big Box Region
Amazon down?
First, let’s address the elephant in the room. Amazon, in its vaunted spot atop the eCommerce world, did not do very well. This surprised us, as the company is well known for its excellence in user experience. Naturally, we investigated – and we found that there were no major surprises or clear keys. Amazon simply did “OK” at aligning to the best practices we used to drive our calculation. Let’s have a look.
Amazon’s time to start render – also known as the time to first paint, for the moment pixels appear in the browser window – was among the fastest in the field for desktops. The three pages tested averaged just under 1 second, which is in line with best practices. But while start render time is rightfully granted importance in the CXi (it’s the key moment when the user can begin engaging) it’s far from the only consideration involved.
Amazon also substantially cut down the number of JavaScripts that load for mobile users when compared to the desktop experience. The mobile pages posted an average of less than 10 JS files, among the fewest of any pages in the competition. While that’s a best practice if the goal is a pure performance, we know that today’s users expect more. We’ve found when comparing mobile conversions against page complexity that companies who find a way to deliver most of the functionality of the desktop experience on mobile see dramatically higher conversions. Our CXi operates on this assumption in its weighting, and Amazon scored poorly there.
Similarly, Amazon’s image load, which is very high on desktops, is cut down by a whopping 90% for mobile users.
One would think that diminishing the payload of the pages so greatly (JS and images comprise much of the page) would mean lightning-fast performance. But relative to the desktop site, mobile performance doesn’t seem to benefit greatly. The sites that scored highest in this study mostly achieve a rough parity between desktop and mobile performance, and Amazon saw mobile performance 4-5x slower than desktop.
Where Amazon unquestionably excels is the ‘intangibles’ – features like one-click ordering, an industry-leading recommendations engine and shipping policy, and the loyalty that’s incumbent with being the biggest in the game. They are the best at what they do. In some ways, one could argue that the rules don’t apply to them: even if they go against certain practices — ones that were generated based on a broad study of eCommerce business performance — they can claim exception because, well, check the scoreboard.
The UX Winner Is…
The Worlds Largest Retailer dominates the competition. Walmart scored consistently higher than anyone else and has the highest score in the competition. (Though we have something different coming up for the final four – so they’re not certain winners yet).
Walmart has fast scores for speed, which is weighted most heavily in the equation. But unlike Amazon, which also scored high on performance, Walmart holds up across the content differentiators, especially on mobile. Based on the content on their pages, the company provides an experience that roughly approximates the desktop experience — still pared down, but not terribly different. Interestingly, Walmart employs a separate mobile site, rather than an adaptive web design solution like Amazon and many other top retailers. We’ve noted that “m.dot” sites have a number of drawbacks, both with user experience (users expect familiar navigation and structure to translate across devices) and with performance (due to redirects). But as with most technology dogma, nothing is absolute, as Walmart proves here.
The top experience in a competitive region is not easily won. To get there, Walmart has made itself a leader in the performance space, through thought leadership, acquisition activity, and massive investment in its Walmart Labs arm. As Walmart has climbed steadily to the top of the Internet Retailer list (they’re currently in the #4 spot) they have made performance and mobile top priorities all along the way.
(click to view larger)
Next up is our Kitchen Sink region – the fastest region of all.
Previous posts:
