E-commerce UX Secrets: What 200,000 Hours of Research Reveals About Conversion

If you run an e-commerce site or work on digital products, this conversation is packed with research-backed insights that could transform your conversion rates.

Apps of the Week

Before we get into our main discussion, we want to highlight a couple of tools that caught our attention recently.

UX-Ray 2.0

We talked about this last week, but it deserves another mention. UX-Ray from Baymard Institute is an extraordinary tool built on 150,000 hours (soon to be 200,000 hours) of e-commerce research. You can scan your site or a competitor's URL, and it analyzes it against Baymard's research database, providing specific recommendations for improvement.

What makes UX-Ray remarkable is its accuracy. Baymard spent almost $100,000 just setting up a test structure with manually conducted UX audits of 50 different e-commerce sites across nearly 500 UX parameters. They then compared these line by line to how UX-Ray performed, achieving a 95% accuracy rate when compared to human experts. That accuracy is crucial because if a third of your recommendations are actually harmful to conversions, you end up wasting more time weeding those out than you saved.

Currently, UX-Ray assesses 40 different UX characteristics. They could assess 80 parameters if they dropped the accuracy to 70%, but they chose quality over quantity. Each recommendation links back to detailed guides explaining the research behind the suggestion.

For anyone working in e-commerce, particularly if you're trying to compete with larger players, this tool is worth exploring. There's also a free Baymard Figma plugin that lets you annotate your designs with research-backed insights, which is brilliant for justifying design decisions to stakeholders.

Snap

We also came across Snap this week, which offers AI-driven nonfacilitated testing. The tool claims to use AI personas that go around your site completing tasks and speaking out loud, mimicking user behavior.

These kinds of tools do our heads in a bit. On one hand, we're incredibly nervous about them because they could just be making things up. There's also the concern that they remove us from interacting with real users, and you don't build empathy with an AI persona the way you do with real people. But on the other hand, the pragmatic part of us recognizes that many organizations never get to do testing because management always says there's no time or money. Tools like this might enable people who would otherwise never test at all.

At the end of the day, it comes down to accuracy and methodology. Before using any such tool, you should ask them to document their accuracy rate and show you that documentation. That will tell you how much salt to take their output with.

E-commerce UX Best Practices with Christian Holst

Our main conversation this month is with Christian Holst, Research Director and Co-Founder of Baymard Institute. We've been following Baymard's work for years, and having Christian on the show gave us a chance to dig into what nearly 200,000 hours of e-commerce research has taught them about conversion optimization.

The Birth of Baymard Institute

Christian shared the story of how Baymard started about 15 years ago. His co-founder Jamie was working as a lead front-end developer at a medium-sized agency, and he noticed something frustrating about design decision meetings. When the agency prepared three different design variations, the decision often came down to who could argue most passionately (usually the designer who created that version), the boss getting impatient and just picking one, or the client simply choosing their favorite.