A complete solution for growth teams
The end-to-end platform for content teams
Actionable Analytics insights in your inbox
Whether you need an in-company growth workshop,
hire and train a new growth team, deploy a growth
strategy or simply get a growth assessment before
planning the next steps, we've got you covered!
The #1 community for scalable growth.
Learn with the best minds in the world.
No results found for your search
Speed vs. Accuracy: How do you test?
Running multiple UX tests runs the risk of making your data less accurate. However, running tests sequentially until statistical significance can be quite slow. How does your team approach this?
Great question, and I often wonder about the same thing as well. I usually lean towards speed rather than accuracy.
Meanwhile, I think for small companies with limited traffic, you should be even more thoughtful about prioritization, in term of deciding where to test. In term of what to test, you should always lean towards big bold test, which requires relatively less time.
Thanks for answering Hila! Our team leans in the same direction - however, recently I've read a lot of articles favoring the contrary. I'm curious about how the rest of the community tackles this issue.
Good question, and if your website has low traffic numbers then the toss up between speed vs accuracy is a very tough call to make. Adjusting the main goal of the test(s) can allow you to run multiple tests across the site. For example, test 1 on homepage main goal is increased interaction with the altered UX element, test 2 on product page's main goal is progression to next step, test 3 on payment page goal is conversion.
If your website has more traffic, running multiple tests at the same time is easier as you can segment the audience. I currently work for a travel company, these are some of the ways we segment our audiences:
Traffic source (PPC/email/natural search)
Customer type (families/couples/groups)
Destination (Asia/North Europe/America/Canaries)
Date of travel (Summer/Winter/Month)
This gives you more time to run the test without it clashing with others. Some of these segments are also an example of early personalisation so you can learn how different customers react to messages and build upon those findings.
For both these size websites, I agree with Hila's comment about prioritization of tests and it's importance to enable you to get the best ROI on your tool.
This makes perfect sense - thanks for sharing!
Use the feedback box below if you have a question, comment or general feedback.
Your feedback has been sent.
Sweet! The link has been copied to your clip boardy board!
Flash isn't supported. Please copy the link manually.