What is A/B testing? What is multivariate testing?
A/B testing is when users are presented with one of two versions of a design -- an A version or a B version -- and conversion rates for each version are compared to see which one performs best. A variation of A/B testing is multivariate testing, which is when you test 3 or more versions of a design.
Why is A/B and multivariate testing helpful?
A/B and multivariate testing is helpful because you can evaluate the effectiveness of two or more different approaches with reliable data very quickly.
A/B and multivariate testing is most valuable when only one element is different in each version . For example, using a different call to action for each design approach, while keeping everything else the same, allows you to draw strong conclusions about why one design performed better than another.
UserTesting does not conduct A/B or multivariate testing, but we can help inform your current or planned tests. For example:
We can help you generate ideas of what to A/B or multivariate test.
We suggest collecting users’ feedback via the UserTesting platform to quickly identify areas of weakness, confusion, or oversight that can spark ideas for a round of A/B or multivariate testing.
We can help you choose the best options for your A/B or multivariate test.
If you have a handful of ideas you’d like to A/B or multivariate test on your live site, users can provide their feedback via the UserTesting platform. This will give you more confidence on the designs you launch.
We can help you understand why a design performed better than another during an A/B or multivariate test.
A/B and multivariate testing can answer the question: “What version had a higher conversion rate?” Usability testing via the UserTesting platform can answer the question: “Why did that version have a higher conversion rate?” The UserTesting platform captures a video and audio recording of users reviewing and commenting on various design options. This allows you to pair a story with your A/B or multivariate test results.
When should A/B and multivariate testing be conducted?
A/B and multivariate testing can only be conducted on a live site or product. However, you can run usability testing: before an A/B or multivariate test to gather insight on what to A/B test and after an A/B or multivariate test to understand why one version performed better than another
UserTesting enables you to get the most out of your A/B testing program. Using Optimizely or another optimization platform?
Contact your Customer Success Manager to plan an A/B testing study or to learn how to partner best with another optimization platform.