What Is A/B Testing?

At a Glance

A/B testing is when users are presented with one of two versions of a design -- an A version or a B version -- and conversion rates for each version are compared to see which one performs best. A variation of A/B testing is multivariate testing, which is when you test 3 or more versions of a design.


Why are A/B and multivariate testing helpful?

A/B and multivariate tests are helpful because you can evaluate the effectiveness of two or more different approaches with reliable data very quickly.

A/B and multivariate testing are most valuable when only one element is different in each version. For example, using a different call to action for each design approach, while keeping everything else the same, allows you to draw strong conclusions about why one design performed better than another.

UserTesting does not conduct A/B or multivariate testing, but we can help inform your current or planned tests.

For example:

Generate ideas of what to A/B or multivariate test

We suggest collecting feedback via the UserTesting platform to quickly identify areas of weakness, confusion, or oversight that can spark ideas for a round of A/B or multivariate testing.

Choose the best options for your A/B or multivariate test

If you have a handful of ideas you’d like to A/B or multivariate test on your live site, users can provide their feedback via the UserTesting platform. This will give you more confidence on the designs you launch.

Understand why a design performed better than another during an A/B or multivariate test

A/B and multivariate testing can answer the question: “What version had a higher conversion rate?” Usability testing via the UserTesting platform can answer the question: “Why did that version have a higher conversion rate?” The UserTesting platform captures a video and audio recording of participants using and commenting on various design options. This allows you to pair a story with your A/B or multivariate test results.

When should A/B and multivariate testing be conducted?

A/B and multivariate testing can only be conducted on a live site or product. However, you can run usability testing before an A/B or multivariate test to gather insight on what to A/B test and after an A/B or multivariate test to understand why one version performed better than another.

Using Optimizely or another optimization platform?

Contact your Customer Success Manager to plan a comparison study or to learn how to partner best with another optimization platform.

Learn more about UserTesting's Professional Services.

Was this article helpful?
8 out of 15 found this helpful