Using the Balanced Comparison Feature


At a Glance

Turn on Balanced Comparison in your test plan when you want to compare two parts in your study. The order of the compared sections alternates for each participant.


Many UserTesting customers create tests that require a test participant to compare two designs, such as: two different web pages, two prototypes, or even two marketing emails. The order in which those designs are presented can sometimes influence a participant’s reactions, creating a subtle bias within a test. The Balanced Comparison feature is designed to reduce the risk of that bias.

When you use Balanced Comparison, you create two sections in your study, Part A and Part B. The order in which participants see the two parts is automatically alternated: the first participant will see option A first and B second, the next will see option B first and option A second, and so on. This balances any biases created by the order in which the two alternates are presented.

Note: Balanced Comparison is only available on the Advanced and Ultimate edition of the Flex plan and on the Professional and Premium editions of the Seat-based plan.

How It Works

Create a new test and select your participants. You can use Balanced Comparison to test any website, prototype, or app. You can also go to the Template gallery for further guidance.

Select your tasks and questions

To use Balanced Comparison, begin building your test plan and toggle Balanced Comparison from off to on.


You'll see a new part of your study labeled Group A and Group B. Drag tasks and questions into these areas normally. You can also put questions and tasks before and after the balanced part of the study.  

Note: When comparing two different URLs we recommend beginning on a blank page, then placing the first URL in section A and the second URL in section B.

Launch your test

When you're done, launch your test as you would normally. When participants take the test, the first participant will see the balanced section in A/B order. The next participant will see it in B/A order. They'll continue to alternate until the test is completely filled, or you turn it off. This way you'll receive a balance of A/B and B/A responses even if you stop the study early.

Review your videos

Completed tests will appear on your dashboard as they would normally.

You can see which version a test participant saw by selecting Metrics from your study dashboard:


The order of tasks in an individual study is displayed in the Tasks tab in the video player.



Learn More

Need more information? Read these related articles.

Want to learn more about this topic? Check out our University courses.

Was this article helpful?
38 out of 53 found this helpful