Balanced Comparison

Many UserTesting customers create tests that require a test participant to compare two designs, such as: two different web pages, two prototypes, or even two marketing emails. The order in which those designs are presented can sometimes influence a participant’s reactions, creating a subtle bias within a test. The Balanced Comparison feature is designed to reduce the risk of that bias.

When you use Balanced Comparison, you create two sections in your study, part A and Part B. The order in which participants see the two parts is automatically alternated: the first participant will see option A first and B second, the next will see option B first and option A second, and so on. This balances any biases created by the order in which the two alternates are presented.

How it works

Create a new test and select your participants. You can use Balanced Comparison to test any website, prototype, or Android app. You can also use our Balanced Comparison test template.

Select your tasks and questions

To use Balanced Comparison, begin building your test plan and toggle Balanced Comparison from off to on.

BalancedComparison.png

 You'll see a new part of your study labeled Group A and Group B. Drag tasks and questions into these areas normally. You can also put questions and tasks before and after the balanced part of the study.  

Note: When comparing two different URLs we recommend beginning on a blank page, then placing the first URL in section A and the second URL in section B.

Launch your test

When you're done, launch your test as you would normally. When participants take the test, the first participant will see the balanced section in A/B order. The next participant will see it in B/A order. They'll continue to alternate until the test is completely filled, or you turn it off. This way you'll receive a balance of A/B and B/A responses even if you stop the study early.

Review your videos

Completed tests will appear on your dashboard as they would normally.

You can see which version a test participant saw by selecting Metrics from your study dashboard: Balanced_comparisons2.png
 

You can also see this information by checking the participant's info in the video player:
Balanced_comparisons3.png

 You can view various individual metrics in the drop down menu next to the video player.

Balanced_comparisons4.png

The order of tasks in an individual study is displayed in the Tasks tab in the video player.

Balanced_comparisons5.png

 

Visit UserTesting University today to learn more about Balanced Comparison!

Tell us what you think!

We’d love to get your feedback on balanced comparison. Tell us what you like and what we could do better, by e-mailing sharefeedback@usertesting.com.

Was this article helpful?
29 out of 38 found this helpful

Comments

0 comments

Article is closed for comments.