What Is Preference Testing?

At a Glance

In preference testing, contributors interact with and review multiple designs to determine which one they prefer and why. They can compare aesthetics, interactions, and/or content.


Why is preference testing helpful?

If you have more than design option, this type of test can provide insight on which one contributors prefer.

When should preference testing be conducted?

Preference testing should be conducted in the early design and prototyping phases. You can also conduct preference testing with your existing design (before a redesign effort) against a competitor’s design.

Setting up a preference test plan

Use Balanced Comparison to setup your preference test.

  • Add preliminary tasks to understand contributors’ background and initial thoughts before they view the websites.
  • Write tasks for where contributors to evaluate each page individually.
  • Finish with a set of tasks asking contributors to compare and contrast the two pages/images they’ve just reviewed. You can provide both URLs again for contributors to reference. Or for a side-by-side comparison, you can share a URL to a screenshot of the content in a task. Be sure to call out key differences that you want contributors to focus their feedback on.

Tip: To minimize recency bias, half the contributors should see Page A first, while the other half should see Page B first. With Balanced Comparison, the UserTesting Platform will counterbalance your test for you.


Here are example objectives for a preference test comparing two versions of the UserTesting contributor applicant page (where they go to sign up to be a contributor). You can use this format to develop your own objectives!

How is each of the following criteria rated?

  • Overall appeal of design
  • Overall appeal of content
  • Trustworthiness of the offer

The likelihood of contributors applying after reading through it.

Which version sets more accurate expectations of what it means to be a test contributor?

* Note any differences relating to the “test” language and the “review” language [the differences between Page A and Page B]

Which version makes signing up to be a UserTesting contributor more enticing?

Tip: You can ask contributors to answer questions without looking at the page. "Now, minimize the browser window and answer the following:"

Get started by creating a test
Learn more about UserTesting's Professional Services.


Learn More

Need more information? Read these related articles.

Want to learn more about this topic? Check out our University courses.

Was this article helpful?
17 out of 21 found this helpful