What is Preference Testing?

What is preference testing?

In preference testing, participants interact with and review multiple designs to determine which one they prefer and why. Participants can compare aesthetics, interactions, and/or content.

Why is preference testing helpful?

If you have more than design option, this type of test can provide insight on which one participants prefer.

When should preference testing be conducted?

Preference testing should be conducted in the early design and prototyping phases. You can also conduct preference testing with your existing design (before a redesign effort) against a competitor’s design.

Setting up a preference test plan

Use Balanced Comparison to setup your preference test.

  • Add preliminary tasks to understand participants’ background and initial thoughts before they view the websites.
  • Write tasks for where participants to evaluate each page individually.
  • Finish with a set of tasks asking participants to compare and contrast the two pages/images they’ve just reviewed. You can provide both URLs again for participants to reference. Or for a side-by-side comparison, you can share a URL to a screenshot of the content in a task. Be sure to call out key differences that you want participants to focus their feedback on.

Tip: To minimize recency bias, half the participants should see Page A first, while the other half should see Page B first. With Balanced Comparison, the UserTesting platform will counterbalance your test for you.


Here are example objectives for a preference test comparing two versions of the UserTesting participant applicant page (where they go to sign up to be a participant). You can use this format to develop your own objectives!

How is each of the following criteria rated?

  • Overall appeal of design
  • Overall appeal of content
  • Trustworthiness of the offer

Participants’ likelihood of applying after reading through it.

Which version sets more accurate expectations of what it means to be a test participant?

* Note any differences relating to the “test” language and the “review” language [the differences between Page A and Page B]

Which version makes signing up to be a UserTesting panelist more enticing?

Tip: You can ask participants to answer questions without looking at the page. "Now, minimize the browser window and answer the following:"

Get started by creating a test
Learn more about UserTesting's Professional Services.

Was this article helpful?
16 out of 20 found this helpful