What sample size do I need?

Learn how many participants you should recruit for your study.

This article applies to: ut logo tiny.pngUserTesting 

On this page:

 


 

About remote experience research

Technology has changed how companies gather insights and perspectives from their users for both unmoderated and moderated experience research. Previously, recruiting participants was a slow process limited by geography, time, and resources. Now, teams can access participants globally with the click of a button, accelerating the research process significantly. With AI and machine learning capabilities, teams can synthesize results into findings in seconds.

 

 

Sample size considerations

With these changes in mind, consider the following to determine the optimal number of participants for your test:

 

Test type

  • Depending on your test type, you might consider a different number of participants.
  • For instance, a benchmarking study might need more participants than an early-stage concept test.
  • See the table below for recommendations based on the type of test you are building.

Testing goals

  • What are the goals of the test?
  • For instance, if you’re trying to identify glaring usability issues, a sample size of 5 may be enough.
  • But if you want to go deeper to find more issues, use larger sample sizes, like 15 participants.

Stakeholders

  • With whom will you be sharing the insights?
  • Do you need a certain sample size to be persuasive to your stakeholders?

Confidence levels

  • Depending on the approach, you might need sufficient data to build confidence around surfacing themes.
  • Read more about how confidence and intervals can impact your sample size.

Availability

  • The UserTesting Network offers broad coverage in the US, UK, and Canada - including niche audiences.
  • In other markets like Australia, France, India, Mexico, Spain, and Germany, reaching niche audiences may be slightly harder.
  • Our Audience Solutions team is always available to help find your target audience through our partner networks regardless of where you’re testing, but you may consider adjusting your sample size based on availability.

 

 

Sample size recommendations

Our research experts at UserTesting recommend the following sample sizes based on your study type, goals, and approach.

Approach Description Ideal sample size How do we do it a UserTesting? 
A/B testing: before Gain confidence in the alternatives you want to test by asking people to interact with and comment on each option. 5-8 Participants complete activities as they think aloud, along with rating scale questions to gauge or "measure" their experience.
A/B testing: after Understand why one version "won" over another by asking people to interact with and comment on the options tested. 5-8 Participants complete activities as they think aloud, along with rating scale questions to gauge or "measure" their experience.
Best-in-class analysis Gather insights on highly-rated and well-known experiences to understand how to emulate in your own experience. 10 Participants complete key tasks on "great" experiences.
Card sorting Understand how and why people group and label related items or information to inform navigational structure and organization. 30 Participants "think aloud" as they sort items, concepts, etc. into categories. Tests may be open (participants name the categories) closed (the user defines categories for contributors) or hybrid (some categories are predefined, but more can be added by contributor).
Case study Build a story around an individual to establish thinking around a potential hypothesis that merits further investigation. 1-2 Participants are asked questions about their experiences, impressions, or preferences, or to interact with one or more interfaces to provide feedback.
Competitive testing Understand participant preferences between competitive experiences (ease of use, understandability, visual appeal, etc.). 10-12 Each participant completes key tasks on a single competitive experience. Responses to rating scale questions (like SUS, SUPR-Q) and performance metrics are compared across competitors.
Concept testing Gauge people's reactions to new ideas or concepts with or without a design to view or interact with. 5-8 Participants view or comment on a concept. A concept could be just a description or an experience to view or interact with.
Copy testing Learn whether copy is clear, effective, and aligned with organizational or brand values and goals. 8-10 Participants read copy and are asked to describe, provide commentary on their perceptions, or rate aspects of the language or messaging. Research objectives might include resonance, understandability, or clarity, among others.
Email copy and design Get feedback on email subject lines, content, layout, clarity, etc. 8-10 Participants view subject lines and email messages and are asked to comment on comprehension, clarity, appeal, tone of voice, visual design, and/or other elements.
Field studies Understand behaviors and/or needs in someone's natural environment. 10 Participants turn on the front-facing camera using our mobile recorder to record real-life experiences.
Image testing Determine whether the message or intent of imagery, their perceptions, and whether it is effectively communicating value is understood. 8-10 Participants view the image and are asked to describe, provide commentary on their perceptions, or rate aspects of the image. Research objectives might include clarity, visual appeal, or comprehension and/or other attributes.
Interviews Gather impressions, attitudes, expectations, and/or goals through a self-guided test or live conversation. 15 Participants are asked questions about their impressions, attitudes, expectations and/or goals in a self-guided test or live conversation.
Longitudinal/Diary study Understand people's behaviors & needs over a period of time. 15 Participants record multiple interactions over a period of time. The experience could be across channels or on a single channel.
Omnichannel and crosschannel Capture the experience as participants move between channels to accomplish a goal. 8-10 Participants record interactions on more than one channel. May require use of all recording solutions: desktop, mobile (on-screen and front-facing camera), and IPEVO.
Prototype testing Observe people as they interact with an early version of a design or experience. 5-8 Participants view a prototype on Invision, Axure, Dropbox, or some other publically accessible URL. Could be static or interactive.
Qualitative benchmarking Baseline existing experience and measure progress on a regular cadence over time. 10-20 Participants complete key tasks and rate their experience regularly (monthly, quarterly, semi-annually, or annually). Rating questions can be industry-standard (SUS, SUPR-Q) or custom informal metrics defined by an organization.
Social networks Gather insights on people's perceptions and behaviors as they view and interact with social posts. 8-10 Participants view a brand or company's presence on a social network and are asked to complete tasks and answer questions related to comprehension, clarity, appeal, tone of voice, and/or other elements.
Tree testing Learn whether or not people can find what they need within a web navigation. 30 Participants "think aloud" as they complete a task that asks them to find a destination within a navigation "tree". The ability of contributors to find the right destination reflects how well the information is organized and named.
Usability testing Observe people as they complete tasks to identify what's working well and where they stumble. 5-8 Participants complete activities and tasks. A larger sample size can be used to test high-risk or complex applications if previous work has yielded inconclusive results and if you need to ensure that you uncover all potential barriers (15 contributors to find 90% of problems, 50 contributors to find 98% of problems).
Video testing Learn whether people understand content and value being communicated and/or find it appealing 8-10 Participants view video content and are asked to comment on comprehension, clarity, appeal, tone of voice, visual design, and/or other elements. They may also speak to calls to action, expectations versus reality, or complete tasks.

 

 

How many participants do I need?

See our short video for information on sample sizes.

 

 

Related content

information icon.png

knowledge icon.png

Want to learn more? Check out these Knowledge Base articles... 

Interested in growing your skills? Check out our University courses...

video icon 2.png

team icon.png

Need hands-on training?

Can't find your answer?

Please provide any feedback you have on this article. Your feedback will be used to improve the article and should take no more than 5 minutes to complete. Article evaluations will remain completely confidential unless you request a follow-up. 

Was this article helpful?
0 out of 0 found this helpful