Avoid bias when collecting feedback

This article applies to: UserTesting 

On this page:

• About research bias
• Avoid being overly directive
• Use neutral language

About research bias

• Participants aim to succeed in tests by answering questions and completing tasks.
• Since you know more about the test, they may look for cues from you to help guide them.
• These cues (tone, body language, facial expressions) can unintentionally influence their answers or task performance, leading to biased findings.
• To avoid bias, maintain a neutral expression and posture during the session.
• Unmoderated tests reduce the risk of moderator bias by eliminating verbal and nonverbal signals.
• Bias can still be introduced through task wording, especially if tasks are:
  
  • Overly directive
  • Lack neutral language

Avoid being overly directive

• Most bias that surfaces in task wording is due to overly explicit instructions about how to complete a task.
• When writing activities for your test, provide a goal to be accomplished, but don’t give away the answer or have respondents simply follow your instructions.
• You can always follow up on goal-directed tasks with more directive tasks. 

Poor example: "Go to the filters and select on size XL, then sort by price to find the most inexpensive sweater that is a size XL."

• This example is too explicit. It tests whether someone can follow your directions rather than if they can use the design.

Better example

"Find a sweater under $100 in size XL."

—THEN—

"If you haven’t already done so, organize the list of sweaters to display from highest price to lowest price."

• This better example shows whether the respondents even tried to use the sort and filter controls.
• The follow-up task is more directive, with the intent of exercising the sort controls.
• Notice that it still does not use the word “sort” because that matches the control label on the page.

Use neutral language

Another bias-inducing practice you want to avoid is asking "leading" questions, in which you inadvertently point the participant to the correct answer.

Here are some examples of leading questions:

• “How much better is the new version than the original homepage?”
• “Was it hard to find the Preferences page?”
• “What would you improve about this page?”

Note that even subtle wording can bias the respondent. These questions assume the following:

• The new version is better.
• Finding the Preferences page is hard.
• The page needs improvement.

Use balanced language to address this issue. In so doing, your questions will elicit more accurate feedback.

• “How does the new version of the homepage compare to the original?”
• “How difficult or easy was it to find the Preferences page?”
• “What, if anything, would you improve about this page?”

Related content