Write non-leading questions

Refer to this guide to write non-leading questions.

Plan Availability: All Plans
👥 User roles: Owner, Admins, and Researchers
For more information, see our article on how to find your plan and user role.

 

On this page:

 


 

Avoid leading words and questions

  • A leading question subtly encourages participants to answer in a particular way.
  • Small differences in wording can have a large impact on your data.
  • Non-specific wording leads to ambiguity and confusion in your study.
  • Words like could, should, or would produce significantly different results in a question, even though these words sound familiar.

 

Don't Ask

Do you find anything about the website to be challenging?

  • This question prompts participants to question their experience on the website.
  • In a subtle way, it raises the prospect that there are challenges even if the participant didn't perceive them initially.

 

Ask this instead

Please rate your experience on the website in terms of usability (scale below).

  • This question doesn't imply challenges using the website.
  • Instead, it asks participants to evaluate their experience and report findings from their point of view.

 

 

Avoid Yes or No questions

  • Similar to leading questions, avoid using questions with Yes or No answers.
  • Some participants in online research are trained to recognize these questions as disqualify points.
    • They are encouraged to select Yes to qualify for the study.
    • This potentially biases the resulting data.

 

Don’t ask

Have you taken out a mortgage in the last 6 months?

  • Yes
  • No [Disqualify]

Participants are more likely to select Yes to qualify for the study.

 

Ask this instead

Which of the following have you done in the last 6 months? Please select all that apply.

  • Moved homes
  • Taken a mortgage [Must select]
  • Bought a car
  • Taken a vacation
  • Opened a new bank account
  • None of the above [Disqualify]

Participants won't know which answer to select to qualify for the study and are more inclined to give honest answers.

 

 

Ask direct questions

Questions that are vague and ambiguous can limit the usefulness of your results. Make sure your questions are direct enough so the respondent has no doubt about what you are asking. Participants are not professional testers, so if you expect to get insights on something specific, make it known in the question.

 

Example 1

Don’t ask

How would you suggest improving the website landing page?

Although you're looking for suggestions about the product visuals on your website, you may end up getting responses about the navigation structure, the amount of wording, or the color scheme.

 

Ask this instead

How would you suggest improving the product visuals on the website?

 

Example2

Don’t ask

Do you watch television regularly for work?

  • What does regularly mean in this question?
  • Some participants may interpret it as meaning every week.
  • Others may interpret it as meaning a couple of times a year.

 

Ask this instead

How often do you watch television?

  • Every day
  • 2-3 times a week
  • A few times a month
  • I rarely watch television

 

 

Prefer not to answer options

  • There are instances where participants aren’t comfortable with you, as the researcher, collecting specific data points.
    • Questions about finances, family life, occupation, sexual orientation, and religious beliefs can be seen as too intrusive to some participants.
    • Privacy is a highly important factor in online research and is equally as important to respondents.
  • Rather than rejecting these participants or forcing them to drop from the study, offering a prefer not to answer option allows you to collect their feedback on the true research aim of the study.
  • Unless the information is key to the research objective, including a prefer not to answer option is always recommended.

 

 

One question at a time

It is often tempting to ask multiple questions at one time, but this can cause issues for respondents and influence their responses.

 

Don't ask

What is the fastest and most economical cable television provider for you?

  • This question is really asking two different things:
    • What is the fastest cable television provider for you?
    • What is the cable television provider for you
  • The answers may be completely different.
  • It would be best to separate these into two questions or provide answer options for each part, separately.

 

Ask this instead

What is the fastest cable television provider for you?

What is the most economical cable television provider for you?

 

 

Cover all answer choices

  • Do your closed-ended answer choices include all possible respondent answers?
  • A rule of thumb to follow: If 10% of respondents (in a pretest or otherwise) select Other, you are likely missing a crucial answer choice.
  • Review the Other responses and add the most frequently mentioned answer(s) to your question.

 

Don't ask

You have answered that you visit Alice’s Grocery once a month. Why don’t you visit the store more often?

  • I find better prices elsewhere
  • Alice’s Grocery doesn’t carry all of the products I want to purchase
  • Other grocery stores are in more convenient locations for me

 

  •  
  • The closed-ended answer choices in the above question don't cover all possible reasons a participant may not visit the store more often.
  • For example, Alice’s Grocery store may not carry specific brands, offer a variety of organic products, or may not have a rewards program.
  • The answers don't include an open-ended Other option for respondents to select.
  • Over 10% of respondents will likely have trouble answering this question accurately.
  • By adding the Other option, participants can give an answer that hasn't been thought of by the researcher.

 

Ask this instead

You have answered that you visit Alice’s Grocery once a month. Why don’t you visit the store more often?

  • I find better prices elsewhere
  • Alice’s Grocery doesn’t carry all of the products I want to purchase
  • Other grocery stores are in more convenient locations for me
  • Other ___________________

 

 

More resources

 

Please provide any feedback you have on this article. Your feedback will be used to improve the article and should take no more than 5 minutes to complete. Article evaluations will remain completely confidential unless you request a follow-up. 

Was this article helpful?
3 out of 3 found this helpful