|
Refer to this guide to write non-leading questions. |
This article applies to: UserZoom
✅ Plan Availability: All Plans
👥 User roles: Owner, Admins, and Researchers
For more information, see our article on how to find your plan and user role.
On this page:
- Avoid leading words and questions
- Avoid yes or no questions
- Ask direct questions
- Prefer not to answer options
- One question at a time
- Cover all answer choices
Avoid leading words and questions
- A leading question subtly encourages participants to answer in a particular way.
- Small differences in wording can have a large impact on your data.
- Non-specific wording leads to ambiguity and confusion in your study.
- Words like could, should, or would produce significantly different results in a question, even though these words sound familiar.
Don't ask
Do you find anything about the website to be challenging?
- This question prompts participants to question their experience on the website.
- Subtly, it raises the prospect that there are challenges even if the participant didn't perceive them initially.
Ask this instead
Please rate your experience on the website in terms of usability (scale below).
- This question doesn't imply challenges using the website.
- Instead, it asks participants to evaluate their experience and report findings from their point of view.
Avoid yes or no questions
- Similar to leading questions, avoid using questions with yes or no answers.
- Some participants in online research are trained to recognize these questions as disqualifying points.
- They are encouraged to select Yes to qualify for the study.
- This potentially biases the resulting data.
Don't ask
Have you taken out a mortgage in the last 6 months?
- Yes
- No [Disqualify]
Participants are more likely to select Yes to qualify for the study.
Ask this instead
Which of the following have you done in the last 6 months? Please select all that apply.
- Moved homes
- Taken a mortgage [Must select]
- Bought a car
- Taken a vacation
- Opened a new bank account
- None of the above [Disqualify]
Participants won't know which answer to select to qualify for the study and are more inclined to give honest answers.
Ask direct questions
- Questions that are vague and ambiguous can limit the usefulness of your results.
- Make sure your questions are direct enough so the respondent has no doubt about what you are asking.
- Participants are not professional testers, so if you expect to get insights on something specific, make it known in the question.
Example 1
Don't ask
How would you suggest improving the website landing page?
Although you're looking for suggestions about the product visuals on your website, you may end up getting responses about the navigation structure, the amount of wording, or the color scheme.
Ask this instead
How would you suggest improving the product visuals on the website?
Example 2
Don't ask
Do you watch television regularly for work?
- What does regularly mean in this question?
- Some participants may interpret it as meaning every week.
- Others may interpret it as meaning a couple of times a year.
Ask this instead
How often do you watch television?
- Every day
- 2-3 times a week
- A few times a month
- I rarely watch television
Prefer not to answer options
- There are instances where participants aren’t comfortable with you, as the researcher, collecting specific data points.
- Questions about finances, family life, occupation, sexual orientation, and religious beliefs can be seen as too intrusive to some participants.
- Privacy is a highly important factor in online research and is equally as important to respondents.
- Rather than rejecting these participants or forcing them to drop out of the study, offering a prefer not to answer option allows you to collect their feedback on the true research aim of the study.
- Unless the information is key to the research objective, including a prefer not to answer option is always recommended.
One question at a time
It is often tempting to ask multiple questions at one time, but this can cause issues for respondents and influence their responses.
Don't ask
What is the fastest and most economical cable television provider for you?
- This question is really asking two different things:
- What is the fastest cable television provider for you?
- What is the cable television provider for you
- The answers may be completely different.
- It would be best to separate these into two questions or provide answer options for each part, separately.
Ask this instead
What is the fastest cable television provider for you?
What is the most economical cable television provider for you?
Cover all answer choices
- Do your closed-ended answer choices include all possible respondent answers?
- A rule of thumb to follow: If 10% of respondents (in a pretest or otherwise) select Other, you are likely missing a crucial answer choice.
- Review the Other responses and add the most frequently mentioned answer(s) to your question.
Don't ask
What You have answered that you visit Alice’s Grocery once a month. Why don’t you visit the store more often?
- I find better prices elsewhere
- Alice’s Grocery doesn’t carry all of the products I want to purchase
- Other grocery stores are in more convenient locations for me
- The closed-ended answer choices in the above question don't cover all possible reasons a participant may not visit the store more often.
- For example, Alice’s Grocery store may not carry specific brands, offer a variety of organic products, or may not have a rewards program.
- The answers don't include an open-ended Other option for respondents to select.
- Over 10% of respondents will likely have trouble answering this question accurately.
- By adding the Other option, participants can give an answer that hasn't been thought of by the researcher.
Ask this instead
You have answered that you visit Alice’s Grocery once a month. Why don’t you visit the store more often?
- I find better prices elsewhere
- Alice’s Grocery doesn’t carry all of the products I want to purchase
- Other grocery stores are in more convenient locations for me
- Other ___________________
Related content
|
|
|
|
Want to learn more? Check out these Knowledge Base articles... |
Interested in growing your skills? Check out our University courses... |
|
|
|
|
Need hands-on training?
|
Can't find your answer? Reach out to Support or our Professional Services team. |