When creating a user test, you need to ask the right question in the right way. You want results you can trust and that you can comfortably base your decisions on.
It’s important to structure questions accurately and strategically in order to maximize your user tests and gain the insights that will really help you move your project forward.
Here are some tips to help you gather a range of data (both factual and subjective) by asking your questions just the right way.
1. Don’t use industry jargon
Terms like “sub-navigation” don’t make sense to the average user. Define key terms or concepts in the questions themselves (unless the goal of your study is to see if they understand these terms/concepts).
2. Define the timeline
If you are asking about some sort of frequency, such as how often a user visits a particular site, make sure you define the timeline clearly. Always put the timeline at the beginning of the sentence.
Bad: How often do you visit Amazon.com? Better: How often did you visit Amazon.com in the past six months? Best: In the past six months, how often did you visit Amazon.com?
3. Break up complicated questions
Don’t try to pack all the concepts into one question.
4. Ask specific questions to get specific answers
If your respondent can give you the answer, “It depends,” then you probably need to make your question more specific.
5. Ask about first hand experiences
Ask about what people have actually done, not what they will do or would do. Try your best to avoid hypotheticals and hearsay.
1. Leave breadcrumbs
Remind them where they should be on the site. To ensure they’re where you want them to be, provide a URL for them to click so they are looking at the right part of the site or app.
2. Don’t make the user feel guilty or dumb
Place the burden of blame on the website, product, or app so it’s not the user’s fault, it’s the site’s fault.
3. Make sure your rating scale questions aren’t skewed
Be fair, realistic, and consistent with the two ends of a rating spectrum.
Bad example: “After going through the checkout process, to what extent do you trust or distrust this company?” (I distrust it just a tad ←→ I trust it with my life) Good example: “After going through the checkout process, to what extent do you trust or distrust this company?” (I strongly distrust this company ←→ I strongly trust this company)
4. Subjective states are relative
“Happy” in one context can mean something very different than “happy” in another context. Be clear about whether “happy” means the opposite of neutral or the opposite of unhappy.
- Option 1: Happy / Not Happy (Happy = Opposite of not happy)
- Option 2: Happy / Neutral / Unhappy (Happy = The best!)
- Option 3: Very Happy / Happy / Unhappy / Very Unhappy (Happy = Just above neutral)
5. Ask many specific questions, rather than a few all-encompassing questions
When you ask questions about vague or complex concepts, users often don’t know how to answer them. Break concepts up when you’re asking the questions and put them back together when you’re analyzing the results.
For more information, visit the original blog post “How to Write Great Questions for Your Next User Test”.