Gathering Factual Responses vs. Subjective Data

At a Glance

Asking the right questions in the right way is necessary to gain accurate insight from your user tests. This article has effective tips on how you can you gather factual and subjective data that you can trust.

When creating a user test, you need to ask the right question in the right way. You want results you can trust and that you can comfortably base your decisions on.

It’s important to structure questions accurately and strategically in order to maximize your user tests and gain the insights that will really help you move your project forward.

Here are some tips to help you gather a range of data (both factual and subjective) by asking your questions just the right way.

Tips for gathering factual responses

1. Don’t use industry jargon

Terms like “sub-navigation” don’t make sense to the average user. Define key terms or concepts in the questions themselves (unless the goal of your study is to see if they understand these terms/concepts).

2. Define the timeline

If you are asking about some sort of frequency, such as how often a user visits a particular site, make sure you define the timeline clearly. Always put the timeline at the beginning of the sentence.

Bad: How often do you visit Better: How often did you visit in the past six months? Best: In the past six months, how often did you visit

3. Break up complicated questions

Don’t try to pack all the concepts into one question.

4. Ask specific questions to get specific answers

If your respondent can give you the answer, “It depends,” then you probably need to make your question more specific.

5. Ask about first-hand experiences

Ask about what people have actually done, not what they will do or would do. Try your best to avoid hypotheticals and hearsay.

Tips for gathering subjective data

1. Leave breadcrumbs

Remind them where they should be on the site. To ensure they’re where you want them to be, provide a URL for them to click so they are looking at the right part of the site or app.

2. Don’t make the user feel guilty or dumb

Place the burden of blame on the website, product, or app so it’s not the user’s fault, it’s the site’s fault.

3. Make sure your rating scale questions aren’t skewed

Be fair, realistic, and consistent with the two ends of a rating spectrum.

Bad example: “After going through the checkout process, to what extent do you trust or distrust this company?” (I distrust it just a tad ←→ I trust it with my life) Good example: “After going through the checkout process, to what extent do you trust or distrust this company?” (I strongly distrust this company ←→ I strongly trust this company)

4. Subjective states are relative

“Happy” in one context can mean something very different than “happy” in another context. Be clear about whether “happy” means the opposite of neutral or the opposite of unhappy.

  • Option 1: Happy / Not Happy (Happy = Opposite of not happy)
  • Option 2: Happy / Neutral / Unhappy (Happy = The best!)
  • Option 3: Very Happy / Happy / Unhappy / Very Unhappy (Happy = Just above neutral)

5. Ask many specific questions, rather than a few all-encompassing questions

When you ask questions about vague or complex concepts, users often don’t know how to answer them. Break concepts up when you’re asking the questions and put them back together when you’re analyzing the results.

Learn More

Need more information? Read these related articles.

Visit the original blog postHow to Write Great Questions for Your Next User Test”.


Was this article helpful?
10 out of 11 found this helpful