Rating Scale questions are the most utilized of the four types of UserTesting metrics questions. The responses provide terrific data for the creation of charts and graphs, which can help you convey to others how your team’s product is performing.
Here are some common examples of Rating Scale questions:
- How difficult (1) or easy (5) was it to log in to the app using Facebook? (1=Very difficult, 5=Very easy)
- How unlikely (1) or likely (5) would you be to recommend this game to a friend? (1=Not at all likely, 5=Extremely likely)
- How cluttered (1) or organized (5) do you consider the home page? (1=Cluttered, 5=Organized)
- How untrustworthy (1) or trustworthy (5) do you consider the site? Please explain your answer aloud. (1=Not at all trustworthy, 5=Entirely trustworthy)
- How disinterested (1) or interested (5) are you in purchasing boosters for this game? (1=Not at all interested, 5=Extremely interested)
- Please rate your agreement with the following statement: “I understand who this site is for.” (1=Strongly disagree, 5=Strongly agree)
Here are some tips to keep in mind when you write rating scale questions:
1. Clarify the endpoints (the labels for 1 and 5, 7, 9, or 11, depending on which point-scale you selected) of your scale
Rating scales are highly adaptable and can be used to measure many different things, but only if the people taking your test understand what you’re asking them to measure!
Use the dropdown in the question to select pre-written endpoint levels, or create your own.
2. Include both ends of the scale in your question in addition to the endpoint fields
People are naturally eager to please, so if you ask participants how easy it was to accomplish something, their inclination is to say it was easy—even if they struggled.
To avoid introducing bias into the response, it’s best to mention both ends of the spectrum as you ask your question (in this example, how difficult or easy). Including the both sides of the scale in the body of your question also reinforces the endpoints you’ve defined and lowers the chance that a participant will misread your question.
3. 1 should always be the “pain” point of the scale, and 5, 7, 9, or 11 (whichever point-scale you selected) should always be the “positive” point
Because we tend to think of the highest number as the positive point in a rating scale, we can easily overlook a scale that has been labeled the opposite way.
The UserTesting Research Team has repeatedly observed participants providing a positive verbal response (i.e. “It was great!”), but clicking the “5 = Low Quality” radio button, despite the negative endpoint label.
Pro tip: If you set up a rating scale question and realize that the two endpoints are neutral and/or the positive point is right in the middle, consider turning it into a multiple-choice question instead.
4. Measure one element or action at a time
Write individual rating scale questions for each task that your participants perform. If you’re worried about participants mistaking the second question as a duplicate, you can always capitalize the task you are asking participants to rate, for additional emphasis.
Want to try out Rating Scale questions for yourself?
Now you’re all set to start writing some stellar Rating Scale questions and getting some solid quantitative data with your next study. If you have any questions, contact your Customer Success Manager.
For more information, visit the original blog post Using Rating Scale Questions in Your UX Research.