Rating Scale questions are the most utilized of the three types of UserTesting metrics questions. The responses provide terrific data for the creation of charts and graphs, which can help you convey to others how your team’s product is performing.
Here are some common examples of Rating Scale questions:
- How difficult (1) or easy (5) was it to log in to the app using Facebook? (1=Very difficult, 5=Very easy)
- How unlikely (1) or likely (5) would you be to recommend this game to a friend? (1=Not at all likely, 5=Extremely likely)
- How cluttered (1) or organized (5) do you consider the home page? (1=Cluttered, 5=Organized)
- How untrustworthy (1) or trustworthy (5) do you consider the site? Please explain your answer aloud. (1=Not at all trustworthy, 5=Entirely trustworthy)
- How disinterested (1) or interested (5) are you in purchasing boosters for this game? (1=Not at all interested, 5=Extremely interested)
- Please rate your agreement with the following statement: “I understand what I can do here and who this site is for.” (1=Strongly disagree, 5=Strongly agree)
Here are some tips to keep in mind when you write rating scale questions:
1. Clarify the endpoints (the labels for 1 and 5) of your scale.
Rating scales are highly adaptable, and they can be used to measure many different things, but only if the people taking your test understand what you’re asking them to measure!
Make sure that your question involves two ends of a spectrum (and that you’re effectively communicating those endpoints).
2. Include both sides of the scale in your question, as well as in the endpoint fields.
People are naturally eager to please, so if you ask users how easy it was to accomplish something, their inclination is to say it was easy—even if they struggled.
To avoid introducing bias into the response, it’s best to mention both ends of the spectrum as you ask your question (in this example, how difficult or easy). Including the both sides of the scale in the body of your question also reinforces the endpoints you’ve defined and lowers the chance that a user will misread your question.
3. 1 should always be the “pain” point of the scale, and 5 should always be the “positive” point.
Because we tend to think of the highest number as the positive point in a rating scale, we can easily overlook a scale that has been labeled the opposite way.
The UserTesting Research Team has repeatedly observed that if confronted with a question like this, users will often provide a positive verbal response (i.e. “It was great!”), but click the “5” radio button, despite the label saying that 5 = Low Quality.
So always label 1 as the “pain” point, and 5 as the “positive” point.
Pro tip: If you set up a rating scale question and realize that the two endpoints are neutral and/or the positive point is right in the middle, consider turning it into a multiple-choice question instead.
4. Measure one element or action at a time.
Write individual rating scale questions for each task that your users perform. If you’re worried about users mistaking the second question as a duplicate, you can always capitalize the task you are asking users to rate, for additional emphasis:
Want to try out Rating Scale questions for yourself?
Now you’re all set to start writing some stellar Rating Scale questions and getting some solid quantitative data with your next study. If you have any questions, contact your Customer Success Manager.
For more information, visit the original blog post "Using Rating Scale Questions in Your UX Research".