Rank order questions

This article applies to: UserTesting 

On this page:

• What is a rank order question?
• Add ranking questions
• Ranking question in tasks
• Review ranking question
• Participant experience
• Ranking question results
• Mean calculation

What is a rank order question?

Rank order questions ask contributors to arrange items in order of priority. They help you:

• Identify top preferences at a glance
• Reduce bias with answer randomization
• Gather insights for faster decision-making

Add ranking questions

1. Go to the Test builder and click +Add on the left.
2. Choose Ranking under the Questions options.
3. Type your question.
4. You may select pre-made ranking labels or create your own.
   
    
5. Type the items that participants will be asked to rank. You can also paste all your answers into the first row.
   
    
6. Apply the randomize items toggle if you want the items to be displayed to each participant in a random order.
   
   If you want to add branching logic, create an additional question, then return to the first question and select Add logic.

Ranking question in tasks

• In Interaction tests, ranking questions can also be added as a follow-up question in Navigation and Figma tasks.
• In this instance, they are shown to the participant within the floating taskbar while the participant still has context of the task.

Review ranking question 

The Review tab shows your ranking question and the items you’ve added to it.

Participant experience

• In the Desktop view, participants can drag items from left to right to place the items in their preferred position.

• In the Mobile experience, participants can drag items from bottom to top, or they can choose the position for an item from the available dropdown within each item.

Ranking question results

Ranking results are presented in this tabular view.

• The table shows the ranking position distribution of each item, as well as the mean position for each item
• The table can be sorted ascending or descending by Mean
• Filters can be applied through the table to show or hide all participants who placed an item in a specific position

Mean calculation

• The mean calculation used is a weighted mean, which is the same approach used in the Matrix question.
• The weighted mean for each item in a Rank Order question is calculated by multiplying the number of times the item appears in each rank position by the numerical value of that position (e.g., 1st = 1, 2nd = 2, etc.).
• These values are summed across all positions and then divided by the total number of responses to get the average ranking for that item.
• Lower scores indicate higher preference.

Related content