Card Sorting with UserTesting: Analyzing the Test Results

Click on the following header titles to skip to that section of the article:

On this page:

• View Card Sort Results on the UserTesting Platform
• View Card Sort Results on the Subdomain
• Share Card Sort Results with Team Members

View Card Sort Results on the UserTesting Platform

If you set up your card sort with the integrated card sorting tool, you can view your results on the Metrics tab on the UserTesting Platform. To observe contributors attempting to complete the card sort test, go to the Metrics tab in the Platform. Navigate to the Card sort task. You'll see...

1. A highlight reel (Watch task...) of all contributors doing the card sort activity.
2. An Overview tab that shows the card and category labels, and numbers showing how contributors matched labels and categories, which you can sort by percentage or total number (count). You can also merge categories for open and hybrid card sorts if contributors created similar categories.
3. An Individual Responses tab that contains clips of individual contributors doing the card sort activity, as well as a summary of how they sorted the cards.

View Card Sort Results on the Subdomain

If you set up your card sort on ia.usertesting.com, you can view your results on the subdomain.

Note: If you're not the test creator or an admin, you'll not be able to access the results in the subdomain.

Select the test you wish to review.

On this test page, you can view the contributors. If you have contributors who did not complete much of the card sort activity, you can exclude them here.

Note that you may see more contributors in the card sort than in your UserTesting test. The UserTesting system will automatically replace any contributors who abandon their test. So, if any contributors start the card sort activity but do not complete their UserTesting test, their data will be included in the card sort activity. But they will not be reflected in the list of completed sessions.

The card sort results include different views where you can see how contributors grouped content and how often other cards were grouped.

• • Raw numbers: This table represents how many cards were dropped into each category by all contributors combined. This view of the data is useful to export and manipulate externally - this is especially helpful in comparing results from multiple similar studies (i.e., the same card sort distributed to different audiences).
    

Images are also visible when you review the results of your card sort. View images when you hover over the thumbnails on the raw data, percentage table, grouped percentages, and distance matrix tables. Images are not viewable on the MDS or dendrogram tabs.

• • Percentages: This table represents how many cards were dropped into each category above a threshold you can adjust. The default is a 5% threshold, which means that the results below that threshold have been omitted. A higher percentage indicates the card was placed in that category by more contributors. A lower percentage split across multiple categories indicates some ambiguity in terms of how to classify a particular card.
    
    

• • Grouped percentages: This table represents how many cards were dropped into each category above a threshold and grouped based on the most cards associated with these categories. The default is a 5% threshold, which means that the results below that threshold have been omitted. You can adjust the threshold level. This visual grouping of cards makes it easier to understand which cards are related.
    • If no cards are highlighted for a particular category, that category may not be relevant or may need to be merged with another category.
    • If evenly split between multiple categories (i.e. same percentage) a card will be grouped with the first category on the list.
      
  • Distance matrix: This table captures the relationship between individual cards (without consideration for the specific categories chosen). The number for each cell is based on the number of contributors that did not group the two cards in the same category. The lower the number, the more similar the two cards are, and the higher the number, the more dissimilar they are.
    
  • MDS (Multidimensional Scaling): This diagram is an analysis and visual representation of the distance matrix. You can analyze any kind of similarity or dissimilarity matrix using multi-dimensional scaling. Plotting these data sets on a multi-dimensional scale allows for easier interpretation and comparison than a linear dataset permits. When reading an MDS map, it represents the relative distances between cards - the scale of the axes has no impact on the relative distance between cards. Based on the data, similar cards will be clustered together. The proximity of the clusters to one another also indicates how similar (close) or dissimilar (far apart) they are from one another.
    
    
  • Dendrogram: This diagram represents a Hierarchical Cluster Analysis, which groups the cards that were considered most similar by the contributors and places them on branches that are close together. This is sometimes called a tree diagram. The longer the line (or branch) that connects to cards, the more dissimilar they are. Shorter lines indicate cards that are more similar to each other. The key to interpreting a dendrogram is to focus on where any two cards are joined together .
    • If the cards are joined together on the left of the dendrogram then those cards are more different—i.e. the relative distance between them on the MDS is large.
    • If the cards are joined together on the right of the dendrogram, then the cards are more similar—i.e. the relative distance between them on the MDS is small.
    • Dragging the red bar horizontally (right-left) highlights how the clustering of cards changes depending on where the line is drawn. This can be helpful as you consider merging categories and finding the right number of clusters for your application.
      

Share card sort results with team members/colleagues

To share the results of a card sort, open the Advanced Options drop-down menu under the Test Details section...

...and select the Shared button found at the bottom of the drop-down.

This will generate a SHARED link that appears at the bottom of Test Details...

A couple of things to note about this SHARED link:

• It will be generated only when the Shared button has been activated.

• It will return data only when contributors have taken the test.
• It will only show team members the Raw Numbers tab in the ia.usertesting subdomain. It will NOT show percentages, grouped percentages, distance matrix, MDS, and dendrogram results.

The XLS or CSV file for raw numbers, percentages, grouped percentages, and distance matrix can be downloaded from the card sort Results tab and shared with team members. There is no export option for the dendrogram or MDS visualizations; if you wish to share these results, you can take a screenshot of the visualizations.

Please provide any feedback you have on this article. Your feedback will be used to improve the article and should take no more than 5 minutes to complete. Article evaluations will remain completely confidential unless you request a follow-up.