Card sort tasks in the new experience

On this page:

• Availability
• Create a Card Sort
• Analyze a Card Sort
  • Dendrogram tab
  • Placement tab
  • Categories tab
  • Cards tab
  • Responses tab
• FAQs

Availability

New Card sort tasks are available in the new experience for:

As for the card sort types available in the new experience:

Create a Card Sort

Follow these steps to add a Card Sort to your test:

1. Go to the Test Builder tab of your test.
2. Click on the Add button on the left-hand side menu.
3. Choose Card Sort.
   Currently only Closed card sorts are available. Open and Hybrid are coming soon.
   Refer to the article "What is Card Sorting?" regarding how these options differ.
4. Configure the card sort task
   1. The instructions are pre-filled, but are editable.
      Participants will get instructions on how the card sort feature works prior to the task. See FAQ section for a screenshot.
   2. Under Cards
      • Type in your cards one by one 
        or
      • Paste all the cards into the field for the first card.
        Each item should be on a new line.
   3. Under Categories
      • Type in your categories one by one 
        or
      • Paste all the categories into the field for the one.
        Each item should be on a new line.
         

Unsure how many participants you need for a card sort? Refer to our sample size recommendations article. 

Analyze a Card Sort

The results section offers various tabs for card sort tasks. Each tab shows the results of your card sort from a different angle, giving you a fuller picture of how participants grouped the cards.

Dendrogram tab

A dendrogram is an interactive visual representation in the form of a tree that shows how participants grouped items together (based on co-placement behavior, not based on categories). 
 

A dendrogram consists of various elements including:

• Clusters which are the groupings created by participants and are shown as light blue boxes. 
• Agreement line which indicates the percentage of participants (33% in the example below) that agree that the current clusters are the "right" clusters. You can slide it left and right (refer to the "Agreement and clusters" section below).

How to interpret a dendrogram

A dendrogram illustrates the strength of relationships between items, and between groups of items:

• The further left two branches merge, the more participants agreed that these cards belong together.
  In the example above, the mango, apple, banana, and strawberry branches merge on the very left because they were grouped together by all participants.
• The further right two branches merge, the fewer participants agreed that these cards belonged together.
  In the example above, the fruits and vegetables branches merge first, before they merge with the furniture branch on the very right.

Agreement and clusters

The number of clusters is not static and depends on:

• the placement of the agreement line (at 33% in the example above) or
• the number of clusters you want (dropdown at the bottom left)

As you slide the agreement line between 100% and 0%, the number of clusters changes:

• Sliding it left gives you more clusters, but smaller and more homogenous ones.
  In the example, at 100% agreement we get 6 clusters: furniture, fruit, vegetables, and three singles — cucumber, tomato, and avocado — since participants disagreed on where they belong.
• Sliding it right gives you fewer clusters, but bigger and less homogenous ones.
  In the example, at 1% agreement, we get 2 groups: fruits/vegetables (a broader group) and furniture.

Linkage methods
Linkage methods define how the distance between clusters is calculated. Different linkage methods can produce different trees from the same data. You can choose between 3 linkage methods:

• Average linkage (most balanced)
  Uses the average distance between all pairs of cards across two clusters
• Single linkage (broader)
  Uses the distance between the closest pair of cards across two cluster
• Complete linkage (tighter) 
  Uses the distance between the furthest pair of cards across two clusters

Placement tab

It provides a table where each cell shows what % of participants placed that card in that category. 

Categories tab

The categories tab provides a table which indicates for each category:

• all the cards placed in the category
• how many participants placed each card
• the agreement score that indicates the percentage of participants who agreed that these cards belong in a category.

About the agreement score

The agreement score for categories is expressed as a color-coded percentage:

Cards tab

The cards tab provides a table showing for each card:

• how many categories it was placed into
• the names of the categories it was placed into
• how many participants placed it in each category

Responses tab

The responses tab shows for each participant's:

• their clusters as well
• their recording 
• their time-on-task

FAQs

Q: Do I need to add instructions around how a card sort works?
A: No, participants will receive instructions on how to sort the cards before the task  (see screenshot below). Participants can open the instructions back up whenever they need a reminder (“Open instructions” option at the top left).

Q: Will participants see the Card Sort while answering follow-up questions?
A: Participants will see their Card Sort responses in read-only view while they answer any follow-up questions added within the task group ("Popular follow-up questions" or "Add follow-up question" options).

Q: Do participants need to sort all the cards?
A: Yes, participants cannot continue to the next task until all cards have been moved into a category and will get a warning about it if needed.

Q: Can participants complete a card sort task using their keyboard?
A: Yes. In addition to dragging and dropping cards into a category, participants can also use the dropdown on each card to move it into a category. This allows them to complete the task using their keyboard.

 

Q: Can I add an image to a card?
A: This functionality isn't available at the moment, but is being assessed. If you need to use images in your card sort, you can create a classic Card Sort instead.

Q: How can I create an open or hybrid card sorting in UserTesting?
A: At the moment, in the new experience, only closed card sort tasks are available. For open and hybrid card sorts, you'd need to create a classic Think-out-loud test instead.