Tree Testing with Treejack and UserTesting

This article explains how to use Optimal Workshop’s tree testing tool, Treejack, in conjunction with the UserTesting platform. To learn about tree testing, including why it’s useful and when you should do it, read this article here.  

 

Why should you use Treejack and UserTesting together? 

When you invite participants to complete your Treejack tree test via the UserTesting platform, you’ll get two valuable types of data for making informed design decisions:

  • Treejack will help you understand how many people could find the right information, how many people failed, and the paths people took through your information before settling on an answer. You can use the results of your Treejack studies delivered via in-depth data visualizations (such as success, directness, and failure pietrees, participant paths, and first clicks) to make more informed decisions related to the organization and labeling of information on your website or app. 
  • UserTesting will give you behavioral insights from watching and listening to participants complete your tree test; and answers to questions such as “Did people find any task particularly confusing?” and “Why did people go the tree one way and not another?”

How to set up a Treejack study in the UserTesting platform

 Part 1 - Set up your tree test in Treejack

1. Create or upload your tree

Your tree is a text-only version of your website that looks similar to a sitemap. Click on the ‘Tree’ tab in your Treejack setup. There, you can upload your tree from a spreadsheet or create it manually using the tree builder:

TJ1.png

Using a spreadsheet is a fast way to create your tree, and is useful for iterating different 

versions of the same tree (especially when you want to A/B test different versions, or benchmark the before and after of one design).

On your spreadsheet, make sure each label has its own line, with the first column as “Home” and the subsequent columns as level two, level three, and so on (check out the example on the left below). When you upload the tree, your higher-level labels become ‘parent nodes’ and your lowest level labels become ‘child nodes’ (check out the example on the right below):

TJ2.png

2. Write your tasks

Once you’ve built your tree, you can start adding tasks on the ‘Tasks’ tab in your Treejack set up:

TJ3.png

Your tasks represent the typical actions your users would take on your website, and will help you to test the findability of different information on your tree. We recommend a maximum of 10 tasks for each tree — your participants will start ‘learning’ the tree after this, and might start using their memory instead of the labels on your tree to make their decisions.

Once you’ve added a task, click ‘Correct answers’ to see your tree and select at least one correct destination for that task.

Note: If you’re testing the tree of a particularly large website (such as government or eCommerce), there might be more than one correct destination, so make sure you select them all on the tree.

TJ4.png

3. Adjust Participant Identifier in the Questionnaire tab

Instead of asking participants for an email address, you should ask users type in their UserTesting username. This will help you easily match their Treejack results with their UserTesting recording when you analyze your data.

You might want to invite participants outside of the UserTesting platform to gather more quantitative data. These participants won’t have a username. They’ll still need to enter something into the identifier box, so you should tell these participants to enter something simple, like 123.



TreeJack4.png

4. Edit the welcome and thank you messages in the Messages tab

Participants see the ‘Welcome’ message when they click on the survey link you give them. The default message welcomes participants to the tree test, gives an estimated time frame and a short explanation of what the tree test is for.

Here’s an example message that you can tailor based on the particulars of your card sort:
'Welcome to this Treekjack study, and thank you for agreeing to participate. This activity shouldn’t take longer than 10 to 15 minutes to complete. Your response will help us to organize the content on our website.'

TreeJackWelcome.png

Participants see the “Thank You” message when they click “Finish”. The default message ends with the sentence, “You may now close this window or navigate to another web page.” If you are recruiting only UserTesting participants, use this message to direct participants back to their task box:

'Thanks for completing our tree test. To continue, please return to the UserTesting task box in the top right corner of the screen.'

TreejackThankYou.png

If you plan to recruit other participants (not via UserTesting) remote participants to get more data, add a note for those participants as well, such as 'If you are not a UserTesting participant, you can now close this window or navigate to another webpage.'

5. Write pre- or post-tree test questions

You can ask your participants to answer pre and post survey questions to give you more qualitative insights, and to enable you to segment your results based on their answers in your analysis.

If you plan to recruit UserTesting participants only, you could choose to only ask these questions in the UserTesting platform. If you plan to recruit participants outside the UserTesting panel as well, then asking the questions in both platforms will mean all your participants get a chance to answer them.

You might want to ask questions like:

  • Overall, how difficult or easy was it to locate what you needed in this menu?
  • What, if anything, would you change about the menu you just tested? Please explain your answer.
  • Which, if any, of the tasks in Treejack were difficult to complete? Please explain your answer.
  • Is the menu you just tested BETTER, WORSE, or ABOUT THE SAME as other web sites you have visited? Please explain your answer.

TreejackQuestion.png

6. Preview your survey before launch to ensure accuracy

The Preview button is your best friend when it comes to getting your tree test right. Preview enables you to take your tree test from your participants’ point of view, and makes spotting errors or inappropriate tasks much easier.

OS6.png

When you preview your survey, check that:

  • Participants are required to give their UserTesting username (or another participant identifier if they aren’t UserTesting participants).
  • The 'Welcome' and 'Thank you' messages are suitable and error-free.
  • Your tree and tasks contain the right content and are error-free.
  • Your pre- or post-tree test questions are clear.

7. Launch your survey and copy the link

Before you can create and launch finalize your UserTesting survey, you’ll need to launch your tree test and copy the link into your UserTesting invitation. You’ll get a link to your tree test when you launch it, and you can also find it on the ‘Setup’ tab.


OS7.png

Part 2 - Set up your UserTesting tree test with your survey link

1. Create a new unmoderated test in UserTesting. 

NewUnmod.jpg

 

 

Next, click Test a Website.

Website_App_Prototype.png

Then, you'll have the option to choose between testing with the UserTesting Panel or your own customers.  In this example, we'll be using the UserTesting Panel.

Panel_or_My_Recruit.png

 

2. Select your sample size and target audience in UserTesting

Usually, when running a tree test, it’s best to opt for a larger sample size so you can get solid quantitative data. However, we recommend only asking a portion of your participants to record themselves via UserTesting as they complete the activity in Treejack. 

Typically, we find that 8-10 users will give you the qualitative insight you need.



TreeJack6.png

3. Choose a starting URL (not the Treejack link) and provide some context

In the Starting Instructions portion of UserTesting, choose a URL to send users to, such as www.google.com. Don’t use the Treejack link just yet. You want to provide more context and instruction before users get to the tree test activity.

You’ll also want to provide some context in the Introduction field. We’ve had success with the following:

Please note: In this test, you’ll be asked to use Treejack. Treejack is an online program that helps companies evaluate how easy or difficult it is to find specific information within their menu.

TreeJack7.png

4. In the first task, include the Treejack link and descriptive instructions, which provides details about how the participant should “end” the test in both UserTesting and Treejack

The UserTesting Research Team recommends the following text for the first task for additional clarity before sending users to the Treejack activity.

'Please visit the following link to access Treejack: [URL]

Follow the Treejack instructions, speaking your thoughts aloud as much as possible. When you finish the last task, answer any follow-up questions in Treejack, and click the “Continue” button.

When you have completed all the tasks in Treejack, click NEXT in the upper right of this UserTesting task box.'

Treejack.png

 

Note: The Treejack URL can be found in the Survey Address field on the Settings tab in the Treejack dashboard.

IMAGE

5. Include follow-up questions to get additional insight from users.

Finally, the UserTesting platform allows you to ask follow-up questions to evaluate the user’s experience as they completed the tree test. Examples are listed below:

  • Overall, how difficult or easy was it to locate what you needed in this menu?
  • What, if anything, would you change about the menu you just tested? Please explain your answer.
  • Which, if any, of the tasks in Treejack were difficult to complete? Please explain your answer.
  • Is the menu you just tested BETTER, WORSE, or ABOUT THE SAME as other web sites you have visited? Please explain your answer.

Other Resources

Was this article helpful?
13 out of 13 found this helpful