Analyze your results
Once you’ve launched your study and gotten your results back, it’s time to get to work on analysis. Before you begin to analyze your results, think back to your objectives. Study sessions are full of rich feedback from participants, and it can be overwhelming to know where to start. Keeping your study objectives front of mind will help you sift through feedback and gain actionable insights into your participant’s experience.
As you analyze your studies, continue to ask yourself the question “Is this directly relevant to my objectives?” For example, if you are curious about the usability of a checkout process and a participant comments on their interest in buying a certain product, this could be an interesting topic for another study, but is not necessarily a usability issue related to your current objectives.
With your objectives in mind, it’s time to jump into the data and videos. Here are some steps to help you get started.
Ideally, when thinking about how to analyze your test results, you’d want to watch all the participant videos in their entirety, and, time permitting, make annotations throughout. This level of focus can result in a deeper, more holistic understanding of each participant’s experience and result in insights that best serve your testing objectives.
If you’re pressed for time, you can use metrics to quickly see a high-level summary of the data from rating scale, multiple-choice, and written response questions. The metrics in our platform are subjective ratings of ease, difficulty, preference, or time on task. These can be used to uncover overall trends in participants’ experiences without having to watch every video.
You can access your study's metrics by selecting the Metrics tab in a study or the Export to Excel button.
Try to answer your study objectives by summarizing the data and seeing if there are clear trends, interesting patterns, or surprising responses. If a participant provides multiple low ratings or if the overall ratings are mixed, it may indicate that there are important qualitative moments in the study sessions. It’s important to remember that metrics don’t tell the whole story; you can investigate these moments by clicking on each rating and seeing what experiences led to that rating.
This is a great opportunity to dig into these sessions and understand where these ratings came from.
After using the metrics to initially understand the data at a high level, you can dive further into the videos to get a sense of where these experiences come from and why they happen.
Please see our “Using the Metrics tab” Knowledgebase article to get an overview of the Metrics tab—why it’s used and what types of tests it best supports.
With sentiment analysis, get quick access to important moments of positive or negative sentiment when you are reviewing a completed session in the UserTesting video player. This feature uses natural language processing of phrases and sentences in Transcripts, so you will need an advanced, ultimate, or premium subscription to access this feature.
If your plan allows, you may have access to smart tags and smart tags for written tasks. With smart tags, you can capture a total of 10 smart tags in the feedback that cover positive, negative, and neutral moments. For written tasks, smart tags are added in-line with the written response on the metrics tab and are color coded according to the overall moment.
Notes and tags
While reviewing your videos, you can create notes to bookmark or highlight interesting moments or observations related to your objectives. You can use them to capture points where the participant has gotten stuck or frustrated, finds something really useful, answered a question, or has reached a certain part of the process.
Tags are a set of keywords used to categorize notes into patterns or trends. Tags should be guided by your objectives, and refer to the specific outcomes or behaviors that you’re interested in (such as, “did they notice the call to action?”). See some of our recommended tags for organizing notes and clips here.
All you have to do is add a #tag to any note or clip summary. Adding an end-timestamp to an annotation will automatically turn it into a clip.
As you go through a video of a study session, create and tag notes wherever you observe an event or behavior that is relevant to your objectives. When you create a note, write down what you observed, when it happened and what may have caused it. When you’re reviewing your data later or sharing it with a colleague, this will help to clarify the context of your observations.
Mark as important
As you create clips, there’s a star icon that you can select to mark notes and clips as important.
You can also see if teammates have marked clips as important:
- Marked as important by you and others
- Marked as important by others but not you
- Marked as important by you
- Not marked as important (default state when you first create the clip)
Here are some suggestions for using this feature:
- Mark clips you think are important. These carry over to the Highlight Reel Editor for immediate access and use.
- Hone in on insightful clips other teammates marked, saving everyone time.
- Focus on the clips with the most engagement, then share these key insights with stakeholders and decision-makers.
As you gather observations, you may notice patterns where participants encounter issues. Documenting these patterns will help you to discover insights, create discussion points for team meetings, and identify opportunities for improvement.
Try to answer the questions you posed at the start of the study by identifying and tabulating the common patterns and trends in the videos you watched. Review your notes and summarize them. What issues are most participants encountering? Next, dig a little bit deeper into these issues and try to understand why they occurred.
In this example, some participants agreed that the homepage of a website was attractive, however, a most participants were on the other end of the scale.
While you may be focused on the issues participants are encountering, it’s also important to recognize the things that people love too. By paying attention to both, you’ll avoid inadvertently trying to “fix” something that’s not broken when attempting to improve the user experience.
Harness the power of the spreadsheet
It’s easy to gather, analyze, and share your findings right within the UserTesting dashboard. You’ll be able to see time on task, responses to metrics questions, and more at a glance.
However, if you find that you want to perform more detailed analysis, you can download the data from your study into an Excel spreadsheet. This can be especially helpful for compiling findings from multiple studies side by side, breaking down patterns in studies with lots of participants, and comparing responses from different demographic groups.
Share your findings
Now that you’ve observed your data, you’re ready to share your insights!
UserTesting highlight reels offer an easy way to capture and share the most relevant behaviors and trends from your study so members of your team can quickly access the key findings. String together clips of your observations into a 2-3 minute highlight reel to share your findings.
To make a highlight reel, select the “Highlight Reels” tab in the dashboard. Drag the clips you’d like to add to the highlight reel into the area on the right. Give the highlight reel a title and then select Save Highlight Reel.
On the highlight reel editor, you can use your browser’s “ctrl+F” function to quickly find tags and annotations to add to a compilation.
Other ways to share your videos
Easily share videos with a colleague or stakeholder by selecting the checkboxes for the specific video(s). You’ll have the option to:
- Export to Excel: Exports all the study information to an Excel file.
- Publish Videos: Publishes a custom page with the videos (this option does not require you to add people to your account.)
- Move to a Folder: Easily organize your videos.
You can also share videos from within the video player:
- Download Videos: Downloads the video to your computer.
- Send Videos: Sends an email with links to the videos
Sharing research findings with stakeholders and colleagues in multiple departments can be a great way to promote a user-centered culture in your company.
Here are a few other ideas for successfully relaying your findings:
- Use charts to represent any interesting metrics data from your questions.
- Back up your claims with user quotes from the studies.
- Be careful not to place blame on any of your teammates. If you have a lot of negative findings, choose your words carefully. “Users found this feature frustrating” is much easier to hear than “This feature is terrible.”
- Encourage team members to ask questions about the findings, but remind them not to make excuses. They’re there to learn about the customer experience, not to defend their design decisions.
- Keep your objective in mind when reviewing data and videos.
- Use metrics to quickly see a high-level summary of your data.
- Create and tag notes wherever you observe an event or behavior that is relevant to your objectives.
- Once you’ve established your findings, it’s time to present them to your stakeholders.
- String together clips of your observations into a 2-3 minute highlight reel to share your findings.