Quick Answers are recorded video feedback of customers or prospects as they complete tasks and answer questions. The tasks and questions are written for you by UserTesting, and designed to answer the most common marketing challenges. You can read more about Quick Answers in Marketing Insight: Getting Started.
“Choose between two options” is one of seven types of Quick Answers in Marketing Insight. With it, you can learn which of two options people find more compelling and why.
After running this Quick Answer test, the Summary tab shows how many test participants prefer Option A or Option B, and provides two automatically generated highlight reels - split based on their responses - of them discussing their answers and explaining why they answered the way they did. If you want to dig into the findings, there are several options. The results table below the highlight reels provides additional feedback from participants about how they feel about that particular experience or activity. The Metrics and Session tabs provide the full recordings broken out by question or participant respectively.
When to use "Choose between two options"
Use “Choose between two options" to:
- Compare two sets of creatives, images, messaging, or designs
- Pre-test variants for A/B testing to understand whether the two options are even the right ones to test
- Compare your image, message, creative, etc. vs a competitors
- Get insights about how customers perceive a new design or image vs. one you used in the past
How to Set up "Choose between two options"
1. Quick Answer Inputs
From the dashboard click Get Quick Answers, then click “Choose between two options.”
Enter a publicly accessible link to a page or document that shows or describes the content. The link needs to point to a single document, hosted online, and accessible and readable by a typical browser. The link can be to a Microsoft or Google doc, a Slides or PowerPoint presentation, a PDF, image, text, or anything similar. This file should include both options being compared.
Then name your test so it’s easy to recognize it from other tests on your dashboard. Marketing Insight displays all your account’s tests on your dashboard. This makes it easy to share tests with other stakeholders. When you’re done, click Select Demographics.
2. Select Demographics
The UserTesting panel is a diverse and global on-demand group of people who take tests. The panel is made up of real people, carefully selected after a thorough application process, who are continually rated by our customers, assuring the highest quality results. To find participants you’ll be able to select device type, set demographic filters, and create custom screener questions. These are detailed in Marketing Insight: Find the Right Participants.
3. Launching a test
- To see a list of all the questions the test asks participants, click the link under Tasks. To see what your test looks like to participants, click Preview Test to the right of the Test Plan Summary, then click Preview Test. When you finish the preview, click the Close button. ⓘ Note If you realize you need to add or remove a participant, or you find something you need to change in your inputs, do either of the following:
- Click Participants to add or remove test participants (requires additional purchases of the Participant Pack).
- Click Edit in the upper right. When you are done editing the test, click "Save and Go to Order Page."
Click Launch Test to send the test to the UserTesting Panel.
4. Reviewing Results
As participants finish, the software uploads the recorded video to our system and notifies you by email. To configure notifications, see Email notification preferences.
Open the email and click the “account dashboard” link in the order summary to go to the dashboard.
For each test, the dashboard displays the type of test—using the same icon from each Quick Answer button—the test name, the day you created it, and how many participants have and have not taken the test.
After all participants finish, a summary appears on your main dashboard.
Click a test title on the dashboard to go to the Summary tab on the Results page.
The Summary tab is a great place to start for Quick Answers. It summarizes the findings from the test: Which options do participants prefer? The answer is split into two automatically generated highlight reels (Watch key moments) to help you summarize and organize the findings based on the participants preferences.
Whether a participant prefers Option A or Option B comes from Question 8. We ask participants, “Which option do you prefer? Explain your answer.?”
The automatic highlight reels show participants reviewing the content on the page and explaining in their own words what they mean. We get this information from Question 2 below. We ask participants: “One by one, review the content and images on the page. Explain in your own words what each means. **Spend no more than 5 minutes on this.**” and we record their comments as they explain their answers.
The results table below the highlight reels provides a quick and easy way to scan through the results in greater detail. We’ve summarized the key findings in the following sub-tabs:
- Difference between options
- Why Option A Is Compelling
- Why Option B is compelling
- Reason for Preference
Most sub-tabs have brief written responses (provided by participants) to help you identify interesting responses more quickly. Once you’ve found feedback you’d like to dig into further, you can watch the video response from that participant by clicking on the written responses. This will take you to that specific moment in the video to hear what else they had to say about that particular experience. Alternatively you can watch their entire response by clicking on “Watch Session” below their screen name.
The results table on the Summary tab is where you are likely to find the most useful feedback and insights because it lets you focus on the most important questions and zero in on specific answers from individual participant’s without having to watch their complete sessions.
Click the Sessions tab to watch each participant’s complete video as they answer all the questions in order. Sessions are available as each participant finishes. There is a text field on the right for you to add a summary of the video. The Summary field is also useful for adding timecodes for moments of interest, such as a participant expressing delight with an experience, or complaining about a difficult interface.
Click the Metrics tab to watch participants’ videos broken down for every single question so you can easily skim the results. It’s essentially an expanded view of the results table on the Summary Tab.
Click the Details tab to see the demographics breakdown and an overview of the test plan’s tasks and questions.
How you get Quick Answers
After you click Launch Test, we send the test to participants in the UserTesting Panel who match the demographics you chose. These participants then answer any screener questions you added in order to qualify to take the test.
If you’d like to see what answering screener questions looks like from the participants’ view, watch this video.
Here’s what we ask participants based on the inputs you provided when you created the test:
- You'll be comparing 2 options, that are still under consideration, and asked to answer some questions and provide feedback. You won’t be able to click on anything on the page. Remember to share your thoughts out loud. Go to the next question to start.
- [Verbal] One by one, review the content and images on the page. Explain in your own words what each means. **Spend no more than 5 minutes on this.**
- [Written] In one sentence, describe the difference between the options. Describe it in your own words, not using language from the page you see.
- [Verbal] Explain what about *Option A* is compelling to you.
- [Written] Summarize your previous answer in a few words. What is **most** compelling about *Option A*?
- [Verbal] Explain what about *Option B* is compelling to you.
- [Written] Summarize your previous answer in a few words. What is **most** compelling about *Option B*?
- [Multiple Choice] Which option do you prefer? Explain your answer. [Option A, Option B]
- [Written] Summarize your previous answer in a few words. Which option did you prefer and why?
As participants finish, the software uploads the recorded video to our system and notifies you by email that the videos are ready to watch. Watch each participant’s entire video on the Session tab or wait for the fifth video session to be uploaded, at which point the highlight reels and results table are available.
Quick Answer: Discover needs and frustrations
Quick Answer: Test Messaging
Quick Answer: Test Creatives
Quick Answer: Understand behavior in mobile apps
Quick Answer: Compare two websites
Quick Answer: Understand Behavior on Websites
Marketing Insight: Live Conversation
Marketing Insight: Getting Started