What are Quick Answers?
Quick Answers are recorded video feedback of customers or prospects as they complete tasks and answer questions. The tasks and questions are written for you by UserTesting, and designed to answer the most common product development problems. You can read more about Quick Answers in Product Insight: Getting Started.
"Validate an idea" is one of seven types of Quick Answers. Each Quick Answer provides a unique insight or answers a question you have.
For "Validate an idea,” the focus is on learning how customers react to an idea for a product or service. Think of this as a concept test, something you do before creating a full prototype. We ask customers a series of questions about your concept, and they also say if they think it is valuable.
After running this Quick Answer test, Summary tab provides a graph of how many people found a concept valuable, and provides a video of them discussing their answers and explaining why they answered the way they did. Here’s an example of five people describing why they did or did not find an online running club valuable.
If you want to dig into the findings, there are several options. The results table below the highlight reels provides additional feedback from participants about how they feel about that particular concept. The Metrics and Session tabs provide the full recordings broken out by question or participant respectively.
When to use "Validate an idea"
Use "Validate an idea" in the Discover phase of the development process to:
- Get reactions to a product or feature idea
- Identify potential questions from customers
How to Set up "Validate an idea"
1. Quick Answer Inputs
From the dashboard click Get Quick Answers, then click "Validate an idea."
Provide a link to an online asset illustrating your idea. The link needs to point to a single document, hosted online, and accessible and readable by a typical browser. The link can be to a Microsoft or Google doc, a Slides or PowerPoint presentation, a PDF, image, text, or anything similar.
Then name your test so it’s easy to recognize from other tests on the dashboard.
At this point, you'll need to determine whether you'd like to test with participants from the UserTesting Panel, or your own customers, via My Recruit. When you’re done, click Next.
For this example, we'll be using the UserTesting panel.
2. Select Participants
- Accept defaults: The default is to use any member of the UserTesting Panel. For many tests, you may find that works well enough. If that’s the case, click Done at the bottom of the page.
- Number of participants: This is set to five unless you’ve purchased the Participant Pack, in which case you can choose between 5 and 15 participants for each Quick Answer. If you’ve purchased the Participant Pack, we also recommend the number of participants for each Quick Answer. For “Validate an idea” we recommend five participants as the default.
- Device: Select the type of device you want participants to use.
Important: Leave this set to Computer unless there is a particular reason why you need someone to answer using a tablet or smartphone. Even when asking them to respond to an idea about a smartphone product, it’s usually best to have people answer using a computer because the screen is larger and participants have an easier time reading your concept document.
For more information on selecting a device and best practices, see:
Mobile Testing Capabilities
Mobile Recorder Best Practices
- Demographic Filters: If you want participants with a particular background, select and configure filters from the options on the right. If you decide not to use a filter after selecting it, click the Remove link in the upper-right corner or deselect it from the list on the right.
For more help with demographic filters, see:
Choose your target audience from the UserTesting panel.
- Screener Questions: Screener questions are multiple choice questions participants must answer correctly before they're able to accept a test. This is an optional feature you can select if the demographic filters above do not cover your requirements. To refine participant selection with screener questions, do any of the following:
- To add a screener question you saved from a previous test, click Saved Screener Questions on the right. If you haven’t saved any screener questions yet, this is empty.
- To create a screener question, click the Screener Question button on the right.
- To add more screener questions, click the Screener Question button or the “Add another screener question” link below the last screener question.
Note: After you create a screener question, you can click the “Add to saved screener,” “Duplicate,” or “Delete” links. To reorder screener questions, drag the question to where you want it to be using the Move icon.
- To preview any screener questions, click Preview Screener.
You can remove any screener questions you add by clicking Delete at the top right of each question’s dialog.
For more help creating screener questions, see:
Using Screener Questions to find your ideal participants
Screener question best practices
3. Launching a Test
- To see a list of all the questions the test asks participants, click the link under Tasks and Questions. To see what your test looks like to participants, click Preview Test to the right of the Test Plan Summary, then click Preview Test. When you finish the preview, click the Close button.
ⓘ Note If you realize you need to add or remove a participant, or you find something you need to change in your test plan—for example, you used the wrong URL or you find a typo, do either of the following:
- Click Participants to add or remove test participants.
Click Edit Details in the upper right. When you are done editing the test, click "Save and Go to Order Page."
Click Launch Test to send the test to the UserTesting Panel.
4. Reviewing Results
As participants finish, the software uploads the recorded video to our system and notifies you by email. To configure notifications, see Email notification preferences.
Open the email and click the “account dashboard” link in the order summary to go to the dashboard.
For each test, the dashboard displays the type of test—using the same icon from each Quick Answer button—the test name, the day you created it, and how many participants have and have not taken the test.
After all participants finish, a summary appears on your main dashboard.
Click a test title on the dashboard to go to the Summary tab on the Results page.
The Summary tab is a great place to start for Quick Answers. It summarizes the findings from the test: Do more people find this idea valuable or not valuable? The answer is split into two automatically generated highlight reels (Watch key moments) to help you summarize and organize the findings based on the participants preferences.
Review the graph
The graph displays the answer, either:
- More people found the concept valuable.
- More people did not find the concept valuable.
We get this information from Question 2 below. We ask participants, “How valuable is this idea to you?” 1=Not at all valuable…5 = Very valuable.
Then we count how many of the five participants did or did not find the idea valuable.
- A score of 1, 2, or 3 = Not valuable
- A score of 4 or 5 = Valuable
Watch key moments
The summary video shows all five participants discussing whether or not they find your idea valuable.
We get this information from Question 2 below. We ask participants: “How valuable is this idea to you?” and we record their comments as they explain their answers.
Note: Until all participants finish, the graph shows a running total as participants complete the test and displays the number of participants yet to finish, but there is no key moments video until all participants finish.
Some participants finished:
All participants finished:
Click the Sessions tab to watch each participant’s complete video as they answer all the questions in order. Sessions are available as each participant finishes. There is a text field on the right for you to add a summary of the video. The Summary field is also useful for adding timecodes for moments of interest, such as a participant expressing delight with a task, or complaining about a difficult interface.
Click the Metrics tab to watch participants’ videos broken down by question so you can easily skim the results. Questions have either written responses or video response.
Metrics is a powerful way to understand the details of your participants’ experiences. After the Summary tab, you are likely to find this tab contains the most useful information because it lets you focus on each test question and each participant’s answer.
Click the Details tab to see the demographics breakdown and an overview of the test plan’s tasks and questions.
How you get Quick Answers
After you click Launch Test, we send the test to people in our UserTesting Panel who match the demographics you chose. These people then answer any screener questions you added in order to qualify to take the test.
If you’d like to see what answering screener questions looks like from the participants’ view, watch this video.
Here’s what we ask participants based on the questions you answered when you created the test:
You'll see a page you can't click on. You'll be asked some questions about it. Remember to think out loud. Don’t pretend to be someone else; we care about your needs and thoughts.
- [Written] In one sentence describe the purpose of this idea. Describe it in your own words, not using language from the page you see.
- [Rating] How valuable is this idea to you? Explain your answer. 1=Not at all valuable…5 = Very valuable.
The answer to this question is where we get the data for the graph and the highlights for the video on the Summary tab.
- [Verbal] Describe how you might benefit from using this. What will it help you do?
- [Written] Write 1-3 words that summarize your previous answer in the box below. How might you benefit?
- [Verbal] Which message on this page is the **most** compelling to you?
- [Written] Write 1-3 words that summarize your previous answer in the box below. How which message is the **most** compelling?
- [Verbal] Which message on this page is the **least** compelling to you?
- [Written] Write 1-3 words that summarize your previous answer in the box below. How which message is the **least** compelling?
As participants finish, the software uploads the recorded video to our system and notifies you by email that the videos are ready to watch. Watch each participant’s entire video on the Session tab or wait for the fifth video session to be uploaded, at which point the highlight reel is available.
New to Product Insight?
Product Insight overview
Quick Answer: Discover needs and frustrations
Quick Answer: Validate a prototype
Quick Answer: Test website usability
Quick Answer: Choose between two options
Quick Answer: Compare two websites
Product Insight: Live Conversation