A practical guide to getting fast feedback at every stage of the product development cycle with Product Insight
- Quick Answers
- Live Conversation
- Situations you may encounter
Product Insight offers two ways to collect fast feedback from customers: Quick Answers and Live Conversation.
Quick Answer tests are recorded video feedback of customers or prospects as they complete tasks and answer questions. These pre-written tasks and questions are designed to answer the most common product development problems.
- Quick Answers are quick to launch and quick to produce feedback, usually within 2 hours.
- Results include highlight reels and a results table that make it easy to share key findings with colleagues.
- Advantages include hearing open and honest thoughts from participants, since they don’t see or hear you as they respond to the questions. The presence of an interviewer often causes people to skew their feedback subconsciously.
- Quick Answers are convenient since you don’t have to conduct live interviews and can review feedback on your schedule.
Live Conversations are recorded videos from one-on-one interviews where you ask questions, listen, interact, and share screens that are at any stage of a project. Scheduling and recruitment typically takes less than a day.
- Conversations unfold naturally so you can zero in on key insights by asking follow-up questions.
- Video interviews are great for sharing complex products or prototypes—especially if you need to control where users go and help them through a prototype that’s not fully functional.
Situations you may encounter
Sometimes it’s hard to know which Quick Answer test you should use to solve a particular business problem or answer a question you have.
Here’s a list of common problems, and the approaches we recommend to solve them. We’ve mapped each situation to the phases of a typical development cycle—from discovery and design, to building and iterating.
- Discovery: Uncover needs and identify new opportunities early in the product development process. Your questions are broad and exploratory.
- Design: Create sketches and prototypes of your product after you identify customer needs.
- Build: Validate design changes and settle any disagreements using Product Insight while building the product.
- Launch: Make sure your product works for its intended purpose.
- Iterate: Measure how your product is doing—sometimes against competitors. Adjust course and make improvements based on what the trends in analytics show you.
During the discovery phase, your goal is to assess the status quo and develop a prioritized roadmap. One of the best ways to do that is to use Quick Answers and Live Conversation to build a shared understanding of your customers with your colleagues. Insights gathered at this stage help you understand your customer, identify unmet needs and opportunities, and generate competitive insights.
Find out what people do today
Find out how people currently accomplish the activities or tasks that you’re hoping to improve with your product. What do they use? How is that experience for them? You can collect this feedback in two ways:
- Live Conversation Conduct exploratory Live Conversations with customers to understand their experience completing a task or activity today. What do they use? Are there any major pain points or opportunities to improve the process? You can use our “Product Insight Live Conversation Script: Discover needs and frustrations” to help guide you through the conversation.
- Quick Answer: Discover needs and frustrations If you don’t have time to conduct a Live Conversation, and don’t need to adjust the script provided above, this Quick Answer will provide insights on how customers accomplish an activity today.
💡 Tip: Run a series of “Discover needs and frustrations” Quick Answer tests, refining the activity you have people talk about in each round. As you learn which specific step is causing the most frustration, hone in on that area.
Here’s an example of what that might look like:
1. First Round: “going for a run”
2. Second Round: “finding a route for a run”
Test participants then provide more specific insight into that part of the process.
Test your current site or app
Before you build anything new or make any changes, test your current product to find what is and isn’t working. This can identify small tweaks to make—or game-changing new features.
- Quick Answer: Test Website Usability This Quick Answer helps you learn how easy it is for a user to complete a specific task on your website and how valuable that experience is to them.
- Quick Answer: Test Mobile Usability This Quick Answer is structured the same way as the website usability test above, but allows you to test an iOS or Android mobile app. The app does not have to be released in either of the app stores—you will be able to upload the app as part of this test.
💡 Tip: To determine which task to test with Product Insight, think about the top three to five things that all users must be able to accomplish using your product. Note that this list might be different for different user groups, so narrow down the list for each group.
- Live Conversation There are several scenarios where using Live Conversation is helpful to understand current use of your product:
- If your product is a physical product or other non-digital property, then you can use Live Conversation to collect feedback about how existing customers use the product. You’ll need to use screener questions to find participants that are existing customers.
- If you want participants to attempt more than one activity in a session, this is most easily accomplished through a real-time conversation.
- If you want the flexibility to ask follow-up questions based on what participants do or say during the conversation.
💡 Tip: Here are some example scripts for getting feedback about usability:
⚠️ Important: You can test released apps only in Live Conversation.
Get the competitive intelligence you need to create differentiated experiences
Don’t reinvent the wheel. To successfully differentiate yourself and gain the advantage, find out what does and doesn’t work for your competitors, and adjust your roadmap accordingly.
- Quick Answer: Test Website Usability or Test Mobile Usability When you go through this test, simply provide the URL to your competitor’s website or app. Yes, this is allowed!
- Quick Answer: Compare Two Websites Get a side-by-side comparison of your website versus your competitor’s website to understand how the experiences compare and whether users find one more compelling than the other.
💡 Tip: You can also have customers compare two competitors; the test doesn’t have to include your own.
Prioritize features on your product roadmap
There are many inputs that help you build a prioritized roadmap, but getting real customer feedback at this stage isn’t usually at the top of the list because it can be so time consuming. This is where Product Insight really shines.
There are two Quick Answer tests we recommend using to incorporate fast customer feedback in your decision making process.
- Quick Answer: Discover Need and Frustrations Understanding customer pain points as people complete a specific task or activity can be eye-opening, and can help crystalize product priorities for a product manager tasked with balancing competing priorities.
- Quick Answer: Compare Two Websites Gathering competitive information is invaluable when building a differentiated product. Understanding how your product compares to your competitor’s highlights gaps and opportunities, and provides a significant competitive advantage.
- Live Conversation Conduct a "Feature prioritization” Live Conversation if you have a list of feature ideas that you need help prioritizing. Use this interview to guide participants through a buy-a-feature exercise to assign a value to each feature and give you a sense of what is most valuable to your target customers.
Once you have a roadmap, the next step is to start designing a solution for the features you want to implement. Your design team will likely provide you with some compelling designs to choose from. At this point, getting feedback from real customers is critical. You want to make sure you’re on the right track before engaging engineering resources. Quick Answers are a fantastic tool to build consensus around the best designs, and avoid costly design decisions.
Test ideas and concepts
Start by showing some concepts to your target customers in order to narrow down the feature set and validate the value proposition.
- Quick Answer: Validate an idea This tests help you get people's reactions to an idea and understand if they find it valuable. You can share a value proposition, landing page mock-up, marketing copy, or any idea you'd like to validate
- Live Conversation You can use our “Validate an idea interview” script if you have more time and want the ability to ask follow-up questions based on people’s first reaction.
💡 Tip: Here are some examples of ways you can present concepts to customers in a test:
⚠️ Important: Make sure URLs you include in the tests are accessible to the public.
Improve and refine product design before implementing
Once you’ve settled on a concept, you’ll still be faced with multiple designs for the same feature. Building more than one version is not realistic, so you need quick feedback to make sure you build it right the first time and avoid unwanted rework. Feedback at this stage also helps define the critical elements that your users cannot do without—also commonly known as a minimum viable product.
There are three Quick Answer tests we recommend using in this situation.
- Quick Answer: Validate a prototype Watch how people engage with your product prototype to find out what they find easy or frustrating about the experience. This provides guidance on how to refine and improve the experience before dedicating resources to bring the product to life.
- Quick Answer: Choose between two options This test makes it easy to get feedback comparing two options side by side. Get feedback on how customers perceive the differences between two UI elements, wording, or even navigation.
- Quick Answer: Compare two websites You can use this test to compare two different versions of a prototype. Let participants explain why one works better than the other.
- Live Conversation Sometimes your prototype needs some explanation to be used effectively or you want participants to attempt more than one activity in a session. For these situations, use the "Validate a prototype" or "Mobile App Usability" Live Conversation interview script. Your prototype may also be sensitive and you don’t want to share the file with participants. In this case you can share your screen during the Live Conversation and give the participant mouse control.
💡 Tip: Start with a Quick Answer test and follow it up with the Live Conversation interview to go deeper.
During the Build phase, it’s common to have to make changes to the original design based on technical limitations. Changes can majorly impact the usability of a feature, its value, or both. Customer feedback helps you avoid making tradeoffs that jeopardize the value of the product you’re releasing.
Evaluate development tradeoffs
- Quick Answer: Validate a prototype Watch how people engage with your updated product prototype to find out how easy or frustrating they find the new experience. This provides invaluable feedback on whether the tradeoffs being considered are justified.
💡 Tip: You can run the same test with a different audience to see whether the feedback is consistent across different target customers.
Before you launch, you want to make sure the final product works well. This is obviously important for all users at all times. In particular, the first-time user experience (FTUE) can make or break a product.
There are a couple of ways you can make sure the experience does what it needs to.
After a product is launched, track usage, gather additional customer feedback, and conduct follow up tests and interviews to confirm the decisions made during the design and build phases. Find out how well your product is received by your customers and whether it solves the problem you intended it to solve.
We also recommend usability testing on the live product with both people who have and haven’t had a chance to use it. All of this leads to incremental improvements and can even lead into the discovery phase of a new product.
- Quick Answer: Test Website Usability This Quick Answer helps you learn how easy it is for a user to complete a specific task on your newly launched website and how valuable that experience is to them.
- Quick Answer: Test Mobile Usability This Quick Answer is structured the same way as the website usability test above, but it allows you to test an iOS or Android mobile app. The app does not necessarily need to be released in either of the app stores—you will be able to upload the app as part of this test.
Narrow down options for A/B tests
Once you launch, it’s time to iterate. A/B testing is a powerful tool when used correctly.
More than half of A/B tests fail to produce a statistically significant winner. Improve your odds by quickly pre-testing the alternates to see how people react to them informally. This lets you identify any wording or visual problems that might derail the A/B test, and you can also pick out the alternates that seem most promising and put them through the A/B test first. Use this Quick Answer test for this situation.
- Quick Answer: Choose between two options This test provides quick feedback on your alternates and helps you determine if you’re better off creating a new alternate to A/B test. If needed, you can also set up several tests on different combos like A vs B, and C vs D, and then A vs D.
Explain changes in analytics
Analytics are great for monitoring usage and spotting problems, but it’s difficult to explain a change, and take action, unless you supplement your analytics with qualitative insight that helps you understand why people are behaving a certain way. To understand what the analytics are telling you, do one of the following:
- Quick Answer: Test Website Usability This Quick Answer helps you learn how easy it is for a user to complete a specific task on your newly launched website and how valuable that experience is to them. If you notice that people drop out of a flow at a point where you don’t expect them to, consider the activity they seem to be trying to accomplish, and use this Quick Answer test to find out what may be causing the issue.
- Quick Answer: Discover needs and frustrations If engagement is low, this Quick Answer provides insights on how customers accomplish an activity today. You should recruit both current customers of your product and new customers. This helps identify a functionality gap between your product and the needs of customers.
- Live Conversation To go deeper, conduct exploratory Live Conversations with customers to understand their experience with your new product. You can use our “Product Insight Live Conversation Script: Discover needs and frustrations” to help guide you through the conversation.