At a Glance
Not sure how to create a plan to collect insights? This article explains why it's important to have a plan and methods for collecting feedback.
Every project requires insights from the people who will be using the product or service that the project is creating. Those involved with the success of the project have an array of tools and tactics at their disposal.
It’s easy to drown in a sea of options. Everything from quantitative data (such as web analytics) to qualitative approaches (such as usability testing) can inform your strategy and design decisions. It’s difficult to know when and how to use each option, and sometimes, there is no single “right” answer.
Observing what users actually do is more insightful than self-reported data
Some methods that focus on collecting user feedback, such as focus groups and surveys, capture self-reported data. They capture what people say they like (or don’t) and whether they consider something to be useful, engaging, or well-designed (or not).
Unfortunately, we can’t use self-reported information as our only data source, because what people say they do is often different from what they actually do.
Also, we have to consider how users may be prompted to provide responses. For example, a required feedback form presented to users as part of a website experience likely won’t gather the most useful or valid data, because the person is trying to complete their task on the site, not necessarily provide feedback. Also, the way a question is written may bias responses and taint the data.
What’s most useful is observing what users actually do. This can be done using methods that capture behavior, such as web analytics or usability studies. Web analytics can tell us what people do on a website. Usability testing can tell us why they behave that way. This is much more insightful than reviewing a self-reported average rating from an online survey. See this course about use cases in the UserTesting platform and this course about running a usability test.
Sometimes it’s easier to fall back on what we are most comfortable with
The team working on a project often has a few research tools or methods they are most comfortable with. Researchers love usability testing and other methodologies that involve observing users doing activities. Marketers and optimization experts focus on web analytics, because conversion rates and other measures often speak for themselves. It’s very easy to go with what we know and love, even if there are better alternatives. See this course about how quantitative and qualitative data fit together.
Use the method that will answer your key questions
It’s important to keep in mind the options you have when approaching the questions that require insights from customers.
For example, a customer recently asked our Professional Services team to run a usability study to test the structure of their site. They wanted to be sure users could find what they were looking for. We suggested that they run a tree test (a method for evaluating the findability of items within a site) with a larger group of participants, along with a usability test with a smaller number of people. Using both methodologies provided more insight than a standalone usability test would.
Knowing the questions you need to answer can lead you to the best approach that provides you with the best data. See this course on defining your test goals.
Use more than one data source to inform your design
In an ideal scenario, you’ll have the option to combine more than one method or data source. For example, running a usability test to find out why a specific design performed better in an A/B test is the perfect marriage of methodologies.
There is no single best option in your toolbox of methods or data sources
Every method and data source has its own set of strengths and weaknesses. The most successful plans are those that integrate both qualitative and quantitative methods.
So, now what? Where do you go from here?
Try not to get overwhelmed. Start small. What project is happening now? And what questions do you need to answer? How can you inform those questions with research?
Once you complete this first project, look ahead. What’s coming in the next quarter? What about next year? And how can you start planning your research approach? Good research is thoughtful and planned ahead of time. If you spend the time to consider the big questions you’re trying to answer, you’ll be well on your way to a successful research plan. See this course on test creation, which includes guides on creating a high-level roadmap for collecting insights.
Need more information? Read these related resources.
Want to learn more about this topic? Check out our University courses.