How Do I Craft a Plan to Collect Insights?

Observe what users actually do

Use the method that will answer your key questions

Use more than one data source to inform your design

Every project requires feedback from the people who will be using the product or service. Those involved with the success of the project have an array of tools and tactics at their disposal.

It’s easy to become overwhelmed by the sheer number of available options. Everything from quantitative data (such as web analytics) to qualitative approaches (usability testing) can inform your strategy and design decisions. It’s difficult to know when and how to use each option, and sometimes, there is no single “right” answer.

Observe what users actually do

Some methods that focus on collecting user feedback, such as focus groups and surveys, capture self-reported data. They capture what people say they do or don't like and whether or not they consider something to be useful, engaging, or well-designed.

Unfortunately, we can’t use self-reported information as our only data source, because what people say they do is often different from what they actually do.

Also, we have to consider how users may be prompted to provide responses. For example, a required feedback form presented to users as part of a website experience likely won’t gather the most useful or valid data because the person is trying to complete their task on the site and not necessarily providing feedback.

Also, the way a question is written may bias responses and therefore taint the data.

What’s most useful is observing what users actually do. This can be done using methods that capture behavior:

• Web analytics can tell us what people do on a website.
• Usability testing can tell us why they behave that way.

These approaches typically generate more meaningful insights than you'd get from reviewing an average rating from a self-reported online survey. See this course about running a usability test.

Use the method that will answer your key questions

It’s important to keep in mind the options you have when approaching the questions that require insights from customers.

For example, a customer recently asked our Professional Services team to run a usability study to test the structure of their site. They wanted to be sure users could find what they were looking for. We suggested that they run a tree test (a method for evaluating the findability of items within a site) with a larger group of participants, along with a usability test with a smaller number of people. Using both methodologies provided more insight than a standalone usability test would.

Knowing the questions you need to answer can lead you to the best approach that provides you with the best data. See this course on defining your test goals.

Use more than one data source to inform your design

In an ideal scenario, you’ll have the option to combine more than one method or data source. For example, running a usability test to find out why a specific design performed better in an A/B test is the perfect marriage of methodologies.

There is no single best option in your toolbox of methods or data sources. Every method and data source has its own set of strengths and weaknesses. The most successful plans are those that integrate both qualitative and quantitative methods.

So, now what? Where do you go from here?:

• Start small. What project is happening now?
• What questions do you need to answer?
• How can you inform those questions with research?

Once you complete this first project, look ahead. What’s coming in the next quarter? What about next year? And how can you start planning your research approach? Good research is thoughtful and planned ahead of time. If you spend the time to consider the big questions you’re trying to answer, you’ll be well on your way to a successful research plan. See this course on test creation, which includes guides on creating a high-level roadmap for collecting insights.

Please provide any feedback you have on this article. Your feedback will be used to improve the article and should take no more than 5 minutes to complete. Article evaluations will remain completely confidential unless you request a follow-up.