Quick Answer: Understand behavior in mobile apps

Overview

Quick Answers are recorded video feedback of customers or prospects as they complete tasks and answer questions. The tasks and questions are written for you by UserTesting, and designed to answer the most common marketing challenges. You can read more about Quick Answers in  Marketing Insight: Getting Started.

“Understand behavior in mobile apps” is one of seven types of Quick Answers in Marketing Insight. With it, you can discover whether people can easily complete a task in a mobile app as well as understanding the behavior behind your metrics and analytics.

xNxGuA-TUiMe4SeYGyuoBxYD1SL2yJMQ1tlUqdL-DRzYBJwHZfeBZB-6r1oenOAooY756u_6Y5kUJrS4LvoT9KC_7gqv3cD15xIm0Ns81tIGJZ0Oma9fMkYHBtKAEsiSYDKQfk5Y.png

After running this Quick Answer test, the Results tab shows how many test participants found the task difficult or easy, and provides two automatically generated highlight reels - split based on their responses - of them discussing their answers and explaining why they answered the way they did.

If you want to dig into the findings, there are several options. The results table below the highlight reels provides additional feedback from participants about how they feel about that particular experience or activity. The Metrics and Session tabs provide the full recordings broken out by question or participant respectively.

 

When to use "Understand behavior in mobile apps"

Use “Understand behavior in mobile apps" to:

  • Explore customer behavior in your mobile app and ensure customers are finding it easy to use
  • Test your app to make your customers understand how to use it and the information you’re trying to convey
  • Test navigation elements to make sure your customers can find what they’re looking for
  • Observing how easy or hard people find completing a specific task in your app and to understand what is driving the behavior reflected in your data and analytics
  • Identifying opportunities to improve conversion rates

How to  Set  up  "Understand behavior  in mobile  apps"

1. Quick Answer Inputs

From the dashboard click Get Quick Answers, then click “Understand behavior in mobile apps”.

 

BehaviorMobile.png

Start by describing the activity you want participants to try on your app. Start your description with a verb, but don’t include exact steps, such as “Click on this button. Then this link. Then go….”

The description is added to a pre-written sentence, so it’s a good practice to use a lowercase letter for the first word and not end with a period or other punctuation. Quick Answers converts your description into the test questions, so the wording you enter is what people will see.

Some examples include:

  • sign up for the newsletter becomes 
    • Take as much time as you need to sign up for the newsletter
  • find out when to report for jury duty becomes 
    • Take as much time as you need to find out when to report for jury duty
  • change your profile picture becomes 
    • Take as much time as you need to change your profile picture.
      MobileBehaviorUpdate.png
      Then name your test so it’s easy to recognize from other tests on your dashboard. Marketing Insight displays all of your account’s tests on your dashboard. This makes it easy to share tests with other stakeholders. At this point, you'll need to determine whether you'd like to test with participants from the UserTesting Panel, or your own customers, via My Recruit. When you’re done, click Next.

2. Select Demographics

UserTesting Panel

The UserTesting panel is a diverse and global on-demand group of people who take tests. The panel is made up of real people, carefully selected after a thorough application process, who are continually rated by our customers, assuring the highest quality results. To find participants you’ll be able to select device type, set demographic filters, and create custom screener questions. These are detailed in Marketing Insight: Find the Right Participants.
Demographics.png

My Recruit

My Recruit gives UserTesting customers the ability to create and launch a study with anyone, anytime—on demand. This feature is completely self-service: you can recruit, manage, and determine appropriate compensation for your customers. You also have the option to add your own branding to the test. After you build your test plan, we'll provide a URL for you to distribute to your participants. Learn more about how My Recruit works.
MyRecruitMI.png

3. Link or upload your app

You have two options to make your app available to the participants. You can either upload an IPA or APK file directly into the platform or simply provide a link to your app in the Google or Apple app stores. If you choose to upload your app directly, we'll securely distribute it to your participants devices (no provisioning required). This is helpful if your testing a mobile app that hasn’t been released yet.

SecureApp.png

If you chose to provide a link to the Google or Apple app stores, and aren’t sure how to locate the URL, please read Finding the app URL.

 

4. Launching a Test
MobileSummary.png

  • To see a list of all the questions the test asks participants, click the link under Tasks. To see what your test looks like to participants, click Preview Test to the right of the Test Plan Summary, then click Preview Test. When you finish the preview, click the Close button. ⓘ Note If you realize you need to add or remove a participant, or you find something you need to change in your inputs, do either of the following:
    • Click Participants to add or remove test participants (requires additional purchases of the Participant Pack).
    • Click Edit in the upper right. When you are done editing the test, click "Save and Go to Order Page."

Click Launch Test to send the test to the UserTesting Panel.

5. Reviewing Results

As participants finish, the software uploads the recorded video to our system and notifies you by email. To configure notifications, see Email notification preferences.

Open the email and click the “account dashboard” link in the order summary to go to the dashboard.

For each test, the dashboard displays the type of test—using the same icon from each Quick Answer button—the test name, the day you created it, and how many participants have and have not taken the test.

After all participants finish, a summary appears on your main dashboard:

Screen_Shot_2020-03-17_at_1.55.06_PM.png

Click a test title on the dashboard to go to the Results tab on the Results page.

Results Tab

The Results tab is a great place to start for Quick Answers. It summarizes the findings from the test: Do more participants find the activity easy or difficult on the app you tested? The answer is split into two automatically generated highlight reels (Watch key moments) to help you summarize and organize the findings based on the participants preferences.

xNxGuA-TUiMe4SeYGyuoBxYD1SL2yJMQ1tlUqdL-DRzYBJwHZfeBZB-6r1oenOAooY756u_6Y5kUJrS4LvoT9KC_7gqv3cD15xIm0Ns81tIGJZ0Oma9fMkYHBtKAEsiSYDKQfk5Y.png

 

Whether a participant finds the experience easy or difficult comes from Question 3. We ask participants, “How difficult or easy was the activity overall? Explain your answer.?” 1=Very difficult…5=Very easy. A score of 1, 2, or 3 means they found the experience difficult. A score of 4 or 5 means they found the experience easy.

The automatic highlight reels show participants doing this activity. We get this information from Question 5 below. We ask participants: “Take as much time as you need to [activity]". and we record their comments as they do this activity.

The results table below the highlight reels provides a quick and easy way to scan through the results in greater detail. We’ve summarized the key findings in the following sub-tabs:

  • Familiarity
  • First Impressions
  • Expected Effort
  • What made it easy
  • What made it hard
  • Perceived value
  • Areas to improve

Most sub-tabs have brief written responses (provided by participants) to help you identify interesting responses more quickly. Once you’ve found feedback you’d like to dig into further, you can watch the video response from that participant by clicking on the written responses. This will take you to that specific moment in the video to hear what else they had to say about that particular experience. Alternatively you can watch their entire response by clicking on “Watch Session” below their username.


jOiiChQ3vwDaTNgHmNzwqBL_KKx864fr4wOSAv3KTmM7OTIhbHYoknkkPNIqhc2LMXA3wMoAZvVl9DY8CPz3vEefnAjjjmxhNJSQVK1X26qbzp_bAwklgkllZwbJ-opJQ_X07DSD.png

The results table on the Results tab is where you are likely to find the most useful feedback and insights because it lets you focus on the most important questions and zero in on specific answers from individual participant’s without having to watch their complete sessions.

Sessions Tab

Click the Sessions tab to watch each participant’s complete video as they answer all the questions in order. Sessions are available as each participant finishes. There is a text field on the right for you to add a summary of the video. The  Results field is also useful for adding timecodes for moments of interest, such as a participant expressing delight with an experience, or complaining about a difficult interface.

SessionsTab.png

 

Metrics Tab

Click the Metrics tab to watch participants’ videos broken down for every single question so you can easily skim the results. It’s essentially an expanded view of the results table on the Results Tab.
MobileMetrics.png

Details Tab

Click the Details tab to see the demographics breakdown and an overview of the test plan’s tasks and questions.

 vHGuagNsBImjOOXyqtu7S8AEcsoxte_b5JFeO33l0yu3GMesFQfGCxMpyZIIk78OoNznlXolR4uQ6A2tvDDjV1FQD3FjcmJWfvxZqhSlMjGMGIdOBXo-5J4r6MFDwlMEZrbvdoj-.png

How you get Quick Answers

After you click Launch Test, we send the test to participants in the UserTesting Panel who match the demographics you chose. These participants then answer any screener questions you added in order to qualify to take the test.

If you’d like to see what answering screener questions looks like from the participants’ view, watch this video.

Once they qualify, we ask them a series of questions about the task or activity you provided. We use the exact phrasing you provide for the task or activity when we ask the participants questions. This is why it’s important to start the sentence with a verb - for example “buy a car” or “find a TV.”

Here’s what we ask participants based on the questions you answered when you created the test:

  1. If you are asked to log in or create an account please do so now. If not, move on to the next step. Don’t use the app yet. The video of this step will be blurred so we won’t capture your personal information.
  2. How familiar are you with this app? [Multiple Choice: Never heard of it, Heard of it, but never used it, Used it once, Used it a few times, Use it regularly , None of the above]
  3. [Written] Take a look at the home screen, but **don’t scroll or tap anywhere yet**. What are your first impressions of the app? Explain your answer. 
  4. Take a look at the home screen, but **don’t scroll or tap anywhere yet**. How difficult or easy do you **expect** it to be to  [activity]? Explain your answer. [5-point Rating Scale: Very difficult to Very easy]
  5. Take as much time as you need to  [activity].
  6. How difficult or easy was the activity overall? Explain your answer. [5-point Rating Scale: Very difficult to Very easy]
  7. [Verbal] What part of the app made it **easy** to accomplish that activity? Write 1-3 words that summarize your previous answer. What part of the app made the activity **easy**? [Written Response]
  8. [Verbal] What part of the app made it **frustrating or hard** to accomplish that activity?
  9. [Written] Write 1-3 words that summarize your previous answer. What part of the app made the activity **frustrating or hard**?
  10. Overall, did the your experience using the app exceed, meet, or fall short of your first **impressions**? Explain your answer. 
    • It exceeded my first impressions.
    • It met my first impressions.
    • It fell short of my first impressions.  
    • Other
  11. How valuable is this app to you? Explain your answer. [5-point Rating Scale: Not at all valuable to Very valuable]
  12. From this list select 1 word that describes the **biggest improvement** you would make to this app: [Multiple Choice: Appearance, Navigation, Trustworthiness, Understandability, Usefulness, None of the Above]

As participants finish, the software uploads the recorded video to our system and notifies you by email that the videos are ready to watch. Watch each participant’s entire video on the Session tab or wait for the fifth video session to be uploaded, at which point the highlight reels and results table are available.

See Also

Quick Answer: Discover Needs and Frustrations
Quick Answer: Test messaging
Quick Answer: Test creatives
Quick Answer: Choose between two options
Quick Answer: Compare two websites
Quick Answer: Understand behavior on websites
Marketing Insight: Live Conversation
Marketing Insight: Getting Started

 

 

Was this article helpful?
0 out of 0 found this helpful