Mobile Tests: Best Practices

At a Glance

Just as mobile apps are different from desktop apps, the process of running a usability study on a mobile device is different from recording one on a desktop computer. Here are some guidelines from our team.

 

General

Here are some recommendations to make your usability study on a mobile device proceed as smoothly as possible:

  • Specify the links for each site in the test plan when you want contributors to see the links when taking a comparison test. (Our iOS browser doesn’t yet support tabbed browsing.) For example, if you’d like contributors to look at Site A and Site B in quick succession, you should write out both URLs in the task so that contributors can access them correctly.
  • Incorporate an SDK into a testing version of your app when testing apps on iOS devices. This does not apply to app tests taken on Android devices.
  • Use the Native Browser option when you require advanced browser functionality (e.g., multiple tabs), or you've noticed issues with your prototype loading on our browser. Contributors will use Safari or Chrome, depending on their operating system. Note that selecting this option will change the recording method in such a way that finger-taps will no longer be captured.

(Check the Native Browser option box when building your test plan in the UserTesting Platform, as seen here, which requires contributors to use the native browser when testing on their mobile devices):

Native_Browser.png

NOTE: Keep in mind that if you're using the in-app browser and something doesn't quite look right in the videos you're receiving, we ask that you recreate the test using a native browser.

Test Length and Tasks

  • With iOS tests, reiterate the site/app you are testing in the first task. Doing so will help contributors understand what they are testing, and what they should be looking at before proceeding through tasks.
  • Android tests automatically display the start URL as the first task, so repeating it is not necessary.
  • Keep tests as close to 15 minutes as possible. Contributors expect tests to take no longer than that. Test sessions longer than 15 minutes may cause contributor fatigue, which often diminishes the quality of the feedback. Keep in mind that test sessions on mobile devices tend to be shorter than those on desktop.
  • Include time frames for some tasks. For example: “Spend up to two minutes reviewing the site. What do you like or dislike about it?”
  • Keep task descriptions short. Because there's a limited amount of space to display the task description on screen, write your tasks as concisely as possible. Also, push the most important details toward the front of the task.
  • Provide the URL to a specific page if you're asking contributors for feedback on that page. Providing the URL will help contributors quickly access that page, and allow them to return to it should they navigate elsewhere.
  • Use shortened URLs. To conserve space, use condensed URLs (created through a service such as bit.ly) when including links in task descriptions. Long links can crowd up the screen!
  • Five-second test: Due to inconsistent loading speeds for mobile connections, avoid using the five-second test when using the Mobile Recorder.

Questions and Ratings

  • Be careful not to use too many written-response questions. Typing on a mobile device can be cumbersome, and can lead to contributor fatigue during the study. Use written-response questions sparingly, opting for verbal responses instead.
  • Direct contributors to write their answers to written-response questions in the “box below.” Some contributors begin answering the written-response questions verbally, so the entire response may not be captured in writing. Reminding them to type their answers in the provided box will help capture the full written response.
  • Multiple choice and rating scales are great for mobile tests. Try to incorporate these types of questions (if available) in your study, because they don’t require lots of typing.

Learn More

Need more information? Read these related articles.

Want to learn more about this topic? Check out our University courses.

Please provide any feedback you have on this article. Your feedback will be used to improve the article, and when you submit your survey you'll be entered into a drawing for a $50 Amazon gift card. This survey should take 5 minutes to complete. Article evaluations will remain completely confidential.

Was this article helpful?
7 out of 16 found this helpful