Mobile Testing FAQ

You can get feedback on a range of mobile experiences, including web, app and real world, by having participants complete tests on smartphones or tablets.


General Questions
Technical Questions
App Testing and Security Questions
Additional mobile testing resources

 

General Questions


1. How do I set up a mobile test?

You can set up either a recorded test or a Live Conversation session to be completed on a mobile device. When creating the audience for your test, select either Tablet or Smartphone as the device type. You can go further and select the OS in the demographics if you want. Test participants will only be able to complete your test on a mobile device, and only if they match the demographics you have selected.  

2. What are some things I should test on mobile devices?

  • Released apps (those found in the App Store and Google Play) 
  • Unreleased apps
  • Web-based experiences
  • Mobile prototypes
  • Real world experiences (by engaging the camera on the mobile device)
  • AR apps

3. Can I use the device's camera with the Mobile Recorder?

Yes, test participants can activate both the front-facing and back-facing cameras. If you expect participants to turn on their camera, it's a best practice to write screeners that share these expectations up front.

4. What is the recording limit on mobile tests?

We recommend that tests be about 15-minutes long. Technically, the recorders can record up to 60-minutes of video, but you’ll need to screen for participants that have 2GB of free space on their device for a test that long. If you're interested in doing longer tests, please contact your Customer Success Manager.

5. Can I conduct mobile tests with my own participants?

Yes, you’re able to create your mobile test and then use My Recruit to share this test with audiences outside of the UserTesting panel.

6. How long does it take to receive completed mobile testing session?

Most Mobile Recorder sessions are completed within hours. Demographics and screening requirements will affect fulfillment time.

7. Do participants have to be sitting at their computer to complete mobile tests?

No. Mobile tests may be completed anywhere that’s safe and legal to use a mobile device (i.e. never while driving). Participants are not "tethered" to a computer. This means a participant can freely move about and test anywhere from the park to the couch at home.

8. Do I need to change how I structure test tasks or questions for mobile?

Yes. Avoid written answers when possible, rather, have people speak them aloud instead.

Additionally, keep tasks simple, and avoid unnecessarily numerous or complex sets of tasks, as participants may have trouble following them. Participants will navigate back and forth from the task/question interface and the website or app. As a result, they will not have the task text readily available at all times.

9. What participant demographics are available for mobile testing?

You'll have the same participant demographic options as you do with desktop studies.

10. Can I test other companies' apps?

Yes, you can conduct competitor app studies. You'll just need to provide the app's URL for test participants

to download.  

11. Do I see any finger input or gestures with a Mobile Recorder test?

On Android tests, you will see where the participant taps and swipes when interacting with your experience. With iOS tests, you will see these gestures unless the "Native Browser" recording option is selected. In that case, you will see the recording of the screen without these details.

12. What happens when a participant runs out of space on their phone?

The app will crash and the recording will be lost. Our recorder checks to make sure the user has 400MB free, enough for a 20-minute test. If you're planning a longer recording, screen for participants who have enough free memory on their device. If this happens to a UserTesting panel participant, we'll re-run the test for you with a new participant.

13. Can I specify that the participants need to be a certain location to do a test?

Yes, but you need to keep some things in mind. If you ask for a general location that the participant would normally be in during the day, they will probably be willing to do that with no special incentive (for example, "take this test outdoors in direct sunlight so we can check how visible the screen is").

However, if you ask the user to make a special trip somewhere in order to take your test (for example, "drive to the grocery store to take this test"), they may not accept the test without proper incentive. For these location-specific tests, you should have a UserTesting subscription with research hours. We'll recruit test participants willing to go to a specified location and will pay them extra compensation for the time involved.

 

Technical Questions 

1. What mobile operating systems and devices are supported?

  • All devices running Android 5.0 and later.
    • Most Kindles with microphones work (ex: Kindle Fire and Amazon Fire phones, but not e-paper tablets or paper readers).
  • All Apple devices running iOS 12 or later.

2. What are the minimum requirements for users to participate in mobile tests?

Test participants must have iOS 12 (or higher), Android operating system 5.0 (or higher), 400MB... 400MB of free space when they try to take a test, and at least 20% battery life

3. Where do I obtain the iOS SDK?

Click here for more information.

4. How is the SDK integrated into my app in the order form?

Upload your iOS app build into the order form, and our order form will take a couple of minutes to attempt to integrate the SDK into your app. You’ll know it’s successful if you are presented with the option to use the Mobile Recorder. If you do not see that option, you will need to ask your developers to manually integrate the SDK.

5. What does integrating the SDK do?

  • Your build will no longer be app store allowable. You should think of these builds as disposable "test" builds. They aren't meant for production, and won't see the light of day outside of a UserTesting test.
  • The app can only be opened during a UserTesting test. Opening the app outside a test will show a message saying, "You must be in a test to use this app".
  • You or your developers will know you've integrated correctly if you see UserTesting messaging when you run your build in the simulator.

6. How do you deal with landscape (horizontal) vs. portrait (vertical) mobile usage with the Mobile Recorder?

The UserTesting video player interface has a rotate function so you can view both landscape and portrait usage easily.

If the participant rotates their device during a test, you can use a button on your dashboard to rotate the playback.

 

7. How is the app distributed to participants for Mobile Recorder tests?

Upload your build into the order form, like you would to TestFlight or HockeyApp. Your app will be distributed through our system to participants who pick up the session.

App Testing and Security Questions

1. How do I test unreleased apps?

You can test unreleased apps (i.e. apps not yet available in a public facing app store) using the following methods:

  • iOS: We need to install an SDK (software development kit) to test an unreleased iOS app. This modified version of the app is what is used during the test.
  • Android: Upload the .apk file during the test creation process. We'll automatically install the app on the participant's device at the start of the test.

2. How do I test released apps?

You can test released apps (i.e. apps that are available for download via the App Store and Google Play) by directing test participants to the respective app store and instructing them to download the app. The test participant would then toggle between the your app to complete tasks and the UserTesting app (where instructions and tasks are displayed), and everything that is displayed on the screen will be recorded along with the participant’s voice in the recording sent to your dashboard. 

3. My team has some questions about app security with the iOS SDK. Can you tell me more?

The UserTesting SDK for iOS locks your app to the test. You can't run the test version of the app outside of the UserTesting environment. If someone tries to run your test app outside of a test, an error message will appear and your app will not be accessible. The downloading of the iOS app to the test participant is also securely facilitated through UserTesting. A user's login is linked to the app download (once they qualify for the requirements of the test).

 

4. Do I need an SDK for Android testing?

No.

Additional mobile testing resources

 

 

Was this article helpful?
8 out of 9 found this helpful

Comments

0 comments

Article is closed for comments.