|
Learn the answers to our most frequently asked questions around mobile testing. |
This article applies to: UserTesting
FAQs:
General questions
How do I set up a mobile test?
- You can set up either a recorded test or a Live Conversation session to be completed on a mobile device.
- When creating the audience for your test, select either Tablet or Smartphone as the device type.
- Optionally, you can select the OS in the demographics.
- Participants take your test only on a mobile device, and only if they match the demographics you have selected.
What are some things I should test on mobile devices?
- Released apps (those found in the App Store and Google Play)
- Unreleased apps
- Web-based experiences
- Mobile prototypes
- Real-world experiences (by engaging the camera on the mobile device)
- AR apps
Can I use the device's camera with the Mobile Recorder?
Yes, test participants can activate both the front-facing and back-facing cameras.
- Only one can be used at a time.
- If you expect participants to turn on their cameras, it's a best practice to write screeners that share these expectations upfront.
What is the recording limit on mobile tests?
We recommend that tests be about 15 minutes long.
- Technically, the recorders can record up to 60 minutes of video, but you'll need to screen for participants that have 2GB of free space on their device for a test that long.
- If you're interested in doing longer tests, contact your Customer Success Manager (CSM).
Can I conduct mobile tests with my own participants?
Yes. You can create your mobile test and use Invite Network to share this test with audiences outside of the UserTesting Contributor Network.
How long does it take to receive a completed mobile testing session?
Most Mobile Recorder sessions are completed within hours. Demographics and screening requirements will affect fulfillment time.
Do participants have to be sitting at their computers to complete mobile tests?
No. Mobile tests can be completed anywhere that's safe and legal to use a mobile device (for example, never while driving). Participants are not tethered to a computer, and they can freely move about and test anywhere.
Do I need to change how I structure test tasks or questions for mobile?
Yes.
-
Avoid written answers whenever possible.
-
Ask participants to speak their answers.
- Keep tasks simple.
- Avoid unnecessary numerous or complex sets of tasks that are hard to follow.
- Participants navigate between the task/question interface and the website or app, and they will not have the task text readily available at all times.
What participant demographics are available for mobile testing?
You'll have the same participant demographics options as you do with desktop studies.
Can I test other companies' apps?
Yes. You'll need their app's URL for participants to download.
What are the implications of selecting the box that will "Require participants on mobile devices to use a native browser, including Safari and Chrome" during test plan creation?
- Box checked:
- Participants go through the developer options flow.
- This gives the UserTesting app permission to record the participant's screen and enables taps on their phone.
- Box not checked: The participant's onscreen tabs will not be recorded if they leave the open UserTesting tab.
Do I see any finger input or gestures with a Mobile Recorder test?
For tests on Android devices, you'll see where the participant taps and swipes when interacting with your experience.
Android OS
- If Native Browser is selected, Android users will be prompted to turn on “Show Taps” in Developer Tools so that taps can be captured in the video.
- They are required to do this before they can proceed with the test.
iOS
- When the Native Browser recording option is selected, you won't see gestures but will see a recording of the screen without those details.
- You will only see gestures when:
- It’s an unreleased app (for app tests)
- You have integrated our SDK
OR - The participant is using an in-app browser (for website tests)
What happens when a participant runs out of space on their phone?
- The app crashes and the recording will be lost.
- Our recorder checks if users have 400MB of free space, enough for a 20-minute test.
- If you're planning a longer recording, screen for participants with enough available memory on their devices.
- If this happens to a UserTesting Contributor Network participant, we'll rerun the test for you with a new participant.
Can I specify that the participants need to be in a certain location to do a test?
Yes, but you need to keep some things in mind.
- If you ask for a general location that the participant would normally be at during the day, they will probably be willing to do that with no special incentive (For example, "Take this test outdoors in direct sunlight so we can check how visible the screen is.").
-
However, if you ask the user to make a special trip somewhere to take your test (for example, "Drive to the grocery store to take this test."), they may not accept the test without proper incentive.
- For these location-specific tests, we recommend having a UserTesting subscription with research hours.
- We'll recruit participants willing to go to a specified location and pay them extra compensation for the time involved.
Technical questions
What mobile operating systems and devices are supported?
- All devices running Android 10.0 and later.
- Most Amazon Kindle devices with microphones work (for example, Kindle Fire and Amazon Fire phones, but not epaper tablets or paper readers).
- All Apple devices running iOS 14 or later.
What are the minimum requirements for users to participate in mobile tests?
- iOS 14 (or higher) or Android operating system 10.0 (or higher)
- 400MB of free space
- At least 20% battery life
Where do I obtain the iOS SDK?
Learn about iOS SDK integration.
How is the SDK integrated into my app in the order form?
Learn how to integrate our SDK and sign your app with an enterprise certificate if you choose to upload your iOS app into your draft.
What does integrating the SDK do?
- Your build will no longer be app-store allowable.
- Think of these builds as disposable "test" builds.
- They aren't meant for production and won't see the light of day outside of a UserTesting test.
- The app can only be opened during a UserTesting test. Opening the app outside a test will show a message saying, "You must be in a test to use this app."
- You or your developers will know you've integrated the SDK correctly if you see UserTesting messaging when you run your build in the simulator.
How do you deal with landscape (horizontal) vs. portrait (vertical) mobile usage with the Mobile Recorder?
The UserTesting video player interface has a rotate function so you can easily view both landscape and portrait usage. If the participant rotates their device during a test, use a button on your Dashboard to rotate the playback.
How is the app distributed to participants for Mobile Recorder tests?
Upload your build into the order form, as you would to TestFlight or HockeyApp. Your app will be distributed through our system to participants who pick up the session.
App testing and security questions
How do I test unreleased apps?
You can test unreleased apps (i.e., apps not yet available in a public-facing app store) using the following methods:
iOS
- Upload your app (.ipa) file when creating your test. If you upload the file to us, we'll automatically install the app on the participant's device at the start of the test.
- After it has been uploaded to us and then distributed to participants, that app file can be used only within the test. (Users won't be able to open the app file outside of the UserTesting test.)
- Instead of uploading your app, you can also choose to make your app available using TestFlight and then insert your TestFlight link into your draft.
- We'll ask users to install the app at the start of the test.
- Users will still have the capability to open the unreleased app after they've completed the test (though you would be able to block future users from downloading the file).
Android
- Upload the public beta link or your .apk file during the test creation process.
- We'll automatically install the app on the participant's device at the start of the test.
- UserTesting does not use an SDK for uploaded Android apps, so participants can still open the app after the test ends.
- If you do not wish for Android users to access the app after the test, ask participants to show themselves uninstalling the app in the last task of their test.
How do I test released apps?
- You can test released apps (for example, apps that are available for download via the App Store and Google Play) by including the URL to the app when building your test.
- The test participant toggles between your app to complete tasks and the UserTesting app (where instructions and tasks are displayed).
- Everything on the screen is recorded along with the participant's voice, and the recording is sent to your Dashboard.
My team has some questions about app security with the iOS SDK. Can you tell me more?
- The UserTesting SDK for iOS locks your app to the test.
- You can't run the test version of the app outside of the UserTesting environment.
- If someone tries to run your test app outside of a test:
- An error message appears.
- Your app will not be accessible.
- Downloading the iOS app is securely completed via UserTesting.
- A user's login is linked to the app download (when qualified to take the test).
Is SDK required for Android testing?
No.
Related content
|
|
|
|
Want to learn more? Check out these Knowledge Base articles... |
Interested in growing your skills? Check out our University courses... |
|
|
|
|
Need hands-on training?
|
Can't find your answer?
|