Testing Voice-Interactive Devices

Voice-Interactive Devices

Insights from Testing

Best Practices

Voice-Interactive Devices

The Internet of Things (IoT) refers to any device that is connected to the internet, including not only smartphones and computers, but an array of connected devices found in places such as: wearable tech, household items, infrastructure and development, healthcare, cars, retail, and more. Gartner projects that by 2025, the number of IoT devices will exceed 30.9 billion worldwide. If you and your organization are developing a smart device and considering the use of voice-user interface (VUI), you are in good company, as VUI becomes more and more common.

With the growing number of devices that use voice interaction however, more and more customers are also becoming frustrated with VUI when it doesn’t function properly, such as when their device misunderstands voice commands. How can you test your VUI with UserTesting to ensure it functions the way you want it to for your customers? In order to ensure your customers adopt and enjoy the VUI you've designed, your team can leverage UserTesting to validate your UX design, meet customer expectations, and optimize how your customers interact with your VUI. Additionally, with the ability to now store native video and audio files safely and securely, the level of feedback and insights are limitless. For more information on native video and audio file storage, please see "Native Video and Audio Testing."

Leveraging testing, you can discover the following insights: 

• Understand customers’ expectations for a variety of voice-interactive devices (and then focus designs to meet those expectations)
• Learn what phrases customers typically use with VUI
• Error correction and prevention around phrasing or other VUI features
• Impact of accents and other regional/geographic nuances
• Impact of visual cues within VUI and their expectations

In addition, we have created several templates in our template library that will give you a head start on creating your tests, and asking helpful questions. They will help you troubleshoot voice-interaction pain points, and identify evolving consumer needs. For example, check out our template for “Voice Interaction Evaluation,” "Connected Device In-the-Wild," "Connected Device Needs and Expectations," and "Connected Device Prototype Test."  

Template from our Template Library in the UserTesting Platform

Insights from Testing

What kinds of contributor feedback will help you gain the most helpful human insights for your VUI device? Here are some categories of test themes, followed by a couple of sample questions from that category that you can ask contributors or your team as you develop your VUI design:

• Testing for usabililty 
• Does the VUI system need to log and/or store all events?
• Is the data display working properly?
• Testing for security 
• Is the user interface password protected? How should that function?
• Is the data encrypted/protected during transfers between devices, apps, etc.?
• Testing for connectivity
• What concerns do you have in connecting your device to the internet?
• Are the directions to connect your device understandable? 
• Testing for performance
• If five or more users are using their devices simultaneously, will the system support that number without slowdown or worse?
• Is the device system scalable for your home? Office, etc.?
• Testing for compatibility
• Is your device dependent on internet speed? Could this affect the use of the device and its features, such as VUI?
• Does your device need specific, compatible products for proper connectivity?
• Testing for pilot use
• The VUI works fine in our lab setting; but does it work in a customer’s home environment? 
• At what distance from the device does it no longer detect someone’s voice?
• Testing for regulatory compliance
• Does this VUI uphold regulatory compliances for your [healthcare, finance, etc.] industry?
• Testing for upgrade considerations
• Is this new upgrade to the VUI still compatible with your customer's [hardware, software, app, OS, etc.]?

Best Practices

• Provide a maximum of two-three VUI options at a time, for customers to compare and provide feedback.
• Give customers a chance to listen to VUI options multiple times, or view the options as text.
• Context is everything. Always be sure to provide your customers with real-life scenarios.

To view more comprehensive data on VUI, please see our datasheet on end-to-end testing of voice interactions: click here.

Please provide any feedback you have on this article. Your feedback will be used to improve the article and should take no more than 5 minutes to complete. Article evaluations will remain completely confidential unless you request a follow-up.