Solving the IoT Mobile App Testing Conundrum with IoT Device Simulators

Releasing a successful IoT product depends in part on ensuring that the device itself delivers the user experience that your customers expect. But it also depends in many cases on testing the IoT mobile apps that customers use to control IoT devices. Many consumer-grade IoT products – such as smart thermostats, internet-connected doorbells, and exercise equipment – rely on mobile apps running on users' phones to provide management interfaces and deliver data to customers. If your IoT mobile app doesn't work properly, your IoT device itself won't work well for your users, either.

That's why finding ways to test the unique interactions between IoT mobile apps and the devices they interact with is an imperative component of the software testing strategy of any business that provides IoT devices. Keep reading for tips on what to consider when addressing this challenge, along with an overview of what an effective IoT mobile app testing strategy typically looks like.

"...finding ways to test the unique interactions between IoT mobile apps and the devices they interact with is an imperative component of the software testing strategy of any business..."

-Frank Moyer

Testing Requirements for IoT Mobile Apps

In certain respects, mobile apps that interface with IoT devices present unique testing challenges. To test the apps reliably, you need to simulate the real-world conditions under which the app operates. That can be tricky to do because most IoT apps rely on Bluetooth networks to connect to devices, and most test automation tools are not designed with this type of configuration in mind.

Other challenges include the fact that IoT mobile apps may use a wide variety of protocols, which translates to more variables that test engineers need to factor in when planning tests. On top of that, the way mobile apps receive data from IoT devices can vary. Some IoT devices stream data to mobile apps continuously and in real-time, for instance, while others issue periodic push updates or are connected only intermittently.

How Not to Test Mobile Apps

Given the unique challenges described above, it may be tempting to settle for an IoT mobile app testing strategy that relies on manual tests to evaluate how well apps interface with the devices they manage. You could send out your engineers with phones in hand to perform the actual activity – like pedaling an IoT bike or ringing a smart doorbell – and evaluate how the mobile app responds.

But that approach is suboptimal. Beyond the obvious fact that it takes a lot of time to test devices manually (and that you'd have to repeat the laborious manual testing process every time you update the app), a manual approach to IoT mobile app testing makes it hard to test across a wide range of devices because you can't take advantage of device farms. Instead, you're limited to whichever handful of devices your engineers have at their disposal, which likely represent only a small subset of the devices that your customers may use. That leads to incomplete test coverage and increases the risk of app performance or reliability problems going undetected until they affect end users.

A Better Approach

Fortunately, there is a better way. You can connect your mobile device farm to physical IoT device simulators, then run automated tests that cover a wide range of scenarios and that reliably emulate real-world conditions.

An IoT device simulator is a special device that connects to mobile apps and sends them the same data that a real IoT device would send – except the simulator gives engineers control over exactly which data is sent. That means testing teams can configure simulators to represent the full range of user actions they want to test for, then push that data to IoT mobile apps using the same protocols and connection methods as customers would use.

In other words, if you want to test how a mobile app responds when a user pedals a smart bike uphill or brakes suddenly, you can configure your IoT simulator to send data to the device that simulates those actions. Or, you could test how an app that controls a smart thermostat responds to a sudden drop in ambient temperature, or how a mobile app handles an IoT doorbell that a user presses repeatedly for a prolonged period. In addition to making it possible to perform these tests automatically across a wide range of devices, this approach allows engineers to take advantage of cloud-based testing software. As long as your mobile devices integrate with your testing cloud, you can test interactions between mobile apps and IoT devices over local connections even if your testing software is not hosted locally.

The Future of IoT Mobile App Testing

The type of IoT mobile app testing strategy I've described here is critical to ensuring that IoT devices and the apps they depend on meet user expectations, even as IoT devices continue to grow in scale and complexity. Manual IoT testing may have worked when consumer IoT devices were newer and customers expected less, but to test effectively in today's high-stakes IoT world, you need the type of automation, broad coverage, and cloud-friendly testing that only IoT device simulators can provide.