How to Avoid 3 Pitfalls of IoT Testing

The future for the Internet of Things (IoT) looks great overall. By most estimates, the IoT market is set to grow dramatically over the course of the next few years. In fact, Bain forecasts that the IoT market will double by 2021 reaching $520 billion.

Fueling this growth are the benefits IoT products offer not only to consumers but also to businesses across a number of industries like healthcare, retail, manufacturing, and transportation.

Nonetheless, there's a multi-billion dollar problem facing the IoT industry. That problem is the quality of its products.

When we take into account the other industries and the similar problems they face compared to IIoT, it's clear to see this is a major challenge for the entire IoT industry.

However, what exactly is contributing to this challenge?

A Lack of Standards

One of the root causes of the problems facing IoT is the lack of industry-wide standards to shaping the development of new products.

Without a set of standards, companies in every industry producing IoT products are free to design and build their products as they see fit. While this may initially seem like a positive, it's having negative ramifications leading to a large number of diverse configurations and specifications that IoT products must contend with in order to connect to one another.  

In short, the lack of standards limits interoperability by making it difficult for IoT products to connect to one another, limiting the technology’s potential, especially when it comes to building an integrated smart home, smart city and/or smart building system.

Fragmentation

Besides the lack of standards, the fragmentation of the smartphone market poses an additional problem. Connecting to and interacting with an IoT product via a smartphone or tablet can be particularly difficult given the unique configurations and constraints placed on smartphone hardware and software.

When combined with the various operating system updates that individual users may or may not install on their devices, the market has become saturated with millions of unique brands, models and smartphone OSs.

Each tile represents a unique smartphone model and Android OS combination on the market, and that's way back in 2015! The size of the tile represents its share of the market. With more brands and models on the market in 2019, fragmentation has continued to increase.

Possible Solutions

While many people in the IoT industry believe a set of standards needs to be adopted, there is a void between the rhetoric and practices that dominate IoT.  As a result, the role of quality assurance (QA) testing is becoming more important.

To contend with development challenges, IoT products require thorough testing to uncover any software anomalies or hardware defects that can taint their performance, reliability, stability, connectivity and overall commercial potential.

Nonetheless, IoT comes with its own testing challenges. Testing phases can be marginalized for a number of reasons, including budget and time constraints, a lack of testing expertise or resources and more.

To avoid the pitfalls of IoT testing, here are three tips companies should consider when designing their next IoT test.

Three IoT Testing Tips

Test Coverage and Compatibility

Due to the problems caused by market fragmentation, or the range of possible IoT and smartphone configurations, it's essential to execute test campaigns with dozens of smartphones, tablets, or other devices that can be used to control or communicate with your IoT product/service.

These testing devices should represent, as closely as possible, the same configurations used by your product’s target audience. Finding as many anomalies as possible on the smartphones, tablets, operating systems most commonly used by a target audience is vital for commercial success.

Websites like Statcounter or Statista can provide helpful information regarding the most used smartphones and operating systems across a collection of markets.

Testing Approaches: UAT and UX

A second, more general mistake companies can make when testing IoT products is failing to carry out a wide variety of tests before launching a new device. Testing an IoT product involves more than testing the actual device. You need to consider the whole context, including intersecting software systems, operational processes, and more, which may be used to communicate and control the product.

Here, it's essential to execute user acceptance testing (UAT) campaigns to validate the product and its software function according to their established requirements. Another key test is user experience (UX) testing, which is designed to measure the user’s experience while using the IoT product and its software.

Insight gained from UX testing can highlight design issues regarding the IoT product or its software that contribute to a positive or negative experience. Although a great UX may not be enough to save poor functioning IoT products from commercial failure, a poor UX is almost guaranteed to limit the commercial success of even well-functioning IoT products.

Besides UAT and UX tests, security testing is also absolutely necessary. The security vulnerabilities facing IoT products have been well documented; they may even limit IoT if the industry fails to take appropriate actions to improve security protocols.

Security tests should be designed to evaluate whether or not data is being properly encrypted and protected during transmission. Executing additional tests is advised to ensure security protocols, like identification, authentication and end-to-end encryption will prevent unauthorized parties from gaining control of an IoT or from accessing its data.

Since IoT doesn't exist in a bubble, performance testing is also key to evaluating how well IoT products perform in a variety of real-world settings.

Connectivity Tests

Last but not least, it's crucial to test an IoT product's ability to connect to other devices via WiFi networks, Bluetooth, near-field communication (NFC), Z-Wave and any other additional forms of connectivity technology an IoT device may have. For IoT products, connecting and remaining connected to another device or smartphone is no easy task.

Connectivity issues can hinder the ability of an IoT device to share data with other connected devices, which in the end will render it only narrowly useful to users. What's the point of a connected device that's disconnected from other connected devices? To avoid these problems, companies should not only examine a product’s ability to connect via different technical possibilities, but also test its ability to remain connected over long periods of time.

For products that will join integrated systems, it's equally important to test their connectivity across various conditions, including those that are less than ideal. Eliminating any connectivity issues, whether caused by a software bug or unreliable hardware, can help the industry produce better products that build consumer and business confidence and fuel future growth.

Conclusion

Overall, whether or not IoT reaches its potential will largely depend on the IoT industry’s ability to produce high-quality products efficiently and at scale. The key will be creating a set of standards and protocols that can mitigate some of the current challenges facing development teams, namely security and interoperability. Until then, product-by-product testing will continue to be a completely essential step in the IoT product development lifecycle.

Written by Jordan Medard of Stardust Testing.