The technology landscape has evolved radically in the last few years and the evolution of the Internet of Things (IoT) could turn out to be one of the biggest testing opportunities.

As per Gartner, by 2020 there will be over 26 billion connected devices; ABI Research goes further and predicts 30 billion wireless devices connected to the Internet by 2020. Other research companies (like Morgan Stanley & Hammersmith Group) predict that it could be up to 100 billion in a few years. This translates to at least 5 times more IoT devices compared to the world’s current population. These vast and networked arrays of smart devices would be enabled by a new-age Internet through the wide adoption of IPv6 and Network Virtualization technologies and smart & cheaper electronics, thereby creating testing opportunities.

IoT and embedded systems testing has much in common with application testing, but there are key differences that need to be brought out. Traditional software testing focusses on Black Box and GUI and doesn’t necessarily consider the source code. Embedded testing involves additional hardware elements and therefore, testing focusses on this hardware, simulation, real time behavior, down time and grey box as the final results depend on these various aspects. Therefore, a new more collaborative or integrated way to work with software before you deploy it is required.

IoT will necessitate a fundamental shift in testing  

Across the board, the Internet of Things could restructure software development and releases into an intuitive cycle.  Accordingly, to facilitate this new paradigm shift, a new test strategy needs to be formulated. The path to a well thought strategy should cover these 5 specific areas:

Testing IoT devices pertains to capturing metrics related to data values which are read from sensors and generated from a variety of home appliances, thermostats, light, bulbs and door locks, to provide a few examples.

Test Automation or Automated testing would be a crucial factor to implementing IoT. Despite the challenges faced with end to end test automation, automated testing is the way forward due to the large adoption of IoT by enterprises and associated mobile computing devices. With the variety of devices available today and further growth expected, automated testing would provide significant benefits during the development cycle.

Simulation Testing would be definitely required for IoT testing. The reasonably extensive number of IoT devices to connectivity options available & growing, it’s imperative that testing tools and procedures verify these communication protocols, including WiFi, BLE, 3G, 4G, LoRa, MiWi, etc.

Security Testing is needed at the device level and the network connectivity through which the data transport happens. This involves access mechanism, authentication, encryption techniques, etc which need to be validated.

Interoperability Testing is usually associated with hardware/software components, systems and platforms that enable machine to machine communication.  This phase is necessary to validate the different devices and things selected to work in the overall solution based on the standards available.

Real time testing is testing performed in a real time environment in which IoT devices must operate. This exposes them to weather, site requirements, and physical impact. This phase is sometimes referred to as Factory Acceptance Testing (FAT) which involves calibration tests and standard based certification.

 

Key considerations for IoT Testing

From an end to end perspective, a generic IoT platform has typically four key components: sensors, application, network communication, and backend database. Identifying the testing scope and coverage is the most important element for formulation of a comprehensive test strategy. The key testing considerations for each of the components of IoT platform are:

1. a) Sensors/Devices – This would involve device hardware, embedded software, sensor response time and performance.
2. b) Network Connectivity – This component pertains to network connectivity, interaction between devices, and data transmission rate.
3. c) Applications – Functionality, user roles & access, multiple request handling, etc
4. d) Database – Data values validation, data encryption/decryption, data consistency and integrity, performance etc

 

Developing a Testing approach that delivers value

The Internet of Things and the rise of the smart devices, ushers in agile IoT platforms which are complex, scalable, and high-performance systems that will require most testing strategies and practices to evolve with both embedded and software components closely integrated and grounded in automated testing.

Since we are focused on IoT embedded systems, we can look at the test approach that will focus on these different embedded test types and target areas, further complimenting the application testing side which is fairly mature today.

 

Different types of Embedded Testing

Target areas at the IoT embedded device level

1. Device Characteristics: these characteristics represent the data model of the device, presented by various data types, which are saved to and fetched from the cloud, for validation of their integrity. Devices supports different data types. The tests validate the data as well as ensure that all the data types are saved properly and retrieved from the cloud.
2. Device Interaction Module: There are protocols like UART which support the reliable data exchange between the microcontroller and the module, are to be tested according to their specifications. The test verifies that data is transmitted in sequence and properly acknowledged.
3. Output Drive Tests – This test is normally at the edge and which is responsible for other high-power entities. This is achieved via a GPIO interface.
4. Firmware upgrade/downgrade: Customers may be using different releases of the firmware, so all upgrade-downgrade paths should be tested.  This is normally tested Over the Air (OTA) i.e. between device to cloud, for example, against a pre-defined compatibility matrix, to ensure all covered firmware releases are upgradable to the new release.
5. CPU, Memory Usage – This is related to RTOS and OS tests pertaining to monitor the CPU and memory consumptions during different nodes of operation.
6. Power Modes – This test is always covered on the edge wherein there are different types of power mode available.
7. Connectivity: The testing focus on validating the wireless configuration for protocols like Bluetooth, BLE, Zigbee, 3G, 4G, LoRa, etc. For a WiFi system, the test validates the wireless configuration, the ability to switch between AP and station modes, and the ability to connect to the various WLANs with different security settings. These standards evolve rapidly and it’s important to validate them.

Conclusion

The Internet of Things and the rise of the smart devices, ushers in agile IoT platforms which are complex, scalable, and high-performing systems that will require most testing strategy and practices to evolve with both embedded and software components closely integrated and grounded in automated testing.

Therefore, IoT Testing is a critical success factor for an IoT deployment. A well thought out and holistic testing approach that must include Embedded systems testing, based on industry best-practices and is developed through innovative and proven expertise provides the much-needed predictability and assurance for a successful IoT implementation.

Adopting a multi-layered and end to end approach provides significant benefits related to early detection of bugs, improved test cycles through automation, increase in confidence levels and the required testing efficiency & sufficiency to ensure a smooth IoT implementation.

Read complete blog