Karthikeyan Lakshminarayanan

Topic: Adopt Non-Functional Testing to realise the full potential of Connected Devices

Abstract: The Internet of Things (IoT) has been a hot topic for decades. It is a matter of fact today that several IoT products have surpassed a huge number of humans on this planet. According to experts, the number of IoT devices is set to reach 25 billion by 2025 and be worth some $1.1 trillion by 2026. It has changed the way people live, do business, and interact.

Smart Homes and Health care are a clear example of how advanced technology and IoT solutions can radically improve the way we approach and solve societal challenges. As we all know, the world is fighting with novel coronavirus and we are isolated in homes depending upon the technology to move on with business. On the other hand, healthcare officials are working tirelessly to provide the best facilities to prevent citizens from infection and to save those who are already infected. In such a scenario, technology innovation is steadily making a difference and countries are mustering all resources at their disposal to deploy cutting-edge technology to mitigate the effects and profile people at risk.

These situations remind us the way business need to enhance their work methodologies and to adopt latest technologies that are secure and scalable. Taken an example of smart homes, people are spending considerable time at home using smart cameras, wearables and telecommunications to remain in touch with their business and day to day life. Home automation provides comfort, convenience and energy efficiency by allowing them to control smart devices, often by a smart home app on their smartphone or other networked devices. These machines transmit the signals every few seconds, gets processed and stored into the backend system for betterment of daily operations. As customers moving to IoT devices it is essential that performance of these communications is transmitted and processed seamlessly under secure conditions. Degradation in performance & security in the data flow might impact the business.

Home Automation typically connects with centralized hub device controlling all other smart devices like bulb, camera, plugs, thermostat etc. Integration of Device to Device through wireless technologies makes this an integral part. When it comes to Non-functional (Performance and Security) the focus must be on device connectivity and data transmission. Testing of these components needed better understanding of required protocol to be used at each layer and the selection of right tool set for message simulations, processing and monitoring.

IoT Non-functional testing teams are facing an extraordinary challenge today in defining & assuring the quality of the IoT ecosystem. From that perspective, the key asks are:
1. How would they ensure the coverage of the entire spectrum of real world IoT use case scenarios are performant & secure?
2. How to orchestrate, execute and analyse the results to focus on the key non-functional metrics?

For successful implementation, IoT Performance & Security needs to shift its focus at macro level and overcome below challenges:
• Simulate traffic from Device to Device (D2D)
• Inadequate mechanism to monitor (like Connectivity, Network Transmission, Node & Routing Mechanism) non-functional aspects of IoT Devices
• Lack of Non-functional requirements to test the IoT devices
• Continuous regression suite implementation to test IoT Devices
• Validating & Verifying the non-functionality of the entire IoT Ecosystem
• Ensuring proper security best practices are in place for IoT Devices

Our solution will focus on modern day Non-functional testing approach for connected devices (IoT), identify its key performance indicators and provide solution for end to end process. It also speaks about the experience in setting up right tool sets and test IoT device components to its fullest capacity.

Solution Overview:
1. Methodology to Simulation & Sniffing calls: Use custom code (Python) to sniff the traffic between devices and manually simulate required business actions using custom chips between IoT devices to measure performance metrics
2. Advanced NFR Requirements: Define performance & security requirements at various approaches like load simulation, network transmission & connectivity, encryption & security standards
3. Continuous Automation: Use automated framework to simulate traffic under different network conditions and implement for continuous delivery
4. End to End approach: Simulate traffic from device/client layer (mobile/web) under different network conditions and validate data ingestion, data processing, data persistence, data consumption on complete ecosystem.
5. Static Firmware Analysis: Unique security testing approach to ensure the security of device firmware

Above solutions aided the business to identify key non-functional bottleneck at each layer including Device to Device (D2D) communication and tune them for efficient performance. This gives a view that validating Non-Functional set of aspects are not just verification process, it is also about influencing the quality of the complete architecture during software development lifecycle. Adopting these Non-Functional techniques to validate & verify an IoT Device ecosystem will help to scale the business to new heights. This diverse solution certainly benefits not only for the Home Automation but also across several industries of which IoT Devices been involved and playing a key role.