#Interactive Session by Pradipta Biswas and Sucheta Saurabh Chitale, "Navigating the IoT Performance Testing Landscape" at #ATAGTR2023.
0 likes19 views
The document discusses the significance of IoT performance testing for ensuring the reliability and efficiency of IoT systems. It covers various topics including the differences between web-based and IoT performance testing, key focus areas and tools, non-functional requirements for different use cases, challenges, and best practices. Additionally, it highlights the importance of conducting thorough performance tests to identify potential issues before they affect users.
#Interactive Session by Pradipta Biswas and Sucheta Saurabh Chitale, "Navigating the IoT Performance Testing Landscape" at #ATAGTR2023.
- 1. December 9, 2023 Navigating the IoT Performance Testing Landscape Sucheta Chitale Pradipta Biswas Global Testing Retreat #ATAGTR2023 2, 3 & 8, 9 December 2023
- 2. Speaker Introduction Performance Test Manager, Sogeti, Capgemini Performance Test Lead, Sogeti, Capgemini Sucheta Chitale Pradipta Biswas
- 3. Agenda 01 What is IoT ? 02 Quick Stats! 03 Performance Testing in IoT 04 Difference between Web-based PT and IoT PT 05 IoT Performance Testing – Focus Areas & tools 06 NFRs for different Use Cases 07 Challenges and best practices 08 Diagrammatic representation 09 Example
- 4. What is IoT ??
- 5. IoT Applications & Types Consumer IoT Industrial IoT Wearables Fitness bands Smart gears Smart home Manufacturing Pharmaceuticals Supply Chain Healthcare Oil & Natural Gas Mining Logistics Agriculture
- 6. Quick Stats 29B Devices by 2027 acc. Statista 483 $B Spending by 2027 acc. Statista
- 7. Performance Testing in IoT Performance testing in IoT (Internet of Things) is a crucial aspect of ensuring the reliability, scalability, and efficiency of IoT systems. IoT performance testing is the process of evaluating the performance and responsiveness of IoT devices, networks, and applications under various conditions. It aims to ensure that IoT systems can handle the expected load, maintain reliability, and provide optimal user experiences. Scalability Key Objectives Reliability Latency and Responsiveness Throughput Security Resource Utilization
- 8. Difference between Web based PT and IoT PT Key Differences Web-based PT IoT PT Communication Protocol HTTP / HTTPS, FTP, SMTP, JDBC, etc. MQTT, CoAP, AMQP, UDP, XMPP, HTTP, etc Simulation End user behaviour simulation Simulation of devices/ sensors Types of Workloads considered Usually, the user-initiated workload is considered Device-initiated and IoT platform-initiated workloads are considered Requests / Responses Users create the requests and receive the responses IoT devices create requests and receive responses as well as requests, and they also provide responses Performance Test Coverage Application Layer Physical, Network and Transport layers Big Data and BI Not all web applications require performance analysis of Big data processing and BI analytics Performance analysis of huge volumes of data from IoT devices are analyzed using Big data, BI and analytics solutions for intelligent decision making in all IoT applications
- 9. End to End IoT Performance Testing API Performance Device Simulation Gateway Scalability Simulation of Different Network Speeds Volume performance assessment, batch jobs , Interface, reports performance End User Customer Experience, Mobile App, Front End Performance Experience IoT Cloud Platform Performance, Scalability, Resilience IoT Performance Testing – Focus Areas Neoload OpenText LoadRunner JMeter Locust IoTify Bevywise IoT Simulator AWS IoT Simulator Azure IoT Device Telemetry Simulator Tools
- 10. Connected Factories Medical Devices Monitoring Remote Fleet Management Smart Grid/Meters Use Case: Industry 4.0, Predictive maintenance, Preventive Maintenance, Equipment Uptime, Connected Floor NFRs: IoT Platform Stability, Scalability & Reliability. Concurrent Assets, Data transmission through Various networks Use Case: Track Vehicle Location, Monitor Vehicle Health, Monitor Parameters such as speed, fuel, brakes etc. NFRs: IoT Platform Stability, Scalability & Reliability. Concurrent Vehicles, Vehicle To Cloud Transmission, App/Dashboard Performance Data transmission through Various networks Use Case: Remote metrics management, safety, billing & metering NFRs: Billing Systems Performance, Responsiveness through different network conditions, Multi meter connectivity, Integrations Use Case: Remote Device management, Upgrades, Multi device connectivity NFRs: Concurrent devices connectivity, Responsiveness through different network conditions, device upgrades NFRs for different Use Cases
- 11. Performance Testing for MQTT using JMeter Simulation On-premise LG Cloud LG PUBLISH A MESSAGE SUBSCRIBE TO A TOPIC APPLICATION UNDER TEST APP SERVER DB SERVER MQTT BROKER
- 12. Example of an IoT Performance Testing Scenario: Smart Home Security System System Components: IoT Cameras: Installed around the house to monitor security. IoT Sensors: Detect motion and door/window openings. Central Hub: Collects data from cameras and sensors and controls the system. Mobile App: Allows users to monitor and control the security system remotely. Performance load testing: simulate large number of users accessing the mobile app simultaneously, checking live camera feeds and receiving real-time alerts from sensors. Performance Stress Testing: Gradually increase the number of IoT devices connected to the hub beyond its recommended capacity. Continuously monitor system behaviour. Network Performance Testing: Simulate varying network speeds and connectivity interruptions to assess how the IoT devices and the mobile app handle such scenarios. Performance Scalability Testing: Gradually increase the number of IoT devices and sensors in the system to evaluate its ability to scale both vertically (adding more resources to a single device) and horizontally (adding more devices). IoT performance testing is crucial for ensuring the reliability and functionality of IoT systems, especially in scenarios where real-time data and responses are critical, such as smart home security systems. By conducting thorough performance testing, we can identify and address potential issues before they impact users or result in system failures.
- 13. Challenges & Best Practices in IoT PT Scalability Diversity of Devices and Protocols Data volume, validation and Integrity Edge and Fog Computing Real-Time Analytics & Insights Firmware Updates Security Challenges Handle large volume of data and connections simultaneously. Require support for various communication standards. Simulate large volumes of data at a high velocity. Evaluate the performance of these distributed processing components. Assessing speed and accuracy of analytics. Firmware updates in IoT devices. Vulnerability to security threats & performance degradation. Create realistic workloads that simulate expected real- world scenarios. Simulate various network conditions, ensure compatibility, performance across the spectrum. Validate data integrity. Test the performance of edge computing components, implement OTA updates. Generate actionable insights based on the collected IoT data. Test with firmware updates, Rollback, Fallback mechanism. Integrate security testing into performance. Best Practices
- 14. THANK YOU