#ATAGTR2021 Presentation : "Use of AI and ML in Performance Testing" by Adolf Patel
Download as PPTX, PDF0 likes373 views
The document discusses the impact of artificial intelligence (AI) and machine learning (ML) on performance testing, emphasizing their ability to automate tasks such as scripting and monitoring for more efficient and accurate results. It highlights methods like data analysis, anomaly detection, and smart service level agreements (SLAs) that enhance the performance testing process and user experience. The integration of AI and ML brings predictive capabilities that can identify and prevent potential performance issues before they occur.
#ATAGTR2021 Presentation : "Use of AI and ML in Performance Testing" by Adolf Patel
- 1. #ATAGTR2021 Use of AI and ML in Performance Testing Adolf Patel
- 2. Overview Artificial Intelligence & Machine Learning Aspect of AI in testing How AI and ML can help in performance testing AI and ML based strategy for performance testing How AI and ML are changing performance testing process Detecting performance anomalies using AI & ML Benefits of AI-ML in performance testing Q&A Agenda
- 3. Overview In today's world of digital era where performance testing is a must to do activity before production rollout, Artificial intelligence(AI) and Machine Learning(ML) is a game changer. Despite of many automation tools available in the market which reduces human effort in developing the scripts, executing the test scenarios and analysing the result, still, there is good deal of manual/human efforts required when it comes to evaluate the results. Use of AI in performance testing will make tasks like scripting, monitoring highly impactful and help to get real time results very quickly and more accurately. AI is crowding the technological landscape at snowballing speed. Appl’s Sri, Microsoft’s Cortana or Google’s Assistance are some of the common AI powered tools we hear in everyday life. How about something similar for performance testing like, you say, “Hey Sri, please send me the performance metrics of XYZ application on my email” and you get a nice report appear in your mailbox or you say, “Hey Sri, please execute an endurance test for 4 hours for XYZ application and send me the report to my email” and your test environment spin up and execute the test and send the report to your email. It sounds very impressive, isn’t it? Can it be done or even possible now? AI is here, and it's bringing a whole new dimension to the performance testing.
- 4. AI is a system with increasingly self-learning capabilities, that can complement human reasoning and activities by understanding the environment, solving human problems, and perform human tasks. In simple, Artificial intelligence is the simulation of human intelligence processes by machines, especially computer systems. Machine learning is a method of data analysis that automates analytical model building using historical data as input to predict new output values. ML is a subset of Artificial Intelligence. The machines learn from the history to produce reliable results. • Supervised Learning - Supervised learning algorithms analyze the training datasets data and produce an inferred function. Example, Credit card fraud detection. Artificial Intelligence & Machine Learning • Unsupervised Learning - Unsupervised learning algorithms analyze and learn the un-clustered data and produce an inferred function. The algorithm itself finds the patterns in the data. Example, Facebook friend request suggestion mechanism. • Reinforcement Learning - Reinforcement learning is defined by characterizing a learning problem and not by characterizing learning methods. Example, bots
- 5. Aspect of AI in testing In the world of Agile, IT Managers want the code to build quickly, test quickly before the production and deploy the code to production. If any errors comes during testing, fix the issue and repeat the testing lifecycle quickly. More than 80% of the teams follow Agile methodology for software development and 70% of the release happens weekly or before weekly basis. Functional issues can be found and tested before production but finding a performance issue or predicting the performance issue or pattern in the production is bit difficult. So, can we use AI solution to make things faster and smarter? AI brings the ability to anticipate problems appearing in a system can prevent any unforeseen issues in app development Not only will AI increase the reliability of predicting system breakdown in general operations, but test engineers will also be able to use AI to create test strategies focused on recognizing dangers most likely to appear.
- 6. How AI and ML can help in performance testing • Quick and Better Data Analysis – Reduce error and improve data usages. • Enhanced user experience – Conventional load testing involves server side performance testing using some protocol like HTTP, This does not give the real user experience performance. AI can be useful to load test the real browser based load testing which give the better user experience performance. Tool like Loadninja, a testing platform on the cloud to load test web applications with real browsers at scale. SMART Identify performance trends with more granularity – Use machine learning(ML) to dig deep into the data and highlight key performance bottleneck for QA and Dev. Create SMART SLA’s – A SMART SLA helps service providers to set stakeholder expectation according to current as opposed to the past system behavior. It also enable test engineers to create performance tests scenarios according to the latest performance data available rather than depending on the past data every time they perform the test. Prevention is better than cure – AI help predict the issues before it occur thus giving time to engineers to fix before it pops up.
- 7. AI and ML based strategy for performance testing Planning Test Design Test Execution Analysis & Reporting Production - Live • NFR Collection • SLA Definition • Script Creation • Workload Modelling • Smart Analysis • Issue finding • Tunning Recommendations • Live Monitoring Issue, Fixes & Retest Application Updates/Maintenance Collect Analyse Learn Predict AL/ML Assisted Components AI-powered application monitoring and scripting tools can detect issues in real time, pinpoint the source of the problems and automatically launch the proper remediation processes — often before end-users even recognize that there’s been any kind of problem. Application/ Environmen t
- 8. How AI and ML are changing performance testing process Make realistic test data – Setting up test data for a performance test scripts are not so easy. Many times, you need to do it manually in csv files, sometimes you capture dynamic data from response packets and use for next course of action in your scripts. An ML assisted tools or simple ML program can help building the vast range of data in no times. Ensure system is resilient - Many of us are familiar with Chaos Monkey, the Netflix tool that intentionally pulls down production instances, given that every system should be redundant. The objective is to test if a system is pulled down, there should be no impact to system performance. By monitoring performance when tests are running, management can determine when redundancy is lacking. Picking systems and randomly pulling them down is arguably a type of AI. By noticing what is failing and what is working, Chaos Monkey can produce more complex strategies for breaking things, a sort of true AI that can improve the customer experience. Provide Realtime insight into a system’s performance - ML solutions can analyze and interpret millions of data in no time, providing near to real time insight into systems performance. They can provide patterns and make predictions to solve performance issues faster and more accurately than human, hence improving the efficiency. Predicting performance in production – Can we predict how the application will perform under certain load in production? The answer is may be. The simplest application of ML is to use data sources to predict trends. Trends help leaders tell if a situation is under control, getting better, or getting worse and how fast it's improving or worsening.
- 9. As enterprise software platforms expand in complexity and usability, performance anomalies have become a serious threat that can result in huge loss in cost and reputation of an enterprise. Machine learning solutions can analyze and interpret thousands of statistics per second, providing real-time insight into a system’s performance. There are several ways machine learning can be utilized to detect performance anomalies in performance. It keeps baselining the real time behavior and comparing against the new normal for any anomalies. Here are few of the most popular methods. Detecting performance anomalies using AI & ML Poor User experienc e Failed Load Balancing Blurred Video Streamin g Decrease d Throughp ut Server Crash Freezing Hanging Increased Latency Delayed Response Time Density-Based – Assuming that normal data points occur around a dense neighborhood and, therefore, anomalies are far always, this method of detection is based on the K-Nearest neighbors' algorithm, or alternatively the local outlier factor. Clustering-based- This method is one of the most popular concepts for unsupervised learning. It assumes that data points that are similar usually belong to similar groups. Here, a K-means algorithm creates ‘k’ similar clusters of data points, with instances that fall outside the clusters being marked as potential anomalies. Support vector machine-based – Support vector machine is generally associated with supervised learning. Here the algorithm learns a soft boundary to cluster the normal data and then identifies instances that fall outside the learned region.
- 10. Benefits of AI-ML in performance testing Codeless automation using natural language processing(NLP) Testing environment developed using ML helps in self healing and corrections Test flows can be automated and can be tested using data Performance test modelling process provide real time workload during test execution SMART SLA preparation and real time workload modelling Adapting the performance rules over time and inventing new rules for automated remediation Extrapolating the resulting data to human experience of performance Increased used of AI assisted APM tools