[Srijan Wednesday Webinars] Why Adopt Analytics Driven Testing
Download as PPTX, PDF•1 like•301 views
Analytics-driven testing uses data on user behavior and software usage to prioritize testing efforts on the most critical areas. This allows testing teams to significantly reduce regression testing time by focusing on the most popular features and environments rather than exhaustively testing all possible combinations. Analytics can provide insights into actual usage patterns to determine the optimal risk-based testing strategy and achieve high test coverage within constrained timelines.
![[Srijan Wednesday Webinars] Why Adopt Analytics Driven Testing](https://image.slidesharecdn.com/srijanwednesdaywebinarsanalyticsdriventesting2-170615131224/85/Srijan-Wednesday-Webinars-Why-Adopt-Analytics-Driven-Testing-27-320.jpg)
![[Srijan Wednesday Webinars] Why Adopt Analytics Driven Testing](https://image.slidesharecdn.com/srijanwednesdaywebinarsanalyticsdriventesting2-170615131224/85/Srijan-Wednesday-Webinars-Why-Adopt-Analytics-Driven-Testing-33-320.jpg)
[Srijan Wednesday Webinars] Why Adopt Analytics Driven Testing
- 1. Why Adopt Analytics Driven Testing Ori Bendet | Inbound Product Manager, HPE Software | @bendet_ori #SrijanWW | @srijan
- 2. ↓80%Reduction in regression time
- 3. 7y in HPE Software in various managerial positions Started in testing Today: Inbound Product Manager StormRunner Functional ABOUT ME
- 4. AGENDA • Applications Development Overview • Today’s Testing Challenges • What are the risks? • Real World Examples • How can you empower your testing by using Analytics
- 5. The World of Software is changing Mega-trends create demands on modern applications MODERN APPLICATION DEVELOPMENT Reduce costs Increase customer attraction/retention Increase the value of your brand Get to market faster 300,000 tweets 200 million+ emails sent 220,000 new photos posted 50,000 apps downloaded $80,000 in online sales 72 hours of new video content uploaded 2.5 million pieces of content shared Agile Every minute*… * Source: “The Data Explosion in 2014 Minute-by-Minute”by ACI InformationGroup. http://aci.info/2014/07/12/the-data-explosion-in-2014-minute-by-minute-infographic/ Mobile Cloud Digital DevOps
- 6. 8
- 7. What we need to test TestingTime
- 8. =Quality
- 9. $59.5BAnnual cost of software defects
- 10. How to mitigate the risk? • Have more testers • Crowdsource • Don’t test • Let your users test for you • Or… use Analytics!
- 11. Analytics
- 12. Example
- 13. Example • My product: 30 features • Supported Matrix: 10 environments (OS/Browser) • My testing team: 10 testers • Testing a feature on an environment = 1 day • Target regression cycle: <3 days!
- 14. 30 features X 10 combinations = 300 days 300 days / 10 testers = 30 days
- 15. Google Analytics • A freemium web analytics service offered by Google that tracks and reports website traffic. • Google launched the service in November 2005 after acquiring Urchin. • Google Analytics is now the most widely used web analytics service on the Internet. • Note: I’m not affiliated with Google Analytics in anyway * Source: https://w3techs.com/technologies/overview/traffic_analysis/all
- 16. Environment Optimization • Focus on the majority of combinations (OS x Browser) • Reduce your regression risk to a minimum • Wait for customer feedback on the other areas
- 17. Let’s see it live
- 18. 30 features X 10 2 combinations* = 60 days 60 days / 10 testers = 6 days (*) with 89% confidence
- 19. Example
- 20. User Behavior • Focus on the majority of the functionality • Target the most active areas • Wait for customer feedback on other areas (% of escaped defects) • Calculated risk(!)
- 21. Let’s see it live - again
- 22. 0 12 features X 10 2 combinations* = 24 day 24 days / 10 testers = 2.4 days (*) with 76% confidence
- 23. Example – part 2
- 24. Building a risk calculator • Test everything (100%) = 300 days • All functionality, top combinations = 60 days • Main features, top combinations = 24 days • Add additional levels based on your needs and time
- 25. Building a risk calculator Top flows 1 OS + Browser Main Functionalitie s top combinations All Functionalitie s top combinations All Functionalitie s Additional combinations Test Everything 50% 75% 90% 100% 95%
- 28. Don’t have analytics? • Use market analysis and statistics • https://netmarketshare.com/ • https://www.w3schools.com/browsers/ • Many more! • Conduct annual user survey • CustomersValidations – before & after
- 29. A word about Test Automation
- 30. Re-use calculator for Automation Top flows 1 OS + Browser Main Functionalitie s top combinations All Functionalitie s top combinations All Functionalitie s Additional combinations Test Everything 50% 75% 90% 100% 95% CI Nightly Full regression
- 31. Summary
- 32. Summary • Don’t test everything! • Get to know your users • Calculate your risk • Measure your un-tested areas • % of Escaped defects • Customer Satisfaction
Editor's Notes
- #3: http://blog.celerity.com/the-true-cost-of-a-software-bug
- #6: Set the stage, have a conversation with the audience about their biggest challenges, changes and painpoints in application delivery
- #11: http://blog.celerity.com/the-true-cost-of-a-software-bug
- #12: http://blog.celerity.com/the-true-cost-of-a-software-bug
- #15: 30 features = 30 days 10 OS + Browsers combinations = 300 days
- #16: 30 features = 30 days 10 OS + Browsers combinations = 300 days
- #17: 30 features = 30 days 10 OS + Browsers combinations = 300 days
- #18: How can Google Analytics help you??
- #19: How can Google Analytics help you??
- #20: Advantage Online Shopping – Browser Overview
- #21: 30 features = 30 days 10 OS + Browsers combinations = 300 days
- #22: 30 features = 30 days 10 OS + Browsers combinations = 300 days
- #24: Google Demo Account – Behavior
- #25: 30 features = 30 days 10 OS + Browsers combinations = 300 days