Tools of the Trade: Load Testing - Ignite session at WebPerfDays NY 14
Download as PPTX, PDF2 likes1,763 views
This document provides insights on load testing, emphasizing its importance for system performance validation and optimization. It reviews various commercial and open-source load testing tools, highlighting their functionalities and suitability based on specific user needs. The author encourages participants to attend a performance and capacity conference for further learning opportunities.
![Tool’s Ecosphere
18
Name Number of found
documents by
Google
Number of found
US positions at
Monster.com
[HP] LoadRunner 894,000 170
[Apache] JMeter 688,000 90
[Borland] SilkPerformer 138,000 12
[Neotys] NeoLoad 87,100 3
[SOASTA] CloudTest 51,500 2
LoadStorm 18,000 -](https://image.slidesharecdn.com/toolsofthetradeloadtesting-ignitesessionatwebperfdaysny14-140924104141-phpapp01/85/Tools-of-the-Trade-Load-Testing-Ignite-session-at-WebPerfDays-NY-14-18-320.jpg)
Tools of the Trade: Load Testing - Ignite session at WebPerfDays NY 14
- 1. WebPerfDays NY 2014 Tools of the Trade: Load Testing Alexander Podelko [email protected] alexanderpodelko.com/blog @apodelko September 18, 2014
- 2. About Me • Specialized in performance for the last 17 years • Currently performance testing and optimization of Hyperion products at Oracle • Board director at CMG http://cmg.org – Organization of performance and capacity professionals – Conference November 3-6, 2014 in Atlanta, GA Disclaimer: The views expressed here are my personal views only and do not necessarily represent those of my current or previous employers. All brands and trademarks mentioned are the property of their owners. 2
- 3. Front- vs. Back-End • Back-end response times depend on load Thanks to Andy Hawkes for the graph 3
- 4. Load Testing Applying multi-user synthetic load to the system – Load testing – Performance testing – Stress testing – Scalability testing – Volume testing – Reliability testing – Concurrency testing – Realistic Testing – Endurance testing – Longevity testing – Soak testing – Stability testing 4
- 5. Can System Handle Peak Load? • You can’t know without testing: 5
- 6. Verify Multi-User Performance • Single-user improvement may lead to multi-user performance degradation 6
- 7. What Else Load Testing Adds • Performance optimization – Apply exactly the same load – See if the change makes a difference • Debugging/verification of multi-user issues • Testing self-regulation functionality – Such as auto-scaling or changing the level of service depending on load 7
- 8. Commercial Tools Gartner Magic Quadrant for Quality Suites: • Leaders: – HP (LoadRunner) – IBM (Rational Performance Testing) – Microsoft (Visual Studio Web Performance…) – Oracle (Application Testing Suite) – SOASTA(CloudTest) – Borland (SilkPerformer) • Peformance: Neotys (NeoLoad), Blazemeter 8
- 9. Open Source Tools • 53 tools listed at http://www.opensourcetesting.org/performance.php • Jmeter – Clear leader in popularity • Gatling – Getting popularity recently • Grinder, OpenSTA – Were popular/looks like no development recently 9
- 10. Mixed Business Models • Advanced functionality – With small tests for free • Freemium models – LoadRunner, NeoLoad (up 50 users free) – CloudTest (Lite up to 100 users free) • BlazeMeter service using open source JMeter • Important when more people get involved – Continuous Integration, Agile development 10
- 11. Load Testing Tools Differ drastically in: – Supported approaches / protocols – Functionality / extendibility – Supported environments – Scalability – Result analysis – Integration (monitoring, APM, CI, etc.) – Cost/Licensing – Available support and skills 11
- 12. Record and Playback: Protocol Level Load Generator Server Load Testing Tool Virtual Users Application Network 12
- 13. Record and Playback: UI Level 13 Load Generator Server Load Testing Tool Virtual Users Application Network Brows ers
- 14. Programming Load Generator Server Load Testing Tool App. Virtual Users Application Network API 14
- 15. Environments • Lab vs. Service (SaaS) vs. Cloud (IaaS) – For both the system and load generators • Test vs. Production • No best solution, depends on your goals / system 15
- 16. Scenarios • System validation for high load – Outside load (service or cloud), production system – Wider scope, lower repeatability • Performance optimization / troubleshooting – Isolated lab environment – Limited scope, high repeatability • Testing in Cloud – Lowering costs (in case of periodic tests) – Limited scope, low repeatability 16
- 17. Integration • Reporting and Analysis • Monitoring / APM – System – Application insights (APM) • AppDynamics, New Relic, Dynatrace, etc. • Continuous Integration – CI servers (Jenkins, Hudson, etc.) – Adding to automatic build process 17
- 18. Tool’s Ecosphere 18 Name Number of found documents by Google Number of found US positions at Monster.com [HP] LoadRunner 894,000 170 [Apache] JMeter 688,000 90 [Borland] SilkPerformer 138,000 12 [Neotys] NeoLoad 87,100 3 [SOASTA] CloudTest 51,500 2 LoadStorm 18,000 -
- 19. Summary • There is no best tool – it depends on your needs • Almost every tool works for a small and simple web site • If using more sophisticated technologies, always check if the tool / approach supports it 19
- 20. Questions? Alexander Podelko [email protected] alexanderpodelko.com/blog @apodelko Need More? Come to the Performance and Capacity conference http://www.cmg.org • November 3-6, 2014 in Atlanta, GA • Perfect complement to Velocity / WebPerfDays