Software performance testing_overview
Download as PPTX, PDF1 like843 views
The document discusses key concepts related to software performance testing. It defines performance testing as determining the speed or effectiveness of a computer system or software. It outlines various terminology used in performance testing like throughput, response time, tuning, benchmarking. It also describes different types of performance testing and factors that should be considered like bottlenecks, test process, tools, and metrics.
Software performance testing_overview
- 1. Software Performance Testing Overview Rohan Bhattarai
- 2. Contents What is Performance Testing? What are the Terminologies I should know? Why Performance Testing? What are the Performance Bottlenecks? Why do we have bottlenecks ? What are the contributors? What are the types of Performance Testing? When is it required? What should be tested? What should be my Performance Testing Process? Why should I use Tool ? What should be my tools combination? What tools are available and what should I use for Performance Testing? What are the Performance Metrics? What should be my best approach?
- 3. What is Performance Testing? Performance testing is the process of determining the speed or effectiveness of a computer, network, software program or device.
- 4. Let’s talk about Software Performance Testing Before going into the details, we should understand the factors that governs Software Performance Testing: Throughput Response Time Tuning Benchmarking
- 5. What are the Terminologies I should know? Application Architecture HTTP vs HTTPS Request Vs Response GET and POST Methods HTTP Status Codes Throughput Response Time Tuning Benchmarking Transaction Mix User Mix User Think Time Ramp-up Pattern
- 6. What are the Terminologies I should know? Application Architecture Client-Server 2-Tier 3-Tier N-Tier
- 7. What are the Terminologies I should know? HTTP vs HTTPS The Hypertext Transfer Protocol (HTTP) is a networking protocol for distributed, collaborative, hypermedia information systems . HTTP functions as a request-response protocol in the client-server computing model. In HTTP, a web browser, for example, acts as a client, while an application running on a computer hosting a web site functions as a server. The client submits an HTTP request message to the server. The server, which stores content, or provides resources, such as HTML files, or performs other functions on behalf of the client, returns a response message to the client. HTTPS is simply HTTP using Secure Sockets Layer (SSL) to encrypt and decrypt data for secure transactions
- 8. What are the Terminologies I should know? Request Vs Response Client Request GET /index.html HTTP/1.1 Host: www.example.com Server Response HTTP/1.1 200 OK Date: Mon, 23 May 2005 22:38:34 GMT Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux) Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT Etag: "3f80f-1b6-3e1cb03b" Accept-Ranges: bytes Content-Length: 438 Connection: close Content-Type: text/html; charset=UTF-8
- 9. What are the Terminologies I should know? GET and POST Methods GET is the simplest HTTP method, and its main job in life is to ask the server to get a resource and send it back. That resource might be an HTML page, a JPEG, a PDF, etc. Doesn't matter. The point of GET is to get something back from the server. POST is a more powerful request. It's like a GET++. With POST, you can request something and at the same time send form data to the server GET and POST basically allow information to be sent back to the webserver from a browser (or other HTTP client for that matter). Imagine that you have a form on a HTML page and clicking the "submit" button sends the data in the form back to the server, as "name=value" pairs.
- 10. What are the Terminologies I should know? GET and POST Methods Choosing GET as the "method" will append all of the data to the URL and it will show up in the URL bar of your browser. The amount of information you can send back using a GET is restricted as URLs can only be 1024 characters. http://www.google.co.uk/#hl=en&source=hp&q= &oq= &aq=f&aqi= g10&aql=&gs_sm=e&gs_upl=1296l6140l0l6328l33l25l5l6l6l0l2 97l2062l2- 8l8l0&bav=on.2,or.r_gc.r_pw.&fp=950be6bfd6a5c219&biw=10 24&bih=563 A POST on the other hand will (typically) send the information through a socket back to the webserver and it won't show up in the URL bar. You can send much more information to the server this way - and it's not restricted to textual data either. It is possible to send files and even binary data such as serialized Java objects!
- 11. What are the Terminologies I should know? GET Method: POST Method: All the name value pairs are submitted as a 1. All the name value pairs are submitted in the query string in URL. Message Body of the request. It's not secured as it is visible in plain text 2. Length of the string (amount of data format in the Location bar of the web browser. submitted) is not restricted. As the data transfers through address bar 3. Post Method is secured because Name- (URL) there are some restrictions in using Value pairs cannot be seen in location bar of space, some characters like ampersand (&) the web browser. etc in the GET method of posting data. We 4. If post method is used and if the page is have to take special care for encoding data if refreshed it would prompt before the request is such special characters are present. resubmitted. Length of the string is restricted. 5. If the service associated with the processing If method is not mentioned in the Form tag, of a form has side effects (for example, this is the default method used. modification of a database or subscription to a If get method is used and if the page is service), the method should be POST. refreshed it would not prompt before the 6. Data is submitted in the form as specified in request is submitted again. enctype attribute of form tag and thus files can One can store the name value pairs as be used in FileUpload input box. bookmark and directly be used while sharing with others - example search results. Data is always submitted in the form of text If the response of the page is always same for the posted query then use GET example database searches
- 12. What are the Terminologies I should know? HTTP Status Codes In HTTP/1.0 and since, the first line of the HTTP response is called the status line and includes a numeric status code (such as "404") and a textual reason phrase (such as "Not Found"). 1xx Informational 2xx Success 3xx Redirection 4xx Client Error 5xx Server Error URL: http://en.wikipedia.org/wiki/List_of_HTTP_status_code s
- 13. What are the Terminologies I should know? Throughput Capability of a product to handle multiple transactions in a give period. Throughput represents the number of requests/business transactions processed by the product in a specified time duration. As the number of concurrent users increase, the throughput increases almost linearly with the number of requests. As there is very little congestion within the Application Server system queues.
- 14. What are the Terminologies I should know? Response Time It is equally important to find out how much time each of the transactions took to complete. Response time is defined as the delay between the point of request and the first response from the product. The response time increases proportionally to the user load.
- 15. What are the Terminologies I should know? Tuning Tuning is the procedure by which product performance is enhanced by setting different values to the parameters of the product, operating system and other components. Tuning improves the product performance without having to touch the source code of the product.
- 16. What are the Terminologies I should know? Tuning It is a process of identifying and eliminating bottlenecks Performance tuning of Webbased systems is a multivariable and iterative process It is a collaborative effort between cross functional teams Performance tester is an enabler since he provides the necessary information in an efficient manner
- 17. What are the Terminologies I should know? Benchmarking A very well-improved performance of a product makes no business sense if that performance does not match up to the competitive products. A careful analysis is needed to chalk out the list of transactions to be compared across products so that an apple-apple comparison becomes possible.
- 18. What are the Terminologies I should know? Transaction Mix CRUD Operations Mission Critical Transactions High Frequency Transactions Hack Search Buy
- 19. What are the Terminologies I should know? User Mix User level and their roles No. of users accessing the application Normal Admin Power
- 20. What are the Terminologies I should know? User Think Time Time that user think to perform any event How long do user think while navigating from one page to another? fixed 5 sec random
- 21. What are the Terminologies I should know? Ramp-up Pattern How many user should login to application at a time? User Load Time
- 22. Defining Performance Testing Again The testing to evaluate the response time (speed), throughput and utilization of system to execute its required functions in comparison with different versions of the same product or a different competitive product is called Performance Testing. Performance testing is done to derive benchmark numbers for the system. Heavy load is not applied to the system Tuning is performed until the System Under Test (SUT) achieves the expected levels of performance.
- 23. Why Performance Testing? The week of Feb 6, 2000: Hackers delivered over 1-Billion transactions concurrently to each one of these sites. Yahoo eBay Buy.com Amazon eTrade How did you think they performed ?
- 24. Why Performance Testing? Is your application scalable ? How reliable is it? How fast is it? How secure is it? What are other performance bottlenecks?
- 25. Why Performance Testing? “Only conducting Performance testing at the conclusion of system or functional testing is like ordering a diagnostic blood test after the patient is dead.” Scott Barber
- 26. Why Performance Testing? Identifies problems early on before they become costly to resolve. Reduces development cycles. Produces better quality, more scalable code. Prevents revenue and credibility loss due to poor Web site performance. Enables intelligent planning for future expansion. To ensure that the system meets performance expectations such as response time, throughput etc. under given levels of load. Expose bugs that do not surface in cursory testing, such as memory management bugs, memory leaks, buffer overflows, etc.
- 27. What are the Performance Bottlenecks?
- 28. Why do we have bottlenecks ? People Reactive than proactive, Not Thinking About “Load” or “Scalability” or “Performance” During Des & Dev Limited Expertise in System Design / Development Process Poor Architecture and System Design Changing requirements Changing project plans Technology Inadequate Hardware Inadequate Network Capacity Poor Implementation / Tuning of Databases and components
- 29. What are the contributors?
- 30. What are the types of Performance Testing? Load Vs Stress Testing Goal based Performance Testing Reactive and Proactive Performance Engineering Performance Validation, Performance Testing and Performance Engineering/Tuning Other Types - Soak Testing / Endurance Testing, Spike Testing, Scalability Vs Stability Testing, Application Speed Vs Application Scalability, Scaling Up Vs Scaling Out
- 31. What are the types of Performance Testing? Load Vs Stress Testing
- 32. What are the types of Performance Testing? Load Testing Process of exercising the system under test by feeding it the largest tasks it can operate with. Constantly increasing the load on the system via automated tools to simulate real time scenario with virtual users. Examples: Testing a word processor by editing a very large document. For Web Application load is defined in terms of concurrent users or HTTP connections.
- 33. What are the types of Performance Testing? Stress Testing Trying to break the system under test by overwhelming its resources or by taking resources away from it. Purpose is to make sure that the system fails and recovers gracefully. Example: Double the baseline number for concurrent users/HTTP connections. Randomly shut down and restart ports on the network switches/routers that connects servers.
- 34. What are the types of Performance Testing? Goal based Performance Testing Reactive and Proactive Performance Engineering
- 35. What are the types of Performance Testing? Performance Validation, Performance Testing and Performance Engineering/Tuning Web Server (s) App Server (s) DB
- 36. What are the types of Performance Testing?
- 37. When is it required? Design Phase: Pages containing lots of images and multimedia for reasonable wait times. Heavy loads are less important than knowing which types of content cause slowdowns. Development Phase: To check results of individual pages and processes, looking for breaking points, unnecessary code and bottlenecks. Deployment Phase: To identify the minimum hardware and software requirements for the application.
- 38. What should be tested? High frequency transactions: The most frequently used transactions have the potential to impact the performance of all of the other transactions if they are not efficient. Mission Critical transactions: The more important transactions that facilitate the core objectives of the system should be included, as failure under load of these transactions has, by definition, the greatest impact. Read Transactions: At least one READ ONLY transaction should be included, so that performance of such transactions can be differentiated from other more complex transactions. Update Transactions: At least one update transaction should be included so that performance of such transactions can be differentiated from other transactions.
- 39. What should be my Performance Testing Process?
- 40. What should be my Performance Testing Process? 1.Planning Determine the performance testing objectives Describe the application to test using a application model 1. Describe the Hardware environment 2. Create a Benchmark (Agenda) to be recorded in Phase 2. A. Define what tasks each user will perform B. Define (or estimate) the percentage of users per task.
- 41. What should be my Performance Testing Process? 1.Planning 2.Record 3.Modify Record Record the defined testing activities that will be used as a foundation for your load test scripts. One activity per task or multiple activities depending on user task definition Modify Modify load test scripts defined by recorder to reflect more realistic Load test simulations. Defining the project, users Randomize parameters (Data, times, environment) Randomize user activities that occur during the load test
- 42. What should be my Performance Testing Process? 4. Execute Virtual Users (VUs): Test Goals Start: 5 Max Response Time <= 20 Sec Incremented by: 5 Maximum: 200 Think Time: 5 sec Test Script: One typical user from login through completion.
- 43. What should be my Performance Testing Process? 5.Monitor 6. Analyze Monitoring the scenario: We monitor scenario execution using the various online runtime monitors. Analysing test results: During scenario execution, the tool records the performance of the application under different loads. We use the graphs and reports to analyse the application’s performance.
- 44. Why should I use Tool ? Manual Testing Limitation Do you have the testing resources? • Testing personnel All of you, • Client machines click the GO button again How do you coordinate and synchronize users? How do you collect and analyze results? How do you achieve test repeatability? Coordinator Analysis? Web server Database Testers server Load Generation System Under Test
- 45. Why should I use Tool ? Manual Testing Limitations Expensive, requiring large amounts of both personnel and machinery. Complicated, especially co-ordinating and synchronising multiple testers Involves a high degree of organization, especially to record and analyse results meaningfully Repeatability of the manual tests is limited
- 46. Why should I use Tool ? Benefit of Tool Solves the resource limitations • Replaces testers with virtual users Analysis • Runs many Vusers on a few machines • Controller manages the virtual users Controller • Analyze results with graphs and reports Vuser Web server Database server host Load Generation System Under Test
- 47. Why should I use Tool ? Using Automated Tools Reduces personnel requirements by replacing human users with virtual users or Vusers. These Vusers emulate the behaviour of real users Because numerous Vusers can run on a single computer, the tool reduces the amount of hardware required for testing. Monitors the application performance online, enabling you to fine-tune your system during test execution. It automatically records the performance of the application during a test. You can choose from a wide variety of graphs and reports to view the performance data. Because the tests are fully automated, you can easily repeat them as often as you need.
- 48. What should be my tools combination?
- 49. What tools are available and what should I use for Performance Testing? Vendor Tool Test Suite - Companion Tools Compuware QALoad QACenter Enterprise Edition+ Empirix e-Load e-TEST suite Rational Performance Rational Suite IBM Tester Mercury LoadRunner Quality Center RadView WebLOAD TestView Suite Apache Apache JMeter™ ? Borland SilkPerformer SilkCentral Test Manager (Segue)
- 50. What are the Performance Metrics? Client-side V/s Server-side Performance Metrics Server-side Metrics – Application level and OS level Client side – Response Time, Pass Fail Statistics, Throughput, Hits/sec, Customized Functionality Validation Counters, User Abandonment Server-side – OS – CPU, Memory, Network and Disk I/O Server-side – Performance related metrics and observations for Web Server (s), Application Server (s) and/or Database Server (s)
- 51. What should be my best approach? How do we find and fix the problems? The framework? PE FRAMEWORK: Performance engineering is the process by which software is reviewed, tested and tuned with the intent of realizing the required performance. This process aims to optimize the most important application performance trait, user experience.
- 52. What should be my best approach?
- 53. Let’s Re-cap What is Performance Testing? What are the Terminologies I should know? Why Performance Testing? What are the Performance Bottlenecks? Why do we have bottlenecks ? What are the contributors? What are the types of Performance Testing? When is it required? What should be tested? What should be my Performance Testing Process? Why should I use Tool ? What should be my tools combination? What tools are available and what should I use for Performance Testing? What are the Performance Metrics? What should be my best approach?
- 54. THANK YOU Rohan Bhattarai