Performance & Automation Testing
Download as PPTX, PDF4 likes947 views
Indium Software is an independent firm specializing in performance and automation testing with over 50 engagements in the past decade. They offer a range of services including load testing, scalability testing, and performance benchmarking, utilizing both commercial and open-source tools. Their approach focuses on providing unbiased analysis and maximizing ROI through innovative engagements and flexible resource models.
Performance & Automation Testing
- 1. INDIUM SOFTWARE An Independent Software Testing Firm Indium Software Performance & Automation Testing Services “Business Assurance through Quality Assurance”
- 2. © 2017 Indium Software Performance Testing Service Overview 2 Focused Performance Testing practice – over past 10 years Over 50 Performance test engagements executed. Expertise in HP Mercury, Microfocus, IBM Rational & Open Source tools. Strong focus on Independent, Objective and unbiased analysis. Performance Test Centre Services: • Load / Stress / Volume Testing • Fail-Over testing • Scalability Testing • Performance Benchmark/Baseline Testing • Performance Test Tool and Process Consulting Differentiators – Simple, Flexible and Scalable • Process (with toll gate rigor) focus • Transaction based pricing (per script model) • High ROI on QA Investments • Innovative engagement models • Just in time resourcing model – core and flexible teams. Validate Architecture | Validate Response times | SLAs | Validate Scalability
- 3. © 2017 Indium Software3 Proficiency on Tools • Commercial - HP Loadrunner, MS VSTS, OATS, MF-Silk Performer, IBM’s RPT • Open Source – Jmeter, Badboy, OpenSTA, Grinder Key Deliverables • Overall Performance Roadmap • Test Plan for all major Releases • Assess high risk aspects of design w.r.t Technology • Comprehensive Test Summary with Observations , Recommendations and Extrapolations Focus Areas - • Application Benchmarking & Predictive Usage Pattern • WAN simulation Using Cloud based load testing • Infrastructure Upgrade – Validation • Mobile application based Performance Testing • Maturity based Performance Engineering Models Value Additions • Load Balancer / Memory Bottleneck • Processor Bottleneck in DB & App. Server • Disc I/O constraints / Network Latency • Server Configuration / Network Bandwidth • DB Deadlock / SQL response time / Indexing issues • JVM / JDBC Connections Constraints • Timeout / TCP Connection stability Performance Test Approach | Overview
- 4. © 2017 Indium Software SMART Performance Test Strategy Prioritization Matrix for Business Scenarios # of Concurrent Users Frequency of transaction(s) usage Customer Facing vs Financial Impact Define High level scenarios Business Processes • Sub-Processes • Transactions or Process Steps Define Test Data Volume of Test Data needed (Static vs Volatile) Mining of historical data for quick test data preparation Performance Test Environment VPN connectivity – Configure Static IP address (Offshore) within client firewall Application access – Test ID creation, user privileges Tools – Depend on the feasibility Readiness to Start People – Performance Testers with multiple test tool experience Process – Performance Testing process, methodology, scripting, execution, Analysis & Reporting Tools – Templates, Jump-Start-Kits 4 SMART stands for Script, Model, Analyse, Report and Tune
- 5. © 2017 Indium Software Performance Test Schema 5 PC 1 Application / Production Server PC 3 PC 4 Performance Test Lab: To conduct load test, monitor performance, analyze and report, remotely. Load Generators: For hosting Controller & Load Generators for load generation via LAN/VPN onto Application Server. These could be at same location or at different geographies. PC 2 VPN/LAN Virtual Users Load PC Offshore (Indium) PC CUSTOMER Infrastructure
- 6. © 2017 Indium Software Performance Test Metrics 6 • OS Resource Utilizations CPU Usage Memory Usage Disk IO Utilization • Network Utilization Bandwidth Latency Traces • Servers Utilization (Web /App) Heap Usage & GC JVM JDBC Threads Application Logs • SLA compliance • Certify the build for production deployment • Trend analysis • SLA compliance Metrics •Test Tool Results •Monitors Data •Diagnostics tool data •Data from Support Teams Analysis Report •Execution & Monitors data analysis •Log analysis •Diagnostics report •Bottlenecks and Root causes Execution Report •Response times •Server resource usage statistics •Defect tracking and comparison Test Summary Report (TSR) •Consolidated report of al the tests conducted •Recommendations
- 7. © 2017 Indium Software7 Recommendations Usage distribution – Response time Server utilization Performance Test metrics Deep Dive Performance Dashboard Artefacts (Samples) – Performance Tests
- 8. © 2017 Indium Software Test Automation Portfolio 8 • Functional Test Automation, Test Data Generation, Testing Tools Administration, Regression automation • Multi Browser / Multi OS • Multi Device – PC / Mobile / Tablets • Usage of reusable functions to ease coding • Reusable Automation Frameworks with proven track record in effort reduction • Process , checklists and templates to act as jumpstart kit • Web Technologies - Java, .NET, LAMP • Enterprise Apps – SAP, Oracle, Peoplesoft, Salesforce.com • BI / DW, Big Data • Mobile, Social Media, Cloud, Analytics Services Jump Start Technologies
- 9. © 2017 Indium Software Tools • Commercial - HP QTP, IBM Rational Functional Tester, Silk Test, TestComplete, Ranorex • Open Source - Selenium RC, SOAPUI, Watir Key Deliverables • Test Strategy / Plan • Test Automation Framework • Automation Suite • Test Automation Execution Summary Focus Areas – <<Customer Name>> : • Define the process for the automation suite management and maintenance Lorem Ipsum • Implement Test Automation Framework and develop Test Automation Suites for Smoke , Sanity & Regression • Automation suite should ensure that , the application build is stable across all major / minor releases which could include defect fixes or new feature enhancements and extend good test coverage Value Additions • Reporting alerts over email / SMS • Cloud ready • Interface with multiple tools • Hybrid automation framework 9 Test Automation Approach | Overview
- 10. © 2017 Indium Software Indium’s Selenium Automation Framework Enhanced (iSAFE) Framework 10 Framework Layer MobileApplicationUnderTest Automation Suite Input Data Driver Script Object Repository Config and Global Variables Reporting Engine Scheduler Start up/Clean Up Properties Execution
- 11. © 2017 Indium Software iSAFE Capabilities 11 iSAFE Capabilities Test Reporting in multiple formats with historical graphs Ability to work with multiple OS / Browsers / Devices Ease of Setup and Execution resulting in minimum learning Fully Configurable Tests using Spreadsheets Support for API and DB Testing Email/SMS Text Status/Voice calls to Stakeholders Target based & parallel execution using Selenium Grid Exception Handling and Screen Shot Management