What is Test Matrix?
1 like5,358 views
The document outlines the importance of software testing and the role of metrics in assessing the quality and effectiveness of testing processes. It describes various types of metrics, including base and calculated metrics, which provide insights into project status and testing efficiency, such as defect density and test effectiveness. Additionally, it emphasizes the need for test execution reports to determine release readiness and overall product health.
What is Test Matrix?
- 1. TEST METRICS
- 2. Agenda ● Why Test Software ● Why Do We Need Metrics ● Types of Metrics ● OS/ Browser compatibility Matrix ● DRE(Defect Resolution Effectiveness) ● Test Execution Burndown Chart ● Test Case Defect Density/ Test Case Efficiency or Test Efficiency ● Test Effectiveness ● TER Copyright © by QA InfoTech. All rights reserved.
- 3. Why Test Software? Any software product, no matter how accomplished by today’s testing technologies, has bugs. Software testing is the process of detecting and removing the bugs from the software product. The goal of software testing is to ensure software product quality according to the customer requirements. Copyright © by QA InfoTech. All rights reserved.
- 4. Some typical software testing questions Copyright © by QA InfoTech. All rights reserved. ● How long will it take to test the software? ● How much will it cost to test? ● What kind of bugs were found? ● How many of the bugs that were found were fixed? ● Will the product be ready on time? ● How much is the risk of release? --Metrics are ways of answering these questions. --There are numerous metrics for software testing.
- 5. Why Do We Need Metrics? TEST METRICS HELPS US: ● Take decisions for the next phase of activities. ● As an evidence on the basis of which we can predict. ● Understand the type of improvement required. ● Take decisions on the change in process or technology. Copyright © by QA InfoTech. All rights reserved.
- 6. Type of metrics Base Metrics (Direct Measure): Base metrics constitute the raw data gathered by a Test Analyst. These metrics are used to provide project status reports to the Test Lead and Project Manager, they also act as an input to the formulas used to derive Calculated Metrics. Eg: # of Test Cases, # of Test Cases Executed Calculated Metrics (Indirect Measure): Calculated Metrics convert the Base Metrics data into more useful information. These types of metrics are generally the responsibility of the Test Lead and can be tracked at many different levels (by module, tester, or project). Eg: % Complete, % Test Coverage Copyright © by QA InfoTech. All rights reserved.
- 7. Test Metrics Test metrics are classified into two categories: ● Product Metrics: Metrics that are related to test results or quality of software under test ● Process Metrics: Metrics that used for evaluation of testing process effectiveness Copyright © by QA InfoTech. All rights reserved.
- 8. OS/Browser Compatibility Matrix: Copyright © by QA InfoTech. All rights reserved.
- 9. DRE(Defect Resolution Effectiveness) Formula to Calculate DRE : Total Valid Defects Resolved -------------------------------------- *100 Total Valid Defects Logged Copyright © by QA InfoTech. All rights reserved.
- 10. OVERALL DRE FOR THE PRODUCT: (CALCULATED BASED ON PRIORITY): Copyright © by QA InfoTech. All rights reserved.
- 11. Test Execution Burndown Chart Copyright © by QA InfoTech. All rights reserved.
- 12. Test Case Defect Density/Test Case Efficiency or Test Efficiency Total number of errors found in test scripts v/s developed and executed. ● (Defective Test Scripts/Total Test Scripts) * 100 Example: Total test scripts developed are 1360, total test scripts executed are 1280, total test scripts passed are 1065, total test scripts failed are 215 So, test case defect density is 215 ------------------------- X 100 = 16.8% 1280 This 16.8% value can also be called as test case efficiency %, which depends on the total number of test cases which uncovered the defects Copyright © by QA InfoTech. All rights reserved.
- 13. Test Effectiveness “Test” Effectiveness ● Number of defects found during “testing” activities in any phase of the SDLC ● Total number of defects found including the ones in production (used a “warranty period”) Test effectiveness is calculated as: # of defects found during testing phase ----------------------------------------------------- total # of defects found (Testing phase + Post release or customer reported defects) Copyright © by QA InfoTech. All rights reserved.
- 14. Test Execution Report (TER) ● Managerial Summary about the health of the product at the end of each testing cycle ● Helps in determining Release Readiness ● Prepared at the end of each testing cycle (Iteration, Sprint) TER Contents: ● Scope of testing ● Environment details ● Coverage details ● Features ● Test Cases ● Pass Rate ● Results ● List of open defects Copyright © by QA InfoTech. All rights reserved.
- 15. ● Defect Resolution Effectiveness ● QA is ➔ Ready to release ➔ Ready to release with recommendations ➔ Not ready to release Copyright © by QA InfoTech. All rights reserved.