Software Testing Foundations Part 5 - White Box Testing
1 like1,203 views
This document discusses white box testing techniques. It defines white box testing as testing based on the internal structure of the code. It then describes various white box techniques including statement coverage, branch coverage, branch condition testing, branch condition combination testing, condition determination testing, and path coverage. It notes that the goal is to execute all code and ensure all decisions are tested with all possible outcomes. Tool support is important to help automate white box testing given its complexity.
Software Testing Foundations Part 5 - White Box Testing
- 1. Software Testing Foundations #5 Dynamic Analysis. White Box Testing. Nikita Knysh [email protected] http://www.facebook.com/groups/istqb/
- 2. Agenda • White Box Testing Overview • White Box Testing Techniques ▫ Statement Coverage ▫ Branch Coverage ▫ Branch Condition Testing ▫ Branch Condition Combination Testing ▫ Condition Determination Testing ▫ Path Coverage • More about White Box Testing
- 3. White Box Testing Overview #1 • Aka Code-based testing, aka structural testing • The source code is known and used for test design (is the basis for white box techniques). • Internal processing of the test object as well as the output is analyzed – Point of Observation (PoO) is inside the test object. Point of Control (PoC) is outside or inside the test object.
- 4. White Box Testing Overview #2 • Test cases are designed to cover the program structure of the test object. • All code should be executed. • In certain cases, it must be possible to manipulate the code, i.e. add code to it. • Expected results should be defined using specifications, NOT the code! • Usually applied in the lower test levels (i.e. component or integration test).
- 5. Technique: Statement Coverage #1 • Test cases should execute a certain quota (or all) statements in the test object.
- 6. Technique: Statement Coverage #2 • If sequences of unconditional statements appear in the program fragment then they are illustrated as one single node because execution of the first statement of the sequence guarantees that all following statements will be executed. • Conditional statements (IF, CASE) and loops (WHILE; FOR) have more than one edge going out from them.
- 7. Technique: Statement Coverage #3 Statement coverage = (number of executed statements / total number of statements) * 100 % • Is also known as C0-coverage. Measured with automated tools. • Value: unreachable code (dead statements) can be detected, empty ELSE-parts are not considered (cannot detect missing statements after ELSE) – not really an advantage.
- 8. Technique: Branch Coverage #1 • Aka decision coverage • Execution of decisions is considered. The following test cases are needed: • a, b, f, g, h, d, e • a, b, c, d, e • a, b, f, g, i, g, h, d, e • a, k, e
- 9. Technique: Branch Coverage #2 • Edges of the graph are the center of attention (all of them should be covered • It is okay to test an edge more than once as it is not always possible to avoid this). • Testing should make sure every decision is executed with both possible outcomes (FALSE and TRUE). • Empty ELSE-parts are considered (every IF should be tested for FALSE, even if it has no ELSE).
- 10. Technique: Branch Coverage #3 Branch coverage = (number of executed branches / total number of branches) * 100 % • Each of the branches is regarded separately and no particular combination of single branches is required. • Also known as C1-coverage. Test cost is higher for higher coverage requirements. • Value: 100 % branch coverage guarantees 100% statement coverage, but not vice versa. Thus, branch coverage is a stronger criterion. • Inadequate for object-oriented systems.
- 11. Technique: Branch Condition Testing • Condition contains logical operators. Complexity of conditions should be taken into account. • The goal is that each atomic (partial) condition shall adopt the values TRUE and FALSE. • This is a weak criterion (weaker than statement or branch coverage). Should be abandoned for complex conditions.
- 12. Technique: Branch Condition Combination Testing #1 (Aka multiple-condition coverage). • Requires that all true-false combinations of atomic conditions be exercised at least once: x=6 (T), y=3 (T), x>3 OR y<5 (T) x=6 (T), y=8 (F), x>3 OR y<5 (T) x=2 (F), y=3 (T), x>3 OR y<5 (T) x=2 (F), y=8 (F), x>3 OR y<5 (F)
- 13. Technique: Branch Condition Combination Testing #2 • The complete condition gives both logical results so this technique meets the criteria of statement and branch coverage. • But it is expensive: number of combinations is 2n with n atomic parts of the condition. • Also, not all combinations can be implemented with test data, for instance for 3<=x and x<5, combination FALSE – FALSE is not possible.
- 14. Technique: Condition Determination Testing #1 • Based on the idea of restriction of combinations. • Combinations to be considered: only those where modification of logical value of an atomic combination can change the logical value of the whole expression. • If it cannot, then failure of an atomic condition will not reveal a defect so the test can be left out.
- 15. Technique: Condition Determination Testing #2 • Example: x=6 (T), y=3 (T), x>3 OR y<5 (T) – can be left out x=6 (T), y=8 (F), x>3 OR y<5 (T) – should be tested x=2 (F), y=3 (T), x>3 OR y<5 (T) – should be tested x=2 (F), y=8 (F), x>3 OR y<5 (F) – should be tested Underlined: Value that can change logical value of entire expression if modified.
- 16. Technique: Condition Determination Testing #3 • The number of combinations lies between n+1 and 2n with n = number of the Boolean operands of the condition, which is considerably smaller that in condition combination test. • Should be used for complex conditions. • Leads to statement and branch coverage, so no need to use them additionally. • It is reasonable to try to achieve 100% coverage as complex conditions are often defect-prone. • However, it may be very expensive to choose input data for test cases. Intensive test of complex conditions can be omitted if they were a subject for a review.
- 17. Technique: Path Coverage #1 • If the test object includes loops or repetitions, previous techniques are not sufficient. • Path coverage requires execution of all different paths through the test object. • It is obvious that 100% path coverage is not feasible in a program as soon as the program is not trivial.
- 18. Technique: Path Coverage #2 6 test cases are possible (3 before edge j multiplied by 2 after edge j) if conditions are independent from each other and edges can be combined freely. • a, b, c, j, k, l, n • a, d, e, f, i, j, k, l, n • a, d, g, h, i, j, k, l, n • a, b, c, j, m, n • a, d, e, f, i, j, m, n • a, d, g, h, i, j, m, n
- 19. Further White Box Testing Techniques • Data flow based techniques primarily verify data usages in the test object: use of variables, read-write access of variables. These techniques may find faults where a value given to a variable in one place leads to failure at another place it is used.
- 20. More on White Box Techniques • Basis for all white box techniques is the source code. • White box techniques are useful at the lower test levels. • Missing implementation of requirements is impossible to find for white box techniques.
- 21. Tool Support • Manual white box testing is too resource consuming and error-prone. • Instrumentation often works this way: The tool inserts counters in the program and initializes them with zero. During program execution, the counters are incremented when they are passed. At the end of the test execution, the counters contain the number of passes through the according program fragments. If a counter stayed zero during the test then the according program fragment has not been executed.
- 22. Thank you! http://www.facebook.com/groups/istqb/