Software testing

Software Testing Strategies
based on

Chapter 13

1

Software Testing

Testing is the process of exercising a program with the
specific intent of finding errors prior to delivery
o the end user.
errors
requirements conformance
performance
an indication
of quality
2

Who Tests the Software?

developer
Understands the system
but, will test "gently"
and, is driven by "delivery "

3

independent tester
Must learn about the system,
but, will attempt to break it
and, is driven by quality

Software Testing
Goals
Types of tests
Levels of tests
Test measures
Test plan

Goals
Verification: Have we built the software right?
Bug-free, meets specs

Validation: Have we built the right software?
Meets customers’ needs

Are Verification and Validation different or variation of the
same idea
My argument: “meeting specs” should equal “meeting
customer needs”...
not generally true (my kitchen, satellite systems)

Software Testing
Goals
Types of tests
Levels of tests
Test measures
Test plan

most of this discussion focuses
on verification
(more specifically bug testing)

Types of tests
Black box
White box

Black box tests
input

output
interfa
ce

1. Does it perform the specified
functions?
2.Does it handle obvious errors in
input?
3.Ariane5 – lousy error handling
4.Classic ints vs floats, yards vs
meters
Black box should catch these if
there is adequate “test coverage”

Example: ordered list of ints
L=create()
L.insert(5)
L.insert(-1)
L.insert(-1)
p=L.getFirst()
print (p)
L.delete(p)
p=L.getFirst()
print(p)
p=L.getNext(p)
print(p)
p=L.getNext(p)

class OrdInts
create
getFirst
getNext
insert
delete
print

-1
-1
5
error

Black box tests
Advantage: black box tester≠developer is unbiased by

implementation details. e.g., Use Case testing, just work through
all the Use Cases
Disadvantage: black box tester is uninformed about
implementation details
unnecessary tests – test same thing in different way
insufficient tests – can miss the extremes, especially if actual use

follows a different pattern

black box tests
Code

Input

x
x
x

x

choose good distribution of input – hope good distribution of code tested

unnecessary tests
Input

Code

large range of input may exercise a small part of code
e.g., operator test of satellite control stations, run through
each
input and output light/key option. Testing same functions,

insufficient tests
Input

Code

a small range of input may exercise a large range of code
but can you ‘know’ this without knowing the code? Did we miss
the 20%

sufficient tests
Input

Code

complex
code

a small range of input may exercise a small but important/error-prone region

White box tests
Based on code
test 1

test 2

class ordInts {
public: …

private:
int vals[1000];
int maxElements=1000;
…
}

bool testMax()
{
L=create();
num=maxElements;
for (int i=0; i<=num; i++)
print i
L.insert(i)
print maxElements;
}

White box tests
Advantage:
design tests to achieve good code coverage and avoid duplication
can stress complicated, error-prone code
can stress boundary values (fault injection)

Disadvantage:
tester=developer may have bias
if code changes, tests may have to be redesigned (is this bad?)

L=create()
L.insert(1)
L.insert(2)
L.insert(3)

class OrdInts
create
getFirst

1
2
3

insert
delete
print

…

L.insert(1001)
p=L.getFirst()
print(p)
p=L.getNext(p)
print p

…

…

getNext

Types of tests

Black box: test based on interface, through interface
White box: test based on code, through code

Testing strategy can/should include all
approaches!
My experience:
Black Box = non developer, outside testing idiots
in your case who?
White Box = developer, part of development process
in your case who?
Gray Box = hated, non developer within developer
organization
in your case who?

Software Testing
White-Box Testing

Black-Box Testing
requirements

output

.
.. our goal is to ensure that all
statements and conditions have
een executed at least once ...
21

input

events

White-Box Testing

Basis Path
Testingwe compute the cyclomatic
First,
complexity:
number of simple decisions + 1
or
number of enclosed areas + 1
In this case, V(G) = 4

22

White-Box Testing

Basis Path
Next, we derive the
Testing independent paths:

1

Since V(G) = 4,
there are four paths

2

3
4
5

7

8

6

Path 1:
Path 2:
Path 3:
Path 4:

1,2,4,7,8
1,2,3,5,7,8
1,2,3,6,7,8
1,2,4,7,2,4,...7,8

Finally, we derive test
cases to exercise these
paths.

A number of industry studies have indicated
that the higher V(G), the higher the probability
23
or errors.

White-Box Testing

Loop Testing

Simple
loop
Nested
Loops
Concatenated
Unstructured
Loops
Loops
24

Why is loop testing important?

White-Box Testing
Equivalence Partitioning

Black-Box
& Boundary Testing

Value Analysis




25

If x = 5 then …

If x > -5 and x < 5 then …

What would be the equivalence classes?

Black-Box Testing

Comparison Testing
Used only in situations in which the reliability of software is

absolutely critical (e.g., human-rated systems)
Separate software engineering teams develop independent

versions of an application using the same specification
 Each version can be tested with the same test data to ensure
that all provide identical output
Then all versions are executed in parallel with real-time
comparison of results to ensure consistency

26

Levels of tests
Unit

white

Integration
System
System integration

black

Testing measures (white box)
Code coverage – individual modules
Path coverage – sequence diagrams
Code coverage based on complexity – test of the risks, tricky

part of code (e.g., Unix “you are not expected to understand
this” code)

Levels of Testing
 Unit testing
 Integration testing
 Validation testing
 Focus is on software requirements
 System testing
 Focus is on system integration
 Alpha/Beta testing
 Focus is on customer usage
 Recovery testing
 forces the software to fail in a variety of ways and verifies that recovery is properly performed
 Security testing
 verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration
 Stress testing
 executes a system in a manner that demands resources in abnormal quantity, frequency, or volume
 Performance Testing
 test the run-time performance of software within the context of an integrated system

29

Unit Testing
Tests the smallest individually executable code units.
Usually done by programmers. Test cases might be
selected based on code, specification, intuition, etc.
Tools:
Test driver/harness
Code coverage analyzer
Automatic test case generator

Integration Testing
Tests interactions between two or more units or
components. Usually done by programmers.
Emphasizes interfaces.
Issues:
In what order are units combined?
How do you assure the compatibility and correctness of
externally-supplied components?

Integration Testing
How are units integrated? What are the implications of
this order?
Top-down => need stubs; top-level tested repeatedly.
Bottom-up => need drivers; bottom-levels tested repeatedly.
Critical units first => stubs & drivers needed; critical units tested

repeatedly.

Integration Testing
Potential Problems:
Inadequate unit testing.
Inadequate planning & organization for integration testing.
Inadequate documentation and testing of externally-supplied
components.

Stages of Testing
Testing in the Large
 System Testing
 End-to-End Testing
 Operations Readiness Testing
 Beta Testing
 Load Testing
 Stress Testing
 Performance Testing
 Reliability Testing
 Regression Testing

System Testing

Test the functionality of the entire system.
Usually done by professional testers.

Realities of System Testing
Not all problems will be found no matter how thorough or

systematic the testing.
Testing resources (staff, time, tools, labs) are limited.
Specifications are frequently unclear/ambiguous and changing
(and not necessarily complete and up-to-date).
Systems are almost always too large to permit test cases to be
selected based on code characteristics.

Software testing

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to Software testing (20)

More from Bala Ganesh (20)

Recently uploaded (20)

Software testing

Editor's Notes