Test Suite Generation for Boolean Conditions with Equivalence Class Partitioning

S. Hallé
Sylvain Hallé
Université du Québec à Chicoutimi
CANADA
Test Suite Generation for
Boolean Conditions with
Equivalence Class Partitioning
CRSNG
NSERC
FormaliSE, May 2022

S. Hallé
Boolean Conditions
Software systems are ﬁlled with conditions that
modulate their behavior.
SELECT name FROM employees
WHERE YEAR(dob) < 1990 AND rank = 3;
if (a % 3 == 0 || (b > c + 6 && o.isReady())
{
...
}
Italic
Bold
+
Underline
OK
Reversed
+
c1
a
(8/1)
=b ≠b
(4/2)
<2 >2
⊤
⊤
=2
c3
(7/2)
=0 ≠0
(3/1)
⊤
⊤
a
≠c =c
(10/3)
⊤
(7/2)
=0 ≠0
(3/1)
⊤
⊤
c2

S. Hallé
Boolean Conditions
Test input generation is the problem of
(automatically) generating values given to a system
if (a % 3 == 0 || (b > c + 6 && o.isReady())
{
...
}

S. Hallé
Boolean Conditions
if (a % 3 == 0 || (b > c + 6 && o.isReady())
{
...
}
a = 3
b = 2
c = 1
o.isReady() = true
test case

S. Hallé
Boolean Conditions
if (a % 3 == 0 || (b > c + 6 && o.isReady())
{
...
}
a = 3
b = 2
c = 1
o.isReady() = true
test case
1
1

S. Hallé
Boolean Conditions
if (a % 3 == 0 || (b > c + 6 && o.isReady())
{
...
}
a = 3
b = 2
c = 1
o.isReady() = true
test case
else
{
...
}
2
a = 1
b = 2
c = 2
o.isReady() = false
2
Diﬀerent values send the
execution on diﬀerent paths
1
1

S. Hallé
...but what if the condition
is incorrect?
Boolean Conditions
if (a % 3 == 0 || (b > c + 6 && o.isReady())
{
...
}
a = 3
b = 2
c = 1
o.isReady() = true
test case
else
{
...
}
2
a = 1
b = 2
c = 2
o.isReady() = false
2
Diﬀerent values send the
execution on diﬀerent paths
1
1

S. Hallé
We must generate a test input for which the
condition sends the execution in the wrong branch
Boolean Conditions
a % 3 == 0 || (b > c + 6 && o.isReady())
a % 3 == 0 || (b ≥ c + 6 && o.isReady())
ACTUAL
EXPECTED

S. Hallé
a = 0
b = 6
c = 0
o.isReady() = true
4
Boolean Conditions
a % 3 == 0 || (b > c + 6 && o.isReady())
a % 3 == 0 || (b ≥ c + 6 && o.isReady())
ACTUAL
EXPECTED

S. Hallé
a = 0
b = 6
c = 0
o.isReady() = true
4
⊤ ⊥ ⊤
⊤ ⊤ ⊤
⊤
⊤
+
Boolean Conditions
a % 3 == 0 || (b > c + 6 && o.isReady())
a % 3 == 0 || (b ≥ c + 6 && o.isReady())
ACTUAL
EXPECTED

S. Hallé
a = 0
b = 6
c = 0
o.isReady() = true
a = 1
b = 7
c = 1
o.isReady() = false
+
4 5
Boolean Conditions
a % 3 == 0 || (b > c + 6 && o.isReady())
a % 3 == 0 || (b ≥ c + 6 && o.isReady())
ACTUAL
EXPECTED

S. Hallé
a = 0
b = 6
c = 0
o.isReady() = true
a = 1
b = 7
c = 1
o.isReady() = false
+
4 5
⊥ ⊥ ⊥
⊥ ⊤ ⊥
⊥
⊥
+
Boolean Conditions
a % 3 == 0 || (b > c + 6 && o.isReady())
a % 3 == 0 || (b ≥ c + 6 && o.isReady())
ACTUAL
EXPECTED

S. Hallé
a = 0
b = 6
c = 0
o.isReady() = true
a = 1
b = 7
c = 1
o.isReady() = false
+
+
a = 1
b = 7
c = 1
o.isReady() = true
4 5 6
Boolean Conditions
a % 3 == 0 || (b > c + 6 && o.isReady())
a % 3 == 0 || (b ≥ c + 6 && o.isReady())
ACTUAL
EXPECTED

S. Hallé
a = 0
b = 6
c = 0
o.isReady() = true
a = 1
b = 7
c = 1
o.isReady() = false
+
+
a = 1
b = 7
c = 1
o.isReady() = true
4 5 6
⊥ ⊥ ⊤
⊥ ⊤ ⊤
⊥
⊤
✓
Boolean Conditions
a % 3 == 0 || (b > c + 6 && o.isReady())
a % 3 == 0 || (b ≥ c + 6 && o.isReady())
ACTUAL
EXPECTED

S. Hallé
Making the condition true/false once is not
suﬀicient to reveal errors
The components of the condition must evaluate to
true/false in various combinations
⇒ Boolean condition coverage
Boolean Conditions
a % 3 == 0 || (b > c + 6 && o.isReady())
a % 3 == 0 || (b ≥ c + 6 && o.isReady())
ACTUAL
EXPECTED

S. Hallé
Several coverage criteria for Boolean conditions
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
Predicate coverage
Each variable has a test case where it is true, and
another test case where it is false

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
Combinatorial ("t-way") coverage
Each t-tuple of variables has test cases for all
combinations of their true/false values
Example: for t=2

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
Clause coverage
Each clause of the condition has a test case where
it is true and another test case where it is false
{
clause

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
Modiﬁed condition/decision coverage (MC/DC)
Predicate coverage + clause coverage + every
clause is shown to independently aﬀect the
outcome

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
outcome
⊥
⊥ ⊥ ⊥
x₀ = ⊥ x₁ = ⊥ x₂ = ⊥ x₃ = ⊥

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
outcome
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
⊥
⊥ ⊤ ⊥
x₀ = ⊥ x₁ = ⊥ x₂ = ⊥ x₃ = ⊥
⊤
x₀ = ⊥ x₁ = ⊤ x₂ = ⊥ x₃ = ⊤

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
MUMCUT Coverage
3 conditions expressed in terms of:

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
MUMCUT Coverage
Unique true point (UTP): test case that makes a
single clause evaluate to ⊤

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
MUMCUT Coverage
single clause evaluate to ⊤ x₀ = ⊥ x₁ = ⊤ x₂ = ⊥ x₃ = ⊤
⊥ ⊤ ⊥

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
MUMCUT Coverage
Near false point (NFP): test case where the
condition is ⊥, but ﬂipping a single variable
changes its value

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
MUMCUT Coverage
Near false point (NFP): test case where the
condition is ⊥, but ﬂipping a single variable
changes its value
⊥ ⊥ ⊥
x₀ = ⊥ x₁ = ⊥ x₂ = ⊥ x₃ = ⊤

S. Hallé
have been proposed
Coverage Criteria
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
MUTP: each variable not in a UTP must be true/
false in at least one test
MNFP: each variable not in a NFP must be true/
CUTP-NFP: all pairs of UTP and NFP for the same
clause that diﬀer by a single variable ﬂip must be
present
MUMCUT Coverage
1
2
3

S. Hallé
Each coverage criterion is expressed informally;
no uniform formal notation to deﬁne them all
Each criterion comes with its own algorithm to
generate test cases; the correctness/optimality of
these algorithms is often not demonstrated
Concrete implementations of these algorithms are
not easy to ﬁnd (research papers); switching
criteria means switching programs
Must restart from scratch if developing a new
coverage criterion
In the current state of things...
Coverage Criteria

S. Hallé
Foundations based on concepts of algebra
Existing criteria become particular cases of this
model
Provides a test generation algorithm that works of
any criterion
Comes with a concrete and freely available
implementation
Goal: deﬁne a formal model of Boolean condition
coverage
Contribution

S. Hallé
Algebraic Definition of Coverage
Let X be a set of arbitrary symbols (variables). A
valuation is a total function ν : X → {⊤,⊥}. We
define as N the set of all valuations.
We denote by φ[ν] the result of evaluating φ by
setting its variables to the values defined by ν.
Let Φ be the set of Boolean formulas with variables
in X. A valuation ν ∈ N is a specific way of
evaluating a condition φ ∈ Φ.
Example: X = {x₀,x₁}
ν(x₀) = ⊤, ν(x₁) = ⊥
φ = x₀ ∧ ¬x₁
φ[ν] = ⊤

S. Hallé
Let τ : N → C be a total function mapping each
valuation ν to an element of a set C. An element c ∈
C is called a category.
Intuitively, τ classiﬁes valuations into categories;
we call it a categorization function. Two valuations
ν and ν' are in the same equivalence class if
τ(ν) = τ(ν'). We note this ν ~ ν'.
The kernel of τ is the partition of N induced by the
quotient N/~. Each subset contains the valuations
that belong to the same category.
algebra!

S. Hallé
Example
X = {x₀,x₁}
There are 4 valuations in N:
ν₀ = {x₀ ↦ ⊤, x₁ ↦ ⊤}
ν₁ = {x₀ ↦ ⊤, x₁ ↦ ⊥}
ν₂ = {x₀ ↦ ⊥, x₁ ↦ ⊤}
ν₃ = {x₀ ↦ ⊥, x₁ ↦ ⊥}

S. Hallé
Deﬁne τ : N → {0,1} as
τ(ν) ={0 if ν(x₀) = ⊤
1 otherwise
Example
X = {x₀,x₁}
ν₀ = {x₀ ↦ ⊤, x₁ ↦ ⊤}
ν₁ = {x₀ ↦ ⊤, x₁ ↦ ⊥}
ν₂ = {x₀ ↦ ⊥, x₁ ↦ ⊤}
ν₃ = {x₀ ↦ ⊥, x₁ ↦ ⊥}

S. Hallé
τ(ν) ={0 if ν(x₀) ≠ ν(x₁)
1 otherwise
Example
X = {x₀,x₁}
ν₀ = {x₀ ↦ ⊤, x₁ ↦ ⊤}
ν₁ = {x₀ ↦ ⊤, x₁ ↦ ⊥}
ν₂ = {x₀ ↦ ⊥, x₁ ↦ ⊤}
ν₃ = {x₀ ↦ ⊥, x₁ ↦ ⊥}

S. Hallé
τ(ν) = the number of true variables in ν
2
1
0
Example
X = {x₀,x₁}
ν₀ = {x₀ ↦ ⊤, x₁ ↦ ⊤}
ν₁ = {x₀ ↦ ⊤, x₁ ↦ ⊥}
ν₂ = {x₀ ↦ ⊥, x₁ ↦ ⊤}
ν₃ = {x₀ ↦ ⊥, x₁ ↦ ⊥}

S. Hallé
ECP: methodology where test cases are divided in
partitions

S. Hallé
Set of all possible
test cases
partitions

S. Hallé
Set of all possible
test cases
Equivalence classes
partitions

S. Hallé
Set of all possible
test cases
Equivalence classes
partitions
Chosen test cases

S. Hallé
partitions
Set of all possible
valuations (N)
Equivalence classes induced by τ
Representatives

S. Hallé
A test suite is a set of valuations (i.e. an element
V ∈ 2 ).
N
Deﬁne 𝜏(V) = ⋃
𝜈 ∈ V
{𝜏(𝜈)}
"the set of categories
present in the test suite"
A Boolean condition coverage criterion is called
algebraic if...
for every Boolean formula 𝜑,
there exists a function 𝜏,
such that a test suite V satisﬁes the criterion
if and only if 𝜏(V) = 𝜏(N). "all categories are
represented in V"

S. Hallé
𝜌(V) ∈ [0,1], and full coverage means 𝜌(V) = 1.
If a test suite V achieves partial coverage, it can
easily be quantiﬁed:
𝜌(V) =
𝜏(V)
𝜏(N)
coverage ratio of V fraction of all
categories that are
present in V
Note that we did not need to specify what speciﬁc
criterion we are talking about!

S. Hallé
Evaluation Trees
∧
∨ ¬
a ¬ c
b
∨
¬ b
c
φ = (a ∨ ¬b ∨ c) ∧ ¬(¬c ∨ b)
A valuation ν applied on a formula φ induces a
structure called an evaluation tree.

S. Hallé
ν = {a ↦ ⊤, b ↦ ⊥, c ↦ ⊤}
Each valuation "colors" the
structure diﬀerently
Each combination of ν and φ
produces a unique tree
Deﬁne as eν(φ) the function that produces
the evaluation tree of φ[ν].
Evaluation Trees
∧
∨ ¬
a ¬ c
b
∨
¬ b
c
φ = (a ∨ ¬b ∨ c) ∧ ¬(¬c ∨ b)
A valuation ν applied on a formula φ induces a
structure called an evaluation tree.

S. Hallé
Tree Transformation
Let T be the set of trees. We deﬁne T* as the set of
trees where nodes can be labeled with .
A tree transformation is a function 𝜏 : T* → T* that
turns an evaluation tree into another one.
Since trees are obtained by evaluating formulas, we
let 𝜏ν : Φ → T* be the function such that
?
^
𝜏ν(φ) = 𝜏(eν(φ))
^
i.e. for a valuation ν, 𝜏ν gets the evaluation tree for
φ and applies the transformation 𝜏 to it.
^

S. Hallé
Tree Transformation
We are free to deﬁne 𝜏 as we want.
∨
∧ ∧ ∧
a ¬ c
b
¬ ¬
a b
¬ b ¬
a c

S. Hallé
Tree Transformation
∨
∧ ∧ ∧
a ¬ c
b
¬ ¬
a b
¬ b ¬
a c
∨
∧
Keep only the root and its immediate
children.
∧ ∧

S. Hallé
Tree Transformation
∨
∧ ∧ ∧
a ¬ c
b
¬ ¬
a b
¬ b ¬
a c
∨
? ∧ ?
Turn every child of the root, except
the second, into , and trim the
descendants of the second node.
?

S. Hallé
Tree Transformation
∨
∧ ∧ ∧
a ¬ c
b
¬ ¬
a b
¬ b ¬
a c
Put under the root a leaf labelled a
and a leaf labelled b, in that order.
∨
a b

S. Hallé
Tree Transformation
∨
∧ ∧ ∧
a ¬ c
b
¬ ¬
a b
¬ b ¬
a c
Under every connective node,
keep only subtrees until the
ﬁrst that determines its color
∨
∧ ∧ ∧
a ¬ c
b
¬ ¬
a b
¬ b ¬
a c

S. Hallé
Tree Transformation
For a given formula φ, some tree transformations
map more than one valuation to the same tree.
∧
∨ ¬
a ¬ c
b
∨
¬ b
c
φ
𝜏 = keep only the root and its immediate children
= (a ∨ ¬b ∨ c) ∧ ¬(¬c ∨ b)

S. Hallé
Tree Transformation
∧
∨ ¬
a ¬ c
b
∨
¬ b
c
φ
= (a ∨ ¬b ∨ c) ∧ ¬(¬c ∨ b)
∧
∨ ¬
{a ↦ ⊥, b ↦ ⊤, c ↦ ⊥}
^
𝜏

S. Hallé
Tree Transformation
∧
∨ ¬
a ¬ c
b
∨
¬ b
c
φ
= (a ∨ ¬b ∨ c) ∧ ¬(¬c ∨ b)
∧
∨ ¬
{a ↦ ⊥, b ↦ ⊤, c ↦ ⊥}
^
𝜏
{a ↦ ⊥, b ↦ ⊥, c ↦ ⊤}
{a ↦ ⊤, b ↦ ⊥, c ↦ ⊤}
∧
∨ ¬
^
𝜏

S. Hallé
Tree Transformation
∧
∨ ¬
a ¬ c
b
∨
¬ b
c
φ
= (a ∨ ¬b ∨ c) ∧ ¬(¬c ∨ b)
∧
∨ ¬
{a ↦ ⊥, b ↦ ⊤, c ↦ ⊥}
^
𝜏
{a ↦ ⊥, b ↦ ⊥, c ↦ ⊤}
{a ↦ ⊤, b ↦ ⊥, c ↦ ⊤}
∧
∨ ¬
^
𝜏
∧
∨ ¬
^
𝜏
{a ↦ ⊥, b ↦ ⊥, c ↦ ⊥}
{a ↦ ⊥, b ↦ ⊤, c ↦ ⊤}
{a ↦ ⊤, b ↦ ⊥, c ↦ ⊥}
{a ↦ ⊤, b ↦ ⊤, c ↦ ⊥}
{a ↦ ⊤, b ↦ ⊤, c ↦ ⊤}

S. Hallé
Tree Transformation
Key observation: for a given formula, a tree
transformation partitions the set of valuations
according to the tree each is mapped to...
∧
∨ ¬
∧
∨ ¬
{⊥⊥⊤,⊤⊥⊤} ∅
∧
∨ ¬
∧
∨ ¬
{⊥⊥⊥,⊥⊤⊤,
⊤⊥⊥,⊤⊤⊥,⊤⊤⊤}
{⊥⊤⊥}
⇒ it deﬁnes an algebraic coverage criterion!

S. Hallé
1.
Questions
Are the coverage criteria mentioned earlier
algebraic?
2. If so, can we ﬁnd a tree transformation for each
that results in the same partitioning of the test
input space?

S. Hallé
1.
Questions
algebraic?
input space?
YES (except MUMCUT*)
*An over-approximation is shown to be algebraic

S. Hallé
1.
Questions
algebraic?
input space?
YES (except MUMCUT*)
*An over-approximation is shown to be algebraic
YES (all details in the paper)

S. Hallé
Deﬁne 𝜏ₙ as:
An Example
Turn the root and every child of the root, except
the n-th, into , and trim the descendants of the
n-th node.
?

S. Hallé
Deﬁne 𝜏ₙ as:
An Example
n-th node.
?
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
Valuations are classiﬁed by 𝜏ₙ depending on the
value they give to the n-th clause.
n=2
? ∧ ? ? ∧ ?
? ?
?
∧ ? ?
∧ ?
? ?
n=1
? ∧
? ? ∧
?
? ?
n=3

S. Hallé
Deﬁne 𝜏ₙ as:
An Example
n-th node.
?
(x₀ ∧ ¬x₁ ∧ x₂) ∨ (x₁ ∧ x₃) ∨ (¬x₀ ∧ x₂ ∧ x₃)
Valuations are classiﬁed by 𝜏ₙ depending on the
value they give to the n-th clause.
n=2
? ∧ ? ? ∧ ?
? ?
?
∧ ? ?
∧ ?
? ?
n=1
? ∧
? ? ∧
?
? ?
n=3
One valuation for each tree ⇒ clause coverage

S. Hallé
We deﬁne appropriate transformations for all
coverage criteria shown earlier... except MUMCUT.
A Word About MUMCUT
present
MUMCUT Coverage
1
2
3

S. Hallé
✓
algebraic
✓
algebraic
not algebraic? ?
A Word About MUMCUT
present
MUMCUT Coverage
1
2
3

S. Hallé
✓
algebraic
✓
algebraic
not algebraic? ?
CUTP-NFP is deﬁned on pairs of valuations, and not
on individual valuations
A Word About MUMCUT
present
MUMCUT Coverage
1
2
3

S. Hallé
Generating Test Suites
A test suite for any algebraic criterion can be
obtained using the equivalence class partitioning
method. A possible way is using a hypergraph.

S. Hallé
⊥⊥⊤
⊥⊥⊥
⊥⊤⊤
⊤⊥⊥
⊤⊤⊥
⊤⊤⊤
⊥⊤⊥
⊤⊥⊤
1. Create one node for each valuation

S. Hallé
∧
∨
1
∧
∨
2
∧
∨
3 a 4 a 5
3
4
4
4
5
5
5
1
2
2
2
2
2. Add one hyperedge linking
the nodes mapped to
each tree induced by 𝜏
⊥⊥⊤
⊥⊥⊥
⊥⊤⊤
⊤⊥⊥
⊤⊤⊥
⊤⊤⊤
⊥⊤⊥
⊤⊥⊤

S. Hallé
∧
∨
1
∧
∨
2
∧
∨
3 a 4 a 5
3
4
4
4
5
5
5
1
2
2
2
2
the nodes mapped to
⊥⊥⊤
⊥⊥⊥
⊥⊤⊤
⊤⊥⊥
⊤⊤⊥
⊤⊤⊤
⊥⊤⊥
⊤⊥⊤
3. Solve the hypergraph vertex
cover problem

S. Hallé
∧
∨
1
∧
∨
2
∧
∨
3 a 4 a 5
3
4
4
4
5
5
5
1
2
2
2
2
the nodes mapped to
⊥⊥⊤
⊥⊥⊥
⊥⊤⊤
⊤⊥⊥
⊤⊤⊥
⊤⊤⊤
⊥⊤⊥
⊤⊥⊤
3. Solve the hypergraph vertex
cover problem
The selected nodes form the
test suite.

S. Hallé
Cons
The hypergraph has 2|X|
nodes
The vertex cover problem has a high complexity
Pros
Guarantees full coverage for any algebraic
criterion
Optimal hypergraph cover ⇔ optimal test suite
Upper bound on complexity
Can solve for multiple criteria at once (instead of
separately + merging)

S. Hallé
Boolean Coverage Toolkit
These principles have been concretely implemented
in an open source Java library:
https://github.com/liﬂab/boolean-coverage-toolkit
Operator o = Or(And(a, Not(b), c),
And(Not(a), Not(b)), And(Not(a), b, Not(c)));
HologramNode n1 = o.evaluate(new Valuation(false,
false, true));
Truncation t = new FailFast(2);
HologramNode n2 = t.applyTo(n1);
Set<Truncation> set = new HashSet<>();
set.add(new KeepTop(2));
set.add(new KeepVariable(a));
HypergraphGenerator g = new HypergraphGenerator();
Hypergraph h = g.getGraph(o, set);
Iterator<?> it =
HittingSetRunner.runHittingSet(h).iterator();
while (it.hasNext()) {
Valuation v = g.getValuation(it.next());
}

S. Hallé
Create Boolean condition
false, true));
Iterator<?> it =
}

S. Hallé
Obtain an expression tree
and apply a transformation
false, true));
Iterator<?> it =
}

S. Hallé
Create tree transformations
false, true));
Iterator<?> it =
}

S. Hallé
Generate and solve
hypergraph
false, true));
Iterator<?> it =
}

S. Hallé
Generate and solve
hypergraph
Iterate over valuations of
the solution
false, true));
Iterator<?> it =
}

S. Hallé
Experiments
We can compare the hypergraph-based approach
with existing tools and techniques:
Random selection
ACTS (combinatorial coverage)
mcdc (MC/DC coverage)
and results published in past papers
(implementation unavailable).

S. Hallé
Experiments
Test suite size
0
2
4
6
8
10
12
14
2 4 6 8 10 12 14
Hypergraph
MCDC
SAT
0
200
400
600
800
1000
1200
1400
1600
1800
0 200 400 600 800 1000 1200 1400 1600 1800
Hypergraph
Chen
G-CUN
MC/DC MUMCUT
The hypergraph approach generates test suites of
comparable size for MC/DC, but larger for
MUMCUT (over-approximated)

S. Hallé
Experiments
Solving for two criteria at once
Criteria Size ratio Time ratio
MC/DC + Clause
MC/DC + Predicate
MC/DC + MUMCUT
MC/DC + 2-way
MC/DC + 3-way
Clause + 2-way
Clause + 3-way
0.87
0.85
0.99
0.76
0.83
0.83
0.87
1.27
1.29
1.38
1.20
1.11
1.24
1.11
The hypergraph approach takes more time, but
produces smaller test suites than when two
independent solutions are merged

S. Hallé
Conclusion
Uniform theoretical framework to deﬁne coverage
criteria as equivalence classes induced by a tree
transformation
Generic algorithm to produce test suites according
to any algebraic coverage criterion
Opens the way to experiments with new coverage
criteria and benchmarking of existing ones.
Future work: extension to other logics, such as
temporal/description logics

Test Suite Generation for Boolean Conditions with Equivalence Class Partitioning

More Related Content

Similar to Test Suite Generation for Boolean Conditions with Equivalence Class Partitioning (20)

More from Sylvain Hallé (20)

Recently uploaded (20)

Test Suite Generation for Boolean Conditions with Equivalence Class Partitioning