100% Code Coverage in Real World Software

100% Code Coverage
in Real World Software
W-JAX 2017
2017-11-09

Once upon a time
*** *** Shipment Tracking
 Only a few
deployments slots per year
 per deployment -> $$$
 Re-deployment?
 wait for next slot
 pay more $$$
 -> Quality or die!

Why should we want
Automated Testing?

Why automated testing?
It‘s 2017, so a part of me wants to say…
Srsly?
Pixabay, License: CC0 Creative Commons

Why automated testing? (1/4)
OK… we want automated testing in order to…
 Protect against (regression) bugs
 Keep confidence & control
 Manage knowledge
 Ensure long term adaptability
 Work faster
 Build upon working parts
Q = 1..n qi

Systems Engineering: Software Lifecycle (2/4)
”Be nice to future-you“ – unknown
People working on software after…
 … 1 year? -> new people
 … 3 years? -> new > original
 … 6 years? -> 0 originals anymore
Future “generations“ of team members must be prepared for…
 … changes, enhancements etc.
 … patches, updates, upgrades, replacements etc.
1 2 5 †*

Reduce complexity to merely binary information (3/4)
”Complexity”by Mark Skipper, License: CC BY 2.0

Testability & Code Design (4/4)
TDD promotes these ideas:
 If it‘s hard to test…
 …it likely has design issues
 So don‘t try to find more and
more clever ways to test…
 … instead change the code to
make it more testable
 -> Testable is the new clever
Test Driven Development & Design

Why should we want
Code Coverage?

Why Code Coverage?
We want to find out what we don‘t know.
 Use it as a checklist
 Find blind spots
 Find dead, unused code
static double divide(double a, double b) {
if (b == 0) {
}
throw new DivisionByZeroException();
}
return a / b;

Why should we want
100% Code Coverage?

Why 100% Code Coverage? (1/6)
 Tests help a lot…
 … for managing the code
the tests cover
 … for code they don‘t cover
test offer no help
 That‘s like feeling bigly about
having the engine running…
 … when you should have all 4 of them running

 Maybe you had a reason not to
cover some code… at that time…
 Will the future team still
know that reason?
 And, after the usual and
inevitable drifts in the code base…
 …will the reason still
be valid in the future?
”U Turn“ by André-Pierre du Plessis,
License: CC BY 2.0

 Often, bug clump in some areas
 As the saying goes: “90% of problems
are caused by 10% of the code“
 90% Code Coverage means you don‘t
cover 10%
 What if that‘s THOSE 10%?
 No? How can you be sure w/out tests?

 OK, that was number magic…
 …let‘s put it a different way:
 Which code will likely
cause more problems:
 highly tested code?
 untested, potentially even
untestable code?
Edvard Munch, “The Scream”

100% Code Coverage to prevent decay (4/6)
”It wasn’t me. It was that way already.“ – my children
Prevent decay by
“Broken Windows“:
 Some code sub-standard…
-> more code will follow
 … and developers can blame it
on the darned legacy code
 Instead, have a
“No code left behind“ strategy
“Broken windows, Northampton State Hospital”
by Karan Jain, License: CC BY-SA 2.0

100% Code Coverage as the Team’s Rule (5/6)
Use the power of metrics
 Rules that everyone needs to stick to
 … objectively measured by a 3rd party
 … let the metric “reward“ the team
when at 100%
 … use the metric to
reduce complexity to…
“Complexity kills. It sucks the life out of developers […]“ – Ray Ozzie
Source: Geek & Poke

23
For professional developers,
automated testing is an obligation.
Uncovered Code does not comply
to that obligation.

25
There is no reason why you
SHOULDN‘T WANT 100% Code Coverage.
(There may be reasons why you
CAN‘T get to 100%, though…)

Code Coverage is a fallacy just a tool.lielie fallacy
It depends on what you use it for…
… and how you interpret its output.

How to trick Code Coverage
“Ohh, I‘m a liar. Yeah.“ – Henry Rollins
 When you run a Test Case
it will always cover some code
 E.g. assertNotNull(new MyClass())
 -> x% Code Coverage
 -> 0 value
@Test
void test_init() {
assertNotNull(
new MyClass());
}

Just because Code Coverage can be faked
does not mean it is irrelevant.

Code Coverage != Test Coverage
Code Coverage means…
 … code is covered by tests
 … but you still can have bugs
Code Coverage does NOT mean…
… all scenarios are covered
-> that would be Test Coverage
static double divide(double a,
double b) {
return a / b;
}
@Test
void test_divide() {
double actual = divide(6, 2);
assertEquals(3, actual, .1);
}
// no validations,
// weak precision check,
// division by zero is possible!
// ...
Understand the metric

Where Code Coverage cannot be measured
“I don't think anybody tests enough of anything.“ – James Gosling
Views
 JSP, JSTL, JSF and others that
allow for markup + conditional code
Configuration
 Did we test ALL configuration entries?
 How can we spot obsolete configs?
“Game over”by porpom, License CC BY 2.0,
<c:if test="${temp > 18 && temp < 27}">
<p>Let‘s take a walk.</p>
</c:if>

Where Code Coverage breaks:
Aliens, Trade Offs & Lack of Knowledge

Alien Code
Generally: full coverage can be impossible
if you cannot change the production code
 3rd party code
 pasted “alien“ code
 generated alien code
e.g. XSD to Java, UML to code exports,
RAD tools, scaffolding scripts etc
 -> consider moving it out (JAR, src, …)

Legacy Code
(Your own, untested) legacy code…
 static calls
 global calls
 global state
 wrong dependencies
 …
 …is like alien code to new team members
https://www.amazon.de/dp/0131177052/

Conscious Trade Offs
Sometimes you find yourself saying:
“I don‘t really know…
 … what I want yet.“
 … how to do it yet.“
-> You need a Proof of Concept (PoC)
 But TDD and 100% Code Coverage
are hard to do when you don‘t know what to assert
 However, professionals should create PoCs only as throw-away code

Lack of Knowledge
Things you don‘t know how to test (yet)
 Sometimes you need to
master the technology first
 It may take some time & effort
to find solutions for testing
 -> Don‘t write testing off for good,
keep looking for solutions
 + look beyond “The Common Practice™”
Image: “Digging the Drain”by
International Disaster Volunteers, License: CC BY 2.0

3 tips that work in
The Real World™

Advice #1:
Start @ 100%, Stay @ 100%

TDD – Test Driven/First Development
Start @ 100%, Stay @ 100%
Image by user “Jgu7“
Get familiar with
 Test Driven Development
 … and the Test First approach

TDD DON‘Ts
Start @ 100%, Stay @ 100%
“Hexagonal Architecture”a.k.a.
“Ports and Adapters”by Alistair Cockburn,
Don‘t start on the “outside“,
i.e. do NOT start with:
 Controllers / Commands
 Views
 Data Access Objects / Queries
 File System Accessors
 …

Also, it‘s a psych thing
If you start on the outside you‘ll
 have slow progress
 feel the need for non-Unit tests
(i.e. slow tests)
 feel the need to mock stuff
 have a Coverage below 100%
 have less fun (always in debt)
Sigmund Freud, Wikipedia

Also, it‘s a psych thing
You might make bad decisions
 You might put more into the
class, e.g. the Controller,
than you should…
 … to kind of make the effort
spent “pay off“
 -> poor design and wrong
dependencies are possible

The Dependency Rule
Inspired by “The Clean Architecture”
by Uncle Bob Martin
Enterprise
Domain
specific
Controllers
“source code
dependencies
can only point
inwards”
Actros, MPC,
Enterp. Rules
Trucker Profile,
App Rules
DI Container, OSGi
SysDate
DAOs
FS
SQL DB
NoSQL DB
SOAP/REST
SQL, Queries

Ports & Adapters / Dependency Inversion
 “But the Use Cases and Domain objects
need data from the outer layers!”
 “How can I read from and write
to the outer layer without depending on it?”
 -> Dependency INVERSION
 -> “Ports”
(simplified: outer layers behind interfaces)

Advice #2: Identify the Domain

The “No Logic“ argument
”Tests are for logic. My code does not contain logic “
 You might not see any logic…
 …but, usually, it‘s there…
 …it‘s probably just
in the wrong place
BIG
Controller
BIG
DAO
tiny
Service
“Weight Training Crossfit Fitness Models”
by ThoroughlyReviewed.com, License: CC BY 2.0

Identify the Domain
“The Clean Architecture”by Uncle Bob Martin
 … yes, it exists
 Start at the core: the Domain
 … also, it’s the code that’s the
easiest to test!
 you’ll start at 100% coverage
 … then stay at 100%
 … while progressing outwards
“source code dependencies can only point inwards”

Advice #3: When in doubt,
trust the test

Hard to test? -> keep it out
Keep your Domain, Rules and
APIs clean, i.e. keep
 hard to test stuff out
 e.g. 3rd party frameworks
& libraries
 … (especially) if they
interfere with testability

Trust the test:
If (your own) code is hard to test
then it likely has a design problem.

Reality Check
Our experience shows:
 100% Code Coverage is
feasible & relevant
 However, you might need to
code and design differently…
 … but that will eventually
lead you to Clean Code and
Clean Architecture
“Big_Data_Higgs”by KamiPhuc, License: CC BY 2.0

Useful Links
 Ports & Adapters
 Dependency Rule
 Clean Architecture (new Book)
 DAO / Repository Pattern
 3 Rules of TDD
 Bowling Game Kata
 “The more testing you do…”

#HappyHacking!
Andreas Czakaj
Managing Director IT / CTO
mensemedia Gesellschaft für Neue Medien mbH
Neumannstr. 10
40235 Düsseldorf, Germany
Email: andreas.czakaj@mensemedia.net
Twitter: @AndreasCzakaj
Blog: https://blog.acnebs.com/

100% Code Coverage in Real World Software

More Related Content

What's hot (6)

Similar to 100% Code Coverage in Real World Software (20)

Recently uploaded (20)

100% Code Coverage in Real World Software