How to Build Your Own 

Cyber Security Framework 

using a Balanced Scorecard"
Russell Cameron Thomas!
EnergySec 9th Annual Security Summit!
September 18, 2013!
Twitter: 

@MrMeritology!
Blog: 

Exploring Possibility Space!
Who here loves frameworks?!
Who here loves frameworks?!
NIST Cyber Security Framework?!
Other?!
Frameworks can matter (a lot) 

Frameworks can matter (a lot) 

if they are instrumental in
driving new levels of
Cyber Security Performance
What the hell is 

“Cyber Security Performance”?!
Yes, “Cyber”!
Yes, “Cyber”!
Confluence of…!
•  Information Security!
•  Privacy!
•  IP Protection!
•  Critical Infrastructure Protection & Resilience!
•  Digital Rights!
•  Homeland & National Security!
•  Digital Civil Liberties!
What the hell is 

“Cyber Security Performance”?!
“Cyber security performance” is… "
… systematic improvements in an
organization's dynamic posture
and capabilities relative to its 

rapidly-changing and uncertain
adversarial environment.”!
“Cyber security performance” is… "
…Management By Objectives!
(Drucker)!
“Cyber security performance” is… "
…Management By Objectives!
…Performance Mgt, incentives!
“Cyber security performance” is… "
…Management By Objectives!
…Performance Mgt, incentives!
…Staffing, training, organizing!
“Cyber security performance” is… "
…Management By Objectives!
…Performance Mgt, incentives!
…Staffing, training, organizing!
…Organization learning, agility!
“Cyber security performance” is… "
…Management By Objectives!
…Performance Mgt, incentives!
…Staffing, training, organizing!
…Organization learning, agility!
… and good practices!
“Performance” vs “Practices”!
Using the Universal Language of Executives….
Using the Universal Language of Executives….
Keep
your head
still
Keep
your head
still
“Keep
your arm
straight”
Keep
your head
still
“Keep
your arm
straight”
“Swing on
one plane”
Keep
your head
still
“Keep
your arm
straight”
“Swing on
one plane”
“Swing easy”
Keep
your head
still
“Grip it
and rip it!
“Keep
your arm
straight”
“Swing on
one plane”
“Swing easy”
Best practices are like golf tips…
Best practices are like golf tips… 

Golf tips alone don't make good golfers
Why Agility? 

Why Rapid Innovation?!
State of

the Art!
Lagging
InfoSec
Program!
Time for some drama!
Time for some drama!
Set in the Summer of 2017!
“
I	
  in central Texas.”	
  
t was another
long heat wave
Spare generating capacity was dangerously low!
You run information security!
at a large industrial company!
that includes several
and cogeneration.!
Thanks to
deregulation
and incentives,
microgrids
have taken off, 

especially 
in Texas
=	
  10+	
  microgrids	
  
Microgrid Adoption, 2017
In recent days,
instead of selling
its excess power,
your firm was
buying at peak
spot prices.


This was strange.!
18	
  months	
  earlier	
  
YouEnergy Ops 
Manager
Business
Continuity
Manager
Effective Response, 

Recovery  Resilience
Your Microgrid Automation

hosted
auto-configuring
software
reporting/trending!
system config!
diagnostics!
Internet	
  
Microgrid
Supervisory
Controller
12	
  months	
  earlier	
  
Spot trading was largely automated
via microgrid automation software.
12	
  months	
  earlier	
  
Optimize Exposure
Insiders?
Threat Intelligence
Business Partners?
Contractors?
Criminals?
APT?
Error?
Hactivist?
Terrorist?
24	
  months	
  earlier	
  
Our New Capability:

Attack-driven Defense
1.  Raise cost to attackers
2.  Increase odds of detection
3.  Iterate defense based on real attack patterns
24	
  months	
  earlier	
  
source:	
  Etsy	
  
h7p://www.slideshare.net/zanelackey/a7ackdriven-­‐defense	
  
Insiders?
Business Partners?
Contractors?
Criminals?
APT?
Error?
Hactivist?
Terrorist?
Threat IntelligenceYesterday	
  
Effective 

Threat Intelligence
Sensors  Pattern Detection for

Anomalous User Behavior
24	
  months	
  earlier	
  
Any
Non-
Tech. Tech.
source:	
  Etsy	
  
h7p://www.slideshare.net/zanelackey/a7ackdriven-­‐defense	
  
User	
  
Class	
  
Insiders?
Business Partners?
Contractors?
Criminals?
APT?
Error?
Hactivist?
Terrorist?
X
Threat Intelligence
X
Yesterday	
  
Quality of

Protections  Controls
Insiders?
Business Partners?
Contractors?
Criminals?
APT?
Error?
Hactivist?
Terrorist?
X
X
Threat IntelligenceYesterday	
  
Efficient/Effective

Execution  Operations
12	
  months	
  earlier	
  
Effective

External Relationships
The Crime:
ArDficially	
  Congested	
  
Subsided	
  Generators	
  
Manipulation of
Wholesale Market
Subsidies
Conges'on	
  pa+erns,	
  July	
  14,	
  2017	
  
Losers: You and hundreds of other microgrids forced to
generate spot market bids during price spikes.
(Botnet-style. Each loses a little $$)
Scam: Generate losing trades in one market
to make money in another market
Attack: Compromised Hosted
Auto-Configuration Software
hosted
auto-configuring
software
reporting/trending!
system config!
diagnostics!
Internet	
  
Microgrid
Supervisory
Controller
The Attackers
Insider: Contractor
at web application
software company
Outsider: Hedge
fund manager
bribed contractor
with profit sharing
Gold Man Hacks Bid Probe 
2017
2017
Gold Man Hacks Faces Record Fine Over Energy
Over	
  the	
  last	
  24	
  months	
  
Adap've	
  
Threat	
  
Intelligence	
  
A+ack-­‐	
  
driven	
  
Defense	
  
Expanded	
  
External	
  
Engagement	
  
Expanded	
  
Detec'on	
  
	
  Response	
  
Metrics	
  
Effective

Agility  Learning
Over	
  the	
  last	
  24	
  months	
  
Effective

Design  Development
Over	
  the	
  last	
  24	
  months	
  
Optimize 

Cost of Risk
Over	
  the	
  last	
  24	
  months	
  
Accountability 

Responsibility
The End
Summary:

The Ten Dimensions of 

Cyber Security Performance!
Actors	
  
Systems	
  
The	
  
Organiza7on	
  
Events	
  
Context
Actors	
  
Systems	
  
1.	
  Exposure	
  
Events	
  
Dimension 1:

Optimize Exposure
Actors	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
Events	
  
Dimension 2:

Effective Threat
Intelligence
Actors	
  
Systems	
  
1.	
  Exposure	
  
3.	
  Design	
  	
  Dev.	
  
2.	
  Threats	
  
Events	
  
Dimension 3:

Effective Design 
Development
Actors	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  Protec'ons	
  	
  
	
  Controls	
  
Events	
  
Dimension 4:

Quality of Protection 

 Controls
Actors	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  ProtecDons	
  	
  
	
  Controls	
  
5.	
  Execu'on	
  
	
  Opera'ons	
  
Events	
  
Dimension 5:

Effective/Efficient

Execution  Operations
Events	
  
Actors	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  ProtecDons	
  	
  
	
  Controls	
  
5.	
  ExecuDon	
  
	
  OperaDons	
  
6.	
  Response,	
  	
  
Recovery	
  
	
  Resilience	
  
Dimension 6:

Effective Response,
Recovery  Resilience
Opera7onal	
  
Cyber	
  Security	
  
Dimensions 1 – 6 

Measure Core
Performance
Events	
  
Actors	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  ProtecDons	
  	
  
	
  Controls	
  
5.	
  ExecuDon	
  
	
  OperaDons	
  
6.	
  Response,	
  	
  
Recovery	
  
	
  Resilience	
  
First	
  Loop	
  Learning	
  
“First Loop Learning”

is Continuous Improvement

in Daily Operations
Events	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  ProtecDons	
  	
  
	
  Controls	
  
5.	
  ExecuDon	
  
	
  OperaDons	
  
Actors	
  
7.	
  External	
  Engagement	
  
The	
  
Organiza7on	
  
Other	
  
Organiza7ons	
  
Government	
  	
  
Law	
  Enforcement	
  
Dimension 7:

Effective External

Engagement
6.	
  Response,	
  	
  
Recovery	
  
	
  Resilience	
  
Events	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  ProtecDons	
  	
  
	
  Controls	
  
5.	
  ExecuDon	
  
	
  OperaDons	
  
Actors	
  
7.	
  External	
  Engagement	
  
Other	
  
Organiza7ons	
  
Government	
  	
  
Law	
  Enforcement	
  
8.	
  Agility	
  	
  Learning	
  
Dimension 8:

Effective Agility

 Learning
6.	
  Response,	
  	
  
Recovery	
  
	
  Resilience	
  
Events	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  ProtecDons	
  	
  
	
  Controls	
  
5.	
  ExecuDon	
  
	
  OperaDons	
  
Actors	
  
7.	
  External	
  Engagement	
  
8.	
  Agility	
  	
  Learning	
  
9.	
  Total	
  Cost	
  of	
  Risk	
  
Other	
  
Organiza7ons	
  
Government	
  	
  
Law	
  Enforcement	
  
Dimension 9:

Optimize

Total Cost of Risk
6.	
  Response,	
  	
  
Recovery	
  
	
  Resilience	
  
Events	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  ProtecDons	
  	
  
	
  Controls	
  
5.	
  ExecuDon	
  
	
  OperaDons	
  
Actors	
  
7.	
  External	
  Engagement	
  
Total	
  Cost	
  of	
  Risk	
  
10.	
  Accountability	
  
	
  Responsibility	
  
Stakeholders	
  
9.	
  Total	
  Cost	
  of	
  Risk	
  
8.	
  Agility	
  	
  Learning	
  
Other	
  
Organiza7ons	
  
Government	
  	
  
Law	
  Enforcement	
  
Dimension 10:

Accountability

 Responsibility
6.	
  Response,	
  	
  
Recovery	
  
	
  Resilience	
  
Dynamic	
  Capabili7es	
  
Dimensions 7 – 10 

Measure Systemic

Agility
Events	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  ProtecDons	
  	
  
	
  Controls	
  
5.	
  ExecuDon	
  
	
  OperaDons	
  
Actors	
  
Total	
  Cost	
  of	
  Risk	
  
10.	
  Accountability	
  
	
  Responsibility	
  
Stakeholders	
  
9.	
  Total	
  Cost	
  of	
  Risk	
  
8.	
  Agility	
  	
  Learning	
  
Other	
  
Organiza7ons	
  
Government	
  	
  
Law	
  Enforcement	
  
7.	
  External	
  Engagement	
  
6.	
  Response,	
  	
  
Recovery	
  
	
  Resilience	
  
Second	
  Loop	
  Learning	
  
“Second Loop Learning”

is Innovation

and Reinvention*
*	
  Individual	
  and	
  CollecDve	
  
Events	
  
Systems	
  
1.	
  Exposure	
  
2.	
  Threats	
  
3.	
  Design	
  	
  Dev.	
  
4.	
  Protec'ons	
  	
  
	
  Controls	
  
5.	
  Execu'on	
  
	
  Opera'ons	
  
Actors	
  
7.	
  External	
  Engagement	
  
Stakeholders	
  
10.	
  Accountability	
  	
  
	
  Responsibility	
  
9.	
  Total	
  Cost	
  of	
  Risk	
  
8.	
  Agility	
  	
  Learning	
  
Other	
  
Organiza7ons	
  
Government	
  	
  
Law	
  Enforcement	
  
Ten Dimensions of

Cyber Security

Performance
6.	
  Response,	
  	
  
Recovery	
  
	
  Resilience	
  
Last thought…!
“Can’t you make it simpler?”!
“Can’t you make it simpler?”!
“We need a crayon version
for executives and other
business and policy types”!
Sure!
Sure!
•  “Transcendental numbers hurt my head”!
Sure!
•  “Transcendental numbers hurt my head”!
•  Declare π = 3.0!
Sure!
•  “Transcendental numbers hurt my head”!
•  Declare π = 3.0!
•  But we lose something essential!
“Circle”	
  
russell.thomas@meritology.com



http://exploringpossibilityspace.blogspot.com/



@MrMeritology!

How to Build Your Own Cyber Security Framework using a Balanced Scorecard