JAXLondon 2015 "DevOps and the Cloud: All Hail the (Developer) King"

DevOps and the Cloud:
All Hail the Developer King
Developer Mode:
Daniel Bryant
@danielbryantuk
Steve Poole
@spoole167

Outline
• On the previous episode of JAX London
– The rise of DevOps and the promise of microservices
• Today
– The reality of microservices, containers and DevOps
– A modern architecture of an application
• Monolitically Complex -> modularly complex++
• It’s real, it makes sense, it’s scary
– Tooling, continual learning and befriending ops
• Your choices…

I (we) am the one who knocks…
Steve Poole
IBM Developer
@spoole167
Daniel Bryant
Principal Consultant,
OpenCredo
@danielbryantuk
Making Java Real Since Version 0.9
Open Source Advocate
DevOps Practitioner (whatever that means!)
Driving Change
“Biz-dev-QA-ops”
Leading change in organisations
All over Docker, Mesos, k8s, Go, Java
InfoQ, DZone, Voxxed contributor

2014 -> Moving to DevOps: Easy, Hard
or Just Plain Terrifying
• Extending agility across your IT org
• From waterfall to agile, (and agile-fall in Ops)
• Breaking down the silos
• CI/CD is key (automate all the things)
• Business need to react -> DevOps and Cloud
• Recommended Ruby, Docker and Vagrant

This Year
• Containers are mainstream
– Orchestration platforms are
where the action is at
– And the container bolt-ons are
evolving (storage, networking)
• #DOES14 showed enterprises
are embracing DevOps
– ‘Bimodal IT’ is really a thing
https://www.flickr.com/photos/tristantaussac/

Part 1- Painful Lessons…
https://www.flickr.com/photos/sarahmstewart/

All I hear is microservices…
“In computing, microservicesis a software
architecture style in which complex
applications are composed of small,
independent processes communicating with
each other using language-agnostic APIs. These
services are small, highly decoupled and
focus on doing a small task, facilitating a
modularapproach to system-building.”
https://en.wikipedia.org/wiki/Microservices

Microservices
Turn applications into small,
independent, highly decoupled,
modular services
https://www.flickr.com/photos/daikrieg/
You want to make
my life more
complicated?

https://www.flickr.com/photos/tahini/
Where’s the problem?

browser
App A
V1.0
Database
browser
You
Your
Customers
server
Data Centre

browser
Load
balancer
App A
V2.0
App A
V2.0
App B
V1.0
Database
browser
Database
Ops

browser
Load
balancer
App A
V2.0
App A
V2.0
App B
V1.0
Database
browser
Database
browser
App A
V2.0
App A
V2.0
App B
V1.0
browser

browser
Load
balancer
App A
V2.0
App A
V2.0
App B
V1.0
Database
Database
browser App A
V2.0
App A
V2.0
App B
V1.0
browser
browser
browser
browser
browser
Load
balancer
Database

browser
Load
balancer
App A
V2.1
App A
V2.1
App B
V1.0
Database
Database
browser App A
V2.1
App A
V2.1
App B
V1.0
browser
browser
browser
browser
browser
Load
balancer
Database

A simple upgrade or a major impact?
lost revenue or going out of business?

https://www.flickr.com/photos/24151087@N00/
What lessons have we learnt?
Sharing datastores sounds like it saves effort but
introduces cohesion between applications
Big-bang versioning of applications means putting
existing unchanged usecases at risk
Scaling is challenging when you try to duplicate whole
systems
Infrastructure – it’s much more important than we
realized

Part 2 - Build Your Own Platform?

Alexis Richardson’s Thoughts

http://wikibon.com/wp-content/uploads/container_implementations.png
Technology Choices

The Anatomy of a Modern Platform?
(Maybe in the Googlesphere…)
http://www.eightypercent.net/post/layers-in-the-stack.html

https://www.flickr.com/photos/miriamdelirium/
Tooling – what’s left to do?

Dynamic Development
Capacity
Predefined static
VM’s LPARs etc
OpenStack
Cloud(s)
Docker
Cloud
Infrastructure as Code
Chef, Puppet, UCD …
OS
Infra On Prem Data
Centres
Cloud Providers
SoftLayer / Amazon etc
Config Containerized
Applications
Continuous Availability
Mesos etc
Deploy
Pipeline
Block Architecture of Hybrid Cloud Dev SaaSPrimary Audience
GIT / Jenkins / Junit …. Selenium, Jmeter…
✔
✔
✔
✔
✔
Application DIY

Dynamic Development
Capacity
Predefined static
VM’s LPARs etc
OpenStack
Cloud(s)
Docker
Cloud
OS
Infra On Prem Data Centres Cloud Providers
Applications
Mesos etc
Deploy
Pipeline
Other static
hosts (‘BYOD’)
DIY
Application DIY

Dynamic Development
Capacity
Predefined static
VM’s LPARs etc
OpenStack
Cloud(s)
Docker
Cloud
OS
Infra On Prem Data Centres Cloud Providers
Applications
Mesos etc
Deploy
Pipeline
Other static
hosts (‘BYOD’)
DIY
Compliance
/ Security ContainersVM Images
Application DIY

Tooling Status
• Cloud OS technology is mainstream
• As you rise up the stack the quality (and options )
vary
• Your main challenge:
– Create a robust (self-healing), secure, compliant
environment with a bag of parts that is still evolving
• Roll your own?
– We still do

Some
suggestions
• Applications
– Serenity BDD, Junit
– Infrastructure
– RoleSpec, ChefSpec,
– ServerSpec
• ‘ilities’
– JMeter, Gatling,
flood.io
– ZAP, Google Cloud
Scanner
https://www.flickr.com/photos/mulliganstu/

You do understand about security and
compliance right?
https://www.flickr.com/photos/adulau/

Where’s your data?
Legal restrictions on data location
Vary by country even within the EU.
Different rules depending on types of data
You already know this?
But now you’re putting the data in the
cloud.
DO YOU understand where its going
Can you control / manage / audit the
situation?
Now it’s your problem.
Having fun finding tools to help

How’s your security knowledge?
• Again – now it’s your problem
– Your code is running in the cloud
– You created the services & the containers
– Are they secure?
– How do you test?
– Are you sure?
– Are those web services you’re buying secure?
• How much do you know about networking?
– Not enough…
• Cyber crime is big business – you will get targeted.
https://www.flickr.com/photos/61423903@N06/

Cybercrime
The estimated cost of Cybercrime is $100 billion per year
500 million victims per year
600,000 facebook accounts compromised per day
28% of cyber attacks are around SQL injection
Only 5% of successful attacks are against Banks, Credit, Financials
etc 35% of successful attacks are against businesses in general
50% of attacks come from hackers – just for the fun if it
60% caused losses to the business

Talk to your Ops team
• They are your best friends.
– They know about security and networking.
• You need to know too
– They know (some) of the answers
• It’s a whole new domain for you
– It’s not a new problem for them
• Time to learn

The Results of the Survey Are In…
• Puppet Labs 2015 State of DevOps
– Available: puppetlabs.com/2015-devops-report
• Accelerates deployment
– High performers 30x more deploys
– Code committed to production 200x faster
• Prevents failures and streamlines recovery
– High performers 60x fewer failures
– Recovery 168x faster

gotocon.com/dl/goto-london-2015/slides/NicoleForsgren_DevOpsNext.pdf

DevOps: Share the Pain...
13/10/2015 @danielbryantuk

“Dev-on-call”
An occasional spike to the head
is a good thing...
...metaphorically speaking
• You build it, you run it
– Shared responsibility
– Communication

Why Take Responsibility?
www.infoq.com/news/2015/06/too-big-to-fail

Cross-functional Teams (FTW)
• Spotify (bit.ly/1C46ZKo)
– Culture
• Amazon (bit.ly/1F3Dgkm)
– Communication
• Gilt (gi.lt/1rgyWvO)
– Strategic alignment

When Things (Inevitably) Go Bad…

“It” Happens
• Failure happens all the time in the cloud
– Distributed systems are complex
– Container/cluster tech promotes transiency
• Get used to failing… literally
– Run ‘Gamedays’
– Disaster in Recovery Training (DiRT)
– bit.ly/1L43U4H

False Start, Dead, or Alive and Kicking?
• We see lots of ‘DevOps’ teams in the wild
– They often look suspiciously similar to ops teams
that are using Puppet/Chef
• Advanced companies leaning towards
– (Full stack) developers
– Platform (tooling) teams
– Site Reliability Engineers (SREs)

“Programmable” Infrastructure?

But wait! What about the “-ilities”?

The Elephant in the Room
• Security…
• “What is Rugged All About?”
– gotocon.com/dl/goto-london-
2015/slides/JoshuaCorman_WhatIsRuggedAllAbo
ut.pdf
• “The Future of Containers”
– www.youtube.com/watch?v=_jBTHyo0mEQ

Be Wary of Shiny SaaS
• SaaS tools can be like crystal
– Easy to get started
– Initially you have fun
– Encourages you to attend the
wrong (root cause) parties
– Soon you’re selling Grandma
for the next (support) hit
– You eventually lose friends
(and your teeth may fall out) https://www.flickr.com/photos/perfected/

Key Messages
• Containers are here to stay
– You need to learn how to design applications in a new
way (‘microservices’)
• The stack is still not baked – but is getting there
• “Safety first”
– It’s a wild world out there – you must understand
about security, networking and the the rise of the
cyber criminal
• Make your ops team your best friend

Other Tech Tips
• Golang dominates infrastructure and tooling
• Learn about clustering and scheduling
– Mesos, Kubernetes, Nomad
– Borg, Omega, Google’s DaaC
• Network skills essential (virtualization, overlays, OSI)
• Multi-cloud appears a solid bet
– Neutrality / abstraction
– Resource managers, OpenStack, Terraform, Ansible…

Thanks – Questions?
Daniel Bryant
@danielbryantuk
Steve Poole
@spoole167

JAXLondon 2015 "DevOps and the Cloud: All Hail the (Developer) King"

More Related Content

What's hot

Viewers also liked

Similar to JAXLondon 2015 "DevOps and the Cloud: All Hail the (Developer) King"

More from Daniel Bryant

Recently uploaded

JAXLondon 2015 "DevOps and the Cloud: All Hail the (Developer) King"

Editor's Notes