EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications

1© Copyright 2016 EMC Corporation. All rights reserved. 1© Copyright 2016 EMC Corporation. All rights reserved.
DEEP DIVE WITH MESOS & PERSISTENT
STORAGE FOR APPLICATIONS
DAVID VONTHENEN, EMC {code}

2© Copyright 2016 EMC Corporation. All rights reserved.
• David vonThenen, Developer Advocate - EMC {code}
– Twitter: @dvonthenen
– Blog: http://dvonthenen.com
– Github: https://github.com/dvonthenen
PRESENTER

• Quick Review of Mesos
• Frameworks and Two Layer Scheduling
• Production Applications
• Demo
• Q&A
AGENDA

QUICK REVIEW OF MESOS

• From a resource perspective, it’s a cluster manager:
– Pools resources to be centrally managed as a single unit
• From an application perspective, it’s a scheduler:
– Dispatches workloads to consume pooled resources
• Often described as a Data Center Operating System
(DCOS)
– Static versus Dynamic Partition of Resource
– Efficient utilization of CPU, Memory, Disk
• Proven resource scheduler for production workloads
QUICK REVIEW OF MESOS

SCHEDULERS ARE DIFFICULT TO BUILD

• Master
– Manages Mesos Agents
– Enables fine-grained sharing of resources (CPU, RAM, …)
across frameworks in the form of resource offers.
• Zookeeper
– Enables highly reliable distributed coordination (cluster
state)
• Agent (aka Slave)
– The entity that will be running the task
MESOS COMPONENTS

MESOS ARCHITECTURE

• Generic
– Anything that can be run on a standard Linux shell
– Provides a mechanism for deploying applications packages
– Supports Docker workloads
• Framework
– A plug-in to Mesos that allows you to implement a sub-
scheduler
– specialize a workload by providing custom placement and
resource allocation
APPLICATION SUPPORT

GENERIC APPLICATIONS

FRAMEWORKS AND
TWO LAYER SCHEDULING

• On the surface…
• Allows specialization of
workload by providing
custom placement and
resource allocation
• Provides monitoring of
these workloads to protect
and react when things fail
MESOS FRAMEWORK

• Scheduler
– a scheduler that registers with the master to be offered
resources
• Executor
– an executor process that is launched on slave nodes to run
the framework’s tasks
FRAMEWORK COMPONENTS

FRAMEWORK ARCHITECTURE

• Odds are you have been using at least 1 framework
without realizing it…
AS IT TURNS OUT…
• Marathon is a framework
– A framework for performing container
orchestration for long running tasks
• Chronos is a framework
– A framework for performing batch or
scheduled tasks

• Zooming into the “Marathon” box…
• Marathon is a Scheduler
• Launching a Marathon task, the “generic”
applications are created inside a “generic” (or
default) executor
• Dogfooding!
MARATHON DISSECTED

MESOS FRAMEWORKS

• 2 layer scheduling or sub-
scheduling of resources
• Manages the lifecycle of your
workload through application
specific monitoring
• What does this mean
specifically?
DIGGING DEEPER

BUILDING HADOOP CLUSTERS

MANAGING HADOOP CLUSTERS

APPLICATIONS USING HADOOP

THIS SEEMS SILLY

• Allows you to subsume the default scheduler such
that:
• You can install and configure your application
• Resources can be allocated in a way that optimizes
your applications
• Allows operation real-time automated improvements
for your applications
– Scaling up for load balancing peak times
– Scaling back to free up resources
2 LAYER SCHEDULING

• The details of your applications are hidden from the
end user!
– Deployment
– Operations
– Growth
– Disaster recovery
– Performance
• Just a consumer of the application as a service!
THE NON-TECHNICAL VERSION

• CouchBase Framework (doesn’t exist, let’s imagine it
does)
– CouchBase is a NoSQL DB
• Monitor the CouchBase instances, when nearing full
– Dynamically add an node using “server-add”
– Then call “rebalance”
• Reactive and Proactive but fully automated
• I think we had all seen this movie…
FOR EXAMPLE

SKYNET

PRODUCTION APPLICATIONS

• Containers are ephemeral therefore stateless
• Quickly found the need for state
– Configuration
– Data
– Logging / Auditing
• Containerizers like Mesos and Docker introduced
exporting to local storage
• Failure of a node? hardware? A better way?
EVOLUTION OF STATEFUL

• Persistent
– Containers can come and go,
but continue where they left off
– Application Data, Logs, Audit
Trail, etc
• External
– Storage lives outside the Mesos
Agent node
– High Availability
– Tolerates node failure
PERSISTENT EXTERNAL STORAGE

• REX-Ray
– Provides a vendor agnostic storage orchestration engine
– AWS, GCE, ScaleIO, XtremIO, many many more
– Github: https://github.com/emccode/rexray
• mesos-module-dvdi
– Provides a hook to Mesos agent nodes to provision, attach,
and detach external storage
– Github: https://github.com/emccode/mesos-module-dvdi
HOW WE ACHIEVE THIS?

• We can run “real world” applications
• Persistence – DBs
• Highly Available, Fault Tolerant
• Audit trail – Transaction Logs
• Not worry about the “2am call”
STATELESS TO STATEFUL

LET’S FACE IT…

• Trouble is… this still represents the “old way” of
thinking
• PREVENT THE CRASH OR FAILURE!
• Problem is you can’t
• What is the alternative?
STATEFUL 2.0?

• Mesos frameworks can proactively monitor for thing
going sideways
• Leveraging external persistent storage and
frameworks fail fast and recover quicker!
• Skynet!
FRAMEWORKS + STATEFUL 2.0

DEMO

• Running on Amazon EC2
– Mesos
– Marathon
– Mesos DNS
• Production HA Cluster
– 3 x Master + Zookeeper Nodes (t2.micros)
– 5 x Slave Nodes with Docker Support (t2.medium)
DEMO CONFIGURATION

• Elastic Search Framework
– Create and mount external volume
– Pour some data into Elastic Search
– Github: https://github.com/mesos/elasticsearch
• Elastic Search Instances backed by Persistent
External Storage
– Using REX-Ray
– Using mesos-module-dvdi
DEMO: FRAMEWORK + EXTERNAL STORAGE

• Deploy Elastic Search Framework using REST API
– Observe Scheduler creation
– Using browser, log into the Elastic Search Framework UI
– Observe Executor nodes get created and attaching to EBS
volumes
• Simulate Failure
– Kill EC2 Instance
– Watch executor recreation and reattach to EBS volumes
DEMO: FRAMEWORK + EXTERNAL STORAGE

DEMO: AUTO DEPLOY ELASTIC SEARCH

Data Persistence in the New Container World
Wednesday 3PM
Joshua Bernstein
VP of Technology for ETD
Tobi Knaup
CEO & Co-Founder of Mesosphere
Guru Session

@EMCcode
@dvonthenen
emccode.com
community.emccode.com
Come visit us at
Booth #1044
or in the vLab
Questions?

EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications

EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications (20)

Recently uploaded (20)

EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications