Thinking in Terms of a Data Center Operating System
1 like425 views
Weston Bassler will discuss the concept of a data center operating system and introduce Apache Mesos. A data center operating system treats all the machines in a data center as a single machine by abstracting resources into a single pool and allowing for better utilization of resources and ease of deploying and scaling applications. Apache Mesos is an open source project that implements this concept through a master-slave architecture and frameworks that manage tasks and services. Bassler will provide demonstrations of launching an application with Marathon, scaling an application on Marathon, and configuring Jenkins to run jobs on Mesos. He will also discuss ways to get started using Mesos through documentation, books, his GitHub repository, or by contacting him directly.
Thinking in Terms of a Data Center Operating System
- 1. Thinking in Terms of a Data Center Operating System Weston Bassler Systems Engineer, Klarna Inc
- 2. Who am I? Weston Bassler, Systems Engineer, Klarna Inc ● currently working at Klarna, a new startup in Columbus, as a Systems Engineer. Previous experience as a Cloud and Linux Engineer for LexisNexis and Verizon Wireless. ● Highly Involved in Linux and Open Cloud Technologies ● Lover of Gifs and Memes
- 3. What I will be Discussing ● Data Center Operating System? ● Quick Introduction to Apache Mesos ● What makes it so interesting? ● Current ways we are trying to utilize ● Demos!!!! ○ Launching an App with Marathon ○ Scaling your App properly ○ Jenkins on Mesos ● Ways to get started using it
- 4. Data Center Operating System ● What is it and what do you mean? ○ All of the machines of your DC and treating it as a single Machine ○ Abstracting resources into a single pool ○ CLUSTER ALL THINGS ● What problem(s) does this solve? ○ Less complexity of managing multiple nodes ○ Better utilization of your resources ○ More control over resources ○ Ease of Deploying and Scaling Apps ● How do I acheive this concept of a “Data Center Operating System”?
- 5. Enters The Apache Mesos Project
- 6. Mesos Explained ● What exactly is Mesos? ○ From mesos.apache.com - “Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.” ● Master/Slave Architecture ○ Zookeeper elects a Leader ○ Masters tell the Slave(s) what to do ○ Slave(s) are the workers ● Frameworks Manage Tasks/Services
- 8. How Does Mesos Work? source: Anton Kirillov
- 9. So why is this interesting? ● Get the most out of your infrastructure -- Utilization ● High Availability / Fault Tolerance / Peace of Mind ● Containers! -> Speed, Flexibility, Isolation, Docker, Docker, Docker ● Scalability -- Nodes and Apps ● Ease of Use -- Web UI for ALL & Native Clusterting ● Use of Frameworks -- Hadoop, Kubernetes, Chronos, Marathon, Spark……
- 10. Some Examples “Launching a Docker container is so yesterday. Today, it's all about 50,000 Docker containers... launched in 100 seconds... on a Mesos-powered cluster.”
- 12. Another Example Apple and Siri - Apple redesigned the Siri Infra to run on Mesos. They developed a Framework called JARVIS which acts in the same way as Marathon. 100 different Services Stores Data on HDFS
- 13. What made me interested? ● The Use of Containers ● Being able to Scale and Manage Containers So Quickly ● Fault Tolerance ● Container Orchestration ● PaaS
- 14. Demo 1 ● Launching an App with Marathon ○ What is Marathon? Overview ○ API/JSON ○ Docker/Mesos Containers ○ Launch Script
- 15. Demo 2 ● Scaling an App on Marathon ○ From the UI ○ Killing Tasks ○ HAProxy ○ Marathon API -> Examples from Git Repo
- 16. ● Jenkins Mesos Plugin ○ What is the Plugin? ○ Take through the process Jenkins Configuration ○ Create and Run a Jenkins Job -- ■ Dreaded “Hello, World” ■ Using Docker to build Docker ○ Show Jenkins Job running on Mesos Demo 3
- 17. Ways to Get Started ● http://mesos.apache.org/documentation/latest/ ● Books: ○ Apache Mesos Essentials - By: Dharmesh Kakadia ○ Mesos In Action - By: Roger Ignazio ● Visit My Github Page ○ Mesos Sandbox: git clone https://github.com/geekbass/mesos-sandbox.git ● Contact Me!!! ○ [email protected] ○ LinkedIn: Weston Bassler ○ @w_bassler ● Companies Using Mesos
- 18. Questions?