Cloud infrastructure on Apache Mesos
This document discusses Apache Mesos, an open source cluster manager that provides efficient resource isolation and sharing across distributed applications or frameworks. Mesos abstracts CPU, memory, storage, and other compute resources away from machines in a datacenter to enable resource-sharing and isolation among application frameworks like Marathon for deploying applications and Chronos for distributed cron jobs. Mesos supports running distributed services efficiently and at large scale by sharing resources between frameworks and allowing applications to dynamically use resources from an elastic resource pool based on demand. The document demonstrates Mesos concepts through examples and concludes with an overview of how to deploy applications on Mesos using Docker containers and frameworks.
Cloud infrastructure on Apache Mesos
- 1. Cloud infrastructure on Apache Mesos AHMED BACHA Abdelkrim A. Trainee @WORLDLINE Lyon Apero-Tech | 13.08.2015 1
- 2. Agenda • Computer and Datacenter evolution • Apache Mesos project • Mesos Frameworks o Marathon o Chronos o How to run applications ? • Demo 2
- 3. Computer and Datacenter evolution (1) • Computers became more powerful than ever • OS (kernel) manage resources abstraction and scheduling. 3
- 4. Computer and Datacenter evolution (2) • Datacenters are big • « Ops » & « Dev » still plumb infrastructure using tools and small piece of code • Building distributed applications become a challenge 4
- 5. Computer and Datacenter evolution What if our computers were operated like our datacenters ? 5
- 6. Computer and Datacenter evolution 6
- 7. Apache Mesos project Introduction 7 • Top Apache foundation project • Cluster resource manager (Datacenter Kernel)
- 10. Apache Mesos project Introduction 10 • Used by
- 11. Apache Mesos project Introduction 11 “We have two datacenters, one is running on virtualization technology and needs 8-10 full- time engineers, the second one uses Apache Mesos and needs only 1-2 engineers.”
- 12. Apache Mesos project Introduction 12 “Our latest Siri Backend (version 3) is powered by Apache Mesos”
- 13. Apache Mesos project Introduction 13 “ Mesos, software for automatic scaling, was in fact built around five years ago and is already in use on more than 50,000 cores at Twitter ”
- 15. Apache Mesos project Abstraction 15 • Resource (CPU, RAM, Storage) abstraction and aggregation • Efficiency & HA • Elastic sharing
- 16. Apache Mesos project Resource sharing 16
- 17. Apache Mesos project Scheduling (1) 17
- 18. Apache Mesos project Scheduling (2) 18
- 19. Mesos Frameworks Services 19 How do we use this Mesos cluster for our daily needs ?
- 20. Mesos Frameworks Services 20 • Marathon : a PaaS like, to run applications (Docker images or binaries) • Chronos : a distributed CRON table
- 21. Mesos Frameworks WaaS : Marathon-HA Proxy 21 Woldline (WaaS) Marathon HA-Proxy load balancer & reverse proxy. • Load balance deployed instances • Dynamic resolution of apps names
- 23. Mesos Frameworks Services 23 • Heterogeneous tasks on elastic partitions • Shared tasks executions • Multiple frameworks
- 25. Mesos Frameworks Apps 25 What is the best way to run applications on top of a Mesos cluster ?
- 27. Mesos Frameworks Apps 27 • Simple & portable packaging format • Tasks isolation
- 28. Demo 28
- 29. Bibliography Ispired from 29 • Apache Mesos and Mesosphere: Live webcast by CEO and Co-Founder Florian Leibert http://fr.slideshare.net/mesosphere/apache-mesos-and-mesosphere-live-webcast- by-ceo-and-cofounder-florian-li • Apache Mesos at Twitter (Texas LinuxFest 2014) http://fr.slideshare.net/caniszczyk/apache-mesos-at-twitter-texas-linuxfest-2014 • Deploying Docker Containers at Scale with Mesos and Marathon http://fr.slideshare.net/DiscoverPinterest/connor-doyle-mesosphere