Open Source Toolchains to Manage Cloud Infrastructure
11 likes1,613 views
The document discusses the use of open source toolchains for managing cloud infrastructure, featuring insights from Mark R. Hinkle, a VP at Cloud.com. It covers the importance of automation in cloud management, the types of management tools available, and challenges such as Jevons's Paradox that lead to increased consumption with improved efficiency. The presentation advocates for integrating various open source tools to optimize cloud operations and emphasizes the necessity of automation to ensure efficient and responsive cloud management.
Open Source Toolchains to Manage Cloud Infrastructure
- 1. Open Source Toolchains to Manage Cloud Infrastructure Mark R. Hinkle VP of Community Cloud.com Twitter: @clouddotcom @mrhinkle Email: [email protected] [email protected]
- 2. 2 Introduction • Responsible for Driving Adoption of CloudStack Open Source Cloud Computing Software • Former manager of Zenoss Open Source project 100,000 users, 1.5 million downloads • Former Linux Desktop Advocate (Zealot?) • Former LinuxWorld Magazine Editor-in-Chief • Open Management Consortium Conspirator • Open Desktop Consortium Instigator • Author - “Windows to Linux Business Desktop Migration” - Thomson • NetDirector Project - Open Source Con guration Management Project • Sometimes Author and Blogger at SocializedSoftware.com/NetworkWorld • Start-up junkie, Glutton for punishment
- 3. 3 Agenda • Introductions • Cloud & Systems Management Overview • Open Source Tools for Cloud Management • Creating Open Source Toolchains • Questions (and Hopefully Answers)
- 4. Cloud Computing & Systems Management Overview
- 5. 5 Jevon’s Paradox and the Cloud …the efficiency with which a resource is used tends to increase (rather than decrease) the rate of consumption of that resource. …in cloud computing increased consumption can introduces signi cant management overhead despite higher hardware utilization and other bene ts 1865, the English economist William Stanley Jevons
- 6. 6 Balancing the Equation: Creating the Cloud + =
- 7. 7 Balancing the Equation: Managing the Cloud + ? =
- 8. Automation Balances the 8 Equation • MeatCloud, Can’t Keep up with Cloud Computing • Devops & Agile IT Philosophy • Script Repetitive Tasks • Integrate Tools • Automate, Automate, Automate
- 9. DevOps – Agile Management for 9 the Cloud • Agile, fast moving • Smaller incremental change • Increased coordination across groups (development/operations) • Eye towards Automation
- 10. 10 Management Toolchains Toolchain (n): Monitoring A set of tools where Patching and Provisioning the output of one tool becomes the Con guration input of another tool
- 11. 11 The Myth of the Nines Availability % Downtime per Downtime per Downtime per Year Month Week 99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes 99.95% 4.38 hours 21.56 minutes 5.04 minutes 99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes 99.999% ( ve nines) 5.26 minutes 25.9 seconds 6.05 seconds 99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds Average polling interval for monitoring? 5 minutes? Even superhuman operations people can’t be alerted and take action in under 5 minutes. One outage per year could drop service level to three nines or worse.
- 12. Open Source Tools to Manage the Cloud
- 13. 13 2011 CLOUD COMPUTING OUTLOOK SURVEY http://www.cloud.com/cloud-computing-outlook
- 14. 14 4 Types of Management Tools Provisioning Installation of operating systems and other software Configuration Management Sets the parameters for servers, can specify installation parameters Orchestration/Automation Automate tasks across systems Monitoring Records errors and health of IT infrastructure
- 15. 15 Cloudy Tools • Network Capable • Cloud “Aware” • Easy-to-Integrate • Adhere to Open Standards • Lend Themselves to Automation
- 16. Open Source 16 Provisioning Tools Year Started Language License Installation Targets Cobbler 2007 Python GPL Red Hat, OpenSUSE Fedora, Debian, Ubuntu Fully Automatic 2000 Perl GPL Debian Installation (FAI) Kickstart ? Python GPL Most .dep and RPM based Linux distros Spacewalk 2008 Perl, Python, Java GPL Fedora, Centos Crowbar/Chef 2011 Ruby Apache ?
- 17. Open Source 17 Configuration Management Tools Year Started Language License Client/Server Bcfg2 2003 Python BSD Yes Cfengine 1993 C Apache Yes Chef 2009 Ruby Apache Chef Solo – No Chef Server - Yes Puppet 2004 Ruby GPL yes
- 18. Open Source 18 Monitoring Tools Year Started License Language Type of Collection Monitoring Methods Cacti (RRDTool) 2001 GPL PHP Performance SNMP, syslog Nagios 1999 GPL C/PHP Availability SNMP,TCP, ICMP, IPMI, syslog OpenNMS 2000 GPL Java Availability/ SNMP, Performance Zabbix 2001 GPL C/PHP Availability/ SNMP, TCP/ICMP, Performance and IPMI, Synthetic more Transactions Zenoss 2005 GPL Python Availability, SNMP, ICMP, SSH, Performance, Event syslog, WMI Management
- 19. Open Source 19 Automation/Orchestration Tools Year Language License Client/ Support Started Server Organization Capistrano 2006 Ruby MIT Yes None RunDeck 2010 Java Apache Yes DTO Solutions Func 2007 Python GPL Yes Fedora Project MCollective 2009 Ruby Apache Yes PuppetLabs
- 20. 20 Cloud APIs Aren’t Created Equal Open Source Abstractions API Language jclouds java deltacloud python Lib cloud python Fog Ruby
- 22. 22 Toolchain Concept Source:http://code.google.com/p/devops-toolchain/wiki/ArchitectureComponents
- 23. 23 Automated Toolchain Capistrano Command and Application Service RunDeck Fabric Control Orchestration Func Provisioning Activity BCFG2 Con guration Cfengine System Con guration Chef Puppet Cloud: OS Install: Eucalyptus Kickstart Cloud Image OS OpenStack Cobbler Bootstrapping Launch Install CloudStack Spacewalk Abiquo
- 24. 24 Example: GeekNet • Servers are automatically built using con guration management software (Puppet) • Discovery tool (Zenoss) nds infrastructure and populates a CMDB then spits out Hundreds of servers, information to scripts that translate serving web, databases, information to BIND con gurations for DNS and other infrastructure • Monitoring tool adds hosts to polling tool for some of the world’s (Nagios) to check servers for availability most highly trafficked • As infrastructure changes systems are updated websites – over 40 million automatically visitors per month. • Servers can be spun up and managed in minutes, not hours automatically with little or no human interaction
- 25. Example Toolchain: 25 Online Gaming Studio
- 26. Questions?
- 27. 27 Contact Me Professional: [email protected] Personal: [email protected] Professional: [email protected] Personal: [email protected] Professional: http://open.cloud.com Personal: http://socializedsoftware.com Mark R. Hinkle VP of Community Twitter: Open Source Enthusiast @clouddotcom @mrhinkle
- 28. Open Source Toolchains to Manage Cloud Computing http://www.slideshare.net/socializedsoftware by Mark R. Hinkle is licensed under a Creative Commons Attribution-ShareAlike 3.0 United States License.
Editor's Notes
- #3: How many users are using
- #6: IN the late 1800s English economist William Stanley Jevons noticed that despite huge increases in the efficiency of coal burning, demand actually increased rather than decreased. In cloud computing despite huge cost savings the consumption of cloud computing is and will continue to grow as demand increases. The downside is that this increased consumption has the ability to increase management burden if users don’t adopt leveraged methods for managing cloud computing.
- #7: With a credit card and a single administrator can spin up 100s of servers in minutes.
- #8: How does the management personnel for the cloud scale? …Automation.
- #10: DevOps is a popular movement among cloud operations professionals who are advocating many of the same methodologes used in Agile software development.
- #11: Ideally for the cloud you create management toolchains that automate the management of your cloud.
- #12: Cloud computing promises highly available systems, but if you have a reactive approach you won’t achieve that goal. Build redundant, highly automated systems that can self-heal.
- #15: Other disciplines like back-up, log management, performance and security (virus,intrusion detection) are important but not core to the delivery of cloud computing systems
- #17: Tools that can automatically build your systems/install systems
- #18: Configuration management tools abstract infrastructure into variables that are implemented via “recipes”Interesting new development is the PXE boot Crowbar extension for Chef Server used to provision virtual machines in OpenStack.
- #21: API lets you develop to one API and move between cloudseasilyTypesof Tasks Accomplished by an APIProvisioning (creating, re-creating, moving, or deleting components e.g. virtual machines, vlans)Configuration (assigning or changing attributes of the architecture such as security and network settings)Cloud ProvidersJclouds – java API Abstraction Libcloud – started by CloudKick (now Rackspace) to abstract clouds, Apache incubator projectDeltacloud – started by Red Hat to abstract clouds, Apache incubator projectFog – started by EngineYard is a ruby gem that allows you to interact with number of cloud API.
- #26: Java applications are developed and checked into Source Control Manager (SCM)Applications built and dependencies resolved via MavenBuilt and unit-tested by HudsonChanges recorded in Change Management Database (CMDB)Deployed by ControlTier (RunDeck)