DevOps 101+: From collaboration to microservices
Download as PPTX, PDF1 like1,071 views
The document explores the evolution of DevOps, focusing on the transition from traditional IT practices to microservices and containerization. Key themes include the importance of culture, automation, and measurement in fostering agile development, as well as the current fragmentation in DevOps tools and practices. It highlights that while adoption rates are increasing, significant challenges remain in terms of speed and resource dependency.
![33
Loosely coupled services
“ The only
communication allowed
[at Amazon] is via
service interface calls
over the network.”
– Steve Yegge, Google, Oct 2011,
paraphrasing Jeff Bezos memo
https://plus.google.com/+RipRowan/posts/eVeouesvaVX](https://image.slidesharecdn.com/dberkholzosndevops-160609201253/85/DevOps-101-From-collaboration-to-microservices-33-320.jpg)
DevOps 101+: From collaboration to microservices
- 1. DevOps 101+: From collaboration to microservices Donnie Berkholz, Ph.D. Research Director — Development, DevOps, & IT Ops Open Source North, June 2016
- 3. 3 Source: 451 Research/Microsoft Cloud+Hosting commissioned research Minimizing risk, maximizing agility
- 4. The need for speed 4 Source: 451 DevOps study, Q3 2014; n=237 63% want more
- 6. 6 Source: 451/Microsoft Hosting + Cloud Study 2015
- 7. The new stack? 7 An infinite array of possible stacks.
- 8. Polyglot programming There’s no obvious choice for the right language, based on community adoption. 8 Donnie Berkholz Source: http://redmonk.com/dberkholz/2014/05/02/github-language-trends-and-the-fragmenting-landscape/
- 11. DevOps: Putting IT into high gear 11
- 12. 12 Agile, truly tip to tail Business to customer
- 13. 3 pillars of DevOps 13 Culture Automation Measurement
- 14. 14 Culture: Tear down all the silos Flickr: kalandrakas
- 15. 15 Flickr: respresFlickr: hartvig, snapeverything, roymaloon Automation: Pets vs Cattle
- 16. Automation: Infrastructure as code 16 Wikipedia: Magnus Manske
- 17. Knight Capital and the $460 million bug 17 Wikipedia: Jericho
- 19. Where are we today? 19 10.0% 54.7% 27.9% 6.8% 0.7% Highly Manual Manual with Limited Automation Tools Automated with Manual Exception Handling Policy Based Automation and Orchestration Other n = 843 Source: 451 VotE Cloud, Q3 2015
- 21. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% < 250 employees 250-999 employees 1,000-9,999 employees >10,000 employees Agile adoption: still not universal 21 451 Research, Voice of the Enterprise: Software-Defined Infrastructure, Q4 2015 (n=670)
- 22. 0% 10% 20% 30% 40% 50% 60% < 250 employees 250-999 employees 1,000-9,999 employees >10,000 employees DevOps adoption: resource-dependent? 22 451 Research, Voice of the Enterprise: Software-Defined Infrastructure, Q4 2015 (n=568)
- 23. DevOps tools in use still vary widely 23 33.83% 37.31% 39.30% 39.30% 40.80% 40.80% 44.28% 45.77% 51.24% 51.74% 63.18% Infrastructure as a Service Build and continuous integration… Application… Traditional middleware and… Release management Platform as a Service QA planning and automation tools Configuration management &… Performance Monitoring and… Project management tools Testing Source: 451 Research/Red Hat, Q1 2016, n=201
- 24. 6.0% 27.9% 34.3% 23.4% 3.5% 1.0% 3.5% 0.5% 0% 5% 10% 15% 20% 25% 30% 35% 40% Release speed still lags demand 24 Source: 451 Research/Red Hat, Q1 2016, n=201
- 25. Enter containers: The future of virtualization
- 26. 26 Aren’t they just like VMs? No. Source: 451 Research, “Now Shipping: The Docker and containers ecosystem rapidly takes shape”
- 27. Containers vs VMs: no clear approach 27 451 Research, Voice of the Enterprise: Software-Defined Infrastructure, Q4 2015 10.9% 14.6% 9.0% Containers Run Separately from VMs Containers Run On Top Of VMs Containers Are Replacing VMs n = 458
- 30. 56.1% 10.7% 3.9% 4.2% 2.1% 22.9% 31.5% 10.2% 8.4% 9.4% 4.7% 35.8% Discovery and Evaluation Running Trials/Pilot Projects In Test and Development Environment Initial Implementation of Production Applications Broad Implementation of Production Applications No Plans Q1 2015 Q3 2015 Docker is not just a toy 30 14.1%} Source: 451 VotE Cloud, 2015; Q1 n=991; Q3 n=960 of cloud-using orgs Prod in 3Q15:
- 31. 56.1% 10.7% 3.9% 4.2% 2.1% 22.9% 31.5% 10.2% 8.4% 9.4% 4.7% 35.8% Discovery and Evaluation Running Trials/Pilot Projects In Test and Development Environment Initial Implementation of Production Applications Broad Implementation of Production Applications No Plans Q1 2015 Q3 2015 Docker is not just a toy 31 Source: 451 VotE Cloud, 2015; Q1 n=991; Q3 n=960 32.7%}of cloud-using orgs Pilot+ in 3Q15:
- 32. Fragmentation drives microservices — enabled by containers 32
- 33. 33 Loosely coupled services “ The only communication allowed [at Amazon] is via service interface calls over the network.” – Steve Yegge, Google, Oct 2011, paraphrasing Jeff Bezos memo https://plus.google.com/+RipRowan/posts/eVeouesvaVX
- 34. Loosely coupled teams “ One of the biggest changes is that we no longer have an official ‘architecture’ team. Instead, we have made ‘architecture’ an ‘ingredient’ on each of our teams.” 34 http://tech.gilt.com/post/102628539834/making-architecture-work-in-microservice – Lauri Apple, Gilt Groupe, 14 Nov 2014
- 35. The foundation of microservices 35
- 37. Container orchestration is limited (∴ adoption immature) 37 451 Research, Voice of the Enterprise: Cloud, Q3 2015 9.4% 36.1% 39.9% 14.6% Currently use Considering using in the next two years Not familiar with these tools Have no plans to use in the next two years n = 534
- 43. Complexity is the new normal 43
- 44. Real-world example #2: REA (realestate.com.au) 44 http://techblog.realestate.com.au/a-microservices-implementation-retrospective/
- 45. REA microservices timeline 45 0 20 40 60 0 6 12 18 24 Microservices Months http://yowconference.com.au/slides/yow2014/SkurrieBottcherEvans-MonolithsToMicroservices.pdf “ Microservices is a long term strategy.” – Evan Bottcher, ThoughtWorks/REA, 9 Dec 2014
- 46. Real-world example #3: Ctrip (Chinese travel site) 46 http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11
- 47. Real-world example #3: Ctrip (Chinese travel site) 47 http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11
- 48. Real-world example #3: Ctrip (Chinese travel site) 48 http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11
- 49. The cloud-native movement is just about to take off 49 32% 32% 13% 13% 9% Developing and running web-based applications Migrating legacy workloads and applications to the cloud Developing and running cloud native applications Managing legacy workloads, applications and assets on the cloud Testing new technologies and methods Source: 451 Research/Red Hat, Q1 2016, n=201
- 50. From primitives to platforms 50 ServerlessPaaSCaaS Container orchestration IaaS / Containers OpinionatedFlexible
- 51. How? DevOps (Culture, Automation, Measurement) What? Microservices Why? Survival 51
- 52. 52 Thank you! Donnie Berkholz Twitter: @dberkholz [email protected]
- 53. Some content from this presentation is Creative-Commons licensed. https://creativecommons.org/licenses/by-sa/ https://creativecommons.org/licenses/by/ 53
- 54. Minimizing risk, maximizing agility Architecture: Microservices, composable monitoring Code: Continuous integration, feature flags Servers: Continuous delivery, infrastructure as code Services: Rolling updates, resilience engineering Product: Continuous deployment, restricted audience 54
Editor's Notes
- #3: 0 talks at Glue last year, 12 this year 1 proposal at OSCON last year, 30 this year Intro myself and credibility re containers/microservices
- #4: Q: “Allocate 100 points among the following five goals as they relate to your company or organization?” Overall business goals Enabled by cloud, open source, self-service IT
- #5: Does your company struggle with increased demands for shorter release cycles, with business managers expecting weekly, daily or even hourly releases and updates?
- #6: Cathedral, indulgences to bazaar Open source, cloud, DigitalOcean Shadow IT: IT as service provider, as vendor Changing sales models – 7 red lines, demand gen (content mktg) vs lead gen
- #7: This one was hosting and cloud service providers
- #8: Languages, databases, frameworks … What’s driving the way we build technology?
- #11: Excepting Angular and React
- #13: OODA loops
- #14: Culture: Collaboration, Sharing Automation: Digital Measurement: KPIs As compared to ITIL: push down authority, plan iteratively, peer review, collaboration, automation
- #15: Why? Empathy drives quality Internal customers Barriers slow delivery
- #16: Why? Replicable – env parity Scalable Accelerated
- #17: Configuration management CFEngine 1.0: 1993
- #19: Developers vs DevOps Release management – quarters to weeks to days to hours Gary Gruver, HP Little’s Law – work in system proportional to arrival speed plus time in system. Reduce batch sizes. Lean.
- #20: For cloud users: In general how would you describe your organization’s workload or service provisioning process?
- #21: Why? Can’t improve what you can’t measure Cattle vs pets (scale, transience) Nagios (NetSaint): 1999 (MN co) Community, UX
- #22: Overall 65.1%
- #23: Overall 39.6%
- #24: Data, not just code Note this is screened for DevOps-leaning audience Most orgs doing testing, but few automated pipelines
- #25: Q: “Approximately how often do you deploy software applications to production?” Audience: People influential in cloud-services development
- #26: The next step in DevOps How do we cope with these demands for agility, scalability, automation, transience?
- #28: Q30. What is your organization’s strategy (if any) around the use of containers (e.g. Docker)? How does this change the infrastructure you’ve invested in for virtualization?
- #29: VotE shows most orgs are largely moved to virtualization, a minority to automation, few to orchestration/private cloud Immutable infrastructure
- #30: Make sure to clarify difference between Docker and K8s
- #31: Of cloud-using orgs
- #32: Of cloud-using orgs
- #34: Business-defined separations. Bounded context based on cross-organizational empathy. Steve Yegge memo — Amazon must be SOA, or you’re fired.
- #35: DevOps + microservices Bounded contexts, empathy defined DevOps is how you build and run microservices.
- #36: Nomad out of HashiCorp, new competitor to Mesos/Kubernetes Azure Container Service, building on Docker & Mesos. Beta by EOY 2015 Also note PaaS providers moving to containers
- #37: Windows catching up quickly with DevOps and containers No need to leave the Microsoft half of your environment behind
- #38: Q16. What are your organization’s plans regarding a container orchestration tool?
- #40: Hard to add new features, slow provisioning, SPOF
- #45: Worked well: template project, resilience, idempotence, automation, exposing verbs Wrote their own testing library to test consumers Problems: right-sizing, code sharing (inheriting from common git repo and adding files worked)
- #47: Fit service into existing infra, don’t fit infra into service Conway’s law – services fit org Skill, ops overhead, complexity, testability
- #48: Fit service into existing infra, don’t fit infra into service Conway’s law – services fit org Skill, ops overhead, complexity, testability
- #49: Fit service into existing infra, don’t fit infra into service Conway’s law – services fit org Skill, ops overhead, complexity, testability
- #50: Q: “What are the top two uses of cloud-based platforms in your organization?” Audience: Those responsible for or influencing cloud-services decisions
- #51: Most private PaaS options have bought into containers at this point
- #52: The future of service delivery.
- #53: DevOpsDays Minneapolis, July 20-21. $249, or $199 if you’re a startup, self-financed, etc