Montreal Kubernetes Meetup: Developer-first workflows (for microservices) on Kubernetes
2 likes698 views
1. The document discusses developer-first workflows for building and operating microservices on Kubernetes. 2. It recommends creating self-sufficient, autonomous teams and using Kubernetes, Docker, and Envoy to provide the basic infrastructure primitives needed for distributed workflows. 3. The strategies suggested depend on the service maturity level and include using similar development and production environments for prototyping, implementing software redundancy for production services, and defining service level objectives and network observability for internal dependencies.
Montreal Kubernetes Meetup: Developer-first workflows (for microservices) on Kubernetes
- 1. Developer-first workflows (for microservices) on Kubernetes Richard Li
- 2. datawire.io ● Building a cloud application using microservices in 2013 ● Distributed systems engineers ● Multiple services ● Prototyping was really fast ● … then velocity slowed down by a lot when we started getting users and needing to add more features 2 Microservices at Datawire ...
- 3. datawire.io Part 1: How do I gain the benefits of microservices? (How do I break up my monolith?) (How do I architect my app with microservices?) (What infrastructure do I need in place before I can benefit from microservices?) 3
- 4. datawire.io 4 Stability/Maturity Velocity Prototype Production Mission critical
- 5. datawire.io 5 Stability/Maturity Velocity Prototype Production Mission critical
- 6. datawire.io How do I break up my monolithic architecture? How do I break up my monolithic process into multiple independent processes? 6
- 8. datawire.io 8 Define Code Test Release Prod Centralized process ● Specialized teams ● Fixed policies (e.g., release criteria)
- 9. datawire.io 9 Microservices lets you run multiple processes!
- 11. datawire.io Microservices is a distributed development architecture workflow. 11
- 12. datawire.io 12 Stability/Maturity Velocity Prototype Production Mission critical ● How do I get to Continuous Deployment incrementally? ● How do I limit the scope of PCI (audit process)? ● How do I ship feature X as fast as possible?
- 13. datawire.io Microservices is ... ● Multiple workflows ○ Including your existing workflow! ○ Workflows designed for different stability/velocity tradeoffs ● Simultaneous workflows 13
- 14. datawire.io Doing things this way shifts how people operate! ● Specialists become generalists → design better holistic systems ● Dev vs Ops vs QA → roadmap / API conversations ● Challenging for engineers → More fun! 14
- 15. datawire.io Part 2: How do I create multiple, simultaneous workflows? 15
- 16. datawire.io Create self-sufficient, autonomous software teams. 16
- 17. datawire.io Why self-sufficiency and autonomy? ● Self-sufficient ○ Team does not need to rely on other teams to achieve its goals ● Autonomy ○ Team is able to independently make (process) decisions on how to achieve its goals 17
- 18. datawire.io Centralized specialist functions are a common barrier to self-sufficiency and autonomy 18 Centralized architecture Centralized infrastructure / ops* (You might need a platform team, eventually)
- 19. datawire.io 19
- 22. datawire.io I’m stranded on API island. Do I have to build my workflow from scratch? 22
- 23. datawire.io No! Kubernetes / Docker / Envoy give you the infra you need 23
- 24. datawire.io Docker ● A consistent build/runtime for your service ● Run locally or in Kubernetes cluster ● Massive community & ecosystem ○ 14M Docker hosts ○ 3300+ contributors ○ Lots of third party tools 24
- 25. datawire.io Kubernetes ● Cloud POSIX ● Declarative definition of your cloud infrastructure ● Supported by all major vendors: Google, Microsoft, Mesos, Docker, Oracle, Red Hat, IBM all support it ● … except for Amazon. ○ Yet 62% of K8S workloads on AWS 25
- 26. datawire.io Envoy ● Modern L7 proxy, designed for cloud applications ○ Resilience: global rate-limiting, circuit breakers ○ APIs for managing fleets of Envoys ○ L7 observability (incl distributed tracing) ● Fast growing ecosystem & community ○ Maintained by Lyft ○ Google, IBM, Apple, Datawire, Verizon ○ CNCF project 26
- 27. datawire.io Cloud Native Compute Foundation 27 ● Istio ● SPIFFE ● App dev
- 28. datawire.io Summary 1. Microservices is a distributed workflow: multiple & simultaneous processes. 2. Start building your distributed workflow by creating a self-sufficient, autonomous team. 3. Kubernetes / Docker / Envoy give you the basic operational primitives you need. 28
- 29. datawire.io Part 3: How do I actually use these technologies to build my workflow? 29
- 30. datawire.io 30 Monolith Prototype new feature, FAST
- 31. datawire.io 31 Stage 1: Prototyping Existing workflow: Designed for mature app, not prototyping Challenge: Prototyping workflow that is fast but not one-off Strategies: ● Dev infra ~= Prod infra ● Self-service operational infrastructure
- 32. datawire.io Strategy: Dev infrastructure ~= Prod infrastructure 32 ● End-to-end ○ Have developers access the service the same way as end users (e.g., through an API Gateway, ingress, etc.) ● Environmental consistency ○ Insure that your dev runtime is the same as your initial production runtime ○ Environmental differences between dev & prod will cause bugs. So eliminate them upfront. Your dev infrastructure should resemble your production infrastructure as much as possible.
- 33. datawire.io Strategy: Self-service operational infrastructure 33 Infrastructure needs to be configured by the service team, not by operations (otherwise, there’s a bottleneck.) Service team needs to be able to publish new services, without operations. Service team has access to the Kubernetes manifest (maybe they use a standard template).
- 34. datawire.io Implementing Stage 1 strategies ● Eliminate environmental differences ○ Containerize your service and use the same container for dev & prod ○ Template your Kubernetes manifests so you can use the same manifest for dev & prod ● End-to-end ○ Use an API Gateway for dev and prod (you can even use the same one) ● Self-service operational infrastructure ○ Expose the right configuration options, in a rational way, for your infrastructure to the service team ■ Kubernetes manifest ■ Configuring your API Gateway 34
- 35. datawire.io 35 Monolith Now in production, and need to ITERATE
- 36. datawire.io 36 Stage 2: Production users Existing workflow: App-level integration testing & roll out; dedicated QA team; unit tests Challenge: Update service quickly, while not letting bugs impact the end user Strategies: ● Software redundancy
- 37. datawire.io Strategy: Multiple versions for software redundancy 37 End user Primary version Canary version Dev version
- 38. datawire.io 38 Stage 3 Internal service consumption Existing workflow: None Challenge: Communicate to consumers the service contract, and support that contract in the service Strategy: ● Service level objectives ● Network-level observability & resilience
- 39. datawire.io ● Define the target level of your service as part of your API ● Common SLOs relate to throughput, latency, and availability ○ This API will support up to 5,000 requests per second per consumer ○ The expected latency at p50, p90, p99 ● Simple implementation of SLOs can be via monitoring Strategy: Service level objectives 39
- 40. datawire.io Strategy: Network level observability & resilience ● Observability ○ Figure out which service is causing the problem ○ Need visibility at a network level ○ Distributed tracing (e.g., Jaeger, Zipkin) ● Resilience ○ Avoid cascade failure ○ Network-level rate limiting, circuit breaking, etc. ● Service mesh ○ Strategy for implementing network-level observability & resilience that’s transparent to end users ○ Istio just proposed to the CNCF 40
- 41. datawire.io Summary 1. Microservices is a distributed workflow. 2. Start building your distributed workflow by creating a self-sufficient, autonomous team. 3. Kubernetes / Docker / Envoy give you the basic operational primitives you need. 4. Your workflow requirements depend on your service maturity (prototype, production, internal dependencies). 5. Strategies for building your workflow a. Prototype: Dev ~= Prod, self-service operational infrastructure b. Production: Software redundancy c. Internal dependencies: Service level objectives, network-level observability/resilience 41
- 42. datawire.io So how does this work in practice? 42
- 43. datawire.io Your traditional monolith workflow ... ● Intended for iterative improvement & prevent downtime ○ Code locally in dev environment ○ Pull request ○ Branch for release ○ QA branch & integration test on staging environment ○ Release on production environment ● Different environments, specialized teams 43
- 44. datawire.io Versus microservice infrastructure & workflow ... ● Eliminate environmental differences ○ Container -- same run time environment ○ Kubernetes -- same deployment environment ● Self-service operational infrastructure ○ Templated Kubernetes manifest ○ Self-service API Gateway ● Multiple versions for software redundancy ○ Facilitated by API Gateway / router 44 Will let you prototype / launch / update a microservice with a workflow that’s tailored to feature development.
- 45. datawire.io Thank you! ● [email protected] ● Getting ready to launch hands-on tutorials on building your own distributed workflow. If you’re interested in being notified, sign up here: ○ https://d6e.co/montrealk8s ● If you’re interested in any of our open source tools, check them out: ○ https://forge.sh for deployment ○ https://www.telepresence.io for real-time live coding ○ https://www.getambassador.io self-service API Gateway built on Envoy 45
- 46. END