Microservices at Scale with Istio

Editor's Notes

• #5: Functions, monitoring, messaging, many managed services
• #6: Audience check
• #7: I’m not here to tell you that monolithic apps are bad. In fact, nice and simple.
• #8: The problem is, they typically contain a lot of complexity and moving parts, all tightly coupled into a blob of stuff Any one change effectively touches the entire body, and everything needs to be tested. This can be expensive. Typically, features get tacked on, along with bug fixes, and that’s where we end up with these long tails on releases. That’s not the best way to get features and fixes out the door fast.
• #9: Microservices allow development teams to deploy highly portable services But, these services at scale can put an incredible burden on Devops teams responsible for keeping the applications healthy
• #10: Microservices allow development teams to deploy highly portable services Features can be decupled from each other, dependencies swapped in and out as needed And, features and components can be implemented in langauges that fit them – maybe reports is in java, and your cart is ruby
• #11: Microservices allow development teams to deploy highly portable services But, these services at scale can put an incredible burden on Devops teams responsible for keeping the applications healthy
• #12: Istio a service mesh that allows us to connect, secure, control and observe services at scale, often requiring no service source code modification What is a service mesh, and what does all of this mean? We will get there, but first some history for context
• #13: When computing initially became ubiquitious in modern business environments, systems were both hardware and software, purpose- built by a vendor Programmers created proprietary software applications and services, with proprietary tools, often leveraging other proprietary software applications Systems analysts are kind of the first DevOps - sometimes they build the system, as well!
• #14: So let’s talk briefly about DevOps as a methodology This were a lot of moving parts in the classic Old World IT Dept. More mass means more friction, and with the advent of web applications and services, time to market became even more critical But, there's a lot of value in how we were doing things. A shift was needed, but one that didn't lose all we had gained Eventually a new methodology began to take hold which had a goal of reducing the friction, while preserving the useful expertise and painful lessons learned
• #15: Advent of DevOps DevOps buidls a bridge between development and operations, preserving and sharing expertise between the disciplines DevOps is as much a cultural shift as it is a technical one
• #16: Microservices – iterative development, rapid release, super fact pivot and time to market CI / CD - if you’re going to release fast, you need a system to keep up with it Cloud adoption - while not strictly required, at this point… yeah. Containers. - just as with microservice architecture and ci / cd – this was a good fit for the methodology these technologies became prolific and ubiquitous because they were needed by the methodology, we did not adapt the methodology to fit the tech
• #17: A great example of a need driving innovation…. Docker disrupted how we build and ship software Rather than installing binaries on a set of painfully curated hosts, <bespoke, artisinal host environments> now we just "use docker” Application binaries are bound with runtime dependencies in a portable environment that can run anywhere Containers on Linux were kind of a mess before Docker, which gave a workable substrate and common abstractions
• #18: Portability is accomplished by decoupling the app from the host where it runs If we lose sight of the host, where compute environment hits the metal (or vm), how do we run containers at scale?
• #19: Kubernetes provides abstractions for deploying software in containers at scale Again, out of necessity - containers were everywhere, and various orchestration options arose. Mesos, Docker Swarm, others… Kubernetes won
• #20: Infrastrcture resources are abstracted in a cluster of worker nodes, and the cluster has a scheduler which deploys work to those nodes So, everything we need…
• #21: so now rather than a monolithic application, running in bespoke compute environments
• #22: this is day one. what do we pick up on day two? or put another way, what happens when we succeed, and our prototyped happy path software needs to scale?
• #23: this is day one. what do we pick up on day two? or put another way, what happens when we succeed, and our prototyped happy path software needs to scale?
• #24: <nice k8s image duplicated over and over, then add fires from docker slide> yesterday we had 5 or 10 microservices all humming along happily, where we can actually still put our hands on them if we wanted today we might have hundreds or thousands of services running, at different versions, across multiple environments, in multiple clouds putting a finer poitn on this, how do you know where the problem is when user latencies spike for a given feature? maybe that's simple, but maybe requests come into an ingress and run through several services, and then unwind back up the stack all the way back out to the user... how do you know what went wrong where?
• #25: We need a way to simply and repeatably deploy, and simply and recoverably modify. Kubernetes has our backs there. We need to re-establish telemetry, observability, and diagnosibility as table stakes for computing at scale. Here, we need to bring some stuff to the party.
• #26: ingress and traffic managemnet, e.g. nginx - how do we get traffic from the user to the application? tracing and observabilty, e.g. opentracing - how do we see how the various services which make up the application work together, or their current status? metrics and analytics, prometheus - can we surface the data about our overall system performance in a meaningful way? security, e.g. vault - how do we make sure all of these services get the resources they need, sometimes through secure means, and ensure they aren't bad guys?
• #27: abstractly, once we move from on-prem monoliths to containerized microervices running in k8s on the cloud, we still need to address the need for <iterate>
• #28: some do one of these things, maybe well these are hard problems
• #29: Service mesh is an infrastructure layer for controlling and monitoring service-to-service traffic in your stack This layer has both a data plane, deployed alongside application services, and a control plane which is used to manage the mesh
• #30: A service mesh is an ideal component in a DevOps environment, as it provides operators with a stable and extensible platform for all of the work needed to maintain and improve the platform, while it remains completely invisible to developers
• #31: Service Mesh This is not a new solution which solves all the world's problems It allows for integration of all existing (and future) best in class solutions for All the Things First time I’ve used that gif unsarcastically So, how does it do that? For that…
• #32: Ok, so now we know what this stuff is… , Istio is a service mesh that sits in the path between services, allowing for transparent telemetry, policy and security
• #33: fine-grained traffic control with rich routing rules, retries, failovers, and fault injection Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress
• #34: security - a strong identity-based authentication and authorization layer which is secure by default for ingress, egress and service-to-service traffic policy - layered over all of this is a pluggable policy engine supporting access controls, rate limits and quotas typically we get all of this with little to no application code changes
• #35: Envoy we’ve talked about this, the proxy pilot converts high level routing rules that control traffic behavior into Envoy-specific configurations, and propagates them to the sidecars at runtime Mixer Mixer enforces access control and usage policies across the service mesh, and collects telemetry data from the Envoy proxy and other services citadel provides strong service-to-service and end-user authentication with built-in identity and credential management.
• #36: What does this look like: two services talking to each other – HTTP GET, simple
• #37: What does this look like: Magic happens in the data plane IP Tables rules are automated to intercept all service traffic and reroute to proxy The proxy has rules and policies to follow, and after considering policy, routing rules, etc, it forwards the traffic to the appropriate service Let’s talk about that little proxy box
• #38: We should talk about Envoy for a minute. Definitely could be its own talk, and there are many out there to check out. Envoy is a good example of Istio surfacing other features of a best-in-class component through its mesh .
• #39: Ok so, back to this
• #40: HTTP HTTP/2 supported, gRPC, or anything over TCP… with or w/o mTLS
• #41: pilot managing the proxies Mixer handling enfocement and telemtrey pickup citadel authN authZ
• #43: We get most of this for free
• #44: Differentiate API gateway – primaritly north-south, vs service mesh east-west Istio has gateway’s which provide ingress for the mesh Betyond that, a lot of day-to-day becomes really simple – canary, traffic mixing for blue/green, AB testing
• #45: More on testing Integrated benchmarking virtually free, making it incredibly easy to catch version-to-version regression
• #46: And all of this is safe out of the box, secure by default in depth with multiple components