#SREcon Immutable Infrastructure: rethinking configuration mgmt
5 likes3,162 views
The document discusses the concept of immutable infrastructure and the challenges of configuration management in traditional deployment processes. It emphasizes the shift towards immutable designs that enable faster, safer, and more scalable operations by treating infrastructure as code and allowing platforms like Kubernetes to manage state. Additionally, it highlights different provisioning methodologies and the importance of image creation within automation build processes.
#SREcon Immutable Infrastructure: rethinking configuration mgmt
- 1. Immutable Infrastructure Rethinking Configuration in the Age of Easy Redeployment RackN, Inc February, 2018 Note: Graphics mainly from http://pexels.com <<< Shift Left <<<
- 2. 2 @zehicle #immutable Involved in Open Ops Software: Digital Rebar Project Kubernetes ClusterOps SIG OpenStack Board Your Humble Presenter I’m all about automating infrastructure. Rob Hirschfeld (aka @zehicle) Co-Founder of RackN [email protected]
- 3. 3 @zehicle #immutable Storytime! “Self-Bootstrapping Kubernetes” Kubecon in Nov 2017 we created this demo Simple “immutable” Idea: 1) In Memory Boot Machines 2) Install Docker 3) Elect Leader 4) Run Kubeadm on Leader 5) Run Kubeadm on Remainder But….it’s shockingly hard to maintain. Dependencies breaks the installation And they are constantly changing.
- 4. 4 @zehicle #immutable Storytime! “Self-Bootstrapping Kubernetes” So, while it’s pretty cool, it’s not “real” immutability Presentation & Demo https://youtu.be/OowxF6GqK4I sa !
- 5. Why is configuration fragile?
- 6. Why is configuration fragile? mu on V
- 7. 7 @zehicle #immutable But… I Infrastructure as Code?! Sorry. Mutability adds complexity Traditional “build-in place” approaches ● Have hidden dependency graphs ● Create variation between environments ● Are harder to “lock down” due to config AND OMG… updates and patches are even harder ● Idempotent operations are difficult ● Roll backward is next to impossible! ● Creating indeterminate state
- 8. 8 @zehicle #immutable Traditional “build-in place” approaches ● Have hidden dependency graphs ● Create variation between environments ● Are harder to “lock down” due to config AND OMG… updates and patches are even harder ● Idempotent operations are difficult ● Roll backward is impossible ● Creating indeterminate state But… I Infrastructure as code?! Sorry. Mutability adds complexity Let’s o k w !
- 9. What is Immutable Infrastructure?
- 10. What is Immutable Infrastructure?Pre-de y fi d V
- 11. 11 @zehicle #immutable Traditional Deploy and Configure System is configured in situ from a least common denominator baseline. This can be “immutable-like” under the right conditions. We’ll come back to that... Delivery Pipeline Deployment Code Build Integrate Run Configure
- 12. 12 @zehicle #immutable Shifting Configuration BEFORE Deployment In our ideal delivery pipeline, configuration is before deployment. Running systems are delivered as a complete runnable unit for deployment. Delivery Pipeline Deployment Code Build Integrate Run Configure
- 13. 13 @zehicle #immutable Shifting Configuration BEFORE Deployment In reality, it’s very hard to create a distinct artifact for every running instance; instead, we create incremental versions. So we do some initialization of the reusable versioned instance. Cloud init is the most commonly known pattern for this. Delivery Pipeline Deployment Code Build Integrate Run Configure Ini iz ! V
- 14. Cloud Native Infrastructure CNIbook.info Justin Garrison & Kris Nova “Infrastructure as software”
- 15. 15 @zehicle #immutable Which Enables… Delegating Operations If you can make your artifacts immutable then you can delegate management of them to a platform like Kubernetes. Kubernetes does not configure infrastructure. It maintains state based on a manifest. StateManager (e.g.Kubernetes) Code Build Integrate Run Configure Delivery Pipeline
- 16. 16 @zehicle #immutable Which Enables… Delegating Operations If you can make your artifacts immutable then you can delegate management of them to a platform like Kubernetes. Kubernetes does not configure infrastructure. It maintains state based on a manifest. StateManager (e.g.Kubernetes) Code Build Integrate Run Configure Delivery Pipeline Kub te ? W ! Is ut > K8s?
- 17. Immutable is a DevOps Pattern <<< Shift Left & Create/Delete
- 18. 18 @zehicle #immutable The Problem Immutability <<< Shifting Left package server image provision server initial config
- 19. 19 @zehicle #immutable The Problem Immutability <<< Shifting Left patch 1 package server image provision server initial config
- 20. 20 @zehicle #immutable The Problem Immutability <<< Shifting Left patch 1 patch 2 package server image provision server initial config
- 21. 21 @zehicle #immutable The Problem Immutability <<< Shifting Left patch 1 patch 2 t e d s o n't s o t c 2! package server image provision server initial config
- 22. 22 @zehicle #immutable The Problem Immutability <<< Shifting Left patch 1 patch 2 t e d s o n't s o t c 2! What Madness? ● We have to maintain root access ● Patches assume system state ● Patches create dependency graphs ● Coordination? Should we halt work? ● Drift is inevitable! package server image provision server initial config
- 23. 23 @zehicle #immutable The Problem Immutability <<< Shifting Left patch 1 patch 2 SA !!package server image provision server initial config
- 24. 24 @zehicle #immutable Apply cloud and container lessons to our Bare Metal … Immutability <<< Shifting Left package server image provision server initial config destroy!!
- 25. 25 @zehicle #immutable Apply cloud and container lessons to our Bare Metal … Immutability <<< Shifting Left destroy!! destroy!!patch 1 package server image provision server initial config package server image provision server initial config
- 26. 26 @zehicle #immutable Apply cloud and container lessons to our Bare Metal … Immutability <<< Shifting Left destroy!! destroy!!patch 1 depatch 2 package server image provision server initial config package server image provision server initial config package server image provision server initial config
- 27. 27 @zehicle #immutable Apply cloud and container lessons to our Bare Metal … Immutability <<< Shifting Left destroy!! destroy!!patch 1 patch N depatch 2 package server image provision server initial config package server image provision server initial config package server image provision server initial config package server image provision server initial config
- 28. Cloud like behavior …
- 29. 29 Immutable Provisioning systems treat infrastructure as a black box Cloud-like Integration and Staged Workflow Provisioning System Requested State Returned State REST API Event Hook
- 30. 30 Cloud-like Integration and Staged Workflow Immutable Provisioning systems treat infrastructure as a black box Provision requests are for a system state with optional parameters. The intermediate changes to achieve the state are not exposed to the requester. Provisioning System Reset Join Install Config Test Requested State Returned State REST API Event Hook
- 31. 31 Cloud-like Integration and Staged Workflow Immutable Provisioning systems treat infrastructure as a black box Provision requests are for a system state with optional parameters. The intermediate changes to achieve the state are not exposed to the requester. REMEMBER: Operators of the provisioning system require high transparency, stages and control. Provisioning System Reset Join Install Config Test Requested State Returned State REST API Event Hook No h n op i n !
- 32. Immutable Patterns 1) Baseline + Configuration 2) Live Boot + Configuration 3) Image Deploy
- 33. 33 Provision 1: Baseline + Configuration Benefit: Easiest to achieve with current tools, Safer than Patching Challenge: Lots of Post-Configuration, Not Really “Immutable”, Slow Instead of relying on patches, rely on starting from a pristine image ResetBaseline Configure Run Additional Reference https://thenewstack.io/immutable-hardware-ops-hygiene-security-efficiency/
- 34. 34 Benefit: Fast reset times, forces good behavior Challenge: Provisioning becomes critical path, still have dependency graph Like #1 but clean-up is simply a reboot. Favors smaller footprint O/S. 2: Live Boot + Configuration Provision RebootBaseline Configure Run
- 35. 35 3: Image Deploy Benefit: Shorter time to ready, highly controlled (“shift left”), rollback Challenge: Harder to create and deploy images Image is deployed from source instead of Baseline + Configure Provision Deploy Image Run Provision Deploy Image Run
- 36. 36 3: Image Deploy Benefit: Shorter time to ready, highly controlled (“shift left”), rollback Challenge: Harder to create and deploy images Image is deployed from source instead of Baseline + Configure Provision Deploy Image Run Provision Deploy Image Run Ini iz ! V Ini iz ! V
- 37. 37 So… Let’s talk Image Creation Ideally in an automation build process. You DO THE CONFIGURATION on a live system (so you still need configuration tools) and then capture the image into a portable format. Tools like Hashicorp Packer, Image Builder, WBIC or raw images are used to create source files (e.g. AMI, OVS).
- 38. 38 So… Let’s talk Image Creation Ideally in an automation build process. You DO THE CONFIGURATION on a live system (so you still need configuration tools) and then capture the image into a portable format. Tools like Hasicorp Packer, Image Builder, WBIC or raw images are used to create source files (e.g. AMI, OVS). Tha n li ot of k & re y ow!
- 39. Yes, But… It’s faster, safer & more scalable.
- 40. 40 Build Pipeline Immutable Demo Prep: Image is pre-created from reference system. Reference System Image Read
- 41. 41 Build Pipeline Deploy Immutable Demo Prep: Image is pre-created from reference system. Stage: Boot RAM image and write image to disk(s) Reference System Image Target System RAM BOOT WriteRead
- 42. 42 Build Pipeline Deploy Immutable Demo Prep: Image is pre-created from reference system. Stage: Boot RAM image and write image to disk(s) Deploy: Reboot and run Reference System Image Target System RAM BOOT Target System RUNNING Reboot Write RunRead
- 43. Thank you! Questions? Interested in IMMUTABLE METAL? It’s complicated, but we can get you there. Start at http://portal.rackn.io • Quickstart takes about 30 minutes • Use your own hardware, VirtualBox or Packet.net account – use “RACKN100” on Packet.net for credit