Journey to Docker Production: Evolving Your Infrastructure and Processes - Bret Fisher, Independent Docker Consultant
4 likes•857 views
Bret Fisher's document provides insights on effectively transitioning to Docker production, emphasizing the importance of starting small and incrementally building a production environment. Key points include avoiding common Dockerfile anti-patterns, focusing on foundational stability, and leveraging container orchestration with Docker Swarm for scalability. The document ultimately aims to simplify the containerization process for organizations by providing practical advice and encouraging agility in infrastructure planning.
Journey to Docker Production: Evolving Your Infrastructure and Processes - Bret Fisher, Independent Docker Consultant
- 1. Journey to Docker Production Bret Fisher Independent DevOps Sysadmin Docker Captain
- 2. • Docker Dev+Ops user since 2014 • Used for my co-founded startup • Now, fill time container ops indi consultant • Helped over dozen orgs go prod Who Am I?
- 3. • You want Docker in production • Containerizing existing infrastructure/apps • Need to break down barriers of getting into Docker production • Often these are "decision barriers", lets make some good defaults • Sometimes it's made-up requirements that aren't required day one • Hopefully I will help you today with all of these! Why Are We Here?
- 4. • Containerize As-Is • What To Focus On First • Building Prod Layer by Layer • Agile Infrastructure Planning • Other Ways To Make An Impact Agenda
- 5. Production Docker: A New Hope "Containerize as-is!"
- 6. New to containers? Limit other changes Sounds obvious, but many container projects start out too big in scope Problems you likely don't need to solve before container day one • Fully automatic CI/CD • Dynamic performance scaling • Containerizing all of an existing solution at once • Service discovery (dynamic updates to LB's, DNS, etc.) • Starting with persistent data Limit Your Simultaneous Innovation
- 7. • Microservice conversion isn't required • 12 Factor is a horizon we're always chasing • Don't let these ideals delay containerization • Maybe we need a 12 Factor 12-step program Legacy Apps Work In Containers Too At its core, a container is just a process
- 8. You need a good foundation What To Focus On First
- 9. is the path to the dark side A Bad Dockerfile...
- 10. Good Dockerfiles are more important than fancy orchestration • It's your new build and environment documentation • At first, focus on "it works" rather then fancy or lean (or multi-stage) • Use FROM Official distros that are most familiar What To Focus On First: Dockerfiles
- 11. Dockerfile Production Anti-patterns New teams often have anti-patterns or wrong defaults Lets look at some of these shall we:
- 12. • Anti: Not storing unique data in volumes Dockerfile Anti-pattern: Trapping Data 👍😺 • Not to be confused with Bind Mounts for sharing data with host
- 13. • Anti: Letting image builds pull FROM latest Dockerfile Anti-pattern: No Version Pinning 👍😻 • Anti: Letting image builds install latest packages • You don't do this with your code, why do it with dependencies?
- 14. • Anti: Not changing defaults in container like you would on a VM • e.g. php.ini, mysql.conf.d, java memory Dockerfile Anti-pattern: Leaving Default Config 👍😽 • NOTE: Don't just copy over config files from VM's into image
- 15. • Anti: Copying in environment config at image build Dockerfile Anti-pattern: Environment Specific 👎🙀 • Goal: Single Dockerfile with default ENV's, and overwrite per- environment at run time
- 16. Building Prod Layer by Layer
- 17. • Stick with what you know at first • Do some basic performance testing of your existing and planned setup. Even if just defaults with sample data. You will learn lots! • Recent Docker Inc. and HPE whitepaper on MySQL benchmark • (authored by yours truly, and others) • bretfisher.com/dockercon17 Containers-on-VM or Container-on-Bare-Metal Do either, or both. Lots of pros/cons to either
- 18. • Docker is kernel and storage driver dependent • Innovations/fixes are still happening here • "Minimum" version != "best" version • No pre-existing opinion? Ubuntu 16.04.2 LTS (latest LTS) • Popular, well-tested with Docker • One of latest LTS kernels out there • Wide storage driver support • Get correct Docker for your distro from store.docker.com Linux Distribution/Release Matters
- 19. • Don't make a decision based on image size (remember it's SIS) • At first: match your existing deployment process • Consider changing to Alpine later, maybe much later Container Base Distribution: Which One?
- 21. • General advice on how to start small and grow big • Based off learnings from • Docker internal testing • Docker reference architectures • Real world deployments • Swarm3k lessons learned Good Defaults: Swarm Architectures
- 22. • "docker swarm init" done! • This happens every day with VM's, why not with containers • Why bother with swarm? Keeping same workflow across environments, secrets, etc. 1-Node Swarm Baby Swarm
- 23. • All Managers and Workers • One node can fail • Great when on a very small budget • pet projects • test/CI setups 3-Node Swarm Minimum for HA
- 24. • All Managers and Workers • Two nodes can fail • My minimum for production that affects business $$ 5-Node Swarm Better High-Availability
- 25. • 5 dedicated managers • Maybe some workers in DMZ • Anything beyond 5 nodes, stick with 5 Managers and rest workers • Control container placement with labels + constraints 10-Node Swarm Separating Out Managers
- 26. > docker node update --label-add data-drive=ssd worker1 > docker node update --label-add net-segment=dmz1 worker2 Constraints with Swarm Services > docker service create --constraint node.labels.data-drive==ssd mysql
- 27. • 5 dedicated managers • Resize managers as you grow • Multiple worker subnets on Private/DMZ • Control container placement with labels + constraints 100-Node Swarm Aww she's all grown up!
- 28. • Different hardware configurations • Different subnets or security groups • Solution: Use node labels with container constraints Not-So-Good Reasons for Multiple Swarms
- 29. • Geographical boundaries between nodes (e.g. different regions) • Security boundaries for compliance (PCI, etc.) • Personnel boundaries for Swarm management using Docker API • (could mitigate Docker EE UCP, auth plugin) Good Reasons for Multiple Swarms
- 30. Do You Have An Externally Driven Deadline?
- 31. • Beware the "Not Implemented Here" Syndrome • Similar to "not invented here" • Opportunities for Outsourcing • Where it can still be a challenge to implement and maintain • And, where the SaaS/On-prem market is mature • For Your Consideration • Image registry • Log aggregation, storage, search, and alerting • 17.05 gets service logs, good for getting started • Monitoring and alerting Outsource Well-Defined Plumbing
- 32. Other Docker Products that Compliment Docker CE • Swarm/Stacks/Secrets/Routing Mesh • CLI Management Docker for AWS/Azure • Cloud Resource Template • Moby Instance OS Docker EE • DTR/UCP • GUI Mgmt/RBAC/Layer 7 Proxy/Image Cache More Effort More Decisions Less Effort Less Decisions
- 33. Remember To Be Agile, You'll Build The Next One Better
- 34. • Already have good infrastructure automation (one click VM)? • Maybe you have great VM autoscale with single-service VM's? • Like the security boundary of the VM OS? You Want Even Less Ops Change? Lets accelerate this even more
- 35. • Least amount of infrastructure change while letting you: • Run on Dockerfiles recipes rather then Puppet etc. • Improve your Docker management skills • Simplify your VM OS build • Moby Project & LinuxKit will likely make this more popular • Windows is doing similar with Hyper-V Containers • Linux is doing similar with Intel Clear Containers One Container Per VM Why don't we talk about this more?
- 36. Other Ways To Make An Impact
- 37. • Use containers for tech support mock environments • Temp test environments, customer demos • Any process where you "wait for human to deploy/configure VM" Other Ways To Make An Impact Docker can be used internally and still add value
- 38. Thank You! My New Docker Udemy Course 90% off for DockerCon: dockermastery.com slides: bretfisher.com/dockercon17