Common primitives in Docker environments
0 likes•1,017 views
This document discusses common primitives needed in Docker environments when deploying applications across multiple machines. It outlines several key primitives like persistence, service discovery, monitoring, logging, authentication and authorization. For each primitive, it describes different approaches, lists relevant open source projects, and provides recommendations on how to implement the primitive in a way that is standardized, scalable and works for both new and legacy applications. The goal is to abstract infrastructure and treat all machines similarly while achieving reliability, reproducibility and reducing manual labor.
Common primitives in Docker environments
- 1. Common primitivesCommon primitives in Dockerin Docker environmentsenvironments Alex Giurgiu ([email protected])
- 3. Until you want to deploy yourUntil you want to deploy your new application in production...new application in production... on multiple machineson multiple machines
- 4. You thought you have thisYou thought you have this
- 5. When in fact you have thisWhen in fact you have this
- 6. We are trying to get hereWe are trying to get here
- 7. This problem is intenselyThis problem is intensely debated at the moment...debated at the moment... with many competingwith many competing projects...projects... thatthat approachapproach it in oneit in one way or another...way or another...
- 8. Just look atJust look at Mesos Google's Omega Kubernetes CoreOS Centurion Helios Flynn Deis Dokku etc.
- 9. What do they have inWhat do they have in common?common? they abstract a set of machines, making it look like its one machine they provide a set of primitives that deal with resources on that set of machines
- 11. Why not use one of the mentioned solutions? Most of them require you to write your application/workload in a custom way. To totally give in to their way of doing things. But (I)we want to run the old/legacy applications, while gaining the same advantages
- 12. Our goals are similar standardize the way we interact with our infrastructure treat all machines in a similar way achieve reliability, through software and not through hardware achieve reproducible infrastructure reduce manual labor
- 13. Our building blockOur building block Container Inputs (binaries,code, packages, etc) External services Build process State
- 14. Common primitivesCommon primitives "common enough that a generalized solution can be devised" "should be applicable to both in-house or external applications"
- 15. Common primitivesCommon primitives persistence service discovery monitoring logging authentication and authorization image build service image registry
- 17. PersistencePersistence one of the hardest problems to solve in a clean and scalable way should be transparent for the application most people just avoid Docker-izing services that require persistence
- 18. LocalLocal - bring the state locally, relative to where the container runs - should be taken care by your deployment/PaaS solution - advantages: write/read speeds, reliability - disadvantages: potentially slow deploys, complex orchestration RemoteRemote - keep state remotely and "mount" it where the application is deployed - can be done by your PaaS solution or by the container itself - advantages: simpler to orchestrate, fast deploys - disadvantages: write/read speeds, (un)reliability
- 19. Projects that try toProjects that try to solve persitencesolve persitence Flocker - https://github.com/ClusterHQ/flocker ?
- 21. Service discoveryService discovery and registrationand registration
- 22. Service discoveryService discovery most worked on aspect of Docker orchestration quite a few different open source projects that tackle this problem multiple approaches: environment variables, configuration files, key/value stores, DNS, ambassador pattern etc.
- 24. Open source projectsOpen source projects Consul (my personal favorite) etcd (CoreOS's favorite) ZooKeeper (many people's favorite) Eureka (Netflix's favorite) Smartstack (Airbnb's favorite) ... (service discovery)
- 25. choose a solution that can accommodate both legacy and custom applications: discovery using DNS or HTTP choose a solution that can be manipulated using a common protocol: HTTP make sure to remove died out applications from your SD system Ideally it should have no single point of failure Consul satisfies all the above requirements How to do itHow to do it (service discovery)
- 26. ConsulConsul (service discovery) can be queried over DNS and HTTP distributed key:value store consistent and fault tolerant(RAFT) fast convergence(SWIM) Service checks
- 27. Service registrationService registration Can be done by your application - simple HTTP call to Consul a separate script/application inside your container another container that inspects running containers - progrium/registrator Most importantly, each container should provide metadata about the service its running.
- 29. MonitoringMonitoring 2 perspectives service monitoring - can be done as in pre-Docker times container monitoring
- 30. Service monitoringService monitoring (monitoring) can be done with tools like Nagios your monitoring system should react dynamically to services that start and stop containers should define what needs to be monitored services should register themselves in the monitoring system Consul supports service checks
- 31. Container monitoringContainer monitoring (monitoring) monitor container state(up/down) - Docker event API provides this information gather performance and usage metrics about each container - Google's cAdvisor provides this cAdvisor provides an API to pull the data out, so you can feed it to your trending system
- 32. Monitoring principlesMonitoring principles (monitoring) have a layer of system monitoring - that trusts humans have a layer of behavior tests - doesnt trust humans. Used to make sure that a certain environment is up reduces manual labor
- 33. enables detailed insights inside the kernel and applications they have a new "cloud" version same thing can be achieved on your private Docker platform SysdigSysdig (DTrace for Linux)
- 34. LoggingLogging
- 35. LoggingLogging logs will be used by engineers to troubleshoot issues ... but now your application is a distributed moving target the need for centralized log aggregation is big
- 36. How to do itHow to do it (logging) Multiple approaches applications write logs to STDOUT and you pick up the logs using the Docker API or client. Logspout can be used to ship the logs remotely applications write logs inside the container and a logging daemon inside the container(RSYSLOG) ships the logs to a centralized location applications write logs in a volume which is shared with another container that runs a log shipping daemon
- 37. How to do itHow to do it (logging) Choose an approach that fits your needs and send the logs to a centralized location logstash-forwarder is a great to forward your logs(please dont choose python-beaver) elasticsearch is a great way to store your logs Kibana is a great way to visualize your logs
- 38. What do we do about log ordering?
- 41. AuthentificationAuthentification how can you prove that a container/service is who it says it is? useful to have a generalized way of authenticating all your containers that way you can count on the reported identity when allowing access to certain resources
- 42. How to do itHow to do it (authentication) Largely unsolved Docker 1.3 tries to check image signatures if they come from the public registry and if they are marked as an "official repo" A PKI setup fits the problem, with a unique certificate for every container(not image) Docker promised some PKI based solution in future releases - I would wait for that
- 43. AuthorizationAuthorization builds on top of authentication will keep track of what resources a container/service can access should hand over details like user/pass pairs, API tokens and ssh keys
- 44. How to do itHow to do it (authorization) Do NOT bake in credentials and ssh keys into images (security and coupling) Easy way - mount external volume that contains credentials, ssh keys or even ssh agent sockets - doesn't require authentication - increases the complexity of your deployment solution Hard way - store credentials in a centralized service - requires some form of authentication - decreases complexity in your deployment solution
- 45. How to do itHow to do it (authorization) Crypt and Consul(or etcd) tries to solve the problem by using OpenPGP each container needs access to a private key. Can be made available through volume credentials are stored encrypted in Consul credentials get retrieved and decrypted in container
- 46. Image build serviceImage build service
- 47. Image build serviceImage build service Build gets triggered when code gets changed and committed to your repository Can perform basic checks to make sure the image complies with some basic rules Commits image to image registry If other images depend on it, a build job should be triggered for those images Extra tip: more control over the input sources for your images will in turn improve the reliability of your builds
- 48. How to do itHow to do it (image build service) Git and Jenkins? probably any vcs and CI tool will work but Git and Jenkins work great Simple workflow commits code Git post commit hook Github webhook Jenkins test and build Push to registry
- 49. Container Inputs (binaries,code, packages, etc) Build process Basic build process
- 51. Image registryImage registry a central place to store your Docker images Docker Hub is the public one you can easily run a private registry
- 52. Open source projectsOpen source projects Docker registry https://github.com/docker/docker-registry Artifactory http://www.jfrog.com/open-source/ (image registry)
- 53. How to do itHow to do it (image registry) USE a registry and dont rely on building images on every machine tag your images with specific versions make version requirements explicit
- 55. Where are we now?Where are we now? a lot of hype, experience needs to follow the sheer number of projects and work put in the ecosystem is impressive this momentum fuels on itself and ignites rapid development in projects that are required to achieve certain things can you program?
- 56. Some conclusionsSome conclusions reduce coupling between components think about your platform as a functional program with side effects - identify the logic and identify the state architect your system in a service oriented way - this way any required service can be placed inside a container avoid running services on your Docker host all container operations should be programmable, and ideally idempotent
- 57. The network is the last bastion of inflexibility. trade-off between flexibility and performance (throughput,latency) detailed analysis of performance?