The Docker Ecosystem
2 likes1,527 views
The document provides a comprehensive overview of Docker, detailing its architecture, key components, and various use cases in production environments. It highlights how Docker simplifies application deployment and management through features like containerization, Docker Machine, and Docker Compose, while also emphasizing its effectiveness in rapid deployment, server consolidation, and multi-tenancy. Additionally, it discusses networking capabilities and the orchestration of container clusters using Docker Swarm.
![32CONFIDENTIAL
docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
rnd-saas/service-discovery latest 9f7499191ada 10 seconds ago 722.7 MB
java 8 bdd93cb6443c 4 days ago 641.9 MB
busybox latest c51f86c28340 4 days ago 1.109 MB
nginx latest 914c82c5a678 7 days ago 132.8 MB
ubuntu precise 38f2c35e1b51 13 days ago 136.1 MB
32
DOCKERFILE
FROM java:8
EXPOSE 8761
VOLUME /tmp
ADD service-discovery-0.1.0.jar app.jar
RUN bash -c 'touch /app.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
Dockerfile content
docker build -t rnd-saas/service-discovery .
Build new image
Show images](https://image.slidesharecdn.com/docker-160518160405/85/The-Docker-Ecosystem-32-320.jpg)
![49CONFIDENTIAL
49
In order to meet the specific requirements of each container, their
placement can be fine-tuned using constraints.
CONSTRAINTS
docker run -d -e constraint:storage==ssd mysql
Constraints operate on Docker daemon labels. To make the previous
example work, Docker must be started with the–label storage=ssd option.
More advanced expressions are also supported:
docker run --rm -d -e constraint:node!=fed*
docker run --rm -d -e constraint:node==/ubuntu-[0-9]+/](https://image.slidesharecdn.com/docker-160518160405/85/The-Docker-Ecosystem-49-320.jpg)
The Docker Ecosystem
- 1. 1CONFIDENTIAL The Docker Ecosystem DZMITRY SKAREDAU, SOLUTION ARCHITECT NOVEMBER 5, 2015
- 2. 2CONFIDENTIAL 2 AGENDA • Introduction to Docker • Docker’s Key Use Cases • Docker Toolbox • Docker Machine • Docker Compose • Docker Swarm • Multi-Host Docker Networking
- 4. 4CONFIDENTIAL 4 WHAT IS DOCKER? Open source engine that leverage LXC and AUFS to package an application and its dependencies in a virtual container that can run on any Linux server. WHAT!? We are using Windows! most of us
- 6. 6CONFIDENTIAL 6 LXC Wikipedia https://en.wikipedia.org/wiki/LXC “ „ Linux Containers (LXC) provide a means to isolate individual services or applications as well as of a complete Linux operating system from other services running on the same host. To accomplish this, each container gets its own directory structure, network devices, IP addresses and process table. The processes running in other containers or the host system are not visible from inside a container. Additionally, Linux Containers allow for fine granular control of resources like RAM, CPU or disk I/O. LXC combines kernel's cgroups and support for isolated namespaces to provide an isolated environment for applications.
- 7. 7CONFIDENTIAL 7 CGROUPS “ „cgroups (abbreviated from control groups) is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes. Wikipedia https://en.wikipedia.org/wiki/Cgroups
- 8. 8CONFIDENTIAL 8 NAMESPACE ISOLATION “ „ namespace isolation, where groups of processes are separated such that they cannot "see" resources in other groups. For example, a PID namespace provides a separate enumeration of process identifiers within each namespace. Also available are mount, UTS, network and SysV IPC namespaces. Wikipedia https://en.wikipedia.org/wiki/Cgroups
- 9. 9CONFIDENTIAL 9 AUFS “ „aufs (short for advanced multi layered unification filesystem) implements a union mount for Linux file systems. Wikipedia https://en.wikipedia.org/wiki/Aufs
- 10. 10CONFIDENTIAL 10 AUFS A typical Linux start to run to two FS: • bootfs (boot file system) Including bootloader and kernel, bootloader is the main kernel boot loader, when after the success of the boot kernel is loaded into memory after the bootfs was umount • rootfs (root file system) Is a typical Linux system consists of /dev, /proc, /bin, /etc and other standard directory and file.
- 11. 11CONFIDENTIAL 11 AUFS Thus for different Linux distributions, the bootfs is basically the same, the rootfs will be different, so different distributions can be public bootfs as shown below: • Debian is a Unix-like computer operating system and a Linux distribution Size: 136.1 MB • BusyBox is software that provides several stripped-down Unix tools in a single executable file. It was specifically created for embedded operating systems with very limited resources. Size: 1.109 MB
- 12. 12CONFIDENTIAL 12 AUFS 2 custom images: 1. With Apache/Emacs over Debian 2. Over BusyBox
- 14. 14CONFIDENTIAL 14 DOCKER UNDER THE HOOD
- 15. 15CONFIDENTIAL 15 DOCKER UNDER THE HOOD
- 16. 16CONFIDENTIAL 16 DOCKER AT LINUX AND MACOS/WINDOWS
- 17. 17CONFIDENTIAL 17 DOCKER CONTAINERS IN PRODUCTION There is currently a pervasive (and faulty) perception that Docker containers are only being utilized in dev-test and proof-of- concept projects. In fact, the question I am most often asked by IT colleagues and customers goes like this: “Is anyone using Docker containers for critical workloads, or even in production?” The answer is an unequivocal “Yes” – critical workloads are being run in Docker containers, and much more pervasively than is commonly understood. Here are a few examples: • Global financial services corporation ING is using Docker containers to help accelerate its continuous delivery process and drive 500 deployments/week, meeting speed to market goals • Global investment bank Goldman Sachs uses Docker containers to centralize application builds and deployments • Streaming music leader Spotify uses Docker containers to make software deploymentsrepeatable, straightforward, and fault-tolerant • Application performance management leader New Relic is using Docker containers to solve its most challenging deployment issues
- 18. 18CONFIDENTIAL DOCKER’S KEY USE CASES
- 19. 19CONFIDENTIAL 19 SIMPLIFYING CONFIGURATION Cloud Services with built-in Docker support
- 20. 20CONFIDENTIAL 20 CODE PIPELINE MANAGEMENT The immutable nature of Docker images, and the ease with which they can be spun up, help you achieve zero change in application runtime environments across dev through production. ENV DEV Private Docker Hub ENV INT ENV QA ENV PRE PROD ENV PROD
- 21. 21CONFIDENTIAL 21 DEVELOPER PRODUCTIVITY In a developer environment, we have two goals that are at odds with each other: 1. We want it be as close as possible to production; and 2. We want the development environment to be as fast as possible for interactive use.
- 22. 22CONFIDENTIAL 22 APP ISOLATION A couple of such cases to consider are server consolidation for decreasing cost or a gradual plan to separate a monolithic application into decoupled pieces.
- 23. 23CONFIDENTIAL 23 SERVER CONSOLIDATION Just like using VMs for consolidating multiple applications, the application isolation abilities of Docker allows consolidating multiple servers to save on cost. However, without the memory footprint of multiple OSes and the ability to share unused memory across the instances, Docker provides far denser server consolidation than you can get with VMs.
- 24. 24CONFIDENTIAL 24 MULTI-TENANCY Using Docker, it was easy and inexpensive to create isolated environments for running multiple instances of app tiers for each tenant.
- 25. 25CONFIDENTIAL 25 RAPID DEPLOYMENT Docker creating a container for the process and not booting up an OS, brings it down to seconds.
- 27. 27CONFIDENTIAL 27 • Docker Machine for running the docker-machine binary • Docker Engine for running the docker binary • Kitematic, the Docker GUI • a shell preconfigured for a Docker command-line environment • Oracle VM VirtualBox DOCKER TOOLBOX
- 28. 28CONFIDENTIAL 28 The Docker VM is lightweight Linux virtual machine made specifically to run the Docker daemon on Windows. The VirtualBox VM runs completely from RAM, is a small ~29MB download, and boots in approximately 5s. DOCKER MACHINE docker-machine create --driver virtualbox my-default Creating VirtualBox VM... Creating SSH key... Starting VirtualBox VM... Starting VM... To see how to connect Docker to this machine, run: docker-machine env my-default docker-machine --native-ssh create -d virtualbox dev
- 29. 29CONFIDENTIAL 29 DOCKER MACHINE docker-machine --native-ssh create -d virtualbox dev docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM dev virtualbox Running tcp://192.168.99.100:2376 List your available machines Create a new Docker VM Get the environment commands for your new VM docker-machine env dev --shell cmd set DOCKER_TLS_VERIFY=1 set DOCKER_HOST=tcp://192.168.99.100:2376 set DOCKER_CERT_PATH=C:UsersDzmitry_Skaredau.dockermachinemachinesdev set DOCKER_MACHINE_NAME=dev # Run this command to configure your shell: # copy and paste the above values into your command prompt
- 30. 30CONFIDENTIAL 30 DOCKER MACHINE docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM dev * virtualbox Running tcp://192.168.99.100:2376 List your available machines docker run ^ -d ^ -p 80:80 ^ -v $(pwd)/src/vhost.conf:/etc/nginx/sites-enabled/vhost.conf ^ -v $(pwd)/src:/var/www ^ nginx Run container pwd: The pwd command will allow you to know in which directory you're located (pwd stands for "print working directory")
- 31. 31CONFIDENTIAL 31 DOCKER MACHINE docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ef9b3f99a05f nginx "nginx -g 'daemon off" 13 seconds ago Up 9 seconds 0.0.0.0:80->80/tcp, 443/tcp sad_elion Show containers docker-machine ip 192.168.99.100 Find machine IP
- 32. 32CONFIDENTIAL docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE rnd-saas/service-discovery latest 9f7499191ada 10 seconds ago 722.7 MB java 8 bdd93cb6443c 4 days ago 641.9 MB busybox latest c51f86c28340 4 days ago 1.109 MB nginx latest 914c82c5a678 7 days ago 132.8 MB ubuntu precise 38f2c35e1b51 13 days ago 136.1 MB 32 DOCKERFILE FROM java:8 EXPOSE 8761 VOLUME /tmp ADD service-discovery-0.1.0.jar app.jar RUN bash -c 'touch /app.jar' ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"] Dockerfile content docker build -t rnd-saas/service-discovery . Build new image Show images
- 34. 34CONFIDENTIAL 34 Running multiple containers • Run your stack with one command: docker-compose up • Describe your stack with one file: docker-compose.yml DOCKER COMPOSE
- 35. 35CONFIDENTIAL 35 HOW TO RUN WORDPRESS FROM orchardup/php5 ADD . /code Dockerfile web: build: . command: php -S 0.0.0.0:8000 -t /code ports: - "8000:8000" links: - db volumes: - .:/code db: image: orchardup/mysql environment: MYSQL_DATABASE: wordpress docker-compose.yml
- 36. 36CONFIDENTIAL 36 The features of Compose that make it effective are: • Multiple isolated environments on a single host DOCKER COMPOSE
- 37. 37CONFIDENTIAL 37 Compose uses a project name to isolate environments from each other. You can use this project name to: • on a dev host to create multiple copies of a single environment (ex: you want to run a stable copy for each feature branch of a project) • on a CI server to keep builds from interfering with each other, you can set the project name to a unique build number • on a shared host or dev host to prevent different projects which may use the same service names, from interfering with each other ISOLATED ENVIRONMENTS
- 38. 38CONFIDENTIAL 38 The features of Compose that make it effective are: • Multiple isolated environments on a single host • Preserve volume data when containers are created DOCKER COMPOSE
- 39. 39CONFIDENTIAL 39 Compose preserves all volumes used by your services. When docker- compose up runs, if it finds any containers from previous runs, it copies the volumes from the old container to the new container. This process ensures that any data you’ve created in volumes isn’t lost. PRESERVE VOLUME DATA
- 40. 40CONFIDENTIAL 40 The features of Compose that make it effective are: • Multiple isolated environments on a single host • Preserve volume data when containers are created • Only recreate containers that have changed DOCKER COMPOSE
- 41. 41CONFIDENTIAL 41 Compose caches the configuration used to create a container. When you restart a service that has not changed, Compose re-uses the existing containers. Re-using containers means that you can make changes to your environment very quickly. RECREATES ONLY CHANGED CONTAINERS
- 42. 42CONFIDENTIAL 42 The features of Compose that make it effective are: • Multiple isolated environments on a single host • Preserve volume data when containers are created • Only recreate containers that have changed • Variables and moving a composition between environments DOCKER COMPOSE
- 43. 43CONFIDENTIAL 43 Your configuration options can contain environment variables. Compose uses the variable values from the shell environment in which docker- compose is run. For example, suppose the shell contains POSTGRES_VERSION=9.3 and you supply this configuration: VARIABLE SUBSTITUTION db: image: "postgres:${POSTGRES_VERSION}"
- 44. 44CONFIDENTIAL 44 Common use case is multiple compose files: changing a Compose app for different environments MOVING A COMPOSITION BETWEEN ENVIRONMENTS docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
- 46. 46CONFIDENTIAL 46 Docker Swarm is used to host and schedule a cluster of Docker containers. DOCKER SWARM
- 47. 47CONFIDENTIAL 47 Since Swarm ships as a standard Docker image with no external infrastructure dependencies, getting started is a simple, three-step process: 1. Run one command to create a cluster. 2. Run another command to start Swarm. 3. On each host where the Docker Engine is running, run a command to join said cluster. SETUP
- 48. 48CONFIDENTIAL 48 Swarm is aware of the resources available in the cluster and will place containers accordingly. To choose a ranking strategy, pass the --strategy flag and a strategy value to the swarm manage command. Swarm currently supports these values: • spread • binpack • random RESOURCE MANAGEMENT docker run -d -m 1g redis
- 49. 49CONFIDENTIAL 49 In order to meet the specific requirements of each container, their placement can be fine-tuned using constraints. CONSTRAINTS docker run -d -e constraint:storage==ssd mysql Constraints operate on Docker daemon labels. To make the previous example work, Docker must be started with the–label storage=ssd option. More advanced expressions are also supported: docker run --rm -d -e constraint:node!=fed* docker run --rm -d -e constraint:node==/ubuntu-[0-9]+/
- 50. 50CONFIDENTIAL 50 In some cases, the placement of a container must be relative to other containers. Swarm lets you define those relationships through affinities. The following will run two Redis servers, while guaranteeing they don’t get scheduled on the same machine: AFFINITY docker run -d --name redis_1 -e ‘affinity:container!=redis_*’ redis docker run -d --name redis_2 -e ‘affinity:container!=redis_*’ redis
- 51. 51CONFIDENTIAL 51 At some point, Swarm will be able to reschedule containers upon host failure. Let’s say you schedule a frontend container with some constraints: FAULT-TOLERANT SCHEDULING docker run -d -e constraint:storage==ssd nginx If the host of this container goes down, Swarm will be able to detect the outage and reschedule this container on another host that can respect the constraint storage==ssd
- 53. 53CONFIDENTIAL 53 Networking is a feature of Docker Engine that allows you to create virtual networks and attach containers to them so you can create the network topology that is right for your application. NETWORKING
- 54. 54CONFIDENTIAL 54 1. Connect containers to each other across different physical or virtual hosts 2. Containers using Networking can be easily stopped, started and restarted without disrupting the connections to other containers 3. You don’t need to create a container before you can link to it. With Networking containers be created in any order and discover each other using their container names NETWORKING
- 55. 55CONFIDENTIAL 55 You can create a new network with docker network create. In this example, we’ll create a network called “frontend” and run an nginx container inside it: NETWORKING docker network create frontend docker run -d --net=frontend --name web nginx Then we could run a web application in a network called “app” and then use the docker network connect command so our Nginx container can forward connections to it. docker network create app docker run -d --name myapp --net=app <my application container> docker network connect app web Now Nginx should be able to connect to your application using the hostname “myapp.app”