![Docker CLI
docker pull image[:tag|@digest], aka docker
image pull
● tag can is a version, digest is a sha256
digest (like git commit hash)
docker push image:tag, aka docker image push
docker rmi image:tag, aka docker image rm
docker build, aka docker image build
● use --no-cache to rebuild from scratch
● use -t image:tag to add name and version
docker images, aka docker image ls
docker image inspect
docker image inspect <imageid> | jq -r '.[].RootFS'
shows all layers of an image](https://image.slidesharecdn.com/bgougsummer182-190109162923/85/Accelerate-your-software-development-with-Docker-22-320.jpg)
![Dockerfile Instructions
ARG <name>[=<default value>]
● Declares build time argument to the
Dockerfile. Pass valu to docker build.
FROM <image>[:<tag> | @<digest>] [AS <name>]
● Declares the base image to inherit from
● FROM can use ARG
● AS is for multistage builds
RUN ( <command> | [“exe”, “param1” …] )
● Execs a command in own layer
● ENV var setting is allowed by prefixing the
command with key=value
CMD
● The command to execute when starting
the container
● One per file
● This is not for executing statements
● See also ENTRYPOINT, it might use it when
no executable is declared
LABEL <key>=<value> <key>=<value> …
● For setting metadata which can be queried
later
● LABEL version=”1.0” vendor=”com.dnhsoft”
● Use LABEL instead of MAINTAINER](https://image.slidesharecdn.com/bgougsummer182-190109162923/85/Accelerate-your-software-development-with-Docker-26-320.jpg)
![But there is more...
EXPOSE
● Tell docker daemon the port will be
exposed
● Doesn’t expose the port automagically, to
do so use docker run -p XXXX:YYYY
ENV (key value | key=value …)
● Sets a ENV variable which is valid until the
end of the Dockerfile
● The ENV will also exist during container
runtime
COPY [--chown=<user>:<group>] <src>... <dest>
● Copies files, dirs into the container at <dst>
● Allows chowing to user:group
● Wildcards are possible
● If <dst> is relative than WORKDIR is used for
resolving the path
● You can’t send as <src> files/dirs up the tree
● Use .dockerignore if you want to skip files
when using wildcards.
ADD [--chown=<user>:<group>] <src>... <dest>
● Same as COPY but also
● Supports <src> from URL
● Local tar.gz|bz2|xz can be decompressed](https://image.slidesharecdn.com/bgougsummer182-190109162923/85/Accelerate-your-software-development-with-Docker-27-320.jpg)
![Hungry and ready for lunch?
ENTRYPOINT ["executable", "par1", "par2"]
● Makes from the container a command
● When you run a container the command
you pass is appended to the ENTRYPOINT
● http://www.johnzaccone.io/entrypoint-vs-c
md-back-to-basics/
VOLUME /path/to/dir
● Shows the intent to mount at the location
● The real mount happens with docker run -v
hostdir:/path/to/dir
SHELL ["executable", "parameters"]
USER <UID>[:<GID>]
● Sets the uid:gid of subsequent commands
● Sets the uid:gid at container runtime
● Please use it, otherwise root = too much
rights
WORKDIR /path/to/workdir
● Sets $(PWD)
● Parameter can be absolute or relative
● When relative appended to current value
● Very much like cd /path/to/workdir](https://image.slidesharecdn.com/bgougsummer182-190109162923/85/Accelerate-your-software-development-with-Docker-28-320.jpg)
![Here come the last ones before the demo
ONBUILD [INSTRUCTION]
● Schedule INSTRUCTION to be executed
when building a child image. A trigger.
● Multiple ONBUILD triggers are executed in
the same order
● Allows one-liners child Dockerfiles : FROM
base-onbuild:1.2
STOPSIGNAL
● Sets the signal number to send when
stopping.
● Could be a number, like 9, or name
SIGKILL
HEALTHCHECK [OPTIONS] CMD
● Allows Docker to check the healthiness of
the container by executing CMD
● CMD should return 0 for healthy and 1 for
unhealthy
● docker ps shows the status
● --interval=TIME , runs every TIME
● --timeout=TIME, probe fails after TIME
● --retries=N , run the probe up to N times
consecutively
● --start-period=TIME , wait TIME after
container start before running the probe.
Useful for containers with long boot time](https://image.slidesharecdn.com/bgougsummer182-190109162923/85/Accelerate-your-software-development-with-Docker-29-320.jpg)
Accelerate your software development with Docker
- 1. Accelerate your software development with Docker Andrey Hristov CTO, DNH Soft
- 2. $ whoami Started programming at age of 13 MSc in Computer Technologies (TU-Sofia) and Software Engineering (HFT Stuttgart) Professional developer since year 2000 PHP core developer since 2002 Spent 11 years working at MySQL, SUN Micro and Oracle improving the MySQL client and server side Last 2 years spent as freelancing technical team lead / consultant Lately became CTO of DNH Soft Software Development practices and architectures appassionato.
- 3. How many of you have used Docker / Linux containers?
- 4. What to expect from this talk? Quick intro into 12 Factor Applications What is Docker? Containers from Linux POV Description of technologies related to containers Overview of Docker Live Demo
- 5. 12 Factors App ● Methodology for building SAAS ● Drafted around 2011 at Heroku 1. Codebase 2. Dependencies 3. Config 4. Backing services 5. Build, release, run 6. Processes 7. Port binding 8. Concurrency 9. Disposability 10. Dev / Prod parity 11. Logs 12. Admin processes
- 6. Codebase / Dependencies / Config Codebase There should be exactly one codebase for a deployed service with the codebase being used for many deployments. Dependencies All dependencies should be declared, with no implicit reliance on system tools or libraries. Config Configuration that varies between deployments should be stored in the environment.
- 7. Backing Services / Build, Release, Run / Processes Backing Services All backing services are treated as attached resources and attached and detached by the execution environment. Build, release, run The delivery pipeline should strictly consist of build, release, run. Build stage artefacts should not be available to release and run stage. Build once, run everywhere. Processes Applications should be deployed as one or more stateless processes with persisted data stored on a backing service.
- 8. Port Binding / Concurrency / Disposability Port Binding Self-contained services should make themselves available to other services by specified ports. Concurrency Scale out via the process model Processes Maximize robustness with fast startup and graceful shutdown
- 9. Dev - Prod Parity / Logs / Admin Processes Dev / Prod Parity All environments should be as similar as possible. Logs Applications should produce logs as event streams and leave the execution environment to aggregate. Admin Processes Any needed admin tasks should be kept in source control and packaged with the application. They should run in the same environment as the application itself.
- 10. To begin with, what is Docker? Docker Inc. is a company, previously dotCloud However in the past 6 years the name meant containers Some people say dockerize when they mean containerize (similarly to the verb “to google”) Containers were not invented by Docker Inc. The company made them available to the masses.
- 11. Then, what is a container? Containerization is OS environment virtualization It feels like a VM but ain't one. Some people call them lightweight VMs. “One kernel to rule them all” compared to “one hypervisor to rule them all”. Can't boot a different OS or kernel. Can't load other kernel modules. Can boot different distro, however. Typically only one process / service (forking apps) runs inside the container. Examples of previous/other works : Solaris Zones, FreeBSD Jails
- 12. Containers on Linux Containers on Linux rely on a couple of kernel features Linux Namespaces, that provide isolation Currently existing namespaces are : cgroup, IPC, network, mount, PID, user (UIDs & GIDs), UTS Control Groups (cgroups), that provide means for hierarchical organization for metering and limiting of resources (memory, CPU, I/O, network) for group (collections of processes)
- 13. Who is running them? Containers are executed on lower level by runtimes LXC/LXD - LXD, written in Go, uses LXC rkt - App Container compliant, deprecated, by CoreOS, now Red Hat. Natively ACI, but supports also Docker and OCI images. Forked very recently runC - OCI compliant implementation in Golang by Docker Inc., a spin off from Docker Engine since Docker 1.11 containerD - works with runC for the high level details, while runC is low level railcar - OCI compliant implementation in Rust by Oracle OCI has two specs, released in July'17 : Image and Runtime CRI-O, implementation of the Kubernetes (1.5+) Container Runtime Interface (CRI) using OCI compatible runtimes.
- 14. But there is more! Containers are managed at a higher level by orchestrators. Docker Compose (single host only) and Docker Swarm both are part of Docker Engine Marathon on Apache Mesos Cattle, obsoleted, by Rancher. Rancher 2.0 runs k8s Kubernetes (k8s). Recently won the Orchestrator wars. If you plan to use containers k8s should be your orchestrator of choice KaaS is available from all major cloud providers - AWS (beta), Azure and GKE
- 15. In short, what’s in for me? Containers are lightweight, or at least lighter than VMs, both in run-time resources usage and size Containers are immutable Containers can be even read-only Containers are meant to be ephemeral Every container contains all needed dependencies and doesn't need anything else Implications: Dep hell is gone. DLL hell memories resurface? XAMPP is dead Linux distro software choice is dead Less software installed means less exploit surface
- 16. Hosting of container images (registries) Docker Inc. runs Docker Hub Library of public images Docker Store - commercially available containers and Docker plugins. Docker Hub supports automated builds triggered on a commit in Github / BitBucket. Storage for your images ● free of charge for you public ones ● has a cost for you private images Alternatives are: ● Host a registry in a container on own VPS or on premise ○ Docker Trusted Registry (Docker EE) ○ RedHat OpenShift CR ○ JFrog Artifactory ○ Sonatype Nexus ● Amazon Elastic Container Registry, you need AWS SDK ● Google Container Registry, you need Google Cloud SDK ● Azure Container Registry
- 18. Docker (the software) Flavors Supported OS for Docker CE: ● Linux (x86-64, ARM, ARM64, ppc64le, s390x( ● MacOS, comes bundled with k8s ● Windows, comes bundled with k8s ● AWS ● Azure Supported Platforms for Docker EE ● CentOS (x86-64) ● OL (x86-64) ● RHEL, SUSE Linux ES, Ubuntu (x86-64 / ppc64le / s390x) ● MS Windows Server 2016 (x86-64) ● AWS ● Azure ● IBM Cloud
- 19. Docker Compose Originally known as Fig Orchestrator that uses IaC “Cluster” configuration is stored in an YAML file ( ./docker-compose.yml ) Features are constantly added, thus there are many compose file versions. Latest is 3.6 as of 18.02 First line in the file states minimum version The file is split in 3 main sections - higher level abstractions, since 2.0 : services, networks, volumes If you plan to use Docker Swarm, then you have to use version 3. Docker EE also now supports K8s deployments from docker-compose.yml
- 20. Docker Compose Entities services - The containers = instances of images. With Swarm you can have multiple instance per service - scaling up and down. volumes / mounts - Persistently stored data. Otherwise data is gone when the container get removed. Mounts import data from the host and are shareable Volumes are BLOBs and are shareable too Volumes are abstracted thru plugins networks - The actual glue between the services DC creates a default network, if are lazy to not create one. This network is called <projectName>_default <projectName> is derived from CWD, pass -p to docker-compose for smth else. Networks can be seen by other projects and they are namespaced by project name. Network frontend in P1 can be attached in project P2 as external network under the foreign name P1_frontend.
- 21. Docker, where is my data? Container images are made of layers aufs (/var/lib/docker/aufs), superseed by overlayfs, shipped with Linux Kernel 4.0 cat /proc/filesystems to see what FSs your kernel support
- 22. Docker CLI docker pull image[:tag|@digest], aka docker image pull ● tag can is a version, digest is a sha256 digest (like git commit hash) docker push image:tag, aka docker image push docker rmi image:tag, aka docker image rm docker build, aka docker image build ● use --no-cache to rebuild from scratch ● use -t image:tag to add name and version docker images, aka docker image ls docker image inspect docker image inspect <imageid> | jq -r '.[].RootFS' shows all layers of an image
- 23. More Docker CLI docker run, aka docker container run docker exec, aka docker container exec docker rm, aka docker container rm docker ps, aka docker container ls docker stop, aka docker container stop (SIGTERM) docker kill, aka docker container kill (SIGKILL) docker kill `docker ps -q` to kill'em all (you might also need to remove them) docker inspect ● inspects networks, containers, images ● gives you tons of info in JSON format. Use jq to process it. docker container diff docker network ls docker network rm docker network prune docker system prune
- 24. Building a container image docker build ● Simple - just run the command ● Transparent - the recipe how to build is in the Dockerfile ● Self-contained everything is one place; the Dockerfile, the assets ONBUILD Strategy ● The Dockerfile is a simple “FROM baseimage” ● Intransparent, as the sysadmin defines what will happen Asset Generation Pipeline Strategy ● Run different asset generators as separate containers ● SASS, composer, etc. ● External driver is needed, like make, gulp, or just whatever your CI provides ● Pro - smaller images ● Con - complicated because of multiple moving parts Multistage Builds Strategy
- 25. Multistage builds Build different artifacts during different stages Opt-in what to pull from a previous stage In short, install the compile time deps in first stage, compile the app, pull only the compiled code in the next stage which will eventually be the delivered image Pro: No need for an external driver like make, gulp, etc Pro: The recipe is in one place - the Dockerfile Con: The Dockerfile become longish
- 26. Dockerfile Instructions ARG <name>[=<default value>] ● Declares build time argument to the Dockerfile. Pass valu to docker build. FROM <image>[:<tag> | @<digest>] [AS <name>] ● Declares the base image to inherit from ● FROM can use ARG ● AS is for multistage builds RUN ( <command> | [“exe”, “param1” …] ) ● Execs a command in own layer ● ENV var setting is allowed by prefixing the command with key=value CMD ● The command to execute when starting the container ● One per file ● This is not for executing statements ● See also ENTRYPOINT, it might use it when no executable is declared LABEL <key>=<value> <key>=<value> … ● For setting metadata which can be queried later ● LABEL version=”1.0” vendor=”com.dnhsoft” ● Use LABEL instead of MAINTAINER
- 27. But there is more... EXPOSE ● Tell docker daemon the port will be exposed ● Doesn’t expose the port automagically, to do so use docker run -p XXXX:YYYY ENV (key value | key=value …) ● Sets a ENV variable which is valid until the end of the Dockerfile ● The ENV will also exist during container runtime COPY [--chown=<user>:<group>] <src>... <dest> ● Copies files, dirs into the container at <dst> ● Allows chowing to user:group ● Wildcards are possible ● If <dst> is relative than WORKDIR is used for resolving the path ● You can’t send as <src> files/dirs up the tree ● Use .dockerignore if you want to skip files when using wildcards. ADD [--chown=<user>:<group>] <src>... <dest> ● Same as COPY but also ● Supports <src> from URL ● Local tar.gz|bz2|xz can be decompressed
- 28. Hungry and ready for lunch? ENTRYPOINT ["executable", "par1", "par2"] ● Makes from the container a command ● When you run a container the command you pass is appended to the ENTRYPOINT ● http://www.johnzaccone.io/entrypoint-vs-c md-back-to-basics/ VOLUME /path/to/dir ● Shows the intent to mount at the location ● The real mount happens with docker run -v hostdir:/path/to/dir SHELL ["executable", "parameters"] USER <UID>[:<GID>] ● Sets the uid:gid of subsequent commands ● Sets the uid:gid at container runtime ● Please use it, otherwise root = too much rights WORKDIR /path/to/workdir ● Sets $(PWD) ● Parameter can be absolute or relative ● When relative appended to current value ● Very much like cd /path/to/workdir
- 29. Here come the last ones before the demo ONBUILD [INSTRUCTION] ● Schedule INSTRUCTION to be executed when building a child image. A trigger. ● Multiple ONBUILD triggers are executed in the same order ● Allows one-liners child Dockerfiles : FROM base-onbuild:1.2 STOPSIGNAL ● Sets the signal number to send when stopping. ● Could be a number, like 9, or name SIGKILL HEALTHCHECK [OPTIONS] CMD ● Allows Docker to check the healthiness of the container by executing CMD ● CMD should return 0 for healthy and 1 for unhealthy ● docker ps shows the status ● --interval=TIME , runs every TIME ● --timeout=TIME, probe fails after TIME ● --retries=N , run the probe up to N times consecutively ● --start-period=TIME , wait TIME after container start before running the probe. Useful for containers with long boot time
- 30. Live Demo
- 31. Q&A / Resources Anatomy of a container: https://bit.ly/2v0EEGj https://github.com/andreyhristov/bws2018-docker https://coreos.com/rkt/docs/latest/rkt-vs-other-projects.html https://docs.docker.com/install/linux/docker-ce/ubuntu/ https://docs.docker.com/compose/install/ https://docs.docker.com/compose/compose-file/ https://docker-software-inc.scoop.it/t/docker-by-docker https://nickjanetakis.com/blog/tag/docker-tips-tricks-and-tutorials https://opensource.com/article/18/5/navigating-container-security-eco system https://dashtainer.com/ https://landscape.cncf.io/ https://traefik.io/ https://leanpub.com/the-devops-2-toolkit https://leanpub.com/the-devops-2-2-toolkit https://leanpub.com/the-devops-2-3-toolkit https://thenewstack.io/ https://www.katacoda.com/ https://github.com/google/gvisor https://www.ianlewis.org/en/almighty-pause-contai