Introduction to Docker, Devops Virtualization and configuration management

UNIT -1:Introduction to Docker
INT 332:DEVOPS VIRTUALIZATION AND
CONFIGURATION MANAGEMENT

What is Docker?
Docker is a containerization platform utilized to package software,
applications, or code into docker containers.
Docker enables seamless deployment to various environments,
preventing dependency issues that might exist between
developers and operations.

x
Docker simplifies the DevOps Methodology by allowing
developers to create templates called “images,” using which
we can create lightweight virtual machines called
“containers.”
Docker makes things easier for software developers by
giving them the capability to automate infrastructure, isolate
applications, maintain consistency, and improve resource
utilization.
There might arise a question that such tasks can also be
done through virtualization, so why choose Docker over it?
This is because virtualization is not as efficient.

x
Containerization vs Virtualization

Docker hub
• Docker Hub is a service provided by Docker for finding and
sharing container images.
• It's the world’s largest repository of container images with an
array of content sources including container community
developers, open source projects, and independent software
vendors (ISV) building and distributing their code in
containers.
• Docker Hub is also where you can go to change your Docker
account settings and carry out administrative tasks.

Key Features of Docker Hub
• Storage, management, and sharing of images with others are
made simple via Docker Hub.
• Docker Hub runs the necessary security checks on our images
and generates a full report on any security flaws.
• Docker Hub can automate the processes like Continuous
deployment and Continuous testing by triggering the Webhooks
when the new image is pushed into Docker Hub.
• With the help of Docker Hub, we can manage the permission for
the users, teams, and organizations.
• We can integrate Docker Hub into our tools
like GitHub, Jenkins which makes workflows easy

Advantages of Docker Hub
• Docker Container Images are light in weight.
• We can push the images within a minute and with help of a
command.
• It is a secure method and also provides a feature like pushing
the private image or public image.
• Docker hub plays a very important role in industries as it
becomes more popular day by day and it acts as a bridge
between the developer team and the testing team.
• If a person wants to share their code, software any type of file
for public use, you can just make the images public on the
docker hub.

• We will learn and do the practical work that how to create the
repository in docker hub
• Source: https://www.geeksforgeeks.org/what-is-docker-hub/

Docker Architecture
Docker uses a client-server architecture.
The Docker client consists of
• Docker build
• Docker pull and
• Docker run.
The client approaches the Docker daemon which further helps in
building, running, and distributing Docker containers.
Docker client and Docker daemon can be operated on the same
system; otherwise, we can connect the Docker client to the
remote Docker daemon.
Both communicate with each other by using the REST API/command
line , over UNIX sockets or a network.
• learn from the below-given docker architecture diagram.

x
Docker Client Docker host Docker Container Docker Registry

1. Docker Client
• The Docker Client serves as the primary interface for interacting
with the Docker Engine.
• It provides a user-friendly command-line interface (CLI) that
enables users to execute various operations, such as building,
running, managing, and deleting Docker images and containers.
• The Docker Client acts as a bridge between the user and the
underlying Docker Engine. This helps in, translating user
commands into instructions that the Docker Engine can
understand and execute.

Key Functions of the Docker Client
• Manage Containers: Run, create, stop, start, and delete containers
effortlessly with intuitive commands. For instance, docker run,
docker stop, and docker rm are some basic commands to manage
containers.
• Work with Images: Pull, push, build, and manage Docker images
using commands like docker pull, docker push, docker build, and
docker rmi. These commands help you handle the blueprints for your
containers.
• Control Networks and Volumes: Create and manage networks for
your containers and handle data persistence using volumes.
Commands like docker network and docker volume come in handy
for these tasks.
• View System Information: Check the status of your Docker
environment, see running containers, monitor resources, and
troubleshoot using commands like docker ps, docker info, and
docker stats.
•

Docker Host
• In the Docker host, we have a Docker daemon and Docker objects
such as containers and images.
• First, let us understand the objects on the Docker host, and then
we will proceed toward the functioning of the Docker daemon.

Docker objects:
– Docker Image: A template that can be used for creating Docker containers. It
includes steps for creating the necessary software.
- Docker Container : A type of virtual machine that is created from the
instructions found within the Docker image. It is a running instance of a Docker
image that consists of the entire package required to run an application.

Insight for Docker Image
• A Docker image is a file used to execute code in a Docker container.
Docker images act as a set of instructions to build a Docker container ,
like a template.
• Docker images also act as the starting point when using Docker.
• An image is comparable to a snapshot in virtual machine (VM)
environments.
• Docker is used to create, run and deploy applications in containers.
• A Docker image contains application code, libraries, tools,
dependencies and other files needed to make an application run.
• When a user runs an image, it can become one or many instances of a
container.

..contd..
• Docker images have multiple layers, each one originates from the
previous layer but is different from it.
• The layers speed up Docker builds while increasing reusability and
decreasing disk use.
• Image layers are also read-only files. Once a container is created, a
writable layer is added on top of the unchangeable images, allowing a
user to make changes.
• References to disk space in Docker images and containers can be
confusing.
• It's important to distinguish between size and virtual size.
• Size refers to the disk space that the writable layer of a container uses,
while virtual size is the disk space used for the container and the
writeable layer.
• The read-only layers of an image can be shared between any
container started from the same image.

Docker image use cases
• A Docker image has everything needed to run a containerized
application, including code, config files environment variables, libraries
and runtimes.
• When the image is deployed to a Docker environment, it can be
executed as a Docker container.
• The docker run command creates a container from a specific image.
• Docker images are a reusable asset -- deployable on any host.
Developers can take the static image layers from one project and use
them in another.
• This saves the user time, because they do not have to recreate an
image from scratch.

Anatomy of a Docker image
• A Docker image has many layers, and each image includes everything
needed to configure a container environment -- system libraries, tools,
dependencies and other files. Some of the parts of an image include:
• Base image. The user can build this first layer entirely from scratch
with the build command.
• Parent image. As an alternative to a base image, a parent image can
be the first layer in a Docker image. It is a reused image that serves as
a foundation for all other layers.

• Layers. Layers are added to the base image, using code that will
enable it to run in a container.
• Each layer of a Docker image is viewable under
/var/lib/docker/aufs/diff, or via the Docker history command in the
command-line interface (CLI).
• Docker's default status is to show all top-layer images,
including repository, tags and file sizes. I
• ntermediate layers are cached, making top layers easier to view.
Docker has storage drives that handle the management of image layer
contents.
• Container layer. A Docker image not only creates a new container,
but also a writable or container layer. This layer hosts changes made
to the running container and stores newly written and deleted files, as
well as changes to existing files. This layer is also used to customize
containers.

• Docker manifest. This part of the Docker image is an additional file. It
uses JSON format to describe the image, using information such as
image tags and digital signature.

Docker image repositories
• Docker images get stored in private or public repositories, such as
those in the Docker Hub cloud registry service, from which users can
deploy containers and test and share images. Docker Hub's Docker
Trusted Registry also provides image management and access control
capabilities.
• Official images are ones Docker produces, while community
images are images Docker users create.
• CoScale agent is an official Docker image that monitors Docker
applications .
• Datadog/docker-dd-agent, a Docker container for agents in the
Datadog Log Management program, is an example of a community
Docker image.

..contd..
• Users can also create new images from existing ones and use the
docker push command to upload custom images to the Docker Hub.
• To ensure the quality of community images, Docker provides feedback
to authors prior to publishing.
• Once the image is published, the author is responsible for updates.
Authors must be cautious when sourcing an image from another party
because attackers can gain access to a system through copycat
images designed to trick a user into thinking they are from a trusted
source.
• The concept of a latest image may also cause confusion. Docker
images tagged with ":latest" are not necessarily the latest in an
ordinary sense. The latest tag does not refer to the most recently
pushed version of an image; it is simply a default tag.

Docker Image Storage
A Docker container consists of network settings, volumes, and images.
The location of Docker files depends on your operating system. Here is
an overview for the most used operating systems:
• Ubuntu: /var/lib/docker/
• Fedora: /var/lib/docker/
• Debian: /var/lib/docker/
• Windows: C:ProgramDataDockerDesktop
• MacOS: ~/Library/Containers/com.docker.docker/Data/vms/0/
In macOS and Windows, Docker runs Linux containers in a virtual
environment. Therefore, there are some additional things to know.

Docker for Mac
• Docker is not natively compatible with macOS, so Hyperkit is used to
run a virtual image. Its virtual image data is located in:
• ~/Library/Containers/com.docker.docker/Data/vms/0
• Within the virtual image, the path is the default Docker
path /var/lib/docker.
• You can investigate your Docker root directory by creating a shell in
the virtual environment:
• $ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
• You can kill this session by pressing Ctrl+a, followed by
pressing k and y.

Docker for Windows
• On Windows, Docker is a bit fractioned. There are native Windows
containers that work similarly to Linux containers. Linux containers are
run in a minimal Hyper-V based virtual environment.
• The configuration and the virtual image to execute linux images are
saved in the default Docker root folder.
• C:ProgramDataDockerDesktop
• If you inspect regular images then you will get linux paths like:

• $ docker inspect nginx ... "UpperDir":
"/var/lib/docker/overlay2/585...9eb/diff" ...
You can connect to the virtual image by:
• docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -i sh
• There, you can go to the referenced location:
• $ cd /var/lib/docker/overlay2/585...9eb/ $ ls -lah drwx------ 4 root
root 4.0K Feb 6 06:56 . drwx------ 13 root root 4.0K Feb 6 09:17 ..
drwxr-xr-x 3 root root 4.0K Feb 6 06:56 diff -rw-r--r-- 1 root root 26
Feb 6 06:56 link -rw-r--r-- 1 root root 57 Feb 6 06:56 lower
drwx------ 2 root root 4.0K Feb 6 06:56 work

The internal structure of the Docker root folder
• Inside /var/lib/docker, different information is stored. For example, data
for containers, volumes, builds, networks, and clusters.
• $ ls -la /var/lib/docker total 152
• drwx--x--x 15 root root 4096 Feb 1 13:09 .
• drwxr-xr-x 13 root root 4096 Aug 1 2019
• .. drwx------ 2 root root 4096 May 20 2019 builder
• drwx------ 4 root root 4096 May 20 2019 buildkit
• drwx------ 3 root root 4096 May 20 2019 containerd
• drwx------ 2 root root 12288 Feb 3 19:35 containers
• drwx------ 3 root root 4096 May 20 2019 image
• drwxr-x--- 3 root root 4096 May 20 2019 network

Docker Volumes
• It is possible to add a persistent store to containers to keep data longer
than the container exists or to share the volume with the host or with
other containers.
• Volumes
• Volumes are the preferred mechanism for persisting data generated by
and used by Docker containers. While bind mounts are dependent on
the directory structure and OS of the host machine, volumes are
completely managed by Docker.

Volumes have several advantages over bind mounts:
• Volumes are easier to back up or migrate than bind mounts.
• You can manage volumes using Docker CLI commands or the Docker
API.
• Volumes work on both Linux and Windows containers.
• Volumes can be more safely shared among multiple containers.
• Volume drivers let you store volumes on remote hosts or cloud
providers, encrypt the contents of volumes, or add other functionality.
• New volumes can have their content pre-populated by a container.
• Volumes on Docker Desktop have much higher performance than bind
mounts from Mac and Windows hosts.
• In addition, volumes are often a better choice than persisting data in a
container's writable layer, because a volume doesn't increase the size
of the containers using it, and the volume's contents exist outside the
lifecycle of a given container.

Storage drivers versus Docker volume
• Docker uses storage drivers to store image layers, and to store data in
the writable layer of a container.
• The container's writable layer doesn't persist after the container is
deleted, but is suitable for storing ephemeral data that is generated at
runtime.
• Storage drivers are optimized for space efficiency, but (depending on
the storage driver) write speeds are lower than native file system
performance, especially for storage drivers that use a copy-on-write
filesystem.
• Write-intensive applications, such as database storage, are impacted
by a performance overhead, particularly if pre-existing data exists in
the read-only layer.
• Use Docker volumes for write-intensive data, data that must persist
beyond the container's lifespan, and data that must be shared between
containers

• This Dockerfile contains four commands.
• Commands that modify the filesystem create a layer.
• The FROM statement starts out by creating a layer from
the ubuntu:22.04 image.
• The LABEL command only modifies the image's metadata, and doesn't
produce a new layer.
• The COPY command adds some files from your Docker client's current
directory.
• The first RUN command builds your application using
the make command, and writes the result to a new layer.
• The second RUN command removes a cache directory, and writes the
result to a new layer.

• Finally, the CMD instruction specifies what command to run within the
container, which only modifies the image's metadata, which doesn't
produce an image layer.
• Each layer is only a set of differences from the layer before it. Note
that both adding, and removing files will result in a new layer. In the
example above, the $HOME/.cache directory is removed, but will still
be available in the previous layer and add up to the image's total size.

Images and layers
A Docker image is built up from a series of layers.
• Each layer represents an instruction in the image's Dockerfile.
• Each layer except the very last one is read-only. Consider the following
Dockerfile:
# syntax=docker/dockerfile:1
FROM ubuntu:22.04
LABEL org.opencontainers.image.authors="org@example.com"
COPY . /app
RUN make /app
RUN rm -r $HOME/.cache
CMD python /app/app.py

• The layers are stacked on top of each other.
• When you create a new container, you add a new writable layer on top
of the underlying layers.
• This layer is often called the "container layer".
• All changes made to the running container, such as writing new files,
modifying existing files, and deleting files, are written to this thin
writable container layer.
•

The diagram below shows a container based on
an ubuntu:15.04 image.

Container and layers
• The major difference between a container and an image is the top
writable layer. All writes to the container that add new or modify
existing data are stored in this writable layer. When the container is
deleted, the writable layer is also deleted. The underlying image
remains unchanged.
• Because each container has its own writable container layer, and all
changes are stored in this container layer, multiple containers can
share access to the same underlying image and yet have their own
data state.

The diagram below shows multiple containers sharing the
same Ubuntu 15.04 image.

The copy-on-write (CoW) strategy
• Copy-on-write is a strategy of sharing and copying files for maximum
efficiency.
• If a file or directory exists in a lower layer within the image, and
another layer (including the writable layer) needs read access to it, it
just uses the existing file.
• The first time another layer needs to modify the file (when building the
image or running the container),
• the file is copied into that layer and modified.
• This minimizes I/O and the size of each of the subsequent layers. e
depth below.

These advantages are explained as:
• Sharing promotes smaller images
• Copying makes containers efficient

Docker Daemon
• Docker daemon helps in listening requests for the Docker API
and in managing Docker objects such as images, containers,
volumes, etc. Daemon issues building an image based on a
user’s input and then saving it in the registry.
• In case we do not want to create an image, then we can simply
pull an image from the Docker hub, which might be built by some
other user. In case we want to create a running instance of our
Docker image, then we need to issue a run command that would
create a Docker container.
• A Docker daemon can communicate with other daemons to
manage Docker services

Docker Registry
• The Docker registry is a repository for Docker images that are
used for creating Docker containers.
• We can use a local or private registry or the Docker hub, which is
the most popular social example of a Docker repository.
• Now that we are through the Docker architecture and understand
how Docker works, let us get started with the installation and
workflow of Docker and implement important Docker commands.

Docker Engine
• It is the underlying client-server technology that supports the tasks and
workflows involved in building, shipping and running containerized
applications using Docker's components and services.
• Used alone, the term Docker can refer either to Docker Engine or to
the company Docker Inc., which offers various editions of
containerization technology around Docker Engine

Components of Docker Engine
• Docker Engine is an Open Source technology comprising a server with
a daemon process called dockerd, a REST API and a client-side
command-line interface (CLI) called docker.
• The engine creates a server-side daemon process that hosts
images, containers networks and storage volumes. The CLI lets users
interact with the Docker daemon via the API.
• Docker Engine is declarative, meaning that administrators program a
specific set of conditions as the desired state .
• The engine then automatically adjusts the actual settings and
conditions to ensure they match the desired state at all times.

Docker Engine vs. Docker Machine
• Docker Engine was initially developed for Linux systems and has since
been extended to operate natively on both Windows and macOS.
• Docker Machine is a tool used to install and manage Docker Engine on
various virtual hosts or older versions of macOS and Windows.
• When Docker Machine is installed on the local system, executing a
command through Docker Machine not only creates virtual hosts, but
also installs Docker and configures its clients.
• As of 2021, Docker Inc. no longer actively maintains Docker Machine
and recommends the Docker Desktop application for macOS and
Windows container development

Docker Engine plugins and storage volumes
Docker Engine can use a range of plugins, available as images hosted
in a private registry or public repository such as GitHub or Docker Hub
Admins can manage a plugin's entire lifecycle with Docker Engine,
from installation to deletion.
• Plugins create items such as data volumes, which are directories that
exist in a container. There are three types of volumes:
• Host volumes live in the file system.
• Named volumes are managed by Docker on the disk where the
volume is created and named.
• Anonymous volumes are similar to named volumes, but are not
associated with a specific source outside the container, making them
more difficult to reference.

Docker Container Life Cycle
The process of a Docker container is not simple. An overview of
its life cycle goes as such:
• Creating the container
• Running the Docker container with the necessary images and
commands
• Pausing the processes that are running inside the container
• Unpausing the processes in the container
• Starting, stopping, and restarting the container
• Killing and destroying the container

• Docker Engine creates a data volume concurrently with a container
image and can include data copied from a parent image. Containers
can share and reuse volumes, and volumes are not deleted when a
container is deleted.
• Because Docker Engine does not delete or collect alienated data
volumes, users are responsible for their data volumes.

Networking in Docker Engine
• Docker Engine provides default network drivers for users to create
unique bridge networks that control container communication. Docker
Inc. recommends that users define their own bridge networks for
security purposes.
• Containers can connect to multiple or no networks, and can connect
and disconnect from networks without disrupting container operation.
Docker Engine includes three network models:
51

• Bridge adds containers to the default docker0 network.
• None adds containers to a container-specific network stack, but does
not give containers external network access.
• Host adds containers to the host's network stack, with no isolation
between the host machine and containers.
52

• Users can also create network driver plugins if Docker's three standard
network types don't suit their needs.
• These plugins feature the same restrictions and rules as other plugins
and use the plugin API
53

• Created: A container that has been created but not started
• Running: A container running with all its processes
• Paused: A container whose processes have been paused
• Stopped: A container whose processes have been stopped
• Deleted: A container in a dead state
•

Phase of Docker Containers
• Create -> Destroy
• Create -> Start -> Stopped -> Destroy
• Create -> Start -> Pause -> Unpause
• Create -> Start -> Restart

Difference between Docker Create, Docker Start And Docker
Run

Difference Between Docker Pause And Docker Stop container

Docker rm Vs. Docker Kill
• Docker container rm: Using docker rm, we can remove one or
more containers from the host node and for doing container
name or ID can be used.
• Docker container kill: The main process inside each container
specified will be sent SIGKILL or any signal specified with option
–signal.

Concept of Hyper -V
• A hypervisor can be defined as the software creating an abstraction layer
between the virtual OS and the physical host machine. This helps create and
run multiple VMs on a single physical machine.
• Similarly, Hyper-V is also known as the virtualization technology using
Windows hypervisor to perform its primary function.
• However, it requires a physical processor with specific features,
including VM monitor mode extensions, a 64-bit processor with second-level
address translation (SLAT), and up to 4GB of RAM.
• The purpose of a hypervisor is to manage interactions between the physical
Hyper-V server and the VMs. The hypervisor provides an isolated
environment to the VMs by controlling the access of the host hardware
resources.
• This helps eliminate system crashes and makes VMs more flexible, efficient,
and convenient.

Practice questions:
• How does Docker differ from traditional virtualization?
• How does Docker facilitate portability of applications?
• Can Docker containers run on Windows and Linux interchangeably?
• What is the significance of Docker volumes in data management?
• How does Docker contribute to a DevOps workflow

Introduction to Docker, Devops Virtualization and configuration management

More Related Content

Similar to Introduction to Docker, Devops Virtualization and configuration management (20)

Recently uploaded (20)

Introduction to Docker, Devops Virtualization and configuration management