Introduction to Docker and Monitoring with InfluxData
0 likes883 views
This document provides an overview of Docker, its applications, and how it manages time series data in container orchestration. It discusses Docker's features, such as images and containers, the use of Docker Compose and Swarm for deployment, and new updates in Docker 17.06. Additionally, it introduces InfluxData's tools for handling time series data, including Telegraf and InfluxDB, and outlines their architecture and performance capabilities.
Introduction to Docker and Monitoring with InfluxData
- 1. © 2017 InfluxData. All rights reserved.1 © 2017 InfluxData. All rights reserved.1 An introduction to Docker and how to handle time series data in container orchestration 25 SEP 2017 Introduction to Docker & Monitoring with InfluxData
- 2. © 2017 InfluxData. All rights reserved.2 © 2017 InfluxData. All rights reserved.2 Our Speakers Today Gary Forghetti Technical Alliance Engineer Gunnar Aasen Partner Engineering
- 3. © 2017 InfluxData. All rights reserved.3 © 2017 InfluxData. All rights reserved.3 Agenda ¨Introduction to Docker, Images & Containers ¨Docker Compose & Swarm ¨What’s new in Docker 17.06 ¨Understanding the time series problem ¨Introduction to InfluxData ¨Q & A
- 4. Deploying Applications with Docker Gary Forghetti Technical Alliance Engineer Business Development Docker, Inc.
- 5. ● Overview of Docker ● Images and Containers ● docker-compose ● Docker Swarm ● What's new in Docker 17.06 Agenda
- 6. ● Founded in 2013 as Linux developer tool. ● Docker solves the “works on my machine” problem. ● Docker allows you to transform and modernize your applications and infrastructure, and reduce costs. ● Docker provides portability, agility and efficiency and has built-in security. ● Docker can be run on physical hardware, desktops, and on virtual machines in public or private clouds. Docker Overview Docker is the world's leading software container platform.
- 7. ● The Docker platform contains multiple products and tools for developing, testing, deploying, and managing applications packaged in containers. ● Docker technology focuses on convenience, ease of use and enablement. ● Docker runs on Linux, macOS and Windows ● Docker Editions ○ Docker Community Edition (free, community support) ○ Docker Enterprise Edition (subscription, business day and business critical support) ● Docker Datacenter (container as a service platform) ○ Docker Universal Control Plane (management UI) ○ Docker Trusted Registry (image storage) ● Docker Datacenter is included with Docker EE Standard and Advanced Docker Overview Docker is the world's leading software container platform.
- 8. Docker Images and Containers
- 9. ● A Docker Image is a template which is used to build a running Docker Container. ● It contains the application code, required binaries, libraries, configuration files and metadata needed to run the application. ● Docker Images do not contain a full operating system. ○ No system kernel or kernel modules. ● Docker Images are stored and shared in registries. ○ Docker Hub (default), Docker Store, and private registries. ● Think of a Docker Image as a "container at rest". Docker Images
- 10. Understanding Image Layers ● An image is a collection of files and some meta data. ● Images are comprised of multiple layers. ● A layer is also just another image. ● Each image contains the software you want to run. ● Every image contains a base layer. ● Docker uses a copy on write system. ● Layers are read only. Docker Images
- 11. ● Docker Containers are processes running applications. ● Docker Containers are created from Docker Images. ● Docker Containers are lightweight, standalone and portable. ● Unlike virtual machines they do not contain a full operating system. ● No system kernel or kernel modules. ● Docker Containers use the system kernel on the host machine (Docker Node). Docker Containers Docker containers are based on open standards and run on all major Linux distributions, macOS, Microsoft Windows, and on any infrastructure including VMs, bare-metal and in the cloud.
- 12. Comparing an application running on physical hardware, on virtual machines and in Docker Containers.
- 13. Limitations: ● Slow deployment times ● High costs ● Wasted resources ● Difficult to scale ● Difficult to migrate ● Vendor lock in Application running on a Physical Server One Application on One Physical Server
- 14. Benefits: ● Multiple containers can run on the same physical machine or virtual machine. ● Containers run as isolated processes and share the system resources CPU, Ram and Storage. ● Containers do not contain a full operating system, they all share the same operating system kernel. ● Containers take up less space and start quicker. ● Smaller attack surface from a security perspective. ● Containers are easily scaled. ● Containers are portable. Application running in a Docker Container Docker Containers Used together, Docker Containers and VMs provide a tremendous amount of flexibility to optimally deploy and manage applications.
- 15. Benefits: ● Better resource pooling ● One physical machine divided into multiple virtual machines ● Easier to scale ● VMs in the cloud ● Rapid elasticity ● Pay as you go model Application running on a Virtual Machine Hypervisor-Based Virtualization Limitations: ● Each VM stills requires: ○ CPU allocation ○ RAM ○ Storage ○ An entire guest operating syst ● Full guest OS means wasted resources ● Application portability not guaranteed
- 17. ● Stack - a collection of Services that make up an application. ● Service - provides a function and is based on a docker image. ○ Some examples of Services: ■ Load Balancer ■ Web Frontend Application ■ Business Logic Backend Application Database ● Task - an individual container running in a Service. Service can have 1-n Tasks (replicas) Terminology
- 18. Manual Application Stack deployment with Docker.
- 19. Manual Application Stack deployment
- 20. Automate Application Stack deployments with docker-compose.
- 21. docker-compose is a Docker tool which allows you to deploy multi-container applications (Stacks) on a docker node in an automated fashion. ● You define your application Stack as one or more Services in a compose yaml file Compose file documentation: https://docs.docker.com/compose/compose-file/ ● The single command "docker-compose" is then used to create and start the Services in the application Stack. ● The "docker-compose" command is also used to terminate and remove the Services in the application Stack. ● docker-compose is intended for development and testing. What is docker-compose?
- 22. Docker Application Stack Yaml file template
- 24. Create the Stack
- 25. Stack Deployed and running
- 26. Docker Swarm
- 27. Clustering and container scheduling. ● Allows you to scale up/down ● Allows you to perform upgrades and roll backs (blue/green deployments) ● Has built-in service discovery and load balancing ● Ensures containers are restarted if they fail ● Allows you to control where containers run (which nodes) ● Has built-in End to End Security ○ Ensure only trusted servers are running your containers ○ Provides a mechanism to store and retrieve secure secrets, passwords and keys, and restrict which containers can access them ● Supports Windows and Linux workloads in the same swarm ● Application Stack is defined as one or more services in a yaml file ● Single command deploys the Stack ("docker stack deploy") and destroys the Stack ("docker stack rm") Docker Swarm is not enabled by default. What is Docker Swarm?
- 29. ● docker swarm (manage the swarm) https://docs.master.dockerproject.org/swarm/reference/ ○ ca - Display and rotate the root CA ○ init - Initialize a swarm ○ join - Join a swarm as a node and/or manager ○ join-token - Manage join tokens ○ leave - Leave the swarm ○ unlock - Unlock swarm ○ unlock-key - Manage the unlock key ○ update - Update the swarm ● docker node (manage nodes in the swarm) https://docs.docker.com/engine/reference/commandline/node/ ○ demote - Demote one or more nodes from manager in the swarm ○ inspect - Display detailed information on one or more nodes ○ ls - List nodes in the swarm ○ promote - Promote one or more nodes to manager in the swarm ○ ps - List tasks running on one or more nodes, defaults to current node ○ rm - Remove one or more nodes from the swarmu ○ update - Update a node Docker Swarm commands
- 30. ● docker stack (manage stacks) https://docs.docker.com/engine/reference/commandline/stack/ ○ deploy - Deploy a new stack or update an existing stack ○ ls - List stacks ○ ps - List the tasks in the stack ○ rm - Remove one or more stacks ○ services - List the services in the stack ● docker service (manage services) https://docs.docker.com/engine/reference/commandline/service/ ○ create - Create a new service ○ inspect - Display detailed information on one or more services ○ logs - Fetch the logs of a service or task ○ ls - List services ○ ps - List the tasks of one or more services ○ rm - Remove one or more services ○ scale - Scale one or multiple replicated services ○ update - Update a service ● docker secret (manage secrets) https://docs.docker.com/engine/reference/commandline/secret/ ○ create - Create a secret from a file or STDIN as content ○ inspect - Display detailed information on one or more secrets ○ ls - List secrets ○ rm - Remove one or more secrets Docker Swarm commands
- 31. Docker Service
- 32. Automate and manage Application Stacks with Docker Swarm.
- 33. Docker Swarm
- 34. Create the Docker Swarm
- 36. ● Pets is a simple multi service dockerized application written by a Docker employee ○ Displays random pictures of pets and allows you to "vote" on your favorites. ● Pets is written in Python and uses the Flask Python Web Framework ● Pets consists of 2 Docker Services: ○ Pets Python Flask Frontend application ■ Custom Docker image: chrch/docker-pets:1.0 ○ Database backend key value store used by the Pets application to store state data ■ Official Docker image: consul:0.7.2 ● Pets is used a lot for demos and has also been used in labs at DockerCon ● Pets is available in a public GitHub repo -> https://github.com/mark-church/docker-pets Pets - Example Docker Application
- 37. Docker Application Stack Yaml file
- 38. Deploy the Stack
- 39. Display the Stack, Services, and Tasks
- 40. Display the Tasks for a Service
- 41. Docker Swarm Overlay Multi-Host Networking
- 42. Docker Swarm Ingress Network Routing
- 43. Docker Swarm Ingress Network Routing
- 44. Docker Swarm Ingress Network Routing
- 45. Recovery
- 46. Update the pets_db Service with a new Docker Image
- 47. Update the pets_db Service
- 48. What's new in Docker 17.06
- 49. ● Support for IBM Z and Windows 2016 Server ● Custom Roles ○ Now have the ability to create custom roles with very granular access (down to a specific API function) ● Role Based Access Control for nodes ○ Restrict which users and teams can deploy to which nodes ● Mixed cluster support ○ Run Linux and Windows services in same cluster ○ Use placement constraints to run Windows containers on Windows nodes and Linux containers on Linux nodes in the cluster ● Policy-Based Automation ○ Automate image promotion using pre-defined policies to move images from one repository to another within the same registry ○ Create immutable repositories to prevent image tags from being modified or deleted, ensuring that production application repositories are secure and locked down ● Multi-stage Builds ○ Create Dockerfile with multiple FROM statements and copy artifacts from 1 stage to another. Allows you to build a docker image with less clutter and reduce it's size ● More info: https://blog.docker.com/2017/08/docker-enterprise-edition-17-06/ What's new in Docker 17.06
- 51. ● What's new in Docker Enterprise Edition ○ https://www.youtube.com/watch?v=yGkoSsnKlQk ● Docker Modernizing Traditional Applications Program ○ https://blog.docker.com/2017/04/modernizing-traditional-apps-with-docker/ ● DockerCon Europe 2017 – October 16-19 ○ https://europe-2017.dockercon.com/ ● Docker Documentation ○ https://docs.docker.com/ ● Docker Community ○ https://www.docker.com/docker-community ● Docker Store ○ https://store.docker.com/ ● Docker Technology Partner Guide ○ https://www.docker.com/sites/default/files/community/Docker-Technology-Partner-Program-Guide-082517. pdf ● Docker Store Vendor Partner ○ https://store.docker.com/publisher/signup ● Docker Maintenance Lifecycle ○ https://success.docker.com/Policies/Maintenance_Lifecycle ● Docker Pricing ○ https://www.docker.com/pricing Additional Information
- 52. Vagrantfile for a 3 node Docker Swarm
- 53. Vagrantfile for a 3 node Docker Swarm
- 55. © 2017 InfluxData. All rights reserved.55 © 2017 InfluxData. All rights reserved.55 What makes time-series unique ¨Needs fast ingestion of data ¨High & low precision data have different retention policy ¨Software agent or DB must be able to compute ¨Unique & changing time query Containers generating huge data load ¨Regular vs. irregular ¨Containers themselves + components within Understanding the time series problem
- 56. © 2017 InfluxData. All rights reserved.56 © 2017 InfluxData. All rights reserved.56 Time to metrics… graphs… value… ¨Primary developers of InfluxDB ¨Created the TICK stack: platform for time series ¨Telegraf and Chronograf fully open source ¨InfluxDB and Kapacitor open core ¨Single nodes scale to 1 million points ingested per second ¨Enterprise clustered and high availability solution ¨InfluxCloud managed platform Introduction
- 57. © 2017 InfluxData. All rights reserved.57 © 2017 InfluxData. All rights reserved.57 Platform Components Telegraf ¨Collection of metrics and events ¨Many input plugins, deployed to worker nodes InfluxDB ¨High performance time-series database Chronograf ¨User Interface for InfluxDB & Kapacitor Kapacitor ¨Metrics processing and alerting engine
- 58. © 2017 InfluxData. All rights reserved.58 © 2017 InfluxData. All rights reserved.58 Demo
- 59. © 2017 InfluxData. All rights reserved.59 © 2017 InfluxData. All rights reserved.59 Learn More URL or other notes on where to find further info. URL or other notes on where to find further info. https://hub.docker.com/_/influxdb/ http://webinars.devops.com/how-to-use-the-tick-stack-coreos-docker-to-make-your-sa as-offering-better https://www.influxdata.com/integration/docker-monitoring-tools/ https://github.com/influxdata/TICK-docker https://github.com/influxdata/telegraf/tree/release-1.3/plugins/inputs/docker https://github.com/influxdata/telegraf/issues/2054 (contributions welcome!) https://docs.docker.com/compose/ https://docs.docker.com/engine/swarm/
- 60. © 2017 InfluxData. All rights reserved.60 © 2017 InfluxData. All rights reserved.60 Q & A?