Devoxx 2016 - Docker Nuts and Bolts
Download as PPTX, PDF6 likes1,461 views
The document presents an overview of Docker and its evolution from 1995 to 2015, highlighting the company's emphasis on developer experience, orchestration capabilities, and containerization technologies. It discusses the cultural shift towards DevOps, continuous delivery, and the use of Docker to streamline deployment processes, including examples of practical applications and tools such as Docker Compose and Swarm. Furthermore, it outlines Docker's goals in enhancing agility, portability, and control across complex enterprise environments, alongside a brief introduction to the Infrakit toolkit for managing infrastructure.
![docker-compose: running multiple containers
Run your stack with one command: docker-compose up
Describe your stack with one file: docker-compose.yml
version: '2'
services:
web:
image: chanezon/spring-doge
ports:
- "8080:8080"
links: ["mongo"]
environment:
- MONGODB_URI=mongodb://mongo:27017/test
mongo:
image: mongo](https://image.slidesharecdn.com/docker-oct-2016-devoxx-university-161108173836/85/Devoxx-2016-Docker-Nuts-and-Bolts-34-320.jpg)
![Configuration
Example config file (zk.conf): Group configuration = Instance + Flavor
{
"Properties": {
/* raw configuration */
}
}
{
"groups" : {
"my_zookeeper_nodes" : {
"Properties" : {
"Instance" : {
"Plugin": "instance-vagrant",
"Properties": {
"Box": "bento/ubuntu-16.04"
}
},
"Flavor" : {
"Plugin": "flavor-zookeeper",
"Properties": {
"type": "member",
"IPs": ["192.168.1.200", "192.168.1.201", "192.168.1.202"]
}
}
}
}
}
}](https://image.slidesharecdn.com/docker-oct-2016-devoxx-university-161108173836/85/Devoxx-2016-Docker-Nuts-and-Bolts-79-320.jpg)
Devoxx 2016 - Docker Nuts and Bolts
- 1. Patrick Chanezon @chanezon, Docker Inc. Docker Nuts and Bolts
- 2. French Polyglot Platforms Software Plumber San Francisco Developer Relations @chanezon
- 3. 1995 2015
- 7. Agility
- 10. Agile methodologies (circa 1999)
- 12. Low MTBIAMSH MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)
- 13. Agility == $$
- 14. Devops
- 15. 25 Mainframe
- 16. Client-Server 26
- 17. 27 Web
- 19. Devops • Cultural movement • Inspired by agile methods • People, Processes & Tools • Continuous delivery • Infrastructure as code • Cross silo collaboration • Small iterations • Feedback loop, measurement Image from Patrick Debois http://www.slideshare.net/jedi4ever/devops-the-war-is-over-if-you-want-it http://www.slideshare.net/jedi4ever/devopsdays-downundervfinal
- 21. 28 http://highscalability.com/blog/2013/11/19/we-finally-cracked-the-10k-problem-this-time-for-managing-se.html Server/Sysadmin 1999: 5(Windows) - 50 (Linux) 2015: 10k-20k x2000
- 22. 28 https://blog.docker.com/2014/12/dockercon-europe-keynote-continuous-delivery-in-the-enterprise-by-henk-kolk-ing/ Henk Kolk, ING, DockerCon EU 2014 People, Processes, Products deployment time: 9 months -> 15 min 1500 deployments/week
- 23. Docker Platform
- 24. The world needs tools of mass innovation
- 25. A programmable Internet would be the ultimate tool of mass innovation
- 26. A commercial product, built on a development platform, built on infrastructure, built on standards. Docker is building a stack to program the Internet
- 27. Isolation using Linux kernel features namespaces pid mnt net uts ipc user cgroups memory cpu blkio devices
- 28. Image layers
- 30. Using Docker to compile your jar/war https://registry.hub.docker.com/_/maven/ docker run -it --rm -v $PWD:/usr/src/spring-doge -v maven:/root/.m2 -w /usr/src/spring-doge maven:3.3-jdk-8 mvn package
- 31. Analyzing images docker images java docker history java:8
- 32. Build an image docker build -t chanezon/spring-doge . FROM java:8 MAINTAINER Patrick Chanezon <[email protected]> EXPOSE 8080 COPY spring-doge/target/*.jar /usr/src/spring-doge/spring- doge.jar WORKDIR /usr/src/spring-doge CMD java -Dserver.port=8080 - Dspring.data.mongodb.uri=$MONGODB_URI -jar spring-doge.jar
- 33. Run a container docker run —env MONGODB_URI=mongodb://mongo:27017/test -p 8090:8080 chanezon/spring-doge
- 34. docker-compose: running multiple containers Run your stack with one command: docker-compose up Describe your stack with one file: docker-compose.yml version: '2' services: web: image: chanezon/spring-doge ports: - "8080:8080" links: ["mongo"] environment: - MONGODB_URI=mongodb://mongo:27017/test mongo: image: mongo
- 35. What’s New?
- 37. 1. Get out of the way The best tools… 2. Adapt to you 3. Make the powerful simple
- 38. Docker for Mac Docker for Windows
- 39. 2. Orchestration
- 40. ng the best way to orchestrate Docke Docker 1.12: now with orchestration built-in.
- 41. Swarm mode Service API Cryptographic node identity Built-in routing mesh Docker 1.12: now with orchestration built-in.
- 42. Using the beta? You already have 1.12 installed. > docker swarm init > docker service create
- 44. Deep integration with native load-balancers, templates, SSH keys, ACLs, scaling groups, firewall rules… beta.docker.com
- 45. Distributed Application Bundle www.docker.com/dab A portable format for multi-container applications
- 46. Docker CaaS
- 47. Goals + + Agility Portability Control
- 48. Cloud Zone 1 Cloud Zone 2 Data Center Development Center Headquarters Containerization: standard containers on a standardized container engine Orchestration: build and deploy complex systems easily Enterprise: Enable delivery and time to value across a large, complex, rapidly evolving enterprise The Global Enterprise Supply Chain
- 49. BUILD Development Environments SHIP Registry: Secure Content & Collaboration RUN Control Plane: Deploy, Orchestrate, Manage, Scale Networking Volumes MonitoringLoggingConfig MgtCI/CD IT Operations Developers IT Operations Docker CaaS Workflow
- 50. Docker Containers as a Service platform 50 BUILD Developer Workflows SHIP Registry Services RUN Management Docker for Mac and Windows Docker Trusted Registry Docker Universal Control Plane Docker Cloud Docker Container Engine Ecosystem Plugins and Integrations
- 51. Docker UCP 1.1 - DTR 2.0 • HA • Unified Auth • Compose deployment • UI to add nodes
- 53. Security scanning in Docker Cloud
- 54. Plumbing
- 56. 2013-05 2013-06 2013-07 2013-08 2013-09 2013-10 2013-11 2013-12 2014-01 1,000,000 0 2014-02 2014-03 2014-04 2014-05 2014-06 2014-07 2014-08 2014-09 2014-10 2014-11 2014-12 2015-01 2015-02 2015-03 2015-04 2015-05 2015-06 2015-07 2015-08 2015-09 2015-10 2015-11 2015-12 2016-01 ~ 2016-09 1,000,000,000 ~ 10,000,000 9,000,000 8,000,000 7,000,000 6,000,000 5,000,000 4,000,000 3,000,000 2,000,000 6,000,000,000 5,750,000,000 5,500,000,000 5,250,000,000 5,000,000,000 4,750,000,000 4.500,000,000 4,250,000,000 4,000,000,000 3,750,000,000 3,500,000,000 3,250,000,000 3,000,000,000 2,750,000,000 2,500,000,000 2,250,000,000 2,000,000,000 1,750,000,000 1,500,000,000 1,250,000,000 Notary runC • containerd • HyperKit , VPNKit, DataKit • SwarmKit • libcontainer • libnetwork • • Docker 1.8 : Docker Content Trust • Docker for Mac Docker for Windows • Docker 1.12 with built-in orchestration • Docker 0.9 : Pluggable execution • Docker 1.7 : Multi-Host Networking • Docker 1.11: OCI support
- 57. Notary “Let’s stop using curl|sh” Trusted collections for any content Transport-agnostic Reliable updates, proof of origin, resistant to untrusted transport, survivable key compromise Build on industry-leading standards and research
- 58. RunC The universal container runtime https://runc.io
- 59. containerd A daemon to control runC built for performance and density http://containerd.tools/
- 60. containerd
- 61. Docker 1.11
- 62. Docker for Mac architecture (simplified)
- 63. Hypervisor Framework vmnet Framework Docker Container Engine Hypervisor Linux VPN Data Service Interface Client Libraries Admin GUI CLI Security Sandbox Docker for Mac internals
- 65. Hypervisor Framework vmnet Framework Docker Container Engine Hyperkit Linux VPNKit DataKit Client Libraries Admin GUI CLI Security Sandbox Improving Docker with unikernel tech
- 66. InfraKit
- 67. Problem: Managing Docker on different infrastructure is difficult and not portable.
- 68. Consistent User Experience 68 How do we handle updates to a cluster??
- 69. Docker for AWS EBS ELB Container Engine Storage plugin Infrastructure Management Network plugin Orchestration IAM CloudFormation EC2VPC Admin interface Linux User Applications / Services
- 70. Docker for AWS EBS ELB Container Engine Storage plugin InfraKit Network plugin Orchestration IAM CloudFormation EC2VPC Admin interface Linux User Applications / Services
- 71. InfraKit A toolkit for building declarative, self-healing infrastructure.
- 72. Declarative • JSON configuration for desired infrastructure state: • Specification of instances — vm image, instance type, etc. • Group properties — size, logical identifiers, etc. • Design patterns encourage • encapsulation • composition • Config is input to all operations — system figures out what to do 72
- 73. Self-healing • Composed of a set of active components / processes that • monitor infrastructure state • detect state divergence • take actions • Continuous monitoring and reconciliation — always on • No downtime — rolling update 73
- 74. Toolkit • Primitives for managing collections of resources • create, scale, destroy • rolling update • Abstractions & Developer SPI • Group - manages collection of resources • Instance - describes the physical resource • Flavor - extra semantics for handling instances • A collection of executable, active components — plugins • Initially, Go daemons in the toolkit • Soon, easy management via Docker Plugins (runc)
- 75. Architecture
- 76. Instance Plugin • Spec: specification / model of an instance (e.g. vagrant, EC2): • Logical ID, Init, Tags, and attachment • Platform-specific properties • Methods: • /Instance.Validate • /Instance.Provision • /Instance.Destroy • /Instance.DescribeInstances • Examples: instance plugins for EC2, Azure VM, Vagrant, … 76
- 77. Flavor Plugin • Gives more context about the group members: • Size, or list of Logical ID’s (e.g. IP addresses for ‘pets’) • Application-specific notions of ‘health’ Is the node not only present but also joined a swarm? • Methods: • /Flavor.Validate • /Flavor.Prepare • /Flavor.Healthy • Examples: flavor for Zookeeper members, Docker swarm nodes 77
- 78. Group Plugin • Main entry point for user interaction: • Create, describe update, update, destroy • Config JSON is always the input • Composed of Instance and Flavor — mix and match to manage cattle (fungible) or pets (special) • Methods: • /Group.Watch • /Group.Unwatch • /Group.Inspect 78 • /Group.DescribeUpdate • /Group.Update • /Group.StopUpdate • /Group.Destroy
- 79. Configuration Example config file (zk.conf): Group configuration = Instance + Flavor { "Properties": { /* raw configuration */ } } { "groups" : { "my_zookeeper_nodes" : { "Properties" : { "Instance" : { "Plugin": "instance-vagrant", "Properties": { "Box": "bento/ubuntu-16.04" } }, "Flavor" : { "Plugin": "flavor-zookeeper", "Properties": { "type": "member", "IPs": ["192.168.1.200", "192.168.1.201", "192.168.1.202"] } } } } } }
- 80. Operations • Make sure the plugins are running: • infrakit/group &; infrakit/zookeeper &; infrakit/vagrant &; • “Watch” the group starts management: • infrakit/cli group watch zk.conf • Update the config, e.g. change size or add IP address • Describe changes before committing — infrakit/cli group describe zk.conf • Begin update — infrakit/cli group update zk.conf 80
- 81. Today 81 • InfraKit is just getting started… only primitives for working with groups like clusters of hosts • But we have big plans • Improve group management strategies • More resource types — networking, load balancers, storage… • A cohesive framework for active management of infrastructure — physical, virtual, or containers
- 82. Get Involved • Help define and implement new and interesting plugins • Instance plugins for different infrastructure providers • Flavor plugins for systems like etcd or mysql clusters • Group controller plugins — metrics-driven auto scaling and more • Help define interfaces and implement new infrastructure resource types — load balancers, networks and storage volume provisioners 82
- 83. More Info • Github: https://github.com/docker/infrakit • A quick tutorial: https://github.com/docker/infrakit/blob/master/docs/tutorial.m d 83
- 84. Demo
- 85. Linux X11 Apps on Docker for Mac https://github.com/chanezon/docker-tips/x11
- 86. • Docker Swarm, Compose and networking • docker 1.11 • swarm 1.1.0 • compose 1.6.0 with networking • Run/Debug with STS IDE in a container Spring Boot App using MongoDB https://github.com/joshlong/spring-doge https://github.com/chanezon/docker-tips/ https://github.com/chanezon/spring-doge
- 87. Docker Java Labs https://github.com/docker/labs/tree/master/developer-tools/ • Wildfly and Couchbase J2EE App • Debugging a Java app in Docker using Eclipse
- 88. Docker for AWS https://beta.docker.com/docs/aws/
- 89. THANK YOU
Editor's Notes
- #50: Local development environments Self service app images Build, Test, Deploy applications Define app behavior and infra needs Registry services for image storage, management and distribution IT Ops maintains library of secure base content Manage role based access to repos/images Management consoles Provision, manage infrastructure resources Monitor, manage, scale infrastructure and applications
- #69: Docker for Mac is super easy to use. It self-updates. But how to do you do that for other platforms??
- #70: Infrastructure management holds an interesting position — it has to prepare the environment for the container engine.
- #71: Infrastructure management sets up the environment for the container engine. It then collaborates with it to service the user’s needs.
- #75: integration in the future as Docker Plugins (as runc containers) goes back to initial goal of simplifying user experience through a common Docker plugin install experience
- #76: InfraKit can be used set up to run with leader election (e.g. integrated with Docker swarm mode) to achieve high availability. Multiple plugin sets are running as hot standby but only one is active (the leader).
- #77: no method to update !!— opinionated immutable infrastructure
- #78: Prepare modifies the Spec for the instance. It can inject additional init scripts, tags. Healthy goes beyond just presence — is a node present AND part of a swarm?
- #79: No operations to create: watching nodes that don’t exist —> will create them … declarative state explain cases for using each. why explicit is better than implicit (use unwatch / destroy example)
- #80: Simple patterns Compose them into a large configuration. Group is made up of Instance and Flavor
- #81: Note the config is used for all inputs. The system figures out what needs to be done. Only signals to system to start managing Docker plugins in a future implementation