Best Practices for Developing & Deploying Java Applications with Docker
- 1. Best Practices for Developing & Deploying Java Applications with Docker Eric Smalling - Solution Architect, Docker Inc. @ericsmalling JavaOne 2017 | CON7957
- 2. 2 Who Am I? ● Eric Smalling ○ Solution Architect Docker Customer Success Team ● ~25 years in software development, architecture, version control admin, etc… ● ~10 years in build & test automation ● Docker user since pre-1.0 days ● Java developer since 1.1.x days
- 3. Agenda ● Docker 101 ● Running a simple Java web application in Docker ● Services, stacks & deploying to clusters ● Application management & troubleshooting ● Application Configuration ● Q & A
- 4. Docker 101 A quick overview of Docker
- 5. Some Docker vocabulary Docker Image The basis of a Docker container. Represents a full application Docker Container The standard unit in which the application service resides and executes Docker Engine Creates, ships and runs Docker containers deployable on a physical or virtual, host locally, in a datacenter or cloud service provider Registry Service (Docker Hub or Docker Trusted Registry) Cloud or server based storage and distribution service for your images
- 6. Docker File System Images, Layers & Containers ● Logical file system by grouping different file system primitives into branches (directories, file systems, subvolumes, snapshots) ● Each branch represents a layer in a Docker image ● Allows images to be constructed / deconstructed as needed vs. a huge monolithic image (ala traditional virtual machines) ● When a container is started a writeable layer is added to the “top” of the file system
- 7. Docker File System Containers & Copy on Write ● Super efficient: Sub second instantiation times for containers New container can take <1 Mb of space ● Containers appears to be a copy of the original image But, it is really just a link to the original shared image ● If someone writes a change to the file system, a copy of the affected file/directory is “copied up”
- 8. Docker File System What about data persistence? ● Volumes allow you to specify a directory in the container that exists outside of the docker file system structure ● Can be used to share (and persist) data between containers ● Directory persists after the container is deleted Unless you explicitly delete it ● Can be created in a Dockerfile or via CLI
- 9. Dockerfile - Linux + Java Example: Initial state
- 10. Image Layers Kernel Ubuntu Linux 16:04 Update apt catalogs Install JDK and curl Download Tomcat Install Tomcat Copy webapp Start tomcat Initial State
- 11. Building the image The docker client command “build” = build an image “-t” = apply a name and optional build Image name and optional tag Path to build context and Dockerfile
- 12. Running the image in a container The docker client command “run” = start a container “--rm” = delete container when it exits “-t” = run with a tty (for console i/o) “-i” = run in interactive mode These often are used in combination like this Image name and optional tag
- 13. Demo Build and run demonstration
- 14. Dockerfile - Linux + Java Example: Optimization step 1
- 15. Image Layers Optimization Step 1 Kernel Ubuntu Linux 16:04 Update apt catalogs, install JDK and curl, clean up Download Tomcat Install Tomcat Copy webapp Start tomcat
- 16. Dockerfile - Linux + Java Example: Optimization step 2
- 17. Image Layers Optimization Step 2 Kernel OpenJDK:8-alpine Update apk catalogs, install curl Download Tomcat Install Tomcat Copy webapp Start tomcat
- 18. Dockerfile - Linux + Java Example: Fully Optimized
- 20. Deploying to Clusters Services, Stacks and Swarms
- 21. More terminology ● Swarm ○ A group of docker hosts, connected and running as a cluster ○ 1-n managers ○ 1-n workers ● Service ○ An application (or part of an application) that provides a specific function (catalog lookup, web front end, payment processing) ● Stack ○ A way of representing multi-service applications ○ Made up of 1-n services
- 22. Stack deploy demo Simple J2EE application deployment with 2 containers: ● React based front end ● Java based back end
- 23. Application Management Monitoring & Troubleshooting
- 24. Health Checks Helping Docker help you ● HEALTHCHECK instruction in DockerFile ● Tells Docker how to test a container to check that it is still working ● New status added to container lists ● Adds “(healthy)” to Status column in a “docker ps response”
- 25. Health Checks Helping Docker help you ● Examples: ○ HEALTHCHECK CMD curl --fail http://localhost || exit 1 ○ HEALTHCHECK --interval=12s --timeout=12s --start-period=30s CMD node /healthcheck.js ● References: ○ Documentation: https://docs.docker.com/engine/reference/builder/#healthcheck ○ Elton Stoneman blog about not using curl/iwr: https://t.co/Zgdd1lyzhk
- 26. JVM Memory Tips and tricks ● Always explicitly specify JVM heap size with “-Xmx” arguments ○ By default, J2SE 5.0+ will use up to 25% of the host machine’s RAM or 1GB (whichever is smaller) ○ Container memory limits (enforced via cgroups) are ignored* (*cgroup awareness is planned for Java 9) ○ It’s just a good practice to specify it anyway ● Do use Docker cpu and memory reservations and limits to avoid over-subscribing your host machines ○ --memory ○ --memory-reservation ○ --cpus ○ etc… ● If limiting cpu, be sure to update GC Thread limiter in JVM ○ -XX:ParallelGCThreads
- 27. Logging Dealing with application logs ● Docker EE Reference Architecture document about this: http://dockr.ly/logging ● Do not output logs into the container’s RW layer ○ slow ○ have to exec or cp out of the container to see them ● Option 1: send logs to stdout (see logging drivers below) ○ Visible via “docker logs” command ○ Visible via Docker UCP web console ● Option 2: send logs to volume ○ Many use a centralized NAS/SAN volume for this ● Option 3: Docker logging drivers
- 28. Docker Log Drivers Log drivers available (as of 9/4/17) Latest always available at: https://docs.docker.com/engine/admin/logging/overview/#supported-logging-drivers
- 29. Application Log Drivers Consider the following when selecting application log drivers: ● syslog and splunk: ○ Good options if log data is highly sensitive since they can be configured to use TLS for transporting logs. ● journald: ○ great for retaining the usage of docker logs as well as logging Docker daemon logs ○ allows for easier troubleshooting and log portability at the same time ○ logs write first locally, so that there is less reliance on logging infrastructure. ● awslogs or gcplogs: ○ Only if cluster exist solely on a single cloud provider
- 30. Application Log Drivers (continued) Consider the following when selecting application log drivers: ● gelf and fluentd: ○ good choice if there's a NoSQL database somewhere in the environment where the logs can be stored. Again, see http://dockr.ly/logging for much more detail on logging.
- 31. Troubleshooting How to use Java tools with container based JVMs ● JVM command line tools via docker exec ○ GC Stats: jstat --gcutil ○ Heap dumps/histograms: jmap ● Expose JMX ports for jconsole or other utilities ● Intelligent health checks ○ More than just “port 8080 is listening” ● Check third party monitoring tools for updated to be “container aware” ○ i.e. Licensing issues with older monitoring tools because each container appears as a new host ● Also, docker specific commands/tools: ○ docker stats ○ ctop
- 32. Application Configuration Managing multi-environment config’s
- 33. Application Configuration Deploying to disparate environments with identical images ● Build artifacts are your Docker images, not .war files or similar ● Build images in CI, store in registry, deploy same images everywhere ● Patterns and tools to deal with configuration differences ○ Separate Stack yaml files ○ Docker secrets ○ Application configuration via volume mounts ○ Third party configuration tools such as Consul and/or Vault ■ consul-template ■ Joyent Containerpilot ■ Roll-your-own
- 34. Environment specific Stacks ● Different environment variable values ● Services that mock production endpoints ○ db ○ web service prod.yml dev.yml
- 35. Docker Secrets ● Stored encrypted in swam ● Exposed only to nodes that run services that need them ● Presented in container via RAM only tmpfs files ○ never persisted to disk in encrypted format ○ when container stops, secret is no longer present ● All communications between swam nodes via TLS, so secret never in the clear on the wire either ● Different secret values per environment using tags ● UCP can manage who/where secrets are available
- 36. Application configuration in volume mounts ● Use volumes that are only available in physical environment they apply to ● Contain environment-specific application configuration properties ● DO NOT store secrets in these (use Docker Secrets or other secure mechanism) ● You can bind mount files (doesn’t have to be full directory structures)
- 37. Resources So much to talk about, so little time to do so!
- 38. Resources So much to talk about, so little time to do so! ● Docker Resources: https://www.docker.com/products/resources ○ Logging Reference Architecture: http://dockr.ly/logging ○ Training: https://training.docker.com ■ Instructor led ■ Self paced with “Play With Docker” ○ Containerizing legacy applications? ■ https://docker.com/MTA ● SquareSpace Blog: Understanding Linux Container Scheduling (with JVMs) https://engineering.squarespace.com/blog/2017/understanding-linux-container-scheduling