Docker for Development

Docker for Development

• 1.
  D O CK E R F O R D E V E L O P M E N T J E F F N I C K O L O F F
• 2.
  • To providea simple scaffold for: • rapid iteration • build, test, lint… release pipeline automation • Dependency (library, service, etc) management W H A T I S T H E G O A L ? A D E V E L O P M E N T E N V I R O N M E N T
• 3.
  • Runtime environmentrealism • Develop and iterate in isolation B E S T P R A C T I C E S A D E V E L O P M E N T E N V I R O N M E N T
• 4.
  • Build toolmanagement • “Did you install <tool X>?” “I dunno bud. It works in IntelliJ” • “We’re upgrading from Maven to <whatever>” *eyes roll* • Multi-day new team member ramp-up • Service dependency modeling (database, etc) • Remote resources: “I need VPN so I can work from the coffee shop / home / etc.” • Service state (data mutation, schema version) • Isolation from other local environments • Environment variable collisions • Log file collisions • Port collisions • Shared (remote) databases C H A L L E N G E S A D E V E L O P M E N T E N V I R O N M E N T
• 5.
  – E VE R Y O N E A T S O M E P O I N T “I know! We’ll just Dockerize the things…”
• 6.
  F I RS T Y O U M A K E A N I M A G E R I G H T ? • “It is easy! I just put my binaries in an image right?” • “Wait, you mean I have to rebuild my image every time?” • “Do my build tools need to go all the way to production if I use them in my development env?”
• 7.
  F I RS T , T H I N K A B O U T Y O U R P R O B L E M • I want to iterate quickly when I’m developing local • I want to model my service dependencies locally • I want to version my build tooling and configuration • I want to automate a local build/launch pipeline
• 8.
  T H RE E I M A G E / C O N T A I N E R P A T T E R N S • Static(ish) iteration tooling, variable source • heavy use of volumes • Static(ish) source, full build tooling • ONBUILD images • Release quality artifact, minimal overhead, minimal attack surface • FROM scratch or alpine
• 9.
  E X AM P L E E N V I R O N M E N T • Owned by a team named, “team” • Go based service named, “proj” • Repo is “go getable” • Binaries can be produced outside of the context of a Go workspace • Build tools: • Node and GulpJS for workflow automation • Mocha for integration testing
• 10.
  L O CA L . D F ( F O R I T E R A T I O N ) # Based on minimal Go tooling FROM golang:alpine # Environment modeling (Go workspace, path, and runtime flags) ENV GOPATH=/src/proj PATH=$PATH:/src/proj/bin GODEBUG=netdns=cgo RUN mkdir -p /src/proj/src/bitbucket.org/team/proj && mkdir -p /src/proj/bin # Install Git (to pull library deps), Node, and Gulp (for build automation) RUN apk --update add --no-cache git nodejs RUN npm install --global gulp # Install library dependencies in image. RUN go get github.com/bitly/go-nsq && go get github.com/codegangsta/cli && go get github.com/gin-gonic/gin && go get github.com/rcrowley/go-metrics && go get github.com/allingeek/go-metrics-influxdb # Explicitly define volumes that should be overridden at runtime VOLUME ["/src/proj/src/bitbucket.org/team/proj", "/src/proj/pkg", "/src/proj/bin"] # Set the context for local iteration WORKDIR /src/proj/src/bitbucket.org/team/proj # Gulp has the iterative build instructions # The iterative build instructions can change without rebuilding this image CMD ["gulp"]
• 11.
  B U IL D . D F ( F O R B U I L D I N G B I N S ) # Based on minimal Go tooling FROM golang:alpine # Install Git for pulling library dependencies RUN apk --update add git # Basic environment stuffs ENV GODEBUG=netdns=cgo CMD proj --debug serve # Copy in sources (and just the sources) COPY *.go /go/src/bitbucket.org/team/proj/ COPY common /go/src/bitbucket.org/team/proj/common COPY service /go/src/bitbucket.org/team/proj/service # Set context, install deps, and (simple) build WORKDIR /go/src/bitbucket.org/team/proj RUN go get ./... && go install
• 12.
  R E LE A S E . D F ( R U N N A B L E I M A G E ) # No Go tooling, just a minimal Linux FROM alpine # Set the context and runtime flags WORKDIR / ENV PATH=$PATH:/ GODEBUG=netdns=cgo CMD proj serve # Add other stuff like EXPOSE / USER here # Copy in the binary built in another container. # In a warm env, this is the only layer that should # ever change. (Fast) COPY ./bin/proj-exported /proj
• 13.
  T H RE E I M A G E / C O N T A I N E R P A T T E R N S • Nobody said you could only use one image, container, etc. • Each of the three patterns is appropriate for a different development phase (see demo). • Nobody said you can ONLY use Docker. • Use other common tools like Make or Compose
• 14.
  M O DE L E N V I R O N M E N T S W I T H C O M P O S E • Local development is an environment • Model service dependencies with known initial state • Include special tooling like integration tests • Include as many components to mirror a realistic runtime as possible (databases, dashboards, logging, mock traffic, etc)
• 15.
  D O CK E R - C O M P O S E . Y M L ( I T E R A T E ) lb: image: nginx volumes: - ./myconf.conf:/etc/nginx/conf.d/myconf.conf ports: - 8080:80 proj: build: . dockerfile: local.df
• 16.
  D O CK E R - C O M P O S E . Y M L ( I T E R A T E )lb: image: nginx volumes: - ./proj-proxy.conf:/etc/nginx/conf.d/proj-proxy.conf ports: - 8080:80 links: - proj:proj proj: build: . dockerfile: local.df links: - db:dbhostname - cache:cachehostname db: image: postgres cache: image: redis integ-test: build: ./integ links: - proj:proj
• 17.
  P U TT I N G I T A L L T O G E T H E R • All of the tools in the stack should be accessible to the developer. • Few of those tools should “define” the experience. • Pick a primary touch point: • docker? docker-compose? make? Eclipse? something else?
• 18.
  A M AK E F I L E PWD := $(shell pwd) # cleanup the environment clean: rm bin/proj-exported docker rmi team/proj:dev # Run NPM in a container and install tools (Gulp & Mocha) in the PWD prepare: docker run -it --rm -v "$(shell pwd)":/work -w /work node:4.1.2 npm install # Start iterating live. This uses docker-compose.yml and local.df. # GulpJS is performing a full format/build/spawn workflow on every source change. iterate: docker-compose up -d # Stop iterating. stop: docker-compose stop # Build proj during image build. Produce runnable image & copy out the binary build: docker build -t team/proj:dev -f dev.df . docker run --rm -v $(PWD)/bin:/xfer team/proj:dev cp /go/bin/proj /xfer/proj-exported # Produce a production quality image by injecting only the binary release: build docker build -t team/proj -f release.df .
• 19.
  D E MO