Docker @ RelateIQ Presentation
18 likes•4,772 views
This document summarizes RelateIQ's experience using Docker to manage their development environments and services. [1] They conducted a hack day to build Docker files and orchestration scripts for common services like Kafka, MongoDB, Redis and Cassandra. This allowed them to run all services with a single command. [2] They want to replace Chef with Docker in production for four reasons: dynamic configuration is too complex in Chef; external dependencies cause flaky provisioning; configuration changes create inconsistent states; and Docker is better for developer workflows. [3] Their future plans for Docker include replacing Chef search with Elasticsearch; integrating monitoring with StatsD or Datadog; and improving the Dockerfile build/management process
![Orchestration scripts
devenv.sh (outer)
[up|update|ssh]
●
●
Controls Vagrant
Controls inner script
devenv-inner.sh (inner)
[stop|start|kill|update|status|restart]
●
Controls Docker containers
update(){
start(){
apt-get update
mkdir -p $APPS/zookeeper/data
apt-get install -y lxc-docker
mkdir -p $APPS/zookeeper/logs
ZOOKEEPER=$(docker run
docker pull server:4444/zookeeper
-d
docker pull server:4444/redis
-p 2181:2181
docker pull server:4444/cassandra
-v $APPS/zookeeper/logs:/logs
docker pull server:4444/elasticsearch
server:4444/zookeeper)
docker pull server:4444/mongo
echo "Started ZOOKEEPER in
container $ZOOKEEPER"
echo "Wiring containers together… "
docker pull server:4444/kafka
}
kill/stop(){
(later slide)
echo "Killing all docker containers:"
docker ps | tail -n +2 |cut -d ' ' -f 1 | xargs docker kill
}](https://image.slidesharecdn.com/dockerrelateiq-presentation-131020185517-phpapp01/85/Docker-RelateIQ-Presentation-7-320.jpg)
![Dockerfile’s
Dockerfile
FROM server:4444/oracle-java7
Std Image
RUN apt-get update
RUN apt-get install -y git curl build-essential make gcc wget
RUN cd /opt && wget http://apache.mirrors.pair.com/cassandra/1.2.9/apachecassandra-1.2.9-bin.tar.gz
RUN cd /opt && tar zxf apache-cassandra-*.tar.gz
RUN rm /opt/*.tar.gz
RUN mv /opt/apache-cassandra-* /opt/cassandra
RUN apt-get install -y lsof
#!/bin/bash
echo "Cassandra node
configuration:"
echo $CASS_SEEDS
echo $CASS_TOKEN
start.sh
echo $CASS_LOCAL_IP
HOST=`hostname`
echo "127.0.0.1 $HOST" >>
/etc/hosts
/opt/cassandra/bin/cassandra -f
ADD cassandra.yaml /opt/cassandra/conf/cassandra.yaml
ADD cassandra-env.sh /opt/cassandra/conf/cassandra-env.sh
ADD log4j-server.properties /opt/cassandra/conf/log4j-server.properties
ADD start.sh /opt/cassandra/bin/start.sh
ADD cassandra-topology.properties /opt/cassandra/conf/cassandra-topology.
properties
RUN chmod 755 /opt/cassandra/bin/start.sh
RUN mkdir /logs
...
VOLUME [ "/logs" ]
...
EXPOSE 7000
….
CMD "/opt/cassandra/bin/start.sh"](https://image.slidesharecdn.com/dockerrelateiq-presentation-131020185517-phpapp01/85/Docker-RelateIQ-Presentation-8-320.jpg)
Docker @ RelateIQ Presentation
- 1. 10/17/2013 - Docker @ RelateIQ - Scott Bessler & John Fiedler
- 2. Docker @ RelateIQ @ Scott Bessler [email protected] @scottbessler John Fiedler [email protected] @johnfiedler Blog blog.relateiq.com RelateIQ Twitter @relateiq File Repo https://github.com/relateiq/docker_public
- 3. Agenda ● Part 1 - What we did with Docker ● Part 2 - Why we want to replace Chef ● Part 3 - Future plans using Docker
- 4. Part 1. Things were getting out of control... Bring on hack day! + + Hack Day! = =
- 5. What we needed to build Docker Files Minimum Kafka MongoDB Redis Cassandra Zookeeper Elasticsearch Crushing IT Docker Files Storm Jetty Nginx + Private Docker Registry
- 6. End product MAC OS X $ devenv start One Command! $ devenv-inner start Kafka VM Containers MongoDB Redis Cassandra Zookeeper ubuntu Elasticsearch
- 7. Orchestration scripts devenv.sh (outer) [up|update|ssh] ● ● Controls Vagrant Controls inner script devenv-inner.sh (inner) [stop|start|kill|update|status|restart] ● Controls Docker containers update(){ start(){ apt-get update mkdir -p $APPS/zookeeper/data apt-get install -y lxc-docker mkdir -p $APPS/zookeeper/logs ZOOKEEPER=$(docker run docker pull server:4444/zookeeper -d docker pull server:4444/redis -p 2181:2181 docker pull server:4444/cassandra -v $APPS/zookeeper/logs:/logs docker pull server:4444/elasticsearch server:4444/zookeeper) docker pull server:4444/mongo echo "Started ZOOKEEPER in container $ZOOKEEPER" echo "Wiring containers together… " docker pull server:4444/kafka } kill/stop(){ (later slide) echo "Killing all docker containers:" docker ps | tail -n +2 |cut -d ' ' -f 1 | xargs docker kill }
- 8. Dockerfile’s Dockerfile FROM server:4444/oracle-java7 Std Image RUN apt-get update RUN apt-get install -y git curl build-essential make gcc wget RUN cd /opt && wget http://apache.mirrors.pair.com/cassandra/1.2.9/apachecassandra-1.2.9-bin.tar.gz RUN cd /opt && tar zxf apache-cassandra-*.tar.gz RUN rm /opt/*.tar.gz RUN mv /opt/apache-cassandra-* /opt/cassandra RUN apt-get install -y lsof #!/bin/bash echo "Cassandra node configuration:" echo $CASS_SEEDS echo $CASS_TOKEN start.sh echo $CASS_LOCAL_IP HOST=`hostname` echo "127.0.0.1 $HOST" >> /etc/hosts /opt/cassandra/bin/cassandra -f ADD cassandra.yaml /opt/cassandra/conf/cassandra.yaml ADD cassandra-env.sh /opt/cassandra/conf/cassandra-env.sh ADD log4j-server.properties /opt/cassandra/conf/log4j-server.properties ADD start.sh /opt/cassandra/bin/start.sh ADD cassandra-topology.properties /opt/cassandra/conf/cassandra-topology. properties RUN chmod 755 /opt/cassandra/bin/start.sh RUN mkdir /logs ... VOLUME [ "/logs" ] ... EXPOSE 7000 …. CMD "/opt/cassandra/bin/start.sh"
- 9. Networking - port forwarding Question: Dude, where’s mongo? Answer: Port forwarding Example: > Docker file expose 9999 (dockerfile) > Docker Machine localhost:27019 vagrant config.vm.forward_port 9999,27019 docker run -p 9999:27018 docker run -p 8000:9999 > VAGRANT config.vm.forward_port 27018, 8000 > localhost:8000 docker file expose 27018 MongoDB Tip: Always use the same port if you can Containers
- 10. Networking - connecting two containers Question: Kafka: Dude, where’s Zookeeper? Answer: Pipework Machine vagrant https://github.com/jpetazzo/pipework Example: docker > startup the container KAFKA = docker run -e zooip=192.168.1.1 pipework br1 > Pipework pipework br1 $KAFKA 192.168.1.2 pipework br1 $ZOOKEEPER 192.168.1.1 (dockerfile) Zookeeper Tip: Use this for clustering Kafka 192.168.1.1 192.168.1.2 Containers
- 11. Best practices What we found Machine *Abstraction ● ● ● ● Data Logs Don’t end with tailing End with foreground execution ● Start script for runtime configuration ● 42 layers, combine Dockerfile lines ● Up Vagrant RAM default VM /logs/mongo /data/mongo /logs/kafka /data/kafka *Standard Data and log dirs *Static IP MongoDB Kafka *Start Script *Foreground Execution *42 layers only Containers
- 12. Chef 4 reasons why we want to replace Chef with Docker in prod
- 13. 1. Dynamic Configuration Is Too Complex Chef ● Dynamic configuration through complex attribute system Docker Dockerfile ● Admittedly less powerful ● What you see is what you get ● Environment variables ● Inspect environment at any step
- 14. 2. External Dependencies Cause Flaky Provisioning Chef Docker ● Once it’s in, it’s in. ● Dependencies are ● Even if you change an external forever image, only incremental ● Slower. You pay the price changes need to be sent on every node deployed to hosts ● Under load? Want a new node? Uh oh, the file download is missing. ● Node creation needs to be foolproof ● Or just use chef-solo to bake images
- 15. 3. Configuration Changes Create an Inconsistent State Chef Docker ● Configuration change? Try ● Ship configuration changes as image deltas to apply it to running node ● Nearly instant restarts ● Which changes restart ● Easier to be disciplined which services? and have each node be ● Removed something? identical ○ knife ssh ○ tombstone ● Another reason to just bake images
- 16. 4. Developers! Developers! Developers! Chef Dockerfile ● Typo? Fix it and start from ● Error? Throw it all out. the previous successful ● Every iteration/test takes: command ○ Boot time ● Containers encourage ○ Every step single-purpose instances ● Can’t test every host (put your monitoring on ● Can’t test every the host) combination of cookbooks ○ Monitoring ○ Logging
- 17. Part 3. Whats next? What do we want? What’s Next for Docker @ RelateIQ? ● Replace Chef search (simple node database in elasticsearch) ● Monitoring via StatsD and/or Datadog Wish List ● Dockerfile repo/trusted builds (mentioned on dockerdev) ● Centralized Docker host management ● Mac host (no more virtualbox!)