Service discovery in Docker environments
21 likes8,413 views
The document discusses the importance of service discovery in Docker environments due to the increasing complexity of applications and the need for effective management of state changes. It focuses on the Consul tool by Hashicorp as a solution for service discovery, emphasizing its features like service health checks, multi-datacenter support, and its scalability through a gossip-based communication protocol. An example implementation illustrates how containers register with Consul, enabling automated service discovery across an infrastructure of physical or virtual machines.
![curl -X PUT -d 'true' http://$GWIP:8500/v1/kv/update/puppetmaster
sleep 3
curl http://jenkins.service.consul:8080/job/puppet-run/build
while [ true ]; do
echo "Waiting for git update(index higher than $MI)"
MI=`curl -s "http://$GWIP:8500/v1/kv/update/puppetmaster?index=
$MI" | awk -F "," '{ print $2 }' | cut -d ":" -f 2`
echo "New index is $MI"
echo "Modules updated. Pulling repo."
cd /etc/puppet/modules && git pull && cd -
done
On puppetmaster:
On git:](https://image.slidesharecdn.com/docker-meetup-140425055451-phpapp02/85/Service-discovery-in-Docker-environments-14-320.jpg)
Service discovery in Docker environments
- 1. Service discovery in Docker environments Alex Giurgiu ([email protected])
- 2. Why? • Average application size has increased tremendously; from single machine to an army of machines • Big trend of splitting monolithic architectures in SOA’s • The need to see the infrastructure as a homogeneous platform rather than a mix of specialised machines.
- 3. Why? • Docker based architectures encourage compartmentalisation => a multitude of specialised containers that need to find each other in an automated way • Rapidly changing environments demand an effective way of managing state change => Puppet driven DNS changes don't cut it anymore
- 4. How?
- 5. How? • Plenty of tools, of various sizes and ages, among which: directory services(DNS,LDAP), etcd, skydock, smartstack, pacemaker, zookeeper, doozerd, Consul, etc. • Many ways to do it; no one-size-fits-all solution; it all depends your infrastructure • This presentation focuses on just one of the ways; a way which applies well to Docker based environments.
- 6. Consul
- 7. Consul ! • Similar to etcd(from CoreOS) • Made by Hashicorp(Vagrant, Packer and Serf) • “Built on top of a foundation of rigorous academic research” - read SWIM and RAFT papers • Consistency through a quorum of nodes(RAFT) • Excellent scalability because of its gossip based communication protocol(SWIM)
- 8. Features • DNS recursor • Service health checks • Support for multiple datacenters • Key/Value store • Query using DNS or HTTP 8
- 9. Concepts • Consul agent can act as a client or as a server • The client doest not store any data, forwards requests to server • Quorum forms between servers • Each DC acts as a separate cluster, with its own data
- 10. Example implementation • Simple test environment for puppet code • 4 types of containers: puppetmaster, git, jenkins and puppetvalidation • Each container registers with CONSUL when it starts • Git notifies puppetmaster of any new commits using CONSUL
- 11. Example implementation • each node(physical or VM) runs its own Consul agent(can be client or server) • the Consul agent binds to the docker0 interface; all containers started on the node can query it using their default gateway • when a container starts it makes a call to the HTTP api to register itself with consul using a JSON formatted string echo "Registering PM service with consul curl -X PUT -d {"ID": "puppetmaster1", ”Name”:”puppetmaster", "Port":8600} http://$GWIP:8500/v1/agent/service/register
- 12. • once its registered the information will be gossiped to all other Consul agents; • all containers from all nodes should now be able to access the service using DNS at <name>.server.consul or using the HTTP API. • any DNS request for names outside the .consul domain will be forwarded to a configurable external DNS server. • git notification done using the Consul key/value HTTP API
- 14. curl -X PUT -d 'true' http://$GWIP:8500/v1/kv/update/puppetmaster sleep 3 curl http://jenkins.service.consul:8080/job/puppet-run/build while [ true ]; do echo "Waiting for git update(index higher than $MI)" MI=`curl -s "http://$GWIP:8500/v1/kv/update/puppetmaster?index= $MI" | awk -F "," '{ print $2 }' | cut -d ":" -f 2` echo "New index is $MI" echo "Modules updated. Pulling repo." cd /etc/puppet/modules && git pull && cd - done On puppetmaster: On git:
- 15. Demo
- 16. Questions