DockerCon EU 2015: Trading Bitcoin with Docker
1 like•5,869 views
The document outlines the development and deployment process of the Bity bitcoin broker using Docker and Ansible. It details the challenges faced in implementing a scalable infrastructure and achieving continuous integration and deployment, as well as the lessons learned in improving team collaboration and environment reproducibility. The future directions include exploring better orchestration tools and optimizing Docker for production environments.
![- name: Start Backend VM
local_action:
module: cloudstack_vm
name: backendpublic
template: "{{ template }}"
service_offering: "{{ instance_type }}"
security_groups: [ 'backend_public' ]
ssh_key: "{{ ssh_key }}"
user_data: "{{ lookup('file', '../files/backend_userdata.yml') }}"
register: backend_public
tags: create_vm
Ansible CloudStack module
- name: backend SecGroup
local_action:
module: cloudstack_sg
name: database
description: Backend public
tags: secgroup](https://image.slidesharecdn.com/vyklsy3xrqeddaxhej8z-signature-4a4feb2d4f9f39f3ef364d4fb746cbebdc9dcb65abba22f35109f540494110a8-poli-151123215111-lva1-app6891/85/DockerCon-EU-2015-Trading-Bitcoin-with-Docker-23-320.jpg)
![26
Ansible and logdriver
- name: Start backend public
docker:
name: backend
image: sbex/backend
restart_policy: always
volumes:
- /app/_env:/usr/src/app/_env:ro
detach: true
ports: 8000:8000
log_driver: syslog
log_opt:
syslog-address: udp://{{hostvars['logserver'].ansible_ssh_host}}:5000
syslog-facility: local0
syslog-tag: backendpublic](https://image.slidesharecdn.com/vyklsy3xrqeddaxhej8z-signature-4a4feb2d4f9f39f3ef364d4fb746cbebdc9dcb65abba22f35109f540494110a8-poli-151123215111-lva1-app6891/85/DockerCon-EU-2015-Trading-Bitcoin-with-Docker-26-320.jpg)
DockerCon EU 2015: Trading Bitcoin with Docker
- 1. A bitcoin broker on Docker Mathieu Buffenoir @MBuffenoir Sebastien Goasguen @sebgoa 1
- 2. Mathieu Buffenoir CTO Bity.com VP of Swiss Bitcoin Association @MBuffenoir Sebastien Goasguen VP Apache CloudStack Author of O’Reilly Docker cookbook @sebgoa
- 3. Outline What is Bity ? From nothing to Docker Docker-compose in dev env Ansible for cloud providers Ansible for docker orchestration Lessons learned Future
- 4. 4 What is Bity ? Buy, sell and store bitcoins Regulated Small team hosted in Switzerland fast-moving space
- 5. 5
- 6. 6 Our needs Follow the “Twelve factor” app recommendations. Scalable, CI/CD -> Docker Cloud (Paas) + Hosted in Switzerland -> Exoscale
- 9. 9 Zero to Prod in 8 months ?
- 10. “It works on my machine” syndrome 10 Increase team collaboration Gain Velocity w/ Increased reproducibility Easy portability
- 11. Difficulty on-boarding developers Difficulty developing across team due to time to setup environment Teams working on different part of the infrastructure Challenges to gain velocity
- 12. Steps
- 13. Nothing to Docker Code on developer laptop with custom environment Zero portability (i.e /source/tree ) Use of Vagrant box Reproducibility of development environment (i.e / source/tree/Vagrantfile ) Use of Vagrant box and Docker Build image for applications and publish for collaboration (i.e /source/tree/Dockerfile +Vagrantfile) $ docker build -t sbex/bity . $ docker run -d -p 80:80 sbex/bity
- 14. Private repositories on Bitbucket Private repositories on Docker Hub Automated builds Web Hooks from Bitbucket to Docker hub Web Hooks from DockerHub to Jenkins Docker Hub
- 15. dev server: $ docker-compose up CI/CD for dev
- 16. 16 One docker-compose file to deploy entire infra Great for developers and testing Used to test parts of applications with latest image from repo Used prior to merging in staging Docker-compose for dev env
- 17. 17 Impossible to run command inside containers How to deal with secrets ? At the time, no Swarm so compose was a single host dev tool Limitations of compose
- 18. 18 Going to production in the cloud
- 19. 19 Choosing a Cloud and “config” tool Need a Swiss sovereign cloud Need a tool to configure: security groups key pairs manage instances Not a configuration management tool to deploy apps.
- 20. Dev (server or laptop + docker-compose) on bitcoin-testnet Staging (cloudstack + ansible) on bitcoin-testnet Prod (cloudstack + ansible) on bitcoin-mainnet separate branches for code and different image tags 20 Environments
- 21. $ git merge dev staging tags $ ansible-playbook deploy.yml staging environment Deploying on staging env
- 22. 22 Create Cloud Infrastructure at Will
- 23. - name: Start Backend VM local_action: module: cloudstack_vm name: backendpublic template: "{{ template }}" service_offering: "{{ instance_type }}" security_groups: [ 'backend_public' ] ssh_key: "{{ ssh_key }}" user_data: "{{ lookup('file', '../files/backend_userdata.yml') }}" register: backend_public tags: create_vm Ansible CloudStack module - name: backend SecGroup local_action: module: cloudstack_sg name: database description: Backend public tags: secgroup
- 24. 24 Ansible Docker module in Ansible core Deploying/Managing containers with Ansible
- 25. 25 - name: Set Docker login credentials command: docker login -u foobar -e {{ hub_email }} -- password={{ hub_password }} - name: Docker pull sbex/angular-frontend command: docker pull sbex/angular-frontend - name: Start bity docker: image: sbex/angular-frontend detach: true restart_policy: always name: bity ports: 80:80 tags: start_container Ansible docker module
- 26. 26 Ansible and logdriver - name: Start backend public docker: name: backend image: sbex/backend restart_policy: always volumes: - /app/_env:/usr/src/app/_env:ro detach: true ports: 8000:8000 log_driver: syslog log_opt: syslog-address: udp://{{hostvars['logserver'].ansible_ssh_host}}:5000 syslog-facility: local0 syslog-tag: backendpublic
- 27. 27 Ansible to configure containers - name: Create directory for settings file: path=/app/_env state=directory - name: Create json settings from template template: src=env.j2 dest=/app/_env/env.json ... - name: Create tables command: docker exec -ti backend ./manage.py migrate
- 28. 28 Dealing with secrets No secrets in container images Use Ansible vault to encrypt all secrets in playbooks stored in bitbucket $ ansible-vault create /path/to/file.yml $ ansible-vault encrypt /path/to/file.yml $ ansible-vault decrypt /path/to/file.yml $ ansible-vault rekey /path/to/file.yml
- 29. 29 Container “Orchestration” Every application has its role Several playbooks $ ansible-playbook deploy.yml $ ansible-playbook upgrade.yml $ ansible-playbook stop.yml $ ansible-playbook start.yml
- 30. 30 Early on: Logspout to ELK Now: Logdriver (ansible 2.0) syslog driver to logstash with ELK Test/deploy monitoring with docker-compose. Logging
- 31. 31 ElasticSearch 1.7 (+data container) Logstash 1.5.3 (+conf for elk logs) Kibana 4 (+Dashboard for elk logs) cAdvisor (Collect & View containers performance) Ngnix Proxy 1.9.3 (for SSL + password access). One docker-compose runs
- 32. 32
- 33. 33 Container restart -> thanks to restart policy (docker > 1.6) Weird Ansible docker behavior at times Config as volume mounts (Too many env variables to handle) Cannot use compose in prod yet (vault, execute commands inside containers ) Lessons Learned
- 34. 34 Currently using Ubuntu 14.04 Investigate the use of Docker optimized OS (e.g coreOS, Atomic, RancherOS) Need Easy upgrade of Docker versions With new versions every 2 months, and possible change of recommended storage driver, we need an easy way to cleanly upgrade production systems Investigate the use of a Docker orchestrator, possible replacing Ansible docker module (e.g Swarm, Kubernetes, tutum…) Future