A TRUE STORY ABOUT DATABASE ORCHESTRATION
0 likes281 views
The document details a workshop led by Gianluca Arbezzano, a Site Reliability Engineer at InfluxData, focusing on database orchestration and infrastructure management using cloud services like AWS and DigitalOcean. It covers orchestration methods such as using EC2 and container management, emphasizing the importance of building scalable architectures and maintaining desired system states. Additionally, it highlights best practices for monitoring and managing logs, alongside various tools utilized in the orchestration process.
A TRUE STORY ABOUT DATABASE ORCHESTRATION
- 2. Agenda: Seasoned Developers Track WORKSHOPAGENDA
- 3. Gianluca Arbezzano Site Reliability Engineer @GianArb A True Story About Database Orchestration Gianluca Arbezzano is an SRE at InfluxData. He is a big Open Source contributor for several projects including and not limited to Zend Framework, Docker, and InfluxDB. He is also a Docker Captain and a member of the Doctrine ORM developers team. He is passionate about developing best practices for developers worldwide and has recently been focused on building scalable infrastructures on top of AWS, DigitalOcean and OpenStack. He is familiar with several programming languages, such as Javascript and Golang and is an active speaker at conferences throughout Europe, sharing his experiences and knowledge on projects that he is leading.
- 4. © 2018 InfluxData. All rights reserved.4 Once upon a time
- 5. © 2018 InfluxData. All rights reserved.5 InfluxCloud Fully managed and hosted service offering of the open source core (TICK Stack)
- 6. © 2018 InfluxData. All rights reserved.6
- 7. © 2018 InfluxData. All rights reserved.7 Cloud Provider
- 9. © 2018 InfluxData. All rights reserved.9
- 10. © 2018 InfluxData. All rights reserved.10
- 11. © 2018 InfluxData. All rights reserved.11 This looks like a EC2/Container orchestrator/scheduler Infrakit, k8s, Swarm, Mesos, Openshift...
- 12. © 2018 InfluxData. All rights reserved.12 Orchestration Really?
- 13. © 2018 InfluxData. All rights reserved.13 Why? • We have the right use case • Make stuff simple for support team and developer • Write the right amount of code • Build a use case for the TICK Stack
- 14. © 2018 InfluxData. All rights reserved.14 CoreOS
- 15. © 2018 InfluxData. All rights reserved.15 Etcd & SkyDNS
- 16. © 2018 InfluxData. All rights reserved.16 Process management
- 17. © 2018 InfluxData. All rights reserved.17 © 2018 InfluxData. All rights reserved.17 We use cloud-init to configure the worker.
- 18. © 2018 InfluxData. All rights reserved.18 “The worker asks for the cluster configuration in order to start the right containers”
- 19. © 2018 InfluxData. All rights reserved.19
- 20. © 2018 InfluxData. All rights reserved.20 Cloud Provider (AWS) • EC2 • EBS • ELB. Moving to ELBv2 • Security Groups, VPC bla bla • Route53 • Massive use of CloudInit
- 21. © 2018 InfluxData. All rights reserved.21 Creation steps 1. Validating request 2. Storing our expectation in etcd 3. Starting to build AWS resources a. Security group b. EC2 c. Load balancer d. Route53 4. Every server download a container that contains configuration file (InfluxDB, Kapacitor, Telegraf and so on) 5. The worker starts and it get cluster configuration 6. It merges cluster conf and service conf
- 22. © 2017 InfluxData. All rights reserved.22 © 2018 InfluxData. All rights reserved.22 Telegraf specification from configuration container
- 23. © 2018 InfluxData. All rights reserved.23 © 2018 InfluxData. All rights reserved.23 Monitoring - that’s easy!
- 24. © 2018 InfluxData. All rights reserved.24 Every containers send logs to a centralized place. in our case, papertrail https://github.com/gliderlabs/logspout
- 25. © 2017 InfluxData. All rights reserved.25 © 2018 InfluxData. All rights reserved.25 The History of life It’s time for Cloud v2
- 26. © 2017 InfluxData. All rights reserved.26 © 2018 InfluxData. All rights reserved.26 IFQL
- 27. © 2018 InfluxData. All rights reserved.27 © 2018 InfluxData. All rights reserved.27 Achieve and maintain desired state take away from v1 ● Recover EC2 from failure ● Recover Containers ● Secure and Stable from S1 to S2
- 28. © 2018 InfluxData. All rights reserved.28 Make a plan before to take action take away from v1
- 29. © 2018 InfluxData. All rights reserved.29
- 30. © 2018 InfluxData. All rights reserved.30 Sandbox https://github.com/influxdata/sandbox