TIAD : Automating the modern datacenter
1 like•3,093 views
The document discusses the automation of modern datacenters using HashiCorp tools like Terraform and Consul, emphasizing the transition from traditional server management to a more efficient, automated approach. It highlights the importance of infrastructure as code, enabling rapid provisioning and deployment, as well as managing application delivery and service discovery. Key features include consistent application deployment, resilient infrastructures, and enhanced teamwork through modular designs and codified knowledge.
![Service Discovery
Service Discovery via DNS or HTTP
$ dig web-frontend.service.consul. +short
10.0.3.89
10.0.1.46
$ curl http://localhost:8500/v1/catalog/service/web-frontend
[{
“Node”: “node-e818f1”,
“Address”: “10.0.3.89”,
“ServiceID”: “web-frontend”,
…
}]](https://image.slidesharecdn.com/tiad-automatingthemoderndatacenter-150325042237-conversion-gate01/85/TIAD-Automating-the-modern-datacenter-53-320.jpg)
TIAD : Automating the modern datacenter
- 1. Automa'ng the Modern Datacenter
- 3. Powering the so9ware-‐managed datacenter. HashiCorp hashicorp.com
- 9. Virtualiza'on Datacenter Server Server VM VM VM VM VM VM VM VM
- 11. Service Prolifera'on Datacenter Server Server Server Server Server Server Server Server Server DNS Database CDN
- 12. Etc… • Hybrid cloud: Physical datacenter vs. Cloud provider • Mul'-‐paradigm: Physical, virtual, container • IaaS, PaaS, SaaS depending on the app • Opera'ng systems: Windows, Linux, Mac, Other • Realis'cally a mixture of everything just shown
- 13. But… why?
- 14. Common Goal: Efficiently deliver and maintain applica9ons.
- 15. Applica'on Delivery • Consistent • Shareable • Readily Available • High producCon parity. • Start and configure servers / services • Deploy and run applicaCon • Update servers or applicaCons • Reconfigure, feature flag • Monitor health • Orchestrate complex changes Development Deployment Maintenance
- 16. HashiCorp’s Open Source Tools Development Deployment Maintenance
- 17. Taming the Datacenter Deployment + Maintenance
- 18. Deployment + Maintenance 1. Acquisi'on 2. Provision 3. Update 4. Destroy
- 19. Historically • Servers: Days, weeks • Provisioning: Hours, days • SaaS: <didn’t exist>
- 20. Today • Servers: Minutes • Provisioning: Minutes • SaaS: Minutes
- 21. Historically • Rela'vely fixed set of servers • Fewer applica'ons to deploy • Fewer SaaS • Less demanding web traffic
- 22. Today • Poten'ally elas'c set of servers of varying sizes • Push towards SoA • SaaS for everything • More internet connected devices than ever before => higher traffic
- 23. What do we need? • Zero to deployed in one command • Resiliency through distributed systems • Autoscaling, autohealing • Beder teamwork through codified knowledge
- 25. HashiCorp’s Open Source Tools Development Deployment Maintenance
- 26. Automa9ng the Datacenter Deployment + Maintenance
- 27. terraform.io
- 28. Build, combine, and launch infrastructure safely and efficiently. terraform.io
- 29. What If I asked you to… • create a completely isolated second environment to run an applica'on (staging, QA, dev, etc.)? • deploy a complex new applica'on? • update an exis'ng complex applica'on? • document how our infrastructure is architected? • delegate some ops to smaller teams? (Core IT vs. App IT)
- 30. What If I asked you to… • create a completely isolated second environment to run an applica'on (staging, QA, dev, etc.)? One command. • deploy a complex new applica'on? Code it, diff it, pull request. • update an exis'ng complex applica'on? Code it, diff it, pull request. • document how our infrastructure is architected? Read the code. • delegate some ops to smaller teams? (Core IT vs. App IT) Modules, code reviews.
- 31. But how?
- 32. Terraform • Create infrastructure with code: servers, load balancers, databases, email providers, etc. • One command to create, update infrastructure. • Preview changes to infrastructure, save diffs. • Use code + diffs to treat infrastructure change just like code change: make a pull request, show the differences, review it, and accept. • Break infrastructure into modules to encourage/allow teamwork without risking stability.
- 33. Infrastructure as Code DigitalOcean Droplet with DNS in DNSimple resource "digitalocean_droplet" "web" { name = "tf-web" size = "512mb" image = "centos-5-8-x32" region = "sfo1" } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = "${digitalocean_droplet.web.ipv4_address}" type = "A" }
- 34. Infrastructure as Code DigitalOcean Droplet with DNS in DNSimple resource "digitalocean_droplet" "web" { name = "tf-web" size = "512mb" image = "centos-5-8-x32" region = "sfo1" } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = "${digitalocean_droplet.web.ipv4_address}" type = "A" }
- 35. Infrastructure as Code DigitalOcean Droplet with DNS in DNSimple resource "digitalocean_droplet" "web" { name = "tf-web" size = "512mb" image = "centos-5-8-x32" region = "sfo1" } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = "${digitalocean_droplet.web.ipv4_address}" type = "A" }
- 36. Infrastructure as Code DigitalOcean Droplet with DNS in DNSimple resource "digitalocean_droplet" "web" { name = "tf-web" size = "512mb" image = "centos-5-8-x32" region = "sfo1" } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = "${digitalocean_droplet.web.ipv4_address}" type = "A" }
- 37. Infrastructure as Code • Human friendly config, JSON compa'ble • Text format makes it version-‐able, VCS-‐friendly • Declara've • Infrastructure as code on a level not before possible
- 38. Zero to Done in One Command Terraform Apply $ terraform apply digitalocean_droplet.web: Creating… dnsimple_record.hello: Creating… Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
- 39. Zero to Done in One Command • Idempotent • Highly parallelized • Will only do what the plan says
- 40. Safely Change/Iterate Terraform Plan + digitalocean_droplet.web backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>" ttl: "" => "<computed>" type: "" => "A" value: "" => "${digitalocean_droplet.web.ipv4_address}"
- 41. Safely Change/Iterate Terraform Plan + digitalocean_droplet.web backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>" ttl: "" => "<computed>" type: "" => "A" value: "" => "${digitalocean_droplet.web.ipv4_address}"
- 42. Safely Change/Iterate Terraform Plan + digitalocean_droplet.web backups: "" => "<computed>" image: "" => "centos-5-8-x32" ipv4_address: "" => "<computed>" ipv4_address_private: "" => "<computed>" name: "" => "tf-web" private_networking: "" => "<computed>" region: "" => "sfo1" size: "" => "512mb" status: "" => "<computed>" + dnsimple_record.hello domain: "" => "example.com" domain_id: "" => "<computed>" hostname: "" => "<computed>" name: "" => "test" priority: "" => "<computed>" ttl: "" => "<computed>" type: "" => "A" value: "" => "${digitalocean_droplet.web.ipv4_address}"
- 43. Safely Change/Iterate • Plan shows you what will happen • Save plans to guarantee what will happen • Plans show reasons for certain ac'ons (such as re-‐create) • Prior to Terraform: Operators had to “divine” change ordering, paralleliza'on, rollout effect.
- 44. Workflow • Make code changes • `terraform plan` • Pull request with code changes + plan to make changes • Review and merge • `terraform apply pr1234.pplan`
- 45. Knowledge Sharing: Modules Terraform Plan module “consul” { source = “github.com/hashicorp/consul/terraform/aws” servers = 3 } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = “${module.consul.server_address}” type = "A" }
- 46. Knowledge Sharing: Modules Terraform Plan module “consul” { source = “github.com/hashicorp/consul/terraform/aws” servers = 3 } resource "dnsimple_record" "hello" { domain = "example.com" name = "test" value = “${module.consul.server_address}” type = "A" }
- 47. Knowledge Sharing: Modules • Self-‐contained infrastructure components • Allows delega'on of responsibility to mul'ple teams • Some teams create modules, other teams consume modules
- 48. Terraform • Zero to fully deployed in one command • Change/maintain infrastructure predictably • Teamwork-‐oriented workflow to infrastructure • Goal: Sta'c deploy/provisioning of infrastructure. Real'me monitoring, discovery, configura'on provided by Consul (discussed next).
- 49. consul.io
- 50. Service discovery, configura9on, and orchestra9on made easy. Distributed, highly available, and datacenter-‐aware.
- 51. Ques'ons that Consul Answers • Where is the service foo? (ex. Where is the database?) • What is the health status of service foo? • What is the health status of the machine/node foo? • What is the list of all currently running machines? • What is the configura'on of service foo? • Is anyone else currently performing opera'on foo?
- 52. Service Discovery Where is service foo?
- 53. Service Discovery Service Discovery via DNS or HTTP $ dig web-frontend.service.consul. +short 10.0.3.89 10.0.1.46 $ curl http://localhost:8500/v1/catalog/service/web-frontend [{ “Node”: “node-e818f1”, “Address”: “10.0.3.89”, “ServiceID”: “web-frontend”, … }]
- 54. Service Discovery • DNS is legacy-‐friendly. No applica'on changes required. • HTTP returns rich metadata. • Discover both internal and external services (such as service providers)
- 55. Failure Detection Is service foo healthy/available?
- 56. Failure Detec'on
- 57. Failure Detec'on • DNS won’t return non-‐healthy services or nodes. • HTTP has endpoints to list health state of catalog.
- 58. Key/Value Storage What is the config of service foo?
- 59. Key/Value Storage Serng and Gerng a Key $ curl –X PUT –d ‘bar’ http://localhost:8500/v1/kv/foo true $ curl http://localhost:8500/v1/kv/foo?raw bar
- 60. Key/Value Storage • Highly available storage of configura'on. • Turn knobs without big configura'on management process. • Watch keys (long poll) for changes • ACLs on key/value to protect sensi've informa'on
- 62. Mul'-‐Datacenter Service Discovery $ dig web-frontend.singapore.service.consul. +short 10.3.3.33 10.3.1.18 $ dig web-frontend.germany.service.consul. +short 10.7.3.41 10.7.1.76
- 63. Mul'-‐Datacenter Serng and Gerng a Key $ curl http://localhost:8500/v1/kv/foo?raw&dc=asia true $ curl http://localhost:8500/v1/kv/foo?raw&dc=eu false
- 64. Mul'-‐Datacenter • Local by default • Can query other datacenters however you may need to • Can view all datacenters within one UI
- 65. Orchestration Events, Exec, Watches
- 66. Events, Exec, Watches Dispatching Custom Events $ consul event deploy 6DF7FE … $ consul watch -type event -name deploy /usr/bin/deploy.sh … $ consul exec -service web /usr/bin/deploy.sh …
- 67. Events, Exec, Watches • Powerful orchestra'on tools • Pros/cons to each approach, use the right tool for the job • All approaches proven to scale to thousands of agents
- 68. Easiest Distributed System Deploy Deploy Consul to AWS $ terraform apply github.com/hashicorp/consul/terraform/aws var.servers The number of Consul servers to launch. Default: 3 Enter a value: 3 …
- 69. Easiest Distributed System Deploy Deploy Consul to AWS (manually) $ consul agent -atlas-join -atlas=USERNAME/NAME -atlas-token=API_TOKEN
- 70. Workflow • Server is started (via Terraform, etc.) • Consul agent is started, joins cluster • Star'ng services (ex. web app) query Consul for configura'on • Once healthy, services are discovered via DNS!
- 71. Opera'onal Bullet Points • Leader elec'on via Ra9 • Gossip protocol for aliveness • Three consistency models: default, consistent, and stale • Encryp'on, ACLs available • Real world usage to thousands of agents per datacenter