Continuous Delivery Automation of Cloud Infrastructure and Applications at Choice Hotels
1 like315 views
The document discusses how Choice Hotels solved problems with their software delivery process by implementing continuous delivery automation of cloud infrastructure and applications. They standardized infrastructure configuration with infrastructure as code and application deployment descriptors. They also implemented a continuous delivery pipeline with multiple stages for tasks like testing, deployment to environments, and production deployment. This pipeline aimed to provide consistent, automated testing and deployment of applications to help improve software delivery.
![Infrastructure as Code
• Base AMIs defined with Puppet
− Linux (based on Amazon Linux)
− Web Server [httpd] (based on Linux AMI)
− Java Server (based on Linux AMI)
− Application Server [Tomcat] (based on Java AMI)
− Cassandra Server (based on Java AMI)
− ActiveMQ Server (based on Java AMI)
• Baked by launching an instance with Cloud Formation, run Puppet, create
image, teardown instance
• Re-bake at least every 30 days
Page 9 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016](https://image.slidesharecdn.com/brianmericlechoicehotels-160527165921/85/Continuous-Delivery-Automation-of-Cloud-Infrastructure-and-Applications-at-Choice-Hotels-9-320.jpg)
Continuous Delivery Automation of Cloud Infrastructure and Applications at Choice Hotels
- 1. Continuous Delivery Automation of Cloud Infrastructure and Applications at
- 2. Agenda Page 2 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016 • What challenges did we face with software delivery? • How we solved these problems using: − Technology − Process
- 3. A little about Choice Hotels… • Founded in 1939 (77 years ago!) • 1,150 Employees (~500 in IT) • Publicly traded (CHH) • $859.9 million in revenue (2015) • $7+ billion flows through reservation system each year • Franchise model (Economy, Mid-Scale, Upscale) • 6,400+ Hotels (500,000+ rooms) • 35+ Countries • Vacation Rentals (launched 2016) Page 3 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 4. A little about me… • Distinguished Engineer (1st at Choice) • Formerly, Senior Director, Platform Engineering • Specialize in OO Design, SOA and CI/CD • Sixteen years software engineering experience • Prior: Director of IT @ O’Reilly Media, Inc. • Married with three kids Page 4 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 5. What challenges did we face with software delivery? Page 5 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 6. Prior Deployment Process Page 6 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 7. Software Delivery Concerns Page 7 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016 • Inconsistent and manual deployment process • Formal change control process (meets once a week) • Inconsistent or lack of automated testing • Load testing performed by one team • Unchecked code quality analysis • No deployment process measurements • Requests for infrastructure takes weeks • Inconsistent and non-standard VM configuration
- 8. How we solved these problems using: Technology Page 8 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 9. Infrastructure as Code • Base AMIs defined with Puppet − Linux (based on Amazon Linux) − Web Server [httpd] (based on Linux AMI) − Java Server (based on Linux AMI) − Application Server [Tomcat] (based on Java AMI) − Cassandra Server (based on Java AMI) − ActiveMQ Server (based on Java AMI) • Baked by launching an instance with Cloud Formation, run Puppet, create image, teardown instance • Re-bake at least every 30 days Page 9 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 10. Application Deployment Descriptors • Written in JSON • Defines required infrastructure resources and application dependencies − Allows definition of the following infrastructure resources: • EC2, ASG, ELB, Scaling Policy, Security Groups, Subnets, RDS, SQS, SNS, Cloud Watch alarms • Descriptors are composable with other descriptors to enable chaining application dependencies together − An entire stack can be deployed by deploying highest level descriptor Page 10 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 11. Application Deployment Descriptor Example Page 11 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 12. Application Deployment Service • Java CLI application • Cloud provider agnostic (sort of) • Communicates to cloud provider through SDK • Traverses descriptor graph (depth-first) − Creates infrastructure resources (idempotent) − Deploys application (idempotent) Page 12 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 13. How we solved these problems using: Process Page 13 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 14. Continuous Delivery Pipeline Page 14 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 15. Commit Stage • Compile source • Execute unit tests • Execute static code analysis • Package artifact • Deploy artifact to Nexus • Deploy site to Nexus • Publish pipeline stage metrics Page 15 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 16. Functional Acceptance Testing Stage • Bake and copy AMI • Launch test environment • Wait for test environment • Execute functional testing with SoapUI • Publish pipeline stage metrics Page 16 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 17. Exploratory Acceptance Testing Stage (Optional) • Reuse functional acceptance testing environment • Execute manual testing (QA Approval Gate) • Publish pipeline stage metrics Page 17 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 18. Operational Acceptance Testing Stage • Reuse functional acceptance testing environment • Execute load testing with JMeter • Execute security testing with ZAP • Execute resiliency testing with Chaos Monkey • Teardown acceptance test environment • Publish pipeline stage metrics Page 18 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 19. Deployment to Systems Integration Test QA Environment • Execute blue/green deployment to SIT (QA Approval Gate) − Launch Green ASG with New Image − Wait For Green ASG − Teardown Blue ASG • Execute smoke tests against SIT (QA Approval Gate) • Publish pipeline stage metrics Page 19 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 20. Deployment to Production Environment • Submit change report to CAB (CAB Approval Gate) • Promote machine image to release • Execute canary deployment to production (Operations Approval Gate) • Execute blue/green deployment to production (Operations Approval Gate) • Rollback if necessary • Publish pipeline stage metrics Page 20 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 21. Page 21 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016
- 22. Thank You! Page 22 | Continuous Delivery Automation of Cloud Infrastructure and Applications | May 26, 2016 www.choicehotels.com [email protected] @bpmericle https://www.linkedin.com/in/brianpmericle
Editor's Notes
- #7: What happens to a process over time without innovation.
- #10: Manifests versioned in repo Re-bake every 30 days to ensure security updates are installed
- #12: Broken up into discrete reusable components
- #17: AMI Categories: Base linux, web server, java, application server, Cassandra Copies across dev account regions and shares the AMI with the prod account
- #18: Runs parallel to Operational Acceptance Testing stage
- #19: Load tests run when appropriate, not on every pipeline run ZAP - OWASP Zed Attack Proxy
- #21: AMI statuses: [SNAPSHOT > RELEASE] [RELEASE > STABLE] [STABLE > ARCHIVE]