Introduction to DevOps on AWS
Download as PPTX, PDF13 likes3,821 views
The document provides an introduction to DevOps principles and practices specifically on AWS, including infrastructure as code, continuous integration, and deployment techniques. It outlines key services like CloudFormation, Elastic Beanstalk, and OpsWorks, emphasizing automation and effective collaboration between development and IT operations. Additionally, it discusses deployment models such as blue/green deployment, infrastructure provisioning, and resource management using AWS technologies.
![Here’s some infrastructure as Code
"WebServer": {
"Type": "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"httpd" : [],
"php" : [],
"php-mysql" : [],
"php-gd" : [],
"php-xml" : [],
"php-mbstring" : [],
"mysql" : []
}
},
"sources" : {
"/var/www/html" : "http://ftp.drupal.org/files/projects/drupal-7.8.tar.gz",
"/home/ec2-user" : "http://ftp.drupal.org/files/projects/drush-7.x-4.5.tar.gz"
},
AWS
CloudFormation
template](https://image.slidesharecdn.com/introduction-to-devops-on-aws-141124165930-conversion-gate01/85/Introduction-to-DevOps-on-AWS-8-320.jpg)
Introduction to DevOps on AWS
- 1. Introduction to DevOps on AWS October 2014
- 2. Agenda • Intro to DevOps (evolution, principles, practices) – Infrastructure as Code – Continuous Integration / Continuous Deployment CI/CD – IT/Infrastructure Automation • DevOps on AWS – CloudFormation – Elastic Beanstalk – OpsWorks – BlueGreen deployment – Deployment Models (AMIs)
- 3. DevOps What is it ? – A philosophy? Cultural change? Paradigm shift ? – Alignment of development and IT operations with better communication and collaboration ? – Improvement in software deployment ? – Breaking down the barriers between development and IT operations ? – Akin to Agile software development applied to infrastructure and IT operations It’s all of the above !!! Principles Code W a l l Developer IT Operations
- 4. Evolution of DevOps from Agile Business Case Requirements Use Case Features Plan Go to market Business • Iterative development • Scrum, sprints, stories • Velocity Design Code Refactor Unit Test Bug Fix Deploy Developers (application) Provision Configure Orchestrate Deploy Report Monitor IT Operations (infrastructure) Agile Development DevOps • Continuous Integration • Continuous Deployment • IT Automation • Application Management Business Agility IT Agility
- 5. DevOps Principles • Collaboration • Breakdown the barriers • Work as one team end to end • Treat Infrastructure as code • Support business and IT agility • Automate everything • Test everything • Measure & monitor everything
- 6. DevOps Practices • Infrastructure as code • IT Automation • Continuous Integration – Application • Compile, test, optimize (code coverage) – Infrastructure • Logical, valid, secure • Continuous Deployment – Application – Rollout & Rollback • Version control integration • Application and Infrastructure version management • Monitoring and logging
- 7. Infrastructure as code – why ? • Scalability (anything manual is not scalable) • Reliability • Reproduction/Duplication • Environment consistency • Auditability/Record Keeping • Security • Governance
- 8. Here’s some infrastructure as Code "WebServer": { "Type": "AWS::EC2::Instance", "Metadata" : { "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "httpd" : [], "php" : [], "php-mysql" : [], "php-gd" : [], "php-xml" : [], "php-mbstring" : [], "mysql" : [] } }, "sources" : { "/var/www/html" : "http://ftp.drupal.org/files/projects/drupal-7.8.tar.gz", "/home/ec2-user" : "http://ftp.drupal.org/files/projects/drush-7.x-4.5.tar.gz" }, AWS CloudFormation template
- 9. Automation and configuration management Declarative Approach to: – Provisioning – Configuration – Orchestration – Reporting Elastic Beanstalk CloudFormation OpsWorks
- 10. Continuous Integration & Continuous Deployment • Application AND Infrastructure • Nothing Manual – Automate as much as possible • Define infrastructure declaratively • Architect infrastructure carefully including security • Treat definitions and configurations like application code • Store in version control • Infrastructure is part of the application • Automate testing (end to end) • Plan for rollback • Monitor, log and audit
- 11. Continuous Integration / Continuous Delivery • Help prove code quality and function repeatedly with predefined results • Lots of options; self hosted, open source, closed source, and SaaS • Monitoring, testing, validation • Plugins
- 12. Continuous Integration / Deployment & Automation Build/ Compile Code Version Control Dev Unit Test App Code IT Ops Dev Env Test Env DR Env Prod Env Application Write App Code Infrastructure tar, war, zip Deploy yum, rpm App CloudFormation Package Application Deploy application only Artifact Repository Deploy infrastructure only AMI Build AMIs Validate Templates Write Infra Code Deploy Infras Automate Deployment
- 13. DevOps on AWS: Deployment & Management – Cloud Formation – Elastic Beanstalk – OpsWorks – BlueGreen deployment
- 14. Monitoring Identity & Access OpsWork CloudTrail Storage S3 EBS Glacier Storage Gateway Foundation Services Networking VPC Direct Connect ELB Route53 Databases RDS Dynamo ElastiCache RedShift Content Delivery CloudFront Analytics EMR DataPipeline Kinesis Compute EC2 WorkSpaces AWS Global Infrastructure Deployment & Managemen t IAM Federation CloudWatch Deployment & Management BeanStalk Cloud Formation AWS Global Infrastructure Applicatio n Services Application Services SES SNS SQS Elastic Transcoder CloudSearch SWF AppStream
- 15. AWS Elastic AWS OpsWorks AWS CloudFormation Beanstalk DevOps framework for application lifecycle management and automation Templates to deploy & update infrastructure as code Automated resource management – web apps made easy DIY / On Demand DIY, on demand resources: EC2, S3, custom AMI’s, etc. Control Deployment and Management Convenience Control
- 16. Example Supported DevOps Practices on AWS • IT automation – Built in and can be combined with 3rd party tools • Version control Integration (Integration with Git, SVN) • Application version management • Infrastructure as code • Infrastructure version management • Deployment • Rollback • Monitoring & logging
- 17. Amazon Elastic Beanstalk AWS Elastic Beanstalk • Automated infrastructure management & code deployment for your application • Includes: • Load balancing • Health monitoring • Auto Scaling • Application platform management • Code deployment
- 18. Amazon Elastic Beanstalk Supports: Java PHP Python Ruby .NET Node.js docker
- 19. Example Elastic Beanstalk Architecture Route 53 Hosted Zone Availability Zone Web Server Web Server Auto scaling Group RDS DB Instance Availability Zone Web Server Web Server Auto scaling Group RDS DB Instance Standby Elastic Load Balancing S3 Bucket
- 20. Amazon Elastic Beanstalk Scalability Security Storage Fault tolerance Software updates and patches Content delivery Elastic Beanstalk
- 21. AWS OpsWorks • Application infrastructure management • Linux and Chef • Primary components: • Stacks • Layers • Instances • Apps AWS OpsWorks
- 22. Application Management - OpsWorks Scalability • Auto healing • Auto scaling • Load balancing • Scaling – time • Scaling - load Application Architecture • Load balancers • Web layer • Elastic IP’s • Security groups • Database layer Infrastructure Provisioning • Region • Availability Zone • Operating system • Keys Configure Application • Source of packages • Git, svn, S3 Deployment • Environments • Dev, Test, Prod Monitoring • Logs • Monitor AWS OpsWorks stack layers instances applications deployments monitoring
- 23. Amazon CloudFormation • Infrastructure as Code • Integrates with version control • JSON format • Templates • Stacks • Supports all AWS resource types AWS CloudFormation
- 24. AWS CloudFormation: Model Your App • Document, version control, and share your applications and infrastructure as a JSON document • Provision app and other AWS resources (VPC, DynamoDB, etc) from a template • Repeatable, reliable deployments for test/dev/prod in any AWS Region
- 25. Resource Property Types • Autoscaling • CloudFront • CloudWatch • DynamoDB • EC2 • Elastic Beanstalk • Elastic Load Balancer • IAM • OpsWorks • RDS • S3 • SNS/SQS Architecting on AWS – Overview of Services for Web Applications
- 26. Example options for a VPC resource • VPN Access • DHCP Options • Customer Gateways • Virtual Private Gateways • Network ACLs • Security Groups • More … Architecting on AWS – Overview of Services for Web Applications
- 27. AWS CloudFormation: Application stack example Amazon Route 53 Elastic Load Balancer CloudFront Distribution S3 Bucket Web ASG Architecting on AWS – Overview of Services for Web Applications Master Standby RR 1 RR 2 RR 3 RR 4 ElastiCache Cluster Web Servers Web Servers App App Elastic Beanstalk
- 28. AWS CloudFormation: Application stack example (continue) Template File Defining Stack Architecting on AWS – Overview of Services for Web Applications Git Subversion Mercurial Dev Test Prod The entire application can be represented in an AWS CloudFormation template. Use the version control system of your choice to store and track changes to this template Build out multiple environments, such as for Development, Test, and Production using the template
- 29. AWS CloudFormation Example (1 of 3) { "Description" : "Create an EC2 instance running the Amazon Linux 32 bit AMI.”, "Parameters" : { "KeyPair" : { "Description" : "The EC2 Key Pair to allow SSH access to the instance", "Type" : "String" } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyPair" }, "ImageId" : "ami-75g0061f”, “InstanceType” : “m1.medium” } } }, "Outputs" : { "InstanceId" : { "Description" : "The InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance” } } } } Architecting on AWS – Overview of Services for Web Applications
- 30. AWS CloudFormation Example (2 of 3) { "Description" : "Create an EC2 instance running the Amazon Linux 32 bit AMI.”, "Parameters" : { "KeyPair" : { "Description" : "The EC2 Key Pair to allow SSH access to the instance", "Type" : "String" } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyPair" }, "ImageId" : "ami-75g0061f”, “InstanceType” : “m1.medium” } } }, "Outputs" : { "InstanceId" : { "Description" : "The InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance” } } } } Architecting on AWS – Overview of Services for Web Applications Notice that you need to use an EC2 KeyPair for the CloudFormation template to work.
- 31. AWS CloudFormation Example (3 of 3) { "Description" : "Create an EC2 instance running the Amazon Linux 32 bit AMI.”, "Parameters" : { "KeyPair" : { "Description" : "The EC2 Key Pair to allow SSH access to the instance", "Type" : "String" } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyPair" }, "ImageId" : "ami-75g0061f”, “InstanceType” : “m1.medium” } } }, "Outputs" : { "InstanceId" : { You can define exactly what type of EC2 instance you want to launch. "Description" : "The InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance” } } } } Architecting on AWS – Overview of Services for Web Applications
- 32. Example - Putting it all together !!
- 34. Sample Architecture with CloudFormation
- 35. Sample Architecture with CloudFormation
- 36. AWS Elastic Beanstalk & OpsWorks Elastic Beanstalk: • Application container framework similar to a PaaS • Deploy your application into Elastic Beanstalk and it takes care of building a self healing, auto-scaling, multi-AZ infrastructure • Allows you to turn some of the knobs under the hood to tweak • Considered one of the easiest places to start with hosting an application on AWS OpsWorks: • Build multi-layer application stacks • Ties in with Chef for a large degree of flexibility and customization • Makes deploying applications easier • More flexible than Elastic Beanstalk, but requires a bit more knowledge
- 39. AMI Deployment Method • Code gets bundled into an AMI, we then deploy that AMI – Pluses • Very atomic • New shouldn’t effect older versions • Can deploy alongside current • Easy tools to automate – Cons • Bit more work involved • Have to think about where your data is persisting • Schema updates potentially harder to package in • Leverage configuration management tools in automation process
- 40. AMI Deployment Method - Building
- 41. AMI Deployment Method - Building Fully Functional AMI OS-Only AMI Partially Configured AMI
- 42. AMI Deployment Method - Building Fully Functional AMI OS-Only AMI Partially Configured AMI Least flexible to maintain
- 43. AMI Deployment Method - Building Fully Functional AMI OS-Only AMI Partially Configured AMI Most amount of post-boot work Least flexible to maintain
- 44. AMI Deployment Method - Building Fully Functional AMI OS-Only AMI Partially Configured AMI Most amount of post-boot work Least flexible to maintain Try and find a happy medium here
- 45. AMI Deployment Method - Deploying Blue/Green Deploys – We stand up a duplicate part of our infrastructure and slowly cut traffic over to it • Shift via DNS • Makes it easy to do testing of new features • Makes it easy to roll back – As we shift more traffic over, let auto-scaling grow/shrink our instances of the new or old application • Shut down the old when no traffic there Amazon Route 53 100% ELB EC2 Instances DynamoDB MySQL RDS Instance ElastiCache Cache Node
- 46. AMI Deployment Method - Deploying Blue/Green Deploys – We stand up a duplicate part of our infrastructure and slowly cut traffic over to it • Shift via DNS • Makes it easy to do testing of new features • Makes it easy to roll back – As we shift more traffic over, let auto-scaling grow/shrink our instances of the new or old application • Shut down the old when no traffic there Amazon Route 53 ELB 90% 10% EC2 Instances ELB EC2 Instances DynamoDB MySQL RDS Instance ElastiCache Cache Node
- 47. AMI Deployment Method - Deploying Blue/Green Deploys – We stand up a duplicate part of our infrastructure and slowly cut traffic over to it • Shift via DNS • Makes it easy to do testing of new features • Makes it easy to roll back – As we shift more traffic over, let auto-scaling grow/shrink our instances of the new or old application • Shut down the old when no traffic there Amazon Route 53 ELB 50% 50% EC2 Instances ELB EC2 Instances DynamoDB MySQL RDS Instance ElastiCache Cache Node
- 48. AMI Deployment Method - Deploying Blue/Green Deploys – We stand up a duplicate part of our infrastructure and slowly cut traffic over to it • Shift via DNS • Makes it easy to do testing of new features • Makes it easy to roll back – As we shift more traffic over, let auto-scaling grow/shrink our instances of the new or old application • Shut down the old when no traffic there Amazon Route 53 ELB 0% 100% EC2 Instances ELB EC2 Instances DynamoDB MySQL RDS Instance ElastiCache Cache Node
- 49. AMI Deployment Method - Deploying Blue/Green Deploys – We stand up a duplicate part of our infrastructure and slowly cut traffic over to it • Shift via DNS • Makes it easy to do testing of new features • Makes it easy to roll back – As we shift more traffic over, let auto-scaling grow/shrink our instances of the new or old application • Shut down the old when no traffic there Amazon Route 53 ELB 0% 100% EC2 Instances ELB EC2 Instances DynamoDB MySQL RDS Instance ElastiCache Cache Node
- 50. AMI Deployment Method - Deploying Blue/Green Deploys – We stand up a duplicate part of our infrastructure and slowly cut traffic over to it • Shift via DNS • Makes it easy to do testing of new features • Makes it easy to roll back – As we shift more traffic over, let auto-scaling grow/shrink our instances of the new or old application • Shut down the old when no traffic there Amazon Route 53 ELB 100% EC2 Instances DynamoDB MySQL RDS Instance ElastiCache Cache Node
- 51. AMI Deployment Method - Deploying Blue/Green Deploys – We stand up a duplicate part of our infrastructure and slowly cut traffic over to it • Shift via DNS • Makes it easy to do testing of new features • Makes it easy to roll back – As we shift more traffic over, let auto-scaling grow/shrink our instances of the new or old application • Shut down the old when no traffic there Amazon Route 53 100% ELB EC2 Instances DynamoDB MySQL RDS Instance ElastiCache Cache Node
- 52. Further Reading • AWS Documentation - http://aws.amazon.com/documentation • AWS Technical Whitepapers – http://aws.amazon.com/whitepapers • AWS Architecture Center – http://aws.amazon.com/architecture
Editor's Notes
- #4: WIKIPEDIA – DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals. Development and IT operations can be siloes having different: goals, mgt, processes and procedures Developers are paid to change things i.e. write code, Ops folks are paid to NOT change things and keep things stable Failed deployment have caused serious corporate issues and have potentially put companies out of business (or put them on the front page of the newspaper)
- #5: Agile originates in the development space and improved the collaboration and communication between business and developers Business Agility = ability to react to the market and client needs quickly Agile approaches are moving downstream towards infrastructure and operations
- #6: No more MANUAL HACKING. Infrastructure should be treated like the application source code. It should be maintained in version control Application management include Application Source Code and Infrastructure defined in Code
- #9: Cloud Formation is a core component of deployment and infrastructure and application management on AWS Uses JSON (Javascript object notation) format, basically key value pairs Contains all the meta data about the resources Supports wide range of AWS resources: DynamoDB, EC2, Elastic Beanstalk, IAM, RDS, Redshift, S3, SNS, SQS, VPC ……..
- #10: Use programming languages like Ruby and Python to declare configurations. Can use CloudFormation with any of the above. E.g. use CloudFormation to setup Puppet Master and Puppet Client
- #12: Building/testing software projects continuously, improve software quality Monitoring executions of externally-run jobs Scheduling, cron jobs Dashboards & Reports Numerous plugins – version control, Mavern, ant, Jenkins – Open Source, Industry Standard
- #13: Pulling it all together Two work streams APPLICATION and INFRASTRUCTURE Version control for code AND infrastructure configs CICD for application code AND infrastructure configs Can deploy independently or together using AMIs All process are iterative
- #18: Elastic Beanstalk – application container - Setup and managing an application's infrastructure Provides support for common architectures Can be customized
- #19: AWS Elastic Beanstalk supports several platforms, including Java, Windows (and .NET), Node.js, PHP, and Ruby.
- #20: Example Elastic Beanstalk Architecture
- #21: Scalability. Do you need to scale up, or scale out? Also, is your application as stateless as possible? Security. What are the security requirements of your organization? Persistent storage. Elastic Beanstalk does not use persistent storage. Apps need to leverage services like Amazon Elastic Block Store or Amazon S3. Fault tolerance. When you set up an Elastic Beanstalk environment, you can decide how many availability zones to use. We recommend that you use at least two (more is better) availability zones to help keep your system as available and fault tolerant as possible. Content delivery. How will users access your application? Leveraging tools like Route 53 and CloudFront can be advantageous. Software updates and patching. Running Elastic Beanstalk environments do not get automatically updated. Instead, you have to launch a new environment – or manage updates and patching separately.
- #22: OpsWorks – application management OpsWorks divides app deployment into four categories: stacks, layers, instances, and apps.
- #23: STACK = container of resources, LAYER = set of resources performing a purpose, INSTANCE = an EC2 instance. APP – defines application, type and its repository info Part of AWS Deployment and Management offerings – is FREE !!! OpsWorks makes it easy to deploy AND operate operations. Define the application’s architecture and the specification of each component including package installation, software configuration and resources such as storage. Use existing templates or build your own Mention Chef recipes used in OpsWorks – for stack definition and deployment
- #28: Notes: Example application stack running in AWS.
- #29: Notes: The entire application can be represented in an AWS CloudFormation template. You can use the version control system of your choice to store and track changes to this template. You can use the template to quickly build out multiple environments, such as for Development, Test, and Production.