Auto scaling using Amazon Web Services ( AWS )
176 likes55,891 views
The document provides an overview of AWS Auto Scaling, detailing its need, functionality, configurations, and integration with other AWS services. It explains terms such as scale out, scale up, and scale down, and outlines how to manage auto scaling policies and the costs involved. Additionally, it discusses considerations for load scenarios and the importance of using Amazon CloudWatch for monitoring and triggering scaling actions.
Auto scaling using Amazon Web Services ( AWS )
- 1. Auto Scaling using AWS Harish Ganesan Co founder & CTO Contact: [email protected] www.twitter.com/harish11g http://www.linkedin.com/in/harishganesan
- 2. Introduction • Need for Auto Scaling • How AWS Auto scaling can help us ? • How to configure an Auto scaling policy in AWS • Things to remember before Scaling out and down • Understand the Intricacies while integrating Auto scaling with other Amazon Web Services • Risks involved in AWS Auto Scaling
- 3. Key AWS terms used EC2 : Elastic Compute Cloud EBS : Elastic Block Storage S3 : Simple Storage Service SQS : Simple Queue Service ELB : Elastic Load Balancing AMI : Amazon Machine Images AZ : Amazon Availability Zones ( Data centers) VPC : Amazon Virtual Private Cloud RDS : Amazon Relational Database Service
- 4. Key Scaling terms Scale out : Achieving scalability by increasing the number of Amazon EC2 instances Scale up : Achieving scalability by resizing the capacity( Compute, Memory and EBS) of existing Amazon EC2 instances Scale down : Decreasing the number of Amazon EC2 instances or the configuration for existing EC2 instances
- 5. Need for Auto Scaling
- 7. Actual Demand vs Fixed Capacity Source: AWS
- 8. Problem 1: Wasted Capacity Source: AWS
- 9. Problem 2 : Lost Customers Source: AWS
- 10. Need for Auto Scaling Source: Internet
- 11. What is AWS Auto Scaling ? AWS Auto Scaling allows us to scale our Amazon EC2 capacity out or down automatically according to the load patterns . Example : • We can expand the number of Amazon EC2 instances from 1 to 100+ automatically during load peaks • We can reduce the number of Amazon EC2 instances from 100+ to 1 automatically during load valleys
- 12. AWS Auto Scaling We can closely align our Infrastructure with our load requirements and save costs Source: AWS
- 14. Candidates for Auto Scaling (1) 1 Predictable Bursts • Applications which has Compute following Load Volatility characteristics like – Predictable bursts of request load Time – A Pattern in Spikes and lows – “On” and “Off” loads – Seasonal trends
- 15. Candidates for Auto Scaling (2) 2 Un Predictable Bursts • Applications which has Compute following Load Volatility characteristics like – Un Predictable bursts of request load Time – No specific pattern in Spikes and lows – Highly elastic ,unexpected and scalable demands
- 16. Candidates for Auto Scaling (3) 3 Constantly Growing • Applications which has Compute following Load Volatility characteristics like – Constantly growing – Successful and growing very Time fast – Complex lead time for deployment
- 17. What Amazon Auto Scaling can do ? • Handle all the 3 load scenarios (Candidates) • Scale out Amazon EC2 instances seamlessly and automatically when demand increases • Scale down unwanted Amazon EC2 instances automatically and save money when demand subsides • Decide the scaling based on AWS CloudWatch metrics • Auto Scale your Web servers(Amazon EC2) in combination with AWS Elastic Load Balancing
- 18. How much does it Cost ? Cost for using AWS Auto Scaling service = 0$ Value= PRICELESS Note : AWS Auto scaling needs Amazon CloudWatch monitoring service to function . Amazon CloudWatch is billed on usage basis.
- 19. Some AWS Auto scaling Concepts Auto Scaling group : Logical grouping of multiple Amazon EC2 instances for easy scaling and Management Health Check: Calls to check on the health status of each Amazon EC2 instance in an Auto Scaling group Launch Configuration: Captures the parameters necessary to create new EC2 instances in Auto Scaling mode
- 20. Some AWS Auto scaling Concepts Triggers: A CloudWatch alarm and an Auto Scaling policy that describes the actions when the alarm threshold is crossed . Two Triggers – Scaling out and Scaling down needs to be created Policy : Set of instructions for Auto Scaling that tells the service how to respond to AWS CloudWatch alarm messages
- 21. Amazon AutoScaling Architecture EC2 Instance Elastic Load Balancer … Amazon CloudWatch Scale Up Key Concepts : Scale Up Rule EC2 Instance • Launch Config Scale Down Scale Down • Auto Scaling Group Rule • Trigger Scale Up • Policy Predefined Scaling • Alarm Auto Scaling Scale Down Activity Group
- 22. Notable parameters on Amazon Auto Scaling • The percentage of allocated EC2 compute units that are CPUUtilization currently in use on the instance • Filters and responds with data that we request about all AutoScalingGroupName the Amazon EC2 instance part of the Auto scaling group • Time taken between a request and the corresponding Latency response as seen by the AWS Elastic Load Balancer • The number (sum) of requests processed by the AWS RequestCount Elastic Load Balancer • The number (average) of healthy instances behind the HealthyHostCount AWS Elastic Load balancer • The number (average) of unhealthy instances behind the unHealthyHostCount AWS Elastic Load Balancer
- 23. Steps to configure Amazon Auto Scaling with AWS Elastic Load Balancing
- 24. Step 1: Configuring AWS Auto Scaling with AWS ELB elb-create-lb my-load-balancer --headers --listener "lb-port=80,instance- port=8080,protocol=HTTP" --availability-zones us-east-1c App server port to Add a name to your The load which requests needs load balancer balancer port to be forwarded
- 25. Step 2: Create a launch configuration as-create-launch-config my-lconfig --image-id ami-e3826c8a --instance-type m1.small --key my-key-pair --group my-security-group Key pair / Security Name your launch Amazon Machine Amazon EC2 group settings for the configuration image(AMI) to be Instance Size Amazon EC2 instances launched during scaling
- 26. Step 3: Create an AWS Auto Scale Group as-create-auto-scaling-group my-as-group --availability-zones us-east-1c –launch-configuration my-lconfig --max-size 11 --min-size 3 --cooldown 180 --load- balancers my-load-balancer Load balancer name in Name your Auto Availability Zone in Minimum/Maximum which the new Amazon scale group which the Auto number of Amazon EC2 instances launched scaled Amazon EC2 EC2 instances will be attached instances will be maintained by Auto launched Scale
- 27. Step 4: Configure the Auto scaling Triggers as-create-or-update-trigger my-as-trigger --auto-scaling-group my-as-group --namespace "AWS/EC2" --measure CPUUtilization --statistic Average --dimensions "AutoScalingGroupName= my-as-group " --period 60 --lower-threshold 20 --upper-threshold 80 --lower-breach-increment"=-2" --upper-breach-increment 4 --breach-duration 180 Measure the Scale out by 4 Lower CPU Limit is average CPU of Amazon EC2 20% and Upper CPU the Auto Scale instances. Limit is 80% Group Scale down by 2 Amazon EC2 instances
- 28. Applying Amazon Auto Scaling for 3 different load scenarios
- 29. Auto Scaling Policy : Predictable Bursts 1 Predictable Bursts • Configure Time based Auto Scaling plan Compute • PutScheduledUpdateGroup Action - specify the date and time of execution, Time minimum, maximum, and desired Instance size of capacity % as-put-scheduled-update-group-action my-group –name “My-scale- up” -- time “2011-04-05T02:00:00Z” -min 10 -max 50
- 30. Auto Scaling Policy : Un predictable Bursts 2 Un Predictable Bursts • Configure On demand Auto Compute Scaling policy • Create policies for both Scaling out and Scaling down scenarios Time • Auto Scaling will respond to changing conditions dynamically %as-create-auto-scaling-group as-group --availability-zones us-east-1c – launch-configuration my-lconfig --max-size 20 --min-size 2 --cooldown 180 --load-balancers my-load-balancer
- 31. Auto Scaling Policy : Constantly Growing 3 Constantly Growing • Periodically monitor the Compute load requirements and manually configure the Auto scaling parameters • Increase the minimum fixed Time capacity gradually • Leverage On-Demand Amazon EC2 instances %as-create-auto-scaling-group my-as-group --availability-zones us-east- 1a –launch-configuration my-lconfig --max-size 4 --min-size 2 -- cooldown 360 --load-balancers my-load-balancer
- 32. Important points to remember before Scaling out / down using AWS Auto scaling
- 33. While Scaling out • Very frequent (minutes) scaling out activity is not cost effective • Scaling out with smaller instance type units closely aligns our capacity with our load requirements • We can create a maximum of 125 scheduled actions per Auto Scaling group. This allows scaling four times a day for a 31-day month for each Auto Scaling group • We can configure a scheduled Scaling action for up to a month in the future
- 34. While Scaling down • Amazon Auto scaling does not close the Amazon EC2 instances in the same of order of launch • To avoid data inconsistency architects should ensure the following points on Amazon EC2 servers of Auto scale group • Log files generated by Auto Scaled Amazon EC2 are moved to Amazon S3 periodically • Data files uploaded in the Web/App EC2 instances are moved to S3 or GlusterFS • Sessions of Web/App layer and Application Cache layers are replicated to a centralized /distributed resource
- 35. Candidates for Auto Scaling • Web Tier ( Apache , Nginx etc) • Application Tier ( Tomcat , Jboss etc) • Load Balancing Tier ( HAProxy, Nginx etc) • Other Stateless Tiers
- 36. Should not use Amazon Auto Scaling for? • Database Tier ( MySQL , SQL , Oracle etc) automate RDS Read Replica’s or Read Slaves Scaling out using custom Scripts • Use Custom Scaling programs / Scripts to add new nodes into the replications or clusters or Shards in the following • Distributed Caching Tier ( memCached ) • Solr Search engine • NoSQL ( Redis , MongoDB) .
- 37. Tips ,Pointers and Notes: While integrating Amazon Auto Scaling with other AWS components
- 38. Amazon Auto scaling and Amazon Machine Images (AMI’s)
- 39. Amazon Auto Scaling and AMI’s • Amazon Auto Scaling can launch new EC2 instances from S3 backed AMI’s and EBS backed AMI’s • AMI’s should be present on the Same Region where the EC2 instances will be launched in Auto scale group • AMI’s need not be present on the Same AZ where the EC2 instances will be launched in Auto scale group
- 40. S3 Backed AMI’s for Auto Scaling 1 1 Web requests are sent to the Amazon ELB Amazon Elastic Load Balancer 2 AWS ELB transfers the requests to 2 AZ : us-east-1a Amazon EC2 instances launched in us-east-1A availability zone AWS Security Groups 3 Amazon EC2 instances are configured to be part of AWS Auto Scale group 3 EC2 Instances 4 AWS Auto Scaling launches the new Amazon EC2 instances from the S3 backed AMI’s Amazon Auto Scaling 4
- 41. EBS Backed AMI’s for Auto Scaling 1 1 Web requests are sent to the Amazon ELB Amazon Elastic Load Balancer 2 AWS ELB transfers the requests to 2 AZ : us-east-1b Amazon EC2 instances launched in us-east-1B availability zone AWS Security Groups 3 Amazon EC2 instances are configured to be part of AWS Auto Scale group 3 EC2 Instances 4 AWS Auto Scaling launches the new Amazon EC2 instances from the EBS backed AMI’s EBS EBS EBS Amazon Auto Scaling 4
- 42. Amazon Auto Scaling and AMI’s • EBS backed AMI’s are faster to launch compared to S3 backed AMI’s • EBS backed AMI’s (Windows) launch in ~20 seconds (approx) • S3 backed AMI’s (Windows) launch in 3 minutes (approx) • Sometimes it is better to have EC2 instances launched faster in Auto scaling when overall CPU threshold is breached • EBS backed AMI’s option in AWS Auto scaling may not be cost effective compared to S3 backed AMI’s
- 43. AWS Auto scaling and Amazon EC2 instances
- 44. Amazon Auto Scaling and Amazon EC2 • A single Amazon Auto Scaling group can launch only one type of EC2 instances • Example , We cannot configure Amazon Auto scaling group to launch combinations of both m1.small and m1.large instance types • A single Amazon Auto Scaling group can launch only one version of AMI • Example , We cannot configure Amazon Auto scaling group to launch 2 versions of AMI bundled
- 45. Amazon Auto Scaling and Amazon EC2 • We can combine Reserved Instances (RI) and On- Demand EC2 Instances in our Scaling architecture • Amazon Auto scaling cannot be used to scale Amazon RDS instances
- 46. Amazon Auto scaling and AWS regions/AZ’s
- 47. Design 1: Amazon Auto Scaling across AZ’s HTTP requests are redirected to Amazon EC2 instances configured across the AWS availability zones in US East Amazon Elastic Load balancer AZ: US-EAST-1A AZ:US-EAST-1B Amazon Security Groups Amazon Security Groups C C L L Amazon EC2 O Amazon EC2 O instances U instances U D D W W Amazon Auto Scaling A Amazon Auto Scaling A T T S3 S3 C Backed C Backed H AMI H AMI
- 48. Design 2: Amazon Auto Scaling across AWS Regions Architecture Not Possible Amazon Elastic Load balancer AWS USA East Region AWS Europe Region Amazon Security Groups Amazon Security Groups C C L L Amazon EC2 O Amazon EC2 O instances U instances U D D W W Amazon Auto Scaling A Amazon Auto Scaling A T T S3 S3 C Backed C Backed H AMI H AMI
- 49. Amazon Auto Scaling ,AWS Regions and AZ’s • AWS Auto Scaling groups can work across multiple AWS Availability Zones inside same Region • If an AWS Availability Zone becomes unavailable, Auto Scaling will automatically redistribute applications to a different Availability Zone • AWS Auto Scaling groups cannot work across Regions • Example , we cannot configure a same AWS Auto scaling group to distribute and create Amazon EC2 instances in both USA and Europe
- 50. Amazon Auto scaling and AWS Elastic Load balancer
- 51. Case 1: AWS Auto Scaling with AWS Elastic Load balancing 1 1 This Architecture is suitable for Web Layer only . AWS ELB is attached to the AWS Auto Scaling group Amazon Elastic Load Balancer 2 AWS ELB acts as the gateway and 2 AZ : us-east-1a transfers the HTTP requests it receives to Amazon EC2 instances in round robin schedule AWS Security Groups 3 Amazon EC2 instances are configured to be part of AWS Auto Scale group 3 EC2 Instances 4 AWS Auto Scaling launches the new Amazon EC2 instances Amazon Auto Scaling 5 AWS Auto Scaling ,AWS CloudWatch and AWS Elastic Load Balancing works in union 4 S3 backed AMI
- 52. Case 2: Amazon Auto Scaling without AWS ELB Amazon EC2 2 1 Reference Auto scaling Architecture instances of in AWS not using AWS ELB Auto scale group reads messages from AWS SQS and processes them SQS 2 Messages needed for processing are put in the AWS SQS using message producer clients 3 Master EC2 instance node controls the Auto scaling EC2 Master EC2 Node 3 Master Amazon EC2 instance node instance group (scale out/down) controls the scale up / scale down of activities programmatically the processing nodes using AWS Auto Scaling programmatically 4 EC2 Instances 4 Auto scaled Amazon EC2 instance processing nodes picks the messages from Amazon SQS Amazon Auto Scaling Amazon EC2 instances are auto scaled programmatically S3 backed AMI depending upon SQS Load
- 53. Amazon Auto Scaling and Amazon ELB • Auto Scaling supports, but does not require, AWS Elastic Load Balancing • We can add Elastic Load Balancing to our Auto Scaling group and use Elastic Load Balancing metrics (request latency or request count) to scale our application • During the Scaling down process, Amazon Auto Scaling instructs the AWS load balancer to remove the Amazon EC2 instance from the load balancing group first
- 54. Amazon Auto scaling and Amazon Elastic IP
- 55. Amazon Auto Scaling and Amazon Elastic IP • Every new Amazon EC2 instances launched by Amazon Auto scaling has a Private and Public IP address • Configuring Elastic IP for Amazon EC2 instances launched by Amazon Auto Scaling is not a cost effective option • We need to pay Amazon for unused Elastic IP’s after the Scale down process • Combining Elastic IP & Auto scaling should be analyzed fully before applying in specific use cases
- 56. Amazon Auto scaling and Monitoring
- 57. Amazon Auto Scaling and Monitoring • Many traditional infrastructure monitoring systems rely on Agents to be installed on Amazon EC2 instances and get registered with Centralized monitoring server for receiving alerts • Since the Amazon EC2 servers are created and terminated dynamically by AWS Auto scaling, it will cause problems like • When new EC2 instances are created by Auto scaling the monitoring agents installed in them cannot auto register with the Centralized monitoring server
- 58. Amazon Auto Scaling and Monitoring • When EC2 instances are killed by Auto scaling , the centralized monitoring server will misunderstand this action as instance non-availability and creates unnecessary noise • Hence before integrating monitoring systems into our AWS Auto Scaled Infrastructure we should ensure that it can monitor dynamically spawned Amazon EC2 instances Note: Contact us if you want Nagios monitoring and Amazon AutoScaling to work together
- 59. Amazon Auto Scaling and AWS CloudWatch • AWS CloudWatch can monitor Auto scaled EC2 instances • AWS CloudWatch should be configured “detailed monitoring” option for the collection of Auto Scaling group metrics • Configure Auto Scaling with group metrics to send aggregated data to Amazon CloudWatch every minute • Auto scaling takes appropriate Scale out /down actions based on the alarms it receives from CloudWatch
- 60. Amazon Auto Scaling and AWS CloudWatch • Custom CloudWatch Metrics with Amazon Auto Scaling is powerful combination Example : Monitor Tomcat valves and send Number of sessions input to CloudWatch, Now AutoScaling can trigger new EC2 based on number of sessions in Tomcat.
- 61. Amazon Auto scaling and Launch Configurations
- 62. Auto Scaling and Launch Configurations • The maximum number of launch configurations per AWS account is 100 • If we modify our launch configuration, AWS Auto Scaling will not apply our new settings to existing Amazon EC2 instances • We can attach only one launch configuration to an AWS Auto Scaling group at a time • When AWS Auto Scaling wants to scale down, it first terminates instances that have an older launch configuration
- 63. Amazon Auto Scaling and Elastic Beanstalk • Amazon Elastic Beanstalk is a deployment tool used to automatically handle the infrastructure details like capacity provisioning, load balancing, auto- scaling, and application health monitoring • We cannot leverage Elastic bean stalk based Auto scaling in following scenarios – If our system demands Non Java applications to be configured on Auto scaling mode – If our application contains Java based Background programs (not a web app ) and they need auto scaling facility
- 64. Amazon Auto scaling and Amazon VPC
- 65. Amazon Auto Scaling and Amazon VPC • Amazon Auto Scaling can scale out /down Amazon EC2 instances inside a Amazon VPC network • AWS Elastic Load balancer cannot be tied to Amazon Auto Scaling inside Amazon VPC – Now possible
- 66. AWS Auto Scaling illustration Following slides illustrates a visual demo of AWS AutoScaling case • Image 1: Amazon EC2 Servers at initial Stage • Image 2: Load Increases : Amazon EC2 Servers are ramped up using AWS AutoScaling • Image 3: Load Decreases • Image 4: Amazon EC2 Servers are dynamically scaled down using AWS AutoScaling The below screenshots represent a custom product developed by us using AWS AutoScaling
- 67. Image 1 : Initial Stage Screenshot represents a custom product developed by us
- 68. Image 2 : Load Increases
- 69. Image 3 : Load Decreases Screenshot represents a custom product developed by us
- 70. Image 4 : Scaling down the EC2 instances
- 71. Risks involved in AWS Auto Scaling Risk 1: AWS Auto Scaling takes between 30 – 180 seconds sometimes to launch a new instance(s) . This intermediate time may cause impaired performance for our customers Risk 2: AWS Auto Scaling cannot differentiate between valid (vs) malicious traffic , it can scale out servers even for malicious traffic
- 72. Auto Scaling is an Art • Auto Scaling will not work for every situation and every application • Just by deploying our application in AWS , does not mean our application will leverage Auto Scaling and scale out automatically • To leverage Auto scaling effectively in-depth architectural expertise is needed in both Application and Infrastructure front
- 73. Auto Scaling is an Art • In some cases Auto Scaling is an on-going(monitor and fine tune based on trends/patterns) and not an one time activity • Badly configured Auto Scaling parameters will – increase the cost of infrastructure – cause revenue leakage rather than reducing them – create unnecessary excess capacity
- 74. General Cost Savings using Auto Scaling 1 30 % savings 2 35 % savings Compute Compute Weekly Weekly 30 % savings 3 35 % savings Compute Yearly
- 75. How do I leverage Amazon Auto Scaling effectively ?
- 76. Leave it to the experts , we will handle this Cloud Architecture Consulting Cloud Application Development Cloud Migration & Implementation Cloud Adoption Strategy “Let's get the job done”
- 77. Contact Us “All you need is an idea and the cloud will execute it for you.” (Structure 2010 event) - Dr Werner Vogels , CTO of Amazon For more details on how we can help your business , contact [email protected] http://harish11g.blogspot.com www.twitter.com/harish11g http://www.linkedin.com/in/harishganesan