SlideShare a Scribd company logo
Resource Replication &
Automated Scaling Listener
Dr Hitesh Mohapatra
School of Computer Engineering
Associate Professor
KIIT (Deemed to be) University
Contents
Resource Replication:
• Concept of replication in cloud
• Types of replications (synchronous vs asynchronous)
• Benefits: High availability, disaster recovery
• Example: Replication in distributed systems
Automated Scaling Listener:
• What is automated scaling and why it's important?
• How listeners help detect when scaling is required
• Auto-scaling groups in AWS, GCP auto-scaler
• Configuration of scaling policies
2/7/2025 Dr Hitesh Mohapatra 2
Resource Replication {Follow the hyperlink}
Resource Replication
• Concept of replication in cloud
• Types of replications (synchronous vs asynchronous)
• Benefits: High availability, disaster recovery
• Example: Replication in distributed systems
2/7/2025 Dr Hitesh Mohapatra 3
Automated Scaling Listener
Objective
• What is automated scaling and why it's important?
• How listeners help detect when scaling is required
• Auto-scaling groups in AWS, GCP auto-scaler
• Configuration of scaling policies
2/7/2025 Dr Hitesh Mohapatra 4
Automated Scaling Listener
•Service Agent Definition:
•Known as the automated scaling listener mechanism.
•Function:
•Tracks and monitors communications between cloud service users and cloud services.
•Purpose:
•Supports dynamic scaling.
•Installation:
•Installed in the cloud, typically close to the firewall.
•Monitoring:
•Continuously tracks data on the status of the workload.
•Assessment Criteria:
•Based on the number of requests made by cloud users.
•Also based on the demands placed on the backend by particular types of requests.
•Example:
•Processing a small amount of incoming data can take a significant amount of time.
2/7/2025 Dr Hitesh Mohapatra 5
Working
2/7/2025 Dr Hitesh Mohapatra 6
Why it is required ?
• Automatically Adjusting IT Resources based on previously set
parameters by the cloud consumer (Auto Scaling).
• Automatic Notification of the cloud consumer when
workloads go above or below predetermined thresholds. This
gives the cloud user the option to change how its present IT
resources are allocated. (Auto Notification)
2/7/2025 Dr Hitesh Mohapatra 7
Architecture
2/7/2025 Dr Hitesh Mohapatra 8
Steps
1. Service Agent Roles:
•The service agents that perform the role of autonomous scaling listeners go by many
names depending on the cloud provider vendor.
2. Initial Access Attempt:
•Three users of cloud services simultaneously try to access one cloud service.
3. Creation of Instances:
•The autonomous scaling listener creates three duplicate instances of the service when it
grows out.
4. Additional Access Attempt:
•A fourth user of a cloud service tries to access the service.
5. Limit Exceeded Alert:
•The automated scaling listener rejects the fourth attempt and alerts the cloud consumer
that the intended workload limit has been exceeded because the cloud service is only
designed to enable up to three instances.
6. Administrator Action:
•To modify the provisioning configuration and raise the redundant instance limit, the cloud
resource administrator of the cloud consumer logs into the remote administration
environment.
2/7/2025 Dr Hitesh Mohapatra 9
How listeners help detect when scaling is
required?
1. Monitoring Workloads:
• Continuously track data on the status of workloads.
• Monitor the number of incoming requests and the load on the backend services.
2. Analyzing Data:
• Assess the patterns and trends in the data to identify peaks and troughs in usage.
• Evaluate the types of requests and their impact on resource consumption.
3. Thresholds and Alerts:
• Set predefined thresholds for various metrics like CPU usage, memory usage, and
network traffic.
• Trigger alerts when these thresholds are crossed, indicating a need for scaling.
2/7/2025 Dr Hitesh Mohapatra 10
Cont.
4. Resource Allocation:
• Determine the appropriate number of resources needed to handle the current
and projected workload.
• Automatically allocate or deallocate resources based on real-time demand.
5. Preventing Overload:
• Prevent overloading of services by ensuring that additional instances are created
when demand spikes.
• Reject new requests or scale down resources when the demand decreases,
maintaining optimal performance.
6. Feedback Loops:
• Implement feedback loops to continuously improve the scaling process.
• Adjust the thresholds and resource allocation strategies based on past
performance and usage patterns.
2/7/2025 Dr Hitesh Mohapatra 11
Cont.
By effectively monitoring and analyzing the workload data, automated
scaling listeners help maintain the efficiency, reliability, and
performance of cloud services. They ensure that resources are
dynamically scaled to meet the demands of the users, preventing both
underutilization and overload.
2/7/2025 Dr Hitesh Mohapatra 12
Difference between Auto Scaling vs Load
Balancing
Feature Auto Scaling Load Balancing
Primary Function
Automatically adjusts the number of
instances based on demand
Distributes incoming traffic across multiple
instances
Purpose
To ensure optimal resource utilization
and handle varying loads
To ensure high availability and reliability by
balancing load
Operation Adds or removes instances as needed
Distributes traffic based on predefined rules
or algorithms
Focus Resource scaling Traffic distribution
Usage Scenario
Scaling in and out instances based on
application needs
Balancing traffic load across running
instances
Reduction of Backend
Duties
Manages instance scaling, reducing
manual intervention
Balances load, manages traffic, and monitors
server health
Combination
Often used together for optimal
performance and scalability
Often used together with auto-scaling for
efficient traffic management
Example Tools AWS Auto Scaling, Azure Autoscale
Elastic Load Balancing (ELB), Azure Load
Balancer
2/7/2025 Dr Hitesh Mohapatra 13
Auto-scaling groups in AWS
Definition:
•Auto Scaling Groups (ASGs) are a
collection of Amazon EC2 instances
managed as a logical grouping for
automatic scaling and management.
Components:
•Launch Configuration: Defines the
instance type, AMI ID, key pair, security
groups, and other configurations.
•Scaling Policies: Determine how and
when the ASG should scale in or out
based on predefined criteria.
2/7/2025 Dr Hitesh Mohapatra 14
Cont.
Dynamic and Predictive Scaling:
•Dynamic Scaling: Adjusts the number of instances based on real-time demand (e.g.,
CPU utilization).
•Predictive Scaling: Uses historical data to predict and provision resources ahead of
time.
Health Checks:
•Continuously monitors the health of instances within the group.
•Automatically replaces unhealthy instances to ensure high availability.
Benefits:
•Automatically adjusts capacity to maintain steady, predictable performance at the
lowest possible cost.
•Ensures application availability by automatically adding or removing instances as
needed.
2/7/2025 Dr Hitesh Mohapatra 15
GCP auto-scaler
Definition:
The GCP Auto Scaler automatically
adjusts the number of VM instances in
a managed instance group based on
the current load.
Components:
•Instance Group Manager: Manages
the lifecycle of VM instances within the
instance group.
•Scaling Policies: Define the metrics
and thresholds for scaling actions.
2/7/2025 Dr Hitesh Mohapatra 16
Cont.
Scaling Metrics:
•Uses various metrics such as CPU utilization, HTTP load balancing serving
capacity, and stackdriver custom metrics to decide when to scale.
Auto-Healing:
•Automatically recreates failed instances to maintain the desired state of the
instance group.
Predictive Autoscaler:
•Uses machine learning to analyze historical usage data and predict future
demand.
•Provision resources proactively to meet anticipated demand.
Benefits:
•Provides cost savings by automatically adjusting resources based on demand.
•Enhances application performance by ensuring sufficient capacity to handle the
load.
2/7/2025 Dr Hitesh Mohapatra 17
Example: Autoscaling based on CPU
Utilization
2/7/2025 Dr Hitesh Mohapatra 18
For example, assume
the load balancing
serving capacity of a
managed instance
group is defined as 100
RPS per instance. If
you create an
autoscaler with the
HTTP(S) load
balancing policy and
set it to maintain a
target utilization level
of 0.8 or 80%, the
autoscaler will add or
remove instances from
the managed instance
group to maintain 80%
of the serving capacity,
or 80 RPS per instance.
2/7/2025 Dr Hitesh Mohapatra 19
2/7/2025 Dr Hitesh Mohapatra 20
Difference between Horizontal vs Vertical
Auto Scaling
Aspect Horizontal Auto Scaling Vertical Auto Scaling
Definition
Addition of more servers or computers to the auto-
scaling group
Scaling by supplying more power (e.g., more RAM) to an
existing system
Scalability Expands the resource pool with more machines Boosts the power of an already-running system
Handling High
User Load
Can handle queries from thousands of users Limited in handling very high user loads
Components Clustering, distributed file systems, load balancing Increasing resources like CPU and RAM
Stateless
Servers
Crucial for handling large number of users; sessions can
move across servers
Not applicable
Downtime
No downtime required; creates new instances
separately
Requires downtime for upgrades and reconfigurations
Availability
Improves availability and performance due to
independence
No improvement in availability; dependent on a single
machine
Performance
Enhances user experience with browser-side session
storage
Improves performance
Redundancy Supports redundancy with multiple instances No redundant server; dependent on single location
Elastic Load
Balancing
Scales incoming requests across instances Not applicable; deals with vertical resource allocation
Best Use Cases Ideal for applications with a large user base Suitable for applications with fewer scalability demands
Challenges Requires effective clustering and load balancing Architectural issues due to single machine dependency
Overall Impact Enhances scalability, availability, and user experience Boosts performance but limited in scalability and availability
2/7/2025 Dr Hitesh Mohapatra 21
Configuration of scaling policies
Step 1: Define Your Scaling Goals
• Identify Key Metrics: Determine which metrics (e.g., CPU utilization, memory
usage, request count) will trigger scaling actions.
• Set Desired Performance Levels: Establish the target performance levels for
your application, such as response time and availability.
Step 2: Create Scaling Policies
• Threshold-Based Policies: Define specific thresholds for your key metrics. For
example, if CPU utilization exceeds 70%, trigger a scale-out action.
• Scheduled Policies: Schedule scaling actions based on predictable usage
patterns. For example, scale out during peak hours and scale in during off-
peak hours.
• Predictive Policies: Use machine learning models to predict future demand
and proactively scale resources accordingly.
2/7/2025 Dr Hitesh Mohapatra 22
Cont.
Step 3: Configure Alarms and Triggers
• Set Alarms: Create alarms that monitor the key metrics and trigger scaling
actions when thresholds are reached.
• Define Triggers: Specify the conditions under which scaling actions should be
triggered, such as exceeding or falling below the defined thresholds.
Step 4: Define Scaling Actions
• Scale-Out Actions: Specify how many instances to add when scaling out. For
example, add two instances if CPU utilization exceeds 70%.
• Scale-In Actions: Specify how many instances to remove when scaling in. For
example, remove one instance if CPU utilization falls below 30%.
2/7/2025 Dr Hitesh Mohapatra 23
Cont.
Step 5: Configure Cooldown Periods
• Set Cooldown Periods: Define cooldown periods to prevent rapid, repetitive
scaling actions. This allows the system to stabilize before triggering another
scaling action.
Step 6: Implement and Test Policies
• Deploy Policies: Implement the scaling policies in your cloud service
configuration.
• Test Policies: Test the policies under different load conditions to ensure they
work as expected and make adjustments as needed.
Step 7: Monitor and Optimize
• Continuous Monitoring: Continuously monitor the performance and
effectiveness of your scaling policies.
• Optimize Policies: Regularly review and optimize the scaling policies based on
performance data and changing application requirements.
2/7/2025 Dr Hitesh Mohapatra 24
Questions
1. What is the primary function of an automated scaling listener mechanism?
2. How do automated scaling listeners support dynamic scaling in the cloud?
3. Where are automated scaling listeners typically installed in the cloud?
4. What data do automated scaling listeners continuously track to assess workloads?
5. How does an autonomous scaling listener respond when multiple users try to access the same cloud
service?
6. What action does an automated scaling listener take when the intended workload limit is exceeded?
7. What is the main difference between horizontal auto-scaling and vertical auto-scaling?
8. Why is stateless server architecture crucial for horizontal auto-scaling?
9. How does load balancing improve the availability and performance of cloud services?
10.Can you explain the connection between load balancing and application auto-scaling?
11.What are the key steps involved in configuring scaling policies for cloud services?
12.How do threshold-based scaling policies differ from scheduled scaling policies?
13.What are the primary components of AWS Auto Scaling Groups?
14.How does the GCP Auto Scaler use metrics to decide when to scale?
15.Why is it important to have effective scaling policies in place for cloud applications?
16.How can feedback loops improve the scaling process in cloud services?
2/7/2025 Dr Hitesh Mohapatra 25

More Related Content

What's hot (20)

PDF
Resource replication in cloud computing.
Hitesh Mohapatra
 
PDF
Multitenancy in cloud computing architecture
Hitesh Mohapatra
 
PDF
Logical Network Perimeter in Cloud Computing
Hitesh Mohapatra
 
PDF
Server Consolidation in Cloud Computing Environment
Hitesh Mohapatra
 
PDF
Networking in Cloud Computing Environment
Hitesh Mohapatra
 
PDF
The life cycle of a virtual machine (VM) provisioning process
Hitesh Mohapatra
 
PPTX
Load balancing in cloud computing.pptx
Hitesh Mohapatra
 
PPT
Cloud security
Tushar Kayande
 
PPTX
Characteristics of cloud computing
GOVERNMENT COLLEGE OF ENGINEERING,TIRUNELVELI
 
PPTX
Cluster Computing
Hitesh Mohapatra
 
PPTX
Cloud Computing
Biswajit Pratihari
 
PPT
Virtualization VMWare technology
sanjoysanyal
 
PPTX
Azure Storage Services - Part 01
Neeraj Kumar
 
PPTX
Cloud computing and data security
Mohammed Fazuluddin
 
PPTX
What is private cloud Explained
jeetendra mandal
 
PPT
Cluster Computing
BOSS Webtech
 
PPTX
Deployment Models of Cloud Computing.pptx
Jaya Silwal
 
PDF
Emerging Trends in Hybrid-Cloud & Multi-Cloud Strategies
Chaitanya Atreya
 
PPTX
Virtualization security threats in cloud computing
Nitish Awasthi (anitish_225)
 
PPTX
cloud computing architecture.pptx
SourodeepChakraborty3
 
Resource replication in cloud computing.
Hitesh Mohapatra
 
Multitenancy in cloud computing architecture
Hitesh Mohapatra
 
Logical Network Perimeter in Cloud Computing
Hitesh Mohapatra
 
Server Consolidation in Cloud Computing Environment
Hitesh Mohapatra
 
Networking in Cloud Computing Environment
Hitesh Mohapatra
 
The life cycle of a virtual machine (VM) provisioning process
Hitesh Mohapatra
 
Load balancing in cloud computing.pptx
Hitesh Mohapatra
 
Cloud security
Tushar Kayande
 
Characteristics of cloud computing
GOVERNMENT COLLEGE OF ENGINEERING,TIRUNELVELI
 
Cluster Computing
Hitesh Mohapatra
 
Cloud Computing
Biswajit Pratihari
 
Virtualization VMWare technology
sanjoysanyal
 
Azure Storage Services - Part 01
Neeraj Kumar
 
Cloud computing and data security
Mohammed Fazuluddin
 
What is private cloud Explained
jeetendra mandal
 
Cluster Computing
BOSS Webtech
 
Deployment Models of Cloud Computing.pptx
Jaya Silwal
 
Emerging Trends in Hybrid-Cloud & Multi-Cloud Strategies
Chaitanya Atreya
 
Virtualization security threats in cloud computing
Nitish Awasthi (anitish_225)
 
cloud computing architecture.pptx
SourodeepChakraborty3
 

Similar to Resource Replication & Automated Scaling Listener (20)

PDF
Architecting for the cloud scability-availability
Len Bass
 
PPTX
shhsbshsuhsbsjsjshsvsbsjsiusegbejsjsjsbsbs
m23aid005
 
DOCX
Cloud Scaling Techniques: Best Practices
Xcelligen Inc
 
PPTX
AcademyCloudFoundations_Module_10 (2).pptx
rawwatchtime
 
PPTX
Scaling in cloud computing for resource utilization
Vasavi Bande
 
PDF
Workload-Aware: Auto-Scaling A new paradigm for Big Data Workloads
Vasu S
 
PDF
Auto scaling using Amazon Web Services ( AWS )
Harish Ganesan
 
PPTX
Scale net apps in aws
Codecamp Romania
 
PPTX
Scale net apps in aws
Codecamp Romania
 
PDF
VietOpenStack meetup 7th Auto-scaling
Vietnam Open Infrastructure User Group
 
PDF
aws-autoscaling.pdf
DewantoroManU
 
PDF
Autoscaling Best Practices - WebPerf Barcelona Oct 2014
Marc Cluet
 
PPTX
Automated listening (1)
ssuser6ff999
 
PDF
Auto Scaling Guided by Code
Pablo Cantero
 
PDF
Cloud Technology_Concepts
eGuvernare_Moldova
 
PDF
Sameer Mitter - Management Responsibilities by Cloud service model types
Sameer Mitter
 
PDF
LOAD BALANCING IN AUTO SCALING-ENABLED CLOUD ENVIRONMENTS
ijccsa
 
PDF
Load Balancing in Auto Scaling Enabled Cloud Environments
neirew J
 
PDF
Scalable Web Apps - Journey Through the Cloud
Ian Massingham
 
PPTX
Deploy a Scalable Website
RightScale
 
Architecting for the cloud scability-availability
Len Bass
 
shhsbshsuhsbsjsjshsvsbsjsiusegbejsjsjsbsbs
m23aid005
 
Cloud Scaling Techniques: Best Practices
Xcelligen Inc
 
AcademyCloudFoundations_Module_10 (2).pptx
rawwatchtime
 
Scaling in cloud computing for resource utilization
Vasavi Bande
 
Workload-Aware: Auto-Scaling A new paradigm for Big Data Workloads
Vasu S
 
Auto scaling using Amazon Web Services ( AWS )
Harish Ganesan
 
Scale net apps in aws
Codecamp Romania
 
Scale net apps in aws
Codecamp Romania
 
VietOpenStack meetup 7th Auto-scaling
Vietnam Open Infrastructure User Group
 
aws-autoscaling.pdf
DewantoroManU
 
Autoscaling Best Practices - WebPerf Barcelona Oct 2014
Marc Cluet
 
Automated listening (1)
ssuser6ff999
 
Auto Scaling Guided by Code
Pablo Cantero
 
Cloud Technology_Concepts
eGuvernare_Moldova
 
Sameer Mitter - Management Responsibilities by Cloud service model types
Sameer Mitter
 
LOAD BALANCING IN AUTO SCALING-ENABLED CLOUD ENVIRONMENTS
ijccsa
 
Load Balancing in Auto Scaling Enabled Cloud Environments
neirew J
 
Scalable Web Apps - Journey Through the Cloud
Ian Massingham
 
Deploy a Scalable Website
RightScale
 
Ad

More from Hitesh Mohapatra (17)

PDF
Uniform-Cost Search Algorithm in the AI Environment
Hitesh Mohapatra
 
PPT
Software Product Quality - Part 1 Presentation
Hitesh Mohapatra
 
PDF
Software Measurement and Metrics (Quantified Attribute)
Hitesh Mohapatra
 
PDF
Software project management is an art and discipline of planning and supervis...
Hitesh Mohapatra
 
PDF
Software project management is an art and discipline of planning and supervis...
Hitesh Mohapatra
 
PDF
Traditional Data Center vs. Virtualization – Differences and Benefits
Hitesh Mohapatra
 
PDF
Inter-Cloud Architecture refers to the design and organization of cloud services
Hitesh Mohapatra
 
PDF
Route Finder Using Bi-Directional BFS/DFS
Hitesh Mohapatra
 
PDF
Python Program for Depth First Search or DFS for a Graph
Hitesh Mohapatra
 
PDF
The Importance of Software Quality: Benefits and Implications for Organizatio...
Hitesh Mohapatra
 
PDF
Breadth-first search is a graph traversal algorithm
Hitesh Mohapatra
 
PDF
Cloud integration with IoT enables seamless data collection, storage, and pro...
Hitesh Mohapatra
 
PPTX
Advancements in Smart Air Pollution Monitoring: Innovations for a Sustainable...
Hitesh Mohapatra
 
PPTX
Smart Weather Monitoring System Using IoT
Hitesh Mohapatra
 
PPTX
Smart Surveillance & Emergency Response Using IoT
Hitesh Mohapatra
 
PPTX
Smart Structural Health Monitoring Through IoT and Sensor
Hitesh Mohapatra
 
PPTX
Smart Road and Application of IoT and Sensor Network
Hitesh Mohapatra
 
Uniform-Cost Search Algorithm in the AI Environment
Hitesh Mohapatra
 
Software Product Quality - Part 1 Presentation
Hitesh Mohapatra
 
Software Measurement and Metrics (Quantified Attribute)
Hitesh Mohapatra
 
Software project management is an art and discipline of planning and supervis...
Hitesh Mohapatra
 
Software project management is an art and discipline of planning and supervis...
Hitesh Mohapatra
 
Traditional Data Center vs. Virtualization – Differences and Benefits
Hitesh Mohapatra
 
Inter-Cloud Architecture refers to the design and organization of cloud services
Hitesh Mohapatra
 
Route Finder Using Bi-Directional BFS/DFS
Hitesh Mohapatra
 
Python Program for Depth First Search or DFS for a Graph
Hitesh Mohapatra
 
The Importance of Software Quality: Benefits and Implications for Organizatio...
Hitesh Mohapatra
 
Breadth-first search is a graph traversal algorithm
Hitesh Mohapatra
 
Cloud integration with IoT enables seamless data collection, storage, and pro...
Hitesh Mohapatra
 
Advancements in Smart Air Pollution Monitoring: Innovations for a Sustainable...
Hitesh Mohapatra
 
Smart Weather Monitoring System Using IoT
Hitesh Mohapatra
 
Smart Surveillance & Emergency Response Using IoT
Hitesh Mohapatra
 
Smart Structural Health Monitoring Through IoT and Sensor
Hitesh Mohapatra
 
Smart Road and Application of IoT and Sensor Network
Hitesh Mohapatra
 
Ad

Recently uploaded (20)

PPTX
Growth and development and milestones, factors
BHUVANESHWARI BADIGER
 
PPTX
SPINA BIFIDA: NURSING MANAGEMENT .pptx
PRADEEP ABOTHU
 
PPTX
Soil and agriculture microbiology .pptx
Keerthana Ramesh
 
PPTX
How to Set Maximum Difference Odoo 18 POS
Celine George
 
PDF
ARAL_Orientation_Day-2-Sessions_ARAL-Readung ARAL-Mathematics ARAL-Sciencev2.pdf
JoelVilloso1
 
PDF
LAW OF CONTRACT ( 5 YEAR LLB & UNITARY LLB)- MODULE-3 - LEARN THROUGH PICTURE
APARNA T SHAIL KUMAR
 
PPSX
HEALTH ASSESSMENT (Community Health Nursing) - GNM 1st Year
Priyanshu Anand
 
PPTX
Stereochemistry-Optical Isomerism in organic compoundsptx
Tarannum Nadaf-Mansuri
 
PDF
The History of Phone Numbers in Stoke Newington by Billy Thomas
History of Stoke Newington
 
PDF
CEREBRAL PALSY: NURSING MANAGEMENT .pdf
PRADEEP ABOTHU
 
PDF
Knee Extensor Mechanism Injuries - Orthopedic Radiologic Imaging
Sean M. Fox
 
PPTX
grade 5 lesson matatag ENGLISH 5_Q1_PPT_WEEK4.pptx
SireQuinn
 
PDF
The Constitution Review Committee (CRC) has released an updated schedule for ...
nservice241
 
PPTX
2025 Winter SWAYAM NPTEL & A Student.pptx
Utsav Yagnik
 
PPTX
Unit 2 COMMERCIAL BANKING, Corporate banking.pptx
AnubalaSuresh1
 
PDF
Biological Bilingual Glossary Hindi and English Medium
World of Wisdom
 
PPTX
I AM MALALA The Girl Who Stood Up for Education and was Shot by the Taliban...
Beena E S
 
PPTX
A PPT on Alfred Lord Tennyson's Ulysses.
Beena E S
 
PDF
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - GLOBAL SUCCESS - CẢ NĂM - NĂM 2024 (VOCABULARY, ...
Nguyen Thanh Tu Collection
 
PPTX
How to Convert an Opportunity into a Quotation in Odoo 18 CRM
Celine George
 
Growth and development and milestones, factors
BHUVANESHWARI BADIGER
 
SPINA BIFIDA: NURSING MANAGEMENT .pptx
PRADEEP ABOTHU
 
Soil and agriculture microbiology .pptx
Keerthana Ramesh
 
How to Set Maximum Difference Odoo 18 POS
Celine George
 
ARAL_Orientation_Day-2-Sessions_ARAL-Readung ARAL-Mathematics ARAL-Sciencev2.pdf
JoelVilloso1
 
LAW OF CONTRACT ( 5 YEAR LLB & UNITARY LLB)- MODULE-3 - LEARN THROUGH PICTURE
APARNA T SHAIL KUMAR
 
HEALTH ASSESSMENT (Community Health Nursing) - GNM 1st Year
Priyanshu Anand
 
Stereochemistry-Optical Isomerism in organic compoundsptx
Tarannum Nadaf-Mansuri
 
The History of Phone Numbers in Stoke Newington by Billy Thomas
History of Stoke Newington
 
CEREBRAL PALSY: NURSING MANAGEMENT .pdf
PRADEEP ABOTHU
 
Knee Extensor Mechanism Injuries - Orthopedic Radiologic Imaging
Sean M. Fox
 
grade 5 lesson matatag ENGLISH 5_Q1_PPT_WEEK4.pptx
SireQuinn
 
The Constitution Review Committee (CRC) has released an updated schedule for ...
nservice241
 
2025 Winter SWAYAM NPTEL & A Student.pptx
Utsav Yagnik
 
Unit 2 COMMERCIAL BANKING, Corporate banking.pptx
AnubalaSuresh1
 
Biological Bilingual Glossary Hindi and English Medium
World of Wisdom
 
I AM MALALA The Girl Who Stood Up for Education and was Shot by the Taliban...
Beena E S
 
A PPT on Alfred Lord Tennyson's Ulysses.
Beena E S
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 - GLOBAL SUCCESS - CẢ NĂM - NĂM 2024 (VOCABULARY, ...
Nguyen Thanh Tu Collection
 
How to Convert an Opportunity into a Quotation in Odoo 18 CRM
Celine George
 

Resource Replication & Automated Scaling Listener

  • 1. Resource Replication & Automated Scaling Listener Dr Hitesh Mohapatra School of Computer Engineering Associate Professor KIIT (Deemed to be) University
  • 2. Contents Resource Replication: • Concept of replication in cloud • Types of replications (synchronous vs asynchronous) • Benefits: High availability, disaster recovery • Example: Replication in distributed systems Automated Scaling Listener: • What is automated scaling and why it's important? • How listeners help detect when scaling is required • Auto-scaling groups in AWS, GCP auto-scaler • Configuration of scaling policies 2/7/2025 Dr Hitesh Mohapatra 2
  • 3. Resource Replication {Follow the hyperlink} Resource Replication • Concept of replication in cloud • Types of replications (synchronous vs asynchronous) • Benefits: High availability, disaster recovery • Example: Replication in distributed systems 2/7/2025 Dr Hitesh Mohapatra 3
  • 4. Automated Scaling Listener Objective • What is automated scaling and why it's important? • How listeners help detect when scaling is required • Auto-scaling groups in AWS, GCP auto-scaler • Configuration of scaling policies 2/7/2025 Dr Hitesh Mohapatra 4
  • 5. Automated Scaling Listener •Service Agent Definition: •Known as the automated scaling listener mechanism. •Function: •Tracks and monitors communications between cloud service users and cloud services. •Purpose: •Supports dynamic scaling. •Installation: •Installed in the cloud, typically close to the firewall. •Monitoring: •Continuously tracks data on the status of the workload. •Assessment Criteria: •Based on the number of requests made by cloud users. •Also based on the demands placed on the backend by particular types of requests. •Example: •Processing a small amount of incoming data can take a significant amount of time. 2/7/2025 Dr Hitesh Mohapatra 5
  • 7. Why it is required ? • Automatically Adjusting IT Resources based on previously set parameters by the cloud consumer (Auto Scaling). • Automatic Notification of the cloud consumer when workloads go above or below predetermined thresholds. This gives the cloud user the option to change how its present IT resources are allocated. (Auto Notification) 2/7/2025 Dr Hitesh Mohapatra 7
  • 9. Steps 1. Service Agent Roles: •The service agents that perform the role of autonomous scaling listeners go by many names depending on the cloud provider vendor. 2. Initial Access Attempt: •Three users of cloud services simultaneously try to access one cloud service. 3. Creation of Instances: •The autonomous scaling listener creates three duplicate instances of the service when it grows out. 4. Additional Access Attempt: •A fourth user of a cloud service tries to access the service. 5. Limit Exceeded Alert: •The automated scaling listener rejects the fourth attempt and alerts the cloud consumer that the intended workload limit has been exceeded because the cloud service is only designed to enable up to three instances. 6. Administrator Action: •To modify the provisioning configuration and raise the redundant instance limit, the cloud resource administrator of the cloud consumer logs into the remote administration environment. 2/7/2025 Dr Hitesh Mohapatra 9
  • 10. How listeners help detect when scaling is required? 1. Monitoring Workloads: • Continuously track data on the status of workloads. • Monitor the number of incoming requests and the load on the backend services. 2. Analyzing Data: • Assess the patterns and trends in the data to identify peaks and troughs in usage. • Evaluate the types of requests and their impact on resource consumption. 3. Thresholds and Alerts: • Set predefined thresholds for various metrics like CPU usage, memory usage, and network traffic. • Trigger alerts when these thresholds are crossed, indicating a need for scaling. 2/7/2025 Dr Hitesh Mohapatra 10
  • 11. Cont. 4. Resource Allocation: • Determine the appropriate number of resources needed to handle the current and projected workload. • Automatically allocate or deallocate resources based on real-time demand. 5. Preventing Overload: • Prevent overloading of services by ensuring that additional instances are created when demand spikes. • Reject new requests or scale down resources when the demand decreases, maintaining optimal performance. 6. Feedback Loops: • Implement feedback loops to continuously improve the scaling process. • Adjust the thresholds and resource allocation strategies based on past performance and usage patterns. 2/7/2025 Dr Hitesh Mohapatra 11
  • 12. Cont. By effectively monitoring and analyzing the workload data, automated scaling listeners help maintain the efficiency, reliability, and performance of cloud services. They ensure that resources are dynamically scaled to meet the demands of the users, preventing both underutilization and overload. 2/7/2025 Dr Hitesh Mohapatra 12
  • 13. Difference between Auto Scaling vs Load Balancing Feature Auto Scaling Load Balancing Primary Function Automatically adjusts the number of instances based on demand Distributes incoming traffic across multiple instances Purpose To ensure optimal resource utilization and handle varying loads To ensure high availability and reliability by balancing load Operation Adds or removes instances as needed Distributes traffic based on predefined rules or algorithms Focus Resource scaling Traffic distribution Usage Scenario Scaling in and out instances based on application needs Balancing traffic load across running instances Reduction of Backend Duties Manages instance scaling, reducing manual intervention Balances load, manages traffic, and monitors server health Combination Often used together for optimal performance and scalability Often used together with auto-scaling for efficient traffic management Example Tools AWS Auto Scaling, Azure Autoscale Elastic Load Balancing (ELB), Azure Load Balancer 2/7/2025 Dr Hitesh Mohapatra 13
  • 14. Auto-scaling groups in AWS Definition: •Auto Scaling Groups (ASGs) are a collection of Amazon EC2 instances managed as a logical grouping for automatic scaling and management. Components: •Launch Configuration: Defines the instance type, AMI ID, key pair, security groups, and other configurations. •Scaling Policies: Determine how and when the ASG should scale in or out based on predefined criteria. 2/7/2025 Dr Hitesh Mohapatra 14
  • 15. Cont. Dynamic and Predictive Scaling: •Dynamic Scaling: Adjusts the number of instances based on real-time demand (e.g., CPU utilization). •Predictive Scaling: Uses historical data to predict and provision resources ahead of time. Health Checks: •Continuously monitors the health of instances within the group. •Automatically replaces unhealthy instances to ensure high availability. Benefits: •Automatically adjusts capacity to maintain steady, predictable performance at the lowest possible cost. •Ensures application availability by automatically adding or removing instances as needed. 2/7/2025 Dr Hitesh Mohapatra 15
  • 16. GCP auto-scaler Definition: The GCP Auto Scaler automatically adjusts the number of VM instances in a managed instance group based on the current load. Components: •Instance Group Manager: Manages the lifecycle of VM instances within the instance group. •Scaling Policies: Define the metrics and thresholds for scaling actions. 2/7/2025 Dr Hitesh Mohapatra 16
  • 17. Cont. Scaling Metrics: •Uses various metrics such as CPU utilization, HTTP load balancing serving capacity, and stackdriver custom metrics to decide when to scale. Auto-Healing: •Automatically recreates failed instances to maintain the desired state of the instance group. Predictive Autoscaler: •Uses machine learning to analyze historical usage data and predict future demand. •Provision resources proactively to meet anticipated demand. Benefits: •Provides cost savings by automatically adjusting resources based on demand. •Enhances application performance by ensuring sufficient capacity to handle the load. 2/7/2025 Dr Hitesh Mohapatra 17
  • 18. Example: Autoscaling based on CPU Utilization 2/7/2025 Dr Hitesh Mohapatra 18
  • 19. For example, assume the load balancing serving capacity of a managed instance group is defined as 100 RPS per instance. If you create an autoscaler with the HTTP(S) load balancing policy and set it to maintain a target utilization level of 0.8 or 80%, the autoscaler will add or remove instances from the managed instance group to maintain 80% of the serving capacity, or 80 RPS per instance. 2/7/2025 Dr Hitesh Mohapatra 19
  • 20. 2/7/2025 Dr Hitesh Mohapatra 20
  • 21. Difference between Horizontal vs Vertical Auto Scaling Aspect Horizontal Auto Scaling Vertical Auto Scaling Definition Addition of more servers or computers to the auto- scaling group Scaling by supplying more power (e.g., more RAM) to an existing system Scalability Expands the resource pool with more machines Boosts the power of an already-running system Handling High User Load Can handle queries from thousands of users Limited in handling very high user loads Components Clustering, distributed file systems, load balancing Increasing resources like CPU and RAM Stateless Servers Crucial for handling large number of users; sessions can move across servers Not applicable Downtime No downtime required; creates new instances separately Requires downtime for upgrades and reconfigurations Availability Improves availability and performance due to independence No improvement in availability; dependent on a single machine Performance Enhances user experience with browser-side session storage Improves performance Redundancy Supports redundancy with multiple instances No redundant server; dependent on single location Elastic Load Balancing Scales incoming requests across instances Not applicable; deals with vertical resource allocation Best Use Cases Ideal for applications with a large user base Suitable for applications with fewer scalability demands Challenges Requires effective clustering and load balancing Architectural issues due to single machine dependency Overall Impact Enhances scalability, availability, and user experience Boosts performance but limited in scalability and availability 2/7/2025 Dr Hitesh Mohapatra 21
  • 22. Configuration of scaling policies Step 1: Define Your Scaling Goals • Identify Key Metrics: Determine which metrics (e.g., CPU utilization, memory usage, request count) will trigger scaling actions. • Set Desired Performance Levels: Establish the target performance levels for your application, such as response time and availability. Step 2: Create Scaling Policies • Threshold-Based Policies: Define specific thresholds for your key metrics. For example, if CPU utilization exceeds 70%, trigger a scale-out action. • Scheduled Policies: Schedule scaling actions based on predictable usage patterns. For example, scale out during peak hours and scale in during off- peak hours. • Predictive Policies: Use machine learning models to predict future demand and proactively scale resources accordingly. 2/7/2025 Dr Hitesh Mohapatra 22
  • 23. Cont. Step 3: Configure Alarms and Triggers • Set Alarms: Create alarms that monitor the key metrics and trigger scaling actions when thresholds are reached. • Define Triggers: Specify the conditions under which scaling actions should be triggered, such as exceeding or falling below the defined thresholds. Step 4: Define Scaling Actions • Scale-Out Actions: Specify how many instances to add when scaling out. For example, add two instances if CPU utilization exceeds 70%. • Scale-In Actions: Specify how many instances to remove when scaling in. For example, remove one instance if CPU utilization falls below 30%. 2/7/2025 Dr Hitesh Mohapatra 23
  • 24. Cont. Step 5: Configure Cooldown Periods • Set Cooldown Periods: Define cooldown periods to prevent rapid, repetitive scaling actions. This allows the system to stabilize before triggering another scaling action. Step 6: Implement and Test Policies • Deploy Policies: Implement the scaling policies in your cloud service configuration. • Test Policies: Test the policies under different load conditions to ensure they work as expected and make adjustments as needed. Step 7: Monitor and Optimize • Continuous Monitoring: Continuously monitor the performance and effectiveness of your scaling policies. • Optimize Policies: Regularly review and optimize the scaling policies based on performance data and changing application requirements. 2/7/2025 Dr Hitesh Mohapatra 24
  • 25. Questions 1. What is the primary function of an automated scaling listener mechanism? 2. How do automated scaling listeners support dynamic scaling in the cloud? 3. Where are automated scaling listeners typically installed in the cloud? 4. What data do automated scaling listeners continuously track to assess workloads? 5. How does an autonomous scaling listener respond when multiple users try to access the same cloud service? 6. What action does an automated scaling listener take when the intended workload limit is exceeded? 7. What is the main difference between horizontal auto-scaling and vertical auto-scaling? 8. Why is stateless server architecture crucial for horizontal auto-scaling? 9. How does load balancing improve the availability and performance of cloud services? 10.Can you explain the connection between load balancing and application auto-scaling? 11.What are the key steps involved in configuring scaling policies for cloud services? 12.How do threshold-based scaling policies differ from scheduled scaling policies? 13.What are the primary components of AWS Auto Scaling Groups? 14.How does the GCP Auto Scaler use metrics to decide when to scale? 15.Why is it important to have effective scaling policies in place for cloud applications? 16.How can feedback loops improve the scaling process in cloud services? 2/7/2025 Dr Hitesh Mohapatra 25