AWS DevDay Vienna - Resiliency and availability design patterns for the cloud

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
V I E N N A
24.10.19
Resiliency and availability design
patterns for the cloud
Cobus Bernard
Senior Technical Evangelist
Amazon Web Services
@cobusbernard
cobusbernard
cobusbernard
B A R 3

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Can you guess whatwillhappen?

Distributed Systems are hard

Failures areagivenand
everythingwilleventuallyfail
over time.
Werner Vogels
CTO – Amazon.com
“ “

Resiliency:Abilityfor asystemtohandle and
eventuallyrecover from unexpected conditions
conditions

Partialfailure mode

Why do we build resilient software systems?

Thecostof downtime
Annual Fortune
1000 application
downtime costs
(IDC)
$1.25 to
$2.5B
Average cost of a
data breach
(Ponemon
Institute)
$3.6M
Cost/hr of a
critical
application
failure (IDC)
$500K
to $1M
Average cost/hr
of downtime
(Ponemon
Institute)
$474K
Average cost per
lost or stolen
record (Ponemon
Institute)
$141

How do we build resilient software systems?

People
Application
Network & Data
Infrastructure

Let’s talk aboutAvailability

Systemavailability
Availability =
Normal Operation Time
Total Time
MTBF**
MTBF** + MTTR*
=
* Mean Time To Repair (MTTR)
**Mean Time Between Failure (MTBF)

Reading homework

Availabilityinparallel
A = 1 – (1 – Ax)2
Part X
Part X

Availabilityinparallel
Component Availability Downtime
X 99% (2-nines) 3 days 15 hours
Two X in parallel 99.99% (4-nines) 52 minutes
Three X in parallel 99.9999% (6-nines) 31 seconds

Componentredundancyincreases availability
significantly!

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Global Infrastructure
• 22 Regions with 69 Availability Zones
• 3 Regions coming soon: Cape Town
Jakarta and Milan
• 100Gbps redundant network
• 99.99% SLA

Fully-scaledAvailabilityZone

Highlyredundant regional network

AWS Region and availabilityzones
Region
Availability zone a Availability zone b Availability zone c
data center
data center
data center
1 or more data centers per AZ
2 or more AZs per region (new regions min 3)
data center
data center
data center
data center
data center
data center

How about a global architecture?

Once upon a time …
Origin

And Now …
Origin
~300ms

Origin
Improve latency for end-users
Origin

Improve availabilityand disasterrecovery
Applications in
US West
Applications in
US East
Users from
San Francisco
Users from
New York
Service 1
Service 2
Service 3
Service 4
Service 1
Service 2
Service 3
Service 4

So should we go for a global architecture?

Perfect your regional architecture first!

Let’s talk about Multi-AZ

Multi-AZ architecture
Region
Instances Instances Instances
DB Instance DB instance
standby
Elastic Load
Balancing (ELB)

Region
new master
Elastic Load
Balancing (ELB)

• Enables fault-tolerant applications
• AWS regional services designed to
withstand AZ failures
• Leveraged by AWS regional
services such as Amazon S3,
Amazon DynamoDB, Amazon
Aurora, Amazon ELBs, etc.
Region
standby
Elastic Load
Balancing (ELB)

Let’s talk about auto scaling

Auto-Scaling
FixedVariable

Availability zone 1
Auto Scaling group
AWS Region
Availability zone 2
Auto-scaling for self-healing
Elastic Load
Balancing (ELB)
X

Let’s talk about the AWS responsibility
models

AWSoperationalresponsibilitymodels
On-Premises Cloud
Less More
Compute Virtual Machine
EC2 Elastic Beanstalk AWS LambdaFargate
Databases MySQL MySQL on EC2
RDS MySQL RDS Aurora Aurora Serverless DynamoDB
Storage Storage
S3
Messaging ESBs
Amazon MQ Kinesis SQS / SNS
Analytics
Hadoop Hadoop on EC2 EMR Elasticsearch Service Athena
Firehose

Let’s talk about databases

Common resiliencyissueswithDatabases??
REPLICATION BACKUPSSCALING

Amazon DynamoDB: purpose-built for thecloud
Fast and flexible key value database service for any scale
Comprehensive
security
Encrypts all data by default
and fully integrates with
AWS Identity and Access
Management for robust
security
Performance at scale
Consistent, single-digit
millisecond response times at any
scale; build applications with
virtually unlimited throughput
Global database for
global users and apps
Build global applications with fast
access to local data by easily
replicating tables across multiple
AWS Regions
Serverless
No hardware provisioning,
software patching, or upgrades;
scales up or down
automatically; continuously
backs up your data

RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
AZ 1
AZ 2
AZ 3
Network
RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
Storage
Node
Leader
PutItem

RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
AZ 1
AZ 2
AZ 3
RR RR RR RR RR
RR RR RR RR RR
RR RR RR RR RR
Storage
Node
Leader
GetItem
Network

Amazon.com, Nike, Netflix, Duolingo, Lyft, Airbnb, Samsung,
Toyota, and Capital One depend on the scale and performance of
DynamoDB to support their workloads.
10 trillion requests
per day
20 million
requests per second

AmazonAurora: purpose-built for thecloud
MySQL and PostgreSQL compatible relational database
Availability
and durability
Fault-tolerant, self-healing
storage; six copies of data
across three AZs; continuous
backup to S3
Fully managed
Managed by RDS:
no hardware provisioning,
software patching, setup,
configuration, or backups
Highly secure
Network isolation,
encryption at rest/transit
Performance
and scalability
5x throughput of standard MySQL
and 3x of standard PostgreSQL;
scale-out up to
15 read replicas

BINLOG DATA DOUBLE-WRITELOG FRM FILES
MYSQL WITH REPLICA
EBS mirrorEBS mirror
AZ 1 AZ 2
Amazon S3
EBS
Amazon Elastic
Block Store (EBS)
Primary
Instance
Replica
Instance
1
2
3
4
5
AZ 1 AZ 3
Primary
Instance
AZ 2
Replica
Instance
Replica
Instance
AMAZON AURORA
TYPE OF WRITE
AmazonAurora: distributed architecture

Capital One, GE, Dow Jones, Netflix, Verizon, Expedia and Amazon depend on
the scale and performance of Aurora to support their workloads.
5X the
throughput of
standard
MySQL
15 low
latency read
replicas
Failover in
less than 30
seconds
3X the
throughput of
standard
PostgreSQL

Purpose-built databases
Relational Key-value Document In-memory Graph Time-series Ledger
DynamoDB NeptuneAmazon RDS
Aurora CommercialCommunity
Timestream QLDBElastiCacheDocumentDB

Read-Writeseparation
Master Read Replica Read Replica Read Replica
Instance InstanceInstance

DatabaseFederation
Users
DB
Products
DB
Master
(Read) Replica
Master
(Read) Replica

DatabaseSharding User ShardID
002345 A
002346 B
002347 C
002348 B
002349 A
CBA
Master
(Read) Replica
Master
(Read) Replica
Master
(Read) Replica

Let’s talk about backups

AWS Backup service

PreventingAccidentalTableDeletion
https://aws.amazon.com/blogs/database/preventing-accidental-table-deletion-in-dynamodb/

PreventingAccidentalTableDeletion (sql)

Practice and testrecoveryfrom your backups!!

Let’s talk about timeouts, backoff &
retries!

Users
App
DB
Conn
Pool
INSERT
INSERT
INSERT
INSERT
What happens if the DB “slows down”?
Timeout client side Timeout backend side ??

User 1
App
DB
Conn
Pool
INSERT
Timeout client side = 10s Timeout backend side = default = Infinite
Retry INSERT
Retry INSERT
ERROR: Failed to get connection from pool
Retry

https://docs.microsoft.com/en-us/dotnet/api/system.net.httpwebrequest.timeout

https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html

@timeout_decorator.timeout(5, timeout_exception=StopIteration)
def timed_get(url):
return requests.get(url)
https://pypi.org/project/timeout-decorator/

Set thetimeouts!

How elsecould wehaveprevented theerror?
User 1
DB
Conn
Pool
INSERT
Retry INSERT
Retry INSERT
Retry
ERROR: Failed to get connection from pool

User 1
DB
Conn
Pool
INSERT
Timeout client side = 10s Timeout backend side = 10s
Wait 2s before Retry
INSERT
INSERT
Backing off betweenretries
Releasing connectionsBackoff

No jitter With jitter
https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
SimpleExponentialBackoffisnotenough:AddJitter

Example: add jitter0-1000ms
def get_item(self, url, n=1):
MAX_TRIES = 12
try:
res = requests.get(url)
except:
if n > MAX_TRIES:
return None
n += 1
time.sleep((2 ** n) + (random.randint(0, 1000) / 1000.0))
return self.get_item(url, n)
else:
return res

Idempotent operation
No additional effect if it is called more than
once with the same input parameters.

Let’s talk about health checking!

Auto Scaling group
Service A
Availability zone 1
Auto Scaling group
AWS Region
Service A
Availability zone 2
Service BService B
database Email
Probing for health
Cluster

Shallowhealthcheck
Instance
Cache node
Email
database
Cluster
Are you healthy?
yes

Deep healthcheck
Instance
Cache node
Email
database
Cluster
Are you healthy?
yes
Are you healthy?
yes
yes
yes
yes

Deep healthcheck
Instance
Cache node
Email
database
Cluster
Are you healthy?
no
Are you healthy?
no
yes
yes
yes

Prioritize shallow health checks during
hard times.
Cache.

Let’s talk about load shedding.

Cheaply reject excess work

Be careful when selecting the right metric

Don’tbeoverlyoptimisticandtakeonmorethanyoucan.
Findanoperationalmetrictorejectwhatyoucannottakein.
Favorcachedandstaticcontent
PrioritizeELBhealthcheck(shallow)pings
Inanoverloadsituationyouhavepreciousresources,donotletany
ofitgotowaste.
Load Shedding

Let’s talk aboutresiliency (chaos) engineering

Fire Drills

GameDay atAmazon
CreatingResiliencyThroughDestruction
https://www.youtube.com/watch?v=zoz0ZjfrQ9s

Chaosengineering
https://github.com/Netflix/SimianArmy

“Chaos Engineeringis the discipline of
experimentingon a distributedsystem
in orderto buildconfidence in the system’s
capabilitytowithstand turbulentconditionsin
production.”
http://principlesofchaos.org

Failureinjection
• Start small & build confidence
• Application level
• Host failure
• Resource attacks (CPU, memory, …)
• Network attacks (dependencies, latency, …)
• Region attacks
• “Paul” attack
https://www.gremlin.comhttps://github.com/Netflix/SimianArmy https://chaostoolkit.org

STEADY
STATE
HYPOTHESIS
RUN
EXPERIMENT
VERIFY
FIX!
PhasesofChaosEngineering

https://aws.amazon.com/wellarchitected

Thank you!
@cobusbernard
cobusbernard
cobusbernard

AWS DevDay Vienna - Resiliency and availability design patterns for the cloud

More Related Content

Similar to AWS DevDay Vienna - Resiliency and availability design patterns for the cloud (20)

More from Cobus Bernard (20)

Recently uploaded (20)

AWS DevDay Vienna - Resiliency and availability design patterns for the cloud