![data store
[A-G]
Option 2: Clustering
machine 1
data store
[H-S]
machine 2
data store
[T-Z]
machine 3
NoSQL](https://image.slidesharecdn.com/5levelsofhighavailabilityfrommulti-instancetohybridcloud-200421103934/85/5-Levels-of-High-Availability-From-Multi-instance-to-Hybrid-Cloud-41-320.jpg)
![data store
[A-G]
Option 2: Clustering
machine 1
data store
[H-S]
machine 2
application service
application service
application service
data store
[T-Z]
machine 3](https://image.slidesharecdn.com/5levelsofhighavailabilityfrommulti-instancetohybridcloud-200421103934/85/5-Levels-of-High-Availability-From-Multi-instance-to-Hybrid-Cloud-42-320.jpg)
![data store
[A-G]
Option 2: Clustering
machine 1
data store
[H-S]
machine 2
application service
application service
application service
data store
[T-Z]
machine 3](https://image.slidesharecdn.com/5levelsofhighavailabilityfrommulti-instancetohybridcloud-200421103934/85/5-Levels-of-High-Availability-From-Multi-instance-to-Hybrid-Cloud-43-320.jpg)
![data store
[A-G]
Option 2: Clustering
machine 1
data store
[H-S]
machine 2
data store
[T-Z]
machine 3
NoSQL](https://image.slidesharecdn.com/5levelsofhighavailabilityfrommulti-instancetohybridcloud-200421103934/85/5-Levels-of-High-Availability-From-Multi-instance-to-Hybrid-Cloud-45-320.jpg)
5 Levels of High Availability: From Multi-instance to Hybrid Cloud
- 1. 5 Levels of High Availability From Multi-instance to Hybrid Cloud Rafał Leszko @RafalLeszko rafalleszko.com Hazelcast
- 2. About me ● Cloud Software Engineer at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland
- 3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast
- 4. ● Introduction ● High Availability Levels ○ Level 0: Single Instance ○ Level 1: Multi Instance ○ Level 2: Multi Zone ○ Level 3: Multi Region ○ Level 4: Multi Cloud ○ Level 5: Hybrid Cloud ● Summary Agenda
- 5. Introduction
- 13. Stateless
- 18. application servicedata store queue other service
- 19. Data is the problem!
- 20. ● Introduction ✔ ● High Availability Levels ○ Level 0: Single Instance ○ Level 1: Multi Instance ○ Level 2: Multi Zone ○ Level 3: Multi Region ○ Level 4: Multi Cloud ○ Level 5: Hybrid Cloud ● Summary Agenda
- 21. Level 0: Single Instance
- 22. application service data store Level 0: Single Instance machine request
- 23. Level 0: Single Instance YOLO!
- 26. What does "Level 0: Single Instance" mean to You? No high availability! No scalability! Super low latency: ● in-process memory ● no network ● local file system Data consistency
- 27. ● Introduction ✔ ● High Availability Levels ○ Level 0: Single Instance ✔ ○ Level 1: Multi Instance ○ Level 2: Multi Zone ○ Level 3: Multi Region ○ Level 4: Multi Cloud ○ Level 5: Hybrid Cloud ● Summary Agenda
- 28. Level 1: Multi Instance
- 29. If one machine is down, the system is still available
- 30. application data Level 1: Multi Instance request data load balancer application application application application machine 1 machine 2 machine 3 machine 4
- 31. application data Level 1: Multi Instance data application application application application machine 1 machine 2 machine 3 machine 4
- 32. data Level 1: Multi Instance data machine 3 machine 4
- 33. data store Level 1: Multi Instance machine 1 data store machine 2 Assumptions: ● Local network ● Fast ● Reliable For example: ● EC2 Instances in the same availability zone ● GCP VM instances in the same zone ● Your on-premises server machines connected with LAN
- 35. data store Level 1: Multi Instance machine 1 data store machine 2
- 36. Data replication
- 37. data store Option 1: Active-Passive (Master-Slave) Replication machine 1 data store machine 2 application service application service application service active passive SQL
- 38. data store Option 1: Active-Passive (Master-Slave) Replication machine 1 data store machine 2 application service application service application service active passive
- 39. data store Option 1: Active-Passive (Master-Slave) Replication machine 1 data store machine 2 application service application service application service down active
- 40. data store Option 1: Active-Passive (Master-Slave) Replication machine 1 data store machine 2 application service application service application service active passive SQL
- 41. data store [A-G] Option 2: Clustering machine 1 data store [H-S] machine 2 data store [T-Z] machine 3 NoSQL
- 42. data store [A-G] Option 2: Clustering machine 1 data store [H-S] machine 2 application service application service application service data store [T-Z] machine 3
- 43. data store [A-G] Option 2: Clustering machine 1 data store [H-S] machine 2 application service application service application service data store [T-Z] machine 3
- 44. 1 Hazelcast Member 1 data partitions 4 7 data partition backups 3 5 9 3 Hazelcast Member 3 data partitions 6 9 data partition backups 2 4 8 2 Hazelcast Member 2 data partitions 5 8 data partition backups 1 6 7
- 45. data store [A-G] Option 2: Clustering machine 1 data store [H-S] machine 2 data store [T-Z] machine 3 NoSQL
- 47. Synchronous (Consistency) or Asynchronous (Latency)? data store machine 1 machine 2 machine 3 data store machine 1 data store machine 2 active passive data store data store
- 51. Synchronous (Consistency) or Asynchronous (Latency)? data store machine 1 machine 2 machine 3 data store machine 1 data store machine 2 active passive data store data store synchronous
- 52. Synchronous (Consistency) or Asynchronous (Latency)? data store machine 1 machine 2 machine 3 data store machine 1 data store machine 2 active passive data store data store synchronous?
- 53. What does "Level 1: Multi Instance" mean to You? Data consistency! Most tools supported Cloud-specific toolkit (e.g. AWS SQS) Simple setup (even on-premises) High latency if accessed multi regions
- 54. ● Introduction ✔ ● High Availability Levels ○ Level 0: Single Instance ✔ ○ Level 1: Multi Instance ✔ ○ Level 2: Multi Zone ○ Level 3: Multi Region ○ Level 4: Multi Cloud ○ Level 5: Hybrid Cloud ● Summary Agenda
- 55. Level 2: Multi Zone
- 56. If one availability zone is down, the system is still available
- 58. application data Level 2: Multi Zone request data load balancer application application application application zone 1 zone 2
- 59. Is multi-zone deployment any different?
- 60. No
- 61. No but… Yes
- 64. No but… Yes
- 65. Level 2: Multi Zone zone 1 zone 2 data data data data
- 66. Level 2: Multi Zone zone 1 zone 2 data data data data Assumptions: ● Machines in at least 2 AZ ● Fast and reliable network For example: ● EC2 Instances in 2 AWS Availability Zones ● Azure VM instances in 2 Availability Sets
- 67. 1 Hazelcast Member 1 data partitions 4 7 data partition backups 3 5 9 3 Hazelcast Member 3 data partitions 6 9 data partition backups 2 4 8 2 Hazelcast Member 2 data partitions 5 8 data partition backups 1 6 7
- 68. Level 2: Multi Zone machine 1 machine 2 Hazelcast Member 2 zone 1 zone 2 Hazelcast Member 1 machine 3 machine 4 Hazelcast Member 4Hazelcast Member 3
- 69. Hazelcast configuration: hazelcast: partition-group: enabled: true group-type: ZONE_AWARE Hazelcast Zone Aware Feature
- 70. Hazelcast Zone Aware Feature
- 71. Level 2: Multi Zone machine 1 machine 2 Hazelcast Member 2 zone 1 zone 2 Hazelcast Member 1 machine 3 machine 4 Hazelcast Member 4Hazelcast Member 3
- 72. No but… Yes
- 73. Make sure your data store is ZONE AWARE
- 74. What does "Level 2: Multi Zone" mean to You? Currently top 1 choice! Data consistency! Cloud-specific toolkit (e.g. AWS SQS) High latency if accessed multi regions Not all tools are "zone aware"
- 75. ● Introduction ✔ ● High Availability Levels ○ Level 0: Single Instance ✔ ○ Level 1: Multi Instance ✔ ○ Level 2: Multi Zone ✔ ○ Level 3: Multi Region ○ Level 4: Multi Cloud ○ Level 5: Hybrid Cloud ● Summary Agenda
- 76. Level 3: Multi Region
- 77. If one region is down, the system is still available
- 78. application data Level 3: Multi Region geo load balancer region 1 load balancer load balancer region 2 application application application application application application application data data data data data geo replication
- 79. data Level 3: Multi Region region 1 region 2 data data data data data geo replication Assumptions: ● Machines in at least 2 geographical regions ● Network may be slow and unreliable For example: ● EC2 Instances in regions: eu-central- 1 and us-west-2
- 80. 10 000 km
- 81. Speed of light: 300 000 km/s Distance: 10 000 km RTT (Round Trip Time) = 60 ms Level 3: Multi Region
- 82. Geo-replication
- 83. data data data geo replication Level 3: Multi Region (Geo-replication) data data data
- 84. Geo-replication ● It's asynchronous ● Your data store must support it ● You must be prepared for data loss ● Two modes: ○ Active-Passive ○ Active-Active
- 86. data data data geo replication Active-Passive Geo-replication data data data active passive ● data loss possible ● (eventual) consistency
- 88. data data data geo replication Active-Active Geo-replication data data data active active ● data loss possible ● eventual consistency ● conflict resolution
- 89. Hazelcast WAN Replication hazelcast: wan-replication: batch-publisher: target-endpoints: 35.184.122.109
- 90. Do I really need to lose consistency?
- 95. Strong Consistency in Multi Region ● NewSQL (Spanner, CockroachDB) ● Multi-region distributed transactions ● Consensus algorithms (Paxos, Raft) ● Always a trade-off: consistency vs latency
- 96. What does "Level 3: Multi Region" mean to You? Super high available! Low latency if accessed from multi regions Sometimes possible to use Cloud-specific toolkit (e.g. Google Spanner - yes, AWS Elasticache - no) Geo-replication (asynchronous)! Eventual consistency (conflict resolution)
- 97. ● Introduction ✔ ● High Availability Levels ○ Level 0: Single Instance ✔ ○ Level 1: Multi Instance ✔ ○ Level 2: Multi Zone ✔ ○ Level 3: Multi Region ✔ ○ Level 4: Multi Cloud ○ Level 5: Hybrid Cloud ● Summary Agenda
- 98. Level 4: Multi Cloud
- 99. If one cloud provider is down, the system is still available
- 100. app data Level 4: Multi Cloud global load balancer load balancer app app data data replication app data load balancer app app data data
- 101. data Level 4: Multi Cloud data data replication data data data cloud provider 1 cloud provider 2 Assumptions: ● Machines in at least 2 cloud providers ● Network may be slow and unreliable ● Machines may be in different geo regions For example: ● EC2 Instances in eu-central-1 and GCP VM Instances in us-west1-a
- 103. Level 4: Multi Cloud ● No Cloud-specific tools ● No VPC Peering across Cloud providers ○ Latency ○ Security ● Cost
- 104. Is High Availability the only reason for Multi-Cloud?
- 105. Reasons for Multi-Cloud ● High Availability / Disaster Recovery ● Avoiding vendor lock-in ● Cloud cost optimization ● Risk Mitigation ● Low latency ● Data Protection / Regulations / Compliance ● Best-Fit Technology (Cloud-specific portfolios)
- 106. What does "Level 4: Multi Cloud" mean to You? No vendor lock-in! Cloud cost negotiations Low latency if accessed from multi-cloud Complex setup! No Cloud toolkit (e.g. AWS SQS)
- 107. ● Introduction ✔ ● High Availability Levels ○ Level 0: Single Instance ✔ ○ Level 1: Multi Instance ✔ ○ Level 2: Multi Zone ✔ ○ Level 3: Multi Region ✔ ○ Level 4: Multi Cloud ✔ ○ Level 5: Hybrid Cloud ● Summary Agenda
- 108. Level 5: Hybrid Cloud
- 109. If all cloud providers are down, the system is still available
- 111. Is it possible that all cloud providers are down?
- 112. No!
- 113. Reasons for Hybrid Cloud ● Data requirements / regulations ● Data security ● Moving to Cloud ● Cost reduction ● All mentioned already in Multi-Cloud
- 114. Level 5: Hybrid Cloud global load balancer On-Premises
- 116. What does "Level 5: Hybrid Cloud" mean to You? No Cloud lock-in! Low latency if accessed from custom networks Super complex setup! Usually extra layer needed (e.g. Kubernetes, OpenShift) Costs a fortune!
- 117. ● Introduction ✔ ● High Availability Levels ○ Level 0: Single Instance ✔ ○ Level 1: Multi Instance ✔ ○ Level 2: Multi Zone ✔ ○ Level 3: Multi Region ✔ ○ Level 4: Multi Cloud ✔ ○ Level 5: Hybrid Cloud ✔ ● Summary Agenda
- 118. Summary
- 119. Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
- 120. distributed system (data replication) Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
- 121. distributed system (data replication) Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud zone aware
- 122. distributed system (data replication) Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud long distance (geo- replication) zone aware
- 123. distributed system (data replication) Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud no cloud tools long distance (geo- replication) zone aware
- 124. distributed system (data replication) Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud no cloud tools own infrastructure long distance (geo- replication) zone aware
- 125. Which High Availability Level is for me?
- 126. Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
- 127. Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
- 128. Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
- 129. Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
- 130. Single Instance Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud