Load Balancing Algorithms - Which one to choose?
- 7. 1. Random
- 9. Random algorithm: • Can you “truly randomly” distribute load? • It is simple to implement and scales well when you have a distributed load balancer setup. • Latency is not being accounted for. • Servers which are underperforming can have more requests being sent to them.
- 10. 2. Round Robin
- 11. 1 1 1
- 12. 2 2 2
- 13. 3 3 3
- 14. Round Robin algorithm • Guarantees same number of requests to all servers. • We again face the same latency problem. • Does not scale well for distributed Load balancers as the logic of where to start from needs to be statically hardcoded. • What if the backend servers have different configs?
- 15. 3. Weighted Round Robin
- 17. Weighted Round Robin algorithm • Servers can take load based on their H/W configs. • Can introduce another service which check for latency and update the weights dynamically. • The weights are present on the load balancer and not the servers, so if it is distributed, all need to be in sync. • Also called “Join shortest queue” algorithm.
- 19. Served Requests 1 1 2 23 3 456 7 4 5 6 7 1 3
- 20. Least Connections algorithm • Routes requests considering the current load on each servers. • In a distributed setup, the LBs should be in sync about the active connections on each server. • We can have race condition of all LBs sending to the same server if all requests come to the LBs at once.
- 21. 5. Least connections with randomness
- 22. Each LB, when receives a request chooses a set of random servers. Then, the least connection algorithm is applied to those servers and the backend server is chosen for the request.
- 23. Least connections with randomness • We are trying to solve the problem of race condition in Least connections algorithm by using this approach. • This works only in a distributed LB setup. • Adds one more layer of complexity to choose random set of servers. • The number of random servers that need to be chosen needs to be hardcoded in each LB. This number cannot be too close to the actual number of servers ? And cannot be too less as well?
- 24. • All the techniques that we have looked at so far assume that the application is stateless and any of the servers can serve requests. • Maintaining persistent connections is a much more difficult task and shall be discussed in the future videos.
- 25. • Join idle queue algorithm – Link • Source IP hash algorithm, based on consistent hashing – Link1, Link2 • This is used for maintaining persistent connections. • Global Server Load Balancing – Link Further reading:
- 26. References • Comparison between basic LB algorithms: Link • Tyler McMullen - Load Balancing is Impossible: Link • “Balls into Bins” — A Simple and Tight Analysis - Link • The Power of Two Choices in Randomized Load Balancing - Link