Scale Your Load Balancer from 0 to 1 million TPS on Azure

Ludicrous Scale on Azure:
Turn Your Load Balancer
Up from 0 to 1 million TPS
Ranga Rajagopalan, CTO/Co-Founder, Avi Networks

Agenda
1. Why ludicrous speed ?
2. Why hardware or virtual
ADCs can’t achieve the scale?
3. Modern load balancing
4. Achieve the scale on Azure
5. Demo

Poll: Do you have applications
that need elastic requirements?
Peak workload vs. Quiet time

Use Cases for Elastic Scale
● HTTPS usage reaching > 80%
Source: Google transparency
report @
https://transparencyreport.google.
com/https/overview
● Black Friday ?
220 % of regular traffic
Source: eCommerce stats @
https://econsultancy.com/blog/685
87-black-friday-cyber-monday-
2016-ecommerce-stats-bonanza
● Product launch ?
● Special Event ?
● Software update ?
● DoS attack ?

Copyright © 2018 Avi Networks
Delivering applications reliably
and securely both on-premises
and in the cloud
Adobe Accelerates Cloud Initiatives with Microsoft Azure
By reliably and securely
delivering our applications,
Avi Networks has become a
key partner for our adoption
of Azure.
Brandon Pulsipher
Vice President, Product Technical Operations
Migrate the Adobe Experience Cloud to the
Azure with enterprise-grade support
Deliver thousands of SSL transactions per
second, and elastic scale of load balancers
and app servers based on real-time traffic
Improve operations with app insights and
central management of all load balancers

Autoscaling to Improve Application Responsiveness
Elasticity and agility
• Auto scale LBs based on load
• Elastic to save resources
• Self-healing to handle any failure
• Adaptive application scale out
Internet

Legacy
• 2RU Appliance – 12K TPS
(PFS/ECC)
• For 1 Million TPS
– 84*2 for HA = 166
Appliances
– 332 RU: ~8+ Racks
• Power
– Active: 800W max,
Passive: 400W max
– Total: ~100 KW
Scale elastically: 1 Million TPS Comparison
Avi: 1/10th Space, 1/10th the Cost, 1/2 the Power
Avi on Commodity x86
• 1RU Dell Server – 40K TPS
(PFS/ECC)
• For 1 Million TPS
– 25*1.5 (N-way HA) = 38
Servers
– 38 RU: ~1 Rack
• Power
– Active: 800W Max, Passive:
400W Max
– Total: ~45KW

Challenges in scaling SSL performance
Performance
Time
(CPU, PPS Optimizations in Intel DPDK / Hypervisors)
30% improvement in vSwitch
performance in last 12
months
• Performance factors:
– CPU – SSL TPS, SSL Throughput
– PPS / IO – L4/L7 Throughput
• Vertical Scaling
– Faster CPUs
– More NICs
– Multi-vCPU LBs
– High Speed Pkt processing (DPDK)
improvements
• Horizontal Scaling
– Multiple LBs
– Traffic distribution across LBs
– Optimizations in SSL/TLS libraries
– KVM/ESXi performance optimizations

Avi Networks Overview
Software load balancer, web application firewall, and service mesh, with built-in predictive analytics
UNIVERSAL SOLUTION
For Modern and
Traditional Use Cases
Across Any Cloud
G2000 CUSTOMERS
20% of F50
10% of F500
Across Industries
EXPONENTIAL GROWTH
3x YoY Growth
5x Increase in Customers
Strategic Partnerships (Cisco)

Poll: Do you have initiatives to
run some applications in Azure?

Modern Distributed Architecture
Software-defined and elastic
CONTROLLER
SERVICE ENGINE
SEPARATE CONTROL
& DATA PLANE
ELASTICITY

BARE METAL VIRTUALIZED CONTAINERSON PREMISES PUBLIC CLOUDVIRTUALIZED CONTAINERS
Across Any Environment
Consistent experience across clouds
CONTROLLER
SERVICE ENGINE
MULTI-CLOUD

Analytics Drives Intelligence
SERVICE ENGINE
INTELLIGENCE
• Connection log analytics
• Security insights: DDoS
• App performance metrics
• End user experience
CONTROLLER
END TO END TIMING
TOTAL RESPONSE TIME
End User
Client
RTT
Server
RTT
App
Response
Load
Balancer
Server App

Automation Operationalizes Digital Transformation
Built-in ecosystem integration
CONTROLLER
SERVICE ENGINE
AUTOMATION
• 100% REST APIs
• Full lifecycle management
• On-demand autoscaling

Elastic Services Fabric for AWS, Azure, GCP
Public cloud agility with enterprise class capabilities
ENTERPRISE-CLASS FEATURES
Full Featured L4-L7 ELB -
persistence, performance
Distributed Web App Firewall
ELB-LIKE AUTOMATION
1-click app provisioning
API integration for DNS, IPAM,
AZs, Autoscaling
APPLICATION ANALYTICS
Real time app performance
monitoring
Connection log analytics
MULTI-CLOUD
Consistency across public cloud &
on-prem
Central control and monitoring

Record Performance – On Commodity Servers or Cloud
•32 cores
•80,000 SSL TPS
Performance scales with
cores (Moore’s Law)
•2 core
•5,000 SSL
TPS
Avi Performance
• 4 Tbps
• 12M SSL TPS
•2x 1-core SEs
•5,000 SSL TPS Fabric performance
scales with Engines
Centralized API,
Mgmt, Monitoring
ECC- 48K, RSA 80K
•1 core
•2,500 SSL TPS
VIPRION 4800, ~1M TPS
Single Fabric - Central Management and Monitoring

Capacity Scaling – Flexibility to Match the Use Case
LargerSEs(MoreCores/Mem)
More SEs (Can Autoscale)
Maximum
Flexibility
More Service Engines (LBs) or Larger SEs or Both

Avi Vantage Azure Integration

1. Client request reaches
VS IP hosted on ALB
2. ALB routes request to
one of its pool members
(Avi SEs)
3. Avi SE forwards request
to one of its pool
members (application
servers)
Traffic Flow for Application

4. Application server
responds to the Avi SE
where the request
originated
5. Avi SE sends the
response directly to client,
bypassing ALB on the
return path

6. Simply add more Service
Engines as traffic scales up

60 Avi ServiceEngines -
F8s_v2 (8 core)
Test Setup
120 clients - D8s_v3
Time to scale to ludicrous
speed - 15 min
Cost - $200

Thank You
@AviNetworks
IDC REPORTCUSTOM COST SAVINGS AND ROI REPORT DEEP DIVE
Next Steps
1. Azure Test Drive
1. Performance White Paper
1. Schedule a demo

Scale Your Load Balancer from 0 to 1 million TPS on Azure

More Related Content

Similar to Scale Your Load Balancer from 0 to 1 million TPS on Azure (20)

More from Avi Networks (20)

Recently uploaded (20)

Scale Your Load Balancer from 0 to 1 million TPS on Azure