ScaleBase Webinar: Strategies for scaling MySQL

Strategies for scaling MySQL

Matthew Aslett, 451 Research
Doron Levari & Paul Campaniello, ScaleBase

© 2013 by The 451 Group. All rights reserved

Agenda

1. 451 Research – The MySQL SPRAIN

2. ScaleBase - Scale-Out your MySQL Database

3. Customer Case Studies

4. Q & A
(please type questions directly into the GoToWebinar side panel)


Company Overview

 One company with 3 operating  200+ staff
divisions  1,300+ client organizations:
 Syndicated research, advisory, enterprises, vendors, service
professional services, datacenter providers, and investment firms
certification, and events  Organic and growth through
 Global focus acquisition


 Matthew Aslett
• Research Manager, Data Management and Analytics
 matthew.aslett@451research.com
 www.twitter.com/maslett

 Responsible for data management
and analytics research agenda

 Focus on operational and analytic
databases, including NoSQL,
NewSQL, and Hadoop

 With 451 Research since 2007


In a nutshell
 MySQL became the default database of choice for Web applications

 Success breeds complexity:
• Greater scalability demands
• Predictable performance problems

 New requirements:
• Proliferation of cloud
• Geo-distributed data

 Multiple strategies for dealing with scalability issues
• Each with their plusses and minuses

 The emergence of new vendors aimed at masking the complexity


Previously on ScaleBase/451 webinars…
 The traditional relational database has been stretched beyond its
normal capacity by the needs of high-volume, highly distributed or
highly complex applications.

 There are workarounds – such as DIY sharding – but manual,
homegrown efforts can result in database administrators being
stretched beyond their normal capacity in terms of managing
complexity.

 Scalability
 Performance
 Relaxed consistency Increased willingness to look towards
 Agility emerging alternatives
 Intricacy
 Necessity


Focus on MySQL
 MySQL filled a gap in the market:
• Low cost and freely available with source code
• Better suited to emerging application workloads

 MySQL benefits
• Lightweight architecture
• Fast read capabilities
• Ease of use

 MySQL primarily adopted for:
• in-house-developed applications
• single-function applications
• customer-facing Web applications
• development-and-test environments


Early adoption drivers: 2008 survey


Rapid increase in adoption: 2012 database survey

90.0
80.0
70.0
60.0
50.0
40.0
30.0
20.0
10.0
0.0

Survey conducted: Jan/Feb 2012
Sample: 205


Scalability, and other challenges
 As usage of MySQL has grown, so has the usage of applications that
depend on MySQL

 This has highlighted a number of challenges
• Scalability of master-slave architecture
• Performance
• Predictability of performance at scale
• Manageability of increasing database/app sprawl

 External factors driving greater complexity:
• Distributed computing architectures
• Proliferation of cloud, and elasticity requirements
• Geo-distributed application requirements


The road to SPRAIN: approaches to scaling MySQL
 Example:
 Photo sharing website
users  Started with single MySQL
database

database


 Example:
users users users  Started with single MySQL
database
 Growth through acquisition and
customer adoption

database


 Example:
database
customer adoption
 Expanded its MySQL footprint
every two years

master

slave slave


 Example:
database
customer adoption
every two years

 Larger master
master
• Pros: Add more CPUs, disk and
memory to cope with increased
data
• Cons: Doesn’t necessarily help
slave slave with performance, temporary fix


 Example:
database
customer adoption
every two years

 More slaves
master
• Pros: Good for scaling our read
queries
• Cons: Asynchronous replication
can lead to write slave lag
slave slave slave slave
 By 2009 had 16 slaves and was
beginning to hit the wall


 Example:
database
customer adoption
every two years

 Considered sharding:
database database database
• Pros: increases scalability to serve
high-traffic websites
• Cons: Horizontal (across masters),
required application re-write
• Cons: Vertical (across the slaves),
considered a temporary fix


 Example:
database
customer adoption
every two years

 Read/write splitting
master • Pros: Better utilization of available
resources
• Cons: Asynchronous replication
can lead to slave lag
slave slave slave slave


 Example:
database
customer adoption
memcached  Expanded its MySQL footprint
every two years

 Memcached
master • Pros: Improves read performance
for frequently accessed data
• Cons: Not persistent, doesn’t solve
write performance issues
slave slave
• Looked instead for alternatives


Alternatives
 NewSQL – retaining support for ACID transactions and SQL

• New databases

• Advanced storage engines, particularly for MySQL

• Advanced clustering/shard management approaches


Comparing NewSQL approaches
 New databases
• Pros: Designed specifically to support distributed architecture
• Cons: May lack compatibility with existing applications

 Advanced storage engines, particularly for MySQL
• Pros: Retain familiarity with with MySQL skills, tools
• Cons: Re-architecting from the inside out.

 Advanced clustering/shard management approaches
• Pros: Retain application compatibility while adding scalability
• Cons: An extra layer of complexity?

 Issues to consider:
• Does it require a forklift move of your entire application ecosystem
• Can you continue to leverage your existing MySQL skill set?
• Is there a risk for your data, e.g. memory reliability?


Spotlight on ScaleBase
 Creates a shared nothing architecture from standard databases

 Elastic load balancing for MySQL (other databases on the roadmap)

 Scale Out via read/write splitting or automatic data distribution

 Data Traffic Manager serves as a proxy between the apps and DB

 Provides a single point for administering the shared nothing cluster
(for performance, HA, change management)

 And the ability to add scalability without the need to migrate to a
new database architecture or make any changes to existing apps.


Conclusion
 Scalability isn’t simply a matter of a theoretical number of nodes

 It’s also about the ability to deliver predictable performance

 And to do so without adding management complexity

 Distributed computing, proliferation of cloud, and geo-distributed
applications are adding to the complexity

 MySQL has never been under so much strain

 The combination of advanced clustering/load balancing and
management technologies provides a potential solution


Thank you.
matthew.aslett@451research.com
@maslett

Photo credit: swisscan on Flickr http://bit.ly/HCZCRQ


How to Scale Out your MySQL Database

Who We Are

Presenters: Paul Campaniello,
VP of Global Marketing
25 year technology veteran with
marketing experience at Mendix,
Lumigent, Savantis and Precise.

Doron Levari, Founder & CTO
A technologist and long-time
veteran of the database industry.
Prior to founding ScaleBase, Doron
was CEO to Aluna.

25

ScaleBase Mission

“MySQL can cost-effectively
scale to support an infinite
number of application users,
with NO disruption to the existing infrastructure”

26

ScaleBase: Scale-Out MySQL - Fix the SPRAIN

Don’t throw out the baby with the bath water!

• Keep your MySQL - keep your InnoDB
• Ecosystem compatibility, preserve skills
• 100% application compatibility
• Smoother migration, no down-time, no forklift
• Your data is safe!
• No “in-memory” magic
• No “in-memory” size limit

27

Single-Box Scalability Improvements in MySQL 5.6

Partitioning Improvements Replication Improvements
– Explicit Partition Selection: – Optimizations to Row-Based
SELECT * FROM employees Replication
PARTITION (p0, p2);
– Multi-Threaded Slaves
– Import / Export for Partitioned Tables:
Bring a new data set into a partitioned – Improvements to Data Integrity
table, or export a partition to manage it
as a regular table ALTER TABLE e – Crash-Safe Slaves
EXCHANGE PARTITION p0 WITH – Replication Checksums
TABLE e2;

SCALABILITY issues remain due to the limitations of a single box:
To ensure ACID, you still face limitations with:
- Memory management - Thread management
- Semaphores - Locking
- Recovery tasks
No new functionality for sharing workloads across multiple boxes

28
http://dev.mysql.co/tech-resources/articles/whats-new-in-mysql-5.6.html

Two Approaches to Scale-Out MySQL

Scale Out
Read

Write

R/W Splitting

Replication

Data
Distribution

30

Read/Write Splitting: Application Experience
Current: App sees master plus multiple slaves

Read Replicas

With ScaleBase: App sees ONE database

31

Read/Write Splitting: Application Experience
Current: App sees master plus multiple slaves
Problems:
• Writing code, maintaining code
• Maintaining DB ops and IPs in the app
• Master DB is too occupied
• Slaves are not balanced Read Replicas
• Replication could break
• Stale data may be read

With ScaleBase: App sees ONE database
ScaleBase Benefits:
• Reads/Writes throughput increases
• Better resource utilization/load balancing
• Improved data consistency/transaction isolation with:
• Replication state lag awareness
• Read stickiness after writes
• Handling of edge cases

32

Data Distribution: Application Experience

Current: App is customized to support multiple shards

With ScaleBase: App sees ONE database…

…and does not require any customization
33

Data Distribution: Application Experience

Current: App is customized to support multiple shards
Problems:
• Major app rewrite, maintaining code
• Maintaining DB ops and IPs in the app
• Administration/3rd party tools are blind
• Non-optimized data distribution policy

With ScaleBase: App sees ONE database…
ScaleBase Benefits:
• Unlimited scalability while improving performance
• Real time elasticity
• Verified data consistency
• Real time monitoring, analysis
• Carefully analyze distribution policy

…and does not require any customization
34

Typical ScaleBase Data Traffic Manager Deployment

Database A Replica A

Application
Servers

Database B Replica B

BI

Unlimited Scale
Database C Replica C

Fault Tolerant
Management Architecture
Centrally Manage
Complex DB Database D Replica D

Environment
35

ScaleBase Features and Benefits

Feature Benefit
Automatic data distribution Scale data-, read-, write- intensive applications

 Expert System Analysis Optimization of data distribution policy

 Parallel query execution Best option for cross-db queries & maintenance commands
Support of sophisticated cross-db queries, including ORDER
 Query result aggregation
BY, GROUP BY, LIMIT, Aggregate functions…
Flexibility: no need to over-provision
 Online data redistribution
No downtime
Read/Write splitting Optimal for scaling of read-intensive applications
 Replication lag-based routing Improves data consistency and isolation
 Read stickiness after writes Ensure consistent and isolated database operation
100% compatible MySQL proxy Applications unmodified
Standard MySQL tools and interfaces
MySQL databases unmodified Data is safe within MySQL InnoDB/MyISAM/etc.
Data consistency verifier Data consistency validation

Real-time monitoring and alerts Simplify management, reduce TCO
36

Scale…without SPRAIN

Scale Your Database With Your Business
160000

140000

120000

100000
Throughput

80000 84000 Throughput (TPM)
Total DB Size (MB)
60000 60000 # Connections
48000
40000
36000
24000 2500
20000 2000
12000 1500 1500
6000 1000
0 500 500
1 2 4 6 8 10 14
Number of Databases
37

ScaleBase Delivers Scalability

Real-time DB
Traffic Visibility
Scale
to Automated
Unlimited Failover

Users
…all with NO
App Rewrites

38

Detailed Scale-Out Case Studies

Mozilla AppDynamics Large Chip Co Solar Edge
• New Product/ • Next gen APM • Scalability • Next Gen
Next Gen App/ company • Multiple Apps Monitoring App
AppStore • Scalability for the • Multiple growing • Massive Scale
• Scalability Netflix users • Monitors real
• Geo-clustering implementation • Availability time data from
• MySQL DB thousands of
distributed
systems

39

Economic Impact

Customer Experience Internal Development
– Faster, more responsive apps, – Developers focus on app, not
even during peak loads writing complex sharding code
– Always on: 24/7 – Eliminate development and
– Improve customer acquisition ongoing maintenance costs
(reduce churn) and brand
Centralized Management
Scale – Troubleshoot problems and
– Use commodity hardware vs. scale- investigate incidents in seconds
up hardware with real-time monitoring
– Cloud ready – deploys easily High Availability
Non-MySQL Options – Eliminate database downtime
– Protect existing investment with no with automated fail-over
ecosystem forklift – Improve business continuity
– Leverage existing MySQL skill set

40

ScaleBase Deployment

Environments Path to Scale-Out:
– Public Cloud 1. Data Distribution
Policy Analysis
– AWS, Rackspace, any
2. Functional Test
– Private cloud
3. Load Test
– Hosted / on-premise
4. Production Migration
Databases Supported (safe, online)
– MySQL 5.1, 5.5, 5.6 (under
certification)
– AWS RDS MySQL 5.1, 5.5
– Maria DB 10.0 (under
certification)

41

Summary

• MySQL is vast and adoption is growing
• MySQL scalability is a significant problem (SPRAIN)
– App explosion (web & mobile), Big Data transactions…
• ScaleBase provides cost-effective Scale-Out solutions
– Seamlessly scale to an infinite number of users
– Improve DB visibility with real-time monitoring
– Enhance business continuity with
automated failover
– Expert analysis and simple deployment
– No application rewrites
– Leverage existing MySQL ecosystem/skills
42

Questions (please enter directly into the GTW side panel)

matt.aslett@451research.com paul.campaniello@scalebase.com

doron.levari@scalebase.com
@maslett
@scalebase
@451research
www.ScaleBase.com

www.451research.com 617.630.2800

43

ScaleBase Webinar: Strategies for scaling MySQL

More Related Content

What's hot (6)

Viewers also liked (6)

Similar to ScaleBase Webinar: Strategies for scaling MySQL (20)

More from ScaleBase (9)

ScaleBase Webinar: Strategies for scaling MySQL