Next-gen Flash-based MySQL and NoSQL Solutions (Real World Case Studies of Extreme Performance, Scalability and Availability)
- 1. Next-gen Flash-based MySQL and NoSQL Solutions Real World Case Studies of Extreme Performance, Scalability, and Availability Darpan Dinker Vice President, Database Technologies Partnership Power > © 2010 Schooner Information Technology. All rights reserved.
- 2. MySQL Conference & Expo, 2010 Agenda • Datacenter trends and challenges • Next-gen flash-based solutions • MySQL case studies • NoSQL case studies Partnership Power > © 2010 Schooner Information Technology. All rights reserved. 2
- 3. MySQL Conference & Too Much Rack, Power, Pipe, and Expo, 2010 Complexity For every 100 U.S. data- The number of units of energy Datacenter installed servers From 2003 to 2008 piped into a data centers use equipment is more energy in the U.S. will the data size of the center, only only utilized increase from 2.2 average web page has three are used than the entire 6% to 10%. nation of million in 2007 to more than tripled. for actual Sweden. - William Forrest 6.8 million in 2010. –websiteoptimization.com computing. Forbes - Frost & Sullivan - EE Times - U.S. Department of Energy © 2010 Schooner Information Technology. All rights reserved. 3
- 4. MySQL Conference & Expo, 2010 Typical Datacenter Deployment Data Access Tier Web/App Tier Caching Tier Database Tier PHP, Perl, Memcached MySQL Ruby, Java Slaves POOL 1 Master POOL 2 Ensure Scale End Minimize User Quality of to Meet Costs Service NoSQL Tier Demand Key-Value Store, Document Store, etc. POOL3 4 © 2010 Schooner Information Technology. All rights reserved. 4
- 5. MySQL Conference & Expo, 2010 Key Challenges Pain of sharding and re-sharding Scale Complexity of adding new servers to Meet Demand Headache of managing server sprawl Poor response time and availability Ensure Trade-offs in consistency models Quality of Service Complex, defensive app development Too much underutilized hardware Wasted power, pipe, and cooling Minimize Costs Integration & management burden © 2009 Schooner Information Technology. Confidential 5
- 6. MySQL Conference & Vision: Tightly Coupled, Scalable MySQL Expo, 2010 and NoSQL Building Blocks Integrated, optimized, scalable MySQL & NoSQL solutions: Effectively leverage flash memory, multi-core processors, high- speed networking, scalable data access software Incorporate highly optimized, balanced hardware platform, operating environment, integrated MySQL and NoSQL Provide efficient, higher level scalable building blocks Eliminate complex integration projects and leverage out of the box performance, scalability and availability Deliver enterprise class reliability © 2010 Schooner Information Technology. All rights reserved. 6
- 7. MySQL Conference & Tightly Coupled Software Expo, 2010 Architecture Networked Clients MySQL NoSQL Operating Environment Intelligent Caching Optimized Flash Multi-Core Transparent Hierarchy Memory Access Scalability HA/DR • Optimized buffer mgmt & • Highly parallel read and • Fine grain locking • Synchronous and async scan resistant algorithms write access replication • Scalable and concurrent • Write-through and write- • Intelligent flash wear data structures • Failure detection and back caching flexibility algorithms automated VIP failover • Optimized thread to core • Adaptive , fine-grained • Durability with high- allocation • Fast, incremental data memory management performance recovery • Efficient handling of • Efficient object meta- • High-performance, network interrupts • Incremental,/full online data for persistence integrated RAID backup and restore Optimized, Balanced Hardware Platform © 2010 Schooner Information Technology. All rights reserved. 7
- 8. MySQL Conference & Schooner MySQL and Expo, 2010 NoSQL Solutions • Integrated, turnkey appliance 1 The Schooner Appliance for MySQL EnterpriseTM with InnoDB – Integrated hardware and software – Enterprise-class support – 100% compatibility, fully certified • High performance + – 8x performance improvement – 1/8x the power and rack space – 50% lower TCO over 3 years 2 The Schooner Appliance for Memcached / NoSQL • High availability – Complete data & service availability – Transparent and fully integrated High Persistent 100% – 90% higher availability Capacity Cache + Key-Value Store + Memcapable © 2010 Schooner Information Technology. All rights reserved. 8
- 9. MySQL Conference & Expo, 2010 Schooner Powered Datacenter Data Access Tier Web/App Tier Caching Tier Caching Tier Database Tier Database Tier PHP, Perl, Memcached Memcached MySQL MySQL Ruby, Java Slaves POOL 1 Master POOL 2 Ensure Scale Minimize End NoSQL Tier User Quality of to Meet Costs Service Key-Value Store, NoSQL Tier Demand Document Store, etc. Key-Value Store, Document Store, etc. POOL3 9 © 2010 Schooner Information Technology. All rights reserved. 9
- 10. MySQL Conference & Schooner Appliance for Expo, 2010 MySQL Enterprise™ with InnoDB High Performance High Availability Turnkey Appliance • Highly parallel, optimized • Fully ACID-compliant with • Multi-instance consolidation flash memory access data durability on single appliance • Advanced buffer pool • Integrated replication and • Web-based GUI/CLI for caching algorithms automated failover centralized management • Multi-core scalability with fine • Integrated, high-performance • Integration with 3rd party grained locking backup and restore mgmt & monitoring tools • Delivered on proven IBM • RAID across SSDs and • 100% compatible and fully server with up to 1TB of flash HDDs certified by Sun/MySQL DBT2 Performance (TPM) Connection Scalability 75000 40000 Master Server TPS 60000 30000 45000 20000 30000 10000 15000 0 64 256 1024 4096 16384 0 Legacy MySQL Legacy MySQL with SSDs Schooner Appliance Schooner MySQL Measured DBT2 throughput at 1000 warehouses,32 connections, 8 SSDs/HDDS with RAID 5. All databases configured for durability and consistency. Legacy results were on MySQL version 5.1.44 (most commonly used today). Schooner MySQL Appliance results were on Schooner-optimized 5.1.44 © 2010 Schooner Information Technology. All rights reserved. 10
- 11. MySQL Conference & MySQL Expo, 2010 Consolidation and Cost Savings TCO: $832,000 TCO: $282,000 TCO SAVINGS: $550,000 3 Year TCO (2 TB MySQL) OpEx OpEx THE BOTTOM LINE CapEx: 32 Servers CapEx: 4 • Immediate capex savings Appliances • 66 % TCO savings ($550,000) Without Schooner With Schooner over 3 years 32 servers, 17.9 kW 4 Schooner • Power & space reductions appliances, 1.8 kW enable green datacenter © 2010 Schooner Information Technology. All rights reserved. 11
- 12. MySQL Conference & Expo, 2010 MySQL What Can I Do With It? Reduce sharding and Scale performance and consolidate slaves process queries faster Apps Masters Slaves Apps Schooner 70K TPM (DBT2) 20K Connections ½ TB or 1TB Flash … … Reduce planned and Eliminate integration and unplanned downtime optimization headaches • Software MySQL Replication + MMM Failover Install/Config • Hardware HA Tools • Support SSD RAID & SSD RAID & SSD RAID & Data Durability Data Durability Data Durability • Certified Tuning Services • Complete © 2009 Schooner Information Technology. Confidential 12
- 13. MySQL Conference & Expo, 2010 Case Study: Financial Services Scale Performance and Process Queries Faster High-performance, out-of-the-box 100% compatible "Our ad-hoc MySQL queries run at least five times faster after installing the Schooner Appliances. They deliver a huge performance benefit and are a breeze to install and manage.“ – Darryl Weatherspoon, VP of Eng at Xoom. © 2010 Schooner Information Technology. All rights reserved. 13
- 14. MySQL Conference & Expo, 2010 Case Study: Social Networking Consolidate Slaves Apps Masters Slaves Apps Masters Schooner "In our business, website performance and efficiency is key to the success of our web properties. The Schooner MySQL Appliances have significantly helped GuteFrage improve their overall website response time while at the same time allowing them to consolidate their database slaves onto a single Schooner appliance, dramatically reducing the time necessary for database administration.“ – Frank Penning, CTO of Holzbrinck Digital © 2010 Schooner Information Technology. All rights reserved. 14
- 15. MySQL Conference & Schooner Appliance for Memcached Expo, 2010 / NoSQL High Performance High Availability Easy Appliance • Highly parallel, optimized • Persistent key-value store • Dynamic containers for flash memory access mode and cache mode consolidation & multi-tenancy • Fast, efficient DRAM-to- • Transparent replication and • Web-based GUI/CLI for Flash caching algorithms automated failover centralized management • Multi-core scalability with • Non-disruptive, rolling • Integration with 3rd party parallel thread allocation upgrades mgmt & monitoring tools • Delivered on a proven IBM • RAID & high-performance • 100% compatible and fully server with a ½ TB of flash backup and restore memcapable compliant Key-Value Store Service Memcached Caching Service • 350GB capacity, 150K TPS • 150GB capacity, 250K TPS • Replication & Failover • Replication & Failover • Persistence, RAID, and Backup • 100% Memcapable • Memcached-based key-value API © 2010 Schooner Information Technology. All rights reserved. 15
- 16. MySQL Conference & Schooner Memcached / NoSQL: Expo, 2010 Capacity and Availability Cache Capacity per Node (GB) Reduction in Downtime (%) 600 100 500 80 400 60 300 40 200 100 20 0 0 Traditional Schooner Appliance Traditional Persistent Storage Replication Memcached Memcached Caching RAID & Failover © 2010 Schooner Information Technology. All rights reserved. 16
- 17. MySQL Conference & Memcached / NoSQL Consolidation Expo, 2010 and Cost Savings TCO: $610,000 TCO: $282,000 TCO SAVINGS: 3 Year TCO (2 TB Memcached) OpEx $328,000 OpEx THE BOTTOM LINE CapEx: 32 CapEx: 4 Servers Appliances • Immediate capex savings • 54% TCO savings ($328,000) Without Schooner With Schooner over 3 years 32 servers, 17.9 kW 4 Schooner • Power & space reductions appliances, 1.8 kW enable green datacenter © 2010 Schooner Information Technology. All rights reserved. 17
- 18. MySQL Conference & Expo, 2010 What Can I Do With It? Consolidate and reduce Reduce planned and server sprawl unplanned downtime Traditional Memcached Schooner Application Transparent POOL 1 Synchronous Replication + Automated Failover POOL 2 SSD RAID & SSD RAID & Persistence Persistence Scale cache capacity and Reduce application process requests faster development complexity Application Tier Application Tier Failure Detection Replication Group Delete Data Durability 250K TPS M$D API ½ TB M$D API Flash Failure Detection Replication Group Delete Data Durability Traditional Memcached Schooner © 2010 Schooner Information Technology. All rights reserved. 18
- 19. MySQL Conference & Expo, 2010 Case Study: Social Networking Consolidate and Reduce Server Sprawl Traditional Memcached POOL 1 Schooner POOL 2 Scaling with 12:1 Consolidation using Persistence, Multi-Get, Replication and Recovery Scaling the data tier is a common challenge, and Schooner is helping us do just that. Power is the big constraint right now, so anything we can do to reduce that footprint right now is helpful. From an administrative perspective, fewer machines is always better, from a monitoring and scripting standpoint, and it also means reductions in potential failures due to fewer boxes.“ – Saran Chari, CTO and Founder at Flixster © 2010 Schooner Information Technology. All rights reserved. 19
- 20. MySQL Conference & Expo, 2010 Case Study: Media Reduce Planned and Unplanned Downtime Application Transparent Synchronous Replication + Automated Failover SSD RAID & SSD RAID & Persistence Persistence 40 Million Users Address Books in Key Value Store with 8:1 Consolidation In the wonderful Schooner world, failovers go away. Schooner replication means that you’re sure that what you have on one node will also be on the other. Our developers don’t have to worry about cache coherency. They can plan on the data being available so they don’t have to program defensively. – Ethan Erchinger, Director of Ops at Plaxo © 2010 Schooner Information Technology. All rights reserved. 20
- 21. MySQL Conference & Expo, 2010 Market Segments Social Networking Business Analytics Gaming Cloud Media Telco Finance © 2010 Schooner Information Technology. All rights reserved. 21
- 22. MySQL Conference & Expo, 2010 The Schooner Advantage • Vastly Higher Performance and Scalability: 8x compared to traditional servers • Significantly Lower TCO: Replacing 8 traditional servers with 1 reduces TCO by more than 50% • Quick Deployment: Easy plug-and-play installation and configuration • Seamless Operation: 100% compatible with existing client applications and management tools • Higher Reliability: Delivers enterprise-class reliability by leveraging persistence, replication, and recovery software • Easy Management: Simple but powerful centralized management and reporting with integrated CLI and GUI • More Revenue: Powers new revenue-producing applications enabled by fast access to terabyte-scale data • World-Class Support: IBM provides 24/7/365, single-point-of-contact service and support for every Schooner appliance, worldwide © 2010 Schooner Information Technology. All rights reserved. 22
- 23. MySQL Conference & Expo, 2010 Q&A © 2010 Schooner Information Technology. All rights reserved. 23
- 24. MySQL Conference & Expo, 2010 Abstract Scale out data centers are realizing order of magnitude improvements in performance, scalability, and availability while reducing TCO with innovative MySQL and NoSQL solutions. These architectures tightly couple MySQL and NoSQL with flash memory, multi-core processors and high performance networking into balanced, highly available, scalable solutions . In this presentation, Darpan Dinker, Vice President of Database Technologies, Schooner Information Technology, will discuss: business and technology challenges tightly coupled MySQL and NoSQL scale out architectures case studies of large scale web site deployments in premier web 2.0, enterprise, and cloud companies, discussing their realized order of magnitude improvements in performance, scalability, downtime and TCO. © 2010 Schooner Information Technology. All rights reserved. 24