MariaDB 10 and what's new with the project
7 likes4,862 views
This document provides an overview of MariaDB 10.0 and what's new compared to previous versions. Some of the key highlights include backporting features from MySQL 5.6 such as InnoDB, Performance Schema, and online ALTER TABLE. MariaDB 10.0 also includes new features like multi-source replication, persistent statistics, and integration with NoSQL databases. The goals are to have feature parity with MySQL 5.6 and provide an open source alternative to Oracle's MySQL with more active development.
MariaDB 10 and what's new with the project
- 1. MariaDB 10.0 & What’s New with The Project Michael “Monty” Widenius Colin Charles
- 2. Who are we • Michael “Monty” Widenius – Creator of MySQL – Project leader at MariaDB, founder of Monty Program Ab – CTO, MariaDB Foundation – [email protected] | twitter: @montywi • Colin Charles – Chief Evangelist, MariaDB @ Monty Program Ab (soon SkySQL) – Formerly of MySQL – [email protected] | twitter: @bytebot 2
- 3. Agenda • MariaDB 10.0 series introduction • Where we stand towards MySQL 5.6 • Features, including MariaDB Galera Cluster 3
- 4. What’s MariaDB? • 38 months: major releases (5.1, 5.2, 5.3, 5.5, Galera Cluster) and 10.0 series • Still community developed, feature enhanced & backward compatible • maria-captains: 50% MP, 50% community including Sphinxsearch, SkySQL, Taobao, Percona, Codership, Facebook, & more... • GPLv2 server 4
- 5. Goal hasn’t changed • uninstall mysql-server • install mariadb-server • it should “just work!” (& be compatible) • (or more recently, you just apt-get/yum install mariadb, and it removes mysql) • MySQL compatible: – data on disk & on the wire is same – same file names, sockets, port 5
- 6. Why MariaDB 10.0? • The 5.5 merge took about a year (!) • In MariaDB 5.5, we have over 1.5 million lines of extra code ~61MB diff • MySQL 5.6 refactored with huge losses in commit history • We’re not a patch set against MySQL – MariaDB clearly does not depend on MySQL for future development 6
- 7. MariaDB 10.0: In a nutshell... • Built on MariaDB 5.5 • Backported features from MySQL 5.6 • Multiple new features 7
- 8. First, let’s start with some history • MariaDB 5.1 (MySQL-5.1 base) – Table elimination, new storage engines, cleanup, better tests, pool of threads • MariaDB 5.2 (MariaDB-5.1 base) – Virtual columns, extended user statistics, segmented MyISAM keycache • MariaDB 5.3 (MariaDB-5.2 base) – Biggest changes to optimizer (faster subqueries, joins, etc.) – Microsecond precision, faster HANDLER (HANDLER RED 50% faster w/530,000 qps), dynamic columns – Better replication (group commit, etc.) – HandlerSocket 8
- 9. MariaDB 5.5 (MariaDB 5.3 + MySQL 5.5) • Opensource, more efficient threadpool • Non-blocking client library • New LIMIT ROWS EXAMINED option • Extended keys for XtraDB/InnoDB • New SphinxSE • Lots of security fixes, new status variables, etc. 9
- 10. MariaDB 10.0 10 Date Version Status 12 Nov 2012 10.0.0 Alpha 6 Feb 2013 10.0.1 Alpha 24 Apr 2013 10.0.2 Alpha 10.0.3 Beta
- 11. Backported features • InnoDB including TRANSACTION READ ONLY • PERFORMANCE_SCHEMA • Online ALTER TABLE (in-progress) • Optimizer – ORDER BY...LIMIT optimization (shows only few rows of a result set) • Re-implemented: – Error messages (w/system error string) – CURRENT_TIMESTAMP as DEFAULT for DATETIME columns – Global Transaction ID (10.0.2 - MDEV-26) – Parallel replication • New (optimizer) – EXISTS-TO-IN optimization 11
- 12. Only in MariaDB 10.0: Multi-source replication • Work from Taobao • Many users partition data across many masters... now you can replicate many masters to a single slave • Great for analytical queries, complete backups, etc. 12
- 13. Only in MariaDB 10.0: • SHOW EXPLAIN for <thread_id> gets the query plan for a running statement • Faster ALTER TABLE with unique keys for Aria & MyISAM • Per-thread memory usage (Taobao) – INFORMATION_SCHEMA.PROCESSLIST has MEMORY_USAGE & EXAMINED_ROWS now – SHOW STATUS has memory usage too 13
- 14. Only in MariaDB 10.0: CassandraSE, LevelDB support • MariaDB as a “data platform” • Integration with NoSQL/Big Data DB, Apache Cassandra cluster, seen as a storage engine to MariaDB • Combine (join) data between Cassandra & MariaDB & Oracle (via CONNECT) • Write to Cassandra from SQL (SELECT, INSERT, UPDATE, DELETE) • LevelDB (in-progress) 14
- 15. Only in MariaDB 10.0: Engine independent persistent statistics • InnoDB has persistent statistics in MySQL 5.6; we have an engine-independent version • These statistics aren’t limited by the SE API, and are used by query optimizer to choose best execution plan for each statement • Statistics collected for non-indexed columns too (unlike InnoDB’s) 15
- 16. Dynamic columns • Store a different set of columns for every row in the table (kinda NoSQL- like) • Database interoperability is now a major feature; CassandraSE makes uses of this • Can now request a row in JSON format • And yes, you can name columns finally ;) 16
- 17. Segmented MyISAM keycache • Solves major read bottlenecks for MyISAM • MyISAM usage with many readers 17
- 18. BKA speedups select max(l_extendedprice) from orders, lineitem where o_orderdate between $DATE1 and $DATE2 and l_orderkey=o_orderkey 18
- 19. Group commit in the binary log 19 Source: Mark Callaghan, https:// www.facebook.com/note.php? note_id=10150261692455933
- 20. Group commit even faster in 10.0 • Slow fsync (removing checkpoint per commit saving 1-in-3 fsyncs) • Fast fsync (reduced stall on LOCK_log gives speedup) 20
- 21. Group commit benchmark against 5.6 • MySQL 5.6 removes checkpoint per commit • More complex thread scheduling, less grouping of commits • Does not remove stall on LOCK_log, less grouping of commits 21
- 22. Threadpool 5.5 vs 5.1 22
- 23. MariaDB 10.0.2 • Support for atomic writes on FusionIO DirectFS • Optimizer collects & can use histogram- based statistics for non-indexed columns • Better table discovery, so FederatedX has assisted discovery, Sequence engine (creates ascending/descending sequences, useful in joins) 23
- 24. Roadmap • MariaDB is already a superset of features in MySQL • Merge is in 2 steps – 10.0.x – 10.1.x • Plan is to have all important features of MySQL 5.6 by the time MariaDB 10.1.x is released as stable – For all practical purposes MariaDB 10.1.x will be a drop in replacement to MySQL 5.6 24
- 25. LGPL client libraries • LGPL client libraries for C & Java • Works with MariaDB/MySQL/Percona Server • Developed by Monty Program Ab & SkySQL Ab, released as LGPL 29 Nov 2012 • ODBC is in the works, sponsors are welcome! 25
- 26. MariaDB Galera Cluster • MariaDB Galera Cluster is made for today’s cloud based environments. It is fully read-write scalable, comes with synchronous replication, allows multi- master topologies, and guarantees no lag or lost transactions. • Currently 5.5-based, we see this as important for 10.0 as well (merges will happen closer to GA) 26
- 27. TokuDB • It’s now opensource • Official MariaDB binaries out within days that includes TokuDB • Tests & builds of TokuDB on multiple platforms (think greater distribution) 27
- 28. CONNECT storage engine • Made by Olivier Bertrand • Read, write & update files in different storage formats: – .DBF (dBASE format) – .CSV – .INI – XML – ODBC 28
- 29. Benchmarks • “Lies, damned lies, and statistics” - Mark Twain – http://blog.mariadb.org/sysbench-oltp- mysql-5-6-vs-mariadb-10-0/ – http://dimitrik.free.fr/blog/archives/ 2013/02/mysql-performance-mysql-56- vs-mysql-55-vs-mariadb-55.html • Yes, we’ve gotten Oracle to notice MariaDB :-) 29
- 30. Continued commitments • Security – Since about a year now, we’re the go-to people for security - good track record • We don’t like regressions – http://www.skysql.com/blogs/hartmut/nasty-innodb- regression-mysql-5525 – http://www.skysql.com/blogs/kolbe/heads-no-more- query-cache-partitioned-tables-mysql-5523 • We care about backward compatibility & introduce features carefully • XtraDB innodb_adaptive_checkpoint=none|reflex| estimate|keep_average (no more reflex...) 30
- 31. Community involvement • Many features since MariaDB 5.2 have come from the community or are sponsored features (5.3, 5.5) • Knowledgebase has 2,600+ articles in English • Yearly downloads of half a million (more users from mirrors + distributions) • Active mailing lists (stats on KB) • Google Summer of Code 2013 • MariaDB User Groups 31
- 32. MariaDB is gaining popularity • Wikipedia (English, German), Wikidata running MariaDB 5.5 • Fedora, OpenSUSE shipping MariaDB as a default • Slackware, Chakra Linux, ArchLinux have followed suit • Many success stories at KB: Limelight Networks, Nimbuzz, Paybox, FictionPress, OLX, SlashGear, Web of Trust, SpamExperts, etc. 32
- 33. Monty Program Ab + SkySQL Ab join forces • Monty Program Ab & SkySQL Ab have entered into a merger agreement • Resulting company has best available development & support people in the MySQL ecosystem • MariaDB Foundation ensures that there never has to be a new fork 33
- 34. MariaDB Foundation • Foundation is driver of MariaDB project • Custodian of code, guardian of community • Foundation can never be controlled by single entity or person • Designed to be self-sustaining 34
- 35. MariaDB Foundation Goals • Increase adoption of MariaDB • Ensure sustainable high-quality efforts to build, test and distribute MariaDB • Ensure that community patches are reviewed and adopted • Guarantee a community voice • Keep MariaDB compatible with MySQL • Maintain mariadb.org 35
- 36. MariaDB Foundation • More founders & sponsors are welcome • Michael Widenius, CTO: [email protected] • Andrew Katz, Legal Counsel: [email protected] • Simon Phipps, CEO: [email protected] • Jeremy Zawodny (Craigslist), Mike Milinkovich (Eclipse Foundation) on Board 36
- 37. Coming up next • Merge SPIDER storage engine • Column level encryption via MyDiamo from Penta Security • New optimizations for Fusion-IO • You can shape the roadmap 37
- 38. Well supported • Everyone supports MariaDB from a support standpoint with the exception of Oracle • Visit the Expo Hall: SkySQL, Percona, AccelerationDB, etc. • Jelastic has it as PaaS too 38
- 39. FAQ • Can I replicate from MySQL 5.6 to MariaDB 10.x? – Yes • Can I replicate from MariaDB 10.x to MariaDB 5.5? – Yes • Can I replicate from MariaDB 10.x to MySQL 5.5/5.6? – No 39
- 40. FAQ on tools • SELECT VERSION() returns 10.0.1- MariaDB • Version string in handshake packet will be 5.5.30-mysql-10.0.2-MariaDB (mysql#68187, MDEV-4088) • Tools should start recognising MariaDB for additional feature-set (mytop, HeidiSQL, etc.) 40
- 41. Conclusion • MariaDB has 30 man years more of development on-top of MySQL 5.5 with many different features • MariaDB is maintained by the people that created original MySQL code • We have 1 million man years of QA (by using the community for QA) • MariaDB is binary compatible with MySQL • MariaDB is opensource, has more features, and have no commercial extensions 41
- 42. Resources • bugs: mariadb.org/jira • [email protected] • [email protected] • fb.com/MariaDB.dbms • twitter: @mariadb • google plus: +MariaDB • #maria on irc.freenode.net • http://kb.askmonty.org/ 42
- 43. Q&A? Thanks Michael “Monty” Widenius [email protected] | @montywi Colin Charles [email protected] | @bytebot