My sql 5.6_replwebinar_may12
2 likes•3,126 views
The document outlines enhancements in MySQL 5.6 replication focusing on performance, scalability, and data integrity, aimed at supporting high-scale web and cloud services. Key features include multi-threaded slaves, automated failover with global transaction identifiers, and optimized row-based replication for improved efficiency. Additionally, it emphasizes tools for replication management, monitoring, and reduced operational overhead to ensure robust database performance.
My sql 5.6_replwebinar_may12
- 1. MySQL 5.6 Replication: Supporting High Scale Web & Cloud Services Mat Keep Andrew Morgan MySQL Product Management MySQL Product Management [email protected] [email protected]
- 2. Session Agenda <Insert Picture Here> • Development Priorities • MySQL 5.6 Replication Enhancements • Resources to Get Started The image cannot be Copyright 2012 Oracle Corporation 2
- 3. The presentation is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. The image cannot be Copyright 2012 Oracle Corporation 3
- 4. Industry Leaders Rely on MySQL Replication is Key to Scale & HA Web & Enterprise OEM & ISVs Cloud The image cannot be Copyright 2012 Oracle Corporation 4
- 5. MySQL Database 5.6: A Better MySQL. • MySQL 5.6 builds on MySQL 5.5 by improving: • Performance and Scalability • Optimizer for better query execution times, diagnostics • Performance Schema for better instrumentation • InnoDB for better transactional throughput • NoSQL API for better flexibility • Replication largest set of enhancements ever released! • Available now under GPL • Development Milestone Release model enabling early evaluation The image cannot be
- 6. MySQL Replication: Development Priorities HIGH PERFORMANCE AUTOMATED FAILOVER & RECOVERY DATA INTEGRITY DEV/OPS AGILITY The image cannot be Copyright 2012 Oracle Corporation 6
- 7. Translating to User Needs PERFORMANCE • Improves read consistency from slaves • Reduces risk of data loss if master fails FAILOVER & RECOVERY • Minimize service interruption • Self-Healing DATA INTEGRITY • Correct, consistent & accessible DEV/OPS AGILITY • Reducing TCO • Quickly responding to business demand The image cannot be Copyright 2012 Oracle Corporation 7
- 8. MySQL 5.6 Replication PERFORMANCE • Multi-Threaded Slaves • Optimized Row-Based Replication FAILOVER & RECOVERY • Global Transaction Identifiers • Replication Failover & Admin Utilities • Crash Safe Slaves & Binlog DATA INTEGRITY • Replication Event Checksums DEV/OPS AGILITY • Replication Utilities • Time-Delayed Replication • Informational Log Events • Remote Binlog Backup • Server UUIDs The image cannot be Copyright 2012 Oracle Corporation 8
- 9. PERFORMANCE The image cannot be
- 10. 5x Higher Performance with MySQL 5.6 Multi-Threaded Slave Performance 300 250 Queries per Second 200 150 100 50 0 0 5 10 QPS 58.11 144.4 282.53 Worker Threads The image cannot be
- 11. Multi-Threaded Slaves Relay • Increases slave throughput, Log reducing lag T4 T3 T2 T1 I/O • All transactions received into slave’s relay log Coordinator • Implements multiple SQL threads, based on database • Applies events to different SQL-A SQL-B SQL-C T4 databases in parallel T1 T2 T3 • Great for systems which isolate application data using DB-A DB-B DB-C databases – e.g. multi-tenant The image cannot be Copyright 2012 Oracle Corporation 11
- 12. Multi-Threaded Slaves • Throughput of slave increased by allowing multiple slave threads: • 0 – functionality disabled • Up to 1024 • Exec_Master_Log_Posn in SHOW SLAVE STATUS now shows the low-water-mark • Configure using: slave-parallel-workers=4 • Only ensures sequencing within a database • Unsuitable if ordering of updates between databases is not required The image cannot be Copyright 2012 Oracle Corporation 12
- 13. Multi-Threaded Slaves Master Slave thread1 Slave thread2 INSERT INTO dba.tab1 ... dba.tab1 dbb.tab1 INSERT INTO dba.tab2 ... dba.tab2 dbc.tab2 INSERT INTO dbb.tab1 ... dba.tab1 dbb.tab3 INSERT INTO dbc.tab2 ... dba.tab3 dbc.tab2 INSERT INTO dba.tab1 ... dba.tab4 INSERT INTO dbb.tab3 ... • Transactions applied in parallel on INSERT INTO dba.tab3 ... the slave INSERT INTO dbc.tab2 ... • Sequencing of transactions within a INSERT INTO dba.tab4 ... given database will be the same The image cannot be Copyright 2012 Oracle Corporation 13
- 14. Optimized Row Based Replication Primary Key Changed Columns • Increases replication throughput for master and slave • Reduces Binlog size, memory requirements & network bandwidth • Only replicates those elements of the Row image that have changed The image cannot be Copyright 2012 Oracle Corporation 14
- 15. Optimized Row Based Replication • Default is to include full before & after image for every changed row • New option: binlog-row-image=minimal • Reduces space: • Inserts: only after image • Deletes: only before image • Updates: after image + primary key or indexed columns of before image The image cannot be Copyright 2012 Oracle Corporation 15
- 16. AUTOMATED FAILOVER & RECOVERY The image cannot be
- 17. Global Transaction Identifiers Master GTID=123456 GTID=123456 GTID=123456 GTID=123456 • Foundation for reliable, automatic failover & recovery • Unique identifier for each replication event written to the Binlog • Simple to track & compare replication across the cluster • Automatically identify the most up-to-date slave for failover • Deploy complex replication topologies • Eliminates Dev/Ops overhead The image cannot be Copyright 2012 Oracle Corporation 17
- 18. Before Global Transaction IDs Master {my-bin.000101, 873} {my-bin.000132, 782} {my-bin.000099, {my-bin.00088, 123} 1027} The image cannot be Copyright 2012 Oracle Corporation 18
- 19. Before Global Transaction IDs Failover & Recovery Must convert {file,pos} from Master old master to one from the new master {my-bin.000101, 873} CHANGE MASTER TO?? {my-bin.000132, 782} {my-bin.000099, {my-bin.00088, 123} 1027} The image cannot be Copyright 2012 Oracle Corporation 19
- 20. With Global Transaction IDs Master GTID=123456 GTID=123456 GTID=123456 GTID=123456 The image cannot be Copyright 2012 Oracle Corporation 20
- 21. With Global Transaction IDs Failover & Recovery Slave(s) can start from same GTID Master GTID=123456 CHANGE MASTER TO?? GTID=123456 GTID=123456 GTID=123456 The image cannot be Copyright 2012 Oracle Corporation 21
- 22. Global Transaction ID Utilities • Self-Healing, Low Administration replication cluster • Automated failover & recovery • mysqlfailover Utility • Switchover & administration • mysqlrpladmin Utility • Delivers HA within the core MySQL distribution • Eliminates the need to integrate 3rd party HA frameworks • Allows extensibility to support variety of HA mechanisms The image cannot be
- 23. Fail- Replication Failover Utility Over • Automatic failover & slave promotion • Default is to promote most up-to-date slave, based on GTID Failed • Slave promotion policies are fully Master configurable; select slave(s) that are candidates for promotion • Implemented as a command-line utility Promoted Monitoring Failover Master Auto-Failover & Utility Slave Promotion • Option to bind in your own pre/post failover scripts • For example, initiate VIP failover • Never lose replicated events Slaves • Nominated slave automatically retrieves any missing updates from other slaves The image cannot be Copyright 2012 Oracle Corporation 23
- 24. Fail- Use of mysqlfailover Over $ mysqlfailover --master=root@rod:3306 --discover-slaves-login=root MySQL Replication Monitor and Failover Utility Failover Mode = auto Next Interval = Mon Mar 19 15:56:03 2012 Master Information ------------------ Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB mysql-bin.000001 571 Replication Health Status +------------+-------+---------+--------+------------+---------+ | host | port | role | state | gtid_mode | health | +------------+-------+---------+--------+------------+---------+ | rod | 3306 | MASTER | UP | ON | OK | | jane | 3306 | SLAVE | UP | ON | OK | | jane | 3307 | SLAVE | UP | ON | OK | | freddy | 3306 | SLAVE | UP | ON | OK | +------------+-------+---------+--------+------------+---------+ Q-quit R-refresh H-health G-GTID Lists U-UUIDs L-log entries The image cannot be Copyright 2012 Oracle Corporation 24
- 25. Fail- mysqlfailover – master failed Over Failover starting... # Candidate slave jane:3306 will become the new master. # Preparing candidate for failover. # Creating replication user if it does not exist. # Stopping slaves. # Performing STOP on all slaves. # Switching slaves to new master. # Starting slaves. # Performing START on all slaves. # Checking slaves for errors. # Failover complete. # Discovering slaves for master at jane:3306 Failover console will restart in 5 seconds. The image cannot be Copyright 2012 Oracle Corporation 25
- 26. Fail- mysqlfailover – monitoring Over resumes MySQL Replication Monitor and Failover Utility Failover Mode = auto Next Interval = Mon Mar 19 16:05:12 2012 Master Information ------------------ Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB mysql-bin.000001 1117 UUIDs +------------+-------+---------+--------+------------+---------+ | host | port | role | state | gtid_mode | health | +------------+-------+---------+--------+------------+---------+ | jane | 3306 | MASTER | UP | ON | OK | | jane | 3307 | SLAVE | UP | ON | OK | | freddy | 3306 | SLAVE | UP | ON | OK | +------------+-------+---------+--------+------------+---------+ The image cannot be Copyright 2012 Oracle Corporation 26
- 27. Replication Administration Utility Admin • Perform switchover to eliminate downtime during planned Master maintenance • Start and stop slaves Status & Switchover Administration Utility • Slave discovery & monitoring • Slave status, thread status • Replication processing, including any lag • Configure slave promotion policies • Non GTID-functionality can be used Slaves with pre-MySQL 5.6 releases The image cannot be Copyright 2012 Oracle Corporation 27
- 28. Switch- Planned switchover to new master Over with mysqlrpladmin $ mysqlrpladmin --master=root@rod:3306 --slaves=root@jane:3306,root@jane:3307,root@freddy:3306 --new-master=root@jane:3306 --demote-master switchover # Performing switchover from master at rod:3306 to slave at jane:3306. # Checking candidate slave prerequisites. # Waiting for slaves to catch up to old master. # Stopping slaves. # Performing STOP on all slaves. # Demoting old master to be a slave to the new master. # Switching slaves to new master. # Starting all slaves. # Performing START on all slaves. # Checking slaves for errors. # Switchover complete. # ...done. The image cannot be Copyright 2012 Oracle Corporation 28
- 29. Crash-Safe Slaves & Binlog Atomic Data • Automatic recovery of a slave and CRASH! Binlog after a failure Position Info Time • Binlog and table data are transactionally consistent Before: – Transaction Data: in tables • Resumes replication without Dev/Op – Replication Info: in files intervention • Automatically rolling back replication to Atomic Data last committed event • Eliminates risk of data loss or Position Info Time corruption MySQL 5.6 – Transaction Data: in tables – Replication Info: in tables The image cannot be Copyright 2012 Oracle Corporation 29
- 30. Crash-Safe Slaves • Writes and reads back only complete events or transactions from the binary log • master.info & relay-log.info files can be replaced with tables • ACID transaction prevent them getting out of sync with InnoDB tables --master-info-repository=TABLE --relay-log-info-repository=TABLE mysql.slave_master_info mysql.slave_relay_log_info The image cannot be Copyright 2012 Oracle Corporation 30
- 31. DATA INTEGRITY The image cannot be
- 32. Replication Event Checksums • Ensures replicated data is Master Slave correct, consistent and accessible • Detects corrupt replication events before they’re applied • Returns an error • Protects entire replication path • Memory # # • Disk • Network • Bugs The image cannot be Copyright 2012 Oracle Corporation 32
- 33. Replication Event Checksums • Implemented in the binary and relay logs • New mysqld options: • binlog-checksum=CRC32 Checksums written to the binary log • master-verify-checksum=1 Master validates checksum read from the binary log • slave-sql-verify-checksum=1 Slave validates checksum when reading from the relay log The image cannot be Copyright 2012 Oracle Corporation 33
- 34. DEV/OPS AGILITY The image cannot be
- 35. MySQL Utilities • Automate common Dev/Ops tasks • Replication: provisioning, testing, monitoring and failover • Database comparisons: consistency checking • Database administration: users, connections, tables, etc • New utilities in development, ie log analysis • Implemented as Python scripts, plug-in for MySQL Workbench • Also available from LaunchPad • Extensible to include custom scripting • Resources: Documentation & Community Forum • On-Demand Webinar: http://t.co/7bFAV083 • http://dev.mysql.com/doc/workbench/en/mysql-utils-man.html • http://forums.mysql.com/list.php?155 The image cannot be Copyright 2012 Oracle Corporation 35
- 36. Utility Workflow for Replication Fail-Over Check Repl Show & Admin • Check: Verifies pre-requisites for Replication • Repl: Initiates Replication to the new slave • Show: Display Replication topology • Fail-Over & Admin: Detects and failovers (or switches) from master to slave. Status monitoring The image cannot be
- 37. Time Delayed Replication Slave 1 Relay Log :3307 • Configure time period before replication events applied to slave Master • Per-slave, via execution of SQL :3306 Binary Logs Thread • Second-level granularity, up to 68 Slave2 years! :3308 Relay Log • Protects against operational error • Dropping a table, etc. • Allows database to be inspected 10 Minute Delay without loading a back-up The image cannot be Copyright 2012 Oracle Corporation 37
- 38. Time Delayed Replication - configuring slave2> CHANGE MASTER TO -> MASTER_HOST = 'localhost‘, -> MASTER_PORT = 3306, -> MASTER_USER = 'repl_user‘, -> MASTER_PASSWORD = 'pw‘, -> MASTER_DELAY = 600; slave2> START SLAVE; The image cannot be Copyright 2012 Oracle Corporation 38
- 39. Time Delayed Replication – Doh! slave2> STOP SLAVE; Breath slowly! master> SHOW BINLOG EVENTSG *************************** 12. row *************************** Log_name: ws2-bin.000001 Pos: 984 Event_type: Query Server_id: 1 End_log_pos: 1096 Info: use `clusterdb`; DROP TABLE `towns` /* generated by server */ slave2> START SLAVE UNTIL -> MASTER_LOG_FILE='ws2-bin.000001', -> MASTER_LOG_POS=984; The image cannot be Copyright 2012 Oracle Corporation 39
- 40. Informational Log Events • Simplifies debugging and auditing when using Row Based Replication • Original query written to the Binlog • Replicated with row event to the slave • Activate with binlog-rows-query-log-events=TRUE master> INSERT INTO simples VALUES (20), (21),(22); The image cannot be Copyright 2012 Oracle Corporation 40
- 41. Informational Log Events BEFORE: mysqlbinlog (-v): ### INSERT INTO clusterdb.simples ### SET ### @1=20 /* INT meta=0 nullable=0 is_null=0 */ ### INSERT INTO clusterdb.simples ### SET ### @1=21 /* INT meta=0 nullable=0 is_null=0 */ ### INSERT INTO clusterdb.simples ### SET ### @1=22 /* INT meta=0 nullable=0 is_null=0 */ The image cannot be Copyright 2012 Oracle Corporation 41
- 42. Informational Log Events AFTER: mysqlbinlog (-vv): # at 443 #111128 16:04:24 server id 1 end_log_pos 504 Rows_query # INSERT INTO simples VALUES (20),(21),(22) # at 504 The image cannot be Copyright 2012 Oracle Corporation 42
- 43. Other Useful Enhancements • Remote BinLog Backup • Creates real-time backup of binary log on remote server • Eliminates requirement for: • MySQL instance to parse binary log into SQL statements • User needing privileges to each master for binary log backup • Universally Unique Server IDs • Enables auto-discovery and analysis of replication topology • Simplifies management of large and highly dynamic MySQL replication environments The image cannot be
- 44. How to Evaluate MySQL 5.6 http://dev.mysql.com/downloads/mysql/#downloads Select Development Release tab The image cannot be Copyright 2012 Oracle Corporation 44
- 45. Next Steps • Evaluate the new features • DevZone Article: http://dev.mysql.com/tech-resources/articles/mysql-5.6-replication.html • Questions & Feedback: forums.mysql.com/list.php?26 • Bugs: bugs.mysql.com/ • Replication documentation • dev.mysql.com/doc/refman/5.6/en/replication.html The image cannot be Copyright 2012 Oracle Corporation 45
- 46. MySQL 5.6 Replication: Supporting High Scale Web & Cloud Services
- 47. Remote Binary Log Back-up MySQL • Enhances operational efficiency Slave • Creates real-time backup of binary MySQL Master log on remote server • Adds a “raw” flag to the Relay Log mysqlbinlog command • Sent via the replication channel Backup • No need for….. • MySQL instance to parse binary log into SQL statements • User needing privileges to each Binary Log Backup master for binary log backup The image cannot be Copyright 2012 Oracle Corporation 47
- 48. Remote Binary Log Backup • Writes to a local file with the same name as the original $> mysqlbinlog --read-from-remote- server --raw –h secret_server -P 3306 -u root mysql-bin.000001 The image cannot be Copyright 2012 Oracle Corporation 48
- 49. Universally Unique Server IDs ! • Enables auto-discovery and analysis of replication topology • UUID automatically generated for each new server that becomes part of the replication cluster • Can be viewed with systems monitoring tools, i.e. MySQL Enterprise Monitor • Simplifies management of large and highly dynamic MySQL replication environments The image cannot be Copyright 2012 Oracle Corporation 49
- 50. Universally Unique Server IDs • A UUID generated when the MySQL server first starts • Stored in auto.cnf file • Available as: • server_uuid system variable for local host • SHOW SLAVE HOSTS for UUIDs of slaves • SHOW SLAVE STATUS for UUIDs of master slave> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event .... Master_UUID: e43ac311-19cc-11e1-a06e-002318c418fd .... The image cannot be Copyright 2012 Oracle Corporation 50
- 51. How to get Utilities • Available on Launchpad • https://launchpad.net/mysql-utilities • bzr branch lp:mysql-utilities • Requires Connector/Python • https://launchpad.net/myconnpy • bzr branch lp:myconnpy • Available as a plugin in MySQL Workbench • http://www.mysql.com/downloads/workbench/ • Documentation is here: • http://dev.mysql.com/doc/workbench/en/mysql- utilities.html The image cannot be