Tungsten University: Setup and Operate Tungsten Replicators
0 likes•719 views
The document provides an overview of Tungsten Replicator, a tool for high-performance data replication for MySQL, detailing its installation, configuration, and management processes. It outlines the steps to set up and operate the replicator, commands to check and modify configurations, and troubleshooting tips. Additionally, it highlights the importance of community engagement and resources for further information.
![Commands to Check Replication
• trepctl services -- All services
• trepctl status -- Overall pipe status
• trepctl status -name stages -- Stage summary
• trepctl status -name stores -- Store summary
• trepctl status -name tasks -- Task detail
• trepctl [-service name] heartbeat -- Is it Alive?
©Continuent 2013 9](https://image.slidesharecdn.com/tungstenuniversity01-31-2013-130212215047-phpapp02/85/Tungsten-University-Setup-and-Operate-Tungsten-Replicators-9-320.jpg)
![The Scene of the Crime
trepctl -host db2 status
Processing status command...
NAME VALUE
---- -----
...
pendingError : Event application failed:
seqno=1957469 fragno=0 message=java.sql.SQLException:
Statement failed on slave but succeeded on master
pendingErrorCode : NONE
pendingErrorEventId : mysql-bin.000008:0000000461736666;0
pendingErrorSeqno : 1957469
pendingExceptionMessage: java.sql.SQLException: Statement
failed on slave but succeeded on master
create table trouble(id int primary
key) /* ___SERVICE___ = [university] */
...
Finished status command...
©Continuent 2013 16](https://image.slidesharecdn.com/tungstenuniversity01-31-2013-130212215047-phpapp02/85/Tungsten-University-Setup-and-Operate-Tungsten-Replicators-16-320.jpg)
Tungsten University: Setup and Operate Tungsten Replicators
- 1. Tungsten University: Set Up and Operate Tungsten Replication Robert Hodges, CEO Giuseppe Maxia, Director of QA ©Continuent 2013
- 2. About Continuent • The leading provider of clustering and replication for open source DBMS • Tungsten Clustering - Commercial-grade HA, performance scaling and data management for MySQL • Tungsten Replication - Real-time, "exible, high-performance replication ©Continuent 2013 2
- 4. Tungsten Replicator Overview Master Replicator THL Download transactions via network (Transactions + Metadata) DBMS Logs Slave Replicator THL Apply using JDBC (Transactions + Metadata) ©Continuent 2013 4
- 5. Tungsten Replication Service Pipeline Stage Stage Stage Extract Filter Apply Extract Filter Apply Extract Filter Apply Master Transaction In-Memory Slave DBMS History Log Queue DBMS ©Continuent 2013 5
- 6. Overview of Installation Process 1. Set up hosts 2. Prepare MySQL replicas 3. Download software 4. Install using tungsten-installer Amazon Setup: https://docs.continuent.com/wiki/display/TEDOC/ Preparing+EC2+Servers ©Continuent 2013 6
- 7. How tungsten-installer Works Staging copy db1 of files db2 check prereqs copy code db3 configure ©Continuent 2013 7
- 8. What Is Tungsten Doing? ©Continuent 2013 8
- 9. Commands to Check Replication • trepctl services -- All services • trepctl status -- Overall pipe status • trepctl status -name stages -- Stage summary • trepctl status -name stores -- Store summary • trepctl status -name tasks -- Task detail • trepctl [-service name] heartbeat -- Is it Alive? ©Continuent 2013 9
- 10. Changing Replicator Con!guration ©Continuent 2013 10
- 11. Commands to Change Con!guration • con#gure-service - Alter con#guration • update - Upgrade the replicator to a new version Hint: Don’t change replicator configuration files directly... ©Continuent 2013 11
- 12. Recon!guring a Replicator • No Restart trepctl offline tools/configure-service ... trepctl configure trepctl online • With Restart tools/configure-service ... replicator restart ©Continuent 2013 12
- 13. Enabling Parallel Replication ©Continuent 2013 13
- 14. Parallel Replication of Sharded Data Pipeline Stage Stage Stage Extract Filter Apply Extract Filter Apply Extract Filter Apply Extract Filter Apply Extract Filter Apply Master Transaction Parallel Slave DBMS History Log Queue DBMS ©Continuent 2013 14
- 16. The Scene of the Crime trepctl -host db2 status Processing status command... NAME VALUE ---- ----- ... pendingError : Event application failed: seqno=1957469 fragno=0 message=java.sql.SQLException: Statement failed on slave but succeeded on master pendingErrorCode : NONE pendingErrorEventId : mysql-bin.000008:0000000461736666;0 pendingErrorSeqno : 1957469 pendingExceptionMessage: java.sql.SQLException: Statement failed on slave but succeeded on master create table trouble(id int primary key) /* ___SERVICE___ = [university] */ ... Finished status command... ©Continuent 2013 16
- 17. Correcting problems 1. Look at the transaction (thl list -seqno NNN) 2. Look at the replicator message log 3. Fix the problem and go online OR 4. Skip over the problem and go online OR 5. Reprovision the slave ©Continuent 2013 17
- 19. Simple Facts about Logs • Replicator log = THL (Transaction History Log) • One THL per replication service • 2x the size of MySQL binlog • Default retention of 7 days • Take service o$ine to move or delete logs safely • The thl tool shows you log contents ©Continuent 2013 19
- 21. Resetting and Reinstalling Reset 1. Stop replicator 2. Remove THL #les 3. Remove tungsten_<svc> databases Reinstall 1. All of the above, plus delete /opt/continuent/* 2. Rerun installation ©Continuent 2013 21
- 22. Backup and Restore ©Continuent 2013 22
- 23. What Backup Tools Does MySQL Have? MySQL mysqldump Server Process (dataset < 10Gb) xtrabackup DBMS Files (10Gb < dataset < 1Tb tar/rsync (dataset < 1Tb) file system snapshot File System (1Tb < dataset) ©Continuent 2013 23
- 24. Advanced Installation using Cookbooks ©Continuent 2013 24
- 25. How Do I Install Fan-In Replication? db1 Replicator Service db1 Replicator Service db1 Service db2 db2 Replicator Service db2 db3 ©Continuent 2013 25
- 26. How Do I install Multi-Master? db1 Replicator Service db1 Service db2 db3 Service db3 Service db1 Service db2 db2 Service db3 Service db1 Service db2 Service db3 Replicator ©Continuent 2013 26
- 27. Tungsten Cookbook Helps! • Installs complex topologies automatically • Directions in cookbook/README • Parameters in cookbook/USER_VALUES.sh • Node names in COMMON_NODES.sh • Installation script for each supported topology ©Continuent 2013 27
- 28. Joining the Community ©Continuent 2013 28
- 29. Tungsten Replicator is Open Source • Project home: http://code.google.com/p/tungsten-replicator/ • Log bugs, #nd builds, post in replicator discussion group • Documentation: https://docs.continuent.com/wiki/display/TEDOC/ Tungsten+Documentation+Home ©Continuent 2013 29
- 30. Wrapping Up ©Continuent 2013 30
- 31. Tungsten University Sessions • Unleashing the Power of Tungsten Connectors (February 21 & 26) • Managing Multi-Master and Fan-in Topologies (March) • Zero-downtime DBMS Migration and Maintenance (March) Send any feedback to: [email protected] ©Continuent 2013 31
- 32. 560 S. Winchester Blvd., Suite 500 Our Blogs: San Jose, CA 95128 http://scale-out-blog.blogspot.com Tel +1 (866) 998-3642 http://datacharmer.org/blog Fax +1 (408) 668-1009 http://www.continuent.com/news/blogs e-mail: [email protected] Continuent Web Page: http://www.continuent.com Tungsten Replicator 2.0: http://code.google.com/p/tungsten-replicator ©Continuent 2012.