Training Slides: 104 - Basics - Working With Command Line Tools
0 likes108 views
The document outlines the basics of working with Tungsten Cluster command line tools, including TPM, CCTRL, and TREPCTL. It provides an overview of the installation requirements, key commands, and functions for managing a MySQL cluster. Additionally, it discusses monitoring and controlling cluster nodes, as well as the management of the replicator process.
![tpm
• Tungsten Package Manager
• As well as using tpm for installs and updates, it can also be used for a number of other actions. You
can issue tpm help for a list of possible options.
• Most common options:
• tools/tpm validate[-update]
• [tools/]tpm update [--replace-release] [--no-connectors]
• tools/tpm install
• tpm diag – Gathers package of stats for support!
• tpm connector – Launches the MySQL command-line client and connects to the current
Primary node via locally-running Connector process
• tpm mysql – Launches the MySQL command-line client and connects to the MySQL server
process running on the local host](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-6-320.jpg)
![tpm connector
• Simple and quick way to connect to the current MySQL Primary node via the command line
• Only works on hosts where the Connector is running
• Will authenticate using credential supplied via application-user and application-password properties
• Provides additional commands to query database and cluster stats:
• Connector-based Tungsten commands are NOT available in Bridge Mode
• This is a good way to tell if you are in Bridge mode – if no commands are available, then you are in Bridge mode
• `tungsten help` will show all commands available
mysql> tungsten help;
+---------------------------------------------------------------------------------------------------------------------------------+
| Message |
+---------------------------------------------------------------------------------------------------------------------------------+
| tungsten connection status: display information about the connection used for the last request ran |
| tungsten connection count: gives the count of current connections to each one of the cluster datasources |
| tungsten cluster status: prints detailed information about the cluster view this connector has |
| tungsten show [full] processlist: list all running queries handled by this connector instance |
| tungsten show variables [like '<string>']: list connector configuration options in use. The <string> may contain '%' wildcards |
| tungsten flush privileges: reload user.map and refresh user credentials |
| tungsten mem info: display memory information about current JVM |
| tungsten gc: calls garbage collector |
| tungsten help: display this help message |
+---------------------------------------------------------------------------------------------------------------------------------+
9 rows in set (0.00 sec)](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-7-320.jpg)
![cctrl
ls
• COORDINATOR[db3:AUTOMATIC:ONLINE]
• Identifies current Coordinator host and Current Cluster Mode
• ROUTERS:
• Lists all connectors associated with the installation, each connector shows status, and
created/active connection counts
• DATASOURCES:
• Lists all nodes that form the cluster](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-10-320.jpg)
![trepctl
• trepctl services
• Short list output of all services running on the host
• Shows basic information
• trepctl [-service SERVICENAME] status [-r N]
• Shows the full status of the replicator
• Specify –service if multiple services available
• Specify –r N to refresh every N seconds or until CTRL+C
• trepctl [-service SERVICENAME] status -name stages
• A more complete status view showing detailed output of each replicator stage](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-17-320.jpg)
![trepctl
• trepctl [-service SERVICENAME] qs [-r N]
• Shows a quick summary of the replicator progress
• Specify –service if multiple services available
• Specify –r N to refresh every N seconds or until CTRL+C
• trepctl [-service SERVICENAME] perf [-r N]
• Shows the status of each stage of the replication pipeline
• Output differs between Primary and Replicas
• Specify –service if there are multiple services available
• Specify –r N to refresh every N seconds or until CTRL+C](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-18-320.jpg)
![trepctl
• trepctl [-service SERVICENAME] reset {OPTIONS}
• Performs a FULL reset of the replicator
• VERY destructive if used incorrectly
• Resets SEQNO to 0
• trepctl [-service SERVICENAME] offline|online {OPTIONS}
• Bring a service online or offline
• Can be used with various options to control how/when
• Used with –skip-seqno to skip errors](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-19-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-33-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)
Global Sequence number for the event](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-34-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)
Associated THL File on Disk](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-35-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)
Commit time to Binary Logs](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-36-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)
Associated Binary Log File and Position](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-37-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)
Source of transaction (Should be a Primary!)](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-38-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)
Metadata](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-39-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)
The database schema that the
following SQL is being applied to](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-40-320.jpg)
![thl
SEQ# = 2 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:50:52.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000000939;12297
- SOURCEID = db1
- METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;
- service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1,
unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1,
sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,
ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33,
collation_connection = 33, collation_server = 8]
- SCHEMA = hr
- SQL(0) = CREATE TABLE regions
( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, region_name VARCHAR(25)
)
DDL Statement](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-41-320.jpg)
![thl
SEQ# = 5 / FRAG# = 0 (last frag)
- FILE = thl.data.0000000001
- TIME = 2020-01-08 13:51:38.0
- EPOCH# = 0
- EVENTID = mysql-bin.000005:0000000000001746;-1
- SOURCEID = db1
- METADATA = [mysql_server_id=101;dbms_type=mysql;tz_aware=true;service=training1;shard=hr]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [foreign_key_checks = 1, unique_checks = 1, time_zone = '+00:00', ##charset = UTF-8]
- SQL(0) =
- ACTION = INSERT
- SCHEMA = hr
- TABLE = regions
- ROW# = 0
- COL(1: ) = 1
- COL(2: ) = europe
Row Change Data](https://image.slidesharecdn.com/104-working-with-the-cli-200803130540/85/Training-Slides-104-Basics-Working-With-Command-Line-Tools-42-320.jpg)
Training Slides: 104 - Basics - Working With Command Line Tools
- 1. The MySQL Availability Company Tungsten Cluster Master Class Basics: Working with Command Line Tools Chris Parker, Customer Success Director, EMEA & APAC
- 2. Topics In this short course, we will • Re-cap the previous Installation • Explore the main Command Line Tools • tpm • cctrl • trepctl • thl
- 4. Tungsten Cluster Standalone Cluster • Minimum 3 nodes • 1 Primary • 2 Replicas • Odd number of nodes • Single datacenter/region
- 6. tpm • Tungsten Package Manager • As well as using tpm for installs and updates, it can also be used for a number of other actions. You can issue tpm help for a list of possible options. • Most common options: • tools/tpm validate[-update] • [tools/]tpm update [--replace-release] [--no-connectors] • tools/tpm install • tpm diag – Gathers package of stats for support! • tpm connector – Launches the MySQL command-line client and connects to the current Primary node via locally-running Connector process • tpm mysql – Launches the MySQL command-line client and connects to the MySQL server process running on the local host
- 7. tpm connector • Simple and quick way to connect to the current MySQL Primary node via the command line • Only works on hosts where the Connector is running • Will authenticate using credential supplied via application-user and application-password properties • Provides additional commands to query database and cluster stats: • Connector-based Tungsten commands are NOT available in Bridge Mode • This is a good way to tell if you are in Bridge mode – if no commands are available, then you are in Bridge mode • `tungsten help` will show all commands available mysql> tungsten help; +---------------------------------------------------------------------------------------------------------------------------------+ | Message | +---------------------------------------------------------------------------------------------------------------------------------+ | tungsten connection status: display information about the connection used for the last request ran | | tungsten connection count: gives the count of current connections to each one of the cluster datasources | | tungsten cluster status: prints detailed information about the cluster view this connector has | | tungsten show [full] processlist: list all running queries handled by this connector instance | | tungsten show variables [like '<string>']: list connector configuration options in use. The <string> may contain '%' wildcards | | tungsten flush privileges: reload user.map and refresh user credentials | | tungsten mem info: display memory information about current JVM | | tungsten gc: calls garbage collector | | tungsten help: display this help message | +---------------------------------------------------------------------------------------------------------------------------------+ 9 rows in set (0.00 sec)
- 8. cctrl • cctrl stands for Cluster Control • Can be executed from any node running the manager in the cluster. For example, running cctrl on a Connector-only host will fail. • Allows control of all nodes in the cluster • Main uses are • Check cluster state • Isolate nodes for maintenance • Switch Primary nodes • Recover nodes following failure/recovery • Perform backup/recovery operations • Requires the Manager to be running
- 9. cctrl help • Outputs list of commands available in cctrl • Use help <command> for more detail on each command
- 10. cctrl ls • COORDINATOR[db3:AUTOMATIC:ONLINE] • Identifies current Coordinator host and Current Cluster Mode • ROUTERS: • Lists all connectors associated with the installation, each connector shows status, and created/active connection counts • DATASOURCES: • Lists all nodes that form the cluster
- 11. cctrl Host, Role, Status Last seqno Extracted Latency from DB commit to THL Last seqno Applied Latency Applying to Target Database
- 12. cctrl Status of manager process Role and status of replicator process Status of MySQL DatabaseConnections through Connectors to DB Role and status of replicator process. On a replica, shows primary the replica is connected to
- 13. cctrl cluster validate • Checks the validity of the cluster and ensures that all nodes are reachable
- 14. cctrl cluster heartbeat • Inserts an entry into the Primary ‘heartbeat’ table • Good way to check that replication is flowing properly • Check the progress= entries in the ls output
- 15. cctrl Cluster Policy Modes • AUTOMATIC • Normal Operational state • Ensures automatic failover • Will attempt auto-recovery of components (MySQL server and Replicator) • MAINTENANCE • Prevents automatic operations mentioned above • Should ONLY be used during Maintenance Windows
- 16. trepctl • Used to control and manage the replicator Java process • Most common uses are • View replicator status • Stop/Start replication • Skip “safe” errors • trepctl help to see all options
- 17. trepctl • trepctl services • Short list output of all services running on the host • Shows basic information • trepctl [-service SERVICENAME] status [-r N] • Shows the full status of the replicator • Specify –service if multiple services available • Specify –r N to refresh every N seconds or until CTRL+C • trepctl [-service SERVICENAME] status -name stages • A more complete status view showing detailed output of each replicator stage
- 18. trepctl • trepctl [-service SERVICENAME] qs [-r N] • Shows a quick summary of the replicator progress • Specify –service if multiple services available • Specify –r N to refresh every N seconds or until CTRL+C • trepctl [-service SERVICENAME] perf [-r N] • Shows the status of each stage of the replication pipeline • Output differs between Primary and Replicas • Specify –service if there are multiple services available • Specify –r N to refresh every N seconds or until CTRL+C
- 19. trepctl • trepctl [-service SERVICENAME] reset {OPTIONS} • Performs a FULL reset of the replicator • VERY destructive if used incorrectly • Resets SEQNO to 0 • trepctl [-service SERVICENAME] offline|online {OPTIONS} • Bring a service online or offline • Can be used with various options to control how/when • Used with –skip-seqno to skip errors
- 20. trepctl status appliedLastEventId : mysql-bin.000005:0000000051631947;-1 appliedLastSeqno : 166764 appliedLatency : 0.769 autoRecoveryEnabled : false autoRecoveryTotal : 0 channels : 1 clusterName : alpha currentEventId : mysql-bin.000005:0000000051631947 currentTimeMillis : 1578578135591 dataServerHost : trainingdb1 extensions : host : trainingdb1 latestEpochNumber : 9 masterConnectUri : thl://localhost:/ masterListenUri : thl://trainingdb1:2112/
- 21. trepctl status appliedLastEventId : mysql-bin.000005:0000000051631947;-1 appliedLastSeqno : 166764 appliedLatency : 0.769 autoRecoveryEnabled : false autoRecoveryTotal : 0 channels : 1 clusterName : alpha currentEventId : mysql-bin.000005:0000000051631947 currentTimeMillis : 1578578135591 dataServerHost : trainingdb1 extensions : host : trainingdb1 latestEpochNumber : 9 masterConnectUri : thl://localhost:/ masterListenUri : thl:// trainingdb1 :2112/ On a Primary, the last ending binary log position written to the THL along with the Seqno for that event, and the latency between the database commit to the binlog and the THL write completion. On a replica, displays the last event written to the target database with the corresponding Seqno, and the latency between the source database commit and the completed apply of that event to the target database.
- 22. trepctl status appliedLastEventId : mysql-bin.000005:0000000051631947;-1 appliedLastSeqno : 166764 appliedLatency : 0.769 autoRecoveryEnabled : false autoRecoveryTotal : 0 channels : 1 clusterName : alpha currentEventId : mysql-bin.000005:0000000051631947 currentTimeMillis : 1578578135591 dataServerHost : trainingdb1 extensions : host : trainingdb1 latestEpochNumber : 9 masterConnectUri : thl://localhost:/ masterListenUri : thl:// trainingdb1 :2112/ Auto-Recovery properties
- 23. trepctl status appliedLastEventId : mysql-bin.000005:0000000051631947;-1 appliedLastSeqno : 166764 appliedLatency : 0.769 autoRecoveryEnabled : false autoRecoveryTotal : 0 channels : 1 clusterName : alpha currentEventId : mysql-bin.000005:0000000051631947 currentTimeMillis : 1578578135591 dataServerHost : trainingdb1 extensions : host : trainingdb1 latestEpochNumber : 9 masterConnectUri : thl://localhost:/ masterListenUri : thl://trainingdb1:2112/ Current Binlog position of the database (NONE on Replicas)
- 24. trepctl status appliedLastEventId : mysql-bin.000005:0000000051631947;-1 appliedLastSeqno : 166764 appliedLatency : 0.769 autoRecoveryEnabled : false autoRecoveryTotal : 0 channels : 1 clusterName : alpha currentEventId : mysql-bin.000005:0000000051631947 currentTimeMillis : 1578578135591 dataServerHost : trainingdb1 extensions : host : trainingdb1 latestEpochNumber : 9 masterConnectUri : thl://localhost:/ masterListenUri : thl://trainingdb1:2112/ masterConnectUri shows the source THL server we are connected to masterListenUri shows the THL server listener protocol, host and port information for replicas to connect with
- 25. trepctl status maximumStoredSeqNo : 166764 minimumStoredSeqNo : 0 offlineRequests : NONE pendingError : NONE pendingErrorCode : NONE pendingErrorEventId : NONE pendingErrorSeqno : -1 pendingExceptionMessage: NONE pipelineSource : /var/lib/mysql relativeLatency : 580.591 resourceJdbcDriver : org.drizzle.jdbc.DrizzleDriver resourceJdbcUrl : jdbc:mysql:thin://trainingdb1:13306/${DBNAME}. . . resourcePrecedence : 99 resourceVendor : mysql rmiPort : 10000
- 26. trepctl status maximumStoredSeqNo : 166764 minimumStoredSeqNo : 0 offlineRequests : NONE pendingError : NONE pendingErrorCode : NONE pendingErrorEventId : NONE pendingErrorSeqno : -1 pendingExceptionMessage: NONE pipelineSource : /var/lib/mysql relativeLatency : 580.591 resourceJdbcDriver : org.drizzle.jdbc.DrizzleDriver resourceJdbcUrl : jdbc:mysql:thin:// trainingdb1:13306 /${DBNAME}. . . resourcePrecedence : 99 resourceVendor : mysql rmiPort : 10000 When the Replicator goes into an OFFLINE:ERROR state, these fields will show all the associated information. Always check the trepsvc.log file for more detail as needed.
- 27. trepctl status maximumStoredSeqNo : 166764 minimumStoredSeqNo : 0 offlineRequests : NONE pendingError : NONE pendingErrorCode : NONE pendingErrorEventId : NONE pendingErrorSeqno : -1 pendingExceptionMessage: NONE pipelineSource : /var/lib/mysql relativeLatency : 580.591 resourceJdbcDriver : org.drizzle.jdbc.DrizzleDriver resourceJdbcUrl : jdbc:mysql:thin://trainingdb1:13306/${DBNAME}. . . resourcePrecedence : 99 resourceVendor : mysql rmiPort : 10000 The current source of THL. Most cases will match the masterConnectUri unless using an MSAA topology. A Primary will show the binary log directory.
- 28. trepctl status maximumStoredSeqNo : 166764 minimumStoredSeqNo : 0 offlineRequests : NONE pendingError : NONE pendingErrorCode : NONE pendingErrorEventId : NONE pendingErrorSeqno : -1 pendingExceptionMessage: NONE pipelineSource : /var/lib/mysql relativeLatency : 580.591 resourceJdbcDriver : org.drizzle.jdbc.DrizzleDriver resourceJdbcUrl : jdbc:mysql:thin://trainingdb1:13306/${DBNAME}. . . resourcePrecedence : 99 resourceVendor : mysql rmiPort : 10000 Latency between NOW and the timestamp of the last event in the local THL.
- 29. trepctl status role : master seqnoType : java.lang.Long serviceName : alpha serviceType : local simpleServiceName : alpha siteName : default sourceId : trainingdb1 state : ONLINE timeInStateSeconds : 85641.738 timezone : GMT transitioningTo : uptimeSeconds : 85673.511 useSSLConnection : false version : Tungsten Clustering 6.1.4 build 44
- 30. trepctl status role : master seqnoType : java.lang.Long serviceName : training1 serviceType : local simpleServiceName : training1 siteName : default sourceId : trainingdb1 state : ONLINE timeInStateSeconds : 85641.738 timezone : GMT transitioningTo : uptimeSeconds : 85673.511 useSSLConnection : false version : Tungsten Clustering 6.1.4 build 44 Current role : master, slave or relay Current State, can be : • ONLINE • ONLINE:DEGRADED • ONLINE:DEGRADED-BINLOG-FULLY-READ • OFFLINE:NORMAL • SUSPECT • OFFLINE:ERROR • GOING-ONLINE:SYNCHRONISING • GOING-ONLINE:RESTORING • GOING-ONLINE:PROVISIONING
- 31. Applied Latency vs Relative Latency The appliedLatency is the latency between the commit time of the source event and the time the last committed transaction reached the end of the corresponding pipeline within the replicator. Within a primary, this indicates the latency between the transaction commit time and when it was written to the THL. In a replica, it indicates the latency between the commit time on the primary database and when the transaction has been committed to the destination database. Clocks must be synchronized across hosts for this information to be accurate. The latency is measured in seconds. Increasing latency may indicate that the destination database is unable to keep up with the transactions from the primary. In replicators that are operating with parallel apply, appliedLatency indicates the latency of the trailing channel. Because the parallel apply mechanism does not update all channels simultaneously, the figure shown may trail significantly from the actual latency. The relativeLatency is the latency between now and timestamp of the last event written into the local THL. This information gives an indication of how fresh the incoming THL information is. On a primary, it indicates whether the primary is keeping up with transactions generated on the primary database. On a replica, it indicates how up to date the THL read from the extractor is. A large value can either indicate that the database is not busy, that a large transaction is currently being read from the source database or from the primary replicator, or that the replicator has stalled for some reason. An increasing relativeLatency on the replica may indicate that the replicator may have stalled and stopped applying changes to the database.
- 32. thl • Interface for viewing the contents of the THL • thl help to view all command options • thl info – Show a summary of the THL available on disk • thl list will product a lot of output, always use with options to filter the result set • -low|from SEQ – Start from supplied seqno • -high|to SEQ – Stop at supplie seqno • -first – Show first seqno available • -first N – Show first N entries • -last – Show last seqno available • -last N – Show last N entries • thl index – re-index THL – can help to speed up replicator restarts • thl purge – Use with CARE since this command will REMOVE ALL THL on disk for that service
- 33. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) )
- 34. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) ) Global Sequence number for the event
- 35. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) ) Associated THL File on Disk
- 36. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) ) Commit time to Binary Logs
- 37. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) ) Associated Binary Log File and Position
- 38. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) ) Source of transaction (Should be a Primary!)
- 39. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) ) Metadata
- 40. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) ) The database schema that the following SQL is being applied to
- 41. thl SEQ# = 2 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:50:52.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000000939;12297 - SOURCEID = db1 - METADATA = [mysql_server_id=101;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true; - service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [##charset = UTF-8, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, auto_increment_increment = 2, auto_increment_offset = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES, ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE', character_set_client = 33, collation_connection = 33, collation_server = 8] - SCHEMA = hr - SQL(0) = CREATE TABLE regions ( region_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY , region_name VARCHAR(25) ) DDL Statement
- 42. thl SEQ# = 5 / FRAG# = 0 (last frag) - FILE = thl.data.0000000001 - TIME = 2020-01-08 13:51:38.0 - EPOCH# = 0 - EVENTID = mysql-bin.000005:0000000000001746;-1 - SOURCEID = db1 - METADATA = [mysql_server_id=101;dbms_type=mysql;tz_aware=true;service=training1;shard=hr] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [foreign_key_checks = 1, unique_checks = 1, time_zone = '+00:00', ##charset = UTF-8] - SQL(0) = - ACTION = INSERT - SCHEMA = hr - TABLE = regions - ROW# = 0 - COL(1: ) = 1 - COL(2: ) = europe Row Change Data
- 43. When the PrimaryKey (pkey) filter is enabled, the key information is optimized to only contain the actual primary keys for the row-based THL record. - SQL(0) = - ACTION = UPDATE - SCHEMA = hr - TABLE = regions - ROW# = 0 - COL(1: ) = 1 - COL(2: ) = Europe - KEY(1: ) = 1 - KEY(2: ) = europe - SQL(0) = - ACTION = UPDATE - SCHEMA = hr - TABLE = regions - ROW# = 0 - COL(1: ) = 1 - COL(2: ) = Europe - KEY(1: ) = 1 - SQL(0) = - ACTION = DELETE - SCHEMA = hr - TABLE = regions - ROW# = 0 - KEY(1: ) = 1 - KEY(2: ) = Europe - SQL(0) = - ACTION = DELETE - SCHEMA = hr - TABLE = regions - ROW# = 0 - KEY(1: ) = 1 UPDATE DELETE Without pkey filter With pkey filter
- 44. Other useful tools • dsctl • Advanced method to reset replicator position • tungsten_find_position • Can be used to interrogate THL for a specific binary log position and return a dsctl command to reposition a replicator • tungsten_find_seqno • Works in the same way as tungsten_find_position but searches based on a sequence number • multi_trepctl • Useful for Multi-Site/Active-Active deployments
- 45. Summary What we have learnt today • How to use the command line tools • tpm • cctrl • trepctl • thl • How to interpret status output
- 46. Next Steps In the next session we will • Discuss Maintenance Operations • Isolating Cluster Nodes • Updating paramaters • Failover/Switches • Rolling Maintenance • Upgrades
- 47. THANK YOU FOR LISTENING continuent.com The MySQL Availability Company Chris Parker, Customer Success Director, EMEA & APAC