Oracle OpenWorld 2013 - HOL9737 MySQL Replication Best Practices

MySQL Replication
Best Practices
Luís Soares
Sr. Software Engineer
Sven Sandberg
Sr. Software Engineer

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing
decisions.
The development, release, and timing of any features or functionality
described for Oracle’s products remains at the sole discretion of
Oracle.

Agenda
 Replication Basics
 Crash-Safe Slaves: Replication Metadata in System Tables.
 On-line Data Verification: Replication Event Checksums
 Tuning and Optimizing Row-based Replication
 Improving the Slave Scalability: Multi-Threaded Slave Applier
 Automated Fail-over and Slave Positioning: Global Transaction Ids
 Summary

Replication Basics
 MySQL Master Server
– Changes data
– Logs changes (Events) into a file (Binary Log)
 MySQL Slave Server
– Retrieves events from the master
– Replays the events on the slave's databases
MySQL Replication Components.

Replication Basics
 The Binary Log
– File based log that records the changes on the master.
– Statement or Row based format (may be intermixed).
– Split into transactional groups.
BEGIN ...Ev1 Ev2 COMMIT
MySQL Replication Components: Binary Log.
Binary Log FileBinary Log File
Event Layout on
a Binary Log File

Replication Basics
MySQL Replication Components: Binary Log.
Binary Log
Binary
log
files
Index
Under the Hood
Binary log files: mysql-bin.000001, mysql-bin.000002, …
- The actual data.
Index: mysql-bin.index
- An index over the binary log files.
Log coordinate:
- binlog file name + event offset in the file (3.23.15+)
- Global Transaction Identifiers (5.6+)

Replication Basics
MySQL Replication Architecture.
Insert...
Insert...
B
binary log
Insert...
relay log
Insert...
A
binary log
Client
Dump
thread
IO
thread
SQL
thread
Network
Master
info
Relay
Log
Info
Replication
Metadata

Replication Basics
 Asynchronous Replication (MySQL 3.23.15+)
– Transactions are committed and externalized without interaction with
the replication layer.
– Events are sent to slave after the commit operation is acknowledged.
– Faster but vulnerable to lost updates on server crashes and
inconsistency.
– Built into the server.
 Semi-synchronous Replication (MySQL 5.5+)
– Master commits transaction but waits until one slave acknowledges
having received and stored the event before replying to the client.
Changes Propagation.

Replication Basics
Hands-On

Replication Basics
Getting MySQL.
 Head to
– http://dev.mysql.com
 ... and get the latest MySQL 5.6 generic tar.gz package from there
 … or alternatively, you can find MySQL 5.6 linux generic package at
your local directory /usr/local/mysql56 .

Replication Basics
Setting Up Working Directories.
$ tar xfvz /usr/local/mysql....tar.gz
directory=$HOME/hol9737
$ mkdir $HOME/hol9737/
$ cd $HOME/hol9737/
Create a directory
for storing the binaries
(e.g., under your home).
Unpack the
package
(e.g., using tar).
$ mv $HOME/hol9737/mysql... $HOME/hol9737/mysql56
Rename the directory for simplicity

Replication Basics
Setting Up MySQL.
$ mysql56/scripts/mysql_install_db
  basedir=$HOME/hol9737/mysql56
  datadir=$HOME/hol9737/master
  user=`whoami`
Create
Two Data
Directories
$ mysql56/scripts/mysql_install_db
  basedir=$HOME/hol9737/mysql56
  datadir=$HOME/hol9737/slave
  user=`whoami`
Create
two Data
Directories

Replication Basics
Setting Up MySQL Master Server.
[mysqld]
serverid=1
logbin=masterbin
logerror=master.err
port=12000
Create a defaults file called
$HOME/hol9737/master.cnf
Set the unique server id.
Turn ON the binary log.

Replication Basics
$ mysql56/bin/mysqld
  defaultsfile=$HOME/hol9737/master.cnf
  lcmessagesdir=$HOME/hol9737/mysql56/share
  datadir=$HOME/hol9737/master/
Start the
master server.
$ mysql5.6/bin/mysql u root port 12000 protocol=tcp
  prompt='master> '
Log in to the
master server.
Setting Up MySQL Master Server.

Replication Basics
Inspecting the Master Status.
master> SHOW BINARY LOGS;
...
master> SHOW BINLOG EVENTS;
...
master> SHOW MASTER STATUS;
What binlog
files are in use?
What's in the
first binary log?
What binary log is in
use what is its position?

Replication Basics
The Binary Log Layout.
Log Events
Format description
Rotate
Log Events
Format description
Rotate
Log Events
Format description ● Multiple Files.
● Files begin with
Format Description
event.
● Each log file ends
with a Rotate event.

Replication Basics
Setting Up Replication User (that the slave will connect with).
master> CREATE USER `rpl_user`@`localhost`;
master> GRANT REPLICATION SLAVE ON *.*
TO `rpl_user`@`localhost`
IDENTIFIED BY 'secret';
master> FLUSH PRIVILEGES;
We could use
any user.
Needs replication
grants to read any change
on the master
Better use
a dedicated
user for connecting
the slave.

Replication Basics
Setting Up MySQL Slave Server.
[mysqld]
serverid=2
relaylog=slaverelaybin
logerror=slave.err
port=12001
Set the unique server id.
Set the name for the
relay log.
$HOME/hol9737/slave.cnf

Replication Basics
Starting Up The MySQL Slave Server.
$ mysql5.6/bin/mysqld
  defaultsfile=$HOME/hol9737/slave.cnf
  lcmessagesdir=$HOME/hol9737/mysql5.6/share
  datadir=$HOME/hol9737/slave/
Start the
slave server.
$ mysql5.6/bin/mysql u root port 12001 –protocol=tcp
  prompt='slave> '
Log in to the
slave server.

Replication Basics
Starting the Slave Threads.
slave> CHANGE MASTER TO
  MASTER_HOST = 'localhost',
  MASTER_PORT = 12000,
  MASTER_USER = 'rpl_user',
  MASTER_PASSWORD = 'secret';
Point the
slave server
to the master
server.
Use the credentials
we had granted
before.
slave> START SLAVE;
Start the slave
threads.

Replication Basics
Inspecting the Slave Status.
slave> SHOW SLAVE STATUS;
Inspect the slave
status.
(...)
               Slave_IO_State: Waiting for master to send event
                  Master_Host: localhost
                  Master_User: rpl_user
                  Master_Port: 12000
                Connect_Retry: 60
              Master_Log_File: masterbin.000002
          Read_Master_Log_Pos: 120
               Relay_Log_File: slaverelaybin.000003
                Relay_Log_Pos: 284
        Relay_Master_Log_File: masterbin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
(...)
Both slave threads
are up and running!

Replication Basics
Replicating From Master to Slave.
master> USE test;
master> CREATE TABLE t1 (a INT);
slave> USE test;
slave> SHOW TABLES;
master> INSERT INTO t1 VALUES(1);
slave> SELECT * FROM t1;
Use the
test db.
Create a table.
Slave has
replicated
the table.
Slave has
replicated
the data.

Replication Basics
What's in the Binary Log?
master> SHOW BINLOG EVENTS [IN 'masterbin.0000002'];
+++++++
+++++++
| masterbin.000002 | 120 | Query       |         1 |         217 | use `test`; CREATE TABLE t1 (a INT)     |
| masterbin.000002 | 217 | Query       |         1 |         296 | BEGIN                                   |
| masterbin.000002 | 296 | Query       |         1 |         395 | use `test`; INSERT INTO t1 VALUES (1)   |
| masterbin.000002 | 395 | Xid         |         1 |         426 | COMMIT /* xid=21 */                     |
+++++++
5 rows in set (0.00 sec)
The “CREATE TABLE” statement.
The “INSERT” statement.

Replication Basics
Replicating From Master to Slave – binary log formats.
• Three formats:
– STATEMENT – every change logged as a statement.
●
Re-executed on the slave.
– ROW – every change logged as data.
●
Data changes are applied on the slave.
– MIXED – every change logged either as data or statements.
●
Automatically switches from statement to row on
non-deterministic statements.

Replication Basics
Replicating From Master to Slave – binary log formats.
master> set binlog_format=ROW;
+++++++
+++++++
| masterbin.000002 | 498 | Table_map   |         1 |         543 | table_id: 70 (test.t1)                  |
| masterbin.000002 | 543 | Write_rows  |         1 |         583 | table_id: 70 flags: STMT_END_F          |
+++++++
Let's change
the format.
The second “INSERT”
statement.

Replication Basics
Master replication files.
$ ls la master/
(...)
rwrw 1 XXXXXX XXXXXX      143 Sep 26 11:10 masterbin.000001
rwrw 1 XXXXXX XXXXXX      614 Sep 26 11:40 masterbin.000002
rwrw 1 XXXXXX XXXXXX       40 Sep 26 11:10 masterbin.index
(...)
The binary log files.
Index file over the
existing binary log
files.

Replication Basics
Slave replication files.
$ ls la slave/
(...)
rwrw 1 XXXXXXX XXXXXXX      128 Sep 26 11:40 master.info
(...)
rwrw 1 XXXXXXX XXXXXXX       57 Sep 26 11:40 relaylog.info
(...)
rwrw 1 XXXXXXX XXXXXXX      337 Sep 26 11:10 slaverelaybin.000002
rwrw 1 XXXXXXX XXXXXXX      778 Sep 26 11:40 slaverelaybin.000003
rwrw 1 XXXXXXX XXXXXXX       50 Sep 26 11:10 slaverelaybin.index
(...)
The relay log files.
Index file over the
existing binary log
files.
Persists IO thread repliaction metadata
(master host, username, … and
positioning on the master's binlog).
Persists SQL thread replication metadata.
Basically, the positioning in the relay log.

Crash-Safe Positioning:
Storing Replication
Metadata in Tables.

Storing Replication Metadata in Tables.
 Store replication metadata in tables, within the context of regular
transactions.
– Commit both data and replication positions together.
– Rollback both data and replication positions together.
– Robust, highly available setups.
– No metadata in files (master.info and relay-log.info).
– Slave knows a valid position to resume replication after a crash.
 Often referred to as the “Crash-Safe Slave” or “Transactional
Replication” feature.
 Non-transactional engines will not work with this for obvious reasons.
Self-healing slaves.
MySQL 5.6

Configuring the slave to use tables as metadata repository.
$ mysql5.6/bin/mysqladmin shutdown
  user=root port=12001 protocol=tcp
[mysqld]
(...)
relayloginforepository=TABLE
masterinforepository=TABLE
Start the
slave server.
Shutdown
the slave
server.
Configure slave
to use tables,
instead of files.

slave> SELECT * FROM mysql.slave_master_infoG
mysql> select * from mysql.slave_master_infoG
*************************** 1. row ***************************
             Master_id: 2
       Number_of_lines: 23
       Master_log_name: masterbin.000002
        Master_log_pos: 614
                  Host: localhost
             User_name: rpl_user
(...)
master.info
was automatically
migrated!
Let's look at
the slave table.

slave> SHOW CREATE TABLE mysql.slave_relay_log_infoG
mysql> SHOW CREATE TABLE mysql.slave_relay_log_infoG
*************************** 1. row ***************************
       Table: slave_relay_log_info
Create Table: CREATE TABLE `slave_relay_log_info` (
  `Master_id` int(10) unsigned NOT NULL,
  `Number_of_lines` int(10) unsigned NOT NULL (...)
  `Relay_log_name` text CHARACTER SET utf8 (...)
  `Relay_log_pos` bigint(20) unsigned NOT NULL (...)
  `Master_log_name` text CHARACTER SET utf8 COLLATE utf8_bin NOT NULL (...)
  `Master_log_pos` bigint(20) unsigned NOT NULL (...)
  `Sql_delay` int(11) NOT NULL (...)
  `Number_of_workers` int(10) unsigned NOT NULL,
  PRIMARY KEY (`Master_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Relay Log Information'
InnoDB engine by
default.
What's the table
schema?

On-line Data Verification:
Replication Event
Checksums

Replication Event Checksums
 MySQL 5.6 Replication adds replication events checksums.
 Each event is appended a 32 bit cyclic redundancy check (CRC32).
 Master generates CRC32 when writing events to the binary log.
 Multiple verifications points (on the master and/or on the slave).
– Can be turned on/off independently.
Cyclic Redundancy Checks Per Events.
C
R
C
Binary Log FileBinary Log File
C
R
C
C
R
C
C
R
C
C
R
C
C
R
C
C
R
C
C
R
C
One CRC per Event.

 --binlog-checksum = CRC32 | NONE
– Turns on/off generation of CRCs on the master.
– SET @@global.binlog_checksum
●
--master-verify-checksum=0 | 1
– Dump thread and user sessions verify checksums.
– SET @@global.master_verify_checksum
●
--slave-sql-verify-checksum=0 | 1
– SQL thread verifies checksums.
– SET @@global.slave_sql_verify_checksums
User Interface for Controlling CRC.

Configuring the Master to Generate Checksums.
[mysqld]
(...)
binlogchecksum=CRC32
Start the
master server.
Shutdown
the master
server.
Configure master
to generate
event checksums.
This is actually the default currently.

Inspecting the Replication Event Checksums.
Let's insert
another row into
test.t1.
$ mysql5.6/bin/mysqlbinlog master/masterbin.000002
# at 120
#120926 12:26:24 server id 1  end_log_pos 199 CRC32 0x9a914962 Query thread_id=1 exec_time=0 error_code=0
(...)
BEGIN
/*!*/;
# at 199
#120926 12:26:24 server id 1  end_log_pos 298 CRC32 0x4de04ef5 Query thread_id=1 exec_time=0 error_code=0
use test/*!*/;
SET TIMESTAMP=1348658784/*!*/;
INSERT INTO t1 VALUES (3)
/*!*/;
# at 298
#120926 12:26:24 server id 1  end_log_pos 329 CRC32 0xe4f8efc1 Xid = 20
COMMIT/*!*/;
Valid CRCs.

Tuning and Optimizing
Row-based Replication

Row-based Replication Enhancements
 RBR works with partial rows in addition to full rows.
– Before Image – only fields required to find the row.
– After Image – only fields that actually changed.
 May be also configured to exclude BLOBS only (when not needed).
 Reduces memory footprint
 Reduces network bandwidth usage
 Reduces binary log size.
 SET binlog_row_image={MINIMAL|NOBLOB|FULL} – default is FULL
Optimized Row-based Replication Logging.

Before Image After Image
Full Rows
Rows without Blobs
Minimal Rows
Blob
Primary
Key
Changed
Columns

Using Full Rows.
master> FLUSH LOGS;
master> SET binlog_format=ROW;
master> CREATE TABLE t2 (int1 INT, int2 int,
float float, txt text, PRIMARY KEY (a));
master> INSERT INTO t2 VALUES (1, 1, 1.0, 'ola');
master> UPDATE t2 SET int2=2;
Create a new
binlog file.
Need
Row
Format.
Update
the table.

Using Full Rows.
$ mysql5.6/bin/mysqlbinlog v master/masterbin.000003
# at 242
#120926 12:55:17 server id 1  end_log_pos 664 CRC32 (...)
BINLOG '
Je1iUBMBAAAAMgAAAFACAAAAAEgAAAAAAAEABHRlc3QAAn (...)
(...)
'/*!*/;
### UPDATE test.t2
### WHERE
###   @1=1
###   @2=1
###   @3=1
###   @4='ola'
### SET
###   @1=1
###   @2=2
###   @3=1
###   @4='ola'
# at 314
The full rows are
written to the binary
log.

Using Rows Without Blobs.
master> FLUSH LOGS;
master> SET binlog_row_image=NOBLOB;
Create a new
binlog file.
Change to
NoBlob
images.

Using Rows Without Blobs.
# at 242
#120926 13:04:09 server id 1  end_log_pos 304 (...)
BINLOG '
Oe9iUBMBAAAAMgAAAPIAAAAAAEgAAAAAAAEABHRlc3QAAnQyAAQ (...)
(...)
### UPDATE test.t2
### WHERE
###   @1=1
###   @2=2
###   @3=1
### SET
###   @1=1
###   @2=3
###   @3=1
# at 304
The blob field was not
included in the before
and after images since
it was not needed for
the update operation.

Using Minimal Rows.
master> FLUSH LOGS;
master> SET binlog_row_image=MINIMAL;
Create a new
binlog file.
Change to
Minimal
images.

Using Minimal Rows.
# at 242
#120926 13:08:09 server id 1  end_log_pos 288 (...)
BINLOG '
KfBiUBMBAAAAMgAAAPIAAAAAAEgAAAAAAAEABHRlc3QAAn (...)
(...)
### UPDATE test.t2
### WHERE
###   @1=1
### SET
###   @2=4
# at 288
Only the PK was stored
in the binary log.
Only the changed field
was stored in the
binary log.

 The different types of images take:
– FULL: 314 – 242 = 72 bytes
– NOBLOB: 304 – 242 = 62 bytes
– MINIMAL: 288 – 242 = 46 bytes
 With only one row and a very simple update statement, very simple
table, we have saved (FULL vs MINIMAL):
– 72 – 46 = 26 bytes

 Log the original query along with the RBR events:
– Enhances auditing and debugging.
– Extra event preceding the Rows events.
– Replicated everywhere in the topology, together with the
Rows events themselves.
– User can turn the behavior on and off.
Informational Log Events for Row-based Replication.

Enabling the logging of the Rows_query_log_event.
master> SET binlog_format=ROW;
master> SET binlog_rows_query_log_events=ON;
master> use test;
master> INSERT INTO t1 VALUES (6);
master> UPDATE t1 SET a=7;
master> SHOW BINLOG EVENTS IN 'masterbin.000005';
Turn on
rows query log events.

Testing the logging of the Rows_query_log_event.
+++++++
+++++++
| masterbin.000009 | 192 | Rows_query  |         1 |         241 | # INSERT INTO t1 VALUES (6)             |
| masterbin.000009 | 286 | Write_rows  |         1 |         326 | table_id: 70 flags: STMT_END_F          |
| masterbin.000009 | 429 | Rows_query  |         1 |         470 | # UPDATE t1 SET a=7                     |
| masterbin.000009 | 515 | Update_rows |         1 |         591 | table_id: 70 flags: STMT_END_F          |
+++++++
Queries are logged
along with the
changes.

Testing the logging of the Rows_query_log_event.
(...)
# at 192
#120926 14:34:14 server id 1  end_log_pos 241 CRC32 0xddde516c Rows_query
# INSERT INTO t1 VALUES (6)
# at 241
#120926 14:34:14 server id 1  end_log_pos 286 CRC32 0xb10b6695 Table_map: `test`.`t1` mapped to number 70
(...)
# at 429
#120926 14:34:21 server id 1  end_log_pos 470 CRC32 0xd3a90e9b Rows_query
# UPDATE t1 SET a=7
# at 470
#120926 14:34:21 server id 1  end_log_pos 515 CRC32 0xdd3c50a9 Table_map: `test`.`t1` mapped to number 70
(...)
$ mysql5.6/bin/mysqlbinlog vv master/masterbin.000005
Queries are logged
along with the
changes.
Extra verbose so that
mysqbinlog outputs the
Rows_query_log_event.

Improving the Slave
Scalability: Multi-Threaded
Slave Applier

Multi-Threaded Slave
 Multiple execution threads to apply replication events to the slave(s).
 Splits processing between worker threads based on schema.
– Multi-tenant systems friendly.
 Concurrent transactions commit independently.
– Intra-schema consistency is always guaranteed.
– Inter-schema eventual consistency.
Properties of the Multi-Threaded Slave.

Multi-threaded Slaves.
RELAY
LOG
CoordinatorCoordinator
ThreadThread
T1T2
DB2
DBn
DB1
IO ThreadIO Thread WorkerWorker
ThreadsThreads
Transaction apply path at the slave.Transaction apply path at the slave.

Setting up a load that will use two workers.
slave> STOP SLAVE;
slave> SET GLOBAL slave_parallel_workers=2;
master> SET binlog_format=STATEMENT;
master> CREATE DATABASE db1;
master> CREATE DATABASE db2;
master> CREATE TABLE db1.t1 (a INT);
master> CREATE TABLE db2.t1 (a INT);
master> INSERT INTO db1.t1 SELECT sleep(5);
master> INSERT INTO db2.t1 SELECT sleep(5);
Not a requirement
for MTS, but needed
for this example.
Set 2
worker
threads.
Use sleep to make
the slave threads apply
concurrently.
Create two
databases.

Setting up a load that will use two workers.
slave> SELECT * FROM mysql.slave_worker_infoG
mysql> select * from mysql.slave_worker_infoG
Empty set (0.00 sec)
slave> START SLAVE; wait 56 seconds
slave> SELECT * FROM mysql.slave_worker_infoG
Positioning table for
the slave threads.

mysql> select * from mysql.slave_worker_infoG
*************************** 1. row ***************************
                 Master_id: 2
                 Worker_id: 0
            Relay_log_name: ./slaverelaybin.000017
             Relay_log_pos: 1098
            Master_log_pos: 1159
(...)
          Checkpoint_seqno: 5
     Checkpoint_group_size: 64
   Checkpoint_group_bitmap: (
*************************** 2. row ***************************
                 Master_id: 2
                 Worker_id: 1
            Relay_log_name: ./slaverelaybin.000017
             Relay_log_pos: 882
            Master_log_pos: 943
(...)
     Checkpoint_group_size: 64
   Checkpoint_group_bitmap:
 Even though these tables are
not for monitoring, we can
see two entries filled,
meaning that two slave
workers were engaged.
 Since they commit
independently, there is a
bitmap saved to pinpoint
which positions have been
applied by each worker.

Automatic Fail-over and
Replication Positioning:
Global Transaction Identifiers

Motivation: Seamless Fail-over.
 Servers sometimes crash (hardware error, bug, meteor hit...)
 Promote a slave to replace crashed master
 Method to represent positions in replication stream is CRUCIAL!
– Traditional: FILENAME+OFFSET
●
local, absolute, manual
– We introduce TRANSACTION IDENTIFIERS
●
global, logical, automatic

A
(master)
B
(slave)
C
(slave2)
 Enable fail-over if master crashes

A
(master)
B
(slave)
C
(slave2)
Crash!

A
(master,
crashed)
B
(slave)
C
(slave2)

A
(master,
crashed)
B
(slave)
C
(slave2)
fail-over

Global Transaction Identifiers.
 Generate a global transaction identifier on commit:
– server_uuid:number
a61678ba488942799e5845ba840af334:1
– server_uuid identifies the server; globally unique
– number is incremented by 1 for each transaction on this server

a61678ba488942799e5845ba840af334:1
 Write GTID to binary log
GTID BEGIN ...Ev1 Ev2 COMMIT
Transaction 1
GTID BEGIN ...Ev1 Ev2 COMMIT
Transaction 2

a61678ba488942799e5845ba840af334:1
 Write GTID to binary log
 Preserve GTID when slave re-executes transaction

New protocol.
 New replication protocol:
– Slave sends to master:
range of identifiers of executed transactions to master
– Master sends all other transactions to slave

New protocol.
(slave)
id1,trx1,
id2,trx2
binlog
(master)
A
id1,trx1,
id2,trx2,
id3,trx3
binlog
B

New protocol.
(slave)
id1,trx1,
id2,trx2
(master)
id1,trx1,
id2,trx2,
id3,trx3
binlog
A 2. id3, trx3, …
1. id1…id2
binlog
B

New protocol used in failover.
(slave)
id1,trx1,
id2,trx2,
id3,trx3
B
binlog
A
(crashed)(master)
A
id1,trx1,
id2,trx2,
id3,trx3
binlog
(slave)
C
id1,trx1,
id2,trx2
binlog
 Failover using new protocol

A
(crashed)(master)
A
id1,trx1,
id2,trx2,
id3,trx3
binlog
(slave)
C
id1,trx1,
id2,trx2
binlog
Crash! (slave)
id1,trx1,
id2,trx2,
id3,trx3
B
binlog

A
(crashed)(crashed)
A
id1,trx1,
id2,trx2,
id3,trx3
binlog
C
id1,trx1,
id2,trx2
binlog
id1,trx1,
id2,trx2,
id3,trx3
B
binlog

(new master)
id1,trx1,
id2,trx2,
id3,trx3
B
binlog
A
(crashed)(crashed)
A
id1,trx1,
id2,trx2,
id3,trx3
binlog
(slave)
id1,id2
id3, trx3,...
C
id1,trx1,
id2,trx2
binlog

Configure the master and slave to use Global Transaction Identifiers.
Shutdown
the master
server.
Shutdown
the slave
server.
master> RESET MASTER;
slave> STOP SLAVE;
slave> RESET SLAVE;
Clear non-GTID state
from binary logs
Clear non-GTID state
from relay logs

[mysqld]
(...)
gtidmode=on
enforcegtidconstistency
logslaveupdates
configure master to use GTIDs
[mysqld]
(...)
gtidmode=on
enforcegtidconstistency
logslaveupdates
logbin=slavebin
configure slave to use GTIDs
GTIDs ON only
works if binary
logging is ON.
Master and slave must either
both have GTIDs ON,
or both have GTIDs OFF

Start
the master
server.
Start
the slave
server.

slave> CHANGE MASTER TO MASTER_AUTO_POSITION = 1,
slave> START SLAVE;
Start the slave threads.
Tell the slave threads to use
GTIDs for positioning.

Try it out.
master> CREATE TABLE t3 (a INT);
master> SELECT @@GLOBAL.GTID_EXECUTED;
mysql> SELECT @@GLOBAL.GTID_EXECUTED;
++
| @@GLOBAL.GTID_EXECUTED |
++
| 8D4B6B8509AE11E2BCA8002318251294:12 |
++
Create and populate
a table.
Create and populate
a table.
See what GTIDs
were assigned
to the transactions.
slave> SELECT @@GLOBAL.GTID_EXECUTED;
++
++
| 8D4B6B8509AE11E2BCA8002318251294:12 |
++
The same GTIDs
have been applied
on the slave.

Try it out – GTIDs in master's binary log.
master> SHOW BINLOG EVENTS IN 'masterbin.000002';
+++++++
+++++++
| masterbin.000002 | 120 | Previous_gtids |         1 |         151 |                                         |
| masterbin.000002 | 151 | Gtid           |         1 |         199 | SET @@SESSION.GTID_NEXT= '[uuid]:1'     |
| masterbin.000002 | 296 | Gtid           |         1 |         344 | SET @@SESSION.GTID_NEXT= '[uuid]:2'     |
+++++++
8 rows in set (0,00 sec)

Try it out – GTIDs in slave's binary log.
slave> SHOW BINLOG EVENTS IN 'slavebin.000001';
+++++++
+++++++
| slavebin.000001 |   4 | Format_desc    |         1 |         120 | Server ver: 5.6.6m9log, Binlog ver: 4 |
| slavebin.000001 | 120 | Previous_gtids |         1 |         151 |                                         |
| slavebin.000001 | 151 | Gtid           |         1 |         199 | SET @@SESSION.GTID_NEXT= '[uuid]:1'     |
| slavebin.000001 | 199 | Query          |         1 |         296 | use `test`; CREATE TABLE t3 (a INT)     |
| slavebin.000001 | 296 | Gtid           |         1 |         344 | SET @@SESSION.GTID_NEXT= '[uuid]:2'     |
| slavebin.000001 | 344 | Query          |         1 |         423 | BEGIN                                   |
| slavebin.000001 | 423 | Query          |         1 |         522 | use `test`; INSERT INTO t3 VALUES (1)   |
| slavebin.000001 | 522 | Xid            |         1 |         553 | COMMIT /* xid=17 */                     |
+++++++
8 rows in set (0,00 sec)

Set Up A Second Slave.
$ mysql5.6/scripts/mysql_install_db
basedir=$HOME/hol9737/mysql5.6
datadir=$HOME/hol9737/slave2 user=`whoami`
Create
new Data
Directory
[mysqld]
serverid=3
relaylog=slave2relaybin
logerror=slave2.err
port=12002
gtidmode=on
enforcegtidconsistency
logslaveupdates
logbin=slavebin
$HOME/hol9737/slave2.cnf

Set Up a Second Slave.
  defaultsfile=$HOME/hol9737/slave2.cnf
  datadir=$HOME/hol9737/slave2/
$ mysql5.6/bin/mysql u root port 12002 –protocol=tcp
  prompt='slave2> '
Start the
slave2 server.
Log in to the
slave2 server.

Set Up a Second Slave.
slave2> CHANGE MASTER TO
  MASTER_AUTO_POSITION = 1,
Point slave2
to the master
server.
Use the credentials
we had granted
before.
slave2> START SLAVE;
Start the slave
threads.
Setup slave2
to use GTIDs for
positioning

Try out the second slave.
slave2> SHOW BINLOG EVENTS IN 'slavebin.000001';
+++++++
+++++++
| slave2bin.000001 |   4 | Format_desc    |         1 |         120 | Server ver: 5.6.6m9log, Binlog ver: 4 |
| slave2bin.000001 | 120 | Previous_gtids |         1 |         151 |                                         |
| slave2bin.000001 | 151 | Gtid           |         1 |         199 | SET @@SESSION.GTID_NEXT= '[uuid]:1'     |
| slave2bin.000001 | 199 | Query          |         1 |         296 | use `test`; CREATE TABLE t3 (a INT)     |
| slave2bin.000001 | 296 | Gtid           |         1 |         344 | SET @@SESSION.GTID_NEXT= '[uuid]:2'     |
| slave2bin.000001 | 344 | Query          |         1 |         423 | BEGIN                                   |
| slave2bin.000001 | 423 | Query          |         1 |         522 | use `test`; INSERT INTO t3 VALUES (1)   |
| slave2bin.000001 | 522 | Xid            |         1 |         553 | COMMIT /* xid=17 */                     |
+++++++
8 rows in set (0,00 sec) The new slave
has replicated from
the master

Recap: we want to simulate master crash and fail-over when one slave is behind.
slave
id1,trx1,
id2,trx2,
id3,trx3
binlog
A
(crashed)master
id1,trx1,
id2,trx2,
id3,trx3
binlog
slave2
id1,trx1
id2,trx2
binlog
Crash!
 The situation we want to handle

slave
id1,trx1,
id2,trx2,
id3,trx3
binlog
A
(crashed)master
id1,trx1,
id2,trx2,
id3,trx3
binlog
slave2
id1,trx1
id2,trx2
binlog
failover

slave
id1,trx1,
id2,trx2,
id3,trx3
binlog
A
(crashed)master
id1,trx1,
id2,trx2,
id3,trx3
binlog
slave2
id1,trx1
id2,trx2
binlog
fail-over

Make slave2 fall behind.
slave2> STOP SLAVE;
Stop the slave threads to ensure
that slave2 does not receive updates
master> SELECT @@GLOBAL.GTID_EXECUTED;
slave> SELECT @@GLOBAL.GTID_EXECUTED;
slave2> SELECT @@GLOBAL.GTID_EXECUTED;
++
++
| [uuid]:13             |
++
++
++
| [uuid]:13             |
++
++
++
| [uuid]:12             |
++
Execute transaction on master
to make slave2 fall behind.

Kill the master.
$ pkill 9 f mysqld.*master
Kill the
master server.
master> SHOW TABLES;
The master
is dead
ERROR 2013 (HY000): Lost connection to MySQL server during query

Failover: make 'slave' a master of 'slave2'.
slave2> CHANGE MASTER TO
Fail-over, i.e.,
switch to use 'slave'
as the master.
slave2> STOP SLAVE;
Stop slave threads.
slave2> START SLAVE; Start slave threads.

Fail-over worked by just pointing the slave to another server!
slave2> SELECT @@GLOBAL.GTID_EXECUTED;
++
++
| 8D4B6B8509AE11E2BCA8002318251294:13 |
++
'slave' is up to date now.
The last transaction
was copied from 'slave'slave2> SELECT * FROM test.t3;
slave2> SELECT * FROM test.t3;
++
| a    |
++
|    1 |
|   99 |
++
'slave2' is up to date now.
The last transaction
was copied from 'slave'

Summary

Summary
 This HOL showcased some MySQL Replication best practices
 Several features from MySQL 5.6 that have direct user visibility were highlighted:
– Multi-threaded slave
– Global Transaction Identifiers
– Optimized Row-based Replication
– Transactional Replication Positions
– Replication Event Checksums
 In the end the attendee was able to get a glimpse of some of the new MySQL 5.6 features.
– But there is much more to explore.

Oracle OpenWorld 2013 - HOL9737 MySQL Replication Best Practices

More Related Content

What's hot (20)

Similar to Oracle OpenWorld 2013 - HOL9737 MySQL Replication Best Practices (20)

Recently uploaded (20)

Oracle OpenWorld 2013 - HOL9737 MySQL Replication Best Practices