Oracle OpenWorld 2013 - HOL9737 MySQL Replication Best Practices

transcript

MySQL ReplicationBest Practices

Luís Soares Sr. Software Engineer

Sven Sandberg Sr. Software Engineer

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.

The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Agenda

Replication Basics

Crash-Safe Slaves: Replication Metadata in System Tables.

On-line Data Verification: Replication Event Checksums

Tuning and Optimizing Row-based Replication

Improving the Slave Scalability: Multi-Threaded Slave Applier

Automated Fail-over and Slave Positioning: Global Transaction Ids

Summary

Replication Basics

MySQL Master Server– Changes data

– Logs changes (Events) into a file (Binary Log)

MySQL Slave Server

– Retrieves events from the master

– Replays the events on the slave's databases

MySQL Replication Components.

Replication Basics

The Binary Log– File based log that records the changes on the master.

– Statement or Row based format (may be intermixed).

– Split into transactional groups.

BEGIN ...Ev1 Ev2 COMMIT

MySQL Replication Components: Binary Log.

Binary Log FileBinary Log File

BEGIN ...Ev1 Ev2 COMMIT

Event Layout ona Binary Log File

Replication BasicsMySQL Replication Components: Binary Log.

Binary Log

Binarylogfiles

Under the Hood

Binary log files: mysql-bin.000001, mysql-bin.000002, …

- The actual data.

Index: mysql-bin.index

- An index over the binary log files.

Log coordinate:

- binlog file name + event offset in the file (3.23.15+)

- Global Transaction Identifiers (5.6+)

Replication BasicsMySQL Replication Architecture.

Insert...

binary log

Insert...

relay log

Insert...

binary log

ClientDumpthread

IOthread

SQLthread

Network

Masterinfo

RelayLogInfo

ReplicationMetadata

Replication Basics

Asynchronous Replication (MySQL 3.23.15+)– Transactions are committed and externalized without interaction with

the replication layer.

– Events are sent to slave after the commit operation is acknowledged.

– Faster but vulnerable to lost updates on server crashes and inconsistency.

– Built into the server.

Semi-synchronous Replication (MySQL 5.5+)

– Master commits transaction but waits until one slave acknowledges having received and stored the event before replying to the client.

Changes Propagation.

Replication Basics

Hands-On

Replication BasicsGetting MySQL.

Head to

– http://dev.mysql.com

... and get the latest MySQL 5.6 generic tar.gz package from there

… or alternatively, you can find MySQL 5.6 linux generic package at your local directory /usr/local/mysql56 .

Replication BasicsSetting Up Working Directories.

$ tar xfvz /usr/local/mysql....tar.gz directory=$HOME/hol9737

$ mkdir $HOME/hol9737/$ cd $HOME/hol9737/

Create a directoryfor storing the binaries (e.g., under your home).

Unpack the package (e.g., using tar).

$ mv $HOME/hol9737/mysql... $HOME/hol9737/mysql56

Rename the directory for simplicity

Replication BasicsSetting Up MySQL.

$ mysql56/scripts/mysql_install_db basedir=$HOME/hol9737/mysql56 datadir=$HOME/hol9737/master user=`whoami`

CreateTwo Data

Directories

$ mysql56/scripts/mysql_install_db basedir=$HOME/hol9737/mysql56 datadir=$HOME/hol9737/slave user=`whoami`

Createtwo Data

Directories

Replication BasicsSetting Up MySQL Master Server.

[mysqld]serverid=1logbin=masterbinlogerror=master.errport=12000

Create a defaults file called $HOME/hol9737/master.cnf

Set the unique server id.

Turn ON the binary log.

Replication Basics

$ mysql56/bin/mysqld defaultsfile=$HOME/hol9737/master.cnf lcmessagesdir=$HOME/hol9737/mysql56/share datadir=$HOME/hol9737/master/

Start themaster server.

$ mysql5.6/bin/mysql u root port 12000 protocol=tcp prompt='master> '

Log in to themaster server.

Setting Up MySQL Master Server.

Replication BasicsInspecting the Master Status.

master> SHOW BINARY LOGS;... master> SHOW BINLOG EVENTS;... master> SHOW MASTER STATUS;

What binlog files are in use?

What's in the first binary log?

What binary log is in use what is its position?

master> SHOW BINARY LOGS;

master> SHOW BINLOG EVENTS;

master> SHOW MASTER STATUS;

Replication BasicsInspecting the Master Status.

+++| Log_name | File_size |+++| masterbin.000001 | 120 |+++1 row in set (0.00 sec)

Replication BasicsThe Binary Log Layout.

Log Events

Format description

Rotate

Log Events

Format description

Rotate

Log Events

Format description ● Multiple Files.

● Files begin with Format Description event.

● Each log file ends with a Rotate event.

Replication BasicsSetting Up Replication User (that the slave will connect with).

master> CREATE USER `rpl_user`@`localhost`;master> GRANT REPLICATION SLAVE ON *.* TO `rpl_user`@`localhost` IDENTIFIED BY 'secret';master> FLUSH PRIVILEGES;

We could useany user.

Needs replicationgrants to read any change

on the master

Better use a dedicated

user for connectingthe slave.

Replication BasicsSetting Up MySQL Slave Server.

[mysqld]serverid=2relaylog=slaverelaybinlogerror=slave.errport=12001

Set the unique server id.

Set the name for the relay log.

Create a defaults file called $HOME/hol9737/slave.cnf

Replication BasicsStarting Up The MySQL Slave Server.

$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/slave.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/slave/

Start theslave server.

$ mysql5.6/bin/mysql u root port 12001 –protocol=tcp prompt='slave> '

Log in to theslave server.

Replication BasicsStarting the Slave Threads.

slave> CHANGE MASTER TO MASTER_HOST = 'localhost', MASTER_PORT = 12000, MASTER_USER = 'rpl_user', MASTER_PASSWORD = 'secret';

Point the slave serverto the master

server.

Use the credentialswe had granted

before.

slave> START SLAVE;Start the slave

threads.

Replication BasicsInspecting the Slave Status.

slave> SHOW SLAVE STATUS;

Inspect the slave status.

(...) Slave_IO_State: Waiting for master to send event Master_Host: localhost Master_User: rpl_user Master_Port: 12000 Connect_Retry: 60 Master_Log_File: masterbin.000002 Read_Master_Log_Pos: 120 Relay_Log_File: slaverelaybin.000003 Relay_Log_Pos: 284 Relay_Master_Log_File: masterbin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes(...)

Both slave threads are up and running!

Replication BasicsReplicating From Master to Slave.

master> USE test;master> CREATE TABLE t1 (a INT); slave> USE test; slave> SHOW TABLES;master> INSERT INTO t1 VALUES(1); slave> SELECT * FROM t1;

Use thetest db.

Create a table.

Slave has replicated the table.

Slave has replicated the data.

Replication BasicsWhat's in the Binary Log?

master> SHOW BINLOG EVENTS [IN 'masterbin.0000002'];

+++++++| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |+++++++| masterbin.000002 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 || masterbin.000002 | 120 | Query | 1 | 217 | use `test`; CREATE TABLE t1 (a INT) || masterbin.000002 | 217 | Query | 1 | 296 | BEGIN || masterbin.000002 | 296 | Query | 1 | 395 | use `test`; INSERT INTO t1 VALUES (1) || masterbin.000002 | 395 | Xid | 1 | 426 | COMMIT /* xid=21 */ |+++++++5 rows in set (0.00 sec)

The “CREATE TABLE” statement.

The “INSERT” statement.

Replication BasicsReplicating From Master to Slave – binary log formats.

• Three formats:– STATEMENT – every change logged as a statement.

● Re-executed on the slave.

– ROW – every change logged as data.

● Data changes are applied on the slave.

– MIXED – every change logged either as data or statements.

● Automatically switches from statement to row on non-deterministic statements.

Replication BasicsReplicating From Master to Slave – binary log formats.

master> set binlog_format=ROW;master> INSERT INTO t1 VALUES(2);master> SHOW BINLOG EVENTS;

+++++++| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |+++++++| masterbin.000002 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 || masterbin.000002 | 120 | Query | 1 | 217 | use `test`; CREATE TABLE t1 (a INT) || masterbin.000002 | 217 | Query | 1 | 296 | BEGIN || masterbin.000002 | 296 | Query | 1 | 395 | use `test`; INSERT INTO t1 VALUES (1) || masterbin.000002 | 395 | Xid | 1 | 426 | COMMIT /* xid=21 */ || masterbin.000002 | 426 | Query | 1 | 498 | BEGIN || masterbin.000002 | 498 | Table_map | 1 | 543 | table_id: 70 (test.t1) || masterbin.000002 | 543 | Write_rows | 1 | 583 | table_id: 70 flags: STMT_END_F || masterbin.000002 | 583 | Xid | 1 | 614 | COMMIT /* xid=27 */ |+++++++9 rows in set (0.00 sec)

Let's changethe format.

The second “INSERT” statement.

Replication BasicsMaster replication files.

$ ls la master/

(...)rwrw 1 XXXXXX XXXXXX 143 Sep 26 11:10 masterbin.000001rwrw 1 XXXXXX XXXXXX 614 Sep 26 11:40 masterbin.000002rwrw 1 XXXXXX XXXXXX 40 Sep 26 11:10 masterbin.index(...)

The binary log files.

Index file over the existing binary log files.

Replication BasicsSlave replication files.

$ ls la slave/

(...) rwrw 1 XXXXXXX XXXXXXX 128 Sep 26 11:40 master.info(...) rwrw 1 XXXXXXX XXXXXXX 57 Sep 26 11:40 relaylog.info(...)rwrw 1 XXXXXXX XXXXXXX 337 Sep 26 11:10 slaverelaybin.000002rwrw 1 XXXXXXX XXXXXXX 778 Sep 26 11:40 slaverelaybin.000003rwrw 1 XXXXXXX XXXXXXX 50 Sep 26 11:10 slaverelaybin.index(...)

The relay log files.

Index file over the existing binary log files.

Persists IO thread repliaction metadata (master host, username, … and positioning on the master's binlog).

Persists SQL thread replication metadata. Basically, the positioning in the relay log.

Crash-Safe Positioning:

Storing Replication Metadata in Tables.

Store replication metadata in tables, within the context of regular transactions.

– Commit both data and replication positions together.

– Rollback both data and replication positions together.

– Robust, highly available setups.

– No metadata in files (master.info and relay-log.info).

– Slave knows a valid position to resume replication after a crash.

Often referred to as the “Crash-Safe Slave” or “Transactional Replication” feature.

Non-transactional engines will not work with this for obvious reasons.

Self-healing slaves.MySQL 5.6

Storing Replication Metadata in Tables.Configuring the slave to use tables as metadata repository.

$ mysql5.6/bin/mysqladmin shutdown user=root port=12001 protocol=tcp

[mysqld](...) relayloginforepository=TABLEmasterinforepository=TABLE

Start theslave server.

Shutdownthe slaveserver.

Configure slaveto use tables,

instead of files.

Storing Replication Metadata in Tables.Inspecting the filesystem and the tables.

$ ls la slave/

master.infoand

relay-log.infoare gone!!

slave> SELECT * FROM mysql.slave_relay_log_info;

relay-log.infowas automatically

migrated!

Let's look atthe relay infoslave table.

(if you see errors likeERROR 2013 (HY000): Lost connection to MySQL server during query- don't worry and just try again)

slave> SELECT * FROM mysql.slave_master_info\G

mysql> select * from mysql.slave_master_info\G*************************** 1. row *************************** Master_id: 2 Number_of_lines: 23 Master_log_name: masterbin.000002 Master_log_pos: 614 Host: localhost User_name: rpl_user(...)

master.infowas automatically

migrated!

Let's look atthe slave table.

slave> SHOW CREATE TABLE mysql.slave_relay_log_info\G

mysql> SHOW CREATE TABLE mysql.slave_relay_log_info\G*************************** 1. row *************************** Table: slave_relay_log_infoCreate Table: CREATE TABLE `slave_relay_log_info` ( `Master_id` int(10) unsigned NOT NULL, `Number_of_lines` int(10) unsigned NOT NULL (...) `Relay_log_name` text CHARACTER SET utf8 (...) `Relay_log_pos` bigint(20) unsigned NOT NULL (...) `Master_log_name` text CHARACTER SET utf8 COLLATE utf8_bin NOT NULL (...) `Master_log_pos` bigint(20) unsigned NOT NULL (...) `Sql_delay` int(11) NOT NULL (...) `Number_of_workers` int(10) unsigned NOT NULL, PRIMARY KEY (`Master_id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Relay Log Information'1 row in set (0.00 sec)

InnoDB engine by default.

What's the table schema?

On-line Data Verification:

Replication Event Checksums

MySQL 5.6 Replication adds replication events checksums. Each event is appended a 32 bit cyclic redundancy check (CRC32). Master generates CRC32 when writing events to the binary log. Multiple verifications points (on the master and/or on the slave).

– Can be turned on/off independently.

Cyclic Redundancy Checks Per Events.

BEGIN ...Ev1 Ev2 COMMITCRC

Binary Log FileBinary Log File

BEGIN ...Ev1 Ev2 COMMITCRC

One CRC per Event.

--binlog-checksum = CRC32 | NONE– Turns on/off generation of CRCs on the master.

– SET @@global.binlog_checksum

● --master-verify-checksum=0 | 1

– Dump thread and user sessions verify checksums.

– SET @@global.master_verify_checksum

● --slave-sql-verify-checksum=0 | 1

– SQL thread verifies checksums.

– SET @@global.slave_sql_verify_checksums

User Interface for Controlling CRC.

Replication Event ChecksumsConfiguring the Master to Generate Checksums.

[mysqld](...) binlogchecksum=CRC32

$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/master.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/master/

Start themaster server.

Shutdownthe master

server.

Configure masterto generate

event checksums.

This is actually the default currently.

Replication Event ChecksumsInspecting the Replication Event Checksums.

master> INSERT INTO t1 VALUES(3);

Let's insertanother row into

test.t1.

$ mysql5.6/bin/mysqlbinlog master/masterbin.000002

# at 120#120926 12:26:24 server id 1 end_log_pos 199 CRC32 0x9a914962 Query thread_id=1 exec_time=0 error_code=0(...)BEGIN/*!*/;# at 199#120926 12:26:24 server id 1 end_log_pos 298 CRC32 0x4de04ef5 Query thread_id=1 exec_time=0 error_code=0use test/*!*/;SET TIMESTAMP=1348658784/*!*/;INSERT INTO t1 VALUES (3)/*!*/;# at 298#120926 12:26:24 server id 1 end_log_pos 329 CRC32 0xe4f8efc1 Xid = 20COMMIT/*!*/;

Valid CRCs.

Tuning and Optimizing Row-based Replication

Row-based Replication Enhancements

RBR works with partial rows in addition to full rows.– Before Image – only fields required to find the row.

– After Image – only fields that actually changed.

May be also configured to exclude BLOBS only (when not needed).

Reduces memory footprint

Reduces network bandwidth usage

Reduces binary log size.

SET binlog_row_image={MINIMAL|NOBLOB|FULL} – default is FULL

Optimized Row-based Replication Logging.

Row-based Replication EnhancementsOptimized Row-based Replication Logging.

Before Image After Image

Full Rows

Rows without Blobs

Minimal Rows

PrimaryKey

Changed Columns

Row-based Replication EnhancementsUsing Full Rows.

master> FLUSH LOGS;master> SET binlog_format=ROW;master> CREATE TABLE t2 (int1 INT, int2 int, float float, txt text, PRIMARY KEY (a));master> INSERT INTO t2 VALUES (1, 1, 1.0, 'ola');master> UPDATE t2 SET int2=2;

Create a new binlog file.

NeedRow

Format.

Updatethe table.

Row-based Replication EnhancementsUsing Full Rows.

$ mysql5.6/bin/mysqlbinlog v master/masterbin.000003

# at 242#120926 12:55:17 server id 1 end_log_pos 664 CRC32 (...)

BINLOG 'Je1iUBMBAAAAMgAAAFACAAAAAEgAAAAAAAEABHRlc3QAAn (...)(...)'/*!*/;### UPDATE test.t2### WHERE### @1=1### @2=1### @3=1 ### @4='ola'### SET### @1=1### @2=2### @3=1 ### @4='ola'# at 314

The full rows are written to the binary log.

Row-based Replication EnhancementsUsing Rows Without Blobs.

master> FLUSH LOGS;master> SET binlog_row_image=NOBLOB;master> UPDATE t2 SET int2=3;

Change toNoBlob images.

Row-based Replication EnhancementsUsing Rows Without Blobs.

# at 242#120926 13:04:09 server id 1 end_log_pos 304 (...)

BINLOG 'Oe9iUBMBAAAAMgAAAPIAAAAAAEgAAAAAAAEABHRlc3QAAnQyAAQ (...)(...)### UPDATE test.t2### WHERE### @1=1### @2=2### @3=1 ### SET### @1=1### @2=3### @3=1 # at 304

The blob field was not included in the before and after images since it was not needed for the update operation.

Row-based Replication EnhancementsUsing Minimal Rows.

master> FLUSH LOGS;master> SET binlog_row_image=MINIMAL;master> UPDATE t2 SET int2=4;

Change toMinimal images.

Row-based Replication EnhancementsUsing Minimal Rows.

# at 242#120926 13:08:09 server id 1 end_log_pos 288 (...)

BINLOG 'KfBiUBMBAAAAMgAAAPIAAAAAAEgAAAAAAAEABHRlc3QAAn (...)(...)### UPDATE test.t2### WHERE### @1=1### SET### @2=4# at 288

Only the PK was stored in the binary log.

Only the changed field was stored in the binary log.

Row-based Replication EnhancementsOptimized Row-based Replication Logging.

The different types of images take:– FULL: 314 – 242 = 72 bytes

– NOBLOB: 304 – 242 = 62 bytes

– MINIMAL: 288 – 242 = 46 bytes

With only one row and a very simple update statement, very simple table, we have saved (FULL vs MINIMAL):

– 72 – 46 = 26 bytes

Row-based Replication Enhancements

Log the original query along with the RBR events:

– Enhances auditing and debugging.

– Extra event preceding the Rows events.

– Replicated everywhere in the topology, together with the Rows events themselves.

– User can turn the behavior on and off.

Informational Log Events for Row-based Replication.

Row-based Replication EnhancementsEnabling the logging of the Rows_query_log_event.

master> SET binlog_format=ROW;master> SET binlog_rows_query_log_events=ON;master> use test;master> INSERT INTO t1 VALUES (6);master> UPDATE t1 SET a=7;master> SHOW BINLOG EVENTS IN 'masterbin.000005';

Turn onrows query log events.

Row-based Replication EnhancementsTesting the logging of the Rows_query_log_event.

+++++++| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |+++++++| masterbin.000009 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 || masterbin.000009 | 120 | Query | 1 | 192 | BEGIN || masterbin.000009 | 192 | Rows_query | 1 | 241 | # INSERT INTO t1 VALUES (6) || masterbin.000009 | 241 | Table_map | 1 | 286 | table_id: 70 (test.t1) || masterbin.000009 | 286 | Write_rows | 1 | 326 | table_id: 70 flags: STMT_END_F || masterbin.000009 | 326 | Xid | 1 | 357 | COMMIT /* xid=81 */ || masterbin.000009 | 357 | Query | 1 | 429 | BEGIN || masterbin.000009 | 429 | Rows_query | 1 | 470 | # UPDATE t1 SET a=7 || masterbin.000009 | 470 | Table_map | 1 | 515 | table_id: 70 (test.t1) || masterbin.000009 | 515 | Update_rows | 1 | 591 | table_id: 70 flags: STMT_END_F || masterbin.000009 | 591 | Xid | 1 | 622 | COMMIT /* xid=82 */ |+++++++11 rows in set (0.01 sec)

Queries are logged along with the changes.

Row-based Replication EnhancementsTesting the logging of the Rows_query_log_event.

(...)# at 192#120926 14:34:14 server id 1 end_log_pos 241 CRC32 0xddde516c Rows_query# INSERT INTO t1 VALUES (6)# at 241#120926 14:34:14 server id 1 end_log_pos 286 CRC32 0xb10b6695 Table_map: `test`.`t1` mapped to number 70(...)# at 429#120926 14:34:21 server id 1 end_log_pos 470 CRC32 0xd3a90e9b Rows_query# UPDATE t1 SET a=7# at 470#120926 14:34:21 server id 1 end_log_pos 515 CRC32 0xdd3c50a9 Table_map: `test`.`t1` mapped to number 70(...)

$ mysql5.6/bin/mysqlbinlog vv master/masterbin.000005

Queries are logged along with the changes.

Extra verbose so that mysqbinlog outputs the Rows_query_log_event.

Improving the Slave Scalability: Multi-Threaded Slave Applier

Multi-Threaded Slave

Multiple execution threads to apply replication events to the slave(s). Splits processing between worker threads based on schema.

– Multi-tenant systems friendly.

Concurrent transactions commit independently.

– Intra-schema consistency is always guaranteed.

– Inter-schema eventual consistency.

Properties of the Multi-Threaded Slave.

Multi-Threaded SlaveMulti-threaded Slaves.

RELAYLOG

Coordinator Coordinator ThreadThread

IO ThreadIO Thread Worker Worker ThreadsThreads

Transaction apply path at the slave.Transaction apply path at the slave.

Multi-Threaded SlaveSetting up a load that will use two workers.

slave> STOP SLAVE;slave> SET GLOBAL slave_parallel_workers=2;master> SET binlog_format=STATEMENT;master> CREATE DATABASE db1;master> CREATE DATABASE db2;master> CREATE TABLE db1.t1 (a INT);master> CREATE TABLE db2.t1 (a INT);master> INSERT INTO db1.t1 SELECT sleep(5);master> INSERT INTO db2.t1 SELECT sleep(5);

Not a requirement for MTS, but needed

for this example.

Set 2worker threads.

Use sleep to makethe slave threads apply

concurrently.

Create twodatabases.

Multi-Threaded SlaveSetting up a load that will use two workers.

slave> SELECT * FROM mysql.slave_worker_info\G

mysql> select * from mysql.slave_worker_info\GEmpty set (0.00 sec)

slave> START SLAVE; wait 56 secondsslave> SELECT * FROM mysql.slave_worker_info\G

Positioning table for the slave threads.

Multi-Threaded Slave

mysql> select * from mysql.slave_worker_info\G*************************** 1. row *************************** Master_id: 2 Worker_id: 0 Relay_log_name: ./slaverelaybin.000017 Relay_log_pos: 1098 Master_log_name: masterbin.000008 Master_log_pos: 1159(...) Checkpoint_seqno: 5 Checkpoint_group_size: 64 Checkpoint_group_bitmap: ( *************************** 2. row *************************** Master_id: 2 Worker_id: 1 Relay_log_name: ./slaverelaybin.000017 Relay_log_pos: 882 Master_log_name: masterbin.000008 Master_log_pos: 943(...) Checkpoint_group_size: 64 Checkpoint_group_bitmap: 2 rows in set (0.00 sec)

Even though these tables are not for monitoring, we can see two entries filled, meaning that two slave workers were engaged.

Since they commit independently, there is a bitmap saved to pinpoint which positions have been applied by each worker.

Automatic Fail-over and Replication Positioning:Global Transaction Identifiers

Motivation: Seamless Fail-over.

Servers sometimes crash (hardware error, bug, meteor hit...) Promote a slave to replace crashed master Method to represent positions in replication stream is CRUCIAL!

– Traditional: FILENAME+OFFSET

● local, absolute, manual

– We introduce TRANSACTION IDENTIFIERS

● global, logical, automatic

Global Transaction Identifiers

(master)

(slave)

(slave2)

Global Transaction IdentifiersMotivation: Seamless Fail-over.

Enable fail-over if master crashes

(master)

(slave)

(slave2)

Crash!

(master,crashed)

(slave)

(slave2)

(master,crashed)

(slave)

(slave2)

fail-over

(master,crashed)

(slave)

(slave2)

fail-over

Global Transaction IdentifiersGlobal Transaction Identifiers.

Generate a global transaction identifier on commit:– server_uuid:number

a61678ba488942799e5845ba840af334:1– server_uuid identifies the server; globally unique

– number is incremented by 1 for each transaction on this server

Write GTID to binary log

GTID BEGIN ...Ev1 Ev2 COMMIT

Transaction 1

GTID BEGIN ...Ev1 Ev2 COMMIT

Transaction 2

Write GTID to binary log

Preserve GTID when slave re-executes transaction

Global Transaction IdentifiersNew protocol.

New replication protocol:– Slave sends to master:

range of identifiers of executed transactions to master

– Master sends all other transactions to slave

(slave)

id1,trx1,id2,trx2

binlog

(master)

Aid1,trx1,id2,trx2,id3,trx3

binlog

(slave)

id1,trx1,id2,trx2

(master)

id1,trx1,id2,trx2,id3,trx3

binlog

A 2. id3, trx3, …

1. id1…id2 binlog

Global Transaction IdentifiersNew protocol used in failover.

(slave)

binlog

(crashed)(master)

binlog

(slave)

Cid1,trx1,id2,trx2

binlog

Failover using new protocol

(crashed)(master)

binlog

(slave)

Cid1,trx1,id2,trx2

binlogCrash! (slave)

binlog

(crashed)(crashed)

binlog

Cid1,trx1,id2,trx2

binlog

(new master)

binlog

(crashed)(crashed)

binlog

(slave)

id1,id2

id3, trx3,...C

id1,trx1,id2,trx2

binlog

Global Transaction IdentifiersConfigure the master and slave to use Global Transaction Identifiers.

Shutdownthe master

server.

Shutdownthe slaveserver.

master> RESET MASTER;

slave> STOP SLAVE;slave> RESET SLAVE;

Clear non-GTID statefrom binary logs

Clear non-GTID statefrom relay logs

[mysqld](...) gtidmode=onenforcegtidconstistencylogslaveupdates

configure master to use GTIDs

[mysqld](...) gtidmode=onenforcegtidconstistencylogslaveupdateslogbin=slavebin

configure slave to use GTIDs

GTIDs ON only works if binary logging is ON.

Master and slave must eitherboth have GTIDs ON,

or both have GTIDs OFF

$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/master.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/master/

Startthe master

server.

Startthe slaveserver.

slave> CHANGE MASTER TO MASTER_AUTO_POSITION = 1, MASTER_HOST = 'localhost', MASTER_PORT = 12000, MASTER_USER = 'rpl_user', MASTER_PASSWORD = 'secret';slave> START SLAVE;

Start the slave threads.

Tell the slave threads to use GTIDs for positioning.

Global Transaction IdentifiersTry it out.

master> CREATE TABLE t3 (a INT);master> INSERT INTO t3 VALUES (1);master> SELECT @@GLOBAL.GTID_EXECUTED;

mysql> SELECT @@GLOBAL.GTID_EXECUTED;

++| @@GLOBAL.GTID_EXECUTED |++| 8D4B6B8509AE11E2BCA8002318251294:12 |

Create and populatea table.

See what GTIDswere assigned

to the transactions.

slave> SELECT @@GLOBAL.GTID_EXECUTED;mysql> SELECT @@GLOBAL.GTID_EXECUTED;

The same GTIDshave been applied

on the slave.

Global Transaction IdentifiersTry it out – GTIDs in master's binary log.

master> SHOW BINLOG EVENTS IN 'masterbin.000002';

+++++++

+++++++

| masterbin.000002 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 |

| masterbin.000002 | 120 | Previous_gtids | 1 | 151 | |

| masterbin.000002 | 151 | Gtid | 1 | 199 | SET @@SESSION.GTID_NEXT= '[uuid]:1' |

| masterbin.000002 | 199 | Query | 1 | 296 | use `test`; CREATE TABLE t3 (a INT) |

| masterbin.000002 | 296 | Gtid | 1 | 344 | SET @@SESSION.GTID_NEXT= '[uuid]:2' |

| masterbin.000002 | 344 | Query | 1 | 423 | BEGIN |

| masterbin.000002 | 423 | Query | 1 | 522 | use `test`; INSERT INTO t3 VALUES (1) |

| masterbin.000002 | 522 | Xid | 1 | 553 | COMMIT /* xid=17 */ |

+++++++

8 rows in set (0,00 sec)

Global Transaction IdentifiersTry it out – GTIDs in slave's binary log.

slave> SHOW BINLOG EVENTS IN 'slavebin.000001';

+++++++

| slavebin.000001 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 |

| slavebin.000001 | 120 | Previous_gtids | 1 | 151 | |

| slavebin.000001 | 151 | Gtid | 1 | 199 | SET @@SESSION.GTID_NEXT= '[uuid]:1' |

| slavebin.000001 | 199 | Query | 1 | 296 | use `test`; CREATE TABLE t3 (a INT) |

| slavebin.000001 | 296 | Gtid | 1 | 344 | SET @@SESSION.GTID_NEXT= '[uuid]:2' |

| slavebin.000001 | 344 | Query | 1 | 423 | BEGIN |

| slavebin.000001 | 423 | Query | 1 | 522 | use `test`; INSERT INTO t3 VALUES (1) |

| slavebin.000001 | 522 | Xid | 1 | 553 | COMMIT /* xid=17 */ |

+++++++

8 rows in set (0,00 sec)

Global Transaction IdentifiersSet Up A Second Slave.

$ mysql5.6/scripts/mysql_install_db basedir=$HOME/hol9737/mysql5.6 datadir=$HOME/hol9737/slave2 user=`whoami`

Createnew DataDirectory

[mysqld]serverid=3relaylog=slave2relaybinlogerror=slave2.errport=12002gtidmode=onenforcegtidconsistencylogslaveupdateslogbin=slavebin

Create a defaults file called $HOME/hol9737/slave2.cnf

Global Transaction IdentifiersSet Up a Second Slave.

$ mysql5.6/bin/mysqld defaultsfile=$HOME/hol9737/slave2.cnf lcmessagesdir=$HOME/hol9737/mysql5.6/share datadir=$HOME/hol9737/slave2/

$ mysql5.6/bin/mysql u root port 12002 –protocol=tcp prompt='slave2> '

Start theslave2 server.

Log in to theslave2 server.

Global Transaction IdentifiersSet Up a Second Slave.

slave2> CHANGE MASTER TO MASTER_AUTO_POSITION = 1, MASTER_HOST = 'localhost', MASTER_PORT = 12000, MASTER_USER = 'rpl_user', MASTER_PASSWORD = 'secret';

Point slave2to the master

server.

Use the credentialswe had granted

before.

slave2> START SLAVE;

Start the slavethreads.

Setup slave2to use GTIDs for

positioning

Global Transaction IdentifiersTry out the second slave.

slave2> SHOW BINLOG EVENTS IN 'slavebin.000001';

+++++++

| slave2bin.000001 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.6m9log, Binlog ver: 4 |

| slave2bin.000001 | 120 | Previous_gtids | 1 | 151 | |

| slave2bin.000001 | 151 | Gtid | 1 | 199 | SET @@SESSION.GTID_NEXT= '[uuid]:1' |

| slave2bin.000001 | 199 | Query | 1 | 296 | use `test`; CREATE TABLE t3 (a INT) |

| slave2bin.000001 | 296 | Gtid | 1 | 344 | SET @@SESSION.GTID_NEXT= '[uuid]:2' |

| slave2bin.000001 | 344 | Query | 1 | 423 | BEGIN |

| slave2bin.000001 | 423 | Query | 1 | 522 | use `test`; INSERT INTO t3 VALUES (1) |

| slave2bin.000001 | 522 | Xid | 1 | 553 | COMMIT /* xid=17 */ |

+++++++

8 rows in set (0,00 sec) The new slavehas replicated from

the master

Global Transaction IdentifiersRecap: we want to simulate master crash and fail-over when one slave is behind.

binlog

(crashed)master

binlog

slave2

id1,trx1 id2,trx2

binlog

Crash!

The situation we want to handle

binlog

(crashed)master

binlog

slave2

id1,trx1 id2,trx2

binlog

failover

binlog

(crashed)master

binlog

slave2

id1,trx1 id2,trx2

binlog

fail-over

Global Transaction IdentifiersMake slave2 fall behind.

slave2> STOP SLAVE;

Stop the slave threads to ensurethat slave2 does not receive updates

master> INSERT INTO t3 VALUES (99);master> SELECT @@GLOBAL.GTID_EXECUTED; slave> SELECT @@GLOBAL.GTID_EXECUTED;slave2> SELECT @@GLOBAL.GTID_EXECUTED;

++| @@GLOBAL.GTID_EXECUTED |++| [uuid]:13 |

++| @@GLOBAL.GTID_EXECUTED |++| [uuid]:12 |

Execute transaction on masterto make slave2 fall behind.

Global Transaction IdentifiersKill the master.

$ pkill 9 f mysqld.*master

Kill themaster server.

master> SHOW TABLES;

The masteris dead

ERROR 2013 (HY000): Lost connection to MySQL server during query

Global Transaction IdentifiersFailover: make 'slave' a master of 'slave2'.

slave2> CHANGE MASTER TO MASTER_HOST = 'localhost', MASTER_PORT = 12001, MASTER_USER = 'rpl_user', MASTER_PASSWORD = 'secret';

Fail-over, i.e.,switch to use 'slave'

as the master.

slave2> STOP SLAVE;Stop slave threads.

slave2> START SLAVE; Start slave threads.

Global Transaction IdentifiersFail-over worked by just pointing the slave to another server!

slave2> SELECT @@GLOBAL.GTID_EXECUTED;mysql> SELECT @@GLOBAL.GTID_EXECUTED;

++'slave' is up to date now.

The last transactionwas copied from 'slave'slave2> SELECT * FROM test.t3;

slave2> SELECT * FROM test.t3;

| 99 |

'slave2' is up to date now.The last transaction

was copied from 'slave'

Summary

This HOL showcased some MySQL Replication best practices Several features from MySQL 5.6 that have direct user visibility were highlighted:

– Multi-threaded slave

– Global Transaction Identifiers

– Optimized Row-based Replication

– Transactional Replication Positions

– Replication Event Checksums

In the end the attendee was able to get a glimpse of some of the new MySQL 5.6 features.

– But there is much more to explore.

Oracle OpenWorld 2013 - HOL9737 MySQL Replication Best Practices

Technology