Date post: | 07-Nov-2014 |
Category: |
Technology |
Upload: | giuseppe-maxia |
View: | 2,485 times |
Download: | 0 times |
Advanced MySQL Replication for the
masses
Giuseppe Maxia, QA Director, Continuent, Inc
Facts. And Demos. Possibly fun
1
Once Upon A Time, In The Life Of A
Database Consultant ...
2
The story of a steel foundry
• Used MySQL databases to store production monitoring data
• Inserted a zillion records per second.
• Slaves often lagged behind
3
BINARY LOG
REPLICATION
MySQLDBMS
trans
actio
n
trans
actio
n
trans
actio
n
trans
actio
n
trans
actio
ntra
nsac
tion
trans
actio
ntra
nsac
tion
trans
actio
ntra
nsac
tion
trans
actio
n
MySQLDBMS
transactiontransaction
transactiontransaction
transactiontransaction
transactiontransaction
transactiontransaction
transactiontransaction
4
The story of a shoe maker
• Had a successful business, spread to a dozen stores.
• Needed to aggregate the data from the stores in his headquarters database.
5
MySQLDBMS MySQL
DBMS
MySQLDBMS
MySQLDBMS
headquarters
store storestore
6
The story of a widgets seller
• Had a successful business, designed for one server.
• Products were created in several sites.
• Needed to allow insertions from more than one master at the time
7
MySQLDBMS
MySQLDBMS
MySQLDBMS
MySQLDBMS
mastermaster
master master
8
All these stories tell us:
Nice dream, but MySQL
can’t do it9
Enter Tungsten Replicator
10
Tungsten Replicator 2.0
• What is it?
11
http://code.google.com/p/tungsten-replicator
100% GPL v2Open Source
12
What can it do?
• Easy failover
• Multiple masters
• Multiple sources to a single slave
• Parallel replication
• Replicate to Oracle and PostgreSQL database
13
© Continuent 2010
MySQL to foreign services
BinLogs
Master DB
(Binlogs enabled) Slave DB
MySQL setup to run as
MySQL master
Data is applied to PostgreSQL or
Oracle
14
From the beginning ...
15
Fail-over (1)
16
Fail-over (2)
17
Fail-over (3)
18
Fail-over (4)
19
Fail-over (5)
20
Fail-over (6)
21
Fail-over (7)
22
Fail-over (8)
23
Failover
• DEMO
24
master/slave with an attitude
25
The steel foundry dreamor parallel replication
From here ...
26
The steel foundry dreamor parallel replication
To here.
27
Parallel replication facts
• Sharded by database
• Good choice for slave lag problems
• Bad choice for single database projects
28
Slave Lagging: why?
• Complex queries on the master
• Long lasting DDL
• Slave restart
29
Testing parallel replication
30
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
db1
db2
db3
db4
db5
db6
db7
db8
db9
preparation (1)db0
31
preparation (2)db0
db1
db2
db3
db4
db5
db6
db7
db8
db9
32
before the test (1)db0
db1
db2
db3
db4
db5
db6
db7
db8
db9
33
before the test (2)
binary logs
MySQL slave
Tungsten slave
RELAY logs
RELAY logs
IO thread
SQL thread
replicator alpha
direct: alpha(slave)
34
starting the test
binary logs
MySQL slave
Tungsten slave
RELAY logs
RELAY logs
IO thread
SQL thread
replicator alpha
direct: alpha(slave)
35
MySQL native replication
slave catch up in 00:59:30
36
Tungsten parallel replication
slave catch up in 00:25:40
37
The widget seller dream, or multi masters
• Tungsten Replicator recipe: use more services
38
Bi-directional replication
39
Bi-directional replication with slaves
40
True multiple master
We’ll see that in a moment.But first
41
The shoe maker dream, or multiple sources
• Tungsten Replicator recipe is still valid: use more services
42
Multiple source replication
43
Multiple masters replication: 3 nodes
44
Multiple masters replication: 4 nodes
45
Updating 4 masters : 1 flow
46
Updating 4 masters : 2 flows
47
Updating 4 masters : 3 flows
48
Updating 4 masters : 4 flows
49
Tungsten in practice
• installation
50
Installation
• Get the binaries
• Expand the tarball
• Run ./tools/tungsten-installer
51
Tools
• replicator
• trepctl
• thl
52
replicator
• It’s the service provider
• You launch it once when you start
• You may restart it when you change config
53
trepctl
• Tungsten Replicator ConTroLler
• It’s the driving seat for your replication
• You can start, update, and stop services
• You can get specific info
54
thl
• Transaction History List
• Gives you access to the Tungsten relay logs
55
ADVERTISING
56
http://www.continuent.com/about/careers
WE ARE HIRING!
QA and supportengineers
57
© Continuent 2011
Conclusion and Questions
58
© Continuent 2011
Worldwide560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel (866) 998-3642 Fax (408) 668-1009e-mail: [email protected]
Contact Information
Continuent Web Site:http://www.continuent.com
Tungsten Projecthttp://tungsten-replicator.org
59