Date post: | 17-May-2015 |
Category: |
Technology |
Upload: | giuseppe-maxia |
View: | 4,259 times |
Download: | 0 times |
Scaling your web app with MySQL replication
Giuseppe MaxiaMySQL Community Team Lead
1This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License.
1Thursday, 30 September 2010
2
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
2Thursday, 30 September 2010
3
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
3Thursday, 30 September 2010
a simple web application
scheme
4
database server
web server
clients
r/w requests
4Thursday, 30 September 2010
database server
web servers
load balancer
clients
r/w requests
scaling web requests
55Thursday, 30 September 2010
writeread
database load on a simple
web application
6
85% 15%
6Thursday, 30 September 2010
readwrite
database load on a successful web
application7
20% 80%
7Thursday, 30 September 2010
8
database server
web servers
load balancer
clients
r/w requests✘
scaling up means buying
a bigger database
server
8Thursday, 30 September 2010
9
readwrite
the bigger database server will eventually
have the same problem
80%20%
9Thursday, 30 September 2010
read/writemaster
read/onlyslaves
web servers
R/W
R/O
load balancer
load balancer
clients
a web application
scheme with replication
1010Thursday, 30 September 2010
read/writemaster
read/onlyslaves
readwriteread
database load with
replication11
85% 15% 100%
11Thursday, 30 September 2010
read/writemaster
read/onlyslaves
readwriteread
scaling database load
with replication
12
85% 15% 100%
12Thursday, 30 September 2010
Replication assessment
13
without replication with replication
database handling
performance
Point in Time recovery
failover
write scaling
backup
read scaling
easy harder
high lower (binary logs)
none easy
none possible
none minimal
with downtime without downtime
none easy
13Thursday, 30 September 2010
14
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
14Thursday, 30 September 2010
master
slaveIO thread
SQL thread
reads
reads
client
transaction
binary log
relay log replication concepts
1515Thursday, 30 September 2010
16
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
16Thursday, 30 September 2010
17
1 SHUT DOWN THE DATABASE SERVER
Master
17Thursday, 30 September 2010
18
2 MAKE A BACKUP COPY
Master
18Thursday, 30 September 2010
19
3 ENABLE THE MASTER
Configuration file[mysqld]log-bin=mysql-binserver-id=1
Master
19Thursday, 30 September 2010
20
4 RESTART THE MASTER
Master
20Thursday, 30 September 2010
21
5 CREATE REPLICATION USER
SQL commandGRANT REPLICATION SLAVE ON *.* to 'slave_user@'10.10.100.%' IDENTIFIED BY 'slave_pass';
Master
21Thursday, 30 September 2010
22
6 INSTALL MySQL on the slave
Slave 1
Make sure that:• You're using the same version of MySQL • You have the same directory structure• The server is not started yet
22Thursday, 30 September 2010
23
7 COPY THE MASTER DATA to the slave
Slave 1
23Thursday, 30 September 2010
24
8 ENABLE THE SLAVE
Configuration file[mysqld]server-id=2relay-log=mysql-relayread-only# optional:log-bin=mysql-bin
Slave 1
24Thursday, 30 September 2010
25
9 START THE SLAVE SERVER
Slave 1
25Thursday, 30 September 2010
26
10 INITIALIZE THE SLAVE
SQL commandSET MASTER TOMASTER_HOST=master_IP,MASTER_PORT=3306,MASTER_USER=slave_user,MASTER_PASSWORD='slave_pwd';
Slave 1
26Thursday, 30 September 2010
27
11 START THE SLAVE SERVICE
SQL commandSTART SLAVE;
Slave 1
27Thursday, 30 September 2010
28
12 CHECK THE SLAVE
SQL commandSHOW SLAVE STATUS \G... Slave_IO_Running: YesSlave_SQL_Running: Yes...
Slave 1
28Thursday, 30 September 2010
Troubleshooting
• SHOW SLAVE STATUS says SLAVE_IO_RUNNING=No
• Make sure that the slave host can connect to the master
• Make sure that master and slave have different Server-id
• Check the error log of both master and slave
2929Thursday, 30 September 2010
Testing the slave
• Create a table in the master.
• Make sure that the slave has replicated the table.
• Insert data in the master
• read that data in the slave
3030Thursday, 30 September 2010
31
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
31Thursday, 30 September 2010
32
1 NO NEED TO STOP THE MASTER!
Master
32Thursday, 30 September 2010
33
2 STOP THE SLAVE
SQL commandSTOP SLAVE IO_THREAD;# wait until the SQL_THREAD# has done everythingSTOP SLAVE SQL_THREAD;# STOP THE SERVER
Slave 1
33Thursday, 30 September 2010
34
3 MAKE A COPY OF THE DATA DIRECTORY
Slave 1
34Thursday, 30 September 2010
35
4 RESTART THE SLAVE
Slave 1
35Thursday, 30 September 2010
36
5 INSTALL MySQL on the new slave
Slave 2
Make sure that:• You're using the same version of MySQL • You have the same directory structure• The server is not started yet
36Thursday, 30 September 2010
37
6 COPY THE old slave DATA on the slave
Slave 2
37Thursday, 30 September 2010
38
7 ENABLE THE NEW SLAVE
Configuration file[mysqld]server-id=3relay-log=mysql-relayread-only # optional:log-bin=mysql-bin
Slave 2
must be unique!
38Thursday, 30 September 2010
39
8 START THE NEW SLAVE
Slave 2
39Thursday, 30 September 2010
40
9 CHECK THE SLAVE
SQL commandSHOW SLAVE STATUS \G... Slave_IO_Running: YesSlave_SQL_Running: Yes...
Slave 2
40Thursday, 30 September 2010
Why it works
• No need to issue a CHANGE MASTER TO command.
• Because we cloned the old slave
• The new slave gets its parameters from the .info files in the data directory
4141Thursday, 30 September 2010
42
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
42Thursday, 30 September 2010
From single server application
43
r/w requests
43Thursday, 30 September 2010
To replication-aware application
44
read/writemaster
read/onlyslaves
R/WR/O
load balancer
44Thursday, 30 September 2010
Single server application$link = mysql_connect(
$server_IP, 'mysql_user', 'mysql_password');$result = mysql_query('INSERT INTO table_name (x) VALUES (1)',$link);$result = mysql_query('SELECT * FROM table_name WHERE x=1',$link);
4545Thursday, 30 September 2010
Making an application aware of replication
46
connectreadwrite
IPuserpassword
<<R/W database handling>>
db
connect
IPuserpassword
<<database handling>>db
read
IPuserpassword
<<read-only database handling>>
db
connectreadwrite
IPuserpassword
<<R/W database handling>>
db
46Thursday, 30 September 2010
Using replication: the WRONG way
47
R/W split by
statement
Write statement?
Connect to the master
Yes
Write to the master
Stop
No
Connect to the next available
slave
Read from slave
47Thursday, 30 September 2010
Why statement split is wrong
• Breaks transactions
• High risk of inconsistency
• Loses or corrupts data
4848Thursday, 30 September 2010
Using replication:the RIGHT way
49
R/W split by function
Write function?
Connect to the master
Yes
Read and write from master
Stop
No
Connect to the next available
slave
Read from slave
more queries?
No
Yesmore
queries?
No
Yes
49Thursday, 30 September 2010
50
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
50Thursday, 30 September 2010
Managing replication
•MONITORING• … or die
5151Thursday, 30 September 2010
52
Sample monitoring
master
slave
Get master binlog and position
get slave status
Running?
Yes
No
alert
Same or later binlog/position?
check table contents
NoYes
52Thursday, 30 September 2010
53
monitoringcontents
GET table CRC
GET table CRC
GET table CRC
master slave
same?
Yes
No
alert
GET table CRC
Stop
53Thursday, 30 September 2010
54
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
54Thursday, 30 September 2010
55
Replacing a slave
Slave crashes
are there more slaves?
STOP the master STOP one
slave
YesNo
add the first slave
add another slave
Stop
55Thursday, 30 September 2010
56
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
56Thursday, 30 September 2010
57
Replacing the master
Master crashes
Let all slaves catch up with
execution
FIND the most up to date slave
STOP replication in
all slaves
make it the master
Stop
FIND which transactions are
missing from other slaves
connect all slaves to the new master
run missing transactions
to other slaves
57Thursday, 30 September 2010
58
Table of contents
Why replication
How to set replication
Using replication
Leveraging replication
From single server to
replication
Adding a slave
Replacing a slave
Replacing the master
Managing replication
58Thursday, 30 September 2010
59
read/writemaster
read/onlyslaves
R/WR/O
load balancer
load balancing
59Thursday, 30 September 2010
60
backup master
slaves
STOP SLAVE
remove slave from load balancer
START SLAVE
perform backup
attach slave to load
balancer
Let slave catch up
60Thursday, 30 September 2010
master
slaves
STOP SLAVE
remove slave from load balancer
START SLAVE
calculate summary
tables
attach slave to load
balancer
Let slave catch up
61
makesummary
tables
61Thursday, 30 September 2010
62
master
slave
innodbnon partitioned
slave
innodbnon partitionedinnodb
partitioned by range
slave
MyISAMpartitioned by range
Partitionsfor heavy statistics
62Thursday, 30 September 2010
63
master
slave
innodbnon partitioned
slave
innodbnon partitioned
ARCHIVEpartitioned by range
(date)
slave
ARCHIVEpartitioned by range
(product)
slave
ARCHIVEpartitioned by range
(location)
Simulating multiple
dimensions
63Thursday, 30 September 2010
Semi-synchronous replication
• Available in 5.5 and higher
• Makes sure that at least one slave has copied the data.
• Increases reliability
6464Thursday, 30 September 2010
65
transaction with regular replication
clientmaster
slave
commit
binary log
execute
returns to client
replication
65Thursday, 30 September 2010
66
transaction with semi-
synchronous replication
clientmaster
slave
commit
binary log
execute
returns to client
sends transaction
to slave
gets acknowledgement
relay log
66Thursday, 30 September 2010
READ MORE
6767Thursday, 30 September 2010
The MySQL online manual
68http://dev.mysql.com/doc68Thursday, 30 September 2010
High Performance MySQL
6969Thursday, 30 September 2010
MySQL High Availability
7070Thursday, 30 September 2010
Web Operations
7171Thursday, 30 September 2010
Cloud Application Architectures
7272Thursday, 30 September 2010
What we didn't cover
73
(And are matter for more presentations)
73Thursday, 30 September 2010
Partial replication
• Replicating only one or more objects
• Specialized slaves
• Different storage engines
• Different data structures
7474Thursday, 30 September 2010
Row-based replication
• Available in 5.1 and higher
• Makes your data more consistent.
• Fixes many problems with statement-based replication
7575Thursday, 30 September 2010
Delayed replication
• Available in 5.6 and higher
• Protect replication against human mistakes and data corruption
7676Thursday, 30 September 2010
Tools
• Monitoring
• Testing and simulating
• Repairing
• Filtering, improving
7777Thursday, 30 September 2010
THANKS FOR YOUR
ATTENTION
78
This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License.
78Thursday, 30 September 2010