Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
1
MySQL Backups: From strategy to Implementation
Mike Frank
Senior Product Manager
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
2
Program Agenda
Introduction
The 5 Key Steps
Advanced Options
References
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
3
Backups are a DBAs Top Priority
Be Prepared
Automate
Test and Practice
Be Ready to recover when a data disaster strikes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
4
Backups
Data loss will happen
– OS crashes
– Power Failures
– Data center outages
– Are you in a cloud?
– File system crashes
– Human error
– Application Error
Just a reminder – of course - you know all that ….
As a DBA you must have a backup strategy
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
5
MySQL Enterprise Backup
Highly efficient and Consistent
Performs the backup while the database is running
Necessary when
– The database can’t be taken offline
Those are “Cold Backups”
The database is large – backups can take a long time
– Export backups are too slow, lock tables, …
– Also lets you perform “Warm Backups” for non-InnoDB tables.
Table level locks
Performs “Hot Backup”
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
6
What we’ll cover today
The goal today is to get you running scheduled backups regularly.
– The first key step to protecting your data
Later you can explore more powerful work flows and features
– Partial data moment
– Compression
– Task synchronization
– Hot and Cold support
There’s more than we can cover this session
http://www.mysql.com/products/enterprise/backup.html
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
8
The 5 Steps
1. Download and Install
2. Setup Backup Directories
3. Creating Full and Incremental Backups
4. Scheduling
5. Recovery – Prepare and Restore
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
10
Download
https://edelivery.oracle.com
Login
Pick MySQL Database and then OS platform
Hit Go
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
11
Find the download package
Your looking for MySQL Enterprise Backup …
Half way down the page – select Specific OS/Version and Format -
either TAR or RPM
Note for Windows -
MySQL Installer for Windows
includes
MySQL Enterprise Backup
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
12
Install
Uncompress the downloaded file - then
ZIPs and Tars
– Basically you can unzip, untar MEB where ever you want
RPM
– sudo rpm -i package_name.rpm
Deb packages
– sudo dpkg –i package_name.rpm
Windows
– Run the MSI – installs the product in
– C:\Program Files\MySQL\MySQL Enterprise Backup 3.8
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
14
Windows – After running MSI installer
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
15
Once installed set path or note location
Default Windows from MSI
– C:\Program Files\MySQL\MySQL Enterprise Backup 3.8\mysqlbackup.exe
– Used in 5 step example
Default Linux from RPM / DEB
– /opt/mysql/meb-3.8
If tar or zip – will be where you unpacked the files
– For linux 5 step example used
/home/mfrank/meb-3.8.1-linux2.6-x86-64bit/
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
16
Step 2
Setup Backup Directories
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
17
Where do you want the backups to go
Make sure you have plenty of space
Monitor that space
Linux
– mkdir /MySQLBackupHome
– chown mysql –R /MySQLBackupHome
– chgrp mysql –R /MySQLBackupHome
– chmod 600 –R /MySQLBackupHome
Windows –
– mkdir C:\mydatabasebackups\
– Make sure only the user that has rights to mysql file can read and write to the directory
– Right Click properties and Edit permissions for only that user (mysql) – Permissions - Modify, Read & Execute,
List Folders, Read, Write
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
19
Backup Types
Full – all the database data
– Note – but not the mysql programs and control files
Incremental – only the database changes –
– Prerequisite – a full backup
– Can be more than one incremental backups
The changes between last full or a prior incremental
Partial – just part of the database
Binary Logs – contains each event/transaction
http://dev.mysql.com/doc/refman/5.6/en/binary-log.html
http://dev.mysql.com/doc/mysql-enterprise-backup/3.8/en/meb-backup-overview.html
http://dev.mysql.com/doc/mysql-enterprise-backup/3.7/en/mysqlbackup.tasks.html
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
20
Keeping it simple
You’ll want to run backups on a schedule
For example – here we will show
– 1 full once week
– An incremental daily
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
21
Commands and Scripts
You need to have scripts that you schedule
In this case a minimum of 2
– FULL
– INCREMENTAL
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
22
Create Backup User
For this example the backup user is – mysqlbackup
http://dev.mysql.com/doc/mysql-enterprise-backup/3.8/en/mysqlbackup.privileges.html
mysql -u root –p
Mysql> CREATE USER 'mysqlbackup'@'localhost' IDENTIFIED BY '<password>';
mysql> GRANT RELOAD ON *.* TO 'mysqlbackup'@'localhost';
mysql> GRANT RELOAD ON *.* TO 'mysqlbackup'@'localhost';
mysql> GRANT CREATE, INSERT, DROP ON mysql.ibbackup_binlog_marker TO
'mysqlbackup'@'localhost';
mysql> GRANT CREATE, INSERT, DROP ON mysql.backup_progress TO 'mysqlbackup'@'localhost';
mysql> GRANT CREATE, INSERT, DROP ON mysql.backup_history TO 'mysqlbackup'@'localhost';
mysql> GRANT REPLICATION CLIENT ON *.* TO 'mysqlbackup'@'localhost';
mysql> GRANT SUPER ON *.* TO 'mysqlbackup'@'localhost';
mysql> GRANT CREATE TEMPORARY TABLES ON mysql.* TO 'mysqlbackup'@'localhost';
mysql> FLUSH PRIVILEGES;
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
24
FULL
Script will
– Create a time stamped subdirectory in the “home”
backup area
– Create the backup command
– Run the backup command
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
25
Linux FULL Bash Script
#!/bin/bash
dtstring=`date +%F`
new_dir="/MySQLBackupHome/mysqlfullweeklybackup_"$dtstring
mkdir $new_dir
cd /MySQLBackupHome
/home/mfrank/meb-3.8.1-linux2.6-x86-64bit/bin/mysqlbackup --defaults-file=mysqlback.cnf
--backup-dir=$new_dir/backup backup &> $new_dir/backup.log
echo $?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
26
Linux Configuration File -
[client]
port = 3306
protocol = TCP
user = <your backup user>
password = <your backup user password>
[mysqld]
datadir = /var/lib/mysql
[mysqlbackup]
datadir = /var/lib/mysql
innodb_data_home_dir =
incremental_base = history:last_backup
comments = My main backup - Full once a week, Incrementals daily
mysqlback.cnf
Note: MEB supports
MySQL External
Authentication
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
27
Windows FULL VBScript
Set shell = WScript.CreateObject("WScript.Shell")
Dim fso
Set fso = CreateObject("Scripting.FileSystemObject")
new_dir = "C:\MySQLBackupHome\mysqlfullweeklybackup_" &
replace(FormatDateTime(now,2),"/","")
If Not fso.FolderExists(new_dir) Then
Set f = fso.CreateFolder( new_dir )
End If
backup_command = "cmd.exe /c ""C:\Program Files\MySQL\MySQL Enterprise Backup
3.8\mysqlbackup.exe"" --defaults-file=mysqlback.cnf --backup-dir=" & new_dir &
"\backup backup > " & new_dir & "\backup.log 2>&1"
Set oExec = shell.Exec(backup_command)
Do While oExec.Status = 0
WScript.Sleep 100
Loop
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
28
Windows Configuration File
[client]
port = 3306
socket = MySQL
user = <your backup user>
password = <your backup user password>
[mysqld]
datadir = C:\ProgramData\MySQL\MySQL Server 5.6\data\
[mysqlbackup]
datadir = C:\ProgramData\MySQL\MySQL Server 5.6\data\
innodb_data_home_dir =
c:\MySQLBackupHome\mysqlback.cnf
Note: MEB supports
MySQL External
Authentication
(Native Windows User)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
30
Linux Incremental Bash Script
#!/bin/bash
dtstring=`date +%F`
new_dir="/MySQLBackupHome/mysqlincrementaldailybackup_"$dtstring
rm -Rf $new_dir
mkdir $new_dir
cd /MySQLBackupHome
/home/mfrank/meb-3.8.1-linux2.6-x86-64bit/bin/mysqlbackup --defaults-file=mysqlback.cnf
--incremental --backup-dir=$new_dir/backup --incremental-backup-
dir=$new_dir/backup/inc backup &> $new_dir/backup.log
echo $?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
31
Windows Incremental VBScript
Set shell = WScript.CreateObject("WScript.Shell")
Dim fso
Set fso = CreateObject("Scripting.FileSystemObject")
new_dir = "C:\MySQLBackupHome\mysqlincrementaldailybackup_" &
replace(FormatDateTime(now,2),"/","")
If Not fso.FolderExists(new_dir) Then
Set f = fso.CreateFolder( new_dir )
End If
Dim backup_command
backup_command = "cmd.exe /c ""C:\Program Files\MySQL\MySQL Enterprise Backup
3.8\mysqlbackup.exe"" --defaults-file=mysqlback.cnf --incremental --backup-dir=" & new_dir &
"\backup --incremental-backup-dir=" & new_dir & "\backup\inc backup > " & new_dir &
"\backup.log 2>&1“
Set oExec = shell.Exec(backup_command)
Do While oExec.Status = 0
WScript.Sleep 100
Loop
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
34
Linux CRON
Weekly Full – Sun at 2:00 am
crontab –e
0 2 * * 0 /MySQLBackupHome/MyFullBackup.sh
Daily Incremental Mon-Sat at 2:00 am
crontab –e
0 2 * * 1-6 /MySQLBackupHome/MyIncrementalBackup.sh
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
36
Windows Scheduler Create a Basic Task …
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
37
Weekly for Full Daily for Incremental (6 days not including the day the full runs)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
38
Select Time, Specify Start a Program
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
39
Path to your full or incremental script
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
40
When scripts run – MEB details to this log
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
42
Recovery - Preparing
Determine time to recover to
– If the time is for an incremental
Applying an Incremental Backup changes reflected in incremental backup files
bring the backup up-to-date
Apply those changes for each - oldest to newest
For example mysqlbackup
--backup-dir=/MySQLBackup/mysqlfullweeklyfullbackup_2013-04-22/backup
--incremental-backup-dir =/MySQLBackup/mysqlfulldailyincrementalbackup_2013-04-
23/backup/inc apply-incremental-backup
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
43
Recovery – Restore the data
Shutdown the database to be restored
– Stop the service or run a mysqladmin shutdown
Run Copy-Back
– Copies the datafiles, logs, and other backed up files into place
For example mysqlbackup --defaults-file=/etc/my.cnf
--backup-dir =/MySQLBackup/mysqlfullweeklyfullbackup_2013-04-22/backup copy-back
http://dev.mysql.com/doc/mysql-enterprise-backup/3.8/en/restoring.html
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
44
Advanced - Backup
Schedule the mysqlbinlog utility to dump all the SQL activity that
happened after the binlog position of the last backup
Direct the output of the mysqlbinlog command to an output file
$ mysqlbinlog --start-position=27183537 /sqldata/binlog.000005 >partial_binlog
$ mysqlbinlog /sqldata/binlog.000006 >> partial_binlog
$ mysqlbinlog /sqldata/binlog.000007 >> partial_binlog ...
$ mysqlbinlog /sqldata/binlog.00000N >> partial_binlog
Getting finer grained for Point in Time
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
45
Advanced - Recovery
To recover the database to the latest possible time,
– Pipe the output from mysqlbinlog --start-position=...
Directly to mysql
This replays all the SQL statements after the last backup.
Or to get to a point in time say be before an “oops” happened
– Edit the bin log and remove the “oops” and all statements after it.
Then
$ mysql < partial_binlog
Getting finer grained for an exact Point in Time
http://dev.mysql.com/doc/refman/5.6/en/point-in-time-recovery.html
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
46
Advanced – Single File
Recommended
– Has been seen to be faster than directory backup
Gives you an additional facility to verify the backup
Does take a little longer to restore
BUT - Can be prepared for restore
– Extract
– Apply log
– Compress back again http://dev.mysql.com/doc/mysql-enterprise-backup/3.8/en/backup-single-file-options.html
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
47
Advanced – Backup Direct to Tape
Meb Is a complete tape solution - Direct to Tape using SBT
– SBT is Oracle’s API for backing up and restoring data via media
management products
– use SBT for integration with Oracle Secure Backup
– use SBT for integration with Symantec Net Backup
– use SBT for integration with Tivoli Storage Manager
– Other products that support SBT
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
49
Advanced - Compression
The compression value needs to be determined
– 1 is normally a good value
– Unless you have an idle CPU
– And a very strict disk limitation
Compression will save disk but eat up more CPU and will take longer
The restore will also be longer
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
50
Advanced – Internal Table
MySQL Enterprise backup writes status to internal backup table
Useful to see detailed history of backups
Provide an audit trail to managers and auditors
backup_history table
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
51
Finally
You also need to decide on
– Retention
What to save and for how long
Schedule and automate this process
Its often nice to keep recent backups “close” for fast recovery
– Archiving
Whether to go to tape
– MEB also supports direct to tape backups
– Move files to archive storage – tape, HDFS, Cloud - S3 ….
This is just the most basic of scripts
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
52
Resources
Product Page – http://www.mysql.com/products/enterprise/backup.html
Latest Release – https://blogs.oracle.com/mysqlenterprisebackup/entry/mysql_enterprise_backup_3_8
User Guide – http://dev.mysql.com/doc/mysql-enterprise-backup/3.8/en/
Team Blog – https://blogs.oracle.com/mysqlenterprisebackup/
Questions, Ideas, … – http://www.mysql.com/about/contact/
– Phone: USA=+1-866-221-0634; Outside USA = +1-208-338-8100
– Email: [email protected]