1

MySQL DatabaseMySQL 5.7

Lalit Choudhary

2

Agenda1 MySQL History

2 MySQL Architecture

3 MySQL Installation

4 MySQL Storage engines and configuration

5 MySQL Administration and Monitoring

6 MySQL Replication

7 MySQL backup and restore

8 Upgrade and Downgrade

3

MySQL History The initial release in 1995 had a SQL interface and a dual license model, a free and an embedded

version. David Axmark, Monty and Allen Larrson founded MySQL AB in 1995, it was taken over by Sun Microsystems in 2008 and Sun itself was taken by Oracle in 2010

MyISAM engine InnoDB transactional engine MySQL 5.0 store procedure, triggers etc. MySQL 5.1 row-based replication, partitioning MySQL development boots in oracle, Including adding and improving existing features. MySQL comes with different choices depending on your needs.

- Community Server (GPL License) - Standard Edition - Enterprise Edition

4

MySQL Architecture• Physical view of the MySQL (5.7) server architecture

Directory location will be different as per OS and type of installation.

5

MySQL Architecture• Physical view of the MySQL (5.7) server architecture

Data directory• Server log files• Status file• Innodb log files• Innodb system tablespace• Innodb log buffer• Innodb tmp tablespace

Program executable files

Data sub-directory (Per Database)

• Data and index files (.ibd)• Object structure files(.frm,.opt)

Program log files

6

Configuration files:◦ data_dir/auto.cnf : Contains server_uuid◦ /etc/my.cnf : MySQL Configuration file.

Misc:• --basedir=dir_name

◦ The path to the MySQL installation directory.• --datadir=dir_name

◦ The path to the MySQL data directory.• --pid-file=file_name

◦ The path name of the file in which the server should write its process ID.• --socket=file_name, -S file_name

◦ On Unix, the name of the Unix socket file to use, for connections made using a named pipe to a local server.• --log-error=file_name

◦ Log errors and startup messages to this file.

MySQL Architecture

7

MySQL Architecture• Logical view of the MySQL server architecture

PLUGIN Connectors : ODBC, JDBC, NET,PHP, Python etc.

Connections/Thread handling

Query Cache Parser

Optimizer

Table Metadata Cache

Keys Cache[MyISAM]

inInnoDB MyISAM NDB MEMORY ……

SERVER

Storage Engines

CLIENT

[ Storage Engines buffer/s]

8

MySQL Installation Ways to install MySQL

• Source code• Binaries : ref http://dev.mysql.com/doc/refman/5.7/en/binary-installation.html• Packages : Ref: http://dev.mysql.com/doc/refman/5.7/en/linux-installation-rpm.html• MySQL Installer MSI and ZIP Archive• Yum repository

Configuration• Storage engine• Server Variables • User management and Access control

9

Storage EnginesInnoDB:• Fully transactional ACID.• Offers REDO and UNDO for transactions.• Data storage in tablespace :

Multiple data files Logical object structure using innodb data and log buffer

• Row level locking.

10

InnoDB Architecture

Log files

Tablespace

MEMORY DISK

Innodb_buffer_pool[Buffered data

pages,MVCC,UNDO,etc.]

Innodb_log_buffer[Buffered logs record]

[tablespace IDs and page IDs]

TABLESPACE

ibdata1

ibdata2

ib_logfile0

ib_logfile1

ib_logfile2

REDO LOG FILESCheckpoint

Checkpoint

Commit

UNDO

Other Memory pools

Ib_buffer_pool file Recovery

11

MySQL ArchitectureInnoDB Ref: http://dev.mysql.com/doc/refman/5.7/en/innodb-storage-engine.html

Tablespace : ibdata◦ The file ibdata is the system tablespace for the InnoDB engine. Apart from the table data storage, InnoDB's functionality requires looking

for table metadata and storing and retrieving MVCC info to support ACID compliance and Transaction Isolation. It contains several types for information for InnoDB objects.

• Table Data Pages• Table Index Pages• Data Dictionary• MVCC Control Data• Undo Space• Rollback Segments• Double Write Buffer (Pages Written in the Background to avoid OS caching)• Insert Buffer (Changes to Secondary Indexes)

◦ Example: innodb_data_file_path = /ibdata/ibdata1:10M:autoextend innodb_data_file_path = /ibdata/ibdata1:988M;/disk2/ibdata2:50M:autoextend

12

MySQL Architecture Innodb redo log file : ib_logfile

◦ It will read across ib_logfile0 and ib_logfile1 to check for any data changes that were not posted to the double write buffer in ibdata1. It will replay (redo) those changes. Once they are replayed and stored, mysqld becomes ready for new DB Connections.

◦ Example :Innodb_log_file_in_group=3 Innodb_log_file_size=150M

Innodb buffer pool file : ib_buffer_pool◦ At the time of mysqld startup to reduce the warmup period after restarting the server, it loads few percent of most

recently used pages for each buffer pool at server shutdown and this data i.e. tablespace ID and page ID data information is derived from the INNODB_BUFFER_PAGE_LRU INFORMATION_SCHEMA table. Tablespace ID and page ID data is saved in a file named ib_buffer_pool under InnoDB data directory.

◦ The file name and location can be modified using the innodb_buffer_pool_filename configuration parameter.

13

MySQL Architecture Transactional logs: Binary logs

◦ Variable name : --log-bin◦ Enable binary logging. The server logs all statements that change data to the binary log, which is used for backup

and replication.◦ Using mysqlbinlog client tool we can extract and view binlog contents in text format.

14

Storage EnginesNDB:• Fully Transactional and ACID Storage engine.• Distribution execution of data and using multiple mysqld.• NDB use logical data with own buffer for each NDB engine.• Offers REDO and UNDO for transactions.• Row level locking.

15

Storage EnginesMyISAM:• Non transactional storage engine• Speed for read• Data storage in files and use key ,metadata and query cache

.FRM for table structure .MYI for table index .MYD for table data

• Table level locking.

16

Storage EnginesMemory:• Non transactional storage engine• All data store in memory other then table metadata and structure.• Table level locking.

17

Storage EnginesArchive:• Non transactional storage engine,• Store large amounts of compressed and unindexed data.• Allow INSERT, REPLACE, and SELECT, but not DELETE or UPDATE sql operations.• Table level locking.

18

Storage EnginesCSV:• Stores data in flat files using comma-separated values format.• Table structure needs be create within MySQL server (.frm)

19

MySQL Production ready configuration

Hardware• Storage and file types : SSD/HDD and EXT4 or XFS on Linux• Memory : Optimal performance, Large transaction ,indexes, faster response time for ongoing changes and utilize Disk IO• CPU : Faster processors with many cores provide better throughput

MySQL configuration◦ innodb_file_per_table◦ innodb_buffer_pool_size◦ innodb_buffer_pool_instances◦ innodb_log_file_size◦ innodb_data_file_path◦ Innodb_log_file_in_group◦ innodb_flush_log_at_trx_commit◦ innodb_thread_concurrency◦ innodb_flush_method

20

MySQL Administration and Monitoring MySQL administrative tools

• Mysql : Login, database access/manipulation , • Mysqladmin : Administration purpose • Mysqldump / mysqlpump : Logical backup utility• Mysqlbinlog : Utility for processing binary log files and display binlog contents in text format.• Mysql enterprise backup : Provides enterprise-grade backup and recovery for MySQL.• Percona xtrabackup : Open source tool for mysql binary backup. Ref: http://dev.mysql.com/doc/refman/5.7/en/programs-overview.html

GUI tools • MySQL enterprise monitor• MySQL workbench/ SQLyog etc.

21

MySQL Administration and Monitoring Monitoring

• MySQL enterprise monitor : MySQL Enterprise Monitor provides real-time visibility into the performance and availability of all your MySQL databases.

• Metadata and : From Information schema and mysql database.• DB statistic: performance_schema databases we can get ongoing as well as historical database activities.• SHOW PROCESSLIST• Mysql utilities • Audit logs & General logs, slow query logs• Third-party tools : Percona tool kit , Nagios , cacti etc

22

MySQL ReplicationMaster Slave

● server_id● log_bin● binlog_format● binlog-do-db● binlog-ignore-db● GTID (optional)

• server_id• log_bin• binlog_format• log_slave_updates• relay_log=file_name • replicate-do-db=db_name• replicate-ignore-db=db_name

ON MASTER:Created replication user and grant privileges:mysql> CREATE USER 'repl'@'%.mydomain.com' IDENTIFIED BY 'slavepass';mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%.mydomain.com';Mysql> SHOW MASTER STATUS;

ON SLAVE:mysql> CHANGE MASTER TO MASTER_HOST='master_host_name',MASTER_USER='replication_user_name', MASTER_PASSWORD='replication_password',MASTER_LOG_FILE=‘master_log_file_name', MASTER_LOG_POS=master_log_position;

23

MySQL Replication

24

MySQL Replication Best practices

− Use GTID enabled replication ◦ log_bin◦ log-slave-update◦ expire_log_days◦ gtid_mode◦ Enforce-gtid-consistency

− Crash-safe slaves ◦ master-info-repository=TABLE◦ Relay-log-info-repository=TABLE◦ Relay-log-recovery

− Secure slave◦ read-only◦ Skip-slave-start

− Avoid replication Lag ◦ slave_compressed_protocol◦ MTR replication

25

MySQL Replication Without GTID

◦ Need Master_Log_File and Master_Log_Pos

GTID− What problems GTID solves?

◦ It is possible to identify a transaction uniquely across the replication servers. Make the automation of failover process much easier. There is no need to do calculations, inspect the binary log and so on. Just MASTER_AUTO_POSITION=1.

− Enable GTID replication

◦ gtid_mode: It can be ON or OFF (not 1 or 0). It enables the GTID on the server.

◦ log_bin: Enable binary logs. Mandatory to create a replication environment.

◦ log-slave-updates: Slave servers must log the changes that comes from the master in its own binary log.

◦ Enforce-gtid-consistency : Statements that can’t be logged in a transactionally safe manner are denied by the server.

26

Backup and Restore

Backup

Binary Backup

− MySQL Enterprise Backup

Logical Backup

− mysqldump / mysqlpump

eg. mysqldump [options] > dump.sql

Hot Backup

− Replication slave

Customized scripts

27

Backup and Restore

Restore

Logical backup restore using mysql client eg. mysql [options] < dump.sql

Binary backup restore : Replace old binaries or package and run mysql_upgrade script.

28

MySQL Upgrade and Downgrade MySQL Upgrade

Supported upgrade methods include:In-Place Upgrade: Involves shutting down the old MySQL version, replacing the old MySQL binaries or packages with the new ones, restarting MySQL on the existing data directory, and running mysql_upgrade.

Logical Upgrade: Involves exporting existing data from the old MySQL version using mysqldump, installing the new MySQL version, loading the dump file into the new MySQL version, and running mysql_upgrade.

MySQL Downgrade

Supported downgrade methods include: In-Place Downgrade: Involves shutting down the new MySQL version, replacing the new MySQL binaries or packages with the old ones, and restarting the old MySQL version on the existing data directory. In-place downgrades are supported for downgrades between GA versions within the same release series. For example, in-place downgrades are supported for downgrades from 5.7.10 to 5.7.9.

Logical Downgrade: Involves using mysqldump to dump all tables from the new MySQL version, and then loading the dump file into the old MySQL version. Logical downgrades are supported for downgrades between GA versions within the same release series and for downgrades between release levels. For example, logical downgrades are supported for downgrades from 5.7.10 to 5.7.9 and for downgrades from 5.7 to 5.6.

29

Additional Resources MySQL Tutorial: http://dev.mysql.com/doc/refman/5.7/en/tutorial.html

MySQL Administration: http://dev.mysql.com/doc/refman/5.7/en/server-administration.html

Blogs: https://planet.mysql.com/

30

THANK YOU