Performance Advantages of Running an Oracle® MySQL ... · Performance Advantages of Running an...

An Oracle White Paper

March 2014

Performance Advantages of Running an Oracle

® MySQL Replicated Database on Oracle

Linux with Oracle’s Sun Flash Accelerator F80 PCIe Card

Performance Advantages of Running an Oracle® MySQL Replicated Database on Oracle Linux with Oracle’s Sun Flash Accelerator F80 PCIe Card

Introduction ....................................................................................... 1

Advantages of Using Flash-based Storage with a MySQL Replicated Database ......................................................................... 2

Overview of Oracle’s Sun Flash Accelerator F80 PCIe Card ............. 2

Configuring Oracle Linux and the MySQL Database for Optimum I/O Performance ................................................................. 3

Hardware/Software Used for Benchmarks ..................................... 3

Configure Oracle’s Sun Flash Accelerator F80 PCIe Card to use as a File System ................................................................. 4

Configuring the MySQL Database for Semi-Synchronous Master/Slave Replication ................................................................... 5

Master Database .................................................................................... 5

Slave Database ...................................................................................... 6

Starting MySQL Replication ................................................................... 7

Benchmark Results ........................................................................... 8

Baseline Results ............................................................................ 8

Implementing Sun Flash Accelerator F80 PCIe Card ..................... 8

Test 1 Results ......................................................................................... 8

Test 2 Results ......................................................................................... 9

Conclusion ........................................................................................ 9

Resources ......................................................................................... 9


1

Introduction

This document demonstrates how to increase MySQL database performance on the master

server while simultaneously reducing replication lag time on the slave server by implementing

the PCIe®

flash based Sun Flash Accelerator F80 PCIe card. This guide demonstrates

optimization of MySQL master and slave servers running Oracle Linux with Unbreakable

Enterprise Kernel. By offering high performance with low latency and minimal CPU burden, the

F80 PCIe Card can help maximize transactional I/O performance for applications in an Oracle

Linux environment like online transaction processing (OLTP), data warehousing (DW), and

data mining. By delivering the I/O performance of hundreds or even thousands of traditional

hard disk drives (HDDs), the F80 PCIe Card is designed to improve user satisfaction and

productivity by enabling applications that use the MySQL database to meet challenging

service-level agreements (SLAs) for response times and throughput.

Oracle Linux with Unbreakable Enterprise Kernel (UEK) was chosen for this study because of

its upstream support for the latest hardware relevant to modern data center operations. In

addition, MySQL database workloads benefit from the platform’s deep integration with the

solution stack, optimizations resulting from industry collaborations and enhancements in the

UEK.

This solution guide provides a basis for planning and deploying PCIe compatible flash storage

in a MySQL environment on Oracle Linux with a typical use case scenario.


2

Advantages of Using Flash-based Storage with a MySQL Replicated Database

OLTP and data warehouse (DW) applications have demanding requirements for short response times and

high throughput, making it difficult for database administrators (DBAs) to maintain these systems as the

number of users grows and the amount of data increases. During the application life cycle, performance

bottlenecks might originate in one or more areas, including the network, processor, or storage devices, such as

HDDs. Correcting a bottleneck in one area might cause a bottleneck in another area.

In addition, if a customer wants to take advantage of MySQL’s built-in replication features to horizontally

“scale out” their database, replication process performance can become a concern. MySQL offers different

methods of replicating a database to provide for high availability, disaster recovery, scalability for high

performance, or a combination of any of these. MySQL replication provides the capability for all the slave

database(s) to synchronize data to the master database. The default replication operation mode in MySQL is

asynchronous, meaning that the master server pushes changes to the slaves once the updates are committed.

Replication has thus little overhead in transaction execution latency and the slaves eventually become

consistent with the state of the master. In the following use case, the semi-synchronous replication method

was used. When deploying semi-synchronous replication, the master server halts the commit of a transaction,

similar to synchronous replication, until the master receives at least one acknowledgement from any of the

slave servers. Another benefit of using semi-synchronous replication is that the acknowledgement from the

slave server only states that the new data has been received and saved to the relay log file; the

acknowledgement doesn’t mean that the slave server has actually applied the data to the database and the

result is better performance.

Flash-based storage provides a new tool for DBAs to help solve their MySQL database performance issues.

For example, 5 ms is the typical response time for a small data read from a HDD. Flash-based devices such as

the F80 PCIe Card can complete the same reading <50us (micro seconds) on average, an improvement of

several orders-of-magnitude in response time.

Flash-based storage also provides performance levels that fall between the performance levels of HDDs and

DDR3 memory. Initial implementations of flash-based drives (SSDs) were intended to replace HDDs in

direct-attach storage or RAID subsystems. The recent innovation of mounting SSDs on a PCIe card alleviates

throughput constraints that are caused by the storage interface. The F80 PCIe Card supports bandwidth up to

4GB/s, up to 260,000 input/output operations per second (IOPs) and capacity up to 800GB.

Overview of Oracle’s Sun Flash Accelerator F80 PCIe Card

The Sun Flash Accelerator F80 PCIe Card offers high performance with low latency and minimal CPU

burden. The F80 PCIe Card is designed to maximize transactional I/O performance for MySQL databases

and for other applications that require high-performance computing cycles. The F80 PCIe Card performs

consistently across reads and writes, regardless of workload, by using industry-standard and widely deployed

LSI SAS software for easier system integration and management, as well as a faster time-to-market.


3

.

Figure 1 High level technology overview and benefits of the Sun Flash Accelerator F80 PCIe card

Configuring Oracle Linux and the MySQL Database for Optimum I/O Performance

You can achieve significant database I/O improvement by following some simple guidelines in setting up

Oracle Linux and the MySQL database with or without replication. This section will describe the

configuration steps for both Oracle Linux and the MySQL replicated database with the Sun Flash Accelerator

F80 PCIe Card.

Hardware/Software Used for Benchmarks

Server Platform

for both Master

and Slave:

Supermicro Dual 6 Core Xeon 5650 2.67 GHz

32 GB 1333 MHz Memory

PCIe 3x8

10G Ethernet

Operating System: Oracle Linux 6.4 with

Unbreakable Enterprise Kernel (UEK) 2.6.39-300

Storage: HDD configured for RAID 10

Sun Flash Accelerator F80 PCIe Card

Database: MySQL 5.6


4

Configure Oracle’s Sun Flash Accelerator F80 PCIe Card to use as a File System

This section explains some actions you can take to tune the Sun Flash Accelerator F80 PCIe Card for

enhanced performance using the Oracle Linux with UEK. The configuration steps that follow provide the

performance improvements needed when used with MySQL and F80 PCIe Card. Here are the steps to

configure the F80 PCIe Card as a single filesystem for each of the master and slave servers for MySQL

databases. Other options would be to use multiple Sun Flash Accelerator F80 PCIe Cards and apply RAID to

provide fault tolerance for extra data protection.

1. Align the F80 PCIe Card on a 1MB boundary in the Oracle Linux operating system. The following

example shows how to partition the PCIe Flash Card as a single partition starting on a 1MB boundary:

echo "2048,," | sfdisk -uS /dev/sda --force

2. For creating an EXT-4 filesystem:

$ mkfs –t ext4 /dev/sda1 or mkfs.ext4 /dev/sda1

3. When mounting the new EXT-4 device (options are described below):

$ mount -o noatime,nodiratime,max_batch_time=0,nobarrier,discard \

> /dev/sda1 /mountpoint

4. Modify the kernel I/O scheduler. The I/O schedulers in Oracle Linux have new I/O capabilities,

including options to modify these settings at boot time. In the tests described in this document, the

DEADLINE I/O scheduler was used with the above file system mount options.

To invoke the DEADLINE scheduler for the entire system, add this line to the /etc/grub.conf file,

and bounce the system.

$ kernel /vmlinuz-2.6.39-300.28.1.el6uek.x86_64 ro \

> root=/dev/mapper/vg_nytro-lv_root elevator=deadline

5. An alternative way to invoke the DEADLINE IO scheduler is to modify the /etc/rc.local file with the

following command:

$ echo "deadline" > /sys/block/sda/queue/scheduler

6. To verify that the DEADLINE scheduler is enabled, issue the following statement as root.

$ cat /sys/block/sda/queue/scheduler

noop anticipatory [deadline] cfq

7. In addition to changing the I/O scheduler, the file system mount options stated above were invoked,

which was added to the /etc/fstab file. This example shows how the /etc/fstab entry invokes these

options:

/dev/sda1 /osfc ext4 defaults,noatime,nodiratime,max_batch_time=0,nobarrier,discard 1 2

An alternative to invoking these options is to specify them when executing the mount command:

$ mount -o noatime,nodiratime,max_batch_time=0,nobarrier,discard \

> /dev/sda1 /osfc


5

8. Performance of the F80 PCIe Card can be improved by increasing Queue Depth (QD) from the

default of 128 to 512 or higher, depending on the database workload. Since the latency of the F80

PCIe Card is so small, more I/O operations/commands can be run in parallel on the F80 PCIe Card

compared to a HDD. In order to modify queue depth, the nr_requests parameter will need to be

modified to the same or larger value of the new queue depth value. Here are examples of modifying

both the nr_requests and queue_depth parameters for /dev/sda which was used in our tests:

$ echo "512" > /sys/block/sda/queue/nr_requests

$ echo "512" > /sys/block/sda/device/queue_depth

9. Further performance benefits can be recognized by telling the operating system to complete an I/O

request on the same CPU that initiated it. In some workloads this can provide a nice jump in

performance. To enable this feature on the Sun Flash Accelerator F80 PCIe card:

echo "2" > /sys/block/sda/queue/rq_affinity

10. Another operating system setting that can provide more performance is setting the rotational setting

to 0 which means that the device is non-rotational, it is a flash device:

echo 0 > /sys/block/sda/queue/rotational

Note: For these settings to persist across reboots, place all of these commands in the /etc/rc.local file.

Configuring the MySQL Database for Semi-Synchronous Master/Slave Replication

This section describes the modifications needed to enable semi-synchronous replication using MySQL.

To enable semi-synchronous master/slave replication in MySQL 5.6, the following configuration settings

need to be applied.

Master Database

Install Plugins

The semi-synchronous plugin will need to be installed before starting the master server with the my.cnf

changes for replication.

From a MySQL prompt:

INSTALL PLUGIN rpl_semi_sync_master SONAME ’semisync_master.so’;

Configure Database

These are the database settings for the master to enable replication used for this test case:

/etc/my.cnf

[mysqld]

log_bin=/u04/binlog/mysql-bin


6

sync_binlog=1

binlog_format=’MIXED’

server_id = 10

# semi-sync replication

rpl_semi_sync_master_enabled=1

datadir = ’/u04/datadir’

max_connections = 600

innodb_flush_log_at_trx_commit=1

innodb_buffer_pool_size = 4G

innodb_thread_concurrency=49

slow_query_log=1

slow_query_log_file = "/u04/slow_query_logfile.log"

innodb_log_group_home_dir = ’/u04/logs’

innodb_data_home_dir = ’/u04/data’

innodb_file_per_table=1


innodb_log_files_in_group=4

innodb_log_file_size=2G


innodb_flush_method = O_DIRECT

innodb_write_io_threads=64

innodb_io_capacity=20000

expire-logs-days=1

Slave Database

Install Plugins

The semi-synchronous plugin will need to be installed before starting the slave server with the my.cnf changes

for replication.

From a MySQL prompt:

INSTALL PLUGIN rpl_semi_sync_slave SONAME ’semisync_slave.so’;

Configure Database

These are the database settings for the slave to enable replication used for this test case:

/etc/my.cnf

[mysqld]

log_bin=/u04/binlog/mysql-bin

binlog_format=’MIXED’

datadir = ’/u04/datadir’


7

server_id = 2

# for semi-sync replication

rpl_semi_sync_slave_enabled=1


innodb_buffer_pool_size = 4G


slow_query_log_file = "/u04"

innodb_log_group_home_dir = ’/u04/logs’

innodb_data_home_dir = ’/u04/data’

innodb_file_per_table=1

innodb_log_files_in_group=4

innodb_log_file_size=2G

innodb_flush_method = O_DIRECT

innodb_io_capacity=20000

relay_log = ’/u04/logs/mysql-relay-log’

log_slave_updates = 1

skip_slave_start

read_only = 1

expire-logs-days=1

Starting MySQL Replication

To start the replication process, start both the master and slave databases then execute the following

commands from each database:

MySQL Commands to Execute on Master Database

mysql> show master status\G;

File: mysql-bin.000064

Position: 120

Binlog_Do_DB:

Binlog_Ignore_DB:

Executed_Gtid_Set:

1 row in set (0.00 sec)

MySQL Commands to Execute on Slave Database

mysql> Reset slave;

mysql> change master to master_host=’master_hostname’, master_ user=’repl’,

master_password=’mysql’, master_log_file=’mysql-bin.000064’, master_

log_pos=120;

mysql> start slave;

mysql> start slave IO_THREAD;


8

Benchmark Results

Before adding the Sun Flash Accelerator F80 PCIe Card to the test configuration, Oracle Linux with UEK

and the MySQL database were configured using techniques described in the above sections. OLTP

benchmarks were executed to establish a baseline before and after implementing the Sun Flash Accelerator

F80 PCIe Card. Quest® Benchmark Factory® software was used for these benchmarks with transactions per

second (TPS) and average response time results documented. The slave metric “Seconds_Behind_Master”,

indicating the number of seconds that the slave is behind the master in processing, was captured during these

tests as well.

Baseline Results

The results from the benchmark before installing the Sun Flash Accelerator F80 PCIe Card on the servers

were the following:

Average TPS: 312

Average Response Time: .32 seconds

Slave Seconds_Behind_Master: 1172 seconds

These tests were executed with a configuration that resulted in the best results from HDD right before

performance started to drop off due to overloading the HDDs.

Implementing Sun Flash Accelerator F80 PCIe Card

After replacing the HDD where the database files are located on both servers with Sun Flash Accelerator F80

PCIe Cards, the following benchmarks were executed:

Test 1: The baseline user load was determined when the HDD was almost 90% busy. When the F80

PCIe Card was implemented, performance increased dramatically. Instead of using the same load as

the all HDD test, the user load of the benchmark was increased. The results show that even when

increasing the load, implementing the Oracle F80 PCIe Card was dramatically faster than the all HDD

setup.

Test 2: The Slave Seconds Behind Master test using the same configuration as with HDD

Test 1 Results

Average TPS: 996

Average response time: .1 second

Note: By installing the F80 PCIe Card in both servers, it was possible to double the user load on the system, while improving

average response times using the same 4G buffer size of the database.


9

Test 2 Results

Slave Seconds_Behind_Master: 0

Conclusion

In our benchmark testing with a Sun Flash Accelerator F80 PCIe Card, Oracle Linux with UEK and a

MySQL database utilizing the master/slave semi-synchronous replication feature, large performance gains

were realized and could directly improve enterprise environments using the configuration presented in this

guide for workloads that are:

1. I/O bound due to large number of HDDs

2. I/O bound due to large HDD throughput requirements (MB/s)

By implementing the Sun Flash Accelerator F80 PCIe Card in an Oracle Linux with UEK and MySQL

replicated environment, not only will the F80 PCIe Card increase performance on the master, but it will also

reduce the number of seconds that the slave is behind the master in replicating data.

In addition, there are benefits from Oracle’s deep integration of the MySQL database solution stack, as well as

optimizations resulting from industry collaborations and enhancements in Oracle Linux with the Unbreakable

Enterprise Kernel.

Resources

Oracle's Sun Flash Accelerator F80 PCIe Card

Oracle MySQL Database

Oracle Linux

http://www.oracle.com/us/products/servers-storage/storage/flash-storage/f80/overview/index.html

http://www.oracle.com/us/products/mysql/overview/index.html

http://www.oracle.com/linux


10


White Paper: Performance Advantages of

Running an Oracle® MySQL Replicated

Database on Oracle Linux with Oracle’s Sun

Flash Accelerator F80 PCIe Card

March 2014 REV.03

Author: Rick Stehno, LSI

Contributing Author: Phillip Goerl, Oracle

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000

Fax: +1.650.506.7200

oracle.com

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

This document is provided for information purposes only, and the contents hereof are subject to change without notice. This

document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in

law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any

liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This

document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our

prior written permission.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and

are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are

trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0114

Date post:	11-Oct-2019
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Performance Advantages of Running an Oracle® MySQL ... · Performance Advantages of Running an...

Documents