Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
Dave Stokes
MySQL Community Manager slideshare.net/[email protected] @stoker
Ten Things to Ten Things to Make Your MySQL Make Your MySQL Servers Faster and HappierServers Faster and Happier
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing
decision. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.
Safe Harbor Statement
Radiologists & Gorillas
83 Percent of Radiologists Didn’t Spot the Gorilla Hiding in This CT Scan
– Picture of gorilla hidden in last slide of ten lung scans.
– They wanted to see if the radiologists, focused on the telltale nodules, would be blind to the easily detectable and highly anomalous gorilla
– The gorilla was miniscule, but huge compared to the nodules. It was about the size of a box of matches-or 48 times the size of a typical nodule.
Gorillas responded
Databases Do Not Play Well With Others
Databases Systems are the colicky, nasty children of the software world with bad tempers and leaky noses that stubbornly insist on their way and will become disruptive at the most inopportune time, ruining man-years of work in an instant before demanding more
--Said by just about every software developer ever
So how do you get happy MySQL Servers?
This session for those who are not MySQL DBAs but are tasked to take care of MySQL Instances among other tasks. This will not make you a DBA any more than a 1 hour session on programming will make you a programmer.
How does a Database server work
Client
SELECT phone
FROM friends
WHERE name = ‘Joe’;
Server
How does a Database server work
Client
SELECT phone
FROM friends
WHERE name = ‘Joe’;
Server
PARSE
find Joe in friends table in memory
return phone
How does a Database server work
Client
SELECT phone
FROM friends
WHERE name = ‘Joe’;
. . .
Process phone data
Server
PARSE
find Joe in friends table in memory
return phone
How does a Database server work
Client
SELECT phone
FROM friends
WHERE name = ‘Joe’;
. . .
Process phone data
Server
PARSE
find Joe in friends table in memory
return phone
What was that about memory???
Thing to do #1 – Lots of memory
• Databases love data in memory
What if it is not in memory?
MySQL
Please give me the data from the city table
OS
What if it is not in memory?
MySQL
Please give me the data from the city table
OS
Get inode
What if it is not in memory?
MySQL
Please give me the data from the city table
OS
Get inode
Ask disk for data
What if it is not in memory?
MySQL
Please give me the data from the city table
OS
Get inode
Ask disk for data
Get data into buffer
What if it is not in memory?
MySQL
Please give me the data from the city table
Load data into memory
OS
Get inode
Ask disk for data
Get data into buffer
Hand buffer off
What if it is not in memory?
MySQL
Please give me the data from the city table
Load data into memory
OS
Get inode
Ask disk for data
Get data into buffer
Hand buffer off
Much longer than just reading from
memory
Rule #2 – Use Proper Hardware
Databases have to do unpredictable queries, random I/O, and sequential scans so slow I/O kills performance
Buffers
What happens when you read a file into memory
DISKDiskController
PageCache
UserBuffer
Buffers
Memory Lookup – 100 nanoseconds, 12GB/sec
DISKDiskController
PageCache
UserBuffer
Buffers
Memory Lookup – 100 nanoseconds, 12GB/secDISK seek – 10 milliseconds, 760MB/sec
DISKDiskController
PageCache
UserBuffer
Disk Reads
A disk read is 100,000 slower than reading memory● For comparison
– 100,000 minutes is about 69.5 days– 100,000 feet is about 19 miles– 100,000 kilometers is 15.7 times around
Earth's equator
SSD
Use standard RAID controller● SSD as writeback cache● Battery backed● $2.5/GB verses .075/GB● Very fast seek time● Density
Hardware recommendations1. Memory – lots of it, ecc
2. DISKs – more spindles, high speed, fast controllers, RAID 10, write back cache, and XFS/ZFS/ext4 not ext2/3
3. Write-through caches with battery backup units for disks must be monitored, and have life span much longer than planned outages. Set 'learning cycles' to off hours
4. CPUs, Core less important (spend money on Memory and IO)
Thing to do #3 –Understand Login/Privs
The default MySQL Login and Privilege system is a little … primitive.
Be Stingy with privs as they are hard to get back
Some tools may not give the privs you want – DROP PRIV is given by default by one popular admin tool
Quick Security Warning!
MySQL login security is primitive.● Database mysql has users table● 'jsmith'@'co.com' or 'fred'@'10.10.%'
– Matches host, then user, then password● Be explicit
● Proxy and Pluggable logins in MySQL 5.6● 'joe'@'10.10.%' may not be the same as
'joe'@'yourco.com' or 'joe'@'home.net'
MySQL privilege security● GRANT functions or access ● Can get Byzantine quickly● Use a GUI
MySQL Workbench – set up roles
Thing to do #4 – Use Current Release
Speed– MySQL 5.5 is 20% faster than 5.1
– MySQL 5.6 is ~15% faster than 5.5
Features– Better optimizer– Sub-queries– NoSQL/memcached– Online DDL
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31
MySQL 5.6: Scalability
● Users can fully utilize latest generations of hardware and OS● Scales as data volumes and users grow
Thing to do #5 – Monitor
You need 'something' monitoring your instances as you need a quick way to check status & record events
– MySQL Enterprise Monitor
– MySQL Workbench
– Nagios, Cacti, phpMyAdmin, etc
Turn on logging
1. Slow query log -- not all long running queries are bad
2. Log queries not using indexes
3. READ the logs!!!
4. Use monitoring software – MEM, Innotop, Cacti, Munin, Nagios, etc – and pay attention to it
Just because you can not see all of the problem does not mean the problem is not there!
Things to do #6 – Backups
Backups are usually some sort of disk snap shot or serializing data to a file
The more the better but you need to know steps to recover dropped table, lost databases, or mangled data.
Use data replication to a slave and then backup slave.
Be paranoid!!!!!
Thing to do #7 -- Replication
Replication for MySQL is the binary log for the master being copied to a slave. The slave then updates its copy of the data
Two types:1. Asynchronous – server does not check changes sent to
slave before proceeding
2. Semi Synchronous – server checks that slave received changes before proceeding
Replication – was messy!
MySQL 5.6 is multi threaded and has GTIDs
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38
Simple to track & compare replication across the cluster● Unique identifier for each transaction written to the Binlog
Automatically identify the most up-to-date slave for failover Deploy n-tier replication hierarchies
Master
GTID=123456
GTID=123456
GTID=123456 GTID=123456
MySQL 5.6: ReplicationGlobal Transaction Ids
Eliminates the need for complex 3rd party
solutions
Replication -- network
Network latency will affect MySQL replication. So plan network topology to minimize bandwidth competition with other systems/services.
Slaves do not need to be as fast as the master but try to keep things reasonably close
Do not have to replicate all tables/databases to all slaves. Cut down on traffic by replicating what is needed!
Writes & Reads Reads Reads
• Write to one master• Read from many slaves, easily add more as needed• Perfect for read/write intensive apps
Application
MySQL Replication
Load Balancer
MySQL DatabaseReplication Enables Scalability
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41
New option: binlog-row-image=minimal
Increases throughput for master and slave– Reduces Binlog size, memory & network bandwidth
Only replicates elements of the Row image that have changed
Primary Key Changed Columns
MySQL 5.6: ReplicationOptimized Row Base Replication
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42
Before:– Transaction Data: in tables
– Replication Info: in files
MySQL 5.6– Transaction Data: in tables
– Replication Info: in tables
Data
Position Info
CRASH!
Time
Data
Position InfoTime
Automatic recovery of a slave after a failure● Binlog and table data are transactionally
consistent Resumes replication without Dev/Op
intervention● Automatically rolling back replication to
last committed event
Atomic
Atomic
MySQL 5.6: Crash safe Slaves
Eliminates risk of data loss or corruption
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43
Master
#
Slave
#
MySQL 5.6: Replication Event Checksums
Eliminates risk of data loss or corruption
Ensures replicated data is correct, consistent and accessible
Detects corrupt replication events before they’re applied
– Returns an error
Protects entire replication path– Memory– Disk– Network– Bugs
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44
Thing to do #8 – Use InnoDB
● Transactional, speedy, crash safe, and where Oracle is concentrating development.
● Online DDL, SSD Optimizations, Dump/Restore warm buffer pool (great for cloud)
● NoSQL access via memcached API
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45
● Enables export/import of tables between running MySQL instances
MySQL 5.6: InnoDBTransportable Tablespaces
CREATE TABLE t(c1 INT) engine=InnoDB;
FLUSH TABLE t FOR EXPORT; -- quiesce the table and create the meta data file
$innodb_data_home_dir/test/t.cfg
UNLOCK TABLES;
Export:
Import:CREATE TABLE t(c1 INT) engine=InnoDB; -- if it doesn't already exist
ALTER TABLE t DISCARD TABLESPACE;
-- The user must stop all updates on the tables, prior to the IMPORT
ALTER TABLE t IMPORT TABLESPACE;
● Better Elasticity - Quickly spin up new instances to meet demand● Great for Cloud, Hosted, SaaS, On-premise deployments
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46
Same app can leverage: Key-value access to
InnoDB via familiar Memcached API
SQL for rich queries, JOINs, FKs, etc.
Fully transactional
MySQL 5.6: InnoDBNoSQL Key Value Access to InnoDB
● Up to 9x performance boost for updates
● Great for fast data ingestion in Big Data pipeline
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47
● Subquery Optimizations● File sort optimizations for most web use cases
● 4x better execution time – 40s to 10s Index Condition Pushdown
● 160x better execution time – 15s to 90ms Batched Key Access and Multi Range Read
● 280x better execution time – 2800s to 10s
MySQL 5.6: Optimizer
● Better complex query execution times ever growing data sets (Big Data!)● MEM + Query Analyzer key to utilizing full benefits of 5.6 Optimizer● MySQL Consultative Support provides guidance on configuration
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48
Things to do #9 – Indexes
Rough rule – look at columns used in joins or after a WHERE
Indexes are good
Without Index
DB needs to scan entire table or table scan
With Index
DB can go right to record
Indexes, the bad
• Overhead -- space, speed, maintenance (you can have too many)
• Not a panacea – does not cure all problems• Will not help if you need to perform a table
scan• Composite indexes can be tricky – YearMonthDay
usually better than DayMonthYear
Slide to check if audience is still awake
Thing to do #10 – Tuning
ASet innodb_buffer_pool_size to 70-80% of memory
Turn off query cache
Where to get help
Lots of Help Available● High Performance MySQL – Schwartz et al● MySQL Administrator's Bible – Cabral● Effective MySQL series – Bradford● OurSQL podcast● Forums.MySQL.Com
Planet.MySQL.Com
Local MySQL User Group
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54
Optimized for Web, Cloud-based, Embedded use cases Simplified, Pluggable architecture
– Maintainability, more extensible– More NoSQL options (HTTP, JSON, JavaScript, etc.)
Refactoring– Data Dictionary in InnoDB– Optimizer/Parser/Protocol
InnoDB – Optimized for SSD – GIS
Easy HA, Replication and Sharding
MySQL Database Development Priorities
MySQL Connect ConferenceMySQL Connect Conference
Sept 21st - 23rd in San Francisco
The Call for Proposals (CFP) for MySQL Connect 2013 just closed. Conference attendees will hear the best ideas from MySQL experts from around the globe: developers, visionaries, customers, partners. Book hotel early do to Americas Cup races in San Francisco.
MySQL Marinate!
Virtual self-study of MySQL through the Boston MySQL Users Group (
http://www.meetup.com/mysqlbos/)
http://www.meetup.com/Virtual-Tech-Self-Study/events/84103332/
Q&AQ&[email protected] - slideshare.net/davestokes