Date post: | 21-Jul-2015 |
Category: |
Technology |
Upload: | mark-swarbrick |
View: | 216 times |
Download: | 5 times |
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
How To Manage & Tune MySQL
Mark Swarbrick Principle Presales Consultant
MySQL Webinar Series 4/4
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement The following is intended to outline our general product direcFon. It is intended for informaFon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcFonality, and should not be relied upon in making purchasing decisions. The development, release, and Fming of any features or funcFonality described for Oracle’s products remains at the sole discreFon of Oracle.
3
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Manage & Tune MySQL
Tuning ConsideraFons + Go-‐live factors to consider
Planning for outages
Upgrading MySQL
4
1
2
3
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Tuning + Go-‐Live ConsideraFons
5
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Performance Tes0ng – How to measure success
Monitoring – Make sure you are monitoring the correct metrics
Backup Strategy – Will your backups be suitable for purpose?
High Availability – Does the up;me requirement match the design?
Failover – Have you got the policies and procedures in place to failover to a backup data center or server?
Growth – Have you planned for growth of data and throughput?
Tuning – How to tune and monitor for performance
6
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
MySQL Performance Schema • IdenFfy performance boXlenecks • IdenFfy problemaFc queries • Get real Fme insight into locks • See exactly what is happening within MySQL
• Get real Fme insight into MySQL internals
• Get real Fme insight into query execuFons
7
mysql> select * from host_summary_by_stages; +-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ | host | event_name | total | wait_sum | wait_avg | +-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ | hal | stage/sql/Opening tables | 889 | 1.97 ms | 2.22 us | | hal | stage/sql/Creating sort index | 4 | 1.79 ms | 446.30 us | | hal | stage/sql/init | 10 | 312.27 us | 31.23 us | | hal | stage/sql/checking permissions | 10 | 300.62 us | 30.06 us | | hal | stage/sql/freeing items | 5 | 85.89 us | 17.18 us | | hal | stage/sql/statistics | 5 | 79.15 us | 15.83 us | | hal | stage/sql/preparing | 5 | 69.12 us | 13.82 us | | hal | stage/sql/optimizing | 5 | 53.11 us | 10.62 us | | hal | stage/sql/Sending data | 5 | 44.66 us | 8.93 us | | hal | stage/sql/closing tables | 5 | 37.54 us | 7.51 us | | hal | stage/sql/System lock | 5 | 34.28 us | 6.86 us | | hal | stage/sql/query end | 5 | 24.37 us | 4.87 us | | hal | stage/sql/end | 5 | 8.60 us | 1.72 us | | hal | stage/sql/Sorting result | 5 | 8.33 us | 1.67 us | | hal | stage/sql/executing | 5 | 5.37 us | 1.07 us | | hal | stage/sql/cleaning up | 5 | 4.60 us | 919.00 ns | +-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
MySQL Workbench Overview
• MySQL Database IDE • Millions of Downloads • Windows, OS X, Linux
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Workbench 6.0
New Look, Redesigned GUI, Simplified Workflow • Home Screen, Managing ConnecFons, Edit & Admin, Color Scheme, Icons
For Developers • Schema Inspector, Maintenance, Cascading Delete, VerFcal Results, Table Data Search, Context SensiFve Help, Table Templates
For MigraFon • New Sybase Anywhere, SQLite
For Administrators • Server Status / Server Data, Backup GUI, Audit Log Inspector
For Design • Improved SynchronizaFon, Improved model prinFng to PDF files
What’s New
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Performance Tasks In Workbench
Monitor
Assess
OpFmize Tune
Configure
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Start by looking at Server Status
§ Management Tab – Server Status
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Basics
• Enabled/Disabled Features
§ Directory Setup § On best disks? § Adequate space? § Slow Query Log On?
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Performance Tes0ng – How to measure success
Monitoring – Make sure you are monitoring the correct metrics
Availability
Use a decent query
Logs
Logs filling up disk space
Slow query log filling space
Disk Space
Disks geing full is the most common problem
Ideally alarm on how soon disk space changes rather than absolute disk space
13
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Performance Tes0ng – How to measure success Monitoring – Make sure you are monitoring the correct metrics
Stalls / Spikes
Table Locks
CPU Spikes
Memory Paging
Connec0ons
Are you reaching max_connecFons limit?
Can the applicaFon connect?
Processes
Long running Processes
14
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Performance Tes0ng – How to measure success Monitoring – Make sure you are monitoring the correct metrics
Transac0ons Long running or long idle transacFons Queued transacFons (size of trans queue) (show innodb status) Replica0on Is ReplicaFon running Whats the replicaFon lag
15
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Performance Tes0ng – How to measure success Monitoring – Make sure you are monitoring the correct metrics
Query Performance Top Queries to opFmise – (interacFve monitor not generate alerts on) Response Fme outliers Queries not using index Queries using full/ parFal scans / table scans Queries that return errors / warning Opera0onal Aspects Server restarts Server config change events Messages in the error Log
16
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Backup Strategy – Will your backups be suitable for purpose
MySQLdump -‐ It can be used to dump a database or a collecFon of databases for backup or transfer to another SQL server (not necessarily a MySQL server). The dump typically contains SQL statements to create the table, populate it, or both. However, mysqldumpcan also be used to generate files in CSV, other delimited text, or XML format.
MySQL Enterprise Backup -‐ "Hot" Online Backups -‐ Backups take place enFrely online, without interrupFng MySQL transacFons
High Performance -‐ Save Fme with faster backup and recovery
Incremental Backup -‐ Backup only data that has changed since the last backup
ParFal Backup -‐ Target parFcular tables or tablespaces
Full Instance Backup -‐ Backs up data, as well as configuraFon and other informaFon to easily create a complete "replica"
17
MySQL Backup Types: Comparison mysqldump LVM Snapshots MySQL Replica0on MySQL Enterprise Backup
Full Backup
✔ ✔ ✔ ✔
Incremental Backups ✖ ✔ ✖ ✔
Par0al Backups ✔ ✖ ✖ ✔
Compression Support ✖ ✖ ✖ ✔
Allows updates ✖ ✖ ✔ ✔
Point in Time -‐ Consistent ✖ ✔ ✔ ✔
Backup Speed Poor Good Very Good Very Good
Recovery Speed Very Poor Good Very Good Very Good
Par0al Restore ✔ ✖ ✖ ✔
LZ4 Compression ✖ ✔
AES 256 Encryp0on ✖ ✔
Corrup0on Detec0on ✔ ✖ ✖ ✔
Meets Regulatory Archive Req.
✔ ✖ ✖ ✔
Supports DDL ✔ ✖ ✖ ✔
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
High Availability – Does the up;me requirements match the design?
See Planning for outages sec;on ;-‐) But always good to refer back to the original design spec!
19
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Failover – Have you got the policies and procedures in place to failover to a backup data center or server?
Most HA strategies fail not due to the hardware or sooware, but to the lack of awareness and tesFng of policies and procedures in place!
20
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Growth – Have you planned for growth of data and throughput?
21
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Tuning – How to tune and monitor for performance
22
Hardware
Config OpFons
Schema
InnoDB
Performance Monitoring
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Hardware
Schema Changes Data Growth
Indexes
SQL
90% of Performance Problems
Source of Database Performance Problems
23
Hardware: The Perfect MySQL Server
• The more cores the beXer (especially for 5.5 and later) • x86_64 -‐ 64 bit for more memory is important – The more the beXer • Fast HD (10-‐15k RPM SATA) or NAS/SAN…… – RAID 10 for most, RAID 5 OK if very read intensive – Hardware RAID baXery backed up cache criFcal! – More disks are always beXer! -‐ 4+ recommended, 8-‐16 can increase IO • …Or SSD (for higher throughput) – Intel, Fusion-‐IO good choices; good opFon for Slaves • At least 2 x NICs for redundancy • Slaves should be as powerful as the Master
Config OpFons The MySQL server is controlled by “System Variables”
• Set Via: – my.cnf / my.ini – SET [GLOBAL] <variable>=<value> – client, i.e mysql • Can be local (session) or global
The World’s Most Popular Open Source Database Copyright 2010 Oracle
The World’s Most Popular Open Source Database Copyright 2010 Oracle
Schemas • Size = performance, smaller is beXer – Size right! Do not automaFcally use 255 for VARCHAR • Temp tables, most caches, expand to full size
• Use “procedure analyse” to determine the opFmal types given the values in your table – hXp://dev.mysql.com/doc/refman/5.1/en/procedure-‐analyse.html – mysql> select * from tab procedure analyse (64,2000) \G
• Consider the types: – enum : hXp://dev.mysql.com/doc/refman/5.1/en/enum.html – set : hXp://dev.mysql.com/doc/refman/5.1/en/set.html
• Compress large strings – Use the MySQL COMPRESS and UNCOMPRESS funcFons
The World’s Most Popular Open Source Database Copyright 2010 Oracle
InnoDB Tuning
• Unlike MyISAM -‐ InnoDB uses a single cache for both index and data – innodb_buffer_pool_size -‐ should be 70-‐80% of available memory. – It is not uncommon for this to be very large, i.e. 34GB on a system with 40GB of memory – Make sure its not set so large as to cause swapping! – mysql>show status like 'Innodb_buffer%' ; • InnoDB can use direct IO on systems that support it -‐ Linux, FreeBSD, and Solaris – innodb_flush_method = O_DIRECT • For more InnoDB tuning see – hXp://dev.mysql.com/doc/refman/5.1/en/innodb-‐tuning-‐troubleshooFng.html
A Virtual MySQL DBA Assistant!
§ Web-‐based, global view of MySQL/Cluster applicaFons (on-‐premise and Cloud deployments)
§ Automated, rules-‐based monitoring and alerts (SMTP, SNMP enabled)
§ Query capture, monitoring, analysis and tuning, correlated with Monitor graphs
§ Real-‐Fme ReplicaFon Monitor with auto-‐discovery of master-‐slave topologies
§ Integrated with Oracle Support
MySQL Enterprise Monitor
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Planning For Outages
29
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. September 29th, 2012
The Lowdown on Down0me…. Maintenance Failures Disasters Operator / User
Revenue Customer Sat Employee Regulatory
CAUSE
EFFECT
IMPACT
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. September 29th, 2012
HA Considera0ons
• SLA requirements to support business objecFves • OperaFonal capabiliFes • Service agility & Fme to market • Budgetary constraints
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. September 29th, 2012
Not Everything Needs 99.999% Up0me
• Recovery Time ObjecFve – Maximum length of downFme before there is break in “business conFnuity”
• Recovery Point ObjecFve – Point in Fme to which data must be recovered when service is re-‐established
• Tier 1, Mission-‐criFcal services • Tier 2, Business-‐criFcal services • Tier 3, Task-‐criFcal services • Tier 4, Non-‐criFcal services
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and / or its affiliates. All rights reserved. September 29th, 2012
What are the Business HA requirements? Typical Availability levels
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and / or its affiliates. All rights reserved. September 29th, 2012
High Availability Solu0ons
MySQL CGE with Geo-‐ReplicaFon
MySQL Carrier Grade EdiFon (aka MySQL Cluster)
Shared storage cluster
DRBD with clustering sooware
MySQL ReplicaFon with clustering sooware
MySQL ReplicaFon
Best PracFce AdministraFon
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. September 29th, 2012
Mapping Up0me to Availability
%"9" 9" 9" 9"9"35 days 4 days 50 mins 5 mins 8 hours
Cos
t & C
ompl
exity
SPs & Line of
Business
Web & Cloud
Services
eCommerce Telecoms Military
.
Shared-Nothing, Geo-Replicated
Clusters
Clustered & Virtualized Systems
Replicated Systems
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2012, Oracle and / or its affiliates. All rights reserved. September 29th, 2012
HA Comparison Chart Overall
HA *1
Manual failover
autom. f/o
Active/ Active
RPO *1 RTO *1 No SPOF DR
MySQL 90.000% ü *2 ? ?
Best Practice 95.000% ü *2 1d 2h
MySQL Replication 98.000% ü ü *3 ü *4 1m 30m
MySQL semi-sync Repl. 98.000% ü ü *3 ü *4 0s 30m
DRBD 99.000% ü ü 0s 2m ü DRBD + Geo Repl. 99.500% ü ü 0s 2m ü ü Shared Storage 99.900% ü ü 0s 1m ü *5
Shared Storage + Geo Repl. 99.950% ü ü 0s 1m ü *5 ü MySQL Cluster 99.990% ü ü ü 0s 0,1s ü MySQL Cluster + Geo Repl. 99.999% ü ü ü 0s 0,1s ü ü
*1 Only guidelines *3 Requires addiFonal cluster framework s/w *5 Requires special hardware *2 Requires backup/restore *4 Slaves can be used read-‐only
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Upgrading MySQL
37
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 38
MySQL Repo’s YUM -‐ MySQL Yum repository supports the following Linux Distros: Red Hat Enterprise Linux 5,6,7 / Oracle Linux 5,6,7 Fedora 18, 19, and 20 MySQL APT -‐ The MySQL APT repository provides a simple and convenient way to install and update MySQL products with the latest sooware packages using Apt. The APT repository supports the following Linux Distros: Debian – 7, Ubuntu -‐ 12.04 LTS, Ubuntu -‐ 14.04 LTS, Ubuntu -‐ 14.10 MySQL SUSE Repo -‐ The MySQL repository for SUSE Linux provides a simple and convenient way to install and update MySQL products with the latest sooware packages using the Zypper package management tool. The repository at this Fme supports the following distro versions: SLES 12 for x86-‐64 SLES 11.3 for x86-‐64 edelivery.oracle.com – All Commerical binaries for all Oracle Sooware. RPM’s and tarball’s available for most popular distro’s dev.mysql.com – All GPL binaries for MySQL including source tarballs.
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 39
Windows Installer
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 40
Upgrade Concepts
• Back up your databases, including the mysql database that contains the grant tables.
• Read all the notes in the upgrade package. These notes enable you to idenFfy upgrade issues that apply to your current MySQL installaFon. Some incompaFbiliFes discussed in that secFon require your aXenFon before upgrading. Others require some acFon aoer upgrading.
• Read the Release Notes as well, which provide informaFon about features that are new in MySQL 5.6 or differ from those found in earlier MySQL releases.
• Aoer upgrading to a new version of MySQL, run mysql_upgrade. This program checks your tables, and aXempts to repair them if necessary. It also updates your grant tables to make sure that they have the current structure so that you can take advantage of any new capabiliFes. (Some releases of MySQL introduce changes to the structure of the grant tables to add new privileges or features.)
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 41
Upgrade Best Prac0ces
• Use virtualizaFon snapshots wherever possible for easy failback • You can run mulFple versions of MySQL on the same server in different directories, just make a copy of the data files for easy fallback
• Try not to skip versions ie upgrade from 5.0 -‐> 5.1 > 5.5 > 5.6 etc • Read the release notes • Have some sample queries and record the output before and aoer an upgrade.
• For absolute data integrity you can dump the data out of the old version and then reimport into the new version
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Thank You!
42