Learn more from Oracle University at education.oracle.com

MySQL Performance Tuning Student Guide

D61820GC30

Edition 3.0 | January 2017 | D89524

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Disclaimer This document contains proprietary information and is protected by copyright and other intellectual property laws. You may copy and print this document solely for your own use in an Oracle training course. The document may not be modified or altered in any way. Except where your use constitutes "fair use" under copyright law, you may not use, share, download, upload, copy, print, display, perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express authorization of Oracle. The information contained in this document is subject to change without notice. If you find any problems in the document, please report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not warranted to be error-free. Restricted Rights Notice

If this documentation is delivered to the United States Government or anyone using the documentation on behalf of the United States Government, the following notice is applicable: U.S. GOVERNMENT RIGHTS The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or disclose these training materials are restricted by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract.

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Authors Mark Lewin Jeremy Smyth

Technical Contributors and Reviewers Jesper Wisborg Krogh Ligaya Turmelle Mark Leith Sveta Smirnova Mayank Prasad Vasil Dimova KimSeong Loh

Editors Anwesha Ray Smita Kommini

Graphic Designers Divya Thallap Seema Bopaiah Publishers Sumesh Koshy Veena Narasimhan

iii

Contents

1 Introduction

Objectives 1-2 Course Goals 1-3 Course Lesson Map 1-4 Introductions 1-6 Classroom Environment 1-7 A Modern Database for the Digital Age 1-8 High Scalability 1-9 MySQL Enterprise Edition 1-10 Oracle Premier Support for MySQL 1-11 Why MySQL Enterprise Edition? 1-12 MySQL and Oracle Integration 1-13 MySQL Cloud Service 1-14 MySQL Cloud Service: Product Overview 1-15 Use Cases 1-17 Tuning MySQL Cloud Service Instances 1-18 MySQL Websites 1-19 Community Resources 1-20 MySQL Certification 1-21 MySQL-Supported Operating Systems 1-22 Summary 1-23 Practices 1-24

2 Performance Tuning Concepts

Objectives 2-2 Improving Performance 2-3 Areas to Tune 2-4 Performance Tuning Terminology 2-5 Response Time 2-6 Measuring Response Times 2-7 Throughput 2-8 Scalability 2-9 Queuing Theory 2-10 Quiz 2-11 Benchmarking 2-12

iv

Benchmarking Best Practices 2-13 More Benchmarking Best Practices 2-14 Benchmarking Bad Practices 2-15 More Benchmarking Bad Practices 2-16 Quiz 2-17 Tuning Steps 2-18 General Tuning Session 2-19 Deploying MySQL 2-20 RAM Requirements 2-21 Maintaining MySQL 2-22 Summary 2-23 Practice 2-24

3 Performance Tuning Tools

Objectives 3-2 MySQL Monitoring Tools 3-3 SHOW [SESSION | GLOBAL] STATUS 3-4 Filtering SHOW STATUS Output 3-5 SHOW STATUS Example 3-7 SHOW ENGINE INNODB STATUS 3-9 SHOW PROCESSLIST 3-10 Quiz 3-11 mysqladmin 3-12 Information Schema 3-13 MySQL Enterprise Monitor: Overview 3-14 MySQL Enterprise Monitor: Architecture 3-15 MySQL Enterprise Monitor: Query Analyzer 3-16 MySQL Enterprise Monitor: Dashboard 3-17 MySQL Enterprise Monitor: InnoDB Performance 3-18 MySQL Utilities 3-19 Quiz 3-20 Community Monitoring Tools 3-21 Linux Tools 3-23 iostat 3-24 vmstat 3-25 sar 3-27 top 3-28 Benchmarking Tools 3-29 The MySQL BENCHMARK() Function 3-30 Stress Tools 3-31 The mysqlslap Stress Tool 3-32

v

SysBench 3-33 Using SysBench 3-34 Sample SysBench Output - 1 3-35 Sample SysBench Output - 2 3-36 Summary 3-37 Practices 3-38

4 Performance Schema

Objectives 4-2 Performance Schema 4-3 The performance_schema Database 4-4 Configuring Performance Schema 4-5 The setup_actors Table 4-6 The setup_instruments Table 4-7 The setup_consumers Table 4-8 The setup_timers Table 4-9 Configuring Instrumentation 4-10 The Connection Tables 4-11 The Instance Tables 4-12 The Event Tables 4-13 Example: Examining Raw Event Data 4-14 Example: Retrieving Statement Summary Information 4-16 Event Nesting 4-17 Quiz 4-19 Using MySQL Workbench for Performance Monitoring 4-20 Configuring Performance Schema in MySQL Workbench 4-21 The MySQL sys Schema 4-22 Using sys Views to Analyze Statements 4-23 Using sys Views for Schema Statistics 4-24 The sys Utility Functions 4-25 The sys Stored Procedures 4-26 The sys Stored Procedures Example: Tracing Threads 4-27 Summary 4-28 Practices 4-29

5 Memory, Connections, and Threads

Objectives 5-2 Major Components of the MySQL Server 5-3 Quiz 5-4 Tuning the MySQL Server 5-5 MySQL Memory Usage 5-6

vi

CPU and I/O Saturation 5-7 Global Buffers 5-8 MySQL Thread Handling 5-9 Per-Thread Buffers 5-10 Calculating Maximum MySQL Memory Usage 5-14 Scenario: Total MySQL Memory Usage 5-15 Quiz 5-16 Simultaneous Connections in MySQL 5-17 Connection Status Variables 5-18 Monitoring Idle Connections 5-19 Scenario: Users Unable to Connect 5-20 Diagnosing Network Problems 5-21 Reusing Threads 5-22 Quiz 5-24 Other Thread Status Variables 5-25 Scenario: Calculating Thread Cache Effectiveness 5-26 Scenario: Setting thread_cache_size 5-27 The MySQL Enterprise Thread Pool 5-28 Summary 5-30 Practices 5-31

6 Tables, Files, and Logs

Objectives 6-2 Reusing Tables 6-3 How MySQL Caches Tables 6-4 Setting table_open_cache 6-5 Setting table_definition_cache 6-6 Setting table_open_cache: Scenario 1 6-7 Setting table_open_cache: Scenario 2 6-8 Setting table_open_cache: Scenario 3 6-9 Quiz 6-10 Tables and Files 6-11 Managing Files 6-12 Setting File Descriptors in the OS 6-13 Quiz 6-14 Binary Logs 6-15 Transactions 6-16 Transactions and the Binary Log 6-17 Sizing the Binary Log Caches 6-18 Monitoring Binary Logs 6-19 Binary Log Group Commit Settings 6-20

vii

Improving Binary Log Performance 6-21 Scenario: Binary Log File Cache Effectiveness 6-22 Quiz 6-23 Summary 6-24 Practices 6-25

7 Statement Monitoring, Sort Buffer, and Query Cache

Objectives 7-2 General Statement Status Variables 7-3 SELECT Statement Status Variables 7-5 Quiz 7-7 Sizing the Sort Buffer 7-8 Sort Status Variables 7-9 Order of Sort Operations 7-10 Quiz 7-11 The MySQL Query Cache 7-12 MySQL Query Cache Settings 7-13 When Not to Use the MySQL Query Cache 7-15 When to Use the MySQL Query Cache 7-16 Quiz 7-17 MySQL Query Cache Status Variables 7-18 Determining Query Cache Effectiveness 7-21 Improving Query Cache Results 7-22 Quiz 7-23 Summary 7-24 Practices 7-25

8 InnoDB Storage Engine

Objectives 8-2 InnoDB Storage Engine 8-3 Operating System Buffers 8-4 InnoDB Buffers and Logs: Reading Data 8-5 InnoDB Buffers and Logs: Writing Data 8-6 Sizing the InnoDB Buffer Pool 8-7 Dumping and Restoring the Buffer Pool 8-8 Sizing the InnoDB Transaction Log Files 8-9 Other InnoDB Transaction Log Settings 8-11 InnoDB Tablespace Settings 8-12 Thread Concurrency 8-14 InnoDB Buffer and Log File Flushing 8-15 Adaptive Flushing 8-17

viii

Purge Behavior 8-18 Undo Logs 8-20 Recommended InnoDB Settings for OLTP and Benchmarking 8-21 Recommended InnoDB Settings for Replication 8-23 Linux Filesystem Recommendations 8-24 InnoDB Support in INFORMATION_SCHEMA 8-25 Information Schema InnoDB Metrics 8-27 InnoDB Support in performance_schema 8-29 SHOW ENGINE INNODB STATUS 8-30 SHOW ENGINE INNODB STATUS: Semaphores and Transactions 8-31 SHOW ENGINE INNODB STATUS: File I/O 8-33 SHOW ENGINE INNODB STATUS: Insert Buffer and Adaptive Hash Index 8-34 SHOW ENGINE INNODB STATUS: Log 8-35 SHOW ENGINE INNODB STATUS: Buffer Pool and Memory 8-36 SHOW ENGINE INNODB STATUS: Row Operations 8-37 SHOW ENGINE INNODB STATUS: Latest Foreign Key Error 8-38 SHOW ENGINE INNODB STATUS: Latest Detected Deadlock 8-39 InnoDB Monitors 8-40 Enabling InnoDB Monitors for Periodic Output 8-41 Quiz 8-42 Summary 8-43 Practices 8-44

9 Schema Design and Performance

Objectives 9-2 Schema Design Considerations 9-3 Schema Design Tasks 9-4 Normalization and Performance 9-5 Denormalizing Specific Data for Performance 9-6 Quiz 9-8 Data Types 9-9 Indexes 9-10 Indexing 9-12 Optimizing Indexes 9-13 Index Types 9-14 B+TREE Index 9-15 Quiz 9-17 InnoDB Table Compression 9-18 Tuning Compression for InnoDB Tables 9-19 Partitioning 9-20 Partitioning Types 9-21

ix

RANGE Partitioning: Example 9-22 LIST Partitioning: Example 9-23 HASH Partitioning: Example 9-24 KEY Partitioning: Example 9-25 Partitioning and Performance 9-26 Partitioning Limitations 9-27 Retrieving Partition Information 9-28 Quiz 9-29 Summary 9-30 Practices 9-31

10 Query Optimization

Objectives 10-2 MySQL Query Processing 10-3 Optimizer Main Stages 10-4 Logical Transformations 10-5 Logical Transformation: Examples 10-6 Cost-Based Optimization 10-7 Cost Model Inputs 10-8 Access Method 10-9 Join Order 10-10 Subquery Optimizations 10-12 Plan Refinement 10-13 Index Condition Pushdown 10-14 Quiz 10-15 Understanding the Query Plan 10-16 Using EXPLAIN 10-17 EXPLAIN Output 10-18 EXPLAIN select_type 10-20 EXPLAIN type 10-21 EXPLAIN key 10-23 EXPLAIN Extra 10-24 EXPLAIN Extra: Join Buffer and Multi-Range Read 10-26 EXPLAIN EXTENDED 10-27 EXPLAIN EXTENDED Query Text 10-28 EXPLAIN PARTITIONS SELECT 10-29 Structured EXPLAIN 10-30 Sample Structured EXPLAIN Output 10-31 Visual EXPLAIN 10-32 Quiz 10-33 Optimizer Trace 10-38

x

Improving Query Performance 10-39 Data Retrieval: Best Practices 10-40 Filtering Data with WHERE 10-41 Indexing for Query Performance 10-42 Query Using WHERE Clause 10-43 Query Using Covering Index 10-44 Query Using Index Condition Pushdown 10-45 Costs of Indexing 10-46 Indexing: Best Practices 10-47 Leftmost Prefixes 10-48 Improving the Performance of SELECT Statements 10-49 Optimizing Table Joins 10-50 Optimizing DML Statements 10-51 Creating Summary Tables 10-52 MySQL Enterprise Monitor Query Analyzer 10-53 Quiz 10-54 Summary 10-55 Practices 10-56

11 Troubleshooting Performance Issues

Objectives 11-2 Key Steps in Troubleshooting 11-3 Before You Start: Establish a Performance Baseline 11-4 Establishing the Nature of the Problem 11-5 Starting to Troubleshoot 11-6 Identifying Performance Issues 11-7 Application Profiling 11-8 Localizing Database Problems 11-9 Troubleshooting Locks 11-10 Explicit Locks 11-11 Implicit Locks 11-12 Avoiding Locks with Multiversion Concurrency Control 11-13 InnoDB Lock Modes 11-14 Lock Type Compatibility Matrix 11-15 Quiz 11-16 Troubleshooting Locks with SHOW PROCESSLIST 11-17 Troubleshooting Locks with SHOW ENGINE INNODB STATUS 11-19 Troubleshooting Locks with Information Schema 11-20 INNODB_TRX View 11-21 INNODB_LOCKS View 11-22 INNODB_LOCK_WAITS View 11-23

xi

Linking Blocked Statements to Locks 11-24 Troubleshooting Locks with Information Schema: Example Query 11-26 Quiz 11-28 Identifying Slow Queries 11-29 Identifying Frequent Queries 11-30 General Query Log 11-31 Slow Query Log 11-32 mysqldumpslow 11-34 Finding Slow Queries with Performance Schema 11-35 Finding Slow Queries with the sys Database 11-37 Finding Slow Queries with MySQL Enterprise Monitor 11-39 Quiz 11-40 Troubleshooting Specific Queries 11-41 EXPLAIN: Possible Indicators of Slow Queries 11-42 Profiling Queries 11-43 Finding the Query Identifiers 11-44 Displaying the Query Profile 11-45 Improving Query Performance 11-47 Quiz 11-48 Summary 11-49 Practices 11-50

12 Optimizing MySQL for your Application Environment

Objectives 12-2 Improving Connector Performance 12-3 Buffered and Unbuffered Result Sets 12-4 Creating Unbuffered Result Sets 12-5 Connection Pools 12-6 Configuring Connection Pools 12-7 Improving Connector Performance in Your Application 12-8 Prepared Statements 12-9 Encryption Overhead 12-10 Compressing Connector Protocol Traffic 12-11 Viewing Connection Attributes with Performance Schema 12-12 Connector/J Connection String: useConfigs=maxPerformance 12-13 Connector/ODBC Tracing 12-14 Application-Specific Optimizations 12-15 Improving InnoDB Performance with the Memcached NoSQL API 12-16 Advantages and Disadvantages of Using the Memcached API 12-17 Memcached API Performance Considerations 12-18 Reducing Transactional Overhead 12-19

xii

Benchmarking with memslap 12-20 Quiz 12-21 Backup Speed 12-22 Backup Types 12-23 Backups: Performance Trade-offs 12-24 Improving the Performance of Logical Backup and Restore Operations 12-25 Improving Backup Performance with Replication 12-27 Improving Database Performance with Replication 12-28 Improving Replication Performance 12-29 Improving Application Performance with MySQL Cluster 12-30 Accessing Data in MySQL Cluster 12-31 Evaluating MySQL Cluster 12-32 Summary 12-34

13 Hardware Optimization

Objectives 13-2 Hardware Limitations in MySQL 13-3 Avoiding CPU Saturation 13-4 Avoiding I/O Saturation 13-5 Redundancy 13-6 Solid State Devices (SSDs) 13-7 Choosing Between HDDs and SSDs 13-8 Configuring MySQL for SSDs 13-9 Storage Area Networks (SAN) and Network Attached Storage (NAS) 13-10 MySQL and Virtualization 13-11 Database-Driven Website Reference Architectures 13-12 Web Reference Architecture Sizes 13-13 Quiz 13-14 Summary 13-16

14 Conclusion

Course Goals 14-2 MySQL Curriculum Footprint 14-3 MySQL Community Support 14-4 MySQL Websites 14-5 We Need Your Evaluation 14-6 Thank You 14-7 Questions and Answers 14-8