MySQL DBA

Spring 2009

Topics in the MySQL AB DBA Course I

• Describe the MySQL Architecture, general operational characteristics and resources utilized for running it.

• Perform the most common Database Administration (DBA) tasks utilizing the various programs contained within the MySQL application

• Utilize the MySQL Administrator Graphical User Interface (GUI) to manage a MySQL server

• Use the INFORMATION_SCHEMA database to access metadata• Install and Upgrade MySQL 5.0 for the most common operating systems• Perform the MySQL start and shutdown operations• Configure MySQL server options at runtime• Activate and manage numerous server logs• Evaluate data types and character sets for performance issues• Understand data locking concepts and the different levels of locking in MySQL• Differentiate between the multiple storage engines available in MySQL

Topics in the MySQL AB DBA Course II

• Create, Maintain and Manage MySQL tables• Perform backup and restore operations utilizing multiple MySQL tools• Manage user maintenance and access to a MySQL Server• Maintain integrity of a MySQL installation utilizing security protocols• Use stored routines and triggers for administration tasks• Manage, apply and understand the reason for using views• Improve performance through query optimization• Optimize Schemas (Databases) utilizing multiple techniques• Utilize MySQL monitoring tools to improve server performance• Compare multiple environment options to improve server performance• Scale MySQL operations up

WHAT IS A DBA?

What is a DBA?

• DBA’s are responsible for …• Some DBA’s also do…• Job Prospects and Pay

Basic DBA Tasks

• Installing and upgrading DBMS software• Starting and stopping the server• Configuring the DBMS (parameters, logs, etc.)• Managing user accounts • Security• Backup and Restore Data• Performance tuning• Porting data • Troubleshooting user problems

MYSQL – ARCHITECTURE AND PROCESSES

Architecture

Processes

• mysqld- the server program

MYSQL STORAGE ENGINES

Storage Engines

• 9+ storage engines – Some handle transaction-safe tables (safer)

• Even if MySQL crashes or you get hardware problems, you can get your data back, either by automatic recovery or from a backup plus the transaction log.

• can provide better concurrency for tables that get many updates concurrently with reads.

• Disadvantage: slower, higher disk space requirements, more memory required to perform updates

• Can use a different storage engine for each table in your schema

• Documentation includes a table to help you choose

MyISAM Storage Engine

• MyISAM — The default MySQL storage engine and the one that is used the most in Web, data warehousing, and other application environments.

• MyISAM is supported in all MySQL configurations, and is the default storage engine unless you have configured MySQL to use a different one by default.

• No transaction or foreign key support • Table level locking

InnoDB

• Used for transaction processing applications– ACID transaction support – foreign keys – row level locking

• included by default in all MySQL 5.1 binary distributions.

• In source distributions, you can enable or disable either engine by configuring MySQL .

Some other storage engines

• Memory — Stores all data in RAM for extremely fast access in environments that require quick lookups of reference and other like data.

• Archive —for storing and retrieving large amounts of seldom-referenced historical, archived, or security audit information.

• Federated — Offers the ability to link separate MySQL servers to create one logical database from many physical servers. Very good for distributed or data mart environments.

Storage Engines

• more to add…

STARTING AND STOPPING MYSQL

Starting MySQL

• To check if the server is running– (to be filled in later)

• To start the server (multiple ways)– Enter "mysqld" command with options in a command

window. – Create a launch shell script with "mysqld" command and its

options. Then run the script to start the server. – Create a Windows system service with "mysqld" command

and its options. Then run the service to start the server. – mysqld_safe is considered a safer way to start the server

(eventually call mysqld) on Unix

Startup MySQL Options• "mysqld" offers a big list of command line options. Here are some commonly used

options:• "--help" - Displays a short help message on how to use "mysqld".• "--verbose --help" - Displays a long help messages on how to use "mysqld".• "--console" - Specifies that the server to send error messages to launching window.• "--datadir=dirName" - Specifies that server data goes to the specified director.• "--init-file=fileName" - Specifies that server to execute all commands in the specified file

at startup time.• "--log=fileName" - Specifies that server to write query logs to the specified file.• "--log-bin=fileBaseName" - Specifies that server to write binary logs to a file with the

specified file base name.• "--log-error=fileName" - Specifies that server to write error logs to the specified file.• "--port=portNumber" - Specifies that server to listen client connections at the specified

port number.• "--version" - Returns the version information of the server.

• http://dba.fyicenter.com/faq/mysql/mysql_server_mysqld_adminitration.html

Stopping MySQL

• Concern:What about current activity?• Multi-step activity– Initiate shutdown process

• mysqladmin shutdown (used on any platform)

– The server creates a shutdown thread if necessary– The server stops accepting new connections – The server terminates current activity (next page)– Storage engines are shut down or closed

• The table cache is flushed and all open tables are closed

– The server exits

More about stopping MySQL

• For threads that have an open transaction, the transaction is rolled back.

• If a thread is updating a non-transactional table, an operation such as a multiple-row UPDATE or INSERT may leave the table partially updated, because the operation can terminate before completion.

MYSQL SETTINGS

Settings

• Three places to adjust settings– my.cnf file (MySQL configuration file) – start-up switches– current session settings (both globally and for

individual sessions). • Packages that work with MySQL may require

certain settings– e.g. turn off unsigned subtraction for financial

calculations – Challenge to ensure settings work for everyone

Checking Settings

• Check current MySQL server mode by connecting to the MySQL database and run the following queries– SELECT @@global.sql_mode; – SELECT @@session.sql_mode;

• To see the current values used by a running server, use the SHOW VARIABLES statement.

• To see the values that a server will use based on its compiled-in defaults and any option files that it reads, use this command: – mysqld --verbose --help

MYSQL TUNING

Strategies

• Benchmarking– find out the problems– tools exist out there

• Examine schema and query design• Where’s the problem– the hardware (memory, network, disks)– the operating system – disk i/o and CPU settings– MySQL

Tuning:Operating System• http://www.devshed.com/c/a/MySQL/MySQL-Server-Tuning-Tips-and-Tricks/1/

• Keep in mind that the OS is also responsible for the way your MySQL server performs. The server instance is a single process and, thus, the OS should be configured to permit really large process sizes. Additionally, enable the --large-pages option in MySQL if the OS you are running has this kind of capability. Also, since tables are quite large (usually) consider using a larger file system block size.

Tuning• http://www.devshed.com/c/a/MySQL/MySQL-Server-Tuning-Tips-and-Tric

ks/1/• http://20bits.com/articles/10-tips-for-optimizing-mysql-queries-that-dont-

suck/

• Caching schemes• Fine-tune the read-ahead kernel technique

(???)• Tip: “do not even attempt to use the default

MySQL configuration. It won't be enough; the default config file is really weak and small.”

Tuning Queries

• EXPLAIN <select statement>• Profiling: timing data – set profile=1; – show profiles; – show profile for query 7; // to see a particular one

• MySQL query Browser in mySQLadmin• Can purchase the MySQL Enterprise Monitor to

“automatically examine a MySQL server’s configuration, security, and performance levels, and identify any deviations from best practice rules”.

Tuning Queries

• ANALYZE– Helps the query optimizer– Analyzes and stores the key distribution for a table• MySQL uses the stored key distribution to decide the

order in which tables should be joined when you perform a join on something other than a constant. In addition, key distributions can be used when deciding which indexes to use for a specific table within a query.

Cleaning House

• OPTIMIZE TABLE– InnoDB– like VACUUM in POSTGRES– OPTIMIZE TABLE should be used if you have deleted a

large part of a table or if you have made many changes to a table with variable-length rows (tables that have VARCHAR, VARBINARY, BLOB, or TEXT columns). Deleted rows are maintained in a linked list and subsequent INSERT operations reuse old row positions. You can use OPTIMIZE TABLE to reclaim the unused space and to defragment the data file.

MYSQL LOG FILES

Log Files• General Query Log

– The query log logs all connections and all executed queries. It very quickly becomes big log (performance implications). Switched off by default.

• Slow Query Log– All queries that took more than long_query_time seconds to execute or didn't

use indexes • Error Log

– logs any MySQL errors that occur while the server is running, as well as startup and shutdown data. This includes data about invalid passwords, as well as syntax errors in the configuration file (my.cnf or my.ini)

• Binary Log– All statements that change data (also used for replication and incremental

backups) • Relay Log

– used for replication stuff

Slow Query Log I

• If you need to identify slow queries on a production MySQL server you may benefit from using the MySQL Slow Query Log. When the MySQL server is started with the --log-slow-queries option, it writes all queries that take longer than a configurable number of seconds to a log file. The queries in the Slow Query Log can be further examined and optimized.

• The Slow Query Log can also be activated by adding the log-slow-queries directive to the [mysqld] section of your server option file, or through the MySQL Administrator.

• From http://forge.mysql.com/wiki/MySQL_User_Guide_Part_IV_Advanced_MySQL_Administration

Slow Query Log II• The Slow Queries Log determines whether a query is slow by how long the

query takes to execute in seconds, not counting the time required to acquire table locks. The default time is two seconds and can be adjusted by setting the long_query_time option in the [mysqld] section of the server configuration file. The long_query_time option can also be set using the Log Files tab of the Startup Variables screen of MySQL Administrator.

• It should be noted that queries can appear in the Slow Query Log even if they are properly optimized if the server load is high enough to cause the query to take longer than the long_query_time.

• If you wish to log all queries that do not use indexes, regardless of how long the queries take to execute, add the log-queries-not-using-indexes option to the [mysqld] section of your MySQL server configuration file, or check the Log queries that don't use indexes option of the Log Files tab of the Startup Variables screen of MySQL Administrator.

More About The Other Log Files

Log Maintenance

• Per MySQL Documentation, when you have logging enabled:– clean up these files regularly to ensure that the logs do not take up

too much disk space. • For the binary log, you can set the expire_logs_days system variable to

expire binary log files automatically after a given number of days

– you may want to back up and remove old log files from time to time and tell MySQL to start logging to new files.

– You can force MySQL to start using new log files by issuing a FLUSH LOGS statement or executing mysqladmin flush-logs or mysqladmin refresh.

– The session sql_log_off variable can be set to ON or OFF to disable or enable general query logging for the current connection

MYSQL USER ACCOUNT MANAGEMENT

User Accounts

• to be added

MYSQL BACKUP AND RESTOREREFS: HTTP://WWW.DEVSHED.COM/C/A/MYSQL/BACKING-UP-AND-RESTORING-YOUR-MYSQL-DATABASE/HTTP://WWW.WEBCHEATSHEET.COM/SQL/MYSQL_BACKUP_RESTORE.PHP

Backup and Restore Examples• Backup

– #mysqldump --opt db_name > backup-file.sql– Ex. mysqldump --opt test_test > test.sql

• contains the SQL statements to recreate and populate the database tables when you wish to restore your database.

– To backup all databases• #mysqldump --all-databases > all_databases.sql

– Disadvantage is that mysqldump locks tables: can lock out users for a long period of time• Restore a dumped file

– #mysql db_name < backup-file.sql– Ex. mysql test_test < test.sql

• Can also compress dumped files• Can also use phpMyadmin for backup and restore• MySQL Backup is available as of MySQL 6.0.5. This feature comprises the

BACKUP DATABASE and RESTORE statements. (official documentation)• In case of an operating system crash or power failure, InnoDB itself does all the job of

recovering data. (official documentation)

Other comments

• Make and follow a standard policy– weekly full; daily incrementals are common for OS files

• Make sure you can USE your backups – horror stories of backup to tapes that later were found

to be unreadable– make sure media used stays readable (e.g. mag tapes)

• Consider offsite storage• Some consider MySQL not enterprise quality; fine

for small to medium shops

slashdot:scenarios to think about

• #1. Server crash.The data on the disk is destroyed. The OS is destroyed. But the hardware is okay.

#2. The building burns down.All of your servers are now smoking heaps of plastic. So's your desk. And all the CD's you had. (note: tornados; hurricanes; floods…)

#3. 5 years from now someone wants a critical policy that was deleted 3 years ago.

MYSQL NEXT TOPIC…

MYSQL NEXT TOPIC…