Practical SQL Server Performance Monitoring & Optimization

Anil Desaihttp://AnilDesai.net

Austin CodeCamp 2010

Anil Desai◦ Independent consultant (Austin, TX)◦ Author of numerous IT books◦ Instructor, “Implementing and Managing SQL Server 2005”

(Keystone Learning)◦ Info: http://AnilDesai.net or Anil@AnilDesai.net

Speaker Information

I. Performance Monitoring Overview

II. Monitoring Database Servers

III. Using SQL Profiler

IV. Using the Database Engine Tuning Advisor

V. Application Design Tips

VI. Managing Processes, Locking, and Deadlocks

Agenda and Overview

Performance Monitoring Overview

Developing processes and approaches for performance optimization

Best Practices:◦ Optimize for real-

world workloads

◦ Monitor/review performance regularly

◦ Focus on specific issues

Performance Monitoring Approaches

Establish a baseline

Identify bottlenecks

Make one change at a time

Measure performance

Repeat (if desired)

Overview of Performance Monitoring Tools and Methods

System/OS

Windows Performance Monitor

Alerts (Performance-Based)

SQL Server

Activity Monitor

SQL Profiler / SQL Trace

Database Engine Tuning Advisor

Dynamic Management Views (DMVs)

Query-Level

Database Engine Tuning Advisor

Query Execution Plans

Monitoring and Troubleshooting Scenarios

Server-Level Issues

• Users are reporting database timeouts• Intermittent transaction timeouts• “The server/application seem sluggish”

Application-Specific

Issues

• “Application A is running more slowly than usual”

• “The End-of-Month report is taking too long to run.”

Other Issues• Ad-hoc reports are running slowly• CPU, memory, disk, or network alerts

are being generated

Monitoring SQL ServerUsing SQL Server tools and features to monitor database activity

Available in all current versions of Windows Statistics are organized into:◦Objects◦ Counters◦ Instances

Data Collector Sets◦Windows Vista / Windows 7 / Windows Server 2008◦ Used to report on performance data that is collected over

time◦ Includes built-in System Diagnostics and System Performance

collectors and reports

Windows Performance Monitor

Anil Desai 10

Backup Device◦ Device throughput Bytes/sec

Buffer Manager◦ Buffer cache hit ratio◦ Page reads / sec

Buffer Partition◦ Free Pages

Cache Manager◦ Cache Hit Ratio

Databases◦ Active Transactions◦ Data File Size◦ Log Growths◦ Percent Log Used◦ Transactions / sec

Useful SQL Server Performance Counters

General Statistics◦ Logins | logouts / sec◦ User Connections

Latches◦ Latch waits / sec

Locks◦ Lock requests / sec◦ Lock waits / sec

Memory Manager◦ Everything!

Replication◦ Depends on configuration

SQL Statistics◦ Batch requests / sec◦ SQL compilations / sec

SQL Server Activity Monitor◦ Provides a quick overview of database server activity◦ CPU, Processes, Resource Waits and Disk I/O◦ “Recent Expensive Queries”

SQL Server Management Studio Reports◦Quick overview of SQL Server usage◦ Can export to Excel or PDF

SQL Server Management Studio Reports

SQL Server Report Examples

Server-Level Reports Database-Level Reports

Server Dashboard Memory Consumption Activity – All Block

Transactions Activity – Top Sessions Performance – Batch

Execution Statistics Performance – Top Queries

by Average CPU Object Execution Statistics

Disk Usage All Transactions All Blocking Transactions Index Usage Statistics Top Transactions by Age Schema Changes History

Windows Event Logs / Event Viewer ◦Application and System Event Logs

SQL Server Management Studio◦ SQL Server Logs

Can configure max. # of log files◦ SQL Server Agent Error logs

Can configure logging levels (Errors, Warnings, Information) Using the Log File Viewer◦ Can Export / Load log information◦ Can search for specific errors/messages

Monitoring SQL Server Logs

Purpose:◦ Monitoring and troubleshooting◦ View server state and performance details◦ Returns relational result sets (use standard SELECT statements)

Full list can be viewed in “Views System Views” section of the properties of the database

Scopes:◦ Server level◦ Database level

Using Dynamic Management Views (DMVs)

DMV Examples

• sys.dm_exec_requests

Query Execution / Processes

• sys.DM_DB_File_Space_Usage

Storage Engine

• sys.DM_DB_Index_Operational_Stats• sys.DM_DB_Index_Physical_Stats

Indexes

• sys.DM_IO_Pending_IO_Requests• sys.DM_IO_Virtual_File_Stats

Disk I/O

• Schema information | Common Language Runtime (CLR)• Database Mirror / Clustering | Replication

Other Categories

Data Collected:◦ System Information (MSINFO)◦Windows Event Logs◦ SQL Server configuration

Command-Line Utility (SQLDiag.exe)◦ Stores output to files◦ Configuration file: SQLDiag.xml◦ Can run as a service (/R)◦ Can run in continuous mode

Using SQLDiag

Using SQL ProfilerMonitoring SQL Server Activity

Purpose / Features: ◦ GUI for managing SQL Trace◦Monitor important events◦ Capture performance data / resource usage◦ Replaying of workloads / transactions◦ Identifying performance bottlenecks◦ Correlation of data with System Monitor◦Workloads for Database Tuning Advisor

Examples:◦ Generate a list of the 100 slowest queries◦Monitor all failed logins (Security)

Understanding SQL Profiler

SQL Profiler Terminology◦ Trace Definitions◦ Events◦ Columns◦ Filters

Creating and Managing SQL Traces◦ SQL Profiler (GUI)◦ System Stored Procedures (Transact-SQL)

Trace Templates (Built-In)◦ Standard (Default), SP_Counts◦ TSQL, TSQL_Duration, TSQL_Grouped,TSQL_Replay, TSQL_SPs◦ Tuning

SQL Server Profiler Architecture

Anil Desai 20

SQL Profiler Terminology Trace

◦ A set of events, data columns and filters that specify what data should be collected

◦ Data can be saved to a file or a database table Trace File

◦ Trace data that is saved to a binary file◦ Default extension is “.trc”

Trace Table◦ A SQL Server database table in which trace information is stored◦ Profiler will automatically create the structure of this table when you start

running a new trace Trace Template

◦ Saved specifications that can be used as the basis for new traces◦ E.g., an environment may have a “Security Monitoring template”, a “CRM

Application Performance”, etc.◦ Default extension is “.tdf”

Groupings:◦ Event Categories ◦ Event Classes◦ Events

Examples:◦ TSQL◦ Stored Procedures◦ Performance◦ Errors and Warnings◦ Security auditing

Configuring Trace Events

Event Categories

Event Classes

Events

Specifies the details to be monitored/recorded Configuring columns◦ Columns can be ordered and grouped◦ Values can be filtered

Examples of Columns:◦ StartTime / EndTime◦ TextData◦ Duration◦ Resource Usage (CPU, Reads, Writes)◦ Information: User, Database, App. Names

Configuring Trace Columns

Interactive◦ Good for “live” monitoring of small sets of data

Trace Files (*.trc)◦ Can enable file rollover based on size◦ “Server processes trace data” option

Trace table◦Will automatically create the table◦ Can set maximum number of rows

Scheduling of traces (stop time)

Trace Output Options

Launching SQL Profiler Connecting to a database instance Configuring output options Create a trace definition Specifying events, columns, and filters Running and viewing a trace

Demo: Creating Profiler Traces

Creating new templates using SQL Profiler

Scripting trace definitions◦ sp_trace_create◦ sp_trace_setfilter◦ sp_trace_GenerateEvent◦ sp_trace_SetEvent◦ sp_trace_SetStatus

Extracting SQL Server Events◦ Transact-SQL Events◦ ShowPlan Events◦ Deadlock Events

Other SQL Profiler Options

Purpose / Goal:◦ Correlate server performance with database performance

Process:◦Define and start a counter log◦Define and start a SQL Profiler trace◦ Import Performance Data in SQL Profiler

Required Trace properties◦ StartTime◦ EndTime

Using System Monitor with SQL Profiler

Using the Database Engine Tuning Advisor

Analyzing workloads to optimize physical database structures

Reviews sample workloads and makes performance recommendations

Evaluates Physical Design Structures (PDS)◦ Indexes (clustered, non-clustered)◦ Indexed Views◦ Partitions

Numerous analysis options Output◦ Generates modification scripts◦ Generates Reports for later analysis

Database Engine Tuning Advisor

Files◦ Transact-SQL Files◦ XML Files◦ Should represent commonly-used queries

SQL Profiler Trace Files / Tables◦ Use Tuning built-in trace template◦ Events:

Transact-SQL Batch Remote Procedure Call (RPC)

◦ Columns: Event Class and Text Data

Workload Sources

Limit tuning time Tuning Options◦ Allowed Physical Design Structures (PDS)◦ Keep all/specific existing objects◦Maximum storage space◦Online or offline recommendations◦ Partitioning

DTA Execution Options

Reports can be exported to XML files

Report Examples:

DTA Reports

Workload analysis Column access Table access View-Table

Relations

Statement cost Event frequencyIndex Usage

(current / recommended)

Process:◦Generate a workload (file or table)◦ Select tuning options◦Run the analysis◦View reports◦ Save and/or apply recommendations

Running the DTA:◦Database Engine Tuning Advisor Application (GUI)◦Dta.exe command-line utility

Using the Database Engine Tuning Advisor

Application Design TipsPractical ways to improve database application performance

Anil Desai 34

Create an abstraction layer between the database and the presentation code◦ Separates presentation and logic (esp. in Web Apps)◦ Example: ADO.NET Datasets

Database design:◦ Understand typical use-cases before designing the database◦ Create and enforce naming conventions◦ Balance write (OLTP) and read (reporting) performance

requirements◦ Use strategic denormalization

Application Design Tips

Never include actions that require user input within a transaction

Use connection pooling, whenever possible◦Open connections late and close them early◦ Avoid unnecessary server round-trips

Use client-side caching whenever possible◦Optimistic concurrency◦ Pessimistic concurrency

Distribute processing◦ Some operations are more efficient on the DB server (e.g.,

aggregations, sorting, etc.)

Application Design Tips (cont’d.)

Managing Processes, Locking, and Deadlocks

Troubleshooting common SQL Server performance problems

Processes◦ Interactive users

SQL Server Management Studio◦ Applications (Connection Pooling)

SQL Profiler Database Engine Tuning Advisor Replication Service Broker

Process IDs < 50 are system-related

Understanding Processes

SQL Server Activity Monitor◦ Processes (connected users)◦ Locks (by Process / by Object)◦ Filtering options◦Auto-refresh option

System Stored Procedures / Views◦Sys.DM_Exec_Sessions ◦Sys.DM_Exec_Requests◦Sys.SysProcesses◦sp_who / sp_who2

Monitoring Processes

Process Information◦ Current Process ID: @@SPID◦ Session Options: DBCC USEROPTIONS

Killing Processes◦ KILL ProcessID [WITH STATUSONLY]

Viewing Last Activity◦ DBCC INPUTBUFFER(ProcessID)◦ DBCC OUTPUTBUFFER(ProcessID)

Managing Processes

Coordinates multiple accesses to the same data Ensures ACID Properties for transactions (Atomic,

Consistent, Independent, Durable) Contention can reduce performance Locking granularity: ◦Row-Level, Page-Level, Table-Level, etc.

Lock Modes: ◦ Shared, Exclusive, etc.

Lock escalation

Understanding Locking

Blocking◦When transaction(s) must wait for a lock on a resource◦ LOCK_TIMEOUT setting (default = wait forever)

Locking Models:◦ Pessimistic◦Optimistic

Understanding Blocking

Balance of concurrency (performance) vs. consistency◦ Affects SELECT queries◦ SET TRANSACTION ISOLATION LEVEL

Transaction Isolation Levels◦ READ UNCOMMITTED◦ READ COMMITTED (default)◦ REPEATABLE READ◦ SERIALIZABLE◦ SNAPSHOT

Row-Versioning:◦ ALLOW_SNAPSHOT_ISOLATION ◦ READ_COMMITTED_SNAPSHOT

Transaction Isolation Levels

Activity Monitor SQL Profiler◦ Locks Event Category

System Monitor: ◦ SQL Server Locks Object

System Views◦Sys.DM_Tran_Locks◦Sys.DM_Exec_Requests

System Stored Procedures◦sp_Lock

Monitoring Locking Activity

Deadlocks:◦ Two or more tasks permanently block each other based on

resource locks◦Default resolution is within 5 seconds

Deadlock victim ◦ Transaction is rolled-back◦ Process receives a 1205 error

Example:◦ Process 1 locks the Customers table and requires access to

the Orders Table◦ Process 2 locks the Orders table and requires access to the Customers Table

Understanding the Deadlock Process

Minimize transaction times◦ Commit / Rollback transactions as quickly as possibly◦ Avoid user-related time within a transaction

Access objects in a consistent order Change the transaction isolation level◦ Use a lower level isolation level, if appropriate◦ Use snapshot-based isolation levels

Avoiding Deadlocks

Deadlock priorities:◦ SET DEADLOCK_PRIORITY (LOW, NORMAL, HIGH, integer)

Deadlock resolution:◦ Lower priority is killed first◦ If equal priorities, least expensive transaction becomes the

victim◦ Application or user should attempt to re-run the transaction

Deadlock Victims

SQL Server Error Log SQL Profiler ◦ Locks Event Category

Lock:Deadlock Chain Lock:Deadlock Deadlock Graph

◦ Events Extraction Trace Property◦ Export deadlock XML (.xdl) file

Viewing Deadlock Files◦ SQL Server Management Studio (File Open SQL

Deadlock Files (*.xdl)

Monitoring Deadlocks

Deadlock Graph

Questions & DiscussionFor more information:http://AnilDesai.net Anil@AnilDesai.net