Building the Fastest SQL Servers

Brent OzarMicrosoft Certified Solutions Master (MCSM)Brent Ozar PLF, LLC

DBI328

Brent Ozar

BrentO@BrentOzar.com www.BrentOzar.com

@BrentO

Photo Licensed with Creative Commons: http://www.flickr.com/photos/philly22/4279056817/

How I Built SQL Servers

How I Thought I Did It:Carefully baseline application to determine load patternsHand-select components perfectly matched to application needsGo live with a faster, more reliable environment

Photo Licensed with Creative Commons: http://www.flickr.com/photos/dangoodphoto/5202499523/

How I Built SQL Servers

How I Thought I Did It:Carefully baseline application to determine load patternsHand-select components perfectly matched to application needsGo live with a faster, more reliable environment

How I Actually Did It:Took a few rush measurements with Perfmon at random timesTried to figure out what parts were rightGo live without testingSomebody changes the app and my work was useless anyway

Photo Licensed with Creative Commons: http://www.flickr.com/photos/icathing/26603225/

How Microsoft Designs SQL Server Appliances

Systematically review thousands of SQL ServersDistill use cases down to a few common patternsChoose hardware components that are very likely to work great for those patternsPublish an incredibly detailed test checklist to make sure the hardware is working as designed

How Microsoft Designs SQL Server Appliances

Systematically review thousands of SQL ServersDistill use cases down to a few common patternsChoose hardware components that are very likely to work great for those patternsPublish an incredibly detailed test checklist to make sure the hardware is working as designed

Let’s steal their hard work.

Session Agenda

Define common SQL Server use patternsUnderstand the right hardware for a patternRecognize the server designs we can reuseLearn how to test our own hardware

Define

Common SQL Use Patterns

OLTP: Transactional Processing

How It’s Accessed:Inserts, Updates, Deletes throughout the day

How It’s Stored:Large numbers of tables with lots of complex relationships, but usually under a terabyte of data.

Examples:Web sites, ecommerce, payroll

Data Warehousing

How It’s Accessed:Loaded in short windows overnight,then read-only with big reads through the day for reports

How It’s Stored:Just a few tables, but many historical records in each table, and often over a terabyte of data.

Examples:10 years of sales history, stock prices, patient history

The Real Difference

An OLTP user says, “I need to fetch Mr. Jones’s order. This transaction takes 3 seconds, but I want it to finish instantly.”

A data warehouse user says,“I need all sales data for bicycles from last year. This report takes 15 minutes, but I want it to finish in 30 seconds.”

Understand

The Right Hardware for Pattern

SQL Server Hardware At Its Simplest

OLTP scenario: 50GB table.

CPU

Memory

(64GB)

Drives(100GB

)

End User

The Right Hardware for the Fastest OLTP

Instant queries = cache all data in memoryMaximize memory sizeMinimize data size (drop extra indexes, right-size fields)

Instant transactions = blazing fast log fileSSD RAID 10 for multiple databasesCan get away with dedicated RAID 10 magnetic for a single DB

Avoiding locking issues = blazing fast TempDB with RCSI

Read Committed Snapshot Isolation

I Didn’t Say Blazing Fast Data Drives.

Once the data’s in cache, data drive speed rarely matters.

SQL Server restarts will mean slow performanceAggressively monitor data size, memory sizeWhen you run out, things get ugly fast

Note: this session is about building really fast servers,not really easy servers. Easy = RAID 10 everything.

SQL Server Hardware At Its Simplest

Data warehouse scenario: 500GB table.

CPU

Memory

(256GB)

Drives(1TB)

End User

The Right Hardware for the Fastest DWs

Instant queries = cache all data in memory (we can’t)

Maximize memory sizeMinimize data size (drop extra indexes, right-size fields)Maximize storage throughput to retrieve what isn’t cached

Instant transactions = blazing fast log fileSSD RAID 10 for multiple databasesCan get away with dedicated RAID 10 magnetic for a single DB

Avoiding locking issues = blazing fast TempDB with RCSI

Read Committed Snapshot Isolation

Recognize

Server Designs We Can Reuse

Photo Licensed with Creative Commons: http://www.flickr.com/photos/leejordan/2484541616/

Download: BrentOzar.com/go/fastdw

Fast Track Data Warehouse Reference Arch.

Microsoft provides sample framework for vendorsVendors like Dell, HP, IBM provide specific server, storage, switching parts as an approved whole

Microsoft gives you the testing instructions for validationVendors provide the proof, and you re-validate it

You can use this same knowledge with your gear too

Sample Fast Track Reference Architecture

Dual Xeon processors, 6 cores each: MCR 3,168 MB/sec

Four 8Gbps Host Bus Adapters (HBAs):Delivers 3,123 MB/sec

48 15k drives in multiple RAID 5s:Delivers 3,146 MB/sec

Notice the balance

SQL Server Can Handle More Than You Think

Maximum Consumption Rate (MCR):How fast each core can consume data from storageStarting point: 200MB/sec of sequential reads per core

Dual socket, quad core: 1,600MB/secFour socket, quad core: 3,200MB/secFour socket, ten core: 8,000MB/sec

Your Hardware Does Less Than You Think

From BrentOzar.com/posters

Typical Storage Throughput

New SSDs Do More Than You Think

Learn

How to Test Our Own Hardware

Photo Licensed with Creative Commons: http://www.flickr.com/photos/boostinchick/4956027368/

Test Storage Quickly with CrystalDiskMark

Free tool with instructions:BrentOzar.com/go/cdmPick 5 tests, 4000MB testfile, drive letterSequential:roughly akin to backups, large table scans4K QD32:vaguely similar to activeOLTP server or TempDB

Test Storage Slowly with SQLIO

Free tool with instructions:BrentOzar.com/go/sqlioLots of possible options,collect the whole setUse a test file larger than yourSAN’s cache (say, 20GB)Don’t run this on a live server

Your Goals

Test with CrystalDiskMark to get a quick ideaTry two simultaneous CrystalDiskMark tests against two different drive letters to see if your multipathing worksWhen that works, amp up to SQLIO and really push itProve the hardware can achieve the Maximum Consumption Rate before you bother installing SQL Server

Remember

Storage Isn’t the Only Solution

How to Reduce Storage Throughput Needs

Keep memory free for SQL Server data cachingDesign and maintain your indexesCompress dataUse partitioned views to segment old archive data into tables with 100% fill factor, perfect statistics, read-only filegroupsConsider columnstore indexes for reporting queriesMerry-go-round scans with SQL Server Enterprise Edition

Merry-Go-Round Scans in Enterprise Edition

tblSales – 1,000GB

Brent’s Query

Merry-Go-Round Scans in Enterprise Edition

tblSales – 1,000GB

Brent’s Query

Merry-Go-Round Scans in Enterprise Edition

tblSales – 1,000GB

Brent’s Query

Dandy’s Query

Merry-Go-Round Scans in Enterprise Edition

tblSales – 1,000GB

Brent’s Query

Dandy’s Query

Merry-Go-Round Scans in Enterprise Edition

tblSales – 1,000GB

Brent’s Query

Dandy’s Query

Merry-Go-Round Scans in SQL Server Type R

tblSales – 1,000GB

Dandy’s QueryDandy’s Query

What We Covered

Defined common SQL Server use patternsOLTP: “I want the query to finish instantly.”DW: “I want the query to finish in 30 seconds.”

Understood the right hardware for a patternOLTP: Cache it all in memoryDW: Get blazing fast

Recognized the server designs we can reuseDW: MS Fast Track Data Warehouse Reference Architecture

Learned how to test our own hardware and compare itRemembered that storage isn’t the only solution

Photo Licensed with Creative Commons: http://www.flickr.com/photos/atxjen/4014857345/

Track Resources

@sqlserver@TechEd_NA#msTechEd

mvaMicrosoft Virtual Academy

SQL Server 2012 Eval Copy

Get Certified!

Hands-On Labs

Resources

Connect. Share. Discuss.

http://northamerica.msteched.com

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources for IT Professionals

http://microsoft.com/technet

Resources for Developers

http://microsoft.com/msdn

Complete an evaluation on CommNet and enter to win!

MS Tag

Scan the Tagto evaluate thissession now onmyTechEd Mobile

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.