SQL Server High Availability

transcript

HA cover

Paul Bertucci• Founder Database Architechs –www.dbarchitechs.com

– Specializing in HA, Database Design, Data Architecture, Data Replication, and P&T for SQL Server, Sybase, DB2 and Oracle

– Over 28+ years experience in Data Base industry• Co-Author of SQL Server 2000 Unleashed! (SAMS)• Co-Author of SQL Server 2005 Unleashed! (SAMS)• Co-Author of SQL Server 2008 Unleashed! (SAMS) – Summer 2009 !• Co-Author of ADO.NET in 24 hours (SAMS)• Author MS SQL Server High Availability (SAMS)• Author Sybase Performance & Tuning• Author Sybase Physical DB Design• Veritas SQL Server Performance Series • Former Chief Data Architect Symantec Corporation• Current Chief Architect Autodesk Corporation

pbertucci@dbarchitechs.com

Agenda

� What is High Availability?

� How do you assess your HA Requirements?

� What are the MS SQL Server related options for HA?

� How each option delivers HA…

� Performance and Tuning is critical too – SQL Shot!

� Q & A

1. What is the quickest way to test if your SQL Server Clusteringconfiguration is failing over properly?

2. What is the SQL Server feature in SQL Server 2005/2008that replaces Log Shipping?

What is Availability?ApplicationAvailability

PlannedDowntime

UnplannedDowntime

RecoverableDisaster

Failure causes:- Human- Hardware- Software

Uptime

The cost of Un-Availability

� Airline Reservation Systems - $67K to $112K per hour

� ATM Service Fees - $12K to $17K per hour

� Brokerage (Retail) - $5.6M to $7.3M per hour

What is your cost of downtime?

Across all layers of your systems

NetworkApplication

MiddlewareDatabase

Operating System HARDWARENetwork Components

ServersDisk Systems

Memory

Availability across planned operation

Feb 14-28 Mar 1 – Apr 15 Apr 16 – 20

Availability Goals

Starting Date Date of Failure Days Hours Minutes MBU (minutes) TU Avail %Period 1 2/14/2008 2/28/2008 15.00 24.00 60.00 21600.00 38.00 99.82407Period 2 3/1/2008 4/15/2008 46.00 24.00 60.00 66240.00 68.00 99.89734Period 3 4/16/2008 4/20/2008 5.00 24.00 60.00 7200.00 442.00 93.86111

Overall 2/14/2008 4/20/2008 66.00 24.00 60.00 95040.00 548.00 99.4234

Extreme Availability

High Availability

Standard Availability

Acceptable Availability

Marginal Availability

Near zero downtime!

Minimal downtime

With some downtimetolerance

Non-critical Applications

Non-production Applications

8,760 hours/year | 168 hours/week | 24 hours/day

525,600 minutes/year | 7,200 minutes/week | 1,440 minutes/day

(99.5% - 100%)

(95% - 99.4%)

(83% - 94%)

(70%-82%)

(up to 69%)

Availability ContinuumCharacteristic Availability Range

Availability Range describes the percentage of time relative to the “planned” hours of operations

ZeroUnplannedDowntime

ZeroPlanned

Downtime

Marginal Availability

Acceptable Availability

Standard AvailabilityHigh Availability

Extreme AvailabilityApplications and Availability

eCommerce

HRAccounting

MarketingMailers

InventoryMgmt

Five 9’s (99.999%) ~ 6 minutes/year downtime

What do you need?

It’s as simple as 1, 2, 3 +

� Step One – Launch of a brief “Phase 0” HA Assessment

� Step Two – Complete an HA Primary Variables gauge

� Step Three – Match your need to the optimal HA solution

� Step + (optional) – Determine the ROI of the HA solution

Assessing HA with Primary Variables

Uptime Requirement

Time to Recover

Data Resiliency

Application Resiliency

Cost of Downtime ($$ lost/hr)

Cost of the High Availability Solution ($$)

Degree of Distributed Access/Synchronization

Scheduled Maintenance Frequency

Performance/Scalability

0% 100%

Long Short

Low High

Often Never

Low High

Tolerance of Recovery TimeHigh Low

Assessment(scope)

Development Methodology“With High Availability built in”

RequirementsRequirements

DesignDesign

Code & TestCode & Test

System Test &Acceptance

ImplementationImplementation

0. Assessment- Project Planning- Project Sizing- Deliverables Identified (SOW)- Schedules/milestones- High-Level Requirements (scope) - Estimate HA Primary Variables (gauges)

1. Requirements- Detail Requirements (process/data/technology)- Early Prototyping (optional)- Detailed HA Primary Variables- Detailed Service Level Agreements/Rqmts- Detailed Disaster Recovery requirements

2. Design- Detail Design (data/process/technology)- Choose and design the matching HA solution for the application

3. Code & Test- Code Development/Unit Testing- Fully integrate the HA solution with the application

4. System Test & Acceptance- Full system Test/User Acceptance- Full HA Test/Validation/Acceptance

5. Implementation- Production Build/Implementation- Production HA build/monitoring begins

Transition

Spiral/Rapid MethodologyIterative approach 0. Initial assessment

- Project Planning- Project Sizing- Deliverables Identified (SOW)- Schedules/milestones- High-Level Functions (scope)-Estimate HA Primary Variables (gauges)

3. Requirements- Detail Requirements

(process/data/technology)- Detail HA Primary Variables- Detailed SLA/Rqmts- Detailed Disaster

Recovery requirements

1. High-level Rqmts/Prototyping- High-level requirements (process/data/technology)

- High-level HA Primary Variables

4. Design- Detail Designs (process/data/technology)- Choose and design the matching HA solution for the application (verified via prototypes)

Elaboration/Prototype

Inception

2. Early Code & Test- Early code and testing of apps/DI

(process/data/technology)- Prototyping of HA options

Construction

5. Code & Test- Code Development/Unit Testing-Fully integrate the HA solutionwith the application

6. System Test & Acceptance- Full system Test/User Acceptance

-Full HA Test/Validation/Acceptance

7. Implementation- Production Build/Implementation- Production HA build/monitoring begins

LogShipping

DBMirroring

SQL Clustering

Data Replication

Cluster Services

OtherHW

DiskMethods

Valid High Availability Options

DiskMethods

OtherHW

Cluster Services

Data Replication

SQL Clustering

DBMirroring

LogShipping

Cluster GroupResources

Windows 2003Enterprise Edition

MSCS Cluster Services

LocalBinaries

Node A

C:Node B

Shared Disk

Q: Quorum

SQL Server 2008 (Virtual SQL Server)

LocalBinaries

COLTST1SQL Clustering

Master DBTempDBAppl 1 DB

COLTST2

SQL Server 2008 (physical)

SQLConnections

Quorum Disk

Q:VSQLDBARCH\VSQLSRV1

MS DTC

SQL Agent

Publication

Server

distribution

Central Publisher/Remote DistributorReplication model

AdventureWorks

SQL Server 2008

AdventureWorks

SQL Server 2008 Subscription

Server

SQL Server 2008

DistributionServer

Data Replication

“Primary”“Replicate”

Can be used as a Warm Standby and/or for Reporting needs

AdventureWorks

SQL Server 2008 Subscription

Server

“Replicate”

Database Mirroring

MSDB DB

SQL Server 2008

WitnessServer

Adventure Works DB

SQL Server 2008 Mirror Server

translog

PrincipalServer

translog

Adventure Works DB

Network

Client Client ClientClient

Database Mirroring

20FIG34

MSDB DB

SQL Server 2008

WitnessServer

AdventureWorksDB

SQL Server 2008

Mirror Server

translog

PrincipalServer

Networ

Database Snapshot

ting U

AdventureWorksDB

translog

Database Mirroring with DB Snapshots

SQL Server 2008

“Source”

CallOne DB

SQL Server 2008

SecondaryServer

“Destination”

translog

\Backup\CallOne_tlog_200405141120.TRN

TxnLogbackupsPrimary

Server

TxnLog Copies

\LogShare\CallOne_tlog_200405141120.TRN TxnLog Restores

MSDB DB

SQL Server 2008

MonitorServer

“Monitor”

Delay between logs loaded

Delay Answer

Log Shipping

RAID Disk I/O Summary

RAID Level Fault Tolerance Logical Reads

Physical I/Os per

Logical Writes

Physical I/Os per Write

RAID 0 None 1 1 1 1

RAID 1 or 10 Best(Optimal for OLTP)

1 1 1 2 writes

RAID 5 Moderate (Optimal for mostly

READ ONLY systems)

1 1 1 2 reads + 2 writes(that’s 4 per write!)

Several RAID vendors are now showing RAID 5 and RAID 10performance almost equivalent now via Cache/Buffer

advancements on their RAID controllers

Fault Tolerance and SQL DB FilesDescription Fault Tolerance

Quorum Drive The quorum drive used with MSCS should be

isolated to a drive by itself (very often mirrored as well for maximum availability)

RAID 1 or

RAID 10

SQL Server

Database

files (OLTP)

For OLTP (online transaction processing) systems,

the database data/index files should be placed

on a RAID 10 disk system.

RAID 10

SQL Server

Database

files (DSS)

For DSS (Decision Support Systems) systems that

are primarily READ ONLY, the database

data/index files should be placed on a RAID 5

disk system.

RAID 5

Temp DB Highly volatile disk I/O (when not able to do all it’s

work in cache)

RAID 10

SQL Server

Transaction

Log files

The SQL transaction log files should be on their own

mirrored volume for both performance and

database protection. (for DSS systems, this

could be RAID 5 also).

RAID 10

Or RAID 1

Example DB data files configuration

TempDB

OLTP X - DB

Master DB

OLTP Y - DB

DSS - DB(read only)

RAID 10

RAID 5

Q: Quorum

RAID 1 or RAID 10

Database Snapshots

Decision Tree approach

Condition/Question

Case A Case B Case CCase D Case nAction

VAction

WAction

XAction

YAction

. . .. . .

DiskMethods

OtherHW

Cluster Services

Data Replication

SQL Clustering

DatabaseMirroring

LogShipping

DistributedTransactions

Decision-Tree Path Traversal

HW/DiskRedundancy

Cluster Services

Data Replication

SQL Clustering

LogShipping

HANot Needed

ab c d

DatabaseMirroring

DatabaseSnapshots

Decision-Tree: ASP Questions 1-3What % of availability must your application have?

A% <= 70%

MarginalAvailability

70% < A% < =83% 83 < A% < =95% A%> 99.5%

AcceptableAvailability

StandardAvailability

ExtremeAvailability

HighAvailability

95% < A% < =99.5%

How much tolerance of downtime by end-users?

Very High

NotCritical

High Medium Very Low

LowCriticality

StandardCriticality

ExtremelyCritical

HighCriticality

What is the per hour cost of downtime for this application?

$C<= $3K

Very LowCost

$3K < $C < =$7K $7K < $C < =$12K $C > $20K

LowCost

ModerateCost

Very HighCost

HighCost

$12K < $C < =$20K

Decision-Tree: ASP Questions 4-6How long does it take to get the application back online?

Very Long

MarginalRecoverability

Long Average Very Short

AcceptableRecoverability

StandardRecoverability

ExtremeRecoverability

FastRecoverability

How much of the application is distributed?

Non-Distributed

A Little Medium All

LowDistribution

ModeratelyDistributed

ExtremelyDistributed

HighDistribution

How much data inconsistency can be tolerated?

Very Little

Very HighConsistency

A Little Medium Very Much

HighConsistency

ModerateConsistency

MinimalConsistency

LowConsistency

Decision-Tree: ASP Questions 7-9How often is scheduled maintenance required?

Very HighDowntime

Average Very Often

HighDowntime

ReasonableDowntime

MinimalDowntime

LowDowntime

How important is high performance and scalability?

Not Very

Very lowPerformance

Somewhat Moderately Extremely

LowPerformance

ReasonablePerformance

ExtremePerformance

HighPerformance

Very Much

How important is the application connection to the end-user?

NotNeeded

Establish newConnection

ConnectionRetry process

ConnectionFail-over

ConnectionRe-established

easily

RarelyNot Often

Not Very Somewhat Moderately ExtremelyVery Much

Decision-Tree: ASP Question 10What is the estimated cost of the HA Solution (budget)?

C$ < $10K

Very LowCost

$10K <= C$ < $100K

LowCost

ModerateCost

ExtremeCost

HighCost

DiskMethods

OtherHW

Cluster Services

SQL Clustering

$100K <= C$ < $250K $250K <= C$ < $500K C$ >= $500K

1. 1e �� Extreme Availability goal2. 1e+2d �� Very low tolerance of downtime3. 1e+2d+3e �� $15k/hr cost of downtime (High Cost)4. 1e+2d+3e+4c �� Average recovery time5. 1e+2d+3e+4c+5a �� No distributed components or synchronization6. 1e+2d+3e+4c+5a+6b �� A little data inconsistency can be tolerated7. 1e+2d+3e+4c+5a+6b+7c �� Average amount of scheduled downtime8. 1e+2d+3e+4c+5a+6b+7c+8d �� Performance is very much important9. 1e+2d+3e+4c+5a+6b+7c+8d+9b �� Connection can be re-established10. 1e+2d+3e+4c+5a+6b+7c+8d+9b+10c �� Moderate HA Cost/Good budget

Best fitting HA Solution (together)

Basic “one-two” Punch approach

Hardware/NetworkRedundancy

Disk BackupsDB Backups

Vendor SLA’s

Training, QA,& Standards

SoftwareUpgrades

Build the proper foundation first

Then, build within the appropriate HA solution that your application requires

Database Snapshots

DiskMethods

OtherHW

Cluster Services

Data Replication

SQL Clustering

DatabaseMirroring

LogShipping

ASP – Scenario #1 with SQL Clustering

SQL Server 2005 (Virtual SQL Server)

LocalBinaries

ASPProd1

Master DB

TempDB

HOE DB

ASPProd2

SQL Server 2005 (physical)

SQL Server 2005 (physical) Quorum Disk

ASQL\ASPSERV1

MS DTC

SQL Agent

MSCSActive/PassiveConfiguration

SQL Server 2000

“Source”

CallOne DB

SQL Server 2000

SecondaryServer

“Destination”

translog

\Backup\CallOne_tlog_200405141120.TRN

TxnLogbackupsPrimary

Server

TxnLog Copies

\LogShare\CallOne_tlog_200405141120.TRN TxnLog Restores

MSDB DB

SQL Server 2000

MonitorServer

“Monitor”

Delay between logs loaded

Delay Answer

Log Shipping

Live REPL solutionPublication

Server

distribution

Central Publisher/Remote DistributorReplication model

MktgDB

SQL Server 2000

MktgDB

SQL Server 2000Subscription

Server

SQL Server 2000

DistributionServer

Headquarters (Santa Clara)

North America (Reporting & “warm/hot” spare)

MktgDB

Server

Europe (Reporting)

MktgDB

Server

Far East (Reporting)

Publication

Server

distribution

Central Publisher(default option)

Northwind

SQL Server 2000 Northwind

Server

DistributionServer

Northwind

Server

Northwind

Oracle Subscription

Server

Northwind

SQL Server 7.0Subscription

Server

Publication

Server

distribution

Central PublisherRemote Distributor

Northwind

SQL Server 2000 Northwind

Server

Northwind

Server

Northwind

Oracle Subscription

Server

Northwind

SQL Server 7.0Subscription

Server

SQL Server 2000

DistributionServer

Distributing DataData Access Latency Autonomy Sites(locations)

Frequency Network Machines Owner Other

Read OnlyReporting short high many high fast/

stable1

server/site1 OLTP

Each site only needs regionaldata

Central PublisherTransactional replfilter by region

REPLICATION

Read OnlyReporting long high many low fast/

stable1

server/site1 OLTP

Central PublisherSnapshot replfilter by region

Read MostlyA few updates short high < 10 medium fast/

stable1

server/site1 OLTP

Regional updateson one table

Central PublisherTransactional replUpdating Subs

Read MostlyA few updates medium high < 10 medium slow/

unreliab1

server/siteAll

update

Regional updateall tables

Central PublisherMerge repl

Inserts (new orders) short high many high fast/

stable1

server/site

1reportsite

Central SubscriberTransactional repl

Hot/WarmSpare

Veryshort

high < 2 high fast/stable

1server/site

1 OLTPsite

Fail-overCentral PublisherRemote DistributorTransactional repl

Read equalEqual updates short high < 10 medium fast/

stable1

server/siteAll

update

Regional updateall tables

Peer-to-PeerTransactional

Database Mirroring

Piecing it togetherHardware/Network

RedundancyDisk BackupsDB Backups

Vendor SLA’s

Training, QA,& Standards

SoftwareUpgrades

NetworkApplication

MiddlewareDatabase

OperatingSystem

HARDWARENetwork Components

ServersDisk Systems

Memory

System

Foundation, Foundation, Foundation

ROI Calculati

Database Mirroring

MSDB DB

SQL Server 2008

WitnessServer

Applx DB

SQL Server 2008

Mirror Server

translog

PrincipalServer

translog

Applx DB

Network

Transparent Client Redirect

“Copy-on-Write” technology

20FIG30

SQL Server 2005 Database Mirroring

20FIG31

SQL Server 2008 Database Mirroring

Database Mirroring with DB Snapshot

MSDB DB

SQL Server 2005

WitnessServer

Applx DB

SQL Server 2005

Mirror Server

translog

PrincipalServer

translog

Applx DB

Networ

Database Snapshot

ting U

04SQL Server 2008

SQLServer

AdventureWorksDB

SnapshotAdventureWorks

Source DataPages

System Catalogof changed pages

SparseFile

SELECT …..data……. FROM AdventureWorks

SNAPSHOT

Snapshot Users

PH Topology With SnapshotsPH Topology With Snapshots

SQL Server 2008

Adventure Works DB

SQL Server 2008

Mirror Server

translog

PrincipalServer

translog

Adventure Works DB

Instance: SQL2008xyz

Instance: SQL2008zzz

Endpoint Name: “endpoint4mirroring”

Role: PARTNER

PH Topology

SQL Server 2008

PrincipalServer

Active

Passive

NetworkNetwork

Less CriticalReporting Users

Database Snapshot

CriticalReportUsers

The Combo PackSQL Server 2008

PrincipalServer

SQL Server 2008

MirrorServer

SQL Server 2008

WitnessServer

SQL Server 2008

PrincipalServer

SQL Server 2008

MirrorServer

SQL Server 2008

Subscriber

SQL Server 2008

Subscriber

Publisher

Distributor

DB Availability Improvement !

SQL Server 2000

SQLServer

SQL Server 2005/2008

SQLServer

RestartStage

SQL Server 2005/2008database is available

SQL Server 2000database is available

Performance and Tuning counts in HA

SQL SHOT – MS SQL Server

Questions

Is there any time left????

SQL Server Resources

Fail-Over via Move Group

ANSWER to question #1

SQL Server 2000

Northwind 2

SQL Server 2000

Distributed Transactions

“Primary Location”

“Secondary Location”

Northwind 1ReadsTry primary first

If not available, try secondaryUpdates

Must succeed together, or be

both rolled back(two-phase commit)

SQL Server High Availability

Technology