37
Building Scalable Applications With SQL Server Chris Adkin
| Date post: | 09-Dec-2014 |
| Category: |
Technology |
| Upload: | chris-adkin |
| View: | 820 times |
| Download: | 4 times |
Building Scalable Applications With
SQL ServerChris Adkin
15 years of IT experience 14 years of DBA experience
across a variety of vertical sectors.
Over 10 years experience of SQL Server from 2000.
Contact◦ Email:
[email protected]◦ Twitter: ChrisAdkin8
About me
The following material contains general rules of thumb.
Always test your assumptions.There will always be edge cases in which conventional wisdom may not apply.
Disclaimer
Characteristic OLTP OLAP
I/O Random Serial
Transactions Short Longer during ETL
Execution plans Simple Complex
Plan nature Serial with index seeks
Parallel with index/table /partition scans
SQL Complexity Simpler Complex, often generated by BI tools
Number of queries Finite Highly variable
(re)compiles impact on performance
Critical Execution plan run time more important
SQL Selectivity Queries typically only affect a few rows
Queries aggregate many rows and affect many rows during ETL
OLAP Compared To OLTP
Where the foundation for scalability is laid.
Consider your most performance critical queries and the number of joins they require.
#1 Schema Design
Data model design “Usual suspects”:◦Over normalisation.◦1:1 relationships, where most / all the attributes from both entities are retrieved together.
◦Logical data model sprawl, as the model evolves over time.
#1 Schema Design
1:1 relationships edge cases:◦Sparse columns, which cannot be compressed or used in clustered indexes.
◦Infrequently accessed columns with wide data types.
#1 Schema Design
“Walking with keys” We only want columns from tables A and E and have to go via B, C and D in order to facilitate the join.
The exception is when entities are introduced to resolve many to many relationships.
#1 Schema Design
A EDCB
Test drive your logical model design as early as possible in the project.
Failure to do so may result in performance issues built into the foundations of the application.
#1 Schema Design
“Chattiness” => heavy network conversation volume between software components.
Chatter takes place via linked servers and /or between tiers.
More time is spent on the network than anywhere else.
#2 “Chatty” Designs
Low IIS and SQL Server CPU utilisation is a symptom.
Cache static / relatively static data outside the database.
Put CRUD intensive operations in stored procedures.
Always put the processing closest to the resource that requires it.
#2 “Chatty” Designs
There is a wider debate about where it is best to put the business logic.
This is beyond the scope of this presentation, however you may wish to look at:◦To sp Or Not sp by Adam Machanic◦Pet vsPet: MS opens .Net benchmarking wars
#2 “Chatty” Designs
Start with a minimal set of indexes and then add.
Use filtered indexes for selective SQL. Indexing GUIDs:
◦Causes page splits and fragmentation◦GUIDS are large, 16 bytes◦Inflates the size of secondary indexes created on clustered indexes.
# 3 Indexing
‘Scatter gun’ indexing strategies can lead to indexes that add little or no value.
Glenn Berry’s diagnostics script includes a query to highlight indexes with lots of writes and little or no reads.
# 3 Indexing
For GUIDs and clustered indexes, refer to “The Clustered Index Debate” by SQL Skills.
If you really need GUIDs, consider:◦NEWSEQUENTIALID().◦SSD technology as blogged about by Tony Rogerson.
Otherwise avoid GUIDs in general.
# 3 Indexing
Know your critical indexes and when to rebuild and re-organise them.
The maintenance plan index rebuild tasks rebuilds ALL indexes.
Use index fragmentation and index read intensity to determine rebuild / reorg order.
Some good free tools are available, the procedures from Ola Hallengren to name one.
# 4 Index Maintenance
A compile takes place when a plan cannot be found for a statement in the plan cache.
Recompiles take place when the execution engine deems that a plan is no longer valid.
All are to be avoided where possible.
# 5 Compiles and Recompiles
Factor Causes A Compile
Causes A Recompile
Use Of temp tables √
Changes in statistics √
DBCC FREEPROCCACHE / instance restart
√
Schema changes √
Use of the RECOMPILE hint √
Non parameterised SQL √
Dynamic SQL √
Plans aged out the plan cache √
Changes in SET options (Refer to Erland Sommarskog’s blog)
√
# 5 Compiles and Recompiles
# 5 Compiles and Recompiles
There are two hints that help to minimise recompiles.
The KEEP PLAN hint causes the execution engine to treat temporary tables as permanent tables.
The KEEPFIXED PLAN hint will prevent recompiles if there are changes to the statistics of base tables used in a query.
Forced parameterisation can minimise compiles, but use it with caution.
There are many causes of RBAR.Cursors.User defined functions, table level functions are the exception.
EXISTS sub queries.Recursive CTEs. . . . and many more, search on SQL Server Central for RBAR
#6 Row By Agonising Row (RBAR)
Avoid RBAR wherever possible.Row by agonising row, as coined by Jeff Moden, is covered extensively in his articles on SQL Server Central.
#6 Row By Agonising Row
Segregate different types of querying activity.
I/O best practise is to segregate random from sequential I/O.
Segregate OLAP/reporting queries from OLTP queries.
# 7 Segregate OLAP and OLTP Workloads
ETL aside, OLTP applications are more write intensive than OLAP ones.
Optimisations for reads can decimate write performance:◦Heavy indexing◦Covering indexes◦Index views◦Column store indexes◦Aggregation
# 7 Segregate OLAP and OLTP Workloads
The Command Query Responsibility Segregation model provides a solution.
Queries insert data into a database optimised for fast writes.
The same data is queued (service broker).
The data is de-queued and written to a database optimised for fast reads.
# 7 Segregate OLAP and OLTP Workloads
An online retailer services thousands of concurrent customers.
A single table holds the balance for the retailer in a single row.
If all transactions can update the balance table at the same time, this creates a unique point of serialisation.
# 8 Unique Points Of Serialisation
Queue the debits credits from the user transactions, two options.
Table with a clustered index, as per Remus Rasanu’s blog and DELETE OUTPUT destructive read.
Service broker is the 2nd option.De-queue values and update the table.
# 8 Unique Points Of Serialisation
Prior to SQL Server 2005, unless dirty reads are used, readers block writers.
This throttles back OTLP through put. The solution; multi-version concurrency control.
Facilitated by two new isolation levels available in SQL Server 2005 onwards.
# 9 Locking
Read committed snapshot◦Data is consistent with beginning of the statement.
◦An extension of the read committed isolation level.
Snapshot isolation◦Data is consistent with the beginning of transaction.
# 9 Locking
Snapshot / RCSI isolation cannot be enabled for databases with filestream file groups.
Queries may have to traverse the row version store => more logical and physical reads.
This Microsoft paper provides guidance as to which isolation level to use.
# 9 Locking
14 bytes of information is added to each row in the database as it is modified or created.
Snapshot isolation can lead to update conflicts, retry logic is required to work around this.
Ensure tempdb resides on fast disks, as this is where the version store lives.
# 9 Locking
Latches protect the integrity of internal SQL Server memory structures.
Three latch wait types:◦Buffer (BUF) : PAGELATCH_* wait type◦Non buffer : LATCH_* wait type◦IO latch : PAGEIOLATCH wait type
# 10 Latching
When multiple threads try to obtain the same latch in incompatible modes, contention takes place.
Some latch contention is to be expected.
Refer to this SQLCAT paper for detailed information on latch trouble shooting.
# 10 Latching
If problems exist, long PAGELATCH_* and PAGEIOLATCH waits will be visible.
Common causes of latch contention include:◦Heavy CRUD activity affecting rows in the same page.
◦Solution is to pad rows out with char[X] columns.
# 10 Latching
An index has a leading column populated by an identity, sequence etc.
Solution: hash partition the table on the indexes leading column.
# 10 Latching
Diagnosing and Resolving Latching Issuesby SQL Customer Advisory Team
Designing High Scale OLTP Systems by Thomas Kejser
Lessons Learned From 128 Core OLTP Testing
by Thomas Kejser
Further Reading
Questions ?
