SQL2008 for Developers

Meinrad WeissPrincipal ConsultantTrivadis AG

Trivadis Solution Portfolio

Core technologies Microsoft, Oracle, IBM, Open Source

Core systems

Individualapplications

(SCM, ERP, CRM)

BusinessIntelligenc

eBusiness

Communicati

on

App

licat

ion

Perf

orm

ance

Man

agem

ent

Ap

plica

tion

Develo

pm

en

t

ManagedServices

Security

Train

ing

Trivadis Facts & Figures

Currently 12 locations with over 500 employeesBaden, Basel, Bern, Lausanne, Zurich Düsseldorf, Frankfurt, Freiburg, Hamburg, Munich, Stuttgart Vienna

Financially independent and sustainably profitable

Key figures in 2007Consolidated income: CHF 98 million / EUR 60 millionOver 1‘500 projects with more than 600 clientsOver 125 Service Level AgreementsAbout 5'000 training participants per year Research budget: CHF 6 million / EUR 3.6 million

Agenda

Simplify existing scenarios

Features for new scenarios

Just profit andInfrastructure

Star Join QueryTransparent Data EncryptionResource Governor

Date Types Table Type ParametersMerge StatementGrouping SetsSparse Columns

Spatial TypeFileStream Type

Star Join Query Processing

Fact Table

Dimension 1

Dimension 2

Dimension 3

Dimension 4Sum (Amount) of red Productsin Basel

Star Join Query Processing (2)

Fact Table Scan

Customer Dim Hash Join

Hash Join

Product Dim

Star Join Query Processing (3)

Fact Table Scan

Customer Dim Hash Join

Hash Join

Product Dim

SQL Server 2005can create one

bitmap filter

Bitmap Filter 1“red”

Star Join Query Processing (4)

Fact Table Scan

Customer Dim Hash Join

Hash Join

Product Dim

SQL Server 2008can create multiple

bitmap filtersBitmap Filter 2“Basel”

Bitmap Filter 1“red”

Star Join Query Processing (5)

Fact Table Scan

Customer Dim Hash Join

Hash Join

Product Dim

SQL Server 2008can move and

reorder the filters

Bitmap Filter 2“Basel”

Implemented as Bloom filter http://en.wikipedia.org/wiki/Bloom_filter

The filters can be pushed down into the fact table scan and eliminate almost all the rows that would be eliminated later by the joins

Bitmap Filter 1“red”

Star Join Query Processing (6)

Enterprise Edition FeatureWorks only on parallel plans

FK relationship must be (Trivadis best practices)

One AttributeNot null, Int/Big Int is preferred

In row optimization, Filter is applied during read of phys. Scan

-3x

-2x-5

0%-4

0%-3

0%-2

0%-1

0%Sa

me

10%

20%

30%

40%

50% 2x 3x

0

5

10

15

20

25

30

“Project REAL”

11 Queries Slower

24 Queries Faster

Source: “Christian Kleinerman Microsoft”

Agenda

Simplify existing scenarios

Features for new scenarios

Just profit andInfrastructure

Star Join QueryTransparent Data EncryptionResource Governor

Date Types Table Type ParametersMerge StatementGrouping SetsSparse Columns

Spatial TypeFileStream Type

New Date Data Types

Simplified programming (date, time) Require less resources

SQL Server Max Precision Range ADO.NET

date 1 day 0001-01-01 to 9999-12-31 DateTime

time 100 nanosec 00:00:00 to 23:59:59.9999999

TimeSpan

datetime2 100 nanosec 1-1-1 to 9999-12-31 DateTime

datetimeoffset

100 nanosec 1-1-1 to 9999-12-31 +/-14:00

DateTimeOffset

More functionality (datetime2, datetimeoffset)

Higher precisionTime zone awareness

Date simple, efficient and fast

Allocates only 3 Bytes(instead of 8 Bytes

datetime)

Convert or Cast will prevent any use of an

index

DateTime, Date and Time

Merge

DML statement combining multiple operations into one

Both sides can be a table or a view

source targetMerge

Match: UPDATE

No Match: INSERT

No Source Match: DELETE

XYZ XYZ

Merge

MERGE Stock S

USING Trades T ON S.Stock_ID = T.Stock_ID

WHEN MATCHED AND (Qty + Delta = 0) THEN DELETE -- delete stock if Qty reaches 0

WHEN MATCHED THEN -- delete takes precedence over update

UPDATE SET Qty += Delta

WHEN NOT MATCHED THEN INSERT VALUES (Stock, Delta)

OUTPUT $action, T.Stock, inserted.Delta;

User-Defined Table Types

A user-defined type represents the definition of a table structureIs used to

Declare table-valued parameters for stored procedures or functionsDeclare table variables

Table-valued Parameter

New parameter type for Stored Procedures and Functions

Data are passed by reference to avoid overhead of a copy

Must be passed as Read-Only Parameters

Replace often temp tablesLess overhead and Cleaner DesignCan be used in select from

RestrictionsNo column statisticsCannot be used in Select Into

Table-valued Parameter (2)

Excellent .Net Client Support

Grouping Sets

One query that produces Multiple groupingsReturns a single result set

Result set is equivalent to a UNION ALL of differently grouped rowsSimplifies writing reports with multiple groupings With improved query performance

Grouping Set (2)select DimDate.calendaryear as Year ,DimDate.calendarquarter as Quarter ,DimTerritory.salesterritorycountry as Country ,sum(FactSales.salesamount) as SalesAmountfrom dbo.factresellersales FactSales inner join dbo.dimtime DimDate on FactSales.orderdatekey = DimDate.timekeyinner join dbo.dimsalesterritory DimTerritory on FactSales.salesterritorykey = DimTerritory.salesterritorykey where DimDate.calendaryear in (2003,2004) group by grouping sets ((calendaryear, calendarquarter, salesterritorycountry) ,(calendaryear, calendarquarter) ,())order by DimDate.calendaryear, DimDate.calendarquarter, DimTerritory.salesterritorycountry

Sparse Columns & Filtered Indexes

Property Bag Scenarios:Distinct Customized property sets associated with data

Property Name, Value pairsLarge number of unique properties, user annotationsExamples

Content/collaboration systems, Content storesShare Point Server, Documentum

Databases with heterogeneous Record Types in one Table

Type specific properties, inherited properties in a type hierarchyExamples

Product Catalogs (commerce Server, Amazon) Location/business specific properties(Virtual Earth)

Sparse Column (1)

Simple implementation of class hierarchies in relational tables

create table Person(ID int not null primary key ,Name varchar(255) not null,BirthDay Date not null,[Type] varchar(25) not null ,Salary money sparse,Department varchar(25) sparse,Rating varchar(2) sparse,LastContact Date sparse )

,...,LastContact Date sparse ,TypeProperties XML COLUMN_SET FOR ALL_SPARSE_COLUMNS)

Sparse Column Details

0 Bytes stored for NULL Values~20% CPU overhead for non-null value access Additional 2- 4 bytes for non-null values

Sparse columns are beneficial when space savings >40%Tables can support large number of sparse columns

30K columns, 1K indexes, 30K statisticsRequires the presence/addition of a sparse column set when # of columns >1024

Select * will return all non-sparse columns + sparse_column_ set

Certain types of columns can not be marked as sparse

Computed columns UDT IdentityRowGuidFileStream etc

Sparse Column (3)

select *from PersonWithout_COLUMN_SET

select *from Person

select ID, Name, BirthDay, [Type], Salary, Department, Rating, LastContact from Person

Filtered Indexes with regular table

Size

TraditionalIndex

US

FR

CHFL………

This Index will never be used

for ‘US‘, 'FR‘, … queries

Not selective enough

CHFL……

Index Nationality

<> 'US', 'FR'

Filtered Index

…

Table

Agenda

Simplify existing scenarios

Features for new scenarios

Just profit andInfrastructure

Star Join QueryTransparent Data EncryptionResource Governor

Date Types Table Type ParametersMerge StatementGrouping SetsSparse Columns

Spatial TypeFileStream Type

Spatial Data Types

geography data type• Geodetic (“Round Earth”)

geospatial model• Define points, lines, and areas

with longitude and latitude• Account for planetary

curvature and obtain accurate “great circle” distances

geometry data type• Planar (“Flat Earth”)

geospatial model• Define points, lines, and

areas with coordinates• Use for localized areas or

non-projected surfaces such as interior spaces

2D Vector Support in SQL Server 2008

Spatial Data Types (2)

Implemented as large CLR user-defined type (UDT)

Methods for computingSTGeomFromText('LINESTRING (100 250, 150 100)', 0)STIntersection(<Spatial Object>)STIntersects(<Spatial Object>)…

SQLServer: Geometry, Geography.Net: SqlGeometry, SqlGeography

Geometry Following Open Geospatial Consortium Simple features for SQL

(OGC/SQL MM, ISO 19125)

Sample

Sample (2)

Spatial Data Multi-Level Grid Indexes

4.2.3

Index on Spatial Types

Requires Clustered unique index on table

id geometry

1 g1

2 g2

3 g3

Base Table T Internal Table for Indexid cell_id

1 7

3 7

3 8

3 9

3 10

1 13

2 20

FileStream

ID: 123Name: HansAge: 32

Mixed (Database + Filesystem)

BLOB in Database

FileStream

ID: 123Name: HansAge: 32

ID: 123Name: HansAge: 32

CheapStreaming PerformanceSynchronizationDouble Management

Integrated ManagementTransact. ConsistencySize Limits (2 GB)Heavy load on Log File

Integrated ManagementTransact. ConsistencyStreaming SupportRead Committed Isolation

FileStream (2)

Unstructured data stored directly in the file system

Requires NTFS

Dual Programming ModelTSQL (Same as SQL BLOB)Win32 Streaming APIs with T-SQL transactional semantics

Data ConsistencyIntegrated Manageability

Back Up / RestoreAdministration e.g. Security

Size limit is the file system volume size

Dual Interface

Begin Transactionselect Get_FileStream_transaction_context() as TxCtx ,PERS_CV.PathName() as UNCPathfrom Person

The Keys to the Win32 Interface

FileStream

Win32 Applicatio

n

IOManage

r

Rdr Srv

SQL Server FS Agent

NTFS

Client SQL Server 2008User Mode

IO Manage

r

SQL Server FS Driver

Kernel Mode

Active Transaction

TxCtx

Performance

FileStream Limitations

Failover clusteringFileStream Filegroups must be put on a shared disk

Some Replication LimitsSee http://msdn2.microsoft.com/en-us/library/bb895334(SQL.100).aspx

SQL Server does not support database snapshots for FileStream data files

Rest of the database can be ‘snapshoted’Database mirroring does not support FileStream

Log Shipping is supported ;-)

FileStream

Don’t forget your DBAInsert, Update and Delete operations have minimal impact on Transaction Log File of DBBut they will affect the size (and time) of Backups

Both Database and TX-Log Backups

FileStream Data can only be stored on “local” Server drivesNever loose the FileStream Data drive

If you loose it then you have the choice between

A probably inconsistent DatabaseLost Transactions

Conclusion

SQL Server 2008 has many improvements for Developers (and also BI and DBA)Simplify existing work

ProductivityOpen the door for new scenarios

Beyond relationalImprove Security and ReliabilityThe new features are already very stable and well documented

“Es git en heisse Summer hür, …”

More Infos

Trivadis TechnoCircleSQL Server 2008:

Was ist neu und relevant für DBA’s und IT ProfessionalsDie essentiellen Neuerungen für EntwicklerDas Herz einer modernen Business Intelligence Plattform

Next Generation Data Centric Applications with Visual Studio 2008 and SQL Server 2008

Mit Buchtaufe von Christian Nagel

Demo an unserem StandWPF goes Windows Trivadis FrameworkDatenbank-Setup “Best Practices” und “SharePoint’s Umsetzung”

Wettbewerb und Gutschein

© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after

the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.