ADO dot Net Advanced PAtterns

8/7/2019 ADO dot Net Advanced PAtterns

1/26


2/26

About Me

Independent Software Developer

20 years developing

Microsoft .NET MVP

ASPInsider

INETA Speaker

Various publications & conferences

Blogs: thedatafarm.com/blog, blog.ziffdavis.com/devlife

Founder and leader ofVermont .NET

INETAAdvisory Board Member


3/26

Agenda

ADO.NET 2.0 DataSet performance

Pushing huge data back to SQL Server

Caching huge amounts of data Querying huge amounts of data in memory


4/26

ADO.NET 2.0 Features for this talk

Batch Updating with DataAdapter

SqlBulkCopy Class

SqlDependency Class


5/26

ADO.NET 2.0 Engine Improvements

DataSet Scalability across the board

Can handle lots of rows

Indexing Engine completely re-written Incremental indexing updates

much more

Low overhead for using Data

Views

Serialization: Faster, more compact

Supports Binary & Schema-less serialization


6/26

DEMO

Indexing Engine Performance


7/26

Moving Huge Amounts of Data

Getting data is easy and fast!

Uploading is a different story

1.1 Update Options DataAdapter.Update one row at a time

Controlled stored procedure update one row at a

time


8/26

Updating in 2.0

DataAdapterBatch Update

Pushes groups of rows up to SQL Server

SQL Server still updates one at a time Find sweet spot for UpdateBatchSize value

Bulk Copy

Streamed Insert using SQL ServerBulk Copy

*slide slightly modified from original printed version


9/26

DEMO

Batch Update & Bulk Copy Class


10/26


11/26

DEMO

Using BCL & DML to process huge

amounts of data


12/26

DataSet Caching

DataSet improvements enable storing lots

of data into cache

SqlDependency enables automatednotification of changes to .NET

Combine SqlDependency and Caching for

serious resource reduction


13/26

DEMO

SqlDependency & DataSet Cache

Part 1


14/26

Big DataSet cache begs granularity

SqlDependency is all or nothing

Create separate dependencies for logical

sections of your cache


15/26

DEMO

SqlDependency & DataSet Cache

Part 2: Achieve granularity


16/26

Divide & Conquer your huge dataset

In-Memory Querying is coming!

LiNQ

VB9

/C#3

What about today?

ADO.NET 2.0 has everything you need to

build a simple query processor

Great example of QP from ADO.NET Team

Technical Lead Pablo Castro (for demo only)


17/26

High Level view of Query Processor

Familiar Query Syntax eg SELECT, WHERE, JOIN

Query is parsed to Algebra Tree Some of the Algebraic Operators

Scan: enumeration over the logical tree Projection: identifying what columns to return

Filter, Join, Sort

Just a data structure

Iterator is created for each operator Does the actual work of processing the data and extracting what

is defined by operator Projection Iterator returns all the rows but only containing the

desired fields

Filter iterator processes all the rows and returns only rows matchingfilter rules


18/26

Roll your own Query Processor

cached

data

ProjectIterator

cached

data

Join

iterator

Narrowed

down to

specific

columns

Flattened tomatch criteria

Enhanced

with related

data


19/26

Our Custom QP Iterators

Scan Enumerates over dataset rows (eg leaf-nodes of tree)

Select (Project)

DataSet Expressions can pull out columns Filter

DataTable.Select for on-the-fly indexing

Consider DataView for a permanent index and its raw.NET 2.0 power

DataTable.Select can leverage this index Sort

Bundle this with the filter leverage DataTable.Select or DataView and indexing


20/26

Use DataViews new power for JOIN

JOIN will be your weakest link in a query

operation

Iterates through joined table for every row in

primary table

Try a DataView instead

DataViews indexing will speed this up


21/26

Beware Operator Constraints

DataView/DT.Select uses == and !=

need to be creative for implementing things

like >, IN orBETWEEN

Cant leverage indexing

Roll your own?


22/26

DEMO

A look at a sample Query

Processor written with

ADO.NET 2.0


23/26

Summary

ADO.NET 2.0s performance andfunctional improvements put new power inour hands

You can work with huge amounts of dataefficiently

Functionality like BatchUpdate gives you

easy access to some of this power With a little more code, you can do

wonders!


24/26

Resources

PDC05 Session DAT408, Pablo Castro,

ADO.NET Team Technical Lead

ADO.NET Team Blog:blogs.msdn.com/dataaccess

MSDN Data Access Dev Center

msdn.microsoft.com/data

Whats New in ADO.NET 2.0, Julia Lerman,

MSDNMagazine April 2005


25/26


26/26

Contact Info

Julie Lerman

[email protected]

Blogs

www.thedatafarm.com/blogblogs.ziffdavis.com/devlife

Date post:	08-Apr-2018
Category:	Documents
Upload:	gagansharma
View:	226 times
Download:	0 times

ADO dot Net Advanced PAtterns

Documents