+ All Categories
Home > Documents > Introduction to ETL Using Microsoft Tools By Dr. Gabriel.

Introduction to ETL Using Microsoft Tools By Dr. Gabriel.

Date post: 14-Dec-2015
Category:
Upload: kalyn-ellender
View: 236 times
Download: 0 times
Share this document with a friend
Popular Tags:
34
Introduction to ETL Using Microsoft Tools By Dr. Gabriel
Transcript

Introduction to ETL Using Microsoft Tools

By Dr. Gabriel

About SSIS

• Microsoft’s ETL tool• Solutions created in packages• Developed in Business Intelligence

Development Studio (BIDS)– Variation of visual studio

How to create SSIS project within BIDS

• Click on Start->All Programs->Microsoft SQL Server 2008> SQL server Business Intelligence Studio

• From the File menu select New->Project• In the Project Type, choose Business Intelligence

Projects->Integration Services Project• Enter name of the project and select storage

location• Click OK button

How to execute SSIS projects

• Within BIDS, right click on the project name in the solution explorer and choose Execute Package– Output window shows results

• Outside of BIDS– dtexec.exe– dtexecui.exe– SQL Server Management Studio– SQL Server Agent Scheduler– Custom .Net application

Tasks We Will Cover

• Bulk insert– Bulk insert data from text files into SQL server

database

• Data flow– transforms, cleans, and modifies data as it is

moved from source to destination

• Execute SQL– Executes SQL statements in specified databases

Connection Managers

• Is a connection to a data source• Must be configured for each data source and

destination• We will use OLE DB and flat file connection

managers– Others connection managers are available

Configuring Connection Manager

• Open a package• Click Control Flow tab• Right click Connection Manager area, then

click New Connection or choose a new connection of a specific type

Flat File Connection Manager

• Configure General, Columns, and Advanced sections

• Use Preview option to view how you configurations were applied

OLE DB Connection Manager

• Using BIDS 2008:– Select Native OLE DB/SQL Server Native Client

10.0– Enter server name– Enter login info– Select database– Test connection– Click OK

Data Migration Best Practices

• Data types from source must match data types from destination– Conversions may be necessary

• Do conversions as early as possible• Bring over only data fields that need to be

loaded into DW

Bulk Insert Task

• Fastest way to copy data to SQL Server• No transformations can be performed when

copying data• Usually is used to bring raw data from sources

into Staging databases in the DW environment• Configure General, Connection, Option, and

Expression sections

Execute SQL Task

• Executes one or more SQL statements or stored procedures

• Executing batches is possible by placing GO command to separate batches

• SQL Statements can be entered directly into the task editor window or may reside in a file or a variable

• May return result that can be captured in a variable• May specify parameters for queries– Use variables

Precedence Constraints• Specify order of task execution• 3 types

– On success• green

– On failure• red

– On completion• Blue

• Configured on the control flow tab– Drag a green arrow that is coming out of a task– Connect it to another tasks– Right click on it to specify type. Default is On Success

Data Flow Task

• Transforms, cleans, and modifies data as it is moved from source to destination

• Is added to the package on the Control Flow tab• Its elements are configured on the Data Flow

tab• Package may contain multiple data flow tasks• Elements are grouped into Source,

Transformation, and Destination categories

Transformations

• Numerous transformations are available• We will look at Lookup transformation only– Essential for loading fact tables– Lookup data from dimension table to retrieve key

values to be loaded into fact table– Caching can be used (either full or partial)

Overall Flow• Wipe out staging table

– Execute SQL task

• Wipe out DW table if data is fully refreshed– Execute SQL task– Do not do this if data should be appended

• Load raw data into staging table– Bulk Insert task/Data flow task

• Load data from staging tables into dimensions– Execute SQL statement task/Data flow task

• Load data into fact table– Data flow task

• Create aggregations and/or OLAP cubes– Execute SQL task and/or SSAS

Load Data Into Fact Table: A closer Look

• Create OLEDB source– use an sql command to select data from staging

table

• Create lookup transformation for each key to dimension table that will be stored in the fact table

• Create OLEDB destination– specify fact table name – Specify mapping

Useful SQL Scripts

• Check if a table exists prior to creating itif not exists (select * from information_schema.tableswhere table_name=‘xyz’)

Useful SQL Scripts• Generate Dates dimension at daily grainDECLARE @gencalendar TABLE (cal_date DATETIME PRIMARY KEY)DECLARE @p_date SMALLDATETIME

SET @p_date = '20010101'

WHILE @date <= '20151231'BEGIN INSERT INTO @gencalendar(cal_date) VALUES(@p_date)

SET @date = dateadd(d, 1, @p_date)END

SELECT * FROM @gencalendar

Useful SQL Scripts• If Dates dimension has key in a YYYMMDD format, this script populates

entire tableDECLARE @p_date SMALLDATETIMESET IDENTITY_INSERT dimdates ONSET @p_date = '19000101'

WHILE @p_date <= '20191231'BEGIN INSERT INTO

dimdates(Datekey,CalendarDate,Calendaryear,Calendarhalfyear,Calendarquarter,Calendarmonth,Calendarday) select CAST(convert(varchar(8),@p_date,112) as int) , CAST(convert(varchar(10),@p_date,101) as date), DATEPART(YY,@p_date) , Case When DATENAME(QQ, @p_date) < 3 then 1 else 2 END, DATEPART (QQ, @p_date), DATEPART(MM,@p_date), DATEPART (DD, @p_date) SET @p_date = dateadd(d, 1, @p_date)END

Useful SQL Scripts• If Dates dimension has key in a YYYMMDD format and the date key

has already been inserted, this script populates remaining columns in this dimension

Update dimDate set CalMonthName = CAST(datekey as varchar) where isEndOfYear = 9

Update dimDate Set CalDate = Cast(substring(CalMonthName,5,2)+'/'+substring(CalMonthName,7,2) +'/'+substring(CalMonthName,1,4) as DATE) where isEndOfYear = 9

Update dimDate Set CalYear = DATEPART(YY,CalDate) ,CalMonth = DATEPART(MM,CalDate) ,CalMonthName = DATENAME (MM, CalDate) + ' ''' + RIGHT(Cast(CalYear as CHAR(4)),2) ,CalQuarter = DATENAME (QQ, CalDate) ,CalQuarterName = CAST(CalYear as CHAR(4)) +' Q' + CAST(CalQuarter as CHAR(1)) ,CalHY = Case When DATENAME(QQ, CalDate) < 3 then CAST(CalYear as CHAR(4)) +' H1' else CAST(CalYear as CHAR(4)) +' H2' END, isEndOfYear = 8 where isEndOfYear = 9

Useful SQL Scripts

• Calculate person’s agecreate function dbo.fCalculateAge(@DOB datetime,@Date datetime) returns int asbeginreturn ( select case when month(@DOB)>month(@Date) then datediff(yyyy,@DOB,@Date)-1 when month(@DOB)<month(@Date) then datediff(yyyy,@DOB,@Date) when month(@DOB)=month(@Date) then case when day(@DOB)>day(@Date) then datediff(yyyy,@DOB,@Date)-1 else datediff(yyyy,@DOB,@Date) end end) end

Useful SQL Scripts

• Sample query for loading data into dimension table

INSERT INTO DimXXX (Col1,Col2)select distinct stgcol1,stgcol2From Staging.dbo.tblWhere ltrim(rtrim(stgcol1))+'|'+ltrim(rtrim(stgcol2)) not in(select

distinct ltrim(rtrim(col1))+'|'+ltrim(rtrim(col2))from DimXXX)

Useful SQL Scripts

• Parsing names in a Last Name, First Name, Middle Name,Suffix format separated by comma, with Middle Name and suffix being optional

create table tmpnames (fullname varchar(255))

The script is presented on the next slide.

select fullname,CASE WHEN charindex(',', fullname) > 1THEN substring(fullname, 1, charindex(',', fullname)-1)WHEN charindex(',', fullname) =0 THEN fullnameEND as lname,CASE WHEN charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))) > 1THEN substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))), 1, charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-1)WHEN charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))<=1 and charindex(',', fullname) >0THEN ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))ELSE ''END as fname,CASE WHEN charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-

charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))))))) > 1and substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',',

fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),1,charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-1) not in ('Sr','Jr','II','III','IV')

THEN substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),1,charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-1)

WHEN charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))<=1 and charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))) >0

and ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))) not in ('Sr','Jr','II','III','IV')

THEN ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))))))

ELSE ''END as mname,CASE WHEN charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)

+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))+1,len(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))))))))))) > 1

and substring(ltrim(rtrim(substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))+1,len(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))))),1,charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))+1,len(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))))))-1) in ('Sr','Jr','II','III','IV')

THEN substring(ltrim(rtrim(substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))+1,len(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))))),1,charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))+1,len(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))))))-1)

WHEN charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))<=1 and charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))) >0

THEN ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))))))

WHEN charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))+1,len(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))))))<=1

and charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',',

fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))))))) >0THEN ltrim(rtrim(substring(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-

charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))),charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))+1,len(ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))))))-charindex(',', ltrim(rtrim(substring(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))),charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))+1,len(ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname)))))-charindex(',', ltrim(rtrim(substring(fullname,charindex(',', fullname)+1,len(fullname)-charindex(',', fullname))))))))))))

ELSE ''END as namesuffixfrom tmpnames

OLAP

• SSAS is used– Cubes– KPIs– Data mining engine– Actions

Cubes• is a multidimensional structure that contains dimensions and measures• Dimensions define the structure of the cube, and measures provide the numerical

values of interest to the end user. • a cube allows a client application to retrieve values as if cells in the cube defined

every possible summarized value. • Cell positions in the cube are defined by the intersection of dimension members

and contain aggregated measures• Cubes can be queried using MDX (Multidimensional Expressions), not SQL

KPI

• Key performance indicators• From business perspective, it is a quantifiable

measurement for gauging business success• a KPI is a collection of calculations that are

associated with a measure group in a cube that are used to evaluate business success

Data Mining Engine

• Uses mathematical analysis to extract patterns and trends from that exist in data

• Some scenarios:– Forecasting sales– Targeting mailings toward specific customers– Determining which products are likely to be sold

together– Finding sequences in the order that customers

add products to a shopping cart

Actions

• SSAS commands that are used by clients– Types:

• Drillthrough actions, which return the set of rows that represents the underlying data of the selected cells of the cube where the action occurs.

• Reporting actions, which return a report from Reporting Services that is associated with the selected section of the cube where the action occurs.

• Standard actions, which return the action element (URL, HTML, DataSet, RowSet, and other elements) that is associated with the selected section of the cube where the action occurs.

Creating Cubes

• Click on Start->All Programs->Microsoft SQL Server 2008> SQL server Business Intelligence Studio

• From the File menu select New->Project• In the Project Type, choose Business

Intelligence Projects->Analysis Services Project• Enter name of the project and select storage

location• Click OK button

Creating Cubes

• Create a new data source– Right click on Data Sources and choose new data source

• Create a new data source view– Right click on Data Source Views and choose new data

source view• Create Dimensions– Right-click on Dimensions and choose new dimension

• Create cubes– Right click on Cubes and choose new cube

Creating Cubes

• Once development completed, it is deployed and processed– Deploy=creating cube structure on the server– Process=data calculations, aggregations, etc.– Right click on the cube name in the solution

explorer and choose Process (prior to this step right click the project name in solution explorer, choose properties and specify deployment location)• Cube is ready to be consumed by users

Questions ?


Recommended