+ All Categories
Home > Documents > SSIS Cheatsheet

SSIS Cheatsheet

Date post: 02-Apr-2015
Category:
Upload: babukomin
View: 753 times
Download: 26 times
Share this document with a friend
1
B.I. MADE SIMPLE http://www.pragmaticworks.com CONVERT. EXTEND. DOCUMENT. DEVELOP. TM Common SSIS Problems and Solutions Problems Solutions Loop over a list of files & load each one Tasks Required: Foreach Loop, Data Flow Task Solution: Configure the Foreach Loop to loop over any particular directory of files. e loop should be configured to output to a given variable. Map the given variable to a connection manager by using expressions. Conditionally executing tasks Solution: Double-click the precedence constraint and set the Evaluation property to Expression and Constraint. Type the condition that you want to evaluate in the Expression box. Pass in variables when scheduling or running a package Solution: Use the /SET command in the DTExec command line or change the Property tab in the Package Execution Utility to have the property path like: \Package.Variables[User::VariableName].Properties[Value] Move & rename the file at the same time Tasks Required: File System Task Solution: Set the File System Task to rename the file and point to the directory you’d like to move it to. is enables you to rename and move the file in the same step. Loop over an array of data in a table & perform a set of tasks for each row Tasks Required: Execute SQL Task, Foreach Loop Solution: Use an Execute SQL Task to load the array and send the data into an object variable. Loop over the variable in a Foreach Loop by using an ADO Enumerator. Perform an incremental load of data Tasks Required: 2 Execute SQL Tasks, Data Flow Task Solution: Have the 1st Execute SQL Task retrieve a date from a control table of when the target table was last loaded and place that into a variable. In the Data Flow Task, create a date range on your query using the variable. en, update the control table using a 2nd Execute SQL Task to specify when the table was last updated. Perform a conditional update & insert Components Required: Data Flow Task, Conditional Split, Lookup Transform or Merge Join, OLE DB Command Transform Solution: Use the Lookup Transform or Merge Join to determine if the row exists on the destination and ignore a failed match. If the row yields blank on the key, then you know the row should be inserted into target (by Condtional Split). Otherwise, the row is a duplicate or an update. Determine if the row is an update by comparing the source value to the target value in the Conditional Split. e update can be done by an OLE DB Command Transform or by loading the data into a staging table. SSIS Expression Cheat Sheet Problems Expression Create a file name with today’s date Use a 2 digit date (ex. “03” for March instead of “3”) Multiple condition if statement. Return the first five characters from a zip code Remove a given character from a string (ex. Remove “-” from a social security number) Uppercase data Replace NULL with another value Remove any non- numeric data from a column Convert text to proper case (ex. 1st leer in each word is uppercase) Expression on the Flat File or File Connection Manager: “C:\\Project\\MyExtract”+(DT_WSTR, 30)(DT_DBDATE) GETDATE()+“.csv” Expression Output Example: C:\Projects\MyExtract2009-03-20.csv RIGHT(”0”+(DT_WSTR,2)MONTH(GETDATE()),2) Expression Output: 03 (if the month is March) In this example, the statement determines that if the ColumnName is blank or NULL, it will be set to unknown. To make a Logical AND condition, use “&&” instead of the “||” operator. ISNULL(ColumnName) || TRIM(ColumnName) == “”? “Unknown” : ColumnName Derived Column Transform in the Data Flow: SUBSTRING(ZipCodePlus4, 1, 5) Derived Column Transform in the Data Flow: REPLACE(SocialSecurityNumber, “-”, “”) Derived Column Transform in the Data Flow: UPPER(ColumnName) Derived Column Transform in the Data Flow: ISNULL(ColumnName) ? “New Value” : ColumnName Replace blanks with NULL values Derived Column Transform in the Data Flow: TRIM(ColumnName) == "" ? (DT_STR, 4, 1252) NULL (DT_STR, 4, 1252) : ColumnName Script Transform in the Data Flow Task with the code as follows ( VB 2008): Imports System.Text.RegularExpressions Public Overrides Sub Input0_ProcessInputRows(ByVal Row As Input0Buffer) If Row.ColumnName_IsNull = False Or Row.ColumnName = “” Then Dim pattern As String = String.Empty Dim r As Regex = Nothing pattern = “[^0-9]” r = New Regex(pattern, RegexOptions.Compiled) Row.ColumnName = Regex.Replace(Row.ColumnName, pattern, “”) End If End Sub Script Transform with the line of partial code as follows: Row.OutputName = StrConv(Row.InputName, VBStrConv.ProperCase) Build dynamic SQL statement Expression on the SQLStatementSource property of Execute SQL Task: "SELECT Column FROM " + @[User::TableName] + " WHERE DateFilterColumn = '" + (DT_WSTR,4)YEAR(@[User::DateTimeVar]) + RIGHT("0" + (DT_WSTR,2)MONTH(@[User::DateTimeVar]), 2) + RIGHT("0" + (DT_WSTR,2)DAY(@[User::DateTimeVar]), 2) + "’" Expression Output: SELECT Column FROM MyTable WHERE DateFilterColumn = '20060915' Calculate beginning of the previous month Expression on component or task: (DT_DATE)(DT_DBDATE)DATEADD ("dd",-1 * (DAY( GETDATE() )-1), DATEADD("month",-1, GETDATE() )) Round to the nearest two decimal mark Expression on Derived Column Transform: ROUND(YourNumber, 2) Expression Output Example: 1.2600000 B.I. Tools and Training from Industry Recognized Experts You Trust
Transcript
Page 1: SSIS Cheatsheet

B.I. MADE SIMPLEhttp://www.pragmaticworks.com

CONVERT. EXTEND.DOCUMENT.DEVELOP.

TM

Common SSIS Problems and SolutionsProblems SolutionsLoop over a list of �les& load each one

Tasks Required: Foreach Loop, Data Flow Task Solution: Con�gure the Foreach Loop to loop over any particular directory of �les. �e loop should be con�gured to output to a given variable. Map the given variable to a connection manager by using expressions.

Conditionally executing tasks

Solution: Double-click the precedence constraint and set the Evaluation property to Expression and Constraint. Type the condition that you want to evaluate in the Expression box.

Pass in variables when scheduling or running a package

Solution: Use the /SET command in the DTExec command line or change the Property tab in the Package Execution Utility to have the property path like: \Package.Variables[User::VariableName].Properties[Value]

Move & rename the �le at the same time

Tasks Required: File System Task Solution: Set the File System Task to rename the �le and point to the directory you’d like to move it to. �is enables you to rename and move the �le in the same step.

Loop over an array of data in a table & perform a set of tasks for each row

Tasks Required: Execute SQL Task, Foreach Loop Solution: Use an Execute SQL Task to load the array and send the data into an object variable. Loop over the variable in a Foreach Loop by using an ADO Enumerator.

Perform an incremental load of data

Tasks Required: 2 Execute SQL Tasks, Data Flow Task Solution: Have the 1st Execute SQL Task retrieve a date from a control table of when the target table was last loaded and place that into a variable. In the Data Flow Task, create a date range on your query using the variable. �en, update the control table using a 2nd Execute SQL Task to specify when the table was last updated.

Perform a conditional update & insert

Components Required: Data Flow Task, Conditional Split, Lookup Transform or Merge Join, OLE DB Command Transform Solution: Use the Lookup Transform or Merge Join to determine if the row exists on the destination and ignore a failed match. If the row yields blank on the key, then you know the row should be inserted into target (by Condtional Split). Otherwise, the row is a duplicate or an update. Determine if the row is an update by comparing the source value to the target value in the Conditional Split. �e update can be done by an OLE DB Command Transform or by loading the data into a staging table.

SSIS Expression Cheat SheetProblems ExpressionCreate a �le name with today’s date

Use a 2 digit date (ex. “03” for March instead of “3”)Multiple condition if statement.

Return the �rst �ve characters from a zip codeRemove a given character from a string (ex. Remove “-” from a social security number)Uppercase data

Replace NULL with another value

Remove any non-numeric data from a column

Convert text to proper case (ex. 1st le�er in each word is uppercase)

Expression on the Flat File or File Connection Manager:“C:\\Project\\MyExtract”+(DT_WSTR, 30)(DT_DBDATE)GETDATE()+“.csv”Expression Output Example: C:\Projects\MyExtract2009-03-20.csvRIGHT(”0”+(DT_WSTR,2)MONTH(GETDATE()),2)Expression Output: 03 (if the month is March)

In this example, the statement determines that if the ColumnName is blank or NULL, it will be set to unknown. To make a Logical AND condition, use “&&” instead of the “||” operator.

ISNULL(ColumnName) || TRIM(ColumnName) == “”? “Unknown” : ColumnNameDerived Column Transform in the Data Flow:SUBSTRING(ZipCodePlus4, 1, 5)

Derived Column Transform in the Data Flow:REPLACE(SocialSecurityNumber, “-”, “”)

Derived Column Transform in the Data Flow:UPPER(ColumnName)Derived Column Transform in the Data Flow:ISNULL(ColumnName) ? “New Value” : ColumnName

Replace blanks with NULL values

Derived Column Transform in the Data Flow:TRIM(ColumnName) == "" ? (DT_STR, 4, 1252) NULL (DT_STR, 4, 1252) : ColumnNameScript Transform in the Data Flow Task with the code as follows ( VB 2008):

Imports System.Text.RegularExpressions

Public Overrides Sub Input0_ProcessInputRows(ByVal Row As Input0Buffer) If Row.ColumnName_IsNull = False Or Row.ColumnName = “” Then Dim pattern As String = String.Empty Dim r As Regex = Nothing pattern = “[^0-9]” r = New Regex(pattern, RegexOptions.Compiled) Row.ColumnName = Regex.Replace(Row.ColumnName, pattern, “”) End IfEnd Sub

Script Transform with the line of partial code as follows:

Row.OutputName = StrConv(Row.InputName, VBStrConv.ProperCase)

Build dynamic SQL statement

Expression on the SQLStatementSource property of Execute SQL Task:"SELECT Column FROM " + @[User::TableName] + " WHERE DateFilterColumn = '" + (DT_WSTR,4)YEAR(@[User::DateTimeVar]) + RIGHT("0" + (DT_WSTR,2)MONTH(@[User::DateTimeVar]), 2) + RIGHT("0" + (DT_WSTR,2)DAY(@[User::DateTimeVar]), 2) + "’"Expression Output: SELECT Column FROM MyTable WHERE DateFilterColumn = '20060915'

Calculate beginning of the previous month

Expression on component or task:(DT_DATE)(DT_DBDATE)DATEADD("dd",-1 * (DAY( GETDATE() )-1), DATEADD("month",-1, GETDATE() ))

Round to the nearest two decimal mark

Expression on Derived Column Transform:ROUND(YourNumber, 2) Expression Output Example: 1.2600000

B.I. Tools and Training from Industry Recognized Experts

You Trust

Recommended