+ All Categories
Home > Documents > Power CenterTraining

Power CenterTraining

Date post: 02-Jun-2018
Category:
Upload: vijayinfor
View: 216 times
Download: 0 times
Share this document with a friend

of 287

Transcript
  • 8/10/2019 Power CenterTraining

    1/287

    1

    Informatica PowerCenter 7.1

    Basics

    Education ServicesVersion PC7B-20041208

    Informatica Corporation, 2003 - 2004. All rights reserved.

  • 8/10/2019 Power CenterTraining

    2/287

    Introduction

  • 8/10/2019 Power CenterTraining

    3/287

    3

    By the end of this course you will:

    Understand how to use the major PowerCentercomponents for development

    Be able to build basic ETL mappings and mapplets*

    Be able to create, run and monitor workflows

    Understand available options for loading target data

    Be able to troubleshoot most problems

    Note: The course does not cover PowerCenter optionalfeatures or XML support.

    Course Objectives

    * A mapplet is a subset of a mapping

  • 8/10/2019 Power CenterTraining

    4/287

    4

    Founded in 1993

    Leader in enterprise solution products

    Headquarters in Redwood City, CA

    Public company since April 1999 (INFA)

    2000+ customers, including over 80% of Fortune 100

    Strategic partnerships with IBM, HP, Accenture, SAP,

    and many others Worldwide distributorship

    About Informatica

  • 8/10/2019 Power CenterTraining

    5/287

    5

    Informatica Products

    PowerCenter ETLbatch and real-time data integration

    PowerAnalyzer BI reportingweb-browser interface with reports,dashboards, indicators, alerts; handles real-timemetrics

    SuperGlue* Centralized metadata browsing cross-enterprise,including PowerCenter, PowerAnalyzer, DBMS,BI tools, and data modeling tools

    PowerExchange Data access to mainframe, mid-size system and

    complex filesPowerCenterConnect products

    Data access to transactional applications andreal-time services

    * Uses PowerCenter to extract metadata and PowerAnalyzer to display reports

  • 8/10/2019 Power CenterTraining

    6/287

    6

    www.informatica.comprovides information (under Services) on:

    Professional Services

    Education Services

    my.informatica.comsign up to access:

    Technical Support

    Product documentation (under Toolsonline documentation) Velocity Methodology (under Services)

    Knowledgebase

    Webzine

    Mapping templates

    devnet.informatica.comsign up for Informatica Developers Network

    Discussion forums

    Web seminars

    Technical papers

    Informatica Resources

  • 8/10/2019 Power CenterTraining

    7/287

    7

    Informatica offers three distinct Certification titles:

    Exam A: Architecture and Administration Exam C: Advanced Administration

    Exam A: Architecture and Administration Exam B: Mapping Design Exam D: Advanced Mapping Design

    Exams A, B, C, D plus Exam E: Enablement Technologies

    For more information and to register to take an exam:http://www.informatica.com/services/Education+Services/Professional+Certification/

    Informatica Professional Certification

    http://www.informatica.com/services/Education+Services/Professional+Certification/http://www.informatica.com/services/Education+Services/Professional+Certification/
  • 8/10/2019 Power CenterTraining

    8/287

    8

    Extract, Transform and Load

    Transaction level data Optimized for transaction

    response time

    Current

    Normalized orDe-normalized data

    Operational Systems

    MainframeRDBMS Other

    Aggregated dataHistorical data

    Decision Support

    Data

    Warehouse

    ETL Load

    Transform

    Extract

    Aggregate dataCleanse dataConsolidate dataApply business rulesDe-normalize data

  • 8/10/2019 Power CenterTraining

    9/287

    9

    PowerCenter Client Tools

    Repository Designer Workflow Workflow Rep ServerManager Manager Monitor Administration

    Console

    Manage repository:ConnectionsFolders

    ObjectsUsers and groups

    Administer repositories on aRepository Server:Create/upgrade/deleteConfigurationStart/stopBackup/restore

    Build ETLmappings

    Build and startworkflows torun mappings

    Monitorand startworkflows

  • 8/10/2019 Power CenterTraining

    10/287

    10

    PowerCenter 7 Architecture

    Not Shown: Client ODBC connections from Designer to sources and targets for metadata

    TargetsSources

    Native Native

    TCP/IP

    HeterogeneousTargets

    Repository

    RepositoryServer

    RepositoryAgent

    TCP/IP

    Native

    Informatica Server

    HeterogeneousSources

    Repository Designer Workflow Workflow Rep ServerManager Manager Monitor Administrative

    Console

  • 8/10/2019 Power CenterTraining

    11/287

    11

    Distributed Architecture and Platforms

    The following components can be distributed across anetwork of host computers: Clients Tools

    PowerCenter Servers

    Repository Servers Repository Databases

    Sources and Targets

    Platforms:

    Client tools run on Windows Servers run on AIX, HP-UX, Solaris, Redhat Linux, Windows

    Repositories on any major RDBMS

  • 8/10/2019 Power CenterTraining

    12/287

    12

    Design and Execution Process

    1. Create Source definition(s)

    2. Create Target definition(s)

    3. Create a Mapping

    4. Create a Session Task

    5. Create a Workflow with Task components

    6. Run the Workflow and verify the results

  • 8/10/2019 Power CenterTraining

    13/287

    13

    Demonstration

  • 8/10/2019 Power CenterTraining

    14/287

    Source Object Definitions

  • 8/10/2019 Power CenterTraining

    15/287

    15

    Source Object Definitions

    By the end of this section you will:

    Be familiar with the Designer interface

    Be familiar with Source Types Be able to create Source Definitions

    Understand Source Definition properties

    Be able to use the Data Preview option

  • 8/10/2019 Power CenterTraining

    16/287

    16

    Import from:Relational databaseFlat fileXML object

    Create manually

    Methods of Analyzing Sources

    Source Analyzer

    Repository

    RepositoryServer

    Repository Agent

    TCP/IP

    DEF

    Native

  • 8/10/2019 Power CenterTraining

    17/287

    17

    Analyzing Relational Database Sources

    TableViewSynonym

    Relational DB Source

    DEF

    Source AnalyzerODBC

    Repository

    RepositoryServer

    Repository Agent

    TCP/IP

    DEF

    Native

  • 8/10/2019 Power CenterTraining

    18/287

    18

    Analyzing Relational Database Sources

    Editing Source Definition Properties

  • 8/10/2019 Power CenterTraining

    19/287

    19

    Analyzing Flat File Sources

    Mapped DriveNFS MountLocal Directory DEF

    Fixed Width Delimited

    Flat FileSource Analyzer

    Repository

    RepositoryServer

    Repository Agent

    TCP/IP

    DEF

    Native

  • 8/10/2019 Power CenterTraining

    20/287

    20

    Flat File Wizard

    Three-stepwizard

    Columns canbe renamed

    within wizard Text, Numeric

    and Datetimedatatypes are

    supported Wizard

    guesses

    datatype

  • 8/10/2019 Power CenterTraining

    21/287

    21

    Flat File Source Properties

  • 8/10/2019 Power CenterTraining

    22/287

    22

    Analyzing XML Sources

    DEF

    XML Schema (XSD),DTD or XML File

    DATA

    Source Analyzer

    Repository

    RepositoryServer

    Repository Agent

    TCP/IP

    DEF

    Native

    Mapped DriveNFS MountingLocal Directory

  • 8/10/2019 Power CenterTraining

    23/287

  • 8/10/2019 Power CenterTraining

    24/287

    24

    Using Data Previewer inSource Analyzer

    Data Preview ExampleFrom Source Analyzer,select Source drop downmenu, then Preview Data

    Enter connection informationin the dialog box

    A right mouse click on the object can also be used to preview data

  • 8/10/2019 Power CenterTraining

    25/287

  • 8/10/2019 Power CenterTraining

    26/287

    26

    Metadata Extensions

    Allows developers and partners to extend themetadata stored in the Repository

    Metadata extensions can be:

    User-definedPowerCenter users can define and createtheir own metadata

    Vendor-definedThird-party application vendor-createdmetadata lists

    For example, applications such as Ariba or PowerCenter Connect

    for Siebel can add information such as contacts, version, etc.

  • 8/10/2019 Power CenterTraining

    27/287

    27

    Metadata Extensions

    Can be reusable or non-reusable

    Can promote non-reusable metadata extensions toreusable; this is irreversible (except by Administrator)

    Reusable metadata extensions are associated withall repository objects of that object type

    A non-reusable metadata extensions is associatedwith a single repository object

    Administrator or Super User privileges are requiredfor managing reusable metadata extensions

  • 8/10/2019 Power CenterTraining

    28/287

    28

    ExampleMetadata Extension for a Source

    Sample User DefinedMetadata, e.g. contact

    information, business user

  • 8/10/2019 Power CenterTraining

    29/287

    Target Object Definitions

  • 8/10/2019 Power CenterTraining

    30/287

    30

    Target Object Definitions

    By the end of this section you will:

    Be familiar with Target Definition types

    Know the supported methods of creating TargetDefinitions

    Understand individual Target Definition properties

  • 8/10/2019 Power CenterTraining

    31/287

    31

    Creating Target Definitions

    Methods of creating Target Definitions

    Import from relational database

    Import from XML object

    Create automatically from a source definition

    Create manually (flat file or relational database)

  • 8/10/2019 Power CenterTraining

    32/287

    32

    Import Definition from Relational Database

    Can obtain existing object definitions from a databasesystem catalog or data dictionary

    Table

    ViewSynonym

    WarehouseDesigner

    Relational DB

    DEF

    ODBC

    Repository

    RepositoryServer

    Repository AgentTCP/IP

    DEF

    Native

  • 8/10/2019 Power CenterTraining

    33/287

    33

    Import Definition from XML Object

    Can infer existing object definitions from a databasesystem catalog or data dictionary

    WarehouseDesigner

    Repository

    RepositoryServer

    Repository AgentTCP/IP

    DEF

    Native

    DEF

    DTD, XML Schema orXML File

    DATA

    Mapped DriveNFS MountingLocal Directory

  • 8/10/2019 Power CenterTraining

    34/287

  • 8/10/2019 Power CenterTraining

    35/287

    35

    Target Definition Properties

  • 8/10/2019 Power CenterTraining

    36/287

  • 8/10/2019 Power CenterTraining

    37/287

    37

    Heterogeneous Targets

    By the end of this section you will be familiar with:

    Heterogeneous target types

    Heterogeneous target limitations

    Target conversions

  • 8/10/2019 Power CenterTraining

    38/287

    38

    Definition: Heterogeneous Targets

    Supported target definition types:

    Relational database

    Flat file

    XML

    Targets supported by PowerCenter Connects

    Heterogeneous targets are targets within a singleSession Task that have different types or have differentdatabase connections

  • 8/10/2019 Power CenterTraining

    39/287

    39

    Step One: Identify Different Target Types

    Oracle table

    Flat file

    Oracle tableTables are EITHER in twodifferent databases, orrequire different (schema-specific) connect strings

    One target is a flat file load

  • 8/10/2019 Power CenterTraining

    40/287

    40

    Step Two: Different Database Connections

    The two databaseconnections aredifferent

    Flat file requiresseparate locationinformation

  • 8/10/2019 Power CenterTraining

    41/287

    41

    Target Type Override (Conversion)

    Example: Mapping has SQL Server target definitions.Session Task can be set to load Oracle tables instead,using an Oracle database connection.

    The following overrides are supported:

    Relational target to flat file target

    Relational target to any other relational database type

    CAUTION: If target definition datatypes are not compatible with datatypes innewly selected database type, modify the target definition

  • 8/10/2019 Power CenterTraining

    42/287

    42

    Lab 1Define Sources and Targets

  • 8/10/2019 Power CenterTraining

    43/287

    43

    Transformation Basic Concepts

  • 8/10/2019 Power CenterTraining

    44/287

    44

    Transformations Objects Used in This Class

    Source Qualifier: reads data from flat file & relational sourcesExpression: performs row-level calculations

    Filter: drops rows conditionally

    Sorter: sorts data

    Aggregator:performs aggregate calculations

    Joiner: joins heterogeneous sources

    Lookup: looks up values and passes them to other objects

    Update Strategy: tags rows for insert, update, delete, rejectRouter: splits rows conditionally

    Sequence Generator: generates unique ID values

  • 8/10/2019 Power CenterTraining

    45/287

    45

    Other Transformation Objects

    Normalizer: normalizes records from relational or VSAM sources

    Rank: filters the top or bottom range of records

    Union: merges data from multiple pipelines into one pipeline

    Transaction Control: allows user-defined commits

    Stored Procedure: calls a database stored procedureExternal Procedure : calls compiled code for each row

    Custom: calls compiled code for multiple rows

    Midstream XML Parser: reads XML from database table or message queue

    Midstream XML Generator: writes XML to database table or message queue

    More Source Qualifiers: read from XML, message queues andapplications

  • 8/10/2019 Power CenterTraining

    46/287

    46

    Transformation Views

    A transformation hasthree views:

    Iconizedshows thetransformation in relation

    to the rest of themapping

    Normalshows the flowof data through thetransformation

    Editshowstransformation ports(= table columns)and properties;allows editing

  • 8/10/2019 Power CenterTraining

    47/287

    47

    Expression Transformation

    Ports Mixed

    Variables allowed

    Create expression in anoutput or variable port

    Usage

    Perform majority ofdata manipulation

    Perform calculations using non-aggregate functions(row level)

    Click here to invoke theExpression Editor

  • 8/10/2019 Power CenterTraining

    48/287

  • 8/10/2019 Power CenterTraining

    49/287

    49

    Expression Validation

    The Validate or OK button in the Expression Editor will:

    Parse the current expression

    Remote port searching (resolves references to ports inother transformations)

    Parse default values

    Check spelling, correct number of arguments in functions,other syntactical errors

  • 8/10/2019 Power CenterTraining

    50/287

    50

    Character Functions Used to manipulate character data

    CHRCODE returns the numeric value(ASCII or Unicode) of the first character of

    the string passed to this function

    CONCAT is for backward compatibility only.Use || instead

    ASCIICHRCHRCODECONCATINITCAPINSTRLENGTH

    LOWERLPADLTRIMREPLACECHRREPLACESTRRPAD

    RTRIMSUBSTRUPPER

    Informatica FunctionsCharacter

  • 8/10/2019 Power CenterTraining

    51/287

    51

    TO_CHAR (numeric)

    TO_DATETO_DECIMALTO_FLOATTO_INTEGER

    Informatica FunctionsConversion

    Conversion Functions Used to convert datatypes

  • 8/10/2019 Power CenterTraining

    52/287

    52

    Informatica FunctionsData Cleansing

    INSTRIS_DATEIS_NUMBERIS_SPACESISNULLLTRIMMETAPHONE

    REPLACECHRREPLACESTRRTRIMSOUNDEXSUBSTRTO_CHARTO_DATE

    TO_DECIMALTO_FLOATTO_INTEGER

    Used to process data during datacleansing

    METAPHONE and SOUNDEX createindexes based on Englishpronunciation (2 different standards)

  • 8/10/2019 Power CenterTraining

    53/287

    53

    Date Functions

    Used to round, truncate, orcompare dates; extract one part

    of a date; or perform arithmeticon a date

    To pass a string to a datefunction, first use the TO_DATE

    function to convert it to andate/time datatype

    ADD_TO_DATEDATE_COMPAREDATE_DIFFGET_DATE_PARTLAST_DAYROUND (Date)SET_DATE_PARTTO_CHAR (Date)TRUNC (Date)

    Informatica FunctionsDate

  • 8/10/2019 Power CenterTraining

    54/287

    54

    Numerical Functions Used to perform mathematical

    operations on numeric data

    ABSCEILCUMEEXPFLOORLNLOGMODMOVINGAVGMOVINGSUMPOWERROUNDSIGN

    SQRTTRUNC

    COSCOSHSINSINH

    TANTANH

    Scientific Functions

    Used to calculategeometric values

    of numeric data

    Informatica FunctionsNumerical and Scientific

  • 8/10/2019 Power CenterTraining

    55/287

    55

    Informatica FunctionsSpecial and Test

    ABORTDECODEERROR

    IIFLOOKUP

    IIF(Condition,True,False)

    IS_DATE

    IS_NUMBERIS_SPACESISNULL

    Test Functions

    Used to test if a lookup result is null

    Used to validate data

    Special Functions

    Used to handle specific conditionswithin a session; search for certainvalues; test conditional statements

  • 8/10/2019 Power CenterTraining

    56/287

    56

    Variable Ports

    Use to simplify complex expressions e.g. create and store a depreciation formula to be

    referenced more than once

    Use in another variable port or an output port expression

    Local to the transformation (a variable port cannot also be aninput or output port)

  • 8/10/2019 Power CenterTraining

    57/287

    57

    Variable Ports (contd)

    Use for temporary storage

    Variable ports can remember values across rows; useful for comparing

    values Variables are initialized (numeric to 0, string to ) when the Mapping

    logic is processed

    Variables Ports are not visible in Normal view, only in Edit view

  • 8/10/2019 Power CenterTraining

    58/287

    58

    Default ValuesTwo Usages

    For input and I/O ports, default values are used to replacenullvalues

    For output ports, default values are used to handle transformationcalculation errors (not-null handling)

    Default

    value for theselectedport

    Selectedport Validate the

    defaultvalueexpression

    ISNULL functionis not required

  • 8/10/2019 Power CenterTraining

    59/287

    59

    Informatica Datatypes

    Transformation datatypes allow mix and matchof source and target database types When connecting ports, native and transformation datatypes must be compatible

    (or must be explicitly converted)

    NATIVE DATATYPES TRANSFORMATION DATATYPES

    Specific to the source and targetdatabase types

    PowerCenter internal datatypes

    Display in source and target tableswithin Mapping Designer

    Display in transformations withinMapping Designer

    Native NativeTransformation

  • 8/10/2019 Power CenterTraining

    60/287

    60

    Datatype Conversions within PowerCenter

    Data can be converted from one datatype to another by: Passing data between ports with different datatypes

    Passing data from an expression to a port

    Using transformation functions

    Using transformation arithmetic operators

    Only conversions supported are:

    Numeric datatypes Other numeric datatypes

    Numeric datatypes String

    Date/Time Date or String

    For further information, see the PowerCenter Client Help > Index> port-to-port data conversion

  • 8/10/2019 Power CenterTraining

    61/287

    Mappings

  • 8/10/2019 Power CenterTraining

    62/287

    62

    Mappings

    By the end of this section you will be familiar with:

    The Mapping Designer interface

    Transformation objects and views

    Source Qualifier transformation

    The Expression transformation

    Mapping validation

  • 8/10/2019 Power CenterTraining

    63/287

    63

    Mapping Designer

    Iconized Mapping

    Mapping List

    Transformation Toolbar

  • 8/10/2019 Power CenterTraining

    64/287

    64

    Source Qualifier Transformation

    Ports All input/output

    Usage Convert datatypes For relational sources:

    Modify SQL statementUser Defined JoinSource Filter

    Sorted portsSelect DISTINCTPre/Post SQL

    Represents the source record set queried by the

    Server. Mandatory in Mappings using relational orflat file sources

  • 8/10/2019 Power CenterTraining

    65/287

    65

    Source Qualifier Properties

    User can modify SQL SELECT statement (DB sources)

    Source Qualifier can join homogenous tables

    User can modify WHERE clause

    User can modify join statement

    User can specify ORDER BY (manually orautomatically)

    Pre- and post-SQL can be provided

    SQL properties do not apply to flat file sources

  • 8/10/2019 Power CenterTraining

    66/287

    66

    Pre-SQL and Post-SQL Rules

    Can use any command that is valid for the databasetype; no nested comments

    Use a semi-colon (;) to separate multiple statements

    Informatica Server ignores semi-colons within singlequotes, double quotes or within /* ...*/

    To use a semi-colon outside of quotes or comments,escape it with a back slash (\)

  • 8/10/2019 Power CenterTraining

    67/287

    67

    Mapping Validation

  • 8/10/2019 Power CenterTraining

    68/287

    68

    Connection Validation

    Examples of invalid connections in a Mapping:

    Connecting ports with incompatible datatypes

    Connecting output ports to a Source

    Connecting a Source to anything but a SourceQualifier or Normalizer transformation

    Connecting an output port to an output port oran input port to another input port

    i i i

  • 8/10/2019 Power CenterTraining

    69/287

    69

    Mapping Validation

    Mappings must:

    Be valid for a Session to run

    Be end-to-end complete and contain valid expressions

    Pass all data flow rules

    Mappings are always validated when saved; can be validatedwithout being saved

    Output Window displays reason for invalidity

  • 8/10/2019 Power CenterTraining

    70/287

    70

    Lab 2Create a Mapping

  • 8/10/2019 Power CenterTraining

    71/287

    Workflows

  • 8/10/2019 Power CenterTraining

    72/287

    W kfl M I f

  • 8/10/2019 Power CenterTraining

    73/287

    73

    Workflow Manager Interface

    TaskTool Bar

    OutputWindow

    NavigatorWindow

    Workspace

    StatusBar

    WorkflowDesignerTools

    W kfl M T l

  • 8/10/2019 Power CenterTraining

    74/287

    74

    Workflow Designer Maps the execution order and dependencies of Sessions,

    Tasks and Worklets, for the Informatica Server

    Task Developer Create Session, Shell Command and Email tasks

    Tasks created in the Task Developer are reusable

    Worklet Designer

    Creates objects that represent a set of tasks

    Worklet objects are reusable

    Workflow Manager Tools

    W kfl St t

  • 8/10/2019 Power CenterTraining

    75/287

    75

    Workflow Structure

    A Workf lowis set of instructions for the InformaticaServer to perform data transformation and load

    Combines the logic of Session Tasks, other types ofTasks and Worklets

    The simplest Workflow is composed of a Start Task, a

    Link and one other Task

    StartTask

    SessionTask

    Link

  • 8/10/2019 Power CenterTraining

    76/287

    Reusable Tasks

    R bl T k

  • 8/10/2019 Power CenterTraining

    77/287

    77

    Reusable Tasks

    Three types of reusableTasks

    SessionSet of instructionsto execute a specificMapping

    CommandSpecific shellcommands to run duringany Workflow

    EmailSends email during

    the Workflow

    R bl T k

  • 8/10/2019 Power CenterTraining

    78/287

    78

    Reusable Tasks

    Use the Task Developertocreate reusable tasks

    These tasks will then appearin the Navigator and can bedragged and dropped intoany workflow

    R bl T k i W kfl

  • 8/10/2019 Power CenterTraining

    79/287

    79

    Reusable Tasks in a Workflow

    In a workflow, a reusable task is representedwith the symbol

    ReusableNon-reusable

    C d T k

  • 8/10/2019 Power CenterTraining

    80/287

    80

    Command Task

    Specify one or more Unix shell or DOS commands torun duringthe Workflow

    Runs in the Informatica Server (UNIX or Windows)environment

    Command task status (successful completion orfailure) is held in the pre-defined task variable$command_task_name.STATUS

    Each Command Task shell command can executebefore the Session begins or after the InformaticaServer executes a Session

    Command Task

  • 8/10/2019 Power CenterTraining

    81/287

    81

    Command Task

    Specify one (or more) Unix shell or DOS (NT, Win2000)commands to run at a specific point in the workflow

    Becomes a component of a workflow (or worklet)

    If created in the Task Developer, the Command task is

    reusable If created in the Workflow Designer, the Command task is

    not reusable

    Commands can also be invoked under the Components

    tab of a Session task to run pre- or post-session

    Command Task (contd)

  • 8/10/2019 Power CenterTraining

    82/287

    82

    Command Task (contd)

    Command Task (contd)

  • 8/10/2019 Power CenterTraining

    83/287

    83

    Command Task (contd)

    Add Cmd

    Remove Cmd

    Email Task

  • 8/10/2019 Power CenterTraining

    84/287

    84

    Email Task

    Configure to have the Informatica Server to send emailat any point in the Workflow

    Becomes a component in a Workflow (or Worklet)

    If configured in the Task Developer, the Email Task is

    reusable (optional) Emails can also be invoked under the Componentstab

    of a Session task to run pre- or post-session

    Email Task (contd)

  • 8/10/2019 Power CenterTraining

    85/287

    85

    Email Task (contd)

  • 8/10/2019 Power CenterTraining

    86/287

  • 8/10/2019 Power CenterTraining

    87/287

  • 8/10/2019 Power CenterTraining

    88/287

    Assignment Task

  • 8/10/2019 Power CenterTraining

    89/287

    89

    Assignment Task

    Assigns a value to a Workflow Variable Variables are defined in the Workflow object

    Expressions Tab

    General Tab

    Timer Task

  • 8/10/2019 Power CenterTraining

    90/287

    90

    Timer Task

    Waits for a specified period of time to execute thenext Task

    General Tab

    Absolute Time

    Datetime Variable

    Relative Time

    Timer Tab

    Control Task

  • 8/10/2019 Power CenterTraining

    91/287

    91

    Control Task

    Stop or ABORT the Workflow

    GeneralTab

    Properties Tab

    Event Wait Task

  • 8/10/2019 Power CenterTraining

    92/287

    92

    Event Wait Task

    Pauses processing of the pipeline until aspecified event occurs

    Events can be:

    Pre-definedfile watch

    User-definedcreated by an Event Raise task elsewhere inthe workflow

    Event Wait Task (contd)

  • 8/10/2019 Power CenterTraining

    93/287

    93

    Event Wait Task (cont d)

    General TabProperties Tab

  • 8/10/2019 Power CenterTraining

    94/287

  • 8/10/2019 Power CenterTraining

    95/287

    Session Task

  • 8/10/2019 Power CenterTraining

    96/287

    96

    Session Task

    Server instructions to run the logic of ONE specific mappinge.g. source and target data location specifications, memoryallocation, optional Mapping overrides, scheduling, processing andload instructions

    Becomes a component of a

    Workflow (or Worklet) If configured in the Task

    Developer, the SessionTask is reusable (optional)

    Additional Workflow Tasks

  • 8/10/2019 Power CenterTraining

    97/287

    97

    Eight additional Tasks are available in the Workflow Designer (covered

    later)

    Command

    Email

    Decision

    Assignment

    Timer

    Control

    Event Wait

    Event Raise

    Additional Workflow Tasks

    Sample Workflow

  • 8/10/2019 Power CenterTraining

    98/287

    98

    Sample Workflow

    Start Task(required)

    Session 1

    Session 2

    Command

    Task

    Sequential and Concurrent Workflows

  • 8/10/2019 Power CenterTraining

    99/287

    99

    Sequential and Concurrent Workflows

    Sequential

    Concurrent Combined

    Note: Although only session tasks are shown, can be any tasks

  • 8/10/2019 Power CenterTraining

    100/287

    Workflow Properties

  • 8/10/2019 Power CenterTraining

    101/287

    101

    Workflow Properties

    Customize WorkflowProperties

    Workflow log displays

    May be reusable ornon-reusable

    Select a WorkflowSchedule (optional)

    Workflow Scheduler

  • 8/10/2019 Power CenterTraining

    102/287

    102

    Workflow Scheduler

    Set and customize workflow-specific schedule

    Workflow Metadata Extensions

  • 8/10/2019 Power CenterTraining

    103/287

    103

    Wo ow etadata te s o s

    Metadata Extensions providefor additional user data

  • 8/10/2019 Power CenterTraining

    104/287

    Conditional Links

  • 8/10/2019 Power CenterTraining

    105/287

    105

    Optional linkcondition

    $taskname.STATUSis a pre-defined

    task variable

    Workflow Variables 1

  • 8/10/2019 Power CenterTraining

    106/287

    106

    Task-specificvariables

    Built-in systemvariables

    Used in decision tasks and conditional linksedit task or link:

    User-defined variables(see separate slide)

    Pre-defined variables

    Workflow Variables 2

  • 8/10/2019 Power CenterTraining

    107/287

    107

    User-defined variables are set in Workflow properties,

    Variables tabcan persist across sessions

    Can be reset inan Assignment task

    Workflow Summary

  • 8/10/2019 Power CenterTraining

    108/287

    108

    y

    1. Add Sessions and other Tasks to the Workflow

    2. Connect all Workflow components with Links

    3. Save the Workflow

    Sessions in a Workflow can be executed independently

    4. Start the Workflow

  • 8/10/2019 Power CenterTraining

    109/287

    Session Tasks

  • 8/10/2019 Power CenterTraining

    110/287

    110

    After this section, you will be familiar with:

    How to create and configure Session Tasks

    Session Task source and target properties

    Creating a Session Task

  • 8/10/2019 Power CenterTraining

    111/287

    111

    Created to execute the logic of a mapping (one

    mapping only)

    Session Tasks can be created in the Task Developer(reusable) or Workflow Developer (Workflow-specific)

    To create a Session Task

    Select the Session button from the Task Toolbar

    OrSelect menu Tasks | Create and select Session from thedrop-down menu

    g

    Session TaskProperties and Parameters

  • 8/10/2019 Power CenterTraining

    112/287

    112

    p

    Session Task

    Sessionparameter

    Properties Tab

    Parameter file

    Session TaskSetting Source Properties

  • 8/10/2019 Power CenterTraining

    113/287

    113

    g

    Set

    properties

    Session Task

    Selectsourceinstance

    Mapping Tab

    Setconnection

  • 8/10/2019 Power CenterTraining

    114/287

  • 8/10/2019 Power CenterTraining

    115/287

    Monitoring Workflows

    Monitoring Workflows

  • 8/10/2019 Power CenterTraining

    116/287

    116

    By the end of this section you will be familiar with:

    The Workflow Monitor GUI interface

    Monitoring views

    Server monitoring modes Filtering displayed items

    Actions initiated from the Workflow Monitor

    Truncating Monitor Logs

    Workflow Monitor

  • 8/10/2019 Power CenterTraining

    117/287

    117

    The Workflow Monitor is the tool for monitoring

    Workflows and Tasks Choose between two views:

    Gantt chart

    Task view

    Gantt Chart view Task view

    Monitoring Current and Past Workflows

  • 8/10/2019 Power CenterTraining

    118/287

    118

    The Workflow Monitor displays only workflowsthat have been run

    Displays real-time information from theInformatica Server and the Repository Server

    about current workflow runs

    Monitoring Operations

  • 8/10/2019 Power CenterTraining

    119/287

    119

    Perform operations in the Workflow Monitor

    Stop, Abort, or Restart a Task, Workflow or Worklet Resume a suspended Workflow after a failed Task is

    corrected

    Reschedule or Unschedule a Workflow

    View Session and Workflow logs Abort has a 60 second timeout

    If the Server has not completed processing andcommitting data during the timeout period, the threadsand processes associated with the Session are killed

    Stopping a Session Task means the Server stops reading data

    Monitoring in Task View

  • 8/10/2019 Power CenterTraining

    120/287

    120

    Start CompletionTask Server Workflow Worklet Time Time

    Status Bar

    Start, Stop, Abort, ResumeTasks,Workflows and Worklets

    Filtering in Task View

  • 8/10/2019 Power CenterTraining

    121/287

    121

    Monitoring filterscan be set usingdrop down menus.Minimizes itemsdisplayed in

    Task View

    Right-click on Session to retrieve theSession Log (from the Server to thelocal PC Client)

    Filter Toolbar

  • 8/10/2019 Power CenterTraining

    122/287

    122

    Display recent runs

    Filter tasks by specified criteria

    Select servers to filter

    Select type of tasks to filter

    Truncating Workflow Monitor Logs

  • 8/10/2019 Power CenterTraining

    123/287

    123

    Workflow Monitor

    Repository Manager

    Repository Managers

    Truncate Log optionclears the WorkflowMonitor logs

  • 8/10/2019 Power CenterTraining

    124/287

    124

    Lab 3Create and Run a Workflow

  • 8/10/2019 Power CenterTraining

    125/287

    125

    Lab 4Features and Techniques I

  • 8/10/2019 Power CenterTraining

    126/287

    Debugger

  • 8/10/2019 Power CenterTraining

    127/287

    127

    By the end of this section you will be familiar with: Creating a Debug Session

    Debugger windows and indicators

    Debugger functionality and options Viewing data with the Debugger

    Setting and using Breakpoints

    Tips for using the Debugger

    Debugger Features

  • 8/10/2019 Power CenterTraining

    128/287

    128

    Wizard driven tool that runs a test session View source / target data

    View transformation data

    Set breakpoints and evaluate expressions

    Initialize variables

    Manually change variable values

    Data can be loaded or discarded

    Debug environment can be saved for later use

    Debugger Interface

  • 8/10/2019 Power CenterTraining

    129/287

    129

    Target Instancewindow

    TransformationInstance

    Data window

    Flashingyellow

    SQLindicator

    Debugger Modeindicator

    Solid yellowarrow is currenttransformation

    indicator

    Output WindowDebugger Log

    Edit

    Breakpoints

    Set Breakpoints

  • 8/10/2019 Power CenterTraining

    130/287

    130

    2. Choose global orspecific transformation

    3. Choose to break on

    data condition or error.Optionally skip rows.

    4. Add breakpoint(s)

    5. Add data conditions

    1. Edit breakpoint

    6. Continue (to next breakpoint)

    Debugger Tips

  • 8/10/2019 Power CenterTraining

    131/287

    131

    Server must be running before starting a Debug Session When the Debugger is started, a spinning icon displays.

    Spinning stops when the Debugger Server is ready

    The flashing yellow/green arrow points to the current active

    Source Qualifier. The solid yellow arrow points to the currentTransformation instance

    Next Instanceproceeds a single step at a time; onerow moves from transformation to transformation

    Step to Instanceexamines one transformation at atime, following successive rows through the sametransformation

  • 8/10/2019 Power CenterTraining

    132/287

    132

    Lab 5The Debugger

  • 8/10/2019 Power CenterTraining

    133/287

    Filter Transformation

    Filter Transformation

  • 8/10/2019 Power CenterTraining

    134/287

    134

    Ports All input / output

    Specify a Filter condition

    Usage Filter rows from

    input flow

    Drops rows conditionally

  • 8/10/2019 Power CenterTraining

    135/287

    135

    Lab 6Flat File Wizard and Filter

    Transformation

  • 8/10/2019 Power CenterTraining

    136/287

    Sorter Transformation

    Sorter Transformation

  • 8/10/2019 Power CenterTraining

    137/287

    137

    Can sort data from relational tables or flat files

    Sort takes place on the Informatica Server machine

    Multiple sort keys are supported

    The Sorter transformation is often more efficient thana sort performed on a database with an ORDER BYclause

    Sorter Transformation

  • 8/10/2019 Power CenterTraining

    138/287

    138

    Sorts data from any source, at any point in a data flow

    Ports Input/Output Define one or more

    sort keys Define sort order for

    each key

    Example of Usage Sort data before

    Aggregator to improve

    performance

    Sort Keys

    Sort Order

    Sorter Properties

  • 8/10/2019 Power CenterTraining

    139/287

    139

    Cache size can beadjusted. Default is 8 Mb.Ensure sufficient memoryis available on theInformatica Server (elseSession Task will fail)

  • 8/10/2019 Power CenterTraining

    140/287

    Aggregator Transformation

    Aggregator Transformation

  • 8/10/2019 Power CenterTraining

    141/287

    141

    By the end of this section you will be familiar with: Basic Aggregator functionality

    Creating subtotals with the Aggregator

    Aggregator expressions Aggregator properties

    Using sorted data

    Aggregator Transformation

  • 8/10/2019 Power CenterTraining

    142/287

    142

    Ports Mixed I/O ports allowed Variable ports allowed Group By allowed

    Create expressions invariable and output ports

    Usage Standard aggregations

    Performs aggregate calculations

    Aggregate Expressions

  • 8/10/2019 Power CenterTraining

    143/287

    143

    Conditional Aggregateexpressions are supported: Conditional SUM format: SUM(value, condition)

    Aggregatefunctions aresupported only inthe AggregatorTransformation

    Aggregator Functions

  • 8/10/2019 Power CenterTraining

    144/287

    144

    Return summary values for non-null data

    in selected ports Use only in Aggregator transformations

    Use in output ports only

    Calculate a single value (and row) for all

    records in a group

    Only one aggregate function can benested within an aggregate function

    Conditional statements can be used with

    these functions

    AVGCOUNTFIRSTLASTMAXMEDIAN

    MINPERCENTILESTDDEVSUMVARIANCE

    Aggregator Properties

  • 8/10/2019 Power CenterTraining

    145/287

    145

    Sorted Input Property

    Set Aggregatorcache sizes forInformatica Server

    machine

    Instructs theAggregator toexpect the data

    to be sorted

    Sorted Data

  • 8/10/2019 Power CenterTraining

    146/287

    146

    The Aggregator can handle sorted or unsorted data

    Sorted data can be aggregated more efficiently, decreasing totalprocessing time

    The Server will cache data from each group andrelease the cached dataupon reaching the firstrecord of the next group

    Data mustbe sorted according to the order of theAggregators Group By ports

    Performance gain will depend upon varying factors

    Aggregating Unsorted Data

  • 8/10/2019 Power CenterTraining

    147/287

    147

    Unsorteddata

    No rows are released from Aggregatoruntil all rows are aggregated

    Group By:- store- department- date

    Aggregating Sorted Data

  • 8/10/2019 Power CenterTraining

    148/287

    148

    Each separate group (one row) is released assoon as the last row in the group isaggregated

    Group By:

    - store- department- date

    Data sorted by:

    - store- department- date

    Data Flow RulesTerminology

  • 8/10/2019 Power CenterTraining

    149/287

    149

    Passive transformation Operates on one row of data at a time AND

    Cannot change the number of rows on the data flow

    Example: Expression transformation

    Active transformation

    Can operate on groups of data rows AND/OR

    Can change the number of rows on the data flow

    Examples: Aggregator, Filter, Source Qualifier

    Data Flow Rules

  • 8/10/2019 Power CenterTraining

    150/287

    150

    Each Source Qualifier starts a single data stream (data flow)

    Transformations can send rows to more than onetransformation (split one data flow into multiple pipelines)

    Two or more data flows can meet only if they originate from acommon active transformation

    Example holds true with Normalizer instead of Source Qualifier.Exceptions are: Mapplet Input and sorted Joiner transformations

    DISALLOWED

    TT

    Active

    ALLOWED

    T

    Passive

    T

  • 8/10/2019 Power CenterTraining

    151/287

    Joiner Transformation

  • 8/10/2019 Power CenterTraining

    152/287

    152

    By the end of this section you will be familiar with:

    When to join in Source Qualifier and when in Joinertransformation

    Homogeneous joins

    Heterogeneous joins

    Joiner properties

    Joiner conditions

    Nested joins

    When to Join in Source Qualifier

  • 8/10/2019 Power CenterTraining

    153/287

    153

    If you can perform a join on the source database, then

    you can configure it in the Source Qualifier The SQL that the Source Qualifier generates, default or

    custom, executes on the source database at runtime

    Example: homogeneous join2 database tables in same

    database

    When You Cannot Join in Source Qualifier

  • 8/10/2019 Power CenterTraining

    154/287

    154

    If you cannotperform a join on the source database, then

    you cannotconfigure it in the Source Qualifier

    Examples: heterogeneous joins

    An Oracle table and a DB2 table

    A flat file and a database table

    Two flat files

    Joiner Transformation

  • 8/10/2019 Power CenterTraining

    155/287

    155

    Active Transformation

    Ports All input or input / output

    M denotes port comesfrom master source

    Examples Join two flat files Join two tables from

    different databases

    Join a flat file with arelational table

    Performs heterogeneous joins on different data

    flows

    Joiner Conditions

  • 8/10/2019 Power CenterTraining

    156/287

    156

    Multiple join conditions aresupported

    Joiner Properties

  • 8/10/2019 Power CenterTraining

    157/287

    157

    Join types: Normal (inner)

    Master outer

    Detail outer

    Full outer

    Joiner can accept sorted data (configure the join condition to use thesort origin ports)

    Set JoinerCaches

  • 8/10/2019 Power CenterTraining

    158/287

    Mid-Mapping Join (Unsorted)

  • 8/10/2019 Power CenterTraining

    159/287

    159

    The unsorted Joiner does not accept input in thefollowing situations: Both input pipelines begin with the same Source Qualifier

    Both input pipelines begin with the same Joiner

    The sorted Joiner does not have these restrictions.

  • 8/10/2019 Power CenterTraining

    160/287

    160

    Lab 7Heterogeneous Join, Aggregator, and

    Sorter

  • 8/10/2019 Power CenterTraining

    161/287

    Lookup Transformation

    Lookup Transformation

  • 8/10/2019 Power CenterTraining

    162/287

    162

    By the end of this section you will be familiar with:

    Lookup principles

    Lookup properties

    Lookup conditions

    Lookup techniques

    Caching considerations

    Persistent caches

    How a Lookup Transformation Works

    F h i t l l k d i

  • 8/10/2019 Power CenterTraining

    163/287

    163

    For each mapping row, one or more port values are looked up in adatabase table or flat file

    If a match is found, one or more table values are returned to themapping. If no match is found, NULL is returned

    Lookup value(s)

    Return value(s)

    Lookup transformation

    Lookup Transformation

  • 8/10/2019 Power CenterTraining

    164/287

    164

    Looks up values in a database table or flat file and

    provides data to other components in a mapping

    Ports Mixed L denotes Lookup port R denotes port used as a

    return value (unconnectedLookup onlysee later)

    Specify the Lookup Condition

    Usage Get related values

    Verify if records exists or ifdata has changed

    Lookup Conditions

  • 8/10/2019 Power CenterTraining

    165/287

    165

    Multiple conditions are supported

  • 8/10/2019 Power CenterTraining

    166/287

    Lookup Properties contd

  • 8/10/2019 Power CenterTraining

    167/287

    167

    Policy on multiplematch:Use first valueUse last valueReport error

  • 8/10/2019 Power CenterTraining

    168/287

  • 8/10/2019 Power CenterTraining

    169/287

  • 8/10/2019 Power CenterTraining

    170/287

    Lookup Caching Properties (contd)

  • 8/10/2019 Power CenterTraining

    171/287

    171

    Set Lookupcache sizes

    Make cachepersistent

    Set prefix forpersistent cachefile name

    Reloadpersistentcache

  • 8/10/2019 Power CenterTraining

    172/287

    172

    Lab 8Basic Lookup

  • 8/10/2019 Power CenterTraining

    173/287

  • 8/10/2019 Power CenterTraining

    174/287

    Setting Default Target Load Type

  • 8/10/2019 Power CenterTraining

    175/287

    175

    Set Target Load Type defaultWorkflow Manager, Tools | OptionsNormal or Bulk (client choice)Override the default in sessiontarget properties

    Target Properties

    Edit Tasks: Mappings Tab

  • 8/10/2019 Power CenterTraining

    176/287

    176

    Session Task

    Select target

    instance

    Row loadingoperations

    Error handling

    pp g

    Target load type

    WHERE Clause for Update and Delete

    PowerCenter uses the primary keys defined in the

  • 8/10/2019 Power CenterTraining

    177/287

    177

    PowerCenter uses the primary keys defined in the

    Warehouse Designer to determine the appropriate SQLWHERE clause for updates and deletes

    Update SQL

    UPDATE SET = WHERE =

    The only columns updated are those which have values linkedto them

    All other columns in the target are unchanged

    The WHERE clause can be overridden via Update Override

    Delete SQL DELETE from WHERE =

    SQL statement used will appear in the Session log file

    Constraint-based Loading

  • 8/10/2019 Power CenterTraining

    178/287

    178

    PK1

    FK1 PK2

    FK2

    To maintain referential integrity, primary keys mustbe loaded before their corresponding foreign keyshere in the order Target1, Target2, Target 3

    Setting Constraint-based Loading

  • 8/10/2019 Power CenterTraining

    179/287

    179

    Constraint-based LoadingTerminology

  • 8/10/2019 Power CenterTraining

    180/287

    180

    Active transformation

    Can operate on groups of data rows and/orcan change the number of rows on the data flow

    Examples: Source Qualifier, Aggregator, Joiner, Sorter, Filter

    Active source

    Active transformation that generates rows Cannot match an output row with a distinct input row

    Examples: Source Qualifier, Aggregator, Joiner, Sorter

    (The Filter is NOT an active source)

    Active group Group of targets in a mapping being fed by the same active

    source

  • 8/10/2019 Power CenterTraining

    181/287

  • 8/10/2019 Power CenterTraining

    182/287

    182

    Lab 9Deleting Rows

  • 8/10/2019 Power CenterTraining

    183/287

    Update Strategy Transformation

    Used to specify how each individual row will be used to

  • 8/10/2019 Power CenterTraining

    184/287

    184

    Used to specify how each individual row will be used to

    update target tables (insert, update, delete, reject)

    Ports All input / output Specify the Update

    Strategy ExpressionIIF or DECODE logicdetermines how tohandle the record

    Example Updating Slowly

    Changing Dimensions

    Update Strategy Expressions

  • 8/10/2019 Power CenterTraining

    185/287

    185

    IIF ( score > 69, DD_INSERT, DD_DELETE )

    Expression is evaluated for each row

    Rows are tagged according to the logic of theexpression

    Appropriate SQL (DML) is submitted to the targetdatabase: insert, delete or update

    DD_REJECT means the row will not have SQL written

    for it. Target will not see that row Rejected rows may be forwarded through Mapping

  • 8/10/2019 Power CenterTraining

    186/287

    186

    Lab 10Data Driven Operations

  • 8/10/2019 Power CenterTraining

    187/287

    187

    Lab 11Incremental Update

  • 8/10/2019 Power CenterTraining

    188/287

    188

    Lab 12Features and Techniques II

  • 8/10/2019 Power CenterTraining

    189/287

    Router Transformation

    Router Transformation

    Rows sent to multiple filter conditions

  • 8/10/2019 Power CenterTraining

    190/287

    190

    Rows sent to multiple filter conditions

    Ports All input/output Specify filter conditions

    for each Group

    Usage Link source data in

    one pass to multiplefilter conditions

  • 8/10/2019 Power CenterTraining

    191/287

    Router Transformation in a Mapping

  • 8/10/2019 Power CenterTraining

    192/287

    192

  • 8/10/2019 Power CenterTraining

    193/287

  • 8/10/2019 Power CenterTraining

    194/287

    Sequence Generator

    Transformation

    Sequence Generator Transformation

    Generates unique keys for any port on a row

  • 8/10/2019 Power CenterTraining

    195/287

    195

    Generates unique keys for any port on a row

    Ports Two predefined output

    ports, NEXTVAL and

    CURRVAL No input ports allowed

    Usage Generate sequence

    numbers

    Shareable across mappings

    Sequence Generator Properties

  • 8/10/2019 Power CenterTraining

    196/287

    196

    Number ofcached values

  • 8/10/2019 Power CenterTraining

    197/287

    Mapping Parameters and

    Variables

    Mapping Parameters and Variables

    B th d f thi ti ill d t d

  • 8/10/2019 Power CenterTraining

    198/287

    198

    By the end of this section you will understand:

    System variables

    Mapping parameters and variables

    Parameter files

    System Variables

    SYSDATE Provides current datetime on the

    Informatica Server machine

  • 8/10/2019 Power CenterTraining

    199/287

    199

    SESSSTARTTIME

    $$$SessStartTime

    Returns the system date value on theInformatica Server Used with any function that accepts

    transformation date/time datatypes

    Not to be used in a SQL override Has a constant value

    Returns the system date value as astring. Uses system clock on machinehosting Informatica Server Format of the string is database type

    dependent Used in SQL override Has a constant value

    Informatica Server machine

    Not a static value

    Mapping Parameters and Variables

    Apply to all transformations within one Mapping

  • 8/10/2019 Power CenterTraining

    200/287

    200

    Apply to all transformations within one Mapping

    Represent declared values

    Variables can change in value during run-time

    Parameters remain constant during run-time

    Provide increased development flexibility

    Defined in Mapping menu

    Format is $$VariableName or $$ParameterName

    Can be used in pre and post-SQL

    Mapping Parameters and Variables

    Sample declarations

  • 8/10/2019 Power CenterTraining

    201/287

    201

    Declare Mapping Variables and Parametersin the Designer Mappings/Mapplets menu

    Set

    aggregationtype

    Set optionalinitial value

    User-definednames

    Set datatype

  • 8/10/2019 Power CenterTraining

    202/287

  • 8/10/2019 Power CenterTraining

    203/287

  • 8/10/2019 Power CenterTraining

    204/287

    Parameters & VariablesInitialization Priority

  • 8/10/2019 Power CenterTraining

    205/287

    205

    1. Parameter file

    2. Repository value

    3. Declared initial value

    4. Default value

  • 8/10/2019 Power CenterTraining

    206/287

    Unconnected Lookups

    Unconnected Lookups

    By the end of this section you will know:

  • 8/10/2019 Power CenterTraining

    207/287

    207

    By the end of this section you will know:

    Unconnected Lookup technique

    Unconnected Lookup functionality

    Difference from Connected Lookup

    Unconnected Lookup

    Physically unconnected from other transformationsNO data flowarrows leading to or from an unconnected Lookup

  • 8/10/2019 Power CenterTraining

    208/287

    208

    arrows leading to or from an unconnected Lookup Lookup data is called from the point in the Mapping that needs it Lookup function can be set within any transformation that supports

    expressions

    Function in the Aggregator

    calls the unconnected Lookup

    Unconnected Lookup Technique

    Use lookup lookup function within a conditional statement

  • 8/10/2019 Power CenterTraining

    209/287

    209

    Condition is evaluated for each row but Lookup function

    is called only if condition satisfied

    IIF ( ISNULL(customer_id),:lkp.MYLOOKUP(order_no))

    Condi t ion

    Lookup func t ion

    Row keys

    (passed to Loo kup)

    Unconnected Lookup Advantage

    Data lookup is performed only for those rows which

  • 8/10/2019 Power CenterTraining

    210/287

    210

    require it. Substantial performance can be gained

    EXAMPLE: A Mapping will process 500,000 rows. For twopercent of those rows (10,000) the item_id value is NULL.Item_ID can be derived from the SKU_NUMB.

    Net savings = 490,000 lookups

    IIF ( ISNULL(item_id), :lkp.MYLOOKUP (sku_numb))

    Condi t ion

    (true for 2 percent of all rows)

    Lookup

    (called only when condition is true)

    Unconnected Lookup Functionality

    OneLookup port valuemay be returned for each Lookup

  • 8/10/2019 Power CenterTraining

    211/287

    211

    Must check aReturn port in the

    Ports tab, elsefails at runtime

    Connected versus Unconnected Lookups

    CONNECTED LOOKUP UNCONNECTED LOOKUP

  • 8/10/2019 Power CenterTraining

    212/287

    212

    Part of the mapping data flow Separate from the mapping dataflow

    Returns multiple values (bylinking output ports to anothertransformation)

    Returns one value - by checkingthe Return (R) port option for theoutput port that provides the

    return value

    Executed for every recordpassing through thetransformation

    Only executed when the lookupfunction is called

    More visible, shows where the

    lookup values are used

    Less visible, as the lookup is

    called from an expression withinanother transformation

    Default values are used Default values are ignored

  • 8/10/2019 Power CenterTraining

    213/287

    213

    Lab 14Straight Load

  • 8/10/2019 Power CenterTraining

    214/287

    214

    Lab 15Conditional Lookup

  • 8/10/2019 Power CenterTraining

    215/287

    Mapplets

    Mapplets

    By the end of this section you will be familiar with:

  • 8/10/2019 Power CenterTraining

    216/287

    216

    By the end of this section you will be familiar with:

    Mapplet Designer

    Mapplet advantages

    Mapplet types

    Mapplet rules

    Active and Passive Mapplets

    Mapplet Parameters and Variables

    Mapplet Designer

  • 8/10/2019 Power CenterTraining

    217/287

    217

    MappletInput and OutputTransformation

    Icons

    Mapplet OutputTransformation

    Mapplet Designer Tool

    Mapplet Advantages

    Useful for repetitive tasks / logic

  • 8/10/2019 Power CenterTraining

    218/287

    218

    Useful for repetitive tasks / logic

    Represents a set of transformations

    Mapplets are reusable

    Use an instance of a Mapplet in a Mapping

    Changes to a Mapplet are inherited by all instances

    Server expands the Mapplet at runtime

    A Mapplet Used in a Mapping

  • 8/10/2019 Power CenterTraining

    219/287

    219

    The Detail Inside the Mapplet

  • 8/10/2019 Power CenterTraining

    220/287

    220

    Unsupported Transformations

    You cannot not use the following in a mapplet:

  • 8/10/2019 Power CenterTraining

    221/287

    221

    Normalizer Transformation

    XML source definitions

    Target definitions

    Other mapplets

    Mapplet Source Options

    Internal Sources

  • 8/10/2019 Power CenterTraining

    222/287

    222

    One or more Source definitions / Source Qualifierswithinthe Mapplet

    External Sources

    Mapplet contains a Mapplet Input transformation

    Receives data from the Mapping it is used in

    Mixed Sources

    Mapplet contains one or more of either of a Mapplet

    Input transformation AND one or more Source Qualifiers

    Receives data from the Mapping it is used in, AND fromthe Mapplet

  • 8/10/2019 Power CenterTraining

    223/287

    Data Source Outsidea Mapplet

    Source data is definedOUTSIDE the Mapplet logic

  • 8/10/2019 Power CenterTraining

    224/287

    224

    Resulting Mapplet HASinput ports

    When used in a Mapping,the Mapplet may occur atany point in mid-flow

    OUTSIDE the Mapplet logic

    Mapplet

    Mapplet Input

    Transformation

    Data Source I nsidea Mapplet

    Source

    Qualifier

    Source data is definedWITHIN the Mapplet logic

  • 8/10/2019 Power CenterTraining

    225/287

    225

    Resulting Mapplet has noinput ports

    When used in a Mapping,the Mapplet is the firstobject in the data flow

    Mapplet

    No Input transformationis required (or allowed)

    Use a Source Qualifier

    instead

    QualifierWITHIN the Mapplet logic

    Mapplet Output Transformation

    Use to contain the results of a Mapplet pipeline. MultipleOutput transformations are allowed.

  • 8/10/2019 Power CenterTraining

    226/287

    226

    Passive TransformationConnected

    Ports

    Input ports only

    Usage

    Only those ports connected toan Output transformation (fromanother transformation) will

    display in the resulting Mapplet One (or more) Mapplet Output

    transformations are required inevery Mapplet

    Output transformations are allowed.

    Mapplet with Multiple Output Groups

  • 8/10/2019 Power CenterTraining

    227/287

    227

    Can output to multiple instances of the same target table

    Unmapped Mapplet Output Groups

  • 8/10/2019 Power CenterTraining

    228/287

    228

    Warning:An unlinkedMapplet Output Groupmay invalidate themapping

    Active and Passive Mapplets

  • 8/10/2019 Power CenterTraining

    229/287

    229

    Passive Mapplets contain only passive transformations Active Mapplets contain one or more active

    transformations

    CAUTION: Changing a passive Mapplet into an active Mappletmay invalidate Mappings which use that Mappletso do an impactanalysis in Repository Manager first

    Using Active and Passive Mapplets

  • 8/10/2019 Power CenterTraining

    230/287

    230

    Multiple PassiveMapplets can populatethe same targetinstance

    Multiple Active Mappletsor Active and PassiveMapplets cannotpopulate the same

    target instance

    Active

    Passive

    Mapplet Parameters and Variables

  • 8/10/2019 Power CenterTraining

    231/287

    231

    Same idea as mapping parameters and variables Defined under the

    Mapplets | Parameters and Variablesmenu option

    A parameter or variable defined in a mapplet is notvisible in any parent mapping

    A parameter or variable defined in a mapping is notvisible in any child mapplet

  • 8/10/2019 Power CenterTraining

    232/287

    232

    Lab 17Mapplets

  • 8/10/2019 Power CenterTraining

    233/287

    Reusable Transformations

    Reusable Transformations

  • 8/10/2019 Power CenterTraining

    234/287

    234

    By the end of this section you will be familiar with: Transformation Developer

    Reusable transformation rules

    Promoting transformations to reusable Copying reusable transformations

  • 8/10/2019 Power CenterTraining

    235/287

    Reusable Transformations

    Define once, reuse many times

  • 8/10/2019 Power CenterTraining

    236/287

    236

    Reusable Transformations Can be a copy or a shortcut

    Edit Ports only in Transformation Developer

    Can edit Properties in the mapping

    Instances dynamically inherit changes

    Caution:changing reusable transformationscan invalidatemappings

    Note: Source Qualifier transformations cannotbe made reusable

    Promoting a Transformation to Reusable

  • 8/10/2019 Power CenterTraining

    237/287

    237

    Check theMake reusable box

    (irreversible)

  • 8/10/2019 Power CenterTraining

    238/287

  • 8/10/2019 Power CenterTraining

    239/287

    239

    Lab 18Reusable Transformations

  • 8/10/2019 Power CenterTraining

    240/287

    Error Logging Objectives

  • 8/10/2019 Power CenterTraining

    241/287

    241

    By the end of this section, you will be familiar with: Setting error logging options

    How data rejects and transformation errors are

    handled with logging on and off

    How to log errors to a flat file or relational table

    When and how to use source row logging

    Error Types

  • 8/10/2019 Power CenterTraining

    242/287

    242

    Transformation error Data row has only passed partway through the mapping

    transformation logic

    An error occurs within a transformation

    Data reject Data row is fully transformed according to the mapping

    logic

    Due to a data issue, it cannot be written to the target

    A data reject can be forced by an Update Strategy

    Error Logging Off/On

    Error Type Logging OFF (Default) Logging ON

  • 8/10/2019 Power CenterTraining

    243/287

    243

    Transformationerrors

    Written to session logthen discarded

    Appended to flat file orrelational tables. Onlyfatal errors written tosession log.

    Data rejects Appended to reject file(one .bad file per target)

    Written to row errortables or file

    Setting Error Log Options

    In Session task

  • 8/10/2019 Power CenterTraining

    244/287

    244

    Log Row DataLog Source Row Data

    Error Log Type

    Error Logging OffSpecifying Reject Files

    In Session task

  • 8/10/2019 Power CenterTraining

    245/287

    245

    1 file per target

    Error Logging OffTransformation Errors

    Details and data are written to session log

  • 8/10/2019 Power CenterTraining

    246/287

    246

    X

    X

    Transformation Error

    Data row is discarded If data flows concatenated, corresponding rows in parallel

    flow are also discarded

    Error Logging OffData Rejects

    Conditions causing data to be rejected include:

    T d b i i l i f l

  • 8/10/2019 Power CenterTraining

    247/287

    247

    Target database constraint violations, out-of-space errors, logspace errors, null values not accepted

    Data-driven records, containing value 3 or DD_REJECT(the reject has been forced by an Update Strategy)

    Target table properties reject truncated/overflowed rows

    0,D,1313,D,Regulator System,D,Air Regulators,D,250.00,D,150.00,D1,D,1314,D,Second Stage Regulator,D,Air Regulators,D,365.00,D,265.00,D2,D,1390,D,First Stage Regulator,D,Air Regulators,D,170.00,D,70.00,D3,D,2341,D,Depth/Pressure Gauge,D,Small Instruments,D,105.00,D,5.00,D

    Sample reject file

    Indicator describes preceding column valueD=Data, O=Overflow, N=Null or T=Truncated

    First column:0=INSERT1=UPDATE2=DELETE3=REJECT

    Log Row Data

    Logs:

  • 8/10/2019 Power CenterTraining

    248/287

    248

    Session metadata

    Reader, transformation, writer and user-defined errors

    For errors on input, logs row data for Iand I/Oports

    For errors on output, logs row data for I/Oand Oports

  • 8/10/2019 Power CenterTraining

    249/287

  • 8/10/2019 Power CenterTraining

    250/287

    Error Logging to a Flat File 1

    Creates delimited Flat File with || as column delimiter

  • 8/10/2019 Power CenterTraining

    251/287

    251

    Flat File LogSettings (Defaults

    shown)

    Logging Errors to a Flat File 2

    Format: Session metadata followed by de-normalized error information

    Sample session metadata

    **********************************************************************

  • 8/10/2019 Power CenterTraining

    252/287

    252

    **********************************************************************Repository GID: 510e6f02-8733-11d7-9db7-00e01823c14d

    Repository: RowErrorLogging

    Folder: ErrorLogging

    Workflow: w_unitTests

    Session: s_customers

    Mapping: m_customers

    Workflow Run ID: 6079Worklet Run ID: 0

    Session Instance ID: 806

    Session Start Time: 10/19/2003 11:24:16

    Session Start Time (UTC): 1066587856

    **********************************************************************

    Row data formatTransformation || Transformation Mapplet Name || Transformation Group || PartitionIndex || Transformation Row ID || Error Sequence || Error Timestamp || Error UTCTime || Error Code || Error Message || Error Type || Transformation Data || SourceMapplet Name || Source Name || Source Row ID || Source Row Type || Source Data

    Log Source Row Data 1

    S t h kb i i t k

  • 8/10/2019 Power CenterTraining

    253/287

    253

    Separate checkbox in session task Logs the source row associated with the error row

    Logs metadata about source, e.g. Source Qualifier,

    source row id, and source row type

    Log Source Row Data 2

    Source row logging is not available downstream of anAggregator, Joiner, Sorter (where output rows are not

    i l l t d ith i t )

  • 8/10/2019 Power CenterTraining

    254/287

    254

    Source row loggingavailable

    Source row loggingnot available

    uniquely correlated with input rows)

  • 8/10/2019 Power CenterTraining

    255/287

    Workflow Configuration

    Workflow Configuration Objectives

    By the end of this section, you will be able to create:

  • 8/10/2019 Power CenterTraining

    256/287

    256

    Workflow Server Connections

    Reusable Schedules

    Reusable Session Configurations

  • 8/10/2019 Power CenterTraining

    257/287

  • 8/10/2019 Power CenterTraining

    258/287

    Relational Connections (Native )

    Create a relational [database] connection Instructions to the Server to locate relational tables

    Used in Session Tasks

  • 8/10/2019 Power CenterTraining

    259/287

    259

    Used in Session Tasks

    Relational Connection Properties

    Define nativerelational database

    connection

  • 8/10/2019 Power CenterTraining

    260/287

    260

    connection

    Optional Environment SQL(executed with each use ofdatabase connection)

    User Name/Password

    Database connectivityinformation

    Rollback Segmentassignment (optional)

    FTP Connection

    Create an FTP connection

    Instructions to the Server to ftp flat files

    Used in Session Tasks

  • 8/10/2019 Power CenterTraining

    261/287

    261

    Used in Session Tasks

    External Loader Connection

    Create an External Loader connection

    Instructs the Server to invoke an external database loader

    Used in Session Tasks

  • 8/10/2019 Power CenterTraining

    262/287

    262

    Used in Session Tasks

  • 8/10/2019 Power CenterTraining

    263/287

    Set up reusable schedules to associate with multiple Workflows

    Defined at folder level

    Must have the Workflow Designer tool open

    Reusable Workflow Schedules

  • 8/10/2019 Power CenterTraining

    264/287

    264

    Must have the Workflow Designer tool open

    Reusable Workflow Schedules

  • 8/10/2019 Power CenterTraining

    265/287

    265

  • 8/10/2019 Power CenterTraining

    266/287

    266

    Reusable Session Configurations

    Session Configuration

    Define properties to be reusable across differentsessions

  • 8/10/2019 Power CenterTraining

    267/287

    267

    sessions

    Defined at folder level

    Must have one of these toolsopen in order to access

    Session Configuration (contd)

  • 8/10/2019 Power CenterTraining

    268/287

    268

    Available from menu orTask toolbar

    Session Configuration (contd)

  • 8/10/2019 Power CenterTraining

    269/287

    269

    Session TaskConfig Object

  • 8/10/2019 Power CenterTraining

    270/287

    270

    Within Session task properties, choose desired

    configuration

    Session TaskConfig Object Attributes

  • 8/10/2019 Power CenterTraining

    271/287

    271

    Attributes

    may beoverriddenwithin theSession task

  • 8/10/2019 Power CenterTraining

    272/287

    Worklets

    Worklets

    An object representing a set or grouping of Tasks

    Can contain any Task available in the Workflow

  • 8/10/2019 Power CenterTraining

    273/287

    273

    Can contain any Task available in the WorkflowManager

    Worklets expand and execute inside a Workflow

    A Workflow which contains a Worklet is called theparent Workflow

    Worklets CAN be nested

    Reusable Workletscreate in the WorkletDesigner

    Non-reusable Workletscreate in the WorkflowDesigner

    Re-usable Worklet

    In the Worklet Designer, select Worklets | Create

  • 8/10/2019 Power CenterTraining

    274/287

    274

    Tasks in a Worklet

    WorkletsNode

    Using a Reusable Worklet in a Workflow

  • 8/10/2019 Power CenterTraining

    275/287

    275

    Workletused in aWorkflow

    Non-Reusable Worklet

    1. Create worklet task inWorkflow Designer

    2. Right-click on new worklet

  • 8/10/2019 Power CenterTraining

    276/287

    276

    gand select Open Worklet

    3. Workspace switches toWorklet Designer

    NOTE: Worklet

    shows only underWorkflows node

  • 8/10/2019 Power CenterTraining

    277/287

    277

    Lab 21Reusable Worklet and Decision Task

  • 8/10/2019 Power CenterTraining

    278/287

    278

    Lab 22Event Wait with Pre-Defined Event

  • 8/10/2019 Power CenterTraining

    279/287

  • 8/10/2019 Power CenterTraining

    280/287

    280

    Parameters and VariablesReview

    Types of Parameters and Variables

    TYPE HOW DEFINED WHERE USED EXAMPLES

    Mapping/Mapplet

    Variables

    Mapping/mappletproperties. Reset by

    variable functions.

    Transformation portexpressions

    $$LastUpdateTime$$MaxValue

    Mappings

  • 8/10/2019 Power CenterTraining

    281/287

    281

    Mapping/MappletParameters

    Mapping/mappletproperties. Constantfor session.

    Transformation portexpressions

    $$FixedCosts$$DiscountRate

    SystemVariables

    Built-in, pre-defined. Transformation portexpressions,Workflow decisiontasks andconditional links.

    SYSDATESESSSTARTIMEWORKFLOWSTARTTIME

    TaskVariables

    Built-in, pre-defined. Workflow decisiontasks andconditional links

    $session1. Status$session1.ErrorCode

    Workflow/

    WorkletVariables

    Workflow or worklet

    properties. Reset inAssignment tasks.

    Workflow decision

    tasks andconditional links

    $$NewStartTime

    SessionParameters

    Parameter file.Constant for session.

    Session properties $DBConnectionORCL$InputFile1

    Mappings& Mapplets

    Workflows& Worklets

  • 8/10/2019 Power CenterTraining

    282/287

    282

    PowerCenter 7.1 Options andData Access Products

    PowerCenter 7.1 Options

    N d dd l i f ti lit i l di

    Profile wizards, rules definitions, profile results tables,

    and standard reportsData Profiling

    Allows export/import of metadata to or from businessintelligence tools like Business Objects and CognosMetadata Exchange with BI

  • 8/10/2019 Power CenterTraining

    283/287

    283

    PowerCenter

    Real-Time/Web ServicesZL Engine, always-on non-stop sessions, JMS

    connectivity, and real-time Web Services provider

    Data CleansingName and address cleansing functionality, includingdirectories for US and certain international countries

    PartitioningData smart parallelism, pipeline and data parallelism,partitioning

    Server engine, metadata repository, unlimited designers,workflow scheduler, all APIs and SDKs, unlimited XML andflat file sourcing and targeting, object export to XML file,LDAP authentication, role-based object-level security,metadata reporter, centralized monitoring

    Server group management, automatic workflow distributionacross multiple heterogeneous serversServer Grid

    Version control, deployment groups, configurationmanagement, automatic promotion

    Team-Based Development

    Virtual Classes

    Watch for short web-based virtual

    classes on most PowerCenterti d XML t

  • 8/10/2019 Power CenterTraining

    284/287

    284

    options and XML support

    Data AccessPowerExchange

    Provides access to all critical enterprise data systems,including mainframe, midrange relational databases,and file-based systems

  • 8/10/2019 Power CenterTraining

    285/287

    285

    a d e based sys e s

    Offers batch, change capture and real-time options

    PowerExchange 5.2 provides tight integration withPowerCenter 7.1.1 through the PowerExchange Clientfor PowerCenter

    Supporting VSAM, IMS, DB2 (OS/390, AS/400), Oracle, ODBC

    Data AccessPowerCenter Connect

    PowerCenter Connect options are currently available for:

    Transactional Applications Real-time Services

  • 8/10/2019 Power CenterTraining

    286/287

    286

    Transactional ApplicationsHyperion EssbasePeopleSoftSAP R/3SAP BWSASSiebel

    HTTPJMSMSMQMQSeriesTIBCOWebMethodsWeb Services

    PowerCenter Connect SDK

    Allows development of new PowerCenter Connect productsAvailable on the Informatica Developer Network

  • 8/10/2019 Power CenterTraining

    287/287


Recommended