+ All Categories
Home > Documents > Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS...

Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS...

Date post: 15-Jul-2020
Category:
Upload: others
View: 4 times
Download: 0 times
Share this document with a friend
55
Metadata Based Impact and Lineage Analysis Across Heterogeneous Metadata Sources Presentation at the THE 9TH ANNUAL Wilshire Meta-Data Conference AND THE 17TH ANNUAL DAMA International Symposium by John R. Friedrich, II, Ph.D. [email protected]
Transcript
Page 1: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Metadata Based Impact andLineage Analysis Across HeterogeneousMetadata Sources

Presentation at theTHE 9TH ANNUAL Wilshire Meta-Data ConferenceAND THE 17TH ANNUAL DAMA International SymposiumbyJohn R. Friedrich, II, [email protected]

Page 2: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 2 or 4125 May 2005

Outline

• Concepts• Harvesting• Mapping• Reporting• Version and Configuration Management• Metadata Management Strategy• Standardization• Proof of Concept• Conclusion

Page 3: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 3 or 4125 May 2005

Caveat

• This presentation is based upon consulting engagements with MetaIntegration Technologies, Inc. (MITI) and facilitated using their tools and methods.

• Meta Integration is the leading "Metadata Component Provider" to major database, data integration, business intelligence, repository, and modeling tool vendors. Meta Integration® OEM components & tools manage and analyze the complete enterprise metadata life cycle across heterogeneous vendors. Solutions range from reusable metadata movement components to repository. Over 50 Model Bridges integrate standards like CWM, and the most popular modeling, ETL/EII, and OLAP/BI tools.

• The Meta Integration components may be found already integrated with its partners’ tools including the Adaptive Repository, Cognos ReportNet, Embarcadero ERStudio, Informatica Power Center and SuperGlue, and more in the future.

• All other trademarks, trade names, service marks, and logos referenced herein belong to their respective companies.

Page 4: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 4 or 4125 May 2005

Concepts

Data Sources

Data Sources

Data Sources

ApplicationsApplications

Applications

ERP Solution

Data Warehouse

Data MartData Mart

Data Mart

Enterprise Application Integration

Business Intelligence Reporting

OLAP CubesOLAP Cubes

ETL

ETL

ETLETL

ETL OLAP CubesStaging

Data FeedsData

Feeds

Data FeedsData

Feeds

Data FeedsData

Feeds

Process & Code

Models

Data Specs

Data Schemas

Mappings

ProcessMetadata

EAI RepositoryETL Repository

Data Schemas

X-froms

ProcessMetadata

EAI Repository

Data Specs

Data Specs

Data Specs

Data Schemas

Mappings

Cubes, Joins, etc.

Message Schema XML

Schema

CASE Design Software

CASE Repository

CASE Design SoftwareCASE Design

Software

CASE RepositoryCASE

Repository

Process & Code

Models Data Models

Page 5: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 5 or 4125 May 2005

Reporting / Business Intelligence

Data WarehouseETLSource Systems

MetaData

CM

Computer Aided Software Engineering

Extract Transform and Load

Development

Business Intelligence Design and Reporting

Data SourcesData Sources

Data Sources

ApplicationsApplications

Data Warehouse

Data Mart

BI Reporting

ETL

OLAP Cube

Staging

ETL Repository

EAI Repository

ETL Engine

Computer Aided Software Engineering

BI DesignETL Design

CASE Repository

CASE Design

Forward/ReverseEngineering

CASE Design

CASE Repository

Forward/ReverseEngineering

Source Data Schemas

Mappings LifecycleMetadata

Data Schemas

Mappings

Cubes, Joins, etc.

Process & Code

Models

Data Models

Target Data Schemas

Staging Data

Models

DW Data Models

Mapping

Mapping Mapping

Page 6: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 6 or 4125 May 2005

Business Intelligence

Data Mart

OLAP Cube

Data Warehouse

Data Warehouse

StagingRDBMSMigrate / Replicate

ETL

Staging Target

AP RDBMS

AR RDBMS

WS-API

XML

Source Systems / ODS

GL

APAR

Accts Rcv’l

Finance

Accts Pay

Data Feeds

XML API

WSAPI

Meta-Data Repository

Business Intelligence Design and Reporting

ReportNet

Cognos Repository

FrameworkManager

StagingData

Model

XML Models

AR Model

AR Model

GL Model

WS-API Model

Computer Aided Software Engineering

Model Mart

ERwin Rose

Forward / Reverse Engineering

Data M

ovement M

apping

XML ETL Source

ETL Process Metadata

AR ETL Source

AP ETL Source

WS ETL Source

ETLTarget

RDBMSSchema

Data M

ovement M

apping

WarehouseData

Model

StandardsMapping

Standard Finance Logical ModelStandards Mapping Standards Mapping

Data M

ovement M

apping

ETL Tool Mappings

Data M

ovement M

apping

BISource

RDBMSSchema

BI Cube

Reports

Views

Queries

Dims

BI Tool M

appings

Meta-Data BridgesDTD/XSD ER1 MDL XMI PDM XMI

Extract Transform and Load Development

Informatica Repository

InformaticaETL Engine

Designer

Computer Aided

Software Engineering

PowerDesigner

Forward/ReverseEngineering

RDBMSAdmin

SQLAdmin

Page 7: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 7 or 4125 May 2005

Harvesting

• Metadata survey• Source/Target survey• Data process survey• Reference and standards survey

• Metadata exchange methodology• Repository Structure• Extraction

Page 8: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 8 or 4125 May 2005

Meta-Data Repositoryh

XML Models

AR Model

AR Model

GL Model

WS-API Model

XML ETL Source

ETL Process Metadata

AR ETL Source

AP ETL Source

WS ETL Source

ETLTarget

RDBMSSchema

WarehouseData

Model

BISource

RDBMSSchema

BI Cube

Reports

Views

Queries

Dims

BI Tool M

appingsStaging

DataModel

Meta-Data BridgesDTD/XSD ER1 MDL XMI PDM XMI

Business Intelligence

Data Mart

OLAP Cube

Data Warehouse

Data Warehouse

StagingRDBMSMigrate / Replicate

ETL

Staging Target

AP RDBMS

AR RDBMS

WS-API

XML

Source Systems / ODS

GL

APAR

Accts Rcv’l

Finance

Accts Pay

Data Feeds

XML API

WSAPI

Business Intelligence Design and Reporting

ReportNet

Cognos Repository

FrameworkManager

Computer Aided Software Engineering

Model Mart

ERwin Rose

Forward / Reverse Engineering

Extract Transform and Load Development

Informatica Repository

InformaticaETL Engine

Designer

Computer Aided

Software Engineering

PowerDesigner

Forward/ReverseEngineering

RDBMSAdmin

SQLAdmin

Page 9: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 9 or 4125 May 2005

Mapping

• Data process mapping• EAI among models• ODS to ETL to DW Staging• Staging to DW• DW to BI

• Metadata management mapping• To/From standards and reference models

Page 10: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 10 or 4125 May 2005

Business Intelligence

Data Mart

OLAP Cube

Data Warehouse

Data Warehouse

StagingRDBMSMigrate / Replicate

ETL

Staging Target

AP RDBMS

AR RDBMS

WS-API

XML

Source Systems / ODS

GL

APAR

Accts Rcv’l

Finance

Accts Pay

Data Feeds

XML API

WSAPI

Meta-Data Repository

Business Intelligence Design and Reporting

ReportNet

Cognos Repository

FrameworkManager

StagingData

Model

XML Models

AR Model

AR Model

GL Model

WS-API Model

Computer Aided Software Engineering

Model Mart

ERwin Rose

Forward / Reverse Engineering

XML ETL Source

ETL Process Metadata

AR ETL Source

AP ETL Source

WS ETL Source

ETLTarget

RDBMSSchema

WarehouseData

Model

BISource

RDBMSSchema

BI Cube

Reports

Views

Queries

Dims

BI Tool M

appings

Meta-Data BridgesDTD/XSD ER1 MDL XMI PDM XMI

Extract Transform and Load Development

Informatica Repository

InformaticaETL Engine

Designer

Computer Aided

Software Engineering

PowerDesigner

Forward/ReverseEngineering

RDBMSAdmin

SQLAdmin

Data M

ovement M

apping

Data M

ovement M

apping

StandardsMapping

Standard Finance Logical ModelStandards Mapping Standards Mapping

Data M

ovement M

apping

ETL Tool Mappings

Data M

ovement M

apping

Page 11: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 11 or 4125 May 2005

Source Message Target Schema

Message Class: EmployeeAttribute: Employee Number

DataType: Char(10)Attribute: Name

DataType: Char(50)Attribute: Address

DataType: Text()Attribute: Age

DataType: Number(2)Attribute: Race

DataType: Char(2)

Table: PersonAttribute: Unique ID

DataType: LongIntAttribute: Name

DataType: Varchar(50)Attribute: DOB

DataType: DatetimeAttribute: Ethnicity

DataType: Char

Table: EmployeeAttribute: Unique ID

DataType: LongIntAttribute: Employee Number

DataType: Varchar(10)Attribute: Date of Employment

DataType: DatetimeAttribute: Status

DataType: Char(10)

Table: AddressAttribute: Unique ID

DataType: LongIntAttribute: Address Line 1

DataType: Varchar(30)Attribute: Address Line 2

DataType: Varchar(30)Attribute: Address City

DataType: Varchar(30)Attribute: Address State

DataType: Varchar(2)Attribute: Address Zip

DataType: Varchar(9)

Table: PersonAddressAttribute: Person Unique ID

DataType: LongIntAttribute: Address Unique ID

DataType: LongIntAttribute: Address Type

DataType: Varchar(10)

Page 12: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 12 or 4125 May 2005

Reporting

• Lineage analysis• Forward (Impact) analysis

• ODS DW Staging DW BI Design Reporting• Reverse lineage analysis

• ODS DW Staging DW BI Design Reporting

Page 13: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 13 or 4125 May 2005

Version and Configuration Management

• Not “one version of the truth”• Multiple versions of sources/targets• Multiple mappings among multiple versions• Multiple configuration of these versions

Page 14: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 14 or 4125 May 2005

Metadata Management StrategySummary .........................................................................................................................................................12 Table of Contents ....................................................................................................................................13 Table of Figures ......................................................................................................................................14 Background .............................................................................................................................................15 Meta-Data Architecture...........................................................................................................................1

5.1 Meta-Data Repository (MDR) Architecture....................................................................................15.2 Meta-Data Sources and Targets.......................................................................................................2

5.2.1 Meta-Data Accessibility ..........................................................................................................25.2.2 Other Databases.......................................................................................................................4

5.3 Server and Workstation Architecture ..............................................................................................45.3.1 Repository Server ....................................................................................................................45.3.2 Repository Administration ......................................................................................................45.3.3 Meta-Data Management ..........................................................................................................45.3.4 Meta-Data Source POC’s ........................................................................................................45.3.5 Business Users.........................................................................................................................4

6 Meta-data Management Process..............................................................................................................56.1 General Meta-data Management Process ........................................................................................5

6.1.1 Meta-Data Repository Transaction Process.............................................................................56.1.2 Criteria for Meta-Data Repository Access ..............................................................................6

6.2 Data Standards Management Process..............................................................................................76.2.1 DSM Process in Detail ............................................................................................................76.2.2 Phase 1—Identify Data Management Requirements...............................................................96.2.3 Phase 2—Data Review Package Development .......................................................................96.2.4 Phase 3—Execute Data Management Plan..............................................................................96.2.5 Phase 4—Evaluate Effectiveness ............................................................................................96.2.6 Types of Data Standards Issues.............................................................................................10

6.3 Meta-Data Management Examples ...............................................................................................126.3.1 Legacy to Oracle ERP ETL...................................................................................................136.3.2 ERP to Warehouse ETL ........................................................................................................146.3.3 Warehouse to BI ETL............................................................................................................146.3.4 Impact Analysis.....................................................................................................................156.3.5 Updated Standard Code Set...................................................................................................12

7 Roles and Responsibilities ....................................................................................................................167.1 Meta-data Manager .......................................................................................................................167.2 Meta-data Repository Administrator .............................................................................................167.3 Data Steward .................................................................................................................................167.4 Data Proponent..............................................................................................................................177.5 Meta-data Integration POC ...........................................................................................................177.6 Data Standardization CM Roles and Responsibilities ...................................................................17

7.6.1 Data Standards Board............................................................................................................177.6.2 Board Chair ...........................................................................................................................187.6.3 Data Standards Manager .......................................................................................................187.6.4 Support Staff .........................................................................................................................187.6.5 Training .................................................................................................................................18

Page 15: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 15 or 4125 May 2005

Customize ERP Model

Develop ETL

Execute Test ETL

Run

Evaluate Results

Execute Production ETL Run

Production Beta Test

Populate Schemas in

MDR

Forward Engineer to Oracle ERP

Customize DW Model

Develop ETL

Execute Test ETL

Run

Evaluate Results

Execute Production ETL Run

Production Beta Test

Populate Schemas in

MDR

Forward Engineer to DW Builder

Legacy to ERP Migration

ERP to Warehouse

Customize DW Model

Develop ETL

Execute Test ETL

Run

Evaluate Results

Execute Production ETL Run

Production Beta Test

Populate Schemas in

MDR

Forward Engineer to DW Builder

Warehouse to BI

Reverse Engineer Changed Version

Impacted POC’s Update

Packages

Test Evaluate Results

Execute Production

Run

Production Beta Test

Get Schema From

Repository

Identify Mapped

(Impacted) Packages

ERP Flex Fields Change Impact Analysis

Page 16: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 16 or 4125 May 2005

Forward-Engineerto Oracle ERP Develop ETL in Informatica

Customize Source/ Target Models

Populate Repository withSource/ Target Models

Retrieve Model from MDR

Edit in ERwin

Import New Version into

MDR

New ERP Schema Design

Completed

Migrate Version from Prior ERP Schema Design

Post in Model Mart

Export Source / Target models

and Transforms from MDR

Import into Informatica

Designer

Export schema and transforms from Designer

Import New Version into

MDR

Store in Informatica Repository

Export complete Model from

ERwin

New ETL Design

Completed

Migrate Version from Prior ETL

Design

Extract Schema from Oracle

ERP into MDR

Export Source /Target Models

from ERwin

Export schema and transforms from Designer

Import New Product into

MDR

Import New Product into

MDR

Compare Models in MDR

Map Models in MDR

Extract Schema from Legacy

into MDR

Export New ERP Design to

ERwin

Generate ERP Update DDL

Page 17: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 17 or 4125 May 2005

Identify Mapped (Impacted) Packages

Impacted POC’s Update PackagesMake Changes in ERwin

Identify Package in

Repository and Reverse Engineer Changed Version

Retrieve Latest Version ERP Schema from

MDR

Make Changes in ERwin

Import New Version into

MDR

New ERP Schema Design

Completed

Migrate Version from Prior ERP Schema Design

Post in Model Mart

Export Source / Target models

and Transforms from MDR

Import into Informatica

Designer

Export schema and transforms from Designer

Import New Version into

MDR

Store in Informatica Repository

Export complete Model from

ERwin

New ETL Design

Completed

Migrate Version from Prior ETL

Design

Generate New ERP Schema or

Extensions

Schema or Transformation

Change Required

Run Web Query to Identify

Impacted ERP Products

Extract Schema from Oracle

ERP into MDR

Compare Models in MDR

Map Models in MDR

Retrieve Latest Impacted

Schema from MDR

Run Web Query to Identify Other Tools’ Impacted

Products

Navigate to Impacted

Schema using MIW

Page 18: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 18 or 4125 May 2005

Identify New meta-data

Updated meta-data to be

committed to physical

instantiation

Updated meta-data to interact with or impact another meta-

data source

Comparison and mapping of meta-data

Updated meta-data from

Standard meta-data package

approval

Editing of meta-data attributes in

ERwin

Import into Meta-Data

Repository as new version of

existing package

Extract meta-data from source

tool

Export meta-data from

Repository into target tool

format

Import into Meta-Data

Repository as first version of new package

Extract meta-data from source

tool

Export meta-data from

Repository into target tool

format

Manage in Model Mart

Integrate meta-data

Integrated meta-data

Page 19: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 19 or 4125 May 2005

Standardization

• Mapping to reference models• Creation of a “canonical form”, I.e., form that may be

transformed to any of the other reference forms.• Standardization process

Page 20: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 20 or 4125 May 2005

ProgramManagement

Organization(s)

DSCMBoard

and/or Chief Architect

DataProponent,

Steward,and/or SDO

Lead Program Management Organization (PMO)

Developers and Deployment

Joint MedicalInformation

Systems

New XML Message Structure Develop and DeployNew Version of XML MessageYes

No

Yes

No

No

YesNo Yes

No

Yes

No

Yes

No

YesNo

Identify SDO,

Proponent & Steward

Identify SDO,

Proponent & Steward

EndorsedEndorsed

New Version of ExistingXML Message

Design Using XML Message Structure

Design Using XML Message Structure

EndorsedEndorsed

Develop and Review Message Structure and/or Extend SDO

Standard

Develop and Review Message Structure and/or Extend SDO

Standard

MHS AdoptedSDO

Standard?

MHS AdoptedSDO

Standard?

Check for SDO Standard

Check for SDO Standard

Need Changeto XML Message

in Registry?

Need Changeto XML Message

in Registry?

Propose Change to Message

in Registry

Propose Change to Message

in Registry

License & Registry Change Control

License & Registry Change Control

Existing Message Change Control

Existing Message Change Control

NeedXMLMessageStructure

Need New

License?

Need New

License?

In the Registry

?

In the Registry

?

Evaluate Lead PMOEvaluate Lead PMO

Deploy Using XML Message Structure

Deploy Using XML Message Structure

Deployment Change Control

Deployment Change Control

NeedsEndorsement

?

NeedsEndorsement

?

New XMLMessage

Program ManagementProgram Management

Evaluate Proponent and StewardEvaluate Proponent and Steward

Propose Change to SDO XML Message

Propose Change to SDO XML Message

ModificationNeeded to SDO

Standard?

ModificationNeeded to SDO

Standard?

Propose New

Message in Registry

Propose New

Message in Registry

New Message Change Control

New Message Change Control

Identify Lead PMO

Identify Lead PMO

Propose Use of SDO

Message

Propose Use of SDO

Message

Yes

No

Yes

Develop Data Requirements, Processes

& Policy

Develop Data Requirements, Processes

& Policy

Page 21: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 21 or 4125 May 2005

Functional Integration Working Group (“FIWG”)Functional Integration Working Group (“FIWG”)

Review Data Package/PlanReview Data Package/Plan

Data Standards Configuration Management Board (“Board”)Data Standards Configuration Management Board (“Board”)

Approve / Reject Data Review

Packages

Approve / Reject Data Review

Packages

Data Standards Configuration Management Board Chair (“Chair”)Data Standards Configuration Management Board Chair (“Chair”)

PrioritizePrioritize

Present Preliminary Data

Review Packages

Present Preliminary Data

Review Packages

Data Standards Configuration Manager (“Manager”)Data Standards Configuration Manager (“Manager”)

Coordinate Support Staff

Efforts

Coordinate Support Staff

Efforts

Assemble Data Review

Packages

Assemble Data Review

Packages

Data Admin Support Staff (“Admin Staff”)Data Admin Support Staff (“Admin Staff”)

Data Analysis and Triage Support Staff (“Analysis Staff”)Data Analysis and Triage Support Staff (“Analysis Staff”)

Impact AnalysisImpact Analysis

Ad Hoc Support Staff (“Add Hoc Staff”)Ad Hoc Support Staff (“Add Hoc Staff”)

Functional and Detailed Impact

Analysis

Functional and Detailed Impact

Analysis

Identify Data Issues/NeedsIdentify Data Issues/Needs

Board Admin Support

Board Admin Support

Staff Admin Support

Staff Admin Support

Staff Admin Support

Staff Admin Support

Staff Admin Support

Staff Admin Support

Present Data Review

Packages

Present Data Review

Packages

Identify / Adopt StandardsDevelop

Procedures

Identify / Adopt StandardsDevelop

Procedures

Identify / Adopt StandardsDevelop

Procedures

Identify / Adopt StandardsDevelop

Procedures

Staff Admin Support

Staff Admin Support

Measure Effectiveness

Coordinate Data Management PlanDevelop Data Review PackageIdentify Data Management Requirements

1

1a

2

2c

2b

2a 3b

3

3a

4f

4e

4d

4

4c

4b

4a

5d

Measure Impact Analysis

Effectiveness

Measure Impact Analysis

Effectiveness

6a

Measure Process Effectiveness and Assemble

Metrics

Measure Process Effectiveness and Assemble

Metrics

Present Data Metrics

Present Data Metrics

Evaluate Data Metrics ReportsEvaluate Data

Metrics Reports

7c

7b

6 & 7

Measure Overall Effectiveness

Measure Overall Effectiveness

6b

Staff Admin Support

Staff Admin Support

7a“Staffing” of

Policy, Guidance,

Agreements, etc.

“Staffing” of Policy,

Guidance, Agreements, etc.

5e

Coordinate Data Management

Plan

Coordinate Data Management

Plan

5a“Staffing” of

Policy, Guidance,

Agreements, etc.

“Staffing” of Policy,

Guidance, Agreements, etc.

5e

“Staffing” of Policy,

Guidance, Agreements, etc.

“Staffing” of Policy,

Guidance, Agreements, etc.

5e

Harmonization and

Functional Analysis

Harmonization and

Functional Analysis

Harmonization,Integration, and

Develop Agreements

Harmonization,Integration, and

Develop Agreements

5b

5c

Page 22: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 22 or 4125 May 2005

Proof of Concept

Implementing An Effective Meta-data Management Environment at Worthington Industries, Inc.

Prepared by John R. FriedrichIn conjunction with

Meta Integration® Technologies, Inc. (MITI) and Worthington Industries, Inc.as part of a

Meta Integration® Based Proof of Concept

Copyright© Meta Integration® Technologies, Inc.

Page 23: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 23 or 4125 May 2005

POC Introduction

• Worthington Industries, Inc.• Worthington Industries is a leading diversified metal

processing company with annual sales of approximately $2 billion.

• Headquartered in Columbus, Ohio, Worthington employs over 8000 people and operates 61 facilities in 10 countries.

Copyright© Meta Integration® Technologies, Inc.

Page 24: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 24 or 4125 May 2005

Informatica ETL Server

Data WarehouseOracle RDBMS

(HP UX)

ETL Staging AreaOracle RDBMS

(HP UX)

Worthington Industries Legacy Systems

Oracle 11iTransactional ERPDatabase (HP UX)

Worthington Data Flows

Data Mart(s)Oracle RDBMS

(HP UX)

Cognos ReportNet

Data Mart(s)Oracle RDBMS

(HP UX)Data Mart(s)

Oracle RDBMS(HP UX)Data Mart(s)

Oracle RDBMS(HP UX)

Copyright© Meta Integration® Technologies, Inc.

Page 25: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 25 or 4125 May 2005

Informatica ETL Server

Data WarehouseOracle RDBMS

(HP UX)

Data Mart(s)Oracle RDBMS

(HP UX)

Cognos ReportNet

Data Mart(s)Oracle RDBMS

(HP UX)Data Mart(s)

Oracle RDBMS(HP UX)Data Mart(s)

Oracle RDBMS(HP UX)

ETL Staging AreaOracle RDBMS

(HP UX)

Worthington Industries Legacy Systems

Oracle 11iTransactional ERPDatabase (HP UX)

InformaticaRepository

Oracle RDBMS(HP UX)

CognosRepository

Oracle RDBMS(HP UX)

Model MartRepository

Oracle RDBMS(HP UX) Informatica

Designer

ErwinCognos

Framework Manager

Other Development

ToolsOther

Development Tools

Other Development

Tools

Meta-data is Everywhere

Copyright© Meta Integration® Technologies, Inc.

Page 26: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 26 or 4125 May 2005

Informatica ETL Server

Data WarehouseOracle RDBMS

(HP UX)

ETL Staging AreaOracle RDBMS

(HP UX)

Worthington Industries Legacy Systems

Oracle 11iTransactional ERPDatabase (HP UX)

InformaticaRepository

Oracle RDBMS(HP UX)

CognosRepository

Oracle RDBMS(HP UX)

Model MartRepository

Oracle RDBMS(HP UX) Informatica

Designer

Erwin

Cognos Framework

Manager

Other Development

ToolsOther

Development Tools

Other Development

Tools

Meta-data Integration

Meta IntegrationRepositoryDatabase

Oracle (HP UX)

Meta Integration

Works

Meta Integration

Model Bridge

Data Mart(s)Oracle RDBMS

(HP UX)

Cognos ReportNet

Data Mart(s)Oracle RDBMS

(HP UX)Data Mart(s)

Oracle RDBMS(HP UX)Data Mart(s)

Oracle RDBMS(HP UX)

Copyright© Meta Integration® Technologies, Inc.

Page 27: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 27 or 4125 May 2005

Meta Data Web Interface

Copyright© Meta Integration® Technologies, Inc.

Page 28: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 28 or 4125 May 2005

Browse Structure

Copyright© Meta Integration® Technologies, Inc.

Page 29: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 29 or 4125 May 2005

Browse Meta-Data

Copyright© Meta Integration® Technologies, Inc.

Page 30: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 30 or 4125 May 2005

Impact Analysis

Copyright© Meta Integration® Technologies, Inc.

Page 31: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 31 or 4125 May 2005

Full MDR Search

Copyright© Meta Integration® Technologies, Inc.

Page 32: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 32 or 4125 May 2005

Search

Copyright© Meta Integration® Technologies, Inc.

Page 33: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 33 or 4125 May 2005

Mapping to Other (Impacted) Systems/Meta-Data

Copyright© Meta Integration® Technologies, Inc.

Page 34: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 34 or 4125 May 2005

Show Standard Vendor

Copyright© Meta Integration® Technologies, Inc.

Page 35: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 35 or 4125 May 2005

Show standard vendor with SOX certification added

Copyright© Meta Integration® Technologies, Inc.

Page 36: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 36 or 4125 May 2005

Mapping to other systems (impacted systems)

Copyright© Meta Integration® Technologies, Inc.

Page 37: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 37 or 4125 May 2005

PowerCenter

Legacy to ERP ETL Example

Oracle ERPASN

OBDC

ERwin XML

Informatica Source Analyzer Informatica Target Analyzer

OBDC

ERwin XML

OB

DC

Meta-Data RepositoryERwin ERwin

OB

DC

Copyright© Meta Integration® Technologies, Inc.

Page 38: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 38 or 4125 May 2005

PowerCenter

Legacy to ERP ETL Example

Oracle ERPASN

OBDC

ERwin XML

Informatica Source Analyzer Informatica Target Analyzer

OBDC

ERwin XML

OB

DC

Meta-Data RepositoryERwinERwin

OB

DC

Copyright© Meta Integration® Technologies, Inc.

Page 39: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 39 or 4125 May 2005

Import From Informatica

Copyright© Meta Integration® Technologies, Inc.

Page 40: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 40 or 4125 May 2005

View Into MDR

Copyright© Meta Integration® Technologies, Inc.

Page 41: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 41 or 4125 May 2005

Export into ERwin

Copyright© Meta Integration® Technologies, Inc.

Page 42: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 42 or 4125 May 2005

Manipulate Model in ERwin

Copyright© Meta Integration® Technologies, Inc.

Page 43: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 43 or 4125 May 2005

Export back to Informatica as ETL Source

Copyright© Meta Integration® Technologies, Inc.

Page 44: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 44 or 4125 May 2005

Update Transformations in Informatica

Copyright© Meta Integration® Technologies, Inc.

Page 45: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 45 or 4125 May 2005

Subset Oracle ERP Model

Copyright© Meta Integration® Technologies, Inc.

Page 46: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 46 or 4125 May 2005

Model in MDR

Copyright© Meta Integration® Technologies, Inc.

Page 47: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 47 or 4125 May 2005

Subset ERP in ERwin

Copyright© Meta Integration® Technologies, Inc.

Page 48: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 48 or 4125 May 2005

Oracle ERP Subset Model as Target of Informatica ETL

Copyright© Meta Integration® Technologies, Inc.

Page 49: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 49 or 4125 May 2005

Standardization Support

Oracle ERP

ASN

Oracle DDL

Create …Select * ...

OBDC

ERwin XML

Informatica

OBDC

ERwin XML

OB

DC

Data Standards Packages inMeta-Data RepositoryStandards Information

Editing in ERwinSchema Forward Engineering

Using ERwin

ASNLegacy Systems

Cognos

Warehouse

DW DDL

Create …Select * ...

Other Meta-Data and Schema XML/XSD

Copyright© Meta Integration® Technologies, Inc.

Page 50: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 50 or 4125 May 2005

Oracle ERP

ASN

Oracle DDL

Create …Select * ...

OBDC

ERwin XML

Informatica

OBDC

ERwin XML

OB

DC

Meta-Data RepositoryStandards InformationEditing in ERwin

Schema Forward EngineeringUsing ERwin

ASNLegacy Systems

Cognos

Warehouse

DW DDL

Create …Select * ...

Other Meta-Data and Schema XML/XSD

Metadata flows

Copyright© Meta Integration® Technologies, Inc.

Page 51: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 51 or 4125 May 2005

Standardization Example: Vendor Meta-Data Package

Copyright© Meta Integration® Technologies, Inc.

Page 52: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 52 or 4125 May 2005

Edit in ERwin

Copyright© Meta Integration® Technologies, Inc.

Page 53: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 53 or 4125 May 2005

Migrate Standard Reference Mappings

Copyright© Meta Integration® Technologies, Inc.

Page 54: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Slide 54 or 4125 May 2005

Export to Other Meta-Data Environments

ERwin

Informatica

Cognos

XML,Rose,Java,Legacy,ETC…..

Copyright© Meta Integration® Technologies, Inc.

Page 55: Metadata Based Impact and Lineage Analysis Across ... · Finance Accts Pay Data Feeds XML API WS API Meta-Data Repository Business Intelligence Design and Reporting ReportNet Cognos

Conclusion


Recommended