Datastage Realtime Projects5

8/13/2019 Datastage Realtime Projects5

1/32

IBM Software Group

2005 IBM Corporation

Lowering the total cost of ownership of SAP deploymentsthrough Enterprise Data IntegrationStphane Heckel IBM Information Integration [email protected]


2/32

IBM Software Group | WebSphere software

2

Tremendous Strain DuringMulti-Year ERP Rollouts

Non-SAP

Non-SAP

Non-SAP

Non-SAP


3/32


3

Data Points from the Marketplace According to The Conference Board "ERP Trends" Research Report, of117 companies surveyed:

40% failed to achieve business case even 12 months after implementation- When benefits were achieved, it took six months longer than

expected75% experienced a moderate to severe "productivity dip duringimplementation

Implementation costs were 25% over budgetSupport costs underestimated by average 20 percent for first year following implementation75 percent report no "hard-stop" to implementation because ERP systems areconstantly evolving


4/32


4

Critical Issues and Concerns Learned From SAP CustomersDisparate, Undocumented Sources and Business Practices

Poor understanding of source systems and business practices complicatesability to map migration and delays project

Double-work Supporting Old & New Infrastructures Simultaneously

Must support current reporting/BI environment while simultaneously migratingto new

Double-work puts undo strain on systems, applications and personnelData Quality and Incompatibility Problems

Difficulty cleansing, transforming legacy data to conform to new standards

Incompatibilities with Diverse Applications (harmonization)

Delays due to understanding and mapping integration routines to conform to3rd party applications


5/32


5

Data Quality and Harmonization IssuesMultiple sources for the same entity (customers, suppliers, material items, etc)Semantic differences in identifiers are different across systems

how do you resolve to the right one?Lack of standards in the dataEmbedded business intelligenceCleansing, transforming, harmonizing and synchronizing the data

is this a manual or automatic process? Is it repeatable?Change data capture implementation

how are new records handled and validated from various data entry points?Business rules for resolving data conflicts and survivorship

how are these developed and enforced?Ownership and accountability

in a cross-system coordination, who owns responsibility for deciding on the rightinformation?

Extreme Scalability how do you efficiently handle and process millions of customers, products,parts, locations, hierarchies?

Results: SAP environments that are cumbersome to implement,

perform under expectations and do not return promised ROI


6/32


6

IBM WebSphere Data Integration Suite

Com plementary To BPM,

EAI, and EII Techn olo gies

Open, Service-Oriented

Architecture Integrated Data Profiling &

Data Quality Complex Data

Transformation and Routing Reusable Components &

Rules

Unlimited Performance withLinear Scalability

Enterprise MetadataManagement

Anytime, AnywhereConnectivity

Industry Standard Compliant(XML, EDI, JMS, JCA) Industry-Ready Integration

Solutions

Un derst and

Se rvice -Ori ented Arc hitecture

SourceidentificationData analysisSource meta datavalidationDiscover dataquality, contentand structureBusiness rulevalidation

Connect

Inte grated Meta data Manage ment

Parallel P rocessing

Cle anse

Standardize,match, andcorrect dataDe-dupe entriesData abstractionidentificationData qualityassurance

Tra nsfor m

Normalize,transform andenrich data.Unique keymanagementDevelop load jobsProcess rules andreference datamanagement

Implementation Methodology

Fed erate

Virtualizeaccess todisparate

information


7/32


7

Rapid Deployment Framework

Deployments, Migrations, Instance Consolidations

SAP

LegacyData

SAP

PeopleSoftEnterprise

OracleApps


One

Other

Data

Siebel

Assessment andValidation

SourceSystems

Standardization &Alignment

Mapping, Transformation& Movement

TargetTargetConnectivity

SourceConnectivity

Cleansing &Harmonization

Underst and

Se r vice -Oriented Architec ture

Source

identificationData analysisSource meta datavalidationDiscover dataquality, contentand structureBusiness rulevalidation

Connect

Integrated Metadata M anage mentParallel P rocessing

Cle anse

Standardize,

match, andcorrect dataDe-dupe entriesData abstractionidentificationData qualityassurance

Tra nsform

Normalize,

transform andenrich data.Unique keymanagementDevelop load jobsProcess rules andreference datamanagement

Implementation Methodology

Fed erate

Virtualizeaccess todisparateinformation

Major Cost Savings OpportunitiesMajor Cost Savings Opportunities


8/32


8

DataHarmonization

Deliverables: An Application that

Translates,Integrates, Enriches,

Standardizes andConsolidates

-The Harmonized Data

Structure-

Cross Ref Tables-

Match Index Tables

DataExtraction

Deliverables:

StagedExtracted Data

DataAssessment

Deliverables:Data Quality Rpt

-Translation

Tables-

Gap AnalysisReports

-Business Rules &

Requirements-

TechnicalSpecs/Design

SolutionImplementation

Deliverables:

An Application thatPrepares,Transforms and

Loads Data into thefinal Solution

- A procedure for

handling dailytransactionalprocessing

Data Integration Methodology & Tools

Metadata ManagementDeliverables: Metadata Repository

MetaStage & MetaBrokers

DataAlignment

Deliverables: An Application

thatSplits Domains

andTranslates each

source-

The AlignedData Structure

DataStage & Connectivity PACKs

QualityStage

ProfileStage & AuditStage

DataStage TX & RTI


9/32


10/32


10

Complementary ValueSAP

Sources &Targets

Non-SAPSources

RFCIDoc

FileJDBCJMSSOAPHTTP

IBMs Complementary Value: extensive native connectivity, support for complexindustry standard data formats, source data profiling, robust data quality & harmonization,

and large bulk data movement for heterogeneous environments

SAP XI

ComplexTransactionData formats

EnterpriseApplications

Mainframe,other legacy

sources

WebSphere Data Integration Suite

PACKfor SAP XI

DataStage TX

PACKs for EDI,HIPAA, SWIFT

RTI

DataStage

PACKs for SAP, PeopleSoft,Siebel, Oracle Applications,

Web services,JMS bindings

Web Services,JMS bindings

Parallelprocessing


11/32


11

Pack Interfaces


OracleApplicationsSiebel

PeopleSoftEnterpriseOne

SAP R/3,mySAP Bus.

Suite

SAPNetWeaver

BI

Direct

Hierarchy

Direct

Hierarchy

EIM

BC Hierarchy Direct

Direct BAPI

IDoc ABAP (extract)

BAPI

OpenHubServicesInterface

EnterpriseApplication

SourcesSources and Targets

IBM WebSphereDataStage Pack

for SAP BW


for SAP R/3


for Siebel

IBM WebSphere

DataStage Packfor PeopleSoftEnterpriseOne

IBM WebSphere

DataStage Packfor PeopleSoftEnterprise

IBM WebSphere

DataStage Packfor OracleApplications

Connec t

Deliver

Enhance


12/32


12

IBM WebSphere DataStage Pack for SAP R/3Bi-directional connectivity

Load non-SAP data into SAP R/3Extract data from SAP R/3

Also supports mySAP Business Suite modulesNative ABAP interfaceEliminates manual coding for extracting data whilespeeding deployment

IDoc interfaceExtracts selected IDoc types from R/3 and translates

them as source data for DataStage job streams.Load interface allows users to load data into R/3 usingbuilt-in validations inherent in SAP IDoc processing.

BAPI InterfaceSupports extracting and loading SAP data via BAPIsLets users work with a business view therebyeliminating any requirement of having complex SAPapplication module knowledge.

Supports Unicode data loads and extractsSupported with DataStage server on Windows, Linux and64-bit HP-UX-Itanium platforms


13/32


13

IBM WebSphere DataStage Pack for SAP BW

Simple, easy and packaged way to load non-SAP data into SAP NetWeaver BI via BAPIs

Imports non-SAP meta data, including creationof InfoObjects, InfoSources, InfoPackages andSource Systems

Delivers performance with increasing data

volumesSupports extracting information from SAPNetWeaver BI for use in other data marts, datawarehouses, reporting applications and otherdownstream targets.

Utilizes OpenHub Services so Pack users cangraphically select OpenHub targets and initiateProcess Chains


14/32


14

Source Identification- Determine system of record info- Define what data is needed

Time

Data Analysis and Discovery

- Understand source data content,quality and completeness- Analyze business rule compliance

Data Correction and EnrichmentAnalysis/Rules-Develop rules to fix data quality-De-dupe system records-Create data consistencyData Abstraction Identification

- Identify key entities such ascustomer, account and theirimportance to the business

Transformations and Data Normalization-Specify transform steps to stdizesource data

-Apply rules to cleanse data

Unique Key Management

- Maintain source/target referencedata for migrated entities to facilitatecreating single source across systems

Process Rules and Reference DataManagement- Develop apps/review programs tomanage and maintain business rulesto migrate data and reference data

Total ProjectTime

Current Manual Effort Ascential Platform Experience

+/- 45% Savings

Conventional vs. IBM Solution Comparison


15/32


15

Source Identification- Determine system ofrecord info- Define what data isneeded

Time

Data Analysis andDiscovery

- Understand sourcedata content, qualityand completeness

- Analyze business rulecompliance

Data Correction andEnrichmentAnalysis/Rules-Develop rules to fix dataquality-De-dupe system records-Create data consistencyData AbstractionIdentification- Identify key entitiessuch as customer,account and theirimportance to thebusinessTransformations and

Data Normalization- Specify transform

steps to stdize sourcedata

- Apply rules to cleansedataUnique Key

Management- Maintain source/targetreference data formigrated entities tofacilitate creating singlesource across systemsProcess Rules andReference DataManagement- Develop apps/reviewprograms to manage andmaintain business rulesto migrate data andreference data

Total ProjectTime

Current Manual Effort Ascential Platform Experience

The table below represents the effort associated with performing the Technical Design and InterfaceDevelopment tasks for a large SAP Instance Consolidation. The client approved this comparative analysisof the project using both conventional development methods and using the Ascential DataStage Integration platform. The Number of Items column represents the number of technical designs, and the number ofinterface specifications associated with estimate. A list of assumptions associated with the estimates is alsoidentified.The documentation process associated with Ascentials Iterations development methodology will be able togreatly reduce the creation time of functional designs as well.

Description # Of Items Current Ascential

Technical Designs 122 867 Days 15 DaysInterface Development

Low Difficulty 35 237 Days 13 DaysMed Difficulty 28 661 Days 42 DaysHigh Difficulty 49 936 Days 147 days

Total Effort 2,701 Days 217 Days

Project Estimate Assumptions: Each Technical Design Diagram can be completed in no more than 1 hour Interfaces that were identified as Low Difficulty were given an allocation of 3 hours for these

items. Interfaces that were identified as Medium Difficulty will take no more than 12 hours to

complete. Interfaces that were identified as High Difficulty were given an allocation of 24 hours for these

items.

Cost Down!!

Conventional vs. IBM Solution Comparison:


16/32


16

Project Savings - By IBM ModuleProject Savings Involve All Phases of the Process. The Following areExamples from Actual Client Engagements:

Source System Analysis -------------> ProfileStage30% of the project effort up to 40% productivity gain

Data Cleansing ---------------------------> QualityStage25% of the project effort up to 50% productivity gainIncreased data accuracy and quality increases confidence of end user in projectdeliverable.

ETL Construction ------------------------> DataStage20% of project effort up to 40 % productivity gain

Data Management Services ------------> MetaStage15% of project effort up to 20% productivity gain

Application System Connectivity -----> PACK for SAP10% of project effort up to 30% productivity gain

On average: approximately 45% savings across the project range is 30% to 60%


17/32


17

The IBM AdvantageOne integrated platform for all tasksOne consistent software architectureIntegrated metadata managementScalable to meet the largest demands ofdata migrationExperience in the largest data migrationprojects

Ascentials strategic relationship with SAPIBMs products are certified by SAPDirect engineering contact betweenSAP and IBMEarly access to software provides aspeed to market advantage

IBMs relationships with SystemsIntegrators make additional projectmanagement and implementationresources available

ASCL ManualDiscover Understand data & data models

Assess cross-source incompatibility Assess/audit data quality Automatically generate ETL jobsUnlimited scalabilityPrepareName and address cleansing

Probabilistic matchingMatching on any user-defined fieldFree-form text standardizationUnlimited scalabilityTransformConnect to source systemsOpen, extensible parallel architect

Cross-tool metadata managementRealtime/batch integrationDesign sequentially, deploy in parallelUnlimited scalability


18/32


18

Customer Examples


19/32


19

Bombardier

BOMBARDIER needed aneasier way to get visibility toparts information and demandacross multiple manufacturinglines to better manage

inventory costs. Many legacysystems with non-SAP data andvery poor data quality.

Selected IBM WebSphere QualityStage to match andstandardize data from legacysystems and IBMWebSphere DataStage to

put standardized data into thecorrect format for loading toSAP R/3 and SAP BW.

IBMs data standardization andBAPI loading capabilitiesdramatically reduces cycle timesand resources

The enhanced ability to viewaccurate consolidated demand forinventory enables better inventorymanagement

Every 1% reduction in overallinventory worth C$104 MM

Problem Solution Result

BOMBARDIER | $21 billion manufacturingand services company of aircraft, rail, and

motorized recreational products.80,000 employees in 24 countries


20/32


21/32


21

Deutsche Bahn

Deutsche Bahn AG | Germanys passengerand freight train company, one of the most important

contracting authorities in Germany with more than200,000 employees

.

For years Deutsche Bahn Grouphad to deal with traditional hand-coding more than 4,000interfaces, very hard to maintainand poorly documented. Their

demand for modernization,company growth and increasedperformance was hindered by animmense lack of integration in thebusiness data of the wholeenterprise.

Automated integration of manyhand-coded applications likeTicketing, PassengerReservation systems etc. as wellas improved quality of data

supporting several datawarehouse initiatives, e.g. forEnergy Saving, FacilityManagement etc. with betterperformance than ever.

Quicker deployment times,better documentation andimproved data managementprocess across the wholeenterprise

Better quality of data entailsimproved decision processessupporting the companys goalof getting in shape for thefuture

ProblemSolution Result


22/32


23/32


23

Novo Nordisk

Critical need to profile proposeddrug products against historicaldrug adoption rates in order toimprove demand forecast andproduction scheduling.

Massive integration problemsconsolidating history from legacysystems.

Implemented IBM WebSphere DataStage , with the IBMWebSphere DataStage Pack forSAP BW to consolidate historicalsales data into SAP BW. Reduced

cycle times dramatically.

Novo Nordisk can conduct 5and 10 year performance trendanalysis to maximize the valuefrom new products that meetthe needs of people with

diabetes.


NOVO NORDISK | Leading diabetes carecompany. Dkr24 billion company with over 18,000

employees in 68 countries


24/32


24

High inventory carrying costsat 22 service centers around thecountry, serving as distributionhubs for lawn and gardeningsupplies. Without visibility intotheir big customers forecasteddemand, Scotts was forced tocarry excess inventory to protectagainst stock-outs and resultingcustomer dissatisfaction.

Scotts deployed IBM WebSphere DataStage ,IBM WebSphere QualityStage and IBM WebSphere DataStagePack for SAP R/3 to collectpoint-of-sale data and thencleanse, match and load thatto data in mySAP BI.

Better insights to demand,reduced inventory through thesupply chain, increased fill ratesfrom 92-98%, and reduceddistribution/holding costs

Entire project reduced on-handinventory by 30% or $99.4 millionin FY2002 compared to FY2001,contributing to a $158 millionincrease in cash provided byoperations

The Scotts Company | producer and marketerof products for do-it-yourself lawn care, gardening,

and professional horticulture


Scotts


25/32


26/32


27/32


27

Use IBM WebSphere DataIntegration Suite to prepare andload data into SAP R/3 in thecorrect format, automaticallyusing the correct SAP formatfor loading; use IBMWebSphere QualityStage andWebSphere DataStage to loadbetter quality data into R/3.

Fast consolidation andharmonization of data from legacysystems into SAP R/3 and theGlobal Part Master Data.

Enhanced ability to viewconsolidated demand for Part andenables better inventorymanagement.

Example: a 20%-30% reduction induplicated parts is worth 10M

Outokumpu | a Finnish dynamicmetals and technology group with a

clear target of becoming the numberone in stainless steel, annual net salesover 6 billion and 19,000 employeesin 40 countries

Outokumpu


Needed a way to get visibilityto parts information anddemand across multiplemanufacturing plants to better

manage inventory costs andcentralize purchasing, butparts information was lockedin several differentincompatible systems andcontained much duplicatedata.


28/32


28

Rich Products | At $1.5 billion in revenue,one of the worlds premier family-ownedfood companies .

To provide faster customer service andincrease supply and demand chainvisibility, Rich wanted to implement anavailable-to-promise application tocomplement their manufacturingplanning application and other legacyand EDI sources. This requiredintegrating 40 interfaces to support amessage volume of over 250,000transactions per week.

Avoided hand-codingby deploying IBMWebSphereDataStage TX as a

centralized hub forSAP interfacing toand from IDocs fordata from EDIsources, i2 and QADsystems.

Justified the project based onanticipated maintenance savings,but also realized reduced effortand faster project development

than originally expected. Added flexibility that will enableRich Products to roll out moreSAP applications, move off themainframe and adopt e-businessapplications in the future.


Rich Products


29/32


29

Customers needed to make thechange from Lira to Euro at over42,000 taget destinations. The goalwas to aggregrate, integrate andmanage this data change using aunified set of platforms. The technicalchallenge was to gather data fromSAP R/3 and load them into MicrosoftSQL or SAP BW depending on thegiven project and need.

Implemented IBMWebSphere DataStage Packfor SAP R/3 andWebSphere DataStage Packfor SAP BW. The solutionenabled extraction andmanagement of native datafrom SAP R/3 and theloading into SAP BW ofdata from diverse andheterogenous sources.

Poste Italiane was able tooptimise and increase thespeed and efficiency of theprocess of introduction ofEuro values across 42,000distributors and offices on theitalian national territory.


Posteitaliane | the leading mailing serviceprovider in Italy. The company also offers FinancialServices with a network of 14.000 post offices.

.

Poste Italiane


30/32


30

Needed to increase theefficiency of internallogistical processes byrationalizing the data flowprocess to and within theinternal SAP BWenvironment and otherapplications (AmosSystem).

Implemented IBMWebSphere DataStageand the WebSphereDataStage Pack for SAPBW to provide a unifiedview of heterogeneouslysourced data.

Saipem was able toachieve its objective ofunifiying and quicklymanaging the processesof data flow within theorganisation.


Saipem | providing services to the oil and gas

industry - as the sole contractor to cover all phasesof the upstream operational cycle: drilling, oil and gasproduction, and transportation to shore.

.

SAIPEM


31/32


32/32

IBM Software Group

2005 IBM Corporation

Thank You

Stphane Heckel IBM Information Integration [email protected]

Date post:	04-Jun-2018
Category:	Documents
Upload:	maheshumbarkar
View:	221 times
Download:	0 times

Datastage Realtime Projects5

Documents