Post on 04-Jun-2018
transcript
8/13/2019 Datastage Realtime Projects5
1/32
IBM Software Group
2005 IBM Corporation
Lowering the total cost of ownership of SAP deploymentsthrough Enterprise Data IntegrationStphane Heckel IBM Information Integration Solutionsstephane.heckel@fr.ibm.com
8/13/2019 Datastage Realtime Projects5
2/32
IBM Software Group | WebSphere software
2
Tremendous Strain DuringMulti-Year ERP Rollouts
Non-SAP
Non-SAP
Non-SAP
Non-SAP
8/13/2019 Datastage Realtime Projects5
3/32
IBM Software Group | WebSphere software
3
Data Points from the Marketplace According to The Conference Board "ERP Trends" Research Report, of117 companies surveyed:
40% failed to achieve business case even 12 months after implementation- When benefits were achieved, it took six months longer than
expected75% experienced a moderate to severe "productivity dip duringimplementation
Implementation costs were 25% over budgetSupport costs underestimated by average 20 percent for first year following implementation75 percent report no "hard-stop" to implementation because ERP systems areconstantly evolving
8/13/2019 Datastage Realtime Projects5
4/32
IBM Software Group | WebSphere software
4
Critical Issues and Concerns Learned From SAP CustomersDisparate, Undocumented Sources and Business Practices
Poor understanding of source systems and business practices complicatesability to map migration and delays project
Double-work Supporting Old & New Infrastructures Simultaneously
Must support current reporting/BI environment while simultaneously migratingto new
Double-work puts undo strain on systems, applications and personnelData Quality and Incompatibility Problems
Difficulty cleansing, transforming legacy data to conform to new standards
Incompatibilities with Diverse Applications (harmonization)
Delays due to understanding and mapping integration routines to conform to3rd party applications
8/13/2019 Datastage Realtime Projects5
5/32
IBM Software Group | WebSphere software
5
Data Quality and Harmonization IssuesMultiple sources for the same entity (customers, suppliers, material items, etc)Semantic differences in identifiers are different across systems
how do you resolve to the right one?Lack of standards in the dataEmbedded business intelligenceCleansing, transforming, harmonizing and synchronizing the data
is this a manual or automatic process? Is it repeatable?Change data capture implementation
how are new records handled and validated from various data entry points?Business rules for resolving data conflicts and survivorship
how are these developed and enforced?Ownership and accountability
in a cross-system coordination, who owns responsibility for deciding on the rightinformation?
Extreme Scalability how do you efficiently handle and process millions of customers, products,parts, locations, hierarchies?
Results: SAP environments that are cumbersome to implement,
perform under expectations and do not return promised ROI
8/13/2019 Datastage Realtime Projects5
6/32
IBM Software Group | WebSphere software
6
IBM WebSphere Data Integration Suite
Com plementary To BPM,
EAI, and EII Techn olo gies
Open, Service-Oriented
Architecture Integrated Data Profiling &
Data Quality Complex Data
Transformation and Routing Reusable Components &
Rules
Unlimited Performance withLinear Scalability
Enterprise MetadataManagement
Anytime, AnywhereConnectivity
Industry Standard Compliant(XML, EDI, JMS, JCA) Industry-Ready Integration
Solutions
Un derst and
Se rvice -Ori ented Arc hitecture
SourceidentificationData analysisSource meta datavalidationDiscover dataquality, contentand structureBusiness rulevalidation
Connect
Inte grated Meta data Manage ment
Parallel P rocessing
Cle anse
Standardize,match, andcorrect dataDe-dupe entriesData abstractionidentificationData qualityassurance
Tra nsfor m
Normalize,transform andenrich data.Unique keymanagementDevelop load jobsProcess rules andreference datamanagement
Implementation Methodology
Fed erate
Virtualizeaccess todisparate
information
8/13/2019 Datastage Realtime Projects5
7/32
IBM Software Group | WebSphere software
7
Rapid Deployment Framework
Deployments, Migrations, Instance Consolidations
SAP
LegacyData
SAP
PeopleSoftEnterprise
OracleApps
PeopleSoftEnterprise
One
Other
Data
Siebel
Assessment andValidation
SourceSystems
Standardization &Alignment
Mapping, Transformation& Movement
TargetTargetConnectivity
SourceConnectivity
Cleansing &Harmonization
Underst and
Se r vice -Oriented Architec ture
Source
identificationData analysisSource meta datavalidationDiscover dataquality, contentand structureBusiness rulevalidation
Connect
Integrated Metadata M anage mentParallel P rocessing
Cle anse
Standardize,
match, andcorrect dataDe-dupe entriesData abstractionidentificationData qualityassurance
Tra nsform
Normalize,
transform andenrich data.Unique keymanagementDevelop load jobsProcess rules andreference datamanagement
Implementation Methodology
Fed erate
Virtualizeaccess todisparateinformation
Major Cost Savings OpportunitiesMajor Cost Savings Opportunities
8/13/2019 Datastage Realtime Projects5
8/32
IBM Software Group | WebSphere software
8
DataHarmonization
Deliverables: An Application that
Translates,Integrates, Enriches,
Standardizes andConsolidates
-The Harmonized Data
Structure-
Cross Ref Tables-
Match Index Tables
DataExtraction
Deliverables:
StagedExtracted Data
DataAssessment
Deliverables:Data Quality Rpt
-Translation
Tables-
Gap AnalysisReports
-Business Rules &
Requirements-
TechnicalSpecs/Design
SolutionImplementation
Deliverables:
An Application thatPrepares,Transforms and
Loads Data into thefinal Solution
- A procedure for
handling dailytransactionalprocessing
Data Integration Methodology & Tools
Metadata ManagementDeliverables: Metadata Repository
MetaStage & MetaBrokers
DataAlignment
Deliverables: An Application
thatSplits Domains
andTranslates each
source-
The AlignedData Structure
DataStage & Connectivity PACKs
QualityStage
ProfileStage & AuditStage
DataStage TX & RTI
8/13/2019 Datastage Realtime Projects5
9/32
8/13/2019 Datastage Realtime Projects5
10/32
IBM Software Group | WebSphere software
10
Complementary ValueSAP
Sources &Targets
Non-SAPSources
RFCIDoc
FileJDBCJMSSOAPHTTP
IBMs Complementary Value: extensive native connectivity, support for complexindustry standard data formats, source data profiling, robust data quality & harmonization,
and large bulk data movement for heterogeneous environments
SAP XI
ComplexTransactionData formats
EnterpriseApplications
Mainframe,other legacy
sources
WebSphere Data Integration Suite
PACKfor SAP XI
DataStage TX
PACKs for EDI,HIPAA, SWIFT
RTI
DataStage
PACKs for SAP, PeopleSoft,Siebel, Oracle Applications,
Web services,JMS bindings
Web Services,JMS bindings
Parallelprocessing
8/13/2019 Datastage Realtime Projects5
11/32
IBM Software Group | WebSphere software
11
Pack Interfaces
PeopleSoftEnterprise
OracleApplicationsSiebel
PeopleSoftEnterpriseOne
SAP R/3,mySAP Bus.
Suite
SAPNetWeaver
BI
Direct
Hierarchy
Direct
Hierarchy
EIM
BC Hierarchy Direct
Direct BAPI
IDoc ABAP (extract)
BAPI
OpenHubServicesInterface
EnterpriseApplication
SourcesSources and Targets
IBM WebSphereDataStage Pack
for SAP BW
IBM WebSphereDataStage Pack
for SAP R/3
IBM WebSphereDataStage Pack
for Siebel
IBM WebSphere
DataStage Packfor PeopleSoftEnterpriseOne
IBM WebSphere
DataStage Packfor PeopleSoftEnterprise
IBM WebSphere
DataStage Packfor OracleApplications
Connec t
Deliver
Enhance
8/13/2019 Datastage Realtime Projects5
12/32
IBM Software Group | WebSphere software
12
IBM WebSphere DataStage Pack for SAP R/3Bi-directional connectivity
Load non-SAP data into SAP R/3Extract data from SAP R/3
Also supports mySAP Business Suite modulesNative ABAP interfaceEliminates manual coding for extracting data whilespeeding deployment
IDoc interfaceExtracts selected IDoc types from R/3 and translates
them as source data for DataStage job streams.Load interface allows users to load data into R/3 usingbuilt-in validations inherent in SAP IDoc processing.
BAPI InterfaceSupports extracting and loading SAP data via BAPIsLets users work with a business view therebyeliminating any requirement of having complex SAPapplication module knowledge.
Supports Unicode data loads and extractsSupported with DataStage server on Windows, Linux and64-bit HP-UX-Itanium platforms
8/13/2019 Datastage Realtime Projects5
13/32
IBM Software Group | WebSphere software
13
IBM WebSphere DataStage Pack for SAP BW
Simple, easy and packaged way to load non-SAP data into SAP NetWeaver BI via BAPIs
Imports non-SAP meta data, including creationof InfoObjects, InfoSources, InfoPackages andSource Systems
Delivers performance with increasing data
volumesSupports extracting information from SAPNetWeaver BI for use in other data marts, datawarehouses, reporting applications and otherdownstream targets.
Utilizes OpenHub Services so Pack users cangraphically select OpenHub targets and initiateProcess Chains
8/13/2019 Datastage Realtime Projects5
14/32
IBM Software Group | WebSphere software
14
Source Identification- Determine system of record info- Define what data is needed
Time
Data Analysis and Discovery
- Understand source data content,quality and completeness- Analyze business rule compliance
Data Correction and EnrichmentAnalysis/Rules-Develop rules to fix data quality-De-dupe system records-Create data consistencyData Abstraction Identification
- Identify key entities such ascustomer, account and theirimportance to the business
Transformations and Data Normalization-Specify transform steps to stdizesource data
-Apply rules to cleanse data
Unique Key Management
- Maintain source/target referencedata for migrated entities to facilitatecreating single source across systems
Process Rules and Reference DataManagement- Develop apps/review programs tomanage and maintain business rulesto migrate data and reference data
Total ProjectTime
Current Manual Effort Ascential Platform Experience
+/- 45% Savings
Conventional vs. IBM Solution Comparison
8/13/2019 Datastage Realtime Projects5
15/32
IBM Software Group | WebSphere software
15
Source Identification- Determine system ofrecord info- Define what data isneeded
Time
Data Analysis andDiscovery
- Understand sourcedata content, qualityand completeness
- Analyze business rulecompliance
Data Correction andEnrichmentAnalysis/Rules-Develop rules to fix dataquality-De-dupe system records-Create data consistencyData AbstractionIdentification- Identify key entitiessuch as customer,account and theirimportance to thebusinessTransformations and
Data Normalization- Specify transform
steps to stdize sourcedata
- Apply rules to cleansedataUnique Key
Management- Maintain source/targetreference data formigrated entities tofacilitate creating singlesource across systemsProcess Rules andReference DataManagement- Develop apps/reviewprograms to manage andmaintain business rulesto migrate data andreference data
Total ProjectTime
Current Manual Effort Ascential Platform Experience
The table below represents the effort associated with performing the Technical Design and InterfaceDevelopment tasks for a large SAP Instance Consolidation. The client approved this comparative analysisof the project using both conventional development methods and using the Ascential DataStage Integration platform. The Number of Items column represents the number of technical designs, and the number ofinterface specifications associated with estimate. A list of assumptions associated with the estimates is alsoidentified.The documentation process associated with Ascentials Iterations development methodology will be able togreatly reduce the creation time of functional designs as well.
Description # Of Items Current Ascential
Technical Designs 122 867 Days 15 DaysInterface Development
Low Difficulty 35 237 Days 13 DaysMed Difficulty 28 661 Days 42 DaysHigh Difficulty 49 936 Days 147 days
Total Effort 2,701 Days 217 Days
Project Estimate Assumptions: Each Technical Design Diagram can be completed in no more than 1 hour Interfaces that were identified as Low Difficulty were given an allocation of 3 hours for these
items. Interfaces that were identified as Medium Difficulty will take no more than 12 hours to
complete. Interfaces that were identified as High Difficulty were given an allocation of 24 hours for these
items.
Cost Down!!
Conventional vs. IBM Solution Comparison:
8/13/2019 Datastage Realtime Projects5
16/32
IBM Software Group | WebSphere software
16
Project Savings - By IBM ModuleProject Savings Involve All Phases of the Process. The Following areExamples from Actual Client Engagements:
Source System Analysis -------------> ProfileStage30% of the project effort up to 40% productivity gain
Data Cleansing ---------------------------> QualityStage25% of the project effort up to 50% productivity gainIncreased data accuracy and quality increases confidence of end user in projectdeliverable.
ETL Construction ------------------------> DataStage20% of project effort up to 40 % productivity gain
Data Management Services ------------> MetaStage15% of project effort up to 20% productivity gain
Application System Connectivity -----> PACK for SAP10% of project effort up to 30% productivity gain
On average: approximately 45% savings across the project range is 30% to 60%
8/13/2019 Datastage Realtime Projects5
17/32
IBM Software Group | WebSphere software
17
The IBM AdvantageOne integrated platform for all tasksOne consistent software architectureIntegrated metadata managementScalable to meet the largest demands ofdata migrationExperience in the largest data migrationprojects
Ascentials strategic relationship with SAPIBMs products are certified by SAPDirect engineering contact betweenSAP and IBMEarly access to software provides aspeed to market advantage
IBMs relationships with SystemsIntegrators make additional projectmanagement and implementationresources available
ASCL ManualDiscover Understand data & data models
Assess cross-source incompatibility Assess/audit data quality Automatically generate ETL jobsUnlimited scalabilityPrepareName and address cleansing
Probabilistic matchingMatching on any user-defined fieldFree-form text standardizationUnlimited scalabilityTransformConnect to source systemsOpen, extensible parallel architect
Cross-tool metadata managementRealtime/batch integrationDesign sequentially, deploy in parallelUnlimited scalability
8/13/2019 Datastage Realtime Projects5
18/32
IBM Software Group | WebSphere software
18
Customer Examples
8/13/2019 Datastage Realtime Projects5
19/32
IBM Software Group | WebSphere software
19
Bombardier
BOMBARDIER needed aneasier way to get visibility toparts information and demandacross multiple manufacturinglines to better manage
inventory costs. Many legacysystems with non-SAP data andvery poor data quality.
Selected IBM WebSphere QualityStage to match andstandardize data from legacysystems and IBMWebSphere DataStage to
put standardized data into thecorrect format for loading toSAP R/3 and SAP BW.
IBMs data standardization andBAPI loading capabilitiesdramatically reduces cycle timesand resources
The enhanced ability to viewaccurate consolidated demand forinventory enables better inventorymanagement
Every 1% reduction in overallinventory worth C$104 MM
Problem Solution Result
BOMBARDIER | $21 billion manufacturingand services company of aircraft, rail, and
motorized recreational products.80,000 employees in 24 countries
8/13/2019 Datastage Realtime Projects5
20/32
8/13/2019 Datastage Realtime Projects5
21/32
IBM Software Group | WebSphere software
21
Deutsche Bahn
Deutsche Bahn AG | Germanys passengerand freight train company, one of the most important
contracting authorities in Germany with more than200,000 employees
.
For years Deutsche Bahn Grouphad to deal with traditional hand-coding more than 4,000interfaces, very hard to maintainand poorly documented. Their
demand for modernization,company growth and increasedperformance was hindered by animmense lack of integration in thebusiness data of the wholeenterprise.
Automated integration of manyhand-coded applications likeTicketing, PassengerReservation systems etc. as wellas improved quality of data
supporting several datawarehouse initiatives, e.g. forEnergy Saving, FacilityManagement etc. with betterperformance than ever.
Quicker deployment times,better documentation andimproved data managementprocess across the wholeenterprise
Better quality of data entailsimproved decision processessupporting the companys goalof getting in shape for thefuture
ProblemSolution Result
8/13/2019 Datastage Realtime Projects5
22/32
8/13/2019 Datastage Realtime Projects5
23/32
IBM Software Group | WebSphere software
23
Novo Nordisk
Critical need to profile proposeddrug products against historicaldrug adoption rates in order toimprove demand forecast andproduction scheduling.
Massive integration problemsconsolidating history from legacysystems.
Implemented IBM WebSphere DataStage , with the IBMWebSphere DataStage Pack forSAP BW to consolidate historicalsales data into SAP BW. Reduced
cycle times dramatically.
Novo Nordisk can conduct 5and 10 year performance trendanalysis to maximize the valuefrom new products that meetthe needs of people with
diabetes.
Problem Solution Result
NOVO NORDISK | Leading diabetes carecompany. Dkr24 billion company with over 18,000
employees in 68 countries
8/13/2019 Datastage Realtime Projects5
24/32
IBM Software Group | WebSphere software
24
High inventory carrying costsat 22 service centers around thecountry, serving as distributionhubs for lawn and gardeningsupplies. Without visibility intotheir big customers forecasteddemand, Scotts was forced tocarry excess inventory to protectagainst stock-outs and resultingcustomer dissatisfaction.
Scotts deployed IBM WebSphere DataStage ,IBM WebSphere QualityStage and IBM WebSphere DataStagePack for SAP R/3 to collectpoint-of-sale data and thencleanse, match and load thatto data in mySAP BI.
Better insights to demand,reduced inventory through thesupply chain, increased fill ratesfrom 92-98%, and reduceddistribution/holding costs
Entire project reduced on-handinventory by 30% or $99.4 millionin FY2002 compared to FY2001,contributing to a $158 millionincrease in cash provided byoperations
The Scotts Company | producer and marketerof products for do-it-yourself lawn care, gardening,
and professional horticulture
Problem Solution Result
Scotts
8/13/2019 Datastage Realtime Projects5
25/32
8/13/2019 Datastage Realtime Projects5
26/32
8/13/2019 Datastage Realtime Projects5
27/32
IBM Software Group | WebSphere software
27
Use IBM WebSphere DataIntegration Suite to prepare andload data into SAP R/3 in thecorrect format, automaticallyusing the correct SAP formatfor loading; use IBMWebSphere QualityStage andWebSphere DataStage to loadbetter quality data into R/3.
Fast consolidation andharmonization of data from legacysystems into SAP R/3 and theGlobal Part Master Data.
Enhanced ability to viewconsolidated demand for Part andenables better inventorymanagement.
Example: a 20%-30% reduction induplicated parts is worth 10M
Outokumpu | a Finnish dynamicmetals and technology group with a
clear target of becoming the numberone in stainless steel, annual net salesover 6 billion and 19,000 employeesin 40 countries
Outokumpu
Problem Solution Result
Needed a way to get visibilityto parts information anddemand across multiplemanufacturing plants to better
manage inventory costs andcentralize purchasing, butparts information was lockedin several differentincompatible systems andcontained much duplicatedata.
8/13/2019 Datastage Realtime Projects5
28/32
IBM Software Group | WebSphere software
28
Rich Products | At $1.5 billion in revenue,one of the worlds premier family-ownedfood companies .
To provide faster customer service andincrease supply and demand chainvisibility, Rich wanted to implement anavailable-to-promise application tocomplement their manufacturingplanning application and other legacyand EDI sources. This requiredintegrating 40 interfaces to support amessage volume of over 250,000transactions per week.
Avoided hand-codingby deploying IBMWebSphereDataStage TX as a
centralized hub forSAP interfacing toand from IDocs fordata from EDIsources, i2 and QADsystems.
Justified the project based onanticipated maintenance savings,but also realized reduced effortand faster project development
than originally expected. Added flexibility that will enableRich Products to roll out moreSAP applications, move off themainframe and adopt e-businessapplications in the future.
Problem Solution Result
Rich Products
8/13/2019 Datastage Realtime Projects5
29/32
IBM Software Group | WebSphere software
29
Customers needed to make thechange from Lira to Euro at over42,000 taget destinations. The goalwas to aggregrate, integrate andmanage this data change using aunified set of platforms. The technicalchallenge was to gather data fromSAP R/3 and load them into MicrosoftSQL or SAP BW depending on thegiven project and need.
Implemented IBMWebSphere DataStage Packfor SAP R/3 andWebSphere DataStage Packfor SAP BW. The solutionenabled extraction andmanagement of native datafrom SAP R/3 and theloading into SAP BW ofdata from diverse andheterogenous sources.
Poste Italiane was able tooptimise and increase thespeed and efficiency of theprocess of introduction ofEuro values across 42,000distributors and offices on theitalian national territory.
Problem Solution Result
Posteitaliane | the leading mailing serviceprovider in Italy. The company also offers FinancialServices with a network of 14.000 post offices.
.
Poste Italiane
8/13/2019 Datastage Realtime Projects5
30/32
IBM Software Group | WebSphere software
30
Needed to increase theefficiency of internallogistical processes byrationalizing the data flowprocess to and within theinternal SAP BWenvironment and otherapplications (AmosSystem).
Implemented IBMWebSphere DataStageand the WebSphereDataStage Pack for SAPBW to provide a unifiedview of heterogeneouslysourced data.
Saipem was able toachieve its objective ofunifiying and quicklymanaging the processesof data flow within theorganisation.
Problem Solution Result
Saipem | providing services to the oil and gas
industry - as the sole contractor to cover all phasesof the upstream operational cycle: drilling, oil and gasproduction, and transportation to shore.
.
SAIPEM
8/13/2019 Datastage Realtime Projects5
31/32
8/13/2019 Datastage Realtime Projects5
32/32
IBM Software Group
2005 IBM Corporation
Thank You
Stphane Heckel IBM Information Integration Solutionsstephane.heckel@fr.ibm.com