+ All Categories
Home > Documents > Enterprise Data Warehousing With SAP BW - Overview

Enterprise Data Warehousing With SAP BW - Overview

Date post: 27-Nov-2015
Category:
Upload: shahulhameed3681
View: 41 times
Download: 3 times
Share this document with a friend
Description:
Enterprise Data Warehousing With SAP BW - Overview
Popular Tags:
49
Enterprise Data Warehousing with SAP BW – An Overview White Paper Version 2.0 August 18th, 2003 [email protected] SAP (SAP AG and SAP America, Inc.) assumes no responsibility for errors or omissions in these materials. These materials are provided “as is” without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party web pages.
Transcript
Page 1: Enterprise Data Warehousing With SAP BW - Overview

Enterprise Data Warehousing with SAP BW – An Overview

White Paper Version 2.0

August 18th, 2003

[email protected]

SAP (SAP AG and SAP America, Inc.) assumes no responsibility for errors or omissions in these materials.

These materials are provided “as is” without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.

SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials.

SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party web pages.

Page 2: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Table of Contents

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW....................................... 1

1 INTRODUCTION........................................................................................................................... 1 1.1 SUPPORTED SOFTWARE VERSIONS ........................................................................................... 1 1.2 THIS WHITE PAPER .................................................................................................................. 1

2 AIMS OF THE WHITE PAPER ..................................................................................................... 1

3 MOTIVATION................................................................................................................................ 2 3.1 THE MARKET............................................................................................................................ 2 3.2 WHY DO YOU NEED A CORPORATE BW STRATEGY?................................................................. 3

4 ENTERPRISE DATA WAREHOUSING WITH SAP BW.............................................................. 5

5 BW ENTERPRISE ARCHITECTURE........................................................................................... 6 5.1 ASPECTS OF A DATA WAREHOUSE ARCHITECTURE .................................................................... 6 5.2 DATA STORE ARCHITECTURE - BW DATA LAYER ..................................................................... 10

5.2.1 BW Architected Data Mart Layer .................................................................................. 11 5.2.2 BW Data Warehouse Layer .......................................................................................... 14 5.2.3 BW Extraction and Staging........................................................................................... 19 5.2.4 BW Support for Operational / Real Time Reporting ..................................................... 20

5.3 DATA ARCHITECTURE - BW DATA MODEL ............................................................................... 24 5.3.1 Operative Data Models and Data Warehouse Data Model .......................................... 24 5.3.2 The BW Data Model ..................................................................................................... 25

5.4 TOPOLOGIES – BW LANDSCAPES ........................................................................................... 29 5.4.1 Consistency in a distributed BW Landscape ................................................................ 30 5.4.2 BW Landscapes und Technical Parameters ................................................................ 31 5.4.3 Inside-Out Landscape Architecture - BW as Central Enterprise Data Warehouse...... 32 5.4.4 Outside-In Landscape Architecture - Decentralized BW Data Warehouses................ 34

6 CENTRAL BW APPLICATION DEVELOPMENT...................................................................... 38

7 DATA AND DATA MODEL INTEGRATION .............................................................................. 40 7.1 CHALLENGES POSED BY DATA INTEGRATION............................................................................ 40 7.2 DATA MODEL INTEGRATION..................................................................................................... 42 7.3 THE ROLE OF SAP MASTER DATA MANAGEMENTS (MDM) ...................................................... 43

8 SUMMARY.................................................................................................................................. 45

TABLE OF ILLUSTRATIONS ………………………………………………………………………………43

2003 SAP AG AND SAP AMERICA, INC. CONTENT

Page 3: Enterprise Data Warehousing With SAP BW - Overview

BW Enterprise Data Warehousing – Overview V2.0

1 Introduction

1.1 Supported Software Versions This document is relevant for BW Versions 3.x.

1.2 This White Paper The subject of this document is very complex. Therefore, we do not claim to cover all the various aspects of this topic, or to offer an exhaustive description of the areas discussed.

To see updated versions of this document, visit the SAP Service Marketplace regularly.

2 Aims of the White Paper This White Paper offers an overview of the implementation of SAP Business Information Warehouse (SAP BW) from a corporate perspective.

It considers the following issues:

• Architectural aspects: data management, data models, topologies etc. on a conceptual level

• Integration aspects: supporting corporate strategy as far as the harmonization of master data is concerned

• Solution development: the efficient, consistent development of BW-based applications.

Enterprises differ in terms of their organization and their business. This implies that there can be no standard solution for a corporate BW implementation strategy. However there are basic truths that always have to be considered, and patterns that arise within specific business types. Above all, this White Paper will describe fundamental aspects of a corporate-wide BW implementation.

This White Paper is no substitute for a business-specific consultation on BW architecture.

This White Paper focuses on the general principles involved in a strategic corporate BW implementation and not on exceptions to this. It follows the 80-20 rule.

The White Paper avoids details wherever possible, so as not to lose sight of the overall context.

Knowledge about BW is useful to read this White Paper but no prerequisite for it.

2003 SAP AG AND SAP AMERICA, INC. 1

Page 4: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

3 Motivation 3.1 The Market At the time of writing, there are more than 6000 BW installations within the market.

The fundamental advantages of BW have led more and more customers to center their corporate data warehousing strategy around BW. Customers frequently stress the end-to-end conception of the BW in this context. In comparison to fragmented technologies, the integrated metadata concept, from data integration right through to analysis, leads to lower total costs for the project overall.

The BW has moved away from isolated instances to incorporate an architecturally-based view. In this way, the BW has come to set new standards of quality in terms of its positioning within an enterprise.

The following graphic shows various rudiments of this strategy:

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 3 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 3

Evolution of SAP BW

Strategic Corporate BW Strategic Corporate BW ImplementationsImplementations

Isolated BW Isolated BW ImplementationsImplementations

BW1

BW2

BW3

BWn

BW1

BW2

BWn

GlobalBW

Others

‘Headquarter’ ‘Headquarter’ ReportingReportingScenarioScenario

‘Local’BW

‘Local’BW

‘Local’BW

GlobalBW

Architected Architected BW LandscapeBW Landscape

ScenarioScenario

EnterpriseBW

BW EnterpriseBW EnterpriseDataData

WarehouseWarehouseScenarioScenario

Issues:SynergyIntegrationConsistency

SpokeBW

SpokeBW

Illustration 1: Evolution of the SAP BW

This leads to one main motivation for this paper. It aims to support the evolution-process of BW offering an overview of the essential criteria for corporate architected BW implementations for customers and consultants.

On the other hand, there is still a large number of customers who do not have a corporate data warehouse strategy and who have implemented BW in a more isolated and restricted way.

Thus, another motive is the hope of making people aware of the prospects and benefits of a business-wide, architecturally-based BW implementation.

2003 SAP AG AND SAP AMERICA, INC. 2

Page 5: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

3.2 Why Do You Need A Corporate BW Strategy? There are many training guides, ‘enhanced documentations’, ‘ASAP Accelerators’ and ‘How to Guides’ available that offer support to BW users. These documents all focus on offering support with concrete challenges that arise during a project.

This is obviously important and necessary. However there are also short-comings as projects are always part of an incremental BW implementation:

From project to project, project goals are modeled and realized in BW. These goals are typically reporting- and analysis-driven, and the overall success of the project is then often measured by decision-makers in terms of how far these reporting and analysis demands are met. As such, the focus is at solution level or using a data warehouse term the focus is at data mart level.

Less attention is accorded to decisive factors for the mid- and long-term success of an investment in BW:

Redundancy has to be controlled in all areas!

Business-wide guidelines rarely exist regarding:

The persistent BW data stores and their design

The BW Data Warehouse data model and its management

The BW landscape (the role of BW instances and which conditions are valid for new instances).

Aspects of data integration are often neglected too, and applications in various organization units that feature a BW are developed asynchronously and redundantly.

Altogether, from an overall business perspective, you are left with a costly and less efficient procedure that delivers neither consistent, reliable, nor integrated information.

The following graphic clarifies the solution-focus issues:

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 4 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 4

Redundancy and Data Mart (Solution) Focus

Real World

InformationRequirements

(grouped to Scopes)

....

Schema-Modeling

InfoCubes/ODS-Objects

Extraction

Sources

Business Rules

Transformation

BW Application Project Team

Successful Data Warehousing means ‘Controlled Redundancy’ !Successful Data Warehousing means ‘Controlled Redundancy’ !Successful Data Warehousing means ‘Controlled Redundancy’ !

Impacts of non-existing corporate BW guidelines:

• Redundant Data• Redundant Extraction• Redundant Transformation• Redundant Business Rules• Redundant Masterdata•......

Illustration 2: Redundancy and Solution focus

2003 SAP AG AND SAP AMERICA, INC. 3

Page 6: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

The next graphic clarifies the multiple BW landscape issues:

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 5 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 5

Redundancy and Multiple BW Landscape

Sub-Division AR/3

Global

Sub-Division B R/3

only Germany

Sales EuropeR/3

BW Local

Global

HQBW

SubDiv BW

MDMS

BWS-America

BWAsia

BWN-America

BWSales Europe

Source Systems

....................DivR/3

DivR/3

DivR/3

12 systemsworldwide

Successful Data Warehousing means ‘Controlled Redundancy’ !Successful Data Warehousing means ‘Controlled Redundancy’ !Successful Data Warehousing means ‘Controlled Redundancy’ !

A Real Life

BW Landscape ExampleImpacts of non-existing corporate BW guidelines:

•Redundant Data Flows•Redundant Extraction•Redundant Development•Redundant Models•......

Illustration 3: Redundancy and multiple BW landscape

The more complex an enterprise’s structures, the more important it is to have corporate-wide guidelines and recommendations for the implementation of BW. They guarantee long-term consistency and reduced costs.

2003 SAP AG AND SAP AMERICA, INC. 4

Page 7: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

4 Enterprise Data Warehousing with SAP BW When we talk about enterprise data warehousing it is first necessary to ask what we understand by this term: Enterprise data warehousing is the sum of all the decisions that have to be taken at corporate level in order to obtain a durable solution that adheres to business-wide, specified guidelines, and fulfills all requirements for integrated, consistent and structured information.

These decisions are mirrored in the enterprise data warehouse architecture.

In this chapter we will consider the aspects of such an architecture so that we can then look more closely at how BW supports an architecturally-based, consistent, corporate data warehouse solution.

As well as a BW architecture that ensures consistency, we want to consider in what ways the BW offers support in generating consistent application solutions.

Furthermore, in discussing a corporate BW strategy, we have to include BW in the data integration strategy of the company. This is done in the last chapter.

As with all important decisions, we concentrate on the fundamental aspects, following the 80:20 rule.

2003 SAP AG AND SAP AMERICA, INC. 5

Page 8: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5 BW Enterprise Architecture An architecture can be seen as a system design decision that has a specified duration. In other words, once it has been implemented it is not easy to change. In defining a corporate BW architecture it is clear that the term ‘architecture’ is used in various senses, and that aspects such as ‘where’, ‘what’ and ‘how’ often become confused. This leads to confusion and is dangerous in that essential factors may be considered from a lop-sided perspective, or not at all.

5.1 Aspects of a Data Warehouse Architecture In discussing an optimal BW architecture, an enterprise often focuses too quickly on physical BW instances. Is more than one BW instance needed and if yes, where are they to be institutionalized? How will they work together? These questions concern a BW landscape, or as we prefer to call it, a consistent enterprise BW topology. This means that the second or even third architecture elements are being considered before the first. How can individual BW instances and their tasks be discussed without having first defined how data is consistently handled and saved in such a BW instance? We must start with the BW data store or data layer architecture: The layer architecture defines persistent data stores and their storage schemas from extraction through to the end-user, and refinement processes between the layers. Qualified data status and the respective layers:

• Staging Area - row • Data Warehouse Layer - integrated, granular, not application-specific, historical • Data Mart Layer - integrated, aggregated, application-specific • Operational Data Store (ODS) – integrated, granular, application-specific, close to event

2003 SAP AG AND SAP AMERICA, INC. 6

Page 9: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 15 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 15

Conceptual Layers of Data Warehousing

Infor-mationAccess

Non-SAPSource

SAPSource

PersistentStaging

Area

Operational Data Store

Archiving & Near-Line Storage

Data Warehouse

Archi-tected DataMarts

Illustration 4: The conceptual layer architecture of data warehousing Repeatedly saving raw data in various refinement situations means on the one hand side redundancy. On the other hand side means a consistent layer architecture the only way to control the redundancy of data warehousing, which derives from its very nature. This has to be taken into consideration with the layer data store schemas. However, a suitable layer support is not the only necessary condition for a corporate-wide data warehouse architecture that can secure consistency in the mid- and long-term. This can only be achieved if the operative data models are mapped consistently to the model of the data warehouse layer and the model of the data mart layer. Such a consistent data (model) architecture therefore describes the model of each layer and how the models relate to each other. As such we need a data architecture that guarantees, for example, that operative material terms, as they are found in the respective sales or distribution area, or in materials management, are reproduced consistently in the material subject area. Ideally you would have a single operative enterprise data model and use that to derive the model for each layer. However, a corporate data model of this sort does not normally exist, particularly if legacy systems are being used. This means that often no or only limited guidelines and rules exist how to map the operative models to a single data warehouse layer model and a single data mart layer model because of the difficulties to achieve this. The result is unsynchronized isolated solutions (‘stovepipe solutions’). This is illustrated in the following graphic:

2003 SAP AG AND SAP AMERICA, INC. 7

Page 10: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 95 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 95

Operational Data Models and the Data Warehouse Data Model

Not aligned operational data models Consistent enterprise data model

Inconsistent data warehouse data models ⇒stovepipe solutions

Consistent data warehouse data model ⇒long term success

A data warehouse A data warehouse consistency challengeconsistency challenge

Illustration 4: Operational Data Models and DWH Data Model A missing data model architecture is one reason why corporate data warehouse strategies fail. Only when there is clarity regarding the data layer and data model does it make sense to discuss the corporate-wide topological aspects of a BW architecture. Because topological aspects are influenced by the layer architecture. Two basic topologies are differentiated: • The BW enterprise data warehouse landscape with BW as the central information hub • Landscape based on ‘local’ BWs with a higher level ‘global’ BW

‘Local’BW

‘Local’BW

‘Local’BW

GlobalBW

OutsideOutside--ININBWBW

LandscapeLandscape

Others

‘Local’BW

‘Local’BW

‘Local’BW

GlobalBW

OutsideOutside--ININBWBW

LandscapeLandscape

Others

EnterpriseBW

InsideInside--OutOutBWBW EnterpriseEnterprise

DataDataWarehouseWarehouse Spoke

BW

SpokeBW

Others

SpokeBW

EnterpriseBW

InsideInside--OutOutBWBW EnterpriseEnterprise

DataDataWarehouseWarehouse Spoke

BW

SpokeBW

Others

SpokeBW

Illustration 5: BW basic topologies When determining the BW topology, political, organizational and technical factors play an important role. As a result you cannot really talk about a generally valid optimal BW topology. We summarize that it is necessary to make definitions that are valid enterprise-wide in the following areas:

2003 SAP AG AND SAP AMERICA, INC. 8

Page 11: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

• BW data layer architecture • Which data statuses does it make sense to save persistently? • Which designs are approved for the various data stores?

• BW data model architecture • Each data warehouse layer requires a data model that describes objects and their

relationships • The data models of the various layers have to be derived from each other consistently

• BW topology architecture • Different corporate cultures, geographical, and business diversifications lead to different

data warehouse landscapes.

2003 SAP AG AND SAP AMERICA, INC. 9

Page 12: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5.2 Data Store Architecture - BW Data Layer It is generally accepted that the data store that supports analysis and reporting is to be kept separate from the data store that deals with any sort of data acquisition. Analysis and reporting are handled by data marts. Close to event-level operative reporting is handled by the operational data store. BW supports multidimensional data marts in an extended star schema. This is made up of BW InfoCubes and over-arching BW InfoObject master data dimensions. Operational data stores are supported by ‘flat’ BW ODS objects. Here too BW master data acts as structures that span ODS objects. Data acquisition (processes that secure quality and integration) are referred to as staging processes. The persistent saving of staging process data occurs in staging areas. Data staging stores are realized by the Persistent Staging Area (PSA) Objects and by ODS objects. The granular, integrated result of the staging process reproduces data in optimal condition. This is referred to as the data warehouse layer. BW ODS objects are the central storage objects that build a BW data warehouse layer. This gives us the following general picture of a BW layer architecture:

Finance

Logistic Ope

n D

istri

butio

n

SAP Business Information WarehouseSAP Business Information Warehouse

StagingStagingAreaArea

CleansingCleansingTransformTransform

MasterReference

Data

Data Data WarehouseWarehouse

ODSODS

PrimaryPrimaryData Data

ManagementManagement

Extra

ctio

n /

Ope

n St

agin

g

DataDataAcquisitionAcquisition

Data Data DeliveryDelivery

any

targ

et

Ext

ende

d St

ar S

chem

asE

xten

ded

Star

Sch

emas

Flow Control Flow Control

MetaMeta DataData

any

sour

ce TransactionData

ArchitectedArchitectedData MartsData Marts

FinanceFinance

LogisticLogistic Ope

n D

istri

butio

n

SAP Business Information WarehouseSAP Business Information Warehouse

StagingStagingAreaArea

CleansingCleansingTransformTransform

MasterReference

Data

Data Data WarehouseWarehouse

ODSODS

PrimaryPrimaryData Data

ManagementManagement

Extra

ctio

n /

Ope

n St

agin

gEx

tract

ion

/ O

pen

Stag

ing

DataDataAcquisitionAcquisition

Data Data DeliveryDelivery

any

targ

et

Ext

ende

d St

ar S

chem

asE

xten

ded

Star

Sch

emas

Flow Control Flow Control

MetaMeta DataData

any

sour

ce TransactionData

ArchitectedArchitectedData MartsData Marts

Illustration 5: BW layer architecture - overview

2003 SAP AG AND SAP AMERICA, INC. 10

Page 13: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5.2.1 BW Architected Data Mart Layer InfoCube data marts allow the flexible generation of queries and navigation to integrated, granular or aggregated data. In order to make comparisons possible, an amount of historical data (defined by you) can be retained. InfoCubes are multidimensional data structures (also called star schemas) that organize data in such a way that key figures (or facts or measures) are affixed to characteristics and their attributes, which form the so-called dimensions (BW InfoCube dimensions and master data). To increase the capacity of an InfoCube schema, dimensions are separated into local and global operative parts. Global, because these dimension-parts are only saved once and are conjointly addressed by various InfoCubes.

FACT Table

Material_Dimension_IDSalesOrg_Dimension_IDTime_Dimension_IDCustomer_Dimension_ID

Sales AmountQuantity

InfoCubeInfoCube

Gebiet 1 Gebiet 2 Gebiet 3

Bezirk 1

Gebiet 3a

Bezirk 2

Region 1

Gebiet 4 Gebiet 5

Bezirk 3

Region 2

Gebiet 6

Bezirk 4

Gebiet 7 Gebiet 8

Bezirk 5

Region 3

Vertriebsorganisation

Material Group

Material Hierarchy Table

Material NumberLanguage Code

Material NumberLanguage CodeMaterial Name

Material Text TableMaterial_Dimension_ID

Material NumberMaterial Dimension

Table

Material Master Table

Material NumberMaterial Number

Material Type

MaterialMaterialDimensionDimension

Shared, globalShared, globalPart of

Material Dimension

LocalLocal Part of Material Dimension

BW Extended Star Schema

FACT Table

Material_Dimension_IDSalesOrg_Dimension_IDTime_Dimension_IDCustomer_Dimension_ID

Sales AmountQuantity

Material_Dimension_IDSalesOrg_Dimension_IDTime_Dimension_IDCustomer_Dimension_ID

Sales AmountQuantity

InfoCubeInfoCube

Gebiet 1 Gebiet 2 Gebiet 3

Bezirk 1

Gebiet 3a

Bezirk 2

Region 1

Gebiet 4 Gebiet 5

Bezirk 3

Region 2

Gebiet 6

Bezirk 4

Gebiet 7 Gebiet 8

Bezirk 5

Region 3

Vertriebsorganisation

Material GroupMaterial Group

Material Hierarchy Table

Material NumberLanguage Code

Material NumberLanguage CodeMaterial NameMaterial Name

Material Text TableMaterial_Dimension_ID

Material NumberMaterial Dimension

Table

Material Master Table

Material NumberMaterial Number

Material TypeMaterial Type

MaterialMaterialDimensionDimension

Shared, globalShared, globalPart of

Material Dimension

LocalLocal Part of Material Dimension

BW Extended Star Schema

Illustration 5: The BW extended star schema In this way BW InfoCubes support the concept of ‘conformed dimensions’, which form a sort of ‘glue’ between InfoCubes.

InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster

Horizontal Consistency:Horizontal Consistency:Conformed Structures Conformed Structures

BW InfoObjectsBW InfoObjectsmaster datamaster data

CUSTOMER

MATERIAL

Horizontal Consistency: BW Architected Data Mart Layer

InfoCubeInfoCube InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster

Horizontal Consistency:Horizontal Consistency:Conformed Structures Conformed Structures

BW InfoObjectsBW InfoObjectsmaster datamaster data

CUSTOMER

MATERIAL

Horizontal Consistency: BW Architected Data Mart Layer

InfoCubeInfoCube

Illustration 6: Horizontal consistency in the BW architected data mart layer

2003 SAP AG AND SAP AMERICA, INC. 11

Page 14: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Conformed dimensions essentially support horizontal consistency within the data mart layer. They are part of the BW concept and help prevent the occurrence of isolated solutions (‘stovepipes’). The BW term for these conformed dimensions is ‘master data’. This is one reason to qualify the BW data marts as Architected data marts. The effectiveness of the BW InfoCube schema does not just support consistency but is obviously also essential for mapping end-users’ demands. Above all we have to mention here the ability to map entities from different historical views to the same key figure. By using surrogate keys in the InfoCube and because of the master data/ conformed dimension concept, the BW makes it possible to simultaneously map different views to key figures in one InfoCube.

SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 6 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 6

BW Support of Different Historical Views

Constellation 09/1998Customer Mother-Company

AAA XBBB XCCC Y

Constellation 10/1998

Customer Mother-Company AAA XBBB Y (changed)CCC Y

Customer Date Revenue

AAA 09/1998 100BBB 09/1998 100CCC 09/1998 100BBB 10/1998 100CCC 10/1998 100

Fact Table

Mother-Comp Rev 09/98 Rev 10/98

X 100 0Y 200 200

Report using today‘s (10/98) constellation

Mother-Comp Rev 09/98 Rev 10/98

X 200 100Y 100 100

Report using 09/98 constellation

Mother-Comp Rev 09/98 Rev 10/98

X 200 0Y 100 200

Report showing historical truth

Mother-Comp Rev 09/98 Rev 10/98

X 100 0Y 100 100

Report showing comparable results

BW supported Views in

a single InfoCube

Extended Star Schema

Illustration 7: BW and slowly changing dimensions • 'Today is Yesterday' : Show key figure values, including historical key figure values, from the

currently valid perspectives (current view of dimension attributes) • 'Yesterday is Today' : Show key figure values from user-defined perspectives (key date oriented

view of dimension attributes) • 'Historical Truth' : Show key figures (transactions) in the same state (historicization of dimension

attributes), as at the time of posting – historically correct

Scenarios can also be presented in which only key figure values with unchanged dimension attribute constellations are reported. In order to reduce redundancy and counteract the ‘one report is equal to one InfoCube’ error, virtual multidimensional structures are supported for persistent InfoCubes. These are called BW MultiProviders and BW Queries.

2003 SAP AG AND SAP AMERICA, INC. 12

Page 15: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 24 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 24

Multi-dimensional Schemas in BW

EndEnd--User User Reporting & Analysis needReporting & Analysis need

MultiMulti--dimensional modeldimensional model

BW InfoCube(s)BW InfoCube(s)physical implementation

basic factsnot report specific

BW MultiProviderBW MultiProvidervirtual Structures

basic factsnot report specific

optional

BW BEX QueryBW BEX Queryvirtual Structures

complex KPIsnavigation behaviorreport (type) specific

BW BEX ReportBW BEX Reportend-user requirement

specific

Illustration 8: Persistent and virtual multidimensional structures in BW Despite the capabilities of BW data mart layers, the limitations and dangers of a data mart-focused implementation are also evident: Limited reusability of data and loss of historical conditions

• Data is dealt with in the data mart layer in a scope-specific manner: Aggregation/ manipulation/ overwriting of data is normal. It follows that the usage of these data for other purposes is limited.

• The data mart layer is the direct access layer for the end-user. Performance, therefore, plays a decisive role. For this reason only data that is actually needed is retained in the data mart layer. Saving data ‘just in case’ will quickly lead to a large data volume and impact negatively on performance. Overwriting old images, as normally happens in the conformed dimensions (master data), is therefore valid and indeed desired. However, this means that history is lost.

Redundant data storage (the same data is held in various InfoCubes and often in different degrees of granularity)

Redundant data acquisition (staging) Redundant data extraction Limited ability to trace data.

Data staging is required by the BW layer architecture here. Factors arising from contact with the data mart data model that threaten consistency are discussed in the chapter on data model architecture.

2003 SAP AG AND SAP AMERICA, INC. 13

Page 16: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5.2.2 BW Data Warehouse Layer In order to overcome the limitations and dangers of a data mart-focused BW implementation, the corporate BW architecture has to answer demands such as:

„extract once deploy many” “single point of truth”

Furthermore, the contradiction between quite sensibly reducing information in the data mart layer if it is not currently required by the end-user, and the claims the BW makes to flexibility, has to be resolved. The BW also has to be able to respond quickly to new requests. The answer is to build a separate layer that saves the integrated results of the data transformation and data quality-ensuring processes in a consistent, granular and historical form: The BW data warehouse layer. Ideally this is built centrally and extends business-wide. We can then talk about a BW enterprise data warehouse (layer).

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 41 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 41

A Matter of Consistency and Reliability –The BW Data Warehouse (Layer)

Finance

Logistic Ope

n D

istri

butio

n

SAP Business Information WarehouseSAP Business Information Warehouse

StagingStagingAreaArea

CleansingCleansingTransformTransform

MasterReference

Data

Data Data WarehouseWarehouse

ODSODS

PrimaryPrimaryData Data

ManagementManagement

Extra

ctio

n /

Ope

n St

agin

g

DataDataAcquisitionAcquisition

Data Data DeliveryDelivery

any

targ

et

Ext

ende

d St

ar S

chem

asE

xten

ded

Star

Sch

emas

Flow Control Flow Control

MetaMeta DataData

any

sour

ce TransactionData

ArchitectedArchitectedData MartsData Marts

Illustration 9: SAP BW data warehouse layer The following description of a BW data warehouse layer is based on William H. Inmon’s definition: A BW data warehouse layer offers:

o Subject-oriented, integrated data o Original data that will not be changed in relation to the goals of a particular project

(‘unflavored data’) o Granular data

Non-aggregated data o Historical data (that cannot be overwritten or deleted)

2003 SAP AG AND SAP AMERICA, INC. 14

Page 17: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

o A ‘single point of truth’ The BW data warehouse layer is the only source for BW architected data marts. (“extract once – deploy many”).

In this way a BW data warehouse offers the following benefits: I. Flexibility

o New InfoCube data mart solutions that include history and new extraction can be built in real-time

o The BW can respond quickly to unpredictable, one-off user requests, without having to build persistent structures (Although the BW data warehouse layer is not to be misused for standard analysis as it does not possess the optimal structures for this).

II. Reliability and Reproducibility o The BW data warehouse layer is centrally administrated and not subjected to arbitrary

projects. The data warehouse layer is the common source for all the data provided in the data marts.

III. Complete history IV. Consistency

o Extraction, staging and integration are managed centrally o Data is extracted only once o Redundant staging processes are avoided.

A BW data warehouse (layer) is an enterprise’s memory, a repository for information for the whole enterprise. In addition to offering horizontal consistency within the BW data mart layer, the BW data warehouse layer offers a sort of vertical consistency that the data mart layer alone is not able to guarantee.

2003 SAP AG AND SAP AMERICA, INC. 15

Page 18: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Vertical Consistency: BW Data Warehouse Layer

Source 1 Source 2 Source 3 Source 4 Source 5

Vertical Consistency:Vertical Consistency:Single Point of Truth Single Point of Truth

BW DWH Layer ODSBW DWH Layer ODS--ObjectsObjects

Transaction / Master Data

InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster

Horizontal Consistency:Horizontal Consistency:Conformed Structures Conformed Structures

BW InfoObjectsBW InfoObjectsmaster datamaster data

CUSTOMER

MATERIAL

InfoCubeInfoCube

Vertical Consistency: BW Data Warehouse Layer

Source 1 Source 2 Source 3 Source 4 Source 5

Vertical Consistency:Vertical Consistency:Single Point of Truth Single Point of Truth

BW DWH Layer ODSBW DWH Layer ODS--ObjectsObjects

Transaction / Master Data

InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster

Horizontal Consistency:Horizontal Consistency:Conformed Structures Conformed Structures

BW InfoObjectsBW InfoObjectsmaster datamaster data

CUSTOMER

MATERIAL

InfoCubeInfoCube InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster

Horizontal Consistency:Horizontal Consistency:Conformed Structures Conformed Structures

BW InfoObjectsBW InfoObjectsmaster datamaster data

CUSTOMER

MATERIAL

InfoCubeInfoCube

Illustration 10: BW data warehouse layer and vertical consistency The question of how to organize data warehouse layers often causes arguments. Extreme positions on this range from 3NF to de-normalized star schemas. We do not want to give a definite answer here, but suggest taking a pragmatic approach. As the data warehouse layer forms the basis for supplying the data marts, data should be saved so that it can be easily transported into conformed dimensions and InfoCubes. This means complex join operations for normalized data warehouse objects on the way to the data marts should be avoided. It is therefore, for example, legitimate to store position data and header information for a sales order together in the data warehouse layer.

To the question, "what kind of database design is appropriate for a data warehouse?", W.H. Inmon offers the following answer: ‘For a data warehouse, it is always better to have a lightly normalized design. Star schemas fit elsewhere (s. Data Marts t. author). The data warehouse design is not perfectly normalized because a few alterations are made for common usage, such as creating arrays of data when data is always used together, creating a small and selective amount of redundancy where appropriate, merging tables that are commonly used together, and so forth. At the end of the day, a data warehouse design is distinctively normalized. This answer presumes that you know the difference between a data warehouse and a data mart.

Restatement or regeneration of some of these higher levels of information requires that we keep available a detailed historical data store. This should be ‘clean’ (free of errors and conformed to the Data Warehouse master data model) to avoid unnecessary reprocessing of source data.’ (W. H. Inmon, Architecture 2002)

2003 SAP AG AND SAP AMERICA, INC. 16

Page 19: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

The BW does not offer a data store specific for the data warehouse layer. Transaction data is saved in flat ODS objects, master data, reference data, in InfoObjects or ODS objects. Handling history is supported in BW by staging processes (transfer/ update rules) and staging ODS objects.

Customer CustomerGroupA YB Z

Customer Source Activ From Activ To CustomerGroup

A S1 20020101 20020331 XB S1 20020101 99991231 ZA S1 20020401 99991231 Y

Customer CustomerGroupA XB ZA Y

Data MartMaster Data Table

DWH Object

Sourcesystem S11st Load: 20020101

after 2nd Load: 20020401

after 2nd Load: 20020401

2nd Load: 20020401

Illustration 11: Example - BW data warehouse: Historization of master data With master data, the source system normally does not deliver versioned or historical data images. Furthermore full-loads are often provided for master data. If the source system does not provide any historization, the BW staging processes must provide this information. The management of activity time slots is supported by BW for data warehouse InfoObjects automatically. For data warehouse ODS objects activity time slots must be maintained during staging. It is also the task of the BW with ‘full loads’ to determine the changed records. This is done using special staging ODS objects. These tasks are omitted when SAP Master Data Management (SAP MDM) historicized Delta images are provided. Handling transaction data is easier, as long as the transaction is not to be posted again. With changeable transactions a simple time stamp is often not enough to prevent it from being overwritten.

2003 SAP AG AND SAP AMERICA, INC. 17

Page 20: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Customer Dimension Material DimensionA 4712

4713

Customer Material TimeAmount

Company Currency

A 4712 200211 100A 4713 200211 150

Time Dimension200211

Customer Time DocNo Pos MaterialAmount Local

Currency

Amount Company Currency

A 20021107-10am 1 10 4711 100 50A 20021107-3pm 1 10 4712 200 100A 20021107-4pm 2 10 4713 300 150

Customer Time DocNo Pos MaterialAmount Local

CurrencyA 20021107-10am 1 10 4711 100 New bookingA 20021107-3pm 1 10 4712 200 Correction bookingA 20021107-4pm 2 10 4713 300 New booking

Sourcesystem

BW Data Warehouse Layer

daily

weekly dailymonthly

other InfoCubeother InfoCube

BW Architected Data Mart Layer

InfoCube

Illustration 12: Example - BW data warehouse: Transaction data The quantity of data in a BW data warehouse layer mounts quickly. The BW/ SAP Archiving Development Kid (ADK) provides the prerequisites for the data warehouse layer to cope with the demand, without the memory space used escalating exorbitantly. If a large data volume is to be made available online, the BW supports the usage of nearline storage.

2003 SAP AG AND SAP AMERICA, INC. 18

Page 21: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5.2.3 BW Extraction and Staging In order to guarantee the quality of data in the BW data warehouse layer, corporate-wide rules with respect to extraction and staging are important. First and foremost, these should avoid redundant extraction and transformation. Using BW as the basis for enterprise data warehousing requires that all data sources can be addressed. For this reason the BW offers a wide spectrum of support for extraction:

• Predefined, enhanceable extractors for practically all SAP application data • The possibility to generate extractors for customer-specific SAP applications like COPA • Generic extractors for customer-specific enhancements of mySAP applications • Direct extraction from databases supported by SAP (BW DB-Connect Feature), based on table- or view-definitions • Flat file extraction • Integration with Ascential’s Datastage • BW 3.5 it is scheduled to support extraction from sources that can be addressed using JDBC or ODBO .

Most extractors for SAP application transaction data are delta-enabled. This means that transactions can be written to a delta queue at the time of posting. They are then extracted from this delta queue into the BW. It is evident that as well as delta-enabling, important design possibilities for BW scenarios are also derived from this, as master data information like price, for example, can also be saved at the time of posting. Extracted data is the first status at which it is relevant to save data persistently. This raw data is saved in the persistent staging area (PSA) on a 1:1 basis. The PSA forms a backup for newly designed staging processes.

StagingStagingArea:Area:PSAPSA

CleansingCleansingTransformTransform

BW Data BW Data WarehouseWarehouse

Extr

actio

n /

Ope

n St

agin

g

BW ODSBW ODSany

sour

ce

StagingStagingArea:Area:PSAPSA

CleansingCleansingTransformTransform

BW Data BW Data WarehouseWarehouse

Extr

actio

n /

Ope

n St

agin

gEx

trac

tion

/ O

pen

Stag

ing

BW ODSBW ODSany

sour

ce

Illustration 13: BW staging Whether and which data statuses between raw and integrated data is saved persistently has to be decided on a case by case basis. This decision depends on the quality of data and the outlay for potential data harmonization. BW ODS objects are used as data stores.

2003 SAP AG AND SAP AMERICA, INC. 19

Page 22: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5.2.4 BW Support for Operational / Real Time Reporting Staging, data warehouse and data mart layers constitute ‘classic’ data warehousing. ‘Classic’ in the sense that data is subjected to an exact and predefined procedure in order to guarantee the quality and integrity of reporting and analysis. ‘Classic data warehousing’ also means the ability to take the burden away from the operative systems.

The classic data warehousing approach:Dedicated load and staging processesHigh qualityOptimized retrieval structuresReduce workload on OLTP systems

DataMart

Finance

ArchitectedArchitectedData MartsData Marts

DataMart

Logistic Tact

ical

/ St

rate

gic

MasterReference

Data

TransactionData

Data Data WarehouseWarehouse

Extra

ctio

n /

Ope

n St

agin

g

DataMart

Finance

DataMart

Finance

ArchitectedArchitectedData MartsData Marts

DataMart

Logistic

DataMart

Logistic Tact

ical

/ St

rate

gic

MasterReference

Data

TransactionData

Data Data WarehouseWarehouse

MasterReference

Data

TransactionData

Data Data WarehouseWarehouse

Extra

ctio

n /

Ope

n St

agin

gEx

tract

ion

/ O

pen

Stag

ing

Illustration 14: Classic data warehousing Alongside the classic layers (staging, data warehouse and data mart) an additional data area is often postulated: The Operational Data Store (ODS). The operational data store supports operative reporting, whereas data marts are data structures for tactical and strategic reporting and analysis. In practice, if operative reporting does not demand real-time data, the physical division between the data mart and the ODS becomes artificial. Mostly loading data two or three times daily is sufficient for operative reporting. Updating data on a daily basis may even be enough. The greater the ‘latency’ (the time difference between the operative event and the acquisition of this event for reporting purposes), the more sensible it is to store granular data in data mart InfoCubes too, and to use this both for tactical analysis and operative reporting. Flat BW ODS objects are better suited to purely listing data. In this way, most BW implementations already support operative reporting and so relieve the pressure on operative systems.

2003 SAP AG AND SAP AMERICA, INC. 20

Page 23: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

In most of the cases operational reporting needs can be supportedin BW using the classic data warehouse process

DataMart

Finance

ArchitectedArchitectedData Marts/ Data Marts/ ODSODS--ObjectsObjects

DataMart

Logistic

Tact

ical

/ St

rate

gic

/ Ope

ratio

nal

MasterReference

DataTransaction

Data

Data Data WarehouseWarehouse

Extra

ctio

n /

Ope

n St

agin

g

ODS-ObjectSales Orders

‘..most of the cases..’: classical data warehousing has its limits if we want to provide event-level or close to event-level operational reporting

(near real time reporting) and/ orif we are confronted with huge data volumes

In most of the cases operational reporting needs can be supportedin BW using the classic data warehouse process

DataMart

Finance

ArchitectedArchitectedData Marts/ Data Marts/ ODSODS--ObjectsObjects

DataMart

Logistic

Tact

ical

/ St

rate

gic

/ Ope

ratio

nal

MasterReference

DataTransaction

Data

Data Data WarehouseWarehouse

Extra

ctio

n /

Ope

n St

agin

g

ODS-ObjectSales Orders

DataMart

Finance

DataMart

Finance

ArchitectedArchitectedData Marts/ Data Marts/ ODSODS--ObjectsObjects

DataMart

Logistic

DataMart

Logistic

Tact

ical

/ St

rate

gic

/ Ope

ratio

nal

MasterReference

DataTransaction

Data

Data Data WarehouseWarehouse

MasterReference

DataTransaction

Data

Data Data WarehouseWarehouse

Extra

ctio

n /

Ope

n St

agin

gEx

tract

ion

/ O

pen

Stag

ing

ODS-ObjectSales Orders

‘..most of the cases..’: classical data warehousing has its limits if we want to provide event-level or close to event-level operational reporting

(near real time reporting) and/ orif we are confronted with huge data volumes

Illustration 15: Classic data warehousing supports operatives reporting BW supports this tactical analysis and operational reporting on the same InfoCube through pre-calculated aggregates, ‘aggregation awareness’ in the OLAP Engine, and ‘query catching’. However it is to be pointed out that the utilization profile of data for operative usage is different in regard to both its history and its granularity to data for analytical usage. With large volumes of transaction data this has to be accounted for in the schema modeling. With delayed operative reporting, which follows the path of ‘classic data warehousing’, there is no need to define a dedicated BW operational data store. You do then have to be aware of the possible consequences of basing analysis and operative reporting on the same structures. One the other hand, if data is requested close to the event time in the operative system, the ‘classic’ pull principle of the BW data warehousing boundaries are set. In this case we talk about ‘real-time’ or ‘near real time operational reporting’. The data extracted (in real time) is written straight to ODS objects, without going through the data warehouse layer. These objects form the BW operational data store.

2003 SAP AG AND SAP AMERICA, INC. 21

Page 24: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 83 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 83

Near Real Time Data Warehousing (Apollo) –BW Operational Data Store

Finance

Logistic Ope

n D

istri

butio

n

SAP Business Information WarehouseSAP Business Information Warehouse

StagingStagingAreaArea

CleansingCleansingTransformTransform

MasterReference

Data

Data Data WarehouseWarehouse

ODSODS

PrimaryPrimaryData Data

ManagementManagement

Extra

ctio

n /

Ope

n St

agin

g

DataDataAcquisitionAcquisition

Data Data DeliveryDelivery

any

targ

et

Ext

ende

d St

ar S

chem

asE

xten

ded

Star

Sch

emas

Flow Control Flow Control

MetaMeta DataData

any

sour

ce TransactionData

ArchitectedArchitectedData MartsData Marts

Illustration 16: BW operational data store: Near real time data warehousing Data from an operational data store is not to be processed directly in the data mart layer for reasons of data consistency. If data from an operational data store is needed in a data mart scenario, you should always go through the (enterprise) data warehouse. To summarize, the functionalities planned and offered by BW to support real time reporting scenarios are: BW Remote InfoCubes – external InfoCube fact tables

BW allows you to define InfoCubes whose fact tables are themselves not persistent in the BW. The data is then read by an extractor during the query runtime and subjected to ‘normal’ InfoSource treatment as far as transfer and update rules are concerned. It can then be made available to the OLAP processor. All data sources that can be addressed by BW can make external fact tables available in this way.

BW Virtual InfoCubes – a program that behaves like a fact table BW ‘near real time data warehousing’ (next main BW release)

With the next main BW release it will be possible to ‘push’ data into BW ODS objects directly after either its emergence or the change event.

Direct reporting to operative structures (next main BW release) Reporting directly to operative structures will also be an option if integration with other data is not required. However the burden produced (lack of data integration and data quality controls) means tight limitations are set for this option.

2003 SAP AG AND SAP AMERICA, INC. 22

Page 25: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 89 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 89

BW and Operational/ Real Time Reporting

Key-Questions:

Data Latency ?Data Integration ?Workload on OLTP Systems ?

Ul

BWBW

OLTP

BW Data IntegrationBW Data Acquisition

BW Near real timedata warehousing

BW ‘classic’data warehousing

BW remoteInfoCube

BW virtualInfoCube

ApolloApollo

Adaptors

JDBC ODBO XMLA SAP Query

Illustration 17: BW and operational / real time reporting .

2003 SAP AG AND SAP AMERICA, INC. 23

Page 26: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5.3 Data Architecture - BW Data Model The BW layer architecture describes the life-cycle of data from the event through to the retrieval-relevant data mart or flat ODS object structure. One thing is intuitively clear: persistent stores and their storage schemata alone are not sufficient to ensure ‘controlled redundancy’. This requires a data warehouse data model.

5.3.1 Operative Data Models and Data Warehouse Data Model The operative application’s entity relationship model serves as the basis for a data warehouse data model. Ideal examples of an business-wide, operational data model are scarce. The challenge then is to derive a consistent data warehouse data model from a multitude of operative models. The following graphic illustrates this:

Not aligned operational data models Consistent enterprise data model

Inconsistent data warehouse data models ⇒stovepipe solutions

Consistent data warehouse data model ⇒long term success

A data warehouse A data warehouse consistency challengeconsistency challenge

Not aligned operational data models Consistent enterprise data model

Inconsistent data warehouse data models ⇒stovepipe solutions

Consistent data warehouse data model ⇒long term success

A data warehouse A data warehouse consistency challengeconsistency challenge

Illustration 18: Operational data model and data warehouse data model Generating a business-wide data warehouse data model based on operative data models, which are not harmonized, is a complex and political procedure. This is why the data warehouse data model architecture is often neglected. The result is isolated applications or ‘stovepipe’ data marts that are not integrated. In this regard BW customers finds themselves in a good position. BW offers an extensible data warehouse data model that represents the operative models of the SAP applications and a large number of industry-specific solutions consistently as part of the BW Business Content. Data models also exist for specific applications from the competitive environment (Oracle Financials, for example). BW customers with a high operative degree of coverage from SAP applications can enjoy being in the situation illustrated in the following graphic:

2003 SAP AG AND SAP AMERICA, INC. 24

Page 27: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

N o t a lig n e d o p e ra tio n a l

d a ta m o d e ls S A P e n te rp ris e d a ta m o d e l

B W C o n s is te n t d a ta w a re h o u se d a ta m o d e l fo rS A P A p p lic a tio n s a n d o th e rs ⇒

lo n g te rm su cce ss

A d a ta w a re h o u s e A d a ta w a re h o u s e c o n s is te n c y c h a lle n g ec o n s is te n c y ch a lle n ge

N o t a lig n e d o p e ra tio n a l d a ta m o d e ls S A P e n te rp ris e d a ta m o d e l

B W C o n s is te n t d a ta w a re h o u se d a ta m o d e l fo rS A P A p p lic a tio n s a n d o th e rs ⇒

lo n g te rm su cce ss

A d a ta w a re h o u s e A d a ta w a re h o u s e c o n s is te n c y c h a lle n g ec o n s is te n c y ch a lle n ge

Illustration 19: Data warehouse data models and the situation for BW customers The BW Business Content data model watches over the incremental generation of InfoCube solutions and ensures that they match, often without the customer even having to be aware of this. This is the true value of BW Business Content from an architecture point of view, and one reason for the success of BW.

5.3.2 The BW Data Model Operative entities and attributes correspond to BW InfoObjects.

Enterprise Data Model:e.g. mySAP Component

Object Models

0MATERIAL

Customer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

0MATTYPE

0MATGR

BW Data Model0CUSTOMER

0DOCNO

0SALESPERS

0AMOUNT

0SALESORG

Entities, Attributes ->BW InfoObjects

operative entities and attributes ⇒ BW InfoObjects

Enterprise Data Model:e.g. mySAP Component

Object Models

0MATERIAL

Customer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

CustomerCustomer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

0MATTYPE

0MATGR

BW Data Model0CUSTOMER

0DOCNO

0SALESPERS

0AMOUNT

0SALESORG

Entities, Attributes ->BW InfoObjects

operative entities and attributes ⇒ BW InfoObjects

Illustration 20: Enterprise data model and enhanceable BW data model: InfoObjects

2003 SAP AG AND SAP AMERICA, INC. 25

Page 28: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Alongside technical properties like format and length, InfoObjects carry data warehousing-relevant properties:

• Aggregation behavior (key figures) • Standard displays (key figures) • Textual description • Language-dependant descriptions • External hierarchies • …

Clustering operative entities and attributes to subject areas takes place through InfoSources. Master data and transaction data InfoSources are differentiated:

Enterprise Data Model:e.g. mySAP Component

Object Models

Enterprise Data Model:e.g. mySAP Component

Object Models

BWData Model defined by

Business Content

BWData Model defined by

Business Content

Customer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

CustomerCustomer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

BCT InfoSources ≡Subject Areas

Master Data Transaction Data

0MATERIAL 0MATGR 0MATTYPE0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT

Clustering operative entities and attributes ⇒ Data Warehouse subject-areas⇒BW InfoSources built of InfoObjects

Illustration 21: BW data model: InfoSources and subject areas InfoSources always represent a set of InfoObjects and describe relations between these. InfoSources are the central bridge between the operative and the BW data model. In the operative system they correspond in form to a DataSource that, in turn, represents an

extractor’s result structure. In the BW data mart layer an InfoSource either defines a conformed dimension (master data

InfoSources), or a relation between conformed dimensions (transaction data InfoSources), in other words, the basis for InfoCube schemata.

The following graphic illustrates the role of BW InfoSources with regard to the data mart layer data model:

2003 SAP AG AND SAP AMERICA, INC. 26

Page 29: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Enterprise Data Model:e.g. mySAP Component

Object Models

Enterprise Data Model:e.g. mySAP Component

Object Models

0MATERIAL0MATGR0MATTYPE......

BW Data Mart Layer Data Model defined by

Business Content

BW Data Mart Layer Data Model defined by

Business Content

Customer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

CustomerCustomer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

BCT InfoSources ≡Subject Areas

0SALESPERS0SALESORG.....

0CUSTOMER00CUSTGR......

0CUSTOMER 0MATERIAL 0AMOUNT

Shared/ Conformed DimensionsShared/ Conformed Dimensions InfoCubes with Local DimensionsInfoCubes with Local Dimensions

BCT Extractors/ DataSource

Master Data Transaction Data

0MATERIAL 0MATGR 0MATTYPE0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT

BW Extended Star SchemasBW Extended Star Schemas

Illustration 22: BW data model: data mart layer data model In the BW (enterprise) data warehouse a master data InfoSource and additional time-slice or

time-stamp attributes defines a data warehouse ODS object, which is able to store different versions of a subject area, which occur over time. Time-slices or time-stamps for master data are normally not offered by the source systems. This information has to be provided in the transfer or update rules during staging. As a rule of thumb a transactional-InfoSources define the structure of a data warehouse ODS object for transactional data (see following graphic):

2003 SAP AG AND SAP AMERICA, INC. 27

Page 30: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Enterprise Data Model:e.g. mySAP Component

Object Models

Enterprise Data Model:e.g. mySAP Component

Object Models

BW Data Warehouse Layer Data Model defined by

Business Content

BW Data Warehouse Layer Data Model defined by

Business Content

Customer Material Sales Person

Sales Transaction

Material group Sales ORGMaterial Type

CustomerCustomer Material Sales Person

Sales Transaction

Material group Sales ORGMaterial Type

BCT InfoSources ≡Subject Areas

BCT Extractors/ DataSource

Master Data Transaction Data

0MATERIAL 0MATGR 0MATTYPE0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT

+ Insert Timestamp/ Activity Time Slice

+ Source

EDW InfoObject/ ODS-Object ODS-Object

If Overwrite:Unique Identifier

+Source

Illustration 23: BW data model: Data warehouse layer data model BW Business Content offers a pre-defined extensible data model for SAP components based on InfoObjects and InfoSources. This data model describes the BW data mart layer and also extends the BW data warehouse layer to include time slices. If the Business Content data model for mySAP components is used throughout, data mart InfoCube scenarios for a particular themed area can be built without producing isolated applications.

2003 SAP AG AND SAP AMERICA, INC. 28

Page 31: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5.4 Topologies – BW Landscapes The topology aspects of a BW architecture often interest people most. Some reasons for this are:

The short-term need to optimize already existing BW landscapes The parallel implementation of BW and R/3 Or simply a short-term focus on factors pertaining to costs.

However, the discussion in the previous chapters has made it clear that focusing on landscape aspects alone is superficial and in no way productive if strategies regarding the persistent layer and its data models are not also developed. If this is now clear, we can pose the question: What does the ideal BW landscape for my enterprise look like?

‘Local’BW

‘Local’BW

‘Local’BW

GlobalBW

OutsideOutside--ININBWBW

LandscapeLandscape

Others

‘Local’BW

‘Local’BW

‘Local’BW

GlobalBW

OutsideOutside--ININBWBW

LandscapeLandscape

Others

EnterpriseBW

InsideInside--OutOutBWBW EnterpriseEnterprise

DataDataWarehouseWarehouse Spoke

BW

SpokeBW

Others

SpokeBW

EnterpriseBW

InsideInside--OutOutBWBW EnterpriseEnterprise

DataDataWarehouseWarehouse Spoke

BW

SpokeBW

Others

SpokeBW

The Ideal Corporate BW Landscape ?

Strategic Corporate Implementation Options:Strategic Corporate Implementation Options:

Illustration 24: The ideal corporate BW landscape? Unfortunately we have to disappoint you: There is no ‘one size fits all’ BW landscape Naturally there are solutions that are to be favored in terms of consistence, but enterprise-specific circumstances often prevent such solutions. Decisions on BW landscape architecture and ultimately on all aspects of the architecture have to be made amidst the following conflicts:

2003 SAP AG AND SAP AMERICA, INC. 29

Page 32: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 109 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 109

Enterprise-wide Strategies

Inside-Out versus Outside-InCentral Data Hub, Local Spokes

Local Hubs, central aggregation

Centralized versus DecentralizedLow dispersal, high centralization

High dispersal, low centralization

High dispersal, high centralization

Global Integrations versus Local ResponsivenessGlobal scale

Global standardization

Global demand

Local customer needs

Local market conditions

Local governmental regulations

“Select the [IT] approach thatfits most closely with corporate strategy”

Prof. Sethi, University of TexasInformation&Management, 2/01

Illustration 25: Enterprise-wide strategies and BW architecture Further parameters relevant to this decision are:

• Master data integration status and strategy • To what extent is an awareness of the essential roles of consistent data available? • Budget • Sponsorship • IT Strategy - Operative System Landscape • Competitive environment.

As a basic principle the following advice should be followed: “Select the [IT] approach that fits most closely with corporate strategy” (Prof. Sethi, University of Texas Information&Management, 2/01) In other words, a business-wide BW enterprise data warehousing strategy will hardly ever be successful when it contradicts the corporate culture.

5.4.1 Consistency in a distributed BW Landscape The corporate BW landscape will often consist of several BW instances and possibly also external data warehouse tools. Alongside general decisions that ensure consistency within and between BW layers, rules must also be defined to ensure the consistency of data and metadata in a distributed data warehouse landscape. BW supports distributed (BW) landscapes with a multitude of functionalities: • Data consistency

The distribution of data in a landscape like this has to be controlled (delta handling). This is guaranteed by BW functionalities:

2003 SAP AG AND SAP AMERICA, INC. 30

Page 33: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

o Data Mart Interface (BW-BW data distribution) o Open Hub Service (BW-Third Party data distribution)

• Metadata consistency This is realized and controlled using the BW development environment and the BW metadata interface (XML for 3rd-party)

• Daily Operations Partnerships with producers of automization tools for data centers guarantee that the data distribution process (BW process chains) can be controlled and scheduled from one central point.

These functionalities are the same whatever BW landscape architecture you choose.

5.4.2 BW Landscapes und Technical Parameters Diverse technical parameters also influence what BW landscape is suitable for your enterprise. • Geographical distribution of users

In the most extreme cases users can be spread around the whole world. This graphic illustrates challenges that then arise:

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 114 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 114

Availability and Maintenance

Availability and Maintenance24 x 7 Access

Performance impacts

Data actuality

Data backup, repair and recovery

System upgrades and maintenance

Zero Downtime B r u s s e l s L . A . T o k y o R e p o r t i n g T i m e

1 2 a m 3 p m 8 a m

2 a m 5 p m 1 0 a m

4 a m 7 p m 1 2 p m

6 a m 9 p m 2 p m

8 a m 1 1 p m 4 p m

1 0 a m 1 a m 6 p m

1 2 p m 3 a m 8 p m

2 p m 5 a m 1 0 p m

4 p m 7 a m 1 2 a m

6 p m 9 a m 2 a m

8 p m 1 1 a m 4 a m

1 0 p m 1 p m 6 a m

B r u s s e l s L . A . T o k y o R e p o r t i n g T i m e

1 2 a m 3 p m 8 a m

2 a m 5 p m 1 0 a m

4 a m 7 p m 1 2 p m

6 a m 9 p m 2 p m

8 a m 1 1 p m 4 p m

1 0 a m 1 a m 6 p m

1 2 p m 3 a m 8 p m

2 p m 5 a m 1 0 p m

4 p m 7 a m 1 2 a m

6 p m 9 a m 2 a m

8 p m 1 1 a m 4 a m

1 0 p m 1 p m 6 a m

B r u s s e l s R e p o r t i n gF r 4 p m

8 p m

S a 1 2 a m

4 a m

8 a m

1 2 p m

4 p m

8 p m

S u 1 2 a m

4 a m

8 a m

1 2 p m

4 p m

8 p m

M o 1 2 a m

4 a m

8 a m

1 2 p m

B r u s s e l s R e p o r t i n gF r 4 p m

8 p m

S a 1 2 a m

4 a m

8 a m

1 2 p m

4 p m

8 p m

S u 1 2 a m

4 a m

8 a m

1 2 p m

4 p m

8 p m

M o 1 2 a m

4 a m

8 a m

1 2 p m

Illustration 26: BW landscape and availability / maintenance • Code pages in different languages have to be supported

BW is Unicode-compatible. Concrete solution scenarios have to be agreed with BW product management.

2003 SAP AG AND SAP AMERICA, INC. 31

Page 34: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

5.4.3 Inside-Out Landscape Architecture - BW as Central Enterprise Data Warehouse It is indisputable that business-wide, consistent information can soonest be guaranteed where there is a central BW instance offering data warehouse layer services. This means the business-wide consolidation of all relevant data in a granular, integrated, non-volatile, unflavored and subject-oriented form. William H. Inmon forwards this solution in the ‘corporate information factory’.

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 111 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 111

BW as Central Enterprise Data Warehouse (EDW)

BW Enterprise Data Warehouse:BW Data Warehouse layer as ‘single point of truth’ for the entire corporation

EnterpriseBW

BW EnterpriseBW EnterpriseDataData

WarehouseWarehouseScenarioScenario

SpokeBW

SpokeBW

Illustration 27: BW as Corporate Information Factory All data that acts as the basis for tactical and strategic data marts has to go through the enterprise data warehouse. Extraction straight into data marts, ignoring the enterprise data warehouse, is not permitted. Exceptions have to be controlled. There must be a strategy in place for using these data marts as the basis for operative reporting, as well as for using the enterprise data warehouse itself for operative reporting. A landscape of this sort with the inherent demands

• „single point of truth” and • “extract once, deploy many”

offers the ideal prerequisites for controlling redundancy at corporate level. It is often assumed that this solution, although ideal, is practically unrealizable. Without going into the arguments in detail, it is sufficient to say that this is not necessarily the case for BW customers.

2003 SAP AG AND SAP AMERICA, INC. 32

Page 35: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Many large companies that operate globally make a large investment in order to unify their operative landscapes by implementing SAP R/3. As a result, the ideal prerequisites for a central enterprise data warehouse approach emerge: integration of data (common coding), integration of data models into the SAP R/3 data model, the building of an operative R/3 landscape, which takes the company organizational and technical conditions into account. In other words, a centralized approach in the operative environment leads directly to a solution option for the enterprise data warehouse topology. This type of ’ideal’ environment is often found in ’single division’ enterprises that find themselves in strongly competitive environments and as a result, have a high sense of awareness for the usage of more reliable and efficient IT solutions. This awareness mostly involves a high degree of support from the highest management level for a BW enterprise data warehouse solution.

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 110 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 110

Inside-Out: Central BW Instance Covers All

BW EDWBW EDW

BW Architected Data MartsBW Architected Data Marts

tactical/operational like

reporting

strategicreporting

integratednon-volatile

no flavor

Inside-Out:Central EDW BW Instance Covers All

rr

PSAPSA

Stag

ing

Area

EnterpriseBW

BW EnterpriseBW EnterpriseDataData

WarehouseWarehouseScenarioScenario

SpokeBW

SpokeBW

BW ODSBW ODSoperational/ near real time

reportingr

ExternalExternal Data MartsData Marts

aggregatedless granulagranula

granula

Illustration 28: Scalability of a BW Corporate Information Factory I A BW enterprise data warehouse based landscape will be implemented incrementally for cost reasons. Often, first a BW instance will be built that offers data warehouse layer as well as data mart layer services, including operative-level reporting. Obviously the aim of the enterprise data warehousing strategy also has to be to source all external data marts with data from the BW enterprise data warehouse only! It is then possible to separate reporting and analysis services as the load increases.

2003 SAP AG AND SAP AMERICA, INC. 33

Page 36: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 112 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 112

Inside-Out: Central EDW BW Instance and BW Spoke Instances

EnterpriseBW

BW EnterpriseBW EnterpriseDataData

WarehouseWarehouseScenarioScenario

SpokeBW

SpokeBW

BW EDWBW EDW

integratedconsistent

Inside-Out:Central EDW BW Instance as Information HUBArchitected Data Marts Spoke Instances

r

strategicreporting

Corporate BWCorporate BWData Mart Data Mart

r

PSAPSA

Stag

ing

Area tactical/

operational likereporting

BW ODSBW ODSoperational reporting

r

BW OBW Ooperational reporting

granular

PSAPSA

BW Architected Data MartsBW Architected Data Marts

tactical/operational like

reporting

r

ri

BW ODSBW ODSoperational/ near real time

reportingr

ExternalExternal Data MartsData Marts

granula

agg egated

granula

DSDS

less granula

aggegaton

granula

Illustration 29: Scalability of a BW Corporate Information Factory II To summarize, the conditions for a central BW enterprise data warehouse solution are always most favorable within a ‘single division company’ with ‘strong’ headquarter.

5.4.4 Outside-In Landscape Architecture - Decentralized BW Data Warehouses

Even where it is preferable to have a single BW enterprise data warehouse for the whole enterprise, the complexity of the different business areas in a company that operates globally can lead to shared BW landscapes. On this issue William H. Inmon says:

‘Simply building a central single data warehouse does not address the size and complexity of the problem posed by the need for integrated, historical easily accessible data across a complex global enterprise.’ W.H. Inmon ‘Managing Multiple Data Warehouse Development‘

Often political circumstances or a company’s lack of an over-arching concept lead to shared BW landscapes too, even when the business conditions seem to be favorable.

Enterprises that choose diversification should generally check that the outlay involved in integrating data on a granular level, as required with the enterprise data warehouse approach, is not greater than the benefit.

Even when the ideal prerequisites exist such as for a ‘single division’ enterprise, certain conditions (a regional enterprise structure, for example, without process integration between the regions), coupled with technical and political factors, can lead to an enterprise moving away from the central BW enterprise data warehouse concept. This leads to a landscape that comprises several ‘local’ BWs. ‘Local’ here means that one of these BWs covers only one part of the organization. The description of what ‘local’ can mean in individual cases depends on the size and the complexity of the company.

2003 SAP AG AND SAP AMERICA, INC. 34

Page 37: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Each ‘local’ BW should operate like an enterprise data warehouse for its own area. In this way, the sum of the ‘local’ enterprise data warehouses forms a quasi virtual enterprise data warehouse. Unlike the central enterprise data warehouse that requires all corporate data to be integrated at granular data level, this is not necessary with ‘local’ enterprise data warehouses, although it is still preferable. Each ‘local’ enterprise data warehouse necessarily ensures a ‘local’ granular data integration. Having a landscape architecture based on ‘local’ BWs means that for over-arching reporting and analysis, at least one additional integration layer is required to consolidate the data from the ‘local’ BWs. A BW that consolidates the data from other BWs and offers this in an integrated form is called a ‘global’ BW. Again, the term ‘global’, and the scope of a particular ‘global’ BW can only be defined on an individual, company-specific basis.

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 128 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 128

Architected Multiple BW Landscape

Divisional

Global

BWBW Europe

LocalR

egional Data Warehouse

ODSData Marts

Global BW Global BW Division AA

Global BWGlobal BWHeadquarter

R/3-ERPAmericas I

R/3-ERPAsia

R/3-ERPEurope I

mySAPCRM

Staging

BWBW Americas

Data Warehouse

ODSData Marts

Staging

R/3-ERPAmericas II

BWBW Asia

Data Warehouse

ODSData Marts

Staging

virtualBW Enterprise

Data WarehouseStaging

Illustration 30: ‘Local - Global’ BW Landscape In terms of an enterprise’s data integration strategy, it is important that master data harmonization takes place on an aggregated level (global BW)1. In these terms a ’global’ BW of this sort is always an integrator.

1 Integration is then only demanded at the higher hierarchy levels of a ’subject area’ or dimension. This is generally easier to achieve than integration at granular level. As a result, a topology of this kind often mirrors the integration strategy of an enterprise that does not choose granular ‘common coding’.

2003 SAP AG AND SAP AMERICA, INC. 35

Page 38: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

The situation of ’local’ BWs or legacy systems with respect to harmonization determines the integration efforts at global BW site: 1. ‘Local’ BWs are to be integrated

I. It is the task of the ’global’ BW to gather data from ‘local’ BWs (Data values are already ’common coded’, local-global data models are integrated, a data model architecture exists)

II. It is the task of the ’global’ BW to consolidate and harmonize data (mapping) (Data values are not ’common coded’, local-global data models are integrated, data model architecture exists)

III. It is the task of the ’global’ BW to consolidate and harmonize data (mapping) and to integrate the local-global data models. (A data model architecture either does not exist, or is not controlled. Statements about model consistency cannot be made without doing individual checks.)

2. ’Local’ BWs and legacy systems are to be integrated I. For legacy systems data and data model integration is necessary.

These scenarios copy the different corporate data warehousing strategies. The essential difference lies in whether ‘local’ and ‘global’ BWs are developed according to common architecture guidelines (scenario 1.I and 1.II). If this is the case we can talk about an ‘architected outside-in BW landscape’. This harmonized, architecture-based interaction of local and global BW instances is a serious option for building a BW based corporate data warehousing strategy from the very beginning. It is supported by the central development of BW templates for ‘local’ reporting and analysis.

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 139 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 139

Architected Multiple BW Landscape: BW Template Roll-Out

Divisional

Global

BWBW Europe

LocalR

egional Data Warehouse

ODSData Marts

Global BWGlobal BWHeadquarter

R/3-ERPAmericas I

R/3-ERPAsia

R/3-ERPEurope I

mySAPCRM

Staging

BWBW Americas

Data Warehouse

ODSData Marts

Staging

R/3-ERPAmericas II

BWBW Asia

Data Warehouse

ODSData Marts

Staging

virtualBW Enterprise

Data WarehouseStaging

Architected Multiple BW Landscape: BW Template Roll-Out

R/3 templateroll-out

BW templateroll-out

Global BW Global BW Division AA

Illustration 31: Architected Multiple BW Landscape and BW Template roll-out

2003 SAP AG AND SAP AMERICA, INC. 36

Page 39: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

However, if dealing with a mature BW landscape, you will probably echo scenario 1.III and scenario 2. There is obviously only a restricted corporate strategy for ‘local’ BWs and it is the task of the ‘global’ BW to take the first step towards a corporate strategy (‘catch the BWs scenario’).

BWlocal

BWlocal

ERP/ Legacy

othersothers

BW GlobalBW Global

ERP/ Legacy Legacy/ Files

BWlocal

BWlocal

Focus on Integration andHeadquarter Reporting

→ Global BW as the Corporate Integrator

BWlocal

BWlocal

ERP/ LegacyERP/ Legacy

othersothers

BW GlobalBW Global

ERP/ LegacyERP/ Legacy Legacy/ FilesLegacy/ Files

BWlocal

BWlocal

Focus on Integration andHeadquarter Reporting

→ Global BW as the Corporate Integrator

Illustration 32: Global BW as corporate Integrator

2003 SAP AG AND SAP AMERICA, INC. 37

Page 40: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

6 Central BW Application Development As well as the need for consistent information on all organizational levels, avoiding redundant application development (data marts) is also a motivation for a business-wide BW strategy.

Redundant application development poses a constant threat to data consistency. Redundant application development is often accompanied by a certain arbitrariness in the management of persistent data stores and BW data models.

The general solution approach is to develop applications or templates for all the organization units of an enterprise centrally.

To what extent applications are generated on a ’local’ level (local-local applications) and/ or corporate templates are adaptable locally, is always company-specific. It is symptomatic that this decision is influenced more by political than functional factors.

Two extreme positions stand out regarding the local management of templates:

• Stable templates

No local changes to the central template are permitted

• Example templates

The centrally produced template acts as an example only.

The first is self-evident: The fewer changes are allowed to the central template, the easier the central further development and maintenance of these templates, and the better it is for overall consistency in the BW.

The description of this scenario sounds very rigid. This is not entirely true as it can work very well if local ‘power users’ are allowed to generate BW queries for template InfoProviders and MultiProviders2 within this scenario. It is then possible to define, for example, local virtual multi-dimensional and flat structures, local KPIs, selections, and layouts.

In the ’stable template’ scenario, the template is transported in the form of the active version of the template metadata (A versions).

If a local application development is intended in addition to central application development, this normally means that local development systems exist. If a central template is changed in this environment, importing a new version of the central template in the A version (see above) will overwrite local adaptations.

As of BW Version 3.5 the generation of customer content is possible to support this scenario.

This enables you to generate and transport meta objects in a D version, as is the case with Business Content. An import into the local target system does not affect the existent active meta objects. Meta objects are merged with existing objects only when the D version is activated. When there are conflicts, how to proceed must be resolved locally.

2 InfoProvider: InfoCube, ODS objects; MultiProvider: Union of InfoCubes and/ or ODS objects.

2003 SAP AG AND SAP AMERICA, INC. 38

Page 41: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 149 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 149

Content Delivery at the Customer Site

M(odified)

create

A(ctive)

activate

11

11

change

11

1122

Customer Content System (Headquarter)

Export (D Object)

import

M(odified)

A(ctive) 11

change

11

11

44

Subsidiary

Content activation

D(elivery)

1111

11 11 11

Illustration 33: BW Customer Content If locally adapting the central template is permitted an additional type of persistent data store is added locally: ‘distilled data’. These are data stores (usually ODS objects) whose structure and method of filling cannot be changed locally as they serve to prepare data for over-arching ‘global’ reporting. If it can be changed locally, the central template can no longer fill these roles.

The different structures of a ‘local’ BW instance illustrates the following picture.

Local BW Data Structures

corporate

local

distilled

source for source for corporate reportingcorporate reporting

standardized corporatecorporatedefined

local reporting

local definedlocal definedlocal reportinglocal reportingLocal BW

Data Structures

corporate

local

distilled

source for source for corporate reportingcorporate reporting

standardized corporatecorporatedefined

local reporting

local definedlocal definedlocal reportinglocal reporting

Illustration 34: BW Customer Content

2003 SAP AG AND SAP AMERICA, INC. 39

Page 42: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

7 Data and Data Model Integration Integration is the central issue from a corporate point of view. In the data warehouse area this primarily concerns the integration of data from different sources and the integration of data models from different operative components. Data integration in BW directly addresses the topic Master Data Management and the SAP solution MDM (Master Data Management).

7.1 Challenges Posed by Data Integration Both the data warehouse layer and the data mart layer make integrated data available by definition. But data integration in corporate terms is easier said than done. As such integrating data from different sources is often the greatest challenge3. The harmonization of data is therefore not a fact of enterprise data warehousing scenarios, but rather the aim. Even when all operative systems deliver ‘common coded’ data, you always have to anticipate that data that cannot be integrated may also be delivered in the future because of company acquisition. The result of this is that data that cannot be integrated during load time has to be saved in the BW. Recognizing this in good time is essential. The status of the master data has to be investigated and the enterprise-wide data harmonization strategy has to be considered carefully. This will allow later data harmonization without having to make adjustments. If data cannot be integrated then the different sets of data have to be separated so that they do not overwrite each other.

Not Common Coded Source-Keys as BW-keys *>> Introducing new Sources later

0MATERIAL TextM1 PIPESS2 CONSULTING

0MATERIAL Date QuantityM1 20020601 100S2 20020601 200

M1 20020601 100 M1 PIPES M1 PUMPS2 20020601 200 S2 CONSULTING S22 CONSULTING

InfoCube Fact-table

Conformed Dimension/InfoObject Master Data

Transaction Data Master Data

R/3-1 R/3-2

Later Introduction of Sourcesystem

?

BW-KEY

BW-KEY

* the technical BW implementation is more sophisticated - the key issue exist anyhow

Master Data

3 With restricted BW implementations data harmonization is often not a central issue. You may therefore only come up against the data harmonization task later when the BW is implemented as an enterprise-wide data warehouse solution.

2003 SAP AG AND SAP AMERICA, INC. 40

Page 43: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Illustration 35: Source system key as BW key? There are several ways in which this situation can be controlled with BW. For one, by using name spaces different data models can be maintained in BW for the different operative systems that cannot be integrated. Data stores are then separated automatically too. This approach is not recommended for an enterprise, however, it is an option for application centers (which offer BW services for different enterprises). As separating data using different data models is only advisable in odd cases, data should actually be separated in data structures defined by a common data model. This is done by extending the key structure of the BW InfoObject affected using a ‘separator’. In the most straightforward of cases this separator will contain origin information (source systems). BW supports separation through concatenation and compounding.

ConcatenationBW-Key

Concatenated-Key

Separator Text

U1004711 U1 AUDIU2004711 U2 BMW

U1 U2

004711 AUDI 004711 BMW

Source-System 1 Source-System 2

Build Key Build Key

Separator

Source-Value

Illustration 36: Key value concatenation Concatenation means the data model is not affected. The key values are simply extended by the separator. This can happen centrally in the BW using general transfer rules. Here we have a data separation option that can be influenced by the user and which it is possible to modify at a later date.

With compounding on the other hand, the key column of the InfoObjects affected are extended to a source system column. This can be done per InfoObject and is automatically supplied by the BW. Separation is handled by the BW and the database. However this has the disadvantage that it cannot be removed again later (for example, after subsequent integration).

We favor concatenation as compounding has further functional limitations when handling complex heterogeneous integration scenarios and is inflexible because of automatism.

Using InfoObject master data navigation attributes (conformed dimension attributes) in queries, instead of concatenated InfoObjects, makes realignment to the common coded values after a migration possible. Adjustments to queries are then obsolete. This is illustrated in the next graphic:

2003 SAP AG AND SAP AMERICA, INC. 41

Page 44: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

0MATERIAL Separator IMATERIAL Text

S1004711 S1 00000123 AUDIS4004711 S4 00000456 BMW00000123 00 00000123 AUDI00000456 00 00000456 BMW

Date 0MATERIAL Amount20020930 S1004711 10020020930 S4004711 20020021001 00000123 300 Query results after migration 20021001 00000456 400

Text 0MATERIAL 2002AUDI S1004711 100BMW S4004711 200AUDI 00000123 300BMW 00000456 400

Text IMATERIAL 2002AUDI 00000123 400BMW 00000456 600

Query access

Group-IDConcatenated-

Key

entries before migration

entires after migration

realigned material numbers after migration

InfoCube Fact Table

Power of BW Navigational Attributes

Illustration 37: Navigation attributes support the flexibility of BW solutions

7.2 Data Model Integration Third party and legacy applications, and even different SAP components such as R/3 and EBP (Enterprise Buyer Professional), use different semanitics for, for example, product terms. For this reason there are different subject areas for products in BW (InfoObjects). These various product views are consolidated again by ‘consolidated’ InfoObjects.

SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 44 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 44

IS: Material IS: Service IS: BBP Prod

0CRM_PROD

IS: CRM Prod

0BBP_PROD0SERVICE0MATERIAL

Global and consolidated views

on products,

independent from source component

Component restricted Analytics

DataSources

Source Components individual data

models

Cross-Component Integration: Product

R/3 Material

R/3R/3R/3

EBP Product

R/3R/3EBP

CRM Product

R/3R/3CRM

R/3 Service

R/3R/3R/3

0PRODUCT

Illustration 38: Model integration with consolidated InfoObjects

2003 SAP AG AND SAP AMERICA, INC. 42

Page 45: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

This obviously does not resolve the challenge of making a common coded key available for the consolidated view 0PRODUCT. SAP MDM functionalities offer support in this respect.

7.3 The Role of SAP Master Data Managements (MDM) Corporate master data management is a topical issue. SAP offers SAP MDM (Master Data Management).4

Active master data management, the central management and distribution of master data, will not be examined any more closely here. If a central master data management exists, BW is a potential client and the overall corporate reporting and analysis is a beneficiary.

More interesting are the possibilities offered by MDM in complex, business-wide BW scenarios without harmonized data. Master data from different source systems can be analyzed for commonalities and a grouping (group ID) corresponding to common coding suggested.

This passive functionality is termed ‘content consolidation’.

SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 45 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 45

mySAP MDM Master Data Consolidation

ClientSAP

Clientnon SAP

Clientnon SAP

ClientSAP

Clientnon SAP

Description of a master data object:Identifying AttributesOther application specific data

Load

Matching

?

ID Mapping=

Content ConsolidationCore Process Steps:

1. Loading identifying attributes of master data objects as they were maintained in their local applications (clients)

2. Matching of uploaded master data objects to identify duplicates

3. Providing ID Mapping Information based on matching results for subsequent usage (I.e. in Business Intelligence)

MDMMDM

Illustration 39: SAP MDM master data consolidation

The mapping information (group ID) is then a navigation attribute of the InfoObject to be harmonized in BW (see Illustration 37: Navigation attributes support the flexibility of BW solutions). This can also be added afterwards without interfering with the existing schemata.

4 Integration between BW and MDM will be offered as of Release BW 3.5.

2003 SAP AG AND SAP AMERICA, INC. 43

Page 46: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

SAP MDM supports the model consolidation scenario described in 7.2 (Data Model Integration) through ’master data harmonization’ by means of consolidated InfoObjects for an analysis that spans components.

Keys of different InfoObjects are checked in MDM for commonalities, mapped to a group ID, and maintained centrally as attributes that span InfoObjects.

SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 46 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 46

Master Data Harmonization

MDMMDMClientSAP

Clientnon SAP

ClientSAP

Clientnon SAP

Local Creation

= ?Matching & ID Mapping

Distribution

+

+

Local Completion

Central Creation

Master data harmonization Core Process Steps:

1. Central Creation of master data objects just covering the global information of the master data object

2. Local Creation within the master data environment of the application system (client) and distributing their global information to MDM

3. Continuous matching processes identify duplicates and result in ID Mapping between them

4. Distribution of global master data information to the various clients

5. Local Completion of master data within the local environment

6. Use of ID Mapping information for MDM Analytics like business-wide analyses etc.

Description of a master data object:Globally relevant Data(Client) specific Data

13.282.401 €=737.108 €

4.002.531 €

634.237 €

6.674.288 €

1.234.237 €

Analytics

Illustration 40: SAP MDM master data harmonization

As well as active master data management, MDM supports key consolidation and complex master data harmonization through deferred (asynchronous) consolidation. The flexibility of the conformed dimension in BW allows the ’extended star schemas’ to be subsequently enhanced with information from the MDM, without interfering with existing solutions.

2003 SAP AG AND SAP AMERICA, INC. 44

Page 47: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

8 Summary Important elements of a corporate BW strategy are architecture, consistent application development and data integration concepts. A corporate BW architecture is essential to mid- and long-term success. An architecture of this kind manifests itself in design and procedural guidelines that are valid enterprise-wide, and in their control. BW supports various aspects of the architecture through a multitude of functionalities that secure consistency. It is up to each individual enterprise to determine how and to what extent these functionalities are used. A central development of BW application templates is an adequate way to support a corporate wide BW architecture and harmonization. Integration of data is an ongoing challenge. BW is tightly integrated with SAP’s MDM and offers in addition various design options to meet integration requirements. This has to be considered in a BW strategy. Coming to the end we want to repeat once more the main statement of this paper: The top priority of any corporate BW strategy must always be to control redundancy. This is true for each individual BW, and for the interaction of all the BWs within an enterprise!

2003 SAP AG AND SAP AMERICA, INC. 45

Page 48: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Table of Illustrations Illustration 1: Evolution of the SAP BW ................................................................................................ 2 Illustration 2: Redundancy and Solution focus ..................................................................................... 3 Illustration 2: Redundancy and multiple BW landscape....................................................................... 4 Illustration 3: The conceptual layer architecture of data warehousing................................................. 7 Illustration 4: The BW extended star schema .................................................................................... 11 Illustration 5: Horizontal consistency in the BW architected data mart layer ..................................... 11 Illustration 6: BW and slowly changing dimensions ........................................................................... 12 Illustration 7: Persistent and virtual multidimensional structures in BW............................................. 13 Illustration 8: SAP BW data warehouse layer..................................................................................... 14 Illustration 9: BW data warehouse layer and vertical consistency ..................................................... 16 Illustration 10: Example - BW data warehouse: Historization of master data .................................... 17 Illustration 11: Example - BW data warehouse: Transaction data ..................................................... 18 Illustration 12: BW staging.................................................................................................................. 19 Illustration 13: Classic data warehousing........................................................................................... 20 Illustration 14: Classic data warehousing supports operatives reporting........................................... 21 Illustration 15: BW operational data store: Near real time data warehousing.................................... 22 Illustration 16: BW and operational / real time reporting .................................................................... 23 Illustration 17: Operational data model and data warehouse data model......................................... 24 Illustration 18: Data warehouse data models and the situation for BW customers............................ 25 Illustration 19: Enterprise data model and enhanceable BW data model: InfoObjects...................... 25 Illustration 20: BW data model: InfoSources and subject areas ....................................................... 26 Illustration 21: BW data model: data mart layer data model ............................................................. 27 Illustration 22: BW data model: Data warehouse layer data model .................................................. 28 Illustration 23: The ideal corporate BW landscape?........................................................................... 29 Illustration 24: Enterprise-wide strategies and BW architecture ........................................................ 30 Illustration 25: BW landscape and availability / maintenance ............................................................ 31 Illustration 26: BW as Corporate Information Factory ........................................................................ 32 Illustration 27: Scalability of a BW Corporate Information Factory I................................................... 33 Illustration 28: Scalability of a BW Corporate Information Factory II.................................................. 34 Illustration 29: ‘Local - Global’ BW Landscape................................................................................... 35 Illustration 30: Architected Multiple BW Landscape and BW Template roll-out................................. 36 Illustration 31: Global BW as corporate Integrator ............................................................................. 37 Illustration 32: BW Customer Content ................................................................................................ 39 Illustration 33: BW Customer Content ................................................................................................ 39 Illustration 34: Source system key as BW key? ................................................................................. 41

2003 SAP AG AND SAP AMERICA, INC. 46

Page 49: Enterprise Data Warehousing With SAP BW - Overview

ENTERPRISE DATA WAREHOUSING WITH SAP BW – AN OVERVIEW V2.0

Illustration 35: Key value concatenation............................................................................................. 41 Illustration 36: Navigation attributes support the flexibility of BW solutions ....................................... 42 Illustration 37: Model integration with consolidated InfoObjects ........................................................ 42 Illustration 38: SAP MDM master data consolidation ......................................................................... 43 Illustration 39: SAP MDM master data harmonization ....................................................................... 44

2003 SAP AG AND SAP AMERICA, INC. 47


Recommended