© 2007 Wellesley Information Services. All rights reserved.
25 Critical Lessons for Teams That Are Running Global SAP NetWeaver BI Projects
Dr. Bjarne Berg
2
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
3
Introduction
• SAP NetWeaver BI (BI) has matured substantially over the last decade
• In this session we will explore the lessons learned from a variety of companies, looking at real examples
• We will pay particular attention to design principles, performance enhancements, and explore what you can learn from other’s mistakes
• We will also look at BI trends and what others are doing in the SAP NetWeaver BI space
4
BI Has “Grown Up” and Is Now a Core Infrastructure Requirement
• 59% of companies now have an enterprise data warehouse This number will grow to 86% by 2009
• However, only 39% of companies have a BI competency center and only 54% have an enterprise information strategy
0% 10% 20% 30% 40% 50% 60%
Enterprise Data Warehouse
Information quality program
Enterprise Info. strategy
BI governance program
BI competency center
No plans
Within 2009
Currently have
Source: Business Week Research, Sept. 2006
Where will your company be in three years?Where will your company be in three years?
5
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
6
Global Master Data Integration and SAP NetWeaver BI
• When an organization has multiple SAP R/3 systems, the SAP NetWeaver BI master data integration can be cumbersome
1. Determine what local master data needs to be maintained in the BI system and load it as separate master data
2. Determine what global master data needs to be maintained in the BI system and load it to the shared areas
To do this you will need master data mapping rules or tables
3. If you have conflicting master data, consider loading the local masterdata to DSOs (make it reportable) and merge the global master data to be used by the InfoCubes
Alternatively, you can load versions of master data into custom master data objects in SAP NetWeaver BI and use the merged master data as the “0 objects”
If you have multiple regional transaction systems, plan on spending 5 - 10% of your global project time on master data integration, design, and testing.
If you have multiple regional transaction systems, plan on spending 5 - 10% of your global project time on master data integration, design, and testing.
7
Global Master Data Integration and SAP NetWeaver BI (cont.)
• You can also decide to use the SAP NetWeaver Master Data Management (MDM) tool to manage and merge the global master data in a central location
If an MDM implementation is warranted, make it a separate project and don’t include this as part of the global BI project. The BI system simply extracts the new merged and integrated master data.
If an MDM implementation is warranted, make it a separate project and don’t include this as part of the global BI project. The BI system simply extracts the new merged and integrated master data.
Source: SAP NetWeaver Magazine, Alan Joch
8
Privacy and Security Concerns
• European and many Asian countries have strict privacy rules
• European Union Commission’s Directive on Data Protection protects personal information from Europe to countries whose privacy practices are not deemed “adequate”
• All European Union (EU) member countries are bound by the European Commission’s finding of adequacy
• Your company can become entangled in many local laws, or seek participation in the “safe harbor” agreements by the Department of Commerce (costs up to $500)
• The safe harbor eliminates the need for prior approval to begin data transfers, and makes approval from the appropriate EU member countries automatic
Some European Restrictions
1. Personal information cannot be collected without consumers’ permission. They have the right to review the data and correct inaccuracies.
2. Data processing companies must register their activities with the government.
3. Employers cannot read, copy, or transfer workers’ private email.
4. Personal information cannot be shared by companies or across borders without express permission from the data subject.
Safe harbor agreements can reduce exposure to such legal issues.
Some European Restrictions
1. Personal information cannot be collected without consumers’ permission. They have the right to review the data and correct inaccuracies.
2. Data processing companies must register their activities with the government.
3. Employers cannot read, copy, or transfer workers’ private email.
4. Personal information cannot be shared by companies or across borders without express permission from the data subject.
Safe harbor agreements can reduce exposure to such legal issues.
9
Tip 1: Capacity and Scalability Is the Top Concern for Your CxO
• Don’t under size your global BI system Spend adequate funding on hardware, memory, processing power
and disk space
7.03
7.21
7.31
7.51
7.62
7.65
7.88
7.9
7 7.25 7.5 7.75 8
Difficult for end users to learn or use
Danger of distributing outdated data
Too many Business Intelligence tools in use
Data Integration
Security
Determining ROI
Server & Desktop processing capacity
Overall System Performance and capacity
Source: Intel, SAP & Business Week "Seizing the BI Opportunity" 2006.
A survey of 353 top C-level officers in large companies, reported that the top BI concern was the scalability of their solutions.
A survey of 353 top C-level officers in large companies, reported that the top BI concern was the scalability of their solutions.
10
Production
12 CPUs
96 GB RAM
Sandbox
4 CPUs
32 GB RAM
Test
4 CPUs
32 GB RAM
Development
4 CPUs
32 GB RAM
• This company reallocated their Sun-6900 box (on Oracle) and have three app servers on the production box
• More memory allows them to take better advantage of the parallel load of the BI system and to cache many of the frequently run queries (BEx Broadcaster)
• The hardware also allows them to compare performance between the boxes
Example of a large BI system
Tip 1: Capacity and Scalability — Global Hardware Example
Key numbers: 3.1 Terabytes of data 2810 named users About 620 active users
Global systems must consider network connections (speed), data encryption, and distributions of application and Web servers for load balancing (due to 24-hour operations).
Global systems must consider network connections (speed), data encryption, and distributions of application and Web servers for load balancing (due to 24-hour operations).
11
Joint Application Design(JAD)
Rapid Application Development(RAD)
Extreme Programming(EP)
System development Life-Cycle based methodologies
(SDLC)
Impact of FailureLow High
Low
High
Time to Delivery
When to Select Different Methodologies
Tip 2: Pick a Formal Methodology — You Have Many Choices
• Accelerated SAP (ASAP) methodology is not your only choice• Even though harder to manage on a global project due to long
communication lines, consider RAD, JAD, or EP based on the time to delivery and impact of failure
Source: Bjarne Berg, DM Review 2004
Look at what your global organizational partners have done.
You may know more about RAD than you think!
Look at what your global organizational partners have done.
You may know more about RAD than you think!
12
Tip 3: Report Dispositioning: Don’t Throw Every Report into BI
• Many tools exist that can report on SAP R/3 data You might have static reports that belong in SAP R/3, which would
not be cost effective to develop in SAP NetWeaver BI• Make cost-effective decisions
Just because the report is not in SAP NetWeaver BI does not mean it cannot be added to a Portal or viewed on the Web
• Not all reports belong in SAP NetWeaver BI Avoid using BI as a “dumping group”
• You need to make conscious decisions on what reporting needs you are going to meet, and how you will accomplish this
You need a formal report dispositioning process and need to evaluate every requirement as BI vs. transaction reporting.
You need a formal report dispositioning process and need to evaluate every requirement as BI vs. transaction reporting.
13
Tip 4: SAP Solutions Manager — EarlyWatch Reports Are Great!
• EarlyWatch reports provide a simple way to confirm how your system is running and to catch problems A “goldmine” for system
recommendations
• Run them periodically and read the details
• A real EarlyWatch report from a mid-sized company that has been running SAP BW for the last three to four years
On a large global project, system issues can be hard to pin-down without access to EarlyWatch reports. The monitoring reports allows you to tune the system before the user community gets access and complaints arise.
On a large global project, system issues can be hard to pin-down without access to EarlyWatch reports. The monitoring reports allows you to tune the system before the user community gets access and complaints arise.
14
Tip 4: SAP Solutions Manager — EarlyWatch Performance Info
1 Performance Indicators The following table shows the relevant performance indicators in various system areas.
Area Indicators Value Trend System Performance Active Users 18 down Hardware Capacity Max. CPU Utilization on DB Server 74 % steady Max. CPU Utilization on Appl. Server 74 % steady Database Space Management DB Size 355.52 GB steady Last Month DB Growth 118.63- GB steady Query Performance Avg. Total Runtime of the BW Queries 11.5 s down Avg. Database Runtime of the BW Queries 8.0 s steady
1 Performance Overview
The performance of your system was analyzed with respect to the average response times and total workload. We did not detect any major problems that could affect the performance of your system.
The following table shows the average response times for various task types:
Task type Dialog Steps
Avg. Resp. Time in ms
Avg. CPU Time in ms
Avg. Wait Time in ms
Avg. Load Time in ms
Avg. DB Time in ms
Avg. GUI Time in ms
DIALOG + RFC
195240 3253.3 728.7 1.8 2.5 1110.9 6.3
UPDATE 5 984.2 28.2 26.0 15.2 585.4 UPDATE2 48 133.2 17.1 0.7 3.3 80.8 BATCH 59288 11599.3 2091.2 0.6 8.5 5772.6 HTTP 257762 693.5 183.7 4.4 2.2 405.0
1.1 Current Workload The following table lists the number of current users (measured from our workload analysis) in your system.
Users Low Activity Medium Activity High Activity Total Users Measured in System 98 11 7 116
In a 24-hour operational systems due to time-zones, you will have less time to react and fix issues. Therefore, early detection of system issues are critical to the success of a global project.
In a 24-hour operational systems due to time-zones, you will have less time to react and fix issues. Therefore, early detection of system issues are critical to the success of a global project.
15
Tip 5: Organizing the Global Team — Six Ways to Balance the Global BI Development Effort
Benefits Risks
1 Single site
2 Distributed analysis
3 Distributed analysis and design
4 Co-located analysis and design
5 Multiple co-located analysis and design
6 Fully distributed development
Option
The more distributed the BI development effort becomes, the more difficult it is to maintain communication and get cohesive requirements.
The more distributed the BI development effort becomes, the more difficult it is to maintain communication and get cohesive requirements.
16
Tip 5: Organizing the Global Team — Small Technical Team Roles
Four to five team members and normally three to six months duration on each go-live depending on scope
BI Basis and functional SAP R/3 support
• These are roles not positions Sometimes one team member can
fill more than one role
Many companies fail to formally assign roles and responsibilities.
As a result, they have many “jack of all trades” and “masters of none.”
Many companies fail to formally assign roles and responsibilities.
As a result, they have many “jack of all trades” and “masters of none.”
Business AnalystPresentation Developer
Business Team
BI ArchitectETL Developer
Technical Team
Project Manager
Project Sponsor
ETL = Extract, transform and load
17
BI Basis and functional SAP R/3 support
15-30 team members and normally 6-18 months duration between each go-live
Tip 5: Organizing the Global Team — Large Functional Teams
Portal Developer(s)
BI Architect
Business Analyst/(sub-team lead)BI DeveloperPresentation Developer(s)ETL Developer
Sales Team
Business Analyst/(sub-team lead)BI DeveloperPresentation Developer(s)ETL Developer
Finance Team
Business Analyst/(sub-team lead)BI DeveloperPresentation Developer(s)ETL Developer
Material Mgmt. Team
Project Manager
Project Sponsor/Steering Committee
In a global BI organization, you simply need to create functional teams (instead of the previous technical team models).
In a global BI organization, you simply need to create functional teams (instead of the previous technical team models).
18
Tip 5: Organizing the Global Team — Localized BI Training
• Training for end-users and the local query developers should be completed in their own language to assure understanding and encourage participation
• Developer training should be in the project language (e.g., English, German, French) Don’t under estimate the value and cost
savings of in-house training.Don’t under estimate the value and cost savings of in-house training.
Reference Title Audience Language Class size (max) Note
BW-310 Intro to SAP BI All Local 25 Bring in house
BW-305 BI Reporting & Analysis Query developers Local 15 Bring in house
BW-350 BI Data Acquisition ETL developers English 10-12 SAP facility
BW-365 BW Authorizations System admin English 1-3 SAP facility
SAP-330 BW Modeling BI developers English 10-20 SAP facility
19
Tip 6: You Have to Plan for Global Cockpits
• 58% of all companies reported that they were already using BI dashboards and 25% more planned to do it in 2007 – 2008 Source: Business Week Research Sept. 2006, survey
• Dashboards are no longer cutting-edge Expected by senior management
Dashboard Scorecard Cockpits
Purpose Displays performance Displays progress Displays status and events
Usage Performance monitoring Performance management Performance management
Updates Real-time feeds Monthly snapshots Daily snapshots
Data Events Summaries Summaries and events
Measures Metrics KPIs Metrics & KPIs
Context Exceptions/alerts Targets and thresholds Trends
Source Linked to systems Linked to plans Linked to BI systemsSources: Wayne Eckerson, 2005; Bjarne Berg 2006
Dashboards and cockpits should be in your long-term global BI strategy (three to five years).
Dashboards and cockpits should be in your long-term global BI strategy (three to five years).
20
Tip 7: Pick a Project Language and Stick with It!
• If you don’t enforce a global project language, BI project documentation becomes fragmented The project team will quickly disintegrate into groups based on the
language with which they are most comfortable• Enforce a project language and require that all emails are
written in it and all notes are taken in the same language • Don’t allow “side bars” in languages that others
don’t understand
Make sure the project language is clear, and that pertinent documents are translated in a timely fashion.
Make sure the project language is clear, and that pertinent documents are translated in a timely fashion.
21
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
22
Billing
Number of billing documentsNumber biling line itemsBilled item quantityNet weightSubtotal 1Subtotal 2Subtotal 3Subtotal 4Subtotal 5Subtotal 6Subtotal ANet valueCostTax amountVolume
Customer
Sold-toShip-toBill-toPayerCustomer classCustomer group~ Customer country~ Customer region~ Customer postal code~ Customer industry code 1End user
Material
Material numberMaterial enteredMaterial groupItem categoryProduct hierarchyEAN/UPC
Time
Calendar yearCalendar monthCalendar weekCalendar day
Unit
Currency KeyUnit of MeasureBase unit of measureSales unit of measureVolume unit of measureWeight unit of measure
Billing information
Billing documentBilling itemBilling typeBilling categoryBilling dateCreation dateCancel indicatorOutput medium~ Batch billing indicatorDebit/credit reason codeBiling categoryReference documentPayment termsCancelled billing documentDivison for the order headerPricing procedure
Organization
Company codeDivisionDistribution channelSales organizationSales group
Logistics
PlantShipping/receiving point
Document details
Sales order document typeSales dealSales docuement
Accounting
Cost centerProfit centerControlling areaAccount assignment group
Personnel
Sales rep number
LEGEND
Delivered in standard extractorsDelivered in LO extractorNot in delivered Content -but in R-3
Standard Content
Tip 8: In the Blueprinting Phase: Model Your Global Solution
Storage Requirements
Storage Objects
+
1. Create a model based on pre-delivered SAP NetWeaver BI content2. Map your data requirements to the delivered content, and identify gaps3. Identify where the data gaps are going to be sourced from
1. Create a model based on pre-delivered SAP NetWeaver BI content2. Map your data requirements to the delivered content, and identify gaps3. Identify where the data gaps are going to be sourced from
Map functional requirements to the standard content before you make enhancements
Map functional requirements to the standard content before you make enhancements
23
Tip 9: Accept Cultural Differences — No Culture Is Dominant!
• Cultural differences should not be tolerated, but embraced • Europe has longer vacations (four to six weeks are common,
not exceptions)• Family time is important — don’t plan 12-hour workdays for
four months• Not everyone is equally interested in hearing how we do
things in the US• Many cultures find it offensive to talk about salaries,
and money Talk about value and deliverables instead
• Consider a co-project manager
24
Tip 10: Meet Local and Global Requirements
• In most global organizations there are varying product hierarchies, financial reporting requirements, mapping of accounts, data integration, and tool requirements
• Try not to solve only the enterprise reporting needs of the corporation, but also the local reporting and analytical needs This way you will get local support and ensure the system will
actually be used
• A successful global BI project has more that one constituent
If you ignore local requirements, they will simply reinvent your effort and find work-arounds and other tools.
If you ignore local requirements, they will simply reinvent your effort and find work-arounds and other tools.
25
Approximate Usage of Standard Content BW 3.5 (percentage of overall development effort)
0
20
40
60
80
100
* Rapidly improving content
Tips 11: Determine Where to Start
• All functional areas are not equally supported by strong standard BI business content Some areas have much you can leverage, others will require
significant enhancement to meet your requirements The differences are often due to customization in SAP R/3 by
companies and/or industry solutionsFocus on an area that solves a problem instead of becoming a “replacement” project.
Gradually, using a priority-phased approach, solve other business problems.
A good way to think of a BI rollout is in terms of business problems.
Focus on an area that solves a problem instead of becoming a “replacement” project.
Gradually, using a priority-phased approach, solve other business problems.
A good way to think of a BI rollout is in terms of business problems.
26
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
27
Tip 12: InfoCube Design — Making the Right Design Decisions
• Dimensions BI allows you to create up to 16 dimensions in a single InfoCube (13
are free) However, using all 16 on a first implementation limits any future
extensions without major redesign of the system Line item dimensions increase query performance
These are physically stored in the fact table and therefore have fewer table joins
• Key figures While no limitations are imposed by BI for the number of key
figures, typical implementations contain 1 - 25 While a higher number may be required (i.e. CO-PA), you’ll
notice tradeoffs of load performance when many records are loaded
While more than 45 key figures are not necessarily wrong, it might be considered unusual — you should perform an impact study of the extract on SAP R/3
28
Tip 12: InfoCube Design — General Guidelines
• Navigational attributes Lends flexibility to the way users can access data Common configuration consists of 1 - 35 attributes While technically not incorrect you should review InfoCubes that do
not contain any navigational attributes Review any InfoCube that contains more than 60
This may be an indicator that too much information is being placed in a single InfoCube
• Hierarchies Hierarchies are ways for users to “drill-down” into the data for
analysis purposes Typical configurations tend to have 1 - 8 hierarchies Review any InfoCube with no hierarchy (or with more than 8) to
validate the design with end-user navigation
For global projects you need to consider global and local currencies as well as various units of measures (e.g., lbs, kilo, tons, gallons, liters). Once decided upon, these units of measures should be available in all relevant InfoProviders.
For global projects you need to consider global and local currencies as well as various units of measures (e.g., lbs, kilo, tons, gallons, liters). Once decided upon, these units of measures should be available in all relevant InfoProviders.
29
Tip 12: InfoCube Design — Evaluating Designs (Real Example)
Name Type Tech_nm Dims* (all)
Characteristics total
Largest dim # of Char
KF # info. Sources
Nav attributes
Billing documents condition values Infocube ZSD_C15 10 47 11 3 1 107Customer Infocube 0SD_C01 8 14 5 16 1 5Delivery service Infocube 0SD_C04 9 23 5 15 2 10Invoice summary Infocube ZSD_C06 16 55 11 19 4 116Order summary Infocube ZSD_C03 16 62 13 23 5 96Sales order condition value Infocube 0SD_C15 10 41 10 2 1 0Sales overview Infocube 0SD_C03 11 34 7 17 7 16Profitability analysis Infocube Z_COPA_X 15 56 14 85 1 70Inventory mgmt plant summary Infocube MRP_MATL 6 9 3 22 1 16Material stock/movements Infocube 0IC_C03 9 18 4 24 - 5Plant & periodic plant stocks Infocube 0IC_C01 8 15 5 21 - 15Ad-hoc query order line details MC ZSD_M01 7 14 3 6 - 0Conditions order & billing document MC OSD_MC01 4 66 56 2 - 0Order and Invoice summary MC ZSD_C04 16 72 20 24 - 88SD Pricing order & billing docs MC ZSD_M02 12 53 12 3 - 94Campaign management Infocube ZDM_C006 15 56 19 29 9 109Commissions Infocube ZDM_C003 14 34 14 8 4 0Daily management Infocube ZSD_C01 9 31 8 10 6 6Disposition summary Infocube ZDM_C001 9 16 4 1 1 0Inquiry summary Infocube ZDC_C005 15 22 4 1 1 29Matrix Infocube ZDM_C002 10 16 3 141 1 0Monthly management report Infocube ZDM_C005 7 9 3 6 1 0Program summary Infocube ZDM_C004 16 23 4 12 8 1
• The observation relates a company’s current BI system to normally observed configuration parameters, which serve as benchmarks to what is commonly seen at other implementations
Cubes with many red or yellow codes should be examined
Cubes with many red or yellow codes should be examined
KF = Key figures
30
Tip 12: InfoCube Design — Evaluating Designs (Another Example)
31
Tip 13: Partitioned InfoCubes That Are No Longer the Same
• Often when InfoCubes are physically partitioned, changes occur as new development and fixes are applied After a while there is a risk that some of the physically partitioned
InfoCubes no longer are identical This can cause many issues (i.e., If archiving is used, you must
ensure copies of these older datastores are maintained to be able to restore data)InfoCubes
Name Technical name
All dimensions
Largest dimension
Characteristics
Key Figures
Nav. Attrib.
Sales Order: ACD 2007 ZCORD_A07 15 5 40 9 59
Sales Order: LPD 2006 ZCORD_L06 14 4 40 9 59
Sales Order: LPD 2007 ZCORD_L07 15 5 40 9 59
Sales Order History: ACD 2006 ZCODI_A06 15 13 58 39 66
Sales Order History: ACD 2007 ZCODI_A07 15 13 60 9 66
Sales Order History: LPD 2006 ZCODI_L06 15 13 58 39 66
Sales Order History: LPD 2007 ZCODI_L07 15 13 60 9 66
1) Removed 30 Key Figures, 2) Added field "Date for inv/bill index and print out" to Date dimension 3) added field "Customer purchase order type" to business reason dimension
1) Removed 30 Key Figures, 2) Added field "Date for inv/bill index and print out" to Date dimension 3) added field "Customer purchase order type" to business reason dimension
NOTESAdded 'Created by' as a Dimension(instead of f ield in Date dim)
Added 'Created by' as a Dimension(instead of f ield in Date dim)
A real exampleA real example
32
Tip 14: Naming Conventions Should Be Followed
InfoCubes should be named:0ABC_C01 orZABC_C02
ODSs should be named:0ABC_O01 orZABC_O02
MultiProviders should be named:0ABC_M01 orZABC_M02
InfoCube Supply Chain Mgmt - Inventory Mgmt. Cube for Risk Management ZRISK
InfoCube Supply Chain Mgmt - Inventory Mgmt. Material stocks/movements (as of 3.0B) 0IC_C03
InfoCube Industry Sectors - Oil & Gas - Exchanges Exchange balance 0OI_EXC01
InfoCube Industry Sectors - Retail Retail Competitor Pricing ZRTL_C01
InfoCube Non SAP Area - Marine Services PEDCO DTHEAD_DTTAIL ZPDCO_C01
InfoCube Non SAP Area - Marine Services PEDCO LOGSUM ZPDCO_C05
InfoCube Non SAP Area - Marine Services PEDCO MTHEAD_MTTAIL ZPDCO_C03
InfoCube Non SAP Area - Marine Services PEDCO RCHEAD_RCTAIL ZPDCO_C02
InfoCube Non SAP Area - SFIO SFIO Movement Position ZSDMVMPOS
InfoCube Non SAP Area - SFIO SFIO Movement Position History ZSDMVMHIS
InfoCube Non SAP Area - SFIO SFIO OIPR ZSD_OIPR
InfoCube Non SAP Area - SFIO SFIO OIPR History ZSD_OIPHS
InfoCube Non SAP Area - EPM (Ent. project mgmt) EPM Cube II ZEPM_C02
InfoCube Non SAP Area - EPM (Ent. project mgmt) Enterprise Project Mgmt (EPM) ZEPM_C01
InfoCube PIW - Profit Improvement Warehouse PROFIT IMPROVEMENT WAREHOUSE ZPIW_P001
InfoCube Monthly Operation Planning 3 MOP3 Pricing Marker ZMOP_CMKR
InfoCube Monthly Operation Planning 3 Monthly Operation Planning 3 ZMOP_P01
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Margin Analysis ZPCA_C04
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. PCA: Summary 1 ZPCA_C03
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. PCA: Summary 1 History ZPCA_C03H
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. PCA: Transaction data 0PCA_C01
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Customer ZPCA_C09
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Other ZPCA_C10
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Periodic Balance ZPCA_C05
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Planning Items ZPCA_C06
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Summary ZPCA_C07
InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Vendor ZPCA_C08
InfoCubeCust. Rel.Mgmt- CRM Analytics- Cross-Scenario Analyses-Case Mgmt Analysis CRM Case Management Analysis ZCRM_CASE
InfoCubeCust. Rel.Mgmt- CRM Analytics- Cross-Scenario Analyses-Activity Activities 0CSAL_C01
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD Billing Cube ZSD_CVF0
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD Billing Cube - History ZSD_CVFH1
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD Billing Summary ZSD_C17
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD Billing: Condition Data Cube ZSD_C06
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD Billing: Tax Conditions Cube ZSD_C06C
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD Daily Lift Report ZSD_ZS561
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD Delivery Cube ZSD_CVL0
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD Delivery Cube - History ZSD_CVLH1
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD PAWS: Pricing Analysis ZPAWS
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD PAWS: Pricing Analysis (1) ZPAWS1
InfoCubeERP Analytics- Sales & Distribution Analyses - SAP R/3 SD PAWS: Pricing Analysis Archive ZPAWS2
InfoCubeERP Analytics- Sales & Distribution Analyses - Commercial Excellence CE Pricing Log ZCE_C01
InfoCubeStrategic Enterprise Mgmt - BPS - Capital Investment Planning Capital Investment Planning ZBPS_P05
ODS = Operational Data Store
33
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
34
Global Performance Issues
• Global projects have many performance challenges Hardware must be optimized for 24/7 operations and
network capabilities BI system must also have optimal backup and disaster recovery
windows and rapid data load processing• Since most of the query rendering time is spent on the
database, it is important that global projects spend serious time optimizing the database reads by leveraging tools such as the BI Accelerator and the classical SAP NetWeaver BI aggregates
The correct definition of global aggregates can dramatically reduce the time spent on database reads by the BI queries, thereby delivering faster query results to the users in various geographical locations.
The correct definition of global aggregates can dramatically reduce the time spent on database reads by the BI queries, thereby delivering faster query results to the users in various geographical locations.
35
Tip 15: Performance Enhancements Are Available — Use Them
• Check indexes periodically Under RSA1 Manage Performance
• Check database statistics to route queries faster At this company, 50% of the InfoCubes
had outdated database statistics that should be updated
• For large InfoCubes, or cubes with many users, the percentage used to build the database statistics can be increased to 15 - 20%
• May yield improved query routing
Name Technical name
Indexes Aggregate index
Stats build
DB Stats
COPA(US) : P&L L'Oreal R110: 2006 YCPAPL_1 10%
COPA(US) : P&L L'Oreal R110: ACD 2007 ZCPAPLA07 10%
COPA(US) : P&L L'Oreal R110: LPD 2007 ZCPAPLL07 10%
FI-AR Accounts Receiv. Line Item IC YCZO_1 10%
FI-AR Historical Indicators US ZCARHI_1 10%
FIAR (CS) : Cube - Hist. indicators Zoom YCZOHI_1 10%
Agreement YC13_AGR 10%
Cancellation and rejection YC11_CR 10%
Carry Over ZCSD_CROV 10%
Consolidated Open Orders ZC_OO 10%
Consolidated Open Orders YC_OO 10%
Delivery YC12_DEL 10%
Historical Invoice LPD ZCHSTLI 10%
Invoice YC13_INV 10%
RGA Data ZC_RGADTL 10%
Sales Order History ZCORDINV 10%
Sales order YC11_ORD 10%
Service rate YC11_SR 10%
Invoice: ACD 2004 ZCINVA04 10%
Invoice: ACD 2005 ZCINVA05 10%
Invoice: ACD 2006 ZCINVA06 10%
Invoice: LPD 2004 ZCINVL04 10%
Invoice: LPD 2005 ZCINVL05 10%
Invoice: LPD 2006 ZCINVL06 10%
Sales Order: ACD 2006 ZCORD_A06 10%
Sales Order: ACD 2007 ZCORD_A07 10%
Sales Order: LPD 2006 ZCORD_L06 10%
Sales Order: LPD 2007 ZCORD_L07 10%
Sales Order History: ACD 2006 ZCODI_A06 10%
Sales Order History: ACD 2007 ZCODI_A07 10%
Sales Order History: LPD 2006 ZCODI_L06 10%
Sales Order History: LPD 2007 ZCODI_L07 10%
Invoice: ACD 2007 ZCINVA07 10%
Invoice: LPD 2007 ZCINVL07 10%
Delivery: ACD 2007 ZCDELA07 10%
Delivery: LPD 2007 ZCDELL07 10%
Service Rate: ACD 2007 ZCSRIA07 10%
Service Rate: LPD 2007 ZCSRIL07 10%
36
Tip 15: Performance Enhancements — Aggregates Are Often Incorrectly Built (Real Example)
• Several cubes have no aggregates, while others can benefit from generating new proposals
• A score above 30% for average aggregate valuations should be a target for a data store
37
Tip 15: Performance Enhancements — Correct Aggregates Are Easy to Build
• This example shows the benefits of aggregates by using system statistics to generate proposals This very large InfoCube had over 160 queries attached to
it and only one aggregate
• Select the run time of queries to be analyzed (e.g., 20 sec)
• Select time period to be analyzed Only those queries executed in
this time period will be reviewed to create the proposal
• High value aggregate proposal (users who had queries that ran over 20 seconds during the last six months, would have benefited from 438 times — each query execution)
38
Tip 16: Memory Cache Is Often Set Too Low
• The cache settings at many companies are too low For example, at one large company the cache was set at 100 MB for
local and 200 MB for global cache, which is too low for a system with thousands of users and 4 TB of data
• Review the settings with the Basis team and look at the available hardware During the review, use the transaction code RSCUSTV14 in SAP
NetWeaver BI to increase the cache if needed Focus particularly on the global cache
• To monitor the usage of the cache, use transaction code RSRCACHE and also periodically review the analysis of load distribution using ST03N – Expert Mode
Example: At one company over 61% of all 35,644 navigation steps in a month accessed the database instead of the cache, even after the query was executed.
Example: At one company over 61% of all 35,644 navigation steps in a month accessed the database instead of the cache, even after the query was executed.
39
Tip 17: Use the BEx Broadcaster to Pre-Fill the Cache
Distribution Types
By broadcasting the query result of commonly used queries to the cache, your users do not need to execute the query from the database. Instead the result is already in the system memory (much faster).
By broadcasting the query result of commonly used queries to the cache, your users do not need to execute the query from the database. Instead the result is already in the system memory (much faster).
40
Tip 18: Focus Performance Enhancements on Large InfoCubes
• Typically 20% of the cubes and queries will see 80% of the usage Focus on these, when performance tuning, or it can become
overwhelming• InfoCubes with over 100 million rows should be analyzed for
performance enhancements, such as broadcasting of queries to cache, aggregates, indexes, database stats, etc.
InfoCube Non SAP Area - SFIO SFIO OIPR History ZSD_OIPHS x x 157,218,110InfoCube Non SAP Area - SFIO SFIO OIPR ZSD_OIPR x x 157,131,010InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Customer ZPCA_C09 x x 119,969,043InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Vendor ZPCA_C08 x x 116,068,014InfoCube Fin. Mgmt & Controlling - Profit Center Acct. Profit Center Accounting - Other ZPCA_C10 x x 116,012,946InfoCube Fin. Mgmt & Controlling - Profit Center Acct. PCA: Summary 1 ZPCA_C03 x x 108,784,493
No of RowsIndex StatsTech NameInfoAreaType Name
Avoid building local queries that are unique to the various geographies or business units. Focus instead on the reusability of global queries that are saved as local views (favorites).
Avoid building local queries that are unique to the various geographies or business units. Focus instead on the reusability of global queries that are saved as local views (favorites).
41
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
42
Tip 19: Avoid “Swiss Army Knife” MultiProviders: It Will Be Slow!
• There is a temptation to use only a few MultiProviders to build all queries This may dramatically slow down queries Hint: This design may prevent queries from being executed
in parallel
Avoid attempting to build a single MultiProvider to do all functions of the data warehouse
43
Tip 19: “Swiss Army Knife” MultiProviders and Parallel Processing
• To avoid an overflow of the memory, parallel processing is cancelled as soon as the collected result contains 30,000 rows or more and there is at least one incomplete subprocess The MultiProvider query is then restarted automatically and
processed sequentially What appears to be parallel processing corresponds to sequential
processing plus the preceding phase of parallel processing up to the termination
• Generally, it’s recommended that you keep the number of InfoProviders of a MultiProvider to no more than 10
Avoid creating InfoCubes that are specific to one country or business unit. If you do this, you will have to later combine them in large MultiProviders that may cause slow performance.
Avoid creating InfoCubes that are specific to one country or business unit. If you do this, you will have to later combine them in large MultiProviders that may cause slow performance.
44
Tip 19: “Swiss Army Knife” MultiProviders and Parallel Processing (cont.)
• Consider deactivating parallel processing for those queries that are MultiProvider queries and have large result sets With SAP BW 3.0B SP14 (SAP BW 3.1 SP8), you can change the
default value of 30,000 rows — refer to SAP Notes 629541, 622841, 607164, and 630500
• A larger number of base InfoProviders is likely to result in a scenario where there are many more base InfoProviders than available dialog processes, resulting in limited parallel processing and many pipelined sub-queries
45
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
46
Tip 20: Clean Up Old Objects That Are No Longer Used
• After a few years of running SAP NetWeaver BI, companies often have many objects that are no longer used
• Cleaning them up makes for a simpler development environment, which is easier to navigate and has a positive impact on analysis on when designing new objects
Keep your environments clean of obsolete junk.Keep your environments clean of obsolete junk.
47
Tip 21: Don’t Replicate Legacy Data Logic in SAP NetWeaver BI
• It’s tempting to replicate the SAP transaction system in the BI environment
• You can spot this when you find many ODSs that serve as lookup tables (easy to recognize, since these systems have many ODSs and each of them have few fields)
Number of fields per ODS
0
2
4
6
8
10
12
14
0-5 6-10 11-15 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 90+
Nu
mb
er o
f O
DS
s
Don't allow the BI system to become a replication of the transaction system, merely because that is what your developers know.
Don't allow the BI system to become a replication of the transaction system, merely because that is what your developers know.
48
Tip 22: Avoid Direct Querying of ODS
• To quickly solve local reporting requirements, it is tempting to bypass the global InfoCubes and query the data stores (ODSs) directly — this is a mistake
• ODSs are for central staging of historical data, detailed analysis, lookups, and data mining — not for local reporting
• If you allow the users to bypass the global InfoCube logic, you will find yourself supporting slow local queries that do not take full advantage of the inherent performance of the OLAP processor
• Many local detailed queries will consume system resources, causing other global users to suffer
• If you follow this path, you will be asked to create InfoSet queries and eventually have minimal advantages of standard content, which will lead you to the road to build a traditional legacy data warehouse
Restrict the number of users that can access the ODSs.Restrict the number of users that can access the ODSs.
49
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
50
Tip 23: Be Aware of Divergent Country Accounting Rules
• Different accounting standards must be dealt with on global projects For example, pension obligations are considered long-term debt in
most European countries and long-term liabilities in the US (unsecured)
• Accounting rules for depreciation, amortization, depletions, and allocations vary from country to country For example, the US allows for accelerated depreciation in
certain instances
Plan on involving accountants and mapping tables if you are consolidating reporting for both local and global purposes.
Make sure you have an audit trail of transformations.
Plan on involving accountants and mapping tables if you are consolidating reporting for both local and global purposes.
Make sure you have an audit trail of transformations.
51
Tip 24: Standardize Currency and Units of Measures
• A global BI requires standardized currencies and Units of Measure (UOM) for aggregate reporting
• Conversions from pounds to kilos, or from liters to gallons are easy, but you have to decide how to handle currencies Possible solution A: Some companies use a pro-forma currency
translation rate for reporting during an interim period and a real currency translation rate when the financial books are closed or funds are transferred
Possible solution B: Some companies use previous month’s currency translation rate for reporting during an interim period and a real currency translation rate when the financial books are closed or funds are transferred
Sites like www.xe.com can provide you pro-forma currency translation daily, but you have to make a decision.
Sites like www.xe.com can provide you pro-forma currency translation daily, but you have to make a decision.
52
Tip 25: Determining Hierarchies — Global vs. Local
• On a global project you can create local and global hierarchies For example, a large European telecom company created a five-level
sales material hierarchy in BI that reflected the Dutch reporting view, a seven-level hierarchy that reflected the German reporting view, and a four-level hierarchy for corporate reporting
This allowed reporting for each subsidiary and a new corporate reporting hierarchy that was not available in any local system
• Lessons learned: You don’t have to select a single hierarchy; local needs can also be accommodated
Trick: MultiProviders can be used to mask the complexity for the query developers.
Trick: MultiProviders can be used to mask the complexity for the query developers.
53
What We’ll Cover …
• Introduction• Scoping the SAP NetWeaver BI project• Getting the project started • Designing InfoCubes — the good, the bad, and the ugly• Improving Performance — what to do, and how to do it• Understanding the curse of MultiProviders• Cleaning up and ODS management• Considering global integration points• Wrap-up
54
Resources
• Steve McConnell, Rapid Development (Microsoft Press 1996, ISBN: 1556159005)
• Jeremy Kadlec, Start to Finish Guide to IT Project Management (NetImpress 2003, ISBN: B0000W86H2, Digital, 109 pages)
• COMERIT.NET (presentations, tools and acccellerators) www.Comerit.net
55
Resources (cont.)
• European privacy laws www.msnbc.msn.com/id/15221111
• European Commission’s Directive on Data Protection www.export.gov/safeharbor
Whereto
FIND it
56
7 Key Points to Take Home
• Be sensitive to other cultures and don’t enforce US working methods
• More than one methodology is available for you to use• Long-term environment sizing and planning is critical • Performance tuning is not an afterthought, but a project task• An SAP NetWeaver BI architecture should be formulated
before a project starts• Plan for BI cockpits and dashboards — they are not nice-to-
haves, but a critical part of BI• Get the global team active and develop the SAP NetWeaver BI
solutions for local and global needs