28
Larry Sackett MDX Reporting and Analytics with SAP NetWeaver ® BW Bonn � Boston
Larry Sackett
MDX Reporting and Analytics with SAP NetWeaver® BW
Bonn � Boston
249_Book.indb 3 8/5/09 12:22:16 PM
Contents at a Glance
1 Why MDX Matters ................................................................. 29
2 Fundamentals of MDX OLAP Structures ................................ 43
3 Basic MDX Statement ............................................................. 61
4 MDX Set Functions ................................................................. 79
5 MDX Member and Tuple Functions ........................................ 107
6 Numeric, Logical, and String Functions .................................. 125
7 MDX and Time Functions ........................................................ 141
8 MDX Query Context ............................................................... 157
9 Advanced Topics in MDX ........................................................ 171
10 SAP NetWeaver BW: The Data Warehouse System ............... 203
11 Data Modeling in SAP NetWeaver BW .................................. 227
12 Query Performance in SAP NetWeaver BW ............................ 261
13 Excel 2007 and SAP NetWeaver BW ...................................... 283
14 Panorama NovaView ............................................................... 305
15 Cognos 8 BI with SAP NetWeaver BW ................................... 329
16 SAP BusinessObjects Roadmap and MDX .............................. 351
A MDX Functions ....................................................................... 367
B The Author ............................................................................... 379
249_Book.indb 5 8/5/09 12:22:16 PM
7
Contents
Introduction and Overview ........................................................................ 15
1 Why MDX Matters ................................................................... 29
1.1 The Business Case for MDX ......................................................... 291.2 What Makes MDX Unique ......................................................... 301.3 How MDX Navigates External SAP NetWeaver BW Hierarchies ... 331.4 History of MDX with SAP ........................................................... 371.5 SAP Adopts ODBO Standard ...................................................... 381.6 MDX with BusinessObjects ........................................................ 391.7 Alternatives to SAP BEx and SAP BusinessObjects ...................... 411.8 Summary ................................................................................... 42
2 Fundamentals of MDX OLAP Structures .................................. 43
2.1 Introduction to MDX Structures and Objects .............................. 432.2 MDX Objects and Definitions ..................................................... 452.3 DalSegno Company Reporting Cube ........................................... 522.4 Flat InfoCube Dimensions ........................................................... 552.5 DalSegno Reporting Cube with MDX .......................................... 572.6 Summary .................................................................................... 59
3 Basic MDX Statement .............................................................. 61
3.1 MDX Test Editor ......................................................................... 623.2 The MDX WHERE Clause – Slicing the Cube ............................... 663.3 Calculated Members and Named Sets ......................................... 693.4 Support for SAP NetWeaver BW Variables .................................. 743.5 Solve Order ................................................................................ 753.6 Dimension Properties ................................................................. 763.7 Summary .................................................................................... 78
249_Book.indb 7 8/5/09 12:22:17 PM
8
Contents
4 MDX Set Functions .................................................................. 79
4.1 Set Functions .............................................................................. 804.1.1 Constructing Simple Sets Using Comma and
Colon Operators ............................................................. 804.1.2 Sets Created with the .Members Function ...................... 814.1.3 Returning Sets from a Parent Member Using the
.Children Function .......................................................... 834.1.4 Crossjoining Two Sets .................................................... 844.1.5 Using the NON EMPTY Keyword to Remove
Empty Cells .................................................................... 854.1.6 Using the Asterisk (*) Operator to Crossjoin Sets ............. 864.1.7 Functions that Compare Members of Different Sets ........ 874.1.8 Filtering Sets by Search Conditions ................................. 914.1.9 Filtering Sets with TopCount, TopSum, and TopPercent ... 924.1.10 Finding Descendants at Lower Levels in a Hierarchy ....... 964.1.11 Ordered Sets .................................................................. 1044.1.12 MDX Functions Can Be Combined Like Lego Bricks ........ 105
4.2 Summary .................................................................................... 106
5 MDX Member and Tuple Functions ......................................... 107
5.1 Member Functions ...................................................................... 1075.2 Combining Member Functions for Advanced Analytics ................ 1115.3 Member Functions for Time Analysis .......................................... 1145.4 Tuple Function .Item() ................................................................ 1185.5 Conditional Function IIF ............................................................. 1225.6 Summary .................................................................................... 124
6 Numeric, Logical, and String Functions ................................... 125
6.1 Mathematic Operators ................................................................ 1256.2 Basic Numeric Functions ............................................................. 126
6.2.1 Count Function ............................................................... 1266.2.2 Average Function ............................................................ 1276.2.3 Median Function ............................................................ 1276.2.4 Sum Function ................................................................. 1276.2.5 Rank Function ................................................................ 128
249_Book.indb 8 8/5/09 12:22:17 PM
9
Contents
6.2.6 Min and Max Functions .................................................. 1316.2.7 Aggregate Function ........................................................ 132
6.3 Logical Functions ........................................................................ 1346.3.1 IsEmpty, IFF, and NOT .................................................... 1346.3.2 CoalesceEmtpy ............................................................... 1366.3.3 AND, OR, and XOR ........................................................ 1376.3.4 Boolean Operators and Comparisons .............................. 139
6.4 String Functions .......................................................................... 1396.5 Summary .................................................................................... 140
7 MDX and Time Functions ......................................................... 141
7.1 Finding the Current Period .......................................................... 1417.2 LastPeriods ................................................................................. 1437.3 Rolling Average ........................................................................... 1447.4 PeriodsToDate ............................................................................ 1447.5 ParallelPeriod .............................................................................. 1477.6 Putting It All Together ................................................................ 1517.7 OpeningPeriod and ClosingPeriod ............................................... 1527.8 PrevMember ............................................................................... 1537.9 Min and Max Values Over Time .................................................. 1547.10 Summary .................................................................................... 155
8 MDX Query Context ................................................................. 157
8.1 OLE DB for OLAP Specification ................................................... 1578.2 MDX Metadata in Schema Rowsets ............................................ 1588.3 MDX Query Resolution ............................................................... 164
8.3.1 The FROM Clause Is Evaluated First ................................ 1648.3.2 Next, the WHERE Clause and Axes Definitions
are Evaluated .................................................................. 1648.3.3 Dependencies in the WITH Clause .................................. 165
8.4 Problems with Query Context ..................................................... 1668.4.1 Invalid WHERE Clause .................................................... 1668.4.2 Ambiguous .CurrentMember .......................................... 167
8.5 Summary .................................................................................... 169
249_Book.indb 9 8/5/09 12:22:17 PM
10
Contents
9 Advanced Topics in MDX .......................................................... 171
9.1 Generate Function ...................................................................... 1719.2 Infinite Recursion ........................................................................ 1739.3 Statistical Functions .................................................................... 176
9.3.1 Covariance ...................................................................... 1789.3.2 Correlation ..................................................................... 1789.3.3 Linear Regression ............................................................ 1799.3.4 Standard Deviation ......................................................... 182
9.4 Utility Time Series Dimension ..................................................... 1849.5 Dimension Properties ................................................................. 1889.6 Automatic Calculations Using Multiple Time Hierarchies ............. 1929.7 Calculations at Different Levels of a Hierarchy ............................. 1939.8 MDX User Interface Functions .................................................... 195
9.8.1 DrillDownMember ......................................................... 1959.8.2 DrillUpMember .............................................................. 1979.8.3 ToggleDrillState .............................................................. 1989.8.4 DrillDownLevel ............................................................... 1999.8.5 The Top and Bottom Drill-Down Functions ..................... 200
9.9 Summary .................................................................................... 201
10 SAP NetWeaver BW: The Data Warehouse System ................. 203
10.1 Why This Matters to You: Knowledge is Power ........................... 20410.2 SAP NetWeaver Platform ............................................................ 20510.3 SAP NetWeaver BW .................................................................... 20610.4 ETL in SAP NetWeaver BW ......................................................... 20810.5 Getting Data into SAP NetWeaver BW ........................................ 21010.6 Data Integration and Transformation ........................................... 21310.7 Overview of DalSegno ETL Process ............................................. 217
10.7.1 DataSource ..................................................................... 21910.7.2 InfoPackage .................................................................... 22010.7.3 Transformations .............................................................. 22010.7.4 Data Transfer Process ...................................................... 222
10.8 Getting Data Out of SAP NetWeaver BW .................................... 22210.8.1 Data Mart Interface ........................................................ 22310.8.2 Open Hub Destination .................................................... 223
249_Book.indb 10 8/5/09 12:22:17 PM
11
Contents
10.8.3 Microsoft Connector 1.0 for SAP NetWeaver BW ........... 22510.9 Summary .................................................................................... 225
11 Data Modeling in SAP NetWeaver BW .................................... 227
11.1 Why Data Modeling Matters to You? .......................................... 23011.2 Multidimensional Data Model .................................................... 23111.3 InfoObjects ................................................................................. 233
11.3.1 InfoObject Naming Conventions .................................... 23311.3.2 Defining InfoObjects as Characteristics ........................... 23411.3.3 Key Figures .................................................................... 23811.3.4 Attributes ....................................................................... 238
11.4 InfoCubes ................................................................................... 24011.5 MultiProviders ............................................................................ 24111.6 Slowly Changing Dimensions ...................................................... 24211.7 Hierarchies .................................................................................. 24411.8 Balanced, Unbalanced, and Network Hierarchies ........................ 24511.9 Virtual Time Hierarchies .............................................................. 24711.10 External Hierarchy: Product Group/Product ................................. 24911.11 Summary .................................................................................... 259
12 Query Performance in SAP NetWeaver BW ............................. 261
12.1 Hardware Sizing .......................................................................... 26212.2 Database Tuning ......................................................................... 26512.3 Analytical Tools ........................................................................... 26712.4 Data Modeling ............................................................................ 269
12.4.1 Dimension Size .............................................................. 26912.4.2 Line Item Dimension ...................................................... 26912.4.3 Fact Table Partitioning .................................................... 27012.4.4 Fact Table Compression .................................................. 27012.4.5 Aggregates ..................................................................... 27012.4.6 DataStore Objects .......................................................... 271
12.5 Analyzing Individual Queries ....................................................... 27312.6 The OLAP Cache ......................................................................... 27812.7 SAP NetWeaver BW Accelerator ................................................. 27912.8 Summary .................................................................................... 281
249_Book.indb 11 8/5/09 12:22:17 PM
12
Contents
13 Excel 2007 and SAP NetWeaver BW ........................................ 283
13.1 Connecting to NetWeaver BW .................................................... 28613.2 Designing a Pivot Table ............................................................... 29213.3 Pivot Charts ................................................................................ 29713.4 OLAP Formulas ........................................................................... 29813.5 Conditional Formatting ............................................................... 30113.6 Excel Services .............................................................................. 30213.7 Summary .................................................................................... 303
14 Panorama NovaView ................................................................. 305
14.1 A Brief History of Panorama and MDX ........................................ 30614.2 NovaView Web Client ................................................................. 30714.3 NovaView MDX Editors .............................................................. 313
14.3.1 Basic MDX Editor in Web Client ..................................... 31314.3.2 Advanced MDX Editor in the Analytics Client ................. 31514.3.3 Defining Parameters ....................................................... 318
14.4 NovaView Smart Reports ............................................................ 32114.5 Drill Through to Detail ................................................................ 32414.6 NovaView Dashboard ................................................................. 32614.7 Security and Caching ................................................................... 327
14.7.1 Security .......................................................................... 32714.7.2 Caching .......................................................................... 327
14.8 Summary .................................................................................... 328
15 Cognos 8 BI with SAP NetWeaver BW .................................... 329
15.1 Cognos Server ............................................................................. 33215.2 Dimensional Functions in Cognos ............................................... 33515.3 Reporting and Analytics in Cognos 8 BI ....................................... 340
15.3.1 Report Studio ................................................................. 34115.3.2 Analysis Studio ............................................................... 344
15.4 Security and Caching ................................................................... 34815.4.1 Security .......................................................................... 34815.4.2 Caching .......................................................................... 349
15.5 Summary .................................................................................... 349
249_Book.indb 12 8/5/09 12:22:17 PM
13
Contents
16 SAP BusinessObjects Roadmap and MDX ............................... 351
16.1 SAP BusinessObjects Frontend BI Tools ....................................... 35316.2 SAP BusinessObjects Web Intelligence (WebI) ............................ 35416.3 Semantic Layer (Universes) ......................................................... 35516.4 Xcelsius ...................................................................................... 35716.5 Crystal Reports ........................................................................... 35916.6 Pioneer ....................................................................................... 36116.7 SAP BusinessObjects Explorer (Polestar) ...................................... 36116.8 Enhancements in Interoperability ................................................ 36216.9 Summary .................................................................................... 364
A MDX Functions .................................................................................... 365A.1 Arguments .................................................................................. 365A.2 Functions .................................................................................... 366
B The Author ........................................................................................... 377
Index ......................................................................................................... 381
249_Book.indb 13 8/5/09 12:22:17 PM
29
MDX is an extraordinarily powerful query and calculation language for both advanced analytics and self-service reporting. It significantly extends the capabilities of SAP NetWeaver BW, making InfoCube data available and useful to users from the boardroom to the shop floor.
Why MDX Matters1
The Business Case for MDX1.1
At the end of the day, business intelligence is about getting the right information into the right hands at the right time to make faster, more profitable business decisions.
A manufacturer’s product manager wants to identify the company’s 10 best custom-EE
ers and, in turn, the top 5 products purchased by these customers.
A financial analyst wants to determine the percent contribution to sales for each EE
of his company’s products to their product subcategory, each subcategory to the product category, and ultimately each product category, subcategory, and product to overall sales.
A sales manager is curious to know the sales rank for each member of her sales team EE
for sales year-to-date versus their ranking for year-to-date sales for the same period last year.
An inventory manager has to track changes to inventory as of the closing period EE
for each of the past 12 months, and he’d also like to know mid-month if any stock-keeping units (SKUs) are dropping below predetermined inventory levels.
A company’s CEO wants to identify potentially at-risk customers, those that have EE
substantially reduced spending in current periods compared to earlier periods.
A risk management officer for an investment bank has to compare dozens of risk EE
measures from the most current close of business to the previous close of business, and earlier dates, for compliance reports and to quickly spot anomalies.
All of these individuals want the ability to get this information through self-service web portals, dashboards, or automated daily emails. Equally important, they want the abil-ity to interact with the data, drill down from summary to detail level, pivot rows and
249_Book.indb 29 8/5/09 12:22:20 PM
30
WhyMDXMatters1
columns, swap or nest dimensions to view results from different perspectives, or slice data to focus on individual products, customers, or time periods.
MDX is uniquely suited to these purposes. That’s why, in less than a decade, MDX has become the de facto standard for reporting and analysis in business intelligence (BI) sys-tems. It is supported by virtually every major vendor of BI servers, including Microsoft, SAP, Cognos (IBM), Hyperion (Oracle), Infor, and SAS.
What Makes MDX Unique1.2
MDX, which stands for MultiDimensional eXpressions, has become popular for two pri-mary reasons: the growth of the BI industry itself, and MDX’s unique ability to navigate, query, and perform calculations against multidimensional structures.
The growth of BI has been fueled by the recognition that business users need fast and intuitive access to company information to make more profitable business decisions. This industry encompasses data warehouses, OLAP databases, and reporting and analy-sis tools.
OLAP stands for online analytical processing, and OLAP databases are usually referred to as cubes. The basis of an OLAP cube is the multidimensional database model, which consists of measures and dimensions joined through a central fact table. Multidimen-sional databases are also called star schemas because dimensions are arrayed around a fact table in a star-like structure (Figure 1.1). They are designed to give users direct access to information that is understandable, quick to find, and easy to use.
SAP NetWeaver BW Enhanced Star Schema
While still encompassing fact tables and dimensions, SAP NetWeaver BW enhances the basic star schema model with links from dimension tables to master data (texts and attri-butes) and external hierarchies. Dimension tables in an InfoCube don’t actually contain data; instead they contain surrogate ID (SID) links to the master data and hierarchies. This is covered in more detail in Chapter 11.
Multidimensional OLAP databases are fundamentally different from the OLTP (online transaction processing) databases used in transaction systems. Instead of the star schema, an OLTP database has a schema that looks more like an intricate spider web, with hundreds of tables joined together through a complex set of primary and foreign key relationships. Designers of OLTP systems, such as SAP, worry more about making database updates and inserts lightning-fast than making them easy for reporting. Data in relational tables is often labeled using coding techniques that make the field names short and unique but that lack descriptions to make the tables more understandable to business users.
249_Book.indb 30 8/5/09 12:22:20 PM
31
WhatMakesMDXUnique 1.2
Time
Date IDWeekMonthQuarter
Year
Channel
Channel IDChannel Description
Fact Table
Customer IDProduct ID
Time IDChannel IDScenario ID
Net SalesUnit SalesDeals Lost
Scenario
Scenario IDBudgetActual
Forecast
Customer
Customer IDCustomer Description
AreaRegionCountry
Product
Product IDProduct Description
Product group
Simple Star Schema Database DesignFigure 1.1
Designers of OLAP systems are more concerned with the information and reporting needs of business users. OLAP systems are designed to:
Extract information of valueEE to business users from the source transaction systems.
Transform the informationEE so that it is easily understood, including creating new measures, adding intelligent descriptions to codes, and standardizing information, for example, putting addresses into U.S. Postal Service format.
Load the dataEE into a multidimensional database so that users can view measures by a wide variety of dimension characteristics, such as sales by product, region, time, and sales channel.
Frequently updateEE information in the OLAP database, typically on a daily basis, but also hourly, and in some cases, within minutes of a transaction.
Make it fast and easyEE for end users to get information relevant to their day-to-day jobs.
Many dimensions in an OLAP database have multilevel hierarchies, making it possible to drill down from summary data to more detailed information. Examples of multilevel dimension hierarchies include time, with drill down from year to quarter to month to day; customer, with drill down from county to region to area to customer; and product, with drill down from product group to product.
This emphasis on multilevel hierarchies and drill down leads to the second reason for MDX’s popularity: It is uniquely able to navigate, query, and perform calculations against multidimensional structures. These include OLAP cubes, dimensions, hierar-
249_Book.indb 31 8/5/09 12:22:21 PM
32
WhyMDXMatters1
chies, levels, attributes, members, and measures. Although it is theoretically possible to query an OLAP cube using SQL, SQL has no inherent knowledge of dimension hier-archies and cannot easily navigate to the parent or children of a particular dimension member.
MDX, however, recognizes all kinds of dimension relationships: from ancestors to descendants, from members to siblings, and from the dimension root (the All level) to the leaf level (most granular data). If you know a particular dimension member, then MDX lets you determine all of these relationships with very little coding. For example, if you know a particular product, you can query for that product’s subcategory or its category with just the Parent or Ancestor MDX functions.
Meta data functions such as Parent, Children, Descendants, and ParallelPeriod enable you to relate levels and members without explicit reference, much like how a cell in an Excel spreadsheet can be related to another cell.
MDX consists of more than 150 expressions that cover mathematical, statistical, logical (Boolean), conditional, member, set, time series, and navigation functions. MDX expres-sions can be combined to create advanced calculations and filters, such as identifying at-risk customers who did business with you on a regular basis during the past 26 weeks, but have not during the most recent 4.
MDX includes several functions for time period comparisons. For example, Parallel-Period compares results from a current period to the same period a year ago. PeriodsTo-Date analyzes data from a particular period in the past through the most current period. LastPeriods lets a user select a set of periods from a current period to a period in the past, for example, the past 3, 6, or 12 months. Other MDX time functions include YTD, QTD, and MTD, which are shorthand for the PeriodsToDate function.
In addition to functions that enable navigation of multidimensional structures, MDX includes a wide variety of functions that support reporting and analytics. Here’s a sampling:
The Filter function lets you select members of a dimension based on Boolean expres-EE
sions, such as all customers that spent more than $500 in the past month. TopCount helps you determine your most popular products based on measures such as net sales or units sold. Related functions include TopSum, TopPercent, BottomCount, BottomSum, and BottomPercent.
CrossJoin makes it possible to nest members from different dimensions on the col-EE
umns and rows of a crosstab report. The Order function sorts dimension members by measures, such as an ordered list of products by their profitability.
Logical functions include Intermediate If (IIF) to test whether a Boolean operation EE
is true or false. Boolean operators are And, Or, Not, and Xor. IsEmpty is used to test whether data exists for a particular member in a cube. NOT and IsEmpty frequently
249_Book.indb 32 8/5/09 12:22:21 PM
33
HowMDXNavigatesExternalSAPNetWeaverBWHierarchies 1.3
are used together to determine the last day for which data has been entered in a cube.
Set functions help you select the Ancestor of a member at a given level of a multi-EE
level hierarchy. Descendants find all of the children of a particular member at a given level of a dimension hierarchy.
One of the most useful MDX functions is CurrentMemberEE . Currentmember provides implicit reference to dimension members on the axes of a crosstab or to dimensions sliced in an MDX WHERE clause. What makes CurrentMember so powerful is that you can create named sets or calculated members that specify the current member of a dimension that is sliced in a dimension, with no explicit reference to that member. That makes it possible for users to simply slice on a member to change a calcula-tion without having to modify the underlying MDX code. We’ll explore the use of CurrentMember, PrevMember, and NextMember to generalize MDX expressions in Chapter 9.
In addition to the full range of mathematical operators and Boolean functions, MDX EE
includes statistical functions such as Correlation, Covariance, Standard Deviation, Variance, and several linear regression functions.
MDX functions are like Lego® bricks. Just as you can combine Lego bricks to build an infinite number of different robots, you can combine MDX functions to build different expressions that react to users’ actions and needs.
How MDX Navigates External SAP NetWeaver BW 1.3 Hierarchies
Much of the power of MDX lies in its ability to navigate up, down, and across hier-archies. ParallelPeriod is an excellent example of an MDX function that navigates up, down, and across hierarchies. Here’s an example of how this function compares May 2005 with May 2004:
ParallelPeriod([Time].[Year], 1, [Time].[Month].[May 2005]
The first element of the expression, [Time].[Year], sets the level at which you want the function to look back. You could just as easily specify the quarter.
The second element, 1, tells the expression how many periods to look back — one year in this case.
The third element, [Time].[Month].[May 2005], identifies the reference period.
May is the fifth member in the set of months for 2005. The ParallelPeriod function will go up the set of month members for 2005 until it reaches the year level, go back one year to 2004, and then go back down the set of month members for 2004 until
249_Book.indb 33 8/5/09 12:22:21 PM
34
WhyMDXMatters1
it reaches the fi fth member, [May 2004]. ParallelPeriod and other functions such as PeriodsToDate, Descendants, and Ancestors need a hierarchy structure such as Year, Quarter, and Month in which to operate. Many SAP NetWeaver BW InfoCubes have fl atdimensions with an [ALL Level 00] and a second [Level 01] that is a list of members, as illustrated in Figure 1.2.
Unrelated DateDimensions
Unrelated Flat Date DimensionsFigure 1.2
Monthmembers
All level
Flat Dimension [0CALMONTH] Only Contains Months with No Linkage to Figure 1.3 Quarters or Years
Figure 1.2 also shows that there are a variety of date characteristics in the DalSegno Company Reporting Cube. However, because there is no structure in the cube that links
249_Book.indb 34 8/5/09 12:22:22 PM
35
HowMDXNavigatesExternalSAPNetWeaverBWHierarchies 1.3
these independent date characteristics, MDX can’t navigate from one to the other. Fig-ure 1.3 shows a fl at dimension, [0CALMONTH], in which you have the All Level and the dates with no other levels.
Fortunately, SAP NetWeaver BW includes of series of prebuilt virtual time hierarchies that can easily be activated for basic date characteristics. These are called virtual time hierarchies because they are stored in main memory rather than in the database tables. An example of an activated virtual time hierarchy is shown in Figure 1.4. The virtual time hierarchy creates a structure that MDX can navigate from month to quarter to year. Activating and using the virtual time hierarchies is covered in Chapter 11.
Virtual Time Hierarchy for Year, Quarter, and MonthFigure 1.4
With the virtual time hierarchy in place, the example in Figure 1.5 shows how the Paral-lelPeriod function works in the SAP NetWeaver BW MDX Test Editor.
The ParallelPeriod expression references the Calendar Year in the Year, Quarter, 1.
Month (YEAQUAMONTH) hierarchy and looks back to the previous year (2004).
The .CurrentMember generalizes the ParallelPeriod expression so that it will refer 2.
to each of the month members shown in the result set in the lower left side of the screenshot. The .Currentmember function will be discussed in more detail through-out this book.
NON EMPTY removes any empty cells in the result set.3.
The Descendants function is used to generate a list of all of the children (months) of 4.
the year 2005.
249_Book.indb 35 8/5/09 12:22:23 PM
36
WhyMDXMatters1
ParallelPeriod function returns the sales resultsfor each month of 2004
ParallelPeriod Used with the .CurrentMember functionFigure 1.5
When the full MDX query is executed, the ParallelPeriod function loops over each month of 2005, which are referenced by .CurrentMember, and returns the sales values for the corresponding months in 2004.
Note
As with all the code illustrations or samples shown in this book, the complete MDX expressions can be found on this book’s web page at www.sap-press.com.
The DalSegno reporting cube also contains separate dimensions for product group and product, and for country, region, area, and customer. The characteristics have to be combined in an external hierarchy to enable MDX navigation.
Non-time hierarchies can be imported from SAP transaction systems, or can be built within SAP NetWeaver BW. They are called external hierarchies because they are main-tained in external database tables. An example of an external hierarchy for product is shown in Figure 1.6. The process of creating an external hierarchy is more complex than simply activating a virtual time hierarchy and is described in depth in Chapter 11.
249_Book.indb 36 8/5/09 12:22:24 PM
37
HistoryofMDXwithSAP 1.4
External Hierarchy for ProductFigure 1.6
History of MDX with SAP1.4
MDX originated in the mid-1990s with Panorama , a small Israeli software fi rm that has since relocated to Toronto, Canada. Panorama had developed an OLAP server and wrote the fi rst iteration of the MDX calculation engine and programming language. In 1996, Microsoft was exploring options for an OLAP server to be part of SQL Server 7.0. In July of that year, Microsoft representatives met with Panorama and quickly decided to acquire the OLAP Server and the query and calculation language.
In January 1997, a small team of Panorama developers, led by Amir Netz , joined Micro-soft as part of the acquisition and relocated to Microsoft’s Redmond, WA, campus. Mosha Pasumansky , a member of the relocated team, is credited by Netz as being the major developer of what was to become MDX.
One of the new team’s fi rst assignments was to defi ne the MDX language, which became part of Microsoft’s OLE DB for OLAP (ODBO) standard . The ODBO standard is built on Microsoft’s OLE DB (Object Linking and Embedding, Database) interfaces for database access. Pasumansky did much of the work in defi ning MDX. He originally referred to MDX as Multi-Dimensional eXtensions because it was defi ned based on its mapping to the SQL query language.
In 1998 Microsoft released SQL Server 7.0, with Panorama’s original OLAP server retooled as SQL Server 7.0 OLAP Manager. (Later this was renamed Analysis Services, with the addition of data mining algorithms in SQL Server 2000). Microsoft published
249_Book.indb 37 8/5/09 12:22:25 PM
38
WhyMDXMatters1
the ODBO standard in 1997. The standard defined the interfaces for connecting to multidimensional data stores, and the definitions of MDX functions, syntax, and SQL mappings. The standard was quickly embraced by OLAP server vendors such as Hype-rion and SAS.
The published standard includes a series of mandatory and optional MDX elements. For this reason, and because of the way the SQL mappings can be interpreted, there are now different “dialects” of MDX. MDX in Microsoft Analysis Services works somewhat differently than in SAP NetWeaver BW. The same is true of the Hyperion OLAP engine owned by Oracle. This book focuses on the dialect in SAP NetWeaver BW.
SAP Adopts ODBO Standard1.5
SAP also adopted the ODBO standard, and turned to Vancouver, Canada--based Simba Technologies to build an ODBO provider for SAP NetWeaver BW. SAP concurrently built an MDX engine to work with the Simba ODBO provider. The combination created an open interface that enabled third-party BI vendors to connect to the BW system. The work was completed in late 1998. Simba continues to work with SAP to this day on upgrades to the ODBO provider, including an updated ODBO interface for Microsoft Excel 2007 to SAP NetWeaver BW. ODBO is a Windows-based interface for component object model (COM) applications.
In April 2001 the first specification for an Internet-based interface was released: XML/A (Analysis). XML/A is an extension of ODBO that supports XML via HTTP/SOAP proto-cols for Web services. The second version of the XML/A specification was published in November 2002. SAP wrote its own XLM/A interface based on the specification.
SAP also added its own interface, OLAP BAPI, which executes remote function calls (RFCs) to SAP NetWeaver BW and SAP NetWeaver BW 3.X servers. The OLAP BAPI is the primary interface for the BusinessObjects Enterprise XI frontend BI tools.
All three interfaces — ODBO, XML/A, and OLAP BAPI — go to the same MDX processor in SAP NetWeaver BW (Figure 1.7). Therefore, MDX functions and syntax are exactly the same regardless of the interface.
Starting in 2006, the SAP NetWeaver BW development team and Panorama engineers worked together to design enhancements to the MDX engine. SAP and Panorama state that nearly 40 enhancements have been made since then. SAP has implemented all mandatory elements of the original ODBO standard and the majority of the optional ele-ments. The result is a very robust set of MDX-based interfaces to SAP NetWeaver BW.
249_Book.indb 38 8/5/09 12:22:25 PM
39
MDXwithBusinessObjects 1.6
Business Explorer
Data Objects
Virtual Objects
InfoProvider
OLA
P Processor
MD
X Processor
Enterprise Portal
Third-Party BI
Tools
XML/A
ODBO
BAPI
Multi-Provider
Info-Set
Three MDX-Based Interfaces to SAP NetWeaver BWFigure 1.7
Panorama rewrote its client interface and MDX query engine to work directly with the Simba ODBO interface and the SAP NetWeaver BW MDX processor. Several demonstra-tions later in this book show how MDX queries in NovaView work with SAP NetWeaver BW, in addition to MDX queries in the SAP NetWeaver BW MDX Test Editor.
Note
SAP NetWeaver BW supports all of the mandatory elements of the ODBO specification, and the majority of optional ones. SAP added MDX functionality to support variables defined in SAP BEx Query Cubes, also referred to as InfoQueries. SAP also added nu-meric functions such as ArcSin, ArcCos, ArcTan, DIV, Log, Log10, and TRUNC that are not part of the ODBO specification. These additional functions are not discussed in this book.
MDX with BusinessObjects1.6
Following the acquisition of BusinessObjects by SAP, the newly combined companies outlined a roadmap for integrating BusinessObjects BI tools with SAP NetWeaver BW. The roadmap was published in January 2008 and updated in May 2009.
The suite of BI tools has been grouped into a common, unified BI platform called SAP BusinessObjects Enterprise XI (currently R3.1). This includes Crystal Reports for format-ted reporting, SAP BusinessObjects Web Intelligence for self-service ad hoc reporting, SAP BusinessObjects Predictive Workbench for enterprise mining and predictive analy-sis, Xcelsius for enterprise dashboards, SAP BusinessObjects Intelligent Search for search and discovery, SAP BusinessObjects Explorer (formerly Polestar) for data exploration,
249_Book.indb 39 8/5/09 12:22:26 PM
40
WhyMDXMatters1
SAP BusinessObjects Text Analysis for semantic analysis and sentiment extraction, and SAP BusinessObjects Voyager for OLAP analysis.
SAP will continue to invest in SAP NetWeaver BW, SAP NetWeaver BW Accelerator, SAP NetWeaver Master Data Management, and SAP extraction capabilities. SAP has stopped development but will continue to support the SAP Business Explorer (BEx) tools, includ-ing the SAP BEx report designer, analyzer (Excel and web), and web application designer. The SAP BEx query designer will continue to be supported indefinitely.
Crystal Reports is the suggested enterprise reporting solution. It can connect directly to SAP NetWeaver BW via the OLAP BAPI for both InfoCubes and InfoQueries. It is offered as a premium to SAP NetWeaver BW customers. The road map includes a lightweight version of Crystal Reports, to be released in 2010. Crystal Reports extracts data from SAP NetWeaver BW via the MDX interface. It includes an internal MDX query editor.
The future tool for OLAP analysis is Pioneer, a combination of the existing SAP Busines-sObjects Voyager and the SAP BEx (Excel and web) analyzer. It will be available as a web client and an Excel client. The client will be based on Ajax (Web 2.0) technology and BI Consumer Services as a universal access layer for SAP and non-SAP multidimensional data. The first shipment of Pioneer is expected in 2010. Later versions of Pioneer will leverage the relational and OLAP universes in the SAP BusinessObjects Enterprise XI semantic layer via the OLAP BAPI.
Customers have to license SAP BusinessObjects Enterprise, which provides the infra-structure for most of the client tools, such as universe repository, scheduling services, caching services, and so on. SAP BusinessObjects Enterprise must run on a server sepa-rate from SAP NetWeaver BW. There are no plans to integrate the two products.
The SAP BEx analyzer and web analyzer will be phased out. Xcelsius will be the primary product for single, stand-alone dashboards. Xcelsius connects to SAP NetWeaver BW via SAP BusinessObjects Enterprise and OLAP BAPI. At some point it will be integrated with Pioneer. Later, SAP BusinessObjects will add direct connectivity to SAP NetWeaver BW (without the use of SAP BusinessObjects Enterprise). Xcelsius will be integrated with SAP NetWeaver Visual Composer.
The SAP BEx report designer will be replaced by Crystal Reports “light” as part of the SAP NetWeaver BW license. Information broadcasting will be extended for distribution of Crystal Reports and Pioneer workspaces and workbooks.
For web- and Excel-based OLAP analysis, SAP and Business Objects recommend staying with the SAP BEx analyzer and web analyzer for new products because they provide superior OLAP features compared to SAP BusinessObjects Voyager. Projects requiring Excel OLAP integration should be done with the SAP BEx analyzer 7.0. SAP customers will be able to upgrade to Pioneer Excel.
249_Book.indb 40 8/5/09 12:22:26 PM
41
AlternativestoSAPBExandSAPBusinessObjects 1.7
Customers can still use the SAP BEx web application designer, but the roadmap rec-ommends shifting to SAP NetWeaver Visual Composer, which will be integrated with Xcelsius.
Alternatives to SAP BEx and SAP BusinessObjects1.7
This chapter would not be complete without mentioning alternatives to both the SAP BEx suite and SAP BusinessObjects.
Any third-party BI tool that adheres to the ODBO or XML/A specification can connect to SAP NetWeaver BW via those two interfaces. Companies that have already invested in OLAP reporting and analysis tools such as Cognos or MicroStrategy do not have to convert to SAP BusinessObjects. By design, the SAP NetWeaver BW development team intends to maintain the open interfaces because they recognize that some SAP clients already have substantial investments in third-party tools.
Microsoft and SAP maintain a close collaboration with products such as Duet for the Microsoft Office suite and Microsoft Reporting Services (packaged with SQL Server 2005 and 2008). The two companies partnered in the development of an SAP BW XML/A provider, which connects Reporting Services to NetWeaver BW via a special XML/A interface. Reporting Services can connect to SAP BEx–defined InfoQueries or directly to InfoCubes. The SAP NetWeaver BW XML/A provider “flattens” MDX queries for use in tabular and matrix reports. The provider also incorporates SAP NetWeaver BW variables.
Microsoft Excel 2007 also connects to SAP NetWeaver BW via the Simba Technologies ODBO provider, which was updated to provide virtually the identical functionality of Excel 2007 as when it’s connected to Microsoft Analysis Services 2005. Microsoft made substantial enhancements to Excel 2007 pivot tables, making it a viable BI tool with SAP NetWeaver BW. Users can drag and drop columns, rows, data values, and filters into pivot tables defined in the InfoQuery or available in an InfoCube. Chapter 13 of this book describes how Excel 2007 works with SAP NetWeaver BW.
Panorama’s NovaView server and BI tools connect to SAP NetWeaver BW via the ODBO interface. As noted earlier in this chapter, Panorama rewrote the NovaView client inter-face to the ODBO provider and optimized its MDX query generator to work natively with the MDX processor in SAP NetWeaver BW. According to the SAP NetWeaver BW development team, Panorama is the only third-party vendor to go to this length to work with the SAP NetWeaver BW system. NovaView’s entire user interface is based on MDX functionality. Chapter 15 covers NovaView reporting and analytics using SAP NetWeaver BW.
Microsoft Analysis Services 2005 or 2008 can extract data from SAP NetWeaver BW InfoCubes through SAP NetWeaver BW’s Open Hub Destination. Using Analysis Ser-
249_Book.indb 41 8/5/09 12:22:26 PM
42
WhyMDXMatters1
vices can be an attractive option for companies that make heavy use of the Microsoft SQL Server BI suite. Microsoft’s Integration Services, part of the SQL Server BI suite, includes an SAP NetWeaver BW connector. The downside of this approach is that none of the SAP NetWeaver BW security settings are extracted with the data, and the dimen-sions have to be redesigned in Analysis Services. Any changes made to InfoCubes also have to be taken into account in Analysis Services.
Summary1.8
MDX is the de facto standard for queries and calculations against multidimen-EE
sional (OLAP) databases, such as SAP NetWeaver BW InfoCubes and SAP BEx InfoQueries.
MDX consists of more than 150 expressions that cover mathematical, statistical, logi-EE
cal, conditional, member, set, time series, and navigation functions.
MDX supports complex business analytics and self-service reporting.EE
SAP was an early adopter of the OLE DB for OLAP standard and continues to EE
enhance its implementation of MDX. The MDX-based interfaces to SAP NetWeaver BW provide open connectivity to any third-party BI tool that adheres to the ODBO or XML/A standards.
To be most effective, MDX requires virtual time hierarchies and external hierarchies EE
to navigate multilevel hierarchies.
Third-party BI tools can be an effective alternative to the SAP BEx or SAP Busines-EE
sObjects frontend BI tools.
The SAP BusinessObjects Enterprise XI server connects to SAP NetWeaver BW EE
through the OLAP BAPI interface.
249_Book.indb 42 8/5/09 12:22:26 PM
379
A
Accelerator, 261Aggregates, 266Amir Netz, 37Analysis and repair of BI objects (Transaction RSRV), 267Application link enabling (ALE), 207Architected data marts, 207Attributes, 227, 238Axes, 50Axis dimensions, 67
B
Basic MDX statement, 61Bill Inmon, 232BI Service API, 210BI statistics, 273BlackBerry, 332Blade servers, 279Business Content (BI Content), 204BusinessObjects, 39
C
Calculated members, 50Calculations at different levels of a hierarchy, 193Characteristics, 227Characteristic values, 228Cognos 8 BI
Analysis Studio, 344Caching, 348Dashboards, 341Dimensional functions, 330Framework Manager, 330Model Advisor, 334Parameter maps, 334Report Studio, 341
Scorecards, 341Security, 348
Cognos 8 BI, 329Cognos Connection Manager, 333Cognos Server, 332Compounding, 235CONCATENATE function, 213Conditional function IIF, 122Conversion routines, 217Crystal Reports, 40.CurrentMember, 46CurrentMember Function, 33
D
DalSegno reporting cube, 57Database statistics, 265Database tuning, 265Data Mart Interface, 222Data marts interface, 208Data modeling, 269
Aggregates, 270DataStore objects, 271Dimension size, 269Fact table compression, 270Fact table partitioning, 270Line item dimension, 269
Database indexes, 265DataSources, 207, 209, 219DataStore objects, 204, 209Data transfer processes (DTPs), 209, 222Data Warehousing Workbench
Modeling section, 211, 217DATE_DIFF, 213DB Connect, 213DBACOCKPIT, 267Designing a Pivot Table, 292Dimension IDs (DIMIDs), 232Dimension properties, 76, 188Dimensions, 45, 227Dimension size, 269
Index
249_Book.indb 379 8/5/09 12:25:14 PM
380
Index
Direct Assignment, 214Distributed data marts, 223DrillDownLevel function, 294DrillDownMember function, 295
E
E. F. ìTedî Codd, 232End routines, 216Enhanced star schema, 229Enhanced Star Schema, 30Excel 2007, 283
Conditional formatting, 301Data Connection Wizard, 289OLAP formulas, 284OLAP Formulas, 298Pivot charts, 297Pivot tables, 283Sub-select query, 296
Excel 2007, 283Excel Services, 302Expert routines, 216External hierarchies
Intermediate nodes, 251Level names, 258Text node, 251
External HierarchiesLeaf-level nodes, 251
External hierarchyProduct Group/Product, 249
Extraction, transformation, and loading (ETL), 203Extractors, 208
F
Fact tableDimension hierarchies, 229Granularity, 229Measures, 229
Finding the Current Period, 141Flat file, 212
G
Generate function, 171Grid computing technology, 279
H
Hardware sizing, 262Hierarchies, 46, 227, 244
Balanced, unbalanced, and network hierarchies, 245Characteristic hierarchies, 244Hierarchical structures formed by characteristic attributes, 244Hierarchies in dimensions of an InfoCube, 244
Hierarchize function, 295Hyperion Essbase, 330
I
IBM DB2, 336Infinite recursion, 173InfoCubes, 229InfoObjects, 208, 228, 233InfoPackages, 209, 220InfoProviders, 208, 209InfoSources, 209
K
Key Figures, 228, 238
L
Levels, 46Logical functions, 134
AND, 134CoalesceEmpty, 134CoalesceEmtpy, 136IIF, 134IsEmpty, 134
249_Book.indb 380 8/5/09 12:25:14 PM
381
Index
NOT, 134NOT IsEmpty, 136OR, 134
M
Master data, 207Mathematic operators, 125MDX-based APIs
OLAP-BAPI, 208OLE DB for OLAP, 208XML/A, 208
MDX expressions, 100MDX Query Context, 157MDX Set functions, 79
Asterisk (*) operator, 86BottomCount function, 92Constructing simple sets, 80Crossjoining Two Sets, 84Descendants function, 96Distinct function, 87Except function, 87Extract function, 89Filter function, 91Intersect function, 89Order function, 104Sets created with .Members function, 81TopCount function, 92Union function, 90Using .Children function, 83
MDX Test Editor, 62, 276Available Objects pane, 64
MDX user interface functions, 195MDX WHERE clause, 66Measures, 48MEMBER_CAPTION, 162Member functions, 107
Ancestor, 110Combining member functions, 111FirstChild, 108FirstSibling, 108Lag, 116LastChild, 108LastSibling, 108Lead, 117NextMember, 109
Parent function, 107PrevMember, 109Siblings, 109
Member properties, 47Members, 46Microsoft, 336Microsoft Analysis Services, 225, 330Microsoft Analysis Services, 41Microsoft Connector 1.0 for SAP NetWeaver BW, 225Microsoft Data Access Components (MDAC), 157Microsoft Excel 2007, 41Microsoft Integration Services, 225Microsoft Reporting Services, 41Microsoft SAP Duet, 41Microsoft SQL Server, 225Mosha Pasumansky, 37MultiProviders, 241
N
Named sets, 50Navigational attributes, 227, 240Navigation attributes, 204NON EMPTY keyword, 72, 85NovaView dashboard, 326NovaView MDX Editors, 313NovaView Smart Reports, 321NovaView Web Client, 307Numeric functions, 126
Aggregate, 132Avg, 127Count, 126Max, 131Median, 127Min, 131Rank, 128Sum, 127
O
OLAP, 30OLAP BAPI, 38, 332
249_Book.indb 381 8/5/09 12:25:14 PM
382
Index
OLAP cache, 262, 278OLE DB for OLAP (ODBO) Standard, 37, 306OLTP (Online Transaction Processing), 30Open hub destination, 208, 222, 223Operational data store, 208Oracle, 336
P
Panorama NovaView, 41, 305Defining parameters, 318Drill through to detail, 324Value filters, 311
Panorama Software, 37ParallelPeriod Function, 33PARENT_UNIQUE_NAME, 162Persistent Staging Area (PSA), 207Pioneer, 40, 209Process chain, 209, 222ProClarity, 306
Q
Query context, 69Query monitor (Transaction RSRT), 267Query performance, 261
R
Ralph Kimball, 232Real-time access, 210Rolling Average, 144Rolling PeriodsToDate, 147Roll-up process, 266Routines for characteristics, 216Routines for key figures, 216RSDDSTAT, 274
S
SAP BW OLE DB Provider, 287SAP GUI, 283
SAP NetWeaver Application Server, 205SAP NetWeaver BW Accelerator, 279SAP NetWeaver BW Variables, 74SAP Quick Sizer, 263Schema Rowsets, 158Service-oriented architecture (SOA), 205, 332Simba, 283Simba Technologies, 38, 283
ODBO provider, 284Slicer, 50Slicer dimensions, 67Slowly changing dimension, 231, 242
Type III SCD, 243Type II SCD, 243Type I SCD, 243
Solve order, 75SQL joins, 267SQL query execution plan, 277SQL query optimizer, 265SQL Server 2005 Analysis Services, 283Staging BAPI, 213, 225Star schema, 229Star Schemas, 30Start routines, 216Statistical functions
Correlation, 176Covariance, 176Linear Regression, 176Standard deviation, 182
String functions, 139MemberToStr, 140SetToStr, 140StrToMember, 140StrToSet, 140StrToTuple, 140TupleToStr, 140
SUBSTRING function, 214Surrogate IDs (SIDs), 231
T
Time functionsClosingPeriod, 152LastPeriods, 143MTD, 146OpeningPeriod, 152
249_Book.indb 382 8/5/09 12:25:14 PM
383
Index
ParallelPeriod, 147PeriodsToDate, 144PrevMember, 153QTD, 146Tail, 142YTD, 146
TopCount function, 70Trace tool (Transaction RSTT), 276Transfer rules, 213Transformation Editor, 220Transformations, 209, 220TREX search engine technology, 279T-shirt sizing, 263Tuple, 48Tuple function .Item(), 118
U
UD Connect, 212Update rules, 213Using multiple time hierarchies, 192Utility time series dimension, 184
V
Virtual Time Hierarchies, 35YEAQUAMON, 247
Visual Composer, 41
W
Web Intelligence, 39Web services, 212WITH MEMBER keyword, 69
X
Xcelsius, 40XML/A (Analysis), 38
249_Book.indb 383 8/5/09 12:25:14 PM
