THE ETWEAVER CCELERATOR

W I N T E R C O R P O R A T I O NW

HIT

E P

AP

ER

S P O N S O R E D R E S E A R C H P R O G R A M

THE SAP NETWEAVER BI ACCELERATOR

Transforming Business Intelligence

Sp

ec

i al i

s ts

i n t

h e Wo r l d

’ s L a r g e s t D a t a b a s e s

http://www.wintercorp.com

411 WAVERLEY OAKS ROAD, SUITE 327WALTHAM, MA 02452

781-642-0300

RICK BURNS AND ROBERT DORIN

September 2006

©2006 Winter Corporation, Waltham, MA. All rights reserved.

Duplication only as authorized in writing by Winter Corporation.

THE SAP NETWEAVER BI ACCELERATOR

Transforming Business Intelligence


The SAP NetWeaver BI Accelerator: Transforming Business Intelligence

©2006 Winter Corporation, Waltham, MA. All rights reserved. Duplication only as authorized in writing by Winter Corporation.

Executive SummaryBusiness intelligence has become an increasingly strategic focus for enterprises over the past several years as they strive to increase profitability and control costs. The highly competitive nature of business markets mandates that information executives search for return-on-investment (ROI) opportunities from within the vast data stores accumulated from state-of-the-art operational systems. SAP NetWeaver Business Intelligence (BI) provides a sophisticated platform for implementing the BI applications necessary to achieve this ROI. But, for many customers, the full ROI potential of BI and data analytics has yet to be realized as the scope and complexity of user requirements has grown beyond initial expectations—more data, more users, and more complex, unforeseen problems to be solved.

The SAP NetWeaver BI accelerator is a new appliance-like (integrated hardware and software) product—available as an optional add-on to SAP NetWeaver BI—that offers performance improvements and query flexibility at a cost that reflects its technology foundation, Intel processors and the Linux operating system. The core value proposition of an appliance is its ease of implementation with superior price-performance. The BI accelerator relies on Intel Xeon processors, the most widely deployed server architecture in the world, to deliver the benefits of an appliance architecture. By basing the accelerator on Intel Xeon processors, SAP can efficiently and effectively offer the benefits of an appliance with the latest platform features—cost, performance, reliability, and security—embodied in Intel’s next-generation technologies.

So, while the BI accelerator incorporates the cost and ease-of-use benefits of an appliance, its specific purpose is query response acceleration—for both tuned and ad hoc queries. In April 2006, WinterCorp, an independent research and consulting firm that specializes in large-scale database scalability, worked jointly with SAP in Walldorf, Germany to run a series of performance tests in order to assess the impact of the BI accelerator. The tests were designed to examine three key areas: single query performance, multi-user scalability, and data load performance. The test results highlighted several performance characteristics of the BI accelerator.

The BI accelerator greatly expands the class of NetWeaver BI queries that perform well and predictably, without requiring customers to define, build, and maintain aggregates. Aggregates are an alternate performance-tuning feature, which is often very effective, but requires that the customer identify each usage scenario and develop highly tuned aggregates for each scenario. As noted by customers that were interviewed, this is not a realistic possibility for BI information workers seeking to find new paths of analysis to reach ROI targets. The BI accelerator is, therefore, an enabler for today’s dynamic BI paradigm. The key performance-related findings can be summarized as follows:

The BI accelerator was effective in reducing response times significantly for a broad range of queries, and is far more practical than the process of building tuned aggregates.The BI accelerator scales as the problem size grows and as the blade chassis configuration is expanded with additional blades.Data loading and indexing onto the BI accelerator occurs at impressive rates, outperforming the construction of optimized aggregates that would be required for comparable query response.

Early customers of the BI accelerator observed these performance improvements, but it was the flexibility to ask any question at any time that was cited as the most important BI accelerator benefit. If SAP and Intel, along with its platform partners, HP and IBM, can continue to deliver this value, then data will be turned into actionable information more effectively and increasingly more demanding ROI expectations will be realized.

•

•

•

3



4

Table of ContentsExecutive Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1 The Promise of the BI Appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Architecture of SAP’s Business Intelligence Accelerator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1 Basic Architecture of the BI Accelerator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 Hardware Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2.1 Intel Technology Foundation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2.2 Exploiting Blade Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2.3 HP Blade System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2.4 IBM BladeCenter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.3 System Management of the BI accelerator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.3.1 Manageability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.3.2 Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Performance of the BI Accelerator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2 Query Performance Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2.1 Standalone query comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2.2 Ad Hoc Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.3 Scalability Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.3.1 Scale-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.3.2 Scale-out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.3.3 Blade Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.4 Load Performance Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.5 Summary of Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Initial Customer Experiences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.1 BP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.2 Brown-Forman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.3 Novartis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5 Summary Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21



The Promise of the BI ApplianceOver the past several years, many enterprises have implemented data warehouses for business intelligence and realized some return on investment in the form of increased profitability or customer satisfaction, successful entry into new markets, or successes with new products. However, many implementation objectives remain unsatisfied. CIOs, today’s information czars, have become concerned about the investment, complexity, and risk involved. At the same time, the volume of information that defines the context for doing business continues to grow at an exponential rate.1 Enterprises struggle to transform torrents of detailed operational data into actionable business intelligence at affordable costs.

The appliance concept is straightforward in principle. By combining hardware and software, designed for a specific purpose—in this case, business analytics of large volumes of data for growing numbers of users—and optimized to take maximum advantage of the other, the BI appliance can offer superior performance at a lower price point, thus providing some relief to the information czars. Delivering on this promise is a significant challenge. But, since executives yearn for simplicity and manageability and they worry about cost, there is a bright future for BI appliance producers if they can deliver on their value proposition.

SAP is applying this concept to the specific challenge of speeding up query performance for SAP NetWeaver Business Intelligence (BI) users—in a manner that is fully transparent to them. The new BI accelerator is an optional product, a plug-in appliance, for SAP NetWeaver BI and, once installed, is fully integrated into the BI architecture. For a particular query, SAP NetWeaver BI checks if the BI accelerator option is available—and will bypass other slower processing options, such as the use of aggregates, if the accelerator can be employed.

The accelerator achieves query performance speedup by exploiting the efficiency of in-memory index processing—indexes that were created from InfoCubes and loaded into the accelerator’s memory. The BI accelerator, developed by SAP and Intel in partnership with HP and IBM, fulfills the cost component of the BI appliance promise running on industry-standard blade server processors using the 64-bit Linux operating system. Hardware is commodity-based; there is no database license; and there is no OS license.

5

1

1The 2005 WinterCorp TopTen Program, a periodic survey of the world’s largest databases, observes that the largest data warehouse has tripled in size every two years since 2001.



Architecture of SAP’s Business Intelligence AcceleratorSAP NetWeaver’s Business Intelligence accelerator is an optional component of SAP NetWeaver 2004s that can be added to an existing SAP NetWeaver installation at any time. The BI accelerator provides added flexibility to ask any question, anytime of NetWeaver managed data, and lowers or eliminates the costs of designing, building, and maintaining alternative performance-tuning features such as aggregates. Its use is completely transparent to analysts and applications, and it requires no changes to a customer’s existing data model. It offers high performance at commodity prices with appliance-like convenience. An examination of the product architecture provides insights regarding how these benefits are realized.

2.1 BASIC ARCHITECTURE OF THE BI ACCELERATORThe BI accelerator is a separate analysis platform, including both hardware and software, which is network attached to the SAP NetWeaver BI server via a gigabit Ethernet link (Figure 1). The BI accelerator is packaged as a high-density blade system from HP or IBM, with pre-installed SAP application software. The blades contain 64-bit Intel Xeon processors running the SUSE Linux operating system. The blade server provides a compact processor cluster, ideal for the highly parallel processing performed by the BI accelerator. The BI accelerator contains copies of selected InfoCubes, reformatted to optimize rapid search and analysis. A shared file system, accessible to all blades, provides persistent storage of the reformatted InfoCubes. The BI accelerator’s performance is the result of five key features of its architecture:

Horizontal data partitioningShared nothing parallelismVertical decomposition, or column-based storageData compressionIn-memory database processing

Horizontal Partitioning. The BI accelerator system is divided into partitions, typically one partition per CPU. For example, a 14-blade system with two processors per blade would have 28 partitions. InfoCube data is associated with either one or many partitions, depending on its size. Fact table data is distributed across all partitions. Dimension data may be partitioned, depending on the size of the dimension.

Shared nothing Parallelism. Querying and loading is divided into multiple steps, one per partition, and each step is executed concurrently by separate processes. The number of partitions determines the degree of parallelism of the accelerator. Since each step works on a separate partition of data, pre-loaded into the memory of a single blade, each step can execute independently, without interference from one another – essentially creating a shared nothing environment. As a result, a 10-blade accelerator with 20 data partitions can search an InfoCube in five percent of the time that it could previously via traditional SAP NetWeaver BI—a 20-fold speedup from parallelism alone.

Vertical decomposition. In addition to partitioning data, the BI accelerator stores data separately by column, instead of the more traditional approach of keeping all columns of a row together. Since a query typically retrieves a small subset of the columns in any row, this can yield significant performance advantages. In addition, storing columns together facilitates data compression.

•••

••

6

2

Fig. 1: Architecture of the BI Accelerator



Data compression. The accelerator compresses data using a smart, dictionary-based algorithm that represents the dictionary index in the smallest possible number of bits. When combined with column-oriented storage of data, the compression technique provides highly compact data storage with low decoding costs. In our testing, for example, 670 GB of source data occupied 55 GB in the BI accelerator, a 12x compression factor.

In-memory database. At first data access, the accelerator dynamically caches the InfoCube data and indexes of each partition into the memory of every node, in effect creating an in-memory database (Figure 2). The accelerator exploits Intel 64-bit processor technology to support large memories, currently 8 GB per blade. Subsequent queries access this in-memory database, eliminating the need to read data from disk, and insuring optimal performance. To avoid performance problems that can result from excessive page swapping, the accelerator should be sized so that the selected InfoCube data can fit within its memory.

At load time, the BI server copies data from its InfoCubes to the BI accelerator, where it is decomposed, compressed, partitioned and stored in partition files in the accelerator’s shared file system. At query time, the BI server determines whether the request involves InfoCubes indexed in the BI accelerator, and, if so, forwards such requests to the accelerator for processing. In the accelerator, data partitions are dynamically copied on demand to the local memory of the blades hosting the partition, where they can be scanned, aggregated, or analyzed in parallel on the fly. The accelerator consolidates results from participating partitions, and returns an answer set to the BI server for further post-processing and formatting.

The architecture of SAP NetWeaver BI, with the BI accelerator, has important implications regarding the types of processing it handles most effectively. The BI accelerator is optimized to resolve data intensive queries quickly. It uses a divide and conquer approach, enabled by parallel processing against its distributed, in-memory database, that is best suited for queries that sift through and analyze large volumes of data to produce a small answer set. When loading InfoCubes into the accelerator, the size of the BI server, the capacity of the network connection to the BI accelerator, and the size of the BI accelerator itself all have a significant effect on the resulting load performance. The performance test results described later in this paper explore these considerations in more detail.

2.2 HARDWARE INFRASTRUCTUREThe hardware platform that delivers the memory-intensive, highly parallel processing capability to the BI accelerator consists of blade systems from HP or IBM. These blade systems are powered by 64-bit Intel Xeon processor technology, with large distributed memory capacity, up to 8 GB per blade. They include a shared storage subsystem that provides persistent storage for accelerator indexes available to all the blades in the system.

2.2.1 Intel Technology FoundationIntel Extended Memory 64 Technology (EM64T) adds 64-bit capability to the Xeon family of processors, providing higher performance and the expanded memory required by the BI accelerator in a cost-effective package. The availability of larger memories means that larger InfoCubes can be analyzed

7

Fig. 2: Map of InfoCube Data in BI Accelerator Memory



by the accelerator, and the improved performance provided by 64-bit technology yields an accelerator platform with higher throughput, capable of processing more queries from more users.

The Intel-SAP partnership has provided more than technology to SAP NetWeaver’s Business Intelligence offering. Intel’s software engineers have provided valuable experience and insight into the exploitation of 64-bit and parallel processing technology. Intel engineers provided 64-bit porting assistance of the accelerator code base and extensive performance optimizations to the product.

Emerging multi-core technologies to be incorporated into upcoming Xeon processors will provide even better performance and throughput capabilities in the future. Coupled with memory advances, which will provide more and faster memory per blade, the next-generation BI accelerator platform will possess an increasing ability to service more complex and ad hoc analysis of larger InfoCubes by a growing population of users across the enterprise.

Intel’s processor technology provides the foundation for the blade systems from HP and IBM that are the foundation of the BI accelerator. The additional benefits provided by the blade platform architecture are described below.

2.2.2 Exploiting Blade PlatformsBlade systems offer a simple configuration of commodity compute clusters in a compact package. The blade system chassis, a 6U to 7U module (10½ to 12 inches high) that fits into a standard 19-inch rack, provides a common enclosure for the processor blades. The processor blades themselves, 64-bit Intel Xeon processor boards in the BI accelerator system, plug into the shared backplane of the blade system chassis (Figures 3 and 4). The backplane supplies power, network, and storage connections to each node, eliminating the need to run separate cables for each function to every node. An external SAN storage subsystem, attached to the blade system via a fibre channel link, provides shared, high performance and highly available storage for accelerator data.

Processor blades themselves are hot swappable, and power, cooling, and network and storage communication services are typically offered in redundant configurations to avoid single points of failure. A plug-in system management module provides centralized monitoring, management and control of the entire blade system, and integrates with popular enterprise system management environments, like HP’s OpenView or IBM’s Tivoli.

The convenience and cost effectiveness of blade computing, combined with the BI accelerator’s appliance-like architecture, provides NetWeaver customers with a high performance, flexible, and easy to use business intelligence environment at an attractive price.

8

Fig. 3: HP Blade System Chassis

Fig. 4: IBM Blade Center Chassis



9

SAP is collaborating with two blade system vendors, HP and IBM, to provide complete blade hardware solutions for the BI accelerator. To simplify installation and shorten time to value, both vendors deliver pre-configured systems with the BI accelerator software pre-loaded. These solutions are profiled below.

2.2.3 HP Blade SystemThe HP blade system for the BI accelerator includes HP ProLiant BL20p blade servers and an HP StorageWorks Enterprise Virtual Array (EVA) storage system. The blade chassis (Figure 3) holds up to eight blades. A large BI accelerator configuration spans two chassis and contains 16 blades. For each project, HP offers an individual customer sizing with regard to number of blades and storage size. Table 1 presents sample HP blade system configurations.

The storage system is a RAID level 5 configuration, and communicates to a single blade that functions as a dedicated NFS server for the remaining blades. Accelerator blades communicate with the NFS server blade over a dedicated subnet to optimize I/O throughput and avoid contention with inter-blade network traffic and communication with the NetWeaver BI server. Blades are connected via Gigabit Ethernet integrated into the backplane. The backplane connects to redundant communication interface modules, which provide links to the external network.

Table 1: Sample HP Blade Server Configurations for BI accelerator

XS S M L

HP BL20p G3 Blade Server




4 x Intel Xeon EM64T 2-way CPUs, 8 GB RAM




HP StorageWorks 4000 Enterprise Virtual Arrays

with 8 x 72 GB discs







2.2.4 IBM BladeCenterThe IBM BladeCenter system for the BI accelerator includes IBM BladeCenter servers and IBM TotalStorage 4700 storage system preconfigured in four package sizes (Table 2). The blade chassis (Figure 4) holds up to 14 blades.

The storage system is a RAID level 10 configuration (striping and mirroring), and communicates to the blade servers via fibre channel. IBM’s General Parallel File System (GPFS) provides shared access to common storage from all blades. Blades communicate with one another via Gigabit Ethernet embedded in the backplane. External network connection is mediated through communication modules that plug into the rear of the chassis.

Table 2: IBM BladeCenter Packages for BI accelerator

XS S M L

IBM eServer Blade Center Chassis




4 x Intel Xeon EM64T 2-way 3.2 GHz CPUs,

8 GB RAM


8 GB RAM


8 GB RAM


8 GB RAM

IBM TotalStorage DS4700 Storage System w 7 x

73 GB, 10K RPM disks

IBM TotalStorage DS4700 Storage System w 14 x 73 GB, 15K RPM disks







2.3 SYSTEM MANAGEMENT OF THE BI ACCELERATOR2.3.1 ManageabilityManagement of the BI accelerator is integrated into the management workbench of SAP NetWeaver BI. The management view provides screens for configuring the accelerator system and its connection to the NetWeaver BI server, and for detecting and diagnosing problems. This view complements the hardware management facilities provided with the blade system.

Common configuration settings, such as the number of accelerator communication processes or RFC servers, are automatically managed, as befits an appliance-like product. In addition, common failures, such as loss of connection between the BI server and the accelerator, are automatically detected and repaired. Detailed accelerator monitoring and diagnosis tools are also available for use by accelerator experts, usually SAP technicians.

SAP’s efforts to make the BI accelerator easy to install, easy to configure, and easy to fix at product launch demonstrate a commitment to ease of use, and auger well for full realization of the appliance concept in the future.

2.3.2 AvailabilityThe ability to recover from a failure is critical to the system availability of the accelerator. All hardware components, except for processor blades, are configured redundantly to avoid a single point of failure. Individual blade failures are immediately detected and the accelerator system management software suggests an appropriate manual workaround. In-flight queries using data partitions owned by the failed blade are aborted and will have to be manually restarted once the problem is repaired.

Repairs are implemented easily. Blades are hot-swappable. If a spare blade is available, the failed one can simply be replaced, and processing then resumed. Alternatively, the partitions owned by the failed node can be redirected to a surviving node via partition reorg command and straightforward metadata change. Processing can then be resumed.

2.4 SUMMARYThe NetWeaver BI accelerator is a powerful analytic engine that sits alongside a BI server and offers fast response to complex, ad hoc queries against large InfoCubes, without requiring the effort to pre-compute aggregates. It relies on blade systems to provide extensive parallelism and large memory space in a compact, appliance-like package. This architecture is ideal for analyzing a large volume of data quickly to return a small answer set.

In the next section, we explore the performance characteristics of the BI accelerator architecture in more depth.

10



Performance of the BI AcceleratorTo analyze the impact of the BI accelerator on the performance of SAP NetWeaver BI, WinterCorp worked jointly with SAP NetWeaver engineers to run a series of performance and scalability tests. These tests were executed at SAP Labs in Walldorf, Germany in April 2006.

3.1 OVERVIEWTests were designed to examine three key areas: query performance, multi-user scalability, and data load performance. The query performances tests compared the performance of NetWeaver BI with and without the BI accelerator attached. In each case, we ran a diverse set of queries covering broad business questions like inventory tracking and management, order fulfillment costs by customer and product class, and sales order summaries with detailed drill-through by region and industry.

These queries consisted of 72 variations on 14 different query templates that required up to five-way joins, a number of aggregate computations, complex where clause restrictions, and up to 17 columns returned per row. In addition, we used these queries to explore variations in query characteristics that might commonly occur with ad hoc queries. Specifically, we varied the amount of data that a query needs to scan or analyze to obtain its answer, and we varied the amount of data that a query returns.

The scalability tests examined two questions. First, how does query throughput change as workload grows? For this question, we varied the workload by incrementally increasing the number of concurrent query streams running the query mix. Second, how does changing the size of the BI accelerator affect the throughput of a given workload?

The load tests determined the rate at which the accelerator can index InfoCubes, to be compared with the performance rate for building aggregates in NetWeaver BI.

The test data consisted of three years of sales and distribution data from an actual NetWeaver customer, a U.S. manufacturer, with their existing data model. The data included 1.3 billion records totaling 670 GB arranged in 9 InfoCubes. A single MultiProvider provided a common view across all InfoCubes.

We ran all tests on a 14 blade, IBM BladeCenter system. Each blade contained two 64-bit Intel Xeon 3.8 GHz processors and 8 GB of memory, totaling 28 CPUs and 112 GB of memory. For scalability testing, we reconfigured the system to disable a subset of the blades. The accelerator system was attached via Gigabit Ethernet to a four-CPU Sun server, acting as the BI database server.

3.2 QUERY PERFORMANCE TESTS3.2.1 Standalone query comparisonsThe first test measured the response time of each of 72 queries in two NetWeaver BI scenarios, one without an attached BI accelerator but with appropriate traditional aggregates, and one with a 14-blade BI accelerator attached, with no aggregates. Figure 5 illustrates two key findings from this test. First, in some cases, the BI accelerator does not outperform a NetWeaver system running without the accelerator. This is true for lightweight queries using well-tuned aggregates. For example, query 5 touches only 65 rows, and query 20 touches only 31 rows. In these cases, the queries run in a few seconds, often in less than one second. Given the overhead of transmitting the query to an external analytic server and returning the answer set (Figure 8), it is unlikely that the accelerator could speed up overall processing for such lightweight queries. On the other hand, when using the accelerator, these queries also finish quickly, usually in less than three seconds, even though they use no pre-computed aggregates. In other words, queries running on the accelerator compute whatever aggregates they need on the fly, in parallel, in roughly comparable time to running them on a standard NetWeaver BI system with expensive, optimized aggregates.

11

3



Figure 5: Query Response Time Statistics

The second important finding from this test is that the accelerator dramatically reduces the variability of response time for all the queries. Without the accelerator, but with a large number of aggregates, response times for the 72 queries ranged from a fraction of a second to 45 seconds—yielding a large standard deviation (Table 3). Running with the accelerator, on the other hand, all queries completed in less than five seconds, with most finishing in less than three seconds. The low standard deviation that results reflects the uniformity of response time, and contributes to a mean response time of less than 1 second, four times better that the mean obtained while running without the accelerator.

Table 3: Query Response Time Statistics

12

without BI accelerator with BI accelerator

Mean 3.5 sec. 0.9 sec.

Standard Deviation 6.6 0.8

The results of this query performance test demonstrate two key benefits of the BI accelerator. First, the BI accelerator increases query flexibility. It expands the user’s ability to ask any question at any time. Second, it introduces more predictable query performance. Minor query variations should no longer yield wildly differing response patterns, making reporting and analysis flexible and practical.

3.2.2 Ad Hoc QueriesTo explore the BI accelerator’s query flexibility a bit further, two classes of query variations likely to occur in ad hoc queries were examined. First, query response time was measured as the number of rows that the query must scan or analyze increases. A single query that first returned a small number of rows was chosen, and by incrementally relaxing query restrictions, the data it touched was expanded from 40 million rows to more than 400 million rows (Figure 6). As the number of rows touched was increased by a factor of ten, both the time spent in the accelerator (red line) and the total query response time (blue line) grew fairly linearly, while the non-accelerator time, a constant amount of work across all



queries, consumed a constant amount of time. This demonstrates excellent performance and scalability of the accelerator as the volume of query processing inside the accelerator increases.

Figure 6: Response Time by Data Touched

For the next test, the amount of data returned was varied (Figure 8). Again, a single query was chosen, and by relaxing some parameters, the number of rows that the query returned was incrementally increased from 1,000 to more than 100,000. In this test, while the amount of work performed in parallel by the accelerator grew (blue line), the amount of work that was performed in the BI server, outside the parallel accelerator environment, increased by a factor of 100. This produces a rapid increase in total response time (blue line) as the serial BI server activity grew to dominate response time performance, especially as the number of rows returned exceeded 10,000.

The results of these ad hoc query tests helps to illustrate two aspects of the BI accelerator architecture. First, the BI accelerator is best suited for data intensive queries that return a small set of rows. This type of query best exploits the ability of the parallel accelerator engine to divide and conquer – to split a large problem into multiple tasks that it can run concurrently (Figure 8). On the other hand, queries that return a large number of rows or that require extensive post-processing in the BI server, inherently serial processing activities, obtain a lower overall response time benefit. Note however, that such queries still benefit from the BI accelerator. In these tests, the query that required 51 seconds to return 100,000 rows when run with the accelerator (Figure 7), was canceled after 10 minutes when run without the accelerator.

Figure 7: Response Time by Data Returned

13



Figure 8: BI Accelerator Query Architecture

3.3 SCALABILITY TESTSTo test the scalability of the BI accelerator, multiple streams of the 72-query set were fed concurrently to NetWeaver BI via Mercury Interactive’s LoadRunner, a popular query driver. The query driver was configured for a minimal inter-query delay, or “think time,” of 10 seconds, and the number of concurrent streams varied between 10 and 50. If one assumes that an average user submits 10 queries per hour, this test would be equivalent to the system workload produced by 300 to 1,800 concurrent users. The scalability test suite was run on three different sized accelerator systems—6 blades, 10 blades, and 14 blades. We measured the effect on performance of increasing the workload for a fixed sized system, or “scale-up”, and the performance of a given workload for systems of different sizes, or “scale-out”.

3.3.1 Scale-upTo measure the ability of the accelerator to handle increasing workload, a series of tests was run on a 14-blade system, tracking the system throughput, defined as queries per hour, while incrementally increasing the workload from 10 to 50 concurrent streams (Figure 9). The actual queries per hour measurement was compared at each workload level (blue line) to a theoretical baseline that plotted 75% linear scalability, a fairly standard lower bound of effective scalability. In this test, the accelerator-based system topped out at over 8,000 queries per hour, and demonstrated better than 75% linear scalability until reaching CPU saturation at the highest workload levels.

Figure 9: Query scalability

14



15

3.3.2 Scale-outNext, the throughput and response time were measured as the same workload was run on different sized BI accelerator platforms. Three platform sizes were tested—6 blades, 10 blades, and 14 blades. At each platform size, a series of tests was executed, incrementally increasing the workload from 10 to 50 concurrent streams (Figures 10-11). The results of these tests demonstrate significantly better throughput and response time as the system size increases from 6 blades (blue line) to 10 blades (red line) to 14 blades (green line).

When examined closely, the results reveal some important nuances. The tests show better scalability going from 6 and 10 blades than when increasing from 10 to 14 blades. For example, with 20 concurrent threads, the throughput increased from 3,000 queries per hour at 6 blades, to 4,600 queries per hour at 10 blades (92% linear scalability), to 5,200 queries per hour at 14 blades (75% linear scalability). This demonstrates reasonable scalability for a newly introduced product like the BI accelerator, but leaves room for scalability improvement in future releases of the accelerator.

Figure 10: Platform scalability - Throughput

Figure 11: Platform scalability – Response time

3.3.3 Blade UtilizationOne key to effective scalability is the ability of a parallel product to divide work equally among the cooperating parallel workers. The following analysis shows that the BI accelerator does an excellent job of performing this task. The accelerator creates a separate data partition for each CPU, and each CPU hosts a parallel query worker to process that partition of data. On a 14-blade system, the work



on each query is divided among 28 parallel threads, two threads per blade. If the work is successfully split equally among the parallel workers, each worker will expend roughly the same resources and finish at roughly the same time. If we can accomplish such even distribution of work, the query will not have to wait for straggling processes to complete, and will yield optimal performance.

To determine distribution of work across the blades, CPU utilization was measured for each blade as a percentage of available CPU resources running a 30-stream scalability test on a 14-blade accelerator platform (Figure 12). The tight band shown in the graph indicates that the blades are sharing work more or less equally, demonstrating effective parallelization based on this significant scalability indicator.

Figure 12: Blade Utilization

3.4 LOAD PERFORMANCE TESTSBefore the query performance and scalability results can be realized, the data must be transferred and indexed onto the BI accelerator. In order for business intelligence queries to use current data, the accelerator must be loaded quickly. The next set of tests measured the rate that accelerator indexes were built from data in the nine InfoCubes residing on a four-CPU BI database server, loading into different sized accelerator platforms (Figures 13-14).

The results of these tests demonstrated the ability to load and index approximately 200 million rows per hour, even on the 6-blade accelerator platform. For the test data, this translates to 22-24 GB per hour. Given that the source BI server was relatively small, these results are impressive. Typically, large NetWeaver sites ingest 20-25 GB of new data per day. These load test results indicate that customers would be able to update their accelerator indexes in roughly an hour. More than likely, this is far less time than it takes to update the aggregates that would be necessary to insure adequate performance in the absence of the BI accelerator.

16

Figure 13: Load Rates in Rows per Hour

BIA Size (Number of Blades)

Figure 14: Load Rates in Gigabytes per Hour

BIA Size (Number of Blades)



Architecturally, accelerator load performance depends on both the ability of the BI server to feed data to the accelerator at a sufficiently fast rate and the ability of the BI accelerator to ingest that data and rapidly index it (Figure 15). In other words, the parallelism and CPU performance of the BI server is as important to load speed as the size and performance of the BI accelerator itself. Load performance will vary depending on the size of the BI server, the network bandwidth between the BI server and the BI accelerator, the size and structure of the InfoCubes, and the scale of the BI accelerator. Actual load rates depend on maximizing the performance of the BI Server feeding the load process. SAP and its hardware providers can provide guidance on configuring your system to maximize data feed rates from the BI server.

Figure 15: BI Accelerator Load Architecture

3.5 SUMMARY OF TEST RESULTSIn summary, these test results highlight several performance characteristics of the BI accelerator.

The BI accelerator greatly expands the class of NetWeaver BI queries that perform well and predictably, without requiring customers to define, build, and maintain aggregates. It performs especially well with queries that scan and analyze a large volume of data to produce a concise answer set. The accelerator provides less performance improvement for queries that return a large answer set or which require extensive post-processing by the BI server. But as long as these queries require touching lots of data, the BI accelerator still provides a significant performance advantage. For lightweight queries, those that touch relatively little data, the accelerator provides comparable performance, but without expensive, optimized aggregates.The BI accelerator provides reasonable scalability as the problem size grows, and as the platform is expanded to add blades up to the limit of a single blade chassis. Effective scalability across multi-chassis configurations has not yet been proven though it is certainly an objective for the future.Data loading and indexing onto the BI accelerator occurs at impressive rates, in many cases significantly outperforming the construction of optimized aggregates that would be necessary to deliver comparable query performance.

•

•

•

17



Initial Customer ExperiencesSeveral customers have installed and tested the BI accelerator in a production application environment prior to its release for general availability in June 2006. As part of its performance assessment of the BI accelerator, WinterCorp interviewed three of these “ramp up” customers. This section summarizes the experiences of these early BI appliance customers.

4.1 BPBP is one of the world’s largest energy companies employing over 96,000 people and operating in over 100 countries worldwide. BP handles tens of millions of transactions a day across the globe and is currently running 10 live instances of SAP NetWeaver BW 3.x, with one version of NetWeaver 2004s and the BI accelerator in pilot operation. Two projects, supporting the Refining and Marketing and the Exploration and Production segments of BP, will go live later in 2006 and early 2007. BP also plans on consolidating its multiple instances of NetWeaver BI into a single instance per business segment.

The business drivers that led BP to select NetWeaver 2004s included the enhanced usability of the front-end toolset and the new integrated planning capabilities, as well as the need for greater performance and scalability. BP’s BW data warehouses currently include 10 Terabytes (TB) of data, supporting 6,000-8,000 users—and both data and user volume are expected to double in the next few years. The performance challenges at BP include:

Data loading—with extensive requirements for data aggregates, load times up to 24 hours are sometimes exceeding available time windows.Query processing—ad hoc and interactive queries can take many minutes, e.g., non-optimized basket analysis can yield 15+ minute run times.Scalability—as data grows to 20 TB and the user BI footprint expands throughout the enterprise, substantial performance improvements are required.

Since March 2006, BP has been testing the BI accelerator, first loading 60 million financial accounting records into a BW cube. The data was loaded from the cube into the BI accelerator at one million rows per minute, a significant improvement and well within the time available. BP developed a control query that accessed 10 million rows, representing all of 2004 data. The query ran in 770 seconds. After building finely tuned aggregates, the query ran in 3.4 seconds and accessed only 140,000 rows. With the BI accelerator’s in-memory index processing, the query took less than 2.6 seconds, without aggregates, effectively accessing all 10 million rows. One of the major benefits for BP is the ability to examine all its data for trend analysis. This kind of ad hoc, drill-down query activity is difficult or impossible to support via tuned aggregates. “You cannot predict user behavior. This is where BI accelerator makes the difference,” said Elie Alam, head of BP’s Management Information Centre of Excellence. “This tool can offer our disparate user groups innovative ways to analyze our massive data assets, and allow them to make faster decisions that increase revenue opportunities and reduce costs.” Based on the initial testing performed, BP is confident that the BI accelerator will meet its data loading, query processing, and scalability challenges.

BP’s BI accelerator is implemented on an HP BladeSystem, consisting of four blades, each with two Intel Xeon 3 GHz processors and 8 GB of RAM. This was the first SAP NetWeaver BI accelerator installation at a customer site.

•

•

•

4

18



4.2 BROWN-FORMANBrown-Forman is one of the largest American-owned companies in the wine and spirits business with $2.1 billion of its $2.7 billion consumer products revenues in 2005 coming from the sales of wine and spirits. Brown-Forman implemented SAP R/3 in 1998 and has added Human Resources, Business Warehouse (BW), and Supply Chain functionality to its core Enterprise Resource Planning (ERP) system with a total of 1,000 SAP users. In 2003, Brown-Forman made a strategic commitment to use BW as its Enterprise Data Warehouse, but two areas of concern remained—the robustness of the BW front-end reporting tools and the response time for ad hoc analysis against large and complex data sources. There was pressure within Brown-Forman to consider other approaches for its newest data analytics applications unless these issues, particularly the query performance problem, could be effectively addressed.

The production data warehouse at Brown-Forman includes more than 2.5 TB of data—one-half coming from external (downstream sales) data sources. Two-thirds of Brown-Forman’s BW workload is ad hoc queries, and while heavy usage of data aggregates satisfies the performance requirements of canned reporting applications, ad hoc queries can run for 15 seconds up to several minutes, which would not meet the requirements of the new BI applications. NetWeaver 2004s, with its new front-end toolset, satisfied Brown-Forman’s reporting and tools-related concerns, and the BI accelerator was seen as a potential solution to the ad hoc query performance dilemma.

To test the BI accelerator, Brown-Forman replicated their entire SAP NetWeaver BW 3.5 data warehouse on a NetWeaver 2004s test system and migrated one of their largest MultiProviders to a BI accelerator environment. This MultiProvider was constructed from five InfoCubes and 160 aggregates containing more than 650 million records that occupied over 300 GB of storage. The BI accelerator configuration consisted of a five blade HP BladeSystem, each blade containing two 3.8 GHz Xeon processors and 8 GB of RAM.

To test standard and ad hoc queries, Brown-Forman ran seven multi-step queries with the BI accelerator enabled, and compared the results against NetWeaver both with and without aggregates. On average, the BI accelerator outperformed NetWeaver with tuned aggregates by a factor of 5, and NetWeaver without aggregates by a factor of 8. For some short queries, the BI accelerator produced higher response times compared with tuned aggregates. For several complex queries, the BI accelerator significantly outperformed tuned aggregates, by up to 50 times. The BI accelerator produced better response times in all cases compared with querying the base InfoCubes, by as high as several hundred times.

Brown-Forman also measured the time to build BI accelerator indexes and the time to build NetWeaver aggregates. For the full database of 650 million records, BI accelerator indexes were built in 6.5 hours compared to 44 hours to build aggregates. Nightly updates of BI accelerator indexes could be performed in less than one minute whereas aggregate updates ran in 30-60 minutes.

The results of these tests convinced Brown-Forman that the BI accelerator offered a solution to their query performance problem, as described by Marc Baldwin, Manager of Business Intelligence at Brown-Forman. “With the front-end tool enhancements of NetWeaver 2004s, our business users agree that the end-user interface issues are solved. And the response times for ad hoc analysis that we observed with the Accelerator will allow me to put the performance issue to bed with my user base. I anticipate greater usage by our business analysts, which should lead to more valuable business insights.”

19



4.3 NOVARTISNovartis is a world leader in pharmaceuticals and consumer health, operating through 360 independent affiliates in 140 countries. Novartis Pharma division is currently running six regional Business Information Warehouse systems for operational reporting that will be consolidated into a global reporting infrastructure based on NetWeaver BI. Novartis is depending on SAP’s BI accelerator to meet its performance and flexibility requirements. More than 1,000 users will be performing ad hoc reporting and analysis on the consolidated global system.

The BI accelerator that Novartis tested was implemented on a 6-blade platform. For the test, a cube of 10 million rows of consolidated financial data was used. The indexing and loading into the BI accelerator was very fast, loading 10 million rows in approximately 10 minutes. Query performance speedup varied depending on the amount of front-end processing required (which is not affected by the BI accelerator). Data management access (back-end only) was up to 102 times faster with the BI accelerator; overall query response time speedup ranged from 3-24 times faster, with less speedup when large numbers of rows were returned for OLAP processing.

For Novartis, the biggest concern is the number of users and the unpredictability of the queries. According to the Olivier Debluts, the Project Leader of BI Solutions, “It’s very difficult to optimize the hundreds of aggregates needed to cover our performance requirements. We simply don’t know what the user community is going to do. It’s not so much about the volumes; it’s about very efficient ad hoc reporting.” Novartis plans to go live on the BI accelerator in November 2006.

20



Summary ObservationsDespite the technology advances of recent years—faster processors, cheaper memory, faster and cheaper disks—executives still struggle to meet the ROI objectives of corporate BI initiatives. Data volumes seem to grow faster than the ability to build scalable, flexible, and affordable BI solutions. The SAP NetWeaver BI accelerator appears to be delivering on the promise on the appliance concept—dedicated commodity-based hardware and open source software delivering ease of use and low cost—with the flexibility, performance, and scalability required to meet strategic BI objectives.

WinterCorp’s tests of the BI accelerator showed that predictable response times for a broad range of queries could be achieved without the need to anticipate user behavior and build data aggregates to cover all known query scenarios. Furthermore, tests showed that performance scaled effectively as the multi-user workload was increased and as the platform configuration—an Intel Xeon processor-based blade server system from HP or IBM—was expanded. User experiences with the BI accelerator prior to its release for general availability have furthered the expectation that the accelerator will transform the SAP NetWeaver BI experience.

Any query at any time at affordable costs? This is the holy grail for BI analysts seeking to turn data into actionable information for competitive advantage. If SAP and its partners can continue to meet their objectives, the BI accelerator will allow analysts, information workers and their CIOs to meet and exceed the ROI goals of corporate BI initiatives.

5

21

A leading center of expertise in very large databases,

WinterCorp provides services in

consulting, research, architecture and engineering.

We help users and vendors understand their opportunities;

select their database and data warehouse platforms;

define and measure the value of their strategies, architectures and products;

plan, architect and design their implementations;

and manage their scalability, performance and availability issues.

Our focus is databases near, at and beyond the frontier of database scalability.

©2006 Winter Corporation, Waltham, MA. All rights reserved.

Duplication only as authorized in writing by Winter Corporation.

411 WAVERLEY OAKS ROAD, SUITE 327WALTHAM, MA 02452

781-642-0300

v i s i t us a t www.wintercorp .com



Date post:	28-Jan-2022
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

THE ETWEAVER CCELERATOR

Documents