Add A BW System to An R/3 Source With No Down Time! Some companies use more than one SAP BW system connected to a lone R/3 instance to satisfy their organizational reporting requirements. Right from the start, there are logistical difficulties deploying and synchronizing data in multiple BW systems and the ERP source. And there is an ever-present risk that users will be forced to endure significant system outages to ensure that the BW systems remain in synch with the transactional system. Initialization is the process of preparing a BW system before performing a delta update loads from the transactional system. Most organizations rely on delta extraction mechanisms to load data from R/3 into BW because a “delta” offers a streamlined ETL approach and eliminates much of the overhead of a full data load. Delta updates allow current data in an InfoProvider to be subsequently refreshed without dumping all the historical data and starting from scratch. Typically, initialization requires a large amount of historical data to be loaded during the first load process when bringing an additional BW system online. Despite all efforts to avoid it, it is necessary to reinitialize occasionally during the BW system lifecycle due to design and requirement changes, and source data extraction failures. If, for example, a single delta request is lost, it might be impossible to recover the associated source data without a complete re-load. During the first load or a re-load, initialization requires a significant amount of processing time and it chews up resources in both the transactional (R/3) and BW systems. During the initialization process, other functions and jobs in both systems may experience diminishing performance or service interruption. Regardless of the number of BW instances, organizations try to achieve a ‘near-zero’ time gap between their R/3 and BW system. Data must be continuously updated in BW data warehouse as it is added, changed, or deleted in the R/3 system. Delta updates are the most efficient way to keep the systems synchronized so they are the preferred method. We’ll show you how BW systems have been improved to better accommodate delta extractions and make the initialization process more efficient. Then we’ll take a look at how you can use some of this enhanced functionality to add another BW system to your existing SAP landscape without losing access to the R/3 system. The initialization process we describe is applicable for established BW instances as well as when your adding a new one to your SAP landscape Help on the Way As the BW system and related R/3 plug-in features, tools and techniques have evolved, several options are available now that can help you simplify the initialization process. Relative to delta update functionality, perhaps the most important upgrade arrived with plug-in PI 2002.1. SAP delivered in this plug-in
three new update methods: Direct delta, Queued delta and Unserialized V3-Update. In this article we refer to the Queued delta method. Updated extractors that feed R/3 data to BW now allow direct load into ODS objects. ODS Objects may play multiple roles in the BW architecture, however, in the scenario we present later, it is used as the basic element of a data warehouse layer. In this role, it is intended to receive data from the transactional source system and to filter duplicate records from this source. The ODS allows us to extract the same record (or same sets of records) multiple times without causing problems related to key figures (such as double-counting). Note: Techniques described in the scenario later in this article cover ‘Queued Delta’ updates only. Other update types have other requirements. Your system must be able to accommodate Queued Delta updates in order to use the approach in this article. Data requests in BW can be loaded in BW as “Initial”, “Delta”, or “Full”. BW 3.x releases now also offer the “Repair Full” request feature. A Repair Full request is an unchecked request, which allows a BW system to avoid the data integrity check applied to normal full loads. However, while this is a useful feature, it also poses a risk to data integrity. If, for example, only Repair Full loads are used to load data, the data integrity has to be enforced by other means, external to BW, since the system will skip the check. Repair Full request allows data to be loaded to an ODS Object, prior to initialization with data, while still allowing it to be used with future delta loads. When a ‘traditional’ full load is applied to an ODS that is already receiving delta loads, the ODS Object will no longer be allowed to receive any further delta loads (without re-extraction and re-initialization). This is the system method of enforcing data integrity. Prior to this new ‘Repair Full’ functionality, if any data in an ODS had to be corrected, a complete re-initialization and re-load would be required before any delta updates could be run again for the ODS. This was a high-cost option in terms of effort and time. Adding a BW System to R/3 Now that you understand how the technology has evolved, let’s look at what you can do with these enhancements. In our example, one BW system is in place and receiving delta loads from a R/3 transactional system (Figure 1). A second BW system is connected to the landscape and the initialization process commences. When understood correctly, this approach can be applied in multiple other scenarios, like data correction. It certainly can be used when you first implement a BW system in your landscape, but it is most powerful when the objective is to minimize the disruption of the OLTP system.
However, this procedure can not be used to directly initialize an InfoCube. In order to avoid this kind of source system disruption, the BW data architecture must be designed accordingly – a layer of properly-configured ODS InfoProviders to receive the data from the OLTP. Subsequent loads within BW can provide data into InfoCubes and other InfoProviders from this ODS InfoProvider (global data warehouse) layer. When there is a large reporting user population on the original BW system, there is no margin for causing data errors, nor is it feasible to require re-initialization of the data sources feeding data to those users. Note: In most cases, the technique we describe is applicable whenever starting a new BW system against an operational R/3 system. Key concerns in this scenario include affecting the existing BW system (BW1) resulting in missed data that forces re-initialization procedure, which is not allowed. There is also a risk of denying end-users access to the R/3 system during extraction resulting in a system outage that could last as long as several days! Figure 1 One BW system receiving delta loads from a R/3 system While the data initialization process always requires serious planning, it is especially true for Logistics (LO) applications in the R/3 system. LO extractions involve large amounts of data, and the extractors in the LO Extraction Cockpit are not always fool-proof because historical data can be altered in the transactional system. Initialization is complicated further by the need to minimize (if not avoid completely) disturbance to the transactional system. In the approach described and illustrated in this article, we show you how to run your Initialization and delta update in parallel with the transactional activity so the R/3 system is completely undisturbed.
Design Requirements As we indicated earlier, the solution we present here applies to queued delta updates only. Other update types require special considerations that are not covered in this article. It is critical that the data sources being initialized are activated and set up for queued delta updates or this technique will not work. There are several design requirements for the target ODS: A) It must have a key, correctly chosen to store data at the individual record level. For example, in case of Sales Delivery Item, the key fields of the ODS are Delivery (0DELIV_NUMB), and Delivery Item (0DELIV_ITEM) and the Source System ID if you have multiple source systems. These fields are sufficient to uniquely identify one delivery item from another. B) In the update rules for the ODS, the update mode must be set to Overwrite for all key figures in order to properly handle the duplicate records sent from R/3. This is maintained in the update rules for each key figure in the ODS. C) The 0RECORDMODE InfoObject in BW is mapped to the R/3 source field ROCANCEL in the transfer rules of the InfoSource feeding the inbound ODS. The ROCANCEL field value is assigned by the R/3 extractor and controls how the ODS engine handles incoming delta records. If this field is not mapped, the delta loads may incorrectly update into the ODS, thereby improperly allowing records deleted in R/3 to still remain visible in BW. The subsequent delta generated by the ODS in the Change Log will be wrong, and consequently the data targets updated from the ODS will receive incorrect data. Tip: Do not use the automatic activation option when setting up the ODS objects in BW. Activation is carried out in a separate step, after all historic data is loaded into the ODS Object, thereby improving load performance. 4-Step Solution Now that we have laid out the challenges and problems, here is the solution. It allows the initialization of an attached BW system without requiring a transactional system outage and without affecting any other existing BW systems currently drawing data from the same transactional system. Step 1: Initialize Data Sources The first step is to initialize the appropriate data sources for the new BW system you are bringing online (BW2) using the Initialize without Data Transfer option.
This is done from within the BW Initialization Data Package by checking the “Initialize Without Data Transfer” checkbox on the “Update” tab.
After the initialization without data transfer function is executed, a new entry appears in the delta queue for the BW2 system’s data source. One can see the entry in transaction code RSA7 in the R/3 source system. After the initialization without data transfer option has been selected, all data from the update jobs (V3 update will be captured in the delta queue.
Figure 2 All data from the update jobs is captured in the delta queue after setting the initialization without data transfer option At this point, the delta queue in the R/3 source system reflects the initialization from BW2. Figure 2 shows the data source (2LIS_12_VCITM) is initialized for the two BW systems (BW1 and BW 2). The delta queue entries for each BW system operate separately. Each system can now request data independent from the other.
Step 2: Fill the Extraction Set-up Tables The BW2 system is now ready to receive delta loads. In this step, you must delete the contents of the extraction set-up tables in the R/3 system before filling them again. Tip! It is critical to empty the set-up tables prior to filling them again, otherwise it is highly likely that data will be duplicated in those tables. Setup tables do not reject duplicates. If the load is not correctly planned, it is possible to get same set of data multiple times. However, this does not affect the approach presented in this article, as we can accept duplicate records in the ODS Objects (designed for Overwrite update). Regardless, tt is a good practice, from performance point of view, to empty the setup tables prior to re-filling them. To accelerate the set-up process, several separate set-up jobs can be executed with discrete document number ranges or date ranges where applicable (Figure 3). Figure 3 Several set-up jobs can be executed to accelerate the set-up process
You should beware that because the delta queue is initialized for BW2, and the set-up jobs pull all historic data up to the most current documents, there is no risk of missing any documents created by R/3 users. This is critical because, as a result, there is no need to block postings in the R/3 applications, or to enforce an R/3 system outage on the R/3 users! Step 3: Repair Full Loads into BW2 Once the initialization without data transfer process is complete, the delta queue begins collecting delta records for the BW2 system (Figure 4). The Repair Full loads can bring all historic data up through the current records into the BW2 inbound target ODS objects. Figure 4 The delta queue collects delta records for BW2 when the initialization without data transfer process is complete Step 4: Begin Delta Updates into the BW #2 System At this point, all historic data has been loaded and activated in the BW2 inbound target ODS. All new or delta document records collecting in the R/3 system delta queue the BW2 data sources are ready to be loaded. Assuming that the V3 update jobs are regularly scheduled, there will be several delta entries. The next step is to schedule your delta update InfoPackage for BW2 data sources (Figure 5). This is accomplished by running Delta extraction from the update tab of the DataPackage.
Once the BW2 delta update InfoPackage is successfully loaded into the inbound target ODS Object, it will appear in the Manage tab screen (Figure 6) Figure 6 - The BW2 delta update InfoPackage appears in the Manage screen after it is loaded into the inbound target ODS Object You may now proceed with the subsequently scheduled delta updates to this ODS Object in the BW2 System, without any negative effects on any other existing Production BW systems, or on the R/3 transactional users.
Timeline: The steps for this procedure are summarized in the timeline in Figure 7. Timing the steps is critical to making this process work properly..
Figure 7 Step for adding and initializing BW2 [Sidebar] How Are Full Loads Created as ‘Repair Full Loads’? In the InfoPackage creation screen, accessed from the RSA1-the Administrator’s Workbench in BW, select the Repair Full Request option in the Scheduler menu. Follow the steps displayed in the following graphic:
Last V3 before Init
Init without data transfer
Extraction Set-up Tables Load
R/3 is active and constantly generating transactions, thereby generating new ‘delta’ records for transfer to BW.
In the ‘Manage’ function of the ODS Object, the completed Repair Full Loads will appear as shown in the following image: Further Comments It is important to understand how and when document records in the SAP R/3 system are 'captured' into the delta queue for transfer into BW. As shown in the graphic below, 'delta' records (created by the system when R/3 documents are created, changed or deleted) begin collecting in the “update queue”. These delta records can contain either new documents, or changes to previously existing documents. When the Initialization without data transfer process is executed, it initializes the delta queue entries for that BW system in the SAP R/3 system. Delta records continue to collect in the SAP R/3 system. When the job to fill the application Setup Tables on the SAP R/3 system begins, it will collect all R/3 transaction records relevant to that application area. This will create document records in the Setup Tables that overlap with the same documents held in the delta queue/update queue.
By the timing of the Repair Full Load strategy, the first documents to be updated into BW are those collected into the application Setup Tables. It is possible that the 'image' of a given document in the Setup Table may not be the 'original image' that was created and captured in the delta queue. In such circumstances, the following example describes how the R/3 and BW systems will communicate to handle this data timing issue. [See diagram below] Example 1: Sales Order 12345 is created originally at 9am (before the setup table load started) with a value of 800kg Order Quantity, and the image captured in the Setup Table has a modified value of 1000kg. The R/3 and BW systems will handle it in the following manner: Obviously, in order for the Setup image to have a different key figure value than what was originally created at 9am, there must have been a change between 9am and the end of the setup table load (around 7PM) - thus triggering a delta record which is captured in the delta queue. As such, the BW system first receives the setup table image of the Sales Order (1000kg). The BW system then receives the first image of the Sales Order 12345 in the delta and then reverts the record in BW to match it. Now the Sales Order has the original image which contains the Sales Order with a value of 800kg Order Quantity. The BW system then receives the second image from the delta, which contains the Sales Order in its final image (1000kg). Thus, the flow of delta data brings the resulting record to the state that was captured in and transferred from the Setup Table. Therefore, the net result to BW is the same, regardless of whether the Setup Table image matches or differs from the original transaction version, as long as all delta records are captured and transferred to BW.
If, after 7PM, there were more changes to the Sales Order 12345, they would also be contained in the Delta. In this case, the Sales Order 12345 will be changed subsequently to its state at the moment of the last V3 update (see previous diagram). Delta loads may also contain ‘Reverse’ images. In this case the line of thought closely follows the example described.
