Oracle BI Applications 11g GoldenGate Integration

7/24/2019 Oracle BI Applications 11g GoldenGate Integration

1/19

Oracle BI Applications 11g:GoldenGate IntegrationBy Praveen Kumar

The release of Oracle Business Intelligence Applications 11g included a major change incomponents, with Oracle Data Integrator 11g (ODI replacing Informatica as the !T"technolog#$ The ne%t logical step was to integrate Oracle&s data replication tool, OracleGoldenGate 11g (OGG, for a real'time load of source s#stem data to the data warehouse$sing GoldenGate replication rather than a con)entional ODI e%tract process, contention onthe source is essentiall# eliminated and the source O"T* data is stored locall# on the datawarehouse, remo)ing networ+ ottlenec+s and allowing !T" performance to increase$ In thisarticle, we-ll wal+ through the architecture and setup for using GoldenGate 11g with OracleBI Applications 11$1$1$.$1$Note:This article will also apply to Oracle Business Intelligence

Applications versions 11.1.1.8.1 and 11.1.1.9.1.

GoldenGate and the Source Dependent Data StoreIf #ou are not familiar with Oracle GoldenGate (OGG, it is the standard Oracle product fordata replication, pro)iding log'ased change data capture, distriution, and deli)er# in real'time$

GoldenGate captures transactional data changes from the source dataase redo log and loads the cinto its own log /le, called a Trail 0ile, using a platform'independent uni)ersal data format$ The !%traprocess understands the schemas and tales from which to capture changes ased on the con/guraset in the !%tract parameter /le$ The data is then read from the ource Trail 0ile and mo)ed across t

networ+ to the Target Trail 0ile using a process called a Data *ump, also dri)en # a parameter /le$ the transactions are loaded into the target dataase tales using the 2eplicat parameter /le con/guwhich maps source tales and columns to their target$ The entire process occurs with su'second laand minimal impact to the source and target s#stems$In Oracle BI Applications, GoldenGate 11g is used for pure replication from the sourcedataase to the target data warehouse, into what is +nown as the ource Dependent Datatore (D schema$


2/19

The D is setup to loo+ e%actl# li+e the source schema, allowing the Oracle Data Integratorpre'uilt Interfaces to change which source the# are using from within the "oading3nowledge 4odule ("34 # e)aluating a )ariale (I5D5D!*"O6!D at )arious pointsthroughout the "34$ sing this approach, the GoldenGate integration can e easil# enaledat an# point, e)en after initial con/guration$ The Oracle BI Applications team did a great joof utili7ing the features of ODI that allow the logical la#er to e astracted from the ph#sical

la#er and data source connection$

User and Schema SetupThe /rst step is to manuall# create the GoldenGate user on the source and target dataases$These users, li+e the ource Dependent Data tore schema, are not created # the Oracle BIApplications installer$ This will e a dedicated user for OGG, and will ha)e pri)ileges speci/cto the needs of the e%tract process on the source and the replicat process on the target$

Create Source GoldenGate UserBeginning with the source, create the user and grant the initial pri)ileges$ Be sure thetalespace has alread# een created$-- Create OGG User on the sourceCREATE USER ogg_user IDENTIFIED BY Passwor!"

DEFAU#T TAB#ESPACE ggs_ata $UOTA UN#I%ITED ON ggs_ata&GRANT CREATE SESSION TO ogg_user&GRANT A#TER SESSION TO ogg_user&GRANT SE#ECT ANY DICTIONARY TO ogg_user&GRANT F#AS'BAC( ANY TAB#E TO ogg_user&

The speci/c tale grants will e made later using a script generated # an ODI *rocedure$The user will temporaril# need A"T!2 A86 TAB"! in order to set up supplemental logging forindi)idual tales$ Afterwards, this pri)ilege can e re)o+ed$


3/19

GRANT A#TER ANY TAB#E TO ogg_user&

0inall#, set up supplemental logging at the dataase le)el, ensuring the necessar#information is logged for each transaction$A#TER DATABASE ADD SUPP#E%ENTA# #OG DATA&

Create Target GoldenGate User8e%t, create the GoldenGate user on the target dataase with target'speci/c pri)ileges$GoldenGate performs D4" on the target tales ased on the change made in the sourcedataase, so the user will need to e granted pri)ileges to I8!2T, *DAT!, D!"!T!$ Again,rather than grant I8!2T A86 TAB"!, etc$, the speci/c tale grants will e generated as ascript )ia an ODI *rocedure$-- Create OGG User CREATE USER ogg_target IDENTIFIED BY Passwor!"DEFAU#T TAB#ESPACE USERS $UOTA UN#I%ITED ON USERS&GRANT CREATE SESSION TO ogg_target&GRANT A#TER SESSION TO ogg_target&GRANT SE#ECT ANY DICTIONARY TO ogg_target&

The chec+point tale will e created from within GoldenGate, so this user will temporaril#need to e granted the 92!AT! TAB"! pri)ilege$ The chec+point tale will +eep trac+ of thelatest position in the target trail /le, allowing a clean reco)er# should the target dataase gooine or some other failure occur$GRANT CREATE TAB#E TO ogg_target&

Create SDS UserA separate D schema must e created for each O"T* source application, with the Dschema essentiall# acting as the source schema$ ;e-ll follow the recommended namingcon)entions for the schema:


4/19

-- Re4u5re GrantsGRANT CREATE SESSION TO BIAPPS_SDS_PSFT_!_CS_.!&GRANT CREATE TAB#E TO BIAPPS_SDS_PSFT_!_CS_.!&

0inall#, the GoldenGate target user must e granted access to use the D talespace forinserts@updates$

-- OGG user 6ust )e grante $uota to 5nsert an u+ate ataA#TER USER ogg_target $UOTA UN#I%ITED ON BIAPPS_SDS_PSFT_!_CS_.!_TS&

Install and Confgure GoldenGate 11gThe schemas are in place, so the ne%t part of the setup is to install and con/gure theGoldenGate application on oth the source and target ser)ers$ The GoldenGate 11ginstallation process is well documented, so we don-t need to go into much detail here$ TheOracle BI Applications documentation also has some e%ample scripts, which wal+ throughthe setup of the e%tract, data pump, and replicat group processes$

The naming standards for the parameter /les are fairl# straightforward, with D8 eing thesame data source numer we used in the D schema name$

!%tract: !T5


5/19

targets$ The result should e GoldenGate process groups setup and read# to go once theparameter /les are in place$

2ememer to re)o+e the 92!AT! TAB"! pri)ilege from the target GoldenGate user once thechec+point tale has een created$RE;O(E CREATE TAB#E FRO% ogg_target&

Enale the SDSSet the SDS Parameter

The /rst step in con/guration is to enale the D in the Oracle BI Applications9on/guration 4anager$ This indicator, set for each source instance, will e used throughoutthe Oracle Data Integrator !T" processing to determine whether to connect directl# to thesource or to the D schema$

9hoose 4anage Data "oad *arameters from the Tas+s list$ In the 4anage Data "oad*arameters window, select the appropriate ource Instance (in this e%ample we are using*eopleoft 9ampus olutions ' so *59 is chosen and search for the *arameter nameI5D5D!*"O6!D$


6/19

elect the parameter and clic+ the !dit utton$ There will e a warning that this parameterwill e set gloall# for this ource Instance$ ince we plan to pull all of our data from

9ampus olutions using OGG, clic+ 6es to continue (otherwise, #ou could choose a speci/cfact group on which to set the parameter$ In the !dit Dialog, change the parameter )alue to

6!$


7/19

Create the SDS Phys!cal SchemaThe *h#sical chema for the D schema must e manuall# created in the Oracle DataIntegrator Topolog# metadata$ Open ODI tudio and select the Topolog# 8a)igator$ nder*h#sical Architecture, drill'down into Oracle and /nd the Data er)er that corresponds to thedata warehouse target for Oracle BI Applications$ 2ight'clic+ and choose 8ew *h#sicalchema$ et oth the chema and ;or+ chema )alues to the D dataase schema thatwas created pre)iousl#$

8e%t, clic+ on the 0le%0ields ta$ ;e need to set the DATAO29!5845ID 0le%0ield tomatch that of the source instance for which the D schema is a target$


8/19

Generate and E"ecute SDS DD# Scr!ptsOracle BI Applications is now con/gured to use the D schema as the source for the D!processes, rather than the actual source, and GoldenGate is con/gured to +eep the Dschema up'to'date with the latest source transactions in real'time$ 8e%t, we will e%ecute anODI *rocedure to generate the DD" scripts for the D schema tales and inde%es, as well

as the grants on the source tales for the GoldenGate user$

To e%ecute the cenario, open ODI tudio, go to the Designer 8a)igator, and in the *rojectsrowse to BI $pps Pro%ect & Components & SDS & 'racle & Generate SDS DD#(E"pand Pac)ages & Generate SDS DD# & Scenar!osand #ou will /nd the cenarioG!8!2AT!5D5DD" Cersion >>1$

2ight'clic+ the cenario and select !%ecute$ After selecting the 9onte%t and Agent, #ou wille presented with a list of ODI Cariales used in the cenario$ These will pro)ide input intothe script generation *rocedure and are descried in detail in section $E$E etup tep:Generate, Deplo#, and *opulate the ource Dependent chema Tales on Target Dataaseofthe Oracle BI Applications documentation$
http://docs.oracle.com/cd/E38317_01/doc.11117/e37988/goldengate.htmhttp://docs.oracle.com/cd/E38317_01/doc.11117/e37988/goldengate.htmhttp://docs.oracle.com/cd/E38317_01/doc.11117/e37988/goldengate.htmhttp://docs.oracle.com/cd/E38317_01/doc.11117/e37988/goldengate.htm


9/19

;hen generating the DD" script for the /rst time, choose 0"" for the 2!02!F54OD!)ariale$ This will attempt to perform a drop and recreate of all ojects$ Also, if #ou wouldli+e to run the scripts manuall#, set the 92!AT!592I*T50I"! e


10/19

ust as we did with the Generate DD" cript cenario, rowse to BI $pps Pro%ect &Components & SDS & Generate SDS 'GG Param ,!les( !%pandPac)ages &Generate SDS 'GG Param ,!les & Scenar!osand /nd the cenarioG!8!2AT!5D5OGG5*A2A450I"! Cersion >>1$ 2ight'clic+ and select !%ecute$ ;e areagain pro)ided with a list of Cariales that need the appropriate )alues added prior tostarting the process$ 4an# of them should loo+ familiar if #ou ha)e wor+ed with theGoldenGate 34 ' ut with much more intuiti)e names$


11/19

et the appropriate )alues for the )ariales, descried in detail in section J$E$ etup tep:Generate and Deplo# Oracle GoldenGate *arameter 0iles to ource and Target 4achines inthe Oracle BI Applications documentation, and clic+ O3 to e%ecute the cenario$ Once thecenario has completed, rowse to the temporar# director# and #ou should /nd a folderD85


12/19

The /nal piece of con/guration is to generate the source de/nitions /le$ This /le pro)idesthe GoldenGate replicat process the tale and column de/nitions from the source dataase,allowing the process to interpret the tale and column mappings in the replicat parameter/le$ The reason it must e used for replication to the D is ecause there are K additionalcolumns mapped to each target tale that do not e%ist in the source tales and cannot emapped to the target tale # name$

CDC=_RP#_#AST_UPDATE_DATE > ?DATENO@ 7CDC=_SRC_#AST_UPDATE_DATE > ?GETEN; GG'EADER7 CO%%ITTI%ESTA%P7CDC=_D%#_CODE > I --U or D

The replicat last update date (9D9L52*"5"AT5*DAT!5DAT! will e the timestamp whenthe record is updated in the target tale$ The source last update date(9D9L5295"AT5*DAT!5DAT! is the commit timestamp for when the change occurred inthe source$ And /nall# the D4" code (9D9L5D4"59OD! represents the t#pe of changeMinsert, update, or delete$

To generate the source de/nitions /le, e%it GG9I and run the following command:,0e1gen +ara615*e ,05r+r60DEFGEN,+r6

Once completed, cop# the source de/nitions /le from the source ser)er to the target ser)erdirdef director#$

In!t!al #oadThere are two diHerent options for performing the initial load of the target D schema fromthe source dataase$ The /rst approach, using the process as deli)ered straight out'of'the'o%, re


13/19

The cenario calls an ODI *rocedure named 9op# D Data$ ;hen e%ecuted, it willgenerate a script with an insert statement for each target D tale using a select o)er adataase lin+ to the O"T* source$ The lin+ must e manuall# created and speci/call#named D.*T'*'#TP, as the ODI *rocedure has the dlin+ name hard'coded$--Dro+ an recreate the ata)ase *5n,ro+ ata)ase *5n D@_TO_O#TP

--create ata)ase *5n D@_TO_O#TPconnect to SYSAD% 5ent515e )< SYSAD% us5ng /PSDE%O/&

--test the *5n,se*ect 9 1ro6 ua*?D@_TO_O#TP&

The /rst approach to the initial load would then ha)e #ou e%ecute the cenario to generatethe insert statements, schedule a downtime for the O"T* application, and run the initial loadscripts$ 2ather than go through those steps, let-s ta+e a loo+ at how to eliminate the sources#stem una)ailailit# with a slight change to the code in the second approach$

/ero Do0nt!me In!t!al #oadThe initial load process will e customi7ed to use the Oracle dataase Nashac+


14/19

Open up the ODI *rocedure 9op# D Data from the copied director#$ 9lic+ on the Detailsta to re)iew the steps$ ;e will need to modif# the step 9op# Data, which generates theD4" script to mo)e data from source to target$ A re)iew of the code will show that it usesthe dictionar# )iews on the source ser)er, across the dataase lin+, to get all tale andcolumn names that are to e included in the script$ The construction of the insert statementis the it of code we will need to modif#, adding the Oracle dataase Nashac+ /truncate ta)*e =>6oe*Sche6a=H,/ co*_rec,ta)*e_na6e&*_s4*_st6t > /INSERT 09J APPEND 90 INTO =>6oe*Sche6a=H,/ co*_rec,ta)*e_na6e / / rtr56*_co*u6n_*5st7 /7 / / / /SE#ECT / rtr56*_co*u6n_e:+r7 /7 / / FRO% /

co*_rec,ta)*e_na6e /?K>DB#5nNa6eKH as o- scn I2ITI$#*#'$D*SC23&,,,

As #ou can see, PI8ITIA"5"OAD598 is a placeholder for an ODI Cariale$ A )ariale is usedto perform the refresh of the 98 from the source rather than eing hard'coded$ 9reate the)ariale I8ITIA"5"OAD598 and set the


15/19

The user setup to connect to the O"T* source will need to temporaril# e granted the selectan# dictionar# pri)ilege in order to allow the select from CLDATABA!$ This pri)ilege canlater e re)o+ed$grant se*ect an< 5ct5onar< to SYSAD%&

8ow that the Cariale is set and the *rocedure code has een modi/ed, we just need to put

it all together in a *ac+age and generate a cenario$ The *ac+age 9op# O"T* to D isalread# setup to call the *rocedure 9op# D Data, so the ODI Cariale I8ITIA"5"OAD598can e added as a refresh step at the eginning of the *ac+age$

After sa)ing the *ac+age, generate a cenario to e%ecute$ ;hen generating the cenario, esure to set all Cariales e%cept for I8ITIA"5"OAD598 as tartup Cariales, as their )alues


16/19

will e set manuall# during the e%ecution of the cenario, while the 98 is going to erefreshed from the source dataase$ Also, rememer to pro)ide a diHerent name than theoriginal cenario, as the# must e uni


17/19

8ow that the transactions are Nowing into the source trail and across the networ+ to thetarget trail, we can e%ecute the cenario to generate the initial load script /les$ ;hene%ecuted, a prompt will appear, allowing #ou to enter the appropriate )alue for each)ariale$ The script can e /ltered down # a speci/c list of tales, if necessar#, # adding acomma'delimited list to the TAB"!5"IT )ariale$ ;e-ll just use a wildcard )alue to generatethe script for all tales$ Other options are to generate a script /le (6 or 8 and to e%ecute theD4" during the e%ecution of the cenario (e)en though the Cariale is named 285DD"$


18/19

The script, named BIA5D59op#5Data5


19/19

The initial load of data to the D schema has een completed and GoldenGate replicationstarted, all without an# impact to the source O"T* application$ The ne%t time the ource

Data !%tract (D! "oad *lan is e%ecuted, it will e just as if it were running directl# againstthe source dataase ' onl# faster ' since it-s pulling data from the D schema on the sameser)er, than+s to Oracle GoldenGate real'time data replication$

Date post:	20-Feb-2018
Category:	Documents
Upload:	praveen
View:	221 times
Download:	0 times

Oracle BI Applications 11g GoldenGate Integration

Documents