Ahmed Alomari

Performance Specialist aalomari@cybernoor.com

Database SIG

Re-platforming the E-Business Suite Database

• Options

• Case Study

• Q & A

Agenda

Options

Re-platforming Options

• Transportable DB

• Transportable Tablespace

• Oracle GoldenGate with the Extended

Datatype Interface

• Data Pump (Full Export/Import)

• Re-implementation

Re-platforming Options

• Transportable DB

• Transportable Tablespace

• Oracle GoldenGate with the Extended

Datatype Interface

• Data Pump (Full Export/Import)

• Re-implementation

Case Study

Case Study

• Source Configuration Database Size: 11 TB

2 Node RAC Cluster with 12 Application Tier nodes

IBM P570 Servers (AIX 5.3) with 10.2.0.5 DB

GPFS

7X24 Global Instance

2,000+ DB files

~20 MB/sec RAC traffic

1 TB+ of redo daily EBZPROD1 EBZPROD2

Private

Interconnect

Public Network

App Tier (1) App Tier (2) App Tier (3)

Shared Disk

App Tier (12)

Case Study

• Target Configuration Database Size: 11 TB

4 Node RAC Cluster with12 Application Tier nodes 11.2.0.2.0 Grid Infrastructure (CRS + ASM)

Intel (x86) Linux Servers (64 bit Kernel) – Quad Core processors Red Hat Enterprise Linux Server release 5.5

10.2.0.5 DB with ASM and then upgraded to 11gR2 DB.

EBZPROD2 EBZPROD3 EBZPROD4EBZPROD1

Private

Interconnect

Public Network

App Tier (1) App Tier (2) App Tier (3)

Shared Disk

App Tier (12)

Case Study

• If Endianness is different, extra steps are

needed.

Check V$TRANSPORTABLE_PLATFORM

• Else

TransportDB option can be used which is much

simpler.

• In this case, AIX-Based Systems (64-bit)[Big]

to Linux x86 (64-bit) [Little] requires file

conversions.

Case Study

• Ensure source and target systems have the

same logical version of the version of the DB

home including PSUs and one-off patches.

• Ensure one-off patches related to Data Pump

and RMAN are in sync on both the source and

target system.

• Apply patch for bug 13001379.

Case Study

• Create the violations table

exec dbms_tts.transport_set_check()

• Mark tablespaces in the transport set as readonly

excluding RDBMS tablespaces such as TEMP,

UNDO, SYSTEM, CTX, SYSAUX, etc..

• Create the transport set

Drop un-used columns (query dba_unused_col_tabs)

Resolve violations.

Export tablespace metadata (transport_tablespace

option) with statistics. Avoids having to re-gather on target system.

Case Study

• Export with the full option and without data

Full=y

Exclude tables, indexes, constraints, MVs, etc..

• Export with the full option to capture all VPD

policies

Full=y

Include=RLS_POLICY

• Export the system schemas

SYS, SYSTEM, ORDSYS, MDSYS, etc..

• Note the AQ settings.

Case Study • Export the GTT table definitions.

• Convert the datafiles using RMAN on the target

system using the CONVERT option with

PARALLELISM option

From platform [AIX-Based Systems (64-bit)]

Use db_file_name_convert to remap path names.

• Create an RMAN script for each set of 20 files

and launch multiple rman sessions.

Leverage the horsepower of the cluster to launch

multiple RMAN sessions on each node.

Very I/O intensive so storage performance is key.

Case Study

• Entire file conversion process took about 12

hours.

Initially took 60+ hours, but after storage tuning and

using multiple streams of RMAN, it reduced to 11.

• The DB file conversion step will take the longest

amount of time out of the entire migration.

So focus on optimizing this step and test with different

numbers of parallelism and number of files per RMAN

script to find the optimal settings in your environment.

Case Study

• Import TTS metadata into the source system.

Exclude PL/SQL units since metadata needs to

be imported first.

• Import datafiles using the

TRANSPORT_DATAFILES option

Exclude Table and Index statistics to avoid the lengthy

step of generating statistics. This optimization alone

can save 10+ hours depending on the volume of the

DB.

• Convert the transportable tablespaces on the

target DB to read write mode.

Case Study

• Update user metadata including default

tablespace and quote information.

• Import PL/SQL units from the full DDL export.

• Import the VPD policies from the RLS export.

• Activate the queue tables.

• Rebuild spatial indexes.

• Import the system schema and GTTs.

• Import the schema statistics (saved earlier).

Avoids having to re-gather statistics.

Case Study

• Post Steps

Reset AQs

Verify grants and synonyms are in sync.

Recompile invalid objects and review list of invalid

objects.

Compile flex fields and recreate grants/synonyms for

Apps.

Run the program DQM Staging Program (ARHDQM) to

build the DQM indexes.

Case Study • Post Steps

Check for missing statistics on Apps related objects

including column level statistics. Avoid sub-optimal execution plans prior to releasing the system

to the users.

Gather System Statistics and Fixed Object Statistics on

the target system (do this during a peak period). Fixed object statistics refreshes the statistics on the X$ tables.

Stale or non-representative statistics on the fixed objects can

result in poor performance of V$ related queries which can

impact Applications performance.

Case Study • Post Steps

Gather System Statistics and Fixed Object Statistics on

the target system (do this during a peak period). Fixed object statistics refreshes the statistics on the X$ tables.

Stale or non-representative statistics on the fixed objects can

result in poor performance of V$ related queries which can

impact Applications performance.

For DB upgrades, normally handled by adstats.sql, however,

timing of when this is run is key.

Run the commands below to refresh the fixed object statistics.

Normally takes about 15-20 minutes depending on CPU speed.

exec dbms_stats.DELETE_FIXED_OBJECTS_STATS;

exec dbms_stats.GATHER_FIXED_OBJECTS_STATS;

Case Study

• Post Steps

Ensure OS huge pages are being used for the SGA vm.nr_hugepages ≅ <SGA size in MB+overhead size>/(2)

Ex: vm.nr_hugepages = 41000 (80 GB SGA)

Ensures SGA memory is pinned and locked.

Set memorylock Shell limit to the size of physical memory on

each node (/etc/security/limits.conf)

– @oradba soft memlock 268435456

– @oradba hard memlock 268435456

Add entry to /etc/rc.d/init.d/ohasd to ensure huge pages are

used on DB startup by CRS.

– ulimit -l unlimited

Case Study

• Post Steps

Tune message buffer size – InterConnect traffic. net.core.rmem_default=4194304

net.core.rmem_max=4194304

Tune timeout and heartbeat frequency in

/etc/sysconfig/o2cb if any OCFS file system is being

used.

Run AutoConfig on all nodes. For App. servers, ensure an internal node is the last node in the

cycle for which AutoConfig is executed.

Start Application Services.

Change Concurrent Manager affinity.

References

Note #: Description:

454574.1 Cross Platform Transportable Tablespaces on 11i with 10gR2

341733.1 Export/Import DataPump Parameters INCLUDE and EXCLUDE - How to Load and Unload

Specific Objects

1377213.1 Oracle E-Business Suite Upgrades and Platform Migration

729309.1 Using Transportable Database to migrate Oracle E-Business Suite 11i Using Oracle

Database 10.2.0 or 11g

Question & Answer

Thank You for Attending.