Upgrades & MigrationsOracle Database 11g Release 2 & ExadataRené Kundersma - Database Technical Architect Oracle Consulting

2

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3

Agenda & Introduction

Upgrades & Migrations Oracle Database 11g Release 2• ‘We are all upgrade specialists’• Upgrade & Migration to Exadata

René Kundersma – Oracle Consulting The Netherlands• Database Technical Architect / Software Engineer• Oracle Certified Master 9i / 10g / 11g• Oracle Exadata Certified Implementation Specialist• Oracle RAC Certified / Linux Certified• Advise, Implement and Troubleshooting• Focus on HA, Exadata, Virtualization and Linux

4

http://blogs.oracle.com/xpsonha

5

We are all upgrade specialists

• Many of you probably have an iP*d or similar kind of device • Many of you probably upgraded the O.S. on it • Those upgrades are generally seen as ‘smooth’operations

6

We are all upgrade specialists ? Are we ?

• The i*-device is monitoring your health• 24 x 7• You need to upgrade •… and you have a bad heart

Imagine…

7

What would you think of to make this upgrade less risky

• Test your upgrade procedure• Test the result of the upgrade• Have an option to fallback

Maybe it is not a bad idea to:

(Maybe it is not a bad idea to have a dr available anyway)..

Of course, there are some similarities between this example and a database. During my work, I fill in these gaps with simple solutions as TTS, DBUA but also for example:• DataGuard • GoldenGate / Streams• Real Application Testing

8

9

Talking about upgrades

• ‘We never upgrade, Oracle 6 still does it for us’• ‘I will make sure my boss hires a consultant• ‘What’s wrong with exp/imp ?’

We have ‘upgrades’ and ‘migrations’, what do you do ?

In case YOU have to do it, there are:• Best practices• A lot of solutions to fight your problem

In other words: STAY-UP-TO-DATE and DON’T FEAR the UNKNOWN

10

Talking about upgrades and migrations

Source system:• 10 node cluster• HP-UX (Itanium)• Oracle RAC + ASM 10.2.0.4 • HP EVA Storage

Target system:• 8 node DB machine • Linux x86-64• Oracle RAC + ASM 11.2.0.1

What databases are we migrating ?• Multiple OLTP and DWH• In this example: a 6TB data warehouse

11

Choices Initial Setup Exadata

• Prepare for ‘migration as is’, meaning:– New Exadata Features that require ‘change’ will be adopted later– New 11g Features that require ‘change’ will be adopted later– Not making any change to the PL/SQL– Not making any change to the application

• One cluster for dev, test, acceptance and production– Relocate Grid Infra and RDBMS HOME to customers wishes– Duplicate standard RDBMS HOME for test and development

• Cater for both DWH and OLTP• As less diskgroups as possible

12

Migrating to Exadata

Requirements:• Only a ‘small’, one time window available for the migration• Migrating ‘as is’ (no changes to code etc)• Want to make sure the migration will succeed at once (no surprises)• Ability to ‘fallback’ in case of failures the first two days

Technical Difficulties:• Large amount of data to transfer in a small window, no staging space• Cross platform Endian change• Implicit upgrade to 11gR2• Existing procedure using exp/imp took five days for 1.8 TB..…

Best Practices• Extent size (8MB) should be a factor of the ASM AU size (4MB)

13

14

Upgrade & Migration options

Physical• Cannot change database extent size for data• Endian and version dependent

– Physical standby (not possible)– TTS / TDB– ASM Redundancy (not possible)

Logical• Extent size can be modified, compress. and part. could be used• Slower on logical level

– Datapump export / import (time intensive)– DataGuard Logical, (not usable, cross Endian)– Streams / Goldengate – CTAS (too much)

HP-UXV1

V1

15

Migration Choice

Transportable tablespace• Robust• Endian Convert• Scalable (details later)

Plan: transfer the file from ASM on HP-UX to ASM on ExadataBut, how to transfer:

• DBMS_FILE_TRANSFER did not work, transferred file unusable• DBMS_STREAMS_TABLESPACE_ADM.PULL_TABLESPACE did

not scale• DBMS_STREAMS_TABLESPACE_ADM.PULL_TABLESPACES

fails when going from Big To Little Endian - 459229.1• RMAN tablespace convert did not scale with one tablespace• RMAN datafile convert was the answer, but was a two step

approach!

16

Migration procedure (1/2)

• Make tablespace ‘USERS’ read only• Step 1:

– ‘backup as copy datafile’ to NFS location on compute node• Step 2:

– convert datafile from local compute node disk into ASM• Import metadata• Total downtime including file transfer of 6TB: 15 hours• Earlier: fallback using Streams was configured after the TTS

procedure

17

Migration procedure (2/2)

On the source:RMAN> backup as copy datafile '+DATATST/tstint/datafile/users.320.637080827' format '/home/oracle/mnt/users.320.637080827';

On the target:RMAN> CONVERT DATAFILE2> '/scratch/users.320.637080827'3> FROM PLATFORM 'HP-UX IA (64-bit)'4> DB_FILE_NAME_CONVERT '/scratch/','+DATATST/';

18

Migration

19

Migration

Step 1: rman ‘copy as datafile’

20

Migration

Step 1: rman ‘copy as datafile’

Step 2: rman ‘convert datafile’

21

Migration

Step 1: rman ‘copy as datafile’

Step 2: rman ‘convert datafile’

This could be done at +/- 15 MByte/s for 1 node

22

Scaling up Migration procedure

for each datafile of tablespace ‘USERS’do

find a node that is not busy transferringif node foundthenbackup as copy datafile to NFS on X-node

elsefi

done

Source ‘daemon’:

23

Scaling up Migration procedure

for each target nodedo

if not already busy convertingthenfor each file found and marked transferreddo

’endian convert’ the file into ASMdone

fidone

Target ‘daemon’:

24

Migration scalability

+/- 30 MByte/s

25

Migration scalability

+/- 60 MByte/s

26

Migration scalability

+/- 420 GByte/hour

27

Overview / Timelines of another similar migration

Task name Duration Start FinishDelivery Exadata 2 days Oct 13 2009 Oct 14 2009

Exadata production ready 1 days Oct 15 2009 Oct 15 2009Engineer and test migr. to Exadata 8 days Oct 16 2009 Oct 27 2009Test application against Exadata 27 days Oct 16 2009 Nov 23 2009

Exadata production Nov 30 2009

28

Exadata In production: lessons learned

• Due to (mostly) technical and business limitations you may get constrained by the number of migration options available.

• With maximum use of existing machines and network resources you can ‘scale out’ the migration

• The hard work needs to be done on the right place (DBM)• Choose a migration procedure that is robust• Not always possible to stick to the best practices• More time on application testing would not have been

‘wasted time’• Calculate enough time for new patches in case you find bugs• Find the proper NFS settings for max throughput.• And…of course…

29

Exadata in production: lessons learned

• Preparation and Planning: key for a successful upgrade• Have a fallback strategy and test it• This is where Oracle Tools often come into play

Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK

ORA-39123: Data Pump transportable tablespace job abortedORA-29345: cannot plug a tablespace into a database using an incompatible character set

Job "SYS"."SYS_IMPORT_TRANSPORTABLE_01" stopped due to fatal error at 15:58:33

source: CHARACTER SET UTF8

target: CHARACTER SET AL32UTF8

30

Oracle Consulting / Next Steps

Why Oracle Consulting ? – Engineers of building blocks for large financials and telcos– Best practices– Subject matter experts. – Knowledge transfer– Link between support and development organization– Access to internal network and competence center– ‘Oracle knows Oracle Best– Multiple successful Exadata Migrations

MAA / Oracle Database HAoracle.com/technetwork/database/features/availability/maa-096107.html

31