173_Oracle RAC From Dream To Production_1.0.0.ppt

In RAC We TrustORACLE - From Dream To ProductionPlamen Zyumbyulev,, Let someone k n o w BGOUG Gabrovo 22.04.2005

Presentation GoalsDescribe the major steps in RAC implementation

Show and explain the main problems and obstacles

Show and explain solution and workarounds

Give some practical ideas and variations

Try to look at the problems from different angles

Non-GoalsExplain RAC concepts and fundamentals

Show ALL aspects of RAC

AgendaIntroduction

RAC Installation

High Availability Configuring

Testing and Tuning RAC

Implementing RAC in production

Introduction

RAC Installation Problems

Solutions

ProblemsNeed to evaluate and purchase cluster hardware

Need of knowledgeable OS, High Availability, Network and Storage professionals

All this takes time

SolutionsRAC on Single NodeRAC on Single VMware NodeRAC on Multiple VMware NodesRAC and Network Block DeviceAll of the solutions presented here are for testing purposes ONLY. These configurations are not certified or supported by Oracle Support ServicesOther Solutions

RAC on Single NodeWhy not?

Metalink Note:241114.1 - Step-By-Step Installation of RAC on Linux - Single Node (Oracle9i 9.2.0 with OCFS)

Key PointsNo need of fencing configurationNo need of using clustered file system or raw devicesNo need of multiple oracle homes (ORACLE_HOME)

RAC on Single Node (contd)Key Points (contd)One oracle user with 2 or more profiles one for every instance. (e.g. .rac1, .rac2, )

zyumix:/# su - oracle oracle@zyumix:~$ . rac1 oracle@zyumix:~$ echo $ORACLE_SID rac1zyumix:/# su - oracle oracle@zyumix:~$ . rac2 oracle@zyumix:~$ echo $ORACLE_SID rac2

RAC on Single Node (contd)Key Points (contd)Oracle Universal Installer needs Clusterware software in order to install RAC option.

Disadvantages

RAC on Single Node (contd)Server AInstance AServer BInstance BDatabaseClient side load balancingServer side load balancingListenerAListenerB

RAC on Single VMware NodeEven easier !!!The Oracle-on-Linux VMware Cookbook - http://www.oracle.com/technology/tech/linux/vmware/cookbook/index.html

An easy, hands-on, step-by-step guide describing how-to install VMware, Linux (RHEL/SLES) and Oracle RAC (again on single node)

VMware Workstation (90-day free eval; registration required) RHEL3 (not free) , SLES8 (not free)

Disadvantages

RAC on Multiple VMware NodesVMware GSX/ESX Server permits the sharing of plain disks with multiple virtual machines running on the same host, provided the disk in question is a SCSI disk.

This approach is very powerful but complex. You can create very complex environments multiple NICs, switches, disks etc.

Now there are a number of nodes although virtual

Disadvantages

RAC and Network Block DeviceThis solution allows you to build scalable and high available database system only with common Intel PCs connected into Ethernet network.

In this solution, a standard shared disk subsystem is replaced by a native Linux technology - Network Block Device (NBD) that maps remote files to local block devices (e.g. /dev/nb0) via TCP/IP network. One computer (not necessarily Linux machine) serves as data storage for all cluster nodes (Linux machines) instead of expensive disk array.

RAC and Network Block Device (contd)With this thing compiled into the kernel, Linux can use a remote server as one of its block devices. Every time the client computer wants to read /dev/nd0, it will send a request to the server via TCP, which will reply with the data requested.

The remote resource doesn't need to be a whole disk or even a partition. It can be a file.

RAC and Network Block Device (contd)Typical configuration Simple NBD configuration

RAC and Network Block Device (contd)InstallationBoth client and server machines are with RHEL3Download source from http://nbd.sourceforge.net/As root dobunzip2 nbd-2.7.3.tar.bz2tar -xvf nbd-2.7.3.tarcd nbd-2.7.3./configuremakemake install

RAC and Network Block Device (contd)Creating new empty files at NBD server [root@rac2 root]# dd if=/dev/zero of=/u01/oradata/rac/system.01 count=300 bs=1M300+0 records in300+0 records out314572800 bytes transferred in 1.683993 seconds (186801738 bytes/sec)[root@rac2 root]#

Running NBD serverSyntax: nbd-server [root@rac2 root]# nbd-server 4101 /u01/oradata/rac/system.01[root@rac2 root]#

RAC and Network Block Device (contd)NBD clientNBD client must be run as root (because of kernel parts of NBD). Before starting NBD client you would have to install Linux kernel NBD module

Installing ndb module RHEL3

[root@rac3 root]# rpm -Uvh kernel-unsupported-2.4.21-4.EL.i686.rpmwarning: kernel-unsupported-2.4.21-4.EL.i686.rpm: V3 DSA signature: NOKEY, key ID db42a60ePreparing... ########################################### [100%] 1:kernel-unsupported ########################################### [100%][root@rac3 root]#

RAC and Network Block Device (contd)NBD client (contd)Loading nbd module[root@rac3 dev]# lsmod | grep nbd[root@rac3 dev]# modprobe nbd[root@rac3 dev]# lsmod | grep nbdnbd 16388 0 (unused)[root@rac3 dev]#running nbd clientSyntax: nbd-client /dev/nb [root@rac3 dev]# nbd-client rac2 4101 /dev/nb0Negotiation: ..size = 307200KBbs=1024, sz=307200[root@rac3 dev]#

RAC and Network Block Device (contd)Now block devices are configured and it is possible to access remote data. Oracle Real application clusters need raw access to shared disk subsystem so mapping raw devices to block devices is needed. This could by done with standard raw command.

Syntax: raw /dev/raw/raw /dev/

[root@rac3 root]# raw /dev/raw/raw1 /dev/nb0/dev/raw/raw1: bound to major 43, minor 0[root@rac3 root]#

Other SolutionsRAC and FireWireBuild Your Own Oracle RAC 10g Cluster on Linux and FireWire http://www.oracle.com/technology/pub/articles/hunter_rac10g.html

RAC and NFSLockingCachingWrite through cache

HA Configuration

Simplified RAC schema DatabaseNET

HA System

Maximum Availability ArchitecturePrimary SiteApplication ServerSecondary SiteApplication ServerData Guard

Extended Distance Clusters Primary SiteRACApplication ServerSecondary SiteApplication ServerVirtualization storage layerWhat about the Quorum Server???

Extended Distance Clusters (contd)Resolving Distance ProblemsApplication partitioninggc_files_to_locksWavelength Division MultiplexingDense Wavelength Division Multiplexing DWDMCoarse Wavelength Division Multiplexing CWDMACTIVE_INSTANCE_COUNT*.active_instance_count = 1*.cluster_database_instances = 2

Testing and Tuning RAC Introduction RAC testing steps Functional Application Tests RAC High Availability tests Scalability tests Digging into RAC performance problems

IntroductionTesting isnt trivial !!!

Classical testing/tuning methods.Always tune single instance first!

Specific RAC issues

RAC aware tools

RAC testing stepsFunctional Application Tests

RAC High Availability testsBe aware about the timeouts!!!

Scalability tests

RAC testing steps (contd)Scalability tests (contd)Patterns of application scalability

Load ( # users, size of tables)Performance (TPS, 1/response time) of one userexponentialconstrainedlinearnearly static

RAC testing steps (contd)Single node2 node RAC Scalability tests (contd) Good scalabilityPerformance -TPS (for all users)# of concurrent users

Chart1

2379.1841364.797

3364.8881778.068

3894.52122807.29

4386.19163116.82

4721.44203299.11

5117.27243363.01

5520.15283662.7

5401.98323476.9

5360.8363541.13

5987.35403596.96

5921.58443536.72

5579.03483545.41

5527.05523516.57

5215.54563598.23

5492.66603664.91

5251.82643601.94

5306.92683530.15

5863.19723418.68

5741.72763506.63

5747.75803452.59

5362.48843575.75

6336.19883450.15

5709.22923436.68

5522.15963394.49

5616.131003456.98

Sheet1

RAC tests for 2 nodes with load3() procedureRAC tests for 1 node with load3() procedure

41657.17

83119.67

123831.83

164274.15

205059.45

245942.43

286288.1

326304.06

366435.12

406132.97

446815.34

486553.27

526422.5

566316.79

605789.69

41887.911672.77

83309.22994.85

124211.763468.63

165395.723621.45

205801.313649.3

245396.84251.86

285484.824184.57

326547.314258.77

365966.034214.36

406109.324336.63

445522.284256.25

486164.714443.89

526390.924479.34

565395.554665.66

606035.664656.49

646299.944729.49

686272.344757.2

724942.275042.58

4453.1941874.63

82060.862779.59

123777.543515.2

164373.083658.19

205022.924187.82

245497.234220.86

286098.084319.08

325876.614380.84

366389.434059.83

406593.864163.68

446380.174349.44

485965.284328.55

526595.444390.43

566020.934472.56

606022.034762

646587.594739.97

685228.294710.7

726354.115143.24

765737.34916.31

805112.114963.7

845147.854943.27

8858565074.24

925815.845212.75

965254.625126.48

1006149.46

1045644.62

1085169.22

1125152.81

1166032.65

1205245.2

1245103.1

1285465.05

1325828.68

1365469.38

1405655.27

&C&A

&CPage &P

Sheet1

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

Sheet2

# of userstps

10751.6821105.643875.974537.21

203424.663398.763515.565838.99

305317.263435.73535.36814.31

405007.593549.43903.446772.06

504923.443535.174282.297242.28

605270.443571.254300.797811.23

705256.33605.664462.277388.14

805666.753505.894976.887516.68

906252.773485.825016.667994

1005949.353504.795196.998054.03

1105981.253379.575145.828646.53

1205790.943511.125187.087894.5

1305834.463436.165190.238830.25

1406654.233386.495299.799262.67

1506335.243437.584863.48967.48

1606967.773452.055947.248479.33

1706648.893525.95804.059730.07

1805556.153403.729284.568600.7

1906364.353352.028465.017714.75

2006501.063490.179351.517949.65

101105.64

203398.76

303435.7

403549.4

503535.17

603571.25

703605.66

803505.89

903485.82

1003504.79

1103379.57

1203511.12

1303436.16

1403386.49

1503437.58

1603452.05

1703525.9

1803403.72

1903352.02

2003490.17

42379.181364.797

83364.881778.068

123894.522807.29

164386.193116.82

204721.443299.11

245117.273363.01

285520.153662.7

325401.983476.9

365360.83541.13

405987.353596.96

445921.583536.72

485579.033545.41

525527.053516.57

565215.543598.23

605492.663664.91

645251.823601.94

685306.923530.15

725863.193418.68

765741.723506.63

805747.753452.59

845362.483575.75

886336.193450.15

925709.223436.68

965522.153394.49

1005616.133456.98

&C&A

&CPage &P

Sheet2

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

Sheet3

N/A/oradata/leapprod/redolog/redo.01 -> /dev/leapnvg/rredo.01500








N/A/oradata/leapprod/ctrl/control.01 -> /dev/leapnvg/rctrlfile.012.5859375


DA_DEFS/oradata/leapprod/da_defs/da_defs.01 -> /dev/leapnvg/rdadefs.01290

DA_FITS/oradata/leapprod/da_fits/da_fits.01 -> /dev/leapnvg/rdafits.01511

DA_MEDIATION/oradata/leapprod/da_mediation/da_mediation.01 -> /dev/leapnvg/rdamed.018191

DA_RATING/oradata/leapprod/da_rating/da_rating.01 -> /dev/leapnvg/rdarat.018191

DRSYS/oradata/leapprod/drsys/drsys.01 -> /dev/leapnvg/rdrsys.0120

IND_MEDIATION/oradata/leapprod/ind_mediation/ind_mediation.01 -> /dev/leapnvg/rindmed.012047

IND_RATING/oradata/leapprod/ind_rating/ind_rating.01 -> /dev/leapnvg/rindrat.012047

SYSTEM/oradata/leapprod/system/system.01 -> /dev/leapnvg/rsystem.012047

TABLES_NO_ANALYZE/oradata/leapprod/tables_no_analyze/tables_no_analyze.01 -> /dev/leapnvg/rtbsnoa.012047

TOOLS/oradata/leapprod/tools/tools.01 -> /dev/leapnvg/rtools.013071

UNDO1/oradata/leapprod/undo/undo.01 -> /dev/leapnvg/rundo.0116382

XDB/oradata/leapprod/xdb/xdb.01 -> /dev/leapnvg/rxdb.0150

DA_GENERAL/oradata/leapprod/da_general/da_general.01 -> /dev/leapnvg/rdagen.0116382



DA_HISTORY/oradata/leapprod/da_history/da_history.01 -> /dev/leapnvg/rdahist.0116382







DA_USAGE/oradata/leapprod/da_usage/da_usage.01 -> /dev/leapnvg/rdausage.0116382









IND_DEFS/oradata/leapprod/ind_defs/ind_defs.01 -> /dev/leapnvg/rindefs.01290

IND_GENERAL/oradata/leapprod/ind_general/ind_general.02 -> /dev/leapnvg/rindgen.028191


IND_HISTORY/oradata/leapprod/ind_history/ind_history.01 -> /dev/leapnvg/rindhist.0116382





IND_USAGE/oradata/leapprod/ind_usage/ind_usage.01 -> /dev/leapnvg/rindusage.0116382








&C&A

&CPage &P

Sheet4

N/A/dev/leapnvg/rredo.01500








N/A/dev/leapnvg/rctrlfile.012.5859375


DA_DEFS/dev/leapnvg/rdadefs.01290

DA_FITS/dev/leapnvg/rdafits.01511

DA_MEDIATION/dev/leapnvg/rdamed.018191

DA_RATING/dev/leapnvg/rdarat.018191

DRSYS/dev/leapnvg/rdrsys.0120

IND_DEFS/dev/leapnvg/rindefs.01290

IND_MEDIATION/dev/leapnvg/rindmed.012047

IND_RATING/dev/leapnvg/rindrat.012047

SYSTEM/dev/leapnvg/rsystem.012047

TABLES_NO_ANALYZE/dev/leapnvg/rtbsnoa.012047

TOOLS/dev/leapnvg/rtools.013071

UNDO1/dev/leapnvg/rundo.0116382

XDB/dev/leapnvg/rxdb.0150

DA_GENERAL/dev/leapnvg/rdagen.0116382



DA_HISTORY/dev/leapnvg/rdahist.0116382







DA_USAGE/dev/leapnvg/rdausage.0116382









IND_GENERAL/dev/leapnvg/rindgen.028191


IND_HISTORY/dev/leapnvg/rindhist.0116382





IND_USAGE/dev/leapnvg/rindusage.0116382








&C&A

&CPage &P

RAC testing steps (contd)Single node2 node RAC Scalability tests (contd) Problem!!!Possible disk bottleneckPerformance -TPS (for all users)# of concurrent users

Chart2

2379.1841364.797

3364.8881778.068

3894.52122807.29

4386.19163116.82

4721.44203299.11

5117.27243363.01

5520.15284262.7

5401.98324776.9

5360.8365241.13

5987.35405596.96

5921.58445536.72

5579.03485545.41

5527.05525516.57

5215.54565598.23

5492.66605664.91

5251.82645601.94

5306.92685530.15

5863.19725418.68

5741.72765506.63

5747.75805452.59

5362.48845575.75

6336.19885450.15

5709.22925436.68

5522.15965394.49

5616.131005456.98

Sheet1


41657.17

83119.67

123831.83

164274.15

205059.45

245942.43

286288.1

326304.06

366435.12

406132.97

446815.34

486553.27

526422.5

566316.79

605789.69

41887.911672.77

83309.22994.85

124211.763468.63

165395.723621.45

205801.313649.3

245396.84251.86

285484.824184.57

326547.314258.77

365966.034214.36

406109.324336.63

445522.284256.25

486164.714443.89

526390.924479.34

565395.554665.66

606035.664656.49

646299.944729.49

686272.344757.2

724942.275042.58

4453.1941874.63

82060.862779.59

123777.543515.2

164373.083658.19

205022.924187.82

245497.234220.86

286098.084319.08

325876.614380.84

366389.434059.83

406593.864163.68

446380.174349.44

485965.284328.55

526595.444390.43

566020.934472.56

606022.034762

646587.594739.97

685228.294710.7

726354.115143.24

765737.34916.31

805112.114963.7

845147.854943.27

8858565074.24

925815.845212.75

965254.625126.48

1006149.46

1045644.62

1085169.22

1125152.81

1166032.65

1205245.2

1245103.1

1285465.05

1325828.68

1365469.38

1405655.27

&C&A

&CPage &P

Sheet1

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

Sheet2

# of userstps

10751.6821105.643875.974537.21

203424.663398.763515.565838.99

305317.263435.73535.36814.31

405007.593549.43903.446772.06

504923.443535.174282.297242.28

605270.443571.254300.797811.23

705256.33605.664462.277388.14

805666.753505.894976.887516.68

906252.773485.825016.667994

1005949.353504.795196.998054.03

1105981.253379.575145.828646.53

1205790.943511.125187.087894.5

1305834.463436.165190.238830.25

1406654.233386.495299.799262.67

1506335.243437.584863.48967.48

1606967.773452.055947.248479.33

1706648.893525.95804.059730.07

1805556.153403.729284.568600.7

1906364.353352.028465.017714.75

2006501.063490.179351.517949.65

101105.64

203398.76

303435.7

403549.4

503535.17

603571.25

703605.66

803505.89

903485.82

1003504.79

1103379.57

1203511.12

1303436.16

1403386.49

1503437.58

1603452.05

1703525.9

1803403.72

1903352.02

2003490.17

42379.181364.797

83364.881778.068

123894.522807.29

164386.193116.82

204721.443299.11

245117.273363.01

285520.154262.7

325401.984776.9

365360.85241.13

405987.355596.96

445921.585536.72

485579.035545.41

525527.055516.57

565215.545598.23

605492.665664.91

645251.825601.94

685306.925530.15

725863.195418.68

765741.725506.63

805747.755452.59

845362.485575.75

886336.195450.15

925709.225436.68

965522.155394.49

1005616.135456.98

&C&A

&CPage &P

Sheet2

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

Sheet3


























































&C&A

&CPage &P

Sheet4


























































&C&A

&CPage &P

RAC testing steps (contd)Single node2 node RAC Scalability tests (contd) Problem!!!Possible interconnect bottleneckPerformance -TPS (for all users)# of concurrent users

Chart3

2379.1841364.797

3364.8881778.068

3894.52122807.29

4386.19163116.82

4721.44203299.11

5117.27243363.01

5520.15284262.7

5401.98324776.9

5360.8365241.13

5987.35405296.96

5921.58445736.72

5579.03485945.41

5527.05526116.57

5215.54566398.23

5492.66606964.91

5251.82646701.94

5306.92687030.15

5863.19727418.68

5741.72767206.63

5747.75807752.59

5362.48847975.75

6336.19888050.15

5709.22928236.68

5522.15968594.49

5616.131009056.98

Sheet1


41657.17

83119.67

123831.83

164274.15

205059.45

245942.43

286288.1

326304.06

366435.12

406132.97

446815.34

486553.27

526422.5

566316.79

605789.69

41887.911672.77

83309.22994.85

124211.763468.63

165395.723621.45

205801.313649.3

245396.84251.86

285484.824184.57

326547.314258.77

365966.034214.36

406109.324336.63

445522.284256.25

486164.714443.89

526390.924479.34

565395.554665.66

606035.664656.49

646299.944729.49

686272.344757.2

724942.275042.58

4453.1941874.63

82060.862779.59

123777.543515.2

164373.083658.19

205022.924187.82

245497.234220.86

286098.084319.08

325876.614380.84

366389.434059.83

406593.864163.68

446380.174349.44

485965.284328.55

526595.444390.43

566020.934472.56

606022.034762

646587.594739.97

685228.294710.7

726354.115143.24

765737.34916.31

805112.114963.7

845147.854943.27

8858565074.24

925815.845212.75

965254.625126.48

1006149.46

1045644.62

1085169.22

1125152.81

1166032.65

1205245.2

1245103.1

1285465.05

1325828.68

1365469.38

1405655.27

&C&A

&CPage &P

Sheet1

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

0000

Sheet2

# of userstps

10751.6821105.643875.974537.21

203424.663398.763515.565838.99

305317.263435.73535.36814.31

405007.593549.43903.446772.06

504923.443535.174282.297242.28

605270.443571.254300.797811.23

705256.33605.664462.277388.14

805666.753505.894976.887516.68

906252.773485.825016.667994

1005949.353504.795196.998054.03

1105981.253379.575145.828646.53

1205790.943511.125187.087894.5

1305834.463436.165190.238830.25

1406654.233386.495299.799262.67

1506335.243437.584863.48967.48

1606967.773452.055947.248479.33

1706648.893525.95804.059730.07

1805556.153403.729284.568600.7

1906364.353352.028465.017714.75

2006501.063490.179351.517949.65

101105.64

203398.76

303435.7

403549.4

503535.17

603571.25

703605.66

803505.89

903485.82

1003504.79

1103379.57

1203511.12

1303436.16

1403386.49

1503437.58

1603452.05

1703525.9

1803403.72

1903352.02

2003490.17

42379.181364.797

83364.881778.068

123894.522807.29

164386.193116.82

204721.443299.11

245117.273363.01

285520.154262.7

325401.984776.9

365360.85241.13

405987.355296.96

445921.585736.72

485579.035945.41

525527.056116.57

565215.546398.23

605492.666964.91

645251.826701.94

685306.927030.15

725863.197418.68

765741.727206.63

805747.757752.59

845362.487975.75

886336.198050.15

925709.228236.68

965522.158594.49

1005616.139056.98

&C&A

&CPage &P

Sheet2

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

000

Sheet3


























































&C&A

&CPage &P

Sheet4


























































&C&A

&CPage &P

Digging into RAC performance problemsInterconnect and shared storage are the two most possible performance problem areas in RACInterconnect speedThroughputLatencyaverage latency of a consistent block request. AVG CR BLOCK RECEIVE TIME should typically be about 15 milliseconds depending -- on your system configuration and volume

Digging into RAC performance problems (contd)Interconnect types

Digging into RAC performance problems (contd)cluster_interconnects parameterIt provides Oracle with information about additional cluster interconnects available for use and can be used to load balance the interconnect traffic to different physical interconnects thus increasing interconnect bandwith.

When you set CLUSTER_INTERCONNECTS in cluster configurations, the interconnect high availability features are not available. In other words, an interconnect failure that is normally unnoticeable would instead cause an Oracle cluster failure as Oracle still attempts to access the network interface which has gone down.

Digging into RAC performance problems (contd)STATSPACK reportsThe STATSPACK report show statistics ONLY for the node or instance on which it was runTop 5 Timed EventsGlobal Cache Service and Global Enqueue Service

Note:135714.1 Script to Collect RAC Diagnostic Information (racdiag.sql)

Digging into RAC performance problems (contd)BAD PERFORMANCE

Top 5 Timed Events~~~~~~~~~~~~~~~~~~ % TotalEvent Waits Time (s) Ela Time--------------------------------------- ------------ ----------- --------global cache cr request 34,568 958 31.44buffer busy global CR 6,513 620 20.35db file sequential read 64,214 455 14.92latch free 13,542 453 14.88buffer busy waits 10,971 295 9.69

GOOD PERFORMANCE

Top 5 Timed Events~~~~~~~~~~~~~~~~~~ % TotalEvent Waits Time (s) Ela Time--------------------------------------- ------------ ----------- --------latch free 10,969 666 51.28buffer busy waits 15,379 159 12.23CPU time 149 11.47PL/SQL lock timer 51 105 8.05db file sequential read 25,163 96 7.36

Digging into RAC performance problems (contd)Global Cache Service - Workload Characteristics BAD GOOD-----------------------------------------------Ave global cache get time (ms): 11.8 2.2 Ave global cache convert time (ms): 51.7 11.2 Ave build time for CR block (ms): 0.7 0.0 Ave flush time for CR block (ms): 0.2 0.2 Ave send time for CR block (ms): 0.0 0.2 Ave time to process CR block request (ms): 0.9 0.4 Ave receive time for CR block (ms): 1.6 0.4 Ave pin time for current block (ms): 0.2 0.2 Ave flush time for current block (ms): 0.0 0.0 Ave send time for current block (ms): 0.1 0.1 Ave time to process current block request (ms): 0.3 0.3 Ave receive time for current block (ms): 33.4 7.5 Global cache hit ratio: 9.5 3.9 Ratio of current block defers: 0.0 0.0 % of messages sent for buffer gets: 6.7 2.5 % of remote buffer gets: 1.8 0.7 Ratio of I/O for coherence: 1.2 1.3 Ratio of local vs remote work: 4.3 4.4 Ratio of fusion vs physical writes: 0.0 0.0

Digging into RAC performance problems (contd)Global Enqueue Service Statistics BAD GOOD---------------------------------Ave global lock get time (ms): 0.2 0.0 Ave global lock convert time (ms): 0.0 0.0 Ratio of global lock gets vs global lock releases: 1.2 1.1

GCS and GES Messaging statistics--------------------------------Ave message sent queue time (ms): 16.5 1.7 Ave message sent queue time on ksxp (ms): 29.4 2.5 Ave message received queue time (ms): 1.9 0.3 Ave GCS message process time (ms): 0.1 0.1 Ave GES message process time (ms): 0.1 0.0 % of direct sent messages: 49.7 63.4 % of indirect sent messages: 50.3 36.6 % of flow controlled messages: 0.0 0.0

Implementing RAC in productionSmooth transition from single instance to RAC

Change ORACLE_HOME

Relinking the RAC Option ON/OFF

CLUSTER_DATABASE = TRUE/FALSE

Start/Stop the second instance

Start gradual movement of clients from one instance to another

Relinking the RAC Option Login as the Oracle software owner and shutdown all database instances on all nodes in the cluster.

2. cd $ORACLE_HOME/rdbms/lib

3. make -f ins_rdbms.mk rac_on (rac_off)

If this step did not fail with fatal errors then proceed to step 4.

4. make -f ins_rdbms.mk ioracle

ReferenceMetalink Note:211177.1 RAC Survival Kit: Rac On / Rac Off - Relinking the RAC Option

Metalink Note:183340.1 Frequently Asked Questions About the CLUSTER_INTERCONNECTS Parameter in 9i.

http://www.fi.muni.cz/~kripac/orac-nbd/

The Oracle-on-Linux VMware Cookbookhttp://www.oracle.com/technology/tech/linux/vmware/cookbook/index.html

Build Your Own Oracle RAC 10g Cluster on Linux and FireWire http://www.oracle.com/technology/pub/articles/hunter_rac10g.html

Note:135714.1 Script to Collect RAC Diagnostic Information (racdiag.sql)

Thank [email protected]

Date post:	26-Sep-2015
Category:	Documents
Upload:	hariprasathdba
View:	225 times
Download:	0 times

173_Oracle RAC From Dream To Production_1.0.0.ppt

Documents