Author – A.Kishore http:/www.appsdba.info
Build Your Own Oracle RAC 11g Release 2 Cluster on Red Hat 4 and iSCSI Here is my setup which I have used for Installing Oracle RAC 11gR2
OpenFiler To access linux1 and linux2 2 GB 500 GB HDD
Linux1 4 GB 250 GB Oracle Apps R12.1.1 Node 1
Linux2 4 GB 250 GB Node 2
Author – A.Kishore http:/www.appsdba.info
High Level Steps > Install openfiler on linux1 using VMWARE > Install Grid Infrastructure on linux1 and linux2 > Install Oracle 11gR2 software on linux1 and linux2 Download the software from the below site http://www.oracle.com/technetwork/database/enterprise-edition/downloads/112010-linuxsoft-085393.html
Author – A.Kishore http:/www.appsdba.info
1. Install Oracle 11gR2 Grid Infrastructure (Both the nodes should be ready) -- clean the old installation (on both nodes) rm -rf /etc/oracle rm -rf /etc/oraInst.loc rm -rf /etc/oratab cd /d01/oracle/app rm -rf oracle rm -rf oraInventory rm -rf /d01/oracle/app/*.* rm -rf /d01/oracle/app/11.2.0/grid/*.* -- On one of the node – Using only one diskgroup /etc/init.d/oracleasm deletedisk VOL1 /dev/sdb1 /etc/init.d/oracleasm createdisk VOL1 /dev/sdb1 /etc/init.d/oracleasm listdisks Note – I am using only one diskgroups -- If there are 3 diskgroups then use the below commands /etc/init.d/oracleasm deletedisk VOL1 /dev/sdc1 /etc/init.d/oracleasm deletedisk VOL2 /dev/sdd1 /etc/init.d/oracleasm deletedisk VOL3 /dev/sde1 /etc/init.d/oracleasm createdisk VOL1 /dev/sdc1 /etc/init.d/oracleasm createdisk VOL2 /dev/sdd1 /etc/init.d/oracleasm createdisk VOL3 /dev/sde1 /etc/init.d/oracleasm listdisks Run cluvfy to verify all the prereqs are available su - root cd /tmp rm -rf bootstrap/ su - oracle cd $SOFTWARE_LOCATION/11gR2/grid sh runcluvfy.sh stage -pre crsinst -n linux1,linux2 –verbose See Appendix A for output Only one package has failed and that is fine, red hat comes with older version of glibc, but the cluvfy script is looking for the latest script Check: Package existence for "glibc-2.3.4-2.41" Node Name Available Required Comment ------------ ------------------------ ------------------------ ----------
Author – A.Kishore http:/www.appsdba.info
linux1 glibc-2.3.4-2.39 glibc-2.3.4-2.41 failed Result: Package existence check failed for "glibc-2.3.4-2.41"
cd $SOFTWARE_LOCATION/11gR2/grid ./runInstaller
Author – A.Kishore http:/www.appsdba.info
Check NetworkCheck NetworkCheck NetworkCheck Network Requirements: Requirements: Requirements: Requirements: In this release there are two new network related components
– SCAN and GNS.
Single Client Access Name (SCAN) for the Cluster: Single Client Access Name (SCAN) for the Cluster: Single Client Access Name (SCAN) for the Cluster: Single Client Access Name (SCAN) for the Cluster:
During Typical installation, you are prompted to confirm the default Single Client Access
Name (SCAN), which is used to connect to databases within the cluster irrespective of which
nodes they are running on. By default, the name used as the SCAN is also the name of the
cluster. The default value for the SCAN is based on the local node name. If you change the
SCAN from the default, then the name that you use must be globally unique throughout your
enterprise.
In a Typical installation, the SCAN is also the name of the cluster. The SCAN and cluster
name must be at least one character long and no more than 15 characters in length, must be
Author – A.Kishore http:/www.appsdba.info
alphanumeric, and may contain hyphens (-). If you require a SCAN that is longer than 15
characters, then select an Advanced installation.
Configure the following addresses:Configure the following addresses:Configure the following addresses:Configure the following addresses:
• A public IP address for each node
• A virtual IP address for each node
• A single client access name (SCAN) configured on the domain name server (DNS)domain name server (DNS)domain name server (DNS)domain name server (DNS) or Grid Grid Grid Grid
Naming Service (GNS)Naming Service (GNS)Naming Service (GNS)Naming Service (GNS) for Round Robin resolution to three addresses (recommended) or at
least one address (check documentation)
Some more information on Single Client Access Name ☺
Single Client Access Name (SCAN) for the Cluster
If you have ever been tasked with extending an Oracle RAC cluster by adding a new node (or shrinking a RAC cluster by removing a node), then you know the pain of going through a list of all clients and updating their SQL*Net or JDBC configuration to reflect the new or deleted node! To address this problem, Oracle 11g Release 2 introduced a new feature known as Single Client Access Name or SCAN for short. SCAN is a new feature that provides a single host name for clients to access an Oracle Database running in a cluster. Clients using SCAN do not need to change their TNS configuration if you add or remove nodes in the cluster. The SCAN resource and its associated IP address(s) provide a stable name for clients to use for connections, independent of the nodes that make up the cluster. You will be asked to provide the host name and up to three IP addresses to be used for the SCAN resource during the interview phase of the Oracle grid infrastructure installation. For high availability and scalability, Oracle recommends that you configure the SCAN name so that it resolves to three IP addresses. At a minimum, the SCAN must resolve to at least one address.
The SCAN virtual IP name is similar to the names used for a node's virtual IP addresses, such as racnode1-vip. However, unlike a virtual IP, the SCAN is associated with the entire cluster, rather than an individual node, and can be associated with multiple IP addresses, not just one address. Note that SCAN addresses, virtual IP addresses, and public IP addresses must all be on the same subnet.
The SCAN should be configured so that it is resolvable either by using Grid Naming Service (GNS) within the cluster, or by using Domain Name Service (DNS) resolution.
In this article, I will configure SCAN to resolve to only one, manually configured static IP address using the DNS method (but not actually defining it in DNS)
Author – A.Kishore http:/www.appsdba.info
192.168.1.187 racnode-cluster-scan – make the entry in /etc/hosts
Author – A.Kishore http:/www.appsdba.info
Enter Oracle password and click Setup
Author – A.Kishore http:/www.appsdba.info
The above error can safely be ignored
Author – A.Kishore http:/www.appsdba.info
[root@linux1 ~]# /d01/oracle/app/oraInventory/orainstRoot.sh Changing permissions of /d01/oracle/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /d01/oracle/app/oraInventory to oinstall. The execution of the script is complete. [root@linux2 ~]# /d01/oracle/app/oraInventory/orainstRoot.sh Changing permissions of /d01/oracle/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /d01/oracle/app/oraInventory to oinstall. The execution of the script is complete.
Author – A.Kishore http:/www.appsdba.info
Run root.sh on linux1 and linux2 cd /d01/oracle/app/11.2.0/grid/ [root@linux1 grid]# sh root.sh See Appendix B and C for outputs
Click OK
Author – A.Kishore http:/www.appsdba.info
INFO: Checking Single Client Access Name (SCAN)... INFO: Checking name resolution setup for "racnode-cluster-scan"... INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "racnode-cluster-scan" (IP address: 216.24.138.153) failed INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "racnode-cluster-scan" (IP address: 192.168.1.187) failed INFO: ERROR: INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "racnode-cluster-scan" INFO: Verification of SCAN VIP and Listener setup failed
Provided this is the only error reported by the CVU, it would be safe to ignore this check and continue by clicking the [Next] button in OUI and move forward with the Oracle grid infrastructure installation. This is documented in Doc ID: 887471.1 on the My Oracle Support web site.
Author – A.Kishore http:/www.appsdba.info
If on the other hand you want the CVU to complete successfully while still only defining the SCAN in the hosts file, simply modify the nslookup utility as root on both Oracle RAC nodes as follows.
First, rename the original nslookup binary to nslookup.original on both Oracle RAC nodes:
[root@racnode1 ~]# mv /usr/bin/nslookup /usr/bin/nslookup.original
Next, create a new shell script named /usr/bin/nslookup as shown below while replacing 24.154.1.34 with your primary DNS, racnode-cluster-scan with your SCAN host name, and 192.168.1.187 with your SCAN IP address:
#!/bin/bash HOSTNAME=${1} if [[ $HOSTNAME = "racnode-cluster-scan" ]]; then echo "Server: 24.154.1.34" echo "Address: 24.154.1.34#53" echo "Non-authoritative answer:" echo "Name: racnode-cluster-scan" echo "Address: 192.168.1.187" else /usr/bin/nslookup.original $HOSTNAME Fi
Finally, change the new nslookup shell script to executable:
[root@racnode1 ~]# chmod 755 /usr/bin/nslookup Remember to perform these actions on both Oracle RAC nodes.
The new nslookup shell script simply echo's back your SCAN IP address whenever the CVU calls nslookup with your SCAN host name; otherwise, it calls the original nslookup binary.
The CVU will now pass during the Oracle grid infrastructure installation when it attempts to verify your SCAN:
[grid@racnode1 ~]$ cluvfy comp scan -verbose Verifying scan Checking Single Client Access Name (SCAN)... SCAN VIP name Node Running? ListenerName Port Running?
Author – A.Kishore http:/www.appsdba.info
---------------- ------------ ------------ ------------ ------------ ------------ racnode-cluster-scan racnode1 true LISTENER 1521 true Checking name resolution setup for "racnode-cluster-scan"... SCAN Name IP Address Status Comment ------------ ------------------------ ------------------------ ---------- racnode-cluster-scan 192.168.1.187 passed Verification of SCAN VIP and Listener setup passed Verification of scan was successful. =============================================================================== [grid@racnode2 ~]$ cluvfy comp scan -verbose Verifying scan Checking Single Client Access Name (SCAN)... SCAN VIP name Node Running? ListenerName Port Running? ---------------- ------------ ------------ ------------ ------------ ------------ racnode-cluster-scan racnode1 true LISTENER 1521 true Checking name resolution setup for "racnode-cluster-scan"... SCAN Name IP Address Status Comment ------------ ------------------------ ------------------------ ---------- racnode-cluster-scan 192.168.1.187 passed Verification of SCAN VIP and Listener setup passed Verification of scan was successful.
Now retry
Author – A.Kishore http:/www.appsdba.info
cat grid.env export ORACLE_HOME=/d01/oracle/app/11.2.0/grid export PATH=$PATH:$ORACLE_HOME/bin . ./grid.env crs_stat –t
Post-Install Actions
By default, the Global Services Daemon (GSD) is not started on the cluster. To start GSD, change directory to the <CRS_HOME> and issue the following commands:
srvctl enable nodeapps –g srvctl start nodeapps
Author – A.Kishore http:/www.appsdba.info
ora.oc4j is offine, that is fine Successful Oracle Clusterware operation can also be verified using the following command: crsctl check crs
olsnodes -n
Author – A.Kishore http:/www.appsdba.info
srvctl status asm -a
ocrcheck
crsctl query css votedisk
Author – A.Kishore http:/www.appsdba.info
References: Thanks to Jeff Hunter, without his help I couldn’t have done this assignment http://www.rachelp.nl/index_kb.php?menu=articles&actie=show&id=61 11gR2 Grid: root.sh fails to start the clusterware on the second node (Doc ID 981357.1)
1. short-term: disable the firewall on all nodes, on Linux this can be done by running the following command(s) as the root user on each node of the cluster:
service iptables stop service ip6tables stop
To permanently disable the firewall, use:
chkconfig iptables off chkconfig ip6tables off
2. long-term: exclude all traffic on the private network from the firewall configuration.
http://coskan.wordpress.com/ http://coskan.wordpress.com/2009/12/07/root-sh-failed-after-asm-disk-creation-for-11gr2-grid-infrastructure/ http://oracle-base.com/forums/viewtopic.php?f=1&t=11307&start=0
1. How to Proceed from Failed 11gR2 Grid Infrastructure (CRS) Installation [ID 942166.1]
Issues and solutions cd /home/oracle/app/11.2.0/grid/log/linux2 cat alertlinux2.log [ctssd(27935)]CRS-2409:The clock on host linux2 is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization Service is running in observer mode.
Author – A.Kishore http:/www.appsdba.info
2010-07-24 05:26:20.471 [ctssd(27935)]CRS-2412:The Cluster Time Synchronization Service detects that the local time is significantly different from the mean cluster time. Details in /home/oracle/app/11.2.0/grid/log/linux2/ctssd/octssd.log. 2010-07-24 05:26:20.471 [ctssd(27935)]CRS-2409:The clock on host linux2 is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization Service is running in observer mode. http://www.oracle.com/technology/pub/articles/hunter-rac11gr2-iscsi.html http://arjudba.blogspot.com/2010/03/in-11gr2-grid-rootsh-fails-with-crs.html How to Proceed from Failed 11gR2 Grid Infrastructure (CRS) Installation [ID 942166.1] -- If root.sh has failed, run the below commands to clean up -- first node cd /home/oracle/app/11.2.0/grid/crs/install ./rootcrs.pl -verbose -deconfig -force -- second node cd /home/oracle/app/11.2.0/grid/crs/install ./rootcrs.pl -verbose -deconfig -force -lastnode To add vip manually [root@linux2 bin]# ./srvctl add vip -n linux2 -A 192.168.0.201/255.255.255.0/eth0 -k 1 [root@linux2 bin]# ./srvctl config vip -n linux2 http://forums.oracle.com/forums/thread.jspa?messageID=4039293 rpm -q binutils-2.15.92.0.2 \ compat-libstdc++-33.2.3 \ elfutils-libelf-0.97 \ elfutils-libelf-devel-0.97 \ gcc-3.4.6 \ gcc-c++-3.4.6 \ glibc-2.3.4-2.41 \ glibc-common-2.3.4 \ glibc-devel-2.3.4 \ glibc-headers-2.3.4 \ libaio-devel-0.3.105 \
Author – A.Kishore http:/www.appsdba.info
libaio-0.3.105 \ libgcc-3.4.6 \ libstdc++-3.4.6 \ libstdc++-devel-3.4.6 \ make-3.80 \ pdksh-5.2.14 \ sysstat-5.0.5 \ unixODBC-2.2.11 \ unixODBC-devel-2.2.11 http://forums.oracle.com/forums/thread.jspa?messageID=4345930 http://www.oracle.com/technology/pub/articles/wartak-rac-vm_3.html http://wiki.oracle.com/page/11gR2+RAC+on+a+Mac+-+Part+6 –nice one Appendix A sh runcluvfy.sh stage -pre crsinst -n linux1,linux2 -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "linux1" Destination Node Reachable? ------------------------------------ ------------------------ linux1 yes linux2 yes Result: Node reachability check passed from node "linux1" Checking user equivalence... Check: User equivalence for user "oracle" Node Name Comment ------------------------------------ ------------------------ linux2 failed linux1 passed Result: PRVF-4007 : User equivalence check failed for user "oracle" WARNING: User equivalence is not set for nodes: linux2 Verification will proceed with nodes: linux1 Checking node connectivity... Checking hosts config file... Node Name Status Comment ------------ ------------------------ ------------------------
Author – A.Kishore http:/www.appsdba.info
linux1 passed Verification of the hosts config file successful Interface information for node "linux1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.3 192.168.1.0 0.0.0.0 192.168.1.1 00:08:A1:95:2F:61 1500 eth1 192.168.0.101 192.168.0.0 0.0.0.0 192.168.1.1 00:13:72:22:4C:78 1500 Check: Node connectivity of subnet "192.168.1.0" Result: Node connectivity passed for subnet "192.168.1.0" with node(s) linux1 Check: TCP connectivity of subnet "192.168.1.0" Result: TCP connectivity check passed for subnet "192.168.1.0" Check: Node connectivity of subnet "192.168.0.0" Result: Node connectivity passed for subnet "192.168.0.0" with node(s) linux1 Check: TCP connectivity of subnet "192.168.0.0" Result: TCP connectivity check passed for subnet "192.168.0.0" Interfaces found on subnet "192.168.1.0" that are likely candidates for VIP are: linux1 eth0:192.168.1.3 Interfaces found on subnet "192.168.0.0" that are likely candidates for a private interconnect are: linux1 eth1:192.168.0.101 Result: Node connectivity check passed Check: Total memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 3.71GB (3895072.0KB) 1.5GB (1572864.0KB) passed Result: Total memory check passed Check: Available memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 3.32GB (3479748.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Comment ------------ ------------------------ ------------------------ ----------
Author – A.Kishore http:/www.appsdba.info
linux1 5.86GB (6144820.0KB) 3.71GB (3895072.0KB) passed Result: Swap space check passed Check: Free disk space for "linux1:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ ------------ /tmp linux1 / 6.82GB 1GB passed Result: Free disk space check passed for "linux1:/tmp" Check: User existence for "oracle" Node Name Status Comment ------------ ------------------------ ------------------------ linux1 exists passed Result: User existence check passed for "oracle" Check: Group existence for "oinstall" Node Name Status Comment ------------ ------------------------ ------------------------ linux1 exists passed Result: Group existence check passed for "oinstall" Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ linux1 exists passed Result: Group existence check passed for "dba" Check: Membership of user "oracle" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Comment ---------------- ------------ ------------ ------------ ------------ ------------ linux1 yes yes yes yes passed Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed Check: Membership of user "oracle" in group "dba" Node Name User Exists Group Exists User in Group Comment ---------------- ------------ ------------ ------------ ---------------- linux1 yes yes yes passed Result: Membership check for user "oracle" in group "dba" passed Check: Run level Node Name run level Required Comment ------------ ------------------------ ------------------------ ---------- linux1 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- linux1 hard 65536 65536 passed Result: Hard limits check passed for "maximum open file descriptors"
Author – A.Kishore http:/www.appsdba.info
Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- linux1 soft 65536 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- linux1 hard 61416 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- linux1 soft 61416 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 i686 x86 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 2.6.9-67.EL 2.6.9 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 256 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 100 100 passed Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 142 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 4294967295 536870912 passed
Author – A.Kishore http:/www.appsdba.info
Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 2097152 2097152 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 6815744 6815744 passed Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 between 9000 & 65500 between 9000 & 65500 passed Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 262144 262144 passed Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 4194304 4194304 passed Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 262144 262144 passed Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 1048576 1048576 passed Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- linux1 1048576 1048576 passed Result: Kernel parameter check passed for "aio-max-nr" Check: Package existence for "make-3.80" Node Name Available Required Comment
Author – A.Kishore http:/www.appsdba.info
------------ ------------------------ ------------------------ ---------- linux1 make-3.80-6.EL4 make-3.80 passed Result: Package existence check passed for "make-3.80" Check: Package existence for "binutils-2.15.92.0.2" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 binutils-2.15.92.0.2-24 binutils-2.15.92.0.2 passed Result: Package existence check passed for "binutils-2.15.92.0.2" Check: Package existence for "gcc-3.4.6" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 gcc-3.4.6-9 gcc-3.4.6 passed Result: Package existence check passed for "gcc-3.4.6" Check: Package existence for "gcc-c++-3.4.6" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 gcc-c++-3.4.6-9 gcc-c++-3.4.6 passed Result: Package existence check passed for "gcc-c++-3.4.6" Check: Package existence for "libaio-0.3.105" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 libaio-0.3.105-2 libaio-0.3.105 passed Result: Package existence check passed for "libaio-0.3.105" Check: Package existence for "glibc-2.3.4-2.41" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 glibc-2.3.4-2.39 glibc-2.3.4-2.41 failed Result: Package existence check failed for "glibc-2.3.4-2.41" Check: Package existence for "compat-libstdc++-33-3.2.3" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 compat-libstdc++-33-3.2.3-47.3 compat-libstdc++-33-3.2.3 passed Result: Package existence check passed for "compat-libstdc++-33-3.2.3" Check: Package existence for "elfutils-libelf-0.97" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 elfutils-libelf-0.97.1-5 elfutils-libelf-0.97 passed Result: Package existence check passed for "elfutils-libelf-0.97" Check: Package existence for "elfutils-libelf-devel-0.97" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 elfutils-libelf-devel-0.97.1-5 elfutils-libelf-devel-0.97 passed Result: Package existence check passed for "elfutils-libelf-devel-0.97" Check: Package existence for "glibc-common-2.3.4" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 glibc-common-2.3.4-2.39 glibc-common-2.3.4 passed
Author – A.Kishore http:/www.appsdba.info
Result: Package existence check passed for "glibc-common-2.3.4" Check: Package existence for "glibc-devel-2.3.4" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 glibc-devel-2.3.4-2.39 glibc-devel-2.3.4 passed Result: Package existence check passed for "glibc-devel-2.3.4" Check: Package existence for "glibc-headers-2.3.4" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 glibc-headers-2.3.4-2.39 glibc-headers-2.3.4 passed Result: Package existence check passed for "glibc-headers-2.3.4" Check: Package existence for "libaio-devel-0.3.105" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 libaio-devel-0.3.105-2 libaio-devel-0.3.105 passed Result: Package existence check passed for "libaio-devel-0.3.105" Check: Package existence for "libgcc-3.4.6" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 libgcc-3.4.6-9 libgcc-3.4.6 passed Result: Package existence check passed for "libgcc-3.4.6" Check: Package existence for "libstdc++-3.4.6" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 libstdc++-3.4.6-9 libstdc++-3.4.6 passed Result: Package existence check passed for "libstdc++-3.4.6" Check: Package existence for "libstdc++-devel-3.4.6" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 libstdc++-devel-3.4.6-9 libstdc++-devel-3.4.6 passed Result: Package existence check passed for "libstdc++-devel-3.4.6" Check: Package existence for "sysstat-5.0.5" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 sysstat-5.0.5-16.rhel4 sysstat-5.0.5 passed Result: Package existence check passed for "sysstat-5.0.5" Check: Package existence for "unixODBC-2.2.11" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 unixODBC-2.2.11-1.RHEL4.1 unixODBC-2.2.11 passed Result: Package existence check passed for "unixODBC-2.2.11" Check: Package existence for "unixODBC-devel-2.2.11" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 unixODBC-devel-2.2.11-1.RHEL4.1 unixODBC-devel-2.2.11 passed Result: Package existence check passed for "unixODBC-devel-2.2.11" Check: Package existence for "pdksh-5.2.14"
Author – A.Kishore http:/www.appsdba.info
Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 pdksh-5.2.14-30.6 pdksh-5.2.14 passed Result: Package existence check passed for "pdksh-5.2.14" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Check: Current group ID Result: Current group ID check passed Checking Core file name pattern consistency... Core file name pattern consistency check passed. Checking to make sure user "oracle" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ linux1 does not exist passed Result: User "oracle" is not part of "root" group. Check passed Check default user file creation mask Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- linux1 0022 0022 passed Result: Default user file creation mask check passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... The NTP configuration file "/etc/ntp.conf" is available on all nodes NTP Configuration file check passed Checking daemon liveness... Check: Liveness for "ntpd" Node Name Running? ------------------------------------ ------------------------ linux1 yes Result: Liveness check passed for "ntpd" Checking NTP daemon command line for slewing option "-x" Check: NTP daemon command line Node Name Slewing Option Set? ------------------------------------ ------------------------ linux1 yes Result: NTP daemon slewing option check passed Checking NTP daemon's boot time configuration, in file "/etc/sysconfig/ntpd", for slewing option "-x" Check: NTP daemon's boot time configuration Node Name Slewing Option Set? ------------------------------------ ------------------------ linux1 yes Result: NTP daemon's boot time configuration check for slewing option passed NTP common Time Server Check started...
Author – A.Kishore http:/www.appsdba.info
NTP Time Server "202.71.140.36" is common to all nodes on which the NTP daemon is running NTP Time Server "209.81.9.7" is common to all nodes on which the NTP daemon is running NTP Time Server ".INIT." is common to all nodes on which the NTP daemon is running NTP Time Server "218.45.21.199" is common to all nodes on which the NTP daemon is running Check of common NTP Time Server passed Clock time offset check from NTP Time Server started... Checking on nodes "[linux1]"... Check: Clock time offset from NTP Time Server Time Server: 202.71.140.36 Time Offset Limit: 1000.0 msecs Node Name Time Offset Status ------------ ------------------------ ------------------------ linux1 1.711 passed Time Server "202.71.140.36" has time offsets that are within permissible limits for nodes "[linux1]". Time Server: 209.81.9.7 Time Offset Limit: 1000.0 msecs Node Name Time Offset Status ------------ ------------------------ ------------------------ linux1 11.497 passed Time Server "209.81.9.7" has time offsets that are within permissible limits for nodes "[linux1]". Time Server: .INIT. Time Offset Limit: 1000.0 msecs Node Name Time Offset Status ------------ ------------------------ ------------------------ linux1 0.0 passed Time Server ".INIT." has time offsets that are within permissible limits for nodes "[linux1]". Time Server: 218.45.21.199 Time Offset Limit: 1000.0 msecs Node Name Time Offset Status ------------ ------------------------ ------------------------ linux1 23.78 passed Time Server "218.45.21.199" has time offsets that are within permissible limits for nodes "[linux1]". Clock time offset check passed Result: Clock synchronization check using Network Time Protocol(NTP) passed Pre-check for cluster services setup was unsuccessful on all the nodes. _]0;oracle@linux1:/d01/oracle/RAC/11gR2/grid_[oracle@linux1 grid]$ Script done on Tue 27 Jul 2010 08:02:17 AM IST
Appendix B
Author – A.Kishore http:/www.appsdba.info
cd /d01/oracle/app/11.2.0/grid/ [root@linux1 grid]# script akr.txt Script started, file is akr.txt [root@linux1 grid]# sh root.sh Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /d01/oracle/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying dbhome to /usr/local/bin ... The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying oraenv to /usr/local/bin ... The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. 2010-07-27 08:46:51: Parsing the host name 2010-07-27 08:46:51: Checking for super user privileges 2010-07-27 08:46:51: User has super user privileges Using configuration parameter file: /d01/oracle/app/11.2.0/grid/crs/install/crsc onfig_params Creating trace directory LOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys
Author – A.Kishore http:/www.appsdba.info
pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert Adding daemon to inittab CRS-4123: Oracle High Availability Services has been started. ohasd is starting ADVM/ACFS is not supported on Redhat 4 CRS-2672: Attempting to start 'ora.gipcd' on 'linux1' CRS-2672: Attempting to start 'ora.mdnsd' on 'linux1' CRS-2676: Start of 'ora.gipcd' on 'linux1' succeeded CRS-2676: Start of 'ora.mdnsd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'linux1' CRS-2676: Start of 'ora.gpnpd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'linux1' CRS-2676: Start of 'ora.cssdmonitor' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'linux1' CRS-2672: Attempting to start 'ora.diskmon' on 'linux1' CRS-2676: Start of 'ora.diskmon' on 'linux1' succeeded CRS-2676: Start of 'ora.cssd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'linux1' CRS-2676: Start of 'ora.ctssd' on 'linux1' succeeded ASM created and started successfully. DiskGroup DATA created successfully. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. CRS-2672: Attempting to start 'ora.crsd' on 'linux1'
Author – A.Kishore http:/www.appsdba.info
CRS-2676: Start of 'ora.crsd' on 'linux1' succeeded CRS-4256: Updating the profile Successful addition of voting disk 0482638d4e5d4f0fbfd40e025d6ca900. Successful addition of voting disk 63878fa4923f4f5dbfadf6a8be49a18a. Successful addition of voting disk 433cdc1d1b7f4f1fbf7f126bc0452222. Successfully replaced voting disk group with +DATA. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 0482638d4e5d4f0fbfd40e025d6ca900 (ORCL:VOL1) [DATA] 2. ONLINE 63878fa4923f4f5dbfadf6a8be49a18a (ORCL:VOL2) [DATA] 3. ONLINE 433cdc1d1b7f4f1fbf7f126bc0452222 (ORCL:VOL3) [DATA] Located 3 voting disk(s). CRS-2673: Attempting to stop 'ora.crsd' on 'linux1' CRS-2677: Stop of 'ora.crsd' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'linux1' CRS-2677: Stop of 'ora.asm' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'linux1' CRS-2677: Stop of 'ora.ctssd' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'linux1' CRS-2677: Stop of 'ora.cssdmonitor' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'linux1' CRS-2677: Stop of 'ora.cssd' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'linux1' CRS-2677: Stop of 'ora.gpnpd' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'linux1' CRS-2677: Stop of 'ora.gipcd' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.mdnsd' on 'linux1' CRS-2677: Stop of 'ora.mdnsd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.mdnsd' on 'linux1' CRS-2676: Start of 'ora.mdnsd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'linux1' CRS-2676: Start of 'ora.gipcd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'linux1' CRS-2676: Start of 'ora.gpnpd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'linux1' CRS-2676: Start of 'ora.cssdmonitor' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'linux1' CRS-2672: Attempting to start 'ora.diskmon' on 'linux1' CRS-2676: Start of 'ora.diskmon' on 'linux1' succeeded CRS-2676: Start of 'ora.cssd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'linux1'
Author – A.Kishore http:/www.appsdba.info
CRS-2676: Start of 'ora.ctssd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'linux1' CRS-2676: Start of 'ora.asm' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'linux1' CRS-2676: Start of 'ora.crsd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.evmd' on 'linux1' CRS-2676: Start of 'ora.evmd' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'linux1' CRS-2676: Start of 'ora.asm' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.DATA.dg' on 'linux1' CRS-2676: Start of 'ora.DATA.dg' on 'linux1' succeeded linux1 2010/07/27 08:54:48 /d01/oracle/app/11.2.0/grid/cdata/linux1/back up_20100727_085448.olr Configure Oracle Grid Infrastructure for a Cluster ... succeeded Updating inventory properties for clusterware Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 6000 MB Passed The inventory pointer is located at /etc/oraInst.loc The inventory is located at /d01/oracle/app/oraInventory 'UpdateNodeList' was successful. Appendix C sh root.sh Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /d01/oracle/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed.
Author – A.Kishore http:/www.appsdba.info
2010-07-27 08:57:25: Parsing the host name 2010-07-27 08:57:25: Checking for super user privileges 2010-07-27 08:57:25: User has super user privileges Using configuration parameter file: /d01/oracle/app/11.2.0/grid/crs/install/crsc onfig_params Creating trace directory LOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Adding daemon to inittab CRS-4123: Oracle High Availability Services has been started. ohasd is starting ADVM/ACFS is not supported on Redhat 4 CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS d aemon on node linux1, number 1, and is terminating An active cluster was found during exclusive startup, restarting to join the clu ster CRS-2672: Attempting to start 'ora.mdnsd' on 'linux2' CRS-2676: Start of 'ora.mdnsd' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'linux2' CRS-2676: Start of 'ora.gipcd' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'linux2' CRS-2676: Start of 'ora.gpnpd' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'linux2' CRS-2676: Start of 'ora.cssdmonitor' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'linux2' CRS-2672: Attempting to start 'ora.diskmon' on 'linux2' CRS-2676: Start of 'ora.diskmon' on 'linux2' succeeded CRS-2676: Start of 'ora.cssd' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'linux2' CRS-2676: Start of 'ora.ctssd' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.asm' on 'linux2' CRS-2676: Start of 'ora.asm' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'linux2' CRS-2676: Start of 'ora.crsd' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.evmd' on 'linux2' CRS-2676: Start of 'ora.evmd' on 'linux2' succeeded linux2 2010/07/27 09:01:36 /d01/oracle/app/11.2.0/grid/cdata/linux2/back up_20100727_090136.olr Preparing packages for installation... cvuqdisk-1.0.7-1
Author – A.Kishore http:/www.appsdba.info
Configure Oracle Grid Infrastructure for a Cluster ... succeeded Updating inventory properties for clusterware Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 7994 MB Passed The inventory pointer is located at /etc/oraInst.loc The inventory is located at /d01/oracle/app/oraInventory 'UpdateNodeList' was successful.
