VeritasStorageFoundation for Oracle® RAC Release Notes...Technical Support Symantec Technical...

Veritas Storage Foundation™for Oracle® RAC ReleaseNotes

Solaris

6.0.1

August 2012

Veritas Storage Foundation™ for Oracle RAC ReleaseNotes

The software described in this book is furnished under a license agreement andmay be usedonly in accordance with the terms of the agreement.

Product version: 6.0.1

Document version: 6.0.1 Rev 0

Legal NoticeCopyright © 2012 Symantec Corporation. All rights reserved.

Symantec, the Symantec logo, Veritas, Veritas Storage Foundation, CommandCentral,NetBackup, Enterprise Vault, and LiveUpdate are trademarks or registered trademarks ofSymantec corporation or its affiliates in the U.S. and other countries. Other names may betrademarks of their respective owners.

The product described in this document is distributed under licenses restricting its use,copying, distribution, and decompilation/reverse engineering. No part of this documentmay be reproduced in any form by any means without prior written authorization ofSymantec Corporation and its licensors, if any.

THEDOCUMENTATIONISPROVIDED"ASIS"ANDALLEXPRESSORIMPLIEDCONDITIONS,REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TOBELEGALLYINVALID.SYMANTECCORPORATIONSHALLNOTBELIABLEFORINCIDENTALOR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING,PERFORMANCE, OR USE OF THIS DOCUMENTATION. THE INFORMATION CONTAINEDIN THIS DOCUMENTATION IS SUBJECT TO CHANGE WITHOUT NOTICE.

The Licensed Software andDocumentation are deemed to be commercial computer softwareas defined in FAR12.212 and subject to restricted rights as defined in FARSection 52.227-19"Commercial Computer Software - Restricted Rights" and DFARS 227.7202, "Rights inCommercial Computer Software or Commercial Computer Software Documentation", asapplicable, and any successor regulations. Any use, modification, reproduction release,performance, display or disclosure of the Licensed Software andDocumentation by theU.S.Government shall be solely in accordance with the terms of this Agreement.

Symantec Corporation350 Ellis StreetMountain View, CA 94043

http://www.symantec.com

http://www.symantec.com

Technical SupportSymantec Technical Support maintains support centers globally. TechnicalSupport’s primary role is to respond to specific queries about product featuresand functionality. TheTechnical Support group also creates content for our onlineKnowledge Base. The Technical Support group works collaboratively with theother functional areas within Symantec to answer your questions in a timelyfashion. For example, theTechnical Support groupworkswithProductEngineeringand Symantec Security Response to provide alerting services and virus definitionupdates.

Symantec’s support offerings include the following:

■ A range of support options that give you the flexibility to select the rightamount of service for any size organization

■ Telephone and/or Web-based support that provides rapid response andup-to-the-minute information

■ Upgrade assurance that delivers software upgrades

■ Global support purchased on a regional business hours or 24 hours a day, 7days a week basis

■ Premium service offerings that include Account Management Services

For information about Symantec’s support offerings, you can visit our Web siteat the following URL:

www.symantec.com/business/support/index.jsp

All support services will be delivered in accordance with your support agreementand the then-current enterprise technical support policy.

Contacting Technical SupportCustomers with a current support agreement may access Technical Supportinformation at the following URL:

www.symantec.com/business/support/contact_techsupp_static.jsp

Before contacting Technical Support, make sure you have satisfied the systemrequirements that are listed in your product documentation. Also, you should beat the computer onwhich theproblemoccurred, in case it is necessary to replicatethe problem.

When you contact Technical Support, please have the following informationavailable:

■ Product release level

http://www.symantec.com/business/support/index.jsphttp://www.symantec.com/business/support/contact_techsupp_static.jsp

■ Hardware information

■ Available memory, disk space, and NIC information

■ Operating system

■ Version and patch level

■ Network topology

■ Router, gateway, and IP address information

■ Problem description:

■ Error messages and log files

■ Troubleshooting that was performed before contacting Symantec

■ Recent software configuration changes and network changes

Licensing and registrationIf yourSymantecproduct requires registrationor a licensekey, access our technicalsupport Web page at the following URL:

www.symantec.com/business/support/

Customer serviceCustomer service information is available at the following URL:

www.symantec.com/business/support/

Customer Service is available to assist with non-technical questions, such as thefollowing types of issues:

■ Questions regarding product licensing or serialization

■ Product registration updates, such as address or name changes

■ General product information (features, language availability, local dealers)

■ Latest information about product updates and upgrades

■ Information about upgrade assurance and support contracts

■ Information about the Symantec Buying Programs

■ Advice about Symantec's technical support options

■ Nontechnical presales questions

■ Issues that are related to CD-ROMs or manuals

http://www.symantec.com/business/support/http://www.symantec.com/business/support/

Support agreement resourcesIf youwant to contact Symantec regarding an existing support agreement, pleasecontact the support agreement administration team for your region as follows:

[email protected] and Japan

[email protected], Middle-East, and Africa

[email protected] America and Latin America

DocumentationProduct guides are available on the media in PDF format. Make sure that you areusing the current version of the documentation. The document version appearson page 2 of each guide. The latest product documentation is available on theSymantec Web site.

https://sort.symantec.com/documents

Your feedback on product documentation is important to us. Send suggestionsfor improvements and reports on errors or omissions. Include the title anddocument version (located on the second page), and chapter and section titles ofthe text on which you are reporting. Send feedback to:

[email protected]

For information regarding the latest HOWTO articles, documentation updates,or to ask a question regarding product documentation, visit the Storage andClustering Documentation forum on Symantec Connect.

https://www-secure.symantec.com/connect/storage-management/forums/storage-and-clustering-documentation

About Symantec ConnectSymantec Connect is the peer-to-peer technical community site for Symantec’senterprise customers. Participants can connect and share informationwith otherproduct users, including creating forum posts, articles, videos, downloads, blogsand suggesting ideas, as well as interact with Symantec product teams andTechnical Support. Content is rated by the community, and members receivereward points for their contributions.

http://www.symantec.com/connect/storage-management

mailto:[email protected]:[email protected]:[email protected]://sort.symantec.com/documentsmailto:[email protected]://www-secure.symantec.com/connect/storage-management/forums/storage-and-clustering-documentationhttps://www-secure.symantec.com/connect/storage-management/forums/storage-and-clustering-documentationhttp://www.symantec.com/connect/storage-management

Storage Foundation forOracle RAC Release Notes

This document includes the following topics:

■ About this document

■ Component product release notes

■ About Veritas Storage Foundation for Oracle RAC

■ About Symantec Operations Readiness Tools

■ Important release information

■ Changes introduced in SF Oracle RAC 6.0.1

■ No longer supported

■ System requirements

■ Fixed issues

■ Known issues

■ Software limitations

■ Documentation

About this documentThis document provides important information aboutVeritas Storage Foundationfor Oracle RAC (SF Oracle RAC) version 6.0.1 for Solaris. Review this entiredocument before you install or upgrade SF Oracle RAC.

The information in the Release Notes supersedes the information provided in theproduct documents for SF Oracle RAC.

This is "Document version: 6.0.1 Rev 0" of the Veritas Storage Foundation forOracleRACReleaseNotes. Before you start,make sure that you are using the latestversion of this guide. The latest product documentation is available on theSymantec Web site at:


Component product release notesProduct guides are available at the following location on the software media inPDF formats:

/docs/product_name

Symantec recommends copying the files to the /opt/VRTS/docsdirectory on yoursystem.

For information regarding software features, limitations, fixed issues, andknownissues of component products:

■ Veritas Cluster Server (VCS)See Veritas Cluster Server Release Notes (6.0.1).

■ Storage Foundation (SF)See Veritas Storage Foundation Release Notes (6.0.1).

■ Storage Foundation Cluster File System High Availability (6.0.1)See Veritas Storage Foundation Cluster File System High Availability ReleaseNotes (6.0.1).

About Veritas Storage Foundation for Oracle RACVeritas Storage Foundation™ for Oracle® RAC (SF Oracle RAC) leveragesproprietary storage management and high availability technologies to enablerobust, manageable, and scalable deployment of Oracle RAC on UNIX platforms.The solution uses Veritas Cluster File System technology that provides the dualadvantage of easy file systemmanagement aswell as the use of familiar operatingsystem tools and utilities in managing databases.

The solution stack comprises the Veritas Cluster Server (VCS), Veritas ClusterVolume Manager (CVM), Veritas Oracle Real Application Cluster Support(VRTSdbac), VeritasOracleDiskManager (VRTSodm), Veritas Cluster File System(CFS), and Veritas Storage Foundation, which includes the base Veritas VolumeManager (VxVM) and Veritas File System (VxFS).

Storage Foundation for Oracle RAC Release NotesComponent product release notes

8


Benefits of SF Oracle RACSF Oracle RAC provides the following benefits:

■ Support for file system-basedmanagement. SF Oracle RAC provides a genericclustered file system technology for storing and managing Oracle data filesas well as other application data.

■ Support for high-availability of cluster interconnects.For Oracle RAC 10g Release 2:The combination of LMX/LLT protocols and the PrivNIC/MultiPrivNIC agentsprovides maximum bandwidth as well as high availability of the clusterinterconnects, including switch redundancy.For Oracle RAC 11g Release 1/Oracle RAC 11g Release 2:ThePrivNIC/MultiPrivNIC agents providemaximumbandwidth aswell as highavailability of the cluster interconnects, including switch redundancy.See the following Technote regarding co-existence of PrivNIC/MultiPrivNICagents with Oracle RAC 11.2.0.2 and later versions:http://www.symantec.com/business/support/index?page=content&id=TECH145261

■ Use of Cluster File System and Cluster Volume Manager for placement ofOracle Cluster Registry (OCR) and voting disks. These technologies providerobust shared block interfaces (for all supportedOracleRACversions) and rawinterfaces (for Oracle RAC 10g Release 2) for placement of OCR and votingdisks.

■ Support for a standardized approach toward application and databasemanagement. Administrators can apply their expertise ofVeritas technologiestoward administering SF Oracle RAC.

■ Increased availability and performance using Veritas Dynamic Multi-Pathing(DMP). DMP provides wide storage array support for protection from failuresand performance bottlenecks in the Host Bus Adapters (HBA), Storage AreaNetwork (SAN) switches, and storage arrays.

■ Easy administration andmonitoring of multiple SF Oracle RAC clusters usingVeritas Operations Manager.

■ VCS OEM plug-in provides a way to monitor SF Oracle RAC resources fromthe OEM console.

■ Improved file system access times using Oracle Disk Manager (ODM).

■ Ability to configureOracleAutomatic StorageManagement (ASM) disk groupsover CVMvolumes to take advantage ofVeritasDynamicMulti-Pathing (DMP).

■ Enhanced scalability and availability with access to multiple Oracle RACinstances per database in a cluster.

9Storage Foundation for Oracle RAC Release NotesAbout Veritas Storage Foundation for Oracle RAC

http://www.symantec.com/business/support/index?page=content&id=TECH145261

■ Support for backup and recovery solutions using volume-level and filesystem-level snapshot technologies, StorageCheckpoints, andDatabaseStorageCheckpoints.

■ Support for space optimization using periodic deduplication in a file systemto eliminate duplicate data without any continuous cost.For more information, see the Veritas Storage Foundation Administrator'sdocumentation.

■ Ability to fail over applicationswithminimumdowntimeusingVeritas ClusterServer (VCS) and Veritas Cluster File System (CFS).

■ Prevention of data corruption in split-brain scenarios with robust SCSI-3Persistent Group Reservation (PGR) based I/O fencing or Coordination PointServer-based I/O fencing. The preferred fencing feature also enables you tospecify how the fencing driver determines the surviving subcluster.

■ Support for sharing applicationdata, in addition toOracle database files, acrossnodes.

■ Support for policy-managed databases in Oracle RAC 11g Release 2.

■ Fast disaster recoverywithminimal downtime and interruption to users.Userscan transition froma local high availability site to awide-area disaster recoveryenvironment with primary and secondary sites. If a site fails, clients that areattached to the failed site can reconnect to a surviving site and resume accessto the shared database.

■ Support for configuring and administering multiple non-global zones withhigh availability to Oracle RAC instances inside the non-global zone.

■ Verification of disaster recovery configuration using fire drill technologywithout affecting production systems.

■ Support for a wide range of hardware replication technologies as well asblock-level replication using VVR.

■ Support for campus clusters with the following capabilities:

■ Consistent detach with Site Awareness

■ Site aware reads with VxVM mirroring

■ Monitoring of Oracle resources

■ Protection against split-brain scenarios

Storage Foundation for Oracle RAC Release NotesAbout Veritas Storage Foundation for Oracle RAC

10

About Symantec Operations Readiness ToolsSymantec Operations Readiness Tools (SORT) is a Web site that automates andsimplifies some of the most time-consuming administrative tasks. SORT helpsyou manage your datacenter more efficiently and get the most out of yourSymantec products.

SORT can help you do the following:

■ List product installation andupgrade requirements, includingoperating system versions, memory, disk space, andarchitecture.

■ Analyze systems to determine if they are ready to install orupgrade Symantec products.

■ Download the latest patches, documentation, and highavailability agents from a central repository.

■ Access up-to-date compatibility lists for hardware, software,databases, and operating systems.

Prepare for your nextinstallation or upgrade

■ Get automatic email notifications about changes to patches,array-specific modules (ASLs/APMs/DDIs/DDLs), and highavailability agents from a central repository.

■ Identify and mitigate system and environmental risks.

■ Display descriptions and solutions for hundreds of Symantecerror codes.

Manage risks

■ Find and download patches based on product version andplatform.

■ List installed Symantec products and license keys.

■ Tune and optimize your environment.

Improve efficiency

Note: Certain features of SORT are not available for all products. Access to SORTis available at no extra cost.

To access SORT, go to:

https://sort.symantec.com

Important release information■ For important updates regarding this release, review the Late-Breaking News

TechNote on the Symantec Technical Support website:http://www.symantec.com/docs/TECH164885

11Storage Foundation for Oracle RAC Release NotesAbout Symantec Operations Readiness Tools

https://sort.symantec.comhttps://sort.symantec.com/http://www.symantec.com/docs/TECH164885

■ For the latest patches available for this release, go to:https://sort.symantec.com/

■ Thehardwarecompatibility list contains informationabout supportedhardwareand is updated regularly. For the latest information on supported hardwarevisit the following URL:http://www.symantec.com/docs/TECH170013Before installing or upgrading Storage Foundation and High AvailabilitySolutions products, review the current compatibility list to confirm thecompatibility of your hardware and software.

Changes introduced in SF Oracle RAC 6.0.1This section lists the changes in SF Oracle RAC 6.0.1.

New versioning process for SFHA Solutions productsSymantec made some changes to simplify the versioning process to ensure thatcustomers have a unified experience when it comes to deploying our differentproducts across Storage, Availability, Backup, Archiving and Enterprise Securityproducts. With this change, all the products will have a 3 digit version, X.Y.Z. Incomplying with this approach the current SFHA Solutions release is available asversion 6.0.1.

New directory location for the documentation on the software mediaThePDF files of the product documentation are now located in the /docsdirectoryon the software media. Within the /docs directory are subdirectories for each ofthe bundled products, which contain the documentation specific to that product.Thesfha_solutionsdirectory containsdocumentation that applies to all products.

Changes related to installation and upgradesThe product installer includes the following changes in SF Oracle RAC 6.0.1.

Locally-installed installation and uninstallation scripts nowinclude the release versionWhen you run local scripts (/opt/VRTS/install) to configure Veritas products, thenames of the installed scripts now include the release version.

Storage Foundation for Oracle RAC Release NotesChanges introduced in SF Oracle RAC 6.0.1

12

https://sort.symantec.com/http://www.symantec.com/docs/TECH170013

Note: If you install your Veritas product from the install media, continue to runthe installsfrac command without including the release version.

To run the script from the installed binaries, run the installsfraccommand.

Where is the current release version with no periods or spaces.

For example, to configure the 6.0.1 version of your product, run this command:

# /opt/VRTS/install/installsfrac601 -configure

VxVM private region backup pre-checks for disk groups priorto upgradeThe installer verifies that recent backups of configuration files of all the diskgroups inVxVMprivate regionhave been saved in the /etc/vx/cbr/bkdirectoryprior to doing an upgrade. If not, a warning message is displayed.

Warning: Backup /etc/vx/cbr/bk directory.

Support for Solaris 11 Automated InstallerYou can use the Oracle Solaris Automated Installer (AI) to install the Solaris 11operating systemonmultiple client systems in anetwork.AI performsahands-freeinstallation (automated installationwithoutmanual interactions) of both x86 andSPARC systems. You can also useAImedia (AI bootable image, provided byOracle,which can be downloaded from the Oracle Web site) to install the Oracle SolarisOS on a single SPARC or x86 platform. All cases require access to a packagerepository on the network to complete the installation.

Additional installation postcheck optionsThe postcheck option has been enhanced to include additional checks.

You can use the installer’s post-check option to perform the following checks:

■ General checks for all products.

■ Checks for Volume Manager (VM).

■ Checks for File System (FS).

■ Checks for Cluster File System (CFS).

13Storage Foundation for Oracle RAC Release NotesChanges introduced in SF Oracle RAC 6.0.1

Support for tunables file templatesYou canuse the installer to create a tunables file template. If you start the installerwith the -tunables option, you see a list of all supported tunables, and the locationof the tunables file template.

Installer support to configure Coordination Point serversYou can now use the -configcps option in the installer to configure CP servers.This functionality to configure CP servers is now integrated with the installer.The configure_cps.pl script used earlier to configure CP servers is nowdeprecated.

You can also configure CP servers by generating response files. You can use the-responsefile '/tmp/sample1.res' option in the installer to configure CPservers.

See the Veritas Cluster Server Installation Guide for more details.

Fast failover of failed links by PrivNIC and MultiPrivNIC agentsThe PrivNIC and MultiPrivNIC agents can now fail over the IP address of a failedlink to an available link when the link status is detected as trouble. If LLT isunable to reach the peer node in two seconds, the link status is marked troubleand the agents fail over the IP address to an available link if the value of the agentattribute EnableUseTroubleState is set to 1 in the type definition of the PrivNICand MultiPrivNIC resource configuration files.

An additional enhancement includes reducing the value of the MonitorIntervalattribute of the agents to the default value of 10 seconds.

Support for Oracle RAC in a zone environmentThis release supports the installation and configuration of multiple non-globalzones in a global zone.

SF Oracle RAC and the necessary agents run in the global zone. Oracle RAC runsin the non-global zone. Youmust configure non-global zoneswith an exclusive-IPzone. The exclusive-IP zonedoesnot share thenetwork interfacewith global-zone.

Using SF Oracle RAC, you can start, stop, and monitor a non-global zone andprovide high availability to Oracle RAC instances inside the non-global zone.


14

SF Oracle RAC installation and configuration checks removed frominstaller

The option SF Oracle RAC installation and configuration checks is nolonger available in the installer.

Changes related to SFDB toolsThe following sections describe the changes related to Storage Foundation forDatabases (SFDB) tools in SF Oracle RAC 6.0.1.

See “Support for creation ofGolden Image snapshots using FlashSnap forOracle”on page 15.

See “Support for Flashsnap at the VVR Secondary site for Oracle” on page 15.

See “Introduction of the Compression Advisor tool for Oracle” on page 15.

Support for creation of Golden Image snapshots usingFlashSnap for OracleIn this release, the SFDB tools support the creation of Golden Image snapshotsusing FlashSnap for Oracle databases.

Onlinemode, third-mirror-break-off type snapshot i.e. online FlashSnap snapshotof a database instance contains all the information needed to create a clone of thedatabase instance. It can act as a template for creating clone database instances.You can thus allocate a FlashSnap snapshot that can be used as a master copy forcreatingoneormore clone instances. The clone instances created fromaFlashSnapimage, termed as the 'golden image', are incremental copies of the master or thegolden image. These depend on the FlashSnap image for their operations.

Support for Flashsnap at the VVR Secondary site for OracleIn this release, the SFDB tools support Flashsnap operation at theVVRSecondarysite for Oracle databases.

Online mode snapshots (i.e. traditional, third-mirror-break-off snapshots) aresupported in VVR replication environment. Also, support for more than onesecondary site is added. For online mode snapshots in VVR environment, IBC(In-Band Control)messages are used to synchronize activities on the Primary andSecondary sites. Snapshot is initiated from VVR Secondary site.

Introduction of the Compression Advisor tool for OracleIn this release, the SFDB tools provide the Compression Advisor tool for Oracledatabases.

15Storage Foundation for Oracle RAC Release NotesChanges introduced in SF Oracle RAC 6.0.1

Veritas File System (VxFS) provides the vxcompress utility that can be used tocompress individual files transparent to the underlying applications. Anapplication reading a compressed file automatically receives the uncompresseddata that is uncompressed in memory only; the on-disk part of the data remainscompressed. If an application writes to a compressed file, parts of the file areuncompressed on disk.

Compression Advisor provides extended compression functionality for Oracledatabase files in Oracle single instance and Oracle RAC environments. TheCompression Advisor command sfae_comp_adm resides in the /opt/VRTS/bindirectory, and it must be run by the DBA user.

Changes to LLTThis release includes the following change to LLT:

Setting the value of peerinact in the /etc/llttab file

Symantec recommendsnot to set the value of peerinact to 0. To achieve the infinitetimeout functionality for peerinact, you must set peerinact to a large value. Thesupported range of value is between 1 through 2147483647.

Changes to I/O fencingThis section covers the new features and changes related to I/O fencing in thisrelease.

Enhancement to the CoordPoint agentThe CoordPoint agent monitors changes to the Coordinator Disk Groupconstitution, such as when a disk is deleted from the Coordinator Disk Group dueto accidental execution of aVxVMadministrative commandor if theVxVMprivateregion of a disk is corrupted.

The agent performs detailed monitoring on the CoordPoint resource and reportsfaults. You can tune the frequency of the detailed monitoring by setting theLevelTwoMonitorFreq attribute introduced in this release. For example, if youset this attribute to 5, the agentmonitors the CoordinatorDiskGroup constitutionin every fifth monitor cycle.

For more information on the CoordPoint agent, see the Veritas Cluster ServerBundled Agents Reference Guide.

For information on configuring the CoordPoint agent using script-based installerandmanually configuring the CoordPoint agent tomonitor coordinator disks, seethe Veritas Cluster Server Installation Guide.


16

For more information on replacing I/O fencing coordinator disks or coordinatordiskgroupwhen the cluster is online, see theVeritasClusterServerAdministrator'sGuide.

No longer supportedThis section lists software versions and features that are no longer supported.Symantec advises customers to minimize the use of these features.

SF Oracle RAC does not support the following:

■ Oracle RAC 11g Release 1 Clusterware

■ Dissimilar version upgrades of SF Oracle RAC componentsFor example, if youhave SFOracle RAC6.0 installed, you cannot upgrade onlyVCS to version 6.0.1.

■ Use of crossover cablesOracle does not support the use of crossover cables for cluster interconnectsdue to the possibility of data corruption and other software limitations.

Note: Crossover cables are however known to function without any issues inSFOracle RAC.While the SFOracle RACTechnical support teammay continueto provide support on related issues for existing deployments, this supportmaybe constrained in some respects as it is no longer a supported configurationby Oracle.

The use of crossover cables is discouraged for new deployments.

■ Bunker replication is not supported in a Cluster Volume Manager (CVM)environment.

Veritas Storage Foundation for Databases (SFDB) tools features whichare no longer supported

The following Storage Foundation for Databases (SFDB) tools features are notsupported in this release:

■ FlashSnap reverse resync

■ Checkpoint policy and Checkpoint quotas

■ Interactive modes in clone and rollback

17Storage Foundation for Oracle RAC Release NotesNo longer supported

System requirementsThis section describes the system requirements for this release.

Important preinstallation informationBefore you install SF Oracle RAC, make sure you have reviewed the followinginformation:

■ Hardware compatibility list for information about supported hardware:http://www.symantec.com/docs/TECH170013

■ Latest information on support for Oracle database versions:http://www.symantec.com/docs/DOC5081

■ Oracle documentation for additional requirements pertaining to your versionof Oracle.

Hardware requirementsDepending on the type of setup planned, make sure you meet the necessaryhardware requirements.

See Table 1-1 on page 18.For basic clusters

See Table 1-2 on page 20.For campus clusters

Table 1-1 Hardware requirements for basic clusters

DescriptionItem

Two to sixteen systems with two or more CPUs.

For details on the additional requirements for Oracle, see the Oracledocumentation.

SF Oracle RACsystems

A DVD drive on one of the nodes in the cluster.DVD drive

SF Oracle RAC requires that all shared storage disks support SCSI-3Persistent Reservations (PR).

Note: The coordinator disk does not store data, so configure the diskas the smallest possible LUN on a disk array to avoid wasting space.The minimum size required for a coordinator disk is 128 MB.

Disks

Storage Foundation for Oracle RAC Release NotesSystem requirements

18

http://www.symantec.com/docs/TECH170013http://www.symantec.com/docs/DOC5081

Table 1-1 Hardware requirements for basic clusters (continued)

DescriptionItem

You can evaluate your systems for available disk space by runningthe product installation program. Navigate to the product directoryon the product disc and run the following command:

# ./installsfrac -precheck node_name

You can also use the Veritas Web-based installation program todetermine the available disk space.

For details on the additional space that is required for Oracle, see theOracle documentation.

Disk space

Each SF Oracle RAC system requires at least 2 GB.

For Oracle RAC requirements, see the Oracle Metalink document:169706.1

RAM

See the Oracle Metalink document: 169706.1Swap space

Two or more private links and one public link.

Linksmust be 100BaseT or gigabit Ethernet directly linking eachnodeto the other node to form a private network that handles directinter-system communication. These links must be of the same type;you cannot mix 100BaseT and gigabit.

Symantec recommends gigabit Ethernet using enterprise-classswitches for the private links.

Oracle requires that all nodes use the IP addresses from the samesubnet.

You can also configure aggregated interfaces.

Network

At least one additional SCSI or Fibre Channel Host Bus Adapter persystem for shared data disks.

Fiber Channel orSCSI host busadapters

Table 1-2 lists the hardware requirements for campus clusters in addition to thebasic cluster requirements.

19Storage Foundation for Oracle RAC Release NotesSystem requirements

Table 1-2 Hardware requirements for campus clusters

DescriptionItem

■ The storage switch (to which each host on a site connects) must haveaccess to storage arrays at all the sites.

■ Volumesmust bemirroredwith storage allocated fromat least two sites.

■ DWDM links are recommended between sites for storage links.

DWDM works at the physical layer and requires multiplexer andde-multiplexer devices.

■ The storage and networks must have redundant-loop access betweeneach node and each storage array to prevent the links from becoming asingle point of failure.

Storage

■ Oracle requires that all nodes use the IP addresses from the same subnet.

■ Symantec recommends a common cross-site physical infrastructure forstorage and LLT private networks.

Network

I/O fencing requires placement of a third coordinator point at a third site.The DWDM can be extended to the third site or the iSCSI LUN at the thirdsite can be used as the third coordination point. Alternatively CoordinationPoint Server can be deployed at the third remote site as an arbitration point.

I/O fencing

Supported Solaris operating systemsThis section lists the supported operating systems for this release of Veritasproducts.

Table 1-3 shows the supported operating systems for this release.

Table 1-3 Supported operating systems

ChipsetsLevelsOperating systems

SPARCUpdate 8, 9, and 10Solaris 10

x86Update 8, 9, and 10Solaris 10

SPARCSRU1 or laterSolaris 11

x86SRU1 or laterSolaris 11

For Storage Foundation for Oracle RAC, all nodes in the cluster must have thesame operating system version and update level.


20

Supported database software

Note: SF Oracle RAC supports only 64-bit Oracle.

The following database versions are supported:

■ Oracle RAC 10g Release 2

■ Oracle RAC 11g Release 1 Database

Note: Oracle 11g Release 1 Clusterware is not supported.


Solaris 10 (SPARC)



Solaris 10 (x64)

11.2.0.3Solaris 11 (SPARC)

11.2.0.3Solaris 11 (x64)

For the latest informationon supportedOracle database versions, see the followingTechnical Support TechNote:

http://www.symantec.com/docs/DOC5081

Support for minor database versions is also documented in the afore-mentionedTechnical Support TechNote.

Additionally, see the following Oracle support site for information on patchesthat may be required by Oracle for each release.

https://support.oracle.com

Veritas File System requirementsVeritas File System requires that the values of the Solaris variableslwp_default_stksize and svc_default_stksize are at least 0x6000 (for Solaris10) and 0x8000 (for Solaris 11).When you install theVeritas File Systempackage,VRTSvxfs, the VRTSvxfs packaging scripts check the values of these variables inthe kernel. If the values are less than the required values, VRTSvxfs increases thevalues andmodifies the /etc/system filewith the required values. If theVRTSvxfsscripts increase the values, the installation proceeds as usual except that youmust reboot and restart the installation program. A message displays if a rebootis required.

21Storage Foundation for Oracle RAC Release NotesSystem requirements

http://www.symantec.com/docs/DOC5081https://support.oracle.com

To avoid an unexpected need for a reboot, verify the values of the variables beforeinstalling Veritas File System. Use the following commands to check the valuesof the variables:

# echo "lwp_default_stksize/X" | mdb -klwp_default_stksize:lwp_default_stksize: 6000

# echo "svc_default_stksize/X" | mdb -ksvc_default_stksize:svc_default_stksize: 6000

For Solaris 10:

# echo "lwp_default_stksize/X" | mdb -klwp_default_stksize:lwp_default_stksize: 8000

# echo "svc_default_stksize/X" | mdb -ksvc_default_stksize:svc_default_stksize: 8000

For Solaris 11:

If the values shown are less than 6000 (for Solaris 10) and less than 8000 (forSolaris 11), you can expect a reboot after installation.

Note: The default value of the svc_default_stksize variable is 0 (zero), whichindicates that the value is set to the value of the lwp_default_stksize variable.In this case, no reboot is required, unless the value of the lwp_default_stksizevariable is too small.

To avoid a reboot after installation, you canmodify the /etc/system file with theappropriate values. Reboot the system prior to installing the packages. Add thefollowing lines to the /etc/system file:

set lwp_default_stksize=0x6000set rpcmod:svc_default_stksize=0x6000

For Solaris 10:

set lwp_default_stksize=0x8000set rpcmod:svc_default_stksize=0x8000

For Solaris 11:


22

Supported replication technologies for global clustersSF Oracle RAC supports the following hardware-based replication andsoftware-based replication technologies for global cluster configurations:

■ EMC SRDF

■ Hitachi TrueCopy

■ IBM Metro Mirror

■ IBM SAN Volume Controller (SVC)

■ EMC MirrorView

Hardware-based replication

■ Veritas Volume Replicator

■ Oracle Data Guard

Software-based replication

Fixed issuesThis section covers the incidents that are fixed in this release.

Issues fixed in SF Oracle RAC 6.0.1Table 1-4 lists the issues fixed in SF Oracle RAC 6.0.1.

Table 1-4 Issues fixed in SF Oracle RAC 6.0.1

DescriptionIncidentnumber

TheSFOracleRAC installer doesnot support theuse of fully qualified domainnames (FQDN). Specifying the fully qualified domain name of a system resultsin the following error:

The node sys1 doesn't seem to be part of the cluster,or CVM is not running on the node sys1.

2585899

If you install and start SFOracleRAC, but later configure SFOracleRACusinginstallvcs, some drivers may not stop successfully when the installerattempts to stop and restart the SF Oracle RAC drivers and processes. Thereason the drivers do not stop is because some dependent SF Oracle RACprocesses may be in the running state.

2329580

23Storage Foundation for Oracle RAC Release NotesFixed issues

Table 1-4 Issues fixed in SF Oracle RAC 6.0.1 (continued)

DescriptionIncidentnumber

Policy-managed Oracle RAC databases fail to come online on some of thenodes in the server pool.

If the cardinality of a policy-managed Oracle RAC database is set to a numberlesser than the number of nodes in the server pool, and if the Oracle agenttries to bring the database online on all the nodes in the server pool, theoperation fails on some of the nodes in the server pool. The resource onrespective nodes move to the faulted state.

2392741

Setting the UseVirtualIP attribute to 1 overwrites the IP address of thevirtual interface on some nodes in the cluster.

2749412

ThePrivNIC/MultiPrivNICagents fail tomatch the exact IP address configuredin the agent configuration with the IP address configured on the system. Asa result, the agent detects thewrong interface as the active interface resultingin a resource fault.

2757032

Removal of SANcable fromanynode in a global cluster setup takes applicationservice groups offline on all nodes.

In a replicated global cluster setup, the removal of SAN cable from any nodein the cluster causes the CFS mount points to fault. As a result, dependentapplication groups are taken offline and replication to the secondary site isadversely affected.

2580393

The PrivNIC resource faults after the UseVirtualIP attribute is set to 1.2734745

TheSFOracleRAC installer fails to set the value of theCSSD resource attributeOfflineWaitLimit to 3.

2740150

Some drivers fail to add to the system.

Sometimes during bootup, someof the drivers fail to add in the systembecauseof add_drv/rem_drv race between ourmoduleswhich are independent of eachother.

2746948

The MultiPrivNIC agent plumbs network interfaces on the global zone.2729806

Storage Foundation for Databases (SFDB) tools fixed issuesTable 1-5 describes the Veritas Storage Foundation for Databases (SFDB) toolsissues fixed in this release.

Storage Foundation for Oracle RAC Release NotesFixed issues

24

Table 1-5 SFDB tools fixed issues

DescriptionFixedissues

If you provide an incorrect host name with the -r option of vxsfadm, thecommand fails with an error message similar to one of the following:

FSM Error: Can't use string ("") as a HASH ref while"strict refs" in use at /opt/VRTSdbed/lib/perl/DBED/SfaeFsm.pmline 776. SFDB vxsfadm ERROR V-81-0609 Repository location isinvalid.

The error messages are unclear.

2585643

TheFlashSnapvalidationoperation failswith the following error if themirrorsfor data volumes and archive log volumes share the same set of disks:

SFAE Error:0642: Storage for diskgroup oradatadg is notsplittable.

2703881(2534422)

After youhavedoneFlashSnap cloningusing a snapplan, any further attemptsto create a clone from the same snapplan using the dbed_vmclonedbcontinue to use the original clone SID, rather than the new SID specifiedusing the new_sid parameter. This issue is also observed when youresynchronize the snapplan, take a snapshot again without specifying thenew clone SID, and then try to clone with the new SID.

2582694(2580318)

The sfae_auth_op -o auth_user command, used for authorizing users,fails with the following error message:

SFDB vxsfadm ERROR V-81-0384 Unable to store credentialsfor

The authentication setup might have been run with a strict umask value,which results in the required files and directories being inaccessible to thenon-root users.

2579929

LLT, GAB, and I/O fencing fixed issuesTable 1-6 lists the fixed issues for LLT, GAB, and I/O fencing.

25Storage Foundation for Oracle RAC Release NotesFixed issues

Table 1-6 LLT, GAB, and I/O fencing fixed issues

DescriptionIncident

vxfen startup script gives error grep: can't open/etc/vxfen.d/data/cp_uid_db.

The error comes because vxfen startup script tries to read a file thatmightnot be present. This error is typically seenwhen starting vxfen for the veryfirst time after installation.

2845244

Setting peerinact value to 0 in the /etc/llttab file floods the system logfile with large number of log messages.

2554167

Vxfenswap fails when LANG is set to a value other than 'C'. The vxfenswaputility internally uses the tr command. If the LANG environment variableis set to something other than C, it may cause improper functioning of thevxfenswap utility.

2699308

On Solaris 11, vxfen startup scripts do not report the correct status offencing .

2726341

Fencingmay start upwith an errormessageopen failed for device:/dev/vxfen in the log. It happens when the fencing startup script triesto access the driver that is still loading into memory. However, fencingcomes up seamlessly in spite of the error message.

2850926

Logs report errors related to mv command when the vxfen service isdisabled.

2699291

Post-install script of the VRTSllt package reports error while attemptingto disable the SMF service system/llt.

2762660

Post-install script of theVRTSvxfenpackage reports errorwhile attemptingto disable the SMF service system/vxfen.

2762660

Known issuesThis section covers the known issues in this release.

For Oracle RAC issues:

See “Oracle RAC issues” on page 27.

For SF Oracle RAC issues:

See “SF Oracle RAC issues” on page 29.

Storage Foundation for Oracle RAC Release NotesKnown issues

26

Oracle RAC issuesThis section lists the known issues in Oracle RAC.

Oracle Grid Infrastructure installation may fail with internaldriver errorThe Oracle Grid Infrastructure installation may fail with the following error:

[INS-20702] Unexpected Internal driver error

Workaround:

Perform one of the following steps depending on the type of installer you use forthe installation:

■ Script-based installerExport the OUI_ARGS environment variable, before you run the SF Oracle RACinstallation program:

export OUI_ARGS=-ignoreInternalDriverError

For more information, see the Oracle Metalink document: 970166.1

■ Web-based installerWhenyou run theWeb-based installer, in theEntertheargumentstobepassedtotheOracleinstaller text box, enter the value -ignoreInternalDriverError.For more information, see the Veritas Storage Foundation for Oracle RACInstallation and Configuration Guide.

During installationor systemstartup,OracleGrid Infrastructuremay fail to startAfter successful installation of Oracle RAC 11g Release 2 Grid Infrastructure,while executing the root.sh script, ohasd may fail to start. Similarly, duringsystemstartup, OracleGrid Infrastructuremay fail to start though theVCS enginelogs may indicate that the cssd resource started Oracle Grid Infrastructuresuccessfully.

The following message may be displayed on running the strace command:

# /usr/bin/strace -ftt -p pid_of_ohasd.bin

14:05:33.527288 open("/var/tmp/.oracle/npohasd",

O_WRONLY

Forpossible causesandworkarounds, see theOracleMetalinkdocument: 1069182.1

27Storage Foundation for Oracle RAC Release NotesKnown issues

Enabling ODM in Oracle RAC 11 Release 2 installations causeserrors (1913013)EnablingODM inOracle RAC11Release 2 installations causes the following error:

'ODM ERROR V-41-4-1-253-12 Not enough space'

Oracle instance may also crash with same error.

The error is observed if the DISM (Dynamic Intimate Shared memory) feature isenabled. In Solaris, the Oracle database uses DISM if it is available on the system,and if the value of the sga_max_size initialization parameter is larger than thesize required for all SGA components combined.

Workaround:Make sure that the file ORACLE_HOME/bin/oradism is owned by theroot user with "execute" and "setuid" permissions. If the problem persists aftercorrecting the permissions, uncomment the sga_max_size and memory_targetinit.ora parameters.

Oracle VIPConfigurationAssistant failswith an errormessage(1182220)During Oracle RAC 10g Release 2 installation, the VIP Configuration Assistantmay fail with the following error message:

The given interface(s), "net0" is not public.

Public interfaces should be used to configure virtual IPs.

This message appears only when the VIP is not from the regular public IP range(for example, 200.).

Workaround: Invoke the vipca utility manually as the superuser.

# export DISPLAY=nebula:0.0

# $CRS_HOME/bin/vipca

Oracle Cluster Verification utility displays a warning messageDuring the final stage of Oracle RAC 10g Release 2 installation, you may receivea warning message with the Oracle Cluster Verification utility.

For example:

Utility

============================================================

OUI-25031: Some of the configuration assistants failed. It is

strongly recommended that you retry the configuration

assistants at this time. Not successfully running any "


28

Recommended" assistants means your system will not be correctly

configured.

1. Check the Details panel on the Configuration Assistant Screen

to see the errors resulting in the failures.

2. Fix the errors causing these failures.

3. Select the failed assistants and click the 'Retry' button

to retry them.

============================================================

Workaround: You may safely ignore this message if the cluster is operatingsatisfactorily.

Oracle Database Configuration Assistant displays an errorThe Database Configuration Assistant utility displays the following error:

SGA size cannot be greater than maximum shared memory

segment size (0).

Workaround: Ignore this message and manually configure the database memoryparameters for Oracle. In the "Memory" tab of the Oracle Database CreationAssistant (DBCA), select a Custom and Manual shared memory managementconfiguration and enter the appropriate values.

SF Oracle RAC issuesThis section lists the known issues in SF Oracle RAC for this release.

Installation known issuesThis section describes the known issues during installation and upgrade.

Verification of Oracle binaries incorrectly reports as failed during OracleGrid Infrastructure installation

The verification of Oracle binaries may incorrectly report as failed during theOracle Grid Infrastructure installation using the SF Oracle RAC installer. Themessage is erroneously reported due to a break in passwordless SSHcommunication. The SSH communication fails because execution of the root.shscript changes the owner of the operating system root directory to the grid userdirectory.


Stopping the installer during an upgrade and then resuming the upgrademight freeze the service groups [2574731]

The service groups freeze due to upgrading using the product installer if youstopped the installer after the installer already stopped some of the processesand then resumed the upgrade.

Workaround: You must unfreeze the service groups manually after the upgradecompletes.

To unfreeze the service groups manually

1 List all the frozen service groups

# hagrp -list Frozen=1

2 Unfreeze all the frozen service groups:

# haconf -makerw

# hagrp -unfreeze service_group -persistent

# haconf -dump -makero

After Live Upgrade to Solaris 10 Update 10, boot from alternate bootenvironment may fail (2370250)

If your setup involves volumes in a shared disk group that are mounted as CFS ina cluster, then during Live Upgrade using the vxlustart command from anysupported Solaris version to Solaris 10 Update 10, boot from an alternate bootenvironment may fail.

Workaround: Run the vxlufinish command. Before rebooting the system,manually delete the entries of all the volumes of shared disks that are mountedas CFS in the /altroot.5.10/etc/vfstab directory.

On Sparc, Live Upgrade from Solaris 9 to Solaris 10 Update 10 may fail(2424410)

On Sparc, Live Upgrade from Solaris 9 to Solaris 10 Update 10 may fail with thefollowing error:

Generating file list.

Copying data from PBE to ABE .

99% of filenames transferredERROR: Data duplication process terminated

unexpectedly.

ERROR: The output is .

29794 Killed

Fixing zonepaths in ABE.


30

Unmounting ABE .

100% of filenames transferredReverting state of zones in PBE

.

ERROR: Unable to copy file systems from boot environment

to BE .

ERROR: Unable to populate file systems on boot environment .

Removing incomplete BE .

ERROR: Cannot make file systems for boot environment .

This is a known issue with the Solaris lucreate command.

Workaround: Install Oracle patch 113280-10,121430-72 or higher before runningvxlustart.

OnSolaris 10, a flash archive installed through JumpStartmay cause a newsystem to go into maintenance mode on reboot (2379123)

If a Flash archive is created on a golden host with encapsulated root disks, whenthis Flash archive is installed onto another host through JumpStart, the newsystem may go to maintenance mode when you initially reboot it.

This problem is caused by the predefined root disk mirror in the Flash archive.When the archive is applied to a clone system, which may have different harddrives, thenewly cloned systemmayget stuck at root diskmirroringduring reboot.

Workaround: Create the Flash archive on a golden host with no encapsulatedroot disks. Run vxunroot to clean up the mirrored root disks before you createthe Flash archive.

Flash Archive installation not supported if the target system's root disk isencapsulated

Symantec does not support SF Oracle RAC installation using Flash Archive if thetarget system's root disk is encapsulated.

Make sure that the target system's root disk is unencapsulated before startingthe installation.

Web installer does not ask for authentication after the first session if thebrowser is still open (2509330)

If you install or configure SF Oracle RAC and then close the Web installer, if youhave other browser windows open, the Web installer does not ask forauthentication in the subsequent sessions. Since there is no option to log out ofthe Web installer, the session remains open as long as the browser is open on thesystem.

Workaround: Make sure that all browser windows are closed to end the browsersession and subsequently log in again.


After performing a manual rolling upgrade, make sure the CVM is onlineon all nodes without errors (2595441)

Make sure that the CVM is online on all nodes without errors after you performthe first phase of a manual rolling upgrade. The CVM protocol version will notupgrade successfully on the nodes where CVM is offline or has errors.

If the CVM protocol version does note upgrade successfully, upgrade the CVMprotocol on the CVM master node.

To upgrade the CVM protocol on the CVM master node

1 Find out which node is the CVM master:

# vxdctl -c mode

2 On the CVM master node, upgrade the CVM protocol:

# vxdctl upgrade

sfmh discovery issue when you upgrade your Veritas product to 6.0.1(2622987)

If a host is not reporting to anymanagement server but sfmhdiscovery is runningbefore you upgrade to 6.0.1, sfmh-discovery may fail to start after the upgrade.

Workaround: If the host is not reporting to VOM, stop sfmh-discovery manuallybefore upgrading to 6.0.1 by executing the following command on the managedhost:

/opt/VRTSsfmh/adm/vxvmdiscovery-ctrl.sh stop

Stopping the Web installer causes Device Busy error messages (2633924)

If you start theWeb installer, and thenperformanoperation (such as prechecking,configuring, or uninstalling), you may get an error message saying the device isbusy.

Workaround: Do one of the following:

■ Kill the start.pl process.

■ Start the webinstaller again. On the first Web page you see that the session isstill active. Either take over this session and finish it or terminate it directly.

After finishing a kernel upgrade on amaster node the cvm group on a slavenode does not come online (2439439)

After successfully finishing a kernel upgrade on one node, the cvm group doesnot come online on the second node.


32

Workaround: Check that your cluster is not in a jeopardy state before you performa rolling upgrade.

Erroneous resstatechange trigger warning

You may encounter the following warning when you restart resources:

CPI WARNING V-9-40-4317 The installer has detected that resstatechange

trigger is configured by setting TriggerResStateChange attributes.

Workaround:

In future releases, the resstatechange trigger will not be invokedwhen a resourceis restarted. Instead, the resrestart trigger will be invoked if you set theTriggerResRestart attribute. The resrestart trigger is available in the currentrelease. Refer to the VCS documentation for details.

Veritas File System modules may fail to unload if SmartMove is enabledand a break-off snapshot volume has been reattached (2851403)

The Veritas File System modules, vxportal and vxfs, may fail to unload ifSmartMove is enabled and a break-off snapshot volume is reattached. Reattachingthe snapshot causes an extra reference count to the vxportalmodule,which causesthe module unload operation to fail.

Workaround:

Manually unload the Veritas Volume Manager modules (vxspec, vxio, vxdmp)before unloading the vxportal module. This decrements the reference count ofthe vxportal module.

LLT known issuesThis section covers the known issues related to LLT in this release.

Cannot configure LLT if full device path is not used in the llttab file(2858159)

(Oracle Solaris 11) On virtual machines ensure that you use the full path of thedevices corresponding to the links in llttab. For example, use /dev/net/net1 insteadof /dev/net/net:1 in the llttab file, otherwise you cannot configure LLT.

Cannot use CPI response files to add nodes to a cluster that is using LLTover UDP (2869763)

When you run the addnode -responsefile command, if the cluster is using LLTover UDP, then the /etc/llttab file generated on new nodes is not correct. So,the procedure fails and you cannot addnodes to a cluster using CPI response files.


Workaround: None

GAB known issuesThis section covers the known issues related to GAB in this release.

Cluster panics during reconfiguration (2590413)

While a cluster is reconfiguring, GAB broadcast protocol encounters a racecondition in the sequence request path. This condition occurs in an extremelynarrow window which eventually causes the GAB master to panic.

Workaround: There is no workaround for this issue.

GAB may fail to stop during a phased upgrade on Oracle Solaris 11(2858157)

While performing a phased upgrade on Oracle Solaris 11 systems, GAB may failto stop. However, CPI gives a warning and continues with stopping the stack.

Workaround: Reboot the node after the installer completes the upgrade.

Cannot run pfiles or truss files on gablogd (2292294)

When pfiles or truss is run on gablogd, a signal is issued to gablogd. gablogd isblocked since it has called an gab ioctl and is waiting for events. As a result, thepfiles command hangs.

Workaround: None.

(Oracle Solaris 11) On virtual machines, sometimes the common productinstaller (CPI) may report that GAB failed to start and may exit (2879262)

GAB startup script may take longer than expected to start up. The delay in startup can cause the CPI to report that GAB failed and exits.

Workaround: Manually start GAB and all dependent services.

I/O fencing known issuesThis section covers the known issues related to I/O fencing in this release.

Installer is unable to split a cluster that is registered with one or more CPservers (2110148)

Splitting a cluster that uses server-based fencing is currently not supported.

You can split a cluster into two and reconfigure SFOracle RAC on the two clustersusing the installer. For example, you can split a cluster clus1 into clus1A andclus1B.


34

However, if you use the installer to reconfigure the SF Oracle RAC, the installerretains the same cluster UUID of clus1 in both clus1A and clus1B. If both clus1Aand clus1B use the same CP servers for I/O fencing, then the CP server allowsregistration only from the cluster that attempts to register first. It rejects theregistration from the cluster that attempts next. Thus, the installer reports failureduring the reconfiguration of the cluster that uses server-based fencing.


Fencing does not come up on one of the nodes after a reboot (2573599)

If VxFEN unconfiguration has not finished its processing in the kernel and in themeantime if you attempt to start VxFEN, you may see the following error in the/var/VRTSvcs/log/vxfen/vxfen.log file:

VXFEN vxfenconfig ERROR V-11-2-1007 Vxfen already configured

However, the output of the gabconfig -a command does not list port b. Thevxfenadm -d command displays the following error:

VXFEN vxfenadm ERROR V-11-2-1115 Local node is not a member of cluster!

Workaround: Start VxFEN again after some time.

The vxfenswap utility does not detect failure of coordination pointsvalidation due to an RSH limitation (2531561)

The vxfenswap utility runs the vxfenconfig -o modify command over RSH orSSH on each cluster node for validation of coordination points. If you run thevxfenswap command using RSH (with the -n option), then RSH does not detectthe failure of validation of coordination points on a node. From this point,vxfenswap proceeds as if the validation was successful on all the nodes. But, itfails at a later stage when it tries to commit the new coordination points to theVxFENdriver. After the failure, it rolls back the entire operation, and exits cleanlywith anon-zero error code. If you runvxfenswapusingSSH (without the-noption),then SSH detects the failure of validation of coordination of points correctly androlls back the entire operation immediately.

Workaround: Use the vxfenswap utility with SSH (without the -n option).

In absence of cluster details in CP server, VxFEN fails with pre-existingsplit-brain message (2433060)

When you start server-based I/O fencing, the node may not join the cluster andprints error messages in logs similar to the following:

In the /var/VRTSvcs/log/vxfen/vxfen.log file:


VXFEN vxfenconfig ERROR V-11-2-1043

Detected a preexisting split brain. Unable to join cluster.

In the /var/VRTSvcs/log/vxfen/vxfen.log file:

operation failed.

CPS ERROR V-97-1400-446 Un-authorized user cpsclient@sys1,

domaintype vx; not allowing action

The vxfend daemon on the application cluster queries the coordination pointserver (CP server) to check if the clustermembers as seen in theGABmembershipare registered with the CP server. If the application cluster fails to contact the CPserver due to some reason, then fencing cannot determine the registrations onthe CP server and conservatively assumes a pre-existing split-brain.

Workaround:Before youattempt to startVxFENon the application cluster, ensurethat the cluster details such as cluster name, UUID, nodes, and privileges areadded to the CP server.

When I/O fencing is not up, the svcs command shows VxFEN as online(2492874)

Solaris 10 SMFmarks the service status based on the exit code of the startmethodfor that service. The VxFEN startmethod executes the vxfen-startup script in thebackground and exitswith code 0.Hence, if the vxfen-startup script subsequentlyexitswith failure then this change is not propagated to SMF. This behavior causesthe svcs command to show incorrect status for VxFEN.

Workaround: Use the vxfenadm command to verify that I/O fencing is running.

Fencing port b is visible for few seconds even if cluster nodes have notregistered with CP server (2415619)

Even if the cluster nodes have no registration on the CP server and if you providecoordination point server (CP server) information in the vxfenmode file of thecluster nodes, and then start fencing, the fencing port b is visible for a few secondsand then disappears.

Workaround: Manually add the cluster information to the CP server to resolvethis issue. Alternatively, you can use installer as the installer adds clusterinformation to the CP server during configuration.

CP server repetitively logs unavailable IP addresses (2530864)

If coordination point server (CP server) fails to listen on any of the IP addressesthat are mentioned in the vxcps.conf file or that are dynamically added usingthe command line, then CP server logs an error at regular intervals to indicatethe failure. The logging continues until the IP address is bound to successfully.


36

CPS ERROR V-97-51-103 Could not create socket for host

10.209.79.60 on port 14250

CPS ERROR V-97-1400-791 Coordination point server could not

open listening port = [10.209.79.60]:14250

Check if port is already in use.

Workaround: Remove the offending IP address from the listening IP addresseslist using the rm_port action of the cpsadm command.

See theVeritas StorageFoundation forOracleRACAdministrator'sGuide formoredetails.

Server-based fencing comes up incorrectly if default port is notmentioned(2403453)

When you configure fencing in customized mode and do no provide default port,fencing comes up. However, the vxfenconfig -l command output does not listthe port numbers.

Workaround:Retain the "port=" setting in the /etc/vxfenmode file,whenusing customized fencingwith at least oneCP server. The default port valueis 14250.

Secure CP server does not connect from localhost using 127.0.0.1 as theIP address (2554981)

The cpsadm command does not connect to the secure CP server on the localhostusing 127.0.0.1 as the IP address

Workaround: Connect the secure CP server using any of the virtual IPs that isconfigured with the CP server and is plumbed on the local node.

Unable to customize the 30-second duration (2551621)

When the vxcpserv process is not able to bind to an IP address during startup, itattempts to bind to that IP address at an interval of 30 seconds. This interval isnot configurable.


NIC resource gets created with incorrect name while configuring CPSSGwith the configure_cps.pl script (2585229)

The name of the NIC resource created by the configure_cps.pl script does notcome out correct when, for example, mth VIP is mapped to nth NIC and every m isnot equal to n. In this case, although CPSSG continues to function without anyproblem, when you unconfigure CPSSG using configure_cps.pl, it fails.


Workaround:To unconfigure CPSSG, youmust remove the CPSSG configurationfrom the VCS configuration.

CP server configuration fails while setting up secure credentials for CPserver hosted on an SFHA cluster (2621029)

When you configure CP server using the configure_cps.pl utility, theconfiguration fails while setting up secure credentials for CP server that is hostedon an SFHA cluster. You may see the following error:

Creating softlink to credential directory /etc/VRTScps/db/CPSERVER

on node nodename.

Unable to connect to node nodename using /usr/bin/ssh.

Please configure ssh communication and retry. Exiting.

Workaround: You can use any of the following options:

■ Before running the configure_cps.plutility, change the default shell for rootuser to either KSH or bash.

■ Perform the following steps after running the configure_cps.pl utility oneach node of the cluster:

■ Manually remove the old credential directory or softlink. For example:

# rm -rf /var/VRTSvcs/vcsauth/data/CPSERVER

■ Create a new soft-link to the shared location of the credential directory:

# ln -s path_of_CP_server_credential_directory \

/var/VRTSvcs/vcsauth/data/CPSERVER

■ Start the CPSSG service group:

# hagrp -online CPSSG -any

The cpsadm command fails after upgrading CP server to 6.0 or above insecure mode (2846727)

The cpsadm command may fail after you upgrade coordination point server (CPserver) to 6.0 in secure mode. If the old VRTSat package is not removed from thesystem, thecpsadm command loads theold security libraries present on the system.As the installer runs the cpsadm command on the CP server to add or upgrade theSF Oracle RAC cluster (application cluster), the installer also fails.

Workaround:Perform the followingprocedure on all of thenodes of theCP server.


38

To resolve this issue

1 Rename cpsadm to cpsadmbin:

# mv /opt/VRTScps/bin/cpsadm /opt/VRTScps/bin/cpsadmbin

2 Create a file /opt/VRTScps/bin/cpsadm with the following content:

#!/bin/sh

EAT_USE_LIBPATH="/opt/VRTScps/lib"

export EAT_USE_LIBPATH

/opt/VRTScps/bin/cpsadmbin "$@"

3 Change the permissions of the new file to 775:

# chmod 755 /opt/VRTScps/bin/cpsadm

Server-based fencingmay fail to start after reinstalling the stack (2802682)

Server-based fencing may fail to start if you use the existing configuration filesafter reinstalling the stack.

Workaround:

After reinstalling the stack, add the client cluster information on the coordinationpoint server because the client cluster information is removed when the stack isuninstalled. Formoredetails, see theSettingup server-based I/OFencingmanuallysection in the Veritas Storage Foundation for Oracle RAC Installation Guide.Alternatively, you can manually modify the /etc/vxfenmode file and the main.cffile to start fencing in disable mode and then configure fencing.

Common product installer cannot setup trust between a client system onrelease version5.1SP1anda server on release version6.0 or later (2824472)

The issue exists because the 5.1SP1 release version does not support separatedirectories for truststores. But, release version 6.0 and later support separatedirectories for truststores. So, because of thismismatch in support for truststores,you cannot set up trust between client systems and servers.

Workaround: Set up trust manually between the coordination point server andclient systems using the cpsat or vcsat command. Now, the servers and clientsystems can communicate in secure mode.

Hostname and username are case sensitive in CP server (2846392)

The hostname and username on the CP server are case sensitive. The hostnameand username used by fencing to communicate with CP server must be in samecase as present in CP server database, else fencing fails to start.


Workaround:Make sure that the same case is used in thehostnameandusernameon the CP server.

Fencingmay show theRFSMstate as replaying for somenodes in the cluster(2555191)

Fencing based on coordination point clients in Campus cluster environment mayshow the RFSM state as replaying for some nodes in the cluster.

Workaround:

Restart fencing on the node that shows RFSM state as replaying.

Veritas Cluster Server may not come up after rebooting the first node inphased upgrade on Oracle Solaris 11 (2852863)

If any of the kernel level services that depend upon Veritas Cluster Server (VCS)do not come up, then VCS fails to come up. The LLT, GAB, and Vxfen modulesmay also fail to come up because the add_drv command failed to add its driver tothe system. On Solaris 11, add_drvmay fail if there is another add_drv commandthat is being run on the system at the same time.

Workaround:

Check the status of LLT,GABandVxfenmodules. Ensure that all the three servicesare online in SMF. Then, retry starting VCS.

Cannot run the vxfentsthdw utility directly from the install media ifVRTSvxfen package is not installed on the system (2858190)

If VRTSvxfen package is not installed on the system, then certain script files thatare needed for the vxfentsthdw utility to function are not available. So, withoutthe VRTSvxfen package installed on the system you cannot run the utility fromthe install media.

Workaround: Install VRTSvxfen package, then run the utility from either theinstall media or from the /opt/VRTSvcs/vxfen/bin/ location.

Usernames should be case-insensitive on the CP server (2846392)

The hostname and username on the CP server should be case-insensitive. But atpresent the CP server does not support case insensitivity in both username andhostname. The CP client fails to communicate with the CP server if a differentcase is used in hostname and username and hence fencing fails to start.

Coordination point server-based fencing may fail if it is configured on5.1SP1RP1 using 6.0.1 coordination point servers (2824472)

The 5.1SP1 installer (CPI) cannot set up trust between a 5.1SP1 client and a 6.0or later server, because there are no separate directories for truststores in the


40

5.1SP1.When trust cannot be setup, the 5.1SP1 installer cannot configure 5.1SP1clients to work with 6.0 or later CPS in secure mode.

Workaround:

Set up trust manually between the CPS and clients using the cpsat or the vcsatcommand. After that, CPS and client will be able to communicate properly in thesecure mode.

The upper bound value of FaultTolerance attribute of CoordPoint agentshould be less than the majority of the coordination points. (2846389)

The upper bound value of FaultTolerance attribute of CoordPoint agent shouldbe less than the majority of the coordination points. Currently this value is lessthan the number of coordination points.

vxfentsthdwutility fails to launch before you install the VRTSvxfen package(2858190)

Before you install the VRTSvxfen package, the file of/etc/vxfen.d/script/vxfen_scriptlib.shwhere stores the vxfentsthdwutility doesn’texist. In this case, the utility bails out.

Workaround:

Besides installing the VRTSvxfen package, run the vxfentsthdw utility directlyfrom the installation DVD.

PrivNIC and MultiPrivNIC agents not supported with OracleRAC 11.2.0.2 and later versionsThe PrivNIC andMultiPrivNIC agents are not supportedwithOracle RAC 11.2.0.2and later versions.

For more information, see the following Technote:


File system check daemon fails to restart after abnormaltermination (2689195)The file system check daemon (vxfsckd) fails to update the vxfsckd-pid file withthe new process ID (pid) of the vxfsckd process after abnormal termination. As aresult, the CFSfsckd agent fails to detect the status of the vxfsckd daemon.

Workaround: Perform the following steps to resolve the issue on the node wherethe vxfsckd resource faults:

1. Log into the node as the root user.



2. Kill all vxfsckd processes:

# kill -9 `ps -ef|grep vxfsckd|awk '{print $2}'`

3. Remove the vxfsckd-pid file:

# rm /var/adm/cfs/vxfsckd-pid

4. Bring the vxfsckd resource online:

# hares -online vxfsckd_resname -sys node_name

Issue with format of the last 8-bit number in private IPaddresses (1164506)ThePrivNIC/MultiPrivNIC resources fault if theprivate IP addresseshave a leading0 in any of the octets that comprise the IP address, for example X.X.X.01 orX.X.0X.1. or X.0X.X.1 or 0X.X.X.1, where X is an octet of the IP address.

When you configure private IP addresses for Oracle Clusterware, ensure that theIP addresses have a format as displayed in the following two-node example:

■ On galaxy: 192.168.12.1

■ On nebula: 192.168.12.2

Confirm the correct format by viewing the PrivNIC or MultiPrivNIC resource inthe /etc/VRTSvcs/conf/config/main.cf file.

CVMVolDg agent may fail to deport CVM disk groupThe CVM disk group is deported based on the order in which the CVMVolDgresources are taken offline. If the CVMVolDg resources in the disk group containa mixed setting of 1 and 0 for the CVMDeportOnOffline attribute, the disk groupis deported only if the attribute value is 1 for the last CVMVolDg resource takenoffline. If the attribute value is 0 for the last CVMVolDg resource taken offline,the disk group is not deported.

Workaround: If multiple CVMVolDg resources are configured for a shared diskgroup, set the value of the CVMDeportOnOffline attribute to 1 for all of theresources.


42

Warning message displayed on taking cssd resource offline ifLANG attribute is set to "eucJP" (2123122)Whenyou take the cssd resource offlineusing thehares -offline cssd commandand the LANG attribute is set to “eucJP”, the following message may be observedin the hamsg engine_A command output:

VCS INFO V-16-2-13716 Could not find message V-16-2-13716

You may ignore the message.

PrivNIC resource faults in IPMP environments on Solaris 11systems (2838745)The PrivNIC resource faults on Solaris 11 systems when private interfaces usedby IPMP are configured under PrivNIC resource.

Workaround:Avoid usingPrivNIC orMultiPrivNIC agents in IPMPenvironments.

Error displayed on removal of VRTSjadba language package(2569224)Removal of the VRTSjadba language package displays the following error on thescreen:

Executing postremove script.

Generating BMC map file...

bmcmap ERROR V-33-1000-10001 Unable to create BMC map

You may ignore the error.

"Configurationmust beReadWrite : Use haconf -makerw" errormessage appears in VCS engine log when hastop -local isinvoked (2609137)A message similar to the following example appears in the/var/VRTSvcs/log/engine_A.log log file when you run the hastop -localcommand on any system in a SF Oracle RAC cluster that has CFSMountresources:

2011/11/15 19:09:57 VCS ERROR V-16-1-11335 Configuration must be

ReadWrite : Use haconf -makerw

The hastop -local command successfully runs and you can ignore the errormessage.



Multiple system panics upon unmounting a CFS file system(2107152)There is a systempanicwhenyouunmount amntlock-protectedVxFS file system,if that device is duplicate mounted on different directories.


Application group attempts to come online on primary sitebefore fire drill service group goes offline on the secondarysite (2107386)The application service group comes online on the primary sitewhile the fire drillservice group attempts to go offline at the same time, causing the applicationgroup to fault.

Workaround: Ensure that the fire drill service group is completely offline on thesecondary site before the application service group comes online on the primarysite.

Oracle group fails to come online if Fire Drill group is onlineon secondary cluster [2653695]If a parallel global service group faults on the local cluster and does not find afailover target in the local cluster, it tries to failover the service group to theremote cluster. However, if the firedrill for the service group is online on a remotecluster, offline local dependency is violated and the global service group is notable to failover to the remote cluster.

Workaround: Offline the Firedrill service group and online the service group ona remote cluster.

Oracle service group faults on secondary site during failoverin a disaster recovery scenario [2653704]Oracle service group fails to go online in the DR site when disaster strikes theprimary site. This happens if the AutoFailover attribute on the Service Group isset to 1 and when the corresponding service group's FireDrill is online in the DRsite. Firedrill Service group may remain ONLINE on the DR site.

Workaround: If the service group containing theOracle (or any database) resourcefaults after attempting automatic DR failover while FireDrill is online in the DRsite, manually offline the FireDrill Service Group. Subsequently, attempt theonline of the Oracle Service Group in the DR site.


44

Veritas Volume Manager can not identify Oracle AutomaticStorage Management (ASM) disks (2771637)Veritas Volume Manager (VxVM) commands can not identify disks that areinitialized by ASM. Administrators must use caution when using the VxVMcommands to avoid accidental overwriting of the ASM disk data.

Oracle Universal Installer fails to start on Solaris 11 systems(2784560)TheOracleUniversal Installer (OUI) fails to startwhen the SFOracle RAC installerinvokes the OUI for the installation of Oracle Clusterware/Grid Infrastructuresoftware.

Workaround: Install the following packages before installing OracleClusterware/Grid Infrastructure.

SUNWxwplt

SUNWmfrun

For instructions, see the Oracle documentation.

Veritas Storage Foundation for Databases (SFDB) tools known issuesThe following are known issues in this release of Veritas Storage Foundation forDatabases (SFDB) tools.

SFDB commands do not work in IPV6 environment (2619958)In IPV6 environment, SFDB commands do not work for SF Oracle RAC. There isno workaround at this point of time.

Database Storage Checkpoint unmount may fail with devicebusy (2591463)In some cases,when adatabase that is clonedusing aDatabase StorageCheckpointis shut down, an error similar to the following may occur:

SFAE Error:0457: Failed to unmount device

/dev/vx/dsk/datadg/datavol:Ckpt_1317707593_rw_1317708154.

Reason: VxFS returned error : umount: /tmp/clonedb/data: device is

busy


Workaround

As an Oracle user, force shut down the clone database if it is up and then retrythe unmount operation.

Attempt to use SmartTier commands fails (2332973)The attempts to run SmartTier commands such as dbdst_preset_policyordbdst_file_move fail with the following error:

fsppadm: ERROR: V-3-26551: VxFS failure on low level mechanism

with message - Device or resource busy

This error occurs if a sub-file SmartTier command such as dbdst_obj_move hasbeen previously run on the file system.

There is no workaround for this issue. You cannot use file-based SmartTier andsub-file SmartTier simultaneously.

Attempt to use certain names for tiers results in error(2581390)If you attempt to use certain names for tiers, the following error message isdisplayed:

SFORA dbdst_classify ERROR V-81-6107 Invalid Classname BALANCE

This error occurs because the followingnames are reserved and are not permittedas tier names for SmartTier:

■ BALANCE

■ CHECKPOINT

■ METADATA

Workaround

Use a name for SmartTier classes that is not a reserved name.

Clone operation failure might leave clone database inunexpected state (2512664)If the clone operation fails, itmay leave the clone database in an unexpected state.Retrying the clone operation might not work.


46

Workaround

If retrying does not work, perform one the following actions depending on thepoint-in-time copy method you are using:

■ For FlashSnap, resync the snapshot and try the clone operation again.

■ For FileSnap and Database Storage Checkpoints, destroy the clone and createthe clone again.

■ For space-optimized snapshots, destroy the snapshot and create a newsnapshot.

Contact Symantec support if retrying using the workaround does not succeed.

FlashSnap resync fails if there is an existing space-optimizedsnapshot (2479901)If you try a FlashSnap resync operationwhen there is an existing space-optimizedsnapshot, the resync operation fails with the following error:

Error: VxVM vxdg ERROR V-5-1-4597 vxdg join FS_oradg oradg failed

datavol_snp : Record already exists in disk group

archvol_snp : Record already exists in disk group

Workaround

Destroy the space-optimized snapshot first and then perform the FlashSnapresync operation.

Clone command fails if PFILE entries have their values spreadacross multiple lines (1922384)If you have a log_archive_dest_1 in single line in the init.ora file, thendbed_vmclonedbwill work but dbed_vmclonebwill fail if you put inmultiple linesfor log_archive_dest_1.

Workaround

There is no workaround for this issue.

SFDB commands do not work with the ZHS16GBK characterset (2715323)SFDB commands do not work if the character set of the Oracle database is set toZHS16GBK. This occurs because SFDB commands are not supported withmulti-byte character sets except AL32UTF8 and ZHS16GBK is a multi-bytecharacter set.


There is no workaround for this issue.

Data population fails after datafile corruption, rollback, andrestore of offline checkpoint (2869259)Sometimes when a datafile gets corrupted below its reservation size, the rollbackmay not pass and the file may not be rolled back correctly.

There is no workround at this point of time.

Checkpoint clone fails if the archive log destination is sameas the datafiles destination (2869266)Checkpoint cloning fails if the archive logdestination is the sameas the datafilesdestination. The error is similar to:

Use of uninitialized value $path in hash element

at /opt/VRTSdbed/lib/perl/DBED/CkptOracle.pm line 121.

Use of uninitialized value $path in concatenation (.) or string


Use of uninitialized value $path in pattern match (m//)


SFDB vxsfadm ERROR V-81-0564 Oracle returned error.

Reason: ORA-02236: invalid file name (DBD ERROR: error possibly near

indicator at char 172 in 'CREATE CONTROLFILE REUSE SET DATABASE

'TClone03' RESETLOGS NOARCHIVELOG

Workaround: For the 6.0.1 release, create distinct archive and datafile mountsfor the checkpoint service.

FileSnap detail listing does not display the details of aparticular snap (2846382)FileSnap does not support displaying a detailed listing of a snapshot or clone.FileSnap only supports displaying a summary of all the snapshots or clones. Forexample, for the CLI vxsfadm -s filesnap -a oracle --name=snap1 -o list,a summary listing all the snapshots is displayed, instead of a detailed listing of aparticular snapshot.



48

Flashsnap clone fails under some unusual archivelogconfiguration on RAC (2846399)In a RAC environment, when using FlashSnap, the archive log destination tosnapshot must be a shared path, and must be the same across all the nodes.Additionally, all nodesmust use the same archive log configuration parameter tospecify the archive log destination. Configurations similar to the following arenot supported:

tpcc1.log_archive_dest_1='location=/tpcc_arch'



Where tpcc1, tpcc2, and tpcc3 are the names of the RAC instances and /tpcc_archis the shared archive log destination.

Workaround: To use FlashSnap, modify the above configuration to*.log_archive_dest_1='location=/tpcc_arch'. For example,




'vxdbd' process is online after Flash archive installation(2869269)After a Flash archive installation of the SF stack, the vxdbd process is up, even ifthe stack is not configured.

Workaround: You can ignore, or stop the vxdbd process using the/opt/VRTSdbed/common/bin/vxdbdctrl stopcommand.

Checkpoint clone fails in CFS environment if clonedusing samecheckpoint and same clone name on both nodes (2869268)The Checkpoint clone of an oracle database fails in a CFS environment, if youcreate a clone with a clone nam

Date post:	10-Feb-2021
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

VeritasStorageFoundation for Oracle® RAC Release Notes...Technical Support Symantec Technical...

Documents