Symantec ApplicationHA 6.1 Agent for Oracle Configuration...

Symantec™ ApplicationHA6.1 Agent for OracleConfiguration Guide - Solarison Oracle VM Server forSPARC

January 2014

Symantec™ ApplicationHA 6.1 Agent for OracleConfiguration Guide

The software described in this book is furnished under a license agreement and may be usedonly in accordance with the terms of the agreement.

Agent version: 6.1.000.000

Document version: 6.1.000.000 Rev 2

Legal NoticeCopyright © 2014 Symantec Corporation. All rights reserved.

Symantec, the Symantec Logo, the Checkmark Logo, Veritas, Veritas Storage Foundation,CommandCentral, NetBackup, Enterprise Vault, and LiveUpdate are trademarks or registeredtrademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Othernames may be trademarks of their respective owners.

The product described in this document is distributed under licenses restricting its use, copying,distribution, and decompilation/reverse engineering. No part of this document may bereproduced in any form by any means without prior written authorization of SymantecCorporation and its licensors, if any.

THE DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIEDCONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIEDWARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE ORNON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCHDISCLAIMERSAREHELD TOBE LEGALLY INVALID. SYMANTECCORPORATIONSHALLNOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTIONWITH THE FURNISHING, PERFORMANCE, OR USE OF THIS DOCUMENTATION. THEINFORMATION CONTAINED IN THIS DOCUMENTATION IS SUBJECT TO CHANGEWITHOUT NOTICE.

The Licensed Software and Documentation are deemed to be commercial computer softwareas defined in FAR 12.212 and subject to restricted rights as defined in FAR Section 52.227-19"Commercial Computer Software - Restricted Rights" and DFARS 227.7202, "Rights inCommercial Computer Software or Commercial Computer Software Documentation", asapplicable, and any successor regulations, whether delivered by Symantec as on premisesor hosted services. Any use, modification, reproduction release, performance, display ordisclosure of the Licensed Software and Documentation by the U.S. Government shall besolely in accordance with the terms of this Agreement.

Symantec Corporation350 Ellis StreetMountain View, CA 94043

http://www.symantec.com

http://www.symantec.com

Technical SupportSymantec Technical Support maintains support centers globally. Technical Support’sprimary role is to respond to specific queries about product features and functionality.The Technical Support group also creates content for our online Knowledge Base.The Technical Support group works collaboratively with the other functional areaswithin Symantec to answer your questions in a timely fashion. For example, theTechnical Support group works with Product Engineering and Symantec SecurityResponse to provide alerting services and virus definition updates.

Symantec’s support offerings include the following:

■ A range of support options that give you the flexibility to select the right amountof service for any size organization

■ Telephone and/or Web-based support that provides rapid response andup-to-the-minute information

■ Upgrade assurance that delivers software upgrades

■ Global support purchased on a regional business hours or 24 hours a day, 7days a week basis

■ Premium service offerings that include Account Management Services

For information about Symantec’s support offerings, you can visit our website atthe following URL:

www.symantec.com/business/support/index.jsp

All support services will be delivered in accordance with your support agreementand the then-current enterprise technical support policy.

Contacting Technical SupportCustomers with a current support agreement may access Technical Supportinformation at the following URL:

www.symantec.com/business/support/contact_techsupp_static.jsp

Before contacting Technical Support, make sure you have satisfied the systemrequirements that are listed in your product documentation. Also, you should be atthe computer on which the problem occurred, in case it is necessary to replicatethe problem.

When you contact Technical Support, please have the following informationavailable:

■ Product release level

■ Hardware information

http://www.symantec.com/business/support/index.jsp

http://www.symantec.com/business/support/contact_techsupp_static.jsp

■ Available memory, disk space, and NIC information

■ Operating system

■ Version and patch level

■ Network topology

■ Router, gateway, and IP address information

■ Problem description:

■ Error messages and log files

■ Troubleshooting that was performed before contacting Symantec

■ Recent software configuration changes and network changes

Licensing and registrationIf your Symantec product requires registration or a license key, access our technicalsupport Web page at the following URL:

www.symantec.com/business/support/

Customer serviceCustomer service information is available at the following URL:

www.symantec.com/business/support/

Customer Service is available to assist with non-technical questions, such as thefollowing types of issues:

■ Questions regarding product licensing or serialization

■ Product registration updates, such as address or name changes

■ General product information (features, language availability, local dealers)

■ Latest information about product updates and upgrades

■ Information about upgrade assurance and support contracts

■ Information about the Symantec Buying Programs

■ Advice about Symantec's technical support options

■ Nontechnical presales questions

■ Issues that are related to CD-ROMs or manuals

http://www.symantec.com/business/support/

http://www.symantec.com/business/support/

Support agreement resourcesIf you want to contact Symantec regarding an existing support agreement, pleasecontact the support agreement administration team for your region as follows:

[email protected] and Japan

[email protected], Middle-East, and Africa

[email protected] America and Latin America

DocumentationProduct guides are available on the media in PDF format. Make sure that you areusing the current version of the documentation. The document version appears onpage 2 of each guide. The latest product documentation is available on the Symantecwebsite.

https://sort.symantec.com/documents

Your feedback on product documentation is important to us. Send suggestions forimprovements and reports on errors or omissions. Include the title and documentversion (located on the second page), and chapter and section titles of the text onwhich you are reporting. Send feedback to:

[email protected]

For information regarding the latest HOWTO articles, documentation updates, orto ask a question regarding product documentation, visit the Storage and ClusteringDocumentation forum on Symantec Connect.

https://www-secure.symantec.com/connect/storage-management/forums/storage-and-clustering-documentation

About Symantec ConnectSymantec Connect is the peer-to-peer technical community site for Symantec’senterprise customers. Participants can connect and share information with otherproduct users, including creating forum posts, articles, videos, downloads, blogsand suggesting ideas, as well as interact with Symantec product teams andTechnical Support. Content is rated by the community, and members receive rewardpoints for their contributions.

http://www.symantec.com/connect/storage-management

mailto:[email protected]



https://sort.symantec.com/documents




http://www.symantec.com/connect/storage-management

Technical Support ............................................................................................... 4

Chapter 1 Introducing the Symantec ApplicationHA agent forOracle ................................................................................ 9

About the Symantec ApplicationHA agent for Oracle ............................. 9Changes introduced in the 6.1 release .............................................. 10About installing and removing the ApplicationHA agent for Oracle .......... 10Supported software ....................................................................... 10

Supported application versions ................................................. 11Supported virtualization environments ........................................ 11Supported operating systems on guest domains ........................... 11

How the agent makes Oracle highly available ..................................... 12About Symantec ApplicationHA agent for Oracle functions .................... 13

Oracle agent functions ............................................................. 13How the Oracle agent supports health check monitoring ................. 18Netlsnr agent functions ............................................................ 20

Typical Oracle configuration in an ApplicationHA guest domain ............. 21About setting up Oracle in an ApplicationHA guest domain ................... 21

Chapter 2 Installing and configuring Oracle .................................... 23

About installing Oracle in an ApplicationHA environment ...................... 23Before you install Oracle in an ApplicationHA environment .................... 24About ApplicationHA requirements for installing Oracle ........................ 24

Chapter 3 Configuring application monitoring with SymantecApplicationHA ................................................................ 27

About configuring application monitoring with ApplicationHA ................. 27Before configuring application monitoring for Oracle ............................ 28Accessing the Symantec High Availability view .................................. 28Configuring application monitoring for Oracle ..................................... 29

Contents

Chapter 4 Troubleshooting Symantec ApplicationHA agentfor Oracle ........................................................................ 34

About troubleshooting ApplicationHA agent for Oracle ......................... 34Error messages common to the Oracle and Netlsnr agents ................... 35Error messages specific to the Oracle agent ...................................... 36Error messages specific to the Netlsnr agent ...................................... 40Verifying the Oracle health check binaries and intentional offline for

an instance of Oracle ............................................................. 41Verifying the intentional offline behavior of the VCS Oracle

agent ............................................................................. 43

Appendix A Resource type definitions .................................................. 45

About the resource type and attribute definitions ................................. 45Resource type definition for the Oracle agent ..................................... 45

Attribute definition for the Oracle agent ....................................... 46Resource type definition for the Netlsnr agent .................................... 52

Attribute definition for the Netlsnr agent ....................................... 52

Appendix B Detail monitoring ................................................................ 56

Setting the PATH variable .............................................................. 56Setting up detail monitoring for ApplicationHA agent for Oracle .............. 56

Setting up detail monitoring for Oracle ........................................ 57Setting up detail monitoring for ApplicationHA agent for Netlsnr ............. 60

Appendix C Using the SPFILE in an ApplicationHA system forOracle .............................................................................. 61

Using the SPFILE in an ApplicationHA guest domain ........................... 61

Appendix D Best practices ...................................................................... 63

Best practices for multiple Oracle instance configurations in aApplicationHA environment ...................................................... 63

8Contents

Introducing the SymantecApplicationHA agent forOracle

This chapter includes the following topics:

■ About the Symantec ApplicationHA agent for Oracle

■ Changes introduced in the 6.1 release

■ About installing and removing the ApplicationHA agent for Oracle

■ Supported software

■ How the agent makes Oracle highly available

■ About Symantec ApplicationHA agent for Oracle functions

■ Typical Oracle configuration in an ApplicationHA guest domain

■ About setting up Oracle in an ApplicationHA guest domain

About the Symantec ApplicationHA agent for OracleThe ApplicationHA agent for Oracle provides controlling and monitoring for Oracledatabase instances.

Symantec agents do the following:

■ Monitor specific resources within an enterprise application.

■ Determine the status of these resources.

■ Start or stop the resources according to external events.

1Chapter

The agents include resource type declarations and agent executables. The agentfor Oracle monitors the Oracle and listener processes, brings them online, andtakes them offline.

The ApplicationHA package contains the following agents that work together tomake Oracle highly available:

■ The Oracle agent monitors the Oracle database processes.

■ The Netlsnr agent monitors the listener process.

Changes introduced in the 6.1 releaseSymantec ships the build_oraapi.sh script along with the VCS agent for Oraclein the Oracle agent’s /opt/VRTSagents/ha/bin/Oracle directory. Thebuild_oraapi.sh script can be used to link the libraries that are required for Oraclehealth check monitoring.

Note: In this release, the health check monitoring option is not supported for theASMInst agent.

About installing and removing the ApplicationHAagent for Oracle

When you install or uninstall Symantec ApplicationHA, the ApplicationHA agent forOracle is automatically installed or removed. For more information, see theSymantecApplicationHA Installation and Upgrade Guide.

When you run the installer or uninstall program that accompanies the quarterlyagent pack release of high availability agents from Symantec, the latest version ofthe ApplicationHA agent for Oracle is automatically installed or removed. For moreinformation, see the Symantec ApplicationHA Agent Pack Installation Guide.

Supported softwareThe Symantec ApplicationHA agent for Oracle supports the following softwareversions:

■ Symantec ApplicationHA agent for Oracle can be installed and run inside guestdomains that have Symantec ApplicationHA 6.1 installed.

■ The following versions of the Veritas Operations Manager components aresupported:

10Introducing the Symantec ApplicationHA agent for OracleChanges introduced in the 6.1 release

■ Veritas Operations Manager Management Server 6.0 or later

■ Veritas Operations Manager managed host for Solaris: 6.0 or later

Supported application versionsTable 1-1 lists the Oracle versions that Symantec ApplicationHA 6.1 currentlysupports on guest domain.

Table 1-1 Supported application versions

VersionApplication

10gR2, 11gR1, 11gR2, and 12cOracle

Supported virtualization environmentsSymantec ApplicationHA can be installed and run inside guest domains in an OracleVM Server for SPARC virtualization environment. The following versions aresupported:

Table 1-2 Supported versions

Firmware versionServer version

Version 7.3.0 or laterOracle VM Server for SPARC 2.0

Version 7.4.0 or laterOracle VM Server for SPARC 2.1

Refer to Oracle VM Server for SPARC Release Notes, for more information on therequired firmware versions.

Supported operating systems on guest domainsThis section lists the supported operating systems for Symantec ApplicationHA 6.1.

Table 1-3 shows the supported operating systems for this release.

Table 1-3 Supported guest operating systems

ChipsetsLevelsOperating systems

SPARCUpdate 8, 9, 10, and 11Solaris 10

11Introducing the Symantec ApplicationHA agent for OracleSupported software

How the agent makes Oracle highly availableThe Symantec ApplicationHA agent for Oracle continuously monitors the Oracledatabase and listener processes to verify they function properly.

The agent provides the following levels of application monitoring:

■ Primary or Basic monitoringThis mode has Process check and Health check monitoring options. With thedefault Process check option, the agent verifies that the Oracle and listenerprocesses are present in the process table. Process check cannot detect whetherprocesses are in a hung state or a stopped state.The Oracle agent provides functionality to detect whether the Oracle resourcewas intentionally taken offline. The agent detects graceful shutdown for Oracle10g and later. When an administrator brings down Oracle gracefully, the agentdoes not trigger a resource fault even though Oracle is down. The value of thetype-level attribute IntentionalOffline and MonitorOption must be set to 1 toenable Oracle agent's intentional offline functionality.For example, with the intentional offline functionality, the agent faults the Oracleresource if there is an abnormal termination of the instance. The agent reportsthe Oracle resource as offline if you gracefully bring down Oracle usingcommands like shutdown, shutdown immediate, shutdown abort, or shutdowntransactional.

■ Secondary or Detail monitoringIn this mode, the agent runs a perl script that executes commands against thedatabase and listener to verify their status.

The agent detects application failure if the monitoring routine reports an improperfunction of the Oracle or listener processes. When this application failure occurs,the agents attempt to start the application services. After a configurable number ofattempts, if the application services do not start, the agents consider this as anapplication failure and report the status to VCS. Depending on the configuration,VCS can then restart the guest domain. After the machine restarts, the agents startsthe application services and brings the configured resources online. The agent thusensures high availability for the Oracle services and the database.

Note: You can use the ApplicationHA wizards to configure only primary or basicmonitoring. To configure secondary or detailed monitoring, use CLI/VeritasOperations Manager (VOM).

12Introducing the Symantec ApplicationHA agent for OracleHow the agent makes Oracle highly available

About Symantec ApplicationHA agent for Oraclefunctions

The functions an agent performs are called entry points. Review the functions forthe following agents that are part of the Symantec ApplicationHA agent suite forOracle:

■ Oracle agent functionsSee “Oracle agent functions” on page 13.

■ Netlsnr agent functionsSee “Netlsnr agent functions” on page 20.

Oracle agent functionsThe Oracle agent monitors the database processes.

Table 1-4 lists the Oracle agent functions.

Table 1-4 Oracle agent functions

DescriptionAgent operation

Starts the Oracle database by using the following sqlpluscommand:

startup force pfile=$PFile

The default Startup option is STARTUP_FORCE. You can alsoconfigure the agent to start the database using different Startupoptions for Oracle.

See “Startup and shutdown options for the Oracle agent”on page 14.

Online

Stops the Oracle database with the specified options by using thefollowing sqlplus command:

shutdown immediate

The default Shutdown option is IMMEDIATE. You can also configurethe agent to stop the database using different Shutdown optionsfor Oracle.

See “Startup and shutdown options for the Oracle agent”on page 14.

Offline

Verifies the status of the Oracle processes. The Oracle agentprovides two levels of monitoring: basic and detail.

See “Monitor options for the Oracle agent” on page 16.

Monitor

13Introducing the Symantec ApplicationHA agent for OracleAbout Symantec ApplicationHA agent for Oracle functions

Table 1-4 Oracle agent functions (continued)


Forcibly stops the Oracle database by using the following sqlpluscommand:

shutdown abort

If the process does not respond to the shutdown command, thenthe agent does the following:

■ Scans the process table for the processes that are associatedwith the configured instance

■ Kills the processes that are associated with the configuredinstance

Clean

Startup and shutdown options for the Oracle agentYou can specify Startup and Shutdown options for the Oracle instances that areconfigured.

Table 1-5 lists the startup options that the agent supports.

Table 1-5 Startup options

DescriptionOption

Runs the command startup forcepfile='location_of_pfile’ if the pfile is configured.

If the pfile is not configured, the agent runs startup force.It picks up the default parameter files from their defaultlocations.

STARTUP_FORCE

(Default)

Runs the command startuppfile='location_of_pfile’ if the pfile is configured.

If the pfile is not configured, the agent picks up the defaultparameter files from their default locations and runs startup.

STARTUP

Starts the database in the RESTRICTED mode.RESTRICTED

Performs a database recovery on instance startup.RECOVERDB


Table 1-5 Startup options (continued)

DescriptionOption

Uses a predefined SQL script (start_custom_$SID.sql)and runs custom startup options. The script must be in the/opt/VRTSagents/ha/bin/Oracle directory andmust have accessto the Oracle Owner OS user. If the file is not present, theagent logs an error message.

With a custom script, the agent takes the following action:

sqlplus /nolog <<!connect / as sysdba;@start_custom_$SID.sqlexit;!

CUSTOM

Uses the srvctl utility to start an instance of the database.

For RAC clusters, you must manually set the default startupoption as SRVCTLSTART.

SRVCTLSTART

Uses the srvctl utility to start an instance of the databasewith the -o 'read only' option.

SRVCTLSTART_RO

Table 1-6 lists the shutdown options that the agent supports.

Table 1-6 Shutdown options

DescriptionOption

Shuts down the Oracle instance by running shutdownimmediate.

IMMEDIATE

(Default)

Runs the shutdown transactional command.This option is valid only for the database versions thatsupport this option.

TRANSACTIONAL

Uses a predefined SQL script(shut_custom_$SID.sql) and runs customshutdown options. The script must be in the/opt/VRTSagents/ha/bin/Oracle directory and musthave access to the Oracle Owner OS user. If the fileis not present, the agent shuts the agent down withthe default option.

CUSTOM


Table 1-6 Shutdown options (continued)

DescriptionOption

Uses the srvctl utility to stop an instance of thedatabase.

For RAC clusters, you must manually set the defaultoption as SRVCTLSTOP.

SRVCTLSTOP

Uses the srvctl utility to stop an instance of thedatabase with the -o transactional option.

SRVCTLSTOP_TRANSACT

Uses the srvctl utility to stop an instance of thedatabase with the -o abort option.

SRVCTLSTOP_ABORT

Uses the srvctl utility to stop an instance of thedatabase with the -o immediate option.

SRVCTLSTOP_IMMEDIATE

Monitor options for the Oracle agentThe Oracle agent provides two levels of monitoring: basic and detail. By default,the agent does a basic monitoring.

The basic monitoring mode has the following options:

■ Process check

■ Health check

The MonitorOption attribute of the Oracle resource determines whether the agentmust perform basic monitoring in Process check or Health check mode.

Table 1-7 describes the basic monitoring options.

Table 1-7 Basic monitoring options

DescriptionOption

Process check

The agent scans the process table for the ora_dbw, ora_smon,ora_pmon, and ora_lgwr processes to verify that Oracle is running.

0

(Default)

Health check (supported on Oracle 10g and later)

The agent uses the Oracle health check APIs to monitor the SystemGlobal Area (SGA) and retrieve the information about the instance.

If you want to use the Oracle agent's intentional offline functionality,you must enable health check monitoring.

1


Review the following considerations if you want to configure basic monitoring:

■ When the administrator brings an Oracle database online on the ApplicationHAguest domain, an Oracle instance begins, but the database is not mounted. Insuch circumstances, this failure is detected only by the health check monitoringoption of basic monitoring or detail monitoring. Detail monitoring updates thedatabase table after detecting a failure whereas health check monitoring doesnot.If process check monitoring of basic monitoring is set, then such a conflict wouldgo undetected. To set health check monitoring option, use the command lineinterface (CLI) or VOM.

In the detail monitoring mode, the agent performs a transaction on a test table inthe database to ensure that Oracle functions properly. The agent uses this testtable for internal purposes. Symantec recommends that you do not perform anyother transaction on the test table. The DetailMonitor attribute of the Oracle resourcedetermines whether the agent must perform detail monitoring.

Note: Use the ApplicationHA wizards to configure basic monitoring for Oracle. Toenable detail monitoring, use CLI or VOM.

How the agent handles Oracle error codes during detailmonitoringThe Symantec ApplicationHA agent for Oracle handles Oracle errors during detailmonitoring. The agent classifies Oracle errors according to their severity andassociates predefined actions with each error code.

The agent includes a reference file oraerror.dat, which lists Oracle errors and theaction to be taken when the error is encountered.

The file stores information in the following format:

Oracle_error_string:action_to_be_taken

For example:

01035:WARN

01034:FAILOVER


Note:When the oraerror.dat file contains two or more errors, then the errors takean ascending precedence and the ApplicationHA agent for Oracle handles only thefirst error.

When the oraerror.dat file contains two or more errors, the order in which theApplicationHA agent for Oracle handles the errors is: FAILOVER/NOFAILOVER,UNKNOWN, and IGNORE/WARN.

Table 1-8 lists the predefined actions that the agent takes when an Oracle error isencountered.

Table 1-8 Predefined agent actions for Oracle errors

DescriptionAction

Ignores the error.

When the Symantec agent for Oracle encounters an error, the agentmatches the error code in the oraerror.dat file. If the error does not havea matching error code in the file, then the agent ignores the error.

IGNORE

Marks the resource state as UNKNOWN and sends a notification if theNotifier resource is configured.

This action is typically associated with configuration errors or programinterface errors.

UNKNOWN

Marks the resource state as ONLINE and sends a notification if theNotifier resource is configured.

This action is typically associated with errors due to exceeded quotalimits, session limits/restricted sessions so on.

WARN

How the Oracle agent supports health check monitoringThe Symantec Cluster Server agent for Oracle uses the Oracle health check APIsto determine whether the Oracle instance on a node was shut down gracefully orwas it aborted. When an oracle instance is shut down gracefully outside of VCScontrol the agent acknowledges the operation as intentional offline.

The Oracle health check APIs are supported on all UNIX platforms, Oracle Database10g Release and later.

You can use the build_oraapi.sh script to link the libraries that are required forOracle health check monitoring. Symantec ships the build_oraapi.sh script alongwith the VCS agent for Oracle in the Oracle agent’s/opt/VRTSagents/ha/bin/Oracle directory.


Generating the executable required for Oracle health checkmonitoringReview the considerations before using the build_oraapi.sh script.

Prerequisites to link Oracle libraries using thebuild_oraapi.sh scriptEnsure that the following prerequisites are met before building the Oracle healthcheck libraries:

■ Set the ORACLE_HOME environment variable.# export ORACLE_HOME=/u01/oraHome

where /u01/oraHome is the directory path where the ORACLE_HOMEenvironment variable is set.

■ If the ORACLE_HOME variable is not set, use the directory path to theORACLE_HOME variable as a command line option along with the Oracleversion when you run the script.# build_oraapi.sh -version <version> [-oracle_home

<path_to_oracle_home>]

Where version is the Oracle database version and path_to_oracle_home is thepath to access the ORACLE_HOME variable.

■ Ensure that the ORACLE_HOME variable is accessible before the executingthe build_oraapi.sh script.

Considerations to run the build_oraapi.sh script■ Run the build_oraapi.sh script on each node of the cluster where you want

to use Oracle health monitoring.

■ You need to run the build_oraapi.sh script to relink the Oracle librarieswhenever the Oracle database version is upgraded on a node.

■ The Oracle health check binaries generated in the Oracle agent directorycorrespond to the ORACLE_HOME directory and the Oracle database version thatare specified while executing the build_oraapi.sh script. ORACLE_HOMEcan be set as an environment variable or specified as a command line option.The Oracle database version can be specified as a command line option withthe build_oraapi.sh script.

■ Symantec ships the build_oraapi.sh script in VCS 6.1 release. Symantec willnot ship the pre-built Oracle health check binaries (oraapi_32, oraapi_3211g,oraapi_64, and oraapi_6411g).


■ To enable health check support in Oracle agent, you must run thebuild_oraapi.sh script to build the Oracle health check binaries based onOracle version.

Building theOracle agent executable using the build_oraapi.shscriptTo build the Oracle agent executable using the build_oraapi.sh script:

1 Log in as root user on the node.

2 Set the Oracle home directory path.

# export ORACLE_HOME=/u01/oraHome

where, /u01/oraHome is the Oracle home directory that stores files.

3 Access the Oracle agent directory.

# cd /opt/VRTSagents/ha/bin/Oracle

4 Run the following command from the Oracle agent directory.

# ./build_oraapi.sh -version version

where version is the Oracle database version on the node.

5 Alternatively, if you did not set the ORACLE_HOME variable, you can set theoracle_home parameter as a command line option.

#./build_oraapi.sh -version version -oracle_home /u01/oraHome

where version is the Oracle database version on the node and /u01/oraHomeis the Oracle home directory path, which is set as theORACLE_HOME variablevalue.

The script generates binaries under agent directory/opt/VRTSagents/ha/bin/Oracle. Oracle health check binaries generated on anode is based on the Oracle major version (10, 11, or 12) and the systemarchitecture (32 bit or 64 bit) available on that node.

For example, oraapi_6411g is a binary created with Oracle version 11 on a 64-bitsystem.

See “Verifying the Oracle health check binaries and intentional offline for an instanceof Oracle ” on page 41.

Netlsnr agent functionsThe listener is a server process that listens to incoming client connection requestsand manages traffic to the database. The Netlsnr agent brings the listener servicesonline, monitors their status, and takes them offline.


Table 1-9 lists the Netlsnr agent functions.

Table 1-9 Netlsnr agent functions


Starts the listener process by using the following command:

lsnrctl start $LISTENER

Online

Stops the listener process by using the following command:

lsnrctl stop $LISTENER

If the listener is configured with a password, the agent uses thepassword to stop the listener.

Offline

Verifies the status of the listener process.

The Netlsnr agent provides two levels of monitoring, basic anddetail:

■ In the basic monitoring mode, the agent scans the process tablefor the tnslsnr process to verify that the listener process isrunning.

■ In the detail monitoring mode, the agent uses the lsnrctlstatus $LISTENER command to verify the status of theListener process. (Default)

Monitor

Scans the process table for tnslsnr $Listener and kills it.Clean

TypicalOracle configuration in anApplicationHAguestdomain

A typical Oracle configuration in an ApplicationHA environment has the followingcharacteristics:

■ Oracle binaries can be installed locally or on shared disks. Data can reside ondata disks. Ensure that the data disks and oracle installation binary disks areautomatically mounted after the guest domain starts/reboots. To automaticallymount the required disks, add appropriate entries in /etc/fstab

About setting up Oracle in an ApplicationHA guestdomain

Tasks involved in setting up Oracle in an ApplicationHA environment include:

21Introducing the Symantec ApplicationHA agent for OracleTypical Oracle configuration in an ApplicationHA guest domain

■ Setting up an ApplicationHA guest domainRefer to Symantec ApplicationHA Installation Guide for more information oninstalling and configuring ApplicationHA.

■ Installing and configuring OracleSee “About installing Oracle in an ApplicationHA environment” on page 23.

22Introducing the Symantec ApplicationHA agent for OracleAbout setting up Oracle in an ApplicationHA guest domain

Installing and configuringOracle


■ About installing Oracle in an ApplicationHA environment

■ Before you install Oracle in an ApplicationHA environment

■ About ApplicationHA requirements for installing Oracle

About installing Oracle in an ApplicationHAenvironment

The strategy for installing Oracle into an ApplicationHA guest domain is aimed atensuring that installations on all guest domains are uniform.

See the Oracle documentation on Solaris.

You can install Oracle in the following ways in an ApplicationHA environment:

Oracle binaries and Oracle data are installed onshared disks.

$ORACLE_HOME on the shared disk

Oracle binaries are installed locally on each nodeand Oracle data is installed on shared disks.

$ORACLE_HOME on the local disk

Note that Oracle data includes the datafiles, control files, redo log files, and archivelog files.

The user oracle and the group dba must be local and not Network InformationService (NIS and NIS+) users.

2Chapter

Before you install Oracle in an ApplicationHAenvironment

Make sure you meet the following prerequisites:

■ Verify that all guest domains have adequate resources to run Oracle andApplicationHA.

■ Verify that the network supports the TCP/IP protocol.

■ Make sure that you meet the ApplicationHA requirements to install Oracle.See “About ApplicationHA requirements for installing Oracle” on page 24.

About ApplicationHA requirements for installingOracle

Make sure you meet the following requirements to install Oracle in an ApplicationHAguest domain.

Each node on which you want to install Oracle mustmeet the following Oracle configuration requirements:

■ Disk partitions■ Shared memory■ Swap size■ Semaphores■ File handles

See Oracle documentation for the correspondingoperating system for specific requirement details.

Kernel parameter configuration

Symantec recommends you to select the SoftwareDevelopment package option when you installSolaris. This option is required for relinking Oracleat install time and to take advantage of someAdvanced Server features.

Solaris package group

Make sure that Sun’s JDK 1.3.1_02 or Blackdown’sJDK 1.1.8 is installed on the system.

JDK requirements for Solaris

24Installing and configuring OracleBefore you install Oracle in an ApplicationHA environment

Depending on your environment, you can place theOracle home directory ($ORACLE_HOME) in one ofthe following ways:

■ Locally on each server’s disk■ On the shared storage.

Review the advantages of each approach to makea decision.

Location of the $ORACLE_HOME

You can have multiple Oracle instances on a guestdomain. In such cases, the parameter file for eachinstance must be accessible on all guest domains.

Note: If you installed multiple versions of Oracle onthe same system, make sure that the SIDs areunique.

Configurations with multiple Oracleinstances (SIDs)

If you plan to create the tablespaces using regular(EXT2/EXT3 or VxFS) files, the file systems thatcontain these files must be located on shared disks.Create the same file system mount points on eachnode.

If you use raw devices on shared disks for Oracletablespaces, you must meet the followingrequirements:

■ The ownership must be Oracle DBA user.■ The permissions or access mode must be 660

on the raw devices that store the Oracle data.

For example, if you use Veritas Volume Manager,type:

# vxedit -g diskgroup_name set group=dba \user=oracle mode=660 volume_name

Note: The user oracle and the group dbamust belocal and not Network Information Service (NIS andNIS+) users

Location of Oracle databasetablespaces

25Installing and configuring OracleAbout ApplicationHA requirements for installing Oracle

The Netlsnr agent supports OS authentication aswell as password authentication for the listenerprocess. If you use Oracle 10g or later, Symantecrecommends you to configure OS authentication. Ifyou want to configure a listener password, make surethat you configure the password correctly. Amisconfigured password can cause the listener tofault.

Refer to the Oracle documentation for details onconfiguring the listener authentication.

Listener authentication inApplicationHA environment

The hot backup of Oracle database is enabled bydefault in ApplicationHA environment.

A virtual machine can fail during a hot backup of anOracle database. ApplicationHA attempts to restartonly if the following requirements are met:

■ The AutoEndBkup attribute value must be set to1, which is the default.

■ The Startup option of Oracle agent must beSTARTUP, STARTUP_FORCE, or CUSTOM.

If you do not meet ApplicationHA requirements, youmust manually end the hot backup.

Note: If you set the AutoEndBkup attribute value to0, then to avoid unexpected ApplicationHA behavioryou must set the DetailMonitor attribute value to 1.

Hot backup of Oracle database inApplicationHA environment

Note: If you want to modify or tune a particular attribute, use CLI or VOM.

26Installing and configuring OracleAbout ApplicationHA requirements for installing Oracle

Configuring applicationmonitoring with SymantecApplicationHA


■ About configuring application monitoring with ApplicationHA

■ Before configuring application monitoring for Oracle

■ Accessing the Symantec High Availability view

■ Configuring application monitoring for Oracle

About configuring application monitoring withApplicationHA

This chapter describes the steps to configure application monitoring withApplicationHA in a virtualization environment.

Consider the following points before you proceed:

■ You configure an application for monitoring on a guest domain using theSymantec ApplicationHA Configuration Wizard.

■ The Symantec ApplicationHA Configuration Wizard is launched when you clickConfigure Application Monitoring in the Symantec High Availability view ofthe Veritas Operations Manager (VOM) Management Server console.

■ In this release, the wizard allows you to configure monitoring for only oneapplication per guest domain.

3Chapter

To configure another application using the wizard, you must first unconfigurethe existing application monitoring.

■ After you have configured monitoring for an application using the wizard, youcan configure monitoring for other applications residing in the same guestdomain, using Symantec Cluster Server (VCS) commands.For more information read the following technote:http://www.symantec.com/docs/TECH159846

■ After configuring Oracle for monitoring, if you create another Oracle instance,these new components are not monitored as part of the existing configuration.In such a case, you must first unconfigure the existing configuration and thenreconfigure the application using the wizard. You can then select all the instancesand databases for monitoring.

Before configuring applicationmonitoring for OracleEnsure that you complete the following tasks before configuring applicationmonitoring for Oracle on a guest domain:

■ Install Veritas Operations Manager (VOM) Management Server. For moreinformation on working with VOM, see the Symantec ApplicationHA User'sGuide. For information on accessing the Symantec High Availability view: See“Accessing the Symantec High Availability view ” on page 28.

■ Install ApplicationHA guest components on the guest domain that you need tomonitor.

■ Assign ApplicationHA - Configure Application Monitoring (Admin) privileges tothe logged-on user on the guest domain where you want to configure applicationmonitoring.

■ Install the application and the associated components that you wish to monitoron the guest domain.

■ If you have configured a firewall, ensure that your firewall settings allow accessto ports used by ApplicationHA installer, wizards, and services.Refer to the Symantec ApplicationHA Installation Guide for a list of ports andservices used.

Accessing the Symantec High Availability viewTo administer an application on a guest domain that is running in the Oracle VMServer for SPARC environment, you must access the Symantec High Availabilityview of the Veritas Operations Manager (VOM) Management Server console.

28Configuring application monitoring with Symantec ApplicationHABefore configuring application monitoring for Oracle

http://www.symantec.com/docs/TECH159846

From the Symantec High Availability view, you can perform administrative actionssuch as:

■ Start an application

■ Stop an application

■ Configure application monitoring

■ Unconfigure application monitoring

■ Enable application heartbeat

■ Disable application heartbeat

■ Enter maintenance mode

■ Exit maintenance mode

To access the Symantec High Availability view

1 Log on to the VOM Management Server console.

2 Select the Server perspective and expand Manage in the left pane.

3 Expand the Organization, or Uncategorized Hosts to navigate to the guestdomain.

4 Right-click the required guest domain, and then clickManage ApplicationHA.

The Symantec High Availability view appears.

Configuring application monitoring for OraclePerform the following steps to configure monitoring for Oracle on a guest domain.

To configure application monitoring for Oracle

1 In the Symantec High Availability view of the VOMManagement Server console,click Configure Application Monitoring.

This launches the Symantec ApplicationHA Configuration Wizard.

2 Review the information on the Welcome screen and then click Next.

The wizard lists all the supported applications for the system.

3 On the Application Selection page, click Oracle in the Supported Applicationslist.

29Configuring application monitoring with Symantec ApplicationHAConfiguring application monitoring for Oracle

4 Ensure that the Oracle database instances and associated Listener processesare running and then click Next.

An application discovery page appears.

Note: If the Oracle database instances are not running, ensure that they areadded in the oratab file.

5 On Listener Selection panel, review the listed Oracle Listeners.

Name of the Listener.Listener

The $ORACLE_HOME path to Oracle binaries and configurationfiles.

Home

The Oracle user, as the defined owner of executables anddatabase files in /etc/passwd.

Owner

Specify the full path name of the file that is sourced by anyoperation.

EnvFile

The $TNS_ADMIN path to directory in which the Listenerconfiguration file resides (listener.ora).

TnsAdmin

This password is set for the selected Listener.Password


6 On the Oracle Database Selection screen, select the Oracle database instancesand the associated listeners that you want to monitor and then clickConfigure.

Click to select the Oracle instances to monitor.Instances

The Oracle System ID (SID) is used to uniquely identify aparticular database on a system. For this reason, one cannothave more than one database with the same SID on acomputer system.

Database SID

ORACLE_HOME refers to a directory where the Oracle isinstalled

Oracle Home

The Database Owner refers to the owner of the Oracle Homedirectory.

Database Owner

PFILE is a text file that contains parameters and their valueswhich are consumed by the Oracle system .

PFILE Path

Oracle Listener receives incoming client connection requestsand manages the traffic of these requests to the databaseserver.

Listeners

Note:For each selected SID, all discovered Listeners are listed, and associatedlisteners are selected. If discovery phase is not displayed the correct SID -Listener association, you can appropriately select listeners for a selected SID.

7 The wizard performs the application monitoring configuration tasks. TheApplicationHA Configuration screen displays the status of each task.

After all the tasks are complete, click Next.

Note: If the configuration tasks fail, click Diagnostic information to check thedetails of the failure.

You then have to run the wizard again to configure the application monitoring.

8 Click Finish to complete the wizard.

This completes the application monitoring configuration.


9 To view the status of the configured application on a guest domain, on theVOM Management Server console, right-click the appropriate guest domainand then click Manage ApplicationHA.

The Symantec High Availability view appears.

By default, the Component List tab appears. The tab lists each component ofthe configured application and the status description of each component.

For more information on viewing and administering applications through theVeritas Operations Manager, see the Symantec ApplicationHA User’s Guide.


10 To view component dependency for the monitored application, click theComponent Dependency tab.

The component dependency graph appears.

The graph illustrates the dependencies between a selected component group(an application or a group of inter-related components) and its components forthe configured application. The left pane displays component groups and/orconfigured applications. The right pane displays components of the selectedcomponent group or application.

For more information on viewing component dependency for any configuredapplication, see the Symantec ApplicationHA User’s Guide.


Troubleshooting SymantecApplicationHA agent forOracle


■ About troubleshooting ApplicationHA agent for Oracle

■ Error messages common to the Oracle and Netlsnr agents

■ Error messages specific to the Oracle agent

■ Error messages specific to the Netlsnr agent

■ Verifying the Oracle health check binaries and intentional offline for an instanceof Oracle

About troubleshootingApplicationHAagent forOracleReview the information on the error logs that you must access:

■ To check the Oracle installation error log, you must access:

$ORACLE_BASE/oraInventory/logs/installActionsdate_time.log

This file contains the errors that occurred during installation. It clarifies the natureof the error and at exactly which point it occurred during the installation. If thereare any installation problems, you must send this file to Tech Support fordebugging the issue.

■ To check the ApplicationHA log file, you must access:

4Chapter

/var/VRTSvcs/log/engine_A.log

/var/VRTSvcs/log/Oracle_A.log

/var/VRTSvcs/log/Netlsnr_A.log

These files contain all the actions that the ApplicationHA engine and other agentsfor Oracle perform.

Error messages common to the Oracle and Netlsnragents

Table 4-1 lists the ApplicationHA agent for Oracle error messages with thedescription and a recommended solution, if available.

Note:%s in the following table must be replaced by the actual value applicable tothe particular Oracle or Netlsnr instance configured in ApplicationHA.

Table 4-1 ApplicationHA agent for Oracle error messages

Description and solutionMessage

The Home attribute in the Oracle or Netlsnr typehas not been set.

Solution: Set the Home attribute value to thecorrect full path name of the Oracle home.

No ORACLE_HOME specified

The string that is specified for the Home attributein the Oracle or Netlsnr type is incorrect.

Solution: Set the Home attribute value to thecorrect full path name of the Oracle home.

Oracle home directory%s does not exist

The file that the EnvFile attribute specifies forsourcing the environment variables is not present,not readable, or is not a text file.

Solution: Set the EnvFile attribute value to thecorrect full path name. Ensure that the file formatis valid.

File %s is not a valid text file

Internal error.

Solution: Contact Technical Support for furtherassistance.

VCSAgExec returned failure when tryingto execute in-depth test

35Troubleshooting Symantec ApplicationHA agent for OracleError messages common to the Oracle and Netlsnr agents

Table 4-1 ApplicationHA agent for Oracle error messages (continued)


Internal error.


Unable to open pipe from %s

Warning message to indicate that the PID for theOracle process that is specified is different thanthe one registered by the previous monitor cycle.

Process %s restarted

MonScript failed to execute correctly.

Solution: Debug MonScript to assess the exactcause of failure.

Monitor procedure %s returned %s

Internal error while executing MonScript.


Monitor procedure%s did not exit, returnvalue is %s

The Owner attribute in the Oracle type has notbeen set.

Solution: Set the Owner attribute value to thecorrect owner of the database binaries.

No owner for Oracle executables wasspecified

The Operating System user that the Ownerattribute specifies is invalid.

Solution: Set the Owner attribute value to thecorrect owner of the database binaries.

Invalid owner %s for Oracle executableswas specified

The file that the MonScript attribute specifies is notaccessible or not found.

Solution: Make sure that the file name indicates avalid and accessible file.

Access to Monscript %s denied. DetailMonitoring will not be enabled!! Pleasespecify a valid file.

The agent cannot decrypt the password youspecified.

Solution: Use vcsencrypt utility to create a newencrypted password and supply the password.

Encountered errors while decryptingpassword!

Error messages specific to the Oracle agentTable 4-2 lists the error messages for the ApplicationHA agent for Oracle with thedescription and a recommended solution, if available.

36Troubleshooting Symantec ApplicationHA agent for OracleError messages specific to the Oracle agent

Note:%s in the following table must be replaced by the actual Oracle databaseinstance name configured in ApplicationHA.

Table 4-2 Oracle agent error messages


The Sid attribute in the Oracle typehas not been set.

Solution: Set the Sid attribute value tothe correct database instance.

No SID specified

The client utility svrmgrl is not foundin the $ORACLE_HOME/bin directory.

Solution: Verify that the Oracle homehas been correctly specified and thatthis executable is present.

sqlplus not found in %s/bin

The client utility srvctl is not found inthe $ORACLE_HOME/bin directory.

Solution: Verify that the Oracle homehas been correctly specified and thatthis executable is present.

srvctl not found in %s/bin

Warning message to indicate that thefollowing commands were notsuccessful in closing the Oracleinstance in the clean or offline entrypoint:

■ Shutdown immediate■ Shutdown abort

Oracle %s failed to stop

Warning message to indicate that thedatabase instance was not runningeven before the clean or offline entrypoints were executed.

Solution: No action required.

Oracle database %s not running

Warning message to indicate that theOracle processes would be signaledwith SIGTERM.


Oracle (%s) kill TERM %s


Table 4-2 Oracle agent error messages (continued)


Warning message to indicate that theOracle processes would be signaledwith SIGKILL.


Oracle (%s) kill KILL %s

Warning message to indicate thatdatabase is in QUIESCING orQUIESCED mode.

Database in QUIESCING/QUIESCED mode

Warning message to indicate thatdatabase is in RESTRICTED mode.

Database in RESTRICTED mode

Warning message to indicate thatdatabase is in SUSPENDED state.

Database in SUSPENDED state

Refer to Oracle’s alert log for moreinformation.

When a monitor times out as manytimes as the value specified, thecorresponding resource is broughtdown by calling the clean entry point.The resource is then markedFAULTED, or it is restarted, dependingon the RestartLimit attribute value.

Solution: Set theFaultOnMonitorTimeouts attributevalue to 0 so that the monitor failuresare not considered indicative of aresource fault.

Another possible reason could be thatautomatic archiving was not enabledwhile setting up the database.

Solution: Archive the databasemanually. If automatic archival isenabled, set theLOG_ARCHIVE_START parametervalue in the file init.ora to TRUE.

Resource %s - monitor procedure did not completewithin the expected time.




The agent could not find the customscript at the specified location to startthe database.

Solution: Make sure the custom fileexists at the specified location and hasvalid permissions.

Custom script/opt/VRTSagents/ha/bin/Oracle/start_custom.sqldoes not exist. Will not be able to start the database.

The agent could not find the customscript at the specified location to stopthe database.

Solution: Make sure the custom fileexists and the specified location andhas valid permissions.

Custom script/opt/VRTSagents/ha/bin/Oracle/shut_custom.sqldoes not exist. Using default shutdown option.

The file oraerror.dat is not present orhas records in an unsupported format.

Solution: Make sure the file exists andhas data in the supported format.

oraerror.dat did not have records that could beparsed

The MonitorOption value is less than0 or greater than 1.

Solution: Set the MonitorOptionattribute value to 0 or 1.

Incorrect Monitor Option

The health check monitoring option isselected when Oracle version is notOracle 10g or later.

Solution: Set the MonitorOption valueto 0 to select the process checkmonitoring option.

MonitorOption value not applicable for this OracleVersion

Internal error.

Solution: Contact Technical Supportfor further assistance.

VCSAgExec returned failure when trying to executehealth check monitor test

Internal error.

Solution: Contact Technical Supportfor further assistance.

VCSAgExec returned failure while trying to findOracle version




Detail Monitoring has been enabledbut the necessary attributes for detailmonitoring have not been set correctly.

Solution: Set the values of the requiredattributes for detail monitoring correctlyor set DetailMonitor attribute value to0 to disable detail monitoring.

One or more of the attributesUser:Pword:Table:MonScript are not set correctly.Detail monitoring will not be enabled!! Unset theDetailMonitor attribute if you want to disableDetailMonitoring.

Error messages specific to the Netlsnr agentTable 4-3 lists the Netlsnr agent error messages with the description and arecommended solution, if available.

Note:%s in the following table must be replaced by the actual Netlsnr instancename configured in ApplicationHA.

Table 4-3 Netlsnr agent error messages


The agent could not process the /proc entries inthe particular monitor cycle.


Cannot open process directory.

Warning message to indicate that the Listenerprocess was not running even before the clean oroffline entry points were executed.


Listener process %s not running

Warning message to indicate that the Listenerprocess would be signaled with SIGTERM.


Listener %s kill TERM %s

Warning message to indicate that the Listenerprocess would be signaled with SIGKILL.


Listener %s kill KILL %s

40Troubleshooting Symantec ApplicationHA agent for OracleError messages specific to the Netlsnr agent

Table 4-3 Netlsnr agent error messages (continued)


The client utility lsnrctl is not found in the$ORACLE_HOME/bin directory.

Solution: Verify that the Oracle home has beencorrectly specified and that this executable ispresent.

lsnrctl not found in %s/bin

The tnslsnr process does not respond.

Solution: Verify the underlying network protocol.

lsnrctl operation timed out

Verifying the Oracle health check binaries andintentional offline for an instance of Oracle

This section describes the steps to verify the state of Oracle instance, the Oraclehealth check binaries, and intentional offline behavior for an Oracle agent.

Note:The steps listed in the table below should be executed by the operating systemuser specified in Owner attribute.

Table 4-4 lists the checks you can verify with Oracle health check binaries.

Table 4-4 Oracle health check options

SolutionVerify

Run the following command to verify that the$ORACLE_HOME variable is correctly set.

# echo $ORACLE_HOME

Set the ORACLE_HOME environment variable if it is notalready set. For example, run the following command to setthe ORACLE_HOME variable:

# export ORACLE_HOME=/u01/oraHome

where /u01/oraHome is the Oracle home directory path.

If the ORACLE_HOMEvariable is set.

41Troubleshooting Symantec ApplicationHA agent for OracleVerifying the Oracle health check binaries and intentional offline for an instance of Oracle

Table 4-4 Oracle health check options (continued)

SolutionVerify

Run the following command to verify that the $ORACLE_SIDvariable is correctly set.

# echo $ORACLE_SID

If required, run the # export ORACLE_SID=db commandto set the $ORACLE_SID variable.

If the SID for the Oracleinstance is set.

Run the following command to verify that the$LD_LIBRARY_PATH variable is correctly set.

# echo $LD_LIBRARY_PATH

If required, run the # exportLD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATHcommand to set the $LD_LIBRARY_PATH variable.

If the library path is set.

Run the following command to verify that the Oracle healthcheck binary reports the status correctly:

# ./oraapi_<Arch><Oracle_Version>

where <Arch><Oracle_Version> is the system architectureand the Oracle version.

For example, # ./oraapi_6411g, where oraapi_6411gis a binary built for Oracle version 11 on a 64-bit system.

The system displays the following message:

Instance is online

Run # echo $?

If the system displays 110 the instance is online and active.

If the Oracle instance is online.


Table 4-4 Oracle health check options (continued)

SolutionVerify

Run the following command to verify that the Oracle healthcheck binary reports the status correctly:

# ./oraapi_<Arch><Oracle_Version>

where <Arch><Oracle_Version> is the system architectureand the Oracle version.

For example, # ./oraapi_6411g, where oraapi_6411gis a binary built for Oracle version 11 on a 64-bit system.


Failure: Instance Shutdown class: SOFT

Proper Shutdown

Run # echo $?

If the system displays 100 the instance is offline.

If the Oracle instance is offline.

If any of the important Oracle process is killed (for example,ora_pmon_<sid>, ora_smon_<sid>), the Oracle instance isabnormally terminated. In such a case run the Oracle healthcheck binary, and the system displays following message:

For example, run # ./oraapi_6411g

Failure: Abnormal Terminationclass: SOFTAbnormal termination

Run # echo $?

If the system displays 98 the termination of Oracleprocesses. The VCS resource fails over as Oracletermination was abnormal and not intentional.

If the Oracle instance isabnormally terminated.

Verifying the intentional offline behavior of the VCS Oracle agentPerform the following steps to verify if VCS Oracle agent determines a gracefulshutdown of a resource as intentional offline.


1 Bring the resource online.

# hares -online ORA_oraprod -sys system

Where system is the name of the system that has Oracle database installed.

2 Verify that IntentionalOffline and MonitorOption attributes are set to 1.

3 Stop the Oracle instance properly outside VCS control.

# su - oracle

# bash

# bash-3.00$ sqlplus "/ as sysdba"


SQL> shutdown immediate

Database closed.

Database dismounted.

ORA-03113: end-of-file on communication channel

4 You can verify the log files to confirm if the shutdown was graceful.

Sample log message:

VCS INFO V-16-1-13470 Resource ORA_oraprod

(Owner: Unspecified, Group: ORA_PROD_Group) is offline on system.

(Intentional But NOT initiated by VCS)

Oracle agent has identified the Intentional offline for the resource.


Resource type definitions

This appendix includes the following topics:

■ About the resource type and attribute definitions

■ Resource type definition for the Oracle agent

■ Resource type definition for the Netlsnr agent

About the resource type and attribute definitionsThe resource type represents the configuration definition of the agent and specifieshow the agent is defined in the configuration file. The attribute definitions describethe attributes associated with the agent. The required attributes describe theattributes that must be configured for the agent to function.

Resource type definition for the Oracle agentThe ApplicationHA agent for Oracle is represented by the Oracle resource type inApplicationHA.

type Oracle (

static str AgentDirectory = "/opt/VRTSagents/ha/bin/Oracle"

static keylist SupportedActions = { VRTS_GetInstanceName,

VRTS_GetRunningServices, DBRestrict, DBUndoRestrict,

DBResume, DBSuspend, DBTbspBackup,

"home.vfd", "owner.vfd", "getid", "pfile.vfd" }

static str ArgList[] = { Sid, Owner, Home, Pfile, StartUpOpt,

ShutDownOpt, EnvFile, AutoEndBkup,

User, Pword, Table, MonScript, Encoding,

MonitorOption }

AAppendix

str Sid

str Owner

str Home

str Pfile

str StartUpOpt = STARTUP_FORCE

str ShutDownOpt = IMMEDIATE

str EnvFile

boolean AutoEndBkup = 1

str MonScript = "./bin/Oracle/SqlTest.pl"

str User

str Pword

str Table

str Encoding

int MonitorOption = 0

static boolean IntentionalOffline = 0

)

Attribute definition for the Oracle agentReview the description of the Oracle agent attributes. The agent attributes areclassified as required, optional, and internal.

Table A-1 lists the required attributes. You must assign values to the requiredattributes.

Table A-1 Required attributes for Oracle agent

DefinitionType anddimension

Requiredattributes

The variable $ORACLE_SID that represents theOracle instance. The Sid is consideredcase-sensitive by the Oracle agent and by theOracle database server.

string-scalarSid

The Oracle user, as the defined owner ofexecutables and database files in /etc/passwd.

The agent also supports LDAP users as Oracleuser.

string-scalarOwner

46Resource type definitionsResource type definition for the Oracle agent

Table A-1 Required attributes for Oracle agent (continued)


Requiredattributes

The $ORACLE_HOME path to Oracle binariesand configuration files. For example, you couldspecify the path as /opt/ora_home.

Note: Do not append a slash (/) at the end ofthe path.

string-scalarHome

Table A-2 lists the optional attributes for Oracle agent. You can configure the optionalattributes if necessary.

Table A-2 Optional attributes for Oracle agent

DefinitionType andDimension

OptionalAttributes

Startup options for the Oracle instance. Thisattribute can take the following values:

■ STARTUP■ STARTUP_FORCE■ RESTRICTED■ RECOVERDB■ CUSTOM■ SRVCTLSTART■ SRVCTLSTART_RO

Default is STARTUP_FORCE.

See “Startup and shutdown options for the Oracleagent” on page 14.

string-scalarStartUpOpt


Table A-2 Optional attributes for Oracle agent (continued)


OptionalAttributes

Shut down options for the Oracle instance. Thisattribute can take the following values:

■ IMMEDIATE■ TRANSACTIONAL■ CUSTOM■ SRVCTLSTOP■ SRVCTLSTOP_TRANSACT■ SRVCTLSTOP_ABORT■ SRVCTLSTOP_IMMEDIATE

Default is IMMEDIATE.

See “Startup and shutdown options for the Oracleagent” on page 14.

string-scalarShutDownOpt

The full path name of the file that is sourced bythe entry point scripts. This file contains theenvironment variables set by the user for theOracle database server environment such asLD_LIBRARY_PATH, NLS_DATE_FORMAT,and so on.

The syntax for the contents of the file dependson the login shell of Owner. File must bereadable by Owner. The file must not containany prompts for user input.

string-scalarEnvFile

The name of the initialization parameter file withthe complete path of the startup profile.

You can also use the server parameter file.Create a one-line text initialization parameter filethat contains only the SPFILE parameter. Seethe Oracle documentation for more information.

See “Using the SPFILE in an ApplicationHAguest domain” on page 61.

string-scalarPfile




OptionalAttributes

Setting the AutoEndBkup attribute to a non-zerovalue takes the datafiles in the database out ofthe backup mode, during Online.

Default = 1

See “About ApplicationHA requirements forinstalling Oracle” on page 24.

integer-scalarAutoEndBkup

Monitor options for the Oracle instance. Thisattribute can take values 0 or 1.

■ 0—Process check monitoring■ 1—Health check monitoring

You must set the value of this attribute as 1 touse the intentional offline functionality of theagent.

Default = 0

See “Monitor options for the Oracle agent”on page 16.

integer-scalarMonitorOption




OptionalAttributes

This resource-type level attribute specifies thefrequency at which the agent for this resourcetype must perform second-level or detailedmonitoring.

You can also override the value of this attributeat resource-level. The value indicates the numberof monitor cycles after which the agent mustperform second-level or detailed monitoring.

For example, the value 5 indicates that the agentwill perform detailed monitoring every five onlinemonitor intervals.

Note: Upgrade from Symantec ApplicationHA5.1 SP2 to Symantec ApplicationHA 6.1 fails ifthe DetailMonitor attribute had been used fordetailed monitoring. If you are upgrading toSymantec ApplicationHA 6.1, set the value ofthe DetailMonitor attribute to theLevelTwoMonitorFreq attribute and reset thevalue of the DetailMonitor attribute to 0.

Note: If you set the AutoEndBkup attribute valueto 0, then make sure that theLevelTwoMonitorFreq attribute value is set to 1for detail monitoring.

Default = 0

integer-scalarLevelTwoMonitorFreq

Pathname to the script provided for detailmonitoring. The default (basic monitoring) is tomonitor the database PIDs only.

Note: Detail monitoring is disabled if the valueof the attribute MonScript is invalid or is set toan empty string.

The path of the supplied detail monitor script is/opt/VRTSagents/ha/bin/Oracle/SqlTest.pl.

MonScript also accepts a pathname relative to/opt/VRTSagents/ha. A relative pathname shouldstart with "./", as in the path./bin/Oracle/SqlTest.pl.

string-scalarMonScript




OptionalAttributes

Internal database user. Connects to the databasefor detail monitoring.

string-scalarUser

Encrypted password for internal database-userauthentication.

Encrypt passwords only when entering themusing the command-line. Passwords must beencrypted using the ApplicationHA Encrypt utility.

string-scalarPword

Table for update by User/Pword.string-scalarTable

This resource-type level attribute defines howVCS reacts when Oracle is intentionally stoppedoutside of VCS control.

If you stop Oracle out of VCS control, the agentbehavior is as follows:

■ 0—The Oracle agent registers a fault andinitiates the failover of the service group.

■ 1—The Oracle agent takes the Oracleresource offline when Health checkmonitoring is enabled.If Health check monitoring is not enabled, theagent registers a fault and initiates the failoverof the service group.

Note: If you want to use the intentional offlinefunctionality of the agent, you must set the valueof the MonitorOption attribute as 1 to enableHealth check monitoring.

Default = 0

See Symantec ApplicationHA User's Guide.

static-booleanIntentionalOffline

Table A-3 lists the internal attribute for Oracle agent. This attribute is for internaluse only. Symantec recommends not to modify the value of this attribute.


Table A-3 Internal attributes for Oracle agent


OptionalAttributes

Specifies the location of binaries, scripts, andother files related to the Oracle agent.

Default is /opt/VRTSagents/ha/bin/Oracle.

static-stringAgentDirectory

Resource type definition for the Netlsnr agentThe Netlsnr agent of the ApplicationHA agent for Oracle is represented by theNetlsnr resource type in ApplicationHA.

type Netlsnr (

static str AgentDirectory = "/opt/VRTSagents/ha/bin/Netlsnr"

static keylist SupportedActions = { VRTS_GetInstanceName,

VRTS_GetRunningServices, "tnsadmin.vfd" }

static str ArgList[] = { Owner, Home, TnsAdmin, Listener,

EnvFile, MonScript, LsnrPwd, Encoding }

str Owner

str Home

str TnsAdmin

str Listener

str EnvFile

str MonScript = "./bin/Netlsnr/LsnrTest.pl"

str LsnrPwd

str Encoding

static boolean IntentionalOffline = 0

)

Attribute definition for the Netlsnr agentReview the description of the Netlsnr agent attributes. The agent attributes areclassified as required, optional, and internal.

Table A-4 lists the required attributes for Netlsnr agent. You must assign values tothe required attributes.

52Resource type definitionsResource type definition for the Netlsnr agent

Table A-4 Required attributes for Netlsnr agent


Requiredattributes

The Oracle user, as the defined owner ofexecutables and database files in /etc/passwd.

The agent also supports LDAP users as Oracleuser.

string-scalarOwner

The $ORACLE_HOMEpath to Oracle binaries andconfiguration files. For example, you could specifythe path as /opt/ora_home.

Do not append a slash (/) at the end of the path.

string-scalarHome

Table A-5 lists the optional attributes for Netlsnr agent. You can configure theoptional attributes if necessary.

Table A-5 Optional attributes for Netlsnr agent


Optionalattributes

The $TNS_ADMIN path to directory in which theListener configuration file resides (listener.ora).

Default is /var/opt/oracle.

string-scalarTnsAdmin

Name of Listener. The name for Listener isconsidered case-insensitive by the Netlsnr agentand the Oracle database server.

Default is LISTENER.

string-scalarListener

The ApplicationHA encrypted password used tostop and monitor the listener. This password is setin the Listener configuration file.

Encrypt passwords only when entering them usingthe command-line. Passwords must be encryptedusing the ApplicationHA Encrypt utility.

string-scalarLsnrPwd


Table A-5 Optional attributes for Netlsnr agent (continued)


Optionalattributes

Specifies the full path name of the file that issourced by the entry point scripts. This file containsthe environment variables set by the user for theOracle listener environment such asLD_LIBRARY_PATH and so on.

The syntax for the contents of the file depends onthe login shell of Owner. This file must readableby Owner. The file must not contain any promptsfor user input.

string-scalarEnvFile

Pathname to the script provided for detailmonitoring. By default, the detail monitoring isenabled to monitor the listener process.

Note: If the value of the attribute MonScript is setto an empty string, the agent disables detailmonitoring.

The pathname to the supplied detail monitoringscript is/opt/VRTSagents/ha/bin/Netlsnr/LsnrTest.pl.

MonScript also accepts a pathname relative to/opt/VRTSagents/ha. A relative pathname shouldstart with "./", as in the path./bin/Netlsnr/LsnrTest.pl.

string-scalarMonScript

Specifies operating system encoding thatcorresponds to Oracle encoding for the displayedOracle output.

Default is "".

string-scalarEncoding

Do not change the value of this attribute.

Default = 0

static-booleanIntentionalOffline

Table A-6 lists the internal attribute for Netlsnr agent. This attribute is for internaluse only. Symantec recommends not to modify the value of this attribute.

Note: To change the value of any attribute, use CLI /Veritas Operations Manager


Table A-6 Internal attributes for Netlsnr agent


OptionalAttributes

Specifies the location of binaries, scripts, and otherfiles related to the Netlsnr agent.

Default is /opt/VRTSagents/ha/bin/Netlsnr.

static-stringAgentDirectory


Detail monitoring


■ Setting the PATH variable

■ Setting up detail monitoring for ApplicationHA agent for Oracle

■ Setting up detail monitoring for ApplicationHA agent for Netlsnr

Setting the PATH variableApplicationHA commands reside in the /opt/VRTS/bin directory. Add this directoryto your PATH environment variable.

To set the PATH variable

◆ Perform one of the following steps:

For the Bourne Shell (sh or ksh), type:

$ PATH=/opt/VRTS/bin:$PATH; export PATH

For the C Shell (csh or tcsh), type:

$ setenv PATH :/opt/VRTS/bin:$PATH

Setting up detail monitoring for ApplicationHA agentfor Oracle

The ApplicationHA agent for Oracle provides two levels of application monitoring:primary (basic monitoring) and secondary (detail monitoring).

■ In the basic monitoring mode, the agent monitors the Oracle processes to verifythat they are continuously active.

BAppendix

■ In the detail monitoring mode, the agent executes the script defined in theattribute MonScript of the Oracle and the Netlsnr resources. If the scriptsuccessfully executes, the agent considers the resource available. You cancustomize the default script according to your configuration.

See “Monitor options for the Oracle agent” on page 16.

You can use the agent’s detail monitoring capability to monitor the status of adatabase and listener and increase the confidence in their availability. Before settingup detail monitoring, you must have the agent running satisfactorily at the basiclevel of monitoring.

Note:Disable detail monitoring before undertaking any database maintenance thatinvolves disabling database access to external users.

Setting up detail monitoring for OracleDetail monitoring for an Oracle resource verifies whether a database is ready fortransactions by performing an update transaction against a table within the database.The update action is taken by the two scripts, SqlTest.pl and SimpleTest.pl,provided with the ApplicationHA agent for Oracle. The scripts are available underthe directory /opt/VRTSagents/ha/bin/Oracle/. Both scripts update the timestampto monitor the database.

The SqlTest.pl script checks whether the database is open before updating thetimestamp. If the database is found to be in restricted mode, quiesced mode, orsuspendedmode, the monitor returns success. In such a case, only basic monitoringoccurs. The SimpleTest.pl script does not perform database checks but onlyissues update statements against the table.

Before enabling detail monitoring for Oracle, you must create a test table (with atimestamp) in the Oracle database. The agent uses this test table for internalpurposes. Symantec recommends that you do not perform any other transactionon the test table. The detail monitor script, MonScript, must exist and have executepermission for root. You can use a custom monitor script, or the scripts providedwith the agent. In the monitor script, the return code 100 indicates failure. Returncodes from 101 to 110 indicate success.

The example to set up detail monitoring, based on the use of the supplied script,shows how to create and test a table for use by detail monitoring, and how to enabledetail monitoring.

57Detail monitoringSetting up detail monitoring for ApplicationHA agent for Oracle

To set up detail monitoring for Oracle

1 Make the ApplicationHA configuration writable:

haconf -makerw

2 Freeze the service group to avoid automated actions by ApplicationHA causedby an incomplete reconfiguration:

hagrp -freeze DiscoveredOracleSG

3 Log on as an Oracle user.

su - <Owner>

4 Set the environment variables for ORACLE_HOME and ORACLE_SID.

export ORACLE_HOME=<Home>

export ORACLE_SID=<Sid>

5 Start the sqlplus utility to set up a database table:

$ORACLE_HOME/bin/sqlplus /nolog

6 As the database administrator, issue the following statements at the sqlplus

prompt to create the test table:

connect / as sysdba

create user <User>

identified by <Pword>

default tablespace USERS

temporary tablespace TEMP

quota 100K on USERS;

grant create session to <User>;

create table <User>.<Table> ( tstamp date );

insert into <User>.<Table> (tstamp) values (SYSDATE);


7 To test the database table for use, do the following:

disconnect

connect <User>/<Pword>

update <User>.<Table> set ( tstamp ) = SYSDATE;

select TO_CHAR(tstamp, 'MON DD, YYYY HH:MI:SS AM')

from <User>.<Table>;

exit

8 Enable the detail monitoring for the Oracle resource using the followingApplicationHA commands:

hares -modify Oracle_<SID>_res User User

hares -modify Oracle_<SID>_res Pword Pword

hares -modify Oracle_<SID>_res Table Table

hares -modify Oracle_<SID>_res MonScript "./bin/Oracle/SqlTest.pl"

hares -override Oracle_<SID>_res LevelTwoMonitorFreq

hares -modify Oracle_<SID>_res LevelTwoMonitorFreq 1

haconf -dump -makero

hagrp -unfreeze DiscoveredOracleSG

Enabling and disabling detail monitoring for Oracle resourceReview the instructions to enable or disable detail monitoring.

To enable detail monitoring

◆ Set the LevelTwoMonitorFreq attribute to 1.


To disable detail monitoring

◆ Set the LevelTwoMonitorFreq attribute to 0.



Setting up detail monitoring for ApplicationHA agentfor Netlsnr

For Netlsnr agent, the detail monitoring is enabled by default to monitor the listenerprocess.

You can disable detail monitoring by setting the value of the attribute MonScript toan empty string.

You can enable detail monitoring for Netlsnr by specifying a value for the MonScriptattribute. The example to set up detail monitoring uses the supplied monitor scriptfor Netlsnr, /opt/VRTSagents/ha/bin/Netlsnr/LsnrTest.pl. The detail monitoring scriptfor the Netlsnr resource uses the Listener command lsnrctl status $Listener

to test the Listener process.

To disable detail monitoring for Netlsnr

◆ Disable detail monitoring by setting the MonScript attribute to an empty string:

haconf -makerw


hares -modify LSNR_$Listener_res MonScript ""



To set up detail monitoring for Netlsnr

1 Make the ApplicationHA configuration writable:

haconf -makerw

2 Freeze the service group to avoid automated actions by ApplicationHA causedby an incomplete reconfiguration:


3 Enable detail monitoring by entering the following commands:

hares -modify LSNR_$Listener_res MonScript "./bin/Netlsnr/LsnrTest.pl"



60Detail monitoringSetting up detail monitoring for ApplicationHA agent for Netlsnr

Using the SPFILE in anApplicationHA system forOracle


■ Using the SPFILE in an ApplicationHA guest domain

Using the SPFILE in an ApplicationHA guest domainWhen using the ApplicationHA agent for Oracle, you can start a database instanceby specifying a PFILE. If you do not specify the PFILE, the database instance startsup using the default SPFILE.

The agent attribute Pfile must specify the location of the PFILE. If your configurationuses the SPFILE, the contents of the PFILEmust specify the location of the SPFILE,which must be created from the PFILE.

Note: To specify PFILE for Oracle instance, use CLI / Veritas Operation Manager(VOM).

CAppendix

To create the SPFILE from a PFILE

◆ The SPFILE must be created from the PFILE. You must have the sysdba orthe sysoper system privileges to create an SPFILE.

You can run the following command to create the SPFILE:

CREATE SPFILE [= spfile_name] FROM PFILE [= pfile_name ];

If you do not specify the complete path for the SPFILE, this command createsan SPFILE at the default location ($ORACLE_HOME/dbs on Solaris).

To specify the SPFILE location in the PFILE

◆ To specify the location of the SPFILE in a PFILE, create a PFILE and specifythe following entry in the PFILE:

SPFILE = spfile_location

The variable spfile_location represents the complete path of the SPFILE. Forexample:

SPFILE = /database/startup/spfileora1.ora

In this case, to start the database use the following command:

startup pfile=location_of_pfile

62Using the SPFILE in an ApplicationHA system for OracleUsing the SPFILE in an ApplicationHA guest domain

Best practices


■ Best practices for multiple Oracle instance configurations in a ApplicationHAenvironment

Best practices for multiple Oracle instanceconfigurations in a ApplicationHA environment

Review some of the best practices for using multiple Oracle instances in aApplicationHA environment:

■ Define the system parameters such that the allocation of semaphore and sharedmemory is appropriate on the guest domain.

■ Use a dedicated set of binaries for each Oracle instance, even if each instanceuses the same Oracle version.

■ If your configuration uses the same Oracle version for all instances, install aversion on the root disk or preferably on a secondary disk. Locate the pfiles inthe default location and define several listener processes.

■ If your configuration has different versions of Oracle, create a separate$ORACLE_HOME for each Oracle version.

■ Listeners accompanying different versions of Oracle may not bebackward-compatible. So, if you want to create a single listener.ora file, youmust verify that the listener supports the other versions of Oracle in the guestdomain. You must also create a separate Envfile for each version of Oracle.

■ Make sure that each listener listens to a different virtual address. Also, assigndifferent names to listeners and make sure that they do not listen to the sameport.

DAppendix

Date post:	30-Apr-2018
Category:	Documents
Upload:	vuongbao
View:	221 times
Download:	1 times

Symantec ApplicationHA 6.1 Agent for Oracle Configuration...

Documents