+ All Categories
Home > Documents > VERITAS Cluster Server Storage Agent for NetApp...

VERITAS Cluster Server Storage Agent for NetApp...

Date post: 09-Sep-2018
Category:
Upload: truonghanh
View: 222 times
Download: 0 times
Share this document with a friend
34
October 2000 30-000073-399 VERITAS Cluster Server Storage Agent for NetApp and Oracle Installation and Configuration Guide Solaris
Transcript
Page 1: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

VERITAS Cluster Server™

Storage Agent forNetApp™ and Oracle™

Installation and Configuration Guide

Solaris

October 200030-000073-399

Page 2: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Disclaimer

The information contained in this publication is subject to change without notice.VERITAS Software Corporation makes no warranty of any kind with regard to thismanual, including, but not limited to, the implied warranties of merchantability andfitness for a particular purpose. VERITAS Software Corporation shall not be liable forerrors contained herein or for incidental or consequential damages in connection with thefurnishing, performance, or use of this manual.

Copyright

Copyright © 2000 VERITAS Software Corporation. All rights reserved. VERITAS is aregistered trademark of VERITAS Software Corporation in the US and other countries.The VERITAS logo and VERITAS Cluster Server are trademarks of VERITAS SoftwareCorporation. All other trademarks or registered trademarks are the property of theirrespective owners.

Printed in the USA, October 2000.

VERITAS Software Corporation1600 Plymouth St.Mountain View, CA 94043Phone 650–335–8000Fax 650–335–8050www.veritas.com

Page 3: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vii

Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vii

For Customers Outside U.S. and Canada . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vii

Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Chapter 1. Installing and Configuring the Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

About the Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Agent Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

An Example NetApp Configuration in a VCS Environment . . . . . . . . . . . . . . . . . . . . . . 3

Configuring the NetApp Filer in a VCS Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Installing the VCS Storage Agent for NetApp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

What Is Installed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Configuring the Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

VCS NetApp Agent Resource Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

NetAppFiler Resource Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Type Definition for NetAppFiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Sample Configuration of NetAppFiler Resource in main.cf . . . . . . . . . . . . . . . . . 8

NetAppExport Resource Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Type Definition for NetAppExport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Sample Configuration for NetAppExport Resource in main.cf . . . . . . . . . . . . . . 10

Configuring the NetApp Agent with Cluster Manager . . . . . . . . . . . . . . . . . . . . . . . . . . 11

iii

Page 4: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Importing the NATypes.cf File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Adding the NetApp Group Using the Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Configuring the Agent by Editing the Configuration Files . . . . . . . . . . . . . . . . . . . . . . 13

Editing the Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Verifying the Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Setting Up to Handle Possible NIC Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Setting Up Multiple NICs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Using the RebootOption of the NetAppFiler Resource . . . . . . . . . . . . . . . . . . . . . . . 16

Using a Shell Script to Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Modifying the Agent Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Setting Up Oracle to Run with the NetApp Filer in a VCS Cluster . . . . . . . . . . . . . . . . 17

Running Oracle with a Single NetApp Filer in a VCS Cluster . . . . . . . . . . . . . . . . . . . . 17

Running Oracle with a NetApp Cluster Filer in a VCS Cluster . . . . . . . . . . . . . . . . . . . 19

Disabling the Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Removing the Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Requirements for Setting Up Optional VCS Volume on the Filer . . . . . . . . . . . . . . . . . 21

Suggestions for Testing Your Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Testing the Configuration From Cluster Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Testing the Configuration on the Filer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Troubleshooting Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

NetApp Filer etc/exports File Must Not Contain VCS-Monitored Volumes . . 23

Export Options Must be Localized for Each System . . . . . . . . . . . . . . . . . . . . . . 23

Export Options: Order of Hostnames in etc/hosts File is Significant . . . . . . . . 23

NetApp Filer etc/hosts.equiv Must List Root for Each System . . . . . . . . . . . . . 24

Oracle Binaries Must Be Installed on Local Disk . . . . . . . . . . . . . . . . . . . . . . . . . 24

NIC Failures Take Approximately Five Minutes to Fail Over . . . . . . . . . . . . . . 24

Shutdown Process Specified by RebootOption Must be Under 120 Seconds . . 24

VCS Volume on Filer Requires Entry in etc/exports . . . . . . . . . . . . . . . . . . . . . . 24

Mount Options on the Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

iv VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 5: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Rebooting the Filer Could Cause Application to Fault . . . . . . . . . . . . . . . . . . . . . 25

“Broken Pipe” Messages in Engine Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Contents v

Page 6: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

vi VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 7: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Preface

This document provides instructions on how to install and configure the VERITAS ClusterServer (VCS) Agent, version 1.3.0, for NetApp F700 Series File Servers for use with Oracle.For information about VCS, refer to the VERITAS Cluster Server User’s Guide.

If this document is dated more than six months prior to the date you are installing yourstorage agent, contact VERITAS Technical Support to confirm the latest supportedversions of the application and operating system.

Technical SupportFor assistance with this VERITAS product, or for information regarding VERITAS servicepackages, contact Technical Support at 800.342.0652 (U.S. and Canada). You may alsocontact Technical Support via email at [email protected].

For Customers Outside U.S. and Canada

From Europe, the Middle East, or Asia, visit the Technical Support website athttp://support.veritas.com for a list of each country’s contact information.

vii

Page 8: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Conventions

Conventions

Typeface Usage

courier Computer output, files, attribute names, device names, and directories

courier(bold)

User input and commands, keywords in grammar syntax

italic New terms, titles, emphasis

italic Variables

Symbol Usage

% C shell prompt

$ Bourne/Korn shell prompt

# Superuser prompt (for all shells)

viii VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 9: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Installing and Configuring the Agent

1 Welcome to the VERITAS® Cluster Server (VCS)™ agent, version 1.3.0, for NetApp withOracle. This guide describes how to install and configure the agent and includestroubleshooting tips to assist you in specific areas.

About the AgentThe agent monitors NetApp NFS file servers that function as shared storage accessedthrough NFS mounts. The agent supports a single NetApp Filer or a NetApp Cluster Filerconfiguration.

The NetApp agent consists of the NetAppFiler and the NetAppExport resource types. TheNetAppFiler resource monitors the NetApp filer, and the NetAppExport resourcemonitors and controls each volume that the filer exports. There is one NetAppFilerresource for each filer, and one NetAppExport resource for each exported volume.

Agent Operations

The NetAppFiler agent monitors the filer device. If the server can contact the filer, theagent reports the resource as ONLINE. If the system is unable to contact the filer within aspecified time, typically 180 seconds, the agent reports the resource as OFFLINE.

The NetAppExport agent monitors each volume exported by the filer and the mountpoints on the server corresponding to each of the exported volumes. When the filer isrebooted, or if a filer is taken over in a cluster-filer setup, the agent reexports volumes thatare mounted on the server.

1

Page 10: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Prerequisites

Prerequisites✔ Set up the NetApp filer server. See “An Example NetApp Configuration in a VCS

Environment.”

✔ Create volumes on the NetApp filer.

Refer to the appropriate NetApp Filer documentation for instructions on setting upvolumes.

When you create volumes, you can create a volume used exclusively by VCS toprovide additional data protection. For details, see“Requirements for Setting UpOptional VCS Volume on the Filer” on page 21.

✔ VCS 1.3.0 or later must be installed on each of the systems in the cluster.

Hardware Requirements◆ NetApp F700 Series Filer.

◆ A VCS-supported Solaris server.

◆ Ethernet network interface.

To provide higher availability for your network interface, you can configure morethan one NIC (Network Interface Card) on each system in the cluster. See “Setting Upto Handle Possible NIC Failures” on page 15.

Software Requirements◆ Solaris 2.5.1, 2.6, 7, and 8.

◆ VCS 1.3.0

◆ Oracle 8.1.x

◆ VCS Enterprise Agent, Version 1.3.0, for Oracle

2 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 11: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

An Example NetApp Configuration in a VCS Environment

An Example NetApp Configuration in a VCS EnvironmentIn the following illustrated example, a single NetApp filer is set up with a two-systemVCS cluster. An application, Oracle, for example, is set up with a NetApp service groupon sysA consisting of the NetAppFiler and NetAppExport resource types. If sysA fails, theservice group fails over to sysB. The dependency tree is shown below.

Note On an Oracle-NetApp setup, a NIC failure on a cluster system causes the failoverprocess to take approximately five minutes.

Public Network

hb1

Local Disk Local Disk

/export1 /export1hb2

/vol/export1

FilersysBsysA

Filer Disk/Volumes

/vol/vol0

Oracle

NetAppExport IP

NICNetAppFiler

SqlNet

Oracle

oracle_lsn

oracle_home

oracle_exp_data

oracle_filer

oracle_IP

oracle_NIC

Chapter 1 Installing and Configuring the Agent 3

Page 12: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the NetApp Filer in a VCS Environment

Configuring the NetApp Filer in a VCS EnvironmentSuccessful operation of NetApp in a VCS environment depends on the following setuprequirements:

✔ Ensure that VCS controls the volumes on the filer that are exported and mounted.Remove the relevant entries in the file etc/exports on the filer to make sure thevolumes are not autoexported by the filer.

✔ On the filer, from the command line:

◆ Set the option rsh.enable to “on” by entering:

> options rsh.enable on

◆ List the hostname of each system in the cluster in the trusted.hosts option ofrsh (the default setting is “*” signifying all hosts):

> options trusted.hosts sysa sysb ...

You can list current values for options by typing the command: options

✔ On the filer, add lines to the file etc/hosts.equiv to allow rsh access for everysystem in the cluster. You can do this in two ways:

◆ Edit the file etc/hosts.equiv directly, using vi, or

◆ Use the NetApp administration browser (http://<filername>/na_admin)to navigate to FilerView > Security > Rsh Access Editor and edit the fileetc/hosts.equiv.

✔ Oracle system files, or “binaries,” must be installed on the local disk of each system inthe VCS cluster. See “Setting Up Oracle to Run with the NetApp Filer in a VCSCluster” on page 17.

4 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 13: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Installing the VCS Storage Agent for NetApp

Installing the VCS Storage Agent for NetApp

1. Log in as root.

2. Insert the CD into a drive connected to your system.

◆ If you are running Solaris volume-management software, the softwareautomatically mounts the CD as /cdrom/cdrom0. Type the following commandto install the agent:

# cd /cdrom/cdrom0# pkgadd -d .

◆ If you are not running Solaris volume-management software, you must mountthe CD manually. For example:

# mount -F hsfs -o ro /dev/dsk/c0t6d0s2 /mnt

Where, in this example, /dev/dsk/c0t6d0s2 is the default for the CD drive.

# cd /mnt# pkgadd -d .

3. Repeat steps 1 and 2 on each system that will become part of the NetApp servicegroup.

What Is Installed

VCS installs the NetApp Filer agents in /opt/VRTSvcs/bin/NetAppFiler and/opt/VRTSvcs/bin/NetAppExport.

In addition to the agents, you can find the sample configuration files in the directory/etc/VRTSvcs/conf/sample_na:

NAtypes.cfmain.cfmain.cf.oracle_on_single_filermain.cf.oracle_on_cluster_filermain.cf.multinic

Chapter 1 Installing and Configuring the Agent 5

Page 14: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the Agent

Configuring the AgentYou can configure the NetApp agent using two methods:

◆ By using VCS Cluster Manager (the VCS graphical user interface) to edit a resourcegroup template for the NetApp agent. See “Configuring the NetApp Agent withCluster Manager” on page 11), or,

◆ By using the types configuration files and directly editing the sample main.cf filesupplied with the NetApp agent (see “Configuring the Agent by Editing theConfiguration Files” on page 13). This method requires that VCS be stopped andrestarted before the configuration takes effect.

Before you configure the agent, review the following tables that describe the NetAppresource types and their attributes. The resource type definition files and the samplemain.cf configurations are also shown for reference.

VCS NetApp Agent Resource Types

For your reference, the tables and examples on the following pages describe the twoNetApp filer resource types: NetAppFiler and NetAppExport. These resources aredefined in the resource type definition file, NAtypes.cf.

You must customize these resources by modifying the VCS configuration file, main.cf.By supplying specific values for the resource attributes, you can configure the agent tomatch your setup. In the examples that follow the tables, the values correspond to theillustration on page 3.

6 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 15: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the Agent

NetAppFiler Resource Type

Description Monitors the NetApp Filer.

Entry Points Online—Not Applicable.

Offline—Not Applicable.

Monitor—Monitors the IP address of the filer.

State Definitions MONITOR—Pings the IP address specified by the filer IP address attribute. If the ping isunable to contact the NetAppFiler within FilerPingTimeout seconds, the resourceis marked as OFFLINE.

Attributes Type andDimension

Definition

FilerIPAddress scalar-string IP address of the filer resource.

FilerPingTimeout scalar-integer Optional attribute. The number of seconds the ping command waits tohear from the filer. The default value is 180 seconds.

FilerControlPath scalar-string Optional attribute. This attribute, which refers to the volume on thefiler dedicated entirely to VCS, protects data when the heartbeatingbetween cluster systems fails. It also prevents more than one systemfrom mounting a volume.

The volume must be exported with access to the root user on each ofthe cluster systems. It requires an explicit entry in the etc/exportsfile of the filer.

For example, if sysA and sysB are connected to the filer, and theFilerControlPathName is /vol/volvcs/, the entry in the exportsfile is:

/vol/volvcs -access=sysA: sysB,root=sysA: sysB

Refer to “Requirements for Setting Up Optional VCS Volume on theFiler” on page 21.

NicResName scalar-string Optional attribute. This string names the NIC resource in this group.This attribute is required to reboot the system in the event of a NICfailure. See “Setting Up to Handle Possible NIC Failures” on page 15.

RebootOption scalar-string Optional attribute. This string can be used to reboot the server, whichis necessary in case of NIC failures. See “Setting Up to Handle PossibleNIC Failures” on page 15.

Default = "".

To specify a reboot string, use, for example: "sync;sync;reboot"

To specify a file containing a shutdown script, use, for example:"/your_path/your_shutdown_script"

Chapter 1 Installing and Configuring the Agent 7

Page 16: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the Agent

Type Definition for NetAppFiler

type NetAppFiler (static str ArgList[] = { FilerIPAddress,

FilerControlPath,FilerPingTimeout,NicResName,RebootOption }

NameRule = NetAppFiler_ + resource.FilerIPAddressstatic str Operations = Nonestr FilerIPAddressstr FilerControlPathint FilerPingTimeout=180str NicResName = ""str RebootOption = ""static int MonitorTimeout = 200

)

Sample Configuration of NetAppFiler Resource in main.cf

NetAppFiler netapp_filer (FilerIPAddress = "166.98.32.19"

)

8 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 17: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the Agent

NetAppExport Resource Type

Description Brings online, takes offline, and monitors the NFS mount point and the volumeexported on the NetApp filer.

Entry Points Online—Exports the volume on the filer. Mounts the volume on the host.

Offline—-Unmounts the volume on the host. Deports the volume on the filer.

Monitor—-The resource is marked ONLINE if the mount point is mounted on thehost and the volume is exported on the Filer.

Clean— Unmounts the volume on the host. Deports the volume on the filer.

StateDefinitions

ONLINE—NFS mount point is mounted and the volume on the filer is exported.

OFFLINE—NFS mount point is unmounted and the volume on the filer is deported.

MONITOR—Monitors the NFS mount point on the host and the volume on the filer.

Attributes Type andDimension

Definition

FilerPathName scalar-string Name of the volume mounted on the host. This volume must notbe exported by the filer (there should be no entry for this volumein the etc/exports file). VCS controls this volume and issuesexport commands to the filer.

FilerOptions scalar-string Export options for the volume described by theFilerPathName.The options can be localized on each host sothe host can control the volume. See the “Sample Configurationfor NetAppExport Resource in main.cf” on page 10. Thehostname used in the FilerOptions field must be the same as thefirst name specified after the IP address in the filer’s etc/hostsfile. See “Export Options: Order of Hostnames in etc/hosts File isSignificant” on page 23.

FilerResName scalar-string Name of the NetAppFiler resource type.

MountPathName scalar-string Pathname of the NFS mounted directory on the host.

MountOptions scalar-string Options for the NFS mount on the host.

Chapter 1 Installing and Configuring the Agent 9

Page 18: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the Agent

Type Definition for NetAppExport

type NetAppExport (static str ArgList[] = { FilerPathName, FileResName,

"FilerResName:FilerIPAddress", FilerOptions,MountPathName, MountOptions,"FilerResName:FilerControlPath", MonitorOnly,"FilerResName:State" }

NameRule = NetAppExport_ + resource.FilerPathNamestr FilerPathNamestr FilerResNamestr FilerOptionsstr MountPathNamestr MountOptionsstatic int NumThreads = 1static int MonitorTimeout = 200static int OnlineTimeout = 120static int OfflineTimeout = 120

)

Sample Configuration for NetAppExport Resource in main.cf

NetAppExport netapp_export (FilerResName = netapp_filerFilerPathName = "/vol/export1"FilerOptions @SysA = "access=SysA,root=SysA"FilerOptions @SysB = "access=SysB,root=SysB"MountPathName = "/export1"MountOptions = "rw"

)

10 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 19: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the NetApp Agent with Cluster Manager

Configuring the NetApp Agent with Cluster ManagerA template for the NetApp resource groups has been automatically installed when youinstalled the NetApp storage agent. Using the VCS GUI, you can view the template,which displays the NetApp service group, its resources and their attributes. You candynamically modify the attributes’ values as necessary for your configuration.

Importing the NATypes.cf File

To use the Oracle template, import the NATypes.cf file to the VCS engine by usingCluster Manager:

1. Start Cluster Manager.

2. Click on the File menu and select Import Types.

3. In the Import Types dialog box, select the file:

/etc/VRTSvcs/conf/sample_na/NATypes.cf

4. Import the types file for NetApp.

5. Save the configuration.

At this point, the NetApp types have been imported to the VCS engine. The NetAppagent can be configured without interrupting or stopping VCS.

For detailed information about using the VCS Cluster Manager, refer to the chapter“About the VCS GUI” in the VERITAS Cluster Server User’s Guide.

Chapter 1 Installing and Configuring the Agent 11

Page 20: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the NetApp Agent with Cluster Manager

Adding the NetApp Group Using the Wizard

If you have imported the NATypes.cf file, you can download the NAGroup template bystarting the Service Group Configuration Wizard in Cluster Manager.

1. In Cluster Manager, click the service group configuration wizard icon on the toolbar.

2. Click Next on the wizard Welcome window.

a. Enter the service group name. In this instance, enter NetApp.

b. Select the systems on which the service group will run.

3. Confirm that you are basing the service group on a predefined template: click Next.

4. Select the NetApp template (NAGroup) from the list of templates; click Next.

5. The wizard prompts you with alternate names in case the names in the template clashwith existing names; make corrections as necessary and click Next.

6. When the wizard indicates that it is ready to create the service group, click Next; theservice group is dynamically created based on the NetApp template.

Review the service group’s resource attributes, and modify the default values, ifnecessary, in accord with your specific configuration.

12 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 21: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the Agent by Editing the Configuration Files

Configuring the Agent by Editing the Configuration FilesTo configure the NetApp Filer agent, you can edit the sample configuration files installedwith the agent software.

Editing the Configuration Files

1. Copy the NAtypes.cf file into place.

# cp /etc/VRTSvcs/conf/sample_na/NAtypes.cf/etc/VRTSvcs/conf/config/NAtypes.cf

2. Make a backup copy of the main.cf file.

# cd /etc/VRTSvcs/conf/config# cp main.cf main.cf.orig

3. Copy the sample main.cf into place.

# cp /etc/VRTSvcs/conf/sample_na/main.cf/etc/VRTSvcs/conf/config/main.cf

4. Edit the sample main.cf file, using vi or another text editor, to reflect yourconfiguration. Refer to “Sample Configuration of NetAppFiler Resource in main.cf”on page 8, and “Sample Configuration for NetAppExport Resource in main.cf” onpage 10.

5. Verify the syntax of main.cf. Correct any errors indicated in the output.

# hacf -verify /etc/VRTSvcs/conf/config

Verifying the Configuration

1. Start the VCS engine on sysA:

# hastart

2. Verify that the agent brings all NetApp service group resources online:

# hagrp -display

3. Take the service group offline and verify that all resources are stopped:

# hagrp -offline netapp_group -sys sysA# hagrp -display

Chapter 1 Installing and Configuring the Agent 13

Page 22: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Configuring the Agent by Editing the Configuration Files

4. Bring the service group online again and verify that all resources are available:

# hagrp -online netapp_group -sys sysA# hagrp -display

5. Start had on sysB:

# hastart

6. Switch the NetApp service group to sysB:

# hagrp -switch netapp_group -to sysB

7. Verify that all NetApp service group resources are brought online on sysB:

# hagrp -display

14 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 23: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Setting Up to Handle Possible NIC Failures

Setting Up to Handle Possible NIC FailuresIf a NIC fails, a service group containing NetApp resources and a NIC resource can takeapproximately five minutes to fail over. When a NIC faults, NFS mounts on the localsystem hang, preventing the graceful offlining of the resources. To force the service groupto fail over in the event of a NIC failure, the system must be rebooted.

Recommendations for setting up your configuration to handle possible NIC failuresinclude:

◆ Using multiple NICs mapped to the base IP address of each cluster server. The NIC isno longer a single-point of failure.

◆ Using the attributes RebootOption and NicResName of the NetAppFiler resourceto force a failover of the group to another cluster system.

Setting Up Multiple NICs

With multiple NICs on each of the cluster systems, a failure of one NIC does not cause aservice group to fail over to another system. A standby NIC, assigned the same IP addressas the first, takes over. An example section of the main.cf file shows a configurationwhere the systems are sysA and sysB:

. . .IPMultiNic oracle_ip (

Address = "166.98.32.43"MultiNICResName = oracle_nic)

MultiNICA oracle_multinicDevice@sysA = { hme0="166.98.32.12",qfe0="166.98.32.12" }Device@sysB = { hme0="166.98.32.13",qfe0="166.98.32.13" }

). . .

In this example, 166.98.32.12 is the base IP address of sysA, and 166.98.32.13 is thebase IP address of sysB.

Use the sample main.cf.multinic file found in the directory/etc/VRTSvcs/conf/sample_na as a reference.

Chapter 1 Installing and Configuring the Agent 15

Page 24: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Modifying the Agent Configuration

Using the RebootOption of the NetAppFiler Resource

The attributes RebootOption and NicResName of the NetAppFiler resource can be usedto direct the server on which a NIC failure is detected to reboot. The agent kills the VCSprocesses, unconfigures GAB, and reboots the server to force the service group to fault.Rebooting clears stale NFS mounts and forces the service group to fault and fail over toanother system in the cluster.

You can set the RebootOption attribute to a string such as: sync; sync; reboot, oryou can specify the name of a script (see “Using a Shell Script to Shutdown”). TheNicResName attribute should be set to the name of the NIC resource in this group. Thedefault setting for both attributes is a null string. If either of these is not set correctly, aNIC failure prevents the service group from failing over. See the “NetAppFiler ResourceType” on page 7.

Note A typical service group failover, even with the use of the RebootOption, takesapproximately five minutes.

Using a Shell Script to Shutdown

Instead of using a reboot string as a value for the RebootOption attribute, you can use anon-interactive shell script to invoke a systematic shutdown procedure. For example:

RebootOption = "/your_path/your_shutdown_script"

Note The shutdown procedure must not take longer than 120 seconds. If it does, increasethe OfflineTimout attribute’s value for the NetAppFiler resource.

If for some reason the system fails to reboot even with the option set, you can force theservice group to fail over. First issue the command:

# /sbin/gabsync

Then, reboot the server.

Modifying the Agent ConfigurationYou can dynamically configure the VCS storage agent for NetApp from the commandline. See the chapter on reconfiguring VCS from the command line in the VERITAS ClusterServer User’s Guide.

16 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 25: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Setting Up Oracle to Run with the NetApp Filer in a VCS Cluster

Setting Up Oracle to Run with the NetApp Filer in a VCS ClusterFor instructions on how to install the VCS enterprise agent for Oracle, refer to theVERITAS Cluster Server Enterprise Agent for Oracle Installation and Configuration Guide.

Note You must install the Oracle binaries on the local disk. You must install the datafilesand control files and redo logs on the NetApp filer.

You can setup Oracle to work with the VCS NetApp Filer agent in two basicconfigurations: with a single filer or with a cluster filer.

In either case, the following information applies:

◆ Install the Oracle system files, ORACLE_HOME, on the local disk of each system inthe VCS cluster to run Oracle and NetApp. Install the datafiles, the control files, andthe redo logs on the filer.

◆ Install the datafiles on a different volume from the control files and redo logs; higheravailability results.

◆ On each system, make sure the file that specifies the location of the control files andredo logs is the same on each of the cluster systems. Typically, that file isinit<ORACLE_SID>.ora and it can be found where Oracle system files areinstalled, for example:

/u01/app/oracle/product/8.1.6/dbs/init<ORACLE_SID>.ora

◆ On each system in the cluster, use the same set of mount points for the NFS mountedvolumes on the filer.

Running Oracle with a Single NetApp Filer in a VCS ClusterIn the following illustrated example with a two-system cluster, Oracle uses a singleNetApp filer. The VCS agent for NetAppFiler monitors the filer while the NetAppExportagent monitors the volumes /vol/oracle_data and /vol/oracle_ctl on the filerand the NFS mount points /u02 and /u03 on the servers. The Oracle system files areinstalled on the local disks of each system, at mount point /u01.

VCS manages volumes /vol/oracle_data and /vol/oracle_ctl entirely. If sysAfails, the volumes /vol/oracle_data and /vol/oracle_ctl are exclusivelyreexported to sysB and are mounted on /u02 and /u03, respectively, of sysB.

Refer to /etc/VRTSvcs/conf/sample_na/main.cf.oracle_on_single_filer,the sample file for this configuration. The diagram of dependencies follows the two-system cluster illustration.

Chapter 1 Installing and Configuring the Agent 17

Page 26: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Running Oracle with a Single NetApp Filer in a VCS Cluster

Running Oracle with a Single NetApp Filer in a VCS Cluster

Dependency Tree, Oracle with Single NetApp Filer Configuration

Public Network

hb1

Local Disk

hb2

Filer

sysBsysA

Filer Disk/Volume

/vol/vol0

/vol/oracle_data

/vol/oracle_ctl

/u01(Oracle_home)/u02/u03

hme0

hme0

Local Disk

/u01(Oracle_home)/u02/u03

NIC

IP

NetAppExportNetAppExport

Oracle

oracle_lsn

NetAppFiler

SqlNet

oracle_home

oracle_exp_data

oracle_filer

oracle_exp_ctl

oracle_NIC

oracle_IP

18 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 27: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Running Oracle with a NetApp Cluster Filer in a VCS Cluster

Running Oracle with a NetApp Cluster Filer in a VCS ClusterIn a cluster filer environment, it is possible to spread a single database instance over thetwo filers. In such a setup, the datafiles are installed in a single volume of one filer and thecontrol files and the redo logs are installed on a volume of the partner/takeover filer. Onceagain, as when running Oracle in a single NetApp filer, you must install the Oracle systemfiles on the local disks and ensure the init<ORACLE_SID>.ora file is the same on eachof the cluster systems.

Refer to /etc/VRTSvcs/conf/sample_na/main.cf.oracle_on_cluster_filer,the sample file for this configuration. The diagram of dependencies follows.

Running Oracle with a NetApp Cluster in a VCS Cluster

Public Network

hb1

hb2

Filer 2

sysBsysA

Filer Disk/Volumes

/vol/vol0

/vol/oracle_ctl

hm

e0

hm

e0

Filer 1

/vol/vol0

/vol/oracle_data

Local DiskLocal Disk

/u01(Oracle_home)/u02/u03

/u01(Oracle_home)/u02/u03

Chapter 1 Installing and Configuring the Agent 19

Page 28: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Disabling the Agents

Dependency Tree, Oracle with Two NetApp Filers

Disabling the AgentsTo disable the agent on a system, you must first switch the NetApp Filer service groups toan OFFLINE state. You can stop the application completely, or switch the agents to anothersystem.

1. You can determine if the service group is online by entering:

# hagrp -state netapp -sys sysA

2. If the service group is online, take it offline by entering:

# hagrp -switch netapp -to sysB

or:

# hagrp -offline netapp -sys sysA

NIC

IP

NetAppExport

Oracle

SqlNet

oracle_filerNetAppFilerNetAppFiler

oracle_lsn

oracle_IPoracle_home

oracle_exp_data

oracle_filer1

oracle_filer2

oracle_NICoracle_exp_ctl

NetAppExport

20 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 29: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Removing the Agent

3. Stop the NetApp Filer agents on the system:

# haagent -stop NetAppFiler# haagent -stop NetAppExport

When you get the message “Please look for messages in the log file,” checkthe file /var/VRTSvcs/log/engine.log_A for a message confirming the agenthas stopped.

You can also use the ps command to confirm the agent is stopped.

You can now remove the system, service group, or resource type from the VCSconfiguration. See the chapter on reconfiguring VCS from the command line in theVERITAS Cluster Server User’s Guide.

Removing the AgentType the following command on each system to remove the agent. Answer promptsaccordingly:

# pkgrm VRTSvcsna

Requirements for Setting Up Optional VCS Volume on the FilerUsing a dedicated VCS volume on the NetApp filer can provide added data protection. Incase the VCS private heartbeat network fails, for example, the VCS volume would stillcontrol access to the filer volumes. Without the VCS volume, a heartbeating failure couldlead to data inconsistency because each system in the VCS cluster could simultaneouslyaccess data on the filer.

Note You can make the size of the volume as small as possible. NetApp requires the useof two disks to configure one volume, one for data and one for parity.

To set up the VCS volume on the filer:

1. Create a volume on the filer dedicated to VCS; for example, /vol/volvcs.

2. Edit the file etc/exports to export the volume with proper permissions for eachsystem in the cluster. For example, if the systems in the VCS cluster are sysA and sysB,the line in etc/exports would be:

/vol/volvcs -access=sysA:sysB,root=sysA:sysB

Chapter 1 Installing and Configuring the Agent 21

Page 30: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Suggestions for Testing Your Configuration

Suggestions for Testing Your ConfigurationAfter successfully configuring your service group and manually failing over the group,you can perform some of the following tests to simulate failure and test failover.

Testing the Configuration From Cluster Systems

✔ Kill Oracle processes on sysA. The group should fail over to sysB.

✔ Kill the VCS engine (kill the had process) on one of the systems. The group does notfail over to the other system.

✔ Power off one of the systems and initiate a group failover.

✔ Remove the public network cable from one of the systems to test for group failover.Wait approximately five minutes to allow a group failover to complete.

✔ Remove both private network connections to simulate a “split-brain” condition.Typically, the group tries to go online on both systems. Because the NetAppExportresource prevents a concurrency violation, the group faults on one system and goesonline on the other, or faults on both systems, depending on the resources in thegroup.

Testing the Configuration on the Filer

✔ Power off both filers. The service group should be faulted on both systems.

✔ Remove the public network connection to the filers. The service group faults on bothsystems.

✔ Performing the following two tests on the filer may cause an application fault tooccur. However, VCS guarantees that service is available by failing the group over toanother system:

◆ Reboot a NetApp filer by issuing the command, for example:

# rsh -n filer_IPaddress reboot

If the Oracle resource faults, the service group fails over to another system. If theresource does not fault, the service group remains online.

◆ If you have configured a cluster filer, power off one of the filers. The backup filertakes over. If the Oracle resource faults, the service group fails over to anothersystem; if the resource does not fault, the service group remains online.

These tests can cause the application resource to fault and VCS to fail over. See“Rebooting the Filer Could Cause Application to Fault” on page 25.

22 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 31: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Troubleshooting Notes

Troubleshooting Notes

NetApp Filer etc/exports File Must Not Contain VCS-Monitored Volumes

The VCS agent for the NetApp requires that the etc/exports file on the filer have noentries for the volumes that the VCS agent is to monitor. The group does not come onlineotherwise.

Export Options Must be Localized for Each System

In the VCS configuration file, main.cf, the attribute FilerOptions of theNetAppExport resource must specify values for each system that give exclusive access tothat system. The table in “NetAppExport Resource Type” on page 9 and the example in“Sample Configuration for NetAppExport Resource in main.cf” on page 10 containdetails.

When specifying export options, do not use an option that permits universal access, suchas "anon=root".

Export Options: Order of Hostnames in etc/hosts File is Significant

The hostname used in the FilerOptions field of the NetAppExport resource must bethe same as the first name specified after the IP address in the filer’s etc/hosts file.

If the entry in etc/hosts file on the filer is:

166.98.32.12 sysA.veritas.com sysA

The corresponding line for the FilerOptions attribute in the main.cf file wouldresemble:

FilerOptions @sysA ="access=sysA.veritas.com,root=sysA.veritas.com"

Whereas, if the entry in the etc/hosts file on the filer shows the nickname before thesystem name, as in:

166.98.32.12 sysA sysA.veritas.com

The corresponding line for the FilerOptions attribute in the main.cf file wouldresemble:

FilerOptions @sysA ="access=sysA,root=sysA"

Chapter 1 Installing and Configuring the Agent 23

Page 32: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Troubleshooting Notes

NetApp Filer etc/hosts.equiv Must List Root for Each System

Make sure that the filer allows rsh access for every system in the cluster. Modify the fileetc/hosts.equiv on the filer so that it contains each of the system names.

Oracle Binaries Must Be Installed on Local Disk

You must install the Oracle binaries on the local disk. You must install the datafiles andcontrol files and redo logs on the NetApp filer.

NIC Failures Take Approximately Five Minutes to Fail Over

If you are using a NetApp-Oracle setup, you may have to change the Oracle resource’sOfflineTimeout attribute to a value of 120 seconds from the default value to enablefaster failover. Using the MultiNICA agent to deal with NIC failures is recommended.

Shutdown Process Specified by RebootOption Must be Under 120 Seconds

When you use the RebootOption, the shutdown procedure must not take longer than120 seconds. This holds true whether the option specifies a command or a proceduralscript. If the shutdown procedure take longer than 120 seconds, you must increase thevalue of the OfflineTimeout attribute of the NetAppFiler resource.

VCS Volume on Filer Requires Entry in etc/exports

If you have elected to use the optional FilerControlPath attribute of the NetAppFilerresource, the volume on the NetApp filer that VCS uses exclusively requires an entry inthe filer’s etc/exports file.

For example:

/vol/volVCS -access=sysA:sysB, root=sysA:sysB

Mount Options on the Host

VCS mounts all file systems on the host with the llock option, which enables locks to beheld locally on the system. If this option is not enabled, any application that uses NFSlocks cannot fail over if the system shuts down or loses power. To see if the llock optionis enabled, type the command:

# mount

The following example output shows the llock option enabled:

/u03 on 10.129.64.75:/vol/vol1/u read/write/setuid/llock/remote onMon Jun 12 11:33:26 2000

Note that llock is an undocumented feature of the Solaris operating system.

24 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide

Page 33: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Troubleshooting Notes

Rebooting the Filer Could Cause Application to Fault

Because VCS manages the export of volumes it monitors on the filer, a reboot of the filer ora failover of a cluster filer causes VCS to reexport those volumes. During the time ofreexporting, it’s possible that applications can access the filer but not the volumes. Thiscan cause the application to fault and VCS to fail over. It is therefore recommended thatyou do not manually cause failover or reboot the filer while applications under VCScontrol are likely to access the filer.

“Broken Pipe” Messages in Engine Log

Intermittently, messages containing “Broken Pipe” may appear in the engine log. You candisregard these messages.

Chapter 1 Installing and Configuring the Agent 25

Page 34: VERITAS Cluster Server Storage Agent for NetApp …eval.veritas.com/downloads/pro/netapp_agent.pdf · October 2000 30-000073-399 VERITAS Cluster Server™ Storage Agent for NetApp™

Troubleshooting Notes

26 VCS Storage Agent for NetApp and Oracle Installation and Configuration Guide


Recommended