EMC SRDF/Cluster Enabler for MSCS SRDF/Cluster Enabler for MSCS Version 2.1 Product ... Uninstalling...

EMC SRDF/Cluster Enabler for MSCSVersion 2.1

PRODUCT GUIDEP/N 300-001-286

REV A02

EMC CorporationCorporate Headquarters:

Hopkinton, MA 01748-9103

1-508-435-1000www.EMC.com

ii

Copyright © 2001–2006 EMC Corporation. All rights reserved.Published May, 2006

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

All other trademarks used herein are the property of their respective owners.

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide

Contents

Preface............................................................................................................................ xv

Chapter 1 IntroductionSRDF/CE overview......................................................................... 1-2

SRDF/CE components............................................................. 1-3What is SRDF?........................................................................... 1-3

Main features in SRDF/CE............................................................. 1-6SRDF/CE Configuration Utility GUI .................................... 1-6SRDF/CE Installation Wizard ................................................ 1-7SRDF/CE Configuration Wizard ........................................... 1-8Majority Node Set support...................................................... 1-8Forced Failover.......................................................................... 1-8SRDF/Asynchronous compatibility ...................................... 1-9SRDF/CE SRDF swap support............................................. 1-10SRDF/CE configuration with multiple remote adapters . 1-11Pushing or pulling a configuration to a node..................... 1-11Replacing the remote storage array ..................................... 1-12Autoconfiguration .................................................................. 1-12Delay failback.......................................................................... 1-13Logging enhancements .......................................................... 1-15

SRDF/CE: A geographic cluster system .................................... 1-16SRDF/CE modes of operation .............................................. 1-18Pre-SRDF/CE clustering considerations............................. 1-21

Microsoft Cluster Service overview ............................................ 1-23MSCS cluster concepts ........................................................... 1-25MSCS modes of operation ..................................................... 1-26

Application software in a cluster environment ......................... 1-28

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide iii

Contents

Chapter 2 SRDF/CE and Cluster BehaviorSRDF/CE failover operation.......................................................... 2-2

SRDF/CE failover and recovery behavior with MSCS....... 2-4SRDF/CE unique behavior ..................................................... 2-5Complete site failure and recovery........................................ 2-6

Response to complete site failure .................................................. 2-8

Chapter 3 Automatic Installation and ConfigurationSystem requirements ....................................................................... 3-2

Hardware requirements .......................................................... 3-2Software requirements............................................................. 3-3SRDF/CE installation and operating requirements............ 3-6

Install considerations ...................................................................... 3-7Prerequisites .............................................................................. 3-7Before you begin ....................................................................... 3-8Planning requirements........................................................... 3-12

SRDF/CE Installation Wizard ..................................................... 3-13SRDF/CE Configuration Wizard ................................................ 3-20

Configuring SRDF/CE and MSCS for the first time ......... 3-21Converting an MSCS cluster................................................. 3-65Adding an SRDF/CE node to an existing cluster.............. 3-72

Uninstalling SRDF/CE ................................................................. 3-75

Chapter 4 Upgrading SRDF/CEGetting started.................................................................................. 4-2

System requirements................................................................ 4-2Prerequisites .............................................................................. 4-2PC emulation software ............................................................ 4-2Verifying your actions ............................................................. 4-2

Upgrading GeoSpan clusters to SRDF/CE.................................. 4-4Upgrading SRDF/CE V2.0.x to V2.1............................................. 4-8

Chapter 5 SRDF/CE Configuration Utility GUIConfiguration Utility introduction................................................ 5-2SRDF/CE for MSCS icon options.................................................. 5-3

Change User Credentials......................................................... 5-3Configuration Wizard.............................................................. 5-4Discover ..................................................................................... 5-4

Control component.......................................................................... 5-6Quorum Control ....................................................................... 5-7

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guideiv

Contents

Site Mode Operations............................................................ 5-13Advanced Controls ................................................................ 5-15Logging Control ..................................................................... 5-18Logging enhancements ......................................................... 5-21Forced Failover....................................................................... 5-27

Groups component ....................................................................... 5-31Groups right-click menu options......................................... 5-34Device group right-click menu options .............................. 5-39

Storage component ....................................................................... 5-42Nodes component ......................................................................... 5-52

Add Cluster Member Node.................................................. 5-53Update Lateral/Peer Designations...................................... 5-55

Chapter 6 AdministrationOverview .......................................................................................... 6-2

SRDF/CE Configuration Utility ............................................ 6-3Using Cluster Administrator and the Configuration Utility ............................................................... 6-3Multicluster configurations .................................................... 6-3

SRDF/CE unique cluster behavior ............................................... 6-4SRDF/CE administration............................................................... 6-5

Stopping the Cluster Service .................................................. 6-5Uninstalling the Cluster Service on a Windows Server 2003 cluster......................................................................................... 6-5Manually starting the Cluster Server Services..................... 6-6Modifying cluster groups ....................................................... 6-7Renaming groups..................................................................... 6-7Configuring additional disk devices..................................... 6-8

Restore/recovery operations....................................................... 6-10Restoring from site failure .................................................... 6-10Recovery from site failure..................................................... 6-12Recovery from total IP communications failure................ 6-13Recovery from HBA or SCSI channel failure..................... 6-14Recovery from an SRDF link failure ................................... 6-14Recovery from a corrupt quorum log ................................. 6-15Symmetrix replacement from the R1 side .......................... 6-15

Upgrade procedures ..................................................................... 6-17Upgrading Windows 2000 to Windows Server 2003........ 6-17Upgrading a quorum disk .................................................... 6-20

Configuration troubleshooting.................................................... 6-22Overview................................................................................. 6-22

vEMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide

Contents

SRDF/CE group in wrong state (not ready/invalid/suspended)................................................... 6-23Unable to start cluster service............................................... 6-24GeoSpan_res.dll prevents MSCS group failover ............... 6-25GeoSpan_res.dll stays offline after failback of the quorum group ........................................................................................ 6-25External SYMAPI locks.......................................................... 6-25Unable to see a newly added disk ....................................... 6-26

Errors and warnings...................................................................... 6-28GeoSpan_res.dll errors .......................................................... 6-28SRDF warning......................................................................... 6-29IP communication warning................................................... 6-29Error while attempting to connect with MSCS .................. 6-30MSCS error 170 ....................................................................... 6-30

Best practices .................................................................................. 6-31How to delay loading of MSCS service (change service dependencies) ......................................................................... 6-31Enabling groups to come online while SyncInProg .......... 6-33Changing the quorum log size ............................................. 6-33

URLs to additional information .................................................. 6-35

Chapter 7 Manual Installation and ConfigurationSystem requirements ....................................................................... 7-2Install considerations ...................................................................... 7-2

Prerequisites .............................................................................. 7-3PC emulation software ............................................................ 7-3Rebooting................................................................................... 7-3Verifying your actions ............................................................. 7-3

SRDF/CE manual installation procedure .................................... 7-5Overview ................................................................................... 7-5Preinstallation ........................................................................... 7-5Manually installing SRDF/CE................................................ 7-6

Uninstalling SRDF/CE ................................................................. 7-30

Appendix A SRDF/CE Alerts and ErrorsSRDF/CE alerts .............................................................................. A-2SRDF/CE errors and events ......................................................... A-4

Interpreting Windows event log information ..................... A-4Event messages .............................................................................. A-5

System event log...................................................................... A-5Application event log messaging.......................................... A-6

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guidevi

Contents

Appendix B Preinstallation WorksheetPreinstallation worksheet .............................................................. B-2

Glossary........................................................................................................................ g-1

Index ............................................................................................................................... i-1

viiEMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide

Contents

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guideviii

Figures

Figures1-1 Basic SRDF configuration ............................................................................. 1-51-2 SRDF/CE COM server interfaces with SYMAPI and MMC snap-ins ... 1-71-3 Lateral and peer nodes in SRDF backup operations .............................. 1-141-4 A geographically distributed two-node SRDF/CE cluster ................... 1-161-5 A geographically distributed four-node SRDF/CE cluster .................. 1-171-6 Multiple SRDF/CE clusters supported on an SRDF link ...................... 1-181-7 Two-node two-cluster SRDF/CE configuration ..................................... 1-201-8 Recommended cabling configuration ...................................................... 1-211-9 A typical two-node MSCS cluster ............................................................. 1-241-10 A typical four-node MSCS cluster ............................................................ 1-252-1 SRDF/CE failover operation ....................................................................... 2-32-2 Types of complete site failure ...................................................................... 2-63-1 SRDF/CE installation and Configuration Wizard process flowchart . 3-143-2 Welcome window ....................................................................................... 3-153-3 License Agreement window ...................................................................... 3-163-4 Destination Folder window ....................................................................... 3-163-5 Upgrade Type window .............................................................................. 3-173-6 Ready to Install the Program window ..................................................... 3-183-7 InstallShield Wizard Completed window ............................................... 3-183-8 Reboot dialog box ........................................................................................ 3-193-9 Opening the Configuration Wizard from the Configuration Utility ... 3-213-10 SRDF/CE Configuration Wizard welcome screen ................................. 3-223-11 Overview of phase 1 configuration steps ................................................ 3-253-12 Selecting the configuration mode ............................................................. 3-263-13 Selecting the node configuration .............................................................. 3-273-14 Designating the first node and entering cluster number ...................... 3-283-15 Selecting local Symmetrix array ................................................................ 3-293-16 Selecting remote Symmetrix array ............................................................ 3-293-17 Creating synchronous temporary device groups ................................... 3-31

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide ix

Figures

3-18 Creating asynchronous temporary device groups ................................. 3-313-19 RDF status of devices error message ........................................................ 3-323-20 SRDF/Asynchronous group creation warning ...................................... 3-323-21 Write-enabling devices ............................................................................... 3-333-22 Selecting local IP address ........................................................................... 3-343-23 Running Disk Manager .............................................................................. 3-353-24 Disk Management utility ........................................................................... 3-363-25 Write Signature and Upgrade Disk Wizard ............................................ 3-373-26 Selecting the hostname where first node is located ............................... 3-383-27 Retrieving groups from the first node and write-enabling devices ..... 3-393-28 RDF State of Devices message ................................................................... 3-393-29 Selecting local IP address for an intermediate or last node .................. 3-403-30 Running Disk Manager on an intermediate or last node ...................... 3-413-31 Overview of phase 2 configuration steps for first node ........................ 3-443-32 Selecting the configuration mode in phase 2 .......................................... 3-453-33 Adding a cluster member node ................................................................ 3-463-34 Selecting the cluster type ........................................................................... 3-473-35 Assigning quorum control settings .......................................................... 3-493-36 Assigning site failure mode and advanced control settings ................. 3-503-37 End of phase 2 for node 1 ........................................................................... 3-523-38 Phase 2 configuration steps for intermediate or last node .................... 3-533-39 Selecting the configuration mode in phase 2 .......................................... 3-543-40 Retrieving phase 2 options from node 1 .................................................. 3-553-41 Viewing cluster type ................................................................................... 3-563-42 Viewing quorum settings ........................................................................... 3-573-43 Identifying if node is last node .................................................................. 3-583-44 Overview of phase 3 configuration steps ................................................ 3-593-45 Installing MSCS ........................................................................................... 3-603-46 Post-MSCS configuration activities .......................................................... 3-623-47 End of phase 3 configuration for the last SRDF/CE node .................... 3-633-48 Applying MSCS information to SRDF/CE groups ................................ 3-693-49 Assigning site mode failure and advanced control settings ................. 3-703-50 Retrieving group and cluster information from the first node ............ 3-744-1 Creating a checkpoint in the GeoSpan Configuration Utility ................ 4-54-2 Selecting discover in the SRDF/CE Configuration Utility ..................... 4-64-3 Restoring the checkpoint in the SRDF/CE Configuration Utility ......... 4-65-1 SRDF/CE Configuration Utility GUI ......................................................... 5-25-2 SRDF/CE for MSCS icon right-click menu ............................................... 5-35-3 Changing user credentials ........................................................................... 5-45-4 Issuing a discover through the Configuration Utility GUI ..................... 5-55-5 Issuing a manual SYMCLI discover command ........................................ 5-55-6 Expanded control view in the SRDF/CE Configuration Utility ............ 5-65-7 Control right-click menu .............................................................................. 5-7

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guidex

Figures

5-8 Quorum Control expanded view ............................................................... 5-85-9 Quorum Control expanded view (continued) .......................................... 5-85-10 Selecting edit Quorum Control settings .................................................. 5-105-11 Changing quorum properties ................................................................... 5-115-12 Pushing a new quorum group definition to MSCS ............................... 5-125-13 Quorum Control right-click menu ........................................................... 5-135-14 Selecting the quorum type ........................................................................ 5-135-15 Expanded site mode operations view and right-click menu ............... 5-145-16 Changing site mode operations parameters ........................................... 5-155-17 Expanded advanced control view and right-click menu ...................... 5-155-18 Changing Advanced Control parameters ............................................... 5-175-19 Expanded SRDF/CE service logging view ............................................. 5-185-20 Editing log level parameters ..................................................................... 5-195-21 Editing log file path .................................................................................... 5-195-22 Editing the log file path ............................................................................. 5-205-23 Viewing the latest log file .......................................................................... 5-205-24 SRDF/CE Log Viewer ................................................................................ 5-215-25 Expanded Forced Failover view and right-click menu ......................... 5-295-26 Enabling the Forced Failover option ....................................................... 5-305-27 Expanded groups view with the device groups list .............................. 5-315-28 Expanded groups view with the device groups list (continued) ........ 5-315-29 Device groups detail view ......................................................................... 5-325-30 Device groups detail view (continued) ................................................... 5-325-31 Groups right-click menu ............................................................................ 5-345-32 Creating a new device group .................................................................... 5-355-33 Creating an SRDF/Asynchronous-enabled device group ................... 5-365-34 SRDF/Asynchronous group creation warning ...................................... 5-365-35 Async devices in consistent state warning ............................................. 5-375-36 Pulling groups from SYMCLI ................................................................... 5-385-37 Listing device groups ................................................................................. 5-385-38 Device group right-click menu ................................................................. 5-395-39 Release Symmetrix locks warning ........................................................... 5-395-40 Array Operations dialog box .................................................................... 5-405-41 Expanded storage view ............................................................................. 5-425-42 Symmetrix device detail view ................................................................... 5-435-43 Symmetrix device detail view (continued) ............................................. 5-435-44 Selecting columns to view ......................................................................... 5-455-45 Modifying R1/R2 device columns ........................................................... 5-465-46 Adding and removing R1/R2 device rows ............................................ 5-465-47 Storage right-click menu ............................................................................ 5-485-48 Listing Symmetrix information ................................................................ 5-485-49 Unlocking Symmetrix arrays .................................................................... 5-495-50 Selecting the remote Symmetrix array .................................................... 5-50

xiEMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide

Figures

5-51 Configuring the remote storage array ...................................................... 5-505-52 Completing the replacement of the remote storage array .................... 5-505-53 Expanded node view .................................................................................. 5-525-54 Nodes right-click menu .............................................................................. 5-535-55 Adding a host node .................................................................................... 5-535-56 Individual peer node right-click menu .................................................... 5-546-1 Windows Server 2003 Enterprise Edition Setup dialog box ................. 6-18A-1 SRDF/CE alerts display .............................................................................. A-2

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guidexii

Tables

Tables5-1 Quorum Control column headings ............................................................ 5-95-2 Groups column headings ........................................................................... 5-335-3 R1/R2 device description column headings ........................................... 5-447-1 Manual installation: Two-node installation procedure ........................... 7-7A-1 System event log descriptions ................................................................... A-5A-2 Application event log descriptions ........................................................... A-7

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide xiii

Tables

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guidexiv

Preface

As part of an effort to improve and enhance the performance and capabilities of its product line, EMC from time to time releases revisions of its hardware and software. Therefore, some functions described in this document may not be supported by all revisions of the software or hardware currently in use. For the most up-to-date information on product features, refer to your product release notes.

If a product does not function properly or does not function as described in this document, please contact your EMC representative.

Audience This guide is part of the EMC SRDF/CE for MSCS (referred to herein as SRDF/CE) documentation set and is intended for use by system administrators during installation, system setup, and routine operations.

System administrators working with SRDF/CE must be proficient in the use of the following products:

◆ Microsoft products:

• Windows Server 2003 Enterprise and Datacenter Editions

• Microsoft Cluster Services (MSCS)

◆ EMC products:

• EMC Solutions Enabler (SYMCLI/SYMAPI)

• EMC ControlCenter Symmetrix Remote Data Facility (SRDF) Manager, if installed

• EMC PowerPath, if installed

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide xv

xvi

Preface

Organization Here is an overview of where information is located in this guide:

Chapter 1, “Introduction,” introduces clustering concepts that pertain to Microsoft Cluster Services (MSCS), Symmetrix Remote Data Facility (SRDF), and SRDF/CE applications.

Chapter 2, “SRDF/CE and Cluster Behavior,” describes SRDF/CE behavior in various modes of operation.

Chapter 3, “Automatic Installation and Configuration,” provides instructions on and lists the requirements for automatically installing and configuring the SRDF/CE software using the SRDF/CE Installation Wizard and the SRDF/CE Configuration Wizard.

Chapter 4, “Upgrading SRDF/CE,” describes the procedures for upgrading a GeoSpan cluster to an SRDF/CE V2.1 cluster and upgrading an SRDF/CE V2.0 cluster to an SRDF/CE V2.1 cluster.

Chapter 5, “SRDF/CE Configuration Utility GUI,” describes the functionality of the SRDF/CE Configuration Utility graphical user interface (GUI).

Chapter 6, “Administration,” lists administrative procedures, such as routine operations, maintenance, and troubleshooting of SRDF/CE clusters.

Chapter 7, “Manual Installation and Configuration,” provides detailed instructions on and lists the requirements for manually installing and configuring SRDF/CE software.

Appendix A, “SRDF/CE Alerts and Errors,” describes the alerts, event messages, and errors you may encounter while using SRDF/CE.

Appendix B, “Preinstallation Worksheet,” contains a sample worksheet to be used with an EMC technical representative. This worksheet creates a snapshot of the environment that will comprise the SRDF/CE cluster.

The “Glossary” provides terminology and general background information for database administration.

Relateddocumentation

The following documentation from EMC Corporation contains information that may be helpful in an SRDF/CE clustered environment:

EMC ControlCenter

◆ EMC ControlCenter Installation Planning Booklet ◆ EMC ControlCenter Guidebook


Preface

◆ EMC ControlCenter Installation Guide ◆ Symmetrix SRDF Host Component Product Guide

EMC Solutions Enabler

◆ EMC Solutions Enabler Symmetrix Array Management CLI Product Guide

◆ EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide◆ EMC Solutions Enabler Symmetrix TimeFinder Family CLI Product

Guide◆ EMC Solutions Enabler Installation Guide

EMC PowerPath

◆ EMC PowerPath Version Product Guide◆ EMC PowerPath Version Product Overview white paper

Fibre Channel

◆ Symmetrix Fibre Channel Product Guide

Related third-partydocumentation

The following Microsoft documentation contains information about or related to the products discussed in this guide:

◆ Microsoft’s Cluster Server documentation set, especially the Microsoft Cluster Server Administrator’s Guide, Document No. X0327902

◆ Step-by-Step Guide to Installing Cluster Service, Microsoft Technical Library

◆ Introducing Windows 2000 Clustering Technologies, Microsoft Technical Library

Typographicalconventions

The following type style conventions are used in this guide:

bold • User actions (what the user clicks, presses, or selects)• Interface elements (button names, dialog box names)• Names of keys, commands, programs, scripts, applications,

utilities, processes, notifications, system calls, services, applications, and utilities in text

italic • Book titles• New terms in text• Emphasis in text

Courier • Prompts • System output • Filenames • URLs • Syntax when shown in command line or other examples

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide xvii

xviii

Preface

Where to get help EMC support, product, and licensing information can be obtained as follows.

Product information — For documentation, release notes, software updates, or for information about EMC products, licensing, and service, go to the EMC Powerlink website (registration required) at:

http://Powerlink.EMC.com

Technical support — For technical support, go to EMC WebSupport on Powerlink. To open a case on EMC WebSupport, you must be a WebSupport customer. Information about your site configuration and the circumstances under which the problem occurred is required.

Your commentsYour suggestions will help us continue to improve the accuracy, organization, and overall quality of the user publications. Please send your opinion of this guide to:

[email protected]

Courier, bold • User entry• Options in command-line syntax

Courier italic • Arguments in examples of command-line syntax• Variables in examples of screen or file output• Variables in pathnames


http://powerlink.emc.com

1Invisible Body Tag

This chapter provides a high-level overview of clustering, Microsoft Cluster Services (MSCS), Symmetrix Remote Data Facility (SRDF), and explains how EMC SRDF/CE V2.1 for MSCS provides disaster recovery protection in geographically distributed clusters.

Important: EMC strongly recommends reading this chapter in its entirety before installing and configuring SRDF/CE V2.1 for MSCS.

◆ SRDF/CE overview ...........................................................................1-2◆ Main features in SRDF/CE...............................................................1-6◆ SRDF/CE: A geographic cluster system.......................................1-16◆ Microsoft Cluster Service overview ..............................................1-23◆ Application software in a cluster environment ...........................1-28

Introduction

Introduction 1-1

1-2

Introduction

SRDF/CE overviewEMC® SRDF®/Cluster Enabler (CE) for Microsoft Cluster Service (MSCS) is a software extension (MMC snap-in) for MSCS. EMC SRDF/CE V2.1 for MSCS (referred to herein as SRDF/CE) allows Windows Server 2003 Enterprise and Datacenter editions running MSCS to operate across a single pair of SRDF1-connected Symmetrix® arrays in geographically distributed clusters. These clusters are referred to as SRDF/CE clusters. SRDF/CE software can extend the Symmetrix enterprise system to support up to 64 MSCS clusters per Symmetrix pair.

Using an SRDF link, SRDF/CE expands the range of cluster storage and management capabilities while ensuring full business continuance protection. A SCSI or Fibre Channel connection from each cluster node is made to its own Symmetrix array. Two Symmetrix arrays are connected through SRDF to provide automatic failover of SRDF-mirrored volumes during MSCS node failover. This connection effectively extends the distance between cluster nodes up to 60 kilometers and forms a geographically distributed cluster with disaster-tolerant capabilities.

SRDF/CE protects data from the following types of failures, 24x7x365 days per year:

◆ Storage failures◆ System failures◆ Site failures

1. Symmetrix Remote Data Facility (SRDF). For more information on SRDF, refer to “What is SRDF?” on page 1-3.


Introduction

SRDF/CE componentsSRDF/CE integrates MSCS software with EMC Symmetrix hardware and SYMAPI software, allowing the seamless use of Symmetrix disks to function as a single SCSI disk. SRDF/CE achieves this using several components:

◆ SRDF/CE Configuration Utility — An MMC-based (Microsoft Management Console) user interface that allows you to:

• Configure operational parameters• Monitor the SRDF/CE Service• Modify logging settings

◆ COM Server — Component that handles all direct interfaces to the SYMAPI from the MMC snap-in and all intra-SRDF communication.

◆ SRDF/CE Service — The main control service mechanism.

◆ SRDF/CE Resource.dll — A .dll used by MSCS to perform group failover/failback operations for all group resources.

◆ Quorum Filter Driver — Component that performs arbitration or ownership protocol for the MSCS database, the quorum.

What is SRDF? SRDF is a Symmetrix-based business continuance and disaster recovery solution sold as a separate license by EMC Corporation. In basic terms, SRDF is a configuration of multiple Symmetrix arrays whose purpose is to maintain multiple, realtime copies of logical volume data in more than one location.

SRDF duplicates production (source) site data to a recovery (target) site transparently to users, applications, databases, and host processors. If the primary site is not able to continue processing, data at the secondary site is current up to the last I/O transaction.

SRDF can be used in several key areas including, but not limited to:

◆ Disaster Recovery◆ Remote Backup◆ Data Center Migration◆ SDMS—Symmetrix Data Migration Service◆ Data Center Decision Solutions

SRDF/CE overview 1-3

1-4

Introduction

When primary (source) systems are down, SRDF enables fast switchover to the recovery (target) copy of the data, allowing critical information to become available in minutes. Business operations and related applications may resume full functionality with minimal interruption.

Protecting against data loss allows the operations and applications to resume at the secondary site. SRDF can be used:

◆ By itself, and data processing can be resumed by powering up a standby system and manually restarting.

◆ In combination with more sophisticated software to automatically resume operations.

Figure 1-1 on page 1-5 illustrates a basic SRDF configuration.

SRDF/CE combines MSCS and SRDF to provide a more sophisticated solution. SRDF/CE provides both a manual configuration utility (refer to Chapter 5, “SRDF/CE Configuration Utility GUI”) and an automated configuration wizard (refer to Chapter 3, “Automatic Installation and Configuration”) to be used in conjunction with the MSCS Cluster Administrator to administer the SRDF-enabled cluster.

Note: For greater detail on SRDF, consult the SRDF documentation set (refer to “Related documentation” in the Preface).


Introduction

Figure 1-1 Basic SRDF configuration

Site A(production)

Site B(recovery)

LocalHost

RemoteHost

SRDF Links

SymmetrixA

SymmetrixB

RecoveryPath

Active HostPath

Local volume Source volume Target volume

SRDF/CE overview 1-5

1-6

Introduction

Main features in SRDF/CEThe following are the main features of SRDF/CE:

◆ “SRDF/CE Configuration Utility GUI”◆ “SRDF/CE Installation Wizard” on page 1-7◆ “SRDF/CE Configuration Wizard” on page 1-8◆ “Majority Node Set support” on page 1-8◆ “Forced Failover” on page 1-8◆ “SRDF/Asynchronous compatibility” on page 1-9◆ “SRDF/CE SRDF swap support” on page 1-10◆ “SRDF/CE configuration with multiple remote adapters” on

page 1-11◆ “Pushing or pulling a configuration to a node” on page 1-11◆ “Replacing the remote storage array” on page 1-12◆ “Autoconfiguration” on page 1-12◆ “Delay failback” on page 1-13◆ “Logging enhancements” on page 1-15

SRDF/CE Configuration Utility GUIThe SRDF/CE Configuration Utility GUI is an MMC snap-in. This MMC snap-in GUI consists of two main components: the snap-in itself and a COM Server.

The SRDF/CE COM Server functions as the interface between SYMAPI and the MMC snap-ins for information and control requests (both local and remote). The COM Server also makes SRDF/CE configuration changes to the Registry. The COM Server performs all other control operations of the Symmetrix array via SYMAPI calls.

Figure 1-2 on page 1-7 shows how both the local and remote SRDF/CE MMC snap-ins and the SRDF/CE Installation Wizard interface to the SRDF/CE COM Server to access information about the Symmetrix array.


Introduction

Figure 1-2 SRDF/CE COM server interfaces with SYMAPI and MMC snap-ins

The MMC snap-in is the presentation component of the SRDF/CE Configuration Utility GUI, and the COM Server is the control and status mechanism for the MMC client. The SRDF/CE COM Server handles actual configuration changes, status information, and control of Symmetrix devices.

Note: For more information on the Configuration Utility, refer to Chapter 5, “SRDF/CE Configuration Utility GUI.”

SRDF/CE Installation WizardThe SRDF/CE Installation Wizard is an InstallShield-based application that copies the SRDF/CE code in a user-designated location and provides a shortcut for the SRDF/CE MMC snap-in. No actual processing or authentication occurs during this process.

The SRDF/CE Installation Wizard should be invoked when:

◆ Installing SRDF/CEor

◆ Upgrading/removing SRDF/CE

SRDF/CE COMServer

SRDF/CE SYMAPI DB file

MMC Console

SRDF/CE MMCSnap-in

Registry

SRDF/CE Configuration

Wizard

MMC Console

Local Host

Remote SRDF/CECOM Server

Remote SRDF/CECOM Server

1 to 7 Remote Nodes

Main features in SRDF/CE 1-7

1-8

Introduction

After running the Installation Wizard, executing the SRDF/CE MMC snap-in will load the SRDF/CE Configuration Utility GUI.

Note: For more information on the Installation Wizard, refer to Chapter 3, “Automatic Installation and Configuration.”

SRDF/CE Configuration WizardThe SRDF/CE Configuration Wizard simplifies the SRDF/CE configuration process by automating the configuration of SRDF/CE and facilitating the installation of MSCS on all nodes in your system. Configuration settings must be applied on a node-by-node basis.

Note: For more information on the SRDF/CE Configuration Wizard, refer to Chapter 3, “Automatic Installation and Configuration.”

Majority Node Set supportMicrosoft Majority Node Set (MNS) provides an alternative quorum resource for clusters with nodes in separate geographic sites. It is useful for clusters with physically separate storage subsystems that implement mirroring to replicate data. MNS enables you to create a server cluster without any shared disks for the quorum resource. SRDF/CE allows you to configure an MNS cluster on Windows Server 2003 Enterprise and Datacenter Editions.

If you are configuring your cluster on Windows Server 2003, the Quorum Control right-click menu in the SRDF/CE Configuration Utility allows you to choose the cluster quorum type you want to configure: a quorum disk-based cluster (Shared Quorum) or a Microsoft Majority Node Set (MNS) cluster.

Forced FailoverThe Forced Failover option is a configurable parameter found under the Control component in the SRDF/CE Configuration Utility. This option provides an automatic way to keep the cluster running on a particular Symmetrix array or arrays in the event of a total site disaster.

The Forced Failover option performs differently depending on whether the MSCS cluster is defined as a quorum disk-based cluster


Introduction

or an MNS-based cluster. For quorum disk-based clusters, the node that owns the quorum decides what nodes will stay up. For MNS-based clusters, you can configure a list of nodes that will be forced up in a site disaster.

Note: MNS is only supported on Windows Server 2003 Enterprise and Datacenter Editions. For more information on the Forced Failover option, refer to “Forced Failover” on page 5-27.

SRDF/Asynchronous compatibilitySRDF/CE is compatible with EMC SRDF/Asynchronous (SRDF/A). SRDF/A is a high-performance, extended-distance asynchronous replication that uses a delta set architecture for reduced bandwidth requirements and no host performance impact.

Asynchronous mode provides a point-in-time image on the target (R2) device that is only slightly behind the source (R1) device. SRDF/A session data is transferred to the remote Symmetrix system in delta sets, eliminating the redundancy of same-track changes being transferred over the link, thereby reducing the required bandwidth. SRDF/A only needs enough bandwidth to support the average production workload versus peak workloads, provided there is enough Symmetrix cache to support the peak workloads.

SRDF/A is intended for users who require no host application impact while maintaining a consistent, restartable image of their data on the R2 side at all times.

Note: For more information on SRDF/A, refer to the EMC white paper SRDF/Asynchronous: A Technical Description available on www.EMC.com.

SRDF/CE supports Enginuity™ (microcode) releases as outlined in the EMC Support Matrix. At the 5x70 Enginuity level, you can specify a single SYMCLI group whose device members have been previously defined as SRDF/A enabled. Once configured, SRDF/CE automatically fails over this group to the target side as necessary. After the failover, when you want to fail back, SRDF/CE automatically establishes a synchronous connection, waits until the data is propagated back, and then changes the SRDF mode back to asynchronous.


Introduction

Note: SRDF/CE does not support clusters where the target (R2) side is larger than the source (R1) side. When the system fails over to the R2 side, it can never fail back since the R2 cannot resynchronize all its data back to the R1 side.

SRDF/CE SRDF swap supportAn R1/R2 personality swap (or R1/R2 swap) refers to swapping the RDF personality of the RDF device designations of a specified device group, so that source R1 device(s) become target R2 device(s) and target R2 device(s) become source R1 device(s).

R1/R2 RDF swaps are available with Enginuity Version 5567 or higher. There are two types of R1/R2 swaps: FastSwap and Dynamic Swap. A FastSwap occurs immediately after failover if the group is fully synchronized. A Dynamic Swap takes longer because after failover, the tracks are checked to determine if they are synchronized, and then the swap occurs. If you enable an R1/R2 swap for a group, SRDF/CE automatically checks during a failover to determine whether FastSwap is available. If FastSwap is available, SRDF/CE will use it. If FastSwap is not supported, SRDF/CE will automatically use Dynamic Swap.

Using the R1/R2 swapThis section describes several scenarios in which it is beneficial to execute an R1/R2 swap.

Symmetrix load balancingIn today's rapidly changing computing environments, it is often necessary to deploy applications and storage on a different Symmetrix array without having to lose disaster protection. R1/R2 swap can enable this redeployment with minimal disruption, while offering the benefit of load balancing across two Symmetrix storage arrays.

For example, if you want to reconfigure an SRDF/CE environment after having decided where the R1 and R2 devices will sit, this procedure will allow you to go from an active/passive configuration to active/active.

1-10 EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide

Introduction

Primary data center relocationSometimes a primary data center needs to be relocated to accommodate business practices. For example, several financial institutions in New York City routinely relocate their primary data center across the Hudson River to New Jersey as part of their disaster drills. R1/R2 swaps allow these customers to run their primary applications in their New Jersey data centers. The Manhattan data centers then acts as the disaster protection site.

Post-failover temporary protection measureYou can regain a measure of protection after failing over to the remote site. If the hosts on the source side are down for maintenance, R1/R2 swap permits the relocation of production computing to the target site without giving up the security of remote data protection. When all problems are solved on the local Symmetrix array, fail over again and swap the personality of the devices to return to the original configuration.

SRDF/CE configuration with multiple remote adaptersSRDF/CE can be configured with multiple RDF links and remote adapter (RA) groups. SRDF/CE not only allows multiple RAs, but periodically tests them to ensure they are functioning. Multiple RA groups are also allowed, and these RA groups do not have to be symmetrical across all RDF links; any one RA group can be allocated over a subset of the defined RDF links.

If a situation occurs where an RDF link goes down, an Event Log message is posted and an entry is placed in the SRDF/CE log. In the case where an RA group can no longer be failed over because there are no RDF links up to support that operation, an SRDF/CE pop-up event in the System Tray is reported as well as an entry in the SRDF/CE log.

Pushing or pulling a configuration to a nodeSRDF/CE allows new or reconfigured nodes to have a valid configuration pushed (that is, propagated) or pulled (that is, retrieved) to that node. This process completely deletes the entire existing SRDF/CE configuration on that target node and replaces it with the configuration the user specifies as valid. This process does not affect the operation of other nodes in the SRDF/CE cluster.


1-12

Introduction

Replacing the remote storage arraySRDF/CE has the automated ability to change the remote storage array in the case of a remote site disaster, lease swap, or other event where the remote storage array must be replaced. This change is done without disruption to the running local cluster nodes. Once the local cluster nodes are reconfigured using this feature, the remote nodes must each have a configuration pushed or pulled to them for them to recognize the new storage array.

These configuration changes require that the source/target Symmetrix device remapping is changed prior to invoking this function. In addition, the node from which the Replace Storage Array function is invoked will be used as the validation node. The validation node is used to validate each local lateral node device configuration and source/target mapping.

AutoconfigurationThe SRDF/CE COM Server allows the SRDF/CE configuration information to be propagated among cluster peers through the Autoconfiguration utility. Proper time synchronization for all host nodes is critical to ensure the proper functioning of Autoconfiguration. EMC recommends all servers in an SRDF/CE cluster be based on GMT and time servers be employed.

When you open the SRDF/CE snap-in, the timestamps of all nodes are validated. If the timestamp of the nodes is older than the current configuration, you will be prompted to update the configuration. You must decide whether the node you are on has the correct configuration. The Autoconfiguration utility then overloads the local configuration parameters with the newer information.

The Autoconfiguration utility specifies the following configuration information about SRDF/CE:

◆ Quorum Control — Quorum Group Name, Quorum Drive Letter (the physical drive is included, but not used by AutoConfig)

◆ Site Mode — Site Mode setting for node

◆ Advanced Information — Cluster Number, Delay Failback, AutoDel Release, and SRDF/A Failover Method settings

◆ Device Groups — Group names and devices


Introduction

Delay failbackThe Delay Failback feature is an option found in the Advanced Control menu and is propagated as part of the Autoconfiguration functionality. This feature modifies the Preferred Owner list for each MSCS cluster group so a failover will occur to a lateral node first, and if the lateral node is unavailable, to a peer node. Lateral nodes are defined as nodes connected to the same Symmetrix array. Peer nodes are defined as nodes connected to different Symmetrix arrays located across the SRDF link from each other (refer to Figure 1-3 on page 1-14).

SRDF/CE manipulates the MSCS Preferred Owners list whenever a group is brought online. SRDF/CE then examines the group Preferred Owners list and determines which node is the lateral node. It can then modify the MSCS Preferred Owner list so the current node and its lateral partner are the first two in the list.

Therefore, no matter which side (R1 or R2) a group is moved to, the Preferred Owner list is modified to allow a group to fail over to a lateral node, and not fail back or fail over across the SRDF link as a first option. MSCS only moves a group across the SRDF link as a last resort. This prevents MSCS from arbitrarily performing what amounts to a failback/failover across the SRDF link in an automatic fashion. This feature delays the actual failback of a group to an R1 node, and is therefore termed delay failback.


1-14

Introduction

Figure 1-3 Lateral and peer nodes in SRDF backup operations

The Delay Failback feature runs simultaneously on all nodes. Therefore, when a group comes online on any node, the Preferred Owner list is updated, regardless of whether it is an R1 or R2 device. For example, the sequence for Group x on Node 1 will be the following:

1. The Delay Failback feature first determines if it knows the other nodes in the four-node cluster. This information is gathered by the SRDF/CE COM Server during normal operations. If not, the feature is bypassed because it is unable to differentiate between a lateral node, peer node, etc.

2. If the Delay Failback feature knows the other nodes, then it determines if Group x has come online on Node 1.

3. If Group x has come online on Node 1, then the MSCS Preferred Owner list is modified so that Node 1 is the first Preferred Owner, followed by the lateral node and then the peer nodes.

SRDF Link

Symmetrix BSymmetrix A

Site BSite A

Lateral Nodes Lateral Nodes

Peer Nodes

Peer Nodes

Primary (R1) Side Secondary (R2) Side

Node 1 Node 2

Node 4Node 3


Introduction

Logging enhancementsThe logging features of SRDF/CE for MSCS Version 2.1 are enhanced to provide more detailed logging. If there is a problem with SRDF/CE for MSCS, detailed logs will provide customer support with the technical information necessary to help diagnose the problem and help SRDF/CE engineers with debugging.

The logging facility now captures much more detail information, and the size and number of log files being created has increased compared to prior versions of SRDF/CE for MSCS.

To help minimize disk capacity utilized by log files, log compression and log retention are two new features added to address these issues.

Refer to “Logging enhancements” on page 5-21 for more details.


1-16

Introduction

SRDF/CE: A geographic cluster systemSRDF/CE provides disaster-tolerant capabilities that enable the cluster servers to be geographically separated by distances of up to 60 kilometers, or where the network connection provides a guaranteed maximum round-trip latency between nodes of no more than 300 milliseconds. Since many servers can connect to one Symmetrix array, it is possible to implement many clusters across this distance.

Figure 1-4 illustrates the hardware configuration of a two-node SRDF/CE cluster solution.

Figure 1-4 A geographically distributed two-node SRDF/CE cluster

Private Interconnect (Heartbeat Connector)

Enterprise LAN/WAN

SymmetrixArray 1

SymmetrixArray 2

Clients

R1 R2

R1R2

BidirectionalSRDF Interconnect

Host ANode 1

Host BNode 2

Ultra Wide SCSIor

Fibre Channel

Ultra Wide SCSIor

Fibre Channel

(Distance of up to 60 km)


Introduction

Figure 1-5 illustrates the hardware configuration of a four-node SRDF/CE cluster solution.

Figure 1-5 A geographically distributed four-node SRDF/CE cluster

SRDF/CE provides disaster-tolerant capabilities by exploiting SRDF capabilities. SRDF allows two Symmetrix arrays to be attached using direct-connect fiber at distances up to 60 kilometers. The SRDF group when built is established in synchronous or asynchronous mode. Once attached, mirrored pairs of disks are established on the devices, with R1 (read/write) disks on one Symmetrix array and R2 (read-only) mirrors on the other arrays. Additionally, four-way mirroring can be implemented, which locally mirrors each member of the R1/R2 mirror pair.

Symmetrix A Symmetrix B

SRDF Link

Fibre ChannelHub/Switch

Node 1

Node 4Node 3

Node 2Fibre Channel

Hub/Switch

VLAN Switch VLAN Switch

Site APrimary

Site BSecondary

Private Interconnect

Enterprise LAN/WAN

Clients

SRDF/CE: A geographic cluster system 1-17

1-18

Introduction


A Symmetrix array can support multiple MSCS clusters on a single Symmetrix array. Likewise, multiple SRDF/CE clusters can be supported on an SRDF link (refer to Figure 1-6).

Figure 1-6 Multiple SRDF/CE clusters supported on an SRDF link

SRDF/CE modes of operationSRDF/CE supports 8 node clusters Windows Server 2003 Enterprise and Datacenter Editions.

Different cluster designs support different modes of operation and data sharing mechanisms. The configuration for an SRDF/CE two-node or multinode cluster in a geographically distributed cluster environment is either active/passive or active/active. EMC defines active/passive and active/active configurations as follows:

Site A Site B

LAN

SRDF Link

Cluster 1

Cluster 2Cluster 3

Cluster 2

Cluster 3

Cluster 1

Cluster 1Cluster 2

Cluster 3


Introduction

◆ Active/Passive: A cluster of two nodes or more where all processing is done on one node during normal operation and the work is picked up by a remaining passive node (or nodes) only when a failure occurs on the active node. In a two-node configuration, half of the hardware is normally idle. When failover occurs, the application restarts with full performance.

Note: Active/passive multinode clustering provides greater flexibility than the standard active/passive MSCS two-node cluster by providing more options in resolving failures and load distribution after server failures. For example, in a multinode cluster, your configuration may include one or more passive (idle) servers to take over the load from other servers during a site failure, or you may distribute the load among the surviving active nodes.

◆ Active/Active: A cluster of two nodes or more where all nodes are running application software during normal operation. When a failure occurs on a node (or nodes), the work is transferred to a remaining node (or nodes) and restarted. The node(s) that picks up the work must then handle the processing load of both systems, and performance is usually degraded. However, all the computer hardware is used during normal operation.

Note: The terms active/active and active/passive apply to the cluster and to the applications running on the cluster. Both the cluster software and the application software must be designed for active/active operation.

Figure 1-7 presents a typical SRDF/CE two-node two-cluster configuration.


1-20

Introduction

Figure 1-7 Two-node two-cluster SRDF/CE configuration

Symmetrix A Symmetrix B

SRDF/Synchronous Link



VLAN Switch VLAN SwitchExtended IP Subnet

Cluster 1/Host 2

Cluster 2/Host 1Cluster 2/Host 2

Cluster 1/Host 1


Introduction

Pre-SRDF/CE clustering considerationsTo ensure disaster recovery protection in an SRDF/CE-enabled cluster, consider the following prior to its installation and configuration:

◆ Cabling

◆ Booting

◆ SRDF coexistence

Cabling Avoid routing all cables through the same path, both in buildings and between sites. To provide an installation with no single point of failure, use a configuration similar to Figure 1-8.

Figure 1-8 Recommended cabling configuration

Symmetrix

Clients

Symmetrix

Path 1

Node 1 Node 2

LAN ALAN B

Path 2


1-22

Introduction

Booting Currently, MSCS can only boot from separate private disks. (MSCS cannot boot off the same bus.) Therefore, SRDF/CE nodes must contain an internal disk for booting or be attached to a nonclustered Symmetrix disk.

SRDF coexistence Multiple SRDF/CE clusters can share the same SRDF pair. SRDF/CE software can extend the Symmetrix enterprise system to support up to 64 MSCS clusters per Symmetrix pair.


Introduction

Microsoft Cluster Service overviewMicrosoft Cluster Service (MSCS) is the clustering extension to Windows Server 2003 Enterprise and Datacenter editions.

MSCS enables up to eight servers running the same Windows operating system in the same domain to be connected to a shared storage system. Typically, this is a RAID array on a shared SCSI storage bus. With SCSI connections, all servers must be within 40 meters of one other (each less than 20 meters from the storage). With Fibre Channel connections, you can increase the distance between the two servers.

Note: Consult with EMC Customer Support if you need more detail.

MSCS protects against failure of production server hardware or network connections. For data protection, MSCS uses a protected storage subsystem. The standard MSCS cluster relies on a RAID 1 or RAID 5 array storage to guarantee data protection.

In a typical MSCS cluster containing one to eight nodes, server nodes share the application workload. Typically, in a node cluster environment with n nodes, each node serves one-nth of the total number of disks and clients connected by a common SCSI bus. If one server node fails, one or several of the remaining nodes take ownership of all the disks and assume all the application workload.

Microsoft Cluster Service overview 1-23

1-24

Introduction

Figure 1-9 presents a typical two-node MSCS cluster on Windows Server 2003 Enterprise and Datacenter editions.

Figure 1-9 A typical two-node MSCS cluster


Enterprise LAN/WAN

Symmetrix

Clients

Ultra Wide SCSIor

Fibre Channel

Ultra Wide SCSIor

Fibre Channel

Host ANode 1 Host B

Node 2


Introduction

Figure 1-10 presents a typical four-node Windows Server 2003 cluster.

Figure 1-10 A typical four-node MSCS cluster

MSCS cluster conceptsMSCS is a loosely coupled cluster system. It is not a fault-tolerant, closely coupled system. The concept of a cluster is to take two or more, off-the-shelf, independent computers and set them up to work together to provide higher availability and scalability than what you can obtain using a single system. When failure occurs in the cluster, control of a disk, or resource, moves to another cluster node. This process is called a failover. Failovers can be initiated by a number of events, including the following:

◆ Manual failover — The moving of resources from one server to another. Done for system load balancing or for server maintenance.

◆ Failover due to hardware failure — The surviving node takes over when a server, SCSI or Fibre Channel host bus adapter (HBA) card, or network interface card (NIC) fails.

◆ Failover due to application failure — The failure of a virtual server or IP resource can initiate the failover.

VLAN Switch VLAN SwitchHost ANode 1

Host BNode 2 Host D

Node 4

Host CNode 3

Fibre Switch Fibre Switch

Ultra Wide SCSI orFibre Channel

Ultra Wide SCSI orFibre Channel

Symmetrix


1-26

Introduction

By contrast, a fault-tolerant system uses special-purpose hardware to run multiple computers in lock step, which provides nonstop computing with no data loss when a component failure occurs. Therefore, fault-tolerant systems are more expensive.

There are benefits and limitations to using a cluster architecture.

Benefits Clustering provides:

◆ Improved availability by continuing to provide a service even during hardware or software failure.

◆ Increased scalability by allowing new components to be added as system load increases.

◆ Simplified management of groups of systems and their applications by enabling multiple applications on multiple servers to be managed as a single system.

Limitations Clustering cannot protect against:

◆ Software corruption◆ Human-induced failures

Note: Protection of user data through backup, EMC business continuance volumes (BCVs), or other forms of offline data redundancy, remain vitally important to the reliable operation of mission-critical applications.

MSCS modes of operationMSCS supports 8 node clusters Windows Server 2003 Enterprise and Datacenter Editions.

Similar to the modes of operation generally discussed for SRDF/CE (refer to “SRDF/CE modes of operation” on page 1-18), the configuration for an MSCS multinode cluster in a geographically distributed cluster environment is either active/passive or active/active.

Availability MSCS allows active/active application operation. During normal operation, software applications can be running on both nodes. If either node fails, the applications are restarted on the remaining cluster node. This provides high availability by minimizing application downtime. Usually, it takes one to 10 minutes to fail over and restart an application on MSCS. Restart time is highly application dependent.


Introduction

Scalability In addition to availability protection, cluster technology is scalable. You can add new components to the system and run the same application (accessing the same database) on multiple nodes of a cluster to deliver increased processing power. To provide scalability, data sharing is needed.

For additional information on different modes of operation and clustering concepts, refer to the following Microsoft URL:

http://www.microsoft.com/windowsserver2003/technologies/clustering/default.mspx


http://www.microsoft.com/windowsserver2003/technologies/clustering/default.mspx

1-28

Introduction

Application software in a cluster environmentSoftware running on a cluster may, or may not, be cluster aware. When software is cluster aware, it provides a restart mechanism invoked whenever the application resource is moved to another node in the cluster.

Application failover requires a restart of the application whenever failover occurs. Restart is not instantaneous. Unlike a fault-tolerant computer, a distributed cluster does not provide nonstop computing. The time that restart takes, and the completeness of the recovery, is application dependent:

◆ For a transaction-oriented application (such as SQL or Exchange that contain both a database and transaction log files), the application provides a restart mechanism to recover work in progress. Usually a transaction log is used to record all work in progress. When a node fails, the information in host memory is lost, but the work can be reconstructed by applying the transaction log to the database to restart. This mechanism recovers all transactions completed before the failure. Transactions partially complete are lost and must be reentered.

◆ Applications such as Microsoft Word or Microsoft Excel provide a checkpoint capability. If the application experiences a failover, all work since the last disk checkpoint is lost.

◆ If an application has neither a database nor checkpoint capability, and also retains no information (or state) between client requests (such as a Web Browser or an MS-Outlook client), then it can fail over by reissuing the outstanding request. In this scenario, no work is lost, and no restart is needed on the server.

◆ If the application has neither a checkpoint nor restart capability, and it retains the state between client requests to the server, then it must be rerun from the beginning when the node it is running on fails.


2Invisible Body Tag

This chapter describes SRDF/CE behavior in various operation modes. Unless otherwise noted, SRDF/CE behavior is described for a standard two-node cluster.

◆ SRDF/CE failover operation ............................................................2-2◆ Response to complete site failure ....................................................2-8

SRDF/CE and ClusterBehavior

SRDF/CE and Cluster Behavior 2-1

2-2

SRDF/CE and Cluster Behavior

SRDF/CE failover operationClusters are designed to overcome failures. There are several possible failure modes in a cluster configuration. SRDF/CE protects against more failure scenarios than local clusters can. Failure of an individual client affects only one user and is not discussed in this chapter. In an SRDF/CE cluster, eight types of cluster elements (Figure 2-1 on page 2-3) can fail (singly or in combination).

This section describes the following:

◆ ”SRDF/CE failover and recovery behavior with MSCS” on page 2-4

◆ ”SRDF/CE unique behavior” on page 2-5

◆ ”Complete site failure and recovery” on page 2-6



Figure 2-1 SRDF/CE failover operation

The section that follows discusses how a two-node cluster responds to various combinations of element failures. Cluster response during failure modes is similar in a three- or four-node system, but a standard two-node system is used in this section for discussion purposes. The starting condition for each of these failure scenarios is:

◆ Both nodes are operational◆ Node 1 (N1) owns the quorum disk◆ Both the public link (internode LAN link) and the private link

(heartbeat link) are configured in MSCS as enabled for all network access

Symmetrix Symmetrix

Clients Clients

R1 R2

R1R2

Ultra Wide SCSIor

Fibre Channel

Host ANode 1

Host BNode 2

3

8

4

6

7

1

2

SW

5HW

Enterprise LAN/WAN


BidirectionalSRDF Interconnect

Ultra Wide SCSIor

Fibre Channel

SRDF/CE failover operation 2-3

2-4


Simply stated, the failover and recovery operations SRDF/CE provides can be divided into situations where:

◆ The behavior of SRDF/CE is the same as MSCS local clusters.◆ The geographic separation and disaster tolerance of SRDF/CE

causes unique behavior and provides recovery alternatives.

SRDF/CE failover and recovery behavior with MSCSThe following sections introduce SRDF/CE failover and recovery behavior common with MSCS (refer to Figure 2-1 on page 2-3).

LAN linkfailure (1)

If the LAN connection between nodes fails, both servers are still available and can communicate over the heartbeat link. No failover occurs, current processing continues, and client requests from clients connected to the LAN locally continue to be serviced. Client traffic from clients connected through the LAN link fail.

Heartbeat linkfailure (2)

If the heartbeat link fails, MSCS routes heartbeat messages across the public LAN. Operation of the cluster continues with no failover of resources.

Host NICfailure (4)

The host is cut off from all clients. Processing continues uninterrupted on the other host. On the failed host, client input to that host fails, but current processing activities continue. MSCS detects the NIC has failed. The isolated node takes resources offline to halt processing. The other node brings the failed resources online so application failover can occur.

Server failure (5) If the host node hardware fails, or the operating system crashes, all heartbeat messages to the remaining node cease. The remaining node then uses the quorum disk to discover the first host has failed. The remaining node then brings the resources of the failed node online and starts the applications recovery procedures.

Application softwarefailure (6)

If an application module fails, MSCS initiates failover to the remaining node. The SRDF/CE resource monitor is directed to make the storage resource for the failed application available on the other node to allow application failover.

Host bus adapterfailure (7)

An HBA failure is a resource failure that triggers an MSCS failover operation. If both Symmetrix arrays are still running, the failover operation completes normally.



SRDF/CE unique behaviorThe following sections introduce SRDF/CE unique behavior which is different from MSCS behavior (refer to Figure 2-1 on page 2-3).

Symmetrix arrayfailure (8)

When a mirrored disk fails in a Symmetrix array, it is not visible to the host because normal operations continue with the mirror, and the failed drive is hot replaced without disturbing the host. However, if an entire Symmetrix array fails, it appears to its attached server as a resource failure indistinguishable from an HBA failure. MSCS on that server triggers a failover operation. However, because the Symmetrix array itself has failed, the remaining Symmetrix devices recognize that SRDF communication is lost and prevent failover from completing unless overrides are set as described in “Complete site failure and recovery” on page 2-6.

SRDF link failure (3) If the SRDF link between a Symmetrix array fails, the EMC ControlCenter® Symmetrix Manager application notices the condition and reports an error.

The MSCS service does not notice the change (because access to existing disk resources is not disturbed) unless it tries to bring a new resource online. If it does try to activate a disk, the local Symmetrix array does not allow the disk to be brought online. The local Symmetrix array also returns a negative reply to Is Alive? queries from MSCS.

All such SRDF link failures are noted in the Event Log, in the SRDF/CE log, and in the System tray icon pop-up bubble. If MSCS or a user attempts to fail over or fail back a group, and there is no SRDF link available to perform that operation, the operation is not allowed. However, if there are multiple active lateral nodes and the group(s) in question are on that lateral side, lateral-to-lateral failover is permitted.


2-6


Complete site failure and recovery

Local MSCS Cluster In a local MSCS cluster, if an entire site fails (such as from a flood, fire, and so forth) the entire cluster fails. By contrast, with an SRDF/CE cluster, each site contains only one of the two nodes in the cluster (or only one of the n nodes in a multinode cluster).

SRDF/CE Cluster In SRDF/CE, a complete site failure can be caused by either a site failure or a total communication failure. Figure 2-2 illustrates the two types of complete site failure.

Figure 2-2 Types of complete site failure

Site (server and Symmetrix) failures (5+8)Site failure occurs when the host and the Symmetrix array both fail (such as from a natural disaster or human error).

Total communication failure (1+2+3)A total communication failure can occur while the host and Symmetrix array remain operational (such as a backhoe digs up the cable conduit where all communications cables leave a building).

Symmetrix Symmetrix

R1 R2

R1R2

Host A Host B

SRDF Link

Heartbeat

Enterprise LAN/WAN

Clients

Symmetrix Symmetrix

R1 R2

R1R2

Host A Host B

SRDF Link

Clients

Site Failure Total Communication Failure

Heartbeat

Enterprise LAN/WAN



A total communication failure, while both nodes remain operational, is referred to as a split-brain condition and is a potential cause of logical data corruption. For example, if both sides assume the other is dead and begin processing new transactions against their copy of the data, two separate and unreconcilable copies of the data can be created.

Both nodes are isolated from each other, but not from local clients. It is impossible to determine if the other node is alive. No remote client processing is possible, but running processes continue.

Note: There is no way for the surviving node to determine which of these two types of failures caused the site failure.


2-8


Response to complete site failureIn SRDF/CE, site failure modes determine the behavior of a cluster when a failure occurs separating the two Symmetrix arrays and suspends remote data mirroring protection.

If a complete site failure occurs, MSCS on the surviving node first notices that heartbeat messages are no longer being received. MSCS attempts to communicate with the other node using the LAN communication path, to see if communication is still possible.

If both communication links are unresponsive, MSCS queries the quorum disk to attempt to discover whether it should fail over the resources. The query to the quorum disk fails because Node 1 is no longer in contact with the Symmetrix array on Node 2. MSCS then concludes the entire site is down.

MSCS then queries the status of the disk resource and decides whether to bring the disk resources on the local node online or to set them offline. The commands to perform this query from MSCS to SRDF/CE are:

◆ Is Alive?— Determines whether a currently online resource is still healthy and can continue to be used, or whether it and all dependent cluster resources must be taken offline.

◆ Online Request— Changes the state of an offline resource to online for a failover.

Site failure mode settings determine how SRDF/CE responds to queries from Cluster Service. This setting must be manually configured to select the desired failover and recovery behavior.

The SRDF/CE site failure mode settings are:

◆ No New Onlines — Both nodes are allowed to continue processing with the resources currently online. In addition, both nodes are allowed to take resources offline, but are not permitted to bring currently offline resources online. In this mode, MSCS can cause all resources on one or more nodes to enter the offline state.

Note: No New Onlines is the recommended (default) mode.

Important: If MSCS cannot write to the quorum disk when it wants to, it terminates.



Inappropriate user actions that cause groups to bounce back act differently. If you attempt to move the quorum group when the SRDF link is down and No New Onlines is configured, the MSCS destination node terminates, and the group bounces back. Active/active configurations are obviously affected because any applications on the destination node now move. This behavior is a result of the preceding behavior.

◆ Local Override— In an SRDF link failure, this setting will only attempt to move disks laterally. If the SRDF link is up, this setting has no impact.

◆ Failstop— On both nodes, SRDF/CE returns a negative status to an Is Alive? inquiry by Cluster Service. This causes MSCS to take all SRDF/CE-dependent storage resources offline. Shutdown of all processing activity on both nodes occurs. It must be decided then whether to continue with the failover operation at the site that is still active.

Important: The mode selected must be set the same for both nodes in the cluster.

Whenever a failure occurs such that mirrored data protection between sites is lost (for example, the SRDF link is down or a Symmetrix array is down), SRDF/CE responds to the failure in the following ways:

◆ No new disk groups can be brought online until communication with the other node is reestablished (unless the SRDF Override feature is set).

◆ If you have set the No New Onlines mode, then the disks currently online will return a positive Is Alive status to MSCS so the disks can remain online.

◆ If you have set the Failstop mode, then all the disks will return a negative Is Alive status to MSCS so the disks are taken offline by MSCS.

Behavior override In addition to the site failure mode settings, SRDF/CE provides the ability to override the mode behavior and bring resources back online under user direction through the SRDF Override feature. This enables you to decide where processing is allowed to continue.

Response to complete site failure 2-9

2-10


If you determine that one site is actually down, and the other site remains operational, you can use the SRDF Override to:

◆ Override the failure mode.

◆ Allow disk resources to be brought online, even though SRDF is not operating and there is no mirror protection of data.

CAUTION!Use the SRDF Override feature with great care. EMC does not recommend using the SRDF Override feature during normal nondisaster operations.


3Invisible Body Tag S

This chapter provides instructions on and lists the requirements for automatically installing and configuring the SRDF/CE for MSCS software using the SRDF/CE Installation Wizard and the SRDF/CE Configuration Wizard. It also describes how to uninstall SRDF/CE. In the case where a manual installation is required (that is, if you are configuring a dynamic cluster), refer to Chapter 7, “Manual Installation and Configuration.”

◆ System requirements .........................................................................3-2◆ Install considerations.........................................................................3-7◆ SRDF/CE Installation Wizard........................................................3-13◆ SRDF/CE Configuration Wizard...................................................3-20◆ Uninstalling SRDF/CE....................................................................3-75

Automatic Installationand Configuration

Automatic Installation and Configuration 3-1

3-2

Automatic Installation and Configuration

System requirementsThis section lists the hardware, software, and EMC Enginuity requirements for an SRDF/CE cluster.

Hardware requirements

You need the following hardware to create a geographically distributed MSCS cluster using SRDF/CE software:

Symmetrix ◆ A currently supported Symmetrix system with Enginuity 5x69 or later, as documented in the EMC Support Matrix.

◆ Two Symmetrix arrays with Remote Link Director (RLD) cards installed and RDF links established in synchronous or asynchronous mode with auto recovery enabled. The Symmetrix arrays must have RDF-type disks for the device groups.

Note: Each Symmetrix array pair supports up to 64 MSCS clusters using SRDF/CE software.

◆ SRDF must be configured for bidirectional SRDF/Synchronous or SRDF/Asynchronous operation. Dual unidirectional RDF links will work; however, SRDF/CE will not report on down links with a dual unidirectional configuration.

◆ A minimum of one HBA is required for connectivity to the Symmetrix array (either QLogix or Emulex). If EMC PowerPath® software is used, an additional HBA is required. For HBA driver recommendations, refer to “HBA driver firmware and software” on page 3-4. For clustering reliabiity, it is recommended to have at least two paths per device.

◆ Two network interface cards; one for each public and private cluster connections between hosts.

◆ Gatekeepers — Each Symmetrix array must be configured with six or more gatekeeper devices per node, per side for SRDF/CE exclusive use. Gatekeepers should be mirrored with signatures. This prevents MSCS from hiding them. In addition, for performance reasons, an additional gatekeeper should be associated with each SRDF/CE device group.

Note: EMC recommends for each SRDF/CE group created allocating one additional gatekeeper on each host and associated per group for performance. It is also recommended the gatekeepers be mirrored.



◆ All disks need to be Read Write visible to the intended hosts at least once to allow Windows to fully populate its internal data stores.

Servers Two or more x86, x64, or IA-64 servers with separate external SCSI or Fibre Channel buses and at least two external LAN connections— configured for all communications (public and private). All servers in a cluster must be of the same processor architecture.

Note: MSCS configuration refers to different intracluster LAN interconnects as public and private. SRDF/CE requires all intracluster LAN interconnects be public for the nodes to communicate.

Disks SRDF/CE requires all disks have signatures. You can assign disk signatures using Windows device management initialization or other third-party tools.

Mandates EMC mandates that all hardware configurations (servers, adapter cards, cluster system, and so forth) used in the SRDF/CE cluster environment are listed on the EMC Support Matrix. The EMC Support Matrix can be found at http://Powerlink.EMC.com.

Note: Compaq AlphaServer is not supported by SRDF/CE.

Software requirementsTo install and configure SRDF/CE, the following Microsoft software and EMC software must be installed on all nodes that compose the SRDF/CE cluster.

Microsoft software The software requirements for Microsoft products are:

◆ Windows Server 2003 Enterprise or Datacenter Editions, with the appropriate service pack and current hot fixes.

Note: The Windows Server 2003 Enterprise and Datacenter Editions support clusters of up to eight nodes.

Note: Before installing the required services packs and/or hot fixes, refer to the EMC SRDF/CE for MSCS Version 2.1 Release Notes.

System requirements 3-3


3-4


◆ Microsoft Internet Explorer 6.0 and higher (only required for online help). Currently, the latest release is 6.0 (refer to http://www.microsoft.com/windows/ie/default.mspx).

◆ Microsoft Cluster Server (MSCS) for Windows Server 2003 SP1. Depending on the installation type, it can be preinstalled if the MSCS cluster is fully functional and a conversion to SRDF/CE occurs, or it can be installed as part of the SRDF/CE installation/configuration process.

◆ Be aware of known problems dealing with adding new devices in a clustered environment:

Refer to Microsoft Knowledge Base article 834911 - Volume label is blank after you mount a volume in Windows Server 2003 at: http://support.microsoft.com/kb/q834911.

Refer to Microsoft Knowledge Base article 82088 - Computer stops responding (hangs) when it tries to mount a NTFS volume after you restart the computer at: http://support.microsoft.com/kb/q820888.

HBA driver firmware and software

CAUTION!SRDF/CE supports either the SCSIport or the STORport driver; however, there are issues to consider for both drivers.Only Emulux adapters are supported.

It is recommended that two Symmetrix arrays used for the RDF relationship have serial numbers with differing last two digits. If they are the same, the SRDF/CE groups must be built manually using SYMCLI, and then imported into SRDF/CE. SRDF/CE will then populate these groups into MSCS.

SCSIport driver The vendor-dependent data returned by a SCSI Inquiry command and stored in the Windows Registry after a boot or Windows device scan only returns the last two digits of the Symmetrix serial number.

In addition, the latest EMC HBA firmware is required.

STORport driverCurrent versions of the STORport driver do not recognize devices created on Symmetrix arrays with a Symmetrix internal device LUN value greater than 255 decimal. This means devices presented to a


http://www.microsoft.com/windows/ie/default.mspx

http://support.microsoft.com/kb/q834911


http://support.microsoft.com/?id=820888


host for use by MSCS and SRDF/CE must have a Symmetrix internal device LUN less than 255 decimal to be visible to MSCS and SRDF/CE. Microsoft is addressing this problem.

Microsoft Hot Fixes MSKB817789 and MSKB823728 and the latest HBA hardware vendor driver are required.

EMC software Any SRDF-capable Symmetrix array can be used in a geographically distributed MSCS cluster.

Note: For details on the required versions of Symmetrix Enginuity (microcode), refer to the EMC SRDF/CE for MSCS Version 2.1 Release Notes, or contact your EMC representative.

The following EMC software is required to run an SRDF/CE cluster.

◆ EMC SRDF/CE

◆ EMC SRDF

◆ EMC Solutions Enabler Kit Version 6.2 or higher, with licenses for the base component, SRDF, and SRDF/CE features is required. SRDF/CE requires the installation of the full runtime kit. SYMAPI must be installed in its default installation folder at C:\Program Files\EMC\SYMCLI\shlib.

EMC Solutions Enabler Version 6.2 is obtained through EMC Customer Service or the EMC Powerlink™ website at http://Powerlink.EMC.com.

IMPORTANT: If you choose to install the base daemon, storapid, with EMC Solutions Enabler 6.2 you MUST issue the following command which will load the base daemon automatically before SRDF/CE service loads:

stordaemon install storapid –autostart

Note: These products must be installed prior to SRDF/CE installation.

◆ EMC PowerPath is an option, but only required if more than one HBA is attached per host and you want to load balance between the HBAs.

System requirements 3-5

3-6


SRDF/CE installation and operating requirementsSRDF/CE requires a Windows installation and runtime user ID. Either a local or a domain-based user ID can be used to install or configure SRDF/CE. The user ID must be the same in all cases across all cluster member nodes, have local admin rights, be a member of the administrator’s group, and have the following permissions:

◆ Act as part of the operating system◆ Log on as a service◆ Log on as a batch job

Important: You must assign the previous three permissions directly to the SRDF/CE user ID, not to the group to which the SRDF/CE user ID belongs.

SRDF/CE requires a minimum of 16 MB of disk space, not including runtime logging storage.

SRDF/CE uses a runtime licensing methodology. To license this product, the correct SRDF/CE runtime license must be entered into the EMC Solutions Enabler License Management Facility using the symlmf command. This process is documented in the EMC Solutions Enabler Installation Guide.

Note: After installation, to change the SRDF/CE user ID or the password, use the Change User Credentials option in the SRDF/CE Configuration Utility (refer to “SRDF/CE for MSCS icon options” on page 5-3) or uninstall/reinstall SRDF/CE.



Install considerationsThe automatic installation procedure describes how to use the SRDF/CE Installation Wizard (refer to “SRDF/CE Installation Wizard” on page 3-13).

Note: The automatic configuration procedure describes how use the SRDF/CE Configuration Wizard and is described in “Configuring SRDF/CE and MSCS for the first time” on page 3-21.

The SRDF/CE Installation Wizard provides a user-friendly alternative to using a manual installation procedure, which is more difficult to use. Only use a manual installation procedure if installing a dynamic cluster with VERITAS Volume Manager on your system. If this is the case, refer to Chapter 7, “Manual Installation and Configuration.”

Contact EMC Customer Support for assistance if any of the following issues are applicable:

◆ You have applications layered on with dependencies.◆ You need other devices online.◆ You are not confident about installing and configuring new

software within the context of Windows Server 2003, MSCS, and Symmetrix with SRDF.

Prerequisites The automatic installation procedure assumes:

◆ Anyone installing SRDF/CE has a working knowledge of the following:

• MSCS setup and configuration • Windows Server 2003 concepts and administration • Symmetrix operations• SRDF family of software solutions• Cluster-aware applications• Disaster-tolerant/disaster-recovery concepts

◆ Windows Server 2003 SP1 (Enterprise or Datacenter Editions), is installed as a member server of an existing domain on each node that makes up the cluster, and each node is rebooted.

Install considerations 3-7

3-8


◆ A supported Internet browser is installed to view online help.

◆ A copy of the appropriate service packs and/or hot fixes as outlined in the SRDF/CE release notes and the EMC Support Matrix.

Before you begin Before installing SRDF/CE, read through all installation procedures and related information for an overall understanding of the installation process. The configuration guidelines provide references for properly installing hardware and software in an SRDF/CE cluster environment.

Devices A device must be write enabled on each cluster member node at least once before it can be used by a MSCS cluster and SRDF/CE for MSCS. MS Windows will not update the WMI (Windows Management Instrumentation) data structures until the disk resource is write enabled.

The following steps explain how to make them write enabled on both R1 and R2 sides.

Assume, for this example, node A is on the R1 side and node B is on the R2 side.

1. After adding devices to node A on the R1 side, use the EMC Solutions Enabler command line interface to create a temporary device group (temp) containing those devices.

C:\>symdg create -type r1 tempC:\>symld -g temp add dev 1EF, 1F0, 1F1

2. After adding devices to node B on the R2 side, use the EMC Solutions Enabler command line to create a group (temp) containing those devices.

C:\>symdg create -type r2 tempC:\>symld -g temp add dev 1F7, 1F8, 1F9



3. On node A, fail back the temp group to R1 side. If devices are not in a valid RDF state to be failed over, split the devices and then do a full establish.

4. Reboot node A.


3-10


5. After node A is up, perform a Symmetrix discover, and then check the RDF state of the devices. Wait until all newly added devices are in Synchronized state.

Once the disks are in the Synchronized state go to the Computer Management Console, Device Manager and scan for hardware changes. When complete, open Disk Manager and initialize the newly added disks as MBR.



6. On node B, failover the temp group to the R2 side.

7. Reboot node B.


3-12


8. After node B is up, perform a Symmetrix discover.

9. Remove the temp group on both nodes A and B, as follows:

C:\>symdg delete temp -force

The newly added devices are ready for use in an MSCS environment.

Planning requirements

EMC recommends completing the preinstallation worksheet (refer to Appendix B) and your EMC Customer Service Engineer (CSE) reviewing it prior to installing SRDF/CE. Using the worksheet, you and the EMC CSE can gather all the required information (node IDs, device IDs, RA groups, Symmetrix IDs, and so on) to successfully install and configure SRDF/CE. Having this information readily available makes the installation and configuration process smoother.



SRDF/CE Installation Wizard The SRDF/CE Installation Wizard is used to automatically install SRDF/CE. The Installation Wizard is also used to install SRDF/CE in certain upgrade procedures, refer to Chapter 4, “Upgrading SRDF/CE.”

You must install SRDF/CE using the Installation Wizard on each node in your cluster. After installing SRDF/CE on the first node in your cluster, configure the node with SRDF/CE using the Configuration Wizard (refer to “SRDF/CE Configuration Wizard” on page 3-20). After installing SRDF/CE and completing the Configuration Wizard Phase 1 and Phase 2 activities on the first node, repeat this process on each of the remaining nodes in the cluster (that is, install SRDF/CE using the Installation Wizard on the second node, configure it using the Configuration Wizard; then install SRDF/CE on the third node and configure it, and so on).

Figure 3-1 on page 3-14 presents a process flowchart showing the sequence of steps to correctly install and configure SRDF/CE on the individual nodes in your cluster using the Installation and Configuration Wizards.

The step-by-step process the Configuration Wizard leads you through when initially configuring SRDF/CE and MSCS is further explained in “Configuring SRDF/CE and MSCS for the first time” on page 3-21.

SRDF/CE Installation Wizard 3-13

3-14


Figure 3-1 SRDF/CE installation and Configuration Wizard process flowchart

Co

nfi

gu

rati

on

Wiz

ard

Ph

ase

1

Sel

ect c

onfig

urat

ion

mod

eS

elec

t nod

e co

nfig

urat

ion

setu

pId

entif

y as

the

first

nod

e an

d en

ter

clus

ter

#S

elec

t loc

al a

nd r

emot

e S

ymm

etrix

arr

ays

S

RD

F/C

E S

ervi

ce s

tart

sC

reat

e te

mpo

rary

gro

ups

Writ

e en

able

dev

ices

Fa

il ov

er d

evic

es (

if an

y)S

elec

t loc

al IP

add

ress

(es)

Run

Dis

k M

anag

er to

ass

ign

driv

e le

tters

/vo

lum

e la

bels

Configure Node 2

Configure Next Node

Co

nfi

gu

rati

on

Wiz

ard

Ph

ase

1C

on

fig

ura

tio

n W

izar

d P

has

e 1

Fir

st N

od

e in

Clu

ster

No

de

N (

Inte

rmed

iate

No

des

)L

ast

No

de

in C

lust

er

MS

CS

Inst

alla

tio

n

En

d o

f M

SC

S In

stal

lati

on

on

Las

t N

od

e in

Clu

ster

Co

nfi

gu

rati

on

Wiz

ard

Ph

ase

3U

pdat

e M

SC

S p

rope

rtie

sA

dd S

RD

F/C

E r

esou

rce

type

Cre

ate

quor

um g

roup

Con

figur

e qu

orum

arb

itrat

iont

tmeo

uts

Reb

oot n

ode

Install MSCS on Node N

Install MSCS on Node 1 Install MSCS on Node 1

MS

CS

Inst

alla

tio

n

Co

nfi

gu

rati

on

Wiz

ard

Ph

ase

3Jo

in th

e no

de in

the

exis

ting

clus

ter

En

d o

f M

SC

S In

stal

lati

on

on

Fir

st N

od

e in

Clu

ster

SR

DF

/CE

for

MS

CS

MS

CS

SR

DF

/CE

for

MS

CS

En

d o

f M

SC

S In

stal

lati

on

o

n In

term

edia

te N

od

es

MS

CS

Inst

alla

tio

n

C

on

fig

ura

tio

n W

izar

d P

has

e 3

Join

the

node

in th

e ex

istin

g cl

uste

r

Use

r in

pu

tA

uto

mat

ed

Co

nfi

gu

rati

on

Wiz

ard

Ph

ase

2S

elec

t con

figur

atio

n m

ode

Add

clu

ster

mem

ber

node

(s)

Sel

ect c

lust

er ty

peA

ssig

n qu

orum

con

trol

set

tings

Ass

ign

site

failu

re m

ode

and

ad

vanc

ed c

ontr

ol s

ettin

gs

Co

nfi

gu

rati

on

Wiz

ard

Ph

ase

2C

on

fig

ura

tio

n W

izar

d P

has

e 2

Sel

ect c

onfig

urat

ion

mod

eR

etrie

ve P

hase

2 s

ettin

gs fr

om N

ode

1V

iew

quo

rum

set

tings

Iden

tify

if la

st n

ode

in c

lust

er

Sel

ect c

onfig

urat

ion

mod

eR

etrie

ve P

hase

2 s

ettin

gs fr

om N

ode

1V

iew

quo

rum

set

tings

Iden

tify

if la

st n

ode

in c

lust

er

Sel

ect c

onfig

urat

ion

mod

eS

elec

t nod

e co

nfig

urat

ion

setu

pId

entif

y as

not

firs

t nod

e in

clu

ster

S

elec

t loc

al a

nd r

emot

e S

ymm

etrix

arr

ays

S

RD

F/C

E S

ervi

ce s

tart

sS

elec

t the

hos

t nam

e of

firs

t nod

eR

etrie

ve P

hase

1 s

ettin

gs fr

om fi

rst n

ode

and

writ

e en

able

dev

ices

R

etrie

ve te

mpo

rary

gro

ups

from

firs

t nod

e

Fai

l ove

r de

vice

s (if

any

)S

elec

t loc

al IP

add

ress

(es)

R

un D

isk

Man

ager

Sel

ect c

onfig

urat

ion

mod

eS

elec

t nod

e co

nfig

urat

ion

setu

pId

entif

y as

not

firs

t nod

e in

clu

ster

S

elec

t loc

al a

nd r

emot

e S

ymm

etrix

arr

ays

S

RD

F/C

E S

ervi

ce s

tart

sS

elec

t the

hos

t nam

e of

firs

t nod

eR

etrie

ve P

hase

1 s

ettin

gs fr

om fi

rst n

ode

and

writ

e en

able

dev

ices

R

etrie

ve te

mpo

rary

gro

ups

from

firs

t nod

e

Fai

l ove

r de

vice

s (if

any

)S

elec

t loc

al IP

add

ress

(es)

R

un D

isk

Man

ager

Del

ete

tem

pora

ry g

roup

s

Inst

all S

RD

F/C

E v

ia C

D a

nd In

stal

l Wiz

ard

S

RD

F/C

E S

ervi

ce is

reg

iste

red

(

but n

ot s

tart

ed)

Inst

alla

tio

n P

has

eIn

stal

l SR

DF

/CE

via

CD

and

Inst

all W

izar

d

SR

DF

/CE

Ser

vice

is r

egis

tere

d

(bu

t not

sta

rted

)

Inst

alla

tio

n P

has

e

Inst

all S

RD

F/C

E v

ia C

D a

nd In

stal

l Wiz

ard

S

RD

F/C

E S

ervi

ce is

reg

iste

red

(

but n

ot s

tart

ed)

Inst

alla

tio

n P

has

e



To install SRDF/CE, insert the CD into an attached and mounted CD drive. If Autorun is enabled on the host, the installation starts automatically. If Autorun is not enabled, use Windows Explorer to double-click the setup.exe application found at the root of the CD. This launches the SRDF/CE Installation Wizard. The Installation Wizard Welcome window appears as shown in Figure 3-2.

Figure 3-2 Welcome window


3-16


Click Next and the License Agreement window appears as shown in Figure 3-3.

Figure 3-3 License Agreement window

After accepting the license conditions, click Next and the Destination Folder window appears as shown in Figure 3-4.

Figure 3-4 Destination Folder window



Click Next. If a previous version of SRDF/CE or GeoSpan is detected, the Upgrade Type window appears as shown in Figure 3-5. See Chapter 4 for details of the upgrade procedure.

Figure 3-5 Upgrade Type window

Click Next and the Ready to Install the Program window appears as shown in Figure 3-6.


3-18


Figure 3-6 Ready to Install the Program window

Click Install. After installing SRDF/CE, the InstallShield Wizard Completed window appears, as shown in Figure 3-7.

Figure 3-7 InstallShield Wizard Completed window

Click Finish and you will be prompted to reboot, as shown in Figure 3-8 on page 3-19.



Figure 3-8 Reboot dialog box

You must reboot the node to complete the installation of SRDF/CE.

After installing SRDF/CE on the node you are on and rebooting your computer, you can invoke the SRDF/CE snap-in based Configuration Utility by double-clicking the SRDF/CE Configuration Utility desktop icon. The Configuration Utility is the base display for SRDF/CE.

Note: For more information on the Configuration Utility, refer to Chapter 5, “SRDF/CE Configuration Utility GUI.”

The first time you successfully log in to the Configuration Utility, several setup functions are performed on the node you are on, including the setup of the SRDF/CE Services. To change the SRDF/CE user ID or password after this, use the Change User Credentials option in the Configuration Utility (refer to “SRDF/CE for MSCS icon options” on page 5-3) or uninstall/reinstall SRDF/CE.

Once you have successfully installed SRDF/CE on the node you are on, configure the node with SRDF/CE using the Configuration Wizard. The Configuration Wizard is accessed by right-clicking the SRDF/CE for MSCS icon in the left pane of the Configuration Utility and selecting the Config Wizard option (refer to “Configuring SRDF/CE and MSCS for the first time” on page 3-21).


3-20


SRDF/CE Configuration WizardThe SRDF/CE Configuration Wizard enables you to set up cluster configuration parameters automatically in a step-by-step fashion. You can use the SRDF/CE Configuration Wizard for the following three node configuration options:

◆ Configuring both SRDF/CE and MSCS for the first time. In this case, after you install SRDF/CE on the first node in the cluster, use the Configuration Wizard to configure SRDF/CE on this node. Repeat this process for each of the remaining nodes in the cluster (that is, installing and configuring SRDF/CE on each individual node). When all nodes in the cluster are configured with SRDF/CE, use the Configuration Wizard to install and configure MSCS on each node in the cluster starting with the last node on which you configured SRDF/CE. Refer to “Configuring SRDF/CE and MSCS for the first time” on page 3-21. (Most of this chapter describes this configuration option.)

◆ Converting your existing MSCS cluster to an SRDF/CE cluster. Refer to “Converting an MSCS cluster” on page 3-65.

◆ Adding an SRDF/CE node to an existing, running SRDF/CE cluster. Refer to “Adding an SRDF/CE node to an existing cluster” on page 3-72.

Access the Configuration Wizard by right-clicking the SRDF/CE for MSCS icon in the Configuration Utility (refer to “SRDF/CE for MSCS icon options” on page 5-3 in the “SRDF/CE Configuration Utility GUI” chapter).



Configuring SRDF/CE and MSCS for the first timeOpen the Configuration Wizard by right-clicking the SRDF/CE for MSCS icon in the left pane of the SRDF/CE Configuration Utility and selecting the Config Wizard option (Figure 3-9).

Figure 3-9 Opening the Configuration Wizard from the Configuration Utility

To configure both SRDF/CE and MSCS for the first time, the SRDF/CE Configuration Wizard consists of three phases. The Configuration Wizard allows you to perform the configuration steps associated with each of the three phases. The first two configuration phases (Phase 1 and 2) involve SRDF/CE configuration activities. The third phase (Phase 3) involves the installation of MSCS.

The first screen that appears when you launch the Configuration Wizard is shown in Figure 3-10 on page 3-22.

SRDF/CE Configuration Wizard 3-21

3-22


Figure 3-10 SRDF/CE Configuration Wizard welcome screen

Phase 1 and Phase 2 Configuration activities apply to the node you are on; you must apply settings on a node-by-node basis (there is no remote configuration). In other words, you must first performPhase 1 and Phase 2 steps on Node 1, then on Node 2, and so on until Phase 1 and Phase 2 steps are performed on all nodes. Once all your nodes are configured with SRDF/CE, proceed to Phase 3 to install MSCS on all nodes in the cluster, starting with the last node you configured with SRDF/CE.

The Phase 3 Configuration steps consist of installing MSCS and performing post-MSCS activities on a node-by-node basis on all the nodes in the cluster, starting with the last node on which you performed Phase 1 and 2 Configuration activities. The Phase 3 Configuration steps are performed on each node in the reverse node order from which you configured the nodes in Phases 1 and 2. For example, if you configured Nodes 1, 2, 3, and 4 in that order for Phases 1 and 2, then for Phase 3, you will perform Phase 3 Configuration activities on Nodes 4, 3, 2, and 1, in that order.

If you cancel out of any phase during the Configuration Wizard, all settings and temporary groups created in previous completed phases are preserved, but settings are not saved for any uncompleted phase. Therefore, if you cancel out of Phase 1 after creating temporary



groups or restart the Configuration Wizard during Phase 1, manually remove any unnecessary temporary groups using SYMCLI.

If you are configuring an SRDF/CE cluster on Windows Server 2003, it is possible some storage devices will not be enabled correctly. For example, devices may appear with incorrect labels, and there may be inaccessible or unavailable disks. In this case, rebooting the host is recommended. If you reboot during a Configuration Wizard phase, settings for that phase will not be saved, and you may have to manually remove any temporary groups created in that phase.

Figure 3-1 on page 3-14 presents a process flowchart showing the sequence of steps that the SRDF/CE Configuration Wizard leads you through when initially configuring both SRDF/CE and MSCS.

Phase 1 configuration overviewThe Phase 1 Configuration activities are slightly different depending on whether you are on the first node.

Phase 1 configuration steps for the first nodeFor the first node, the Phase 1 Configuration activities consist of the following nine steps. These steps must be performed in exact order.

1. “Step 1. Select configuration mode” on page 3-26.

2. “Step 2. Select node configuration” on page 3-27.

3. “Step 3. Designating the first node and enter cluster number” on page 3-28.

4. “Step 4. Select local Symmetrix array” on page 3-29.

5. “Step 5. Select remote Symmetrix array” on page 3-29.

6. “Step 6. Create temporary device groups” on page 3-30.

7. “Step 7. Write-enable devices” on page 3-33.

8. “Step 8. Select local IP addresses” on page 3-34.

9. “Step 9. Run Disk Manager” on page 3-35.

Phase 1 configuration steps for intermediate or last nodeFor any node other than the first node (that is, intermediate nodes or the last node), the Phase 1 Configuration activities consist of the same nine steps just listed, except the sixth and seventh steps are different. The steps are as follows:

1. “Step 1. Select configuration mode” on page 3-26).


3-24



3. “Step 3. Designating the first node and enter cluster number” on page 3-28.

4. “Step 4. Select local Symmetrix array” on page 3-29.

5. “Step 5. Select remote Symmetrix array” on page 3-29.

6. “Step 6. Select the hostname where first node is located” on page 3-38.

7. “Step 7. Retrieving groups and write-enabling devices” on page 3-38.

8. “Step 8. Select local IP address(es)” on page 3-40.

9. “Step 9. Run Disk Manager” on page 3-41.



Phase 1 configuration steps for the first nodeThe first dialog box to appear in the Phase 1 Configuration lists the nine steps you must complete in Phase 1 for the first node (Figure 3-11).

Figure 3-11 Overview of phase 1 configuration steps

Click Next to start the Phase 1 configuration.


3-26


Step 1. Select configuration modeAs shown in Figure 3-12, you can select the Automatic or Manual mode. Selecting Automatic mode enables you to perform the configuration steps associated with the three phases through the SRDF/CE Configuration Wizard GUI. Selecting Manual mode enables you to perform the configuration steps through the SRDF/CE MMC snap-in GUI.

Important: EMC recommends using the Automatic mode to configure your cluster, and using the Manual mode to install a dynamic cluster using the VERITAS software product Volume Manager for Windows (previously called LDM Pro). You can create and manage dynamic and basic disks on Windows Server 2003 using VERITAS Volume Manager for Windows. VERITAS Volume Manager for Windows enables you to logically group disks under disk groups.

Figure 3-12 Selecting the configuration mode

Select the configuration mode and click Next to continue.



Step 2. Select node configurationSelect the SRDF/CE for MSCS option as your node configuration setup, since you are installing and configuring both SRDF/CE and MSCS for the first time. As shown in Figure 3-13, you must select the type of node to configure from one of the following three options:

◆ SRDF/CE for MSCS — Select this option if you are installing and configuring both SRDF/CE and MSCS for the first time. In this case, you install SRDF/CE first and then MSCS. Most of this chapter pertains to this option.

◆ Convert MSCS Cluster to SRDF/CE — Select this option if you want to convert your existing MSCS cluster to an SRDF/CE cluster. “Converting an MSCS cluster” on page 3-65 describes how this option differs from the first option.

◆ Add a new SRDF/CE Cluster Node to an Existing, Running SRDF/CE Cluster — Select this option if you want to add an SRDF/CE node to an existing, running SRDF/CE cluster. “Adding an SRDF/CE node to an existing cluster” on page 3-72 describes how this option differs from the first option.

Figure 3-13 Selecting the node configuration

Select the type of node configuration and click Next to continue.


3-28


Step 3. Designating the first node and enter cluster numberAs shown in Figure 3-14, you must identify whether the node is the first node in the cluster. A node designated as the first node requires slightly different configuration steps than the other nodes in the cluster. The other cluster node(s) can be intermediate nodes or the last node.

In this step, you can also choose the cluster number. The cluster number can be between 0 and 63. If you are setting up only one cluster and there are no other SRDF/CE clusters defined in your network, you can leave the default cluster number equal to 0.

CAUTION!This feature only works if multicast is active. If it is not active, the cluster numbers must be manually assigned and tracked.

Figure 3-14 Designating the first node and entering cluster number

After designating the node as the first node in the cluster, and entering the cluster number, click Next. After clicking Next, a



Symmetrix Discover is completed, the cluster number is validated, and the SRDF/CE Service starts.

Note: The length of time to complete a Symmetrix Discover depends on the number of disks in the storage subsystem connected to the current node.

If you have only one local Symmetrix array and one remote Symmetrix array, the Configuration Wizard will skip Steps 4 and 5, and proceeds directly to “Step 6. Create temporary device groups” on page 3-30.

Step 4. Select local Symmetrix arraySelect a Symmetrix array from the list. The selected Symmetrix array will be configured as the local Symmetrix array (Figure 3-15).

Figure 3-15 Selecting local Symmetrix array

Note: If you have only one local Symmetrix array and one remote Symmetrix array, the Configuration Wizard will skip Steps 4 and 5 and proceed directly to “Step 6. Create temporary device groups”.

Step 5. Select remote Symmetrix arraySelect a Symmetrix array from the list. The selected Symmetrix array will be configured as the remote Symmetrix array (Figure 3-16).

Figure 3-16 Selecting remote Symmetrix array

Note: If you have only one local Symmetrix array and only one remote Symmetrix array, the Configuration Wizard will skip Steps 4 and 5 and proceeds directly to Step 6.


3-30


Step 6. Create temporary device groupsTemporary device groups must be created in the SRDF/CE configuration process so operations (such as write-enabling) can be performed on the disks. Temporary device groups are then deleted later on in the configuration process at the end of Phase 1 for the last node.

You can create R1 or R2 device groups that are synchronous or asynchronous. In Figure 3-17 on page 3-31, the user has selected an RDF1 device type, and in Figure 3-18 on page 3-31 the user has selected an RDF2-Asynchronous device type. Use the Configuration Wizard to create temporary groups to test their suitability and usability across the cluster. At least one device is required to create a device group.

Note: Device group information automatically propagates to the other nodes in the cluster.

If you cancel out of any phase during the Configuration Wizard, all settings and temporary groups created in previous completed phases are preserved, but settings are not saved for any uncompleted phase. Therefore, if you cancel out of Phase 1 after creating temporary groups or restart the Configuration Wizard during Phase 1, you must manually remove any unnecessary temporary groups using SYMCLI.



Figure 3-17 Creating synchronous temporary device groups

Figure 3-18 Creating asynchronous temporary device groups


3-32


If you are creating a temporary device group, and one of the devices is not in the Synchronized or Failed Over state, you will get the message shown in Figure 3-19.

Figure 3-19 RDF status of devices error message

If you try to create an asynchronous device group in the Failed Over state, the warning shown in Figure 3-20 appears.

Figure 3-20 SRDF/Asynchronous group creation warning



Step 7. Write-enable devicesThe temporary device groups must be write-enabled on this node so that disk letters and volume labels can be assigned to the temporary device groups (Figure 3-21). In this step, all devices are write-enabled on this node, and if devices are not resident on this node, they will fail over to this node and be write-enabled.

Note: If you are configuring an SRDF/CE cluster on Windows Server 2003, it is possible that some storage devices will not be enabled correctly. For example, devices may display with incorrect labels, and there may be inaccessible or unavailable disks. In this case, rebooting the host is recommended. If you reboot during a Configuration Wizard phase, settings for that phase will not be saved, and you may have to manually remove any temporary groups created in that phase.

Figure 3-21 Write-enabling devices

Click Next to continue.


3-34


Step 8. Select local IP addressesSelect a maximum of two available IP addresses from the list to designate as your local IP address (Figure 3-22). The IP addresses selected must be able to communicate with the other SRDF/CE hosts in the cluster because SRDF/CE will use them for intraprocess communication. These IP addresses can be the same or different as those used by MSCS.

Figure 3-22 Selecting local IP address



Step 9. Run Disk ManagerClick Next to assign volume labels and disk letters to the devices in the temporary groups through the Disk Manager utility (Figure 3-23).

Figure 3-23 Running Disk Manager

CAUTION!Be aware of known problems dealing with adding new devices in a clustered environment:

◆ Refer to Microsoft Knowledge Base article 834911 - Volume label is blank after you mount a volume in Windows Server 2003 at http://support.microsoft.com/kb/q834911.

◆ Refer to Microsoft Knowledge Base article 82088 - Computer stops responding (hangs) when it tries to mount a NTFS volume after you restart the computer at http://support.microsoft.com/kb/q820888.




3-36


When you run Disk Manager, the Disk Management utility appears, as shown in Figure 3-24. The Write Signature and Upgrade Disk Wizard also appears when you run Disk Manager. Use this wizard to assign new disk signatures and partition and label disks (Figure 3-25 on page 3-37).

CAUTION!Do not upgrade any disks to dynamic disks. SRDF/CE can only use basic disks with Microsoft Cluster Server.

Figure 3-24 Disk Management utility



Figure 3-25 Write Signature and Upgrade Disk Wizard

This is the last Phase 1 Configuration step for the first node. Now you must perform Phase 2 configuration activities on the first node. Proceed to Phase 2 Configuration “Step 1. Select configuration mode” on page 3-45.

Phase 1 Configuration steps for intermediate or last nodePhase 1 Configuration activities for any node other than the first node (that is, intermediate nodes or the last node) consist of the same nine Phase 1 steps listed in the previous section for the first node, except the sixth and seventh steps are different. The Phase 1 steps for an intermediate or last node are discussed next.

Step 1. Select configuration modeRefer to “Step 1. Select configuration mode” on page 3-26.

Step 2. Select node configurationRefer to “Step 2. Select node configuration” on page 3-27.

Step 3. Designate that the node is not the first node in clusterIdentify whether the node is the first node in the cluster. Select No in the Is this the first node in the cluster? field (refer to Figure 3-14 on page 3-28).

Step 4. Select local Symmetrix arrayRefer to “Step 4. Select local Symmetrix array” on page 3-29.


3-38


Step 5. Select remote Symmetrix arrayRefer to “Step 5. Select remote Symmetrix array” on page 3-29.

Step 6. Select the hostname where first node is locatedThis step is different from Step 6 of Phase 1 for the first node. For an intermediate or last node, perform this step and enter the hostname where the first node is located (Figure 3-26). The first node refers to the node from which you want to pull the configuration. You can enter the hostname in the box or select it by clicking Browse (...).

Figure 3-26 Selecting the hostname where first node is located

Step 7. Retrieving groups and write-enabling devicesThis step is different from Step 7 of Phase 1 for the first node. For an intermediate or last node, this step retrieves the groups from the first node (that is, the node from which you want to pull the configuration) and applies the Phase 1 settings (that is, all the group information) from the first node to the current node. Then, the devices on the current node are write-enabled (Figure 3-27 on page 3-39).



Note: If you are configuring an SRDF/CE cluster on Windows Server 2003, it is possible that some storage devices will not be enabled correctly. For example, devices may appear with incorrect labels, and there may be inaccessible or unavailable disks. In this case, rebooting the host is recommended. If you reboot during a Configuration Wizard phase, settings for that phase will not be saved, and you may have to manually remove any temporary groups created in that phase.

Figure 3-27 Retrieving groups from the first node and write-enabling devices

If devices in the device group cannot be write-enabled, the RDF State of Devices message box appears, as shown in Figure 3-28.

Figure 3-28 RDF State of Devices message


3-40


Step 8. Select local IP address(es)Select a maximum of two available IP addresses from the list to designate as your local IP address (Figure 3-29). The IP addresses selected must communicate with the other SRDF/CE hosts in the cluster because SRDF/CE will use them for intraprocess communication. These IP addresses can be the same or different as those used by MSCS.

Figure 3-29 Selecting local IP address for an intermediate or last node



Step 9. Run Disk ManagerClick Next if you want to assign volume labels and disk letters to the devices in the temporary groups through the Disk Manager utility (Figure 3-30).

Figure 3-30 Running Disk Manager on an intermediate or last node

CAUTION!Do not upgrade any disks to dynamic disks. SRDF/CE can only use basic disks with Microsoft Cluster Server.

Note: You may not see some of the devices in the temporary groups on the intermediate or last nodes. If this is the case, reboot the host. If you reboot in the middle of a Configuration Wizard phase, settings will not be saved, and you may have to manually remove any temporary groups created in that phase.

Note: If this is the last node, the temporary groups will be deleted.


3-42


This is the last Phase 1 step for the intermediate or last node you are on. Now proceed to Phase 2 Configuration “Step 1. Select configuration mode” on page 3-54.

Phase 2 configuration overviewThe Phase 2 Configuration activities are different depending on whether you are on the first node.

Phase 2 configuration steps for the first nodeFor the first node, the Phase 2 Configuration activities consist of the following six steps. These steps must be performed in exact order.


2. “Step 2. Add cluster member node(s)” on page 3-46.

3. “Step 3. Select cluster type” on page 3-47. This step only displays if you are configuring a cluster on Windows Server 2003 Enterprise or Datacenter Editions. On the Windows Server 2003 platform, you have the option of configuring a quorum disk-based cluster or an MNS cluster. On any other platform, the cluster can only be configured as a quorum disk-based cluster.

• If you select Shared Quorum, the next step is Step 4, assigning quorum control settings.

• If you select Majority Node Set, the next step is Step 5, assigning site failure mode and advanced control settings.

4. “Step 4. Assign quorum control settings” on page 3-48.

5. “Step 5. Assign site failure mode and advanced control settings” on page 3-50.

Important: After performing the above Phase 2 Configuration steps on the first node, you do not proceed directly to the Phase 3 Configuration. First you must do the following on each of the remaining nodes in the cluster:

a. Install SRDF/CE.

b. Perform the Phase 1 and 2 Configuration steps.



Phase 2 configuration steps for the intermediate or last nodeFor any node other than the first node (that is, intermediate nodes or the last node), the Phase 2 Configuration activities consist of four steps:


2. “Step 2. Retrieve phase 2 settings” on page 3-55.

3. “Step 3. Viewing cluster type” on page 3-56. This step is for informational purposes only. You cannot change the cluster type because this setting is pulled from the first node.

4. “Step 4. Viewing quorum settings” on page 3-57. This step is for informational purposes only. You cannot change these settings since they were pulled from the configuration node.

5. “Step 5. Designate if node is the last node” on page 3-58.

After performing these four steps, if you are on an intermediate node, you must do the following on each of the remaining intermediate nodes in the cluster:

a. Install SRDF/CE.

b. Perform the Phase 1 and 2 Configuration steps.

If this node is the last node in the cluster, proceed directly to “Phase 3 configuration” on page 3-59.


3-44


Phase 2 configuration steps for the first nodeFigure 3-31 shows the first dialog box to appear in the Phase 2 Configuration when you are on the first node.

Figure 3-31 Overview of phase 2 configuration steps for first node

Click Next to begin the Phase 2 configuration.





Figure 3-32 Selecting the configuration mode in phase 2



3-46


Step 2. Add cluster member node(s)A minimum of one cluster member node is required in the Phase 2 Configuration. As shown in Figure 3-33, select a cluster member node by clicking Browse (...) or enter more than one hostname in the box by separating the hostnames with a comma.

Figure 3-33 Adding a cluster member node




Step 3. Select cluster typeIf you are configuring a cluster on Windows Server 2003 Enterprise or Datacenter Editions, you have the option of configuring a quorum disk-based cluster or an MNS1 cluster (Figure 3-34). On any other platform, the cluster can only be configured as a quorum disk-based cluster, and this step will not appear.

Select Shared Quorum if your cluster quorum data will reside on a shared disk in the cluster. Select Majority Node Set (MNS) if your cluster quorum will reside on a local disk on each node and be managed by MSCS.

Note: MNS is only supported on Windows Server 2003 Enterprise and Datacenter Editions.

Figure 3-34 Selecting the cluster type

1. MNS enables you to create a server cluster without any shared disk for the quorum resource. MNS provides an alternative quorum resource for clusters with nodes in separate geographic sites.


3-48


If you select Shared Quorum, the next step is assigning quorum control settings (refer to “Step 4. Assign quorum control settings” on page 3-48).

If you select Majority Node Set, the next step in the Configuration Wizard is assigning site failure mode and advanced control settings (refer to “Step 5. Assign site failure mode and advanced control settings” on page 3-50).

Important: Once you select and configure a cluster based on the type of quorum management, this value cannot be changed. If you decide later you want to use a different cluster quorum type, you must reinstall SRDF/CE.

Step 4. Assign quorum control settingsIn this step, you select the quorum disk and place it in a group. A quorum disk contains the configuration information of the cluster. You may edit the quorum disk group name and the quorum mount point (Figure 3-35 on page 3-49). The physical drive number can be between zero and 999 and the group name must conform to the following restrictions:

◆ be alphanumeric◆ "-" and "_" are allowed◆ no spaces or special characters are allowed



Figure 3-35 Assigning quorum control settings

The group to which the quorum disk is assigned cannot be defined as an SRDF/A group. If the group is defined as an SRDF/A group, then you are prompted to either cancel the requested action or remove the SRDF/A designation.

Important: The quorum disk you specify in SRDF/CE must be the same as the quorum disk that you specify in MSCS.

Note: You only need to set the Quorum Control settings on one node because the settings will automatically be propagated to the other nodes in the cluster.

Click Next to continue and proceed.


3-50


Step 5. Assign site failure mode and advanced control settings As shown in Figure 3-36, in this step you must select site failure mode options for Local Override (Yes/No), SRDF Override (Yes/No), and Site Mode (No New OnLines or FailStop). You must also assign Advanced Control Settings for the Delay Failback feature (Yes/No).

Figure 3-36 Assigning site failure mode and advanced control settings

Site Failure Mode Settings

Local Override — Selecting Yes means when a site failure occurs (that is, loss of SRDF link status), SRDF/CE will not attempt to automatically try the SRDF link during a failover process. Instead, it will only attempt lateral failovers. The default is No.

SRDF Override — Selecting Yes enables you to move or bring new groups online even though the SRDF link is down and there is no mirror protection of data. The default is No.

CAUTION!EMC does not recommend using the SRDF Override mode during normal nondisaster operations. Use this feature with care.



Site Mode — No New OnLines is the recommended (default) mode. The SRDF/CE site failure mode settings are:

◆ No New OnLines — Both nodes are allowed to continue processing with the resources currently online. In addition, both nodes are allowed to take resources offline, but are not permitted to bring currently offline resources online. In this mode, MSCS can cause all resources on one or more nodes to enter the offline state.

◆ Failstop — This setting causes groups to go offline when there is a server, site, or SRDF failure. On both nodes, SRDF/CE returns a negative status to an Is Alive? inquiry by Cluster Service. This causes MSCS to take all SRDF/CE-dependent storage resources offline. Shutdown of all processing activity on both nodes occurs. Then it must be decided whether to continue with the failover operation at the active site.

Important: The mode selected must be set the same for all nodes in the cluster.

Advanced control settings

Delay Failback — The default for the Delay Failback feature is Yes. The Delay Failback feature is automatically propagated to the other nodes in the cluster.

The Delay Failback feature modifies the Preferred Owner list for each MSCS cluster group such that a failover will occur to a lateral node first, and if the lateral node is unavailable, to a peer node. Lateral nodes are defined as nodes connected to the same Symmetrix array. Peer nodes are defined as nodes connected to different Symmetrix arrays located across the SRDF link from each other.

Note: Since the Delay Failback feature makes use of the MSCS Preferred Owner list any list that preexists or is modified by the administrator will be overwritten.

SRDF/CE manipulates the MSCS Preferred Owners list whenever a group is brought online. SRDF/CE then examines the group Preferred Owners list and determines what node is the lateral node. It can then modify the MSCS Preferred Owner list so the current node and its lateral partner are the first two in the list. Therefore, no matter which side (R1 or R2) a group is moved to, the Preferred Owner list is modified to allow a group to fail over to a lateral node and not fail back or over across the SRDF link as a first option. MSCS only moves a group across the SRDF link as a last resort. This prevents MSCS from arbitrarily performing what amounts to a failback/failover


3-52


across the SRDF link in an automatic fashion. This feature delays the actual failback of a group to an R1 node, and is therefore termed delay failback.

The Delay Failback feature runs simultaneously on all nodes. Therefore, when a group comes online on any node, the Preferred Owner list is updated, regardless of whether it is an R1 or R2 device.

Important: The Delay Failback feature is enabled by default. You must not change the Preferred Owner list; doing so will result in indeterminate failover behavior.

Note: You only need to change the Site Mode and Delay Failback settings on one node because the settings automatically propagate to the other nodes in the cluster.

After assigning the site failure mode and advanced control settings for the first node, the screen shown in Figure 3-37 appears.

Figure 3-37 End of phase 2 for node 1



Important: This is the last step in the Phase 2 Configuration for the first node. Do not proceed directly to Phase 3. You must first do the following on each of the remaining nodes in the cluster:

1. Install SRDF/CE.

2. Perform the Phase 1 and Phase 2 Configuration steps.

When you complete these steps for all the intermediate nodes and the last node in your cluster, you can proceed to the Phase 3 Configuration (refer to “Phase 3 configuration” on page 3-59). Phase 3 involves the installation of MSCS on all the nodes in the cluster (starting with the last node you configured with SRDF/CE).

Phase 2 configuration steps for the intermediate or last nodeFigure 3-38 shows the first dialog box to appear in the Phase 2 Configuration.

Figure 3-38 Phase 2 configuration steps for intermediate or last node

Click Next to begin the Phase 2 configuration.


3-54




Figure 3-39 Selecting the configuration mode in phase 2




Step 2. Retrieve phase 2 settingsIf you are on an intermediate node or the last node, this step retrieves all the Phase 2 settings from the first node and applies them to the current node. The Phase 2 settings retrieved and applied to the current node include all the cluster member node, site failure mode, advanced control, multicast, and cluster type settings (Figure 3-40).

Figure 3-40 Retrieving phase 2 options from node 1



3-56


Step 3. Viewing cluster typeThis step is informational since the cluster type is predetermined by Node 1. You can view the cluster type in the dialog box shown in Figure 3-41.

Figure 3-41 Viewing cluster type




Step 4. Viewing quorum settingsThis step is informational since the quorum disk is predetermined by Node 1. You can view the quorum control settings in the dialog box shown in Figure 3-42. For information relating to what each field describes, refer to “Step 4. Assign quorum control settings” on page 3-48.

Figure 3-42 Viewing quorum settings



3-58


Step 5. Designate if node is the last node

You must identify if the node you are on is the last node in the cluster (Figure 3-43).

Figure 3-43 Identifying if node is last node

If you are on an intermediate node, perform the Phase 1 and Phase 2 configuration steps on each of the remaining intermediate nodes in the cluster.

If you are on the last node, you can proceed to the Phase 3 Configuration (refer to “Phase 3 configuration” on page 3-59).

Note: If this is the last node, it is critical you indicate this. During last node processing, several critical internal steps are performed. If you do not identify the last node as such, you will have to reconfigure the node from the beginning of Phase 1.



Phase 3 configurationThe Phase 3 configuration steps consist of installing MSCS and performing post-MSCS activities on a node-by-node basis on all the nodes in the cluster, starting with the last node on which you performed Phase 1 and 2 configuration activities. The Phase 3 configuration steps are performed on each node in the reverse node order from which you configured the nodes in Phases 1 and 2. For example, if you configured Nodes 1, 2, 3, and 4 in that order for Phases 1 and 2, then for Phase 3, you will perform Phase 3 configuration activities on Nodes 4, 3, 2, and 1, in that order.

The Phase 3 configuration activities consist of the following two steps. These steps must be performed in exact order.

1. “Step 1. MSCS installation” on page 3-60.

2. “Step 2. Post-MSCS configuration activities” on page 3-61.

Figure 3-44 shows the first dialog box to appear in the Phase 3 configuration.

Figure 3-44 Overview of phase 3 configuration steps


3-60


Step 1. MSCS installationThe first step in the Phase 3 Configuration is the installation of MSCS on the last SRDF/CE node (Figure 3-45). The actual installation of MSCS is outside the scope of this document. Refer to the appropriate Microsoft MSCS installation guides.

Figure 3-45 Installing MSCS

After you install MSCS, click Next. (Next is disabled until MSCS installation is complete.)

CAUTION!When you initially install MSCS on the last SRDF/CE node, all disks are resident on this last node. Now, you should only select the quorum disk during the MSCS install for MSCS to manage. Do not include all disks during the initial MSCS install (including all disks is the MSCS default option).If you did not install MSCS first on only the quorum disk (that is, you included all disks during the initial MSCS install), MSCS will create groups for all disks. SRDF/CE will not recognize these



groups, and thus cannot manage them. Therefore, in this case, you must delete the disks and groups from MSCS and create the necessary groups within SRDF/CE.

Important: When you install MSCS on Windows Server 2003, click Advanced and choose the minimal configuration option. (You must do this now for the last SRDF/CE node and later on for all subsequent nodes.) On the next tab, make sure to click Quorum and select the quorum disk selected during the SRDF/CE configuration.

Note: On Windows Server 2003, if you do not click Advanced and choose the minimal configuration option, all subsequent nodes will fail the MSCS configuration process.

Once the installation of MSCS on this node is completed, the SRDF/CE Configuration Wizard automatically detects the MSCS installation. Click Next to proceed.

Note: MSCS reads disk configurations during MSCS installation, loads that information into its Registry, and reads its in-memory copy. Therefore, any disk not present during the MSCS install may not be recognized by MSCS.

Step 2. Post-MSCS configuration activitiesThis is the last step of the Phase 3 Configuration for this node. As shown in Figure 3-46 on page 3-62, in this step, four post-MSCS installation activities are completed:

1. The MSCS service is set to Manual start mode.

2. SRDF/CE is added as a resource type under MSCS.

3. The quorum group is created within MSCS.

Note: If the quorum disk in SRDF/CE and the quorum disk in MSCS are not the same, the following warning message appears.


3-62


4. Quorum arbitration timeouts are automatically set.

Figure 3-46 Post-MSCS configuration activities



You have successfully completed the SRDF/CE configuration for the last SRDF/CE node in the cluster, and the dialog box appears as shown in Figure 3-47. You must reboot the last SRDF/CE node after completing the Phase 3 configuration (that is, MSCS installation and post-MSCS configuration activities). The MSCS quorum arbitration occurs prior to the SRDF/CE drivers being put into place and a reboot of the last SRDF/CE node activates the SRDF/CE arbitration code.

CAUTION!If you ignore the reboot screen, SRDF/CE will work, but the quorum arbitration will not start and the quorum failovers will fail.

Figure 3-47 End of phase 3 configuration for the last SRDF/CE node

After rebooting the last SRDF/CE node, complete the Phase 3 Configuration activities for the remaining nodes in the cluster in the reverse node order from which you configured the nodes in Phases 1 and 2.


3-64


CAUTION!To have a fully functional and supported SRDF/CE cluster, you must have installed it as per the installation documentation and completed all phases. Failure to do so may result in an either a nonfunctional or partly functional SRDF/CE cluster configuration where failover results may vary.

Post phase 3 activitiesOnce SRDF/CE is fully installed and configured on each node and Phase 3 is completed on all nodes, configure disk groups that represent the applications you want SRDF/CE and MSCS to protect. To do this, refer to “Groups component” on page 5-31. It is recommended for each individual application, or group of federated applications, a separate SRDF/CE group be built to ensure disk consistency. When an SRDF/CE group is built, it will be automatically added to MSCS.

After you define new SRDF/CE groups, it is highly recommended the application(s) and the disk group(s) be validated by failing them over to each node and ensuring the application and its attendant resources come up correctly in each case.



Converting an MSCS clusterThis section outlines the procedure to convert a standard operational MSCS cluster to an SRDF/CE cluster. To convert the cluster, you must have a valid MSCS cluster configuration and comply with the hardware and software requirements discussed in “System requirements” on page 3-2.

Pre-conversion considerationsConsider the following requirements and warnings before proceeding with converting a standard MSCS cluster to an SRDF/CE cluster:

◆ Be familiar with SRDF/CE and the process to install a new SRDF/CE cluster as outlined in “Install considerations” on page 3-7 and “SRDF/CE Installation Wizard” on page 3-13 before attempting to do the conversion procedure detailed in this section.

◆ All applicable parts of a clean SRDF/CE install must be fulfilled in addition to the conversion procedure detailed in this section (for example, complete the SRDF/CE configuration worksheet found in Appendix B, “Preinstallation Worksheet.”)

◆ This conversion procedure is more complex than a simple install, and therefore should not be attempted by unfamiliar personnel. Inappropriate or unknowledgeable use can result in data loss.

◆ Ensure a recent, full backup is available before attempting this procedure.

This procedure assumes your MSCS cluster is presently operational and functional, all groups can successfully fail over, all storage for the MSCS cluster is contained on the same, single Symmetrix, and an RDF relationship is established between the source Symmetrix and the alternate Symmetrix array. This procedure also assumes all your MSCS disks are presently running on SRDF devices as either an R1 or an R2. If not, you must make these changes to the devices before attempting this procedure.

This procedure does not address designing or configuring the EMC Symmetrix array or SRDF aspects. The procedure assumes the existing cluster data has been moved to a Symmetrix array with properly configured and tested SRDF devices, and the cluster and applications are verified on the SRDF device hardware.


3-66


The steps you perform when converting an MSCS cluster to an SRDF/CE cluster are different depending on whether you are on the first node. The first node is the node with the quorum disk online.

To ensure you convert the MSCS cluster to an SRDF/CE cluster correctly, follow these steps:

1. Check each node to see if the proper service pack and hot fixes are installed. If not, install them now.

2. Install Solutions Enabler V6.2 on both nodes if necessary.

3. Add the SRDF/CE runtime license to the EMC Solutions Enabler license file.

4. Using SYMCLI commands, ensure you can see all devices (local and remote) and also ping the remote Symmetrix array. Ensure the SRDF is set for Synchronous or Asynchronous and Bidirection.

5. Write down everything about the existing disk configuration (you need physical drive numbers, RA groups, and the Symmetrix device numbers of Node 1) and the MSCS group names. Note this information in the SRDF/CE Configuration Worksheet in Appendix B, “Preinstallation Worksheet.” Ensure you know the physical drive number and the Symmetrix device number for each disk in the cluster, and to what group each disk belongs.

Determine the physical drive number of the quorum disk on each node. This is recommended because MSCS removes the physical drive for the disk where the quorum in not online since MSCS is installed and using the quorum disk.

6. In Cluster Administrator, you must rename any and all MSCS groups that contain spaces or characters other than A-Z, a-z, 0-9. MSCS group names must conform to the following restrictions:

• length less than 31 characters• only A-Z, a-z, 0-9 are allowed ("-" and "_" are not allowed)• no spaces are allowed

The SYMAPI database cannot accept special characters. Create a separate group for the quorum disk and move the quorum to this group so it is isolated from the other resources.

In addition, all MSCS resources with the same name as the MSCS group they are in must be renamed. As a reminder, during the installation process a new group is built to contain the quorum



disk. Any resources in the preexisting quorum disk group (typically the Cluster Group) must not be named the same as the user defined group name.

7. Verify all SRDF communication is enabled and you have documented all the cluster settings.

8. Install SRDF/CE on each MSCS node of the cluster. Prior to converting an existing MSCS cluster, SRDF/CE must be installed on each MSCS node. When SRDF/CE detects MSCS installed during the SRDF/CE configuration process, SRDF/CE attempts to register its resources with MSCS. For this process to be successful, MSCS must find the SRDF/CE code installed on each node.

9. On each node, open the SRDF/CE Configuration Utility. Double-click the SRDF/CE for MSCS icon and enter the correct login information.

10. On each node, right-click the SRDF/CE for MSCS icon and select Discover. The SRDF/CE Services must be configured and started by logging in to the SRDF/CE snap-in, and then issuing an SRDF/CE Discover.

After completing Discover on the node that does not own the quorum, if the following message displays, ignore it and continue.

11. Start the SRDF/CE Service on all nodes.

12. On Node 1, right-click the SRDF/CE for MSCS icon and select Configuration Wizard.

You are now ready to proceed with the conversion steps on the first node. Refer to “MSCS to SRDF/CE cluster conversion steps for the first node” on page 3-68.


3-68


MSCS to SRDF/CE cluster conversion steps for the first nodeFor the first node, the Phase 1 Configuration activities consist of the following nine steps, which must be performed in exact order. The sequence of steps for converting an MSCS cluster to an SRDF/CE cluster on the first node are the following:



3. “Step 3. Designating the first node and enter cluster number” on page 3-28. The MSCS group names are checked at this point to verify they comply with SYMAPI naming standards (for example, spaces and special characters are not allowed). If they do not comply, an error message appears and the Config Wizard exits.

4. Select local Symmetrix array(s). If your configuration only has two Symmetrix arrays, one local and one remote, this step is skipped (refer to “Step 4. Select local Symmetrix array” on page 3-29).

5. Select remote Symmetrix array(s). If your configuration only has two Symmetrix arrays, one local and one remote, this step is skipped (refer to “Step 5. Select remote Symmetrix array” on page 3-29).

6. Select the local IP address(es) (refer to “Step 8. Select local IP addresses” on page 3-34).

7. Retrieve MSCS information to apply to SRDF/CE. Retrieving MSCS information includes retrieving the cluster member nodes and group names from MSCS (Figure 3-48 on page 3-69). The geospan_res dependency is then added as a resource type to MSCS.



Figure 3-48 Applying MSCS information to SRDF/CE groups

A warning message appears indicating the groups online on this node will be brought offline briefly to add the geospan_res dependency to each physical disk resource in each group configured by SRDF/CE. Then the MSCS groups are imported and created in SRDF/CE.

8. Assign the Quorum Control settings (refer to “Step 4. Assign quorum control settings” on page 3-48).

9. Assign site mode failure and delay failback settings (refer to Figure 3-49).


3-70


Figure 3-49 Assigning site mode failure and advanced control settings

10. At the end of the SRDF/CE for MSCS configuration page of the first node, you will be prompted to not restart the machine. It is critical to not reboot this specific host at this step. Otherwise disk resources will fail to move to the other cluster member nodes and the cluster could end up in a failed state.

MSCS to SRDF/CE cluster conversion steps for an intermediate or last nodeFor the intermediate or last node, the Phase 1 Configuration activities consist of the following 10 steps, which are only slightly different than those for the first node. Note the first six steps are the same as those for the first node (refer to “MSCS to SRDF/CE cluster conversion steps for the first node” on page 3-68), and then steps 7 through 10 are different. These steps must be performed in exact order. The sequence of steps for converting an MSCS cluster to an SRDF/CE cluster on the intermediate or last node are the following:

1. Select the configuration mode (refer to “Step 1. Select configuration mode” on page 3-26).



2. Select the Convert MSCS Cluster to SRDF/CE option for your node configuration setup (refer to “Step 2. Select node configuration” on page 3-27).

3. Designate the node as not the first node in the cluster (refer to “Step 3. Designating the first node and enter cluster number” on page 3-28). The Configuration Wizard then performs a storage array discover.



6. Select the local IP addresses (refer to “Step 8. Select local IP addresses” on page 3-34).

7. Select the hostname where the first node is located. The first node refers to the node from which you want to pull the configuration. (refer to “Step 6. Select the hostname where first node is located” on page 3-38).

8. Pull configuration from node. Enter the node from which you want to retrieve the cluster configuration. This step will retrieve all the Phase 2 settings from the first node and apply them to the current node. The Phase 2 settings retrieved and applied to the current node include all the cluster member node, quorum control, site mode failure, and delay failback settings (refer to “Step 2. Retrieve phase 2 settings” on page 3-55).

9. Retrieve MSCS information to apply to SRDF/CE. Retrieving MSCS information includes retrieving the cluster member nodes and group names from MSCS. The MSCS group names are then checked to verify they comply with SYMAPI naming standards (for example, spaces and special characters are not allowed). The geospan_res dependency is then added as a resource type to MSCS (refer to Figure 3-48 on page 3-69).


3-72


10. Assign/view quorum control settings. Since the quorum disk is on Node 1, MSCS makes it invisible to the other nodes. Therefore, you must supply the physical drive number for the quorum disk. All other information previously retrieved from the first node cannot be changed (refer to “Step 4. Assign quorum control settings” on page 3-48).

If the following screen displays, ignore it and continue:

11. At the end of the SRDF/CE for MSCS configuration page for the intermediate or last node, you are prompted to restart the machine. Select Yes, I want to restart the computer, and click Finish.

12. After restarting the last node, go back to the first node (the node that owns the quorum) and restart the first node.

Adding an SRDF/CE node to an existing cluster The procedure for adding an SRDF/CE node to an existing SRDF/CE cluster consists of two phases: pre-MSCS activities and post-MSCS activities.



Pre-MSCS activitiesTo complete the pre-MSCS activities, do the following:

Note: Install EMC Solutions Enabler V6.2, if necessary.

1. Select the configuration mode (refer to “Step 1. Select configuration mode” on page 3-26).

2. Select Add a new SRDF/CE cluster node to an existing SRDF/CE cluster as your node configuration setup (refer to “Step 2. Select node configuration” on page 3-27).



5. Select the local IP addresses (refer to “Step 8. Select local IP addresses” on page 3-34).

6. Select the hostname where the first node is located (the first node refers to the node from which you want to pull the configuration). (refer to “Step 6. Select the hostname where first node is located” on page 3-38).

7. Pull configuration (refer to Figure 3-50 on page 3-74). In this step, group and cluster member node information are retrieved from the first node (that is, the node from which you want to pull the configuration) and applied to the current node (that is, the SRDF/CE node you want to add to the SRDF/CE cluster). Then, the latest cluster member node list is propagated to all nodes in the SRDF/CE cluster.


3-74


.

Figure 3-50 Retrieving group and cluster information from the first node

8. View quorum control settings. This step is informational since the quorum disk is predetermined by Node 1. You can view the quorum control settings in the dialog box shown in Figure 3-42 on page 3-57. For descriptions of each field, refer to “Step 4. Assign quorum control settings” on page 3-48.

Post-MSCS activitiesTo complete the post-MSCS activities do the following:

1. “Step 1. MSCS installation” on page 3-60.

2. “Step 2. Post-MSCS configuration activities” on page 3-61.



Uninstalling SRDF/CEThere are two supported scenarios for uninstalling SRDF/CE from a configured cluster.

1. To remove some cluster nodes and leave SRDF/CE on the remaining cluster nodes.

a. Open Microsoft Cluster Administrator.

b. Ensure no cluster resource groups are owned by the node(s) you will remove. Move any owned resource groups to a different node.

c. Right-click the node(s) to remove and choose Stop Cluster Service. Wait for the cluster service to stop on the node(s) as indicated by a red X.

d. Right-click the node(s) you want to remove and choose Evict. Evicting a node will uninstall the cluster service on that node and remove that node from the cluster.

e. Uninstall SRDF/CE from the evicted node(s). Use the Add/Remove Programs utility in the Control Panel to remove SRDF/CE for MSCS. Reboot when prompted to complete the uninstall.

2. Uninstalling SRDF/CE from all nodes of the cluster. The cluster will be deleted.

a. Follow the above procedure for evicting all but one node in the cluster.

b. Right-click the last node and choose Evict. Evicting the last node will uninstall the cluster service on that node and permanently delete this MSCS cluster.

c. Uninstall SRDF/CE from the last node. Use the Add or Remove Programs utility in the Control Panel to remove SRDF/CE for MSCS. Reboot when prompted to complete the uninstall.

Uninstalling SRDF/CE 3-75

3-76



4Invisible Body Tag

This chapter describes the procedures for upgrading a GeoSpan cluster to an SRDF/CE V2.1 cluster and for upgrading an SRDF/CE V2.0 cluster to an SRDF/CE V2.1 cluster.

◆ Getting started ....................................................................................4-2◆ Upgrading GeoSpan clusters to SRDF/CE ....................................4-4◆ Upgrading SRDF/CE V2.0.x to V2.1...............................................4-8

Upgrading SRDF/CE

Upgrading SRDF/CE 4-1

4-2

Upgrading SRDF/CE

Getting started

System requirementsRefer to “System requirements” on page 3-2 for a list of the hardware, software, and EMC firmware requirements for an SRDF/CE clusters.

PrerequisitesRefer to the following sections before attempting a GeoSpan to SRDF/CE V2.1 upgrade procedure or an SRDF/CE V2.0.x to SRDF/CE V2.1 upgrade procedure:

◆ “Prerequisites” on page 3-7

◆ “Before you begin” on page 3-8

◆ “Planning requirements” on page 3-12

PC emulation softwareAlthough not required, it is useful to have PC emulation software installed on both cluster nodes, particularly if they are in different physical locations. The GeoSpan to SRDF/CE V2.1 and the SRDF/CE V2.0.x to SRDF/CE V2.1 upgrade procedures require that you physically move back and forth between nodes. Using remote access software to control nodes from a remote location enables you to perform all steps on one machine.

Verifying your actionsAfter completing your SRDF/CE upgrade procedure, EMC recommends testing the system to ensure that your actions are accurately carried out. Significant steps include the following:

◆ Moving or copying files

◆ Creating or deleting device groups

◆ Adding or removing devices within groups

◆ Changing the status of SRDF devices (from synchronized to failed over, for example)


Upgrading SRDF/CE

Use any of the following methods to verify your actions:

◆ Look in the folders to which you moved or copied files.

◆ Use MSCS Cluster Administrator or the SRDF/CE Configuration Utility GUI to view node or device configuration information.

◆ Use Windows Server 2003 to view details on disk drives, such as a drive’s current status.

CAUTION!Do not perform configuration changes unless you are familiar with the procedure and are aware of its possible impact on subsequent configuration steps.

Getting started 4-3

4-4

Upgrading SRDF/CE

Upgrading GeoSpan clusters to SRDF/CEThe following procedure describes how to upgrade an EMC GeoSpan for MSCS V1.2.3 cluster to an SRDF/CE cluster.

Note: This upgrade can only be performed on version 1.2.3 of GeoSpan MSCS. In addition, this upgrade requires Solutions Enabler Version 6.2.

1. SRDF/CE requires a Windows installation and runtime user ID. Either a local or a domain-based user ID can be used to install or configure SRDF/CE. The user ID must be the same in all cases across all cluster member nodes, have local admin rights, be a member of the administrator’s group, and it must have the following permissions:

• Act as part of the operating system• Log on as a service• Log on as a batch job


2. The following additional privileges must be assigned to the SRDF/CE user ID during the upgrade:

• Backup files and directories

• Restore files and directories

You can set these privileges from Local Security Settings. After setting these privileges, reboot the machine for them to take effect.

These privileges are only required for the checkpoint restore during the upgrade. Once the upgrade is complete, these privileges can be removed.

3. Bring all groups online on Node 1. This step is essential because the upgrade process must work from the last node back to the first.

4. Perform each of the following steps on each cluster member node in reverse order (that is, starting from the last node going back to the first node). For example, in a four-node cluster, do Step 4a first on Node 4, followed by Node 3, Node 2, and Node 1, and then proceed to Step 4b.


Upgrading SRDF/CE

a. Open the GeoSpan Configuration Utility and create a checkpoint as shown in Figure 4-1. Creating the checkpoint is mandatory for the upgrade process to successfully complete. Once the checkpoint process is started, select Yes when asked to overwrite the backup group and save the configuration. After making the checkpoint file, close the Configuration Utility. It is advisable to backup the checkpoint file.

Figure 4-1 Creating a checkpoint in the GeoSpan Configuration Utility

b. Change the Cluster Service to manual, stop the Cluster Service, and then stop the GeoSpan Service, in that order, on each cluster member node in reverse chronological order.

c. Install SRDF/CE V2.1 on each cluster member node in reverse order and select Upgrade when asked how to upgrade from SRDF/CE 2.0. When the installation completes, reboot when prompted.

Note: Prior to upgrading a GeoSpan V1.2.3 cluster, SRDF/CE must be installed on each node. When SRDF/CE detects MSCS installed during the SRDF/CE configuration process, SRDF/CE attempts to register its resources with MSCS. For this process to be successful, MSCS must find the SRDF/CE code installed on each node.

d. When the system comes back up, open the SRDF/CE Configuration Utility on each cluster member node in reverse order. After logging in to the SRDF/CE Configuration Utility with the user ID set up as described in Step 1, right-click the SRDF/CE for MSCS icon and select Discover as shown in Figure 4-2.

Upgrading GeoSpan clusters to SRDF/CE 4-5

4-6

Upgrading SRDF/CE

Note: The Pull Configuration From Peer dialog box appears when you log in to the SRDF/CE Configuration Utility on the remaining cluster member nodes after Node 1. When this dialog box appears, click Cancel.

Figure 4-2 Selecting discover in the SRDF/CE Configuration Utility

e. On each cluster member node in reverse order, restore the checkpoint. To do so, right-click the Control component, and then select SRDF/CE Config Options, Restore as shown in Figure 4-3.

Figure 4-3 Restoring the checkpoint in the SRDF/CE Configuration Utility


Upgrading SRDF/CE

5. On Node 1, start the SRDF/CE driver by issuing the command net start SRDFCE at the command prompt. After that completes, start the SRDF/CE Service, and then start the Cluster Service, in that order.

6. Repeat Step 5 on each of the remaining cluster member nodes in chronological order (for example, on Node 2, Node 3, Node 4 and so on).

7. Once the upgrade is complete, return to each node and check the service settings for the Cluster Service. It is recommended this is set to Automatic startup.

Upgrading GeoSpan clusters to SRDF/CE 4-7

4-8

Upgrading SRDF/CE

Upgrading SRDF/CE V2.0.x to V2.1The following procedure describes how to upgrade an SRDF/CE V2.0.x cluster to an SRDF/CE V2.1 cluster.

Note: This upgrade requires EMC Solutions Enabler V6.2.

1. SRDF/CE requires a Windows installation and runtime user ID. Either a local or a domain-based user ID can be used to install or configure SRDF/CE. The user ID must be the same in all cases across all cluster member nodes, have local admin rights, be a member of the administrator’s group, and have the following permissions:



2. The following additional privileges must be assigned to the SRDF/CE user ID during the upgrade:

• Backup files and directories

• Restore files and directories

You can set these privileges from Local Security Settings. After setting these privileges, reboot the machine for them to take effect.

These privileges are only required for the checkpoint restore during the upgrade. Once the upgrade is complete, these privileges can be removed.

3. Bring all groups online on Node 1. This step is essential because the upgrade process must work from the last node back to the first.

4. On the last node of the cluster do the following:

a. Open the Configuration Utility and create a checkpoint as shown in Figure 4-3 on page 4-6. Creating the checkpoint is mandatory for the upgrade process to successfully complete. Once the checkpoint process is started, select Yes when asked to


Upgrading SRDF/CE

overwrite the backup group and save the configuration. After making the checkpoint file, close the Configuration Utility. It is advisable to backup the checkpoint file.

b. Change the Cluster Service to manual, stop the Cluster Service, and then stop the SRDF/CE Service, in that order.

c. Install SRDF/CE V2.1 and select Yes when asked to upgrade. When the installation is complete, select Reboot if prompted, and then restart the system.

d. When the system comes back up, open the SRDF/CE Configuration Utility. After logging in to the SRDF/CE Configuration Utility with the user ID set up as described in Step 1, right-click the SRDF/CE for MSCS icon and select Discover as shown in Figure 4-2 on page 4-6.

Note: The Pull Configuration From Peer dialog box displays when you log into the SRDF/CE Configuration Utility on the remaining cluster member nodes after Node 1. When this dialog box appears, click Cancel.

e. Restore the checkpoint by right-clicking the Control component, and then select SRDF/CE Config Options, Restore as shown in Figure 4-3 on page 4-6.

f. Start the SRDF/CE Service, and then start the Cluster Service in that order.

g. Move all groups from Node 1 to this node in MSCS.

5. Repeat Steps 4a through 4f on each cluster member nodes in reverse order. For example, if you began your installation on the last node (for example Node 4), then the next node to be configured should be Node 3, followed by Node 2, and so on.

Upgrading SRDF/CE V2.0.x to V2.1 4-9

4-10

Upgrading SRDF/CE


5Invisible Body Tag

This chapter describes the functionality of the SRDF/CE Configuration Utility graphical user interface (GUI).

◆ Configuration Utility introduction..................................................5-2◆ SRDF/CE for MSCS icon options ....................................................5-3◆ Control component ............................................................................5-6◆ Groups component ..........................................................................5-31◆ Storage component ..........................................................................5-42◆ Nodes component ............................................................................5-52

SRDF/CE ConfigurationUtility GUI

SRDF/CE Configuration Utility GUI 5-1

5-2

SRDF/CE Configuration Utility GUI

Configuration Utility introductionThe SRDF/CE Configuration Utility GUI configures SRDF/CE and controls SRDF disk configurations and MSCS disk resource definitions. The SRDF/CE Configuration Utility GUI allows you to set up and configure Symmetrix-based disk resources and configure these resources automatically within MSCS. This allows MSCS to automatically move geographically dispersed disk resource groups back and forth. MSCS resource groups ensure that application dependencies within resource groups are observed, and the application software is coordinated with the movement of data resources. Figure 5-1 is an example of how the SRDF/CE Configuration Utility GUI appears after configuration.

Figure 5-1 SRDF/CE Configuration Utility GUI

As shown in Figure 5-1, the SRDF/CE Configuration Utility GUI comprises four components under the SRDF/CE for MSCS icon:

◆ Control◆ Groups◆ Storage◆ Nodes

These components are described in the following sections.



SRDF/CE for MSCS icon options The SRDF/CE for MSCS icon is enabled after logging in to SRDF/CE for the first time. By clicking the SRDF/CE for MSCS icon, you can access the right-click menu options to change user credentials, access the Configuration Wizard, or to discover Symmetrix arrays (Figure 5-2).

Figure 5-2 SRDF/CE for MSCS icon right-click menu

Change User CredentialsThe Change User Credentials option allows you to change the login credentials (Figure 5-3 on page 5-4). To change the user ID, do the following:

◆ Coordinate this action between all nodes due to DCOM use.

◆ Ensure you are logged in as the new user ID before making the change.

◆ Ensure the Debug Programs privilege is assigned to the new user ID, or ensure the COM server is not running as the old user ID before making the change.

SRDF/CE for MSCS icon options 5-3

5-4


Figure 5-3 Changing user credentials

Configuration WizardThe Config Wizard option launches the SRDF/CE Configuration Wizard. The Configuration Wizard enables you to set up cluster configuration parameters automatically in a step-by-step fashion. The Configuration Wizard guides you through any of the following configuration tasks:

◆ Configuring SRDF/CE and MSCS for the first time◆ Converting an MSCS cluster to an SRDF/CE cluster◆ Adding an SRDF/CE node to an existing SRDF/CE cluster

Note: For more information on the SRDF/CE Configuration Wizard, refer to “SRDF/CE Configuration Wizard” on page 3-20.

DiscoverThe Discover option scans the storage subsystem connected to the current node and gathers device information. Use Discover when configuring SRDF/CE, making Symmetrix configuration changes, updating the SYMAPI database, or following group failover/ failback operations.

Important: SRDF/CE uses a dedicated SYMCLI/SYMAPI database. If you make storage configuration changes, update the SRDF/CE database by either issuing a Discover through the SRDF/CE Configuration GUI (Figure 5-4 on page 5-5), or manually issuing the set symcli_db_file=<fully-qualified db file location> command and then the manual SYMCLI discover command (Figure 5-5 on page 5-5).



Figure 5-4 Issuing a discover through the Configuration Utility GUI

Figure 5-5 Issuing a manual SYMCLI discover command

SRDF/CE for MSCS icon options 5-5

5-6


Control componentAs shown in Figure 5-6, the Control component of the SRDF/CE Configuration Utility GUI contains five configurable parameters of SRDF/CE. The following list provides the parameters and references the page numbers for more details:

◆ “Quorum Control” on page 5-7◆ “Site Mode Operations” on page 5-13◆ “Advanced Controls” on page 5-15◆ “Logging Control” on page 5-18◆ “Forced Failover” on page 5-27

Figure 5-6 Expanded control view in the SRDF/CE Configuration Utility

The Control component enables you to manage the SRDF/CE configuration locally and on all peer nodes. The Control right-click menu has functionality for additional configuration synchronization options with cluster member nodes (Figure 5-7 on page 5-7). The right-click menu item SRDF/CE Config Options on the Control display node enables you to perform the following options:

◆ Checkpoint — Saves to file the current SRDF/CE Registry entries, SYMCLI device group information, and all control parameters and options. This data can subsequently be used to reconfigure the SRDF/CE-specific data back onto a clean installation with the Restore feature.



Important: EMC recommends a configuration checkpoint file be generated and saved after the configuration is complete. Using checkpoint helps to ensure the procedure can be quickly and reliably performed should you need to replace SRDF/CE and/or perform a recovery into an existing SRDF/CE cluster.

◆ Restore — Loads a checkpoint file to restore saved SRDF/CE control information and device group definitions.

Restore does not restore the MSCS configuration. It only restores what is specific to SRDF/CE.

Figure 5-7 Control right-click menu

Quorum ControlAs shown in Figure 5-8 and Figure 5-9 on page 5-8, the Quorum Control option lists the following parameters in the right pane:

◆ Quorum Type◆ Symmetrix Device◆ Group◆ Owning Node◆ Owner SymDevID◆ Physical Device◆ Mount Point◆ R1/R2◆ Device Status◆ Capacity (MB)◆ RA Group

Control component 5-7

5-8


Figure 5-8 Quorum Control expanded view

Figure 5-9 Quorum Control expanded view (continued)



The Quorum Control parameters are described in Table 5-1.

Table 5-1 Quorum Control column headings

Column heading Description

Capacity (MB) Shows the device capacity in megabytes.

Device Status Indicates the R1/R2 device status. The possible device status states are Ready, Not Ready, and Write-Disabled.

Group Indicates the SRDF/CE group name of the quorum.

Owner SymDevID Indicates the SymDevID on the owning node (that is, the MSCS owning node) corresponding to the local SymDevID. For example, in Figure 5-8 on page 5-8, Symmetrix device 0BC (which is part of the quorum group) is mapped to 0BC on the remote Symmetrix array. Since this device is managed by MSCS and is online on API1190 (which is connected to the remote Symmetrix array), the Owner SymDevId shows 0BC (as seen from API1190).

Mount Point Indicates the mount point of the physical drive on the owning node. If the owning node field is blank, this column shows the local mount point (if it exists).

Note: SRDF/CE only supports drive letters. Windows 2003 Cluster Server supports more than drive letters (that is, mount points), but SRDF/CE does not support anything other than drive letters.

Owning Node Shows the MSCS node name that owns the quorum. This information is obtained directly from MSCS. If the quorum is not part of MSCS, or if MSCS is not installed on the local node, the owning node field will be blank.

Physical Device Shows the physical device number as seen on the local node for a particular Symmetrix device. If that device is managed by MSCS, the physical drive number will be that of the owning node.For example, in Figure 5-8 on page 5-8, Symmetrix device 0BC is part of MSCS and is online on API1190. On API1190 this device is mapped to \\.\PHYSICALDRIVE21.

Quorum Type Indicates whether the quorum is a quorum disk-based cluster (Shared Quorum) or a Microsoft Majority Node Set (MNS) cluster. MNS enables you to create a server cluster that does not have any shared disk for the quorum resource.

R1/R2 Shows the R1 or R2 device type.

RA Group Indicates the RA group to which the quorum belongs.

Symmetrix Device Shows the Symmetrix device ID that is mapped to the host.


5-10


Edit Quorum Controlparameters

To modify the Quorum Control parameters, right-click any of the parameters displayed in the right pane and select the Edit Quorum Control Parameters option (Figure 5-10). The Quorum Properties dialog box appears (Figure 5-11 on page 5-11).

Figure 5-10 Selecting edit Quorum Control settings



Figure 5-11 Changing quorum properties

Two options are available to change the quorum disk properties. The default option allows you to select from a pull-down list the currently mounted disks from which to choose the quorum disk. When you select an available disk using the default option, the selected disk will be force mounted to the defined mount point. If any disk is already at that drive letter, that disk will be dismounted. This is the normal option most users should choose.

There are situations where you might want to force a specific disk to be defined as the SRDF/CE quorum disk and you do not want any other action other than the SRDF/CE configuration definition to be changed. Or, you might want to change the SRDF/CE quorum designation when the quorum disk is not online on the current node. In these situations, select the Edit Quorum values using direct text entry option.


5-12


Note: SRDF/CE only supports drive letters. Windows Server 2003 supports more than drive letters (that is, mount points), but SRDF/CE does not support anything other than drive letters.

Only disks with formatted file systems will be visible since the MSCS quorum requires a file system.

Push to MSCS By right-clicking any of the Quorum Control parameters in the right pane, you can also select Push to MSCS. This option allows you to push a new quorum group definition to MSCS. When you select Push to MSCS, the dialog box in Figure 5-12 appears.

Figure 5-12 Pushing a new quorum group definition to MSCS

Set quorum type This option is only available for Windows Server 2003. If you are configuring your cluster on Windows Server 2003, the Quorum Control right-click menu allows you to choose the cluster quorum type you want to configure: a quorum disk-based cluster (Shared Quorum) or a Microsoft Majority Node Set (MNS) cluster. To select the quorum type, select Set Quorum Type from the Quorum Control right-click menu (Figure 5-13 on page 5-13). The Quorum Type dialog box shown in Figure 5-14 on page 5-13 appears. MNS enables you to create a server cluster without any shared disk for the quorum resource.




Figure 5-13 Quorum Control right-click menu.

Figure 5-14 Selecting the quorum type

Site Mode OperationsThe Site Mode Operations option controls how SRDF/CE behaves when a failure occurs. There are three parameters associated with this option: Local Override, SRDF Override, and Site Mode.

◆ Local Override — In an SRDF link failure, this setting will only attempt to move disks laterally. If the SRDF link is up, this setting has no impact. The default is No.

◆ SRDF Override — You can select either Yes or No. Yes enables you to move or bring new groups online if the SRDF link is down. The default is No.


5-14


◆ Site Mode — The site mode you set must be the same on all nodes of the cluster. You can select either No New Onlines or FailStop. The default is No New Onlines.

• FailStop — On all nodes, SRDF/CE returns a negative status to an Is Alive? inquiry by Cluster Service. This causes MSCS to take all storage resources offline. Shutdown of all processing activity on all nodes occurs. Therefore, you must decide whether to continue with the failover operation at the site still active.

• No New Onlines — All nodes are allowed to continue processing with the resources currently online. In addition, all nodes are allowed to take resources offline, but are not permitted to bring currently offline resources online. In this mode, it is possible that MSCS will cause all resources on all nodes to enter the offline state.

To modify these parameters, right-click Site Mode Operations in the left pane or any of the parameters in the right pane, and select Edit Site Mode Parameters (Figure 5-15). The Site Mode Operations dialog box appears (Figure 5-16 on page 5-15).

Figure 5-15 Expanded site mode operations view and right-click menu



Figure 5-16 Changing site mode operations parameters

Advanced ControlsAdvanced Control settings are for advanced debugging purposes only and should only be used with EMC recommendation. As shown in Figure 5-17, Advanced Control lists the Cluster Number, Delay Failback, SRDF/A Failover Method, Auto Del Release, Multicast IP, and Multicast Port parameters.

Figure 5-17 Expanded advanced control view and right-click menu

◆ Cluster Number — A unique identifier associated with the nodes within an SRDF/CE cluster. The cluster number equals a value between 0 and 63 and is propagated through the Autoconfiguration function.


5-16


The Autoconfiguration function attempts to automatically determine if other SRDF/CE clusters are running and what cluster number they are using. This functionality is limited to SRDF/CE clusters that are up and network-accessible to the new cluster. EMC GeoSpan for MSCS clusters do not support or respond to this automatic discovery feature. If there are other GeoSpan or SRDF/CE clusters not network accessible, ensure the cluster number you choose does not conflict with a preexisting cluster.

◆ Delay Failback — Modifies the Preferred Owner list for each MSCS cluster group such that a failover will occur to the lateral nodes first, and if the lateral nodes are unavailable, then to the peer nodes. If the Preferred Owner list is not coded for an MSCS Group, a list is inserted with the lateral nodes followed by the peer nodes. This setting gets updated each time the MSCS Group comes online. If Delay Failback is not on, MSCS controls the node failover.

Important: The Delay Failback feature is enabled by default. Do not change the Preferred Owner list because this results in indeterminate failover behavior.

Note: Delay Failback uses a UDP-based multicast to differentiate between local and remote nodes.

◆ SRDF/Asynchronous Failover Method — If you have SRDF/A-enabled groups, this option allows you to determine how the groups are split after failover (Normal or Immediate). The Normal option splits the groups at the end of the current bundle cycle, ensuring a consistent copy. Normal is the default. The Immediate option splits the groups immediately after failover and therefore the copy is indeterminate (that is, the data copy may or may not be consistent). Only select the Immediate option if the consistency of the data is not important. The Immediate option deletes the pending tracks and immediately performs the failover, with a potential loss of data.

◆ Auto Del Release — In some host environments, long-running SRDF operations may have left Symmetrix Lock 9 locked. The Auto Del Release function, when enabled during a failover or failback operation, checks if the lock value is held. If the lock remains locked for 180 seconds with the Auto Del Release option set, the lock is automatically removed and the failover/failback operation is allowed to proceed.



◆ Multicast IP — The multicast IP address is used as part of the configuration of the Delay Failback operation. The multicast IP address is part of a thread that periodically keeps track of which hosts are running and connected to the Symmetrix array. This information is fed into the Delay Failback operation to ensure a current list of hosts, their state, and what Symmetrix array they are connected to is maintained.

◆ Multicast Port — The multicast port address is used as part of the configuration of the Delay Failback operation in the same way as that described above for the multicast IP address.

To change any Advanced Control parameters, select the Edit Advanced Control Parameters option on the Advanced Control right-click menu in the left pane, or right-click any of these parameters in the right pane (Figure 5-17 on page 5-15). When you select the Edit Advanced Control Parameters option, the dialog box appears as shown in Figure 5-18.

Note: The default for the Status Display is Off for improved performance.

Figure 5-18 Changing Advanced Control parameters


5-18


The Status Display option at the bottom of the Advanced Control Parameters dialog box allows you to determine whether you want to display the status bubble in the bottom-right corner of the SRDF/CE Configuration Utility. The status bubble provides information on Configuration Utility performance. The default is Off.

Logging ControlLogging Control sets the SRDF/CE logging levels and paths. As shown in Figure 5-19, Logging Control lists the SRDF/CE Log Level, the SRDF/CE Log Path, and the SRDF/CE Log File Viewer parameters.

◆ SRDF/CE Log Level — Sets the SRDF/CE log level. Log level can be set from 1–Normal to 4–Trace high. 1 allows only error, warning, and informational messages to be displayed in the log file; 2 enables minimum trace information; 3 enables additional trace information; 4 enables maximum trace information. 1–Normal is the default.

◆ SRDF/CE Log Path — Sets the directory location for the SRDF/CE log file. This is a trace file where all logging information is placed.

◆ SRDF/CE Log File Viewer — Enables you to view any log file.

Figure 5-19 Expanded SRDF/CE service logging view



To edit the log level parameters, right-click SRDF/CE Log Level in the right pane and select Edit Log Level Parameters (Figure 5-20).

Figure 5-20 Editing log level parameters

To edit the log file path, right-click SRDF/CE Log Path in the right pane and select Edit Log File Path (Figure 5-21). The Select the SRDF/CE Log Path dialog box opens (Figure 5-22 on page 5-20).

Figure 5-21 Editing log file path


5-20


Figure 5-22 Editing the log file path

To view the latest log file, as well as other log files, right-click SRDF/CE Log File Viewer in the right pane and select Open Latest Log (Figure 5-23). You can also view log files by double-clicking SRDF/CE Log File Viewer in the right pane. The SRDF/CE Log Viewer dialog box then appears (Figure 5-24 on page 5-21).

Figure 5-23 Viewing the latest log file

To open the log files in the SRDF/CE Log Viewer, do one of the following:

◆ Select File, Open or File, Open With Other Editor.◆ Double-click the file.



To choose which columns you want to display in the SRDF/CE Log Viewer, do one of the following:

◆ Select View, Add/Remove Columns (refer to Figure 5-24).◆ Right-click column headings and select Select or Hide to sort

columns.◆ Drag and drop column headings to sort columns.

You can also search for text within the log files displayed in the SRDF/CE Log Viewer by selecting View, Find.

Figure 5-24 SRDF/CE Log Viewer

Logging enhancementsThe logging features of SRDF/CE for MSCS Version 2.1 are enhanced to provide more detailed logging. If there is a problem with SRDF/CE for MSCS, detailed logs will provide customer support with the technical information necessary to help diagnose the problem and help SRDF/CE engineers with debugging.


5-2


The logging facility now captures much more detail information, and the size and number of log files being created has increased compared to prior versions of SRDF/CE for MSCS.

To help minimize disk capacity utilized by log files, log compression, and log retention are two new features added to help address these issues.

Log compression Log compression uses Microsoft NTFS compression for the SRDF/CE logs. When logging is enabled the Log Compression feature is enabled, and it is not configurable by the user. The minimum individual log file size for SRDF/CE logs is 20 megabytes. Once this threshold has been reached, the logging facility closes the log file, compresses it, and starts SRDF/CE logging to a new log file.

The naming convention used for SRDF/CE log files is srdfce_ followed by a date and timestamp with a .log suffix. For example, a typical log file name would be srdfce_20051012085417.log. When this log file reaches the maximum individual size threshold, the log file will be compressed and a new log file created with a different date and timestamp.

2 EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide


To obtain the actual log file size for an NTFS compressed file from within Windows Explorer, right click on the file and select Properties. The Size on disk field is the amount of disk space the compressed log file is utilizing. The Size field is the uncompressed size of the file.

Note: If log files are moved to another directory and the directory is not a NTFS compressed directory the log file(s) will be decompressed to their actual size.

Opening log files using SRDF/CE log viewer or other standard editing tools will not require decompressing the log files.

Log retention SRDF/CE for MSCS 2.1 will generate more logging information and a greater number of log files than previous versions of the product. Log Retention is a new feature that customizes the amount of time log files will exist, or the maximum disk space the log files utilize.


5-24


By selecting the Control node in the SRDF/CE user interface and then selecting Logging Control and double-click on the Log Retention element the following dialog box will be presented:

There are three user options: Disable log retention; Number of days; and Allocate disk space for SRDF/CE logs.

Disable log retention The Disable log retention option is mutually exclusive with the Number of days and Allocate disk space for SRDF/CE logs options and causes SRDF/CE for MSCS to retain all log files.

Number of daysWhen the Number of days option is selected, the Log Retention value text box is enabled. The value entered in this text box determines how many days from the current date SRDF/CE for MSCS will keep SRDF/CE log files in the configured log file directory.



For example, if you enter 10 in the text box, any log file less than or equal to the current date minus 10 days is deleted.

Allocate disk space for SRDF/CE logsTo allocate disk space for SRDF/CE logs, select Controls, Logging control, and then SRDF/CE Log Retention. The following dialog will appear.

The value located in the log retention value textbox determines how much physical disk space will be allocated for SRDF/CE logs. The value is calculated by adding the size of all log files. If the sum of log


5-26


files is greater than the value specified for the log retention value, log files are deleted from oldest to newest until the size of all log files is less than or equal to the log retention value.

Immediate log cullingThe algorithm to determine which log files to delete is only invoked when a new log file is being created. If either the time or size log retention options are enabled you can also take immediate action by right clicking the SRDF/CE Log Retention element and selecting Delete log files. This feature is only enabled when you select a log retention feature and is disabled if log retention is disabled.

Once you select this feature a confirmation dialog appears asking if you are sure you want to delete log files using existing log retention values.



Forced FailoverThe Forced Failover option provides an automatic way to keep the cluster running on a particular Symmetrix array or arrays in the event of a total site disaster.

The Forced Failover option performs differently depending on whether the MSCS cluster is defined as a quorum disk-based cluster or an MNS-based cluster. For quorum disk-based clusters, the node that owns the quorum decides what nodes will stay up. For MNS-based clusters, you can configure a list of nodes to be forced up in a site disaster.


Quorum disk-based clustersFor quorum disk-based clusters, the side that remains up with respect to a Symmetrix array is based on what node owns the quorum resource. In a site disaster, MSCS keeps all nodes up on the side owning the quorum. All resources owned by the other side are moved to the surviving side.

In the quorum disk case, SRDF/CE monitors all the nodes. If tracks (that is, data) are not owed to the surviving side, then the move proceeds smoothly. If tracks are owed to the surviving side, then the SRDF Override option is required to make the move successful. Therefore, if SRDF/CE detects a split-brain1 condition during MSCS' normal group failover processing, the Forced Failover option automatically invokes the SRDF Override function so the groups from the failing site successfully transition to the new site.

MNS-based clustersAn MNS cluster is different from the quorum disk-based cluster because MSCS automatically shuts down the entire cluster when it detects the majority of nodes are not running. Under normal MSCS operations, you can evaluate the situation and restart the cluster in

1. A total communication failure, while both nodes remain operational, is referred to as a split-brain condition and is a potential cause of logical data corruption. For example, if both sides assume the other is dead and begin processing new transactions against their copy of the data, two separate and unreconcilable copies of the data can be created.


5-28


the Service Control Manager with the ForceQuorum option. The SRDF/CE Forced Failover option automates this process.

When the entire cluster shuts down due to a loss of the majority of nodes, MSCS then automatically attempts a node restart. On restart, a list of nodes (preconfigured by the user during the initial SRDF/CE configuration) is supplied to MSCS. These nodes now constitute the new cluster majority. If the nodes you were running on are included in the preconfigured list of nodes, the node restart will occur with the ForceQuorum option. As groups from the failed site are brought online, the SRDF Override function is automatically invoked to allow that movement.

Important: All nodes in the preconfigured forced failover list need to be running and communicating with each other or the cluster will not be forced up successfully.

To rejoin nodes into a cluster once the cluster is forced up successfully, perform the following steps on the nodes you want to rejoin:

1. Stop the Cluster Service on the nodes with the ForceQuorum option.

2. Remove the ForceQuorum key.

3. Restart the Cluster Service on these nodes.

Note: The Forced Failover option is no longer enabled after you add the nodes not running to the cluster and the cluster resumes normal MNS operation.

The Forced Failover option therefore enables you to configure a node or list of nodes on the same side of the Symmetrix array to restart in the event the majority of nodes are lost.



As shown in Figure 5-25, Forced Failover displays whether this feature is enabled, and if it is an MNS cluster, it displays the configured list of nodes forced up in a site disaster.

Figure 5-25 Expanded Forced Failover view and right-click menu

To enable or disable the Forced Failover option, do one of the following:

◆ Double-click Forced Failover in the left pane.◆ Right-click Forced Failover in the left or right pane and select

Edit Forced Failover (refer to Figure 5-25).

The dialog box shown in Figure 5-26 on page 5-30 appears. If you are running an MNS cluster, you can select which nodes you want to force up in a total site disaster. In Figure 5-26, the user has selected node L82AP216 as the node to be forced up in a site disaster. If you are not running an MNS cluster, the MNS Node Selection group is disabled and grayed out. When you click OK, SRDF/CE verifies the nodes you selected are lateral nodes with respect to each other. You cannot select peer nodes to be forced up since each node would be connected to a different Symmetrix array. In addition, the following warning message appears:

You should only enable this feature when trying to automatically fail resources over following the loss of a site. It is not recommended you enable this feature because there is the potential for data loss. Do you want to continue?


5-30


Figure 5-26 Enabling the Forced Failover option



Groups componentThe Groups component lists the currently configured SRDF/CE device groups. As shown in Figures 5-27 and 5-28, clicking Groups gives you device group information in the right pane, such as the owning node (the MSCS node name that owns the device), the RDF configuration, the RDF state, the specific devices, whether R1/R2 Swap is enabled, the RA group to which the device belongs, whether the SRDF/Asynchronous feature is enabled, the SRDF/Asynchronous failover method, the number of R1/R2 invalid tracks (if any), and the SRDF/A lag time. Clicking each device group in the left pane displays more detailed information about the specific group, such as its Symmetrix device, owning node, owner SymDevId, physical device, mount point, device status, and so on. The device group detail is shown in Figures 5-29 and 5-30. Table 5-2 on page 5-33 provides a description of the Groups column headings.

Figure 5-27 Expanded groups view with the device groups list

Figure 5-28 Expanded groups view with the device groups list (continued)

Groups component 5-31

5-32


Figure 5-29 Device groups detail view

Figure 5-30 Device groups detail view (continued)



Table 5-2 Groups column headings



Devices Shows all R1/R2 Symmetrix device IDs mapped to the host.


Group Indicates the SRDF/CE group name to which the device belongs.

Owner SymDevID Indicates the SymDevID on the owning node (that is, the MSCS owning node) corresponding to the local SymDevID.



Owning Node Shows the MSCS node name that owns this particular disk.This information is obtained directly from MSCS. If the disk is not part of MSCS, or if MSCS is not installed on the local node, then the owning node field will be blank.

Physical Device Shows the physical device number as seen on the local node for a particular Symmetrix device. If that device is managed by MSCS, then the physical drive number will be that of the owning node.


R1/R2 Invalid Tracks Indicates the number of invalid R1 and R2 tracks (if any).

R1R2 Swap Shows whether R1/R2 swap is enabled. Refer to “SRDF/CE SRDF swap support” on page 1-10 for more information on this feature.

RA Group Indicates the RA group to which the group belongs.

RDF Config Shows the RDF configuration of the group (that is, the R1 or R2 device type).

RDF State Indicates the RDF state of the group. For a listing of the possible RDF states, refer to EMC Solutions Enabler SRDF Family CLI Product Guide.

SRDF/Async Lag Time Indicates the lag time between the target (R2) device and the source (R1) device in an SRDF/Asynchronous environment.


5-34


Groups right-click menu optionsYou can create new device groups, pull groups from SYMCLI, or list device groups by right-clicking Groups, selecting SRDF/CE Group Options (Figure 5-31), and selecting one of the following options:

◆ “Create New Device Group” on page 5-35

◆ “Pull groups from SYMCLI” on page 5-38

◆ “List” on page 5-38.

Figure 5-31 Groups right-click menu

Symmetrix Device Shows the Symmetrix device ID mapped to the host.

SRDF/Asynchronous Shows whether the SRDF/Asynchronous feature is enabled.

SRDF/Asynchronous Failover Method

Shows the SRDF/Asynchronous failover method. It can be normal or immediate. Refer to “Advanced Controls” on page 5-15 for more information.

Table 5-2 Groups column headings (continued)




Create New DeviceGroup

The Create New Device Group option enables you to create new device groups by physical device or Symmetrix device. When you select Create New Device Group, the Select and Group Devices dialog box appears (Figure 5-32). The Device Display Type list also allows you to choose SRDF/A-capable devices.

Figure 5-32 Creating a new device group


5-36


If you select SRDF/A devices as a group, the device listings are replaced with Symmetrix RDF device groups that consist of SRDF/A-capable devices (Figure 5-33). If you try to create an asynchronous device group in the Failed Over state, the warning shown in Figure 5-34 appears.

Figure 5-33 Creating an SRDF/Asynchronous-enabled device group

Figure 5-34 SRDF/Asynchronous group creation warning



You can also create an SRDF/Synchronous group with SRDF/A- enabled devices in a consistent state. In this case, a warning displays indicating in order to create an SRDF/Synchronous group, the asynchronous devices in the same RA group will be changed to synchronous (Figure 5-35).

Figure 5-35 Async devices in consistent state warning

The quorum group, by Microsoft requirement, cannot be designated as SRDF/A-capable. Therefore, if you select a device/group SRDF/A-capable and later choose this as the quorum device, then the SRDF/A option will be deactivated. You cannot use any device from a group that is SRDF/Asynchronous to be the quorum disk. Microsoft requires the quorum disk to be synchronous.

Note: SRDF/CE automatically determines what Enginuity level is running on the local and remote Symmetrix arrays. If the Enginuity level is earlier than 5x70, the asynchronous options in the Device Display Type box will not be available. If the pair of Symmetrix arrays is running on Enginuity level 5x70, you will be allowed to create only one SRDF/A group and the R1-R2 swap option will be dimmed. Enginuity levels 5x71 and higher supports multiple SRDF/A groups and R1-R2 swap.


5-38


Pull groups fromSYMCLI

The Pull Groups from SYMCLI option displays all the groups defined in SYMCLI that are not SRDF/CE groups, and then leads you through the process of importing these groups into SRDF/CE and MSCS (Figure 5-36).

Figure 5-36 Pulling groups from SYMCLI

List The List option lists the device groups as shown in the example in Figure 5-37.

Figure 5-37 Listing device groups



Device group right-click menu optionsWhen you right-click the specific device group in the left pane, the SRDF/CE Group Operations right-click menu option enables you to instruct MSCS to move cluster groups or make them come online or offline, to fail over and fail back device groups, to set the R1-R2 Swap option, and to designate a group as SRDF/Synchronous or SRDF/Asynchronous via the Array Operations option (Figure 5-38).

Figure 5-38 Device group right-click menu

◆ MSCS Move — Instructs MSCS to initiate a cluster group move.

◆ MSCS Online — Brings the selected cluster group online.

◆ MSCS Offline—Takes the selected cluster group offline.

◆ Unlock — The Unlock option unlocks any device group locks currently in place. When you select Unlock, the warning message shown in Figure 5-39 appears.

Figure 5-39 Release Symmetrix locks warning

CAUTION!Use Unlock with caution and only with a thorough understanding of this function.


5-40


◆ Array Operations — When you click Array Operations from the SRDF/CE Group Operations right-click menu, the Array Operations dialog box opens (Figure 5-40). The Storage Group Name list shows the single corresponding SYMCLI group that is the same as the SRDF/CE group. There is only one SYMCLI group per SRDF/CE group.

Figure 5-40 Array Operations dialog box

Current Status shows the current group RDF state, whether the SRDF/Asynchronous failover method is normal or immediate, whether the group is SRDF/Synchronous or SRDF/Asynchronous, and if R1-R2 Swap is enabled.

In the Group Actions area you can Synchronize Disks or set the R1-R2 Swap. The R1-R2 Swap option enables you to automatically swap the RDF device designations of a specified device group so that source R1 device(s) become target R2 device(s) and target R2 device(s) become source R1 device(s). If an existing group is being enabled for dynamic swap and this group is currently live on the R2 side, the change will not take



effect until either the group is failed back, or the group is taken offline, and then placed online. If the group is not dynamic swap-capable, this option will be grayed out.

The SRDF/Asynchronous Failover Method option allows you to determine how the groups (if you have SRDF/A-enabled groups) are split after failover (Normal or Immediate). The Normal option splits the groups at the end of the current bundle cycle, ensuring a consistent copy. Normal is the default. The Immediate option splits the groups immediately after failover and therefore the copy is indeterminate (that is, the data copy may or may not be consistent). Only select the Immediate option if the consistency of the data is not important. The Immediate option deletes the pending tracks and immediately performs the failover, with a potential loss of data.

The Set SRDF Protocol Type area also allows you to designate the group as SRDF/Synchronous or SRDF/Asynchronous. If the group is not SRDF/Asynchronous-capable, this option will be dimmed.

The Group Movement area includes the SRDF Failover option and SRDF Failback option. SRDF Failover performs an SRDF Symmetrix group failover of the selected cluster group. A failover is defined as the process of taking one or more resources offline on one cluster member and bringing them online on another cluster member. The SRDF Failback option performs an SRDF Symmetrix group failback of the selected cluster group. A failback is defined as the action of moving a resource back to the cluster member designated to be the resource's preferred owner. By default, resources are owned by their preferred owners, so a failback would only occur if the resource is moved from its preferred owner. This is likely the result of a failover.

CAUTION!Use the SRDF Failover and Failback options with caution; these options do not interact with MSCS, and therefore could result in data corruption/loss as well as cluster loss.


5-42


Storage componentThe Storage component lists the local and remote storage arrays connected to the node (Figure 5-41). For each EMC Symmetrix array, there are expanded lists of the R1 and R2 devices. For example, if you click the R1 Devices folder icon for a Symmetrix array, all the R1 devices associated with that Symmetrix array are listed, as shown in Figure 5-42 and Figure 5-43 on page 5-43. R1 device information that appears in the right pane is described in Table 5-3 on page 5-44.


SRDF/CE uses a dedicated SYMCLI/SYMAPI database. If you make storage configuration changes, you must update the SRDF/CE database by either issuing a Symmetrix SRDF/CE Discover, or manually issuing the set symcli_db_file=<fully-qualified db file location> command and then the manual SYMCLI discover command, symcfg disc.

Note: A major change in the SRDF/CE configuration is the physical device information is now based on the disk signature instead of the SYMAPI database. This allows you to view disks on any node, regardless of where the disks are online.

Figure 5-41 Expanded storage view



Figure 5-42 Symmetrix device detail view

Figure 5-43 Symmetrix device detail view (continued)

Storage component 5-43

5-44


Table 5-3 R1/R2 device description column headings




Dynamic Swap Capable Indicates whether the device is enabled for dynamic swap. Refer to “SRDF/CE SRDF swap support” on page 1-10 for more information on dynamic swap.

Group Name Indicates the SRDF/CE group name to which the device belongs.



Owning Node Shows the MSCS node name that owns this particular disk. This information is obtained directly from MSCS. If the disk is not part of MSCS, or if MSCS is not installed on the local node, then the owning node field will be blank.

Owner SymDevID Indicates the SymDevID on the owning node (that is, the MSCS owning node) corresponding to the local SymDevID. For example, in Figure 5-42 on page 5-43, Symmetrix device 0B4 (which is part of the quorum group) is mapped to 0B4 on the remote Symmetrix array. Since this device is managed by MSCS and is online on API1190 (which is connected to the remote Symmetrix array), the Owner SymDevId shows 0B4 (as seen from API1190).

Physical Device Shows the physical device number as seen on the local node for a particular Symmetrix device. If that device is managed by MSCS, then the physical drive number will be that of the owning node.For example, in Figure 5-42 on page 5-43, Symmetrix device 0B5 is part of MSCS and is online on API1191. On API1190 this device is mapped to \\.\PHYSICALDRIVE9.


RA Group Indicates the RA group to which the device belongs.



In summary, the columns Symmetrix Device, Group Name, R1/R2, Device Status, Capacity (MB), and RA Group pertain to the local node. The Owning Node column lists the node where the device is online in MSCS. The Owner SymDevID column pertains to the owning node. The Physical Device and Mount Point columns pertain to the owning node if the device is managed by MSCS. If the device is not managed by MSCS, these columns pertain to the local node. The Dynamic Swap Capable, SRDF/A Capable, and Signature columns pertain to the device itself.

Select which columns to view in the right pane by selecting View, Add/Remove Columns in Windows Server 2003 (Figure 5-44).

Figure 5-44 Selecting columns to view

Signature Indicates whether a disk signature was found for a particular device. If a disk signature cannot be found (or is not present), the disk will be unusable from the point of view of SRDF/CE, and you cannot add this disk to a group until a valid signature is written.

SRDF/Async Capable Shows whether the device can be used in an SRDF/Asynchronous environment.

Symmetrix Device Shows all the R1/R2 Symmetrix device IDs that are mapped to the host.

Table 5-3 R1/R2 device description column headings (continued)



5-46


The Modify Columns dialog box appears (Figure 5-45).

Figure 5-45 Modifying R1/R2 device columns

Select which rows to view in the right pane by selecting View, Add/Remove Rows (Figure 5-44 on page 5-45). The Add/Remove Rows dialog box appears, as shown in Figure 5-46. This option can be used to remove individual grouped devices from the display list in the right pane.

Figure 5-46 Adding and removing R1/R2 device rows



A feature of the SRDF/CE MMC snap-in is ungrouped devices can be dragged and dropped into an existing SRDF/CE device group. The user is queried to confirm the move, and device-type checking is performed. If the device being added is found to be a raw device (that is, a device without a NTFS file system installed on it and/or without a drive letter assigned to it), that device will be added to the SRDF/CE device group, but not added to MSCS. To do this, manually create the MSCS resource. Raw device and MSCS interaction has proven to be unpredictable.

Note: Prior to dragging and dropping a disk into a group, verify the disk is not mounted on any node in the cluster and the disk is mounted on the correct drive letter.

Devices in an SRDF/CE group must be defined as either Ready or Write Disabled. Not Ready devices can cause MSCS to hang. As with the case of raw devices, Not Ready devices will be added to the SRDF/CE group but not added to the corresponding MSCS group.

Note: MSCS operates on the principle all storage is local in nature. Because of this limitation, MSCS must see all the devices on all nodes in the cluster. To accomplish this, when you set up the Symmetrix devices to be used by the SRDF/CE cluster, do not define these devices as Not Ready.

You can list information about a Symmetrix array, unlock features on a Symmetrix array, or replace a remote Symmetrix array with another, by right-clicking on the Storage icon and selecting Symmetrix Options (Figure 5-47 on page 5-48). The Symmetrix options include List, Unlock, and Replace Remote Storage Array.


5-48


Figure 5-47 Storage right-click menu

The List option displays statistics about a particular Symmetrix array, as shown in Figure 5-48.

Figure 5-48 Listing Symmetrix information



The Unlock option unlocks any Symmetrix external locks currently in place. The Unlock option allows you to release the arbitration cluster address, arbitration lock, arbitration reserve lock, and the RDF lock (Figure 5-49).

CAUTION!Use Unlock with caution and only with a thorough understanding of this function.

Figure 5-49 Unlocking Symmetrix arrays


5-50


The Replace Remote Storage Array option enables you to replace the remote (and only the remote) Symmetrix array. This function can be used to replace a Symmetrix array in the case of a physical site outage, a lease change, or other event where the remote Symmetrix array must be replaced. Replacing the remote array does not impact the running lateral nodes, but it does presume the peer nodes on the remote side of the SRDF link are down. If you select this option, the dialog boxes shown in Figures 5-50 through 5-52 will appear.

Figure 5-50 Selecting the remote Symmetrix array

Figure 5-51 Configuring the remote storage array

Figure 5-52 Completing the replacement of the remote storage array



If you choose to replace the remote storage array, complete the following steps:

1. Ensure the remote nodes are down.

2. Recable/rezone the new remote Symmetrix array to the remote nodes.

3. Reconfigure the new remote Symmetrix array to be the SRDF pair for the existing local Symmetrix array and all necessary devices.

4. Verify on the local side the necessary R1/R2 mappings have occurred.

5. Run Replace Remote Storage Array. This will validate the R1/R2 mappings on all local nodes. Ensure the local Symmetrix array is connected and configured for all devices to the remote Symmetrix array. This will update all local nodes with the new remote Symmetrix array information.

Once you have successfully completed the Replace Remote Storage Array function, perform a Pull Configuration operation for each remote peer node to synchronize the remote nodes back into the SRDF/CE cluster.

CAUTION!Use Replace Remote Storage Array with extreme caution and only with a thorough understanding of this function.


5-52


Nodes componentThe Nodes component lists the current set of SRDF/CE cluster member nodes (not necessarily the MSCS cluster member nodes). For each node listed as part of the SRDF/CE cluster, the status of the SRDF/CE Service and SRDF/CE COM Server are listed (either running/stopped). The IP address and configuration timestamp for each SRDF/CE cluster member node are also listed (Figure 5-53).

Figure 5-53 Expanded node view



Add Cluster Member NodeTo add a new cluster member node, select Add Cluster Member Node from the Nodes right-click menu (Figure 5-54). If SRDF/CE cannot find a specified host, you are presented with the option to do a Force Add of the host. The Add Host Node dialog box appears as shown in Figure 5-55.

Figure 5-54 Nodes right-click menu

Figure 5-55 Adding a host node

Nodes component 5-53

5-54


As shown in Figure 5-56, by right-clicking on a specific cluster member node name, you can:

◆ Start and stop a cluster member node’s SRDF/CE Service.◆ Perform an IP discover.◆ Edit IP settings.◆ Push or pull configuration to/from peer nodes. Pushing the

configuration to all nodes unconditionally forwards the configuration from the local node to all other peer nodes and resets their configuration timestamps to equal that of the local node.

Note: Pushing or pulling configurations from one node to another overwrites the entire configuration of the target node.

Figure 5-56 Individual peer node right-click menu



Update Lateral/Peer DesignationsIn the event of a manual configuration change, GeoSpan to SRDF/CE upgrade, or other events it is sometimes necessary to update the SRDF/CE configuration to note what SRDF/CE nodes are lateral and which are peer to each other. This option causes all SRDF/CE nodes to update their view of this relationship. To do this, select Update Lateral/Peer Designations from the Nodes right-click menu (Figure 5-54 on page 5-53).

SRDF/CE warnings, or alerts, appear in the bottom-right corner of your desktop. For more information on SRDF/CE alerts, refer to Appendix A, “SRDF/CE Alerts and Errors.”

Nodes component 5-55

5-56



6Invisible Body Tag

This chapter describes administrative procedures for routine operations, maintenance, and troubleshooting of SRDF/CE. SRDF/CE, once installed and configured, requires minimal maintenance.

For those occasions requiring administrative intervention, refer to this chapter, which walks you through the procedures to ensure SRDF/CE operates properly in a complex environment.

◆ Overview.............................................................................................6-2◆ SRDF/CE unique cluster behavior..................................................6-4◆ SRDF/CE administration..................................................................6-5◆ Restore/recovery operations..........................................................6-10◆ Upgrade procedures ........................................................................6-17◆ Configuration troubleshooting ......................................................6-22◆ Errors and warnings ........................................................................6-28◆ Best practices.....................................................................................6-31◆ URLs to additional information.....................................................6-35

Administration

Administration 6-1

6-2

Administration

OverviewSRDF/CE is an EMC application that operates in a complex environment and works with other products such as:

◆ Microsoft — Microsoft Windows Server 2003 Enterprise and Datacenter Editions and Microsoft Cluster Service (MSCS) clusters

◆ EMC Corporation — EMC SRDF, EMC TimeFinder®, Volume Logix, PowerPath, and others

◆ User applications — SQL Server, Exchange, Lotus Notes, and others

Basic function The basic function of SRDF/CE is to relocate data volumes from one system to another, using an SRDF connection. When a volume is set offline on one server and made online on another server, some applications could potentially be impacted by this configuration change.

Likewise, if a server is taken offline to add new equipment, just recently purchased, and then brought back online in an SRDF/CE cluster, the potential exists for impact on MSCS and SRDF/CE operations.

Basic understanding Therefore, to perform administrative functions properly in this complex environment, you must fully understand:

◆ What the functions are doing.

◆ How they may interact with the operations of SRDF/CE.

Note: Chapter 1, “Introduction,” addresses the concepts pertaining to SRDF/CE operation.

That level of knowledge must be combined with an understanding of how other products work together in the complex environment. An example of what administrators of an SRDF/CE cluster should know or recognize is you cannot perform an EMC SRDF commit together with failover/failback procedures.

This chapter is the first step in helping to address those issues.


Administration

SRDF/CE Configuration UtilityAs discussed in Chapter 5, use the SRDF/CE Configuration Utility (refer to Figure 5-1 on page 5-2) to set up and configure SRDF/CE and to control SRDF disk configurations and MSCS resource definitions.

Note: Chapter 5, “SRDF/CE Configuration Utility GUI,” describes the components of the SRDF/CE Configuration Utility. Refer to this chapter prior to making any configuration changes. This information is also documented in the Help utility.

Using Cluster Administrator and the Configuration UtilityCluster Administrator Use MSCS Cluster Administrator for all normal cluster operations.

SRDF/CE ConfigurationUtility

Initially, the SRDF/CE Configuration Utility is used to set up and configure SRDF/CE during installation. Thereafter, use the SRDF/CE Configuration Utility to reconfigure resources. This utility can also be useful in troubleshooting certain error conditions.

Multicluster configurationsMultiple SRDF/CE cluster environments can be supported by a single pair of Symmetrix systems. Anyone who administers the cluster must be aware if several nodes fail simultaneously, the SRDF/CE resource groups fail over one at a time. This is normal SRDF/CE behavior.

Note: The failover of resources from different clusters can be intermixed.

Overview 6-3

6-4

Administration

SRDF/CE unique cluster behaviorPhysical differences between local and SRDF/CE clusters introduce the possibility for failure and operating modes impossible with the other configuration. For example:

◆ With local clusters use shared physical disks, destruction of physical storage while connected to one node eliminates the possibility of the other node picking up the device and operating successfully.

◆ With SRDF/CE clusters, the remote node can resume operation using the copy of the destroyed device data.

SRDF/CEconsiderations

When a site failure occurs, determine:

◆ Whether the other node is actually dead (as can be determined by network and SRDF links).

◆ Whether you want this second copy of the data to be modified.

Example In the event that both nodes in a two-node configuration are still operational but cannot communicate, two distinct copies of data— each modified at a different host by a different application—become possible. This situation is not possible under normal clusters. To work around this fragile situation, proceed carefully.

For planning purposes, decide:

◆ How well the sites and their interconnects are protected.

◆ How important is it to stay online and operational versus the cost of having multiple modified copies of data.

Additional application-specific resources provide additional dependencies. SRDF/CE operates using MSCS defined rules. For example, in the special case of a site failure, care must be taken to determine which type of site failure occurred (refer to “SRDF/CE unique behavior” on page 2-5) before proceeding with the recovery actions.


Administration

SRDF/CE administrationThis section describes the following administrative procedures:

◆ “Stopping the Cluster Service” on page 6-5

◆ “Uninstalling the Cluster Service on a Windows Server 2003 cluster” on page 6-5

◆ “Manually starting the Cluster Server Services” on page 6-6

◆ “Modifying cluster groups” on page 6-7

◆ “Renaming groups” on page 6-7

◆ “Configuring additional disk devices” on page 6-8

Stopping the Cluster Service To stop the Cluster Service:

1. Take all groups except the Quorum group offline.

2. Exit Cluster Administrator.

3. Stop the Cluster Server Service on the node without the Quorum group.

4. Stop the Cluster Server Service on the remaining node.

Note: If the MSCS service fails to stop, go into the task manager and stop the clussvc.exe process and any resrcmon.exe processes.

Stopping the SRDF/CE Service is optional anytime after the Cluster Service is stopped on a node.

Uninstalling the Cluster Service on a Windows Server 2003 clusterIn Windows Server 2003, the Cluster Service is installed along with the operating system. You cannot remove the Cluster Service, but you can return it to an unconfigured state. For more information, refer to the Microsoft Knowledge Base article at:


SRDF/CE administration 6-5


6-6

Administration

In summary, the steps to uninstall the Cluster Service on a Windows Server 2003 cluster are:

1. Start Cluster Administrator (CluAdmin.exe).

2. Right-click the node, and select Stop the Cluster service.

Note: Do not perform this next step if this server is the last node in the cluster.

3. Right-click the node, and select Evict Node. This step returns the cluster to its original unconfigured state. You can add it later to the same or to a different cluster.

If you cannot start the Cluster Service, or if you have trouble removing the node, you can manually deconfigure the Cluster Service by doing the following:

1. Run the Cmd.exe program to open a command prompt.

2. At the command prompt, type cluster node nodename /forcecleanup, and then press Enter.

Note: If the Cluster Service does not exist in the Registry, the command does not respond. To create a place holder, type sc create clussvc at the command line, and then press Enter.

Manually starting the Cluster Server ServicesTo manually start the Cluster Server Services:

1. Boot all nodes

2. Ensure the SRDF/CE Service started on all of the nodes. If you stopped the SRDF/CE Service on any nodes, start the SRDF/CE Service on all nodes.

3. Manually start the Cluster Server Service on the node with the Quorum last (the last node shut down in the previous procedure).

4. Wait for the MSCS Service startup to complete, and then open Cluster Administrator to connect to the cluster. Verify the cluster is operational.

5. If not already started, start the SRDF/CE service on those particular nodes remaining in the cluster. Then start the Cluster Server Service on those nodes.


Administration

6. Use Cluster Administrator to move the groups back to their R1 sides. This happens quickly because no SRDF operation is involved.

7. Bring all groups online.

Modifying cluster groupsIt may be necessary to move devices from one group to another occasionally. Within MSCS, right-click a resource and select the Change Group option to move the disk to another group. When SRDF/CE is installed into the cluster, the move operation is complicated by introducing SRDF and SYMAPI groups. To move a disk from one group to another without updating the SYMAPI database, the moved device would not fail over when commanded to do so.

If you want to remove or delete one disk from a group that contains many disks, do the following:

1. Within the SRDF/CE Configuration Utility, open the Groups component and click the group containing the device in question. In the right pane listing all disks, right-click the disk to remove, and select Delete. This will remove the disk and the corresponding GeoSpan_res resource from this group as well as from MSCS.

2. Now add this device to another group.

Renaming groupsRenaming a cluster group name affects the functionality of the SRDF/CE product. If you rename a MSCS cluster group, you can failover/back without issue — but only if the new group name adheres to the SRDF/CE naming standards. The only time you will experience problems is if you modify the disk group in any way. If you deconfigure the group or add a disk to this group you will experience problems if the cluster group name and the SRDF/CE group name do not match. If you need to permanently rename a group, it is best to deconfigure the group, and then re-create it with the new name.


6-8

Administration

If you do rename a group within an MSCS cluster and later forget the original name , look at the name of the GeoSpan_res resource. Its name will not change unless you deconfigure it through the SRDF/CE Configuration Utility.

Configuring additional disk devicesThis procedure assumes you added the disk to the SCSI bus. If you have not, please do so before continuing.

This procedure describes configuring additional disk devices for a two-node system. If you have more than two nodes, perform the steps described for Node 2 on all the remaining nodes in the cluster.

1. Disable the Cluster Service and reboot all nodes after the storage is added to all nodes. The reason this reboot is necessary is that MSCS does not read a node’s device information except at boot time and we need to add a new disk definition without MSCS involved.

2. After the reboot, use the syminq command on all nodes to verify you can see the new storage.

3. Open the SRDF/CE Configuration Utility on all nodes and right-click EMC and select Discover. This updates the SRDF/CE SYMAPI database to see the newly added device.

4. Within the SRDF/CE Configuration Utility on Node 1, create a temporary group for the newly added disk. Use this temporary group to move the disk between nodes to get the disk signature and drive letter assignments synchronized on both nodes. Move the disk group to the Node 1 side. Once this is done, open Disk Administrator and create a new partition for the device. Format, label, and assign a drive letter. You may need to delete/modify the disk key to make this disk appear normally. Do not proceed to the next step until the disk is formatted and labeled.

5. Once you are done on Node 1, use SRDF/CE to move the disk group to the Node 2 side. Once complete, open Disk Administrator on Node 2 and assign the proper drive letter to the new device. You may need to delete/modify the disk key to make this disk appear normally. If you must delete the disk key, ensure you reassign drive letters to all devices. Do not proceed to the next step until you can see the disk with the proper drive letter on each side.


Administration

6. Once the temporary group is transitioned to all nodes, open the SRDF/CE Configuration Utility on Node 1 and delete the temporary group created for the device.

7. On all nodes, reconfigure the Cluster Service to Automatic startup and reboot.

8. To create a new SRDF/CE group or add a disk to an existing group, use the SRDF/CE Configuration Utility.

9. Test failing over the new group between nodes to ensure the configuration is correct.

10. Once you add a new device, update your Checkpoint file within the SRDF/CE Configuration Utility. Do this by right-clicking Control and selecting SRDF/CE Config Options, Check Point.


6-10

Administration

Restore/recovery operationsThis section describes the following restore and recovery operations:

◆ “Restoring from site failure”

◆ “Recovery from site failure” on page 6-12

◆ “Recovery from total IP communications failure” on page 6-13

◆ “Recovery from HBA or SCSI channel failure” on page 6-14

◆ “Recovery from an SRDF link failure” on page 6-14

◆ “Recovery from a corrupt quorum log” on page 6-15

◆ “Symmetrix replacement from the R1 side” on page 6-15

Restoring from site failureThe following procedure describes how to restore your storage system after a site failure occurs with all links lost (that is, with one side down).

1. Restore the SRDF link.

2. Open the SRDF/CE Configuration Utility and verify the groups moved are in a Split state. All other groups should be Synchronized or Failed over.

3. Go to a command prompt and enter:

set symcli_db_file=SRDF/CE home directory\SRDFCESymapi.db

4. At the command prompt, enter:

symdg list

This should list the SRDF/CE groups.

5. Choose one of the groups that are in a Split status and enter the following:

symrdf –g GroupName query

This should report the status of the disks within this group.

CAUTION!Contact EMC Customer Support before proceeding to the next step.


Administration

6. Before you establish/restore the disks, take the disk group offline within Microsoft Cluster Administrator.

7. To get the disks out of the Split state, decide which side of the R1/R2 pair has the most recent data. Using the wrong command here causes data loss. To synchronize the RDF devices, use one of the following SYMCLI commands:

• If the R1 owns the good data, enter: symrdf –g GroupName establish

• If the R2 owns the good data, enter: symrdf -g GroupName restore

Note: This process could take significant time, as this is a full establish/restore.

8. After you establish/restore the disk, it takes time to resynchronize the drives. To check the status of the establish/restore, enter:

symrdf –g GroupName query

9. When the devices are done with the establish/restore, the devices are left in a synchronized state. This means the device is read/write-enabled on the R1 side and write-disabled on the R2 side. Any devices you did a restore operation on have moved to the R1 side. Use the SRDF/CE Configuration Utility to perform an SRDF failover operation on these device groups to bring them back to the R2 side. Access the SRDF Failover option in the Configuration Utility by right-clicking any group under the Groups component and selecting SRDF/CE Group Operations, Array Operations, SRDF Failover.

10. Before starting the Node 1 services, ensure all the disks are located on the surviving node within SRDF/CE. For example, if the surviving node is the SRDF/CE slave node, then all R1 devices should read Synchronized and all R2 groups will be in a Failed over status.

11. Restore the previously dead node.

12. Start the SRDF/CE Service and then the Cluster Server Service.

13. Use MSCS to move the groups back to their original locations.

Restore/recovery operations 6-11

6-12

Administration

Recovery from site failureThe following procedure describes how to recover from a site failure with all links lost (that is, with one side down).

1. Open the MSCS Administrator on the surviving node and verify which groups have failed. If the quorum group failed, it must come online first before the failover of the other groups can happen.

2. Decide which groups should be brought online on the surviving node. All groups do not have to be brought online. If you do not want to bring any groups online simply restore the SRDF connection and recover from there. Otherwise, use the following process to move groups to the surviving node:

a. Using SRDF/CE, set the SRDF Override to Yes.

b. Use MSCS to bring the quorum group online. The GeoSpan_res may fail to come online. This is acceptable behavior and you should just bring this resource online. Do not proceed until both the quorum disk and the GeoSpan_res are online. On Windows 2000, if you lost the node that owned the quorum disk, you must attempt to restart the Cluster Service to bring the groups online. If this does not work, you must do the following from a command line:

– To set SYMCLI to see the SRDF/CE API database, enter:set symcli_db_file=SRDF/CE home directory\SRDFCESymapi.db

– Use symdg list to verify you can now see the SRDF/CE disk groups

– For each device group, enter the following command:symrdf -g GroupName rw_enable R1|R2

Select R1 or R2 depending on which device is local to this machine.

– Once all devices are read/write enabled on this node, start the Cluster Service

c. Once the quorum is online, bring each group online one at a time. Wait for each resource within a group to completely come online before proceeding to the next group.


Administration

d. Once all groups are online on the surviving node, use SRDF/CE to set the SRDF Override to No.

Note: If Site Mode is set to FailStop, change the Site Mode to No New Onlines or leave the SRDF Override on. Otherwise, the groups go offline when the override is turned off.

Recovery from total IP communications failureWhen all IP communications between nodes are lost, all Cluster groups fail over to the node that owns the quorum device. Additionally, in Windows 2000 only, if the node that owned the quorum no longer has access to any IP communications (NIC failure/switch failure) all IP resources within the cluster fail and any resources that depend on the IP (network names and so on) go offline.

To recover from this scenario:

1. Stop the Cluster Service on the node(s) that does (do) not own the groups.

2. Restore IP communications.

3. Bring all groups online.

4. Start Cluster Service on the node(s) without the groups.

To change group ownership while the IP links are down, do the following:

1. Stop the Cluster Service on the node(s) that does (do) not own the groups.

2. Stop the Cluster Service on the other node(s).

3. Start the Cluster Service on the node(s) you want to bring all groups online.

You cannot have groups active on more than one node during total IP communications failure.


6-14

Administration

Recovery from HBA or SCSI channel failureWhen one node loses connection to the storage, whether it is cable, HBA, or some other type of channel failure, all groups on the node with the channel failure will fail over to the surviving node. Also, the SRDF/CE Configuration Utility will display an SRDF warning on the node with the failure.

To recover from this type of failure:

1. Stop the Cluster Service on the node with the channel failure.

2. Restore connection to the storage.

3. If applicable, reopen the paths to the storage within PowerPath.

4. Once the connection is reestablished to the storage, the SRDF warning within SRDF/CE should go away.

5. Start the Cluster Service.

Note: If the disks are not visible, issue a bus rescan from within the disk scan utility, then issue a Windows dcomcnfg remove command and the manual SYMCLI discover command, symcfg disc. If the disks are still not visible, perform a bus rescan again. It might be necessary to reboot before starting the Cluster Service.

Recovery from an SRDF link failureIf the SRDF link goes down, one of the following happens depending on site mode settings:

◆ Site Mode set to No New Onlines

• SRDF/CE detects the SRDF failure and the SRDF Warning displays within SRDF/CE.

• All MSCS groups remain online and function normally. SRDF/CE-dependent resources go offline.

• If an MSCS group attempts to fail over, SRDF/CE does not allow it and the group goes offline. If you must fail this group over, use the procedure from “Recovery from site failure” on page 6-12 to move the group. The group is in a split state if it is moved.

◆ Site Mode set to FailStop


Administration

• SRDF/CE detects the SRDF failure and the SRDF Warning displays within SRDF/CE.

• All MSCS groups go offline.

If you want to bring any groups online, set the SRDF Override to Yes and bring the groups online within MSCS. Bring the quorum group online first!

With the override set, you can fail over a group and this will cause the groups to be in a split state when the link is restored.

To recover from this failure:

1. Restore the SRDF link.

2. Once the SRDF link is restored, SRDF/CE should no longer display an SRDF warning.

If you moved any groups while the SRDF link went down, use the same recovery procedure described in “Restoring from site failure” on page 6-10 to get groups out of a Split state.

Recovery from a corrupt quorum log The complete Microsoft article can be found on Microsoft Knowledge Base Article 172951 at the following URL:


Symmetrix replacement from the R1 sideThe following process assumes the Symmetrix replacement is being done on the R1 side and there are no Dynamic RDF groups configured on either Symmetrix array.

1. Change the MSCS service start up to manual on all cluster nodes.

2. Ensure all groups are online on the R2 side, and shut down all cluster nodes on R1 and R2.

3. Replace the R1 Symmetrix and perform the appropriate configuration change to the R2 side as well.

4. Once the configuration change is made, and the new Symmetrix array is in place, bring the RDF link up.



6-16

Administration

5. After performing the appropriate zoning and device masking operations, bring up the R1 and R2 nodes. Open the snap-in and perform a Symmetrix discover (symcfg discover) on each node. Start the SRDF/CE service, starting with the surviving node. It is important the discover operation is performed prior to starting the service; failure to do so may result in MSCS malfunction.

6. On the R2 node (the surviving node), right click on the storage icon and do symreplace. Verify and select the new Symmetrix serial number, then click OK. Follow the next step the system prompts, to pull from the surviving node.

7. Using the SYMCLI, perform a full RDF restore for each group.

8. Once all groups are in synchronized state, start MSCS on the R1 node. Verify all groups came online. Then start MSCS on the R2 node.


Administration

Upgrade proceduresThis section describes the following upgrade procedures:

◆ Upgrading a node in an SRDF/CE cluster from Windows 2000 to Windows Server 2003 (refer to the next section)

◆ Upgrading a quorum disk (refer to page 6-20)

Upgrading Windows 2000 to Windows Server 2003The following procedure describes how to upgrade a node in an SRDF/CE cluster from Windows 2000 to Windows Server 2003.

This procedure describes an upgrade for a two-node system. If you have more than two nodes, perform the steps described for Node 2 on all remaining nodes in the cluster.

1. Verify you have moved all the available groups to Node 1 (the primary node).

Note: Verify the SRDF/CE and Cluster Services are set to Manual mode before you start upgrading the system.

2. On Node 2 (the secondary node):

a. Select Start, Programs, Administrative Tools, Services and stop the Cluster Service and SRDF/CE Service, respectively.

b. Close the Cluster Administrator and the SRDF/CE Configuration Utility.

c. Insert the Windows Server 2003 CD in the CD drive.

d. Select Install Windows Server 2003 Enterprise Edition.

e. Select Upgrade (Recommended) in the Welcome to Setup Windows dialog box, and click Next.

f. Select I accept the agreement in the License Agreement dialog box, and click Next.

g. Enter the product key provided and click Next.

Upgrade procedures 6-17

6-18

Administration

h. The Windows Server 2003, Enterprise Edition Setup dialog box appears as shown in Figure 6-1. In the Report System Compatibility window, take notice of any compatibility issues listed. If you do not want to correct any of the listed issues, click Next to continue.

Figure 6-1 Windows Server 2003 Enterprise Edition Setup dialog box

i. Once you have finished updating from Windows 2000 to Windows Server 2003, reboot the system and come up in Safe mode.

j. Once up in Safe mode, change the Cluster Service from Automatic start to Disabled.

k. Reboot normally.

l. You will not be able to see any attached devices. Change the HBA driver from the Windows 2000 version to the Windows Server 2003 version. Once you have changed the driver, reboot normally.

m. Once you validate all devices can be seen change the Cluster Service startup to Automatic and start the Cluster Service.

3. On Node 2:

After the node boots up, follow the procedure detailed below to edit the Volume Shadow Copy Service Access Control Registry Entry to grant access.


Administration

Important: Before attempting this procedure to modify the Registry, EMC recommends reading article number 256986 in the Microsoft Knowledge Base, which can be accessed at the following URL:


This article provides information on how to back up, restore, and edit the Registry. Before you modify the Registry, back it up and understand how to restore the Registry if a problem occurs.

CAUTION!Using the Registry Editor incorrectly can result in serious problems requiring you to reinstall your operating system. Microsoft cannot guarantee you will solve problems resulting from using the Registry Editor incorrectly. Use Registry Editor at your own risk.

a. Click Start, Run, type regedit, and click OK.

b. Locate the following Registry subkey:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\VssAccessControl

Note: If the VssAccessControl subkey does not exist, add it. To do this, right-click VSS, Point to New, and then click Key. In the New Key #1 dialog box, enter VssAccessControl, and press Enter.

c. Right-click VssAccessControl, select New, and click DWORD Value.

d. In the New Value #1 dialog box, enter example.com\cluster account (where example.com is the name of the domain where the cluster account is located, and cluster account is the name of the cluster service account), and press Enter.

e. Right-click the new Registry value you added, and click Modify. In the Value data box, enter 1, and click OK.

f. Quit Registry Editor.


6-20

Administration

4. On Node 2:

a. Select Start, Programs, Administrative Tools, Services and first start the SRDF/CE Service, and then start the Cluster Service.

Note: You can change the SRDF/CE Service and the Cluster Service to Automatic. If you do this, verify the Cluster Service is dependent on the SRDF/CE Service.

b. Open the SRDF/CE Configuration Utility by selecting Start, Programs, SRDF/CE Configuration Utility.

c. Move all groups present on Node 1 to Node 2.

Note: Verify all groups have been moved successfully to Node 2.

5. On Node 1:

a. Perform the same procedures from Steps 2–4.

b. Both nodes should be up and running SRDF/CE and Cluster Administrator, and you should be able to move the groups from one node to another.

Upgrading a quorum diskThe following procedure describes how to upgrade a quorum disk with SRDF/CE:

1. Create a temporary SRDF/CE device group that contains only the new quorum disk.

2. Perform an MSCS Move operation to each of the nodes. This will validate the new quorum disk can be successfully brought online to each node.

Note: At each node note the Window’s physical drive number.

3. Perform an MSCS Move operation on the temporary quorum disk group back to the starting node.

4. Delete the temporary new quorum disk from within SRDF/CE.

5. Add the new quorum disk to the current quorum disk group from within SRDF/CE.


Administration

6. Change the quorum disk from within MSCS, using the published Microsoft MSCS procedures for changing the quorum disk. It is critical you follow this procedure:

a. It is recommended you perform the procedures on all the nodes on one side of the SRDF link, and then move to the other side of the SRDF link.

b. When you start working on the nodes on the remote side of the SRDF link, you will have to perform an SRDF failover of the quorum group.

7. When you have fully configured MSCS with the new quorum disk, go to each SRDF/CE node in the cluster, and use the SRDF/CE Configuration Utility to do the following:

a. Open the Control component.

b. Click Quorum Control.

c. In the right pane, right-click any of the displayed Quorum Control parameters, and select Edit Quorum Control Parameters.

d. Change the quorum disk parameters to match exactly what was specified to MSCS.

CAUTION!This is a critical step. If it is not followed your cluster may not be functional.

8. Reboot all nodes in the cluster.


6-22

Administration

Configuration troubleshooting

Overview This section troubleshoots the following configuration issues you may encounter while using SRDF/CE:

◆ “SRDF/CE group in wrong state (not ready/invalid/suspended)” on page 6-23

◆ “Unable to start cluster service” on page 6-24

◆ “GeoSpan_res.dll prevents MSCS group failover” on page 6-25

◆ “GeoSpan_res.dll stays offline after failback of the quorum group” on page 6-25

◆ “External SYMAPI locks” on page 6-25

◆ “Unable to see a newly added disk” on page 6-26

EMC technicalsupport

For technical support relating to SRDF/CE, contact EMC Customer Support.

Microsoft technicalsupport

The first step you should take in troubleshooting a Microsoft product is to reference the Microsoft Knowledge Base for possible solutions. Access the knowledge base online at:

http://support.microsoft.com

Another excellent resource for troubleshooting MSCS problems is the Microsoft Cluster Server Troubleshooting and Maintenance Guide. This guide is published at:

ftp://ftp.microsoft.com/bussys/winnt/winnt-docs/papers (download the mscs.exe file)


http://support.microsoft.com

ftp://ftp.microsoft.com/bussys/winnt/winnt-docs/papers/mscs.exe

Administration

SRDF/CE group in wrong state (not ready/invalid/suspended)When you create groups, if the status of the group is not synchronized, failed over, or R1 updated, the state is wrong. This is usually caused by an incomplete SRDF configuration. To fix this:

1. Set the SYMAPI database to point to the SRDF/CE database.

2. From a command prompt, enter:

SET SYMCLI_DB_FILE=SRDF-CE home directory\SRDFCESymapi.db

3. Verify the above command worked by typing SYMDG LIST. This should list the groups you created in SRDF/CE.

4. View the status of the groups. For example, to check the status of the group R1_Group (this is case-sensitive) enter:

symrdf –g R1_Group query

This command displays information about the disks within this group. To the far right, it displays the status of the devices. If the status here is anything other than synchronized, failed over, or R1 updated, issue the following command: symrdf –g R1_Group –force resume

5. Once you issue the command, check the group status by typing:

symrdf –g R1_Group query

You should see the devices begin to synchronize.

6. Once the group is completely synchronized or failed over, repeat Steps 3–5 for each device group in an improper status.

Configuration troubleshooting 6-23

6-24

Administration

Unable to start cluster serviceThere are several reasons why the MSCS service may not start:

◆ Attempting to start the Cluster Service on a node without access to the quorum.

◆ The quorum disk or the quorum log is corrupt.

◆ The cluster is in split-brain1 status.

◆ SRDF/CE Service must be started before MSCS. If it is not started, MSCS may not start.

To start Cluster Service:

1. Shut down the services on one node.

2. Power down that node.

3. On the surviving node (s), stop the MSCS and SRDF/CE Services if started.

4. Open the SRDF/CE Configuration Utility and move all disks to this side.

5. Start the SRDF/CE Service, and then start the Cluster Service. The services should come online for this node. If not, try rebooting this node.

6. Once the services have started, power on the other node and start its services. SRDF/CE must be started before MSCS.

7. If the above does not work, you may need to reinstall MSCS on all nodes. If you do reinstall, ensure you format the quorum disk prior to installation to remove all references to the previous cluster.

1. A total communication failure, while both nodes remain operational, is referred to as a split-brain condition and is a potential cause of logical data corruption. For example, if both sides assume the other is dead and begin processing new transactions against their copy of the data, two separate and unreconcilable copies of the data can be created.


Administration

GeoSpan_res.dll prevents MSCS group failoverWhen you complete the SRDF/CE installation, move a group within Cluster Administrator. When the Geospan_res attempts to come online, it tries to restart according to the threshold set, and then fails back to its original node. If the MSCS group does not fail over, an error occurred with the srdfce_svc service account.

Check the setting of the srdfce_svc service and ensure all nodes are using the same service account. Ensure that this account is a member of the domain administrator’s group as well as the local machine administrator’s group. Also check to see if this account has log on as a service rights.

GeoSpan_res.dll stays offline after failback of the quorum groupAfter SRDF/CE and MSCS are installed and properly configured, you attempt to move the quorum group between the cluster nodes. When you move the group, the quorum disk comes online, but the SRDF/CE resource remains offline. This is caused by a timeout value set within Microsoft cluster we have no control over for failover of the quorum disk. This is resolved by using a smaller physical quorum disk, or by simply deleting this resource from the quorum group. SRDF/CE uses the information contained within the Configuration Utility under Quorum Control first when failing over the quorum resource. The SRDF/CE resource is only used if the information placed here is incorrect. This resource can be deleted without having any affects on product functionality.

External SYMAPI locksUnder certain circumstances, the SYMAPI may leave the Symmetrix system in a locked state. The symptoms indicating you are in a locked state may include the following: unable to fail over devices, failure to move groups, or failure to start MSCS service. To check if you are in a locked state, go to a command line and enter the following:

symcfg -lock list

This command should echo the local and remote Symmetrix systems and their lock status (locked/unlocked). If you are in a locked state, identify the lock numbers to release these locks.


6-26

Administration

To locate the lock number, do the following:

1. In the SRDF/CE Configuration Utility, right-click Symmetrix and select Discover.

2. Once a successful Discover is made, right-click Symmetrix and select List. After a few moments, this displays a long list of many things within the Symmetrix array. Scroll through the document until you reach the Lock section (near the top). The Lock section runs through each bus and specifies lock information about each one. The typical report states No lock for 0 SYMAPI_C_SYM_LOCK_NOTFOUND and a lock displays lock information along with the length of time for the lock. Find which device number has the lock and record this number (typically 0 or 15).

To release the locks, enter the following from a command line:

symcfg -lockn LockNum release

Note: You can also unlock the Symmetrix array using the SRDF/CE Configuration Utility by right-clicking Symmetrix and selecting the Unlock option.

If you are unsuccessful, add the -force flag to force the release. After unlocking the Symmetrix array, verify the lock is released by typing symcfg -lock list. Once both Symmetrix arrays are unlocked, your issue should be resolved.

Unable to see a newly added disk Microsoft Cluster may not allow you to see the newly added device. This causes problems when you attempt to assign drive letters or manipulate the disk signature. To get cluster to release its grasp on the disk, do one of two things:

◆ Uninstall MSCS on one node, or

◆ Stop the Cluster Disk service on the Control Panel, Devices list. You can stop this by setting the startup value to manual and rebooting.

If you do not have the disk listed when you are adding the disk resource to your group, do the following:

1. Evict one node from the cluster.


Administration

2. Uninstall MSCS from this node.

3. Configure the disk on this node.

4. Reinstall MSCS software to join the cluster.

5. Reapply service packs and hot fixes.

6. Add the disk resource to your group.

7. Add the dependency to this disk resource.

Note: MSCS operates on the principle all storage is local in nature. Because of this limitation, MSCS must see all devices on all nodes in the cluster. To accomplish this, when you set up the Symmetrix devices to be used by the SRDF/CE cluster, do not define these devices as Not Ready.


6-28

Administration

Errors and warningsSRDF/CE is a service that depends on MSCS and the EMC Symmetrix API (SYMAPI) to function. There are few events caused and/or generated by the SRDF/CE Service. Virtually all SRDF/CE errors can be traced to problems with MSCS or the SYMAPI. Therefore, this section is not intended to be a troubleshooting guide or a definitive reference for the Windows event and application logs. This section explains the following errors/warnings you may encounter:

◆ “GeoSpan_res.dll errors”

◆ “SRDF warning” on page 6-29

◆ “IP communication warning” on page 6-29

◆ “Error while attempting to connect with MSCS” on page 6-30

◆ “MSCS error 170” on page 6-30

For more information on the alerts, event messages, and errors you may encounter while using the SRDF/CE Configuration Utility graphical user interface, refer to Appendix A, “SRDF/CE Alerts and Errors.”

GeoSpan_res.dll errorsWhen you install the GeoSpan_res.dll, you may receive an error within MSCS. To determine the severity of this error:

1. In MSCS, click the Resource Types folder and locate the geospan_res resource.

2. If the resource has a red circle around it, uninstall the resource and try reinstalling it.

3. If the resource fails the second time you install it, uninstall it again and attempt to install it from the opposite node. This may fix the problem.

Note: If the above did not work, check the path to the SRDF-CE home directory. These paths must be exactly the same for this resource to work properly. If you installed one operating system on the C:\ drive and the other on the D:\ drive, this will not work. The %systemroot% path also needs to be exactly the same on all nodes before you can continue with the SRDF/CE installation.


Administration

Important: If you get an error and the geospan_res appears to be operational, make a note of the error and continue with the installation. It is possible to experience false errors at this point; MSCS gets confused.

SRDF warning If the SRDF warning appears, the configuration is not correct. Check the following:

1. Ensure the SRDF/CE Service is started on both sides. This error will appear if the service is not started.

2. Check the cables. Check the SRDF connection as well as the connection to the HBA.

3. From each host, issue a symrdf ping command. If you can ping from both sides, you should have a valid connection.

4. If this does not resolve the problem, contact the EMC Customer Support for further assistance.

IP communication warningMany things can cause an IP communication warning error. Check the following:

1. Check the SRDF/CE and the Cluster Service are running. If not, this error appears.

2. Ping all adapters from each node.

3. If the IP communications warning is displayed on one side only, ping all adapters on the opposite node to verify connectivity.

If you can ping, this message is in error. This is a known issue and can only be cleared by a series of reboots.

Errors and warnings 6-29

6-30

Administration

Error while attempting to connect with MSCSMSCS uses a remote procedural call (RPC) to connect to the cluster. You see this error if the RPC service has not fully started. If you receive this error, do one of three things:

◆ Wait a few minutes and try again.

◆ Try connecting to the IP address of the cluster instead of the NetBIOS name.

◆ Enter a period (.) to connect using LPC to the local node as opposed to the cluster name/address.

Note: This must be resolved before MSCS will operate properly.

MSCS error 170This error is issued when MSCS attempts to arbitrate for a device the opposite node already owns. In an SRDF/CE configuration, this can be caused by improper setup of the SRDF/CE arbitration numbers. Each cluster needs to use a different set of arbitration numbers. Each node within the cluster must use the same arbitration number.

If the cluster member nodes do not use the same numbers, you may not fail groups over to the opposite node. If MSCS logging is enabled, you will see the Error 170 message throughout the cluster log.

If two separate clusters attempt to use the same arbitration numbers, the first cluster to attempt to come online should do so, but the second cluster will fail to start the Cluster Service. If MSCS logging is enabled, you will see the Error 170 message throughout the log.

To correct this problem, ensure each cluster uses a unique set of arbitration numbers. Ensure each node within a cluster uses the same arbitration numbers.

Note: This error can also be caused by two or more nodes attempting to gain control of the device at the same time. This could happen if two or more nodes reboot at the same time. This is described in Microsoft KB article 249194, at: http://support.microsoft.com/kb/q249194.


Administration

Best practicesThis section describes the following best practices that pertain to both MSCS and SRDF/CE:

◆ “How to delay loading of MSCS service (change service dependencies)”

◆ “Enabling groups to come online while SyncInProg” on page 6-33

◆ “Changing the quorum log size” on page 6-33

How to delay loading of MSCS service (change service dependencies)Please read the following article before attempting to implement this procedure:


1. Open regedt32.exe and find the following Registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ClusSvc

2. Select the subkey and in the right-hand side, double-click the DependOnService field.

3. When the Data dialog box appears, enter the name or names of the services you prefer to start before this service with one entry for each line. Enter srdfce_svc, and click OK. Ensure you do not delete the current values.

4. Close the Registry Editor.

Note: The use of any type of fault-tolerant adapter or teaming for the heartbeat is not recommended. If you require redundancy for your heartbeat connection, use multiple network adapters set to Internal Communication Only and define their network priority in the Cluster configuration. Microsoft also does not recommend using a multiported network adapter. Please refer to your NIC manufacturer for information as to its compatibility on a Server Cluster.

5. Click Internet Protocol (TCP/IP), and select Properties.

Best practices 6-31

6-32

Administration

6. On the General tab, verify you have selected a static IP address not on the same subnet or network as another one of the public network adapters. An example of good IP addresses to use for the private adapters is 10.10.10.10 on Node 1 and 10.10.10.11 on Node 2 with a subnet mask of 255.255.255.0.

7. Ensure there is no value set in the Default Gateway box.

8. Verify there are no values defined in the Use the following DNS server addresses box.

9. Click Advanced.

10. On the DNS tab, verify there are no values defined. Ensure the Register this connection's addresses in DNS and Use this connection's DNS suffix in DNS registration checkboxes are cleared.

11. On the WINS tab, verify there are no values defined. Click Disable NetBIOS over TCP/IP.

12. When you close the dialog box, you may receive the following prompt. If you receive this prompt, click Yes:

This connection has an empty primary WINS address. Do you want to continue?

13. If you are using a crossover cable for your private heartbeat interconnect, disable the TCP/IP stack destruction feature of Media Sense.

Add the following Registry value to each node:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\ParametersValue Name: DisableDHCPMediaSenseData Type: REG_DWORDData: 1

For additional information about this, refer to Microsoft Knowledge Base article 254651 Cluster Network Role Changes Automatically at the following URL:


14. Complete the previous steps on all other nodes in the cluster.

15. Start Cluster Administrator.

16. Click the cluster name at the root of Administrator. On the File menu, select Properties.


Administration

17. On the Network Priority tab, verify the private network is listed at the top. If it is not, use the Move Up button to increase its priority.

18. Click the private network, and then click Properties.

19. Select the Enable this network for cluster use checkbox.

20. Click Internal cluster communications only (private network).

Enabling groups to come online while SyncInProgIf you are experiencing long failover times and want to speed up the failback process, manipulate the Registry to allow the cluster groups to come online while the SRDF devices are still being synchronized.

To do so, open up the Registry Editor and modify the following key:

HKEY_LOCAL_MACHINE\SOFTWARE\EMC\SRDF/CE for MSCS\Config\WaitTargetState

The Registry item WaitTargetState controls whether to allow devices to come online while in the SyncInProg state. Default value 0 means do not come online unless the devices are synchronized. Setting this value to 1 allows the devices to come online before the devices are fully synchronized. Restart the SRDF/CE Service for this to take effect.

CAUTION!Not waiting for synchronization can be very dangerous and is not recommended. If you lose the RDF links, a host, or a site during the resynchronization process and you attempt to fail over while the SyncInProg is still in progress, data loss can occur.

Changing the quorum log sizeThe default quorum log size of any cluster is 64 kilobytes (KB). If you increase the number of shares or transactions significantly, the quorum log size may be too small.

You might need to adjust the quorum log size when you use a large number of file shares. For example, an administrator adds new file shares that work properly, and then a failover occurs. The file shares are no longer displayed in a cluster.

Best practices 6-33

6-34

Administration

You may receive an event message indicating the quorum log size is too small.

To change the maximum quorum log size:

1. Open Cluster Administrator.

2. In the console tree, click the cluster. On the File menu, select Properties.

3. Click the Quorum tab.

4. In Reset quorum log at, enter the maximum size (in kilobytes) that the quorum log file can be before the cluster resets it.

Note: You do not need to restart the computer or service unless you are running a third-party tool, such as a quota or virus protection program, which might prevent the change from occurring.

Note: The cluster records all changes to the cluster database in the quorum log file. When the quorum log attains the specified size, the cluster saves the database and resets the log file.


Administration

URLs to additional informationThe following URLs provide additional information.

◆ Microsoft’s Cluster website:

http://www.microsoft.com/ntserver/ProductInfo/Enterprise/default.asp

◆ Microsoft Cluster Server Troubleshooting Guide (download):

ftp://ftp.microsoft.com/bussys/winnt/winnt-docs/papers/mscs.exe

◆ MSCS Installation Resources:


◆ Recommended WINS Configuration for Microsoft Cluster Server:


◆ How to Enable MSCS Cluster Logging:


◆ How to Change Service Dependencies (Delay Loading of MSCS Service):


◆ How to Add Disks to the Shared SCSI Bus:


◆ How to Enable File Auditing in MSCS:


◆ Replace Cluster Disks:


◆ Installing Exchange 5.5 into MSCS:

http://www.microsoft.com/exchange/en/55/help/default.htm

◆ Installing SQL into MSCS


◆ Installing IIS into MSCS:


◆ Configuring a File Share for MSCS:


URLs to additional information 6-35

http://www.microsoft.com/ntserver/ProductInfo/ Enterprise/default.asp

ftp://ftp.microsoft.com/bussys/winnt/winnt-docs/ papers/mscs.exe

http://www.microsoft.com/exchange/en/55/help/default.htm


6-36

Administration

◆ Configuring a Printer Queue for MSCS:


◆ Cluster Server Cannot Use Disk Beyond Device Number 25:


◆ How to Recover From a Corrupt Quorum Log:



7Invisible Body Tag

This chapter provides detailed instructions on and lists the requirements for manually installing and configuring SRDF/CE software. Only use a manual installation procedure if you are configuring a dynamic cluster. This chapter also describes how to uninstall SRDF/CE.

◆ System requirements .........................................................................7-2◆ Install considerations.........................................................................7-2◆ SRDF/CE manual installation procedure ......................................7-5◆ Uninstalling SRDF/CE....................................................................7-30

Manual Installationand Configuration

Manual Installation and Configuration 7-1

7-2

Manual Installation and Configuration

System requirementsRefer to “System requirements” on page 3-2 for a list of the hardware, software, and EMC firmware requirements for an SRDF/CE cluster.

Install considerationsThe manual installation procedure describes how to use the SRDF/CE Configuration Utility GUI to configure SRDF/CE. The SRDF/CE Configuration Utility GUI semiautomates the configuration process and provides an alternative to using Symmetrix utilities and SYMCLI commands.

Important: You should perform the manual installation and configuration procedures outlined in this chapter only if you are installing dynamic cluster with VxVM on your system. Table 7-1 on page 7-7 describes the manual installation/configuration procedure for a two-node system. Changes in the procedures for a multinode system are noted in Table 7-1.

If you are not installing a dynamic cluster, use the SRDF/CE Installation Wizard and the SRDF/CE Configuration Wizard, which are easier to use (refer to Chapter 3, “Automatic Installation and Configuration”).

The manual installation procedure assumes MSCS is not currently installed on your system. Do not install MSCS before installing SRDF/CE. If you do have an existing MSCS installation, use the Configuration Wizard.

Contacting EMC Customer Support is recommended for assistance if any of the following issues are applicable:

◆ You have applications layered on with dependencies.◆ You need other devices online.◆ You are not confident about installing and configuring new

software within the context of Windows Server 2003, MSCS, and Symmetrix with SRDF.



Prerequisites Refer to the following sections before manually installing and configuring SRDF/CE:

◆ “Prerequisites” on page 3-7

◆ “Before you begin” on page 3-8

◆ “Planning requirements” on page 3-12

PC emulation softwareAlthough not required, it is useful to have PC emulation software installed on both cluster nodes, particularly if they are in different physical locations. The manual installation procedure requires physically moving back and forth between the two nodes. Using remote access software to control both nodes from a remote location enables you to perform all the steps on one machine.

Rebooting At various points in the manual installation procedure, you will be instructed to restart (reboot) your system.

Important: Failure to reboot where the SRDF/CE installation procedure explicitly instructs you to do so will result in the SRDF/CE driver failing to load.

Verifying your actionsAfter completing significant steps in the manual installation procedure, EMC recommends testing the system to ensure your actions are accurately carried out. Significant steps include the following:

◆ Moving or copying files

◆ Creating or deleting device groups

◆ Adding or removing devices within groups

◆ Changing the status of SRDF devices (from synchronized to failed over, for example)

Use any of the following methods to verify your actions:

◆ Look in the folders to which you moved or copied files.

◆ Use MSCS Cluster Administrator or the SRDF/CE Configuration Utility GUI to view node or device configuration information.


7-4


◆ Use Windows Explorer to view details on disk drives, such as a drive’s current status.

CAUTION!Do not perform configuration changes unless you are familiar with the procedure and are aware of its possible impact on subsequent configuration steps.



SRDF/CE manual installation procedureSRDF/CE runs in the Windows Server 2003 (Enterprise or Datacenter Editions) environment.

Overview The installation process comprises the following phases:

◆ Preinstall required software and EMC firmware products◆ Create an MSCS Administrative Account for SRDF/CE◆ Install SRDF/CE◆ Initially configure SRDF/CE◆ Build temporary groups for SRDF devices◆ Remove the temporary groups◆ Install MSCS and start all nodes◆ Configure SRDF/CE resources◆ Verify the installation

For clarity, these procedures refer to the cluster nodes as Node 1 and Node 2. This is an arbitrary designation. However, to follow along, decide which one of your servers will act as Node 1 and Node 2. EMC recommends having the server:

◆ at your primary data center act as Node 1.◆ at your secondary (failover) site act as Node 2.

Important: To have SRDF/CE operate as documented on both nodes, explicitly follow the installation and configuration procedures for the appropriate operating system. Note that checkpoints are provided at the completion of major steps to help ensure the procedure is progressing correctly.

Preinstallation The required software and EMC firmware products (refer to “Software requirements” on page 3-3) must be preinstalled on the nodes that will comprise the SRDF/CE cluster. (Refer to the EMC SRDF/CE for MSCS Version 2.1 Release Notes for the required versions of these products.)

Note: If you are unfamiliar with these product installation procedures, contact your EMC Customer Support Engineer or refer to the following EMC publications: EMC ControlCenter Installation Guide and the EMC Solutions Enabler Installation Guide.

SRDF/CE manual installation procedure 7-5

7-6


Manually installing SRDF/CETo ensure SRDF/CE installs correctly and operates as intended, explicitly follow the steps described in Table 7-1 on page 7-7.

Once SRDF/CE is installed, use the SRDF/CE Configuration Utility GUI to complete the initial setup.



Table 7-1 Manual installation: Two-node installation procedure

Step On node Do the following

Step 1: Preinstallation

1 1. Check all hardware used in the SRDF/CE cluster environment is listed in the EMC Support Matrix. The EMC Support Matrix can be found at http://powerlink.EMC.com.

2. Ensure applications (such as Microsoft Exchange, SQL Server, Lotus Notes, and so forth) used in the SRDF/CE cluster are cluster-aware.

Note: Important: Any application-specific dependencies that apply to MSCS also apply to SRDF/CE.

3. Have MSCS software readily available.

4. Check that the current operating service packs and patches are installed.

5. Check that the appropriate supported browser is installed.

6. Check that the following EMC software is installed as documented in their corresponding installation guides, and the products are fully functional:

• EMC Solutions Enabler, which includes EMC SYMAPI • Symmetrix Remote Data Facility (SRDF)

Note: Refer to the EMC SRDF/CE for MSCS Version 2.1 Release Notes for the required software versions.

7. Do the following to ensure SRDF operates appropriately in an SRDF/CE cluster:

• Test SRDF without being part of a cluster.• Ensure SRDF is in synchronous or asynchronous mode and not

domino mode.• Get drive assignments.

For additional information, refer to the EMC SRDF documentation.

Node 1

Node 2



7-8


1

(cont)

8. Ensure:

• Symmetrix devices do not have locks on them.• Gatekeepers are configured and have signatures written on them

(refer to “Hardware requirements” on page 3-2).

Note: Verify there are a minimum of six gatekeepers per node. It is recommended gatekeepers be mirrored and for performance one gatekeeper be assigned per group.

• The volumes to be shared as clustered devices are basic. If they are not, convert these to basic. (Refer to MSCS documentation.)

• The R1 and R2 devices and gatekeepers are visible to each node and the disks are defined as either Read/Write (RW) or Write-Disabled (WD) depending on what side is active, and not Not-Ready (NR). You should also have a minimum of six gatekeepers assigned to each server that participates in the cluster.

9. Have the preinstallation worksheet (refer to Appendix A) readily accessible.

Table 7-1 Manual installation: Two-node installation procedure (continued)


Node 1

Node 2



Step 2: Create an MSCS administrative account for SRDF/CE (node 1)

2 On your domain controller, create a new domain administrative account with the appropriate privileges for administering MSCS and SRDF/CE, if one does not previously exist. The easiest way to do this is:

1. Manually create the domain administrator’s account.

2. Enter a password for this account.

Note: This password is required in Step 5.

3. Give the new account a descriptive name, such as srdfce-admin. You will use this account to install, configure, and administer both MSCS and SRDF/CE. In addition, this account will be used as the running account for SRDF/CE Services. This account must have the following privileges:


Note: To change the user credentials for SRDF/CE (not for MSCS), right-click the SRDF/CE for MSCS icon within the SRDF/CE Configuration Utility and select the Change User Credentials option. This option allows you to change the SRDF/CE logon credentials (refer to “SRDF/CE for MSCS icon options” on page 5-3 for more information).



DomainController


7-10


Steps 3-6: Begin Installing SRDF/CE (both nodes)

3 On all cluster nodes apply operating system maintenance:

For Windows 2003, apply at least Service Pack 1 on both nodes and reboot both nodes.

4 On all cluster nodes install SRDF/CE.

1. As outlined in the EMC Solutions Enabler Installation Guide, add the SRDF/CE runtime license key to the EMC Solutions Enabler licensing database using the symlmf command.

2. Run the SRDF/CE setup program from the installation CD.

3. When the Welcome dialog box appears, click Next.

4. Read the Software License Agreement and click Yes.

5. When the Start Copying Files dialog box displays, click Next to proceed with the installation.

6. In the Setup Complete dialog box, select Yes in response to the prompt I want to restart my computer now.

7. Click Finish.

5 On all cluster nodes, test whether SRDF/CE installed correctly with the correct privileges. Do this by:

1. Double-clicking the EMC SRDF-CE Configuration Utility desktop icon to invoke the SRDF/CE snap-in based Configuration Utility.

2. Double-clicking the SRDF/CE for MSCS icon and entering the Administrative ID information.

If this process succeeds, SRDF/CE is installed correctly. Do not exit the Configuration Utility at this point. Successive steps will use it.

Note: This step cannot be skipped because it validates the installation, registers the necessary services, and sets various default values needed later in the manual process.



Node 1

Node 2

Node 1

Node 2

Node 1

Node 2



6 On all cluster nodes, ensure Time Service is available for all members participating in the cluster by issuing the command net time \\node-1 /set /y for all servers that will join the cluster. If Windows Time service in the Services Control panel is not listed, the cluster members have to point to a common server to achieve time synchronization. The best way to do this is to point the servers to a stable domain controller or an NTP server. Ensure all servers are GMT-based.

At this point:

• There should be an Administrative ID in the domain for Cluster Administration. This ID should have the correct security permissions defined.

• The SRDF/CE software should be installed and licensed.• All nodes in the SRDF/CE cluster are validated.• The prospective nodes of the SRDF/CE cluster should be time-synchronized.



Node 1

Node 2


7-12


Step 7: Create a SYMAPI database and specify Symmetrix IDs

7 On all cluster nodes, obtain local and remote node storage configuration information.

1. If you are not in the SRDF/CE Configuration Utility from previous steps, select Start, Programs, SRDF/CE Configuration Utility.

Note: To use the SRDF/CE Configuration Utility, expand and collapse the tree view by single-clicking the plus (+) or minus (-) icon, and then right-clicking an item to display its menu selection.

2. Double-click the SRDF/CE for MSCS icon. This will bring up the SRDF/CE for MSCS login panel.

Log in to SRDF/CE. Once the login is successful, SRDF/CE will automatically register the SRDF/CE Services.

3. Right-click the SRDF/CE for MSCS icon and select Discover. Discover creates a local SRDF/CE SYMAPI database and populates it with discovered storage array information.

Depending on the storage configuration, this step could take time.

In the case where there are only two Symmetrix arrays, one local and one remote, the Discover process also populates the corresponding local and remote arrays in the Configuration Utility.

Note: In the case where multiple Symmetrix arrays were discovered, a panel will pop up requesting you first designate the local Symmetrix array and then designate the remote Symmetrix array.

Steps 8-10: Add peer nodes, set advanced control and quorum settings

8 On Node 1:

Right-click Nodes, and then select Add Cluster Member Node. This will bring up a dialog box to add nodes to the SRDF/CE configuration. These nodes and the network must be accessible.



Node 1

Node 2

Node 1



9 On Node 1:

1. Expand Control.2. Select Advanced Control.3. Right-click Advanced Control and select Edit Advanced Control

Parameters.The only time the default values in Advanced Control should be changed is when another cluster is installed on a Symmetrix system running another SRDF/CE cluster.

Note: If you are setting up only one cluster, leave the default cluster ID at 0. If your configuration includes more than 64 clusters, contact EMC Customer Support.

Adjust Cluster Number to a value from 0 to 63. Once you choose a value and click OK, SRDF/CE will attempt to validate that no other cluster using either of the two Symmetrix arrays defined above is using that lock value. The results of this validation will be reported to you once you make your selection. If the Cluster Number discovery process indicates that the Cluster Number is in use, select another number.



Node 1


7-14


10 On Node 1, specify the quorum settings:

1. Expand Control.

2. Select the type of cluster to configure by selecting Set Quorum Type from the Quorum Control right-click menu. If configuring a cluster on Windows Server 2003 Enterprise or Datacenter Editions, you have the option of configuring a quorum disk-based cluster or an MNS cluster.

Note: MNS enables you to create a server cluster without any shared disk for the quorum resource. It provides an alternative quorum resource for clusters with nodes in separate geographic sites. MNS is only supported on Windows Server 2003 Enterprise and Datacenter Editions.

Select Shared Quorum if your cluster quorum data will reside on a shared disk in the cluster. Select Majority Node Set if your cluster quorum will reside on a local disk on each node and be managed by MSCS.

Note: You may safely ignore the warning that indicates that SRDF/CE is unable to verify whether your selection matches MSCS. This is because MSCS is not yet installed. You must ensure that it matches the quorum type you intend to use when you install MSCS.

If you select Shared Quorum, proceed to Step 3, which follows. If you select Majority Node Set, proceed to Step 4 on page 7-15.

3. Right-click any of the parameters displayed in the right pane and select Edit Quorum Control Parameters and enter:

• Quorum Group Name. Enter the device group name for the quorum device. This device group name must be the same across all nodes in the cluster.

• Quorum Disk. Select the desired quorum disk from devices present on this node.



Node 1



10 (cont.)Note: When you install MSCS, the quorum device group name you specify in SRDF/CE must be the same as the quorum group name later specified during MSCS installation / configuration. The quorum name is case-sensitive.

4. Start the SRDF/CE Service. This service must be started after the quorum configuration steps are completed.

At this point, the following is specified on both nodes:

• Local and remote Symmetrix IDs• Nodes to become members of the SRDF/CE cluster• The cluster number• The quorum disk information

Step 11: Configuring node 2

11 On Node 2:

1. Select Nodes.

2. Right-click the node name representing the current local node, and then select Peer Configurations, then Get Configuration from Node. A node selection box appears.

3. In the Get Configuration from Node node selection box, specify the node you want to pull the configuration from—usually Node 1. If Node 2 is not specified as a valid node during the Node 1 configuration process, the operation will fail because Node 1 has not authorized Node 2 to have access to the configuration.

Note: This function will pull the configuration from Node 1 and completely overlay the configuration of Node 2. Any changes done locally on Node 2 will be lost.

4. Expand Control.

5. Select Quorum Control and verify that Quorum settings match those in Step 10.

At this point, Node 2 is a fully participating member of the SRDF/CE cluster. Any further changes to either Node 1 or Node 2 will be autoconfigured throughout the SRDF/CE cluster.



Node 2


7-16


Steps 12-16: Build temporary groups for the SRDF devices

12 On Node 1:

To validate a given group can be moved among all of the SRDF/CE nodes, you must define temporary device groups. Once these groups are defined, move the groups from node to node. This verifies that all devices within a group are in the same state, all devices can be failed over between nodes, and Windows builds all the correct device definitions.

To change the SRDF state of the device, first put the device into a disk group.

1. From the SRDF/CE Configuration Utility, right-click Groups and select SRDF/CE Group Options, Create New Device Group. The Create New Device Groups dialog box appears.

2. Specify whether you want to retrieve information about R1 or R2 devices:

a. To request information about R1 devices, select RDF1 from Device Display Type. To request information about R2 devices, select RDF2 from Device Display Type.

b. Provide a name for the device group in the Device Group Name field.

c. Select the devices to make accessible.

d. Move devices to the lower pane. Put R1 devices in one group. Put R2 devices in another group. Separate devices in groups by RA group number.

e. It is recommended for cluster failover performance all groups be enabled for R1-R2 Swap. This feature is a Symmetrix device configuration option enabled within the Symmetrix bin. Once this change is done, select either at group creation time or later to enable R1-R2 swap. Click OK.

Note: You may safely ignore the error that indicates that the MSCS create for this group failed. This is because MSCS is not yet installed. These are temporary groups that will be deconfigured and reinstalled in later steps, at which time they will be successfully created in MSCS.

This group is now autoconfigured on all nodes in the SRDF/CE cluster.



Node 1



12 (cont)

3. Repeat Step 12 (1 and 2) for other groups, as necessary.

Note: EMC recommends the Symmetrix array and the R1-R2 devices be configured to use Dynamic Swap where available. This will speed failover times.

Note: If dynamic Rx devices are used, establish an initial R1/R2 pairing. EMC recommends all R1/R2 devices be set to the Synchronized state prior to building device groups. You can do this using SYMCLI commands.

13 Since a group can contain multiple devices, and since each of these devices can be online on a different node, EMC recommends that each group, once configured, should be write-enabled with respect to the local node. This ensures group consistency and that each node in the cluster has the correct Windows device mappings prior to the installation of MSCS.

On each node in the SRDF/CE cluster, perform the following steps from the SRDF/CE Configuration Utility:

1. From Groups, right-click Refresh or click F5.

2. For each group (including the quorum group), determine the current read/write state. Select Groups, and in the right-hand panel, observe each group's RDF Config and RDF State. For an R1 group, the state must show Synchronized, and for an R2group, the state must show Failed Over.

3. For each group in (2) that is not in the correct state, you must move the devices to the appropriate storage array side by performing an SRDF FailOver or SRDF FailBack operation, so the devices become accessible and read/write from the current node. Right-click the appropriate group name, select SRDF/CE Group Operations, then Array Operations. From the Array Operations panel, under Group Movement, select SRDF Failover (for R2 group) or SRDF Failback (for R1 group) as required.

4. For each group (including the quorum group), move the group to the desired state. Expand Groups, select a group, right-click, and then select SRDF FailOver or SRDF FailBack.



Node 1

Node 1


7-18


13

(cont)

5. Move all devices to the appropriate storage array side so the devices become accessible and read/write from the current node.

6. Repeat the process for each group.

Once all device groups are visible to a given node, run a scan for hardware changes on that node to ensure these devices are registered within Windows:

1. Select Start, Programs, Administrative Tools, Computer Management.

2. Select Device Manager.

3. Right-click in the results pane on Disk Devices and do a Scan for hardware changes.

Now that the disks are added to the Windows Registry, map each disk to a drive letter using Disk Management. This will be the drive letter ultimately used by MSCS.

1. Select Start, Programs, Administrative Tools, Computer Management.

2. Select Disk Management.

Note: In the Initialize and Convert Disks Wizard, when presented with the Select Disks to Initialize panel, you MUST change the partition style from the default to MBR (Master Boot Record). SRDF/CE does not support GPT (GUID Partition Table) disks.

3. Right-click in the results pane on the various physical devices and format, label, and mount on a specific drive letter as necessary. Close Disk Management when done.

Note: The mounted drive letter for each physical disk must be the same across all nodes in the SRDF/CE cluster. SRDF/CE only supports drive letters. Windows Server 2003 supports more than drive letters (that is, mount points), but SRDF/CE does not support anything other than drive letters.

Note: Failure to close Disk Management when completed with a node could corrupt that node’s Registry.



Node 1



13

(cont)

Note: To avoid corrupting the volume labels, do not open Disk Manager on the nodes where the disks are not write-enabled.

4. In the case of the quorum group, once it is read/write on each node, the physical device information is available. This information must be added to each Node’s SRDF/CE configuration:

a. Expand Control.

b. Select Quorum Control.

c. Right-click Shared Quorum in the right hand pane and then select Edit Quorum Control Parameters.

d. In the Quorum Disk field, select the desired quorum disk from those devices present on this node.

14 Repeat Step 13 for Node 2.



Node 1

! CAUTION

Node 2


7-20


15 On the last node in the SRDF/CE cluster, verify that all the disks configured in the previous steps are now visible on this last node.

1. Invoke Disk Management.

2. Verify all disks configured in Step 13 are now visible on this last node.

Note: Drive letters on all nodes should match those assigned by Disk Management.

3. After verification, close Disk Management. The system regenerates the Registry key. The Registry keys for all nodes should now reflect a parallel disk structure for those disks and devices participating in the cluster.

Note: Failure to close Disk Management when completed with a node could corrupt that node’s Registry.

16 1. On the Node 2, deconfigure (remove) the temporary disk groups you previously created.

a. Invoke the SRDF/CE Configuration Utility.

b. Expand Groups.

c. Select each group in turn except the Quorum group, right-click, and then select Delete. This will delete the temporary group from all nodes in the SRDF/CE cluster.

At this point:

• The SRDF/CE installation and device mapping is complete on all nodes.• The temporary groups for validating moving disk devices between nodes are created

and then removed.• All nodes display the same set of drive letters and labels as when Node 1 was

read/write enabled. • All nodes display the shared disks with identical drive letters.

Note: These disks will only be read/write enabled on the last node.



Node 2

Node 2



Steps 17 -18: Complete SRDF/CE and MSCS installation on the last node

17 On Node 2, verify quorum disk configuration information:

1. Invoke the SRDF/CE Configuration Utility.

2. Expand Quorum Control and verify the following:

• Group name

• Drive Number

• Mount Point

• Cluster Type

3. Verify the Advanced values for the following:

• Cluster Number

• Delay Failback

• SRDF/Asynchronous Failover Method

Note: The only time these default values should change is when you want to install another cluster on a Symmetrix array already running another SRDF/CE cluster. If this is the case, the preferred action is to contact EMC Customer Support.

4. Start the SRDF/CE Service. This service must be started after the quorum configuration steps are complete.



Node 2


7-22


18 On Node 2, install and configure MSCS.

1. Install MSCS. (Form a new cluster and select the Advanced (minimum) Configuration from the Select Computer panel of the Create New Cluster Wizard. Configure the network following Microsoft’s recommendations. During the installation, run the Cluster Service from the SRDF/CE administrative account.

2. Configure and verify MSCS dependencies, startup mode, and recovery settings.

a. Select Start, Programs, Administration Tools, Computer Management.

b. Expand Services and Applications and select Services.

c. Right-click in the results pane on Cluster Service:

– In the default General tab, verify that Startup Type is Automatic.– In the Log On tab, verify your account is set to the domain

administrative account that you created in Step 2.– In the Recovery tab, verify the selections are set to the following:

First failure: Restart the serviceSecond failure: Restart the serviceSubsequent failures: Take no actionReset fail count after: 7 daysRestart service after: 3 minutes

3. You must add dependencies on the SRDF/CE Service and Windows Management Instrumentation Driver Extensions to the Cluster Service: From a command prompt, issue the command sc qc clussvc. Note the DEPENDENCIES list returned in the command.

4. Issue the command sc config clussvc depend= SRDFCE_svc/Wmi/<serviceslist>

where <services list> is the list of services noted in step 3, delimited by "/".Example: "sc config clussvc depend= SRDFCE_svc/Wmi/ClusNet/RpcSs/W32Time/NetMan. Important: One space after the equal sign (=) and no spaces between service names. Also note that service names are case sensitive. For further help, consult Microsoft documentation.

(Continued on next page)



Node 2



18 (cont.) 5. To verify step 4, from Computer Management, Services and Applications, Services, right-click on Cluster Service, and select the Dependencies tab. Verify that EMC SRDF/CE for MSCS Service and Windows Management Instrumentation Driver Extensions are now included in the dependencies list.

19 On Node 2, install the SRDF/CE resource .dll file. The file, geospan_res.dll, resides in the SRDF/CE installation folder. Do not change the filename. This file creates a resource type named geospan_res in MSCS.

1. Open a command window.

2. Enter the appropriate command line for the operating system, typing continuously, with no line breaks. Type the following command:

cluster restype "SRDF/CE for MSCS" /create /dllname:geospan_res.dll /type:geospan_res

If you incorrectly typed the command line "geospan_res", to recover from the error:

a. Type cluster restype "xxxxx" /delete /type

Where "xxxxx" is used as an example of an incorrect entry for "geospan_res". You must type in your incorrectly spelled entry for "geospan _res" exactly as you previously typed it.

b. Retype the original command line(s).

3. Invoke the MSCS Cluster Administrator and verify the resource type in step 2 has been added.

4. For consistency and clarity, it is recommended that you rename the Cluster Groupto the same name as the Quorum group you defined in step 10.

5. Reboot cluster node Node 2.

At this point, .After Node 2 restarts, the SRDF/CE and Cluster Server services are started on Node 2. MSCS Cluster Administrator:

• Should be aware of the single-node cluster.



Node 2


7-24


Step 20: Complete SRDF/CE and MSCS installation on node 1

20 On Node 1, install MSCS, have Node 1 join the cluster previously created on Node 2, and set the startup mode.1. Start the SRDF/CE Service on Node 1 and verify the groups configured

on Node 2 are propagated to Node 1.2. Configure and verify MSCS dependencies, startup mode, and recovery

settings.

a. Select Start, Programs, Administrative Tools, Computer Management.

b. Expand Services and Applications.

c. Select Services.

d. Right-click in the results pane on Cluster Service.

1. In the General tab, verify that Startup Type is Automatic.2. In the Log On tab verify that your account is set to the domain

administrative account that you created in Step 2. 3. In the Recovery tab and verify the selections are set to the

following: First failure: Restart the service Second failure: Restart the service Subsequent failures: Take no action Reset fail count after: 7 days Restart service after: 3 minutes

4. You must add dependencies on the SRDF/CE Service and Windows Management Instrumentation Driver Extensions to the Cluster Service: From a command prompt, issue the command: sc qc clussvc. Note the DEPENDENCIES list returned in the command.



Node 1



20 (cont.) 5. Issue the command sc config clussvc depend= SRDFCE_svc/Wmi/<services

list>,where <services list> is the list of services noted in step 4, delimited by "/". Example: "sc config clussvc depend= SRDFCE_svc/Wmi/ClusNet/RpcSs/W32Time/NetMan" Important: One space after the equal sign ( = ) and no spaces between service names. Also note that service names are case sensitive. For further help, Extensions are now included in the dependencies list.

6. To verify step 5, from Computer Management->Services and Applications->Services, right-click on Cluster Service, select the Dependencies tab and verify that EMC SRDF/CE for MSCS Service and Windows Management Instrumentation Driver Extensions are now included in the dependencies list.

7. From a command prompt, issue the following two commands: cluster /prop QuorumArbitrationTimeMax=1800 cluster /prop QuorumArbitrationTimeMin=15

8. Reboot cluster node Node 1.

At this point, .After Node 1 restarts, the SRDF/CE and Cluster Server services are started on Node 1. The MSCS Cluster Administrator:

• Should be aware of the two-node cluster.

This completes the installation procedure. Now configure the remaining SRDF/CE Configuration Utility components.




7-26


21 SRDF/CE uses its own SYMAPI database (SRDFCESymapi.db); this reduces interference with SYMAPI databases used by other applications.

Note: Configuring devices for control by more than one SYMAPI database can interfere with normal operation of software that relies on those devices.

1. Before configuring a device for SRDF/CE control, determine whether that device is configured for control by another SYMAPI database on your system or on some other system. Operations (such as failover) that SRDF/CE or some other software initiates on such devices might not be reflected in all corresponding SYMAPI databases.

2. Invoke the SRDF/CE Configuration Utility.

a. Right-click the SRDF/CE for MSCS icon and select Discover to rebuild the local Symmetrix database.

b. Right-click the Groups component and select SRDF/CE Group Options, Create New Device Group. The Select and Group Devices dialog box appears.

c. Create device groups and assign devices to each group:

– Create the new group, assign its devices, and click OK. d. From Groups, right-click Refresh to view the changes for Node 2.

3. Invoke the MSCS Cluster Administrator and verify the resource type and disks are correct for each group.



Node 2



Step 22: Configure SRDF/CE resources

22 On either one of the two nodes, from the SRDF/CE Configuration Utility:

1. Expand Site Mode Operations to set the SRDF communications:

a. Right-click Site Mode and select the mode that controls the behavior of how you want the nodes to respond to site failure events:

– Site Mode = No New Onlines. This is the default.– Site Mode = FailStop. This setting causes groups to go offline

when there is a server, site, or SRDF failure.

Note: Change the Site Mode settings on one node because the settings will circulate to the other node through Autoconfiguration.

On both nodes, from the SRDF/CE Configuration Utility:

2. Expand Logging Control to enable or disable logging activity, if required.

a. Select SRDF/CE Log Level to turn the SRDF/CE log level on or off. When turned on, logging detail can be set from 1-Low to 4-Mega. 1 enables minimal debugging output; 3 enables a more lengthy debugging output. 0-Off is the default.

b. Select SRDF/CE Log Path to set the directory location for the SRDF/CE log file. This is a trace file where all the logging information is placed.

c. Select SRDF/CE Log File Viewer to view any log file.



Node 1

Node 2


7-28


Step 23: Verify the Installation

23 Verify the installation to ensure MSCS sees nodes and disk resources.

1. Invoke MSCS Cluster Administrator.

2. Verify:

• Both nodes are visible and online.

• All disk resources configured are available.

• The Physical Disk dependencies you created for each group are established:

– Select the Physical Disk resource, right-click, and then select Properties.

– Click the Dependencies tab.

The Physical Disk resource must be dependent on the SRDF/CE resource.

Note: All disks must be dependent on the SRDF/CE resource, except the quorum device. All disks (except the quorum) must be dependent on an SRDF/CE resource, a physical disk resource, a cluster name resource, and a cluster IP resource.

3. Move the groups back and forth between the cluster nodes. For each group:

a. Select the group, right-click, and select Move Group.

b. Select the same group, right-click, and reselect Move Group.

Note: The group should successfully be brought online by the other node in the cluster.



Node 1

Node 2



At this point:

◆ A new MSCS cluster group is created. ◆ An SRDF/CE resource is added to the new cluster group.◆ The devices you specified are moved into the new cluster group. The devices are

dependent on the resource; the resource acquires new dependencies, one for each device.◆ A new SYMAPI database group is created on each node—the database name is the same

on both nodes—and the appropriate Symmetrix devices are moved into each group.___________________________________________

You can rename the MSCS cluster group, the SRDF/CE resource, and the two SYMAPI groups (one on each node). However, the SRDF/CE resource and both SYMAPI groups must always have the same name. Therefore, to change the name of the SRDF/CE resource or either SYMAPI group, you must make all three the same. Some applications, such as SAP, will change the MSCS resource name. If this occurs, reconfigure so all three names are the same.___________________________________________

◆ The quorum control name matches the quorum group name enabling SRDF/CE to see the quorum in the Registry.

Step 24: Create a checkpoint file

24 The checkpoint file is used to recover the SRDF/CE API database if it becomes corrupt. To create the checkpoint file, do the following:

1. On each node, invoke the SRDF/CE Configuration Utility.

2. Right-click Control and select SRDF/CE Config Options, Check Point.

3. A message will pop up asking if you want to proceed. Click Yes.

4. A Checkpoint complete message appears stating the local Registry and group information was written to the RegList.txt file in the SRDF/CE base installation directory. Click OK.

This completes the creation of the checkpoint file. The checkpoint file should be updated anytime changes are made to the group information in the SRDF/CE management utility.

You can now start working with SRDF/CE



Node 1

Node 2


7-30


Uninstalling SRDF/CETo uninstall SRDF/CE, you must stop the cluster on all nodes and use the Add/Remove Programs utility (in the Control Panel) to remove SRDF/CE.

Important: Without SRDF/CE, MSCS cannot be failed over to a cluster member node. Therefore, remove and reinstall MSCS Cluster Service to make the cluster operational again in a local disk configuration.


AInvisible Body Tag

This appendix describes the alerts, event messages, and errors you may encounter while using the SRDF/CE Configuration Utility GUI.

◆ SRDF/CE alerts .................................................................................A-2◆ SRDF/CE errors and events ............................................................A-4◆ Event messages..................................................................................A-5

SRDF/CE Alerts andErrors

SRDF/CE Alerts and Errors A-1

A-2

SRDF/CE Alerts and Errors

SRDF/CE alertsSRDF/CE warnings, or alerts, appear in the bottom-right corner of the SRDF/CE Configuration Utility (Figure A-1). The possible alerts that could display are the following:

◆ SRDF/CE Service ◆ RDF Local Lock◆ RDF Remote Lock ◆ SRDF/CE Driver ◆ IP Communications◆ SRDF Communications◆ Quorum Reserve

Figure A-1 SRDF/CE alerts display

SRDF/CE Service — Indicates the SRDF/CE Service is not started on the local node. Once the service is started, this warning will not appear.

RDF Local Lock — Indicates an application has exclusive access to the SYMAPI database on the local Symmetrix array designated within the SRDF/CE Configuration Utility. This warning briefly displays when a failover to the remote Symmetrix array occurs. If this warning persists, you may need to manually release this lock using SYMCLI commands.

RDF Remote Lock — Indicates an application has exclusive access to the SYMAPI database on the remote Symmetrix array designated within the SRDF/CE Configuration Utility. This warning briefly displays when a failover to the remote Symmetrix array occurs. If this warning persists, you may need to manually release this lock using SYMCLI commands.



SRDF/CE Driver — Indicates the SRDF/CE Service is unable to communicate with the SRDF/CE device driver. Quorum failover will not work properly if this warning appears. If this warning appears after SRDF/CE is installed and configured, you may need to restore the communication links.

IP Communications — Indicates the SRDF/CE Services on the cluster nodes cannot communicate. If this warning appears, failover/failback operations may be affected and MSCS may initiate group movement. If this warning appears after SRDF/CE is installed and configured, you may need to restore the communication links.

SRDF Communications — Reflects the inability of the SRDF/CE Service to ping from its local Symmetrix array to the remote Symmetrix array through the SRDF link. If the SRDF Communications warning appears, failover/failback operations may not work properly. Call EMC Customer Support for resolution.

Quorum Reserve — Indicates the cluster is up and functioning properly and that there is a reserve on the quorum. This warning appears if no reserve exists on the quorum.

SRDF/CE alerts A-3

A-4


SRDF/CE errors and eventsSRDF/CE relies on MSCS and the EMC Symmetrix API (SYMAPI) to function. There are a few events caused and/or generated by the SRDF/CE service. Virtually all SRDF/CE errors can be traced to problems with MSCS or the SYMAPI. Therefore, the sections that follow are not intended to be a troubleshooting guide or a definitive reference for the Windows event and application logs.

Interpreting Windows event log information

Users involved in diagnosing problems in an SRDF/CE cluster must have a thorough understanding of the following logs:

◆ Windows event logs

◆ SRDF/CE application logs

◆ MSCS logs

◆ SYMAPI logs

◆ Logs of major applications deployed on the cluster

Event log entries occur in the order in which they are processed by the system. This can cause confusion as to the actual cause of an error because related errors from different sources may be interspersed by other, nonrelated entries. This is particularly true in the system and application logs. Additionally, since related errors may be entered into different logs, correlating the relevant entries temporarily is necessary.

Example A resource assigned to an SQL virtual server becomes unavailable to MSCS due to a cluster error. This problem will generate errors in the Windows application and system event logs, the MSCS log, the SYMAPI log and the SRDF/CE log. Additionally, SQL Server, SRDF/CE, MSCS and SYMAPI generate entries in the Windows event logs. Depending on where the diagnosis starts, the problem could appear to have originated in any of the applications. Therefore, concise analysis is required to ascertain the actual error path.



Event messagesCurrently, SRDF/CE system-level messaging comprises the following categories:

◆ Device driver messages recorded in the System Event Log

◆ Service error messages recorded in the Application Event Log

System event log The following are System Event logs:

Table A-1 System event log descriptions

Event Description Action

geospan_device_loadedSRDFCD: Driver loaded successfully.

Load success. No action necessary.

geospan_device_attach_failedSRDFCD: Failed to attach to <device name>.

The SRDF/CE failed to set up properly because driver startup failed.

This could be due to environment problems with MSCS. Check the quorum device name and Symmetrix array mapping. Fix problems and restart.

geospan_device_disk_number_invalidSRDFCD: Invalid quorum disk number <disknumber>, registered by application.

SRDF/CE failed to start properly because the quorum disk number is invalid.

Check the MSCS configuration and SYMAPI database.

geospan_device_disk_number_missingSRDFCD: Signature matching diskNumber <disknumber> was not found in <hostname> registry.

SRDF/CE failed to start properly because the quorum disk number could not be read.

Check the MSCS configuration and SYMAPI database.

geospan_device_load_failed SRDF/CE failed to start properly because the quorum filter driver failed to load. This could be due to environment problems with MSCS.

Check the installation.

geospan_device_not_a_signature Quorum signature invalid. Check the MSCS configuration and SYMAPI database.

Event messages A-5

A-6


Application event log messagingInformational: After the SRDF/CE installation completes, the machine is rebooted

and you logon for the first time, you will see an application popup indicating a service or driver failed to start, as follows:

Application popup: Service control Manager : At least one service or driver failed during system startup. Use Event Viewer to examine the event log for details.

In addition, you will see the following event log message:

The following boot-start or system-start driver(s) failed to load: SRDFCE

For more information see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

No action is required. The problem will be gone after MSCS is configured.

The following entries will occur during a normal startup of the SRDF/CE Service:

service start main for GeoSpan service.service_main for WNTServiceClass.AppInit.

The following entries will occur during a normal shutdown of the SRDF/CE Service:

WNTServiceClass::TskSvcEvent=serviceStopEvent.ServiceControlHandle:Exiting. Cleaning up and leaving.cleanup service_main for WNTServiceClass.StartServiceCtrlDispatcher return false.

geospan_device_signature_failed Quorum signature mismatch. Check the MSCS environment and Symmetrix configuration.

geospan_device_signature_missing Quorum signature missing. Check the MSCS environment and Symmetrix configuration.

geospan_device_unable_to_read_signature Cannot read quorum signature. Check the MSCS configuration and SYMAPI database.

Table A-1 System event log descriptions (continued)

Event Description Action



Table A-2 Application event log descriptions

Error Description Action

E_INVALID_PDEV_INFOThe physical device information from the Symmetrix does not match the physical device information from WMI. This can happen if Windows renumbers the physical device ID's.

Under certain conditions Windows can renumber the physical device identifiers. One of these conditions is on reboot. If this happens the physical device identifiers extracted from WMI do not match the physical device identifiers from the Symmetrix array. In this case we use the physical device identifiers from WMI.

A Symmetrix discover is required.

E_R1_R2_SWAP_IGNOREDR1/R2 personality swap is enabled, but will be ignored because the SRDF/CE Override option or the Forced Failover option is also enabled.

R1/R2 personality swap can be enabled for each group, but will be ignored during failover if the SRDF/CE override option or the Forced Failover option is also enabled.

No action required.

E_GATEKEEPER_LOCKEDOne or more Gate Keeper devices are locked.

Each Symmetrix call requires a gatekeeper device to complete the call successfully. This error indicated that no gatekeeper devices were available to service the request. As a result the Symmetrix call failed.

Add more gatekeeper devices and retry the action.

E_SYMINIT_FAILEDUnable to initialize a connection to the Symmetrix. Call to SymInit failed with a status of [<symapierrno>]. SYMAPI error text for [<symapierrno>] is [<symapi description of the error>]. The application can not continue.

An error occured while trying to make a connection to the Symmetrix array.

SRDF/CE can not continue. You must reboot the machine and restart the application.

GROUP_MOVE_FAILED:A failure occurred trying to move group [<groupname>].

Group Move across cluster nodes failed. Check the SRDF/CE logs for more information.

SRDFCE_SRDF_SET_MODE_ASYNCHRONOUS_ERROR:CClusSymInfo::SetSRDFMode: Problem encountered trying to Set Mode [<0/1>] for group [<groupname>]. Retry count [<retries>]. RC = [<symapierrno>] - [<symapi description of the error>].

RDF mode 0 means synchronous, 1 means asynchronous. SRDFCE failed to set RDF mode of a group to Synchronous/Asynchronous.

The SYMAPI description should give you enough information regarding the nature of the problem. Check the RDF configuration and the RA link state. Fix any problems you see and try the command again.

Event messages A-7

A-8


SRDFCE_SRDF_SET_MODE_ASYNCHRONOUS_ERROR:CClusSymInfo::SwapGroup: SymDgRdfControl for group [<groupname>] for FAST SWAP failed after [<retries>] retries. RC = [<symapierrno>] - [<symapi description of the error>]

Swapping the group failed after n number of retries.

The SYMAPI error description should give you enough information regarding the nature of the problem.

SRDFCE_CANNOT_COMMUNICATE_WITH_DRIVER:Cannot open device to send commands to driver\r\nEither SRDF/CE Filter Driver is not Running or not responding.

The filter driver creates \\.\SRDFCEDevice0 control device for SRDFCE application to communite with driver. This device is not present since the driver is not loaded successfully.

Make sure the driver c:\windows\system32\drivers\srdfce.sys is present on the system. If not, run the install wizard to repair the installation. Also make sure that the driver is not disabled.

SRDFCE_SRDF_SET_MODE_ASYNCHRONOUS_WARN:CClusSymInfo::SetSRDFMode: SymDgRdfSet for Set Mode failed for Group Name [<groupname>]. RC = [<symapierrno>] - [<symapi description of the error>]. Retries: [<retry count>].

Setting the RDF mode to sync/async failed. No action is required until all retries failed.

SRDFCE_RES_STATE_INVALID:The RDF pair state for group [<groupname>] was found to be [SYMAPI_RDF_STATE_MIXED] during a MSCS Is / LooksAlive check interval.

One or more devices in the RDF group is different from rest of the members.

Device state must be either Failover/Synchronized/Consistent state depending on RDF mode and the failover state of the group. Any other RDF state must be investigated and fixed as soon as possible.

SRDFCE_SRDF_FAST_SWAP_WARNING: CClusSymInfo::SwapGroup: SymDgRdfControl for group [<groupname>] for FAST SWAP failed. RC = [<symapierrno>] - [<symapi description of the error>]. Retries done: [<number of retries>]. Retries left: [<number of retries left>]."

Swapping group as part of group failover failed.

No action is necessary until all retries failed.

Table A-2 Application event log descriptions (continued)




Unable to handle to node MSCS is not running. Start Cluster Service or attempt to diagnose.

Cannot open device to send commands to driver The SRDF/CE driver is unavailable. Check installation. In addition, this could be due to environment problems with MSCS. Check the quorum device name and Symmetrix array mapping. Fix problems and restart.

Table A-2 Application event log descriptions (continued)


Event messages A-9

A-10



BInvisible Body Tag

This appendix contains a sample of the preinstallation worksheet to use with an EMC technical representative before incorporating SRDF/CE for MSCS into your clustering environment.

◆ Preinstallation worksheet ................................................................ B-2

PreinstallationWorksheet

Preinstallation Worksheet B-1

B-2

Preinstallation Worksheet

Preinstallation worksheetBefore installing SRDF/CE for MSCS, meet with your EMC representative to complete the preinstallation worksheet presented on the next page.

In the worksheet:

◆ The Group Name is the same as the MSCS device group name.

◆ The Pdev value is the numeric designation in the host's physical drive number (for example, the value 5 in \\.\physicaldrive5).

◆ The Label value specifies the name of the disk.

◆ The Drive value is the drive letter designation to be assigned to a given Windows device (disk) when the cluster is operational.

◆ The Symdev value is the Symmetrix device name that is a four-hex number assigned to the given device by the Symmetrix array.

◆ The RA Group value is the Symmetrix remote adapter or RDF group number.

Note: Clustered devices have the same group name and drive letter on both nodes.

Gather all information before installing any software (including the operating system). You must have all the information by the time you verify the configured drive letters on both nodes and before you install MSCS on either node.



Note: The gray area above represents an example only.

Synchronous SRDF Link

Bidirectional

Windows Server

Windows Server

Symmetrix Client(s)

IP Address ___.___.___.__

___.___.___.__

____________

Private IP Address

NETBIOS Name

Windows Server 1

IP Address ___.___.___.__

___.___.___.__

____________

Private IP Address

NETBIOS Name

Windows Server 2

Name of ClusterNETBIOS Name ___.___.___.__

___.___.___.__

____________

MSCSConfiguration

Cluster IP Address

Cluster Subnet Mask

Client(s)

Symmetrix

SRDF/CE for MSCS Configuration Worksheet

Symmetrix 112-Digit Symmetrix ID _ _ _ _ _ _ _ _ _ _ _ _

Symmetrix InformationGroup Name Pdev Drive Symdev

QuorumGrp Quorum-O Q 0006(R1)

Label RA Group

Symmetrix 112-Digit Symmetrix ID _ _ _ _ _ _ _ _ _ _ _ _

Symmetrix InformationGroup Name Pdev Drive Symdev

QuorumGrpphysicaldrive5 Quorum-O Q 0006(R1)

Label RA Group

1 1

physicaldrive5

Preinstallation worksheet B-3

B-4


On the reverse side of the worksheet, answer the following questions.

Description

How many of these gatekeepers per host are dedicated to SRDF/CE?a

SRDF/CE requires all drives formatted NTFS. If No, these drives need to be converted.

What is the size of the drive that will contain the quorum? A quorum size of at least 500 MB is recommended.

Are all data drives for the cluster and quorum configured as RDF devices?

What is the name of the MSCS administrative account?

What is the name of the SRDF/CE Service Account?

a. SRDF/CE requires a minimum of six dedicated gatekeepers per host and six per group, per node.


Glossary

This glossary contains terms related to the EMC SRDF/CE software.

Aagent An installed program designed to control a particular resource type.

Each type of resource supported in a cluster is associated with an agent.

Application ProgramInterface (API)

A language and message format used by an application program to communicate with another program that provides services for it. APIs are usually implemented by writing function calls. Examples of APIs are the calls made by an application program to such programs as an operating system, messaging system, or database management system.

See also SYMAPI.

asynchronous mode See SRDF Asynchronous (SRDF/A).

autoconfiguration The utility the SRDF/CE COM Server uses to propagate the SRDF/CE configuration among cluster peers. Proper time synchronization for all host nodes is critical to ensure the proper functioning of Autoconfiguration. EMC recommends all servers in an SRDF/CE cluster be based on GMT and time servers be employed.

availability The ability to continue to provide a service even during hardware or software failure.

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide g-1

g-2

Glossary

BBCV device A Symmetrix business continuance volume (BCV) that functions as a

mirrored media to a standard device for a protected storage environment.

BCV mirror BCV device upon establishing or reestablishing a BCV pair.

BCV pair A standard device and a BCV device that provide a protected storage environment.

business continuance An SRDF function that ensures business applications continue running despite possible disk failures.

business continuancevolume

See BCV device.

Ccache Random access electronic storage used to retain frequently used data

between the CPU and either a hard disk or slower RAM. It speeds up general data flow because a cache can be accessed quickly.

channel director The component in the Symmetrix array that interfaces between the host channels and data storage. It transfers data between the channel and cache.

client A computer using services or resources provided by a remote machine, called a server. Often, communications software will have a separate version for the client or guest, and the server or host.

Clients create a TCP/IP session with a service in the cluster using a known IP address. This address appears to the cluster software as a resource in the same group as the application providing the service. In a failure, the Cluster Service will move the entire group to another system.

client failover The response of a client machine after resource failure on the server for the client caused a resource failover. A client will detect a failure in the session and reconnect in exactly the same manner as the original connection. The IP address is now available on another machine and the connection is quickly reestablished. In this simple case, all information related to the original session not committed to disk is lost. This provides higher availability, but no fault tolerance for the service. Applications can use transactions to guarantee the


Glossary

client request is committed to the server database to gain fault-tolerant semantics.

cluster A group of two or more independent computers addressed and used as a single system.

cluster aware software Software that provides a restart mechanism invoked whenever the application resource is moved to another node in the cluster.

Cluster Service The collection of software on each node that manages all cluster-specific activity.

Ddata center

migrationsAn SRDF function that reduces application outage to minutes instead of hours.

delay failback An SRDF/CE feature modifying the Preferred Owner list for each MSCS cluster group so that a failover will occur to a lateral node first, and if the lateral node is unavailable, to a peer node.

See also lateral node and peer node.

dependency The requirement of one resource needing another resource to function properly. The SRDF/CE resource becomes a dependency for physical disk resources in the cluster. Therefore, any operations performed on the disk resource cannot be completed until the SRDF/CE resource has been invoked.

device A uniquely addressable part of the Symmetrix array consisting of a set of access arms, the associated disk surfaces, and the electronic circuitry required to locate, read and write data.

device group A grouping of several devices established to provide configuration, status, and performance data on the collective devices within the group.

director The component in the Symmetrix array that allows the Symmetrix array to transfer data between the host channels and disk devices.

See also channel director and disk director.

disaster recovery An SRDF function that recovers data at the disaster recovery site in minutes rather than days.


g-4

Glossary

discover A discover action performed in the SRDF/CE Configuration Utility or Configuration Wizard scans the Symmetrix array connected to the current node and gathers device information.

disk director The component in the Symmetrix array that interfaces between cache and the disk devices.

EEMC Solutions Enabler Also known as SYMCLI, an application written using the Symmetrix

Application Programming Interface (SYMAPI) that retrieves data from a Symmetrix array using special low-level SCSI commands.

The EMC Solutions Enabler allows you to run commands on the host to obtain configuration, status, and performance data from the Symmetrix arrays attached to hosts that are running in an open systems environment.

SYMCLI SRDF and TimeFinder components allow you to perform control operations on RDF and BCV devices.

establish A BCV process that assigns a BCV device as the next available mirror of a standard device.

established The BCV pair condition where the BCV device and standard device are synchronized and functioning as a Symmetrix mirror. A BCV pair is established by the BCV commands establish, reestablish, restore, or incremental restore.

Ffailback The action of moving a resource back to the cluster member

designated to be the resource's preferred owner. By default, resources are owned by their preferred owners, so a failback would only occur if the resource moved from its preferred owner. This is likely the result of a failover.

failover The process of taking one or more resources offline on one cluster member and bringing them online on another cluster member.

fault-tolerant Continuous operation in case of failure. A fault-tolerant system can be created using two or more computers that duplicate all processing, or having one system stand by if the other fails. It can also be built with redundant processors, control units, and peripherals. Fault-tolerant operation requires backup power in a power failure. It


Glossary

may also imply duplication of systems in disparate locations in the event of natural catastrophe or vandalism.

FDDl An acroynym for Fiber Distributed Data Interface.

Fibre Channel A high-speed serial interface capable of data transfer rates of up to 100 MB/s.

Fibre Channel Director The Fibre Channel adapter (FA) in the Symmetrix subsystem that interfaces between the host Fibre Channel interface and data storage. It transfers data between the channel and cache.

forced failover An SRDF/CE feature allowing you to automatically keep a cluster up on a particular Symmetrix array or arrays in a total site disaster.

forced quorum Software functionality allowing the cluster to be forced up in the event that total communication is lost between nodes and MSCS wants to shut down the cluster to avoid a split-brain condition.

See also split-brain condition.

Ggraphical userinterface (GUI)

A method that allows users to interact with the computer and its special applications based on graphics instead of text. GUIs use icons, pictures, and menus and use a mouse as well as a keyboard to accept input.

group A collection of resources to be managed as a single unit. Usually, a group contains all elements needed to run a specific application and for client systems to connect to the service provided by the application. Groups allow an administrator to combine resources into larger logical units and manage them as a unit. Operations performed on a group affect all resources contained within that group.

HHBA See also host bus adapter (HBA).

heartbeat A polling communication mechanism used by the cluster processes to determine whether the other member(s) of the cluster are alive and working or have failed. If the heartbeat is not functioning, a failover is initiated, and another node in the cluster will take over the services.


g-6

Glossary

high availability The characteristic of a computer system/computing environment that allows it to continue to provide applications and access to data if a single component or resource fails. Service is interrupted for only a brief time, and may or may not be apparent to the end users.

host bus adapter(HBA)

A device circuit board that provides an interface between the SCSI bus and the computer I/O bus (for example, PCI, EISA, microchannel).

hyper-volume The term used by the Symmetrix array to make a physical disk appear as multiple physical disks. Each hypervolume has its own unique SCSI address.

II/O Input/output.

identifier (ID) A sequence of bits or characters that identifies a program, device, controller, or system.

incremental establish A TimeFinder BCV or SRDF control operation.

For BCV control operations, an incremental establish causes the BCV device to be incrementally synchronized and functioning as a Symmetrix mirrored device. (The devices must have been previously paired.) This is the same as an establish operation except an incremental establish is much faster: It copies only the differences or new storage data from the standard device to the BCV device. Any changed tracks on the BCV device are overwritten by the data on the corresponding tracks from the standard device.

For SRDF control operations, an incremental establish causes the target (R2) device to be incrementally synchronized and established as a Symmetrix mirrored device. (The devices must have been previously paired.) This is the same as an establish operation except that an incremental establish is much faster: It copies only the differences or new storage data from the source (R1) device to the target (R2) device. Any changed tracks on the device are overwritten by the data on the corresponding tracks from the source (R1) device.

incremental restore A TimeFinder BCV or SRDF control operation.

In BCV control operations, an incremental restore is a control operation that reassigns a BCV device as the next available mirror of the standard device in the pair. However, the standard devices are


Glossary

updated with only the data written to the BCV device during the time of the original pair split. The data written to the standard device during the split is overwritten with data from the BCV mirror.

In SRDF control operations, an incremental restore is a control operation that reassigns a target (R2) device as the next available mirror of the source (R1) device in the pair. The source (R1) devices are updated with only the data written to the target (R2) device during the time of the original pair split. The data written to the source (R1) device during the split is overwritten with data from the target (R2) mirror.

Llateral node Nodes connected to the same Symmetrix array.

MMajority Node Set

(MNS)A quorum-capable resource based on replicating data to local disks associated with a majority of cluster nodes. MNS enables you to create a server cluster without shared disk for the quorum resource. SRDF/CE allows you to configure an MNS cluster on Windows Server 2003 Enterprise and Datacenter Editions.

MicrosoftManagement

Console (MMC)

A Microsoft user interface (UI) framework for use in administrating different components of the Microsoft Windows operating platform. This framework is used to host specific UI/control extensions called snap-ins. Use snap-ins to administer both local and remote computers. Third-party snap-ins can be written for use with MMC, which is the case for the SRDF/CE Configuration Utility.

mirrored pair A device comprising two hypervolumes with all data recorded twice—once on each disk drive.

mirroring A device comprising two hypervolumes with all data recorded twice—once on each disk drive. The Symmetrix array maintains two or more identical copies of a set of data on separate disks. Each copy automatically updates during a write operation. If one disk device fails, the Symmetrix array automatically uses one of the other copies from another disk drive.

MMC See Microsoft Management Console (MMC).


g-8

Glossary

MSCS Microsoft Cluster Service. A shared-nothing cluster solution for Windows Server Family.

Nnetwork interface

card (NIC)A device that provides network communication capabilities to and from a computer system.

nodes Members of a cluster. Also referred to as systems. A node contains a CPU, disk, and network resource.

Ooffline The state of a resource or group that classifies it as unavailable. When

used in context with a cluster member, offline implies the cluster member may not be booted, or the cluster service on the node in question may not be functioning properly.

online The state of a resource or group that classifies it as available. When used in context with a cluster member, online implies the other cluster members are receiving heartbeats from the cluster member in question. See also resource.

Ppeer node Nodes connected to different Symmetrix arrays located across the

SRDF link from each other.

Qquery A command reporting the state of all the BCV devices in the system,

as well as the status of SRDF states.

quorum disk An ordinary disk volume used as a special communication mechanism between server systems. In an MSCS cluster, a small amount of cluster system data (a few megabytes) is stored on this volume. The SCSI-3 Reserve and Reset commands are used to move quorum-disk ownership back and forth between nodes. If the heartbeat mechanism fails, the quorum disk is used for each node to verify whether the other node is still functioning. Because not all disk products implement these multihost SCSI-3 commands, not all disk products will work in an MSCS environment. Thus, Microsoft is very rigorous in providing the Cluster/RAID category of tests to qualify


Glossary

disks (refer to Microsoft’s Hardware Compatibility List) capable of running with MSCS software ).

RR1 device See source (R1) device.

R2 device See target (R2) device.

RDF1/RDF2 A type of SRDF device group. Only RDF devices can be assigned to an RDF group. An RDF1 group type contains source (R1) devices and an RDF2 group type contains target (R2) devices.

RDF State The SRDF/CE Configuration Utility displays the RDF state information in a column in the right pane when you select:

• Control, Quorum Control

• A specific group under Groups

• R1/R2 devices under Storage

The possible RDF states are the following:

Invalid — The device and link are in an unrecognized combination.

SyncInProg — Synchronizing in progress.

Synchronized — The source and target have identical data.

Split — The source is split from the target and the target is enabled.

Suspended — The link is suspended.

Failed Over — The target is write-enabled, the source is write-disabled and the link is suspended.

Partitioned — The communication link to the remote Symmetrix array is down and the device is write-enabled.

R1 Updated — The target is write-enabled, the source is write-disabled and the link is up.

R1 UpdInProg — The target is write-enabled, the source is write-disabled, the link is up, but there are invalid tracks between the target and the source.

Mixed — This state is only set for a SymDgShow() call when the RDF states of the devices in the group are different from each other, thereby making the RDF state of the group mixed.


g-10

Glossary

N/A — Not applicable.

Consistent — R2 data is consistent.

RA Remote adapter. An RA provides the link connection and fiber optic protocol support between the local and remote Symmetrix arrays. The RA cable connection is ESCON fibre (ESCON protocol).

RAID Redundant array of independent disks. Data is stored on multiple magnetic or optical disk drives to increase output performance and storage capacities and to provide varying degrees of redundancy and fault tolerance. Instead of storing valuable data on a single hard disk that could fail at any time, RAID ensures a backup copy of all information always exists by spreading data among multiple hard disks.

reestablish A business continuance process that reassigns a BCV device as the next available mirror of the standard device with which it was previously paired. The BCV mirror is updated with the data written to the standard device during the period the BCV pair was split. The data written to the BCV device during the split is overwritten by data from the standard device.

Remote Link Director(RLD)

RLDs create the data link paths between two data storage units. Each Symmetrix array requires a minimum of two, up to a maximum of eight RLDs, depending on the Symmetrix model in use. Each RLD manages two ESCON fibre link connections. Each RLD can perform a single I/O at a time to its paired RLD in the remote Symmetrix array.

RLDs have either an RA1 or RA2 designation. RA1s reside in the source Symmetrix array. RA2s reside in the target Symmetrix array. These RLDs can also be assigned to an RA group. Up to four RA groups can exist in an SRDF configuration.

See also RA.

resource An object managed by the Cluster Service that sees all resources as identical opaque objects. Resources may include physical hardware devices such as disk drives and network cards, or logical items such as disk partitions, TCP/IP addresses, entire applications, and databases. A resource is said to be online on a node when it is providing its service on that specific node.

resource failback The movement of resources back to their preferred location in the cluster. This is usually done under manual user control to avoid a


Glossary

situation where a resource is failed back, and then immediately fails over again because of an unresolved node problem. MSCS also allows automatic failback and provides a timing window to try to avoid repeated failovers.

resource failover The process where control of a resource moves to another node of a cluster. Failover can be initiated automatically or manually. When initiated automatically, the cluster management software detects a failure of server node hardware or an application. When manually initiated, the cluster administrator uses the MSCS Cluster Administrator software application.

resource group A collection of resources to be managed as a single unit. Usually a group contains all elements needed to run a specific application, and for client systems to connect to the service provided by the application. Groups allow an administrator to combine resources into larger logical units and manage them together. Operations performed on a group affect all resources contained within that group.

restore A TimeFinder BCV or SRDF control operation.

In BCV control operations, a restore copies a full BCV mirror back to the standard device in the pair and reassigns the BCV device as the next available mirror to the standard device.

In SRDF control operations, a restore copies the full target (R2) device back to the source (R1) device in the pair and reassigns the target (R2) device as the next available mirror to the source (R1) device.

See also incremental restore.

RF A remote adapter that provides the link connection and fiber optic protocol support between the local and remote Symmetrix arrays. The RF cable connection is SCSI fibre (SCSI protocol). An RF differs from an RA only in the type of connection; an RA uses an ESCON fibre connection, and an RF uses a SCSI fibre connection.

See also RA.

Sscalability The ability to add new components to a storage system as system

load increases.


g-12

Glossary

SCSI Small Computer System Interface. SCSI is a high-speed parallel interface used to connect microcomputers to SCSI peripheral devices, such as disks, printers, and other computers and local area networks.

semisynchronousmode

An SRDF mode of operation that provides an asynchronous mode of operation. Applications are notified an I/O (or I/O chain) is complete once the data is in the cache of the local RA1 Symmetrix array. Any new data is then written to cache in the remote RA2 Symmetrix array. The remote Symmetrix array acknowledges receipt of the data once it is secure in its cache. If source tracks are pending transfer to a target (R2) device, and a second write is attempted to the source (R1) device, the Symmetrix array disconnects (nonimmediate retry request), and waits for the pending track to transfer to the remote Symmetrix array.

snap-in See Microsoft Management Console (MMC).

Solutions Enabler See EMC Solutions Enabler.

source (R1) device A Symmetrix source (R1) device that is participating in SRDF operations with a target (R2) device. All writes to this device are mirrored to a target (R2) device in a remote Symmetrix array. An R1 device must be assigned to an RDF1 group type.

See also RDF1/RDF2.

source unit In an SRDF configuration, it is the primary data storage subsystem. It initiates many of the SRDF synchronization activities. An SRDF configuration must have at least one source unit and one target unit. See also target unit.

split A business continuance process that removes the BCV mirror from the existing BCV pair and assigns the BCV mirror back to its original device address. The BCV device then holds an instant copy of the data from the standard device.

split-brain condition A total communication failure while both nodes remain operational. A split-brain condition is a potential cause of logical data corruption. For example, if both sides assume the other is dead and begin processing new transactions against their copy of the data, two separate and unreconcilable copies of the data can be created.

SRDF Symmetrix Remote Data Facility. SRDF consists of the microcode and hardware required to support Symmetrix remote mirroring.


Glossary

SRDF Asynchronous(SRDF/A)

A high-performance, extended-distance asynchronous replication using a delta set architecture for reduced bandwidth requirements and no host performance impact.

Asynchronous mode provides a point-in-time image on the target (R2) device only slightly behind the source (R1) device. SRDF/A session data is transferred to the remote Symmetrix system in delta sets, eliminating the redundancy of same-track changes being transferred over the link, reducing the required bandwidth. SRDF/A only needs enough bandwidth to support the average production workload versus peak workloads.

SRDF/A is intended for users who require no host application impact while maintaining a consistent, restartable image of their data on the R2 side at all times.

SRDF link Fiber optic connections and channels between two Symmetrix arrays. A minimum of two to a maximum of eight links can exist between the two units.

SYMAPI Symmetrix Application Program Interface. See Application Program Interface (API).

SYMCLI See EMC Solutions Enabler.

synchronous mode An SRDF mode of operation that ensures 100 percent synchronized mirroring between the two Symmetrix arrays. This is a synchronous mode of operation. Applications are notified that an I/O (or I/O chain) is complete when the RA2 Symmetrix array acknowledges that the data has been secured in cache.

systems See nodes.

Ttarget (R2) device A Symmetrix target (R2) device participating in SRDF operations

with a source (R1) device. It resides in the remote, or target, Symmetrix array. It is paired with a source (R1) device in the local Symmetrix array and receives all write data from its mirrored pair. This device is not accessed by user applications during normal I/O operations. An R2 device must be assigned to an RDF2 group type. See also RDF1/RDF2.

target unit In an SRDF configuration, this subsystem maintains synchronization with the devices it is paired with in the source unit. It can serve as a


g-14

Glossary

source unit during disaster recovery. An SRDF configuration must have at least one source unit and one target unit. See also source unit.

Vvirtual servers See nodes.

Wworkload migrations In SRDF, similar to data center migrations; especially useful for

minimizing outages during preventative maintenance of hardware or software.


Index

Aadministrative procedures 6-1advanced control settings 5-15application

event log messaging A-6software failure 2-4-specific resources 6-4

array operations 5-40Asynchronous SRDF

failover method 5-16, 5-41asynchronous SRDF

compatibility 1-9failover method 7-21group 3-30, 3-49, 5-36, 5-39

auto del release 5-16autoconfiguration 1-12

Bbehavior

common with MSCS 2-4override 2-9SRDF/CE failover operation 2-2

best practices 6-31booting 1-22build temporary group for SRDF devices 7-16

Ccabling 1-21changing

quorum log size 6-33user credentials 5-3

checkpoint 5-6

clusteradministrator 6-3architecture 1-26-aware software 1-28benefits 1-26concepts 1-25limitations 1-26modes of operation 1-26modifying groups 6-7multiple SRDF/CE for MSCS 1-17number 5-15pre-SRDF/CE considerations 1-21SRDF/CE 1-2

cluster servicemanually starting 6-6stopping 6-5unable to start 6-24uninstalling on Windows 2003 6-5

clustering 1-21Configuration Utility GUI 1-6, 5-2

SRDF/CE for MSCS icon 5-3Configuration Wizard 1-8, 3-20, 5-4configuration, troubleshooting 6-22configurations

basic SRDF 1-4multicluster 6-3

configuring additional disk devices 6-8control settings 5-6

advanced controlauto del release 5-16cluster number 5-15delay failback 5-16SRDF/A failover method 5-16, 5-41

forced failover 5-27

EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide i-1

i-2

Index

logging control 5-18log file viewer 5-18log level 5-18, 7-27log path 5-18

quorum control 5-7site mode operations 5-13

converting MSCS cluster to SRDF/CE 3-65create

a resource type 7-23a SYMAPI database 7-12an administrative account 7-9new device group 5-35

credentials 5-3

Ddata protection 1-2delay

failback 1-13, 5-16loading of MSCS Service 6-31

device group 5-35list 5-38unlock 5-39

discover 5-4disk devices 6-8documentation i-xviidynamic swap 1-10, 5-40, 5-44

Eedit log parameters 5-19EMC

business continuance volumes (BCVs) 1-26documentation i-xvisoftware requirements 3-3, 3-5technical support 6-22

error messages 6-28event messages A-5

application event log A-6geospan_device_attach_failed A-5geospan_device_disk_number_missing A-5geospan_device_load_failed A-5geospan_device_loaded A-5geospan_device_not_a_signature A-5geospan_device_signature_failed A-6geospan_device_signature_missing A-6geospan_device_unable_to_read_signature

A-6

system event log A-5external SYMAPI locks 6-25

Ffailover

due to application failure 1-25due to hardware failure 1-25forced 1-8, 5-27manual 1-25operation 2-2

failstop 3-51, 5-14failure

modes 2-8types 2-2

FastSwap 1-10fault-tolerant system 1-26forced failover 1-8, 5-27

GGeoSpan v1.2.3 to SRDF/CE upgrade procedure

4-4GeoSpan_res.dll

errors 6-28prevents MSCS group failover 6-25stays offline after quorum failback 6-25

groupenabling 6-33renaming 6-7

groups settings 5-31array operations 5-40create new device group 5-35list 5-38MSCS move 5-39MSCS offline 5-39MSCS online 5-39pull groups from SYMCLI 5-38set R1R2 swap 5-40unlock 5-39

Hhardware

NIC 2-4requirements 3-2SRDF/CE architecture 1-16

heartbeat


Index

link failure 2-4host bus adapter

driver software requirements 3-4failure 2-4, 6-14

IInstallation

Wizard 1-7, 3-15installation

prerequisites 3-7requirements 3-2, 7-2

IPcommunication warning 6-29communications failure 6-13

LLAN link failure 2-4lateral node 1-13list

device groups 5-38Symmetrix information 5-48

localmirrors 1-17override 5-13

locks 5-49, 6-25log

edit parameters 5-19file viewer 5-18interpreting A-4level 5-18, 7-27path 5-18

logging control 5-18

Mmain SRDF/CE features 1-6majority node set 1-8, 3-47, 7-14manual failover 1-25Microsoft

required software 3-3technical support 6-22

mirroring 1-17MNS 1-8, 3-47, 7-14modes of operation

active/active 1-19active/passive 1-19

modifying cluster groups 6-7MSCS

Cluster Administrator 6-3converting to an SRDF/CE cluster 3-65error 170 6-30error while attempting to connect 6-30move 5-39offline 5-39online 5-39service 6-31

multicluster configurations 6-3multiple

remote adapters 1-11SRDF/CE clusters 6-3

Nnew device group 5-35newly added disk, unable to see 6-26no new onlines 3-51, 5-14node settings 5-52

Ooffline

data redundancy 1-26MSCS 5-39

overridelocal 2-9, 3-50, 5-13SRDF 2-9, 3-50, 5-13, 5-27, 6-12, 6-15

overviewadministrative procedures 6-2Configuration Utility GUI 5-2Configuration Wizard 3-20error messages 6-28Installation Wizard 3-7MSCS 1-23SRDF 1-3SRDF/CE 1-2SRDF/CE and Cluster Behavior 2-1

PPC emulation software 4-2, 7-3peer node 1-13

settings 5-52preinstallation worksheet 3-12, B-1prerequisites

i-3EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide

i-4

Index

for automatic SRDF/CE installation 3-7for manual SRDF/CE installation 4-2, 7-3for upgrading SRDF/CE 4-2

pull groups from SYMCLI 5-38push/pull configuration to a node 1-11, 5-54

Qquorum

control 5-7log size 6-33upgrading 6-20

RR1/R2 swap 1-10, 5-33, 5-40, 5-44, 7-16rebooting 3-63recovery

from corrupt quorum log 6-15from HBA or SCSI channel failure 6-14from site failure 6-12from SRDF link failure 6-14from total IP communications failure 6-13

remoteadapters 1-11storage array 5-50

renaminggroups 6-7groups in MSCS 3-66

replace remote storage array 1-12, 5-50requirements

converting MSCS cluster to SRDF/CE 3-65hardware 3-2HBA driver software 3-4installing and configuring SRDF/CE 3-2, 7-2Microsoft software 3-3pre-SRDF/CE installation planning 3-12SRDF/CE installation and operating 3-6upgrading SRDF/CE 4-2

restore 5-7from site failure 6-10

SSCSI channel failure 6-14SCSIport driver 3-4select and group devices 5-35service dependencies 6-31

site failuremodes 2-8recovery 6-12restoring 6-10

site mode 3-51operations 5-13setting 5-14

software requirements 3-3SRDF 1-3, 6-3

asynchronouscompatibility 1-9failover method 5-16, 5-41, 7-21group 3-30, 3-49, 5-36, 5-39

coexistence 1-22link failure 6-14override 2-9, 3-50, 5-13, 5-27, 6-12, 6-15swap 1-10synchronous 3-30, 5-39warning 6-29

SRDF/Acompatibility 1-9failover method 5-16, 5-41, 7-21group 3-30, 3-49, 5-36, 5-39

SRDF/CEadministration 6-5automatic installation prerequisites 3-7behavior 2-1behavior override 2-9Configuration Utility GUI 1-6, 5-2configuration with multiple RAs 1-11Configuration Wizard 1-8converting from MSCS cluster 3-65failover operation 2-2group in wrong state 6-23hardware architecture 1-16icon 5-3installation requirements 3-6Installation Wizard 1-7log file viewer 5-18log level 5-18, 7-27log path 5-18main features 1-6manual installation prerequisites 4-2, 7-3overview 1-2push/pull configuration to a node 1-11replacing the remote storage array 1-12response to complete site failure 2-8


Index

SRDF swap 1-10uninstalling 3-75, 7-30unique behavior 2-5unique cluster behavior 6-4V2.0 to V2.1 upgrade procedure 4-8

SRDF/CE Configuration Utility GUIcontrol settings 5-6

advanced control 5-15checkpoint 5-6forced failover 5-27logging control 5-18quorum control 5-7restore 5-7site mode operations 5-13

groups settings 5-31nodes settings 5-52storage settings 5-42

storage array, remote 5-50STORport driver 3-4SYMAPI

database 7-26locks 6-25

SYMCLI pull groups 5-38Symmetrix

list 5-48locks 5-49replace remote storage array 5-50settings 5-42unlock 5-49

system event log A-5

Ttechnical support

EMC 6-22Microsoft 6-22

troubleshootingconfiguration 6-22EMC technical support 6-22Microsoft technical support 6-22

Uunable

to see newly added disk 6-26to start cluster service 6-24

uninstallingcluster service on Windows 2003 6-5SRDF/CE 3-75, 7-30

uniquecluster behavior 6-4

unlockdevice groups 5-39Symmetrix 5-49

upgrade procedureGeoSpan v1.2.3 to SRDF/CE 4-4quorum disk 6-20requirements 4-2SRDF/CE V2.0 to SRDF/CE V2.1 4-8Windows 2000 to Windows 2003 6-17

user credentials 5-3

Wwarning

IP communication 6-29SRDF 6-29

Windows 2000 to 2003 upgrade procedure 6-17

i-5EMC SRDF/Cluster Enabler for MSCS Version 2.1 Product Guide

i-6

Index


Date post:	24-Jun-2018
Category:	Documents
Upload:	ngocong
View:	231 times
Download:	0 times

EMC SRDF/Cluster Enabler for MSCS SRDF/Cluster Enabler for MSCS Version 2.1 Product ... Uninstalling...

Documents