7/31/2019 Veritas Cluster Knowledge Sharing

1/23

7/31/2019 Veritas Cluster Knowledge Sharing

2/23

Summary

What is high availability and cluster?VCS Communication Mechanism

VCS Architecture and Agent Monitoring

Fail Over Configurations Installation of Veritas Cluster Server in Unix platforms

7/31/2019 Veritas Cluster Knowledge Sharing

3/23

High Availability & VCS FeaturesCombination of 2 or more servers with appropriate middleware

and interconnects to achieve some or all of the following:

High levels of system reliability (availability)

Monitor, restart and failover

Simplified administration of IT assets (manageability) Java, web interface, Manage multiple clusters

Increased processing power (scalability)

Distributed services, Add systems, storage to running clusters

7/31/2019 Veritas Cluster Knowledge Sharing

4/23

Sample Cluster

7/31/2019 Veritas Cluster Knowledge Sharing

5/23

High Availability Design HA-aware applications

Restart capability Crash-tolerance

HA Management software Site replication Fault detection, notification, and failover Storage management Backup and recovery

Redundant Hardware Power supplies

Network interface cards, hubs, switches storage

7/31/2019 Veritas Cluster Knowledge Sharing

6/23

VCS Terms & Concepts Cluster

A VCS cluster consists of multiple systems connected in various combinations to sharedstorage devices

VCS engines (had) runs on each system in the cluster that controls applications running inthe cluster Systems

Member of a cluster, also referred as node Has its own copy of VCS software Share a common view of cluster configuration which is dynamically communicated whenever

changes are made within the cluster

Connected to public network to provide application services to users Service groups

Related collection of resources All resources within the service group must be available and this cant be spitted across

systems Resources have interdependencies which must be considered during configuration and

management Service groups also have interdependencies

Service group types Failover online on one server at a time. Most applications are configured this way Parallel Online on multiple services at a time. Application must be specifically designed to

run in parallel. E.g., FTP, web servers, Oracle parallel servers

7/31/2019 Veritas Cluster Knowledge Sharing

7/23

VCS Terms & Concepts contd.. Resources

VCS objects that correspond to hardware of software components disks, interface cards, IPs, etc Monitored and controlled by VCS online, offline, monitor Classified by type Identified by unique names and attributes Can depend on other resources within the same service group ( correct order)

Resource Types

General description of the attributes of a resource such as disks, IP, applications, databases Predefined resource types: Mount, Share, IP, NIC Resource Types have inherent interdependencies IP resource type depends on the NIC resource type New resource types can be created as required

Agents Processes that control resources One agent per resource type Agent controls all resources of that type New agents can be added apart from predefined agents

Dependencies The order in which VCS brings the resources and service groups online or offline Parent resource depends on child resource for example Mount depends on Disk Dependencies are homogenous, for example Service group cant depend on a resource

7/31/2019 Veritas Cluster Knowledge Sharing

8/23

VCS Communication Mechanisms Private Network

Minimum of two dedicated communication channels between each system in the cluster Heartbeat communication Determines cluster membership Configuration distribution Configuration & status information for every resource and

service groups are dynamically distributed dynamically to all systems

LLT (Low Latency Transport)

Proprietary networking protocol used by VCS for heartbeat communication and Trafficdistribution Very fast, kernel-to-kernel communication, not routable Uses DLPI over Ethernet

GAB (Group Membership Services/Atomic Broadcast) Manages cluster membership and maintain cluster state Runs over LLT Uses broadcasts to distribute cluster configuration information

VCS Engine (had) Maintains configuration and state information for all cluster resources Uses GAB to communicate among systems Monitored by hashadow process VCS Architecture overview

7/31/2019 Veritas Cluster Knowledge Sharing

9/23

VCS components

VCS: A Replicated State Machine

7/31/2019 Veritas Cluster Knowledge Sharing

10/23

Monitoring and Communication

SG

VCS

HAD

ip agent

nic agent

disk group agent

file system agent

process agent

web dg

web process

web fs web ip

nic

web

7/31/2019 Veritas Cluster Knowledge Sharing

11/23

VCS Architecture Overview

Each cluster system has VCS agents that monitor thestatus of resources

Agents communicate the status of resources to the VCSengine

The VCS engine communicates the status of all localresources to other systems using GAB

GAB uses LLT to communicate between systems over

private network These mechanisms work together to ensure all members

of the cluster maintain identical copy in local memory

7/31/2019 Veritas Cluster Knowledge Sharing

12/23

Fail Over Configurations

Asymmetric

192.1.1.1192.1.1.1

Asymmetric Failover

Before After

7/31/2019 Veritas Cluster Knowledge Sharing

13/23

Fail Over Configurations

Symmetric

192.1.1.1 192.1.1.2 192.1.1.1192.1.1.2

Symmetric Failover

Before After

7/31/2019 Veritas Cluster Knowledge Sharing

14/23

Fail Over Configurations

N to 1

N to 1 failover

7/31/2019 Veritas Cluster Knowledge Sharing

15/23

Application Service Management

Multi Node / Multi GroupClusters

Unlimited numbers of resource(service) groups

Migrate RG without affecting

other RGs Cascading failover

Policy based failover - loaddistribution

Attribute based RGs Easy start, stop and migration

of resource groups - GUI

SGOracle

NBUSGWeb

SGSGOracle

SGWeb

7/31/2019 Veritas Cluster Knowledge Sharing

16/23

VCS Installation on Solaris

Prerequisites: Sun Solaris with latest patches, VxVM, VCS and VxFS

Veritas license keys for all Veritas products(http://vlicense.veritas.com)

A minimum of two physically independent private networks

Crossover cable or hub based on number of nodes Atleast 3 ethernet cards

Shared storage with separate controller

Environment variables PATH=$PATH:/sbin:/usr/sbin:/opt/VRTSvcs/bin

MANPATH=$MANPATH:/opt/VRTS/man

export $PATH, $MANPATH

7/31/2019 Veritas Cluster Knowledge Sharing

17/23

VCS 5.0 InstallationDisk space required:

Operating System Support:

Cluster Manager (Java Console)AIX 5.2 and 5.3

HPUX 11.23 IA and PA - RISCRHEL 4 Update 3 and SLES9 SP3Solaris SPARC 2.8, 2.9, and 2.10

Windows 2000, Windows XP & Windows 2003

7/31/2019 Veritas Cluster Knowledge Sharing

18/23

The installvcs utility VCS packages: VRTSgab, VRTSllt, VRTSperl, VRTSvcs, VRTSvcsag,

VRTSvcsmg, VRTSvcsmn, VRTSvlic, VRTSvcsdc, VRTSweb,VRTSvcsw

It creates VCS communication files: /etc/llttab /etc/gabtab /etc/llthosts

The following configuration files are added: /etc/VRTSvcs/conf/config/main.cf /etc/VRTSvcs/conf/config/types.cf

Installvcs options -installonly -> Only licensing & installation of VCS software -configonly -> creates configuration files without performing

installation -configfile -> unattended installation using VCSvars.pm file

7/31/2019 Veritas Cluster Knowledge Sharing

19/23

Installation parameters Cluster Name

Cluster Number : 0 to 255, Avoid 0 as its default

System Names (host name)

Network interfaces identified for VCS private network Web console configuration Virtual IP, Subnet mask and Network

interface. ClusterService servicegroup is created

SMTP/SNMP notification configuration

7/31/2019 Veritas Cluster Knowledge Sharing

20/23

Managing Cluster Services

Cluster Configuration File: /etc/VRTSvcs/conf/config/main.cf on each system in the

cluster

The cluster configuration in memory is created from main.cf

had daemon maintains configuration in memory and state

information of all cluster resources Starting VCS

hastart command starts the had and hashadow processes

had checks for active configuration in private network

otherwise it loads the configuration from disk. Node-1 starts had & hashadow processes on second node and

process is repeated until all members have identical copies.

7/31/2019 Veritas Cluster Knowledge Sharing

21/23

Managing Cluster Services

Modifying Cluster Configuration haconf makerw -> Cluster configuration opened and .stale file

is created in config directory

haconf dump makero Changes written to disk and .stale fileremoved otherwise it keeps the changes only in memory

In GUI, it prompts to open the configuration when attempting tomodify the configuration

Starting VCS with a stale configuration If .stale file found during startup, it requests the configuration

from another server via private network

.stale file is removed after saving valid configuration to main.cf

7/31/2019 Veritas Cluster Knowledge Sharing

22/23

Managing Cluster Services

Modifying Cluster Configuration haconf makerw -> Cluster configuration opened and .stale file

is created in config directory

haconf dump makero Changes written to disk and .stale fileremoved otherwise it keeps the changes only in memory

In GUI, it prompts to open the configuration when attempting tomodify the configuration

Starting VCS with a stale configuration If .stale file found during startup, it requests the configuration

from another server via private network

.stale file is removed after saving valid configuration to main.cf

7/31/2019 Veritas Cluster Knowledge Sharing

23/23

Thanks & Regards

Kiran Jangam

Fujitsu India Pvt Ltd

References :Symantec Tech notes & VCS Admin Guide 5.0

http://www.symantec.com/connect/forums/top-technotes-veritas-cluster-server-updated-march-2012

http://www.symantec.com/business/support/index?page=content&id=DOC5492

https://sort.symantec.com/public/documents/sf/5.0/solaris/pdf/vcs_users.pdf