EMC Business Continuity for Microsoft Office SharePoint ... · Best practices and recommendations...

EMC Business Continuity for Microsoft Office SharePoint Server 2007

Enabled by EMC CLARiiON CX4,

EMC RecoverPoint/Cluster Enabler, and Microsoft Hyper-V

Proven Solution Guide

Copyright © 2010 EMC Corporation. All rights reserved.

Published October 2010

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

Benchmark results are highly dependent upon workload, specific application requirements, and system design and implementation. Relative system performance will vary as a result of these and other factors. Therefore, this workload should not be used as a substitute for a specific customer application benchmark when critical capacity planning and/or product evaluation decisions are contemplated.

All performance data contained in this report was obtained in a rigorously controlled environment. Results obtained in other operating environments may vary significantly.

EMC Corporation does not warrant or represent that a user can or will achieve similar performance expressed in transactions per minute.

No warranty of system performance or price/performance is expressed or implied in this document. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

All other trademarks used herein are the property of their respective owners.

Part number: H7042

EMC Business Continuity for Microsoft Office SharePoint Server 2007 Enabled by EMC CLARiiON CX4,

EMC RecoverPoint/Cluster Enabler, and Microsoft Hyper-V – Proven Solution Guide 3

Table of Contents

Chapter 1: About this Document .............................................................................. 5

Overview ........................................................................................................................................... 5

Audience and purpose ...................................................................................................................... 6

Scope ................................................................................................................................................ 7

Protecting a SharePoint farm using RecoverPoint/CE ..................................................................... 9

Objectives and findings ................................................................................................................... 10

Reference Architecture ................................................................................................................... 12

Validated environment profile ......................................................................................................... 13

Hardware and software resources .................................................................................................. 13

SharePoint farm content types ........................................................................................................ 15

Prerequisites and supporting documentation .................................................................................. 16

Terminology .................................................................................................................................... 17

Chapter 2: Storage Design ...................................................................................... 18

Overview ......................................................................................................................................... 18

Considerations ................................................................................................................................ 19

Layout .............................................................................................................................................. 19

Storage-system spindle design ....................................................................................................... 26

Space requirements ........................................................................................................................ 26

Cluster details ................................................................................................................................. 28

Chapter 3: File System ............................................................................................. 29

Chapter 4: Application Design ................................................................................ 30

Overview ......................................................................................................................................... 30

Considerations ................................................................................................................................ 31

Application design layout ................................................................................................................ 31

RecoverPoint policy settings ........................................................................................................... 32

Best practices and recommendations ............................................................................................. 33

Chapter 5: Network Design ..................................................................................... 34

Overview ......................................................................................................................................... 34

Considerations ................................................................................................................................ 35

Table of Contents



RecoverPoint network and Fibre Channel configuration ................................................................ 36

Chapter 6: Application Configuration ..................................................................... 37

Overview ......................................................................................................................................... 37

Application configuration ................................................................................................................. 38

Chapter 7: Testing and Validation .......................................................................... 43

Overview ......................................................................................................................................... 43

Test methodology ............................................................................................................................ 44

SharePoint user profiles .................................................................................................................. 45

Tested components ......................................................................................................................... 46

Observations ................................................................................................................................... 47

Section A: Test results summary ............................................................................ 48

Overview ......................................................................................................................................... 48

Testing objectives and results ......................................................................................................... 48

Section B: CLARiiON RecoverPoint splitter testing ................................................ 52

Section C: Baseline RecoverPoint testing .............................................................. 55

Section D: Replication testing ................................................................................. 64

Synchronous test results ................................................................................................................. 64

Asynchronous test results ............................................................................................................... 67

Section E: Virtual machine migration testing .......................................................... 73

Synchronous testing ........................................................................................................................ 73


Section F: Unplanned failover testing ..................................................................... 88



Section G: Planned failback testing ........................................................................ 95

Section H: Query scaling testing ............................................................................. 98


Asynchronous test results ............................................................................................................. 103

Chapter 8: Conclusion ........................................................................................... 109

Chapter 1: About this Document




Overview

Introduction This Proven Solution Guide summarizes a series of best practices that EMC

discovered, validated, or otherwise encountered during the validation of a solution for the business continuity of Microsoft Office SharePoint Server 2007 using EMC® CLARiiON® CX4, EMC RecoverPoint with Cluster Enabler, and Microsoft Windows 2008 R2 Hyper-V.

EMC's commitment to consistently maintain and improve quality is led by the Total Customer Experience (TCE) program, which is driven by Six Sigma methodologies. As a result, EMC has built Customer Integration Labs in its Global Solutions Centers to reflect real-world deployments in which TCE use cases are developed and executed. These use cases provide EMC with an insight into the challenges currently facing its customers.

Use case definition

A use case reflects a defined set of tests that validates the reference architecture for a customer environment. This validated architecture can then be used as a reference point for a Proven Solution.

Contents The content of this chapter includes the following topics.

Topic See Page

Audience and purpose 6

Scope 7

Protecting a SharePoint farm using RecoverPoint/CE 9

Objectives and findings 10

Reference Architecture 12

Validated environment profile 13

Hardware and software resources 13

SharePoint farm content types 15

Prerequisites and supporting documentation 16

Terminology 17




Audience and purpose

Audience The intended audience for the Proven Solution Guide is:

• Internal EMC personnel • EMC partners • Customers

Purpose Microsoft SharePoint Server is a business-critical platform and, as such, should have

the highest levels of availability under all circumstances – for example, a site disaster.

The purpose of this Proven Solution is to demonstrate EMC's replication and automation technology as shown by EMC’s RecoverPoint product suite with RecoverPoint Cluster Enabler (RecoverPoint/CE) in providing fully automated disaster recovery for enterprise-class virtualized SharePoint farm environments.

Using this technology, it is now possible to stretch existing Microsoft Failover Clusters in order to provide geographically separated high availability and disaster recovery. Failover times may vary depending on bandwidth, data change rate, and RecoverPoint configuration, but both the failover and failback times demonstrated by the solution were under 10 minutes. This is significantly less than what can be achieved with native tools.

With RecoverPoint’s data reduction and compression features, SharePoint environments can now be replicated across longer and bandwidth-constrained distances, and customers can expect automated disaster recovery within minutes.




Scope

Scope This Proven Solution describes a virtualized Microsoft Office SharePoint Server 2007

enterprise farm that is protected by RecoverPoint and can serve a particular user count (based on a common user profile). The environment consists of one geographically dispersed six-node Microsoft Windows 2008 R2 Failover Cluster with Hyper-V.

The Proven Solution consists of a SharePoint 2007 Publishing portal that is document-centric. User load can simulate these user actions:

• Browse • Search • Modify

Business challenge

Disaster recovery (DR) for federated applications and environments becomes increasingly difficult to achieve with consistency as the applications and environments grow. As a federated application, SharePoint requires that all server roles, configurations, and data are consistent across the farm.

The challenges and solutions are listed in Table 1.

Table 1. Business challenges and solutions

Challenge Solution

Adjusting to dynamic workloads as data volume grows and user workloads change.

Server virtualization allows for simplified configuration and provisioning and rapid modification of the SharePoint farm when and where necessary.

Maintaining consistency for SharePoint farms to avoid long re-indexing times and severely degraded search capabilities.

RecoverPoint/CE’s support of full farm failover allows the SharePoint farm to resume with crash consistency and minimizes lengthy re-indexing during failover or failback processes.

Performing disaster recovery (DR) for Enterprise SharePoint environments, which is both complex and difficult.

The solution enables fully automated failover. Once configured, failover becomes automatic and planned failback is quick, simple, and causes minimal disruption.

Enterprise SharePoint environments can stretch to tens of servers, with differing roles, such as Index, SQL servers, and application servers.

Writing DR plans to meet failover and recovery service level agreements (SLAs) becomes an arduous task, and failovers become unreliable or fail. Typically, with legacy solutions, critical working components of a SharePoint farm, such as a valid search content index, are lost or broken on failover.




Technology solution

This solution describes a virtualized Microsoft Office SharePoint Server 2007 enterprise farm environment, protected with remote disaster recovery and fully automated failover, enabled by EMC technology.

The environment consists of a six-node Hyper-V Windows Failover Cluster, with three active nodes (production site) and three passive nodes (DR site). The cluster contains the entire host infrastructure required to operate an Office SharePoint Server 2007 farm, for example, domain controllers, application servers, Web front ends (WFEs), and SQL servers.

SharePoint Server 2007 uses Microsoft SQL Server 2008 as its data store.

Microsoft Windows 2008 R2 Enterprise with Hyper-V provides the virtualization platform to maximize hardware utilization and improve SharePoint performance and availability. Hyper-V enables virtual machine high availability through Microsoft Windows Failover Clustering (WFC) and provides both Live and Quick Migration features. In synchronous replication mode, the solution supported Live Migration of virtual machines between sites with minimal disruption to the availability of the virtual machine.

EMC CLARiiON CX4-240 arrays provide consolidated, managed, and highly-available storage for both production and DR sites. In addition, they provide an in-built mechanism (CLARiiON splitter) for RecoverPoint to provide continuous remote replication (CRR) of production data to the DR site.

RecoverPoint CRR can operate in two modes:

• Synchronous replication over an FC inter-site link Remote data is kept in sync with the production site, enabling zero recovery point objective (RPO) and minute-based recovery time objectives (RTO). Distances up to 200 km or 4 ms latencies are supported.

• Asynchronous replication over an IP WAN link Remote data is asynchronously sent to the remote site. RecoverPoint’s Bandwidth Reduction feature allows for replication over longer distances, slower WAN links, or both. Metropolitan to intercontinental distances can be achieved.

Fully integrating with Microsoft Windows failover methodologies, RecoverPoint/CE ensures that a complete site failover can happen with minimal downtime and zero user intervention. Planned failover of a virtual machine from site to site is now possible from Microsoft Windows Failover Cluster Manager. The operator does not need to be trained in RecoverPoint or CLARiiON technologies in order to achieve this.

Integration of this solution in an existing CLARiiON CX4-240 SharePoint environment requires minimal downtime (minutes to convert cluster groups) and a minimal footprint per site:

• Two server racks for RecoverPoint appliances (RPAs) • Eight 4/8 GB FC ports • Four 1 GigE network ports • RecoverPoint/CE software installed on all cluster nodes




Protecting a SharePoint farm using RecoverPoint/CE

Protecting a SharePoint farm

When a failure occurs on a SharePoint farm, it is very important to keep certain farm components consistent, especially the SQL databases and the Search function.

If the Index server is out of sync with the Search database, the Index server will start a full incremental crawl that can take many days to complete. During the crawl, new documents will not be searchable; this may breach your Search SLAs, and farm performance will be hindered.

Certain companies with large data sets, for example 35 TB, quote up to 39 days in order to re-crawl their entire farm.

RecoverPoint provides crash-consistency across the entire SharePoint farm.

Therefore, if a failure occurs, the farm will restart on the local or DR side (depending on the failure) without inconsistency issues. EMC RecoverPoint with RecoverPoint/CE automatically provides crash recovery with minimal service disruption and a fully functional SharePoint farm within minutes of a failure.




Objectives and findings

Objectives and findings

The objectives and findings of this Proven Solution are listed in Table 2.

Table 2. List of objectives

Objective Details Findings

Baseline performance testing

• Conduct performance testing to determine the performance baseline for the SharePoint 2007 farm.

• Measure Microsoft SQL Server load, Microsoft Internet Information Services load, and passed tests/second.

• 248,280 users at 1% concurrency • CPU load highest on WFEs at

close to 100%

Baseline RecoverPoint testing

• Conduct testing to observe and document the performance impact that RecoverPoint replication has on the SharePoint farm.

• 249,360 users at 1% concurrency • 1% increase in CLARiiON SP

utilization

Synchronous distance testing

• Determine the optimum distances and latencies for RecoverPoint replication over synchronous links.

• Understand what, if any, overhead or infrastructure impact RecoverPoint replication incurs.

• Maximum distance 100 km round-trip over a 1 Gb/s link

Asynchronous distance testing

• Determine the optimum distances and latencies for RecoverPoint replication over asynchronous links.

• Understand what, if any, overhead or infrastructure impact RecoverPoint replication incurs.

• Maximum distance of 1,600 km round-trip over a 900 Mb/s link

RecoverPoint migration testing

• Conduct migration tests to determine the impact of virtual machine migration within the production site. (lateral move)

• Conduct migration tests to determine the impact of virtual machines migrating from production to the DR site using RecoverPoint. (peer move)

• Determine the time taken to migrate various farm virtual machines.

• Lateral moves successful, only downtime impact from primary SQL server migration

• Peer moves to DR site and back successful, only downtime impact from primary SQL server migration

RecoverPoint failover testing

• Determine the failover capabilities of RecoverPoint.

• Unplanned synchronous failover to DR of the entire farm resulted in 7 minutes downtime with a fully operational farm failed over automatically




• Unplanned asynchronous failover to DR of the entire farm resulted in 7 minutes downtime with a fully operational farm failed over automatically

Query scaling testing • Determine the distances achievable by using fewer Query servers, which reduces bandwidth usage over synchronous and asynchronous links.

• Query scaling in a synchronous environment increased distance to 300 km round-trip with a 1 Mb/s link

• Query scaling in a synchronous environment increased distance to 2,500 km round-trip with a 900 Mb/s link




Reference Architecture

Corresponding Reference Architecture

This Proven Solution Guide has a corresponding Reference Architecture document that is available on Powerlink®, EMC.com, and KB.Wiki. Refer to EMC Business Continuity for Microsoft Office SharePoint Server 2007 Enabled by EMC CLARiiON CX4, EMC RecoverPoint/Cluster Enabler, and Microsoft Hyper-V – Reference Architecture for details.

If you do not have access to this content, contact your EMC representative.

Reference Architecture diagram

The solution’s overall physical architecture is shown in Figure 1.

Figure 1. Reference Architecture diagram




Validated environment profile

Profile characteristics

EMC validated the solution with the environment profile as shown in Table 3.

Table 3. Profile characteristics

Profile Characteristic Value

SharePoint farm user data 1.5 TB

Concurrency 1%

Site collections 15

Sites per site collection 1

SQL 2008 virtual machine 2 instances (active/active)

Hyper-V cluster (physical) 6 nodes (3 production / 3 DR)

Web Front Ends (virtual machines) 6 (also running Query role)

Excel Services (virtual machine) (hosting Central Admin)

1

Index servers (virtual machine) 1

Application servers (virtual machines) 1

SCVVM (physical) 1

Hardware and software resources

Hardware The hardware used to validate the solution is listed in Table 4.

Table 4. Hardware

Equipment Quantity Configuration

Storage array (Production site)

1 CLARiiON CX4-240 with: • 8 FC ports per SP • FLARE® 29 • 44 * 300 GB 15k FC disks

Storage array (DR site)

1 CLARiiON CX4-240 with: • 8 FC ports per SP • FLARE 29 • 52 * 300 GB 15k FC disks

Distance emulation device

1 Anue H-series GEM + FC dual-blade Network Emulator software v3.3.5

Network switch 4 48-port Cisco trunkable network switches (2 production and 2 disaster recovery)




FC switch 3 48-port FC 4 GB switch (2 production and 1 DR)

Hyper-V server 6 16-core, 48 GB RAM

Infrastructure server 2 8-core, 16 GB RAM

RecoverPoint appliance 4 GEN3

Software The software used to validate the solution is listed in Table 5.

Table 5. Software

Software Version

Windows Server 2008 Enterprise Edition R2 RTM

Microsoft Hyper-V 2008 R2

Microsoft SQL Server 2008 64-bit Enterprise Edition SP1

Microsoft Office SharePoint Server 2007 SP2

Microsoft SCVMM 2008 R2 RTM

PowerPath® (with VE capabilities) 5.3 SP1

Visual Studio Test Suite 2008 SP1

KnowledgeLake DocLoader 1.1

EMC RecoverPoint 3.2 SP2 Patch 2

EMC RecoverPoint/Cluster Enabler 4.0.1




SharePoint farm content types

Farm content types

The SharePoint farm content types are detailed in Table 6.

Table 6. Farm content types

Type Size (KB)

DOC 251

DOCX 102

XLSX 20

MPP 235

PPTX 189

JPG 93

GIF 75

VSD 471

The average document size (including document weighting) was 187 KB, which is indicative of real-world average file sizes.




Prerequisites and supporting documentation

Technology It is assumed the reader has a general knowledge of:

• EMC CLARiiON CX4-240 • EMC RecoverPoint • EMC RecoverPoint/CE • Microsoft SharePoint • Microsoft Hyper-V • Microsoft Cluster services

Supporting documents

The following documents, located on Powerlink.com, provide additional, relevant information. Access to these documents is based on your login credentials. If you do not have access to the following content, contact your EMC representative.

• EMC RecoverPoint Administration Guide • EMC RecoverPoint/CE Administration Guide • Disaster Recovery for Windows Using EMC RecoverPoint/Cluster Enabler -

Applied Technology • Disaster Recovery in a Geographically Dispersed Cross-Site Virtual Environment -

Enabled by the EMC CLARiiON CX4 Platform, EMC RecoverPoint, and Microsoft Hyper-V on Windows Server 2008

• CLARiiON best practices

Third-party documents

The following documents are available on the Microsoft website:

• Hyper-V Deployment and Best Practices (online presentation) • Running SQL Server 2008 in a Hyper-V Environment: Best Practices and

Performance Considerations (SQL Server technical article)

http://www.microsoft.com/virtualization/assets/media/hdbp/local/index.htm�

http://download.microsoft.com/download/d/9/4/d948f981-926e-40fa-a026-5bfcf076d9b9/SQL2008inHyperV2008.docx�

http://download.microsoft.com/download/d/9/4/d948f981-926e-40fa-a026-5bfcf076d9b9/SQL2008inHyperV2008.docx�




Terminology

Introduction This section defines the terms used in this document.

Term Definition

Synchronous replication

Synchronous replication ensures that data replicated at a secondary site is an identical copy of the primary site with no data lag between the primary and secondary site. The secondary site has to acknowledge receipt of each write before the next write can occur.

Asynchronous replication

Asynchronous replication ensures that data replicated at a secondary site is an identical copy of the primary site with some data lag. The writes on the primary site can continue while the acknowledgment from the secondary site can be delayed.

Consistency group A consistency group is a set of mirrors that are managed as a single entity and whose secondary images always remain in a consistent and restartable state with respect to their primary image and each other.

Recovery point objective (RPO)

RPO is the point in time to which systems and data must be recovered after an outage. This defines the amount of data loss a business can endure.

Recovery time objective (RTO)

RTO is the period of time within which the systems, applications, or functions must be recovered after an outage. This defines the amount of downtime that a business can endure and survive.

Microsoft Visual Studio Team System (VSTS)

VSTS is an Application Lifecycle Management solution that has four role-specific editions, all based on Microsoft Team Foundation Server (TFS) as the underlying platform. VSTS was used during test validation to generate and emulate user load.

Live migration Microsoft Hyper-V live migration feature that moves a running virtual machine from one physical host to another without any disruption of service or perceived downtime.

Quick migration Microsoft Hyper-V quick migration feature that rapidly moves a running virtual machine from one physical host system to another with minimal downtime.

Virtual hard disk (VHD)

VHD format is the common virtualization file format that captures the entire virtual machine operating system and the application stack in a single file stores on a file system in the parent partition.

Chapter 2: Storage Design




Overview

Introduction EMC has a number of documents that identify recommendations and guidelines

associated with the operation of:

• EMC CLARiiON and Microsoft SQL Server 2005 • EMC storage design for Microsoft SharePoint Server 2007 • EMC CLARiiON storage systems

For more information, refer to Chapter 1: About this Document > Prerequisites and supporting documentation > Supporting documents.

These best practices form the basis of this Proven Solution. To determine the optimum storage design, follow the guidelines listed below:

• Determine the required number of IOPS that the storage system must support including a factor for utilization growth (and future local/remote replication spindle overhead).

• Determine the maximum user count, user profiles, and user concurrency. • Define the common and uncommon operations: Search; Modify; and Browse. • Define customer response time SLAs for common and uncommon operations. • Determine the size of the database and log LUNs. • Determine the size of the Index and Query LUNs. • Determine the required size of the virtual machine OS drives for various farm roles. • Determine the required size of the RecoverPoint Repository and the Journal LUNs.

Scope The storage design layout instructions presented in this chapter apply to the specific

components used during the development of this solution.

Contents This chapter contains the following topics:

Topic See Page

Considerations 19

Layout 19

Storage-system spindle design 26

Space requirements 26

Cluster details 28




Considerations

Storage Design Considerations

The optimal disk layout was determined from preliminary tests in the environment in order to profile the storage IOPS requirements per virtual machine and role. From this, storage design (RAID group configurations, number of LUNs per RAID group, and so on) could be determined based on tested IOPS calculations.

LUN response times are dependent on what the role of the LUN is in the SharePoint environment, and so can vary greatly. For example, the SQL server TempDBs and SearchDBs require a quicker response time than the Application server LUN. During the design phase, it was ensured that the LUN response times from the CLARiiON were not a limiting factor for SharePoint farm and RecoverPoint performance. For RecoverPoint, the response times of the Repository and Journal LUNs were very important.

Layout

Introduction Storage on the CLARiiON CX4-240 was allocated in order to ensure optimal

performance of the SharePoint farm and RecoverPoint components.

It was determined during testing based on IOPS that the TempDB and SearchDBs required RAID 1/0 LUNs. All other SharePoint LUNs were RAID 5. One hot spare was allocated on each CLARiiON DAE bus.

The RecoverPoint Repository and Journal LUNs were all RAID 1/0. This was to ensure the best performance for RecoverPoint, especially the Repository LUN.

Goal To optimize storage usage and performance of all components in the solution.




SharePoint and RecoverPoint production storage layout

The layout of the RAID groups for the production site is shown in Figure 2.

Figure 2. SharePoint and RecoverPoint production storage layout




SharePoint and RecoverPoint DR storage layout

The layout of the RAID groups for the DR site is shown in Figure 3. Additional RAID 1/0 groups were used for the larger Journal LUNs to accommodate RecoverPoint on the DR site.

Figure 3. SharePoint and RecoverPoint DR storage layout

Production SharePoint farm disk layout

The CLARiiON disk layout legend for Figure 2 in this solution is shown in Table 7.

Table 7. CLARiiON disk layout legend for production

DAE RAID type Allocation

0_0 RAID 5, 4+1 • OS LUNs, WFE OS and Query LUNs

1_0

RAID 5, 4+1 RAID 1/0, 2+2 RAID 1/0 2+2 HS

• ContentDB 1-5 Data LUNs • SearchDB data and log LUNs • TempDB data and log LUNs • RecoverPoint Journals • Hot spare

0_1

RAID 5, 4+1 RAID 1/0 2+2

• ContentDB 11-15 Data LUNs • ContentDB log LUNs • SQL internal DB LUNs • SearchDB data and log LUNs • TempDB data and log LUNs




RAID 1/0 2+2 HS

• RecoverPoint Journals • Hot spare

1_1 RAID 5, 4+1 • ContentDB 6-10 Data LUNs

DR SharePoint farm disk layout

The CLARiiON disk layout legend for Figure 3 in this solution is shown in Table 8.

Table 8. CLARiiON disk layout legend for DR

DAE RAID type Allocation

0_0 RAID 5, 4+1 • OS LUNs, WFE OS and Query LUNs • RecoverPoint Repository

1_0

RAID 5, 4+1 RAID 1/0 2+2 RAID 1/0 2+2 HS

• ContentDB 15 Data LUNs • SearchDB data and log LUNs • TempDB data and log LUNs • RecoverPoint Journals • RecoverPoint Repository • Hot spare

0_1

RAID 5, 4+1 RAID 1/0 2+2 RAID 1/0 2+2 HS

• ContentDB 11-15 Data LUNs • ContentDB log LUNs • SQL internal DB LUNs • SearchDB data and log LUNs • TempDB data and log LUNs • RecoverPoint Journals • Hot spare

1_1 RAID 5, 4+1 • ContentDB 6-10 Data LUNs

0_2 RAID 1/0 2+2 • RecoverPoint Journals

1_2 RAID 1/0 2+2 • RecoverPoint Journals

OS and application LUN usage

The SharePoint farm usage with respective LUN sizes and total LUN space is shown in Table 9.

Table 9. SharePoint farm usage – LUN size and space

CX4-240 array LUN usage LUN sizes Total LUN space

WFEs 6 x 40 GB OS LUNs 6 x 70 GB Query LUNs

660 GB

Domain controllers 2 x 25 GB OS LUNs 50 GB

Index server 50 GB OS LUN 200 GB Index LUN

250 GB




Excel and application servers 2 x 50 GB 100 GB

Total 1.060 TB

RecoverPoint LUN usage – production array

The RecoverPoint array usage with respective LUN sizes and total LUN space on the production array is shown in Table 10.

Table 10. RecoverPoint array usage – LUN size and space – production array


RecoverPoint Repository 1 x 5 GB 5 GB

WFEs, application server Journals 10 x 5 GB 50 GB

Index Journal 1 x 30 GB 30 GB

SQL Journals 2 x 150 GB 300 GB

Total 385 GB

RecoverPoint LUN usage – DR array

The RecoverPoint array usage with respective LUN sizes and total LUN space on the DR array is shown in Table 11.

Table 11. RecoverPoint array usage – LUN size and space – DR array


RecoverPoint Repository 1 x 5 GB 5 GB

WFEs, application server Journals 10 x 10 GB 100 GB

Index Journal 1 x 30 GB 30 GB

SQL Journals 4 x 150 GB 600 GB

Total 735 GB




Storage design for SQL databases

Introduction to Storage design for SQL databases

The first step in determining the storage design for a database is to assess the performance requirements of the applications that run on the database, and the type of load that places on the database and storage system.

The main questions are:

• How many IOPS will the SharePoint farm generate on the storage system? • What is the maximum acceptable LUN response rate in ms?

In a production environment, the best way to obtain this information is to analyze the current application and database performance.

Options for assessing performance requirements

The optimal disk layout was determined based on preliminary tests in the environment. From this, the disks could be laid out based on real IOPS calculations.

LUN response times are dependent on what the role of the LUN is in SQL, and so can vary greatly. During the design phase, it was ensured that the LUN response times from the CLARiiON were not a limiting factor for SharePoint farm performance.

Database storage layout

Based on observed IOPS:

• Database data LUNs were evenly spread over 3 x 5-disk RAID 5 LUNs • Database log LUNs resided on a 1 x 5-disk RAID 5 LUN • Search and Temp databases and log files resided on 2 x 4-disk RAID 1/0 LUNs




Storage design for RecoverPoint

Introduction The first step in determining the storage design for RecoverPoint is to assess the

performance requirements of the applications, and the type of load that places on the database and storage system.

The main questions are:

• How many write IOPS will the SharePoint farm generate on the storage system? • What is the maximum acceptable LUN response rate in ms? • Is this a synchronous or an asynchronous replication environment? • How does this affect the IOPS to the RecoverPoint Repository LUN and Journal

LUNs?

In a production environment, the best way to obtain this information is to analyze the current application and database performance.

Options for assessing performance requirements

The optimal disk layout was determined based on preliminary tests in the environment. From this, the disks could be laid out based on real IOPS calculations.

LUN response times are very important for RecoverPoint because slow response times can lead to high loads or application delays. During the design phase, it was ensured that LUN response times from the CLARiiON were not a limiting factor for RecoverPoint performance. The production Repository LUN had the highest IOPS of all the RecoverPoint LUNs.

Database storage layout

Based on observed IOPS:

• Production RecoverPoint Repository and Journal LUNs were evenly spread over a 1 x 4-disk RAID 1/0 group

• DR RecoverPoint Journal LUNs were evenly spread over 3 x 4-disk RAID 1/0 groups

• The DR Repository LUN resided on a 1 x 5-disk RAID 5 shared group




Storage-system spindle design

Spindle count used

The spindle count used for production is:

• 30 x 300 GB FC, RAID 5, 15k rpm disks • 8 x 300 GB FC, RAID 10, 15k rpm disks • 4 x 300 GB FC, RAID 10, 15k rpm disks (RecoverPoint) • 2 x 300 GB FC, 15k rpm disks (hot spares)

The spindle count used for DR is:

• 30 x 300 GB FC, RAID 5, 15k rpm disks • 8 x 300 GB FC, RAID 10, 15k rpm disks • 12 x 300 GB FC, RAID 10, 15k rpm disks (RecoverPoint) • 2 x 300 GB FC, 15k rpm disks (hot spares)

Space requirements

Space requirement validation

After the spindle requirements for performance have been calculated, space requirements need to be validated.

SharePoint space requirements

SharePoint space requirements are shown in Table 12.

Table 12. SharePoint space requirements

Storage Item Configuration

Array disk WFE OS LUNs 240 GB RAID 5, 5 x 300 GB FC 15k rpm (shared)

Array disk WFE Query LUNs 420 GB RAID 5, 5 x 300 GB FC 15k rpm (shared)

Array disk AD and Application Server OS LUNs

150 GB RAID 5, 5 x 300 GB FC 15k rpm (shared)

Array disk Index server OS and Index LUNs

250 GB RAID 5, 5 x 300 GB FC 15k rpm (shared)




SQL files The space requirements for the SQL database were determined using best practice guidelines in conjunction with the calculations specific to the environment as shown in Table 13.

Table 13. Space requirements for SQL database

Quantity of LUNs

Item LUN size Configuration (formatted capacity)

10 Content DB data

200 GB 3 x 1000 GB RAID 5, 15 x 300 GB FC 15k rpm

10 Content DB log

25 GB 250 GB RAID 5, 5 x 300 GB FC 15k rpm (shared)

3 Search DB data

100 GB 300 GB RAID 1/0, 4 x 300 GB FC 15k rpm (shared)

1 Search DB log


3 Temp DB data


1 Temp DB log


RecoverPoint The space requirements for the RecoverPoint Repository and Journal volumes were

determined using best practice guidelines in conjunction with the calculations specific to the environment. Space requirements for production and DR are shown in Table 14 and Table 15, respectively.

Table 14. RecoverPoint space requirements for production

Item LUN size

Configuration (formatted capacity)

Repository LUN


Journal LUNs


Table 15. RecoverPoint space requirements for DR

Item LUN size

Configuration (formatted capacity)

Repository LUN

5 GB 5 GB RAID 5, 5 x 300 GB FC 15k rpm (shared)

Journal LUNs

730 GB 730 GB RAID 1/0, 4 x 300 GB FC 15k rpm




Cluster details

Hyper-V cluster design and configuration

In Windows Server 2008 R2, Hyper-V has the ability to do live migrations.

The cluster consisted of six active nodes, three production, and three DR. In this way, EMC could test a lateral move as well as a full site failure.

Chapter 3: File System



Chapter 3: File System

Introduction to file system

This chapter details the file system layout for SharePoint and RecoverPoint. Storage enumeration for all Hyper-V virtual machines boot LUNs were configured as virtual hard disks (VHDs). All application LUNs were configured as Pass-through disks.

SharePoint The solution used the Microsoft maximum recommended sizing of 100 GB for each

SQL SharePoint 2007 content database. A total of 1.5 TB of data was split into 15 x 100 GB content databases, each on a 200 GB LUN (file system). All virtual machines used the CX4-240 for their file systems.

The SharePoint file system layout is the same as that listed in the EMC Virtual Architecture for Microsoft SharePoint Server 2007 Enabled by EMC CLARiiON CX3-40, Microsoft Windows 2008, and Hyper-V Integration Guide.

This integration guide is available on EMC Powerlink. If you do not have access to the document, contact your EMC representative.

RecoverPoint Thirteen RecoverPoint Journal LUNs were allocated, one for each consistency group

(CG). One RecoverPoint Repository LUN was allocated as well.

The same number of LUNs for RecoverPoint was allocated on the DR array. The LUNs on the DR array were bigger because, in normal operation with the SharePoint farm on the production side, more data needs to be stored there.

The data that is stored on a Journal LUN consists of periodic bookmarks for a consistency group, which allows you to restore to a specific point in time, if necessary. The larger the Journal LUN, the more bookmarks that can be stored, and therefore providing the ability to go back further in time for recovery.

Chapter 4: Application Design




Overview

Introduction to application design

This chapter details the application design for the Proven Solution.

SharePoint was configured to ensure continued stability and performance as RecoverPoint replication occurred. RecoverPoint was configured to optimize replication performance rates and meet the objectives of the Proven Solution.

This chapter deals with three main elements:

• SharePoint farm design • RecoverPoint design • RecoverPoint/CE design

Contents This section contains the following topics:

Topic See Page

Considerations 31

Application design layout 31

RecoverPoint policy settings 32

Best practices and recommendations 33




Considerations

Design considerations

The RecoverPoint application planning took into account the impact of RecoverPoint on the storage, the SharePoint farm, SQL Server, and the network infrastructure.

Application design layout

SharePoint configuration

The SharePoint farm was designed as a publishing portal. Fifteen SharePoint sites (document centers) were populated with random user data and configured as one site collection.

As the WFEs are CPU-intensive, each WFE virtual machine was allocated four virtual CPUs. The Excel calculation server was configured as the Central Administration server. As crawling is a CPU-intensive activity, the Index server was configured with four CPUs. To improve the crawl speed, the Index server also ran a dedicated WFE role, so all Index requests were maintained within one machine. The incremental crawl schedule was set to 15 minutes. In addition, every front-end WFE server was a Query server for search purposes.

RecoverPoint/ CE configuration

Each virtual machine was configured as a Microsoft cluster group and a RecoverPoint consistency group. Each cluster group and therefore each consistency group consisted of the virtual machine and all associated disks. When a Microsoft cluster group is migrated to a DR site node, RecoverPoint/CE notifies RecoverPoint to migrate the consistency group to the DR array so that the DR node has access to all the cluster group disks. RecoverPoint then reverses the direction of replication so that the production array has a copy of the consistency group data on the DR array.

In order to ensure transportability between sites at an individual virtual machine level, it is important to understand that each virtual machine requires dedicated LUNs and cannot be shared.




RecoverPoint policy settings

RecoverPoint policy settings

The policy settings for each consistency group during synchronous replication were:

• Best compression • RPO set to system optimized lag

The policy settings for each consistency group during asynchronous replication were:

• Best compression • RPO set to a maximum lag of 30 seconds




Best practices and recommendations

Introduction When designing a SharePoint farm for use with RecoverPoint, it is important to find a

balance between the performance of the farm, the storage performance, and the impact of RecoverPoint replication on the farm.

Best practices Failover Clustering and RecoverPoint

If the cluster service is restarted on a cluster node, or a cluster node is rebooted, image access in RecoverPoint must first be enabled for the node to access the LUNs. Once the node can access the LUNs, image access must be disabled for that consistency group or failover to that cluster node will fail. This is a cluster service limitation.

Consistency Groups Each consistency group in RecoverPoint should correspond to a cluster group, which in turn should correspond to a virtual machine. For this Proven Solution, EMC used the convention where the cluster group names were also used as the names for the RecoverPoint consistency groups.

Sufficient Bandwidth There should be sufficient bandwidth allocated to ensure that no consistency group goes into a high load state because this causes an initialization, which is a full sync of that consistency group. If this occurs, the consistency group is out of sync and protection is compromised. Initializations also cause more traffic to be sent across the WAN link.

Networking The public and private networks should be bridged between the production site and DR site so that a failover does not affect connectivity for the users accessing the SharePoint farm.

Query Server scaling If bandwidth is very limited, then consider reducing the number of Query servers to reduce network traffic between the sites. Most of the network traffic in the SharePoint 2007 farm is due to index propagations.

Preferred owners You should set the Preferred Owners order in Cluster Management to ensure balanced failover of all the cluster groups to the correct DR servers.

Chapter 5: Network Design




Overview

Introduction to network design

This chapter details the RecoverPoint network and Fibre Channel connectivity.


Topic See Page

Considerations 35

RecoverPoint network and Fibre Channel configuration 36




Considerations

Considerations To ensure performance of the RecoverPoint components, 1 GB Ethernet

connections were used for all network connections.

The multiple networks with which the servers were configured, comprised of a cluster heartbeat, public, private, iSCSI 1 and 2 for production, iSCSI 1 and 2 for DR, and a Live Migration network, are listed in Table 16.

Table 16. Networks

Network Description

Cluster heartbeat Dedicated cluster heartbeat network

Public External network for client access, for example, WFEs

Private Internal network for SharePoint, for example, SQL servers

Live migration Dedicated network for Microsoft Live Migration




RecoverPoint network and Fibre Channel configuration

RecoverPoint network and Fibre Channel configuration

The RecoverPoint Appliances (RPAs) for the production and DR sites were connected by a trunked network between two switches in order for them to communicate with each other, as shown in Figure 4. Each RPA was connected to the network switches by 2 x 1 GB connections. The network switches were connected to the Distance Emulator using the WAN VLAN. The other networks, for example, public, were trunked between the switches for IP connectivity between the sites.

In synchronous replication mode, the RPAs were connected by Fibre Channel to the distance emulator, and the distance emulator blades were connected by a Fibre Channel connection. The distance emulator could then be set to emulate distance and latency for the Fibre Channel connection.

In asynchronous replication, the distance emulator blades were connected by a 1 GB Ethernet connection. The distance emulator could then be set to emulate distance and latency for the Ethernet connection.

There were 4 x 4 GB Fibre Channel connections from the RPAs to the CLARiiON CX4-240, with two to each CX4-240 storage processor (SP) in each site.

Figure 4. RecoverPoint network configuration

Chapter 6: Application Configuration




Overview

Introduction to application configuration

This chapter shows examples of the configuration that EMC used in this Proven Solution.




Application configuration

Hyper-V virtual machine settings

The Hyper-V virtual machines had the boot drives configured as VHDs and all secondary drives as Pass-through disks.

The drive configuration as well as other basic settings for one of the SQL virtual machines is shown in Figure 5. In Hyper-V, the boot drive must be configured as an IDE device. All other Pass-through disks were configured as SCSI disks. Each SCSI controller could control a maximum number of 15 disks. The maximum number of CPUs per virtual machine in Hyper-V is four.

Figure 5. Hyper-V virtual machine configuration




Cluster resource configuration

The disks for each virtual machine are added as cluster resources in the virtual machine groups once the virtual machine is configured for high availability in Failover Cluster Manager as shown in Figure 6.

There are three production nodes (tce-sp-r900x) and three DR nodes (tce-sp-dr900x) in the cluster.

Figure 6. Cluster resource configuration




Consistency group disk resources

In RecoverPoint, you can see the disk resources for the virtual machine in a consistency group as shown in Figure 7.

Each replication set consists of the production LUN and the DR replica LUN. The LUNs have to be of the same size. Each consistency group has a Journal LUN on the production and DR arrays. The Journal LUNs on the DR arrays are bigger because in normal operation the farm is active on the production array. The bigger the Journal LUN, the more bookmarks that can be kept by RecoverPoint, so that the consistency group can be rolled back if required.

Figure 7. Consistency group disks




Setting the preferred owners for server failover

In Cluster Manager, RecoverPoint/CE automatically sets the preferred owners in a specific order so that, during a server failure event, the virtual machines first fail to other servers in the same production site. This is called a lateral move.

If all the cluster nodes in the production site fail, then the virtual machines fail over to specific cluster nodes on the DR site. This is called a peer move.

The Cluster group property “Auto Start” needs to be enabled for the virtual machine to resume after a failover, as shown in Figure 8.

Figure 8. Preferred Owners settings




Consistency group traffic flow

The normal flow of traffic for one of the consistency groups is shown in Figure 9.

The green lines show that data is flowing from the production host to the local storage. At the same time, it is being replicated to the remote Journal volume and then to the remote storage.

Since the remote RPA will send an acknowledgment of the data being received on the remote site, there is very little delay compared to the data actually being committed to the remote storage before acknowledgment. The system traffic window shows how much data is being committed locally as well as how much data is being sent over the remote link.

Figure 9. Traffic flow for a consistency group

Chapter 7: Testing and Validation




Overview

Introduction to testing and validation

This chapter details the performance testing under user load using Microsoft Visual Studio Team Suite (Microsoft VSTS).


Topic See Page

Test methodology 44

SharePoint user profiles 45

Tested components 46

Observations 47




Test methodology

Data population The data population tool used a set of sample documents. Altering the document title

and document metadata before insertion made each document unique to SharePoint.

There was one load agent host for each WFE, allowing data to be loaded in parallel until the 1.5 TB (approximately 6.8 million documents) data set goal was reached. The data was spread evenly across the 15 site collections (each a unique content database).

Testing tools Microsoft VSTS 2008 was used to simulate load on the SharePoint farm using

custom test code from an independent third party, KnowledgeLake, a Microsoft Gold Partner.

Load generation

To generate and emulate client load, Microsoft VSTS was used in conjunction with KnowledgeLake code to simulate real-world, SharePoint user activity.

The VSTS team test rig in this Proven Solution consisted of seven virtual machines – one controller and six VSTS team agent hosts. The controller evenly distributed client load across the agent hosts.

Passed tests per second

Passed tests per second are the number of user actions (Browse, Search, or Modify) per second that the SharePoint farm can service.




SharePoint user profiles

Profile mix A common mix of the user profiles was used to emulate different types of business

organizations. For example, some organizations are browse-intensive, while others are search-intensive, modify-intensive, or both. The common user profile used consisted of a mix of Browse, Search, and Modify. The ratio used was:

• 80% Browse • 10% Search • 10% Modify

All tests were run from a load controller host that spread the load evenly across each of the six load agent hosts. The load controller host also collected performance metrics from all the load agents and hosts in the farm for analysis.

Browse In a Browse test, the code simulates a user browsing a site until the user reaches an

end document listing that contains no sub-pages.

Search In a Search test, the code simulates a user running a stored procedure in the SQL

database to find a unique number, in this case a Social Security Number (SSN).

The code then performs a web request to search for that unique number.

Modify In a Modify test, the code simulates a user retrieving a document. The document

name is extracted from the database prior to each test run. The code then modifies the metadata for that document before saving it back to the farm in its modified form.

User capacity calculation

All users were run against a Microsoft heavy user profile, that is, 60 requests per hour.

Zero percent think time was applied to all tests. “0% think time” is the elimination of typical user decision time when browsing, searching, or modifying in Microsoft Office SharePoint Server. For example, a single complete user request is completed from start to finish without user pause, therefore creating a continuous workload on the system.

The maximum user capacity is derived from the following formula:

# = seconds per hour / RPH / Concurrency% * RPS

Example: 3600 / 60 / 1% * 54 = 324,000

Example: 3600 / 60 / 10% * 54 = 32,400




Tested components

Tested components

The testing for this Proven Solution is related to the RecoverPoint product protecting data on a SharePoint farm.

The tested components were:

• RecoverPoint functionality and performance • Impact of RecoverPoint replication on the SharePoint farm • RecoverPoint/CE integration with Microsoft Failover Cluster • Distance testing between the production and DR sites • Query scaling • Live and quick migrations • Full farm unplanned failover • Full farm planned failback




Observations

Index server propagation spikes

The Index server consistency group statistics are shown in Figure 10. There are spikes in the charts that represent propagations and updates of data being written to the Index server. These large writes cause time lags, as the data has to be replicated to the remote site.

In the Proven Solution, the Index server did incremental crawls every 15 minutes. Each incremental crawl then started a propagation to the Query servers.

Figure 10. Index server consistency group




Section A: Test results summary

Overview

Introduction to test results summary

This section provides a short summary of all the Proven Solution test results. Details of the testing results are provided in the subsequent sections.

Graphical content

Charts and images shown in the subsequent sections are from tests that were run on the SharePoint farm with RecoverPoint/CE.

Testing objectives and results

Introduction to testing objectives

RecoverPoint reliably and consistently replicated data in the SharePoint farm. A number of tests were performed and are listed as follows.

Objective 1: Baseline performance testing

Baseline performance testing consisted of maximum user load on the production SharePoint farm.

Results The results of the baseline performing testing show:

• 248,280 users

Objective 2: Baseline RecoverPoint testing

Baseline RecoverPoint testing consisted of maximum user load on the production SharePoint farm with the RecoverPoint splitter installed on the CLARiiON array and replicating data to the DR site.

Results The results of the baseline RecoverPoint testing show:

• 249,360 users • 1% more CLARiiON SP utilization • RecoverPoint achieved an average compression ratio of 3.42:1




Objective 3: Synchronous distance testing

Synchronous distance testing consisted of maximum user load testing on the farm with synchronous replication to the DR site over various distances.

Results The results of synchronous distance testing are shown in Table 17 and indicate that

it is possible to support a bandwidth of 1 Gb/s within a 0-100 km distance. Table 17. Synchronous distance testing results

Distance scenario: Synchronous replication (FC)

Achieved results

Round-trip distance (km)

Latency (ms)

Bandwidth (Mb/s)

Baseline (same site) 0 0 1,000

Metro CWDM/DWDM 100 1 1,000

Objective 4: Asynchronous distance testing

Asynchronous distance testing consisted of maximum user load testing on the farm with asynchronous replication to the DR site over various distances.

Results The results of asynchronous distance testing are shown in Table 18. The RPO was a

maximum 30 seconds between the production and DR sites.

Table 18. Asynchronous distance testing results

Distance scenario: Asynchronous replication (IP)

Achieved results

Round-trip distance (km)

Latency (ms)

Bandwidth (Mb/s)

Baseline (same site) 0 0 300

City-to-city, shorter distance 400 4 500

State-to-state/inter-country (Europe)

1,600 16 900

Objective 5: RecoverPoint migration testing

RecoverPoint migration testing consisted of using Microsoft live migration and quick migration to migrate various virtual machines to the local site and the DR site. The synchronous testing was done over a 1 Gb/s link with 1 ms latency, which equates to a 100-km round trip. The asynchronous testing was done over a 300 Mb/s link with no latency.




Results The results of RecoverPoint migration testing are shown in Table 19.

Table 19. Migration testing results

* virtual machine downtime: The amount of time that the virtual machine is unresponsive to application requests.

** Cluster migration time: The amount of time required by the cluster to fail over the cluster group (a subset of the overall virtual machine downtime).

This is the time from an offline request to a virtual machine cluster group on the source node to the time that the virtual machine online action

completes on the target node.

Note Live migration - Virtual machines with large memory configurations take longer to migrate than virtual machines with smaller memory configurations. This is because active memory is copied over the network to the receiving cluster node prior to migration. Live migration requires high bandwidth and low latency networks in order to function.

Quick migration - This migration type is essentially a suspend-to-disk and then resume operation. Virtual machine memory must be committed to disk before migration. This causes a significant number of burst writes, which need to be replicated quickly, and therefore requires high-bandwidth WAN links between sites.

virtual machine role

Failover type

Synchronous Asynchronous

virtual machine downtime (seconds) *

Cluster migration time (seconds) **

virtual machine downtime (seconds) *

Cluster migration time (seconds) **

Live migration

Domain controller

Local to remote

45 35 105 63

Web front end Local to remote

210 38 210 60

SQL Database

Local to remote

105 27 210 52

Web front end Local to local

30 1 30 1

Quick migration

Domain controller

Remote to local

105 26 75 26

Web front end Remote to local

210 26 105 26

SQL Database

Remote to local

750 27 570 55

Web front end Local to local

135 1 90 1




Objective 6: RecoverPoint failover testing

RecoverPoint failover testing consisted of the unplanned failover of the entire farm from the production site to the DR site during synchronous and asynchronous replication.

Results The results of RecoverPoint failover testing show:

• Synchronous failover in 7 minutes; total downtime: 7 minutes • Asynchronous failover in 7 minutes; total downtime: 7 minutes

Objective 7: RecoverPoint planned failback testing

RecoverPoint planned failback testing consisted of planned failback of the entire farm from the DR site to the production site during asynchronous replication.

Results The results of RecoverPoint planned failback testing show:

• Planned failback with 2 minutes downtime

Objective 8: Query scaling testing

Query scaling testing consisted of reducing the number of Query servers, which reduced SharePoint propagation traffic. Because of reduced propagation traffic, less data needed to be replicated, and longer distances between sites could be achieved.

Reducing SharePoint propagation traffic significantly reduces the amount of data that needs to be replicated across the WAN.

Results The results of Query scaling testing are shown in Table 20.

Table 20. Query scaling testing results

Number of Query servers

Round-trip distance (km) Latency (ms) Bandwidth (Mb/s)

Asynchronous Query server scaling

6 1,600 16 900

4 1,800 18 900

2 2,500 25 900

Synchronous Query server scaling

6 100 1 1,000

4 200 2 1,000

2 300 3 1,000




Section B: CLARiiON RecoverPoint splitter testing

SharePoint farm with and without RecoverPoint splitter

The passed tests per second for the SharePoint farm with the RecoverPoint splitter enabled versus not enabled are shown in Figure 11.

Passed tests/sec baseline with no splitter:

• 41.38 – 248,280 users at 1% concurrency

Passed tests/sec baseline with splitter enabled:

• 41.56 – 249,360 users at 1% concurrency

These results show that the RecoverPoint splitter has minimal impact on farm performance.

Figure 11. Passed tests/sec results, with and without the RecoverPoint splitter

0

5

10

15

20

25

30

35

40

45

50

00:1

5:00

00:2

1:15

00:2

7:30

00:3

3:45

00:4

0:00

00:4

6:15

00:5

2:30

00:5

8:45

01:0

5:00

01:1

1:15

01:1

7:30

01:2

3:45

01:3

0:00

01:3

6:15

01:4

2:30

01:4

8:45

01:5

5:00

02:0

1:15

02:0

7:30

02:1

3:45

Passed Tests/Sec no splitter

Passed Tests/Sec baseline




CLARiiON SP utilization, without RecoverPoint splitter

The CLARiiON SP utilization on the SharePoint farm without the RecoverPoint splitter is shown in Figure 12.

The average SP CPU utilization was 12.85%.

Figure 12. CLARiiON SP utilization, without the RecoverPoint splitter

0

5

10

15

20

25

Tue

Mar

23

09:5

8:19

GM

T 20

10Tu

e M

ar 2

3 10

:00:

19 G

MT

2010

Tue

Mar

23

10:0

2:19

GM

T 20

10Tu

e M

ar 2

3 10

:04:

19 G

MT

2010

Tue

Mar

23

10:0

6:19

GM

T 20

10Tu

e M

ar 2

3 10

:08:

19 G

MT

2010

Tue

Mar

23

10:1

0:19

GM

T 20

10Tu

e M

ar 2

3 10

:12:

19 G

MT

2010

Tue

Mar

23

10:1

4:19

GM

T 20

10Tu

e M

ar 2

3 10

:16:

19 G

MT

2010

Tue

Mar

23

10:1

8:19

GM

T 20

10Tu

e M

ar 2

3 10

:20:

19 G

MT

2010

Tue

Mar

23

10:2

2:19

GM

T 20

10Tu

e M

ar 2

3 10

:24:

19 G

MT

2010

Tue

Mar

23

10:2

6:19

GM

T 20

10Tu

e M

ar 2

3 10

:28:

19 G

MT

2010

Tue

Mar

23

10:3

0:19

GM

T 20

10Tu

e M

ar 2

3 10

:32:

19 G

MT

2010

Tue

Mar

23

10:3

4:19

GM

T 20

10Tu

e M

ar 2

3 10

:36:

19 G

MT

2010

Tue

Mar

23

10:3

8:19

GM

T 20

10Tu

e M

ar 2

3 10

:40:

19 G

MT

2010

Tue

Mar

23

10:4

2:19

GM

T 20

10Tu

e M

ar 2

3 10

:44:

19 G

MT

2010

Tue

Mar

23

10:4

6:19

GM

T 20

10Tu

e M

ar 2

3 10

:48:

19 G

MT

2010

Tue

Mar

23

10:5

0:19

GM

T 20

10Tu

e M

ar 2

3 10

:52:

19 G

MT

2010

Tue

Mar

23

10:5

4:19

GM

T 20

10Tu

e M

ar 2

3 10

:56:

19 G

MT

2010

Tue

Mar

23

10:5

8:19

GM

T 20

10

SP A - Utilization (%)

SP B - Utilization (%)




Browse, Search, and Modify test times

The test times for Browse, Search, and Modify are shown in Figure 13.

Browse averaged 1.23 seconds, Search averaged 1.01 seconds, and Modify averaged 1.15 seconds.

Figure 13. Average test time results for Browse, Search, and Modify

0

0.5

1

1.5

2

2.5

3

3.5

4

00:1

5:00

00:2

0:15

00:2

5:30

00:3

0:45

00:3

6:00

00:4

1:15

00:4

6:30

00:5

1:45

00:5

7:00

01:0

2:15

01:0

7:30

01:1

2:45

01:1

8:00

01:2

3:15

01:2

8:30

01:3

3:45

01:3

9:00

01:4

4:15

01:4

9:30

01:5

4:45

02:0

0:00

02:0

5:15

02:1

0:30

02:1

5:45

Avg. Test Time, Browse

Avg. Test Time, Search

Avg. Test Time, Modify




Section C: Baseline RecoverPoint testing

RecoverPoint replication enabled

The passed tests per second with synchronous RecoverPoint replication enabled with 1 Gb/s bandwidth and no latency are shown in Figure 14.

Passed tests/sec baseline with splitter enabled: 41.56, which equates to 249,360 users at 1% concurrency.

Figure 14. Passed tests/sec results, RecoverPoint replication enabled

0

5

10

15

20

25

30

35

40

45

50

00:1

5:00

00:1

9:45

00:2

4:30

00:2

9:15

00:3

4:00

00:3

8:45

00:4

3:30

00:4

8:15

00:5

3:00

00:5

7:45

01:0

2:30

01:0

7:15

01:1

2:00

01:1

6:45

01:2

1:30

01:2

6:15

01:3

1:00

01:3

5:45

01:4

0:30

01:4

5:15

01:5

0:00

01:5

4:45

01:5

9:30

02:0

4:15

02:0

9:00

02:1

3:45

Passed Tests/Sec

Passed Tests/Sec








0

0.5

1

1.5

2

2.5

3

3.5

4

00:1

5:00

00:2

0:15

00:2

5:30

00:3

0:45

00:3

6:00

00:4

1:15

00:4

6:30

00:5

1:45

00:5

7:00

01:0

2:15

01:0

7:30

01:1

2:45

01:1

8:00

01:2

3:15

01:2

8:30

01:3

3:45

01:3

9:00

01:4

4:15

01:4

9:30

01:5

4:45

02:0

0:00

02:0

5:15

02:1

0:30

02:1

5:45







CLARiiON SP utilization, RecoverPoint splitter enabled

The CLARiiON SP utilization on the SharePoint farm with the RecoverPoint splitter enabled is shown in Figure 16.

With an average of 13.81, it is 1% higher than without the RecoverPoint splitter. This is still very low SP CPU utilization overall.

Figure 16. CLARiiON SP utilization, RecoverPoint splitter enabled

0

5

10

15

20

25

30

35

Tue

Feb

02 1

2:28

:10

GM

T 20

10Tu

e Fe

b 02

12:

30:1

0 G

MT

2010

Tue

Feb

02 1

2:32

:10

GM

T 20

10Tu

e Fe

b 02

12:

34:1

0 G

MT

2010

Tue

Feb

02 1

2:36

:10

GM

T 20

10Tu

e Fe

b 02

12:

38:1

0 G

MT

2010

Tue

Feb

02 1

2:40

:10

GM

T 20

10Tu

e Fe

b 02

12:

42:1

0 G

MT

2010

Tue

Feb

02 1

2:44

:10

GM

T 20

10Tu

e Fe

b 02

12:

46:1

0 G

MT

2010

Tue

Feb

02 1

2:48

:10

GM

T 20

10Tu

e Fe

b 02

12:

50:0

9 G

MT

2010

Tue

Feb

02 1

2:52

:09

GM

T 20

10Tu

e Fe

b 02

12:

54:0

9 G

MT

2010

Tue

Feb

02 1

2:56

:09

GM

T 20

10Tu

e Fe

b 02

12:

58:0

9 G

MT

2010

Tue

Feb

02 1

3:00

:09

GM

T 20

10Tu

e Fe

b 02

13:

02:0

9 G

MT

2010

Tue

Feb

02 1

3:04

:09

GM

T 20

10Tu

e Fe

b 02

13:

06:0

9 G

MT

2010

Tue

Feb

02 1

3:08

:09

GM

T 20

10Tu

e Fe

b 02

13:

10:0

9 G

MT

2010

Tue

Feb

02 1

3:12

:09

GM

T 20

10Tu

e Fe

b 02

13:

14:0

9 G

MT

2010

Tue

Feb

02 1

3:16

:09

GM

T 20

10Tu

e Fe

b 02

13:

18:0

9 G

MT

2010

Tue

Feb

02 1

3:20

:09

GM

T 20

10Tu

e Fe

b 02

13:

22:0

9 G

MT

2010

Tue

Feb

02 1

3:24

:09

GM

T 20

10Tu

e Fe

b 02

13:

26:0

9 G

MT

2010

Tue

Feb

02 1

3:28

:09

GM

T 20

10

SP A - Utilization (%)

SP B - Utilization (%)




RecoverPoint Repository disk utilization

The disk utilization for the RecoverPoint Repository LUN is shown in Figure 17.

Disk utilization averaged 37.79% with some peaks close to 100%. The Repository LUN is very busy, with each disk servicing an average of 136 IOPS.

Figure 17. RecoverPoint Repository disk utilization results

0

10

20

30

40

50

60

70

80

90

100

Bus 0 Enclosure 0 Disk 10 -Utilization (%)








Lag in time between production and DR sites

The maximum lag in time between the production and DR consistency groups was 2.76 seconds, while the average was 0.86 seconds as shown in Figure 18. This is the average amount of time difference between the production site and the DR site if the production site goes down and the DR site has to recover.

Figure 18. Results for lag in time between replicas

0

0.5

1

1.5

2

2.5

3

2010

/02/

02 1

0:23

:16.

935

GM

T

2010

/02/

02 1

0:29

:16.

935

GM

T

2010

/02/

02 1

0:35

:16.

935

GM

T

2010

/02/

02 1

0:41

:16.

935

GM

T

2010

/02/

02 1

0:47

:16.

935

GM

T

2010

/02/

02 1

0:53

:16.

935

GM

T

2010

/02/

02 1

0:59

:16.

935

GM

T

2010

/02/

02 1

1:05

:16.

935

GM

T

2010

/02/

02 1

1:11

:16.

935

GM

T

2010

/02/

02 1

1:17

:16.

935

GM

T

2010

/02/

02 1

1:23

:16.

935

GM

T

2010

/02/

02 1

1:29

:16.

935

GM

T

2010

/02/

02 1

1:35

:16.

935

GM

T

2010

/02/

02 1

1:41

:16.

935

GM

T

2010

/02/

02 1

1:47

:16.

935

GM

T

2010

/02/

02 1

1:53

:16.

935

GM

T

2010

/02/

02 1

1:59

:16.

935

GM

T

2010

/02/

02 1

2:05

:16.

935

GM

T

2010

/02/

02 1

2:11

:16.

935

GM

T

2010

/02/

02 1

2:17

:16.

935

GM

T

2010

/02/

02 1

2:23

:16.

935

GM

T

2010

/02/

02 1

2:29

:16.

935

GM

T

2010

/02/

02 1

2:35

:16.

935

GM

T

2010

/02/

02 1

2:41

:16.

935

GM

T

2010

/02/

02 1

2:47

:16.

935

GM

T

2010

/02/

02 1

2:53

:16.

935

GM

T

2010

/02/

02 1

2:59

:16.

935

GM

T

2010

/02/

02 1

3:05

:16.

935

GM

T

2010

/02/

02 1

3:11

:16.

935

GM

T

2010

/02/

02 1

3:17

:16.

935

GM

T

2010

/02/

02 1

3:23

:16.

935

GM

T

RPO—lag in time between replicas

RPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-APP-01 - sec

RPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-DB1 - sec

RPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-DC-01 - sec

RPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-EXCEL-01 - sec

RPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-WFE-01 - sec





RPO - lag in time between replicas during transfer after init - PRODSITE - RPA2 - TCE-SP-INDEX-01 - sec







Transfer rate The average transfer rate of 1.99 MB/s, as shown in Figure 19, shows the average amount of data per consistency group that needs to be replicated from the production site to the DR site. The total average amount of data being transferred is 51.74 MB/s.

Figure 19. Transfer rate results

0

5

10

15

20

25

2010

/02/

02 1

0:23

:16.

935

GM

T20

10/0

2/02

10:

26:1

6.93

5 G

MT

2010

/02/

02 1

0:29

:16.

935

GM

T20

10/0

2/02

10:

32:1

6.93

5 G

MT

2010

/02/

02 1

0:35

:16.

935

GM

T20

10/0

2/02

10:

38:1

6.93

5 G

MT

2010

/02/

02 1

0:41

:16.

935

GM

T20

10/0

2/02

10:

44:1

6.93

5 G

MT

2010

/02/

02 1

0:47

:16.

935

GM

T20

10/0

2/02

10:

50:1

6.93

5 G

MT

2010

/02/

02 1

0:53

:16.

935

GM

T20

10/0

2/02

10:

56:1

6.93

5 G

MT

2010

/02/

02 1

0:59

:16.

935

GM

T20

10/0

2/02

11:

02:1

6.93

5 G

MT

2010

/02/

02 1

1:05

:16.

935

GM

T20

10/0

2/02

11:

08:1

6.93

5 G

MT

2010

/02/

02 1

1:11

:16.

935

GM

T20

10/0

2/02

11:

14:1

6.93

5 G

MT

2010

/02/

02 1

1:17

:16.

935

GM

T20

10/0

2/02

11:

20:1

6.93

5 G

MT

2010

/02/

02 1

1:23

:16.

935

GM

T20

10/0

2/02

11:

26:1

6.93

5 G

MT

2010

/02/

02 1

1:29

:16.

935

GM

T20

10/0

2/02

11:

32:1

6.93

5 G

MT

2010

/02/

02 1

1:35

:16.

935

GM

T20

10/0

2/02

11:

38:1

6.93

5 G

MT

2010

/02/

02 1

1:41

:16.

935

GM

T20

10/0

2/02

11:

44:1

6.93

5 G

MT

2010

/02/

02 1

1:47

:16.

935

GM

T20

10/0

2/02

11:

50:1

6.93

5 G

MT

2010

/02/

02 1

1:53

:16.

935

GM

T20

10/0

2/02

11:

56:1

6.93

5 G

MT

2010

/02/

02 1

1:59

:16.

935

GM

T20

10/0

2/02

12:

02:1

6.93

5 G

MT

2010

/02/

02 1

2:05

:16.

935

GM

T20

10/0

2/02

12:

08:1

6.93

5 G

MT

2010

/02/

02 1

2:11

:16.

935

GM

T20

10/0

2/02

12:

14:1

6.93

5 G

MT

2010

/02/

02 1

2:17

:16.

935

GM

T20

10/0

2/02

12:

20:1

6.93

5 G

MT

2010

/02/

02 1

2:23

:16.

935

GM

T20

10/0

2/02

12:

26:1

6.93

5 G

MT

2010

/02/

02 1

2:29

:16.

935

GM

T20

10/0

2/02

12:

32:1

6.93

5 G

MT

2010

/02/

02 1

2:35

:16.

935

GM

T20

10/0

2/02

12:

38:1

6.93

5 G

MT

2010

/02/

02 1

2:41

:16.

935

GM

T20

10/0

2/02

12:

44:1

6.93

5 G

MT

2010

/02/

02 1

2:47

:16.

935

GM

T20

10/0

2/02

12:

50:1

6.93

5 G

MT

2010

/02/

02 1

2:53

:16.

935

GM

T20

10/0

2/02

12:

56:1

6.93

5 G

MT

2010

/02/

02 1

2:59

:16.

935

GM

T20

10/0

2/02

13:

02:1

6.93

5 G

MT

2010

/02/

02 1

3:05

:16.

935

GM

T20

10/0

2/02

13:

08:1

6.93

5 G

MT

2010

/02/

02 1

3:11

:16.

935

GM

T20

10/0

2/02

13:

14:1

6.93

5 G

MT

2010

/02/

02 1

3:17

:16.

935

GM

T20

10/0

2/02

13:

20:1

6.93

5 G

MT

2010

/02/

02 1

3:23

:16.

935

GM

T

Transfer Rate

Incoming writes rate for link - PRODSITE - RPA1 - TCE-SP-APP-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-DB1 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-DC-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-EXCEL-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-02 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-03 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-DB2 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-DC-02 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-INDEX-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-WFE-04 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-WFE-05 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-WFE-06 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA1 - TCE-SP-APP-01 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA1 - TCE-SP-DB1 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA1 - TCE-SP-DC-01 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA1 - TCE-SP-EXCEL-01 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA1 - TCE-SP-WFE-01 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA1 - TCE-SP-WFE-02 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA1 - TCE-SP-WFE-03 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA2 - TCE-SP-DB2 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA2 - TCE-SP-DC-02 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA2 - TCE-SP-INDEX-01 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA2 - TCE-SP-WFE-04 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA2 - TCE-SP-WFE-05 - Megabytes/secTotal Output rate for link during transfer - PRODSITE - RPA2 - TCE-SP-WFE-06 - Megabytes/sec(blank) - (blank) - (blank) - (blank) - (blank)




WAN throughput

The average WAN throughput of 90.59 Mb/s, as shown in Figure 20, shows how much data needs to be replicated from the production site to the DR site. This equates to 11.32 MB/s, which illustrates the advantage of RecoverPoint data compression.

Figure 20. RPA and site WAN throughput results

0

50

100

150

200

250

300

2010

/02/

02 1

0:23

:16.

935

GM

T20

10/0

2/02

10:

28:1

6.93

5 G

MT

2010

/02/

02 1

0:33

:16.

935

GM

T20

10/0

2/02

10:

38:1

6.93

5 G

MT

2010

/02/

02 1

0:43

:16.

935

GM

T20

10/0

2/02

10:

48:1

6.93

5 G

MT

2010

/02/

02 1

0:53

:16.

935

GM

T20

10/0

2/02

10:

58:1

6.93

5 G

MT

2010

/02/

02 1

1:03

:16.

935

GM

T20

10/0

2/02

11:

08:1

6.93

5 G

MT

2010

/02/

02 1

1:13

:16.

935

GM

T20

10/0

2/02

11:

18:1

6.93

5 G

MT

2010

/02/

02 1

1:23

:16.

935

GM

T20

10/0

2/02

11:

28:1

6.93

5 G

MT

2010

/02/

02 1

1:33

:16.

935

GM

T20

10/0

2/02

11:

38:1

6.93

5 G

MT

2010

/02/

02 1

1:43

:16.

935

GM

T20

10/0

2/02

11:

48:1

6.93

5 G

MT

2010

/02/

02 1

1:53

:16.

935

GM

T20

10/0

2/02

11:

58:1

6.93

5 G

MT

2010

/02/

02 1

2:03

:16.

935

GM

T20

10/0

2/02

12:

08:1

6.93

5 G

MT

2010

/02/

02 1

2:13

:16.

935

GM

T20

10/0

2/02

12:

18:1

6.93

5 G

MT

2010

/02/

02 1

2:23

:16.

935

GM

T20

10/0

2/02

12:

28:1

6.93

5 G

MT

2010

/02/

02 1

2:33

:16.

935

GM

T20

10/0

2/02

12:

38:1

6.93

5 G

MT

2010

/02/

02 1

2:43

:16.

935

GM

T20

10/0

2/02

12:

48:1

6.93

5 G

MT

2010

/02/

02 1

2:53

:16.

935

GM

T20

10/0

2/02

12:

58:1

6.93

5 G

MT

2010

/02/

02 1

3:03

:16.

935

GM

T20

10/0

2/02

13:

08:1

6.93

5 G

MT

2010

/02/

02 1

3:13

:16.

935

GM

T20

10/0

2/02

13:

18:1

6.93

5 G

MT

2010

/02/

02 1

3:23

:16.

935

GM

T

RPA and site WAN throughput

WAN throughput from site - PRODSITE - (blank) - (blank) -Megabits/sec




Compression ratio

The average compression ratio of 3.42:1 with peaks of over 8:1, as shown in Figure 21, shows that the RecoverPoint devices are getting reasonable compression of the data before sending it over the WAN.

This allows less bandwidth to be used and reduces the lag in time between replicas.

Figure 21. Compression ratio results

0123456789

2010

/02/

02 1

0:23

:16.

935

GM

T

2010

/02/

02 1

0:28

:16.

935

GM

T

2010

/02/

02 1

0:33

:16.

935

GM

T

2010

/02/

02 1

0:38

:16.

935

GM

T

2010

/02/

02 1

0:43

:16.

935

GM

T

2010

/02/

02 1

0:48

:16.

935

GM

T

2010

/02/

02 1

0:53

:16.

935

GM

T

2010

/02/

02 1

0:58

:16.

935

GM

T

2010

/02/

02 1

1:03

:16.

935

GM

T

2010

/02/

02 1

1:08

:16.

935

GM

T

2010

/02/

02 1

1:13

:16.

935

GM

T

2010

/02/

02 1

1:18

:16.

935

GM

T

2010

/02/

02 1

1:23

:16.

935

GM

T

2010

/02/

02 1

1:28

:16.

935

GM

T

2010

/02/

02 1

1:33

:16.

935

GM

T

2010

/02/

02 1

1:38

:16.

935

GM

T

2010

/02/

02 1

1:43

:16.

935

GM

T

2010

/02/

02 1

1:48

:16.

935

GM

T

2010

/02/

02 1

1:53

:16.

935

GM

T

2010

/02/

02 1

1:58

:16.

935

GM

T

2010

/02/

02 1

2:03

:16.

935

GM

T

2010

/02/

02 1

2:08

:16.

935

GM

T

2010

/02/

02 1

2:13

:16.

935

GM

T

2010

/02/

02 1

2:18

:16.

935

GM

T

2010

/02/

02 1

2:23

:16.

935

GM

T

2010

/02/

02 1

2:28

:16.

935

GM

T

2010

/02/

02 1

2:33

:16.

935

GM

T

2010

/02/

02 1

2:38

:16.

935

GM

T

2010

/02/

02 1

2:43

:16.

935

GM

T

2010

/02/

02 1

2:48

:16.

935

GM

T

2010

/02/

02 1

2:53

:16.

935

GM

T

2010

/02/

02 1

2:58

:16.

935

GM

T

2010

/02/

02 1

3:03

:16.

935

GM

T

2010

/02/

02 1

3:08

:16.

935

GM

T

2010

/02/

02 1

3:13

:16.

935

GM

T

2010

/02/

02 1

3:18

:16.

935

GM

T

2010

/02/

02 1

3:23

:16.

935

GM

T

RPA compression ratio

Compression ratio - PRODSITE - (blank) - (blank) - (blank)




RPA CPU utilization due to compression

The average RPA CPU utilization due to compression was 17.26% as shown in Figure 22.

There were some peaks over 50%, but all within acceptable CPU usage parameters.

Figure 22. RPA CPU utilization due to compression results

0

10

20

30

40

50

60

2010

/02/

02 1

0:23

:16.

935 …

2010

/02/

02 1

0:28

:16.

935 …

2010

/02/

02 1

0:33

:16.

935 …

2010

/02/

02 1

0:38

:16.

935 …

2010

/02/

02 1

0:43

:16.

935 …

2010

/02/

02 1

0:48

:16.

935 …

2010

/02/

02 1

0:53

:16.

935 …

2010

/02/

02 1

0:58

:16.

935 …

2010

/02/

02 1

1:03

:16.

935 …

2010

/02/

02 1

1:08

:16.

935 …

2010

/02/

02 1

1:13

:16.

935 …

2010

/02/

02 1

1:18

:16.

935 …

2010

/02/

02 1

1:23

:16.

935 …

2010

/02/

02 1

1:28

:16.

935 …

2010

/02/

02 1

1:33

:16.

935 …

2010

/02/

02 1

1:38

:16.

935 …

2010

/02/

02 1

1:43

:16.

935 …

2010

/02/

02 1

1:48

:16.

935 …

2010

/02/

02 1

1:53

:16.

935 …

2010

/02/

02 1

1:58

:16.

935 …

2010

/02/

02 1

2:03

:16.

935 …

2010

/02/

02 1

2:08

:16.

935 …

2010

/02/

02 1

2:13

:16.

935 …

2010

/02/

02 1

2:18

:16.

935 …

2010

/02/

02 1

2:23

:16.

935 …

2010

/02/

02 1

2:28

:16.

935 …

2010

/02/

02 1

2:33

:16.

935 …

2010

/02/

02 1

2:38

:16.

935 …

2010

/02/

02 1

2:43

:16.

935 …

2010

/02/

02 1

2:48

:16.

935 …

2010

/02/

02 1

2:53

:16.

935 …

2010

/02/

02 1

2:58

:16.

935 …

2010

/02/

02 1

3:03

:16.

935 …

2010

/02/

02 1

3:08

:16.

935 …

2010

/02/

02 1

3:13

:16.

935 …

2010

/02/

02 1

3:18

:16.

935 …

2010

/02/

02 1

3:23

:16.

935 …

RPA utilization due to compression

Box utilization - PRODSITE - RPA1 - (blank) - %

Box utilization - PRODSITE - RPA2 - (blank) - %




Section D: Replication testing

Synchronous test results

Synchronous replication testing

The passed tests per second with RecoverPoint replication enabled with 1 Gb/s bandwidth and 1 ms latency are shown in Figure 23.

The passed tests/sec averaged 39.73, which is equivalent to 238,380 users at 1% concurrency. Adding 1 ms latency, which equates to a 100 km round trip time, caused a lower user count as well as a few delays, which can be seen by the downward spikes in the chart.

Figure 23. Passed tests/sec results for synchronous replication

0

5

10

15

20

25

30

35

40

45

50

00:1

5:00

00:1

9:45

00:2

4:30

00:2

9:15

00:3

4:00

00:3

8:45

00:4

3:30

00:4

8:15

00:5

3:00

00:5

7:45

01:0

2:30

01:0

7:15

01:1

2:00

01:1

6:45

01:2

1:30

01:2

6:15

01:3

1:00

01:3

5:45

01:4

0:30

01:4

5:15

01:5

0:00

01:5

4:45

01:5

9:30

02:0

4:15

02:0

9:00

02:1

3:45

Passed Tests/Sec

Passed Tests/Sec






The average search time has a few peaks that happen just after incremental crawls. Incremental crawls cause a lot of write activity on the production farm, which then saturates the WAN link for a short period. All test times were higher because of the increased latency.

The average Browse time was 1.24 seconds, average Search time was 1.24 seconds, and average Modify time was 1.48 seconds.


0

2

4

6

8

10

12

14

16

00:1

5:00

00:2

0:15

00:2

5:30

00:3

0:45

00:3

6:00

00:4

1:15

00:4

6:30

00:5

1:45

00:5

7:00

01:0

2:15

01:0

7:30

01:1

2:45

01:1

8:00

01:2

3:15

01:2

8:30

01:3

3:45

01:3

9:00

01:4

4:15

01:4

9:30

01:5

4:45

02:0

0:00

02:0

5:15

02:1

0:30







DR Journal LUNs disk utilization

The disaster recovery Journal LUNs’ disk usage is shown in Figure 25.

The disk utilization averaged 5.05%. The total average disk IOPS for the Journal LUNs was 135.73.

Figure 25. DR Journal LUNs’ disk utilization results

0

10

20

30

40

50

60

70

80

90

100

11:0

7:30

11:1

3:30

11:1

9:30

11:2

5:30

11:3

1:30

11:3

7:30

11:4

3:30

11:4

9:30

11:5

5:30

12:0

1:30

12:0

7:30

12:1

3:30

12:1

9:30

12:2

5:30

12:3

1:30

12:3

7:30

12:4

3:30

12:4

9:30

12:5

5:30

13:0

1:30

Bus 0 Enclosure 2 Disk 0 - Utilization (%)















Asynchronous test results

Asynchronous replication testing (900 Mb/s)

Figure 26 shows the passed tests per second with RecoverPoint replication enabled with 900 Mb/s bandwidth and 16 ms latency.

Because RecoverPoint was set up as asynchronous, it did not have to wait for an acknowledgement from the DR RPAs to complete a transaction.

Passed tests/sec were 41.24, which is equivalent to 247,440 users at 1% concurrency. Sixteen ms latency equates to 1,600 km round trip distances.

Figure 26. Passed tests/sec results (900 Mb/s)

0

5

10

15

20

25

30

35

40

45

50

00:1

5:00

00:1

9:45

00:2

4:30

00:2

9:15

00:3

4:00

00:3

8:45

00:4

3:30

00:4

8:15

00:5

3:00

00:5

7:45

01:0

2:30

01:0

7:15

01:1

2:00

01:1

6:45

01:2

1:30

01:2

6:15

01:3

1:00

01:3

5:45

01:4

0:30

01:4

5:15

01:5

0:00

01:5

4:45

01:5

9:30

02:0

4:15

02:0

9:00

02:1

3:45

Passed Tests/Sec

Passed Tests/Sec








0

0.5

1

1.5

2

2.5

3

00:1

5:00

00:2

0:15

00:2

5:30

00:3

0:45

00:3

6:00

00:4

1:15

00:4

6:30

00:5

1:45

00:5

7:00

01:0

2:15

01:0

7:30

01:1

2:45

01:1

8:00

01:2

3:15

01:2

8:30

01:3

3:45

01:3

9:00

01:4

4:15

01:4

9:30

01:5

4:45

02:0

0:00

02:0

5:15

02:1

0:30








The passed tests per second with RecoverPoint replication enabled with 500 Mb/s bandwidth and 4 ms latency are shown in Figure 28.

Four ms latency is equivalent to 400 km round trip distances. Passed tests per second of 41.20 equates to 247,200 users at 1% concurrency.

The reason so much bandwidth is needed is that, for each 15-minute incremental crawl, two Query master merges to each Query server, which causes write bursts to the Query LUNs that need to be replicated.


0

5

10

15

20

25

30

35

40

45

50

00:1

5:00

00:1

9:45

00:2

4:30

00:2

9:15

00:3

4:00

00:3

8:45

00:4

3:30

00:4

8:15

00:5

3:00

00:5

7:45

01:0

2:30

01:0

7:15

01:1

2:00

01:1

6:45

01:2

1:30

01:2

6:15

01:3

1:00

01:3

5:45

01:4

0:30

01:4

5:15

01:5

0:00

01:5

4:45

01:5

9:30

02:0

4:15

02:0

9:00

02:1

3:45

Passed Tests/Sec

Passed Tests/Sec








0

0.5

1

1.5

2

2.5

3

00:1

5:00

00:2

0:15

00:2

5:30

00:3

0:45

00:3

6:00

00:4

1:15

00:4

6:30

00:5

1:45

00:5

7:00

01:0

2:15

01:0

7:30

01:1

2:45

01:1

8:00

01:2

3:15

01:2

8:30

01:3

3:45

01:3

9:00

01:4

4:15

01:4

9:30

01:5

4:45

02:0

0:00

02:0

5:15

02:1

0:30








The passed tests per second with RecoverPoint replication enabled with 300 Mb/s bandwidth and no latency are shown in Figure 30.

Passed tests per second of 40.95 equate to 245,700 users at 1% concurrency.


0

5

10

15

20

25

30

35

40

45

50

00:1

5:00

00:1

9:45

00:2

4:30

00:2

9:15

00:3

4:00

00:3

8:45

00:4

3:30

00:4

8:15

00:5

3:00

00:5

7:45

01:0

2:30

01:0

7:15

01:1

2:00

01:1

6:45

01:2

1:30

01:2

6:15

01:3

1:00

01:3

5:45

01:4

0:30

01:4

5:15

01:5

0:00

01:5

4:45

01:5

9:30

02:0

4:15

02:0

9:00

02:1

3:45

Passed Tests/Sec

Passed Tests/Sec








0

0.5

1

1.5

2

2.5

3

00:1

5:00

00:2

0:15

00:2

5:30

00:3

0:45

00:3

6:00

00:4

1:15

00:4

6:30

00:5

1:45

00:5

7:00

01:0

2:15

01:0

7:30

01:1

2:45

01:1

8:00

01:2

3:15

01:2

8:30

01:3

3:45

01:3

9:00

01:4

4:15

01:4

9:30

01:5

4:45

02:0

0:00

02:0

5:15

02:1

0:30







Section E: Virtual machine migration testing

Synchronous testing

Migration testing (1 Gb/s)

The passed tests per second with RecoverPoint replication enabled with 1 Gb/s bandwidth and 1 ms latency are shown in Figure 32. These tests were run at a slightly lower user load of 160,620 at 1% concurrency to simulate normal SharePoint farm activity instead of an extremely busy farm.

The big reductions in passed tests per second visible in the chart occurred during the migration of the SQL server. During this time, the farm was inaccessible to users.

The first downward spike is a live migration. The interruption occurs because the SQL server memory has to be copied across to the DR site before the server can be migrated. The quick migration back to the production site has longer downtime because all the SQL server data has to be committed to disk and replicated before the server can be migrated.

Figure 32. Passed tests/sec results (1 Gb/s)

0

5

10

15

20

25

30

35

40

00:1

5:00

00:1

8:45

00:2

2:30

00:2

6:15

00:3

0:00

00:3

3:45

00:3

7:30

00:4

1:15

00:4

5:00

00:4

8:45

00:5

2:30

00:5

6:15

01:0

0:00

01:0

3:45

01:0

7:30

01:1

1:15

01:1

5:00

01:1

8:45

01:2

2:30

01:2

6:15

01:3

0:00

01:3

3:45

01:3

7:30

01:4

1:15

Passed Tests/Sec

Passed Tests/Sec





Figure 33 shows how the test times were affected during the failover.

Browse peaked at just under 350 seconds.


0

100

200

300

400

500

600

700

00:1

5:00

00:1

9:15

00:2

3:30

00:2

7:45

00:3

2:00

00:3

6:15

00:4

0:30

00:4

4:45

00:4

9:00

00:5

3:15

00:5

7:30

01:0

1:45

01:0

6:00

01:1

0:15

01:1

4:30

01:1

8:45

01:2

3:00

01:2

7:15

01:3

1:30

01:3

5:45

01:4

0:00







Migration of a Domain Controller

The live migration of one of the two Domain Controllers to the DR site and then a quick migration back to the production site is shown in Figure 34.

The live migration took 1 minute and 30 seconds to complete, while the quick migration took 2 minutes. Since there were two Domain Controllers, there was very little impact to the performance of the farm during these migrations.

Figure 34. Percentage processor time during migrations of a Domain Controller

0

10

20

30

40

50

60

70

80

90

100

00:2

0:30

00:2

1:00

00:2

1:30

00:2

2:00

00:2

2:30

00:2

3:00

00:2

3:30

00:2

4:00

00:2

4:30

00:2

5:00

00:2

5:30

00:2

6:00

00:2

6:30

00:2

7:00

00:2

7:30

00:2

8:00

00:2

8:30

TCE-SP-DC-01 % Processor Time





Failover of a Domain Controller

Figure 35 shows the traffic as DC01 is failed over from the production site to the DR site. The migration is initiated in Failover Cluster Manager.

The Total Traffic chart shows that local consistency group traffic is being replaced by site-to-site traffic as the virtual machine is migrated. The spikes in System Traffic, Application Traffic, and Incoming Writes are a result of the migration of the virtual machine.

This view is available in the RecoverPoint Management Application.

Figure 35. Traffic during failover of a Domain Controller




Migration of the SQL server

The live migration of the SQL server from the production site to the DR site, and the quick migration from the DR site to the production site is shown in Figure 36.

The live migration took 2 minutes, while the quick migration took 12 minutes. During these migrations, downtime occurred on the farm because the SQL server was unreachable.

Figure 36. Percentage processor time during migrations of a SQL server

0

10

20

30

40

50

60

70

80

90

100

00:2

8:30

00:3

0:30

00:3

2:30

00:3

4:30

00:3

6:30

00:3

8:30

00:4

0:30

00:4

2:30

00:4

4:30

00:4

6:30

00:4

8:30

00:5

0:30

00:5

2:30

00:5

4:30

00:5

6:30

00:5

8:30

01:0

0:30

01:0

2:30

TCE-SP-DB1 % Processor Time





Migration of a WFE

The live migration and quick migration of one of the WFEs from the production site to the DR site and back are shown in Figure 37.

Since there were six WFEs on the farm, the impact of one WFE being migrated was small. The live migration took 4 minutes, while the quick migration took 3 minutes and 30 seconds.

Figure 37. Percentage processor time during migrations of a WFE

0

10

20

30

40

50

60

70

80

90

100

01:0

5:45

01:0

6:30

01:0

7:15

01:0

8:00

01:0

8:45

01:0

9:30

01:1

0:15

01:1

1:00

01:1

1:45

01:1

2:30

01:1

3:15

01:1

4:00

01:1

4:45

01:1

5:30

01:1

6:15

01:1

7:00

01:1

7:45

01:1

8:30

TCE-SP-WFE-04 % Processor Time





Migration of a WFE

You can view the migration of WFE04 in the RecoverPoint Management Application as well as Failover Cluster Manager as shown in Figure 38.

Figure 38. Migration of a WFE




Migration of a WFE (local to local)

The live migration and quick migration of one of the WFEs from one production site node to another are shown in Figure 39.

This shows the impact of moving a virtual machine within the production site. The live migration took 30 seconds, while the quick migration took 2 minutes and 30 seconds. Since the virtual machine was not migrating to the DR site, it was significantly quicker to migrate.


0

10

20

30

40

50

60

70

80

90

01:2

6:00

01:2

6:45

01:2

7:30

01:2

8:15

01:2

9:00

01:2

9:45

01:3

0:30

01:3

1:15

01:3

2:00

01:3

2:45

01:3

3:30

01:3

4:15

01:3

5:00

01:3

5:45

01:3

6:30

01:3

7:15







Migration testing (300 Mb/s)

The passed tests per second with RecoverPoint replication enabled with 300 Mb/s bandwidth and no latency during failover tests are shown in Figure 40.

The downtime shown in the chart is caused by the migration of the SQL server. Passed tests per second averaged 25.94, which is equivalent to 155,640 users at 1% concurrency.

Figure 40. Passed tests/sec test results (300 Mbps)

0

5

10

15

20

25

30

35

40

00:1

0:00

00:1

4:45

00:1

9:30

00:2

4:15

00:2

9:00

00:3

3:45

00:3

8:30

00:4

3:15

00:4

8:00

00:5

2:45

00:5

7:30

01:0

2:15

01:0

7:00

01:1

1:45

01:1

6:30

01:2

1:15

01:2

6:00

01:3

0:45

01:3

5:30

01:4

0:15

01:4

5:00

01:4

9:45

01:5

4:30

01:5

9:15

02:0

4:00

02:0

8:45

Passed Tests/Sec

Passed Tests/Sec




Lag in time for virtual machine migrations

The lag in time for each of the virtual machines that was failed over during the migration tests is shown in Figure 41.

The Domain Controller and the WFE had a maximum lag in time of 1.25 seconds. The SQL server had a maximum lag in time of 8.25 seconds.

This increase lag in time only occurs while the virtual machine is migrating and then goes back to the baseline lag in time.

Figure 41. Lag in time between replicas during migration failover

0123456789

2010

/03/

11 1

4:22

:02.

462

GM

T

2010

/03/

11 1

4:24

:02.

462

GM

T

2010

/03/

11 1

4:26

:02.

462

GM

T

2010

/03/

11 1

4:28

:02.

462

GM

T

2010

/03/

11 1

4:39

:02.

462

GM

T

2010

/03/

11 1

4:41

:02.

462

GM

T

2010

/03/

11 1

4:43

:02.

462

GM

T

2010

/03/

11 1

4:45

:02.

462

GM

T

2010

/03/

11 1

4:47

:02.

462

GM

T

2010

/03/

11 1

4:49

:02.

462

GM

T

2010

/03/

11 1

4:51

:02.

462

GM

T

2010

/03/

11 1

4:53

:02.

462

GM

T

2010

/03/

11 1

4:55

:02.

462

GM

T

2010

/03/

11 1

4:57

:02.

462

GM

T

2010

/03/

11 1

4:59

:02.

462

GM

T

2010

/03/

11 1

5:01

:02.

462

GM

T

2010

/03/

11 1

5:03

:02.

462

GM

T

2010

/03/

11 1

5:05

:02.

462

GM

T

2010

/03/

11 1

5:07

:02.

462

GM

T

2010

/03/

11 1

5:20

:02.

462

GM

T

2010

/03/

11 1

5:22

:02.

462

GM

T

2010

/03/

11 1

5:24

:02.

462

GM

T

2010

/03/

11 1

5:26

:02.

462

GM

T

2010

/03/

11 1

6:09

:02.

462

GM

T

2010

/03/

11 1

6:11

:02.

462

GM

T

2010

/03/

11 1

6:13

:02.

462

GM

T

2010

/03/

11 1

6:15

:02.

462

GM

T

RPO - lag in time between replicas during transfer

RPO - lag in time between replicas during transfer after init - DRSITE - RPA1 - TCE-SP-DB1 - sec

RPO - lag in time between replicas during transfer after init - DRSITE - RPA1 - TCE-SP-DC-01 - sec

RPO - lag in time between replicas during transfer after init - DRSITE - RPA2 - TCE-SP-WFE-04 - sec






Browse peaked at 500 seconds for a short period of time because of the migration tests.


0

100

200

300

400

500

600

00:1

0:00

00:1

5:15

00:2

0:30

00:2

5:45

00:3

1:00

00:3

6:15

00:4

1:30

00:4

6:45

00:5

2:00

00:5

7:15

01:0

2:30

01:0

7:45

01:1

3:00

01:1

8:15

01:2

3:30

01:2

8:45

01:3

4:00

01:3

9:15

01:4

4:30

01:4

9:45

01:5

5:00

02:0

0:15

02:0

5:30







Migration of a Domain Controller

The live migration and quick migration of a Domain Controller from the production site to the DR site are shown in Figure 43.

Live migration took 1 minute and 45 seconds, while quick migration took 1 minute and 30 seconds. Since there was another Domain Controller, there was minimal impact to the farm.

Figure 43. Percentage processor time during migrations of a Domain Controller

0

10

20

30

40

50

60

70

80

90

100

00:1

0:00

00:1

0:45

00:1

1:30

00:1

2:15

00:1

3:00

00:1

3:45

00:1

4:30

00:1

5:15

00:1

6:00

00:1

6:45

00:1

7:30

00:1

8:15

00:1

9:00

00:1

9:45

00:2

0:30

00:2

1:15






Migration of the SQL server

The live migration and quick migration of the SQL server from the production site to the DR site and back are shown in Figure 44.

Live migration took 3 minutes and 30 seconds, while quick migration took 9 minutes and 30 seconds. Because the SQL server was unreachable, the farm was down during this time.

Figure 44. Percentage processor time during migrations of a SQL server

0

10

20

30

40

50

60

70

80

90

100

00:1

8:45

00:2

0:45

00:2

2:45

00:2

4:45

00:2

6:45

00:2

8:45

00:3

0:45

00:3

2:45

00:3

4:45

00:3

6:45

00:3

8:45

00:4

0:45

00:4

2:45

00:4

4:45

00:4

6:45

00:4

8:45

00:5

0:45

00:5

2:45

00:5

4:45

00:5

6:45






Migration of a WFE

Figure 45 shows the live migration and quick migration of a WFE from the production site to the DR site and back.

The live migration took 4 minutes, while the quick migration took 3 minutes and 45 seconds. Impact to the farm was minimal because there were six WFEs.


0

10

20

30

40

50

60

70

80

90

100

01:0

6:00

01:0

7:00

01:0

8:00

01:0

9:00

01:1

0:00

01:1

1:00

01:1

2:00

01:1

3:00

01:1

4:00

01:1

5:00

01:1

6:00

01:1

7:00

01:1

8:00

01:1

9:00

01:2

0:00

01:2

1:00

01:2

2:00

01:2

3:00

01:2

4:00

01:2

5:00






Migration of a WFE (local to local)

The live migration and quick migration of a WFE from one production site node to another are shown in Figure 46.

The live migration took 30 seconds, while the quick migration took 1 minute and 30 seconds.


0

10

20

30

40

50

60

70

80

90

100

01:4

1:00

01:4

1:45

01:4

2:30

01:4

3:15

01:4

4:00

01:4

4:45

01:4

5:30

01:4

6:15

01:4

7:00

01:4

7:45

01:4

8:30

01:4

9:15

01:5

0:00

01:5

0:45

01:5

1:30

01:5

2:15






Section F: Unplanned failover testing


Unplanned failover testing (1 Gb/s)

The passed tests per second with RecoverPoint replication enabled with 1 Gb/s bandwidth and 1 ms latency during a full failover caused by a production site crash are shown in Figure 47.

The SharePoint farm had a total downtime of 7 minutes before it was fully operational again.

Figure 47. Passed tests/sec during failover (1 Gb/s)

0

5

10

15

20

25

30

35

40

00:0

9:15

00:0

9:45

00:1

0:15

00:1

0:45

00:1

1:15

00:1

1:45

00:1

2:15

00:1

2:45

00:1

3:15

00:1

3:45

00:1

4:15

00:1

4:45

00:1

5:15

00:1

5:45

00:1

6:15

00:1

6:45

00:1

7:15

00:1

7:45

00:1

8:15

00:1

8:45

00:1

9:15

00:1

9:45

00:2

0:15

00:2

0:45

00:2

1:15

00:2

1:45

Passed Tests/Sec

Passed Tests/Sec






Some tests failed during downtime as the farm was failing over to the DR site.


0

50

100

150

200

250

300

350

00:0

9:15

00:1

0:00

00:1

0:45

00:1

1:30

00:1

2:15

00:1

3:00

00:1

3:45

00:1

4:30

00:1

5:15

00:1

6:00

00:1

6:45

00:1

7:30

00:1

8:15

00:1

9:00

00:1

9:45

00:2

0:30

00:2

1:15

00:2

2:00







SQL server downtime and recovery

Figure 49 shows the relation between the SQL server downtime and recovery and the passed tests per second.

When the SQL server recovers, the CPU peaks because it is catching up to user requests before it normalizes.

Figure 49. Percentage processor time compared to passed tests/sec

0

10

20

30

40

50

60

70

80

90

100

00:0

9:15

00:1

0:00

00:1

0:45

00:1

1:30

00:1

2:15

00:1

3:00

00:1

3:45

00:1

4:30

00:1

5:15

00:1

6:00

00:1

6:45

00:1

7:30

00:1

8:15

00:1

9:00

00:1

9:45

00:2

0:30

00:2

1:15

00:2

2:00


Passed Tests/Sec




Incoming writes transition from production site to DR site

Figure 50 shows how the incoming writes transition from the production site to the DR site.

The first set of lines (faded) on the left side of the chart represents the production site writes, which change to the regular lines and represent the DR site incoming writes. The maximum incoming writes during the failover was 1.2 MB/s for the SQL server.

Figure 50. Incoming writes rate

0

0.5

1

1.5

2010

/03/

04

12:3

0:16

.935

GM

T

2010

/03/

04

12:3

1:16

.935

GM

T

2010

/03/

04

12:3

2:16

.935

GM

T

2010

/03/

04

12:3

3:16

.935

GM

T

2010

/03/

04

12:3

4:16

.935

GM

T

2010

/03/

04

12:3

5:16

.935

GM

T

2010

/03/

04

12:3

6:16

.935

GM

T

2010

/03/

04

12:3

7:16

.935

GM

T

2010

/03/

04

12:3

8:16

.935

GM

T

2010

/03/

04

12:3

9:16

.935

GM

T

2010

/03/

04

12:4

0:16

.935

GM

T

2010

/03/

04

12:4

1:16

.935

GM

T

2010

/03/

04

12:4

2:16

.935

GM

T

2010

/03/

04

12:4

3:16

.935

GM

T

2010

/03/

04

12:4

4:16

.935

GM

T

2010

/03/

04

12:4

5:16

.935

GM

T

2010

/03/

04

12:4

6:16

.935

GM

T

2010

/03/

04

12:4

7:16

.935

GM

T

2010

/03/

04

12:4

8:16

.935

GM

T

Incoming writes rate for link

Incoming writes rate for link - DRSITE - RPA1 - TCE-SP-APP-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-DB1 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-DC-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-EXCEL-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-INDEX-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-02 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-03 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-DB2 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-DC-02 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-04 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-05 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-06 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-APP-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-DB1 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-DC-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-EXCEL-01 - Megabytes/sec





Unplanned failover testing (300 Mb/s)

The passed tests per second with RecoverPoint replication enabled with 300 Mb/s bandwidth and no latency during full farm failover are shown in Figure 51.

The downtime for this failover was 7 minutes.

Figure 51. Passed tests/sec results (300 Mbps)

0

5

10

15

20

25

30

35

40

00:0

9:45

00:1

0:30

00:1

1:15

00:1

2:00

00:1

2:45

00:1

3:30

00:1

4:15

00:1

5:00

00:1

5:45

00:1

6:30

00:1

7:15

00:1

8:00

00:1

8:45

00:1

9:30

00:2

0:15

00:2

1:00

00:2

1:45

00:2

2:30

00:2

3:15

00:2

4:00

00:2

4:45

00:2

5:30

Passed Tests/Sec

Passed Tests/Sec





The test times for Browse, Search, and Modify during the failover are shown in Figure 52.


0

50

100

150

200

250

300

350

00:0

9:45

00:1

0:30

00:1

1:15

00:1

2:00

00:1

2:45

00:1

3:30

00:1

4:15

00:1

5:00

00:1

5:45

00:1

6:30

00:1

7:15

00:1

8:00

00:1

8:45

00:1

9:30

00:2

0:15

00:2

1:00

00:2

1:45

00:2

2:30

00:2

3:15

00:2

4:00

00:2

4:45

00:2

5:30







Incoming writes transition from production site to DR site

Figure 53 shows how the incoming writes transition from the production site to the DR site.

The first set of lines (faded) on the left side of the chart represents the production site writes, which change to the regular lines and represent the DR side incoming writes. The group with the maximum write rates of 1.6 MB/s was one of the WFEs.

Figure 53. Incoming writes rate

0

0.5

1

1.5

2

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …

2010

/03/

12 …


Incoming writes rate for link - DRSITE - RPA1 - TCE-SP-APP-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-DB1 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-DC-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-EXCEL-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-03 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-05 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-DB2 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-DC-02 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-INDEX-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-02 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-04 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-06 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-APP-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-DB1 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-DC-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-EXCEL-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-03 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-05 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-DB2 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-DC-02 - Megabytes/sec




Section G: Planned failback testing

Planned failback testing (300 Mb/s)

Once the failover to the DR site is done, it is necessary to manually fail the virtual machines back to the production site. Because the cluster service is restarted on the production hosts, before failing the hosts back to production, it is necessary to enable and disable image access for each consistency group.

The passed tests per second with RecoverPoint replication enabled with 300 Mb/s bandwidth and no latency during asynchronous testing are shown in Figure 54.

EMC scripted the failback so that each virtual machine failed back consecutively and was evenly distributed among the production nodes. During this process, downtime totaled 2 minutes.


0

5

10

15

20

25

30

35

40

00:0

5:30

00:0

7:30

00:0

9:30

00:1

1:30

00:1

3:30

00:1

5:30

00:1

7:30

00:1

9:30

00:2

1:30

00:2

3:30

00:2

5:30

00:2

7:30

00:2

9:30

00:3

1:30

00:3

3:30

00:3

5:30

00:3

7:30

00:3

9:30

00:4

1:30

00:4

3:30

00:4

5:30

00:4

7:30

00:4

9:30

00:5

1:30

00:5

3:30

Passed Tests/Sec

Passed Tests/Sec





The test times for Browse, Search, and Modify during the planned failback are shown in Figure 55.


0

5

10

15

20

25

30

35

40

45

50

00:0

5:30

00:0

7:45

00:1

0:00

00:1

2:15

00:1

4:30

00:1

6:45

00:1

9:00

00:2

1:15

00:2

3:30

00:2

5:45

00:2

8:00

00:3

0:15

00:3

2:30

00:3

4:45

00:3

7:00

00:3

9:15

00:4

1:30

00:4

3:45

00:4

6:00

00:4

8:15

00:5

0:30

00:5

2:45

00:5

5:00







Incoming writes rate for the WAN link

The incoming writes rate for each consistency group as the virtual machines are failed back consecutively from the DR site to the production site as shown in Figure 56. This can be seen by the different times each consistency group has write rate peaks.

The highest peak of 9 Mb/s in writes is for the SQL server failback.

Figure 56. Incoming writes rate for WAN link

02468

10

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …

2010

/03/

07 …


Incoming writes rate for link - DRSITE - RPA1 - TCE-SP-APP-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-DB1 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-DC-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-EXCEL-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-INDEX-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-01 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-02 - Megabytes/secIncoming writes rate for link - DRSITE - RPA1 - TCE-SP-WFE-03 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-DB2 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-DC-02 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-04 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-05 - Megabytes/secIncoming writes rate for link - DRSITE - RPA2 - TCE-SP-WFE-06 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-APP-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-DB1 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-DC-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-EXCEL-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-INDEX-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-01 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-02 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA1 - TCE-SP-WFE-03 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-DB2 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-DC-02 - Megabytes/secIncoming writes rate for link - PRODSITE - RPA2 - TCE-SP-WFE-04 - Megabytes/sec




Section H: Query scaling testing


Query scaling testing (4 Query servers)

The passed tests per second with RecoverPoint replication enabled with 1 Gb/s bandwidth and 2 ms latency with four Query servers are shown in Figure 57.

This configuration supported 40.63 passed tests per second, which is equivalent to 243,780 users at 1% concurrency. Because of the reduction in Query servers, EMC was able to add 2 ms latency, which equates to 200 km round-trip distances.

Figure 57. Passed tests/sec results (4 Query servers)

0

5

10

15

20

25

30

35

40

45

50

00:1

1:00

00:1

6:00

00:2

1:00

00:2

6:00

00:3

1:00

00:3

6:00

00:4

1:00

00:4

6:00

00:5

1:00

00:5

6:00

01:0

1:00

01:0

6:00

01:1

1:00

01:1

6:00

01:2

1:00

01:2

6:00

01:3

1:00

01:3

6:00

01:4

1:00

01:4

6:00

01:5

1:00

01:5

6:00

02:0

1:00

02:0

6:00

02:1

1:00

Passed Tests/Sec

Passed Tests/Sec





The test times for Browse, Search, and Modify during the four-Query server test are shown in Figure 58.

The Search test suffered the most because of the amount of data that needed to be replicated during each incremental crawl.



0

5

10

15

20

25

30

00:1

1:00

00:1

6:15

00:2

1:30

00:2

6:45

00:3

2:00

00:3

7:15

00:4

2:30

00:4

7:45

00:5

3:00

00:5

8:15

01:0

3:30

01:0

8:45

01:1

4:00

01:1

9:15

01:2

4:30

01:2

9:45

01:3

5:00

01:4

0:15

01:4

5:30

01:5

0:45

01:5

6:00

02:0

1:15

02:0

6:30







Lag in time between replicas

The small lag in time between the production site consistency groups and the replicas on the DR site is shown in Figure 59.

The highest peak is 0.45 seconds. Because this replication is synchronous, the lag should be very small.

Figure 59. Lag in time between replicas

00.10.20.30.40.5

2010

/02/

23 1

6:00

:16.

935 …

2010

/02/

23 1

6:02

:16.

935 …

2010

/02/

23 1

6:04

:16.

935 …

2010

/02/

23 1

6:06

:16.

935 …

2010

/02/

23 1

6:08

:16.

935 …

2010

/02/

23 1

6:10

:16.

935 …

2010

/02/

23 1

6:12

:16.

935 …

2010

/02/

23 1

6:14

:16.

935 …

2010

/02/

23 1

6:16

:16.

935 …

2010

/02/

23 1

6:18

:16.

935 …

2010

/02/

23 1

6:20

:16.

935 …

2010

/02/

23 1

6:22

:16.

935 …

2010

/02/

23 1

6:24

:16.

935 …

2010

/02/

23 1

6:26

:16.

935 …

2010

/02/

23 1

6:28

:16.

935 …

2010

/02/

23 1

6:30

:16.

935 …

2010

/02/

23 1

6:32

:16.

935 …

2010

/02/

23 1

6:34

:16.

935 …

2010

/02/

23 1

6:36

:16.

935 …

2010

/02/

23 1

6:38

:16.

935 …

2010

/02/

23 1

6:40

:16.

935 …

2010

/02/

23 1

6:42

:16.

935 …

2010

/02/

23 1

6:44

:16.

935 …

2010

/02/

23 1

6:46

:16.

935 …

2010

/02/

23 1

6:48

:16.

935 …

2010

/02/

23 1

6:50

:16.

935 …

2010

/02/

23 1

6:52

:16.

935 …

2010

/02/

23 1

6:54

:16.

935 …

2010

/02/

23 1

6:56

:16.

935 …

2010

/02/

23 1

6:58

:16.

935 …

2010

/02/

23 1

7:00

:16.

935 …

2010

/02/

23 1

7:02

:16.

935 …

2010

/02/

23 1

7:04

:16.

935 …

2010

/02/

23 1

7:06

:16.

935 …

2010

/02/

23 1

7:08

:16.

935 …

2010

/02/

23 1

7:10

:16.

935 …

2010

/02/

23 1

7:12

:16.

935 …

2010

/02/

23 1

7:14

:16.

935 …

2010

/02/

23 1

7:16

:16.

935 …

2010

/02/

23 1

7:18

:16.

935 …

2010

/02/

23 1

7:20

:16.

935 …

2010

/02/

23 1

7:22

:16.

935 …

2010

/02/

23 1

7:24

:16.

935 …

2010

/02/

23 1

7:26

:16.

935 …

2010

/02/

23 1

7:28

:16.

935 …

2010

/02/

23 1

7:30

:16.

935 …

2010

/02/

23 1

7:32

:16.

935 …

2010

/02/

23 1

7:34

:16.

935 …

2010

/02/

23 1

7:36

:16.

935 …

2010

/02/

23 1

7:38

:16.

935 …

2010

/02/

23 1

7:40

:16.

935 …

2010

/02/

23 1

7:42

:16.

935 …

2010

/02/

23 1

7:44

:16.

935 …

2010

/02/

23 1

7:46

:16.

935 …

2010

/02/

23 1

7:48

:16.

935 …

2010

/02/

23 1

7:50

:16.

935 …

2010

/02/

23 1

7:52

:16.

935 …

2010

/02/

23 1

7:54

:16.

935 …

2010

/02/

23 1

7:56

:16.

935 …

2010

/02/

23 1

7:58

:16.

935 …



















The passed tests per second with RecoverPoint replication enabled with 1 Gb/s bandwidth and 3 ms latency and two Query servers are shown in Figure 60.

This configuration supported 40.76 passed tests per second, which is equivalent to 244,560 users at 1% concurrency. Because of the reduction in Query servers, EMC was able to add 3 ms latency, which equates to 300 km round-trip distances.


0

5

10

15

20

25

30

35

40

45

50

00:0

4:00

00:0

8:45

00:1

3:30

00:1

8:15

00:2

3:00

00:2

7:45

00:3

2:30

00:3

7:15

00:4

2:00

00:4

6:45

00:5

1:30

00:5

6:15

01:0

1:00

01:0

5:45

01:1

0:30

01:1

5:15

01:2

0:00

01:2

4:45

01:2

9:30

01:3

4:15

01:3

9:00

01:4

3:45

01:4

8:30

01:5

3:15

01:5

8:00

02:0

2:45

Passed Tests/Sec

Passed Tests/Sec








0

5

10

15

20

25

30

35

40

00:0

4:00

00:0

9:15

00:1

4:30

00:1

9:45

00:2

5:00

00:3

0:15

00:3

5:30

00:4

0:45

00:4

6:00

00:5

1:15

00:5

6:30

01:0

1:45

01:0

7:00

01:1

2:15

01:1

7:30

01:2

2:45

01:2

8:00

01:3

3:15

01:3

8:30

01:4

3:45

01:4

9:00

01:5

4:15

01:5

9:30









The passed tests per second with RecoverPoint replication enabled with 900 Mb/s bandwidth and 18 ms latency with four Query servers are shown in Figure 62.

This test was done with a normal user load to demonstrate the response times when the farm is not pushed to its performance limits. The passed tests per second average was 30.63, which translates to 183,780 users at 1% concurrency. Eighteen ms latency is equivalent to 1,800 km round-trip distances.


0

5

10

15

20

25

30

35

00:1

5:00

00:1

9:45

00:2

4:30

00:2

9:15

00:3

4:00

00:3

8:45

00:4

3:30

00:4

8:15

00:5

3:00

00:5

7:45

01:0

2:30

01:0

7:15

01:1

2:00

01:1

6:45

01:2

1:30

01:2

6:15

01:3

1:00

01:3

5:45

01:4

0:30

01:4

5:15

01:5

0:00

01:5

4:45

01:5

9:30

02:0

4:15

02:0

9:00

02:1

3:45

Passed Tests/Sec

Passed Tests/Sec







This shows the difference in response times that a normal user load has when compared to a heavy user load. Under a heavy user load, the Search times peaked at 30 seconds, whereas, in this test, they peaked at just over 2 seconds.


0

0.5

1

1.5

2

2.5

00:1

5:00

00:2

0:15

00:2

5:30

00:3

0:45

00:3

6:00

00:4

1:15

00:4

6:30

00:5

1:45

00:5

7:00

01:0

2:15

01:0

7:30

01:1

2:45

01:1

8:00

01:2

3:15

01:2

8:30

01:3

3:45

01:3

9:00

01:4

4:15

01:4

9:30

01:5

4:45

02:0

0:00

02:0

5:15

02:1

0:30








As shown in Figure 64, the lag in time between the production site consistency groups and the replicas peaked at just over 7 seconds, which is acceptable for asynchronous replication.

This represents the maximum time difference between the production and DR replicas. EMC set the maximum acceptable time difference to be 30 seconds.


0

2

4

6

8

2010

/03/

16 0

8:50

:02.

462 …

2010

/03/

16 0

8:53

:02.

462 …

2010

/03/

16 0

8:56

:02.

462 …

2010

/03/

16 0

8:59

:02.

462 …

2010

/03/

16 0

9:02

:02.

462 …

2010

/03/

16 0

9:05

:02.

462 …

2010

/03/

16 0

9:08

:02.

462 …

2010

/03/

16 0

9:11

:02.

462 …

2010

/03/

16 0

9:14

:02.

462 …

2010

/03/

16 0

9:17

:02.

462 …

2010

/03/

16 0

9:20

:02.

462 …

2010

/03/

16 0

9:23

:02.

462 …

2010

/03/

16 0

9:26

:02.

462 …

2010

/03/

16 0

9:29

:02.

462 …

2010

/03/

16 0

9:32

:02.

462 …

2010

/03/

16 0

9:35

:02.

462 …

2010

/03/

16 0

9:38

:02.

462 …

2010

/03/

16 0

9:41

:02.

462 …

2010

/03/

16 0

9:44

:02.

462 …

2010

/03/

16 0

9:47

:02.

462 …

2010

/03/

16 0

9:50

:02.

462 …

2010

/03/

16 0

9:53

:02.

462 …

2010

/03/

16 0

9:56

:02.

462 …

2010

/03/

16 0

9:59

:02.

462 …

2010

/03/

16 1

0:02

:02.

462 …

2010

/03/

16 1

0:05

:02.

462 …

2010

/03/

16 1

0:08

:02.

462 …

2010

/03/

16 1

0:11

:02.

462 …

2010

/03/

16 1

0:14

:02.

462 …

2010

/03/

16 1

0:17

:02.

462 …

2010

/03/

16 1

0:20

:02.

462 …

2010

/03/

16 1

0:23

:02.

462 …

2010

/03/

16 1

0:26

:02.

462 …

2010

/03/

16 1

0:29

:02.

462 …

2010

/03/

16 1

0:32

:02.

462 …

2010

/03/

16 1

0:35

:02.

462 …

2010

/03/

16 1

0:38

:02.

462 …

2010

/03/

16 1

0:41

:02.

462 …

2010

/03/

16 1

0:44

:02.

462 …

2010

/03/

16 1

0:47

:02.

462 …

2010

/03/

16 1

0:50

:02.

462 …

2010

/03/

16 1

0:53

:02.

462 …

2010

/03/

16 1

0:56

:02.

462 …

2010

/03/

16 1

0:59

:02.

462 …

2010

/03/

16 1

1:02

:02.

462 …

2010

/03/

16 1

1:05

:02.

462 …

2010

/03/

16 1

1:08

:02.

462 …

2010

/03/

16 1

1:11

:02.

462 …

2010

/03/

16 1

1:14

:02.

462 …


RPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-APP-01 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-DB1 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-DC-01 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-EXCEL-01 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-WFE-01 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-WFE-03 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA1 - TCE-SP-WFE-05 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA2 - TCE-SP-DB2 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA2 - TCE-SP-DC-02 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA2 - TCE-SP-INDEX-01 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA2 - TCE-SP-WFE-02 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA2 - TCE-SP-WFE-04 - secRPO - lag in time between replicas during transfer after init - PRODSITE - RPA2 - TCE-SP-WFE-06 - sec





Figure 65 shows the passed tests per second with RecoverPoint replication enabled with 900 Mb/s bandwidth and 25 ms latency and with two Query servers at a heavy user load.

The average passed tests per second was 41.72, which equates to 250,320 users at 1% concurrency. Twenty-five ms latency is equivalent to 2,500 km round-trip distances.


0

5

10

15

20

25

30

35

40

45

50

00:1

5:00

00:1

9:45

00:2

4:30

00:2

9:15

00:3

4:00

00:3

8:45

00:4

3:30

00:4

8:15

00:5

3:00

00:5

7:45

01:0

2:30

01:0

7:15

01:1

2:00

01:1

6:45

01:2

1:30

01:2

6:15

01:3

1:00

01:3

5:45

01:4

0:30

01:4

5:15

01:5

0:00

01:5

4:45

01:5

9:30

02:0

4:15

02:0

9:00

02:1

3:45

Passed Tests/Sec

Passed Tests/Sec






Browse averaged 1.23 seconds, Search averaged 1.02 seconds, and Modify averaged 1.15 seconds. The red line in this chart shows how much the propagation of Index server data to the Query servers affects Search response times.


0

1

2

3

4

5

6

00:1

5:00

00:2

0:15

00:2

5:30

00:3

0:45

00:3

6:00

00:4

1:15

00:4

6:30

00:5

1:45

00:5

7:00

01:0

2:15

01:0

7:30

01:1

2:45

01:1

8:00

01:2

3:15

01:2

8:30

01:3

3:45

01:3

9:00

01:4

4:15

01:4

9:30

01:5

4:45

02:0

0:00

02:0

5:15

02:1

0:30








As shown in Figure 67, the lag in time for this test is higher because it was done under full load.

The maximum lag in time was 9.5 seconds, which is still well within the 30-second lag that EMC set as the threshold.


02468

10

2010

/03/

11 1

7:00

:02.

462

GM

T

2010

/03/

11 1

7:03

:02.

462

GM

T

2010

/03/

11 1

7:06

:02.

462

GM

T

2010

/03/

11 1

7:09

:02.

462

GM

T

2010

/03/

11 1

7:12

:02.

462

GM

T

2010

/03/

11 1

7:15

:02.

462

GM

T

2010

/03/

11 1

7:18

:02.

462

GM

T

2010

/03/

11 1

7:21

:02.

462

GM

T

2010

/03/

11 1

7:24

:02.

462

GM

T

2010

/03/

11 1

7:27

:02.

462

GM

T

2010

/03/

11 1

7:30

:02.

462

GM

T

2010

/03/

11 1

7:33

:02.

462

GM

T

2010

/03/

11 1

7:36

:02.

462

GM

T

2010

/03/

11 1

7:39

:02.

462

GM

T

2010

/03/

11 1

7:42

:02.

462

GM

T

2010

/03/

11 1

7:45

:02.

462

GM

T

2010

/03/

11 1

7:48

:02.

462

GM

T

2010

/03/

11 1

7:51

:02.

462

GM

T

2010

/03/

11 1

7:54

:02.

462

GM

T

2010

/03/

11 1

7:57

:02.

462

GM

T

2010

/03/

11 1

8:00

:02.

462

GM

T

2010

/03/

11 1

8:03

:02.

462

GM

T

2010

/03/

11 1

8:06

:02.

462

GM

T

2010

/03/

11 1

8:09

:02.

462

GM

T

2010

/03/

11 1

8:12

:02.

462

GM

T

2010

/03/

11 1

8:15

:02.

462

GM

T

2010

/03/

11 1

8:18

:02.

462

GM

T

2010

/03/

11 1

8:21

:02.

462

GM

T

2010

/03/

11 1

8:24

:02.

462

GM

T

2010

/03/

11 1

8:27

:02.

462

GM

T

2010

/03/

11 1

8:30

:02.

462

GM

T

2010

/03/

11 1

8:33

:02.

462

GM

T

2010

/03/

11 1

8:36

:02.

462

GM

T

2010

/03/

11 1

8:39

:02.

462

GM

T

2010

/03/

11 1

8:42

:02.

462

GM

T

2010

/03/

11 1

8:45

:02.

462

GM

T

2010

/03/

11 1

8:48

:02.

462

GM

T

2010

/03/

11 1

8:51

:02.

462

GM

T

2010

/03/

11 1

8:54

:02.

462

GM

T

2010

/03/

11 1

8:57

:02.

462

GM

T

2010

/03/

11 1

9:00

:02.

462

GM

T

2010

/03/

11 1

9:03

:02.

462

GM

T

2010

/03/

11 1

9:06

:02.

462

GM

T

2010

/03/

11 1

9:09

:02.

462

GM

T

2010

/03/

11 1

9:12

:02.

462

GM

T

2010

/03/

11 1

9:15

:02.

462

GM

T

2010

/03/

11 1

9:18

:02.

462

GM

T

2010

/03/

11 1

9:21

:02.

462

GM

T

2010

/03/

11 1

9:24

:02.

462

GM

T

2010

/03/

11 1

9:27

:02.

462

GM

T















Chapter 8: Conclusion



Chapter 8: Conclusion

Summary RecoverPoint and RecoverPoint with Cluster Enabler provide best-in-class disaster

recovery for active SharePoint farms. You can now expect automated disaster recovery, within minutes (RTO), with minimal (asynchronous) to zero (synchronous) SharePoint data loss (RPO), even in the event of a full site disaster.

RecoverPoint’s data compression allows Microsoft Cluster to be stretched over much longer distances than before, while ensuring data consistency using synchronous or asynchronous technologies.

RecoverPoint’s management allows complete control and monitoring of replication status in a single user-friendly GUI.

This Proven Solution also saves on licensing costs (no virtual machine OS licenses on the DR site) and on time for updating and maintaining a mirror DR farm (software/configuration updates).

This is a unique Proven Solution that proves you can provide an automatic failover and restart of an entire SharePoint farm.

Findings The key findings of the Proven Solution testing were as follows:

• A full site disaster caused only 7 minutes of downtime for users connecting to the SharePoint farm (for both synchronous and asynchronous replication).

• Planned failback of the SharePoint farm had a cumulative downtime of only 2 minutes.

• The impact of the RecoverPoint splitter on the CLARiiON’s SP utilization was minimal at about 1 percent under full SharePoint user load.

• Bandwidth reduction by RecoverPoint was significant (3.42:1). • Quick Migration of virtual machines with large amounts of memory (for example,

SQL server), requires sufficient bandwidth and low enough latency to manage the large amounts of data being committed to disk in burst writes and requiring quick replication.

• Query server propagations caused the most disk writes in the SharePoint farm and were therefore the highest users of replication bandwidth. Reducing the number of Query servers allowed for longer distances between the production and DR sites.

• The entire SharePoint farm including its search services was able to restart and serve clients in about 7 minutes.

Next steps EMC can help to accelerate assessment, design, implementation, and management

while lowering the implementation risks and costs of a backup/disaster recovery solution for a virtualized Microsoft SharePoint 2007 environment.

To learn more about this and other solutions, contact an EMC representative or visit EMC Solutions for Microsoft.

http://www.emc.com/solutions/application-environment/microsoft/index.htm�

Date post:	27-May-2018
Category:	Documents
Upload:	trinhnguyet
View:	221 times
Download:	0 times

EMC Business Continuity for Microsoft Office SharePoint ... · Best practices and recommendations...

Documents