White Paper - Message · PDF fileEMC VMAX ... Large Pre-allocation—test of the ability...

89 Fifth Avenue, 7th Floor

New York, NY 10003

www.TheEdison.com

212.367.7400

White Paper

HP Thin Technologies

A Competitive Comparison

Printed in the United States of America

Copyright 2012 Edison Group, Inc. New York. Edison Group offers no warranty either expressed

or implied on the information contained herein and shall be held harmless for errors resulting

from its use.

All products are trademarks of their respective owners.

First Publication: September 2012

Produced by: Chris M Evans, Sr. Analyst; John Nicholson, Sr. Analyst; Barry Cohen, Editor-in-

Chief; Manny Frishberg, Editor

Table of Contents

Executive Summary _____________________________________________________ 1

Introduction ___________________________________________________________ 2

Objective __________________________________________________________________ 2

Audience __________________________________________________________________ 2

Terminology ________________________________________________________________ 2

Overview ______________________________________________________________ 3

Thin Provisioning Overview ___________________________________________________ 3

Thin Provisioning Drawbacks __________________________________________________ 4

HP 3PAR StoreServ Thin Technology ________________________________________ 5

Competitive Analysis ____________________________________________________ 6

EMC VMAX_________________________________________________________________ 6 Background _______________________________________________________________________ 6 HP 3PAR StoreServ Comparison—Start Thin _____________________________________________ 6 HP 3PAR StoreServ Comparison—Get Thin ______________________________________________ 7 HP 3PAR StoreServ Comparison—Stay Thin ______________________________________________ 7 VMAX Restrictions __________________________________________________________________ 7

NetApp ____________________________________________________________________ 8 Background _______________________________________________________________________ 8 HP 3PAR StoreServ Comparison—Start Thin _____________________________________________ 8 HP 3PAR StoreServ Comparison—Get Thin ______________________________________________ 9 HP 3PAR StoreServ Comparison—Stay Thin ______________________________________________ 9

Hitachi VSP ________________________________________________________________ 9 Background _______________________________________________________________________ 9 HP 3PAR StoreServ Comparison—Start Thin ____________________________________________ 10 HP 3PAR StoreServ Comparison—Stay Thin _____________________________________________ 10

EMC VNX _________________________________________________________________ 11 HP 3PAR StoreServ Comparison—Start Thin ____________________________________________ 11 HP 3PAR StoreServ Comparison—Get Thin _____________________________________________ 11 HP 3PAR StoreServ Comparison—Stay Thin _____________________________________________ 11 Performance Considerations _________________________________________________________ 12

IBM XIV __________________________________________________________________ 12 HP 3PAR StoreServ Comparison—Start Thin ____________________________________________ 12 HP 3PAR StoreServ Comparison—Get Thin _____________________________________________ 12 HP 3PAR StoreServ Comparison—Stay Thin _____________________________________________ 13

Testing Overview and Methodology _______________________________________ 14

Test 1—Zero-Page-Reclaim Performance ________________________________________ 14

Test 2—Large Pre-Allocation__________________________________________________ 15

Test Results ___________________________________________________________ 16

Test 1—Zero-Page-Reclaim Performance ________________________________________ 16

Test 2—Large Pre-Allocation__________________________________________________ 19

Conclusions and Recommendations _______________________________________ 21

Best Practices _____________________________________________________________ 21

Appendix A—Document References _______________________________________ 23

Appendix B—Test Equipment Specification _________________________________ 24

Arrays ____________________________________________________________________ 24

Servers ___________________________________________________________________ 24

Edison: HP Thin Technologies Comparison Page 1

Executive Summary

As the drive to "do more with less" becomes a mantra for many organizations,

optimizing space utilization is a key goal of many IT departments. Storage continues to

be one of the major cost components of today's infrastructure deployments. Thin

technology, including thin provisioning, offers efficiency benefits that can significantly

reduce both capital and operational costs. However implementations of thin

technologies differ with the storage vendors.

HP 3PAR StoreServ is seen as a leader in thin technology, with three key aims:

1. Start Thin—ensure thin provisioned storage occurs with minimum overhead.

2. Get Thin—ensure data moved to HP 3PAR StoreServ remains thin on migration.

3. Stay Thin—ensure data is kept at optimal efficiency over time.

To validate this statement, a literature review, extensive customer interviews, and two

tests were performed:

1. Zero-Page-Reclaim Performance—validation of the ability to reclaim freed resources

as part of normal operations.

2. Large Pre-allocation—test of the ability to create new storage volumes with minimal

overhead.

Overall, HP 3PAR StoreServ was the best performer in achieving the goals of "start

thin," "get thin," and "stay thin."


Introduction

Objective

This report looks at thin provisioning technology from the major storage vendors in

today's marketplace. It compares the thin implementations from seven storage array

platforms, including Hewlett Packard's 3PAR storage arrays. In particular this white

paper highlights three important differentiating aspects of HP 3PAR StoreServ's thin

technology:

1. The ability to "start thin"—provisioned storage is thin at deployment time.

2. Getting thin—the ability to move data from thick to thin.

3. Staying thin—maintaining thin LUNs.

Audience

Decision makers in organizations that are considering implementing a thin technology

strategy will find this paper provides high level information on vendor offerings.

Technical professionals looking to understand more about the implementation of vendor

thin technology solutions will also find the content of this paper useful.

Terminology

This white paper makes reference to the following common terminology:

"Thick" LUN—a storage volume presented from an array in which all of the space

representing the logical size of the LUN (logical unit number) is reserved on the

array for exclusive use by that volume.

"Thin" LUN—a storage volume presented from an array that is not tied to any

physical storage allocation and in which only the physically written space is

consumed on the array.

Thin technologies—a suite of features, including thin provisioning, that optimize the

use of a storage array.


Overview

In recent years storage has become one of the major cost components within the data

center. Although the price of storage continues to fall, the rate of data growth in many

organizations continues to rise steeply, resulting in increasing costs for managing the

storage systems. Every year there is a requirement to "do more with less," using storage

more efficiently without increasing the operational budget. There are a number of key

initiatives being undertaken by organizations in order to reduce their storage

consumption. These relate directly to the use of thin technology.

Reducing Waste—Storage utilization never reaches 100 percent of the physical space

provisioned from an array, as each level of configuration—from the array to the

host—introduces some inefficiency. Reducing waste increases utilization and allows

the deferral of additional capital expenditure.

Reducing Overhead—Deploying storage isn’t a quick task; from purchase order to

deployment on the data center floor, the process can take months to achieve. Storage

administrators usually keep storage in reserve in order to manage the purchase

process.

Flexibility—End users want the minimum disruption to their applications and as a

result, many over-order storage resources, in many cases up to 36 months ahead of

when the space is actually needed. Ideally, end users should be able to lay out their

storage needs based on growth plans and then allocate that storage on-demand.

Improving Cost Efficiency—Storage Tiering (placing data on the most cost-effective

media for the I/O profile required) is a key technology in reducing storage costs.

Dynamic tiering can be used to automate the process of data placement, based on the

use of storage pools for LUN creation. Thin provisioned LUNs directly aid the

deployment of a tiered storage model. A thin LUN is built from blocks of physical

disk capacity from within a pool of storage with metadata to associate the logical

LUN to the physical space. The physical blocks can therefore be taken from multiple

pools, where each pool represents a different tier.

Thin Provisioning Overview

Thin provisioning is a space reduction technology implemented by all of the major

enterprise storage vendors. It enables the utilization of storage within an array to be

increased over traditional "thick" storage deployments.


In traditional "thick" storage deployments, physical space on disk is immediately

reserved for the entire size of a volume (or "LUN") at creation time, regardless of how

much space will subsequently be used by the host. In thin storage deployments, no

space is reserved in advance for the LUN. As the host writes data to the LUN, physical

space is assigned on-demand from the array, usually in blocks that vary from 4KB to

42MB, according to the vendor. Thin provisioned LUNs are therefore much more

efficient and more closely track the actual space in use on the host.

For many reasons, storage utilization on hosts never reaches 100 percent utilization.

However with "thick" LUNs, physical space is reserved out on an array for the entire

size of a volume. Thin provisioned deployments can take advantage of all physical

storage available by creating more logical storage capacity than is physically available.

So called "over-provisioning" enables the utilization of physical space to be pushed to

levels higher than can be achieved in normal deployments.

Thin Provisioning Drawbacks

The ability to over-provision storage does come with a few drawbacks. Should physical

space be completely exhausted, hosts will receive write errors, indicating a physical

problem on the array. Write failures are not usually handled gracefully by the host

operating system and can lead to system crashes. Therefore, physical versus logical

space capacity needs to be carefully managed.

Over time as files are created and deleted, thin LUNs become less efficient. This is due to

the way in which the file system on the LUN manages file allocations, free space and

metadata. Some file system implementations are more "thin friendly" than others and

are designed to re-use released space. However, housekeeping of thin provisioned

storage is required in order to maintain optimal levels of efficiency. Storage vendors

have introduced features that enable the array to recover unused storage resources:

Zero-Page-Reclaim (ZPR)—A storage array identifies an entire block of storage

consisting of binary zeros, the block will be assumed to be unused and is released

back to the free pool. The ability to find unused blocks depends on a number of

factors, including the file system and array block-size and the level of file

fragmentation. Smaller array block-sizes are better for ZPR operations.

SCSI UNMAP—The UNMAP command is a low-level I/O operation that can be

used by the host to signal to the storage array that a block of storage is no longer in

use and can be released to the free pool. Unfortunately very few operating systems

currently support this feature.


HP 3PAR StoreServ Thin Technology

The HP 3PAR StoreServ storage platform has a "thin by design" architecture that places

no restrictions on the use of either thin or thick LUNs. This applies to LUN performance,

and capacity, removing the need for the storage administrator to design the layout of the

array to cater for thin technology. HP 3PAR StoreServ is specifically optimized for thin

provisioning and contains many unique design features that enable "starting thin,"

"getting thin," and "staying thin."

RAID MP (multi-parity)—HP 3PAR StoreServ arrays implement a form of RAID-6

that divides physical disks into "chunklets" of either 256MB or 1GB in size.

Chunklets are combined to form Logical Volumes (LVs) and Common Provisioning

Groups (CPGs) from which Virtual Volumes are created. Thin Provisioning Virtual

Volumes use a block size increment of 16KB, which is the minimum reclaimable unit

of storage within the array.

Hardware ASIC—Now at Generation Four, HP 3PAR StoreServ uses dedicated

custom ASIC (Application Specific Integrated Circuit) processors to perform the

identification and recovery of unused resources that can be reclaimed from thin

provisioned virtual volumes. An ASIC enables processor-intensive tasks to be

offloaded to dedicated hardware, removing the performance impact of features such

as space reclamation from the array and ensures consistent host I/O response times.

The HP 3PAR StoreServ ASIC provides a range of functions, including inline ZPR.

Thin Persistence—An operating system task that identifies and recovers freed

resources.

Thin Conversion—Performs the migration of thick to thin volumes through a

process of inline zero detection. As data is copied to the array, zeroed blocks of data

are identified and logically mapped rather than physically written to disk.

Thin Copy Reclamation—Performs space recovery on volume copies within the

array.

Thin Reclamation API—HP 3PAR StoreServ developed the Thin Reclamation API

in partnership with Symantec. This feature allows the file system to signal when

freed resources can be released on the array. It is supported by Symantec Veritas

Storage Foundation from Version 5 onwards.

Virtualization Support—HP 3PAR StoreServ supports the VMware VAAI API,

including the "block zeroing" command.

Management—HP 3PAR StoreServ arrays provide alerts for specific thin

provisioning space issues. Alerts are issued based on pre-defined thresholds and

enable efficient monitoring of capacity in thin environments.


Competitive Analysis

EMC VMAX

Symmetrix VMAX, EMC's flagship enterprise storage platform, is the first enterprise-

class storage product to move away from custom hardware design. It uses commodity-

based Intel processors with customized hardware managing the interconnect between

storage modules. The VMAX operating system—Enginuity—is an evolution of the code

developed for the first Symmetrix ICDA (Integrated Cache Disk Arrays) in 1991, and it

still retains many of the original architectural design features and constraints. The

discussion of VMAX in this section covers the latest 10K, 20K and 40K models.

Background

Thin provisioning in VMAX is implemented as a feature called Virtual Provisioning—

EMC's brand name for their thin provisioning technology. Thin provisioned LUNs are

known as thin devices and take physical storage from thin pools. A thin pool is created

using standard "thick" LUNs (termed data devices), which are subdivided into allocation

units called thin device extents. Thin pools must use the same emulation and RAID

protection type and EMC recommend building them from disks of the same rotational

speed and data device size.

A thin extent is 12 tracks or 768KB in size and represents both the initial minimum

assigned to all thin devices when they are bound to a thin pool and also the lowest

increment of granularity when the capacity of a thin device is extended. HP 3PAR

StoreServ thin technology uses the much smaller increment of 16KB, which results in

less wastage, particularly with fragmented and thin-hostile file systems. Thin devices are

effectively cache-based objects that simply reference the underlying physical pool of

standard LUNs in array. These LUNs in turn, map to physical disks. A single VMAX

system supports up to 512 pools and 64,000 thin devices.

HP 3PAR StoreServ Comparison—Start Thin

VMAX Virtual Provisioning requires a significant amount of initial planning. EMC

recommends the use of large data devices within pools. As data devices are effectively

standard LUNs, typical configurations create LUNs to be used purely for thin pools and

LUNs to be used for non-thin provisioned usage. Meta devices (a linked series of

multiple standard devices) cannot be used as data devices. This practice leads to waste

and a shortage of the right type of storage. It is possible to dissolve and resize standard

LUNs, however the process is time consuming and can lead to unbalanced performance.


With HP 3PAR StoreServ thin technology, physical disks are simply assigned to a pool

from which either thin or thick LUNs can be provisioned.

When VMAX thin devices are bound to a thin storage pool, a minimum allocation of one

thin extent (768KB) is reserved. As thin devices are effectively cache objects, each device

consumed an additional 148KB of cache, plus 8KB per 1GB based on the size of the thin

device. With HP 3PAR StoreServ thin technology, no initial space reservations are made.

HP 3PAR StoreServ Comparison—Get Thin

There is no functionality within the VMAX array to optimize storage when converting

thick LUNs to thin devices. As thick LUNs are copied to a thin device, the space

occupied by the thin device is 100 percent of the logical allocation. Thin devices must be

optimized, using a process called Space Reclamation. This means data migrations

moving thick to thin LUNs require additional physical capacity to be available for the

migration process. HP 3PAR StoreServ thin technology uses the Thin Conversion feature

to optimize the migration of thick to thin LUNs in real time at line speeds.

HP 3PAR StoreServ Comparison—Stay Thin

EMC VMAX arrays are able to reclaim "empty" or zero pages of a thin device using

Space Reclamation. This runs as a background task on the Disk Adapter associated with

the LUN. An entire thin extent is read into cache and examined, checking the T10-DIF

values for each block of data against a known T10-DIF value for an all-zeros block. If the

entire thin extent contains only zeroed data, then it is released from use. Until the space

reclamation process is run, any zeroed areas of a thin device still consume physical

space. VMAX space reclamation cannot be used with thin devices that are in an active

SRDF (Symmetrix Remote Data Facility) pair or are using local replication. With 3PAR

StoreServ thin technology, the process of zero-detecting is done inline using a custom

ASIC. This has no impact on the controller processor or cache utilization levels and

occurs in real time at line-speed. There are no restrictions on replicated LUNs.

VMAX Restrictions

EMC recommends a utilization level of between 60-80 percent per thin pool in order to

prevent "out of space" issues. With multiple pools (which are required for different

RAID data protection types) this can result in significant waste. HP 3PAR StoreServ thin

technology does not require separate pools for multiple protection types. When using

Synchronous SRDF with VMAX, only one active write is permitted per thin device.

Where thin devices are created into meta-devices, this can result in a performance

impact. There is also a limit of eight read-requests-per-paths for each thin device, which

can result in slow performance with high read miss rates.


NetApp

NetApp storage appliances were originally developed to deliver network-attached

storage using either the CIFS or NFS protocols. Over time, NetApp have developed their

platform to cater for block storage, using either iSCSI or Fibre Channel. The current

versions of NetApp filers can be configured in either 7-mode or cluster-mode and

represent two distinct product lines based on the original Data ONTAP operating

system, and the codebase from the acquisition of Spinnaker, Inc., respectively.

Background

NetApp filers implement block-based storage within Data ONTAP by emulating LUNs

within volumes known as FlexVols. FlexVols are then created on aggregates (pools of

physical storage) and physical disk RAID groups. The underlying architecture uses a

data layout called WAFL (Write Anywhere File Layout) that operates a "write-new"

policy for both new data and updates; no block or file data is ever updated in place.

WAFL uses a page size of 4KB, storing updates in non-volatile RAM before writing an

entire "stripe" of data to disk. In this way, writes are optimized on commit-to-disk using

a RAID-4 physical disk configuration. NetApp LUNs are emulated through files on

volumes; therefore, both block and file data can be mixed within the same storage pool.

LUN creation is a simple process to achieve, however the use of block-based LUNs

involves significant complexity.


All NetApp volumes by default are "thick" provisioned with thin volumes simply

having a no space guarantees. In turn, LUNs within a volume are thin provisioned if

they have space reservation disabled. By default, all LUNs are thick provisioned and

have space reservation enabled. The administrator must turn off space reservation after

the LUN is created to make it thin-provisioned. However, as blocks of data are not

overwritten in place, an additional amount of space (called Fractional Reserve or

Overwrite Reserved Space) must be reserved to manage data updates where snapshots

are used on the LUN. By default Fractional Reserve is set at 100 percent when the space

guarantee is set to "file" for a volume. This is the only way to guarantee enough space is

available within the volume to hold updates to the entire contents of the LUN. Space

guarantees are complex and if used incorrectly can result in LUNs going offline in order

to protect data. The system creates a propensity to over-configure storage in order to

reduce the risk of data access issues. HP 3PAR StoreServ thin technology is implemented

in a much simpler way and does not have the management complexity seen in the

NetApp platform.



NetApp has no native features for importing LUNs from other storage platforms. LUNs

migrated into the NetApp platform using host-based tools allocate physical space

matching the entire logical size of the LUN. There are no native features within Data

ONTAP to identify and reclaim zero or empty pages of data. Data ONTAP does

implement data deduplication (called dedupe) at the block level, and it is possible to use

this feature to deduplicate zeroed pages of data. However there are restrictions on the

size of LUNs that have dedupe enabled. In addition, deduping can have negative

performance impacts on highly utilized LUNs. NetApp quote tests that show the

performance impact for writes on deduplicated volumes becomes worse with larger

systems; for example the FAS6080 can have a performance degradation of up to 35

percent. HP 3PAR StoreServ thin technology has no performance impact.


Over time, as data is written to a NetApp block-LUN, the physical space used trends

towards the logical LUN size. There are no in-built features for enabling zero-block

identification and reclaiming. Instead, NetApp requires the deployment of a host-based

agent called SnapDrive to track updates to the file system. The agent must be deployed

on every server to which NetApp LUNs are presented; otherwise the tracking of file

deletions cannot occur. Platform support for SnapDrive is limited and does not include

common operating systems such as RHEL6. HP 3PAR StoreServ thin technology

automatically detects zero-block data inline with no performance impact and does not

require the implementation of host agents.

Hitachi VSP

The VSP is Hitachi's current enterprise-level storage array and is the evolution of

previous Lightning and USP-V models. The VSP retains the use of custom ASIC

technology, in which the management of storage processes is handled by Virtual Storage

Directors connected to the back-end switch matrix. Custom ASIC usage has been a

feature of all of the Hitachi storage platforms; however it isn't used directly in the thin

provisioning approach or in managing the efficiency of thin provisioned storage.

Background

VSP thin provisioning technology is known as HDP--Hitachi Dynamic Provisioning.

HDP thin LUNs (called LDEVs or logical devices) are created from a HDP pool that

comprises standard LDEV devices. In turn, LDEVs are created from RAID groups, built

from up to 16 disks in one of seven RAID-5 or RAID-6 variations. At the physical level,

data is written in tracks of 256KB per physical disk, which results in a standard logical


page size of 42MB, in order to accommodate all possible RAID levels. This means initial

volume allocations and volume expansion of thin LUNs is in 42MB-page increments that

can result in inefficient use of space with small file block size and thin-unfriendly file

systems. With HP 3PAR StoreServ thin technology, space allocations are made in 16KB

increments, which results in much less wastage in thin-unfriendly environments.


The creation of VSP HDP pools requires considerable planning. Hitachi recommends

that pools be created using large LUNs built from traditional RAID groups. RAID group

creation is typically performed at array installation time and so is a one-off task. The

RAID group size depends on how many physical disks and back-end directors have

been installed within the VSP. However it is normal to build HDP pools from many

RAID groups across all back-end directors to ensure maximum I/O performance—so

called “wide striping”. Although standard "thick" LDEVs can be created from the same

RAID groups used to create HDP pools, for performance reasons, the practice isn't

recommended. The implementation of HDP can result in wasted resources and always

requires the reservation of many RAID groups to thin provisioning. By contrast, with

HP 3PAR StoreServ thin technology physical disks are simply assigned to a pool from

which either thin or thick LUNs can be provisioned.

On VSP, thin LDEVs allocate a minimum of one 42MB page on assignment to an HDP

pool. With HP 3PAR StoreServ thin technology, no initial space reservations are made.


The Hitachi VSP platform enables traditional "thick" LUNs to be imported into the

system using the external virtualization feature of the array, known as Universal

Volume Manager. In addition, "thick" LUNs can be imported from other VSP systems

using TrueCopy replication. Any imported LUNs remain fully allocated at their original

logical size until zero-block reclaim is performed using the zero-page-reclaim feature.

ZPR is a post-processing background task that examines individual LDEVs and releases

42MB pages back to the HDP pool. Hitachi recommends ZPR be executed during

periods of inactivity, as the task is performed by the back-end directors and can have a

performance impact on production I/O. As ZPR is not performed in-line, thin LDEVs

will "grow" over time as data is written to the file system on the LUN. This means thin

pools need to be provisioned with additional capacity to cater for this growth between

ZPR reclaim tasks. With 3PAR StoreServ thin technology, the process of zero-detection is

done inline using a custom ASIC. This has no impact on the controller processor and

occurs in real time at line-speed.


EMC VNX

EMC's VNX platform is an evolution of the previous CLARiiON and Celerra products

(serving block and file protocols respectively). The two platforms were brought together

and marketed as a single platform, using one management tool, called Unisphere. Block-

based storage LUNs are presented from the base hardware unit, with file access

implemented on x-blade modules. Thin provisioning technology is implemented using

the Virtual Provisioning (VP) feature. VP extends the capabilities of LUN configuration

to include both thick and thin LUNs on the same disk pool. VP disk pools can be

comprised of large numbers of disks (greater than the standard disk pool which is

limited to 16 devices), but still configures disks in RAID groups for resiliency.


VNX thin LUNs can be logically defined in sizes from 1MB to 16TB. However each LUN

reserves a minimum of 3GB. This means allocation of a large number of LUNs has a

significant reservation on physical space, in contrast to HP 3PAR StoreServ thin

technology’s no minimum physical spare reservation.


VNX does not support any native inline zero-page reclaim functionality. Instead,

"empty" pages must be reclaimed by using either the LUN Migration (within the array)

or SAN Copy (from external arrays) functions. This means that thin LUN physical

capacity will trend towards their logical size over time. EMC recommends using the

sdelete host command and LUN migration as the method of reclaiming unused space in

the VNX array. By comparison, HP 3PAR StoreServ thin technology detects and

eliminates zero-page data inline with no need to perform additional manual data

migrations. VNX supports the Symantec Thin Reclamation API, however this requires

the deployment of the Veritas File System on every server for which reclaim is required.


VNX thin LUNs grow in increments of 1GB of physical space, with 8KB blocks used

as the minimum level of granularity. The 8KB block refers to the space within which

zero page data can be reclaimed, however each increment of space assigned to a LUN

works in steps of 1GB, so any 8KB holes "punched" out of a 1GB slice can only be reused

by that volume, rather than as free space for all available volumes. HP 3PAR StoreServ

thin technology implements LUN mapping to internal logical disks at 32MB pages, with

16KB representing the minimum level of page granularity for thin provisioning. This

makes the 3PAR system much more efficient with thin LUNs. VNX has no ASIC

technology to perform inline zero space reclamation, unlike HP 3PAR StoreServ.


Performance Considerations

EMC highlight that Virtual Provisioning Thin LUNs do provide more flexibility, but

offer lower performance than traditional thick LUNs and so recommend their use only

for applications requiring "moderate" performance. HP 3PAR StoreServ thin technology

has no performance restrictions.

IBM XIV

The IBM XIV storage array platform was acquired from an Israeli startup, founded by

the inventor of the EMC Symmetrix, Moshe Yanai. The platform takes a radical

departure from traditional arrays and uses only high-capacity SATA or SAS hard drives,

although the configuration has recently been expanded to accelerate I/O using an SSD

cache layer. XIV is now at the third generation of hardware, utilizing either 2TB or 3TB

drives, with 6TB of SSD cache. Each array is comprised of between six and 15 server

nodes, which hold 12 hard drives each, resulting in a maximum configuration of 180

drives. Each node subdivides disks into 1MB chunks, which are then distributed across

all disks as a single large pool of mirrored data. XIV uses the terms "soft size" and "hard

size" to refer to the logical and physical size of a LUN respectively. These terms also

apply equally to pools that can be allocated physical capacity. It is possible for a pool to

deplete hard (physical capacity) and lock access to a volume, despite there being free

physical space in other pools. The overall capacity of an XIV array is referred to as the

"system hard size." A "system soft size"—the degree of over-provisioning permitted at

the array level—is also defined, but can't be modified by the system administrator. This

value has to be set by an IBM engineer and requires the customer to indemnify IBM

against any issues that occur as a result of the change.


By default, all XIV LUNs are thin provisioned and placed into storage pools. As data is

spread equally across all drives in the system, storage pools provide no more than a

logical administrative benefit for thin provisioning; no workload segregation is possible.

Due to the architectural design of the system, all LUNs reserve an initial 17GB at

creation time, which can result in a significant initial waste of space. In addition, all

LUNs are incremented in 17GB chunks. HP 3PAR StoreServ thin technology reserves no

space on initial LUN creation, and uses increments in 16KB, making it highly efficient.


XIV can support the thick-to-thin conversion of volumes as part of the Data Migration

feature. Volumes imported from other systems do have zero-pages identified and

removed during the process. However, Data Migration requires configuration changes


and the placement of the XIV "inline" with the host and the original volume. This task

requires an outage to achieve. With HP 3PAR StoreServ thin technology, data can be

migrated into the array via the host, identifying zero-pages in line and without requiring

a host outage.


XIV implements a zero-page-reclaim feature to identify and eliminate "empty" blocks of

data. However, the feature is implemented as part of the background data "scrubbing"

routine that trawls the array and looks for data integrity and parity issues. Reclamation

of ZPR data can take many days (or as long as three weeks) to complete, so there is an

over-allocation of physical space until zero pages can be identified and recovered. HP

3PAR StoreServ thin technology implements active ZPR detection inline at the time of

data write, ensuring "empty" pages are immediately identified and eliminated before

data is written to physical disk.

XIV supports the Symantec Thin Reclamation API for instant space reclamation,

enabling hosts running Symantec Storage Foundation, version 5 and above, to directly

signal to the array when storage is released. This requires the deployment of the Veritas

File System on each host connected to the array. Instant space reclamation does not

support mirrored volumes, volumes that have snapshots or snapshots themselves,

making the recovery process limited.


Testing Overview and Methodology

The aim of performing vendor comparison tests is to show how HP 3PAR StoreServ

compares to other vendors in terms of performance and efficiency. Although the

implementations from each vendor appear to offer similar features, the implementations

differ greatly in their performance and efficiency. The following tests were performed in

the competitive summary list.

The storage systems tested were not directly comparable for performance targets or

specifications. They varied in the number, size and types of drives, the number of

controllers and other physical specifications. Therefore, the data generated in Edison's

zero-page-reclaim performance test should only be compared for the differences for each

tested array from the test baseline. The exception is the Large Pre-Allocation results,

which demonstrate the effects of the different thin provisioning and storage

architectures on capacity utilization, rather than a change in performance for the

systems.

Details of the hardware tested can be found in the Appendix, at the end of this

document.

Test 1—Zero-Page-Reclaim Performance

This test aims to show the impact of zero-page-reclaim functionality on each array. The

reclaim function is an essential property of "stay thin," ensuring that ongoing allocations

don't turn thin volumes into thick ones over time. Ideally this test should not impact I/O

performance. The test process performed the following steps:

1. Create a single large "thick" 200GB LUN and assign to a Windows host.

2. Quick format the LUN with the NTFS file system.

3. Perform load test with IOMETER, writing binary zeros to the LUN, recording IOPS

and latency figures.

4. Repeat the test with a 200GB thin LUN.

Prior to the test, the zero-page-reclaim task was enabled on the VSP system. For the

EMC platforms, the zero-page-reclaim feature was enabled by performing a LUN

migration, the method recommended by EMC.


Test 2—Large Pre-Allocation

This test aims to show the overhead at the initial creation of thin LUNs and addresses

the requirement to "start thin." Ideally the creation of thin LUNs should reserve the

minimum amount of storage possible on the array. The test process performed the

following steps:

1. Create five 200GB thin LUNs and assign to a Windows host.

2. Quick format the LUNs with the NTFS file system.

3. Measure the amount of space consumed as indicated by the array.


Test Results

Test 1—Zero-Page-Reclaim Performance

The data in these tables represents the performance for each array capable of zero-page-

reclaim within the test parameters. Data for IBM XIV was not included because that

system performs reclaim over a very long period of time that was outside the test

parameters.1

Platform Degradation from Baseline (%)

EMC VMAX 48.19%

Dell Compellent 42.33%

EMC VNX 29.99%

Hitachi VSP 23.34%

HP 3PAR 0.00%

Table 1 - Test 1 - IOPS Performance during ZPR

1 According to an IBM Redbook, IBM XIV Storage System: Copy Services and Migration, it,

"could take up to three weeks for used space value to decrease… This is because recovery of

empty space runs as a background task."(Page 264). Not only is the time required for ZPR outside

the parameters of our research, enabling over-provisioning is, "not within the scope of the

administrator role."(Page 30) This suggests that an IBM engineer must perform an

overprovisioned configuration.

http://www.redbooks.ibm.com/redbooks/pdfs/sg247759.pdf


Figure 1 - Test 1 - IOPS Performance during ZPR

Platform Degradation from Baseline (%)

EMC VNX 42.61%

EMC VMAX 40.58%

Dell Compellent 74.22%

Hitachi VSP 30.33%

HP 3PAR 1.23%

Table 1 - Test 1 - I/O Latency during ZPR


Figure 2 - Test 1 - I/O Latency during ZPR

The results of this test show that ZPR activity has an impact on both the latency and

throughput of each platform except HP 3PAR StoreServ. The greatest effect was seen on

EMC VMAX performance and the latency increase with Dell Compellent.

Edison was able to determine that performance impact shown on EMC VMAX was

because the platform needs to read each thin device extent into cache in order to

perform ZPR processing. This cache load clearly has a direct impact on array

performance.

Edison was unable to diagnose the causes of the increase latency on the Dell Compellent

system.

The HP 3PAR StoreServ array has dedicated ASICs to handle the ZPR workload without

impacting on delivering I/O to hosts.


Test 2—Large Pre-Allocation

The results from this test are shown in the following table and graph.

Platform Space Allocated (MB)

EMC VMAX 17

NetApp FAS 368

Dell Compellent 800

Hitachi VSP 1,230

HP 3PAR 3,125

EMC VNX 20,039

IBM XIV 86,000

Table 2 - Test 2 - Large Pre-Allocation

Figure 3 - Test 2 - Large Pre-Allocation


The results of this test show EMC VNX and IBM XIV performed poorly in pre-

allocations. EMC VNX reserves a minimum of 3GB per LUN; IBM XIV reserves a

minimum of 17GB per LUN. The other platforms performed well. Clearly when systems

have large volumes of LUNs, the minimum reserve can have a detrimental impact on

the aims of "starting thin," resulting in large amounts of unusable storage.


Conclusions and Recommendations

Thin provisioning is a great space optimization feature that can be used to increase

levels of utilization on storage arrays. The direct benefit is reducing capital expenditure

on hardware and operational expenditure on management. However as we have seen

from the tests, not all thin provisioning implementations are equal in terms of their

ability to optimize space with minimal impact on performance.

HP 3PAR StoreServ systems adhere to three basic principles:

1. Start Thin—the creation of new LUNs requires minimal overhead. In Test 2 we saw

all arrays perform efficiently at this, except for the EMC VNX and IBM XIV

platforms. With low overhead on LUN creation, the efficient platforms can scale to

far greater numbers of LUNs and so can deliver storage resources more efficiently.

2. Get Thin—the ability to move data from thick to thin deployments. The import of

existing data into an array requires features that enable data to be optimized as it is

written to disk. Only HP 3PAR StoreServ is able to perform inline zero detection at

write time. EMC VNX, Dell Compellent and IBM XIV are able to zero detect when

data is imported under certain circumstances, for example as part of replication, but

these are not completely flexible solutions.

3. Stay Thin—the ability to detect and free unused space over time. As data is written

to thin volumes, the trend is for LUNs to grow in size to equal the logically allocated

capacity. This can happen because of defragmentation or with "thin unfriendly" file

systems that embed metadata with content, or are inefficient at reusing released

resources. Most vendors, with the exception of NetApp now support some form of

zero-page-reclaim or UNMAP feature, where space is returned to the array when

released by the host. However these background tasks can have a significant impact

on host I/O performance as was demonstrated in Test 1.

Only the HP 3PAR StoreServ platform provides a thin provisioning implementation

that delivers the most efficient storage utilization.

Best Practices

The testing and research in this white paper highlights a number of best practice

considerations:

1. Implement Zero-Page-Reclaim—This feature should be used to ensure LUNs stay

thin, however on most platforms (except 3PAR because of its custom ASIC and XIV


because it runs so slowly) needs to be scheduled out of normal production hours to

minimize performance impact.

2. Be aware of minimum LUN sizes—When setting a standard for thin provisioned

LUNs, ensure that the minimum configured LUN size is not likely to waste capacity.


Appendix A—Document References

The following documents were referenced during the production of this white paper.

TR-3505 - NetApp Deduplication for FAS and V-Series Deployment and

Implementation Guide

TR-3563 – NetApp Thin Provisioning Increases Storage Utilization with On Demand

Allocation

TR-3483 – Thin Provisioning in a NetApp SAN or IP SAN Enterprise Environment

GC27-3913-03 - IBM XIV Storage System Planning Guide

GC27-3912-02 – IBM XIV Storage System Product Overview

4AA3-3516ENW – HP 3PAR Architecture

300-006-718 – Best Practices for Fast, Simple Capacity Application with EMC

Symmetrix

H2222.3 – EMC VNX Virtual Provisioning White Paper

300-011-798 – EMC VNX Series Release 7.0 VNX System Operations

Dell Compellent – Data Progression Data Sheet


Appendix B—Test Equipment Specification

The following equipment was used to perform the testing documented in this white

paper.

Arrays

Hitachi VSP

NetApp FAS3140, running Data ONTAP 8.0.2, RAID-DP across 28 drives.

Dell Compellent—RAID-5 across 72x 600GB SAS drives

EMC VNX5700 running microcode 5.31, RAID-5 across 24x 300GB 15K SAS drives.

HP 3PAR F400 InForm OS 3.1.1 (MU1)

EMC VMAX-20K

IBM XIV Gen2, 72x 1TB SATA drives

Servers

HP BL Blade Servers, 2x Intel X5650 CPU, 16GB RAM, HP Flex10 I/O

Windows 2008R2 SP1 & CentOS 6.2

IOMeter v2006.07.27

4AA4-4079ENW

Date post:	25-Mar-2018
Category:	Documents
Upload:	phungnhan
View:	213 times
Download:	0 times

White Paper - Message · PDF fileEMC VMAX ... Large Pre-allocation—test of the ability...

Documents