Sunil Kumar EMC Software and services India Pvt. Ltd · 2019-02-11 · Advancement in copy data...

2015 Storage Developer Conference - India. © Insert Your Company Name. All Rights Reserved.

Emerging trends in copy management

Sunil Kumar EMC Software and services

India Pvt. Ltd


Agenda

Introduction to copy data Copy management system Emerging use cases


Copy data

What is copy data ? Copies of primary/production data which represents the data stored on disk at any given point in time. Benefits of copy data: No impact to production environment (host and applications ) as it is

typically created off the host by storage controller. Lower RPO by creating frequent copies (any point in time , hourly

basis) Quick recovery in case of data loss/corruption results in lower RTO


Types of copy data

-Snapshot -Full copy -CDP(Continuous data protection)


copy data - Snapshot Snapshot: Space efficient point in time copy data created by using a bunch of pointers (copying only meta data) to the original data stored on disk in a storage device (LUN / File system)

Typically starts with almost zero size as no physical copy of original data is performed when snapshots are created

Size grows as original data

changes on production LUN/Volume/FS

X Y Z

Original LUN

Snapshot Data

. . .


copy data –Full copy

Full copy: Point in time copy data which is created by performing physical copy of all the blocks of original data stored on disk in a storage device (LUN / File system) to another storage device of same size.

May not be available instantly like snapshot (depends on implementation)

Highly available as all blocks of

production data is copied on separate storage device.

Not as space efficient as snapshot as

entire data needs to be copied

X Y Z

Original LUN

Full copy Data

X Y Z


copy data - CDP

CDP: Point in time copy data which is created by capturing every single change made to the original data stored on disk in a storage device (LUN / File system) to a separate storage location

Journal every write individually, hence enabling "Any-Point-In-Time" snapshots

Offers near zero data loss(extremely

low RPO) Space consumption depends on

implementation(compression algorithm) used by particular CDP system.

X Y Z

Original LUN

X’ Y’ X’’

Journal volume

HOST

CDP system


Feature comparison Snapshot Full copy CDP

Recover capability from data corruption error

Yes Yes Yes

Storage Space requirement Low Required only for changed data

High Same size as source

High Maintains local copies and tracks all writes

Requires Original data Yes All unmodified blocks points to original data

No Contains full copy of all blocks from source

No A local copy of original data is maintained.

RPO Offered LOW Space Overhead is low and Creation is much faster, so can be created frequently(mainly hourly)

HIGH Space overhead is high, so typically created on daily basis

Extremely LOW Since every writes are captured, it allows to roll back to any point in time


Copy data market

As per research done by analyst firm IDC More than 60% of all enterprise disk capacity worldwide is filled with

copy data By 2016, spending on storage for copy data will approach $50 billion

and copy data capacity will exceed 315 million terabytes By 2016, companies will spend 8 times as much on copy data by 2016

as they will on storage for big data and analytics In the next 12 months, [IT departments] expect increased use of data

copies for app development and testing, regulatory compliance, multi-user access and long-term archival

https://idc-community.com/groups/it_agenda/storageanddatamanagement/copydataopportunitygetsrecognizedmeanwhilethemarketcontinuestogrow


Challenges with copy data Multiple silo systems generating copy data which leads

to Operational inefficiency Network performance Increased expenditure to procure and maintain copy data

storage Copy data primarily perceived as insurance against data loss Without central lifecycle management, many copies end up

sitting around for long periods of time just taking up space for no purpose.

Copy data Production data

Test/Dev

Analytics

BC/DR

backup

production

..


Copy management system

Removes infrastructure silos by providing a single pane of glass for all the copy management

operation by eliminating multiple copy generations tools and scripts,

reducing redundant copy data storage cost

Brings in new use cases of copy data besides protection. Improves operational efficiency by automating several

tasks related to copy data use cases.


Copy management system Use cases

Oracle

Exchange

SAP

.

Copy Management

System

Oracle clone copy

Restore to production

Oracle snap of clone

Test & Dev


Long term retention



Backup

Analytics

Refreshable copy


Copy management system High level architecture

SQL Oracle Exchange

File systems

Volume Managers

Device driver

HBA

Disk Subsystem

Storage Controller

SQL Oracle Exchange

File systems

Volume Managers

Device driver

HBA

Disk Subsystem

Storage controller/ Copy Creator

module

Application/host module

Database & Business logic

Storage Controller

Copy management system

Host Host

Workflow Engine & Scheduler


Typical copy management system workflow to create a copy

Prepare application and storage for copy data

creation

Put application in backup mode(freeze)

Create point-in-time snapshot of underlying

storage volume

Take application out of backup mode (unfreeze)

Catalog application and Storage specific

information


Copy data consistency

There are primarily three types of data consistency offered by any copy management system Inconsistent: copy contains what is on disks at the time of copy without bothering about interdependency among different files on multiple volumes/disk. Crash-Consistent: Interdependent files across different LUN/volumes guaranteed to be the same version(i.e. same point in time). Application-Consistent: Involves application in the copy creation process and creates a transaction consistent copy by flushing all of application memory and I/O operations so that the application copy is completely consistent.


Copy data consistency

Past: Primarily Crash consistent copy (with the use of storage controller provided consistency via consistency groups) without involving application Emerging: Mostly application aware copy by involving application and OS(VSS) backup component in the copy creation workflow to create an application consistent copy which creates transactional consistent copy data.


Copy data consistency – application consistency options


Copy data restore/recovery

Past: admin driven and complete application instance (database, volume group, File system etc.) restore as copy data is associated with LUN /File systems and restore from copy data roll backs entire LUN data. Emerging: It makes use of advance application and storage level interfaces to catalog complete application and storage layout and offer item level restore capability from the copy data created by

storage controller. e.g. you can restore individual mails boxes from exchange, Table from a database and individual virtual machine or virtual machine disk or files and directory from a virtual machine host.

Self service restore


Copy data restore/recovery – item level restore capability


Copy data retention

Past: Copy data was being primarily used a backup accelerator. For long term retention integration with prominent backup product via script or third party modules was required to store the copy data on long term retention media e.g. tape. Emerging: Advancement in copy data technologies (like increased number of snapshot per device, snapshot shipping and reduced cost of disk based storage) is enabling copy management system to retain, archive and store copy data to long term retention media like de-duplicated disk subsystem or cheaper cloud based storage. This is enabling copy management system itself to act like backup system eliminating the need for a separate backup product.


Copy data retention

Application data

Copy Management

System

Application copy

Long term Retention in cloud

Store copy data in Deduplicated storage


Hypervisor aware

Emerging: Increasingly adopting smarter approach while creating copies

in virtualized infrastructure Becoming aware of hypervisor based platforms and

integrating with hypervisors management framework to offer advanced capabilities like: 1. Virtual machine consistent copy of hypervisor file system which

enables copy creation of 100’s of virtual machines in a very short time

2. In case a virtual machine underlying storage is moved from one storage system to another, copy management platform dynamically follows the virtual machine storage to create appropriate copy on the underlying storage system.


Hypervisor aware


SLA driven approach

Past: Earlier copy management system were used just like an orchestration engine to assist in copy generation. SLA specific to applications RPO and RTO were defined and monitored by external application. Emerging: Copy management system are exposing SLA driven approach where an organization can define the SLA to create and manage application copies and its lifecycle. It lets application admin specify appropriate:

RPO for the application Copy retention period i.e. how long the copy needs to be kept Frequency of copy Long term retention/backup frequency

It is also providing interfaces to application admins to monitor whether the SLA are met or violated based on real time and all using a single pane of glass


SLA driven copy management system


Copy data – DevOps

Emerging: Copy management system have started to offer capabilities like: On demand multiple generation repurpose copies

from one gold copy to use for diff. use cases Refresh capability of gold copy to get the latest copy

data. Exposing interfaces (like REST)/API for integration

with other products Integration with Chef, Puppet and other DevOps

tools


Copy data – DevOps


Disaster recovery

Emerging: Enhanced to create local as well as remote (DR) copies. Both local and remote copies are created in a single

operation resulting in a single freeze/thaw operation on application. This reduces the impact on production environment.

Getting integrated with proven DR solutions like VMware SRM and others

Vendors are building DR orchestration engine inside the copy management system itself to reduce dependency on third party DR products.


Copy data – Analytics

Emerging: Analytics is normally not performed on the

production environment. Most of the Big data analytics requires a Test/Dev

kind of environment involving multiple copies of data Copy management solutions optimizes the space

consumed per copies by using smarter approach of creating copies out of copies for multiple use cases and managing the life cycle of these copies

Many solutions are using smarter approach to recreate the production environment from Copy data by integrating with the native applications

Date post:	04-Jan-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Sunil Kumar EMC Software and services India Pvt. Ltd · 2019-02-11 · Advancement in copy data...

Documents