Download

Information Infrastructure

© 2009 IBM Corporation

Storage for Cloud Computing

Clod Barrera – STG Systems Storage

January 20, 2009

© 2009 IBM Corporation2

Agenda

What and why is Cloud Computing?

Cloud Computing and Storage

Ensembles as Cloud and Enterprise Infrastructure

Storage Ensembles


Cloud Computing

Notes:

Abstraction – alleviates IT consumers from the operations of applications, allowing end users to focus instead on the execution and high-value activities

Virtualization – access to business services on-demand independent of location and resource constraints

Dynamic allocation – dynamically provisions, configures, reconfigures, and de-provisions IT capability as and when needed, transparently and seamlessly

Speed and reliability through standardization, higher level of automation, and abstractionFlexibility through virtualization, dynamic resource sharing, and proactive capacity allocationEfficiency through higher level of automation, catalog standardization, and prioritization

SOA

Service Mgmt

Emerging model where users can have access to applications or compute resources from anywhere through their connected devices through a simplified UI

Applications reside in massively-scalable data centers where compute resources can be dynamically provisioned and shared to achieve significant economies of scale

A strong service management platform results in only marginal incremental management cost, if any, when more workload needs to be handled by the cloud

Virtual-ization

Information


Key Attributes of Cloud Computing

Simple “self service” user interface

where underlying technology and

services are irrelevant to the user.

*A Service Management platform allows an operations management team to collect or discover information about how the resources associated with a service are configured and consumed and then access that information to support integrated management processes that address changes to and problems with the service.

When adding IT resources to

the service, the incremental

cost to manage these new

resources approaches zero

Provides for an infinitely scalable

pool of connected computing

resources

Although often associated with Internet Services, clouds will exist

both inside a data center to support that customers & internal

users as well as supporting third-party managed service offerings.

Efficient, green, and cost effective

infrastructure


AaaS - Architecture as a Service BaaS - Business as a Service CaaS - Computing as a Service CRMaaS - CRM as a Service DaaS - Data as a Service DBaaS - Database as a Service EaaS - Ethernet as a Service FaaS - Frameworks as a Service GaaS - Globalization or Governance as a Service HaaS - Hardware as a Service IaaS - Infrastructure or Integration as a Service IDaaS - Identity as a Service LaaS - Lending as a Service MaaS - Mashups as a Service OaaS - Organization or Operations as a Service SaaS - Software as a Service StaaS - Storage as a Service PaaS - Platform as a Service TaaS - Technology or Testing as a Service VaaS - Voice as a Service

A variety of ‘as-a-Service’ terms have been used to describe the services offered in Clouds:


Who’s Offering Cloud Services today?3Tera Grid-based Platform For Cloud AppsAkamai Application Performance ServicesAmazon.com Amazon Web ServicesAretiInternet Virtual HostingBungee Labs Virtual Labs-as-a-Service softwareCohesiveFT Cloud PlatformElastra Hosted Elastic ComputingEMC Storage Cloud – Decho(Mozy, Pi) +

Cloud Infrastructure & Services DivisionEnki Computing UtilityFlexiscale Utility Computing on DemandFortress ITX DynamicGridGoogle Search ServicesHP Adaptive Infrastructure as a ServiceiCloud Desktop CloudIBM Computing Cloud, Advanced UIs, Deep Capacity on DemandJoyent Accelerator for ApplicationsLayered Technology GridLayerMicrosoft CRM, email and IM ServicesMosso Hosting CloudSalesforce.com Force.comTerremark InfinistructureXCalibre FlexiScale


Cloud Computing Users and Business ModelsMain actors and their interaction mode in the use and delivery of Cloud

Services, enable the identification of the models and evolving technologies that support the Cloud Computing strategy:

• Acquisition Model (Service): "All that matters is results; I don't care how it's done"

• Business Model (Pay for usage): "I don't want to own assets - I want to pay for elastic usage, like a utility"

• Access Model (Internet): "I want accessibility from anywhere, from any device"• Technical Model (Scalable, elastic, shareable): "It's about economies of scale, with

effective and dynamic sharing"

Service Consumers

Service Catalog,ComponentLibrary

CloudAdministrator

DatacenterInfrastructure

Monitor & ManageServices & Resources

Component Vendors /Software Publishers

Publish & UpdateComponents,Service Templates

AccessServices

IT Cloud

Garter, 2008


Cloud Computing: A New Paradigm

Enterprise Cloud

Network Cloud

INNOVATIVE BUSINESS MODELS

End Users / Requestors

Government/ Academics

Industry(Startups/ SMB/ Enterprise) Consumers

• An “Elastic” pool of high performance virtualized compute resources

• Cloud applications enable the simplificationof complex services

• A cloud computing platform combines modular componentson a service oriented architecture

• New combinations of services to form differentiating value propositions at lowercosts in shorter time

• Internet protocol based convergence of networks and devices

SIMPLIFIED SERVICES

Source: Corporate Strategy


Cloud computing perceived benefits and demand drivers

Awareness and demand for cloud computing are fueled by perceived / potential benefits to business entities, individual users, and IT providers (includes enterprises + service providers)

Cloud computing’s “nirvana-like” promise drives higher service level expectations among business entities and

individual users

Which in turn puts pressure on the enterprise data center to deliver higher service quality (at lower

cost)

IT Providers

Key Benefit:

Competitivenes

Lower TCO

Faster Time to Market

Higher Cust Rentention

Service quality

Resource optimization

Resiliency

Flexibility

Efficiency

“Green”

Enhanced chargeback

Business Entities

Key Benefit:

Innovation

Faster, easier innovation

New business models

New products and services

Faster time to market

Lower IT cost

Lower IT risk (brand protection)

Improved IT user productivity

Improved Client Satisfaction

Improved Disaster Recovery

IT Users

Key Benefit:

Quality of Experience

Speed of access

Ease of access (anywhere, anytime)

Ease of use

Minimal software requirements on access device

No long-term commitments


Model for Delivering Cloud Services (single provider)

Fulfillment Services

• End Users make requests using a catalog of services• The Provider delivers service using fulfillment processes and technology• Services are normalized to achieve Abstraction and Encapsulation

Fulfillment Process

Fulfillment + Enterprise Process (e.g. Open Account)

Fulfillment + Enterprise Process + Governance (e.g. ITIL)

Provisioning Services

Usage & Accounting

Services

MonitoringServices

SecurityServices

EnergyManagement

Services

Technology

….

End User Requestors& Operators

…

ServiceRequestor

Service CatalogRequest UI

Operational UI

ServiceProvider

ServiceCreator

Federated Data Repository

ServiceDefinition

Tools

ServicePublishing

Tools

ServiceFulfillment &Configuration

ToolsUI, Fulfillment, Data, MetaData

ServiceMonitoring

ToolsKPIs

ServicePlanning

© 2009 IBM Corporation11 August 24, 2008

Architectural Model for Cloud Computing

End User Requests

& Operators

…

Service Request & Operations

Design & Build

Image Library (Store)

Deployment

OperationalLifecycle of Images

IT Infrastructure & Application

Provider

ServiceCreation & Deployment

Virtual Image Management

Service CatalogRequest UI

Operational UI

Standards Based Interfaces

Virtualized Infrastructure

Service Management

Service Oriented Architecture Information Architecture

Standards Based Interfaces

Service Catalog,Component

Library

DatacenterInfrastructure

Cloud Administrator

AccessServices

Optimized Middleware(image deployment, integrated security, workload mgmt., high-

availability)

Service Oriented Architecture Information Architecture

User Request Management/Self Service Portal

Security: Identity, Access, Integrity, Isolation, Audit & Compliance

Usage Accounting

License Managemen

t

Image Lifecycle

ManagementProvisioning

Performance

Management

Availability/Backup/ Restore

Service Lifecycle Management

Service Management

Virtual Resources & Aggregations

SMP Servers Network HardwareStorage Servers

System Resources

Blades Storage

Virtualized Infrastructure

Server Virt. Storage Virt. Network Virt.


The Role of Storage in Clouds - 1

Deep Infrastructure(What cloud?)

Service Management

SOAApplicationMiddleware

Virtualized InfrastructureServer / Storage / Network

Virtu

al Im

ag

e

Ma

na

ge

me

nt

Clo

ud

Se

rvic

e

Us

er

The ‘Cloud Service’ is an application or upper layer function, or a platform service

e.g. Salesforce.com, hosted computing

“Application computing” is done by the Cloud Service and is co-located with the storage

Incremental cost and time to add a new user is effectively zero

Desired Storage Infrastructure is the same as advanced Enterprise Storage (only cheaper)

Integrated server & storage managementAutomated provisioningPerformance & security controlsChargebackTCO challenges

Value of the Cloud ServiceComplete application stack offload

Compute

Requires some new/better storage function



Storage ApplicationAs a Cloud Service

Service Management

SOAStorage Application

Storage Optimized Infrastructure

Virtu

al Im

ag

e

Ma

na

ge

me

nt

Clo

ud

Se

rvic

e

Us

er

The ‘Cloud Service’ is a Storage Intensive application or upper layer function

e.g. Medical Image ArchiveBackup/DR for SMBData WarehousingContent servingClient could be an individual or a server farm

Desired Storage Infrastructure is an optimized Storage Application

Archive + Indexing + Search, long term retentionVideo/large object access

Local compute limited to:Ingest and playbackOptimization, management

Value in the Cloud ServiceMulti-tenancyContentSpecialty storage application

ComputeCompute

Requires some new/better storage function

Requires specialty storage/function



‘Storage in the Cloud’(like the old SSPs)

Service Management

SOAStorage Application –

Device emulation

Storage Optimized Infrastructure

Virtu

al Im

ag

e

Ma

na

ge

me

nt

Clo

ud

Se

rvic

e

Us

er

Storage devices are seen as local, deployed across the cloud

E: drive on laptop/desktops – Amazon S3‘Capacity on Demand’ for server farms

Desired Storage Infrastructure is a very low cost ‘Network Virtual Disk’ facility

Upsell services? Archive + Indexing + Search, long term retention

Local compute limited to:Ingest and playbackOptimization, management

Value in the Cloud ServiceConvenience, costCapacity on demand

ChallengesMulti-tenancyCostPerformanceApplication Synergy

Compute

Compute

© 2009 IBM Corporation

Making Cloud Computing a Reality requires:

Virt

ualiz

atio

n

Integrated virtualization and management with optimized systems and networks to break the lock between IT resources and business services

Autonomic ManagementAutonomic management methods for both application and infrastructure services to meet user needs and expectations for delivery of high quality of service

Ensembles

Simplified IT infrastructure, reduced complexity and management through the creation of large, consistent pools of resources that are managed as one.

Software

Services

Hardware


Methods for IT Simplification

Business Processes as Services

Service oriented architecture technologies frame business processes as services, ease of deployment, composition, reuse, change…

Topologies of federated services must be mapped onto large numbers of diverse physical and virtual resources

Heterogeneous servers, storage, networks

“Enterprises report that IT operational overhead = 70% of their IT budgetand growing . . . leaving precious few resources for new initiatives.”

– Forrester, 2007

OLD

Virtual Resource Objects/Appliances, Libraries will reduce the labor required for software stack development and management

Ensemble

Ensemble

Ensemble

Ensembles, scalable pools of like systems that are manageable as single systems will replace multitudes of individual servers and reduce the labor required for physical systems management

Service management software spans the diversity of heterogeneous physical and virtual resources, providing unified cross-platform management in support of SOA

NEW


Ensemble Components

An Ensemble generally consists of the following components:

– A pool of compatible system nodes (e.g., N physical servers; need not be homogeneous)

– Virtual resource mobility within an ensemble and with compatible ensembles

– The networks which interconnect the ensemble nodes (may be local / optimized)

– Resource virtualizers (hypervisors, I/O virtualizers, storage virtualizers, …)

– An ensemble manager appliance that provides platform management for the ensemble virtual and physical resources

– Tools for planning, ensemble creation, P2V migration, image mgmt. & composition, …

– Ensemble-local automated optimization software of performance, availability, energy usage, security, … with intelligent defaults

– Multi-system services (locking, caching, message queuing, …) may be integrated with some ensembles

OS OS OS

Hypervisor

Server

OS OS OS

Hypervisor

Server

WorkloadMobility

Server Ensemble Example

EnsembleManager

Ensemble definition:a pool of like systemsthat is manageable asa single system


Ensembles –Building blocks for the New Enterprise Data Center and Clouds

EnsemblesServer Ensemble

Ensemble Manager

Virtualization

Network Ensemble

Ensemble Manager

Virtualization

Storage Ensemble

Ensemble Manager

Virtualization

SEP Storage

Instantiate Virtual System Collections by allocating or assigning resources,

configuring the resources as required, and

managing the resources in the context of the goals

& constraints provided.


Accessing and Exploiting Ensembles

Service Management Center

Service Request Manager

Catalog of Available Services

Policies, Parameters, Options

Specific Subscriber Cloud Instance

Secure & Provision Resources Monitoring

Subscriber

Ensembles

Server Ensemble

Ensemble Manager

Virtualization

Network Ensemble

Ensemble Manager

Virtualization

Storage Ensemble

Ensemble Manager

Virtualization

SEP Storage

Cloud Service Request


# of Thingsto Manage

N virtual elements; M physical elements N virtual elements; one physical ensemble

Create, Test, and Maintain

Do it yourself; few assemblies are alike Standard “off the shelf” assemblies

Management Automation

Add-on software, custom scripts, … Built-in optimizations; intelligent defaults

Management Interfaces

Many individual knobs and variables Menus of selectable standard behaviors

# ofConsoles

Separate consoles for physical & virtual Single console; in-context functionality

Data Center Mgmt. Arch.

Monolithic; spans d. c. heterogeneity Hierarchical; pool-level modularity

The Benefits of Ensembles

Ensembles will significantly reduce IT resource management complexity and cost,and improve IT functionality, in consumable data center increments

Individual Servers Server Ensemble

OS OS OS

Hypervisor

Server

OS OS OS

Hypervisor

Server

WorkloadMobility

OS OS OS

Hypervisor

Server

OS OS OS

Hypervisor

Server

EnsembleManager


Server Ensemble Hierarchy

Level Single System Corresponding Ensemble Examples

VirtualRuntimes

WebSphere VE ensemble

Virtual OperatingSystems

AIX 6.1 WPAR ensemble

Virtual Machines

a.k.a.Logical

Partitions

System z ensemble VMWare ensemble

(e.g. on blade servers) Other x86 ensembles

Server ensembles look like single systems, hiding their internal parallel structure Virtual resource mobility within an ensemble is a key enabling technology

Middleware Ensemble

Server Ensemble

OS Ensemble

SMP Server

Hypervisor

OS

App

OS

App

App App

OS

SMP Server

Runtime MW

App App

OS

SMP Server

Hypervisor

Hypervisor

SMP Server

Hypervisor

OS

App

OS

App

App App

OS

SMP Server

Runtime MW

App App

OS

SMP Server

Hypervisor

Hypervisor

SMP Server

Hypervisor

OS

App

OS

App

App App

OS

SMP Server

Runtime MW

App App

OS

SMP Server

Hypervisor

Hypervisor

Mobility

Mobility

Mobility


Cloud Computing – Requirements on Storage

Scale, Low cost

Dynamic, unpredictable workloads

Automated management, provisioning, performance monitor & tuning, …

Security and data protection

Quality of Storage Service

Geographic distribution

Storage Use Cases

– Archive

– Database

– File

– Block


Storage EnsemblesLevel Single System Storage Ensemble Examples

Archive

Compliance Warehouse

Database

Balanced Warehouse

Data Warehouse cluster

High performance OLTP

File

SoFS Large scale NFS and CIFS

data access Stream servers Compliance stores

Block(LUN)

Virtual storage + SAN + management

Virtual SAN storage as an (enterprise ) service

Building block for larger ensembles

File Server EnsembleFile Server

Database Server Database Server/Storage Cluster

Xaction serviceQuery serviceBackupRemote copy

IntegrationProvisioningQOS, load balanceVirtual storageAutomated mgmtDisaster recovery

SAN Components SAN Ensemble

Cluster FSCluster FSFile APIsQOSStorage TiersBackupRemote copyArchive interface

Archive EnsembleBlock Storage Server

File & Archive APIsIndexing/SearchRetention PolicyCompliance+ File Ensemble

Cluster FS

Content Mgmt


Storage Ensembles are ‘Dual Use’ technology

Improve Enterprise Computing

– Lower complexity, lower costs of operation

– Improved automation

– Improved security

– Server / storage / network integration

Better infrastructure for Cloud Service Offerings

– Same as above +

– Responsive to dynamic workloads

– Responsive to new business modes


Ensemble Pool Management : Dynamic Capacity for Ensembles

New storage is discovered and can be easily added to the Ensemble

The Ensemble will automatically re-balance workloads given the new available capacity

Pooled ResourcesCPU=a+b+cMem=x+y+z

App

OS

Image

App

OS

Image

App

OS

Image

App

OS

Image

App

OS

ImageApp

OS

Image

App

OS

Image

Scheduler


Technologies of Scale - Cluster File Systems

Compute NodeStorage NodeNFS/CIFS Client

external fabric

NFS/CIFS clients

Compute cluster

NSDstorage servers

internal fabric

SAN

Highly Scalable NAS

Compute cluster NSDstorage servers

internal fabric


SoFS – GTS OfferingSimilar implementations of varying degrees of maturity found in other vendor’s products.

General Parallel Filesystem (GPFS) -

IBM’s High end clustered file system

CTDB – Clustered Trivial Database Daemon, Controls the cluster and the file service daemons

Enhanced CIFS Server with NTFS Semantics to Support Active Directory Integration

SoFS Package – Provides Management GUI, Apache file server module, acceleration tools, etc.

IBM Hardware


Storage DevicesPool 1

Storage Virtualization

Archive

Replicate

Physical S

torage M

anagement

Virtu

al S

tora

ge

Ma

na

ge

me

ntBackup/HSM


Storage DevicesPool N

Device Access, Pathing

Storage Provisioning

Archive Replicate Backup/HSM

File Systems

Content Management

DBMS

Applications

Device Access, Pathing

Storage Provisioning

Archive Replicate Backup/HSM

File Systems

Content Management

DBMS

Applications

VirtualStorageTier 1



Virtu

al S

ystem

s Ma

na

ge

me

nt

Ph

ysical S

ystem

s Ma

na

ge

me

nt

Virtual Storage as Block Infrastructure



Block Storage Export – FC, iSCSI, IB, …



Virtu

al S

tora

ge

Ma

na

ge

me

nt





Block Storage Clusters

Archive

Replicate

Backup/HSM

Physical Storage Mgmtt

Fully integrated cluster as a storage block server- Virtual Storage- Thin provisioning- Space-efficient, container based replication- Self balancing, reconfiguration, rebuild- Multiple storage tiers – cost/performance



Block Storage Export – FC, iSCSI, IB, …



Virtu

al S

tora

ge

Ma

na

ge

me

nt





Stacking Storage Ensembles

Archive

Replicate

Backup/HSM

Physical Storage Mgmtt

Higher layer ensembles built on lower ones - File Built on Block - Archive built on File

Cluster File System

Content Management


Standards for Cloud Storage

Service access interfaces

Storage service interfaces– Provisioning

– QOS

– Performance management

– Chargeback accounting

– Data protection

– Storage Security

Storage infrastructure management interfaces (SMIS)

Service Management

SOAApplicationMiddleware

Virtualized InfrastructureServer / Storage / Network

Virtu

al Im

ag

e

Ma

na

ge

me

nt

Clo

ud

Se

rvic

e

Us

er

Compute

Date post:	12-Jan-2015
Category:	Technology
Upload:	cameroon45
View:	1,298 times
Download:	0 times