OpenStack Swift In the EnterpriseAn Introduction to Object Storage and Swift

1

• This webinar is being recorded and an on-demand version will be available at the same URL within 15 minutes after we conclude

• Please submit questions via the button to the upper left of the viewer

• On Twitter? Join the conversation: #objectstorage, #swift, #openstack @HOSTINGdotcom

2

Housekeeping

• Critical factors driving new storage types

• What is Object Storage?

• OpenStack Swift Architecture

• Key Considerations

3

Today’s Agenda

4

Critical factors driving new storage types

• Object storage began when

applications started to store

and serve data to any device

regardless of where they are

connecting from

• Explosion in non-transactional

data (i.e. Machine generated,

rich media content, etc.)

• Cloud architectures

demanding alternatives to

storing and retrieving content

Critical factors driving new storage types cont..

• Retention of data has

increased

• Performance

degrades as number

of files or storage

increases

• IT budgets are not

increasing

5

Data growth is exploding

A look at Traditional Storage Types

Block Based File Based

• Sits on top of Block Based Storage

• Stores Files\Folders that are

humanly readable

• Can increase and decrease size of

storage

• NIFS\CIFS Protocols allow for a

single namespace

• Scaling limitations: Metadata is

held within the file, performance

suffers as number of files increase

• Is designed for LAN based access,

not Internet based devices

6

• Data is structured with a maximum length (i.e. block size)

• Typically stored in SAN environments where data is stored in volumes

• Can increase storage, however, it cannot decrease* (*without some very nifty & complex tooling)

• Scaling the size of a block disk is subject to vendor and partition limitations

• Is designed for to be accessed by low latency, high bandwidth networking

7

Introduction to Object Storage

• Object storage, often called Cloud

Storage, is commonly used by cloud

providers due to it strength in

dynamically increasing or decreasing

storage demand dictates

• All objects (i.e. movies, pictures,

documents, etc.) are stored with

• A unique Identifier

• Meta Data

• Attributes

• Data

• Is accessed via HTTP restful API

allowing applications to access the data

from anywhere

• The most commonly used Object

Platforms are AWS’s S3, and

OpenStack Swift

Object Storage: What this Enables

• Consider an application where

patient records are inputted

• The application has form fields

where the care providers inputs

diagnosis, patient local,

symptoms, etc. (i.e. Metadata of

a patient record)

• Now imagine the care provider

or research facility wants to look

at this data from not just one

hospital or location but abroad

to learn about trends

• How can this data be accessed

independent of any one facility

while ensuring the security?

Source: RightScale 2014 State of the Cloud Report

• Object Storage allows application to retrieve

metadata of a file from billions objects

• The retrieval of a given object is

authenticated and authorized

• The data can be retrieved over the Internet

or over private networks instantly via

HTTP(s)

Swift based Object Storage

9

Feature Benefit

Swift is a core OpenStack Project OpenStack is the most successful open

source project ever that set out to solve

common challenges with cloud

technologies.

Deployment Flexibility Can be deployed\consumed from Public,

Private, Hybrid cloud designs

Unlimited Storage Huge & flat namespace, highly scalable

read\write access. Ability to serve content

directly to from storage systems

Multi-dimensional Scalability (scale out

architecture)

Backup and archive large amounts of data

with linear performance

Easily* add capacity unlike RAID resize Elastic data scaling with ease

No central database Higher performance, no bottlenecks

Real-time visibility into client requests Know what exactly users are requesting

Numerous ways to access storage Users can access storage via CLI, HTTP,

file-system gateways

The Core Components of Swift

10

Swift Proxy

Storage Node

Identity Service

Understands where data lives &

routes requests

Runs account, container, &

object services

Authenticates and authorizes

storage requests

Swift Data Model

Three components to unstructured data

• Account: A unique name storage area that will contain the metadata

• Container: User-defined storage area containing metadata of the container and list of objects

• Object: Is where the data object and its metadata is held

Account: Customer A

Container 1

Object

Orange

Object

Green

Swift Zones and Regions

• Zones:

• Replicates data among nodes

within the environment

• Default replica is 3

• Ensure no networking \ or

hardware single point of failure

• Regions:

• Conceptually designed to

distribute replicas among

geographically dispersed

areas

• Used for offsite DR purposes

• Used in Active\Active Multi-

sharing environments

Region A Region B

Zone A

Node 1

Node 2

Node 3

Node 4

Node 5

Zone B

Node 1

Node 2

Node 3

Node 4

Node 5

• OpenStack Swift and Amazon S3 are eventually consistent platforms

• CAP Theorem: These platforms follow availability and partition tolerance dropping consistency

Data Consistency Processes

• Swifts eventual consistency is not a good fit for applications needing ACID guarantees such as:

• Baking systems that must ensure two transactions don't withdraw the same money

• Ticketing systems that must ensure two people never buy the same ticket

Not a good fit

Common Swift Use Cases

15

Scientific /

Research

Data

Rich Digital

Media

Archive and

Backups

• Manage rich

media

content’s

explosive

growth and file

size

• Consistent

performance

at scale

• Access to a

single system

that manages

billions and

petabytes of

records\objects

• Object metadata

makes it

possible to

conduct complex

searches

• Legal and

governance

requirements

• Monetization

of data

• Cheep and

deep

Key Considerations

Performance

• Separate Account and Container

databases to SSD backed storage

• Add and remove storage

inclemently as these operations

will incur ring rebalancing

Availability

• Use a load balancer in front of the

Proxy nodes for HA

• Consult your storage vendor when

using tradition NAS\SAN storage

as RAID technologies will overlap

with OpenStack’s replica

technology

Private | Public | Hybrid Swift

17

18

• Combining eplexity’s OpenStack Swift expertise…

• OpenStack-focused, deployment specialists

• Over 40PB+ systems management

• … with HOSTING’s World-Class Infrastructure

• 6 privately-owned and managed datacenters

• Specialists in business-critical application availability

• Unique Support Experience

1

8

Our OpenStack Swift Solution

• No DIY required – Get started

and scale!

• Designed from the ground up for

Private Cloud

19

Q&AJohn Clendennen | CTO, eplexity

Mike McCracken | Director, Professional Services, HOSTING

For more information about solutions by HOSTING, please contact our

team at 888.894.4678.