+ All Categories
Home > Software > Openstack Swift Introduction

Openstack Swift Introduction

Date post: 14-Jul-2015
Category:
Upload: youn-sung-park
View: 3,077 times
Download: 3 times
Share this document with a friend
Popular Tags:
34
Swift Highly available, distributed, eventually consistent object storage Youn Sung Park
Transcript
Page 1: Openstack Swift Introduction

SwiftHighly available, distributed, eventually consistent object

storage

Youn Sung Park

Page 2: Openstack Swift Introduction

March 16, 2015 Presentation Title

2

1. Introduction

2. Features - Basic

3. Features - Advanced

Contents

Page 3: Openstack Swift Introduction

Introduction

1

Page 4: Openstack Swift Introduction

Contents

• What is Swift?

Object Storage

Swift component

March 16, 2015

Swift

4

Introduction

Page 5: Openstack Swift Introduction

Object Storage

• Object Storage – What is Object Storage?

March 16, 2015

Swift

5

Introduction

Swift

- Highly available, distributed, eventually consistent object storage

Page 6: Openstack Swift Introduction

Object Storage

• Object Storage – What is Object Storage?

March 16, 2015

Swift

6

Introduction

Page 7: Openstack Swift Introduction

Object Storage

• Object Storage – What is Object Storage?

March 16, 2015

Swift

7

Introduction

Object storage vs File storagevs Volume storage

Page 8: Openstack Swift Introduction

Object Storage

• Object Storage – Legacy Storage

March 16, 2015

Swift

8

Introduction

WAS WAS WAS

NAS / SAN NAS / SAN NAS / SAN NAS / SAN…

Page 9: Openstack Swift Introduction

Object Storage

• Object Storage – Legacy Storage

March 16, 2015

Swift

9

Introduction

WAS WAS WAS

NAS / SAN NAS / SAN NAS / SAN NAS / SAN…

Storage manager

Page 10: Openstack Swift Introduction

Object Storage

• Object Storage – Legacy Storage

• Fault tolerance – RAID5, 6, 10, ...

• Interface – NFS for file, iSCSI for volume

• Multi tenancy - ??

• QoS - ??

• Authentication - ??

• Data Placement - ??

• Metadata management - ??

• Capacity management - ??

• Rebalancing - ??

March 16, 2015

Swift

10

Introduction

Page 11: Openstack Swift Introduction

Object Storage

• Object Storage

March 16, 2015

Swift

11

Introduction

WAS WAS WAS

Object Storage

Page 12: Openstack Swift Introduction

Object Storage

• Object Storage – Legacy Storage

• Fault tolerance – Replication

• Interface – REST API

• Multi tenancy – Defaut. SWAuth, Keystone, Tempauth, LDAP(?)

• QoS – Rate limit, Bandwidth limit, Software defined ...

• Authentication - SWAuth, Keystone, Tempauth

• Data Placement – The Ring algorithm

• Metadata management – Ring and database

• Capacity management – The Ring algorithm

• Rebalancing – The Ring algorithm

March 16, 2015

Swift

12

Introduction

Page 13: Openstack Swift Introduction

Components

• Swift component – Server application

March 16, 2015

Swift

13

Introduction

Proxy Server

Account Server

Container Server

Object Server

Sqlite

Container List

Object List

Objects

Sqlite

Page 14: Openstack Swift Introduction

Components

• Swift component – Daemon

• Replicator – Replicate objects and make a system in a consistent state.

• Recover disk failure, network outages situation.

• Updater – Update metadata

• Recover failure caused by container, account metadata high load.

• Auditor – Delete problematic account, container or objects and replicate from other server

• Recover dbs or files which have bit rot problem.

March 16, 2015

Swift

14

Introduction

Page 15: Openstack Swift Introduction

Components

• Swift component – Key algorithm: Data placement, Metadata management

March 16, 2015

Swift

15

Introduction

RingContainer name/file name Location of a file

ex) 3 copyServer A / sdcServer H / sdyServer D / sdp

Page 16: Openstack Swift Introduction

Features - Basic

2

Page 17: Openstack Swift Introduction

Contents

• Swift features

History – Releases

Architecture

High availability

High scalability

Eventually consistent & Self healing

REST API

March 16, 2015

Swift

17

Features - Basic

Page 18: Openstack Swift Introduction

History - Releases

March 16, 2015

Swift

18

Features - Basic

Page 19: Openstack Swift Introduction

Architecture

• Architecture

March 16, 2015

Swift

19

Features - Basic

Proxy Server Proxy Server Proxy Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone1

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone2

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone3

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone4

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone5

A/C/O Server

Sqlite

Account server

Container server

Object server

Sqlite

Ring Ring Ring

Page 20: Openstack Swift Introduction

Features

• High availability, Fault tolerance

March 16, 2015

Swift

20

Features - Basic

Proxy Server Proxy Server Proxy Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone1

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone2

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone3

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone4

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone5

A/C/O Server

Sqlite

Account server

Container server

Object server

Sqlite

Ring Ring Ring

Page 21: Openstack Swift Introduction

Features

• High scalability

March 16, 2015

Swift

21

Features - Basic

A/C/O Server

HDDHDDHDDHDD

A/C/O Server

HDDHDDHDDHDDHDDHDDHDDHDD

More disk

Zone1Zone1

Zone1Zone1

Zone5

Zone1Zone1

Zone1Zone1

Zone1Zone1

Zone7

More zones

A/C/O Server

A/C/O Server

Zone

A/C/O Server

A/C/O Server

Zone

A/C/O Server

A/C/O Server

More servers

Page 22: Openstack Swift Introduction

Features

• Eventually consistent & self healing

March 16, 2015

Swift

22

Features - Basic

Proxy Server Proxy Server Proxy Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone1

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone2

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone3

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone4

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone5

A/C/O Server

Sqlite

Account server

Container server

Object server

Sqlite

Fail

Eventual consistency

Ring Ring Ring

Page 23: Openstack Swift Introduction

Features

• Eventually consistent & self healing

March 16, 2015

Swift

23

Proxy Server Proxy Server

A/C/O Server

A/C/O Server

Zone1

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone2

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone3

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone4

A/C/O Server

A/C/O Server

A/C/O Server

A/C/O Server

Zone5

A/C/O Server

Sqlite

Account server

Container server

Object server

Sqlite

Server failure

Self healing - replication

A/C/O Server

Proxy Server

Features - Basic

Ring RingRing

Page 24: Openstack Swift Introduction

Features - Advanced

3

Page 25: Openstack Swift Introduction

Contents

• Swift features

History – Features

Features

Large object support

Static web hosting

S3 compatible API

Object expiration

Temp url

Object versioning

Global cluster

Storage policy

March 16, 2015

Swift

25

Features - Advanced

Page 26: Openstack Swift Introduction

History - Features

March 16, 2015

Swift

26

Features - Advanced

Swift 2.2 ReleaseStorage policy support

Swift 1.10 ReleaseGlobal clusters support

Swift 1.8 ReleaseDifferent replica policy for A/C/OStatsD update

Swift 1.7 ReleaseObject versioning supportStatd integration

Swift 1.4.8 ReleaseObject expiration supportTemp url support

Swift 1.4.3 ReleaseS3 compatible API support

Swift 1.3 ReleaseStatic website support

Swift 1.2 ReleaseLarge object support

Swift 1.0 Release

Swift 1.12 ReleaseAccount ACL support

Page 27: Openstack Swift Introduction

Features

• Large object support

• Swift limitation

• Single object: 5GB

• Split object & manage large object

• Manage segmented objects by manifest file

• Ref

• http://docs.openstack.org/developer/swift/overview_large_objects.html

March 16, 2015

Swift

27

Features - Advanced

Page 28: Openstack Swift Introduction

Features

• Static web hosting

• Upload static web file and make web site

• Upload web site file with index and error files

• Use statiscweb middleware

• Ref

• http://docs.openstack.org/developer/swift/middleware.html#staticweb

March 16, 2015

Swift

28

Features - Advanced

Page 29: Openstack Swift Introduction

Features

• S3 compatible API

• Support S3 API

• Support limited API less than 40%

• Use swift3 middleware

• Ref

• https://github.com/stackforge/swift3

March 16, 2015

Swift

29

Features - Advanced

Page 30: Openstack Swift Introduction

Features

• Object expiration

• Schedule deletion of objects

• Use X-Delete-At and X-Delete-After header while using an object PUT or POST

•X-Delete-At: Delete object at specified time

•X-Delete-After: Delete object after specified time

• Ref

• http://docs.openstack.org/developer/swift/overview_expiring_objects.html

March 16, 2015

Swift

30

Features - Advanced

Page 31: Openstack Swift Introduction

Features

• Temp url

• Provide url to access in limited time

• Need temp_url_expires time in header

• Use temporary URL middleware

• Ref

• http://docs.openstack.org/developer/swift/api/temporary_url_middleware.html

March 16, 2015

Swift

31

Features - Advanced

Page 32: Openstack Swift Introduction

Features

• Global cluster

• Make a single cluster in distant region

• Read/Write affinity

• Deferred replication

• Ref

• http://docs.openstack.org/developer/swift/admin_guide.html

• https://swiftstack.com/blog/2012/09/16/globally-distributed-openstack-swift-cluster/

March 16, 2015

Swift

32

Features - Advanced

Page 33: Openstack Swift Introduction

Features

• Storage policy

• Support various policy in sing storage cluster

• Use multiple ring file

• Ref

• http://docs.openstack.org/developer/swift/admin_guide.html

March 16, 2015

Swift

33

Features - Advanced

Page 34: Openstack Swift Introduction

Thanks


Recommended