NeST: Network Storage Technologies

Building I/O Appliances on Commodity SystemsJohn Bent, Andrea Arpaci-Dusseau,Remzi Arpaci-Dusseau and Miron Livny

http://www.ioappliance.com

Outline

IntroductionCase studiesStorage modulesConclusion

Problem Statement

Appliances are attractive because they are robust, reliable, available and especially because they are easy to use. To fulfill these criteria, traditional network appliances impose policy decisions on their users and are built either as kernel modules or upon specially designed kernels.

“How to build portable, configurable I/O appliances?”

Goal

To create a network-storage “template” that produces a range of I/O appliances according to the storage needs of the target application and any constraints of the host system.

NetworkStorage

Technologies

Target App

Host System

PerfectI/O

Appliance

Host system constraints

Thread supportRaw disk accessSelect interface

Target app. storage needs

Invariant and variant storage needsInvariant

Reliable Low latency High bandwidth Easy to administer Cheap

Target app. storage needs

Variant Write concurrency Replacement costs Security and authentication needs Communication protocol Transfer unit

Outline

IntroductionCase studiesStorage modulesConclusion

Building I/O appliances

Four case studies ReqEx WiND Web proxy cache Condor checkpoint server

What is ReqEx?

ReqExStaging Area

Huge tape library (terabytes)

Queue of Reqs

Tape Robot

A robot moves archived data one tape at a time to a temporary staging area.

PerfectI/O

Appliance

Condor Manager

What is ReqEx?

ReqExStaging Area

WAN

Compute cluster

Data is transferred and stored locally to facilitate access by compute nodes.

ReqEx variant storage needsWrite concurrency

No write (or read) concurrency

Replacement costs Tape robot is very slow; objects cannot be lost

Security and authentication needs Only owner can remove object

Protocol ReqEx can be linked with NeST client library

Transfer unit Whole object transfers only

What is WiND?

WiND variant storage needsWrite concurrency

No write concurrency

Replacement costs Unknown

Security and authentication needs Unknown

Protocol Predefined specific WiND protocol

Transfer unit Disk blocks are accessed directly

What is a web proxy cache?

Local Area Network

Internet

PerfectI/O

Appliance

Frequently accessed objects can be stored locally to decrease request latencies.

Cache variant storage needsWrite concurrency

No write concurrency

Replacement costs Negligible

Security and authentication needs None

Protocol HTTP

Transfer unit Whole object transfer only

PerfectI/O

Appliance

What is Condor ckpt server?

A condor job runs on an execute machine.

Keyboard activity causes the job to be evicted. A snapshot of the process is sent to the checkpoint server.

When the job migrates to another idle machine, the checkpoint file is recovered and progress resumes.

CCS variant storage needsWrite concurrency

No write concurrency

Replacement costs The running time of the job (could be months)

Security and authentication needs Unauthorized access cannot be allowed

Protocol Can link with NeST client library

Transfer unit Whole file transfer only

I see you’re discussing checkpointing. Don’t

forget about incremental.

Outline

IntroductionCase studiesStorage modulesConclusion

Storage modules

StorageManagement

Concurrency Architectures

DataSemantics

Static Configuration

Protocols AdministrativeInterface

RuntimeAdaptation

NameSpace

Configurable Components

Concurrency architectureData semantics Protocol layerNamespaceSecurity and authenticationStorage management

Concurrency architecture

NOB POP POT

Easy ... but uninteresting.

“How can multiple storage requests be interleaved to maximize system throughput?”

Data semantics

Must stored objects be protected from concurrent writes?

Is transaction support necessary?What are the recovery costs for lost

objects?

Protocol layer

Most applications can not link with NeST client libraries

Most applications have their own specific communication protocols

“How can a protocol layer easily communicate with arbitrary networking protocols?”

Tower of Babel

Namespace

FlatHierarchical

“How do clients uniquely identify their stored objects?”

Security and authentication

OwnershipPrivacyEncryptionAuthenticationAccess rights

Storage management

Native filesystemRaw disk accessUninteresting from client perspective

Outline

IntroductionCase studiesStorage modulesConclusion

Conclusions and future work

Conclusions None

Future work Lots

Maybe you

should try a little harder.

Conclusions and future work

How to most easily identify the variant storage needs of the target application? Config file? Installation script? Run-time monitoring?

How to ensure that performance is at least as good as an appliance specifically designed for the target application?