2013 Summer Institute: Discover Big Data, August 5-9, San Diego, California SAN DIEGO SUPERCOMPUTER...

2013 Summer Institute: Discover Big Data, August 5-9, San Diego, California

SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA, SAN DIEGO

Dealing with Data:Choosing a Good Storage Technology

for Your Application

Rick WagnerHPC Systems Manager

July 1st, 2014



Application Focus

Storage choices should be driven byapplication need, not just what’s available.

But, applications need to adaptas they scale.

Writing a few small files to anNFS server is fine…

writing 1000’s simultaneously willwipe out the server.

If you use binary files,don’t invent your own format.

Consider HDF5.



Storage Technologies

File Systems Devices Services

memory

block

Cloud

MySQL

CouchDB

ext4

NFS

Lustre

PVFS

FUSE



Storage Technologies

File Systems Devices Services

memory

block

Cloud

MySQL

CouchDB

ext4

NFS

Lustre

PVFS

FUSE

Each has its own performance characteristics

Not all are available everywhere



File Systems

Classic access, POSIX, Windows

Most relevant:• Local• Remote

• NFS, CIFS• Parallel (Lustre, GPFS)

Local file systems are good for small and temporary files

Network file systems very convenient for sharing databetween systems



Parallel File Systems

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5030CONSOLEMGT

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5030CONSOLEMGT

Rail 0

Rail 1

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

0

1

00

1

0 0

1

00

1

0 0

1

00

1

0 0

1

00

1

0

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5030CONSOLEMGT

STATUS

PSU 1

PSU 2

FAN

RST

3433

3231

3635

2827

2625

3029

2221

2019

2423

1615

1413

1817

109

87

1211

43

21

65

IS 5030CONSOLEMGT

16 Compute Nodes

16 Compute NodesFlash I/O Node

Flash I/O Node

Each switch connected to its 6neighbors via 3 QDR links

LustreFilesystem

Dual 10GbE

Dual 10GbE



Parallel File Systems

OSS72TB

32 OSS (Object Storage Servers) Provide 100GB/s Performance and >4PB Raw Capacity

Arista 750810G

Arista 750810G

Redundant Switches for Reliability and Performance

3 Distinct Network Architectures

OSS72TB

OSS72TB

OSS72TB

64 Lustre LNET Routers100 GB/s

Mellanox 5020 Bridge12 GB/s

MDS

MDS

Myrinet 10G Switch25 GB/s

MDS

GORDONIB cluster

TRITONMyrinet cluster

TRESTLES IB cluster

Metadata Servers



A Cautionary Tale

http://www.youtube.com/watch?v=gDfLXAtRJfY&feature=youtu.be





Devices

Raw block device (/dev/sdb) or RAM FS (/dev/shm)

Useful in specific cases, like fast scratch

Can be very good for small I/O



Services

Things accessed programmatically

Frequents the last thought for HPCapplications: A MISTAKE

DatabasesCloud storage (Amazon S3)Document storage (MongoDB, CouchDB)



Know What You Need

http://www.youtube.com/watch?v=F4OIDszDA9E





Order of Magnitude Guide

Storage file/directory file sizes BW IOPs

Local HDD 1000s GB 100 MB/s 100

Local SSD 1000s GB GB/s 10000

RAM FS 10000s GB GB/s 10000

NFS 100s GB 100 MB/s 100

Lustre/GPFS 100s TB 100 GB/s 1000

Cloud Infinite TB 10 GB/s 0

DB N/A N/A N/A 10000



Choosing

My application needs to:

I should consider:

Write a checkpoint dump from memory from a large parallel simulation.

A parallel file system and a binary file formatlike HDF5.



Choosing


I should consider:

Run analysis on remote systems and return the results to a web portal for users.

Cloud storage for results and input, and local scratch space for the job.



Choosing


I should consider:

Randomly access many small files, or read and write small blocks from large files.

A database, RAM FS, or local scratch space.



Many Boxes Make a Sad Panda

http://www.youtube.com/watch?v=N2zK3sAtr-4

Database logos courtesy of RRZEiconshttp://commons.wikimedia.org/



http://commons.wikimedia.org/wiki/User:RRZEicons

Date post:	15-Dec-2015
Category:	Documents
Upload:	micheal-finn
View:	213 times
Download:	0 times

2013 Summer Institute: Discover Big Data, August 5-9, San Diego, California SAN DIEGO SUPERCOMPUTER...

Documents