Integrating Portable and Distributed Storage

Niraj Tolia

Jan Harkes, Michael Kozuch, M. Satyanarayanan

PARALLEL DATA LABORATORYINTEL RESEARCH PITTSBURGH

Carnegie Mellon University

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 2

Motivation

• Explosive growth of portable storage devices

• Are these glorified floppy disks?

• In a world of ever-improving networking, why do we still carry them around?

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 3

Integration Value

• Portable Devices• Performance• Availability• Ubiquity

• Distributed File Systems• Robustness• Sharing and Collaboration• Consistency• Capacity

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 4

10,000 Foot View

• Lookaside Caching - All cache lookups check portable device

• Minimal disruption of existing usage model

Distributed File System Server

Client

NetworkNetwork

PortableDevice

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 5

Outline

• Motivation• Lookaside Caching• Benchmarks and Evaluation• Related Work• Conclusion

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 6

Lookaside Caching

• Technique to combine the strengths of dist. file systems and portable storage

• Design Goals• Be tolerant of user error• No compromise of robustness or consistency

• Simplicity!

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 7

Design Decisions

• File server is the authoritative source• No change made to native file system

• Allows flexibility in the choice of device• Allows user to look at files stored on the

portable device without a file system client• Only addition is an index file

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 8

Lookaside Caching Flowchartopen()

cached

done network fetch

done

Yes No

fetch attributes

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 9

Lookaside Caching Flowchartopen()

cached

done network fetch

done

Yes No

fetch attributes + hash

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 10

Lookaside Caching Flowchartopen()

cached

done

Yes No

fetch attributes + hash

portable

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 11

Lookaside Caching Flowchartopen()

cached

done

Yes No

fetch attributes + hash

portable

done

Yes

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 12

Lookaside Caching Flowchartopen()

cached

done

network fetch

done

Yes No

fetch attributes + hash

portable

done

Yes No

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 13

Implementation• File system metadata extended to

include a cryptographic hash (SHA-1) • Based upon whole-file hashing• Implemented in Coda 6.0

• mkdb utility to generate index file• Index generated from normal file system tree• Lazy update process• Allows users to change data

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 14

Outline

• Motivation• Lookaside Caching• Benchmarks and Evaluation

• Benchmark Descriptions• Evaluation Setup• Results

• Related Work• Conclusion

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 15

Benchmarks

• Kernel Compile • Client File System Trace • Virtual Machine Migration

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 16

Benchmark Description

• Kernel Compile• Compiled Linux 2.4.18• Compile reads 80.7 MB (118 MB source tree)• Portable device contained either 2.4.18,

2.4.17, 2.4.13, 2.4.9, or 2.4.0

• Client File System Traces• Four desktop clients from the DFSTrace traces• No think time modelled

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 17

Benchmark Description

• Virtual Machine Migration• Based on the Internet Suspend/Resume project• Time taken to resume a migrated Virtual

Machine and execute a MS Office-based productivity benchmark

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 18

Evaluation Setup

• Hi-Speed USB Flash device• Evaluated at different bandwidths

• 100 Mb/s, 10 Mb/s, 1 Mb/s + 10 ms, and 100 Kb/s + 100 ms

NIST Net Router

File ServerClient

USB Keychain

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 19

USB Portable Device Bandwidth

• Max below manufacturer ratings• Low bandwidth observed for small files

25.016.764 KB

26.525.8

7.4Write (Mb/s)

29.4100 MB28.61 MB

6.34 KBRead (Mb/s)Size (KB)

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 20

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

100 Mb/s 10 Mb/s 1 Mb/s 100 Kb/s

No

rmal

ized

Ru

nti

me

2.4.0 .9 .13 .17 .18

Linux Kernel Compile

Linux Kernel Versions

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 21

Trace Replay

0.0

0.2

0.4

0.6

0.8

1.0

1.210

0 M

b/s

10 M

b/s

1 M

b/s

100

Kb/

s

100

Mb/

s

10 M

b/s

1 M

b/s

100

Kb/

s

Berlioz Purcell

No

rmal

ized

Ru

nti

me

33% 66% 100%

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 22

Internet Suspend/Resume

• Two metrics of performance• Resume Time• Benchmark Time

VM started VM usable

Time

End of Benchmark

Resume Time

BenchmarkTime

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 23

Internet Suspend/Resume

• Resume Time• Portable device had VM memory image (100%

hit rate)

99.7%124301100 Kb/s

96.2%123171 Mb/s

69.2%123910 Mb/s

7.1%1314100 Mb/s

Win %With Lookaside (s)No Lookaside (s)

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 24

Internet Suspend/Resume

• Benchmark Time• Portable device had an unmodified Windows

XP + Office image (~50% hit rate)

• Slowdown

68.8%953030531100 Kb/s

61.6%103226881 Mb/s

42.7%21237010 Mb/s

6.9%161173100 Mb/s

Win %With Lookaside (s)No Lookaside (s)

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 25

Internet Suspend/Resume

• Suspend Time• Lookaside caching does not currently handle this• Apart from portable storage, you can also use

staging servers, etc.• Topic of ongoing research

VM started VM usable

Time

End of Benchmark

Resume Time

BenchmarkTime

Data Safe

SuspendTime

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 26

Related Work

• Mobile Storage Solutions• Segank, PersonalRaid, Footloose, Personal

Server

• Commercial Offerings• Migo, KeyComputing

• Other Content Addressable Systems• CASPER, LBFS• CFS/Chord, PAST/Pastry, Venti, Pastiche

Niraj Tolia April 2, 2004http://www.pdl.cmu.edu/ 27

Conclusion

• Demonstrated the integration of portable and distributed storage

• Shown that a simple optimization can have a dramatic effect on performance

• Future Work• While invented for portable devices, lookaside

caching can be extended to use off-machine storage