FAWNA Fast Array of Wimpy Nodes*

Bogdan Eremia, SCPD

*by DavidAndersen, Jason Franklin, Michael Kaminsky,Amar Phanishayee,LawrenceTan,Vijay Vasudevan

•

Energy in computing

• Power is a significant burden on computing3-yearTCO soon to be dominated by power

2

Hydroelectric Dam

3

“Energy consumption by…data centers could nearlydouble ...(by 2011) to more than 100 billion kWh,representing a $7.4 billion annual electricity cost” [EPA Report 2007]

“Google’s power consumption ...would incur anannual electricity bill of nearly $38 million”

[Qureshi:sigcomm09]

Annual cost of energy for Google,Amazon,Microsoft=

Annual cost of all first-year CS PhD Students

Monday, October 12, 2009

Can we reduce energyuse by a factor of ten?

Still serve the same workloads

Avoid increasing capital cost4

FAWN Improve computational efficiency ofdata-intensive computing using an arrayof well-balanced low-power systems.

FastArray ofWimpy Nodes

CPU

CPU

Disk

CPU

CPU

Mem

()* %&' +,-#.

()* %&' +,-#.

()* %&' +,-#.

()* %&' +,-#.

()* %&' +,-#.

()* %&' +,-#.

40W

TraditionalServer

FAWN

AMD Geode256MB DRAM

4GB CompactFlash

5220W

{

Goal:reduce peak powerTraditional Datacenter

Power

Cooling

Distribution

1000W

750W

FAWN

100W

<100W

100%

Servers

20%

20% energy loss(good)

6

Overview

• Background

• FAWN Principles

• FAWN-KV Design

• Evaluation

• Conclusion

7

Nanoseconds

CPU Cycle

Towards balanced systems1E+08

1E+07

1E+06

1E+05

1E+04

1E+03

1E+02

1E+01

1E+00

1E-011980 1985 1990 1995 2000 2005

Year

Rebalancing Options

Today’s CPUs Slower CPUsArray of Fast Storage

Fastest Disks

Slow CPUsToday’s Disks

Disk Seek

Wastedresources

DRAM Access

8

Inst

ruct

ions

/sec

/W in

mil

lion

sSpeed vs.Efficiency

Targeting the sweet-spot in efficiency

1000

500

0

1500

2000

2500

1 10 100 1000 10000 100000

Custom ARM Mote

XScale 800Mhz

Xeon7350

Atom Z500

Fastest processorsexhibit superlinear

power usage

Fixed power costs candominate efficiencyfor slow processors

FAWN targets sweet spotin system efficiency when

including fixed costs

Instructions/sec in millions

(Includes 0.1W power overhead)9

1000 1500 2000 250050001

10

100

1000

10000

Inst

ruct

ions

/sec

in m

illi

ons

100000

Custom ARM Mote

Xeon7350

XScale 800Mhz

Atom Z500

Targeting the sweet-spot in efficiencyInstructions/sec/W in millions

FAWN

More efficientToday’s CPU Slower CPU Slow CPU

Array of Fast Storage Today’s DiskFastest Disks

10

••

Overview

• Background

• FAWN Principles

• FAWN-KV DesignArchitecture

Constraints

• Evaluation

• Conclusion11

Data-intensive KeyValue

• Critical infrastructure service

• Service level agreements for performance/latency

• Random-access,read-mostly,hard to cache

12

•

FAWN-KV:Our KeyValue Proposition

• Energy-efficient cluster key-value store

• Goal:improve Queries/Joule

• Prototype:Alix3c2 nodes with flash storage500MHz CPU,256MB DRAM,4GB CompactFlash

13

Monday, October 12, 2009

• Wimpy CPUs, limited DRAM

• Efficient and fast

failover

•

FAWN-KV:Our KeyValue Proposition

• Prototype:Alix3c2 nodes with flash storage500MHz CPU,256MB DRAM,4GB CompactFlash

14

Unique Challenges:

• Flash poor at small random writes

FAWN-KVArchitecture

X

Back-end

Back-end

Back-end

Back-end

Back-end

Front-end

KV Ring

Consistent hashing

FAWN-DS

Manages BackendsActs as GatewayRoutes Requests

15

FAWN-KVArchitecture

16

Front-end X

Back-end

Back-end

Back-end

Back-end

Back-end

FAWN-DS

FAWN-DS FAWN-KVEfficient Failover

Avoid random writesLimited Resources

Avoid random writes

{

Log-structured Datastore• Log-structuring avoids small random writes

18

GetPut

Delete

Random Read

Append

FAWN-KVEfficient Failover

Avoid random writes✔✔

FAWN-DSLimited Resources

Avoid random writes

(H,B]

Hash Index Values

H

G

F C

B

A

On a node addition

D

Node additions, failures require transfer of key-ranges

19

Concurrent Inserts,

Nodes stream data range

21

ConcurrentInserts

A ••Background operations sequentialContinue to meet SLA

from B toDatastore List Stream Atomic Update Aof Datastore List

Minimizes locking

Compact Datastore

FAWN-KVEfficient Failover

Avoid random writes✔✔

✔✔

FAWN-DSLimited Resources

Avoid random writesMonday, October 12, 2009

FAWN-KV Take-aways

• Log-structured datastore

• Avoids random writes at all levels

• Minimizes locking during failover

• Careful resource use but high performing

• Replication and strong consistency

• Variant of chain replication (see paper)

21

Overview

• Background

• FAWN principles

• FAWN-KV Design

• Evaluation

• Conclusion

22

Evaluation Roadmap

• Key-value lookup efficiency comparison

• Impact of background operations

• TCO analysis for random read workloads

23

FAWN-DS LookupsSystem

Alix3c2/Sandisk(CF)

Desktop/Mobi (SSD)

MacbookPro / HD

Desktop / HD

QPS

1298

4289

66

171

Watts

3.75

83

29

87

QPSWatt

346

51.7

2.3

1.96

• FAWN-based system over 6x moreefficient than 2008-era traditional systems

24

Qu

eri

es

per

seco

nd

Queri

es

per

seco

nd

Impact of background ops

0

1600

1200

800

400

Peak Compact SplitMerge

Peak query load

0

1600

1200

800

400

Peak Compact SplitMerge

30% of peak query load

Background operations have:• Moderate impact at peak load• Negligible impact at 30% load

25

26

When to use FAWN forrandom access workloads?

TCO = Capital Cost + Power Cost ($0.10/kWh)

FAWN (10W each)

2TB disk

64GB SATA Flash SSD

2GB DRAM per node

~$250-500 per node

Traditional (200W)

Five 2TB disks

160GB PCI-e Flash SSD

64GB FBDIMM per node

~$2000-8000 per node

./*,

(#

"#$0.12*+*,-./

Ratio of query rate to

cooling,

!$""""

!$"""

!$""

!$"

!$

!"#$!"#$ !$"""

0.12*+*,%34

0.12*+*0)#35

-)*+

%&%'!

!$ !$" !$""

01)23!4&')!56+77+8-(9():;

•••

Conclusion• FAWN architecture reduces energy

consumption of cluster computing

•FAWN-KV addresses challenges of wimpy nodesfor key value storage

Log-structured,memory efficient datastore

Efficient replication and failover

Meets energy efficiency and performance goals

“Each decimal order of magnitude increase inparallelism requires a major redesign and rewrite ofparallel code”- KathyYelick

28

•