Power efficiency as an analogue to memory management

Sara Alspaugh and Arka Bhattacharya

Conflicting goals?

• To save energy one needs to shut down servers

• Shutting down servers incurs huge latency penalty on active-set misses.

Take Lessons from Memory Management .

Memory Management

CACHE(<1 ns)

MEMORY DRAM

(~10 ns)

ACTIVE DISK(~10 ms)

P miss,cache

P miss,memory

Tavg= Tcache + Pmiss,cache .Tmemory + Pmiss,cache.Pmiss,memory .Tdisk

Penalty : 100 Penalty : 1000

REQ

UES

TS

Disk Management

CACHE(<1 ns)

MEMORY DRAM

(~100 ns)

ACTIVE DISK(~10 ms)

Tavg= Tcache + Pmiss,cache .Tmemory + Pmiss,cache.Pmiss,memory .Tdisk + Pmiss,cache.Pmiss,memory .Pmiss,active-disk.Tinactive-disk

IN-ACTIVE DISK

(~5 sec)

Pmiss,active-disk

P miss,cache

P miss,memory

Penalty : 100 Penalty : 1000

Penalty : 5000

REQ

UES

TS

For interactive jobs , average latency needed < 100 ms

Permissible active-disk misses

• Tavg ≈ Tactive-disk + Pmiss,active-disk.Tnon-active-disk

Permissible active-disk misses

• (100ms) ≈ (10ms) + Pmiss,active-disk.(5 sec)

Tactive-disk ≈ (10ms)Tnon-active-disk ≈ (5 sec)

For interactive jobs : Tavg < (100 ms)

Pmiss,active-disk < 2%

Permissible active-disk misses

• Tavg ≈ Tactive-disk + Pmiss,active-disk.Tnon-active-disk

TO CREATE ENERGY-PROPORTIONAL STORAGE , WE THUS HAVE TO MAINTAIN A NON-ACTIVE STORE AND REDUCE Pmiss,active

Techniques

1. Put disks to sleep after a loiter time , and wake-up on demand

2. Caching 3. Prefetching disk blocks4. Dynamic object creation/Intelligent layout

• Tavg ≈ Tactive-disk + Pmiss,active-disk.Tnon-active-disk

Wikipedia 1-day trace

• Replicated thrice• Partitioned into 5 disks

MSR-Cambridge Web/SQL server 7 day trace

• Not replicated• 4 volumes• 10 disks each

1 . Putting disks to sleep after Loiter

ACTIVE DISKACTIVE DISK ACTIVE DISK ACTIVE DISK

Request Stream

NON- ACTIVE

DISK

LOITER TIME ?

What loiter time to set ?

Energy Improvement

Latency Increase

Low Loiter TImes

High Loiter TImes Desired

100ms

Active-set misses vs aggressiveness of loiter

Putting it together

Active-disk misses

Do we really care about average latency ?

95th percentileLoiter Time = 100 sec

MSR

2 . Caching

File access frequency for top 5000 accessed files

5 205 405 605 805 100512051405160518052005220524052605280530053205340536053805400542054405460548050

1000

2000

3000

4000

5000

File Access frequency (Wikipedia)

Access frequency

5 255 505 755 1005 1255 1505 1755 2005 2255 2505 2755 3005 3255 3505 3755 4005 4255 4505 47550

1000

2000

3000

4000

5000

File access frequency - MSR Cambridge Web/SQL Server

Access Frequency

Options

1. Caching on an SSD2. Keep active set on and keep some spinning

reserve

CACHE(<1 ns)

MEMORY DRAM

(~10 ns)

ACTIVE DISK(~10 ms)

SSD cache(~50 μs)

REQ

UES

TS

What to cache on ?

DRAM SSD Hard Disk

Latency 5-10ns 60 μs 10ms

Cost / GB $100 $1-$2 5-10 ¢

Bandwidth 1500 MB/s 300-500 MB/s 50-100MB/s

Energy negligible 1-3W 10W

Increase in Average Idle Time with Increase in Cache Size

0 10000 50000 10000000

1

2

3

4

5

6

7

Wikipedia

CACHE SIZE

LARG

E ID

LE T

IME

(sec

)

0 100000 500000 10000000

50

100

150

200

250

300

350

400

450

MSR-Cambridge

CACHE SIZE

LARG

E ID

LE T

IME

(sec

)

How aggressive are you ? Different loiter times – (1) MSR

How aggressive are you ? Different loiter times – (2) Wikipedia

CACHE(<1 ns)

MEMORY DRAM

(~100 ns)

FIRST REPLICA(~10 ms)

IN-ACTIVE DISK

(~5 sec)

SPINNING RESERVE

Keeping an active set and spinning reserveRE

QU

ESTS

Active Set , Spinning Reserve , Normal

Wikipedia

3.Prefetching

Spatial correlation

Average : 0.3 %

TIM

E W

IND

OW

BLOCK NUMBER

Graph of different prefetching constants - Microsoft

MSR

Latency Analysis

MSR

4 . Intelligent layout of Objects

ACTIVE DISKACTIVE DISK ACTIVE DISK ACTIVE DISK

Request Stream

ACTIVE DISKACTIVE DISK ACTIVE DISK ACTIVE DISK

Request Stream

NON- ACTIVE

DISK

NON- ACTIVE

DISK

Wikipedia

Summary

• Not much different from memory management

• Very simple techniques which could be optimized on the trace.

• Large energy savings to be had if we are willing to pay slight latency penalties.