Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 213 times |
Download: | 0 times |
Cooperative Caching Middleware for Cluster-Based Servers
Francisco Matias Cuenca-Acuna
Thu D. Nguyen
Panic Lab
Department of Computer Science
Rutgers University
Our work
• Goal– Provide a mechanism to co-manage memory of cluster-based
servers
– Deliver a generic solution that can be reused by Internet servers and file systems
• Motivation– Emerging Internet computing model based on infrastructure
services like Google, Yahoo! and others
» Being built on clusters: scalability, fault-tolerance
– It’s hard to build efficient cluster-based servers
» Dealing with distributed memory
» If memories are used independently, servers only perform well when the working set fits on a single node
Previous solutions
Request distribution based on load and
data affinity
Web Server
FS
Web Server
FS
Web Server
FS
Web Server
FS
Web Server
FS
Network
A A
Front end
Previous solutions
Web Server
FS
Web Server
FS
Web Server
FS
Web Server
FS
Web Server
FS
Network
A A
Distributed front end
Request distribution based on load and
data affinity
Round Robin req .distribution
Our approach
Web Server
FS
Web Server
FS
Web Server
FS
Web Server
FS
Web Server
FS
Network
AA
Cooperative block caching and global block replacement
Round Robin req distribution
Our approach
Web Server
FS
Web Server
FS
Web Server
FS
Web Server
FS
Web Server
FS
Network
Round Robin req distribution
Other uses for our CC layer
Why cooperative caching and what do we give up?
• Advantages of our approach– Generality
» By presenting a block-level abstraction
» Can be used across very different applications such as web servers and file systems
» Doesn’t need any application knowledge
– Reusability
» By presenting it as a generic middleware layer
• Disadvantages of our approach– Generality + no application knowledge possible
performance loss
– How much?
Our contributions
• Study carefully why cooperative caching, as designed for cooperative client caching to reduce server load, does not perform as well as content-aware request distribution
• When compared to a web server that uses content-aware request distribution
– Lose 70% when using a traditional CC algorithm
– Lose only 8% when using our adapted version (CCM)
• Adapt cooperative caching to better suit cluster based servers
– Trade lower local hit rates for higher total hit rates (local + global)
Our cooperative caching algorithm (CCM)
• Files are distributed across all nodes– No replication
– The node holding a file on disk is called the file’s home
– Homes are responsible for tracking blocks in memory
• Master blocks and non-master blocks– There is only one master block for each block/file in memory
– CCM only tracks master blocks
• Hint based block location– Algorithm based on Dahlin et. al (1994)
– Nodes have approximate knowledge of block location and may have to follow a chain of nodes to get to it
Replacement mechanisms
• Each node maintains local LRU lists
• Exchange age hints when forwarding blocks– Piggyback age of oldest block
• Replacement– Victim is a local block: evict
– Victim is a master block:
» If oldest block in cluster according to age hints, evict
» Otherwise, forward to peer with oldest block
Assessing performance
• Compare a CCM-based web server against one that uses content-aware request distribution
– L2S (HPDC 2000)
» Efficient and scalable
» Application-specific request distribution
» Maintain global information
» File based caching
• Event driven simulation– The same simulator used in L2S
• The platform we simulate is equivalent to:– 1Gbps VIA LAN
– Clusters of 4 & 8 nodes with single 800Mhz Pentium III
– IDE hard drive on each node
Workload
• Four WWW traces:
• Drive server as fast as possible
Trace Avg. req. size Num. of requests Working set sizeCalgary 13.67KB 567823 128MBClarknet 9.50KB 2978121 250MBNASA 20.33KB 3147685 250MBRutgers 17.54KB 745815 500MB
0
1000
2000
3000
4000
5000
6000
7000
8000
4 8 16 32 64 128 256Memory per node (MB)
Th
rou
gh
pu
t (r
eq/s
ec)
L2S
CCM-Basic
Results
Throughput for Clarknet on 8 nodes
0
1000
2000
3000
4000
5000
6000
7000
8000
4 8 16 32 64 128 256Memory per node (MB)
Th
rou
gh
pu
t (r
eq/s
ec)
L2S
CCM-DS
CCM-Basic
0
1000
2000
3000
4000
5000
6000
7000
8000
4 8 16 32 64 128 256Memory per node (MB)
Th
rou
gh
pu
t (r
eq/s
ec)
L2S
CCM
CCM-DS
CCM-Basic
Hit Rate
0
10
20
30
40
50
60
70
80
90
100
4 8 16 32 64 128 256 512
Memory per node (MB)
Remote hit rate
Local hit rate
0
10
20
30
40
50
60
70
80
90
100
4 8 16 32 64 128 256 512
Memory per node (MB)
Remote hit rate
Local hit rate
Hit rate distribution on CCM-Basic Hit rate distribution on CCM
To
tal
hit
rat
e
To
tal
hit
rat
e
Normalized throughput
Throughput normalized versus L2S
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
4 8 16 32 64 128 256Memory per node (MB)
No
rmal
ized
th
rou
gh
pu
t
Clarknet
Rutgers
Nasa
Calgary
Resource utilization
CCM’s resource utilization
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
4 8 16 32 64 128 256 512Memory per node (MB)
No
rmal
ized
res
ou
rce
usa
ge
Disk
CPU
NIC
Scalability
Throughput when running on varying cluster sizes
0
2000
4000
6000
8000
10000
12000
14000
Number of nodes
Th
rou
gh
pu
t (r
eq
/se
c)
Further results
• Performance differences between CCM and L2S may be affected by:
– L2S’s use of TCP hand-off
– L2S’s assumption that files are replicated everywhere
– Refer to paper for estimates of potential performance difference due to these factors
• Current work– Limit the amount of metadata maintained by CCM
» To reduce memory usage
» Discard outdated information
– Lazy eviction and forwarding notification
» On average finds a block with 1.1 hops (vs. 2.4)
» 10% response time decrease
» 2% throughput increase
Conclusions
• A generic block-based cooperative caching algorithm can efficiently co-manage cluster memory
– CCM performs almost as well as a highly optimized content aware request distribution web server
– CCM scales linearly with cluster size
– Presenting a block-based solution to a file-based application only led to a small performance loss should work great for block-based applications
• CCM achieves high-performance by using a new replacement algorithm well-suited to a server environment
– Trades off local hit rates and network bandwidth for increased total hit rates
– Right trade-off given current network and disk technology trends
Future & related work
• Future Work– Investigate the importance of load-balancing
– Provide support for writes
– Validate simulation results with implementation
• Some Related Work– PRESS (PPoPP 2001)
– L2S (HPDC 2000)
– LARD (ASPLOS 1998)
– Cooperative Caching (OSDI 1994)
– Cluster-Based Scalable Network Services (SOSP 1997)
Thanks to
• Liviu Iftode
• Ricardo Bianchini
• Vinicio Carreras
• Xiaoyan Li
Want more information?www.panic-lab.rutgers.edu
Extra slides – Response time
Response time normalized versus L2S
0.6
0.8
1
1.2
1.4
1.6
1.8
4 8 16 32 64 128 256Memory per node (MB)
No
rmal
ized
res
po
nse
tim
e
Clarknet
Rutgers
Nasa
Calgary
Extra slides – Hops vs. hit rate
Number of hops versus hit rate
0
0.5
1
1.5
2
2.5
4 8 16 32 64 128 256 512Memory per node (MB)
Nu
mb
er
of
ho
ps
0
20
40
60
80
100
120
Hit
ra
te
Hops (w/ notification)HopsGlobal Hit Rate