W
Web Caching
based on:
Web Caching, Geoff Huston
Web Caching and Zipf-like Distributions:Evidence and Implications, L. Breslau, P.Cao, L. Fan, G. Phillips, S. Shenker
On the scale and performance ofcooperative Web proxy caching, A.Wolman, G. Voelker, N. Sharma, N.Cardwell, A. Karlin, H. Levy
Page 1 of 24eb Caching
W
h
Hypertext Transfer Protocol (HTTP)
based on an end-to-end model (the network isa passive element)
Pros:
The server can modify the content;
The server can track all content requests;
The server can differentiate betweendifferent clients;
The server can authenticate the client.
Cons:
Popular servers are under continuous higstress
high number of simultaneously openedconnections
high load on the surrounding network
The network may not be efficiently utilized
Page 2 of 24eb Caching
W
n
Why Caching in the Network?
reduce latency by avoiding slow links betweeclient and origin server
low bandwidth links
congested links
reduce traffic on links
spread load of overloaded origin servers tocaches
Page 3 of 24eb Caching
W
d;
ffic;
on
Caching points
content provider (server caching)
ISP
more than 70% of ISP traffic is Web base
repeated requests at about 50% of the tra
ISPs interested in reducing the transmissicosts;
provide high quality of service to the endusers.
end client
Page 4 of 24eb Caching
W
Caching Challenges
Cache consistency
identify if an object is stale or fresh
Dynamic content
caches should not store outputs of CGIscripts (Active Cache?)
Hit counts and personalization
Privacy concerned user
how do you get a user to point to a cache
Access control
legal and security restrictions
Large multimedia files
Page 5 of 24eb Caching
W
st
e
Web cache access pattern -(commonthoughts)
the distribution of page requests follows aZipf-like distribution: the relative probabilityof a request for the i-th most popular page isproportional to with
for an infinite sized cache, the hit-ratio growin a log-like fashion as a function of the clienpopulation and of the number of requests
the hit-ratio of a web cache grows in a log-likfashion as a function of cache size
the probability that a document will bereferenced requests after it was lastreferenced is roughly proportional to(temporal locality)
1 iα⁄ α 1≤
k1 k⁄
Page 6 of 24eb Caching
W
es
its
t’sns
w
5)
Web cache access pattern - (P. CaoObservations)
distribution of web requests follow a a Zipf-like distribution (fig. 1)
the 10/90 rule does not apply to web access(70% of accesses to about 25%-40% of thedocuments) (fig. 2)
low statistical corellation between thefrequency that a document is accessed andsize (fig. 3)
low statistic correlation between a documenaccess frequency and its average modificatioper request.
request distribution to servers does not folloa Zipf law.; there is no single servercontributing to the most popular pages (fig.
α 0.64 0.83,∈
Page 7 of 24eb Caching
W
ts
f
Implications of Zipf-like behavior
Model:
A cache that receives a stream of reques
N = total number of pages in the universe
= probability that given the arrival oa page request, it is made for page
Pages ranked in order of their popularity,page i is thei-th most popular
Zipf-like distribution for the arrivals:
,
no pages are invalidated in the cache.
PN i( )
i i 1 N,=( ),
PN i( ) Ω iα⁄=
Ω 1 iα⁄
i 1=
N
∑ 1–
=
Page 8 of 24eb Caching
W
or
Implications of Zipf-like behavior
infinite cache, finite request stream:
finite cache, infinite request stream:
page request interarrival time:
H(R) = hit ratio
d(k) = probability distribution that the nextrequest for page iis followed by k-1 requests fpages other than i
H R( )Ω 1 α–( )⁄( ) RΩ( )1 α⁄ 1–
= 0 α 1< <( ),
Ω ΩR( )log= α 1=( ),
H C( )C
1 α–= 0 α 1< <,
Clog= α 1=( ),
d k( )1
k Nlog--------------- 1 1
N Nlog-----------------–
k1 1
Nlog------------–
k–
≈
Page 9 of 24eb Caching
W
Cache Replacement Algorithms(fig. 9)
GD-Size:
performs best for small sizes of cache.
Perfect-LFU
performs comparably with GD-Size in hit-ratio and much better in byte hit-ratio forlarge cache sizes.
drawback: increased complexity
In-Cache-LFU
performs the worst
Page 10 of 24eb Caching
W
to
ve
Web document sharing and proxycaching
What is the best performance one couldachieve with “perfect” cooperative caching?
For what range of client populations cancooperative caching work effectively?
Does the way in which clients are assigned caches matter?
What cache hit rates are necessary to achieworthwhile decreases in document accesslatency?
Page 11 of 24eb Caching
W
al
For what range of client populations cancooperative caching work effectively?
for smaller populations, hit rate increasesrapidly with population (cooperative cachingcan be used effectively)
for larger population cooperative caching isunlikely to bring benefits
Conclusion:
use cooperative caching to adapt to proxyassignments made for political or geographicreasons.
0 5000 10000 15000 20000 25000 30000
Population
0
10
20
30
40
50
60
70
80
90
Req
ues
t H
it R
ate
(%)
Ideal (UW)Cacheable (UW)Ideal (MS)Cacheable (MS)
Page 12 of 24eb Caching
W
Object Latency vs. Population
Latency stays unchanged when populationincreases
Caching will have little effect on mean andmedian latency beyond very small clientpopulation (?!)
0 5000 10000 15000 20000
Population
0
500
1000
1500
2000
Ob
ject
Lat
ency
(m
s)Mean No CacheMean CacheableMean IdealMedian No CacheMedian CacheableMedian Ideal
Page 13 of 24eb Caching
W
Byte Hit Rate vs. Population
same knee at about 2500 clients
shared documents are smaller
0 5000 10000 15000 20000
Population
0
10
20
30
40
50
60B
yte
Hit
Rat
e (%
)
IdealCacheable
Page 14 of 24eb Caching
W
on
Bandwith vs. population
while caching reduces bandwidth consumptithere is no benefit to increased clientpopulation.
0 5000 10000 15000 20000
Population
0
1
2
3
4
5M
ean
Ban
dw
idth
(M
bit
s/s) No Cache
CacheableIdeal
Page 15 of 24eb Caching
W
ct is
Proxies and organizations
there is some locality in organizational membership, but the impanot significant
978
735
480
425
392
337
251
246
239
228
222
219
213
200
192
15 Largest Organizations
0
10
20
30
40
50
60
70
80
90
100
Hit
Rat
e (%
)
Ideal CooperativeIdeal LocalCacheable CooperativeCacheable Local
978
735
480
425
392
337
251
246
239
228
222
219
213
200
192
15 Largest Organizations
0
10
20
30
40
50
60
70
Hit
Rat
e (%
)
UWRandom
Page 16 of 24eb Caching
W
e
Impact of larger population size
unpopular documents are universallyunpopular => unlikely that a miss in one larggroup of population to get a hit in anothergroup.
UW
Idea
l
UW
Cac
heab
le
MS
Idea
l
MS
Cac
heab
le
0
20
40
60
80
100
Hit
Rat
e (%
)
CooperativeLocal
Page 17 of 24eb Caching
W
h
t
Performance of large scale proxycaching - model
N = clients in population
n = total number of documents ((aprox. 3.2billions)
pi = fraction of all requests that are for the i-tmost popular document ( , )
exponential distribution of the requests doneby client with parameter , where = 590reqs/day = average client request rate
exponential distribution of time for documenchange, with parameter (unpopulardocument days, populardocuments days)
pc = 0.6 = probability that the documents arecacheable
E(S) = 7.7KB = avg. document size
E(L) = 1.9 sec = last-byte latency to server
pi1
iα----≈ α 0.8=
λN λ
µµu 1 186⁄=
µp 1 14⁄=
Page 18 of 24eb Caching
W
te
of
ith
Hit rate, latency and bandwidth
Hit rate = f(Population) has three areas:
1. the request rate too low to dominate the raof change for unpopular documents
2. marks a significant increase in the hit ratethe unpopular documents
3. the request rate is high enough to cope wthe document modifications
Latency
Bandwidth
10^2 10^3 10^4 10^5 10^6 10^7 10^8
Population (log)
0
20
40
60
80
100
Cac
hea
ble
Hit
Rat
e (%
)
Slow (14 days, 186 days)Mid Slow (1 day, 186 days)Mid Fast (1 hour, 85 days)Fast (5 mins, 85 days)
latreq 1 HN–( ) E L( )⋅ HN lathit⋅+=
BN HN E S( )⋅=
Page 19 of 24eb Caching
W
etsitn
Document rate of change
Proxy cache hit is very sensitive to the changrates of both popular and unpopular documenwith a decrease on the time scales at which hrate is sensitive once the size of the populatioincreases.
1min
2 5 10 30 1hour
2 6 12 1day
2 7 14 30 180
Change Interval (log)
0
20
40
60
80
100
Cac
hea
ble
Hit
Rat
e (%
)
mu_pmu_u
A
B
C
1min
2 5 10 30 1hour
2 6 12 1day
2 7 14 30 180
Change Interval (log)
0
20
40
60
80
100
Cac
hea
ble
Hit
Rat
e (%
)
mu_pmu_u
AB
C
Page 20 of 24eb Caching
W
ash
ree
ve
Comparing cooperative cachingscheme
Three types of cache schemes: hierarchical, hbased and summary caches;
Conclusions:
the achievable benefits in terms of latency aachieved at the level of city-level cooperativcaching;
a flat cooperative scheme is no longer effectiif the served area increases after a limit
in a hierarchical caching scheme the higherlevel might be a bottleneck in serving therequests.
Page 21 of 24eb Caching
W
in
Hierarchical Caching (Squid):
a hierarchy of k levels
a client request is forwarded up in thehierarchy until a cache hit occurs.
a copy of the requested object is then storedall caches of the request path
Page 22 of 24eb Caching
W
s
d
Hash-based caching system:
assumes a total of m caches cumulativelyserving a population of size N
upon a request for an URL the clientdetermines the cache in charge and forwardthe request to it.
if the cache has the URL the page is returneto the client, otherwise the request isforwarded to the server
Page 23 of 24eb Caching
W
f
a
ter
Directory based system(Summary cache)
a total of m caches serving cumulatively apopulation of size N
the population is partitioned intosubpopulations of size N/m and eachpopulation is served by a single cache
each cache maintains a directory thatsummarizes the documents stored at each othe other cache in the system
when a client request can not be served by local cache, it is randomly forwarded to acache which is registered to have thedocument. If no document has the documenthe request is forwarded to the original serv
Page 24 of 24eb Caching