Date post: | 24-Dec-2015 |
Category: |
Documents |
Upload: | lydia-goodwin |
View: | 222 times |
Download: | 0 times |
Graybox NFS Caching Proxy
By:
Paul Cychosz and
Garrett Kolpin
• First we need to know a little about how the “graybox” works.
• Inference by observation
• No server or client-side modifications necessary
What are Graybox Techniques?
• Figure out what the client is caching
• Maintain low overlap between client and proxy caches
Our Problem
• Treat the client as a graybox
• Monitor NFS traffic
• Infer from NFS traffic the contents of client cache
Our Solution
Cache overlap decreased by 5% to 50% when using graybox techniques
Conclusions
• Approach
• Results
• Conclusions
Outline
Approach
NFS specifics: Implemented in kernel as another FS type
Replaces “file_operations” with its own functions for read, write, open, etc.
Uses VFS abstraction
VFS
NFS kernel code
RPC / XDR
TCP / UDP
retrans., data wrapping
Approach
Client cache / Replacement
• Caches on page level • Not much NFS specific code – Uses global Linux page cache
• Not a static size, no upper-bound
• Timestamps: “jiffies”, per page.
Approach
Reads typically > ~85% of traffic
NFS / RPC “read” structure
• RPC Packets sent out: access to dir, getattr to file, access to file, then read, read, read, …
• Client use of getattr to revalidate cached copies
Exploit this, proxy explicitly look for these to determine client cache contents
Approach
ProxySees:
File handle
Offset
size
--becomes our “cache element”
Store in hash table
Cache on reads, run our cache element replacement policy on getattr (proof of cached client copy)
Approach
Read system call
Cache lookup
Revalidate data
Read setup
OR
Replacement(possible force-out)
Lookup
Return data Add
OR
NFS Client Proxy
hit
miss
hit
miss
getattr
read
Data
Forward req. to server
Typical read()
• Disable swap space
• Use mincore() to view pages that client has cached
• Use “balloon” program to keep client cache contents manageable
Viewing Client Cache Contents
Test Environment:
Approach
Logistics: Initial setup, client/server/proxy all on same PC – doesn’t work
Client cache size changes
-solved: compare percentages, normalize
Repeated RPC calls not idempotent
Difficulties
Not a significant problem, proxy efficiency degrades linearly with NFS performance if network traffic is bad
Approach
Ambiguities:
Different FS activities DON’T generate unique RPC patterns
getattr used for a few things besides reads
-- Null RPC packets
Cannot tell exact blocks client caching, only which files
Need second, local replacement policy on proxy!
Difficulties
Approach
Global Policies
• LRU
• MRU
Local Policies
• Forcing entire file
• MRU
Cache Replacement
• Approach
• Results
• Conclusions
Outline
• Control Experiments– Global LRU, no local force-out– Global MRU, no local force-out
• Combinations– Global LRU, whole file force-out– Global LRU, block-level local MRU– Global MRU, whole file force-out– Global MRU, block-level local MRU
What We Tested
• Used Postmark benchmark
• 12 files with sizes randomly selected between 4 and 50 kilobytes
• Gives us more of a “real-world” workload
How We Tested
LRU Global Replacement
0
20
40
60
80
100
120
NF WF MRU
Force-out Method
Per
cen
t
Proxy Cache Fill
Overlap
Results
MRU Global Replacement
0
20
40
60
80
100
120
NF WF MRU
Force-out Method
Per
cen
t
Proxy Cache Fill
Overlap
Results
All Combinations
0
20
40
60
80
100
120
LRU NF MRU NF LRU WF LRU MRU MRU WF MRU MRU
Replacement Combination
Per
cen
t
Proxy Cache Fill
Overlap
Results
• Approach
• Results
• Conclusions
Outline
• Decrease in overlap by between 5 and 50 percent as compared to control tests
• MRU global file replacement is good by itself
• MRU global policy with whole-file local policy gives highest cache fill and lowest overlap
Conclusions
Questions? Feedback?
?
?
?