Date post: | 02-Dec-2014 |
Category: |
Technology |
Upload: | alexander-alten-lorenz |
View: | 2,544 times |
Download: | 4 times |
February 2012
Filesystems, RPC and HDFSAlexander Lorenz
Agenda
2
1 Linux Kernel I/O Scheduler
2 I/O Stack in Linux
3 VFS Implementation
4 NFS RFC Model
5 RPC
6 HDFS
7 Limitations / Problems (Discussion)
©2011 Cloudera, Inc. All Rights Reserved. Confidential. Reproduction or redistribution without written permission is
prohibited.
Linux Kernel I/O Scheduler
• Disk seek is the slowest operation in a computer
• I/O scheduler arranges the disk head to move in a single direction to minimize seeks
• Prevent Starvation
• Improve overall disk throughput by • Reorder requests to reduce the disk seek time• Merge requests to reduce the number of requests
3
Kernel I/O Scheduler Framework
Block layer
external queue device driver
enqueue
Internal queues
dequeue
IO SchedulerMerge, sort
prioritize
• Linux elevator is an abstract layer to which different I/O scheduler can attach
• Merging mechanisms are provided by request queues• Front or back merge of a request and a bio• Merge two requests
• Sorting policy and merge decision are done in elevators• Pick up a request to be merged with a bio• Add a new request to the request queue• Select next request to be processed by block drivers
4
Filesystem
Userland
KernelspaceSys Calls
Access Locking
Prefetch Flush
Disk Layout MetaData
HDD Driver
Cache
I/O Stack in Linux
5
Application
Bulk writes
VFS Implementation
6
Userland
KernelspaceSys Calls
VFS
ext3 ext2 NFS CIFS
Application
NFS RFC Model
Local HDD Local HDD
Applicationwith
NFS Access
FilesystemNFS Client
RPC
TCP/IPUDP
RPC
TCP/IPUDP
NFS ServerFilesystem
Kernelspace Kernelspace
File Handler
7
NFS - OSI Model
8
RPC
Client Server
Process starts
Process continued
Server waits
Server waits
Server start
PCPE
PR
Termination
RPC Return
RPC Message
Client waits
9
HDFS Layer
10
Local Client
VFS
HDFS Application
POSIX API HDFS API
Network HDFSNFS Driver
HDFS Model
10
add Blck (src)
write
Pipe
line
HDFS Cluster
Namenode
Client
DN
DN
DN
Block received
Block received
Block received
HDFS Write Model
DN
DN
Client NNRPC (ClientProtocol)
RPC (DFSClient.DFSInputStream)
RPC (DataNodeProtocol)
RPC rcv only
FSData stream (socket)
RPC (DataNodeProtocol)
RPC proxy
RPC proxy
DFS
RPCProxy IPC
VFS
HDD
DN intern
xceiver
11
Links / Resources
13
The Hadoop Distributed File System Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler, Yahoo!
http://dtrace.org/blogs/brendan/2011/05/11/
NFS and RPC Chavalit Srisathapornphat, CISC856
Linux I/O Schedulers Hao-Ran Liu