IBP and CondorMicah Beck
Assoc. Prof. & Director
Condor Week, Madison May 6, 2003
Funding• Dept. of Energy
SciDAC• National Science
Foundation ANIR• UT Center for Info
Technology Research
Logistical Networking Research at UTK
University of Tennessee
• Micah Beck• James S. Plank• Jack Dongarra
University of California, Santa Barbara
• Rich Wolski
IBP: The Internet Backplane Protocol
• A scalable mechanism for deploying shared storage resources throughout the network
• A general store-and-forward overlay networking infrastructure
• A state management infrastructure for distributed applications and active services
The Network Storage Stack
Applications
Logistical File System
Logistical Tools
L-Bone
IBP
Local Access
Physical
exNode
• Our adaption of the network stack architecture for storage• Like the IP Stack• Each level encapsulates details from the lower levels, while still exposing details to higher levels
IBP: How it Works
• Storage provisioned on community “depots”• Very primitive service (similar to block service, but
more sharable)• Goal is to be a common platform (exposed)• Also part of end-to-end design
• Best effort service – no heroic measures• Availability, reliability, security, performance
• Allocations are time-limited!• Leases are respected, can be renewed• Permanent storage is to strong to share!
The Network Storage Stack
The L-bone:Resource Discovery& Proximity queries
IBP: Allocating and managing networkstorage (like a network malloc)
The exNode:A data structurefor aggregation
LoRS: The Logistical Runtime System:Aggregation tools and methodologies
L-Bone: January 2003
Current Storage Capacity: 13 TB
Multithreaded Transfers
Caching/Staging
Point-to-Multipoint
Heterogeneous Multicast
Relationship to Work of Condor Group
• Wide Area File Management/Access
• Management of Computation State
• As a Storage Allocation Layer for • Kangaroo, NeSt
• DiskRouter
• If routers can have disks, how about disks with processors?
Routers, Depots and the Network Functional Unit
router
inin
out
depot
disk/RAM
inin
out
depot
RAM
inin
out
NFU
send
store
load
execute
Scalable Operations
• IBP Depots with NFU define a State Transformation Substrate
• Processes run at “endpoints” but can use NFUs to transform data in network
• Is it processor-in-storage or active networking?
• All state is exposed