Date post: | 18-Jan-2018 |
Category: |
Documents |
Upload: | adele-park |
View: | 226 times |
Download: | 0 times |
Douglas Thain, John BentAndrea Arpaci-Dusseau, Remzi Arpaci-Dusseau, Miron Livny
Computer Sciences Department, UW-Madison
Gathering at the Well:
Creating Communities for Grid I/O
www.cs.wisc.edu/condor
4 earth-shattering revelations
1) The grid is big.2) Scientific data-sets are large.3) Idle resources are available.4) Locality is good.
www.cs.wisc.edu/condor
How to optimize job placement on the grid?› Move data to the jobs.› Move jobs to the data.› Allow jobs to access data remotely.
› Need framework for evaluation. I/O communities
www.cs.wisc.edu/condor
I/O Communities
UW INFN
www.cs.wisc.edu/condor
I/O communities are an old idea
› File servers and administrative domains
› But, we want more flexible boundaries simple mechanism by which users can
express I/O community relationships
www.cs.wisc.edu/condor
I/O communities› Mechanism which allow either
jobs to move to data, or data to move to jobs, or data to be accessed remotely
› Framework to evaluate these policies
www.cs.wisc.edu/condor
Grocers, butchers, cops› Members of an I/O community
Storage appliances Interposition agents Scheduling systems Discovery systems Match-makers Collection of CPU’s
www.cs.wisc.edu/condor
Storage appliances› Should run without special privilege
Flexible and easily deployable Acceptable to nervous sys admins
› Should allow multiple access modes Low latency local accesses High bandwidth remote puts and gets
www.cs.wisc.edu/condor
Interposition agents› Thin software layer interposed
between application and OS› Allow applications to transparently
interact with storage appliances› Unmodified programs can run in
grid environment
www.cs.wisc.edu/condor
Scheduling systems and discovery
› Top level scheduler needs ability to discover diverse resources
› CPU discovery Where can a job run?
› Device discovery Where is my local storage appliance?
› Replica discovery Where can I find my data?
www.cs.wisc.edu/condor
Match-making› Match-making is the glue which
brings discovery systems together› Allows participants to indirectly
identify each other i.e. can locate resources without
explicitly naming them
www.cs.wisc.edu/condor
Mechanisms not policies
› I/O communities are a mechanism not a policy
› A higher layer is expected to choose application appropriate policies
› We will however demonstrate the strength of the mechanism by defining appropriate policies for one particular application
www.cs.wisc.edu/condor
Experimental results› Implementation› Environment › Application› Measurements› Evaluation
www.cs.wisc.edu/condor
Implementation› NeST
storage appliance› Pluggable File System (PFS)
interposition agent built with Bypass› Condor and ClassAds
scheduling system discovery system match-maker
www.cs.wisc.edu/condor
Two I/O communities› INFN Condor pool
236 machines, about 30 available at any one time
Wide range of machines and networks spread across Italy
Storage appliance in Bologna• 750 MIPS, 378 MB RAM
www.cs.wisc.edu/condor
Two I/O communities› UW Condor pool
911 machines, 100 dedicated for us Each is 600 MIPS, 512 MB RAM Networked on 100 Mb/s switch One was used as a storage appliance
www.cs.wisc.edu/condor
CMS simulator sample run
› Purposefully choose a run with high I/O to CPU ratio
› Accesses about 20 MB of data from a 300 MB database
› Writes about 1 MB of output› ~160 seconds execution time
on a 600 MIPS machine with local disk
www.cs.wisc.edu/condor
Assume the position› We assumed the role of an Italian
scientist› Database stored in Bologna› Need to run 300 instances of simulator
› How to take advantage of UW pool? Three way matching
www.cs.wisc.edu/condor
Three way matching
Machine NeSTJob
JobAd
MachineAd
StorageAdm
atch
Refers toNearestStorage.
Knows whereNearestStorage is.
www.cs.wisc.edu/condor
Two way ClassAdsType = “job”TargetType = “machine”Cmd = “sim.exe”Owner = “thain”Requirements = (OpSys==“linux”)
Job ClassAd
Type = “machine”TargetType = “job”OpSys = “linux”Requirements = (Owner==“thain”)
Machine ClassAd
www.cs.wisc.edu/condor
Three way ClassAdsType = “job”TargetType = “machine”Cmd = “sim.exe”Owner = “thain”Requirements = (OpSys==“linux”) && NearestStorage.HasCMSData
Job ClassAd
Type = “machine”TargetType = “job”OpSys = “linux”Requirements = (Owner==“thain”)NearestStorage = ( Name = “turkey”) && (Type==“Storage”)
Machine ClassAd
Type = “storage”Name = “turkey.cs.wisc.edu”HasCMSData = trueCMSDataPath = /cmsdata”
Storage ClassAd
www.cs.wisc.edu/condor
Policy specification› Run anywhere where data is available
Requirements = (NearestStorage.HasCMSData)
› Run local only Requirements = (NearestStorage.Name == “nestore.bologna”)
› Run local first Requirements = (NearestStorage.HasCMSData) Rank = (NearestStorage.Name == “nestore.bologna” ) ? 10 : 0
› Arbitrarily complex Requirements = ( NearestStorage.Name == “nestore.bologna”)
|| ( ClockHour < 7 ) || ( ClockHour > 18 )
www.cs.wisc.edu/condor
Policies evaluated› INFN local› UW remote› UW stage first› UW local (pre-staged)› INFN local, UW remote› INFN local, UW stage› INFN local, UW local
www.cs.wisc.edu/condor
Completion Time
www.cs.wisc.edu/condor
CPU Efficiency
www.cs.wisc.edu/condor
Conclusions› Locality is good› I/O communities are a natural
structure to expose this locality› Users can use I/O communities to
easily express different job placement policies
www.cs.wisc.edu/condor
Future work› Automation
Configuration of communities Dynamically adjust size as load
dictates› Automation
Selection of movement policy› Automation
www.cs.wisc.edu/condor
For more info› Condor
http://www.cs.wisc.edu/condor› ClassAds
http://www.cs.wisc.edu/condor/classad› PFS
http://www.cs.wisc.edu/condor/pfs› NeST
http://www.nestproject.org
www.cs.wisc.edu/condor
Local only
www.cs.wisc.edu/condor
Remote only
www.cs.wisc.edu/condor
Both local and remote
www.cs.wisc.edu/condor
Grid applications have demanding I/O needs
› Petabytes of data in tape repositories› Scheduling systems have demonstrated
that there are idle CPUs› Some systems
move jobs to data move data to jobs allow job remote access to data
› No one approach is always “best”
www.cs.wisc.edu/condor
Easy come, easy go› In a computation grid, resources
are very dynamic› Programs need rich methods for
finding and claiming resources CPU discovery Device discovery Replica discovery
www.cs.wisc.edu/condor
Bringing it all together
CPU Discovery System
Replica Discovery System
Device Discovery System
Job Agent
Execution siteStorage appliance
Distributed Repository
Short-haul I/O
Long-haul I/O
www.cs.wisc.edu/condor
Conclusions› Locality is good› Balance point between staging data
and accessing it remotely is not static depends on specific attributes of the job
•data size, expected degree of re-reference, etc depends on performance metric
•CPU efficiency or job completion time
www.cs.wisc.edu/condor