Using Grid Computingat NIKHEF
David Groep, NIKHEF2001-10-01
The One-Liner
• Resource sharing and coordinated problem solving in dynamic multi-institutional virtual organisations
What is Grid computing?
• Dependable, consistent and pervasive access• Combining resources from various organizations
• `Virtual Organizations’ – user-based view on Grid
• Technical challenges:– transparent decisions for the user– uniformity in access methods– secure & crack resistant– authentication, authorization, accounting (AAA) "a
• Globus Project started 1997• de facto-standard• Reference implementation of Gridforum standards
• Large community effort• Basis of several projects, including EU-DataGrid
• Toolkit `bag-of-services' approach
• Successful test beds, with single sign-on, etc…
Grid Middleware
Grid Architecture
Applications
Grid Services GRAM
Grid Security Infrastructure (GSI)
Grid FabricCondor MPI PBS Internet Linux
Application ToolkitsDUROC MPICH-G2Condor-G
GridFTPMDS
SUN
VLAM-G
Make all resources talk standard protocols
Promote interoperability of application toolkit, similar to interoperability of networks by Internet standards
ReplicaSrv
Looking for Resources
• Per Virtual Organization (or test bed)
• Directory of Resources and their Characteristics
• Used to find `best resource out there’
DataGrid http://marianne.in2p3.fr/DutchGrid ldap://giishost.nikhef.nl:30001/o=Grid
Submitting a Job
Sending your Data
• Tape robots, disks, etc. share GridFTP interface• Optimize for high-speed >1Gbit/s networks
• In the future: automatic optimizations, bandwidth reservations, directory-enabled networking, …
DataGrid Test Bed 1
• DataGrid TB1:– 14 countries– 21 major sites
– “Work Package 6”
• Submitting Jobs:– Login only once,
run everywhere– Cross administrative
boundaries in asecure and trusted way
– Mutual authorization
DutchGrid Platform
Amsterdam
UtrechtKNMI
Delft
Leiden
Nijmegen
Enschede
• DutchGrid:– Test bed coordination– PKI security
• Participation byNIKHEF:
FOM, VU, UvA, Utrecht, Nijmegen
KNMI, SARAAMOLFDAS-II (ASCI):
TUDelft, Leiden, VU, UvA, Utrecht
Telematics Institute
Systems around WCW
Start using the grid
• All the necessary “client tools” are on all Linux and Solaris systems
• You just need:– Credentials/tokens for the Grid (see next slides)– Authorization to use resources
(you get all NIKHEF resources by default)– Information on which resources to use effectively
Your Grid Credentials
• You will use resources across several domains– You may not care about security and authorization– But the remote site admin will !
• All communications are authenticated usingX.509 “Public Key” Certificates
• The technology used to securecredit card transactions on the web (https://……)
• Uniquely binds name/affiliation to a digital token
Certification Authorities
• CA’s act as trusted third parties
• Remote sites trust the CA for a proper binding• They will not do authentication again, so
only authorization left.
• CA’s are highly valuable: crack one to impersonate others on the Grid
(and abuse resources)
• Registration Authorities do in-person ID checks
CA’s in DataGrid
• 10 National CA’s (one per EU country)• Each one has a detailed
policy and practice statement
• NIKHEF operates the CA for DutchGridSee http://certificate.nikhef.nl/
• Get a “certificate” from the DutchGrid CAbefore you can start using the Grid
• It’s valuable, protect it with a pass phrase• One cert valid for all DataGrid sites
The Proxy
• A `proxy certificate’ is a limited-lifetime delegationwithout a pass phrase to protect it
• Implements the single sign-on for Grid• Valid for 12 hours (by default)
• Use it to:– Run your jobs– Get access to your data
• Get it, by running grid-proxy-init
Now see for yourself
Getting a Certificate
• Initialize your environment for the Grid• Use the Globus local guide from
http://www.dutchgrid.nl/Org/Nikhef/• Run grid-cert-request• Send the result to [email protected]
you will be contacted by phone• Put the certificate (sent by mail) in your$HOME/.globus/usercert.pem
• Or use the Web at http://certificate.nikhef.nl/cgi-bin/reqconf
Using the Grid
• Request authorization: [email protected]• Look what is out there using grid-info-search or
http://marianne.in2p3.fr/datagrid/giis/giis-browse.html
• Try some local hosts:– bilbo, kilogram, triangel
kilogram:davidg:1009$ globus-job-run dommel.wins.uva.nl /usr/ucb/quota -vDisk quotas for random (uid 12xxx):Filesystem usage quota limit timeleft files quota limit timeleft/home/random 13067 1500000 2000000 0 0 0kilogram:davidg:1010$
• Start running your analysis/MC/other jobs
GridFTP
• Universal high-performance file transfer• Extends the FTP protocol with:
– Single sign-on (GSI, GSSAPI, RFC2228)– Parallel streams for speed-up– Striped access (ftp from multiple sites to be faster)
• Clients: gsincftp, globus-url-copy.
What’s Next?
• Some of the nice user-features to come:
– Finding data files by characteristics(give me all golden decay’s)
– Moving your job to where the data is– Automatic partitioning of jobs– Support true-interactive work– Better network utilisation (faster access to data)– ………
• If you are in the DataGrid project, ask your WP leader for authorization in TB1