18
MyProxy, GridFTP, GRAM & so much more XSEDE Gateways Tutorial Matt McKenzie, PhD
MyProxy, GridFTP, GRAM & so much more
XSEDE Gateways Tutorial Matt McKenzie, PhD
https://www.xsede.org/gateways-‐overview
¡ What type of Gateway? § Who is your audience? ▪ Desktop application ▪ Large portal/website interface
¡ Structured staging plans § Short & long term goals § Precise list of requirements for your gateway § Choose technologies based on resources and time
¡ Bring your development team with you ¡ Make friends at your resource site(s)
¡ For scalability Community Accounts (CA) are used instead of unique XSEDE accounts § A single community user account is created ▪ Gateway uses the CA to launch jobs ▪ SAML attribute (citizenship info) is sent along CA request
§ Gateway user of the CA typically has privileges to only run a limited set of commands ▪ Community Shell – commsh cat commsh.mmcken6.conf
# Generic commands DirectAccess cc ** DirectAccess aprun ** DirectAccess /bin/cat *
¡ A service for securing private keys § Keys stored encrypted with password § Keys never leave the MyProxy server
¡ A service for retrieving proxy credentials § Delegate these credentials to access other resources
¡ A commonly used service for gateway/portal security
¡ myproxy-‐logon § Retrieve stored credential § Requires passphrase that was used when it was stored § First command you should always execute
¡ myproxy-‐info § Get stored credential information
¡ myproxy-‐init § Generate a proxy from credentials on local machine & store it
on the proxy server ¡ myproxy-‐destroy
§ Remove credential
¡ Useful flags ¡ -‐s server_name ¡ -‐l user_name
¡ GridFTP: Globus provides this service § Implements FTP protocol using GSI authentication
¡ globus-‐url-‐copy § Client tool that performs file management tasks
¡ globus-‐url-‐copy Source_URL Destination_URL § URL looks like: § gsiftp://<hostname:port>/path ▪ globus-‐url-‐copy gsiftp://gridftp.kraken.nics.xsede.org/PATH/TO/FILE gsiftp://gridftp.blacklight.psc.teragrid.org/PATH/TO/FILE
§ The file:// protocol is deprecated, please do not use
http://www.loni.org
LONI HPC Enablement Workshop – LaTech University, October 23, 2008
!"#$%&'(
Basic Transfer One control channel, several
parallel data channels
Third-party
Transfer Control channels to each server, several parallel
data channels between servers Striped Transfer
Control channels to each server on one node, several parallel data channels
between servers and data channels spread across nodes
myproxy.teragrid.org
Myproxy-‐logon Globus-‐job-‐run My machine
Gatekeeper LRM
jobmanager jobmanager
jobmanager
Condor pool
PBS: Compute nodes
Fork: Local host
1 2 3
globusrun where_to/who_to_talk_to what_to_do globusrun url_to_resource/jobmanager-‐X RSL file
¡ Globusrun – Execute & manage jobs via GRAM ¡ Useful options
§ -‐p ▪ Verify syntax of the RSL specification
§ -‐a ▪ Authenticate only
§ -‐y or –refresh-‐proxy ▪ Attempt to delegate a new X.509 proxy to the job manager managing a
jobID § -‐status § -‐f RSL_FILENAME § -‐r Resource Contact ▪ Host:Port/Service
§ -‐b ▪ Submit job and exit GRAM service
¡ XML language ¡ Attribute & value pairings ¡ GRAM attributes: executable, arguments, count, directory, maxtime, jobtype, project,… § A different way to describe a PBS script
¡ LRM interprets the RSL attributes to manage the GRAM request
& (project=TG-‐STA110014S) (jobtype=mpi) (directory=/lustre/scratch/mmcken6/apoa1/) (count=24) (executable=/lustre/scratch/mmcken6/namd2) (arguments=apoa1.namd)
#!/bin/bash #PBS –l size=24 #PBS –A TG-‐STA110014S cd /lustre/scratch/mmcken6/apoa1/ aprun –n 24 /lustre/scratch/mmcken6/namd2 apoa1.namd
:~>globusrun -‐o -‐r grid.nics.utk.edu:2119/jobmanager-‐pbs '&(directory=/lustre/scratch/$USER)(executable=/lustre/scratch/$USER/helloworld)(jobtype=mpi)(count=12)(project=my_allocation)'
Gatekeeper
jobmanager jobmanager
jobmanager
Condor pool
PBS: Compute nodes
Fork: Local host
¡ jobmanager-‐pbs '&(directory=/lustre/scratch/user)(executable=/lustre/scratch/$USER/helloworld)(jobtype=mpi)(count=12)(project=my_allocation)'
¡ Jobmanager-‐pbs autogenerates a PBS script ¡ Jobtype=mpi
§ Places “aprun –n” or “mpirun –np” § Site/machine dependent
¡ Jobtype=single § This is solitary, non-‐parallel job § Most flexible option ▪ One can set aprun/mpirun/other exe and its various arguments ▪ '&(executable=aprun)(arguments="-‐n" "24" "helloworld") (directory=/lustre/scratch/user)(jobtype=single)(count=24)(maxtime=10) (project=my_allocation)'
¡ Jobtype=mpi § Generates a PBS for MPI jobs
¡ Jobtype=multiple § Parallel applications that do not depend on MPI § Check your site friend on how this is implemented ▪ @ NICS = many under 1 submission ▪ @ SDSC = many submissions
~> more sub_namd.rsl & (project=TG-‐STA110014S) (jobtype=mpi) (directory=/lustre/scratch/mmcken6/apoa1/) (count=24) (executable=/sw/xt/namd/2.8/cnl3.1_gnu4.6.1/bin/namd2) (arguments=apoa1.namd) ~> globusrun -‐o -‐r grid.nics.utk.edu:2119/jobmanager-‐pbs -‐f sub_namd.rsl GRAM Job failed because the authorization system denied the request – not authorized to run the specified executable (error code 165)
Solution Email [email protected] or talk to your XSEDE site friend Could you please add “/sw/xt/namd/2.8/cnl3.1_gnu4.6.1/bin/namd2” to my commsh?
~>globusrun -‐o -‐r grid.nics.utk.edu:2119/jobmanager-‐pbs '&(directory=/lustre/scratch/$USER) (executable=/lustre/scratch/$USER/helloworld)(jobtype=mpi)(count=12)(project=my_allocation)' Hello world from process 8 of 12 Hello world from process 10 of 12 Hello world from process 7 of 12 Hello world from process 3 of 12 Hello world from process 11 of 12 Hello world from process 4 of 12 Hello world from process 5 of 12 Hello world from process 1 of 12 Hello world from process 0 of 12 Hello world from process 9 of 12 Hello world from process 2 of 12 Hello world from process 6 of 12 Application 5579880 resources: utime 0, stime 0
¡ Good overview of the parts of GRAM5 § http://www.globus.org/toolkit/docs/5.0/5.0.0/execution/gram5/user/
¡ Good manual for RSL files § http://www.globus.org/toolkit/docs/2.4/gram/rsl_spec1.html/#Simple%20RSL%20Examples
¡ Many, many examples and error codes § http://www.nics.tennessee.edu/computing-‐resources/grid_services
Seen ¡ Scientific workflows on XSEDE ¡ ParamChem Gateway ¡ Xbaya, graphical workflow composer ¡ Airavata overview & tutorial
Coming ¡ Airavata software toolkit
§ Does more than build scientific workflows
