Post on 13-Jan-2017
transcript
Managing HPC Clusters with Dasein
Udit AgarwalDept. of Computer ScienceClemson University
Amy Apon, Ph.D.Chair, Dept. of Computer ScienceClemson University
J. Craig Lowery, Ph.D.Sr. Distinguished EngineerDell Software Group
Linh B. Ngo, Ph.D.Research Associate,Dept. of Computer ScienceClemson University
2 Dell Unrestricted Software
Research Engagement• Deliverables
– Mapping of the Dasein model to Palmetto interfaces– Assessment of potential completeness and utility– Sample code implementing key functions– Final report
3 Dell Unrestricted Software
Functional Mapping Analysis
ComputeServices
getAutoScalingSupport()
getImageSupport()
getSnapshotSupport()
……………
listModules()
runJob()
jobStatus()
deleteJob()
PalmettoDasein
module avail
qsub filename.pbs
qstat –anu username
qdel jobid
Provides the list of software packages
Submitting jobs to run on the cluster
To check the status of your job
Killing a job submitted
Impl
emen
ts
Impl
emen
ts
Implements
Class NameMethod Name
New methods added
4 Dell Unrestricted Software
Solution ArchitectureDell Cloud Manager
Dasein
Amazon Web Service
JoyentPBS
Scheduler
Dasein API
AWS API
Joyent API
Common data in Dasein format
Cloud status in AWS data format
Cloud status in Joyent data format
Palmetto super-
computerCluster status in Palmetto data format
Linux Commands, Shell Scripts and Submission Scripts
Result of executed Commands
Code modified for this project
New management domain enabled.
PBS Scheduler API
5 Dell Unrestricted Software
Summary• Dasein (and, therefore, Dell Cloud Manager and similar
software) can meaningfully manage HPC clusters, but requires object model extensions.
• A mature solution could be used by faculty, staff, students and researchers to manage HPC jobs and “regular” cloud resources through a single portal/tool.
• Could simplify the use of HPC clusters for non-technical researchers who lack knowledge/training of the scheduler, Linux shell, etc.