Date post: | 15-Dec-2015 |
Category: |
Documents |
Upload: | alvaro-sams |
View: | 214 times |
Download: | 0 times |
NBCR Science Gateway: Transparent Access to Remote NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop ClientsResources Through Rich Desktop Clients
Luca [email protected]
OutlineOutline
• NBCR/PRAGMA
• NBCR SOA infrastructure– Opal Overview – Opal Toolkit GUI– Opal Integration into Existing Tools– Access to Multiple Resources (TG, PRAGMA)
National Biomedical Computational ResourceNational Biomedical Computational Resource
• Funded by NIH (NCRR)
• Its mission is to conduct, catalyze, and enable biomedical research by harnessing, developing, and deploying forefront computational, information and grid technologies – Develop and deploy advanced
computational, data and visualization tools.– Integrate them in an advanced grid
framework.– Deliver and support this grid/Cyber-
infrastructure– Training (Summer Institute)
• Funded by NIH (NCRR)
• Its mission is to conduct, catalyze, and enable biomedical research by harnessing, developing, and deploying forefront computational, information and grid technologies – Develop and deploy advanced
computational, data and visualization tools.– Integrate them in an advanced grid
framework.– Deliver and support this grid/Cyber-
infrastructure– Training (Summer Institute)
National Biomedical Computational ResourceNational Biomedical Computational Resource
In the Pacific Rim and Globally 29 institutions
Strengthen Existing and Establish New Sustainable Collaborations
Work with Science Teams to Advance Grid Technologies and
Improve the Underlying Infrastructure
Pacific Rim Application and Grid Middleware Assembly (PRAGMA)
http://www.pragma-grid.net
A Practical Collaborative Framework
•Collaborations•Integration of existing applications•Deployment of a Grid infrastructure
MotivationMotivation
• Integration of different tools (client side application, workflow tools, web portals, etc.)– Several user interfaces
• Seamless access to computational resources– Easy from the scientific user perspective
Condor pool SGE Cluster PBS Cluster
Globus Globus Globus
Application Services Security Services (GAMA)
StateMgmt
Web server PMV/Vision Kepler
Big PictureBig Picture
Opal ToolkitOpal Toolkit
• Opal allows application developers to publish command-line applications using Web services– Minimal deployment effort: no coding, only a simple
configuration file – plain web service, SOAP– Common interface: every application uses the same
WSDL (same stubs on the client side)– Simple APIs (Launch job, Query job status, Get job
outputs, Get output as Base64, Destroy job, etc.)
Opal ToolkitOpal Toolkit
– Every application exposed with its own URL http://ws.nbcr.net/opal/services/Pdb2pqrService
– It takes care of data staging– It supports submission via:
• Fork• Globus GRAM• DRMAA• CSF4 (experimental)
– Run all the applications under a single user account– Used in NBCR, CAMERA, GLEON, among other
Opal Toolkit / Client SideOpal Toolkit / Client Side
• Several clients APIs available: Java, Python, PERL.
• Command line generic client:
# java edu.sdsc.nbcr.opal.GenericServiceClient -l
http://localhost:8080/axis/services/PDB2PQRServicePort -r launchJob -a "-ipdb sample.pdb -h -opdb output.pdb“ -f etc/sample.pdb
• Too complex for beginner users • Graphical User Interfaces
getAppMetadata
launchJob
getOutput…
PDB2PQR
getAppMetadata
launchJob
getOutput…
MEME
…Opal server
User
Vision
Kepler
User clients
Browser
Opal GUIOpal GUI
• Main characteristics:– Working out of the box (no configuration)– Multiplatform -> Web interface– Shipped with Opal
• Key features:– Access Statistics – List of services– Simple submission form– Advanced submission form
Access StatisticsAccess Statistics
Go to http://ws.nbcr.net
NBCR production server
List of ServicesList of Services
User has to input command line!
Too error prone.
Submission form should be customized on command line arguments
Simple Submission FormSimple Submission Form
Advanced Submission FormAdvanced Submission Form
• When deploying a scientific application there is an optional part to describe input and output of the application
• It is a command line syntax description language
Group 1
Group 2
Group 3
Group 4
Exclusive group
Input file
String
Exclusive enumeration
Flag
PDB2PQR Advanced Submission FormPDB2PQR Advanced Submission Form
Vision Opal NodeVision Opal Node
Kepler Opal Web Services ActorKepler Opal Web Services Actor
Autodock ToolsAutodock Tools
PBD2PQR Web SitePBD2PQR Web Site
Access to Multiple ResourcesAccess to Multiple Resources
CSF4CSF4
• Community Scheduler Framework:
– Open Source project and can be accessed at http://sourceforge.net/projects/gcsf
– Developed by Lab. of Distributed Computing and System Architecture, Jilin University, China
– It is a metascheduler framework hosted as an Execution Component in GT4 container
– It uses WSRF compliant services– It can submit jobs to Globus
CSF4 Meta-Scheduler
Grid Site GT2
LSF
Grid Site GT2
PBS
Grid Site GT4
SGE
Grid Site GT2
Condor……
•GT2 (gatekeeper)
•GT4 (WS-GRAM)
CSF4 Typical DeploymentCSF4 Typical Deployment
CSF4 New FeatureCSF4 New Feature
Users want to run applications
• Application based scheduling:
– CSF4 keeps a table of available applications
Application name Resources
namd tg1:/usr/local/namd
namd tg2:/usr/share/bin/namd
autodock pragma1:/usr/local/autodock
autodock pragma2:/usr/local/autodock
CSF4 New FeatureCSF4 New Feature
Users want to run applications
• Application based scheduling:
– CSF4 keeps a table of available applications
Application name Resources
namd tg1:/usr/local/namd
namd tg2:/usr/share/bin/namd
autodock pragma1:/usr/local/autodock
autodock pragma2:/usr/local/autodock
Avian Flu Grid
Opal CSF4 IntegrationOpal CSF4 Integration
• Opal plug-in for submission to CSF4
• Deployment architecture:
Opal Enable Clients
App A App B App C
Opal Server
App A App B App C
CSF4
ClusterAappAappB
Globus GRAM
…
Globus GRAM
ClusterBappAappC
WS/SOAP
WSRF
(Pre/WS)-GRAM
Future DirectionsFuture Directions
• Opal 2.0 will be released soon – Improved architecture (plug-in model)– URL support for input and output files– Embedded database (Hibernate)
• Support for other metascheduler (Gridway)• Opal GUI for Gridsphere (Portlet)
AcknowledgmentsAcknowledgments
• Wilfred Li, Peter Arzberg (NBCR vision)• Sriram Krishnan (Opal)• Xiaohui Wei (CFS4)• Many other NBCR developers...
• More information, downloads, documentation:– http://nbcr.net – http://nbcr.net/software/opal/