Microsoft PowerPoint - GARUDA and experience on interoperability-finalEU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Subrata Chattopadhyay
subratac@cdac.in
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Outline
• Q&A
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Project Overview • Precursor to the National Grid Computing Initiative
– Test Bed for grid technology/concepts and applications leading to the plan for the main grid initiative
• Project initiated by the Dept. of IT in November 2004
• Major Deliverables
– Nation-wide high-speed communication fabric
– Aggregation of Grid Resources
– Grid Strategic User Group
• High-speed Networking Component in collaboration with ERNET
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
• Bioinformatics
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
AccessAccess
Gateway
Gateway
AccessAccess
GatewayGateway
GatewayGateway
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Cyber Infrastructure – Network Fabric
• The Proof of Concept network has been established in collaboration with ERNET
• The MPLS Virtual Private Network (VPN) connects 22 institutions at 100 Mbps and 23 institutions at 10 Mbps across 17 Indian cities with SLA agreements
• Collaborative environment enabled through Video Conferencing over IP at the following centres of C-DAC : Bangalore, Pune, Chennai, Hyderabad, Mumbai and Trivandrum
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Cyber Infrastructure – Resources
• In addition to PARAM Padma, Linux Clusters have been setup at Pune, Hyderabad & Chennai
• Grid Labs have been setup at Bangalore, Pune & Hyderabad
• Fourteen of the partner institutions are contributing resources including satellite terminals (compute aggregating to more than 300 CPUs)
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Bangalore :
• IISc & RRI, Bangalore • IMSc & MIT, Chennai • IIT, Guwahati • IIT, Kharagpur
• AIX SMP from University of Hyderabad
• SAC, Ahmedabad is contributing GSAT3 terminals (GSAT4 when launched) and Satellite bandwidth
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
End Users access the grid through the Garuda Access Portal
High Availability
High Availability
Bangalore Users
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
– To Contribute to the aggregation of resources in the Grid
• Current Participation
– ERNET-HQ in Delhi
– 8 centres of C-DAC
– Total of 45 institutions
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
– Enable applications of national importance requiring aggregation of geographically distributed resources
• Deliverables
– Grid enablement of illustrative applications and some demonstrations such as
• Bioinformatics
• Disaster Management
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
• Supports highly optimized Bioinformatics codes on the PARAM Padma
• Web computing portal providing all computational facility to solve related problems
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Flight data Flight data Flight data Flight data
transmission from transmission from transmission from transmission from
nearby Airportnearby Airportnearby Airportnearby Airport
Grid Partner
UserUserUserUser
AgenciesAgenciesAgenciesAgencies
GRID
Communication
Fabric
CommunicationCommunicationCommunicationCommunication
UserUserUserUser
AgenciesAgenciesAgenciesAgencies
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
• Options for interoperability
• Study and experience
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Service
Discovering
Accounting
Auditing
Job
Provenance
Package
Manager
CLI
Network
Monitoring
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Job Submission Flow for EGEE Grid User
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Job Submission Flow for GARUDA Grid User
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
MOAB
GLOBUS
SRB
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Our Planned Activity to support EGEE and GARUDA Interoperability.
Installation and Study of EGEE Infrastructure.
Installing EGEE components to create : – Computing Element
– Worker Node
– User Interface
– Monitoring Node
• Install SL-4.X for CE,WN,UI and SL-3.X for mon.
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Current Status :
• We achieved the successful deployment of all mentioned component at C- DAC,Bangalore,India.
• Thanks to EGEE support for testing and deploying individual EGEE components.
• All components/services are tested internally and externally from User's and Administrator's perspective.
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Experimental Setup – EGEE to GARUDA
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Details
• Jobs are submitted at the LCG - UI using the ‘glite-wms-submit’
• The “-r” option is used to specify the CE to the WMS
• A Simple “Hello World” Job is run by submitting the JDL at the UI
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Results
• Jobs were successfully submitted to the LCG - CE and then transferred to the remote queue of GARUDA head node
• Jobs executed successfully on the GARUDA resources by scheduling through the local PBS
• Problems persist with Data Staging ( both I/P and O/P )
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Issues to be studied
• Security vulnerabilities from password less RSH connection b/w LCG-CE and GARUDA head node.
• Mechanisms to achieve Data Staging – I/P and O/P staging
• Mechanisms to publish and track the Job Status information
• Mechanisms to publish the Garuda Resource information in the GLite Information Services
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Jobs from GARUDA to EGEE
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Issues to be studied • How to interact with the WMS / LB and Information systems for publishing the job information ?
• How does GARUDA Meta-scheduler negotiate on the EGEE resources – Should it interact with WMS/LB or other component?
• How to account for the jobs submitted through remote queues of PBS as they do not come through WMS ?
• Data Staging from GARUDA - specifically from SRB .
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Summary
• Garuda grid POC phase completed
• Interoperability – key issues planned and experimented with EuIndia grid in next phase
• Initial results promising – Job adapter harder and may take longer time
• Challenges – Job Monitoring , Data Staging, Security and related Password less rsh connection
• Collaborative efforts needed
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu