Garuda : the unique platform for Innovative research...

Garuda : the unique platform for

Innovative research collaboration

Subrata Chattopadhyay

CDAC , Bangalore

[email protected]

25-Oct-13

mailto:[email protected]

Outline

Grid & Garuda NGI

Recent achievements

Applications

Services & support

Conclusion

2

Grid Computing

GG-BLR GG-CHE GG-HYD TF BLR TF PUNE IITD YUVA PRL

Climate

Modeling

Disaster

Management

Bio

Informatics CFD

Crypt

analysis

Grid Middleware

3

Grid Computing

Grid allows seamless access to information

AND computing resources

Name derived from the “Power Grid”

You need not know or care where either

comes from

It means

Grid allows users to share resources in a secure,

seamless and transparent way.

Grid also facilitates collaborative environment over

the high speed communication Fabric.

The Super Computing power is accessible to the

end users without the need of huge investment on

Super Computers.

CPU cycles that would otherwise be wasted are put

to good use.

4

Sharing of resources among the community

Seen as a collective pool

Heterogeneous

Geographically distributed

Different Administrative domains

Wide variety of Tools, Interfaces to choose with.

Grid Computing

Components of Grid Middleware

6

Grid Middleware - Definition

Middleware is a layer between grid

applications and low level functionality

of grid.

Grid Middleware is a layer of S/W to

enforce:

Scalability, Transparency, Heterogeneity,

Fault Tolerance and Security of the Grid

below the application layers.

It provides an uniform interface to the

Grid users and handle all the complexity

generated due to heterogeneous systems.

7

8

Popular Middleware

Globus – Globus Alliance

GridBus – University of Melbourne

UNICORE - Uniform Interface to Computing Resource

gLite – CERN / EGEE /EGI

Legion – (Avaki - Corporate Distributor)

Alchemi – (.NET Grid Computing Framework)

Condor

SGE

70 + Partners 10000 CPUs –

550TF

EGI, CHAIN-

REDS

220TB Storage

1500 +

Certificates

NKN

25-Oct-13

• High Capacity, Highly

Scalable Backbone

• Provide Quality of Service

(QoS) and Security

• Wide Geographical Coverage

• Common Standard Platform

• Bandwidth from Many NLD’s

• Highly Reliable & Available by

Design

• Test beds ( for various

implementation)

• Dedicated and Owned.

25-Oct-13

How to join Garuda grid ?

11

Pre-requisites for Accessing

Garuda

Network Connectivity to Garuda

Resources

Valid Grid Certificate

MoU or LoE

12

Mr. A

1. CDAC

Bangalore

BD 08-06-82

Govt

Of

India

Certificates can be compared to identity cards

Name

Issuer

Validity

Signature

Valid Till: 01-02-2012

13

– Indian Grid Certification Authority located at C-DAC, Knowledge

Park, Bangalore, India.

– IGCA is the accredited member of APGridPMA.

– Issues X.509 Certificates to support the secure environment in

Grid. (for GARUDA, institutes that do research in grid from India

and foreign institutes that collaborates with GARUDA).

– http://ca.garudaindia.in

1500 +

• Certificates Issued

338

• Valid User Certificates

41

• Valid Host Certificates

45

• Registration Authorities

25-Oct-13

http://ca.garudaindia.in/

– Indian Grid Certification Authority located at C-DAC,

Knowledge Park, Bangalore, India.

– IGCA is the accredited member of APGridPMA.

– Issues X.509 Certificates to support the secure environment in

Grid. (for GARUDA, institutes that do research in grid from India

and foreign institutes that collaborates with GARUDA).

IGCA

15

GARUDA SLCS provides grid users an instant access to GARUDA grid for a trial period of 30days.

Highlights:

• Hassle free registration

• Get an access in less than 5mins.

• Service over the internet.

Features:

• GARUDA Job submission portal

• GARUDA Compiler Service

Website: http://labs.garudaindia.in 25-Oct-13

Public accessible CA root certificates

Certificate requests

Valid certificates

Revoked certificates

CRL

CP/CPS

User guides/FAQ’s

RA contact lists.

CA Contact info

http://ca.garudaindia.in

Web Repository

17

VOMS

VOMS –Virtual Organization Management/Membership Service

VOMS is a system to classify users that are part of a Virtual Organization (VO) on the:

–Set of attributes that will be granted to them upon request.

–include that information inside Globus-compatible proxy certificates

1. In addition to the standard X.509 based Grid certificates, the users should obtain

separate VO credentials, from the VO Server.

2. The VO credentials identify the membership of a Grid User to a specific VO.

3. The VO credentials also identify the ‘Role’ of the Grid User in the corresponding

VO.

VO Registration

http://voms.garudaindia.in/

19

santhosh119

VO Registration

20

Virtual User Community in Garuda

Group Name Description

Bioinformatics application of statistics and computer science to the molecular biology

ClimateModelling Deals with the dynamics of the climate system.

OSDD

Community dedicated to develop drugs for tropical infectious diseases like malaria, tuberculosis

GeoPhysis Study related to physics of the Earth and its environment in space

CAE

usage of computer software to solve engineering problems

IndianHeritage

Focused on technology products for preserving & processing Heritage texts

HealthInformatics

Focused on utilizing compute power for health informatics

MaterialScience interdisciplinary field applying the properties of matter to science and engineering

Euindia

The vision of a worldwide Grid for Research by both Europe and India

ToolsDeveloper

Forum to communicate and collaborate on developing Garuda Tools

21

VO Registration

22

Affiliating Mail from Garuda

23

Programming Development Environment

Computing Resources and Virtual Organizations

Research Organizations Educational institutions Computing Centers

WSRF+GT4 + other Services + Cloud S/W (Nimbus/ VMware)

NKN

Grid PSE

Virtualization support

Workflows

Grid Security and High-Performance Grid Networking

Data

Grid

Resou

rce E

nab

ler &

Mo

nito

rin

g

CDAC Resource centers

Access Portal CLI Visualization

Federated Information Server

Job Scheduler

Programming Environments Security

Resource

Management User Environments Middleware Data Grid Resources

Hand held devices

GARUDA – enabled Applications

Non – Research

Organizations

Cloud Interface

25-Oct-13

GSRM Garuda Information Registry

Paryavekshanam

25-Oct-13

Garuda Access Portal

Garuda GridFTP GUI

Globus Online

AGSG PSP

25-Oct-13

Megha Scilab

VRGeo Galaxy Workflow – OSDD

Garuda

Garuda User Forum 25-Oct-13

CDAC Resource :

• 4TF HPC clusters each at Bangalore, Chennai & Hyderabad

• PARAM Yuva II at Pune and PARAM Padma at Bangalore

Fourteen of the partner

institutions are also contributing resources including satellite terminals.

Total computing power is more than 10000 CPUs equivalent to 550TF

Storage space 220 TB

25-Oct-13

25-Oct-13

Serving applications of National Importance

‐ Alliance with the Open Source Drug Discovery (OSDD) project

of CSIR

‐ Disaster management applications

‐ Weather forecasting models & Earthquake engineering

‐ Applications from the fields of Bioinformatics, CAE & Material

sciences

First in India

‐ Setting up of Indian Grid Certification Authority (IGCA) in 2009,

to issue digital certificates for grid researchers in India

‐ Digital certificates trusted by other International Certification

authorities

‐ Issued more than 1500 IGCA certificates

Global Integration

‐ Integrated with the European Grid Infrastructure through the

EU-India Grid and CHAIN-REDS

‐ Achieved middleware interoperability between the European

Glite middleware & Garuda middleware components

25-Oct-13

Bioinformatics : Open Source Drug Discovery

OSDD HeadNode

Garuda Middleware

Stack, login service,

Gridway Metascheduler

OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid

Garuda Middleware Stack

LRM- Torque

OSDD Tools – weka, cdk,…

NKN

Internet / NKNOSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


NKN

Internet / NKN

OSDD User Community

Grid Enabled Bioinformatics tools useful in drug discovery pipeline

Internet / NKN

Internet / NKN

OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


NKN


Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


NKN

Internet / NKN

OSDD User Community

OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


NKN


Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


NKN

Internet / NKN

OSDD User Community

OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


NKN


Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

OSDD HeadNode

Garuda Middleware



OSDD Customized

Galaxy

DB

Ext DB

GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


GGHYD

Cluster

Yuva

Cluster

JNU

Cluster

Other

OSDD

Cluster

GARUDA Grid


LRM- Torque


NKN

Internet / NKN

OSDD User Community

Grid Enabled Bioinformatics tools useful in drug discovery pipeline

Internet / NKN

Internet / NKN

• OSDD users given access to Garuda through OSDD VO

• HPC clusters to run drug discovery problems

• Users connected through both NKN and Internet

• Galaxy Workflow for genomics proteomics applications

• Distributed job execution through Gridway

Project Team : OSSD community

25-Oct-13

Climate modeling on Garuda

Seasonal Forecast Model (SFM) implemented on

GARUDA to do ensemble forecasting

A framework developed to do ensemble

forecasting using the existing grid middleware

services

Several simulations were done

Use GSRM for data storage

Model resolution: 40 Km x 40 Km

Needs 27 GB of disk space and 80hrs of wall

clock time (If I use 64 processors – Intel Xeon,

3.16 GHZ, 16GB RAM, Infiniband connectivity)

for 1 season of 1 year with 1 ensemble member

Collaborative Class Room

Project Team : CDAC Hyderabad

Supported Features:- • Interface to Access

grid • GSRM based data

storage for maintaining course repositories

• Indexing of course material based on key words

Website: http://ccr.garudaindia.in

25-Oct-13

Bioinformatics: Taxogrid

Resources

gg-hyd.cdac.org.in

gg-che.local

gg-blr.tfg

xn00.npsf.cdac.ernet.in

yuva.pnq-cdac-hq-100.garuda

Software Versions

Globus – 4.0.7

GridWay – 5.4

DRMAA - 1

Dataset

Database: UniProtKB

Mycobacterium tuberculosis H37Rv

No. of sequences: 3988

Size of input dataset: 1.65 MB

Size of output file: 10.4

Method : Maximum Likelihood

TaxoGrid Benchmark (Time Scale)

0

200

400

600

800

1000

1200

1400

5 10 15 20

Number of cores

Tim

e (

in m

inu

tes)

TaxoGrid Benchmark (Speed-up)

1.00

1.76

2.72

3.31

0

0.5

1

1.5

2

2.5

3

3.5

0 5 10 15 20 25

No. of pipelines (160 cores)Sp

eed-

up

Time for speedup at value ‘1’ : 1144 minutes

Time for speedup at value ‘3.31’ : 346 minutes

Project Team : CDAC Pune

25-Oct-13

Aim: Optimize the noise generated by a 3-D wing with flaps in landing configuration by variation of flap location and orientation.

CAE: Aeroacoustics Optimization

Project Team : Zeus Numerix

• Uses Kepler workflow Framework integrated with native Globus job submission routines

• Optimization Module uses OPT4J framework

• Optimization module includes AFFG (Adaptive Fuzzy Fitness Granule) routine which can reduce the number of fitness function evaluations up to 50%.

• Completion sucessful 40 simultaneous simulations (parallel + serial )e Fuzzy Fitness Granule) routine which can reduce the number of fitness function evaluations up to 50%.

• Completion sucessful 40 simultaneous simulations (parallel + serial)

25-Oct-13

25-Oct-13

New Middleware Stack

New Middleware Stack (v4.0)

released QoS Framework

Megha Cloud interface

Reservation framework

Improved services for login, accounting & compilation

Updated components of Globus Toolkit & Gridway

37

Megha Cloud Interface

Cloud computing platform built on

top of the Garuda resources.

Convergence of Grid and Cloud

technologies

Following apps are available in

the SaaS model GROMACS, OpenFOAM, R, Octave, SFM, FASTA

38

Scilab Open source,

cross-platform

numerical

computational

package and a

high-level,

numerically

oriented

programming

language.

39

• In collaboration with IITB

• scilab.in accesses Megha for executing scilab code and

rendering graphics

• Many textbooks examples are solved and available as

part of text book companion project

http://scilab.in/

VRGeo

40

Open-Source

Collaborative Mapping

Platform for Crowd-

sourcing Geospatial

information

Garuda Challenge

Called upon participants to showcase compute-

data intensive scalable applications on the

GARUDA

Received nearly 20 submissions

Shortlisted candidates benchmarked their

results on production resources of GARUDA

Winner - Rs. 50,000/-

Runner up - Rs. 25,000/-

Results announced during PARCOMPTECH

conference – Feb 2013

41

PARCOMPTECH India 2014

Conference Dates: 27th & 28th February 2014

Venue: YASHADA, Baner Road, Pune

http://parcomptech.cdac.in/

42

25-Oct-13

44

The “Global” Grid…

CNGrid

NKN &

Garuda

EUAsiaGrid

SAGrid &

SANREN GISELA

Genesis II

and the “non-Global” middleware

courtesy : Roberto Barbera, INFN

http://www.egi.eu/

Conclusion

Garuda grid – stable and operational over

NKN

Many tools and services deployed to assist

the users

Huge potential to grow Garuda virtual

community

Unique opportunity for solving national

problems through collaborations 45

Services & Support

Grid Operations : 080-66116474, 6511

Email : [email protected]

Website : https://gridsupport.garudaindia.in

http://www.garudaindia.in

46

https://gridsupport.garudaindia.in/

http://www.garudaindia.in/

धन्यवाद!

ಧನ್ಯವಾದಗಳು!

ధన్యవాదాలు!

நன்றி!

നന്ദി!

Thank you!

ধন্যবাদ!

25-Oct-13

Date post:	07-Jul-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Garuda : the unique platform for Innovative research...

Documents