+ All Categories
Home > Documents > GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th...

GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th...

Date post: 03-Jan-2016
Category:
Upload: ashlyn-jacobs
View: 216 times
Download: 0 times
Share this document with a friend
Popular Tags:
30
Para ver esta película, disponer de QuickTime™ y un descompresor TIFF (LZ Para ver esta película, deb disponer de QuickTime™ y de un descompresor TIFF (LZW). gLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th ,2007.
Transcript
Page 1: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

gLite Information System(s)

Antonio Juan Rubio MonteroCIEMAT

10th EELA Tutorial.Madrid, May 7th-11th,2007.

Page 2: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

This presentation is based on LCG and gLite manuals and

previous EELA tutorials

Page 3: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Outline

1. Grid Information Systems Overview

2. gLite Information Systems3. GLUE Schema

1. Grid elements

4. LCG Information System1. Hierarchy2. LDAP model

5. R-GMA1. Overview2. Server Architecture3. Consumer-Producer model4. User interfaces

6. References

Page 4: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Grid Information Systems Overview

• Collect information of grid resources Discovering new added resources Monitoring load and health status

• Publish these information Periodically updated Well know data model

• Used by Users searching a concrete resource WMS allocating and managing jobs Other monitoring services

Page 5: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

gLite Information Systems

• Basic data model. Grid Laboratory Uniform Environment (GLUE) Schema.

• Two architectures in glite3 LCG Information System

BDII over Globus MDS. OpenLDAP interface. Preserves LCG-2 compatibility.

Relational Grid Monitoring Architecture (R-GMA) Based on the GMA standard from the Grid Global Forum Information in SQL relational databases Web Services. Future replacement of LCG IS.

Page 6: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

GLUE Schema

Page 7: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

GLUE Schema

• A schema of objects and attributes describing Grid resources and its relationships. Originally a EU-DataTAG and US-iVDGL coordinated

effort. Current participants: EGEE, OSG, Globus and NorduGrid. A way to describe Grid info

Statically and dynamically supplied Hierarchically represented Independently of the framework (LDAP, XML, SQL…)

• Actual release (1.2) is mapped into LDAP XML Old ClassAd (used by Condor Matchmaking)

Page 8: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Site

A collection of resources owned by the same organization and managed by the same administrator. Contains info on the location, the administrator, the web homepage and so on.

Service

The description of a deployed Web Service. Contains the URI endpoint of the WS, the WSDL document, the list owners and so on.

StorageElement Cluster

1 1 1

*

*

*

Site Element

Page 9: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Cluster

A set of heterogeneous resources. Contains information on shared temporary directories.

SubCluster

A set of similar resources. Contains the number of Logical and Physical CPUs.

1

*

Host

Contains detailed static information of the type of hosts and related installed software. Data deal with the type of CPU architecture, memory sizes, the operating system installed as well as the type of network adapter. Furthermore it contains some information on performance mesures obtained by executing well known benchmark softwares.

Location

Information on installed softwares, their path and version

1

*

ComputingElement

1

*

Cluster Element

Page 10: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

ComputingElement

Abstraction of a queue of jobs

Policy

Contains info on configuration policies. MaxWallClockTime, MaxRunningJobs, MaxCPUTime . . .

AccessControlPolicyBase

Set of rules defining access control policy rules

Info

Static information on the resource that deal with the type of Loca scheduler adopted, the default Storage Element and so on.

VOview

View for a given Virtual Organization. Contains authorization details for VO members and the amount of available resources.

State

Dynamic information on the status of this queue such as the number of free CPUs and the Estimated Traversal Time (ETT)

Job

Information on jobs in this queue, its owner, its local and global ID and its status

*

*

Computing Element

Page 11: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

LCG Information System

Page 12: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

LCG Information System Levels

• Resource level: Grid Resource Information Server (GRIS) One GRIS on top of each CE, SE, RB, MyProxy (no WNs). Sensors and scripts get status of concrete resources

statically (e.g. GlueCEUniqueID) or dynamically (e.g. GlueCEStateWaitingJobs)

• Site level: Grid Index Information Server (GIIS) Compiles all the information of the different GRISes in a

site. LCG-2 recommends using a BDII instead of a GIIS

Improves robustness and stability. Called the site BDII.

Page 13: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

LCG Information System Levels

• Top level: Berkeley DB Information Index (BDII) Keeps all Grid information about the VOs (generally only

one). Stores information from local BDIIs or GIISes in its

database. Only queries sites that are included in a configuration

file.

Page 14: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

LCG Information System Hierarchy

Page 15: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

A LDAP Model

• Way of collecting info Pull model (higher level servers periodically query

lower level servers) All servers are based on LDAP

Inherit hierarchical structure (tree-like) LDAP Data Information Format (LDIF)

• Users get info with Generic applications

ldapsearch (MDS: 2135 and BDII:2170 ports) Graphical (BDII web; LDAP GUIs) Always can get information about specific resources

(maybe more up-to-date) by querying directly the site BDIIs, GIISes or GRISes.

Querying VO info with lcg-infosites or lcg-info

Page 16: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

LDAP Browser showing BDII Info

Page 17: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Relational-GMA

Page 18: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

R-GMA Overview

• Added from EDG Project• Based on the GMA standard from the GGF• Information in SQL relational databases (a DB

per VO)• Query syntax is a SQL subset• Simple consumer-producer model• Web Services oriented• CLI and Web user interface• Allows self-logging applications• Future replacement of LCG IS

Page 19: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

RGMA Overview

• R-GMA offers a global view of the VO information In one large relational DB: virtual database. Registry stores localization tuples (database rows)

published by producers: Standard Tables: CE state in GLUE Schema (by R-GMA-

GIN) Applications specific tables (e.g. self-logging with Log4j) Access by SQL queries through a WS interface.

Always updated Consumers get information directly from the

resource Can be cached by a secondary producer from

single o multiple producers to improve access.

Page 20: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

R-GMA Server Architecture

Page 21: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

R-GMA Server Description

• R-GMA Servers Server (one or several) Schema Server (There is one common global

schema server for the grid) Registry Server (one or several) Browser (shows grid information via web browser)

Page 22: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

R-GMA Server Description

• R-GMA Clients Generic Client (generic set of APIs for different

languages and CLIs) Servicetool (used by all services to publish its

existence and status) Site Publisher

Client to publish the existence of a site. Each site must have one R-GMA site publisher.

GadgetIN (GIN) Client to extract information from MDS and to republish

it to R-GMA. used by the CE.

Data archiver (flexible archiver) Client to make the data that is coming from the R-GMA

site-publisher, servicetool and GIN constantly available. By default the GLUE tables and service tables are

archived, however this can be configured.

Page 23: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

R-GMA Servers Per Node Type

The gLite R-GMA Server should deployed the first as part of a gLite grid, because all services require it to publish service information.

Deployment module Contains Used / included by

R-GMA Server R-GMA serverR-GMA registry serverR-GMA schema serverR-GMA browserR-GMA site publisherR-GMA archiverR-GMA servicetool

MON-box

R-GMA client RGMA client APIs UI, WN, SD

R-GMA servicetool R-GMA servicetool R-GMA server,VOMS, LB, WMS, CE, DC, FTS, FTA, Hydra, I/O-Server,Torque Server

R-GMA GIN R-GMA GadgetIN CE

Page 24: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

• The Producer stores its location (URL) in the Registry.

• The Consumer looks up producer URLs in the Registry.

• The Consumer contacts the Producer to get all the data.

• Data passes directly from the producer to the consumer: (does not through the registry)

• The user or any user application does not need to know the registry; this will be handled directly by the consumers and producers.

Registry

Producer Consumer

Store

Loc

atio

n

Look up Location

Execute or Stream data

R-GMA Producer- Consumer model

Page 25: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Consumer

Producer 1

Registry

TableName

Value 1 Value2

Value 3 Value 4

TableName

Value 1 Value 2

TableName URL 1

TableName URL 2

• The Consumer will get all the URLs that could satisfy the query.

• The Consumer will connect to all the Producers.

• Producers that can satisfy the query will send the tuples to the Consumer.

• The Consumer will merge these tuples to form one result set.

Producer 2TableName

Value 3 Value 4

Multiple Producers

Page 26: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

CPULoad (Producer 3)

CH CERN ATLAS 1.6 19055611022002

CH CERN CDF 0.6 19055511022002

CPULoad (Producer 1)

UK RAL CDF 0.3 19055711022002

UK RAL ATLAS 1.6 19055611022002

CPULoad (Producer 2)

UK GLA CDF 0.4 19055811022002

UK GLA ALICE 0.5 19055611022002

CPULoad (Consumer)

Country Site Facility Load Timestamp

UK RAL CDF 0.3 19055711022002

UK RAL ATLAS 1.6 19055611022002

UK GLA CDF 0.4 19055811022002

UK GLA ALICE 0.5 19055611022002

CH CERN ALICE 0.9 19055611022002

CH CERN CDF 0.6 19055511022002

Select * from CPULoad

Page 27: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Service

URI VOtype

emailContact

site

gppse01 alice SE [email protected] RAL

gppse01 atlas SE [email protected] RAL

gppse02 cms SE [email protected] RAL

lxshare0404 alice SE [email protected] CERN

lxshare0404 atlas SE [email protected] CERN

ServiceStatusURI VO type up statusgppse01 alice SE y SE is runninggppse01 atlas SE y SE is runninggppse02 cms SE n SE ERROR 101lxshare0404 alice SE y SE is runninglxshare0404 atlas SE y SE is running

Result Set (Consumer)

URI emailContact

gppse02 [email protected]

SELECT Service.URI Service.emailContact FROM Service S, ServiceStatus SS WHERE (S.URI= SS.URI and SS.up=‘n’)

Joins

Page 28: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

R-GMA user interfaces

• R-GMA CLI

• Web interface

Page 29: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

References

• GLUE Schema http://glueschema.forge.cnaf.infn.it/

• LCG-2 User Guide https://edms.cern.ch/file/454439//LCG-2-UserGuide.html

• gLite 3.0 User Guide https://edms.cern.ch/file/722398/1.1/gLite-3-

UserGuide.pdf

• R-GMA home page http://www.r-gma.org/

• R-GMA in EGEE http://hepunx.rl.ac.uk/egee/jra1-uk/

Page 30: GLite Information System(s) Antonio Juan Rubio Montero CIEMAT 10 th EELA Tutorial. Madrid, May 7 th -11 th,2007.

Edificio BroncePlaza Manuel Gómez Moreno s/n28020 Madrid. España

Tel.: 91 212 76 20 / 25Fax: 91 212 76 35www.red.es

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (LZW).

Questions…


Recommended