Post on 28-Dec-2015
transcript
CHEP 2000, 10.02.2000 Roberto Barbera
Roberto Barbera(*)
The GENIUS grid portal
CHEP 2003, 24-28.03.2003
(*)work in collaboration with A. Falzone and A. Rodolico
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Outline
Introduction
Motivations
Current status (through the real use made by the Experiments)
Summary and conclusion
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GRID
MIDDLEWARE
Visualising
Workstation
Mobile Access
Supercomputer, PC-Cluster
Data-storage, Sensors, Experiments
Internet, networks
Roberto Barbera
The Grid metaphor
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
CHEP 2003, 24-28.03.2003
DataGrid job submission work-flowReplicaReplicaCatalogueCatalogue
UIJDL
Logging &Logging &Book-keepingBook-keeping
ResourceResourceBrokerBroker
Job SubmissionJob SubmissionServiceService
StorageStorageElementElement
ComputeComputeElementElement
Information Information ServiceService
Job Status
DataSets info
Author.&Authen.
Job S
ub
mit
Even
t
Job
Qu
ery
Job
Stat
us
Input “sandbox”
Input “sandbox” + Broker InfoGlobus RSL
Output “sandbox”
Output “sandbox”
Job Status
Pu
blis
h
grid
-pro
xy-in
it
Exp
and
ed J
DL
SE & CE info
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto Barbera
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
EDG m/w has been released but…EDG software (Globus, UI, JDL, WP2, WP3, etc.) contains tens of commands/switches which also have their own logical sequences.
Browsing Grid VO “directories” (users, RC’s, DB’s, etc.) requires LDAP “speaking” and tomorrow could require SQL “speaking”.
“User gridification” is a tough task for a “rookie” this does not fit with the claim that “grids” are for everybody and that grid computing will be as easy as surfing the Internet ?
Furthermore, all this holds for DataGrid. What will happen when other grids’ software (especially UI’s) will come up (PPDG, iVDGL, etc.) ? Will users have to learn tens of “grid dialects” ?
Today “grid computing” is a rather complicated experience for un-experienced user only possible at selected machines (UI’s) this does not fit with the claim that one could do “grid computing” even from a PDA ?
Is there any way to set-up a “user-friendly” grid ?
Roberto Barbera
A web portal: why and how ?It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, WAP phone).It can keep the same user interface to several back-ends (grid “dialects” command-line UI’s).It must be redundantly “secure” at all levels: 1) secure for web transactions, 2) secure for user credentials, 3) secure for user authentication, 4) secure at VO level.All available grid services must be incorporated in a logic way, just “one mouse click away”.Its layout must be easily understandable and user friendly.
Roberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
A little bit of history1998-1999graphic interface to LSF (multi) clusters for ALICE
jobs
2000-2001simple but effective web portal to submit ALICE
jobs to the grid using EnginFrame and GLOBUS
2002-nowofficial grant of the INFN Grid Project to
collaborate with NICE srl to integrate in a web portal all services offered by the DataGrid middle-ware; adoption/contributions by other experiments
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto Barbera
GENIUS®
(Grid Enabled web eNvironment for
site Independent User job Submission)
[https://genius.ct.infn.it]
INFN/NICE collaboration
OS & Net services
Basic Services
High level GRID middleware
ALICE ATLAS CMS LHCbApplications’specific layer
Other apps
GLOBUS
toolkit
DataGRID architectur
e
GENIUS web portal
CHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto Barbera
GENIUS: how it works
Apache
EnginFrame
GENIUShttps+java/xml+rfb
WEB Browser
EDGUI
Local
WS
the GridEDG+GSI
3-tier model
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaRoberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
EnginFrame in briefStandards-based GRID portal Java, Tomcat, Apache, XML/XSL → GridML
Solves back-end integration problems Visual rendering for most Grid objects
jobs, job arrays, hosts, services, databases, etc. Multiple Grid technologies support
EDG, GLUE, Globus, Condor (soon) Authentication delegation (GSI,MyProxy, NIS,NT,
Kerberos V, ...) Data management: UL/DL + remote (multi-) file browsing Integration with interactive apps
End-user oriented focus! application integration
Roberto BarberaCHEP 2003, 24-28.03.2003
GENIUS show: the main page
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaCHEP 2003, 24-28.03.2003
GENIUS show: OS authentication
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaCHEP 2003, 24-28.03.2003
GENIUS show: file services
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaCHEP 2003, 24-28.03.2003
GENIUS show: grid authentication
MyProxy auth
enticatio
n !
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto Barbera
Grid authentication “à la MyProxy”
EDGUI
Local
WS
Roberto Barbera
MyProxy
Server
GENIUS
Server(EDG
UI)
grid-proxy-initmyproxy-init
any grid service
myproxy-get-delegatio
n
output
the Grid
execution
WEB Browser
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
ALICE: graphic job description
CHEP 2003, 24-28.03.2003 Roberto Barbera
ALICE: job submission
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaCHEP 2003, 24-28.03.2003
ALICE: job queue
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaCHEP 2003, 24-28.03.2003
ALICE: job output
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaCHEP 2003, 24-28.03.2003
ALICE: personal spooler
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
CHEP 2003, 24-28.03.2003 Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
ALICE: grid file browsing
CHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
ALICE: grid file replication
Roberto Barbera
CHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
ALICE: grid file inspection
Roberto Barbera
CHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
ALICE: grid file downloading
Roberto Barbera
CHEP 2003, 24-28.03.2003 Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
ALICE: interactive analysis
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
US
(cou
rtes
y of
L. V
acca
ross
a)
EU
ATLAS: “GLUE” job submission
Roberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(cou
rtes
y of
L. V
acca
ross
a)
ATLAS: interactive analysis
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(cou
rtes
y of
M. V
erla
to)
CMS grid integration schema
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(cou
rtes
y of
M. V
erla
to)
CMS: GENIUS/Impala integration
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(cou
rtes
y of
M. V
erla
to)
CMS: GENIUS/BOSS integration
CHEP 2003, 24-28.03.2003 Roberto Barbera
GENIUS show: Earth Ob. app.
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: Biomed. app.
Roberto Barbera
Present status and perspectivesCurrent implementation of GENIUS includes more than 100 services. Among them: secure web transactions, user authentication and authorization; grid authentication “à la MyProxy” ! remote interaction with the user’s and “grid” file system; interfaces for job submission/control, data management, VO servers,
and monitoring systems; several Resource Brokers and VO’s available; persistent (user level) book-keeping and spooler system; secure interactive analysis with TightVNC full support for “parallel” multi-jobs (“sequential” ones soon) web guided job description (from DATAMAT) CVS available !
Next future: Participation to the GGF Grid Portal Manifesto and GCE Working
Group.
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Roberto BarberaRoberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
FAQ’s…and answers (1)Q: I want to use GENIUS. Do I have to pay for it ?A: No. GENIUS is “open source” and the underlying portal framework EnginFrame is freefree for the education and research communities.Q: I want to use GENIUS. Do I need any software running on my laptop ?A: No client software needs to be installed apart from the web browser. GENIUS can really be accessed from everywhere.Q: Do I have to be afraid about cached password sent over the web ?A: Access passwords are securely “streamed” only when needed and then destroyed. Only temporary sessions are possible.Q: Can new authentication methods implemented into GENIUS ?A: Of course. Kerberos V is a good example. EnginFrame is already compliant with Kerberos authentication.
Roberto BarberaRoberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
FAQ’s…and answers (2)Q: I want to add a new VO to GENIUS and customize new services for that VO. How can I do that ?A: A new VO can be added to GENIUS in just minutes. New VO specific services can be added just modifying only two files: an XML file and a shell script.Q: Can I use GENIUS to interface other m/w’s ?A: Yes. Although GENIUS is currently based on the DataGrid middleware(w/ and w/o GLUE extensions), it can be very easily interfaced to others. A direct interface to the Globus Toolkit already exists and another one to Condor is in progress.Q: How can I start downloading/using GENIUS ?A: Go to the reference site https://genius.ct.infn.it, click on “GENIUS CVS available” and follow the instructions.
Roberto BarberaRoberto BarberaCHEP 2003, 24-28.03.2003
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Summary and conclusionComputational grids could represent the “natural” environment for next generation high energy physics experiments, computational bio-medicine, Earth observation and many other inter-disciplinary applications. “Grid” could be the Internet “new age” where users can seamlessly and ubiquitously access not only information but also huge computing resources and mass storage systems distributed worldwide with their own applications.However, in order to turn dreams into reality, grid access must be ubiquitous, easy and intuitive especially for the vast majority of non-expert users and these are just the goals of GENIUS. GENIUS is also a good example of implementation of real use cases started in one experiment and followed by others.