Date post: | 28-Dec-2015 |
Category: |
Documents |
Upload: | blaze-sullivan |
View: | 214 times |
Download: | 1 times |
2DataGRID ConferenceMarch 6, 2002
Application objectives for year 1
Define use cases for the applicationsDefine application requirementsDeploy realistic applications on the TestBed 1Evaluate TestBed 1
3DataGRID ConferenceMarch 6, 2002
AchievementsDialogue within the applications on the opportunities of GRID
Dialogue and coordination to understand how to do things differently and not more of the same
Testbed 0 has improved our understanding of the issues A very large amount of unfunded application effort has made
this possible Realistic applications using GLOBUS have been deployed
Requirements expressed in June 2001 Developed detailed “long term use cases” A very large exercise leveraging unfunded effort (30-40
person-month)Funded WP8 effort (LCs) has
Participated to the deployment and configuration of the TestBed
Validated the TestBed with “generic” HEP applications Resource Broker Replica catalogue functionality
4DataGRID ConferenceMarch 6, 2002
AchievementsDetailed TestBed evaluation plans have been elaborated and applied
Few hours after the official opening of the TestBed on December 9 a standard physics simulation job has been run
Although TB1 had not reached stability much middleware functionality was demonstrated by the applications in a short period of time
5DataGRID ConferenceMarch 6, 2002
Achievements
Mechanisms to provide feedback to the developers have been put in place Priority list constantly updated by WP8 and
discussed at weekly WP manager meetings Feedback on the release plan Detailed user requirements for crucial
components (e.g. Storage Element) in preparation
Large unfunded participation from experiments Probably as much as 200 person-month in the
first year
6DataGRID ConferenceMarch 6, 2002
Issues and actionsStability (or even existence) of the testbed
Testbed existed for 10 days in December and then for ~5 days before the review!!
Configuration and management of the VO’s is very difficult
Our needs were not expressed too clearly and were not given enough attention
Procedures should be worked out to automate it
Assignment and escalation of problems is complicated
It has worked well thanks to the heroic efforts of all people involved, but it will have to be better formalised
Need tight coordination between site-managers, TestBed and applications
7DataGRID ConferenceMarch 6, 2002
Issues and actionsCoexistence of pseudo-production and development environment still a problem
We have done it for years on a local environments On a distributed ones is more difficult and we are
learning
Deployment plans to all TestBed sites are still unclear
This needs to be clarified!
8DataGRID ConferenceMarch 6, 2002
Plan for the next yearContinue exploitation of TB1
Deploy larger and more realistic applications on TB1, including “data challenges” and “production challenges”
Use more sites as TB1 expands
Refine the user requirements and long term views A common set of requirements / use cases to define
common solutions This will facilitate the relation with other GRID MW projects Collaboration with DataTag has MW interoperability as
specific goal An RTAG has been launched in the LCG Project with this
mandate This has been possible because of the work of the DataGRID
project and is heavily based on WP8's findings to date
9DataGRID ConferenceMarch 6, 2002
What we want from a GRIDThis is the result of our experience on TB0 & TB1
OS & Net services
Basic Services
High level GRID middleware
LHCVO common application layer
Other apps
ALICE ATLAS CMS LHCb
Specific application layer
Other apps
GLOBUS team
GRID architectur
e
10DataGRID ConferenceMarch 6, 2002
Common use cases
OS & Net services
Bag of Services (GLOBUS)
Specific application layer
GLOBUS team
MiddleWare
MW1 MW2 MW3 MW4 MW5
MiddleWare
MW1 MW2 MW3 MW4 MW5
LHCVO use cases & requirements Other apps
If we manage to define
ALICE ATLAS CMS LHCb Other apps
MiddleWare
MW1 MW2 MW3 MW4 MW5
VO use cases & requirements Common core use case
Or even better
LHC Other apps
MiddleWare
MW1 MW2 MW3 MW4 MW5
VO use cases & requirements
It will be easier to arrive at
Common use cases
LHC Other apps
11DataGRID ConferenceMarch 6, 2002
Detailed plan of workLCs have already interviewed the experiments
A lot of material has been collectedThey will now post-process the information
Hopefully one monthThe result will be the starting point for the RTAG
RTAG will contain representative of experiments and (subject to the agreement of the experiments) of LCs
The RTAG produces results A of detailed use cases for HEP in two-three months
During this time LCs will devote their attention to the testbed (see later)The results will be brought back into WP8/9/10 to
Discuss possible implementation Involve WP9-10
12DataGRID ConferenceMarch 6, 2002
Production Scenario (PRO)
VO MetaDataCatalogLFN1LFN2LFN3LFN4...
JDLWDL file Broker
Production db
UIExp.Frame
VO-ReplicaCatalog
CE
SE
WNConfiguration db GIIS
GRIS
MSS
13DataGRID ConferenceMarch 6, 2002
VO MetaDataCatalogLFN1LFN2LFN3LFN4...
JDL file
Broker
Production db VO-ReplicaCatalog
CE
SE
WNConfiguration db
Rcapifile open
UIExp.Frame
SE
Spawn a job
EvtTagCollection
Physics Analysis (ANA) scenario
Retrieve result
14DataGRID ConferenceMarch 6, 2002
Execution (additional data/info needed)
Job on worker node
Catalog(LFN)
Calibration…
ReplicaCatalogue
InformationServices
Scheduling Grid
15DataGRID ConferenceMarch 6, 2002
Postprocessing
Job on worker node
Catalog(LFN)
Calibration…
ReplicaCatalogue
InformationServices
Storage Element
SEn
16DataGRID ConferenceMarch 6, 2002
Disclaimer
A lot of this work is essentially architectural We want to provide a purely user point of view Inevitably we will trespass the border and say
not only what we want but how we want it
However we understand this and we will be glad to be suggested different ways to do that As long as this still does what we want
17DataGRID ConferenceMarch 6, 2002
Application PrioritiesAPI’s
This is a long standing requirement, now this is needed for prototyping
MPI At least on local clusters, wide area MPI later
Interactive access We have to agree on what it means and how it can be
provided We certainly want to start trying on TB1
Linux 7.2 (summer) This is necessary for a proper deployment
Solaris (release 2) Not a priority per se, but if not this year, we will never do it The DataGRID code should demonstrate some portability to
be seriously considered as a baselineTools for VO management
18DataGRID ConferenceMarch 6, 2002
Application testbed requirements
Three main priorities
19DataGRID ConferenceMarch 6, 2002
Application testbed requirements
Development and production testbeds We need a valid model to have both
Consolidation & Stability of production testbed It has to stay up from tomorrow to release 2Consolidation and complete deployment of uniform information system
The question of FTREE/R-GMA/MDS has to be solved
Support for the testbed This has to work much better than in TB1 till now
Interface to MSS CASTOR, HPSS, DataSTORESolve the question of installation on pre-existing clusters
Light LCFG?Expansion of the testbed
(up to) the famous 40 WP6 sites
20DataGRID ConferenceMarch 6, 2002
Immediate consolidationMay be jump started by a jam session of LCs and WPs Get together for few days till some test
applications run on the testbed without problems
This should be done rather quickly 3-4 days week 22th March or after Easter
These applications may be bundled in a standard test package to run periodically A kind of testbed heartbeat to verify its
stability
21DataGRID ConferenceMarch 6, 2002
SummaryApplications have been able to deploy and demonstrate large applications on the TestBed
The LHC experiments are participating with enthusiasm to the project
The side effect is that the pressure is high from the users for
Support Stable environment Functionality Documentation Wide deployment
These are healthy signs Continued positive reaction to the feedback from the users
will maintain their high level of enthusiasm and participationThe evolution of TB1 leading to TB2 will be a crucial test of the project’s ability to reach its final goals