ARDA Reports to the LCG SC2 L.A.T.Bauerdick for the RTAG-11/ARDA group * Architectural Roadmap...

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

ARDA Reportsto the LCG SC2

L.A.T.Bauerdickfor the RTAG-11/ARDA group

*Architectural Roadmap towards a Distributed Analysis

LATBauerdick/Fermilab ARDA Reports – GriPhyN Applications Meeting, ANL

Oct 15, 2003 2

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.ARDA Mandate

Mandate for the ARDA RTAG

• To review the current DA activities and to capture theirarchitectures in a consistent way

• To confront these existing projects to the HEPCAL II use casesand the user's potential work environments in order to explorepotential shortcomings.

• To consider the interfaces between Grid, LCG and experiment-specific services– Review the functionality of experiment-specific packages, state of

advancement and role in the experiment.

– Identify similar functionalities in the different packages

– Identify functionalities and components that could be integrated inthe generic GRID middleware

• To confront the current projects with critical GRID areas

• To develop a roadmap specifying wherever possible thearchitecture, the components and potential sources ofdeliverables to guide the medium term (2 year) work of the LCGand the DA planning in the experiments.


Oct 15, 2003 3

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.ARDA Schedule and Makeup

Schedule and Makeup of ARDA RTAG

The RTAG shall provide a draft report to the SC2 by September 03.

• It should contain initial guidance to the LCG and the experimentsto inform the September LHCC manpower review, in particular onthe expected responsibilities of– The experiment projects

– The LCG (Development and interfacing work rather than coordinationwork)

– The external projects

The final RTAG report is expected for October 03.

The RTAG shall be composed of

• Two members from each experiment

• Representatives of the LCG GTA and AA

• If not included above, the RTAG shall co-opt or inviterepresentatives from the major Distributed Analysis projects andnon-LHC running experiments with DA experience.

• Alice: Fons Rademakers and Predrag Buncic• Atlas: Roger Jones and Rob Gardner• CMS: Lothar Bauerdick and Lucia Silvestris • LHCb: Philippe Charpentier and Andrei Tsaregorodtsev

• LCG GTA: David Foster, stand-in Massimo Lamanna• LCG AA: Torre Wenaus• GAG: Federico Carminati

Done

Done


Oct 15, 2003 4

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.ARDA mode of operationThank you for an excellent committee -- large expertise, agility and

responsiveness, very constructive and open-minded, and sacrificing quite a bit of the summer Series of weekly meetings July and August, mini-workshop in

September Invited talks from existing experiment’s projects:

• Summary of Caltech GAE workshop (Torre)• PROOF (Fons)• AliEn (Predrag)• DIAL (David Adams)• GAE and Clarens (Conrad Steenberg)• Ganga (Pere Mato)• Dirac (Andrei)

Cross-check w/ other projects of emerging ARDA decomposition of services• Magda, DIAL -- Torre, Rob• EDG, NorduGrid -- Andrei, Massimo• SAM, MCRunjob -- Roger, Lothar• BOSS, MCRunob -- Lucia, Lothar• Clarens, GAE -- Lucia, Lothar• Ganga -- Rob, Torre• PROOF -- Fons• AliEn -- Predrag• DIRAC -- Andrei• VOX -- Lothar

Done

Done


Oct 15, 2003 5


Initial Picture Distributed Analysis (Torre, Caltech w/s)


Oct 15, 2003 6

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Hepcal-II Analysis Use CasesScenarios based on GAG HEPCAL-II report

Determine data sets and eventually event components

•Input data are selected via a query to a metadata catalogue Perform iterative analysis activity

•Selection and algorithm are passed to a workload management system, together with spec of the execution environment

•Algorithms are executed on one or many nodes

•User monitors progress of job execution

•Results are gathered together and passed back to the job owner

•Resulting datasets can be published to be accessible to other users

Specific requirements from Hepcal-II Job traceability, provenance, logbooks Also discussed: support for finer-grain access control and

enabling to share data within physics groups


Oct 15, 2003 7

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Analysis Scenario

This scenario represents the analysis activity from the user perspective. However, some other actions are done behind the scene of the user interface: To carry out the analysis tasks users are accessing shared

computing resources. To do so, they must be registered with their Virtual Organization (VO), authenticated and their actions must be authorized according to their roles within the VO

The user specifies the necessary execution environment (software packages, databases, system requirements, etc) and the system insures it on the execution node. In particular, the necessary environment can be installed according to the needs of a particular job

The execution of the user job may trigger transfers of various datasets between a user interface computer, execution nodes and storage elements. These transfers are transparent for the user


Oct 15, 2003 8


Example: Asynchronous Analysis

Running Grid-based analysis from inside ROOT (adapted from AliEn example)●ROOT calling the ARDA API from the command prompt

// connect + authenticate to the GRID Service arda as “lucia”TGrid *arda = TGrid::Connect(”arda",”lucia”,"","");

// create a new analysis Object ( <unique ID>, <title>, #subjobs)TArdaAnalysis* analysis = new TArdaAnalysis(“pass001",“MyAnalysis",10);

// set the program, which executes the Analysis Macro/Scriptanalysis->Exec("ArdaRoot.sh”,"file:/home/vincenzo/test.C"); // script to execute

// setup the event metadata queryanalysis->Query("2003-09/V6.08.Rev.04/00110/%gjetmet.root?pt>0.2");

// specify job splitting and runanalysis->OutputFileAutoMerge(true); // merge all produced .root filesanalysis->Split(); // split the task in subjobsanalysis->Run(); // submit all subjobs to the ARDA queue

// asynchronously, at any time get the (partial or complete) resultsanalysis->GetResults(); // download partial/final results and merge themanalysis->Info(); // display job information


Oct 15, 2003 9

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Asynchronous Analysis Model

Extract a subset of the datasets from the virtual file catalogue using metadata conditions provided by the user.

Split the tasks according to the location of data sets. A trade-off has to be found between best use of available resources

and minimal data movements. Ideally jobs should be executed where the data are stored. Since one cannot expect a uniform storage location distribution for every subset of data, the analysis framework has to negotiate with dedicated Grid services the balancing between local data access and data replication.

Spawn sub-jobs and submit to Workload Management with precise job descriptions

User can control the results while and after data are processed

Collect and Merge available results from all terminated sub-jobs on request

Analysis objects associated with the analysis task remains persistent in the Grid environment so the user can go offline and reload an analysis task at a later date, check the status, merge current results or resubmit the same task with modified analysis code.


Oct 15, 2003 10

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Synchronous Analysis

Scenario: using PROOF in the Grid environment Parallel ROOT Facility, main developer Maarten

Ballintjin/MIT PROOF already provides a ROOT-based framework to use

a (local) cluster computing resources

•balancing dynamically the workload, with the goal of optimizing CPU exploitation and minimizing data transfers

•makes use of the inherent parallelism in event data

•works in heterogeneous clusters with distributed storage

Extend this to the Grid using interactive analysis services, that could be based on the ARDA services


Oct 15, 2003 11


ARDA Roadmap “Informed” By DA Implementations

Following SC2 advice, reviewed major existing DA projectsClearly AliEn today provides the most complete implementation of a

distributed analysis services, that is “fully functional” -- also interfaces to PROOF Implements the major Hepcal-II use cases Presents a clean API to experiments application, Web portals, … Should address most requirements for upcoming experiment’s

physics studies• Existing and fully functional interface to complete analysis

package --- ROOT• Interface to PROOF cluster-based interactive analysis system• Interfaces to any other system well defined and certainly

feasible Based on Web-services, with global (federated) database to give

state and persistency to the systemARDA approach:

Re-factoring AliEn, using the experience of the other project, to generalize it in an architecture; Consider OGSI as a natural foundation for that

Confront ARDA services with existing projects (notably EDG, SAM, Dirac, etc)

Synthesize service definition, defining their contracts and behavior Blueprint for initial distributed analysis service infrastructure


Oct 15, 2003 12

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.ARDA Distributed Analysis Services

Distributed Analysis in a Grid Services based architecture ARDA Services should be OGSI compliant -- built upon OGSI middleware Frameworks and applications use ARDA API

with bindings to C++, Java, Python, PERL, …

• interface through UI/API factory -- authentication, persistent “session” Fabric Interface to resources through CE, SE services

• job description language, based on Condor ClassAds and matchmaking

Database(ses) through Dbase Proxy provide statefulness and persistence

We arrived at a decomposition into the following key services API and User Interface Authentication, Authorization, Accounting and Auditing services Workload Management and Data Management services File and (event) Metadata Catalogues Information service Grid and Job Monitoring services Storage Element and Computing Element services Package Manager and Job Provenance services


Oct 15, 2003 13


API

User Interface Factory

Auditing

DBD/RDBMS

Registry/Lookup/Config

V.O. directory

Authentication

Storage Element

Gatekeeper

Job Manager

Transfer Manager

File Transfer

Process Monitor

Transfer Broker

Job Broker

Job Optimizer

Transfer Optimizer

Catalogue Optimiser

User Interface

Grid Monitoring

CE

1

1..n1

1

0..n

1..n

1

1

1

1

11

1

1

1

0..n

0..n

0..n

0..n

0..n

1. lookup

2. authenticate

3. register

4. bind

Authorisation

File Catalogue

Metadata Catalogue

Task Queue

DB PRoxy

1

1

1

1

Package Manager

Job Provenance

1

Authorisation

Accounting

111111111111111

1

1

AliEn (re-factored)


Oct 15, 2003 14


ARDA Key Services for Distributed Analysis

Information Service

Authentication

Authorisation

Auditing

Grid Monitoring

Workload

Management

Metadata Catalogue

File Catalogue

Data Management

Computing Element

Storage Element

Job

Monitor

Job Provenance

Package Manager

DB Proxy

User Interface

API

Accounting

7:

12:

5:

13:

8:

15: 11:

9: 10:

1:

4:

2:

3:

6:

14:


Oct 15, 2003 15


ARDA HEPCAL matching: an example

HEPCAL-II Use Case: Group Level Analysis (GLA)

• User specifies job information including• Selection criteria;

• Metadata Dataset (input);

• Information about s/w (library) and configuration versions

• Output AOD and/or TAG Dataset (typical);

• Program to be run;

• User submits job;

• Program is run;

• Selection Criteria are used for a query on the Metadata Dataset;

• Event ID satisfying the selection criteria and Logical Dataset Name of corresponding Datasets are retrieved;

• Input Datasets are accessed;

• Events are read;

• Algorithm (program) is applied to the events;

• Output Datasets are uploaded;

• Experiment Metadata is updated;

• Report summarizing the output of the jobs is prepared for the group (eg. how many evts to which stream, ...) extracting the information from the application and GridMW

•Authentication•Authorization•Metadata catalog

•Workload management

•Metadata catalog

•File catalog•Data management•Storage Element

•Package manager•Compute element

•File catalog•Data management•Storage Element

•Metadata catalog•Job provenance

•Auditing


Oct 15, 2003 16

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.API to Grid servicesARDA services present an API, called by applications

like the experiments frameworks, interactive analysis packages, Grid portals, Grid shells, etc

In particular the importance of UI/API Interface services to higher level software

• Exp. framework

• Analysis shells, e.g. ROOT

• Grid portals and other forms of user interactions with environment

• Advanced services e.g. virtual data, analysis logbooks etc Provide experiment specific services

• Data and Metadata management systems

Provide an API that others can project against Benefits of common API to framework

• Goes beyond “traditional” UIs à la GANGA, Grid portals, etc Benefits in interfacing to analysis applications like ROOT et al Process to get a common API b/w experiments --> prototype

The UI/API can use the Condor ClassAds as a Job Description Language

• This will maintain compatibility with existing job execution services, in particular LCG-1.


Oct 15, 2003 17

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.API and User Interface

API

+ Authentication+ Data Management

+ Grid Service Management+ Job Control

+ Metadata Management+ NewInterface

+ Posix I/O

SOAP(from API)

Grid File Access

(from API)

Experiment Frameworks

POOL/ROOT/...

(from Experiment Frameworks)...)

API (OGSI User Interface Factory)

Storage Element (POSIX I/O service)

Portals

Grid Shells

Grid File System


Oct 15, 2003 18


File Catalogue and Data Management

Input and output associated with any job can be registered in the VO’s File Catalogue, a virtual file system in which a logical name is assigned to a file. Unlike real file systems, the File Catalogue does not own the

files; it only keeps an association between the Logical File Name (LFN) and (possibly more than one) Physical File Names (PFN) on a real file or mass storage system. PFNs describe the physical location of the files and include the name of the Storage Element and the path to the local file.

The system should support file replication and caching and will use file location information when it comes to scheduling jobs for execution.

The directories and files in the File Catalogue have privileges for owner, group and the world. This means that every user can have exclusive read and write privileges for his portion of the logical file namespace (home directory).


Oct 15, 2003 19

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Job Provenance service

The File Catalogue is extended to include information about running processes in the system (in analogy with the /proc directory on Linux systems) and to support virtual data services Each job sent for execution gets an unique id and a

corresponding /proc/id directory where it can register temporary files, standard input and output as well as all job products. In a typical production scenario, only after a separate process has verified the output, the job products will be renamed and registered in their final destination in the File Catalogue. The entries (LFNs) in the File Catalogue have an immutable unique file id attribute that is required to support long references (for instance in ROOT) and symbolic links.


Oct 15, 2003 20

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Package Manager Service

Allows dynamic installation of application software released by the VO (e.g. the experiment or a physics group).

Each VO can provide the Packages and Commands that can be subsequently executed. Once the corresponding files with bundled executables and libraries are published in the File Catalogue and registered, the Package Manager will install them automatically as soon as a job becomes eligible to run on a site whose policy accepts these jobs.

While installing the package in a shared package repository, the Package Manager will resolve the dependencies on other packages and, taking into account package versions, install them as well. This means that old versions of packages can be safely removed from the shared repository and, if these are needed again at some point later, they will be re-installed automatically by the system. This provides a convenient and automated way to distribute the experiment specific software across the Grid and assures accountability in the long term.


Oct 15, 2003 21

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Computing Element

Computing Element is a service representing a computing resource. Its interface should allow submission of a job to be executed on the underlying computing facility, access to the job status information as well as high level job manipulation commands. The interface should also provide access to the dynamic status of the computing resource like its available capacity, load, number of waiting and running jobs.

This service should be available on a per VO basis.

Etc pp.


Oct 15, 2003 22

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Talking PointsHorizontally structured system of services with a well-defined API and a database

backend Can easily be extended with additional services, new implementations can be

moved in, alternative approaches tested and commissionedInterface to LCG-1 infrastructure

VDT/EDG interface through CE, SE and the use of JDL, compatible with existing i/s ARDA VO services can build on emerging VO management infrastructure

ARDA initially looked at file based datasets, not object collection talk with POOL how to extend the file concept to a more generic collection

concept investigate experiment’s metadata/file catalog interaction

VO system and site security Jobs are executed on behalf of VO, however users fully traceable

How do policies get implemented, e.g. analysis priorities, MoU contributions etc Auditing and accounting system, priorities through special “optimizers” accounting of site “contributions”, that depend what resources sites “expose”

Database backend for the prototype Address latency, stability and scalability issues up-front; good experience exists In a sense, the system is the database (possibly federated and distributed) that

contains all there is to know about all jobs, files, metadata, algorithms of all users within a VO

set of OGSI grid services provide “windows”/”views” into the database, while the API provides the user access

allows structuring into federated grids and “dynamic workspaces”


Oct 15, 2003 23

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.General ARDA RoadmapEmerging picture of “waypoints” on the ARDA

“roadmap” ARDA RTAG report

• review of existing projects, common architecture component’a decomposition & re-factoring

• recommendations for a prototypical architecture and definition of prototypical functionality and a development strategy

Development of a prototype and first release

• Integration with and deployment on LCG-1 resources and services

Re-engineering of prototypical ARDA services, as required

OGSI gives framework in which to run ARDA services Addresses architecture Provides framework for advanced interactions with the Grid Need to address issues of OGSI performance and scalability up-

front

• Importance of modeling, plan for scaling up, engineering of underlying services infrastructure


Oct 15, 2003 24

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Roadmap to a GS Architecture for the LHC

Transition to grid services explicitly addressed in several existing projects Clarens and Caltech GAE, MonaLisa

•Based on web services for communication, Jini-based agent architecture

Dirac

•Based on “intelligent agents” working within batch environments

AliEn

•Based on web services and communication to distributed database backend

DIAL

•OGSA interfaces Initial work on OGSA within LCG-GTA

•GT3 prototyping

Leverage from experience gained in Grid M/W R&D projects


Oct 15, 2003 25

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.ARDA Roadmap for PrototypeNo “evolutionary” path from GT2-based grids

Recommendation: build early a prototype based on refactoring existing implementations

Prototype provides the initial blueprint Do not aim for a full specification of all the interfaces

4-prong approach: Re-factoring of AliEn, Dirac and possibly other services into ARDA

• Initial release with OGSI::Lite/GT3 proxy, consolidation of API, release

• Implementation of agreed interfaces, testing, release GT3 modeling and testing, ev. quality assurance Interfacing to LCG-AA software like POOL, analysis shells like ROOT

•Also opportunity to “early” interfacing to complementary projects

Interfacing to experiments frameworks

•metadata handlers, experiment specific servicesProvide interaction points with community


Oct 15, 2003 26


Experiments and LCG Involved in Prototyping

ARDA prototype would define the initial set of services and their interfaces. Timescale: spring 2004

Important to involve experiments and LCG at the right level Initial modeling of GT3-based services Interface to major cross-exp packages: POOL, ROOT, PROOF,

others Program experiment frameworks against ARDA API, integrate with

experiment environments Expose services and UI/API to other LHC projects to allow synergies Spend appropriate effort to document, package, release, deploy

After the prototype is delivered, improve on Scale up and re-engineer as needed: OGSI, databases, information

services Deployment and interfaces to site and grid operations, VO

management etc Build higher-level services and experiment specific functionality Work on interactive analysis interfaces and new functionalities


Oct 15, 2003 27


Major Role for Middleware Engineering

ARDA roadmap based on a well-factored prototype implementation that allows evolutionary development into a complete system that evolves to the full LHC scale

ARDA prototype would be pretty lightweight

• Stability through basing on global database to which services talk through a database proxy

• “people know how to do large databases” -- well founded principle (see e.g. SAM for RunII), with many possible migration paths

•HEP-specific services, however based on generic OGSI-compliant services

Expect LCG/EGEE middleware effort to play major role to evolve this foundation, concepts and implementation re-casting the (HEP-specific event-data analysis oriented) services

into more general services, from which the ARDA services would be derived

addressing major issues like a solid OGSI foundation, robustness, resilience, fault recovery, operation and debugging

Expect US middleware projects to be involved in this!


Oct 15, 2003 28

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Conclusions

ARDA is identifying a services oriented architecture and an initial decomposition of services required for distributed analysis

Recognize a central role for a Grid API which provides a factory of user interfaces for experiment frameworks, applications, portals, etc

ARDA Prototype would provide an distributed physics analysis environment of distributed experimental data for experiment framework based analysis

•Cobra, Athena, Gaudi, AliRoot, for ROOT based analysis interfacing to other analysis packages like JAS; event

displays like Iguana; grid portals; etc. can be implemented easily

Date post:	31-Dec-2015
Category:	Documents
Upload:	jocelin-garrison
View:	216 times
Download:	1 times

ARDA Reports to the LCG SC2 L.A.T.Bauerdick for the RTAG-11/ARDA group * Architectural Roadmap...

Documents