LHCb Development

4th February 2004 GRIDPP9 1

LHCb Development

Glenn PatrickRutherford Appleton Laboratory


LHCb - Reminder

RICH1RICH2

Calorimeters

Muon System

VELO

Magnet

Tracking stations(inner and outer)

20 m

1.2M electronic channelsWeight ~4,000 tonnes

b

d

B meson

b

dAnti-B meson


LHCb GridPP Development

LHCb development has been taking place on three fronts: MC Production Control

and Monitoring Gennady Kuznetsov (RAL)

Data ManagementCarmine Cioffi (Oxford) Karl Harrison (Cambridge)

GANGA Alexander Soroko (Oxford)Karl Harrison (Cambridge)

All developed in tandem with LHCb Data Challenges


Data Challenge DC03

VELO TT

RICH2

RICH1

65M events processed.

Distributed over 19 different centres.

Averaged 830,000 events/day.

Equivalent to 2,300 × 1.5GHz

computers.

34% processed in UK at 7 different institutes.

All data written to CERN.

“Physics” Data Challenge.Used to redesign and optimise detector …


The LHCb Detector

Changes were made formaterial reduction andL1 trigger improvement

Reduced number of layers for M1 (4 2) Reduced number of tracking stations behind the magnet (4 3)

No tracking chambers in the magnetNo B field shielding plate

Full Si stationReoptimized RICH-1 design

Reduced number of VELO stations (25 21)


“Detector” TDRs completed

Only Computing TDR remains


Data Challenge 2004“Computing” Data Challenge.April – June 2004

Produce 10 × more events.

At least 50% to be done via LCG.

Store data at nearest Tier-1(i.e. RAL for UK institutes)

Try out distributed analysis.Test computing model and write computing TDR.

Require stable LCG2 release with SRM interfaced to RAL DataStore


DC04: UK Tier-2 Centres

NorthGridDaresbury, Lancaster, Liverpool,Manchester, Sheffield

SouthGridBirmingham, Bristol, Cambridge,Oxford, RAL PPD

ScotGridDurham, Edinburgh, Glasgow

LondonGridBrunel, Imperial, QMUL, RHUL, UCL

1101100011


DIRAC Architecture

Information Service

Authentication

Authorisation

Auditing

Grid Monitoring

Workload Management

Metadata Catalogue

File Catalogue

Data Management

Computing Element

Storage Element

Job Provenance

Package Manager

User Interface

API

Accounting

DIRAC components

Other project components:AliEn, LCG, …

Resources:LCG, LHCb production sites


MC Control StatusGennady Kuznetsov

Control toolkit breaking down production workflow into components – modules, steps.

To be deployed in DC04. SUCCESS!

DIRACDistributedInfrastructure withRemoteAgentControl


Bookkeeping dataMonitoring info

Get jobs

Site ASite A

Site BSite B

Site CSite C

Site DSite D

Agent

Production service

Monitoring serviceBookkeeping service

AgentAgent

Agent

DIRAC v1.0 Original scheme

“Pull” ratherthan “Push”


Components – MC Control

Module

Module

Module

Module

Module

Module

Module

Step

Step

Step StepStep

Step Step

Step

WorkflowJob

Job Job Job

Job Job

JobJob

Job Job Job

Job Job

Job

Job

Job Job Job

Job Job

Job

ProductionLevels of usage:1. Module – Programmer2. Step – Production

Manager3. Workflow –

User/Production manager

Module is the basic component

of the architecture

Each step generates job as a Python program.

This structure allow the Production Manager to construct any algorithm as a combination of modules.

Gennady Kuznetsov


Module Editor

Python code of single module. Can

be many classes.Module

variables.

Description

Module Name Stored as XML file

Gennady Kuznetsov


Step EditorStep Name

Description

Definitions of Modules

Instances of Modules

Variables of currently selected instance

Selected instance

Stored as XML file, where all modules are embedded

Gennady Kuznetsov

Step variables.


Workflow Editor

Gennady Kuznetsov

Workflow Name

Description

Step Definitions

Step Instances

Variables of currently

selected Step Instance

Selected Step Instance

Workflow Variables.

Stored as XML file


Job Splitting

Gennady Kuznetsov

Step

Step StepStep

Step Step

Step

Workflow DefinitionJob

Job Job Job

Job Job

JobJob

Job Job Job

Job Job

Job

Job

Job Job Job

Job Job

Job

Production

Python List

The input value for the job splitting is a Python list object.

Every single (top level) element of this list applies to the Workflow Definition and propagates through the code and generates a single element of the production (one or several jobs).


Future: Production Console

Once an agent has received a workflow, the Production Manager has no control over any function in a remote centre.

Local Manager must perform all of the configurations and interventions at individual site.

Develop ”Production Console” which will provide extensive control and monitoring functions for the Production Manager.

Monitor and configure remote agents.

Data replication control.

Intrusive system – need to address Grid security mechanisms and provide robust environment.


DIRAC v1.0 Architecture

Application packager


Workfloweditor

Workfloweditor

Production editor

Production editor

Production manager

Production DBProduction DB

Production preparation

EditInstantiateworkflow

Createapplication

tar file

Central Services

MonitoringDB

MonitoringDB

BookkeepingDB

BookkeepingDB

Production resources

Agent AAgent ASite A

Agent nAgent nSite n

…

JobXMLJobXML

JobstatusJob

status

MetaXMLMetaXML

Castor MSSCERN

Castor MSSCERN

Datasetreplica

Datasetreplica

Agent BAgent BSite B

Central Storage

Jobrequest

Jobrequest

Bookkeeping Service

Bookkeeping Service

Monitoring Service

Monitoring Service

Production Service

Production Service



Workfloweditor

Workfloweditor

Production editor

Production editor

Production manager

Production DBProduction DB

Production preparation

EditInstantiateworkflow

Createapplication

tar file

Central Services

MonitoringDB

MonitoringDB

BookkeepingDB

BookkeepingDB

Production resources

Agent AAgent ASite A

Agent nAgent nSite n

…

JobXMLJobXML

JobstatusJob

status

MetaXMLMetaXML

Castor MSSCERN

Castor MSSCERN

Datasetreplica

Datasetreplica

Agent BAgent BSite B

Central Storage

Jobrequest

Jobrequest

Bookkeeping Service

Bookkeeping Service

Monitoring Service

Monitoring Service

Production Service

Production ServiceProduction Manager


DIRAC v2.0 WMS Architecture

Job Receiver

Job Receiver

Job DBJob DB

Optimizer 1Optimizer 1Optimizer 1Optimizer 1

Optimizer 1Optimizer 1

MatchMakerMatchMakerJo

b qu

eue

Agent 1Agent 1

Agent 2Agent 2

Agent 3Agent 3

LCG CELCG CE

LCG WMSLCG WMS

DIRAC CEDIRAC CE

DIRAC Workload Management

Productionservice

Productionservice

GANGAGANGA

Commandline UI

Commandline UI

Computing resources

Job Receiver

Job Receiver

Job DBJob DB




b qu

eue

Agent 1Agent 1

Agent 2Agent 2

Agent 3Agent 3

LCG CELCG CE

LCG WMSLCG WMS

DIRAC CEDIRAC CE


Productionservice

Productionservice

GANGAGANGA

Commandline UI

Commandline UI

Computing resources

ProductionService

Also data stored remotely

Based on central queue service


Data Management Status

Carmine Cioffi

File catalogue browser for POOL

Integration of POOL persistency framework into GAUDI new EventSelector interface.

SUCCESS!


List of LFNs

Tabs for LFN / PFN

mode selection

List of PFNs associated to the

LFN selected from the list of

LFNs on the left sub-panel

Read the next and previous bunch of

files from the catalog

Write mode selection

Import the fragment of a catalog

Reload the catalog

Shows the metadata schema, with the

possibility to change it

List all the metadata value of the catalog List the files

selected

Search text bar

Filter text bar.

Main Panel, LFN Mode Browsing

POOL file catalogue providesLFN & PFN association.

Browser allows user to interact with catalogue via GUI.

Can save list of LFNsfor job sandbox


Main Panel, PFN Mode Browsing

Sub menu with three operations to be done on the file

selected.

In PFN mode, the files are browsed in the same way as Windows Explorer. The folders are shown on the left sub-panel and the value of the folder on the right sub-panel.

Write mode buttonopens WrFCBrowserframe allowing user to write to the catalogue…


Register a PFN

Add a PFN replica

Delete a PFN

Add LFN

Remove a LFN

Add metadata

value

RollbackCommit

Show the action performed

Write Mode Panel


PFN register frame

Frame to show and change the metadata schema of the catalog

This frame allows setting of the

metadata value


This frame shows the

metadata value of the PFN Myfile

Shows the list of the files selected

This frame shows the attribute value of the

PFN


Benefit from investment in LCGRetire parts of Gaudi reduce maintenance.

Designed and implemented a new interface for the LHCb EventSelector. Criteria:

One or more “datasets” (e.g. list of runs, list of files matching a given criteria).

One or more “EventTagCollections” with extra selection based on Tag values.

One or more physical files.

Result of an event selection is a virtual list of event pointers.

GAUDI/POOL Integration


Physicist’s View of Event Data

GaudiBookkeeping

DatasetEvent 1Event 2…

Event 3


Event 3

FileEvent 1Event 2…

Event N

FilesRAW2-1/1/2008RAW3-22/9/2007RAW4-2/2/2008…


Event 3


Event 3

Event tag collctnTag 1 5 0.3Tag 2 2 1.2…

Tag M 8 3.1

Collection SetB -> ππ Candidates (Phy)B -> J/Ψ (μ+ μ-) Candidates…


Future: Data to Metadata

File catalogue holds only a minimal amount of metadata.

LHCb deploys a separate “bookkeeping” database service to store the metadata for datasets and event collections.

Based on central ORACLE server at CERN with query service through XML-RPC interface.

Not scaleable, particularly for Grid, and completely new metadata solution required.

ARDA based system will be investigated.

Vital that this is development is optimised for LHCb and synchronised with data challenges.

Corresponds to ARDA Job Provenance DBand Metadata Catalogue


DIRAC

Metadata: Data Production

Information Flow

Job.xml

•Build newconfiguration•Selection ofDefaults

Productiondone

Prod.Mgr

Configuration

Bookkeeping

Data Production

Production Jobs

File Catalogue


Metadata: Data Analysis

User Job

Information Flow

Job.opts

ModifyDefaults

User

Select input data

Pick-up defaultconfiguration

Bookkeeping

Configuration

File Catalogue

DIRAC


LHCb GANGA StatusAlexander Soroko, Karl Harrison

User Grid Interface.First prototype released in April 2003.To be deployed for LHCb 2004 Data

Challenge.SUCCESS!

LHCb

ATLAS

BaBar+ Alvin TanJanusz Martyniak


GANGA for LHCb

GANGA will allow LHCb user to performstandard analysis tasks:

Data queries.Configuration of jobs, defining the job

splitting/merging strategy.Submitting jobs to the chosen Grid resources.Following the progress of jobs.Retrieval of job output.Job bookkeeping.


GANGA User Interface

Database of Standard Job Options

Job Options Editor

Strategy Database (Splitting scripts)

Strategy SelectionData Selection

(Input/Output Files)

Job Requirements

(LSF Resources, etc)

Job Factory (Job Registry Class)

Ganga Job object Ganga Job object Ganga Job object Ganga Job object Ganga Job object

Local Client

Grid/Batch System Gatekeeper

Submit job

Send job output

Worker nodes

Get

jo

b o

utp

ut

Sen

d

Job script

JDL file

Job Options file

Get

Mo

nit

ori

ng

In

fo

Storage Element

File Transfer


Software Bus User has access to

functionality of Ganga components through GUI and CLI, layered one over the other above a Software Bus

Software Bus itself is a Ganga component implemented in Python

Components used by Ganga fall into 3 categories: Ganga components of

general applicability or Core Components (to right in diagram)

Ganga components providing specialised functionality (to left in diagram)

External components (at bottom in diagram)

User has access to functionality of Ganga components through GUI and CLI, layered one over the other above a Software Bus

Software Bus itself is a Ganga component implemented in Python

Components used by Ganga fall into 3 categories: Ganga components of

general applicability or Core Components (to right in diagram)

Ganga components providing specialised functionality (to left in diagram)

External components (at bottom in diagram)

Job Definition

Job Registry

Job Handling

File Transfer

Python Native

Softw

are Bus

CLI

GUI

Pyth

on

R

oo

t

Gau

di

Pyth

on

PyC

MT

PyA

MI

Py

Mag

da

BaBar Job Definition and

Splitting

Gaudi/Athena Job Options

Editor

Gaudi/Athena Job Definition


GUIs Galore


DIRAC WMS Architecture

Job Receiver

Job Receiver

Job DBJob DB




b qu

eue

Agent 1Agent 1

Agent 2Agent 2

Agent 3Agent 3

LCG CELCG CE

LCG WMSLCG WMS

DIRAC CEDIRAC CE


Productionservice

Productionservice

GANGAGANGA

Commandline UI

Commandline UI

Computing resources

Job Receiver

Job Receiver

Job DBJob DB




b qu

eue

Agent 1Agent 1

Agent 2Agent 2

Agent 3Agent 3

LCG CELCG CE

LCG WMSLCG WMS

DIRAC CEDIRAC CE


Productionservice

Productionservice

GANGAGANGA

Commandline UI

Commandline UI

Computing resources

GANGA


Future Plans

Database ofStandard Job Options

Job-Options Editor

Job-Options Template

Job-OptionsKnowledge Base

Dataset

Dataset Catalogue

Dataset Selection

Job Factory(Machinery for Generating XML Descriptions of Multiple Jobs)

StrategySelection

Job Collection(XML Description)

User Requirements

Database ofJob Requirements

Derived Requirements

Job Requirements

Strategy Database(Splitter Algorithms)

DispatcherScheduler Proxy

Scheduler Service

Remote-Client SchedulerGrid/ Batch-System

SchedulerAgent

(Runs/Validates Job)

Software CacheComponent

Cache

Software/Component Server

Remote Client

Local Client

Execution node

NorduGridLocalDIALDIRACOther

JDL, Classads,

LSF Resources, etc

LSFPBSEDGUSG

Refactorisation of Ganga, with submission on remote client

Refactorisation of Ganga, with submission on remote client

Motivation Ease

integration of external components

Facilitate multi-person, distributed development

Increase customizability/flexibility

Permit GANGA components to be used externally more simple

Motivation Ease

integration of external components

Facilitate multi-person, distributed development

Increase customizability/flexibility

Permit GANGA components to be used externally more simple

2nd GANGA prototype ~ April 2004


Future: GANGADevelop into generic front-end capable of submitting a range of applications to the Grid. Requires central core and modular structure (started with version 2 re-factorisation) to allow new frameworks to be plugged in.

Enable GANGA to be used in complex analysis environment over many years for many users. Hierarchical structure, import/export facility, schema evolution, etc.

Interact with multiple Grids (e.g. LCG, NorduGrid, EGEE…). Needs to keep pace with development of Grid services. Synchronise with ARDA developments.

Interactive analysis? ROOT, PROOF

Date post:	12-Jan-2016
Category:	Documents
Upload:	fionan
View:	31 times
Download:	0 times

LHCb Development

Documents