JOINT INSTITUTE FOR NUCLEAR RESEARCH

 

OFF-LINE DATA PROCESSING GRID-SYSTEM MODELLING FOR NICA

1

Nechaevskiy A.

Dubna, 2012

AGENDA

NICA off-line data processing parameters Tasks for simulation Simulation platform choice Model efficiency estimation First results Conclusion

2

DATA PROCESSING SCHEMA FOR NICA MPD

3

NICA’s data flow parameters:• high speed of the events generation (to 6 KHZ),• in the central collision of Au-Au about 1000 particles are formed,• the size of the file with modelled information from detectors for 100000 events occupies about 5 TB.

№ Parameter Value1 Speed of data collection from all detector’s

components4.7 GB/s

2 Duration of the set of statistics period within a year

120 days

3 Frequency of the event emergence on an installation exit

6 KHz

4 Dead time after event emergence 1 cicle (50%)5 Average of tracks in an event 5006 Average of particles collisions 207 Average of bytes on each collision 458 Average time of event's reconstruction on the

processor in capacity 1КSI2K2 s.

MPD parameters

SOURCE DATA

4

№ Requirements Value

1 Quantity of events to processing in a year 1.87 е10

2 Total data volume to storage in a year 8,4 PB

3 Total Disk space in case storage is RAID6 (+25%) in a year 10 PB

4 Total CPUs in grid structure, minimum necessary for data recovery with the speed equal to a set of events, proceeding from 7000 thousand astronomical clock of work a year

1480

5 Numbers of grid sites 20

6 Minimum of Data transfer speed from JINR to Sites 2,5 Gb/s

The specification of requirements to NICA experiment off-line data processing

The expected number of data processed events is about 19 billions. If data transfer speed from sensors is 4.7 GB/s, the total amount of source data can be estimated as 30 PB annually, or 8.4 PB after processing.

GRID FOR EXPERIMENTS

5

Hierarchical grid infrastructure with some computing centers Tier 0/1/2 already used in ALICE experiment and others. PANDA experiment wants to use it also.

Questions For Simulation

• Grid Infrastructure Architecture?• Number Resource centers?• Amount of the Resources?• Capacity of the network?• Resource distribution between users groups?• etc.

Urgency

Recommendation and specification for NICA grid infrastructure creation

SIMULATION TASKS

6

Task 1.

Task 2.

GRIDSIM SIMULATION PACKAGE

7

•Allows to simulate various classes of heterogeneous resources, users, applications and brokers • There are no restrictions on jobs number which can be sent on a resource; • Capacity of a network between resources can be set;• System supports simulation of statistical and dynamic schedulers;• Statistics of all or the chosen operations can be registered• Implemented in Java• Configuration files are used to set simulation’s parameters • Source code is available• A lot of examples of the GridSim using• Multilevel architecture allows to add new components easily

http://www.gridbus.org/gridsim/

GridSim Architecture

MODEL EFFICIENCY ESTIMATION

8

Parameters of the model efficiency:a) Average network loading by days [%]b) Numbers of the running /waiting jobsc) Number of uses CPUsd)Total Data transfers in hours [GB] e)Total Storage uses [%] f) Cluster uses [%]j) Refused CPUs [%]

MODEL COMPONENTS

1. User Interface (edit/add model)

2. MySQL database to save simulation parameters

3. Simulation System

4. Results Visualization Tools

9

10

TEST SIMULATION

Clusters: 1 Machine 2 CPUsUsers: 1Jobs: 10

EXAMPLE OF GRAPHIC REPRESENTATION OF THE SIMULATION RESULTS

11

1. Waiting and Running Jobs

2. Average Clusters Usage

DONE!

The web interface of the model editing with one test scenario of the grid work is created

key parameters of the model estimate are allocated; Results visualization tools are created; Simulation passed debugging and verification phase.

12

CONCLUSION

13

to estimate some architectures (parameters) of the data processing system by changing entrance data only;

library of scenarios (Data processing, architectures, other) will allow to compare various technical solutions and to choose optimum.

The model will allow :

― the user interface development;

― debugging the model in client-server architecture

― development of a scenarios sets of grid systems work

― user’s editing and adding grid model parameters

Plans:

14

Questions?