Status and plans for online installation
LHCb Installation ReviewApril, 12th 2005
Niko Neufeldfor the LHCb Online team
Niko NeufeldCERN, PH
2
Components of the Online System
•Experiment Control System (ECS): run-control and detector control
•Timing and Fast-Control (TFC): timing and trigger distribution
•Data Acquisition (DAQ): data movement, data storage and data processing
•General infrastructure: control room, general purpose (wireless) networking,
Niko NeufeldCERN, PH
3
ECS
•Physical components of the ECS include:– (A few) fibers (DSS, Ethernet connection
SX85-UX85)– Ethernet network (in UX85 and SX85)– Control PCs– CAN and SPECS cables for detector slow
control (in the counting houses and towards the detector), these are covered in “Cabling of the Experiment” by V. Bobillier
Niko NeufeldCERN, PH
4
DAQ
• Physical components of DAQ:– Ethernet network
•Cables from and in D1, D2, D3 (detector electronics)
•Switches in D2
– Event Filter Farm•Racks in D1
•PCs and switches in D1
– Storage system in SX85– Barrack layout:
EDMS 479595, 479597, 479599
D3
D2
D1
Niko NeufeldCERN, PH
5
TFC
•Physical components of TFC:– Electronics modules (number [[FIXME]], 4
types 2 racks: – Fibers from UX to SX via PZ– Fibers from and in D3 barrack (connecting
TFC modules and detector electronics)
•EDMS xxxyyy vz
Niko NeufeldCERN, PH
6
General (online) infrastructure
•Wire-less and wired networking in SX85 and 2889: has been ordered at IT/CS – expected to be ready by end of July 2005
•Workstations in the Control Room: will be installed gradually as needed from September 2005 on
•Central (disk) servers for ECS and DAQ tests: basic infrastructure will be installed by November 2005
Niko NeufeldCERN, PH
7
Fibers (SX to UX)• Fibers from SX85 to UX85 (D2
barrack) in PZ shaft – 24 pairs of Multi-mode fibers
•DSS (profi-bus) connectivity (2 pairs)
•DSS Ethernet (1 pair)
•ECS Ethernet (1 – 2 pairs)
– 12 pairs of Mono-mode fibers•DAQ (1 pair)
•TTC Machine timing signals (2 pairs)
•DSS PLC Synchronization (2 pairs)
• Installation finished by TS/EL/OF
• Easy to add new fibers, if needed
Niko NeufeldCERN, PH
8
TFC Fibers in UX85
•“L0” front-end fibers: long distance (up to ~ 60 m) from D3 to detector front-end: 35, together with bulk of cables / fibers (c.f. V. Bobillier)
•“L1” fibers: short distance (all in the D3 barrack: max 15 m) to detector crates.Will be installed by end of 05 (TS/EL/OF for connections from patchpanel to pp, LHCb for the patch-cords)
Niko NeufeldCERN, PH
9
Ethernet cabling UX85Logical view of D2
barrack
• 1544 cables from D3 (detector) to D2
• 500 Cables between D1 (farm) and D2
• 406 cables internal in D2 (central DAQ and ECS)
DSS(2E08) (2E06)(2E07)(2E09) (2E02)(2E04)(2E05) (2E03)
Optical
uplinks/Fans.
(2C09)
CC-PCSupport(2C07)
CC-PCLinks
(2C08)
Farm Controls(
2C06)
Controls Switch(2C05)
5 x 5x2x8 Ethernet connections towards farm racks in D1
5x 1(+1 spare) Ethernet Link to Storage Network
50x1 Controls Uplinks from D1
Aggr. Switch (2D07)
Spare(2D01)
Patch Panel
(2D06)
Aggr.Switch(2D03)
Read-out Net-work
(2D05)
PatchPanel
(2D04)
Spare(2D02)
Spare(2D09)
StorageHub
(2B07)(2B09) (2B08)
Spare(2C04)
ControlPCs
(2C03)
ControlPCs
(2C01)
ControlPCs
(2C02)
5x 5x8 Ethernet connections towards Readout Network
SFC(2B03)
SFC(2B01)
SFC(2B02)
SFC(2B04)
SFC(2B05)
D1
Spare(2D08)
D3
~2 x 800 linksFrom FE
~20 CC-PC Links from FE
Surface
(2B06)
(2A08) (2A06)(2A07)(2A09) (2A01)(2A02)(2A04)(2A05) (2A03)
(2E01)
Niko NeufeldCERN, PH
10
Ethernet cabling UX85
• Cables for the DAQ and the ECS Ethernet network – Short distances (farm in UX85/D1) allow using cheap
copper infrastructure
• An important part of the Online installation – Total of ~2400 CAT6 UTP cables (4800 patch-panel
connections)– 50 km total of cables– Maximum cable length: 36 m. LHCb is 10 Gigabit over
copper ready– EDMS 497862 v4
• Work in progress: 65% finished, should be ready by 01/05/05– work done by AMEC-SPIE organised by IT/CS (M. Da
Costa & E. Sallaz)
Niko NeufeldCERN, PH
11
Cabling between D2 and D1D1
D2
D1D1
D2
Niko NeufeldCERN, PH
12
Rack installation for Event Filter Farm in D1 barrack
• Old 59U Delphi racks have been refurbished
• Each of the 50 racks will be preinstalled with:– Power bars (6)– Angles (88) (with 2 screws each)– Spacer bars (4)– Rack-cooler door (1)– Ethernet patch-cables (88)
• The PCs will be mounted late as possible (cost!). Lot of work per PC: unpacking, installing, connecting and testing
• Need more manpower for these tasks
Niko NeufeldCERN, PH
13
Preparation of racks
Niko NeufeldCERN, PH
14
Horizontal Cooling of PC racks
Joint project of LHC experiments, pioneered by LHCb75 pieces build to our specifications by CIAT (10 kW cooling capacity)Will be installed from May 05 in Point 8
Niko NeufeldCERN, PH
15
Online System Integration Test: Real Time Trigger
Challenge• Test of a vertical slice of the DAQ and
software-triggers scheduled for June 2005
• Prototype farm together with readout network, event-builders and trigger algorithms
• Installed, cabled and operated in the manner as anticipated for operation in Point 8
• Complete with ECS for run-control (trigger-algorithms) and slow-control (rack-coolers, CPU fans etc…)
• In a second stage can also test TFC and Readout Boards
Niko NeufeldCERN, PH
16
Planning
Niko NeufeldCERN, PH
17
Conclusions
• Installation of LHCb Online system well on track
• Infrastructure for Detector Commissioning ready for September 2005
• All parts of Online system will be tested together in the Real Time Trigger Challenge
• Additional Manpower needed for Event Filter Farm installation
• Many thanks to the Experimental Area team and the IT/CS and TS/EL groups