HEP UCL

Cambridge UniversityCambridge UniversityImperial College LondonImperial College LondonUniversity of ManchesterUniversity of Manchester

Royal Holloway, University of LondonRoyal Holloway, University of LondonUniversity College London University College London

Matthew Warren, UCLMatthew Warren, UCL

20 October 200620 October 2006

EUDET JRA3EUDET JRA3DAQ StatusDAQ Status

20-Oct-2006

EUDET JRA3 DAQ Status 2 HEP UCL

PC/s

OvervieOvervieww

• General DAQ goalsGeneral DAQ goals

• Details of current design and status: Details of current design and status:

1.1.Front-End interface to sub-detector (FE)Front-End interface to sub-detector (FE)

2.2.Data-link (FE to Off-Detector Receiver)Data-link (FE to Off-Detector Receiver)

3.3.Off-Detector Receiver (ODR)Off-Detector Receiver (ODR)

4.4.Control data-link (Clock, Control to FE)Control data-link (Clock, Control to FE)

5.5.Data StoreData Store

6.6.SoftwareSoftware

• SummarySummary

ASICs

FE

Con

trol

-link

ODR

Store

Data-link

20-Oct-2006

EUDET JRA3 DAQ Status 3 HEP UCL

DAQ DAQ GoalsGoals

•Use commercial components where possible:Use commercial components where possible:-Readout links use standard connectors and protocolsReadout links use standard connectors and protocols-PCs with PCI (-Express) cardsPCs with PCI (-Express) cards

•ModularModular-Generic readout board for all usersGeneric readout board for all users-Detector specific interfaces as plug-in modulesDetector specific interfaces as plug-in modules-Other ‘bespoke’ functionality in firmwareOther ‘bespoke’ functionality in firmware

•Front end control attempts commercial hardware tooFront end control attempts commercial hardware too-Extract ‘fast’ signals from commercial signallingExtract ‘fast’ signals from commercial signalling

•DAQ software genericDAQ software generic•Failure protection (fail-over)Failure protection (fail-over)

-PCs not reliable – reroute signals on-the-flyPCs not reliable – reroute signals on-the-fly

20-Oct-2006

EUDET JRA3 DAQ Status 4 HEP UCL

Front End Interface Front End Interface (FE)(FE)

• Interfaces directly with sub-detector ASICsInterfaces directly with sub-detector ASICs- Collects data (and buffers) for transmissionCollects data (and buffers) for transmission- Provides clock+control signalsProvides clock+control signals

•Formats data for specific data link protocol (S-Link, Ethernet, raw Formats data for specific data link protocol (S-Link, Ethernet, raw etc.)etc.)

•FPGA basedFPGA based•Assume plug-in module at edge of detectorAssume plug-in module at edge of detector•Dependent on ASICs/Detector hardwareDependent on ASICs/Detector hardware

•BUT is the FE part of DAQ?BUT is the FE part of DAQ?- For ECAL, UK groups are active: e.g. Cambridge has test FE ...For ECAL, UK groups are active: e.g. Cambridge has test FE ...

Slab

FEFPGA

PHYASIC

Data

ASIC ASICASIC

Conf

Clock + Control

FE

20-Oct-2006

EUDET JRA3 DAQ Status 5 HEP UCL

FE with ECAL ‘Test-FE with ECAL ‘Test-Panel’Panel’ ‘FE’

FPGAs emulate ASIC using same digital part (VHDL)

‘FE’ reads out these ‘chips’

20-Oct-2006

EUDET JRA3 DAQ Status 6 HEP UCL

FE with ECAL ‘Test-FE with ECAL ‘Test-Panel’Panel’

20-Oct-2006

EUDET JRA3 DAQ Status 7 HEP UCL

Data-Data-linklink•This is the physical link (regardless of protocol)This is the physical link (regardless of protocol)

-Both endsBoth ends•Use commercial components and standardsUse commercial components and standards

-E.g. HSSDC2, SFP with Ethernet, S-Link etc.E.g. HSSDC2, SFP with Ethernet, S-Link etc.•Can be copper or fibreCan be copper or fibre•Assume multi-Gigabit ratesAssume multi-Gigabit rates•No-brainer (if it works at all) – first time we see the No-brainer (if it works at all) – first time we see the data is when presented by a FIFO inside the FPGAdata is when presented by a FIFO inside the FPGA

HSSDC2 SFP

20-Oct-2006

EUDET JRA3 DAQ Status 8 HEP UCL

Off-Detector Receiver Off-Detector Receiver (ODR)(ODR)

•PC based, with PCI-Express cardsPC based, with PCI-Express cards•Cards as generic as possibleCards as generic as possible

- Detector specific firmwareDetector specific firmware- Interfaces to data-link via plug-in modulesInterfaces to data-link via plug-in modules

•Can act as an FE – good for testing FE firmware earlyCan act as an FE – good for testing FE firmware early•Provides capability for fast-controlsProvides capability for fast-controls

PC

Mot

herb

oard

PCI-Receiver

Large FPGA

MemoryMicro-

Processor/s

TX Module

RX Module

RX Module

RX Module

RX Module PCIe Interface

CPUInter-Slot

PC

I-E

xpre

ss B

us

Fast Control Interface

20-Oct-2006

EUDET JRA3 DAQ Status 9 HEP UCL

ODR(2) - ODR(2) - Hardware Hardware

Using development board that suits our needs well:Using development board that suits our needs well:

““PLDA XpressFX” (www.plda.com)PLDA XpressFX” (www.plda.com)•PCI-Express endpoint firmware ‘core’ included. PCI-Express endpoint firmware ‘core’ included. •Based on Xilinx Virtex 4 FX100 FPGA (huge + 2xCPU)Based on Xilinx Virtex 4 FX100 FPGA (huge + 2xCPU)•Suitable for ‘Module 0’ DAQSuitable for ‘Module 0’ DAQ

•4x 2.5 Gbit interfaces (bi-dir)4x 2.5 Gbit interfaces (bi-dir)•4 more via plug-ins4 more via plug-ins•128 MByte RAM128 MByte RAM•8 lane PCI-Express8 lane PCI-Express

20-Oct-2006

EUDET JRA3 DAQ Status 10

HEP UCL

ODR(3) - ODR(3) - StatusStatus

• Cambridge, Manchester, RHUL, UCL each Cambridge, Manchester, RHUL, UCL each have a PLDA board (late June).have a PLDA board (late June).

• Each have a hardware development PCEach have a hardware development PC• Basic Linux driver software working (SLC4)Basic Linux driver software working (SLC4)• PCI-Express core workingPCI-Express core working

- Can read/write to the board (flash LEDs!)Can read/write to the board (flash LEDs!)• Ethernet interface is workingEthernet interface is working• Memory controller in progressMemory controller in progress• Complete data transfer system by end of Complete data transfer system by end of

year.year.

20-Oct-2006

EUDET JRA3 DAQ Status 11

HEP UCL

Control-link Control-link (Clock and (Clock and Control)Control)

•Provides timing synchronisation and configuration for Provides timing synchronisation and configuration for FE and ASICsFE and ASICs

•Use data-link hardware:Use data-link hardware:-Attempt to recover clock and timing from linkAttempt to recover clock and timing from link-Good trigger/timing signals too?Good trigger/timing signals too?

•Fall-back: provide discrete interfacesFall-back: provide discrete interfaces-ClockClock-TriggerTrigger-Power-pulsingPower-pulsing

FE Clock extract/ generate

PHYClock+Control+Config FibreClk

Data

Clk2Trig2

LVDS

20-Oct-2006

EUDET JRA3 DAQ Status 12

HEP UCL

Datastore and Datastore and SoftwareSoftware

Keep it simple!Keep it simple!Raw data written to locally mounted/NFS diskRaw data written to locally mounted/NFS diskIf data volume increases – use generic If data volume increases – use generic firmware algorithms to compress data (ZIP)firmware algorithms to compress data (ZIP)Could use LCIO if feasable …Could use LCIO if feasable …

Status:Status:•Data storage software is under-development Data storage software is under-development

- Part of driver Part of driver •Full-scale DAQ:Full-scale DAQ:

-UK meeting in 2 weeks to discuss.UK meeting in 2 weeks to discuss.-Share load with other detectors?Share load with other detectors?

20-Oct-2006

EUDET JRA3 DAQ Status 13

HEP UCL

SummarSummaryy

•Front End: Front End: IN-PROGRESSIN-PROGRESS- FE ‘Test-Panel’FE ‘Test-Panel’

•Off-Detector Receiver: Off-Detector Receiver: IN-PROGRESSIN-PROGRESS - Hardware in place- Hardware in place- Driver working- Driver working- Firmware under-development- Firmware under-development

•Control Link: Control Link: LATERLATER - Can use some ODR firmware.Can use some ODR firmware.- Fall-back solution not difficultFall-back solution not difficult

•DAQ software: DAQ software: TBATBA

Prototype DAQ using ODR’s as source and destination Prototype DAQ using ODR’s as source and destination by the year end.by the year end.