PSI - Jul. 18th, 2007 1

Status of the electronics and DAQ systems of the MEG

experiment

PSI - Jul. 18th, 2007 2

Electronic chain

216

630

ActiveSplitter

1:11:14:1

ActiveSplitter

1:11:14:1

ActiveSplitter

1:11:14:1

Ramp60 120

Pre-Amp8:1

PMT

LXe

HV

PMT

lateral

front

TC

HV

HV

APD

PMTbars

fibers

DC

HV

Wires

Strips

Pre-Amp

Pre-Amp

576

1156

atten

DRSDRS

DRSDRS

DRSDRS

5 crates512

Hit registers

4 boards

TriggerTrigger

Trigger

32

3 crates

Aux. devices

INFN-Le

INFN-Pv

PSI

PSI

INFN-GeINFN-Ge

PSI - Jul. 18th, 2007 3

5 crates

DRSDRS

DRSDRS

DRS

Hit registers

TriggerTrigger

Trigger

3 crates20 MHz

clock

Trigger signalEvent numberTrigger type

TriggerStart

BusyError

E5 area ‘counting room’

PC (Linux)

PC (Linux)

PC (Linux)PC (Linux)PC (Linux)

PC (Linux)

PC (Linux)PC (Linux)PC (Linux)

Front-End PCs

GigabitEthernet

On-line farm

PC (Linux)PC (Linux)

storage

PC (Linux)

Event builder

Type3

1 crate

clockstartstopsync

Ancillarysystem

DAQ and control

Run startRun stopTrigger config

Main DAQ PC PC (Linux)

INFN-Pi

PSIPSI

INFN-Ge

INFN-Pi

INFN-Pi

PSI - Jul. 18th, 2007 4

HV systemActive down regulation of an external HV supply4 different requirements

• Lxe: 1000V , 100 uA• TC bars: 2400V, 1 mA• TC curved: 500V, <1 uA• DC: 2400V, ~1 uA

• 10 chn per board - 180 chn per crate• 24-bit ADCs for high accuracy (20mV)• Read out every 4 seconds of 900 chnCommercial HV supplies installedMass production completedInstallation completed in January ‘07Tests

– laboratory tests– 60 chn for TC ran stably for 2 weeks in

December

The system is ready

PSI - Jul. 18th, 2007 5

Splitter

4÷1 Trigger or DRS

Power

Inputs• Single coaxial

cable (RG178–9m)

• Negligible crosstalk

1÷1 DRS output• high bandwidth• High density

twisted pairs• crosstalk ~ 0.6%

1÷1 trigger or Type3 output Standard twisted

pairs cable

PSI - Jul. 18th, 2007 6

Splitter Summary

Splitter– 16 inputs– 16 output 1÷1 differential 400 MHz– 16 output 1÷1 differential 100 MHz– 4 outputs 4÷1 attenuated differential 100 MHz– Noise <1 mV rms– Cross-talk ~0.6% with LXe signals– Common calibration levels from backplane

Installation– Completed in September 06

Cables– All cables ready at PSI – Only TC cables installed during December 06 run

Test– Satisfactory test during December 06 run

The system is ready

PSI - Jul. 18th, 2007 7

TC APD-preamplifiers

APD pre amplifiers• First prototype with problems on IC and cross talk• Second prototype design and test completed • Mass production and test completed• Assembly and test completed

• System delivery in Aug ’07 (Glued

to the TC fibers)

PSI - Jul. 18th, 2007 8

TC APD hit registers

• Mezzanine board host on the PSI GP-VME boards (the same boards used for DRS)

• Design, production and test completed

• The system consists of 6 VME boards

• System delivery Aug ‘07 (with the detector)

PSI - Jul. 18th, 2007 9

TC Ramp

PMT ramp generator• Design of the final boards completed• Mass production in progress• system delivery 8 boards Aug 07

PSI - Jul. 18th, 2007 10

TriggerBoards

Type1

Front end

Type2

Ancill

PSI - Jul. 18th, 2007 11

Trigger system statusInstallation

- The entire system has been installed in Jul 06. - Active sub-detectors cabled in Nov 06- Pedestal and TC triggers provided during Dec 06 test run- Integration of the trigger and DAQ Nov-Dec 06

The system is ready

Configuration firmware- Baseline Version 3 ready for download Jul 07

• Scaler readout (implemented for Type 1.3 @Run06)• Memory space arrangement (faster read-out)• Revised trigger list

Trigger tree– Increased number DCH wire sum (individual inputs for each

end) – Cosmic ray counters

Analysis software– Base analysis tools available– Trigger parameters evaluation and download, under

development– Monitoring and efficiency evaluation tools

PSI - Jul. 18th, 2007 12

. . . 14 boards

14 x 48Type1Type1

Type116

4

LXe front face

(216 PMTs)

2 boards

. . . 5+5+2 boards

9 x 48

Type1Type1

Type116

4

LXe lateral facesback (216 PMTs) 4 in 1

lat. (144x2 PMTs) 4 in 1 up/down

(54x2 PMTs) 4 in 1

1 board

Timing counterscurved (512 APDs) 8 in 1

bars (30x2 PMTs)1 board

1 board

2 x48

Type2

Type2

Type2

Type2

Type2

. . . 9 boards

9 x 48Type1Type1

Type116

4

4 boards

2 x 48Type1Type1

16

4

16Drift chambers64 channels

2 x48

1 x48

Type12 x48

16Auxiliary devices16 channels

The trigger tree

Type2

4 x 48

Type1

16CR counters32 channels Type1

16

PSI - Jul. 18th, 2007 13

Trigger list (1)QTHQTL

MeV

DWW

DWN

LXe charge e+- direction e+- timing

trig.# name conditions

0 MEG QSUM > QTH && D < DN && |T| < TWN

1 MEG-Q QSUM > QTL && D < DN && |T| < TWN

2 MEG-D QSUM > QTH && D < DW && |T| < TWN

3 MEG-T QSUM > QTH && D < DN && |T| < TWW

4 RD-narrow QSUM > QTL && |T| < TWN

5 RD-wide QSUM > QTL && |T| < TWW

PSI - Jul. 18th, 2007 14

Trigger list (2)

trig.# name conditions

6 0 QSUM > QTH && QNaI > QN && |T| < TWN

7 0-NaI QSUM > QTL && |T| < TWN

8 NaI QNaI > QN 9 LXe-high QSUM > QTH

10 LXe-low QSUM > QTL

11 CW QTL < QSUM < QTH

12 neutron Qpatch < QTpatch && Qi< QTi

13 pulse-shape && QSUM < QTL

&& QWIRE> QTWIRE

14 laser QLAS > QTLAS

15 led

PSI - Jul. 18th, 2007 15

Trigger list (3)

trig.# name conditions

16 michel NDC 4 && QL > QT && QR > QT && (QL+QR) > QTS

17 DC track out NDC 4 && Iout

18 DC track NDC 4

19 DC + CR single DC && CR

20 DC

21 CR

22 TC QL > QT && QR > QT && (QL+QR) > QTS

31 pedestal internally generated random triggers

PSI - Jul. 18th, 2007 16

Cyclic buffers

Diff. driver FADC

Cyc.buff

Proces. Algor.

LVDS Tx

Cyc.buffType1 FPGA

Cyc.buff

Proces. Algor.

LVDS Tx

Cyc.buffType2 FPGA

LVDS Rx

Cyc.buff

Proces. Algor.

LVDS Tx

Cyc.buffType2 FPGA

LVDS Rx

Type1 layer

Type2 layer

Final Type2

Analog inputs

Trigger output

PSI - Jul. 18th, 2007 17

Monitoring: TRG Waveform

• 512 channels (5120 ns)

• 10 FADC bit data

• Range 0 V --> 1V

• 20 MHz Bandwidth

• Typical pulse 400 mV

• Baseline Fluctuation 1 mV (s 0.4 mV)

T (ns)

T (ns)

PSI - Jul. 18th, 2007 18

Monitoring: system operation

Periodic monitor of the system parameters performed automatically in the DAQ

• Clock locking • Synchronous operation • LVDS data transmission • Computer busy• Rates (trigger, channel)

Automatic alarms in case of failure

PSI - Jul. 18th, 2007 19

Detector monitoring: ratesRate of individual

channels (independent of trigger)

PSI - Jul. 18th, 2007 20

Efficiency: tools

1. Cyclic buffers- dump-mode used to take 5us depth snapshot of the trigger status

for monitoring, debugging and efficiency evaluation(PMT signals, physical quantities estimators: charge, time,

amplitude, position, patterns …)- source-mode used to process simulated events as well real

recorded events

2. Trigger – Prescaled unbiased triggers simultaneously acquired with meg

events

3. Algorithm emulators– Simulation of FPGA algorithms with Xilinx tool intensively used– Emulation by means of c++ code in progress

4. Simulated data– Meg events– Calibration events

PSI - Jul. 18th, 2007 21

DRS2

PSI GVME Board

FPGA with2 Power-PC

Mezzanine boards

PSI - Jul. 18th, 2007 22

DRS System

Optical fiberBack-end PC

Off-line cluster

Ethernet

Front-end PC

All channels equipped with DRS2

• 2636 channels

• 1024 cells per channel

• 0.5 - 2.5 GHz sampling speed

•After calibration pedestal noise at 0.5 mV RMS

PSI - Jul. 18th, 2007 23

DRS2 online calibration

mV

ADC counts / 10

The pedestal is dependent on the cell number

• Need of individual pedestal value for each bin

The non-linear response function depends on the cell number

• Need of different response functions for each bin

– Measure Vin – ADCout characteristics with precise DC power supply at the DRS2 input

– Fit and store parameters online– Write on disc linearized, pedestal

subtracted samples

PSI - Jul. 18th, 2007 24

DRS3 status

50 prototypes of DRS3 available for test

PSI - Jul. 18th, 2007 25

DRS3 linearity

DRS3 linearity

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Uin [V]

Uo

ut

[V]

Uin (V)

Uou

t (V

)

PSI - Jul. 18th, 2007 26

DRS3 nonlinearity

Point deviation from linear interpolation for five arbitrary cells

30 deg. C30 deg. C

50 deg. C50 deg. C

• Integral nonlinearity is below 1mV with only one offset per cell used for correction

• Output changes by ~1mV in 20 deg. C --> Tc = 50 ppm

PSI - Jul. 18th, 2007 27

DRS3 summary

• No dangerous temperature instability• Readout speed increased 16 MHz (DRS2) -> 33 MHz• “Region Of Interest” – Readout mode works• Master clock signal (LVDS) can be digitized differentially

improvement in clock signal• Plan:

– VME boards with DRS3 test in July 2007 – Order engineering run after all tests have been finished– Get chips in ~December, but not in time for this year’s

beam time

PSI - Jul. 18th, 2007 28

Auxiliary digitizer:Type3

Modified Type1 boards to produce an auxiliary digitization of the LXe signals

PSI - Jul. 18th, 2007 29

Auxiliary digitizer status

• Higher input capability (32 channels), no LVDS transmission• 612 PMTs on LXe lateral+back sides 20 boards• Prototype tests: Jan 07

– Bit-stream downloading through VME

– Control signal (CLK, SYNC, START, STOP) distribution – FADC digitization and storage – RAM readout

• Production: Apr 07• Tests: May 07

• Installation: Aug 07

PSI - Jul. 18th, 2007 30

DAQ

80 GB System DiskRAID 1 (Mirror)

/home/meg

1.2 TG Data DiskRAID 5

80 GB System DiskRAID 1 (Mirror)

80 GB System DiskRAID 1 (Mirror)

Back-End

Front-End #1

Front-End #2

. . .NFS

VME-Interface

VME-Interface

SC-FE

Gig

abit

Sw

itch

PSI - Jul. 18th, 2007 31

Multi-threading model

VMETransferThread

CalibrationThread

CalibrationThread

CalibrationThread

CalibrationThread

CollectorThread

VME

Round-Robindistribution

Network

Zero-copy ring

buffers

Zero-copy ring

buffers

PSI - Jul. 18th, 2007 32

DRS Readout rateOptimal readout rate of DRS full waveforms with 4

calibration threads: 30 events/sDuring Dec 06 run max readout at 7 events/s

– Double event readout– Code optimization– Single calibration thread

PSI - Jul. 18th, 2007 33

DRS Readout rate

Assuming 50% occupancy Zero suppression done on the front-endMax transfer rate at 50 events/s

– Hits the VME transfer speedMuch larger than the expected maximum trigger rate

of 20 events/s

PSI - Jul. 18th, 2007 34

Trigger readout rate

• Type 3 boards (32 ch/board)– Standard 2EVME A32, D64 Readout

– For 1 cycle TAS 650 s/board ( v 50 MB/s)

– 20 boards/crate: 70 events/s (%Live) 80% @ trigger rate = 20 s-1

• Type 1 (16 ch/board) & Type 2 (18 ch/boards)– Custom 2EVME A32, D32 Readout (no access to A31-A0 during data broadcast)

– For 1 cycle TAS 1300 s/board ( v 25 MB/s)

– 20 boards/crate: 60 events/s (%Live) 75% @ trigger rate = 20 s-1

PSI - Jul. 18th, 2007 35

Higher DAQ rate

Higher rate is considered only for calibration events

• 50 Hz full waveform readout hits VME transfer limit (83 MB/sec)

• VME transfer size can be reduced by doing zero-suppression and ADC/TDC analysis on VME side

– Use embedded Power-PC CPUs (C-code)– Use FPGA (VHDL-code)

• Tools– Basic zero-suppression in VME in late 2007– ADC/TDC analysis in VME later in summer, need input

from sub-detector groups– Possibility of reaching 100 Hz

PSI - Jul. 18th, 2007 36

DAQ rate vs. amount of data

• DAQ speed is not a limiting factor• The total data size needs solution:

– 30 Hz is maximal VME speed for full waveforms → >270 MB/sec

– Data transmission limit is 20 MB/sec (=250TB/year)→ need online reduction 10x

– Storage limit is 30 TB/year→ need offline reduction 8x

PSI - Jul. 18th, 2007 37

Online data reduction

The factor 10 data size reduction obtained within the DAQ system:

• Level 3 trigger in the Event Builder task:– Fast linear fit of the LXe energy with trigger wfms – Presence of an e+ with a minimal momentum using

DC information

• Waveform data compression– Zero suppression– ADC/TDC like data for calibration– Waveform rebinning

PSI - Jul. 18th, 2007 38

Data size reductionPossible algorithms for data size

reduction• Zero suppression: hit if

max.ampl. > n x (baseline)

• Readout window at the trigger time

• Pile-up flag: Zero-crossings of first derivation

• Re-binning of signal tail4:1, 8:1

• ADC for calibration events: Numerical integration of signals over baseline

0.5 ns bins 4 ns bins

PSI - Jul. 18th, 2007 39

Conclusions

• Splitters: installed, operational, expected performances, test in Dec 06

• Fiber preamp: problem with an IC fixed, test passed, mounted on the TC detector, installation in Aug 07

• Hit registers: mezzanine boards produced, FPGA firmware ready (PSI GPVME board), installation in Aug 07

• Trigger: installed, operational, built-in debugging and control tool, need tuning after detector turn on, test in Dec 06

PSI - Jul. 18th, 2007 40

Conclusions • DRS2: installed, operational, good for timing,

temperature dependence, usable with DC• DRS3: prototype test phase, final solution, not

available in 2007• Aux digitizer: production problem solved, prototype

test completed successfully, ready for 2007 run • DAQ: installed, operational, good performances, test in

Dec 06 run

The electronics and the DAQ systems are expected to be ready for the 2007 run

PSI - Jul. 18th, 2007 41

Rack space

PSI - Jul. 18th, 2007 42

Trigger reminder

Digital approach – Flash analog-to-digital converters (FADC)– Field programmable gate array (FPGA)

trigger observables

– energy, direction and time (Lxe calorimeter)

– e+ time and approx. direction (Timing Counters)

Expected rate – For 108 muon stop rate

e

1-s 20 4

ff

TRfRRe

PSI - Jul. 18th, 2007 43

All triggers can be:• masked• prescaled (up to 32 bits)

Trigger features

Other information

Type 1• 100 MHz, 5 s depth, 10 bits, waveforms for all channels

• Single rate for each channel

Type 2• Rate for each trigger type

• Event Counter (hardware distributed to the DRS boards)

• Trigger pattern (hardware distributed to the DRS boards)

• Live Time and Dead Time

PSI - Jul. 18th, 2007 44

Trigger efficiency: example

events associated with the TRG STOP

( ~1s delay)

s s

low intensity

(slit = 10%)

high intensity

(slit = 100%)

– Extract pulses for

– “Trigger” events

– “Unbiased” events (out of trigger window and with 10 mV threshold)

PSI - Jul. 18th, 2007 45

Trigger efficiency: TC charge

UnbiasedTrigger

Npe

Landau peak for e+

Secondaryparticles

rescaled by

RpulseT

= NT/NU

almost full efficiency at Landau peak (~6MeV, ~500pe)

Interpolation by erf function

PSI - Jul. 18th, 2007 46

DRS2 primer

0.2-1 ns Inverter (“Domino”) chain Rotatingsignal

Input 1

Output 1

16 MHz

Input 2

Channels3 to 10

Analog

outputAnalog switch

Need of external buffer and FADC

PSI - Jul. 18th, 2007 47

DRS2 Temperature Dependence

T [º C]

Vout [V]

Vout vs. Temperature

y = -0.0139x + 1.3938

0.8

0.85

0.9

0.95

1

1.05

20 25 30 35 40 45

DRS2 has a marked dependence on the temperatureDRS2 has a marked dependence on the temperature

Tc ~ 1.4 % / ºCTc ~ 1.4 % / ºC