Yifan Yang--- DAQ Hardware for LS1 1
DAQ Hardware For LS1
IIHE-ULBGilles De Lentdecker Yifan Yang
Erik Verhagen Michael Korntheuer
Yifan Yang--- DAQ Hardware for LS1 2
Overview
• Off-detector electronics • microTCA environment introduction • Brussels activities
• Opto Hybrid FPGA Board• Challenges and possible solution• Prototype board plan
• Summary
Yifan Yang--- DAQ Hardware for LS1 3
Overview
• Off-detector electronics • microTCA environment introduction • Brussels activities
• Opto Hybrid FPGA Board• Challenges and possible solution• Prototype board plan
• Summary
Yifan Yang--- DAQ Hardware for LS1 4
GLIB GLIB GLIB
GE2/1Reference plane
GE1/1Superchamber
AMC13/GLIB
Port 8-11 PCIe or customized protocol(GLIBs to AMC13)
System overview
GLIB
15 Column5 FPGA boards4 Glib 1 AMC13
Yifan Yang--- DAQ Hardware for LS1 5
Overview
• Off-detector electronics • microTCA environment introduction • Brussels activities
• Opto Hybrid FPGA Board• Challenges and possible solution• Prototype board plan
• Summary
Yifan Yang--- DAQ Hardware for LS1 6
uTCA developments• We already got :
– 1 AMC13– 1 GLIB v2
• We should get soon:– 6 GLIBs v3
• In Brussels lab:– 3 uTCA crates (2 NAT & 1 Vadatech VT892)– Already built two AMC (w/o FPGA)– A mezzanine with 2 FPGAs– Already some experience with uTCA world (MCH, MMC, IPMI, etc.)– These developments are also valuable for LS1
Yifan Yang--- DAQ Hardware for LS1 7
Overview
• Off-detector electronics • microTCA environment introduction • Brussels activities
• Opto Hybrid FPGA Board• Challenges and possible solution• Prototype board plan
• Summary
Yifan Yang--- DAQ Hardware for LS1 8
@3.2 Gbps
Glib
μTCA Crate (12 slots)
Eta partitions
123.....
ColumnsA B C
GEM chamber segmentationVFAT2
AMC13
Δϕ=10°
To CSC systemGlib
Opto Hybrid FPGA Board
Yifan Yang--- DAQ Hardware for LS1 9
Space limitation
10
Large number of signals
MOLC 200 4 ways-- potentially more layers are neededPanasonic 160 2 ways-- more signals needed to be shared and 24 VFAT2
Yifan Yang--- DAQ Hardware for LS1
Yifan Yang--- DAQ Hardware for LS1 11
Large number of signals
• VFAT2 from 30 to 24, S bits and Dataout differential signals decrease to 432
• MCLK,T1,RSTb are shared by column instead of channel, decrease to 15
• I2C and DACO_V/I keep at 24• 471 signals in total can be fit into 3 Panasonic
160 2 ways connectors.• By using SN65LVDS386(16 channel LVDS receiver)
,signals on FPGA can be reduced to 249.
Yifan Yang--- DAQ Hardware for LS1 12
High speed link
• As GBT Emulator, 4.8 Gbps transceiver is required.
• GTX--- 6.6Gbps only in Virtex 6 • GTP--- 3.2Gbps in Virtex 5 and spartan 6• By using Virtex 6, part of the CSC system
firmware can be reused and also possible for emulating GBT link which benefit LS2 design
Yifan Yang--- DAQ Hardware for LS1 13
Possible FPGA
Embedded XADC is another advantage
Yifan Yang--- DAQ Hardware for LS1 14
Power and cooling
• 1.5V 4A• 3.3V 3A• 15.9W in total• Cooling by attaching metal to air pipe
Yifan Yang--- DAQ Hardware for LS1 15
LHC clock and synchronization
• Copper cable is the easiest way to get 40.08MHz LHC clock but too far away
• TTC system has to be used• TTCrx+QPLL• Command decoder on opto hybrid • Fixed latency transfer to CSC and Glib
Radiation environment – Snap12 Transmitter: < 1 SEU per year per link– Snap12 Receiver: ~1 SEU per week per link• These typically just affect a single data word
– Finisar Optical Transceiver: ~7 SEU/day/link• Typically just affects a single data word• Low rate, less than one error in 3 *1013 bits
– FPGA GTX Transceivers: ~3 SEU/year/link– FPGA Block RAMs: ~9 SEU/day/chip• These typically affect a single bit in a single cell• Need to investigate mitigation for FPGA BRAMs
– FPGA CLBs: ~6 SEU/day/chip• Need to investigate mitigation for FPGA CLBs
Yifan Yang--- DAQ Hardware for LS1 16
Yifan Yang--- DAQ Hardware for LS1 17
Firmware design
Yifan Yang--- DAQ Hardware for LS1 18
Firmware design
• Trigger data readout • Tracking data readout• VFAT2 control• TTC system • Slow control• SEU control
Yifan Yang--- DAQ Hardware for LS1 19
Overview
• Off-detector electronics • microTCA environment introduction • Brussels activities
• Opto Hybrid FPGA Board• Challenges and possible solution• Prototype board plan
• Summary
Yifan Yang--- DAQ Hardware for LS1 20
Prototype board plan
Evaluate connector signal integrity and other component(TTCrx,QPLL)Estimate logic resource usage for full system and 3.2Gbps connectionTry refreshing firmware and maybe other mitigation methods
Yifan Yang--- DAQ Hardware for LS1 21
Time estimate• Schematic design before April (away for two weeks in February ) from
now 1.5 month• PCB design before mid of May 1.5 month• Assembling before June 0.5 month• Firmware design• Reuse RPC design for TTC system 1.5 month • Reuse TOTEM design for VFTA2 control 1 month• Reuse CSC design for trigger data readout 1.5 month• Slow control and SEU control 1 month• Integration of all function module 1 month
• Trigger and tracking data receiver 1 month• Segment trigger algorithm implementation 2 month• Communication with AMC13 1 month
• DAQ Software design
Opto Hybrid board part
UTCA part
Yifan Yang--- DAQ Hardware for LS1 22
Overview
• Off-detector electronics • microTCA environment introduction • Brussels activities
• Opto Hybrid FPGA Board• Challenges and possible solution• Prototype board plan
• Summary
Yifan Yang--- DAQ Hardware for LS1 23
summary
• Study is ongoing• Prototype design is ongoing• Communication is very important• Suggestions are welcome
Yifan Yang--- DAQ Hardware for LS1 24
Thank you!
Yifan Yang--- DAQ Hardware for LS1 25
Backup
Yifan Yang--- DAQ Hardware for LS1 26
microTCA• Little brother of ATCA (Advanced Telecom-
munications Computing Architecture)– the largest specification effort in the history of the PCI
Industrial Computer Manufacturers Group (PICMG)– Offers a flexible, high density, high performance
backplane based on serial standards used today (GbE, PCIe, …)
– Large effort in CMS to replace VME by microTCA for upgrades
Yifan Yang--- DAQ Hardware for LS1 27
AMC
• Advanced Mezzanine Card (AMC)• microTCA is based on the AMC standard initially
developed for the ATCA cards.• Up to 12 AMC cards can be inserted into the microTCA
backplane• The MCH (MicroTCA Carrier Hub) provides the
connectivity between slots – Provides also crate management functions
• The system can operate in redundant mode with a second MCH
Yifan Yang--- DAQ Hardware for LS1 28
GLIB (see S. Haas talk https://indico.cern.ch/conferenceDisplay.py?confId=182546)
• An evaluation AMC card for bench-top operation or use in a uTCA crate
• Expandable via FMC, two mezzanine slots
4 SFP+To uTCAbackplane
AMC features:Mid-size double-width AMC4 SFP+ transceiver modulesVirtex-6 FPGA with twenty 6.5Gbps transceiversI/O capability can be further en-hanced with 2 FPGA MezzaninesGigabit Ethernet link to PC for bench-top operation
Yifan Yang--- DAQ Hardware for LS1 29
AMC13 (see E. Hazen talk https://indico.cern.ch/conferenceDisplay.py?confId=182546)
• It is not an MCH! It is a 13th AMC in MCH-2 slot(as permitted by MicroTCA standard)
• It distributes LHC clock / timing / controls to AMCs• It collects DAQ data from AMCs• It provides standard interface to CMS subdetectors:– CMS DAQ via optical fibers (currently 2 at ~ 5Gb/s)– TTC via 1300nm fiber @ 160Mb/sec biphase mark code
• Future TTC upgrade may be supported via spare SFP site– TTS via 1300nm fiber with protocol t.b.d.
Yifan Yang--- DAQ Hardware for LS1 30
AMC13 (HCAL example)