NSW Electronics Design Reviews, February 2015 1
Overview of NSW trigger electronics
Lorne Levinson
NSW Electronics Design Reviews, February 2015
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 2
NSW trigger concept• Phase I upgrade: Increased backgrounds, but must maintain existing trigger rate• Filter “Big Wheel” muon candidates to remove tracks that are not from the IP
Only track “A” should be a trigger candidate: pointing: D < 7.5mrad• Challenge is latency: 500nsec for electronics + 500ns fibres to be in time for Big Wheel• Micromegas: 2M strips, 0.4mm• sTGC: 280K strips (3.2mm), 45K pads, 28K wires• sTGC, MM find candidates
independently, list merged for Sector Logic
• Hit per layer:sTGC: hit is centroid of 3-5 stripsMicromegas:hit is address of strip
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 3
Present Small Wheel – defines basic layout and envelopes• 16 detector layers in total• 2 technologies, MicroMegas and sTGC
Micro-Mesh Gaseous detectors (Micromegas): primary precision tracker • Space resolution < 100 μm
independent of track incidence angle • Good track separation due to small
0.5 mm readout granularity (strips) • Excellent high rate capability due to
small gas amplification region and small space charge effects
sTGCs: primary trigger detector • Bunch ID with good timing resolution
– additional suppression of fakes • Good space resolution providing track
vectors with < 1 mrad angular resolution
• Based on proven TGC technology; Pads & strips, instead of only strips as in current detector
Lorne Levinson, Overview of NSW trigger electronics
Detector layout
9.3m
NSW Electronics Design Reviews, February 2015 4Lorne Levinson, Overview of NSW trigger electronics
VMM, TDS, ART, ROC Are ASICs (IBM 130nm)
MM & sTGC versions:different cable to E-link mapping
NSW Electronics Design Reviews, February 2015 5Lorne Levinson, Overview of NSW trigger electronics
VMM, TDS, ART, ROC ASICsPlanned for IBM 130nm
Front end boards
MM & sTGC versions:different cable to E-link mapping
NSW Electronics Design Reviews, February 2015 6
Off- detector architecture with FELIX
• Functions can now be separated, physically, or just logically.
• “ROD” = Event fragment builder/processor for ATLAS central Read Out System
L1DDC
Readout Controller
VMMs
Lorne Levinson, Overview of NSW trigger electronics
“ROD”
……………..
• coarse time counter: 12-bit Gray-code • test pulse generator: 10-bit adjustable • coarse threshold generator: 10-bit adjustable• temperature sensor: ~ 725 mV - 1.85 mV/°C• configuration registers: 80-bit + 24-bit / channel• PROMPT (courtesy of CERN): export regulations (ITAR) compliance circuit
analog1
data1data2
analog2
CA shaper
logic
orneighbor
addr.
6-b ADCflg 1-bitthr 1-bit
addr 6-bitampl 10-bittime 8-bitBC 12-bit
ART (flag + serial address)ART clock
12-b BC
Gray count
10-b ADC
8-b ADC
BC clocklogic
time
peak
VMM2 Front end ASIC architecture
4XFIFO
data/TGC clock
mux
tk clockpulser
trim
bias registerstp clock
TGC out (ToT, TtP, PtT, PtP, 6bADC)
tempDAC reset
test
64 channels
analog mon.
• technology: IBM CMOS 130nm• power dissipation: 4-8+ mV/ch. • transistor count: > 5 million
prompt
7
ATLAS Muon NSW Trigger 8
VMM3 and Readout Controller ASICs• VMM2 has only simple, non-pipelined readout;
needs ATLAS L0, L1 pipelines and collecting all hits belonging to the trigger• RO Controller interfaces VMM3 to the ATLAS GBTx of the readout system.• Collect data from up to 8 VMMs to one Read out Controller
– one ROC per Front end board• Architecture:
– VMM keeps hits until Level-0 Accept; – RO Controller keeps hits until Level-1 Accept, formats output
• GBT: aggregates many 80, 160 or 320 Mb/s “E-links” to one 3.2Gb/s fibre– Aggregate several readout companions into one GBT– One GBT per layer on each detector edge 32 per 1/16th sector– GBT is bi-directional:
other direction carries Trigger, Timing & Control (TTC) data– One bi-dir E-link per Front End board for configuring ASICs and
Detector Control system: monitoring temperatures and voltagesLorne Levinson, Hong Kong, 31 May 2014
sTGC strips: one output per channel• 6-bit flash ADC of strip signal peak
charge; serial output at 160Mb/s• Centroid of 3 to 5 3.2mm strips
gives track coordinate in a layer
sTGC pads: one output per channel• 10nsec pulse at peak, or use
leading edge of Time-over-Threshold
• Coincidence in 8-layer tower of pads chooses which strips to transfer to centroid finder
Front end outputs for trigger path
Lorne Levinson, Overview of NSW trigger electronics 9NSW Electronics Design Reviews, February 2015
Micromegas: one output per 64-channel chip
• Address in Real Time (ART) of first (in time) strip hit in each BC per chip
• Address of the 0.4mm strip gives track coordinate
• “mTPC” mode
probably 1st
sTCG
sTCG
sTCG
sTCG
sTGC trigger scheme
10
On-chamberASICS
On Rim of NSWFPGAs
USA 15
Pad Trigger
StripTDS
PadTDSPad
VMM
sTGCTrigger
Processor
StripVMM
sTCG
sTCG
sTCG
sTCG
StripVMM
Pad trigger uses pad tower coincidence to choose ONLY the relevant band of strips.
Physical pads staggered by ½ pad in both directions
Logical pad-tower defined by projecting from 8 layers of staggered pad boundaries
Pad-tower coincidence = 2 3-out-4 overlapping pads
Router
1/16th sector
Problem: no BW to read all strips
StripTDS
Only oneStrip TDSchosen
Lorne Levinson, Overview of NSW trigger electronics NSW Electronics Design Reviews, February 2015
11Lorne Levinson, Overview of NSW trigger electronics NSW Electronics Design Reviews, February 2015
Strip Trigger Data Serializer ASIC• Prepare strip trigger data to send to centroid finder:
– Use pad tower ID from Pad Trigger to select a band of strips– serialize the FADC values of the band of strips for transmission to the
Router board on the rim• BCID, as determined by the Pad Trigger, is appended to the data
path to Trigger Processor has fixed latency, but is not sync’ed to BC at every step. Sync’ed to BC only on output to Sector Logic
• Rad tol, SEU mitigation, IBM 130nm• 1st TDS prototype delivered, being tested• Requires 5G serial output over twinax to Router
– Serializer modified from GBT and – 5G serializer core submitted Feb 2014: ~1mm2, 300 mW, 1.5 V, – Successful test results were presented in July
12Lorne Levinson, Overview of NSW trigger electronics NSW Electronics Design Reviews, February 2015
sTGC Pad Trigger Data Serializer ASIC• A second mode of the strip TDS ASIC• Assigns each pad signal to a bunch crossing
– 1 bit per pad per BC• Serialize and transmit up to 104 pads in 1 BC to Pad Trigger on rim of wheel.
• Uses same 5G serializer as for strips• Rad tol, SEU mitigation, IBM 130nm
Lorne Levinson, Overview of NSW trigger electronics 13NSW Electronics Design Reviews, February 2015
• One per 1/16th sector, on rim of wheel
• Receives: Pad signals synchronized to BC clock
• Builds tower coincidences from 3-out-4 coincidences in the two quads
• Identifies which bands of strips should be read out to the centroid logic• Priority encode bands so there is at most four (was three)
coincidences per sector• Send band ID, -ID, BCID for each candidate to strip TDS
• Provides the BCID tagging of the strip trigger data
• FPGA, so it can be programmable, butOn periphery of wheel, so SEU mitigation still required
• Readout on Level-1 Accept, but also must report monitoring of non-triggering BCs.
• Being prototyped with Xilinx Kintex
sTGC Pad trigger
Lorne Levinson, Overview of NSW trigger electronics 14NSW Electronics Design Reviews, February 2015
NSW Electronics Design Reviews, February 2015 15
sTGC Router• Routes strip hits from strip TDS to Trigger Processor in USA15• One Router per layer per sector• 5G serial inputs from several TDS ASICs• Sends output from up to 4 TDS ASICs (i.e. 4 Pad Tower triggers) in each
bunch crossing and routes that them to Trigger Processor in USA15– Up to 4 (was 3) output fibres per Router to USA15– Will use FPGA serializer/deserialize, not GBTx
• Reduces latency by discovering which inputs are active before the whole frame is received and sets up routing for that link (“cut-through routing”)
• On periphery of wheel, but SEU mitigation still required• Being prototyped with Xilinx Artix• Option to send output at up to 6.6G (Artix limit) • Uses repeaters to regenerate serial inputs
Lorne Levinson, Overview of NSW trigger electronics
Lorne Levinson, Overview of NSW trigger electronics
sTCG
sTCG
sTCG
sTCG
Micromegas trigger scheme
16
On-chamberASICS
USA 15
ARTVMMStrip
MMTrigger
Processor
VMMStrip
sTCG
sTCG
sTCG
MIcRomegas
VMMStrip
• Each VMMs send address of its first hit in each BC to an ART ASIC.64 channels = 2.6cm
• Use the coordinate of the center of the strip for the slope calculation
• ART chooses up to 8 addresses from 32 VMMs to send
• Less accurate than sTGC centroidSimpler architecture
• Plan to use GBT for transmission
1/16th sector
NSW Electronics Design Reviews, February 2015
Micromegas trigger “ART” ASIC• 32 serial ART inputs from 32 VMM front-end chips
chooses up to 8 hits for transmission to Trigger Processor via GBT• Rad tol, SEU mitigation, IBM 130nm• 1st prototype received, 4mm x 2.85 mm
Lorne Levinson, Overview of NSW trigger electronics 17
1024 chips:32 per 1/16th sector4 per layer
NSW Electronics Design Reviews, February 2015
First prototype
Second prototype
Trigger Processor
One sTGC and one MM Trigger Processor per 1/16th sector • Large Xilinx FPGA; ATCA in USA15 (radiation protected area)• Receives 32 fibres (90m) from ART ASICs or 32 (was 24) fibres from Router• sTGC: centroid finder; MM uses ART address as hit• Find valid track segments that can corroborate hits in the Big Wheel• Combine list from MM and sTGC, removing duplicates• Send valid segments to Sector Logic for matching to the Big Wheel hits.
– Up to 8 per BC• Input and output data read out to ATLAS on Level-1 Accept• Monitoring and diagnostic data sent to a monitor PC
Lorne Levinson, Overview of NSW trigger electronics NSW Electronics Design Reviews, February 2015 18
Lorne Levinson, Overview of NSW trigger electronics NSW Electronics Design Reviews, February 2015 19
Trigger processor platform
20
LAr
SRS
• ATCA-based FPGA boards, one board per octant, 2 crates
• Two MM & two sTGC sectors per carrier• One sTGC or one MM sector per FPGA• Input fibers per sector: MM: 32, sTGC: 32• Transfer candidates between MM & sTGC FPGAs
via low latency lines • Avoid development of yet-another-ATCA-FPGA board
Two candidate carriers & mezz’s: LAr, SRS
NSW Electronics Design Reviews, February 2015
SRS: two FPGAs per mezz ×2 36 input, 36 output fibres
LAr: one FPGA per AMC mezz ×4 48 input, 48 output fibres
NSW Electronics Design Reviews, February 2015 21
LatencyOur major issue: sTGC is at the limit of our allocated latency: 1025ns from IP to Sector Logic inputWant at least 100ns contingencyWork is in progress to understand in detail every element and their interconnections in order to determine how to reduce latency:• Previously used VMM pulse-at-peak for pads;
actually using Time-over-Threshold: saves 10-15ns• Higher speed clocks in the Pad Trigger• Widen the parallel path from Pad Trigger to strip TDS:
2 pairs 10 pairs and new protocol, could save 15-20ns (but more cable volume)• Detailed study of the strip TDS to see where time can be saved between receiving
the Pad Trigger info and sending the data out• Router deserialization–serialization: could reduce by 10-15ns• Fiber length to USA15: needs to be accurately determined.
Currently a worst case number. Could save 25ns• Sector Logic has given us 50ns, 15-25ns to be used for merging candidates
Bottom line from above: reduce by 85 – 110ns promising, but… not proven More to add, e.g. 2m 5m fiber to Sector Logic, gaps in counting, …
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 22
NSW trigger concept• Phase I upgrade: Increased backgrounds, but must maintain existing trigger rate• Filter “Big Wheel” muon candidates to remove tracks that are not from the IP
Only track “A” should be a trigger candidate: pointing: D < 7.5mrad• Challenge is latency: 500nsec for electronics + 500ns fibres to be in time for Big Wheel• Micromegas: 2M strips, 0.4mm• sTGC: 280K strips (3.2mm), 45K pads, 28K wires• sTGC, MM find candidates
independently, list merged for Sector Logic
• Hit per layer:sTGC: hit is centroid of 3-5 stripsMicromegas:hit is address of strip
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 23
Thank you
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 24Lorne Levinson, Overview of NSW trigger electronics
Connections between the ROC and up to 8 VMMs
MM Latencymin
(ns)max
(ns) Notes
TOF from interaction point to MM (z=7.744 m) 30.2 30.2 To periphery of MM @R=4.618m + 5 cm IP
Earliest arrival hit 50 75 Depends onsite of rolling window (2 or 3 BC)
VMM2 chip latency 10 15 ART at threshold crossing instead of peak
FEB to Trigger Driver cable 3 3 Twin-ax cable 0.5 m @ 5 ns/m
Trigger driver latency 41 41 ART encoding (companion chip): from VMM to until first data bunch is output to GBT
Uplink GBT latency (Tx) 99 111 GBTx measured Mar 2014
Fiber to Trigger Processor card in USA15 (80-90 m) 400 450 5 ns/m (fiber length might be reduceable)
Uplink GBT latency (Rx) 44 44 GBT-FPGA (optimized) - includes GTX-TX (Kai Chen)
Trigger Algorithm 56.25 56.25 320 MHz clock
Re-synch to 320 MHz clock driving output serializer 0 3.1 45° phase chosen to best match pipeline length
Output to Sector Logic serializer (Tx only) 25 30 Deserializer on Sector Logic latency budget
Fiber to Sector Logic 5 10 1-2 m fiber @ 5ns/m
Total 763 869 TDR was 785-920 ns
sTGC Latencymin(ns)
max(ns) Notes
TOF from interaction point to NSW (z=7.8 m) 29 31 To periphery of NSW @R=5m
Pad signal jitter in chamber 5 10 Worst case due to tracks midway between wires (late signals due to long drift time)
Pad ASD (VMM) 40 50 ASD latency + time-to-peak
Serialize 32 pads @5 Gbps 16 20
TDS to pad triger on rim, max 4 m 18 23 Twin-ax cable delta R = 3-4 m + delta Z = 0.5 m @ 5 ns/m
Deserialize 32 pads 30 40 On Pad Trigger
Pad trigger (incl deskew) 15 25 Strips are pipelined until pad trigger arrives
Serializer of Pad Trigger output 0 0 25 ns for 32 bits @ 1.28 Gbits/sec (simultaneous with deserializer)
Pad trigger to on-chamber TDS ASIC 18 23 Twin-ax cable delta R = 3-4 m + delta Z = 0.5 m @ 5 ns/m
TDS: Trigger Data Serializer 95 95 (Strip data transferred while waiting for pad trigger)
On chamber cabling (up to 3-4m) to Router 18 23 Twin-ax cable delta R = 3-4 m + delta Z = 0.5 m @ 5 ns/m
Router 85 95 Include deserialization, switch (10 ns), serialization
Fiber to Centroid card in USA15 (80-90) 400 450 5 ns/m (fiber length might be reduceable)
Trigger processor input deserializer 40 40
sTGC trigger algorithm 56 56 8 layers done in parallel, measured to be 13 clocks + 5 estimated
Re-synch to 320 MHz clock driving output serializer 0 3.1 45° phase chosen to best match pipeline length
Centroid to Sector Logic serializer (Tx only) 25 30 Deserializer on Sector Logic latency budget
Fiber to Sector Logic 5 10
Total 890 1024 TDR was 780-896 ns
NSW Electronics Design Reviews, February 2015 27
sTGC trigger algorithm
• Use average of centroids in each quad to define space points R1 & R2 • 1, 2, or even 3 of the 4 centroids of a quadruplet are omitted from averaging if:
– -ray's: wide (>5 strips)– Neutrons: large charge or wide– Noise: single strip– Pileup, i.e. pulse in a component strip is active before the trigger
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 28
How does a muon look in a background environment
Need to reject individual layer measurements to get good coordinates
-ray
neutron
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015
sTGC centroid finder demonstrator
• Cosmic ray test of one quadruplet• Trigger demonstrator using Xilinx
Virtex-6 evaluation board • Custom mezzanine cards to accept
the ToT signals from 8 (16-chan) FE VMMs, 4 strip + 4 pad layers: – Triggers on 3-out-of-4 pad layers– Calculates Time-over-Thresholds
(VMM1 does not have 6-bit FADC)– Finds 4 centroids– Selects and averages centroids– Sends inputs and outputs to
ethernet for recording, playback
• Latency of centroid calc: ~45ns
A cosmic ray passing at an angle thru’ a quadruplet. are the centroids (values on the left)
Vertical line is the calculated average.29Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 30
sTGC centroid calculation & averaging
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 31
6.7 cm
1 2 3 4 5 6 7 8MicroMegas Layers
Multilayer Multilayer
18.5 cm
X plane U plane V plane
Interaction Pointz
Strip #S2Strip #S1
X0
X1
X2
X3
UV
Slope X0 = LookUpTable (#S2 - #S1)
Local straight track selection based on stripnumber difference with strip precision
Layer Pair Slope
Averagerms=1.7 mrad
Theta Resolution at the entrance of the Muon System
Local segment slopes calculation between hits of Layers belonging to the same Layer Pair over 3 BCs.
Layer Pair
X0 X1 X2 X3 U V
Contacts: [email protected], [email protected]
Matching slope = track candidate
Micromegas trigger algorithm I
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015
Implementation for 2048 strips x 8 Layers
32
BC2A
BC1A
BC0A
Layer ABC hits
ShiftRegister
BC3@40 MHz
BC2B
BC1B
BC0B
Layer BBC hits
ShiftRegister
BC3@40 MHz
BC2A-BC2BBC2A-BC1BBC2B-BC1ABC2A-BC0BBC2B-BC0A
READ@320 MHz
SLOPE#0
SLOPE SELECT
Hit A
Hit B
SLOPE#63
Hit A
Hit B
64Slope
Calculation&
Pre-selection
MAX 8 Hits/BC
MATCH/SLOPE STORAGE
Shift RegistersLayer A/Layer B RAW DATA
x Number of Layer Pairs (6) Slope Select Logic
ROI & Track Angle
Selected track(s)
9x31 Match slope bit
BC2A-BC2BBC2A-BC1BBC2B-BC1ABC2A-BC0BBC2B-BC0ABC0A-BC0BBC1A-BC1BBC1A-BC0BBC1B-BC0A
9x64 slopes
Mi: Match slope value i
M1 M31
Slope #0
Slope #575
M1-M31 M1-M316 Layer Pairs
M1-M31 M1-M31
9 BCA-BCB combinations
Mor
OR OR
AND Mor
SelectedSlopes
DataID
Data pointers
Dataaccess
3 ticksA 5 (Storage) + 5 (Accumulate) ticks
B
1 ticks+1 per track
3 ticks
2 ticks
19 ticks@320 MHzA B
59,4 nsFrom A to B
Designed (VHDL) & SimulatedXST Synthesis & P&R
Virtex-7 xc7v485t-2 speed grade
Slice Registers 13%Slice LUTs 17% logic 11% shift Register 2% route-thru's 4%Occupied Slices 27%
Tunable OR/ANDLogic
x4 PR
MatchReset
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 33Lorne Levinson, Overview of NSW trigger electronics
Micromegas trigger algorithm II
θ
• Projective “global” slope easy to estimate from slope road
• Local slope easily calculated through local fit
• “Global” stereo slope calculated to determine ROI in φ
37 ns latency
Arrival of the last hit on the GBT link Coincidence candidate formed
Global horizontal/stereo slopes calculated
Local slope calculated
ROI determined
Projective roads help: • create coincidences
quickly, • reject background
from the start
NSW Electronics Design Reviews, February 2015 34
• Algorithm efficiency essentially 100%• Inefficiencies related to hits with late raise times, detector gaps,
low-ionization hits• Inefficiencies caused by simulated incoherent backgrounds are small• Irreducible inefficiencies due to muon brem @ 1 TeV at 5% from showering• Algorithm intrinsic resolution of measurement of θ local(@NSW) - θ global(@IP)
is 1.33mrad but affected by multiple scattering in the calorimeter
Performance summary -- algorithm II
Lorne Levinson, Overview of NSW trigger electronics
NSW Electronics Design Reviews, February 2015 35Lorne Levinson, Overview of NSW trigger electronics
• Finds candidates in R- ,ftagged by pT
• Mismatch of NSW and BW detector boundaries fan-out to several modules
Big Wheel Regions-of-Interestto be confirmed by NSW.
Red lines are NSW sector boundaries.
Sector Logic