Post on 11-Oct-2020
transcript
CMS Phase II Precision Clock Distribution Studies
P.-A. Bausson, M. Besançon, F. Couderc, M. Dejardin, J.-L. Faure, F. Ferri, S. Ganjour, P. Gras, I. Mandjavidze, Ö. Şahin, M. Titov CEA Saclay-Irfu
HTPD meetingCERN03.12.2018
Ö. Sahin, P.-A. Bausson
Motivation: need for low jitter clock distribution
2
LHC clock 40.078 MHz
Sensor+TDCLHC Clock distribution
• The clock distribution may significantly degrade the precise timing information obtained from the CMS Phase-II detectors.
• A 15 ps random jitter on the clock may have an impact of 3.5 ps (assuming 30 ps [Sensor+TDC] uncertainty).
• The impact will be significantly larger for higher jitter values.
• We need a low-jitter clock distribution not to jeopardise the timing performance.
• Over 10000 channels, FE under harsh radiation conditions.
Pierre-Anne Bausson, Özgür Şahin
Outline
3
• Phase I final full clock distribution chain performances
• Temperature effects
• Phase II hardware measurements
• Pure clock distributions studies
• Clock distribution networks with cascaded PLLs
• Outlook and conclusion
Pierre-Anne Bausson, Özgür Şahin
Last year reminder
4
• Overview of the last year:
• Different clock distribution schemes
• PLL loop bandwidth effect on the reference clock jitter for GBT-FPGA.
• Front-End (GBTx chip based) clock performances
• And we introduced studies outlook on :
• Back-End (MicroTCA standard based) clock performances
• Full chain (Back-End to Front-End) clock performances
• Effect of temperature on clock jitter
Pierre-Anne Bausson, Özgür Şahin
Phase I final clock chain performances
5
MM OM3 (10 m)
4.8 Gbps
Back-end
Clock source
Front-endClock
generator CG635
BNCSMA 80.156 MHz
HDMI 40.078 MHz
MM OM3 (10 m) 80.156MHz
Jitter cleaner Si5344
Oscilloscope Lecroy820zi-b
VLDBe-link
VTRx
HDMI-SMA
HDMI-SMA
SMA - BNC
SMA - BNC
HDMI 40.078 MHz
Jitter cleaner Si5344
Clk Inj
Clk Inj
SMA
SMA
VLDBVTRx
e-link
µTCA crate
AMC
13
FC7
FMC
µTCA crate
AMC
13 FC7
FMC
MM OM3 (10 m) 80.156MHz
Backplane 40.078 MHz
Backplane 40.078 MHz
eLink to eLink resultsRJ 11.5 psDJ 50.7 psRMS 27.8 ps
3ps RJ increase and 30ps DJ increase are observed with respect to the simpler BE clock distribution (see GBTx tests).
Pierre-Anne Bausson, Özgür Şahin
Temperature effects
6
• The impact of temperature is investigated at Saclay PCD lab:
• 20 GHz BW 80 GS/s oscilloscope
• [-40; +180] °C climate chamber
• Setup is as shown below :
Clock source
Back-end
Clock generator CG635 BNC
SMA 40.078 MHz
HDMI 40.078 MHz
Jitter cleaner Si5344 PLL
Xilinx KCU105
SFP+
SFP+
VLDB
VLDB
Oscilloscope SDA 820Zi-B
VTRx
VTRx
e-link
e-link
HDMI-SMA
HDMI-SMA
SMA - BNC
SMA - BNC
SMA 120.234
MHz
HDMI 40.078 MHz
MM OM3 (10 m)
4.8 Gbps
(Climate chamber Front-end)
[-30; +10] °C
Pierre-Anne Bausson, Özgür Şahin
Temperature effects
7
• We observed a constant improvement in the clock quality with the decrease in the temperature in the absolute measurements (wrt. scope’s PLL).
• Similarly, we obtained the cleanest channel-to-channel differential clock at the lowest die temperature.
• A consistent 2 ps decrease in the RMS jitter is observed from 20ºC climate chamber temperature to -30ºC.
Pierre-Anne Bausson, Özgür Şahin
Clock distribution schemes
8
TCDS2 DAQ crate ReadoutASIClpGBT(s)
~100 m MM Fiber
Encoded @ 2.5 Gbps
RF clock
TCDS2 DAQ crate ReadoutASIC
Rad Hard Clock buffer
Pure clock160 MHz
RF clock
Pure clock master
Pure clock node
ReadoutASIC
Rad Hard Clock bufferRF clock
Pure clock160 MHz
Fiber?@ 320 MHz?
BASELINE
HYBRID
PURE
Pierre-Anne Bausson, Özgür Şahin
Clock distribution schemes
9
lpGBT performances (ASIC+FPGA)
Full system redundancy
Back-End clock distribution
DAQ board as the clock
distribution + Monitoring
PCD system + MonitoringMonitoring PCD system +
Monitoring
<10ps >10ps
YesNo >10ps<10ps
Pierre-Anne Bausson, Özgür Şahin
Clock distribution schemes
10
lpGBT performance (asic+fpga)
Full system redundancy
Backend clock distribution
DAQ board as the clock
distribution + Monitoring
PCD system+ MonitoringMonitoring PCD system +
Monitoring
<10ps >10ps
YesNo >10ps<10psBaseli
ne Pure
Pure+
BaselineHybrid
Pierre-Anne Bausson, Özgür Şahin
Phase II hardware measurements
11
• The Serenity platform is an ATCA leaf board designed for the Phase-II backend DAQ applications and it is made by a collaboration led by Imperial College.
• We designed daughter cards dedicated to the platform to perform clock jitter measurements
• Views of the 2 daughter cards :
Pierre-Anne Bausson, Özgür Şahin
Phase II hardware measurements
12
ATCABack-End Front-End
Interposer
Interposer
Firefly
Firefly
Si53345 Clock buffer
Si5345 PLL
Si5345 PLL
Si5345 PLL
Si5345 PLL
Serenity@320.624 MHz - Si5345 9 outputs (North) 1.3 M at 20 MS Acquisition window @ 20GS/s— Reference: clock from the south interposer. The clock (RMS = 1.3 ps) is fed externally.RMS: 2.8 ps, RJ: 1.8 ps, DJ: 4.2 ps
Roger Rusack et al.
Pure clock distribution study
Roger Rusack et al.
Pure clock distribution study
Roger Rusack et al.
Results
Pierre-Anne Bausson, Özgür Şahin
U. Minnesota clock distribution tests
16
• University of Minnesota (R. Rusack et al) designed and presented a simple but effective passive clock distribution system to the collaboration.
• They demonstrated that with using modern fast integrated circuits that it is a possible to construct a clock distribution network with a precision of 200 fs between different branches of the network.
• At CEA Saclay will also welcome their system in our lab and tests the system with our test stand.
• The tests are currently planned to start next Monday (10th of December).
Eduardo Mendes, Özgür Şahin
TCDS
• Together with the CERN HPTD group, we have performed tests evaluating the performance of clock distribution networks with cascaded PLL configurations.
• Particularly important for the Phase II clock distribution networks.
Cascaded PLL measurements
DAQ
SMA-SMA 320.625
MHz
Clock source
SourceSi5345revD
EVB
Si5344revD EVB (L1)
MicroSemi SSA
Si5344revD EVB (L1)
320.625MHz
Si5344revD EVB (L2)
SMA-SMA 320.625
MHz Si5344revD EVB (L2)
320.625MHz
Eduardo Mendes, Özgür Şahin
• The PLL cascades not only provides a very clean clock distribution network, but it also cleans the (already very clean) input clock as well (with RMS of 0.75 ps).
Cascaded PLL measurements
1
2
3
4
5
6
RM
S jit
ter [
ps]
6.29 4.74 4.50 4.67
6.14 3.61 2.81 2.97
4.96 3.26 1.08 0.87
4.37 2.82 0.87 0.32
Si5345D loopBW 4000 Hz
100 400 1000 4000loopBW L1 [Hz]
100
400
1000
4000
loop
BW L
2 [H
z]
Si5345D loopBW 4000 Hz
1 10 210 310 410 510 610Offset freq. [Hz]
140−
120−
100−
80−
60−
40−
Phas
e no
ise
[dBc
]
RMS: 6.29 psRMS: 6.14 psRMS: 4.96 psRMS: 4.37 psRMS: 4.74 psRMS: 3.61 psRMS: 3.26 psRMS: 2.82 ps
RMS: 4.50 psRMS: 2.81 psRMS: 1.08 psRMS: 0.87 psRMS: 4.67 psRMS: 2.97 psRMS: 0.87 psRMS: 0.32 ps
loopBW- L1: 100 Hz - L2: 100Hz
loopBW- L1: 100 Hz - L2: 400Hz
loopBW- L1: 100 Hz - L2: 1000Hz
loopBW- L1: 100 Hz - L2: 4000Hz
loopBW- L1: 400 Hz - L2: 100Hz
loopBW- L1: 400 Hz - L2: 400Hz
loopBW- L1: 400 Hz - L2: 1000Hz
loopBW- L1: 400 Hz - L2: 4000Hz
loopBW- L1: 1000 Hz - L2: 100Hz
loopBW- L1: 1000 Hz - L2: 400Hz
loopBW- L1: 1000 Hz - L2: 1000Hz
loopBW- L1: 1000 Hz - L2: 4000Hz
loopBW- L1: 4000 Hz - L2: 100Hz
loopBW- L1: 4000 Hz - L2: 400Hz
loopBW- L1: 4000 Hz - L2: 1000Hz
loopBW- L1: 4000 Hz - L2: 4000Hz
Si5345lbw4000Hz
Eduardo Mendes, Özgür Şahin
• Encouraged with the results, we introduced additional jitter into one of the the clock distribution channels.
• The jitter functions are chosen as square waves (~70 ps RMS) and sinusoidal (~60 ps RMS) waves with different frequencies.
Cascaded PLL measurements
SMA-SMA 320.625
MHz
Source
Si5344revD EVB (L1)
MicroSemi SSA
Si5344revD EVB (L1)
Si5344revD EVB (L2)
SMA-SMA 320.625
MHz Si5344revD EVB (L2)
320.625MHz
Noise
Eduardo Mendes, Özgür Şahin
1 10 210 310 410 510 610 710Offset freq. [Hz]
160−
140−
120−
100−
80−
60−
40−
Phas
e no
ise
[dBc
]
RMS: 3.89 psRMS: 3.05 psRMS: 2.86 psRMS: 3.22 psRMS: 2.47 psRMS: 5.34 psRMS: 8.98 psRMS: 16.96 ps
RMS: 2.50 psRMS: 9.33 psRMS: 17.24 psRMS: 33.38 psRMS: 2.84 psRMS: 16.48 psRMS: 32.97 psRMS: 61.80 ps
loopBW- L1: 100 Hz - L2: 100Hz
loopBW- L1: 100 Hz - L2: 400Hz
loopBW- L1: 100 Hz - L2: 1000Hz
loopBW- L1: 100 Hz - L2: 4000Hz
loopBW- L1: 400 Hz - L2: 100Hz
loopBW- L1: 400 Hz - L2: 400Hz
loopBW- L1: 400 Hz - L2: 1000Hz
loopBW- L1: 400 Hz - L2: 4000Hz
loopBW- L1: 1000 Hz - L2: 100Hz
loopBW- L1: 1000 Hz - L2: 400Hz
loopBW- L1: 1000 Hz - L2: 1000Hz
loopBW- L1: 1000 Hz - L2: 4000Hz
loopBW- L1: 4000 Hz - L2: 100Hz
loopBW- L1: 4000 Hz - L2: 400Hz
loopBW- L1: 4000 Hz - L2: 1000Hz
loopBW- L1: 4000 Hz - L2: 4000Hz
SquareNoise@1000.0Hz
10
20
30
40
50
60
RM
S jit
ter [
ps]
3.89 2.47 2.50 2.84
3.05 5.34 9.33 16.48
2.86 8.98 17.24 32.97
3.22 16.96 33.38 61.80
SquareNoise@1000.0Hz
100 400 1000 4000loopBW L1 [Hz]
100
400
1000
4000
loop
BW L
2 [H
z]
SquareNoise@1000.0Hz
• The initial findings show that if correctly tuned, the cascaded PLLs do not degrade the clock distribution network.
• It can clean possible noise introduced by the network components.
Cascaded PLL measurements
Pierre-Anne Bausson, Özgür Şahin
Outlook
21
• Most Phase II are still under design or manufacturing :
• Our ATCA crate and power supply are setup at Saclay
• ATCA DTH and other leaf boards currently in manufacturing, expected Q2 2019
• VTRx+ prototypes and lpGBT engineering samples, expected early 2019
• Phase II baseline full clock chain tests will follow as soon as we have them in hand
• We are trying to identify the system need for a rad-hard clock fan-out chip
• We started to work on possible solutions to monitor slow clock phase drifts
• We will also welcome and tests the passive clock distribution system from U. Minn.
• Our short-term main goals are to study various phase monitoring possibilities and to identify the system requirements for an eventual rad-hard clock fan-out chip
• This year we presented our work to PM2018, TWEPP and ISPCS conferences. A CMS note is also in preparation.
Pierre-Anne Bausson, Özgür Şahin
Conclusion
22
! Temperature effects on clock jitter is encouraging given the operating conditions of HgCal at -30 °C
! Our short-term main goal is to study various phase monitoring possibilities and
identify the system requirements for an eventual rad-hard clock fan-out chip
Phase I hardware (MicroTCA+VTRx+GBTx) clock distribution performances are presented "
Phase II hardware (ATCA+VTRx+lpGBT) clock distribution measurements started. Serenity platform seem to perform well and is suitable for clock distribution "
Still waiting components availability. Our test stand is ready to welcome them at
Saclay for clock distribution measurements.
Pierre-Anne Bausson, Özgür Şahin
Backup
23
Pierre-Anne Bausson, Özgür Şahin
Clock gen blocks
PLL320.625
MHzFS725 Rb.
10MHz
320.625MHz
81134A clock gen
FS725 Rb.
10MHz
TG5011A noise gen
Withnoise
320.625MHz
Pierre-Anne Bausson, Özgür Şahin
• The simulation of 30 ps RMS (sensor) convoluted with 11.5 ps RJ and 50.7 ps DJ assuming the dual-dirac jitter model. RMS of the resulting distribution is 40.9 ps.
• DJ becomes significant; it is important to keep (if possible to minimalize) the deterministic jitter under control.
Simulating the clock distribution jitter
25
t [ps]
Pierre-Anne Bausson, Özgür Şahin
• AMC13(XG) is distributing the TTC clock through FCLKA pin of the MicroTCA crate.
• The input clock is 2xLHC clock frequency
• The distributed clock is 40.078 MHz
• In our tests we used FC7 board to emulate the BE clock distribution node.
µTCA Clock distribution
26
In the CMS topology one of the MCH slotsare occupied by the AMC13(XG) board.
Pierre-Anne Bausson, Özgür Şahin
Rad-hard clock buffer
• 2 to n clock fan-out architecture: output characteristics should be similar to eLink clock.
• It needs to be phase aligned with eLink clock OR
• It needs to forward and align the fast control signals as well
27
Rad hard clock buffer
1
2
…
n
lpGBT eLink clock
Pure clock
Fast control
Pierre-Anne Bausson, Özgür Şahin
• Digital Dual Mixer Time Difference circuit aims to measure the ‘time difference’ i.e. ‘latency’ between two clock inputs with a digital circuit.
• Using a slightly different frequency clock, we amplify the clock difference!✴ Waveform is taken from P. Moreira’s thesis
DDMTD
28