Image
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic: o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
TITLE
CEI-112G: THE NEXT WAVE OF ELECTRICAL INTERFACES
Nathan Tracy, TE ConnectivityEd Frlan, SemtechTom Palkert, MACOMBrian Holden, Kandou Bus
SPEAKERSNathan Tracy
OIF VP of Marketing/Board Member, TE Connectivity [email protected]
Ed FrlanOIF TC vice chair, Semtech Senior System Architect [email protected]
Tom PalkertOIF PLL vice chair, MACOM System [email protected]
Brian HoldenOIF MA&E co-chair, Kandou [email protected]
� CEI IA is a clause-based format supporting publication of new clauses over time:� CEI-1.0: included CEI-6G-SR, CEI-6G-LR, and CEI-11G-SR clauses.� CEI-2.0: added CEI-11G-LR clause� CEI-3.0: added work from CEI-25G-LR, CEI-28G-SR � CEI-3.1: added work from CEI-28G-MR and CEI-28G-VSR
� CEI-11G and -28G specifications have been used as a basis for specifications developed in IEEE 802.3, ANSI/INCITS T11, and IBTA.� CEI 56G projects are in progress:
� LR: backplane� MR: chip to chip� VSR: chip to module� XSR: chip to optics engine (separate chips)� USR: chip to optics engine (2.5D or 3D package)
� CEI 112G project has begun!
11G
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017SxI-5 CEI-1.0 CEI-2.0 CEI-3.0 CEI-3.1
3G6G
25G & 28G56G
112G
OIF Electrical Implementation Agreements
4
Name Rate per pair Year Activities that Adopted, Adapted or were influenced by the OIF CEI
CEI-112G 112Gbps 201X The future is bright
CEI-56G 56Gbps 2016 in process: IEEE, InfiniBand, T11 (Fibre Channel)
CEI-28G 28 Gbps 2011 InfiniBand EDR, 32GFC, SATA 3.2, SAS-4,100GBASE-KR4, CR4, CAUI4
CEI-11G 11 Gbps 2008 InfiniBand QDR, 10GBASE-KR, 10GFC, 16GFC, SAS-3, RapidIO v3
CEI-6G 6 Gbps 2004 4GFC, 8GFC, InfiniBand DDR, SATA 3.0, SAS-2, RapidIOv2, HyperTransport 3.1
SxI5 3.125 Gbps 2002-3 Interlaken, FC 2G, InfiniBand SDR, XAUI, 10GBASE-KX4, 10GBASE-CX4, SATA 2.0, SAS-1, RapidIO v1
SPI4, SFI4 1.6 Gbps 2001-2 SPI-4.2, HyperTransport 1.03
SPI3, SFI3 0.800 Gbps 2000 (from PL3)
OIF’s CEI work has been a significant industry contributor
Host ICModule Connector
AC Coupling
CapModuleRetimer IC
USR
LR
MR
VSR
XSR
• Different reaches, number of connectors, channel materials mean we can optimize the application specifications for best efficiency
• Different modulations provide advantage in certain cases
CEI 56G Example Applications
OIF 112Gbps Panel Discussion
Nathan Tracy, TE Connectivity
Ed Frlan, Semtech
Tom Palkert, MACOM
Brian Holden, Kandou Bus
Image
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic: o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
TITLE
CEI-112G: The next wave of electrical interfaces
Ed Frlan, (Semtech, Senior System Architect)OIF Technical Committee Vice Chair
� IEEE 802.3cd Task Force is standardizing 100GBASE-DR Optical Modules:
o 4x 25G NRZ electrical (CAUI-4,100GAUI-4) <-> 1x 100G
PAM-4 optical (500m)
o 2x 50G PAM-4 electrical (100GAUI-2) <-> 1x 100G PAM-4
optical (500m)
� Next-gen 100GBASE-DR Optical Module:
o 1x 100G ?PAM-4? electrical <-> 1x 100G PAM-4 optical
(500m)
112G VSR Application: Next-Gen 100GBASE-DR Optical Modules
28G-VSR, 56G-VSR-PAM4 DR Modules
112G-?PAM4? DR Module
100GAUI4/100GAUI-2
Tx
100G PAM4Encoder
100G PAM4Encoder
E/O
O/E
FIR
FFE/DFE
100GAUI4/100GAUI-2
Rx
100GAUI-1Tx
E/O
O/E
FIR
FFE/DFE
100GAUI-1Rx
� Ability to enable simple, low-power optical module implementations
o E.g. CE-28G-VSR/CAUI-4 electrical interfaces based on low-power CTLE receivers
o CEI-56G-VSR-PAM4/50GAUI also based on CTLE approach
� Channel selection enabling a broad suite of applications
o Nominal 10dB channel was the right choice
� Selection of a modulation scheme which could be the basis for a wide range of electrical reaches
o 25G/28G CEI interfaces (VSR, MR, LR) based on NRZ modulation
o 56G CEI interfaces (XSR, VSR, MR, LR) addressable by PAM-4 modulation
� pJ/bit efficiency of new interface better than previous generation
Elements of successful, past VSR standards
� Possible 112G modulation schemes include: PAM-4, PAM-8, duo-binary, DMT
� Other modulation schemes and/or variants on the above are possible!
112G candidate modulations112G VSR Modulation
Format Pros Cons
PAM-4
• Ability to re-use electronics developed for 112G optical PAM-4
• Analog implementation may be feasible• Familiarity, availability of test equipment
• May not be feasible for longer reachinterfaces
PAM-8 • Possible feasibility for longer reach interfaces
• Likely mandates an ADC/DSP implementation
• Smaller SNR than PAM-4• DMT higher performance than PAM-8
Duo-binary • Analog implementation may be feasible• Feasibility for longer reach interfaces • Requirement for 112G NRZ transmitter
DMT • Ability to deal with poor channels• Feasibility for longer reach interfaces
• Challenge of transmitters having large PAPR
• Challenging ADC• Unfamiliarity, perceived complexity
56G PAM-4 OIF VSR Channel
Example of cobo 7dB VSR channels
� 10dB loss from host IC ball to module CDR IC at 14 GHz
� Key question: can a 112G VSR channel based on an as is 10dB CEI 56G-PAM4 channel be made to work?
112G OIF VSR Channel (1/3)
Example of cobo VSR channel (from previous slide) extended to 10dB loss @ 14 GHz
� The example 112G channel is based on the 56G cobo channel with 3dB additional loss
� Channel exhibits excellent performance for 28 GBd signals
� Much worse for 56 GBd signals (loss > 20dB + non-neglible ILD)
� A 100G PAM-4 solution would require advanced, power-hungry equalizers
� Such a channel is likely not a suitable starting point for 112G VSR
� 10dB channel pulse response including “12mm IEEE” packages at either end
� Two significant pre-cursors, tail extends to > 20 UI
112G VSR Channel – pulse response (2/3)
Cd Cp160 fF 110 fF
Zp 12mm, Zc 85W
IEEE 12mm package
� “COM” = 3.26 dB @ SER = 1E-6,
� VEO (vertical eye opening) = 5mV @ SER = 1E-6
� Channel will require significant improvements in order to improve system margin (Note: simulation included no crosstalk!)
� This type of heavy equalization was required for 56G-PAM4 LR reaches (35dB loss channels)
112G VSR Channel – equalized response (3/3)� Equalization based on 4-tap Tx FIR and Rx CTLE +
20-tap DFE
Summary – Keys to a successful 112G VSR specification� If 112G is required to meet present reach then along with channel improvements (via improved
packages, connectors, etc), losses must also drop – the optical module will not be able to support a high power electrical interface solution!
� How? Possibilities include:o Improved PCB materials
o Channels based on coaxial interconnects
� ….. or, channels need to be shorter; optics must get closer to the host switch
� Likely a combination of all the above will be required
---
QUESTIONS?
Thank you!
Image
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic: o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
TITLE
CEI-112G: Considering Electrical Channels
Nathan Tracy, TE ConnectivityOIF Board Member and Marketing VP
18
VSR - Connecting Chips to Modules - Typical Reach up to 10”
LR - Channels in Chassis - Typical Reach = 1m
4-10”, PWB trace
Connector with footprint0.7-1.5”, PWB trace
• ~1m of improved PWB• 2 backplane connectors
• 0.5m of PWB• 1 orthogonal connector
• 14” of improved PWB• 1m of cable• 2 cable backplane connectors
Channels Considered for this Discussion
• Used 25G (published) and 50G (in development) OIF and IEEE industry standards as starting point
• Based on the shift towards PAM4 with the transition from 25G to 50G we can assume 100G will likely be PAM4
• Other encoding schemes were not considered but new emerging methods could enable next generation high speed links.
• For 100G the actual data rate will likely be 112GBaud/s with a Nyquist frequency around 28GHz (PAM4)
• The bandwidth of interest is assumed to be 10MHz – 56GHz• The Insertion Loss/Return Loss requirements were extrapolated using a
combination of,– Historical trends in data rate leaps, using current 50G targets as reference– Successful demonstrations of actual channels at 50G (PAM4 and NRZ)
19
Assumptions to Determine 100G Targets
25Gbps NRZ [IEEE Std. 802.3bm]28Gbps NRZ [OIF CEI-28G-VSR]50GBaud PAM4 [IEEE Draft 802.3bs]56GBaud PAM4 [OIF Draft oif2014.230.06]56Gbps NRZ, 20dB [OIF Draft oif2016.101.00]56Gbps NRZ, 13dB [OIF Draft oif2016.101.00]112G PAM4 [GUESS]
Target 17.5dB@ 28GHz
Very Short Reach (Chip to Module) Limits
25Gbps NRZ [IEEE Std. 802.3bj]25GBaud PAM4 [IEEE Std. 802.3bj]25Gbps NRZ [OIF CEI-25G-LR]50GBaud PAM4 [IEEE Draft 802.3cd]56GBaud PAM4 [OIF Draft oif2014.380.05]56GBaud ENRZ [OIF Draft oif2014.364.04]112GBaud PAM4 [GUESS]
Target 29dB@ 28GHz
Long Reach (Backplane) Limits
22
10”, PWB trace
Connector with footprint 1” PWB trace
4”, PWB trace
Improved Connector with footprint
0.4” PWB trace 2”, PWB trace
Board mounted cable connector
18” of 33AWG or 23” of 30 AWG cable
Improved Connector with footprint
Two Possible Paths for Improvement
Reduce Channel Length Use Lower Loss Channel
~10 dB gap
N4000-13SI PWB material
0.4” PWB trace
Existing VSR Channel vs New Limits
- 3.5mm thick- 8mil stub- Long via barrel
Conclusions:• Passes up to the Nyquist frequency but may be impractical lengths • Footprint is critical. FP causes significant degradation beyond 33GHz
23
4”, PWB trace0.7” PWB trace
Megtron6EM888
Improved Connector with footprint
Meg6 provides 2” of additionaltrace on DC
Shorter VSR Channels
24
2”, PWB trace
Board mounted cable connector
18” of 33 AWG cableOR
23” of 30 AWG Cable
Improved Connector with footprint
0.7” PWB trace
Conclusions:• Passes up to the Nyquist frequency with margin• Utilizing cable provides extended reach and flexibility
Megtron6, Typical FPMegtron6, Short FP
- 3.5mm thick- 8mil stub- Long via barrel
Using Cables to Extend Channel Length
25
14.00
GHz
12.50
GHz
11.00
GHz
9.500
GHz
8.000
GHz
6.500
GHz
5.000
GHz
3.500
GHz
2.000
GHz
500.0
MHz
0
-2
-4
-6
-8
-10
FREQ
dB
B2_T0B2_EOL
Variable
PCB T0 vs. EOL VarianceSDD21- PCB B2
1250011000950080006500500035002000500
0
-2
-4
-6
-8
-10
Freq (MHz)
dB
Pair 14-15_cable1_1Pair 14-15_cable1_after
Variable
Whisper T0 vs. EOL VarianceSDD21 Pair 14-15
T0 vs EoL: Temperature/Humidity cycling per EIA-364-31 Method III200mm PWB Megtron 6 traces vs. 500mm twinax cable assemblies
0.25dB @14 GHz Change No Change
95
96
97
98
99
100
101
102
103
104
105
4.7E-08 4.705E-08 4.71E-08 4.715E-08 4.72E-08 4.725E-08 4.73E-08 4.735E-08 4.74E-08 4.745E-08 4.75E-08
PCB Trace Differential Impedance
95
96
97
98
99
100
101
102
103
104
105
4.77E-08 4.775E-08 4.78E-08 4.785E-08 4.79E-08 4.795E-08 4.8E-08 4.805E-08 4.81E-08 4.815E-08 4.82E-08
Bulk Cable DIfferential Impedance
5% PWB Impedance Tolerance 2% Cable Impedance Tolerance
IL V
aria
nce
Durin
g Te
mp/
Hum
idity
Typi
cal I
mpe
danc
eVa
rianc
e
200mm FR4 PWB Material 500mm, 30 AWG Cable
PCB vs. Cable Consistency Measurements
26
27” PWB Megtron 6 traces vs. 1m twinax cable assemblies
-50
-45
-40
-35
-30
-25
-20
-15
-10
-5
0
0 2 4 6 8 10 12 14 16 18 20
Mag
nitu
de (d
B)
Frequency (GHz)
SCD21- SDD21All 3 Cables
SCD-SDD (dB) A2A3_C1 SCD-SDD (dB) B2B3_C1 SCD-SDD (dB) C2C3_C1SCD-SDD (dB) D2D3_C1 SCD-SDD (dB) H2H3_C1 SCD-SDD (dB) J2J3_C1SCD-SDD (dB) K2K3_C1 SCD-SDD (dB) L2L3_C1 SCD-SDD (dB) A2A3_C2SCD-SDD (dB) B2B3_C2 SCD-SDD (dB) C2C3_C2 SCD-SDD (dB) D2D3_C2SCD-SDD (dB) H2H3_C2 SCD-SDD (dB) J2J3_C2 SCD-SDD (dB) K2K3_C2
Mode Conversion (Skew) SCD21-SDD21 Measurements
~2-4ps of skew ~6-8ps of skew
PCB & Cable Consistency Measurements
02 03
05 06
37 36
33 34
- 3.5mm thick- 8mil stub- Long via barrel
27
4”, PWB trace0.7” PWB trace
Improved Connector with footprint
Short FPTypical FP
- Micro-via- No stub- Short via barrel
Typical FP Short FP
>20dB SNR @ 28 GHz
Typical Noise in Shorter VSR Channel
Ideal mating zone & short footprintRealistic mating zone & short footprintIdeal mating zone & typical footprint (thick board with 8mil stub)Realistic mating zone & typical footprint (thick board with 8mil stub)
28
4”, PWB trace0.7” PWB trace
VSR Sensitivity to Connector Design
29
Cable termination Impedance varied +/- 10%
Nominal+/-10% Impedance
- Excess solder paste- Inaccurate cable placement - Stripping of signal insulation and shield- Manufacturing control is critical
VSR Sensitivity to Cable Termination Variance
30
Traditional Backplane
Orthogonal
Cabled Backplane
Epic Fail
Fail
Fail
- 1.0m (40”) of Meg6
- 2 BP connectors- 5.1mm (0.200”) thick BP- 2.8mm (0.110”) thick DCs
- 0.5m (20”) of Meg6
- 1 DPO connector- 2.8mm (0.110”) thick DCs
- 0.3m (12”) of Meg6- 1.0m of 30AWG HS cable
- 2 cable connectors- 2.8mm (0.110”) thick DCs
25G limit100G limitExisting channel
25G limit100G limitExisting channel
25G limit100G limitExisting channel
Existing LR Channels with 100G Limits
freq, Hz
freq, Hz
31
6”, PWB trace
• Reduce 10” to 6” DC Traces• Backdrill = 9mil stubs• Improved connector• Improved PWB material
o Meg6 = 6” DCo EM-888 = 6” DC
• Retimers for shorter lengths
Meg6 provides 6” of additionaltrace on each DC
Meg6 PWB materialEM-888 PWB material Conclusions:
• Passes through 40GHz with reasonable reaches
• Single retimers could extend reach with minimal impact
Shorter Orthogonal Channel
32
2”, PWB trace with 8” cable+conn • 16” Cable x 2 + Connector
• 2” DC Trace x 2• 6/6/6 Traces in low cost FR4
• 6” DC Trace• 6/6/6 Traces in
low cost FR4
Conclusion:• Passes through 50GHz
with good reaches
Cabled Orthogonal Channel
• 4” DC Traces• 9mil Stubs
• 1m Cable
• Improved Whisper Connector
• Improved PWB Traces
o Meg6
o EM-888
33
Meg6 provides 3.5” of additionaltrace on each DC
Conclusions:• Passes through 40GHz
with reasonable reaches (7.5” DC with Meg6)
• Single retimers could extend reach with minimal impact
Cable BP Channel
34
4” DC Traces
6/6/6 Traces in FR4
8” Cable + Conn
2” PWB Traces
6/6/6 Traces in FR4
Conclusions:• Passes through 50GHz with
10” reaches and inexpensive material (FR4)
Improving the Cable BP Channel
35
• Orthogonal and Cable Backplane LR channel noise with Meg6 daughtercards
• Both channels have an SNR > 20dB @ 28GHz
Cable BP
Orthogonal Insertion Loss
PowerSum NEXT
PowerSum FEXT>20dB SNR @ 28 GHz
>20dB SNR @ 28 GHz
Noise in 100G LR Channels — PWB DC’s
36
Cable BP
Orthogonal
25mil Via Stubs15mil Via Stubs12mil Via Stubs9mil Via Stubs
Orthogonal Channel:• 6” DC traces• Meg6 materialCable Channel• 4” DC traces• Meg6 material• 1m cable backplane
100G LR Channel - Via Stub Length Impact
37
Cable BP
Orthogonal
HVLP Foils
VLP Foils
Standard Foils
Orthogonal Channel:• 6” DC traces• Meg6 materialCable Channel• 4” DC traces• Meg6 material• 1m cable backplane
100G LR Channel-PWB Foil Roughness Impact
• Don’t bet against 112Gbps copper for LR channels• 100G LR and VSR channels are possible• New lower loss techniques will need to be
implemented• Manufacturing consistency will be even more
critical• Need better definition of silicon requirements
Conclusions
---
QUESTIONS?
Thank you!
Image
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic: o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
TITLE
CEI-112G-VSRChallenges and needed improvements
Tom PalkertMACOM
Image
SPEAKERTom Palkert
OIF CEI-112G-VSR Electrical Chip to Module Interfaces
Optical Module
Host ASIC
�Challenges:– System level – OEM– Channel
• Attenuation
• Modulation
• Signal impairments
– Silicon
CEI-112G-VSR
� Improvement options:– System level – OEM– Channel
• Attenuation
• Modulation
• Signal impairments
– Silicon
Challenges
CEI-112G-VSR
System level Challenges: support TOR DAC and Aggregation switch
3m
300m
OEM challenge: support legacy PCB distances
VSR=8-12in
1m Backplane
MR=20in
Switch card
XSRUSR
DAC 3m
Channel Challenges: Attenuation
10dB channel for 28G-VSR and 56G VSR-PAM4
20dB channel for 56G VSR-NRZ
‘Legacy’ channel from 28G VSR
Channel Challenges: Modulation
20log(1/3)=-9.54dB
20log(1/7)=-16.90dB
PAM4 (56Gbaud) PAM8 (37.3Gbaud)
Channel Challenges: Signal Impairments
Optical Module
Host ASIC
Module connector
AC couplingcapacitorTrace vias
� Higher data rates challenges traditional retimer methods
� Legacy channels– Not well characterized for higher performance
� Power constraints from higher density modules– i.e. QSFP-DD provides 2x bandwidth vs QSFP but uses same module size
Silicon Challenges
Improvements
CEI-112G-VSR
System level improvements 1: Move Switch to center of rack
1.5m
System level improvements 2:Define asymmetric Switch to Server connectionsDefine end to end budgets that take advantage of short NIC traces
Switch ports require long host to module traces Server ports have very short
host to module traces
8-12in1in
OEM improvements 1:
VSR=8-12in
1m Backplane
MR=20in
Switch card
XSRUSR
DAC 3m
Cabled backplaneImproved materials
OEM improvements 2: Asymmetric VSR spec
Optical Module
Host ASIC
Assume ‘high performance’Host SERDES
Assume ‘low performance’Module SERDES
Attenuation improvements: Low Loss PCB
10dB channel for 28G-VSR and 56G VSR-PAM4
20dB channel for 56G VSR-NRZ
channel for 112G-VSR
Modulation improvements: PAM8?
Image
5dB
PAM8PAM4
7.4dB PAM8penalty
Signal Impairments improvements: BipassTM cables
Stacked Optical Modules
Host ASIC twinax
Optical Module
Host ASICModule connector
Trace vias
Signal integrity improved with BipassTM cables
AC couplingcapacitor
� DSP
� Use COM ‘like’ specification to allow flexible silicon design
Silicon Improvements
Noise Available signal
Margin
Courtesy: Rich MellitzSamtec
� 100G serial 2km optical link demonstrated
What will we see this week?
MACOMDSP
� VSR channel demonstrated
What will we see this week?
DSP
World’s first 100G serial VSR electrical link demonstration using an APM DSP over a TE channel with TE COBO connector. BER < 6e-7
---
QUESTIONS?
Thank you!
Image
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic:
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
� Topic: o Nam elementum commodo mattis. Pellentesque
malesuada blandit euismod.
TITLE
The CEI-112G in MCM ProjectBrian Holden, Kandou BusOIF MA&E Co-Chair
KANDOU reinventing the BUS
� On 1/19/17, the OIF started the CEI-112G in MCM project– The goal of this project is to support high rate interconnect within Multi-
Chip Modules (MCMs)
– The project is defined to support the interconnection of large logic devices with both:
• Small driver devices• Other large logic devices• More than one clause may be created as a part of this project
o .
The OIF CEI-112G in MCM Project
KANDOU BUS
� The agreed-upon properties in the project start:
– 0-1 cm reach– DC coupled path, on-die AC coupling optional– 1E-15 Error Ratio (high-latency FEC may not be used
to accomplish this)– Forwarded clock used
o .
CEI-112G in MCM Project Properties
KANDOU BUS
The rest of this presentation presents Kandou’s ideas for this interface
� The reach is not the only question
� Two distinct applications exist:– Logic-Logic - Large logic chip to medium/large logic chip
• In these applications, 1 to 10 Tb/s need to be transferred between two logic dies
– Logic-Driver - Large logic chip to small driver• In these applications, a main logic ASIC needs to connect at 100 to
112 Gb/s to many small SiGe or GaAs driver devices
Kandou’s ideas for the CEI-112G in MCM interface
KANDOU BUS
Logic-Logic application
KANDOU BUS
� Example dies that could be connected, typically using a packet mechanism:
– Packet I/O subsystem– Packet inspection/forwarding engine– Traffic manager– In-package switch fabric
– CPU, GPU, NPU, TPU, DSP processor dies– Processor coherency fabric– Processor I/O bus– HPC fabric
o .
Example dies to connect with the Logic-Logic application
KANDOU BUS
Logic-Driver application
KANDOU BUS
� Five key questions exist for each application:
1. Use DC or AC coupling (or on-die only AC coupling) in order to support a Silicon-to-SiGe (or III-V) connection
2. Use a shared forwarded clock or use Clock-Data Recovery (CDR)
3. Rely on the FEC of the I/O subsystem or not4. Require pair orientation or not5. Use CNRZ-5 or PAM-4 for the modulation technique
o .
Key questions for the CEI-112G in MCM interface
KANDOU BUS
� Logic-Logic: – Wants to have DC coupling to minimize the area required and
signal integrity impairment produced by AC coupling.
– It also wants to save power by not using a line code and scrambling.
� Logic-Driver:– Often requires AC coupling to marry the common-mode
voltage levels of devices made from very different semiconductor processes such as SiGe or GaAs.
– A compromise is to restrict the AC coupling to on-die AC coupling.
DC or AC Coupling
KANDOU BUS
� Logic-Logic: – Since Logic-Logic applications are almost always outside of the
I/O subsystem and use packet mechanisms, there are no relevant line clocks to work with.
• Power can be saved by using a shared forwarded clock
� Logic-Driver:– With the rise of 25GE and the future single lane 50GE & 100GE,
for VSR the incoming clocks on the different lanes can be different since the links can come from different chassis.
• This makes it inconvenient to use a shared forwarded clock when those multiple clocks have to be carried, so CDR is a more universal choice
• Intra-system uses can often use a shared forwarded clock
Shared Forwarded Clock or CDR
KANDOU BUS
� Logic-Logic: – Are typically outside of the I/O subsystem and thus cannot rely
on its Forwarding Error Correcting block. – Often use credit-based flow control that cannot tolerate a high-
latency FEC
� Logic-Driver:– The location w.r.t. the I/O subsystem can be mixed. The lanes
bound for outside of the system are protected by the FEC, but intra-system lanes may not be.
� Both applications generally need good native error performance.
FEC or not
KANDOU BUS
� Logic-Logic: – Just wants the most throughput for the least power
• Does not care about pair-orientation.
– Wide packet busses are common.
� Logic-Driver:– Typically wants the data kept within pairs
• The data is often bound for an optical device.
Pair-orientation or not
KANDOU BUS
� Logic-Logic: – CNRZ-5 is an excellent choice given its NRZ-shaped eyes, high
native signal integrity, and low power.
– CNRZ-5’s eyes are ~ 70% wider and 95% taller than those of PAM-4 in the included simulations. Implementations need much less equalization.
� Logic-Driver:– PAM-4 is a good choice to allow a driver device to have the
same modulation on both of its sides.
� NRZ is not likely to be viable for either given its excessive switching speed, which looks to exceed what is possible in silicon
Modulation technique: CNRZ-5 or PAM-4
KANDOU BUS
• 5 bits on 6 wires• 5 comparators
• Ideal for shorter connections including die-to-die interconnect inside a package
• Delivers NRZ shaped eyes at the decision point
• 69.6 GBaud delivers 116 Gb/s equivalent
• 50 GBaud delivers a useful product
CNRZ-5
KANDOU BUS
� Channel– Real 1.2 cm MCM channel, extended
to 100GHz– Uses GX13 substrate (er = 3.1, tandD =
0.019 @ 10GHz)– Assumed 0.4ps skew between wires
• For example, PAM4 uses 2 wires. The skew between the 2 wires is 0.4ps. CNRZ5 uses 6 wires. The skew between every wire and its neighbor is 0.4ps. The total skew across the 6 wires is 2ps
� Noise and jitter– Gaussian noise with std dev of 2mV– Rj (1 sigma) = 1% UI– Dj p-p = 10% UI
Simulation setup
KANDOU BUS
� Baud rate– CNRZ5 (EE-DR variant): 69.6GBd (50 GBd also shown)– PAM4: 58GBd
� Impedance– 50 ohms throughout
� Tx– Trise/Tfall = 5 ps– Tx peak-to-peak single ended = 0.3V– No FIR filter
� Rx– Auto-adaptive CTLE used– Ranges from 0 to 12 dB (1.2 cm only needed 2 dB)
Simulation configuration
KANDOU BUS
Robust, even at high speed
CNRZ-5 at 69.6 GBd
KANDOU BUS
Eyes are much smaller, both horizontally and vertically
PAM-4 at 58 GBd
KANDOU BUS
Works even better at 50 GBd,
Which is a good match to networking needs
CNRZ-5 at 50 GBd
KANDOU BUS
CEI-112G in MCM 1.2 cm simulation results
Code BaudGBd
UI(ps)
CTLE
Min EH15(mV)
Min EW15(ps)
Min EW15 (UI)
EyeHeights (mV)
Eye Width (ps)
CNRZ5 50 20 0 122.1 11.1 55.5 145.6, 132.4, 123.0, 130.1, 122.1
11.7, 11.3, 11.1, 11.5, 10.7
CNRZ5 69.6 14.37 1 101.6 7.1 49.5 120.4, 107.7, 101.6, 114.1, 102.3
8.7, 7.8, 7.1, 8.1, 7.4
PAM4 58 17.24 2 51.7 4.2 24.5 52.0, 52.1, 52.0,51.7, 52.0, 51.7
4.2, 4.4, 4.2, 4.2, 4.4, 4.2
KANDOU BUS
� This example bump map is runs at 25 GBdwith little equalization
� It gets 500 Gb/s per direction over 26 wires per direction in 2.4mm of chip beachfront
– Uses 150 um conventional bumps
Example bump map from 25 GBd implementation
KANDOU BUS
CNRZ-5 is the best choice for the Logic-Logic application.
Here is a good set of answers to our five key questions:� Logic-Logic:
1. DC coupling, silicon to silicon only2. Shared forwarded clock3. 1E-15 raw BER4. Not pair-oriented5. CNRZ-5
� Logic-Driver:1. DC coupled path, on-die AC coupling optional, silicon to SiGe (and III-V)2. Clock-Data Recovery (CDR)3. 1E-15 raw BER4. Pair oriented5. PAM-4
Conclusions
KANDOU BUS
---
QUESTIONS?
Thank you!