+ All Categories
Home > Technology > OIF 112G Panel at DesignCon 2017

OIF 112G Panel at DesignCon 2017

Date post: 17-Feb-2017
Category:
Upload: deborah-porchivina
View: 139 times
Download: 23 times
Share this document with a friend
85
Image Topic: o Nam elementum commodo mattis. Pellentesque malesuada blandit euismod. Topic: o Nam elementum commodo mattis. Pellentesque malesuada blandit euismod. o Nam elementum commodo mattis. Pellentesque malesuada blandit euismod. Topic: o Nam elementum commodo mattis. Pellentesque malesuada blandit euismod. TITLE CEI-112G: THE NEXT WAVE OF ELECTRICAL INTERFACES Nathan Tracy, TE Connectivity Ed Frlan, Semtech Tom Palkert, MACOM Brian Holden, Kandou Bus
Transcript
Page 1: OIF 112G Panel at DesignCon 2017

Image

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic: o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

TITLE

CEI-112G: THE NEXT WAVE OF ELECTRICAL INTERFACES

Nathan Tracy, TE ConnectivityEd Frlan, SemtechTom Palkert, MACOMBrian Holden, Kandou Bus

Page 2: OIF 112G Panel at DesignCon 2017

SPEAKERSNathan Tracy

OIF VP of Marketing/Board Member, TE Connectivity [email protected]

Ed FrlanOIF TC vice chair, Semtech Senior System Architect [email protected]

Tom PalkertOIF PLL vice chair, MACOM System [email protected]

Brian HoldenOIF MA&E co-chair, Kandou [email protected]

Page 3: OIF 112G Panel at DesignCon 2017

� CEI IA is a clause-based format supporting publication of new clauses over time:� CEI-1.0: included CEI-6G-SR, CEI-6G-LR, and CEI-11G-SR clauses.� CEI-2.0: added CEI-11G-LR clause� CEI-3.0: added work from CEI-25G-LR, CEI-28G-SR � CEI-3.1: added work from CEI-28G-MR and CEI-28G-VSR

� CEI-11G and -28G specifications have been used as a basis for specifications developed in IEEE 802.3, ANSI/INCITS T11, and IBTA.� CEI 56G projects are in progress:

� LR: backplane� MR: chip to chip� VSR: chip to module� XSR: chip to optics engine (separate chips)� USR: chip to optics engine (2.5D or 3D package)

� CEI 112G project has begun!

11G

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017SxI-5 CEI-1.0 CEI-2.0 CEI-3.0 CEI-3.1

3G6G

25G & 28G56G

112G

OIF Electrical Implementation Agreements

Page 4: OIF 112G Panel at DesignCon 2017

4

Name Rate per pair Year Activities that Adopted, Adapted or were influenced by the OIF CEI

CEI-112G 112Gbps 201X The future is bright

CEI-56G 56Gbps 2016 in process: IEEE, InfiniBand, T11 (Fibre Channel)

CEI-28G 28 Gbps 2011 InfiniBand EDR, 32GFC, SATA 3.2, SAS-4,100GBASE-KR4, CR4, CAUI4

CEI-11G 11 Gbps 2008 InfiniBand QDR, 10GBASE-KR, 10GFC, 16GFC, SAS-3, RapidIO v3

CEI-6G 6 Gbps 2004 4GFC, 8GFC, InfiniBand DDR, SATA 3.0, SAS-2, RapidIOv2, HyperTransport 3.1

SxI5 3.125 Gbps 2002-3 Interlaken, FC 2G, InfiniBand SDR, XAUI, 10GBASE-KX4, 10GBASE-CX4, SATA 2.0, SAS-1, RapidIO v1

SPI4, SFI4 1.6 Gbps 2001-2 SPI-4.2, HyperTransport 1.03

SPI3, SFI3 0.800 Gbps 2000 (from PL3)

OIF’s CEI work has been a significant industry contributor

Page 5: OIF 112G Panel at DesignCon 2017

Host ICModule Connector

AC Coupling

CapModuleRetimer IC

USR

LR

MR

VSR

XSR

• Different reaches, number of connectors, channel materials mean we can optimize the application specifications for best efficiency

• Different modulations provide advantage in certain cases

CEI 56G Example Applications

Page 6: OIF 112G Panel at DesignCon 2017

OIF 112Gbps Panel Discussion

Nathan Tracy, TE Connectivity

Ed Frlan, Semtech

Tom Palkert, MACOM

Brian Holden, Kandou Bus

Page 7: OIF 112G Panel at DesignCon 2017

Image

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic: o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

TITLE

CEI-112G: The next wave of electrical interfaces

Ed Frlan, (Semtech, Senior System Architect)OIF Technical Committee Vice Chair

Page 8: OIF 112G Panel at DesignCon 2017

� IEEE 802.3cd Task Force is standardizing 100GBASE-DR Optical Modules:

o 4x 25G NRZ electrical (CAUI-4,100GAUI-4) <-> 1x 100G

PAM-4 optical (500m)

o 2x 50G PAM-4 electrical (100GAUI-2) <-> 1x 100G PAM-4

optical (500m)

� Next-gen 100GBASE-DR Optical Module:

o 1x 100G ?PAM-4? electrical <-> 1x 100G PAM-4 optical

(500m)

112G VSR Application: Next-Gen 100GBASE-DR Optical Modules

28G-VSR, 56G-VSR-PAM4 DR Modules

112G-?PAM4? DR Module

100GAUI4/100GAUI-2

Tx

100G PAM4Encoder

100G PAM4Encoder

E/O

O/E

FIR

FFE/DFE

100GAUI4/100GAUI-2

Rx

100GAUI-1Tx

E/O

O/E

FIR

FFE/DFE

100GAUI-1Rx

Page 9: OIF 112G Panel at DesignCon 2017

� Ability to enable simple, low-power optical module implementations

o E.g. CE-28G-VSR/CAUI-4 electrical interfaces based on low-power CTLE receivers

o CEI-56G-VSR-PAM4/50GAUI also based on CTLE approach

� Channel selection enabling a broad suite of applications

o Nominal 10dB channel was the right choice

� Selection of a modulation scheme which could be the basis for a wide range of electrical reaches

o 25G/28G CEI interfaces (VSR, MR, LR) based on NRZ modulation

o 56G CEI interfaces (XSR, VSR, MR, LR) addressable by PAM-4 modulation

� pJ/bit efficiency of new interface better than previous generation

Elements of successful, past VSR standards

Page 10: OIF 112G Panel at DesignCon 2017

� Possible 112G modulation schemes include: PAM-4, PAM-8, duo-binary, DMT

� Other modulation schemes and/or variants on the above are possible!

112G candidate modulations112G VSR Modulation

Format Pros Cons

PAM-4

• Ability to re-use electronics developed for 112G optical PAM-4

• Analog implementation may be feasible• Familiarity, availability of test equipment

• May not be feasible for longer reachinterfaces

PAM-8 • Possible feasibility for longer reach interfaces

• Likely mandates an ADC/DSP implementation

• Smaller SNR than PAM-4• DMT higher performance than PAM-8

Duo-binary • Analog implementation may be feasible• Feasibility for longer reach interfaces • Requirement for 112G NRZ transmitter

DMT • Ability to deal with poor channels• Feasibility for longer reach interfaces

• Challenge of transmitters having large PAPR

• Challenging ADC• Unfamiliarity, perceived complexity

Page 11: OIF 112G Panel at DesignCon 2017

56G PAM-4 OIF VSR Channel

Example of cobo 7dB VSR channels

� 10dB loss from host IC ball to module CDR IC at 14 GHz

Page 12: OIF 112G Panel at DesignCon 2017

� Key question: can a 112G VSR channel based on an as is 10dB CEI 56G-PAM4 channel be made to work?

112G OIF VSR Channel (1/3)

Example of cobo VSR channel (from previous slide) extended to 10dB loss @ 14 GHz

� The example 112G channel is based on the 56G cobo channel with 3dB additional loss

� Channel exhibits excellent performance for 28 GBd signals

� Much worse for 56 GBd signals (loss > 20dB + non-neglible ILD)

� A 100G PAM-4 solution would require advanced, power-hungry equalizers

� Such a channel is likely not a suitable starting point for 112G VSR

Page 13: OIF 112G Panel at DesignCon 2017

� 10dB channel pulse response including “12mm IEEE” packages at either end

� Two significant pre-cursors, tail extends to > 20 UI

112G VSR Channel – pulse response (2/3)

Cd Cp160 fF 110 fF

Zp 12mm, Zc 85W

IEEE 12mm package

Page 14: OIF 112G Panel at DesignCon 2017

� “COM” = 3.26 dB @ SER = 1E-6,

� VEO (vertical eye opening) = 5mV @ SER = 1E-6

� Channel will require significant improvements in order to improve system margin (Note: simulation included no crosstalk!)

� This type of heavy equalization was required for 56G-PAM4 LR reaches (35dB loss channels)

112G VSR Channel – equalized response (3/3)� Equalization based on 4-tap Tx FIR and Rx CTLE +

20-tap DFE

Page 15: OIF 112G Panel at DesignCon 2017

Summary – Keys to a successful 112G VSR specification� If 112G is required to meet present reach then along with channel improvements (via improved

packages, connectors, etc), losses must also drop – the optical module will not be able to support a high power electrical interface solution!

� How? Possibilities include:o Improved PCB materials

o Channels based on coaxial interconnects

� ….. or, channels need to be shorter; optics must get closer to the host switch

� Likely a combination of all the above will be required

Page 16: OIF 112G Panel at DesignCon 2017

---

QUESTIONS?

Thank you!

Page 17: OIF 112G Panel at DesignCon 2017

Image

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic: o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

TITLE

CEI-112G: Considering Electrical Channels

Nathan Tracy, TE ConnectivityOIF Board Member and Marketing VP

Page 18: OIF 112G Panel at DesignCon 2017

18

VSR - Connecting Chips to Modules - Typical Reach up to 10”

LR - Channels in Chassis - Typical Reach = 1m

4-10”, PWB trace

Connector with footprint0.7-1.5”, PWB trace

• ~1m of improved PWB• 2 backplane connectors

• 0.5m of PWB• 1 orthogonal connector

• 14” of improved PWB• 1m of cable• 2 cable backplane connectors

Channels Considered for this Discussion

Page 19: OIF 112G Panel at DesignCon 2017

• Used 25G (published) and 50G (in development) OIF and IEEE industry standards as starting point

• Based on the shift towards PAM4 with the transition from 25G to 50G we can assume 100G will likely be PAM4

• Other encoding schemes were not considered but new emerging methods could enable next generation high speed links.

• For 100G the actual data rate will likely be 112GBaud/s with a Nyquist frequency around 28GHz (PAM4)

• The bandwidth of interest is assumed to be 10MHz – 56GHz• The Insertion Loss/Return Loss requirements were extrapolated using a

combination of,– Historical trends in data rate leaps, using current 50G targets as reference– Successful demonstrations of actual channels at 50G (PAM4 and NRZ)

19

Assumptions to Determine 100G Targets

Page 20: OIF 112G Panel at DesignCon 2017

25Gbps NRZ [IEEE Std. 802.3bm]28Gbps NRZ [OIF CEI-28G-VSR]50GBaud PAM4 [IEEE Draft 802.3bs]56GBaud PAM4 [OIF Draft oif2014.230.06]56Gbps NRZ, 20dB [OIF Draft oif2016.101.00]56Gbps NRZ, 13dB [OIF Draft oif2016.101.00]112G PAM4 [GUESS]

Target 17.5dB@ 28GHz

Very Short Reach (Chip to Module) Limits

Page 21: OIF 112G Panel at DesignCon 2017

25Gbps NRZ [IEEE Std. 802.3bj]25GBaud PAM4 [IEEE Std. 802.3bj]25Gbps NRZ [OIF CEI-25G-LR]50GBaud PAM4 [IEEE Draft 802.3cd]56GBaud PAM4 [OIF Draft oif2014.380.05]56GBaud ENRZ [OIF Draft oif2014.364.04]112GBaud PAM4 [GUESS]

Target 29dB@ 28GHz

Long Reach (Backplane) Limits

Page 22: OIF 112G Panel at DesignCon 2017

22

10”, PWB trace

Connector with footprint 1” PWB trace

4”, PWB trace

Improved Connector with footprint

0.4” PWB trace 2”, PWB trace

Board mounted cable connector

18” of 33AWG or 23” of 30 AWG cable

Improved Connector with footprint

Two Possible Paths for Improvement

Reduce Channel Length Use Lower Loss Channel

~10 dB gap

N4000-13SI PWB material

0.4” PWB trace

Existing VSR Channel vs New Limits

Page 23: OIF 112G Panel at DesignCon 2017

- 3.5mm thick- 8mil stub- Long via barrel

Conclusions:• Passes up to the Nyquist frequency but may be impractical lengths • Footprint is critical. FP causes significant degradation beyond 33GHz

23

4”, PWB trace0.7” PWB trace

Megtron6EM888

Improved Connector with footprint

Meg6 provides 2” of additionaltrace on DC

Shorter VSR Channels

Page 24: OIF 112G Panel at DesignCon 2017

24

2”, PWB trace

Board mounted cable connector

18” of 33 AWG cableOR

23” of 30 AWG Cable

Improved Connector with footprint

0.7” PWB trace

Conclusions:• Passes up to the Nyquist frequency with margin• Utilizing cable provides extended reach and flexibility

Megtron6, Typical FPMegtron6, Short FP

- 3.5mm thick- 8mil stub- Long via barrel

Using Cables to Extend Channel Length

Page 25: OIF 112G Panel at DesignCon 2017

25

14.00

GHz

12.50

GHz

11.00

GHz

9.500

GHz

8.000

GHz

6.500

GHz

5.000

GHz

3.500

GHz

2.000

GHz

500.0

MHz

0

-2

-4

-6

-8

-10

FREQ

dB

B2_T0B2_EOL

Variable

PCB T0 vs. EOL VarianceSDD21- PCB B2

1250011000950080006500500035002000500

0

-2

-4

-6

-8

-10

Freq (MHz)

dB

Pair 14-15_cable1_1Pair 14-15_cable1_after

Variable

Whisper T0 vs. EOL VarianceSDD21 Pair 14-15

T0 vs EoL: Temperature/Humidity cycling per EIA-364-31 Method III200mm PWB Megtron 6 traces vs. 500mm twinax cable assemblies

0.25dB @14 GHz Change No Change

95

96

97

98

99

100

101

102

103

104

105

4.7E-08 4.705E-08 4.71E-08 4.715E-08 4.72E-08 4.725E-08 4.73E-08 4.735E-08 4.74E-08 4.745E-08 4.75E-08

PCB Trace Differential Impedance

95

96

97

98

99

100

101

102

103

104

105

4.77E-08 4.775E-08 4.78E-08 4.785E-08 4.79E-08 4.795E-08 4.8E-08 4.805E-08 4.81E-08 4.815E-08 4.82E-08

Bulk Cable DIfferential Impedance

5% PWB Impedance Tolerance 2% Cable Impedance Tolerance

IL V

aria

nce

Durin

g Te

mp/

Hum

idity

Typi

cal I

mpe

danc

eVa

rianc

e

200mm FR4 PWB Material 500mm, 30 AWG Cable

PCB vs. Cable Consistency Measurements

Page 26: OIF 112G Panel at DesignCon 2017

26

27” PWB Megtron 6 traces vs. 1m twinax cable assemblies

-50

-45

-40

-35

-30

-25

-20

-15

-10

-5

0

0 2 4 6 8 10 12 14 16 18 20

Mag

nitu

de (d

B)

Frequency (GHz)

SCD21- SDD21All 3 Cables

SCD-SDD (dB) A2A3_C1 SCD-SDD (dB) B2B3_C1 SCD-SDD (dB) C2C3_C1SCD-SDD (dB) D2D3_C1 SCD-SDD (dB) H2H3_C1 SCD-SDD (dB) J2J3_C1SCD-SDD (dB) K2K3_C1 SCD-SDD (dB) L2L3_C1 SCD-SDD (dB) A2A3_C2SCD-SDD (dB) B2B3_C2 SCD-SDD (dB) C2C3_C2 SCD-SDD (dB) D2D3_C2SCD-SDD (dB) H2H3_C2 SCD-SDD (dB) J2J3_C2 SCD-SDD (dB) K2K3_C2

Mode Conversion (Skew) SCD21-SDD21 Measurements

~2-4ps of skew ~6-8ps of skew

PCB & Cable Consistency Measurements

Page 27: OIF 112G Panel at DesignCon 2017

02 03

05 06

37 36

33 34

- 3.5mm thick- 8mil stub- Long via barrel

27

4”, PWB trace0.7” PWB trace

Improved Connector with footprint

Short FPTypical FP

- Micro-via- No stub- Short via barrel

Typical FP Short FP

>20dB SNR @ 28 GHz

Typical Noise in Shorter VSR Channel

Page 28: OIF 112G Panel at DesignCon 2017

Ideal mating zone & short footprintRealistic mating zone & short footprintIdeal mating zone & typical footprint (thick board with 8mil stub)Realistic mating zone & typical footprint (thick board with 8mil stub)

28

4”, PWB trace0.7” PWB trace

VSR Sensitivity to Connector Design

Page 29: OIF 112G Panel at DesignCon 2017

29

Cable termination Impedance varied +/- 10%

Nominal+/-10% Impedance

- Excess solder paste- Inaccurate cable placement - Stripping of signal insulation and shield- Manufacturing control is critical

VSR Sensitivity to Cable Termination Variance

Page 30: OIF 112G Panel at DesignCon 2017

30

Traditional Backplane

Orthogonal

Cabled Backplane

Epic Fail

Fail

Fail

- 1.0m (40”) of Meg6

- 2 BP connectors- 5.1mm (0.200”) thick BP- 2.8mm (0.110”) thick DCs

- 0.5m (20”) of Meg6

- 1 DPO connector- 2.8mm (0.110”) thick DCs

- 0.3m (12”) of Meg6- 1.0m of 30AWG HS cable

- 2 cable connectors- 2.8mm (0.110”) thick DCs

25G limit100G limitExisting channel

25G limit100G limitExisting channel

25G limit100G limitExisting channel

Existing LR Channels with 100G Limits

freq, Hz

freq, Hz

Page 31: OIF 112G Panel at DesignCon 2017

31

6”, PWB trace

• Reduce 10” to 6” DC Traces• Backdrill = 9mil stubs• Improved connector• Improved PWB material

o Meg6 = 6” DCo EM-888 = 6” DC

• Retimers for shorter lengths

Meg6 provides 6” of additionaltrace on each DC

Meg6 PWB materialEM-888 PWB material Conclusions:

• Passes through 40GHz with reasonable reaches

• Single retimers could extend reach with minimal impact

Shorter Orthogonal Channel

Page 32: OIF 112G Panel at DesignCon 2017

32

2”, PWB trace with 8” cable+conn • 16” Cable x 2 + Connector

• 2” DC Trace x 2• 6/6/6 Traces in low cost FR4

• 6” DC Trace• 6/6/6 Traces in

low cost FR4

Conclusion:• Passes through 50GHz

with good reaches

Cabled Orthogonal Channel

Page 33: OIF 112G Panel at DesignCon 2017

• 4” DC Traces• 9mil Stubs

• 1m Cable

• Improved Whisper Connector

• Improved PWB Traces

o Meg6

o EM-888

33

Meg6 provides 3.5” of additionaltrace on each DC

Conclusions:• Passes through 40GHz

with reasonable reaches (7.5” DC with Meg6)

• Single retimers could extend reach with minimal impact

Cable BP Channel

Page 34: OIF 112G Panel at DesignCon 2017

34

4” DC Traces

6/6/6 Traces in FR4

8” Cable + Conn

2” PWB Traces

6/6/6 Traces in FR4

Conclusions:• Passes through 50GHz with

10” reaches and inexpensive material (FR4)

Improving the Cable BP Channel

Page 35: OIF 112G Panel at DesignCon 2017

35

• Orthogonal and Cable Backplane LR channel noise with Meg6 daughtercards

• Both channels have an SNR > 20dB @ 28GHz

Cable BP

Orthogonal Insertion Loss

PowerSum NEXT

PowerSum FEXT>20dB SNR @ 28 GHz

>20dB SNR @ 28 GHz

Noise in 100G LR Channels — PWB DC’s

Page 36: OIF 112G Panel at DesignCon 2017

36

Cable BP

Orthogonal

25mil Via Stubs15mil Via Stubs12mil Via Stubs9mil Via Stubs

Orthogonal Channel:• 6” DC traces• Meg6 materialCable Channel• 4” DC traces• Meg6 material• 1m cable backplane

100G LR Channel - Via Stub Length Impact

Page 37: OIF 112G Panel at DesignCon 2017

37

Cable BP

Orthogonal

HVLP Foils

VLP Foils

Standard Foils

Orthogonal Channel:• 6” DC traces• Meg6 materialCable Channel• 4” DC traces• Meg6 material• 1m cable backplane

100G LR Channel-PWB Foil Roughness Impact

Page 38: OIF 112G Panel at DesignCon 2017

• Don’t bet against 112Gbps copper for LR channels• 100G LR and VSR channels are possible• New lower loss techniques will need to be

implemented• Manufacturing consistency will be even more

critical• Need better definition of silicon requirements

Conclusions

Page 39: OIF 112G Panel at DesignCon 2017

---

QUESTIONS?

Thank you!

Page 40: OIF 112G Panel at DesignCon 2017

Image

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic: o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

TITLE

CEI-112G-VSRChallenges and needed improvements

Tom PalkertMACOM

Page 42: OIF 112G Panel at DesignCon 2017

OIF CEI-112G-VSR Electrical Chip to Module Interfaces

Optical Module

Host ASIC

Page 43: OIF 112G Panel at DesignCon 2017

�Challenges:– System level – OEM– Channel

• Attenuation

• Modulation

• Signal impairments

– Silicon

CEI-112G-VSR

� Improvement options:– System level – OEM– Channel

• Attenuation

• Modulation

• Signal impairments

– Silicon

Page 44: OIF 112G Panel at DesignCon 2017

Challenges

CEI-112G-VSR

Page 45: OIF 112G Panel at DesignCon 2017

System level Challenges: support TOR DAC and Aggregation switch

3m

300m

Page 46: OIF 112G Panel at DesignCon 2017

OEM challenge: support legacy PCB distances

VSR=8-12in

1m Backplane

MR=20in

Switch card

XSRUSR

DAC 3m

Page 47: OIF 112G Panel at DesignCon 2017

Channel Challenges: Attenuation

10dB channel for 28G-VSR and 56G VSR-PAM4

20dB channel for 56G VSR-NRZ

‘Legacy’ channel from 28G VSR

Page 48: OIF 112G Panel at DesignCon 2017

Channel Challenges: Modulation

20log(1/3)=-9.54dB

20log(1/7)=-16.90dB

PAM4 (56Gbaud) PAM8 (37.3Gbaud)

Page 49: OIF 112G Panel at DesignCon 2017

Channel Challenges: Signal Impairments

Optical Module

Host ASIC

Module connector

AC couplingcapacitorTrace vias

Page 50: OIF 112G Panel at DesignCon 2017

� Higher data rates challenges traditional retimer methods

� Legacy channels– Not well characterized for higher performance

� Power constraints from higher density modules– i.e. QSFP-DD provides 2x bandwidth vs QSFP but uses same module size

Silicon Challenges

Page 51: OIF 112G Panel at DesignCon 2017

Improvements

CEI-112G-VSR

Page 52: OIF 112G Panel at DesignCon 2017

System level improvements 1: Move Switch to center of rack

1.5m

Page 53: OIF 112G Panel at DesignCon 2017

System level improvements 2:Define asymmetric Switch to Server connectionsDefine end to end budgets that take advantage of short NIC traces

Switch ports require long host to module traces Server ports have very short

host to module traces

8-12in1in

Page 54: OIF 112G Panel at DesignCon 2017

OEM improvements 1:

VSR=8-12in

1m Backplane

MR=20in

Switch card

XSRUSR

DAC 3m

Cabled backplaneImproved materials

Page 55: OIF 112G Panel at DesignCon 2017

OEM improvements 2: Asymmetric VSR spec

Optical Module

Host ASIC

Assume ‘high performance’Host SERDES

Assume ‘low performance’Module SERDES

Page 56: OIF 112G Panel at DesignCon 2017

Attenuation improvements: Low Loss PCB

10dB channel for 28G-VSR and 56G VSR-PAM4

20dB channel for 56G VSR-NRZ

channel for 112G-VSR

Page 57: OIF 112G Panel at DesignCon 2017

Modulation improvements: PAM8?

Image

5dB

PAM8PAM4

7.4dB PAM8penalty

Page 58: OIF 112G Panel at DesignCon 2017

Signal Impairments improvements: BipassTM cables

Stacked Optical Modules

Host ASIC twinax

Optical Module

Host ASICModule connector

Trace vias

Signal integrity improved with BipassTM cables

AC couplingcapacitor

Page 59: OIF 112G Panel at DesignCon 2017

� DSP

� Use COM ‘like’ specification to allow flexible silicon design

Silicon Improvements

Noise Available signal

Margin

Courtesy: Rich MellitzSamtec

Page 60: OIF 112G Panel at DesignCon 2017

� 100G serial 2km optical link demonstrated

What will we see this week?

MACOMDSP

Page 61: OIF 112G Panel at DesignCon 2017

� VSR channel demonstrated

What will we see this week?

DSP

World’s first 100G serial VSR electrical link demonstration using an APM DSP over a TE channel with TE COBO connector. BER < 6e-7

Page 62: OIF 112G Panel at DesignCon 2017

---

QUESTIONS?

Thank you!

Page 63: OIF 112G Panel at DesignCon 2017

Image

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic:

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

� Topic: o Nam elementum commodo mattis. Pellentesque

malesuada blandit euismod.

TITLE

The CEI-112G in MCM ProjectBrian Holden, Kandou BusOIF MA&E Co-Chair

KANDOU reinventing the BUS

Page 64: OIF 112G Panel at DesignCon 2017

� On 1/19/17, the OIF started the CEI-112G in MCM project– The goal of this project is to support high rate interconnect within Multi-

Chip Modules (MCMs)

– The project is defined to support the interconnection of large logic devices with both:

• Small driver devices• Other large logic devices• More than one clause may be created as a part of this project

o .

The OIF CEI-112G in MCM Project

KANDOU BUS

Page 65: OIF 112G Panel at DesignCon 2017

� The agreed-upon properties in the project start:

– 0-1 cm reach– DC coupled path, on-die AC coupling optional– 1E-15 Error Ratio (high-latency FEC may not be used

to accomplish this)– Forwarded clock used

o .

CEI-112G in MCM Project Properties

KANDOU BUS

Page 66: OIF 112G Panel at DesignCon 2017

The rest of this presentation presents Kandou’s ideas for this interface

� The reach is not the only question

� Two distinct applications exist:– Logic-Logic - Large logic chip to medium/large logic chip

• In these applications, 1 to 10 Tb/s need to be transferred between two logic dies

– Logic-Driver - Large logic chip to small driver• In these applications, a main logic ASIC needs to connect at 100 to

112 Gb/s to many small SiGe or GaAs driver devices

Kandou’s ideas for the CEI-112G in MCM interface

KANDOU BUS

Page 67: OIF 112G Panel at DesignCon 2017

Logic-Logic application

KANDOU BUS

Page 68: OIF 112G Panel at DesignCon 2017

� Example dies that could be connected, typically using a packet mechanism:

– Packet I/O subsystem– Packet inspection/forwarding engine– Traffic manager– In-package switch fabric

– CPU, GPU, NPU, TPU, DSP processor dies– Processor coherency fabric– Processor I/O bus– HPC fabric

o .

Example dies to connect with the Logic-Logic application

KANDOU BUS

Page 69: OIF 112G Panel at DesignCon 2017

Logic-Driver application

KANDOU BUS

Page 70: OIF 112G Panel at DesignCon 2017

� Five key questions exist for each application:

1. Use DC or AC coupling (or on-die only AC coupling) in order to support a Silicon-to-SiGe (or III-V) connection

2. Use a shared forwarded clock or use Clock-Data Recovery (CDR)

3. Rely on the FEC of the I/O subsystem or not4. Require pair orientation or not5. Use CNRZ-5 or PAM-4 for the modulation technique

o .

Key questions for the CEI-112G in MCM interface

KANDOU BUS

Page 71: OIF 112G Panel at DesignCon 2017

� Logic-Logic: – Wants to have DC coupling to minimize the area required and

signal integrity impairment produced by AC coupling.

– It also wants to save power by not using a line code and scrambling.

� Logic-Driver:– Often requires AC coupling to marry the common-mode

voltage levels of devices made from very different semiconductor processes such as SiGe or GaAs.

– A compromise is to restrict the AC coupling to on-die AC coupling.

DC or AC Coupling

KANDOU BUS

Page 72: OIF 112G Panel at DesignCon 2017

� Logic-Logic: – Since Logic-Logic applications are almost always outside of the

I/O subsystem and use packet mechanisms, there are no relevant line clocks to work with.

• Power can be saved by using a shared forwarded clock

� Logic-Driver:– With the rise of 25GE and the future single lane 50GE & 100GE,

for VSR the incoming clocks on the different lanes can be different since the links can come from different chassis.

• This makes it inconvenient to use a shared forwarded clock when those multiple clocks have to be carried, so CDR is a more universal choice

• Intra-system uses can often use a shared forwarded clock

Shared Forwarded Clock or CDR

KANDOU BUS

Page 73: OIF 112G Panel at DesignCon 2017

� Logic-Logic: – Are typically outside of the I/O subsystem and thus cannot rely

on its Forwarding Error Correcting block. – Often use credit-based flow control that cannot tolerate a high-

latency FEC

� Logic-Driver:– The location w.r.t. the I/O subsystem can be mixed. The lanes

bound for outside of the system are protected by the FEC, but intra-system lanes may not be.

� Both applications generally need good native error performance.

FEC or not

KANDOU BUS

Page 74: OIF 112G Panel at DesignCon 2017

� Logic-Logic: – Just wants the most throughput for the least power

• Does not care about pair-orientation.

– Wide packet busses are common.

� Logic-Driver:– Typically wants the data kept within pairs

• The data is often bound for an optical device.

Pair-orientation or not

KANDOU BUS

Page 75: OIF 112G Panel at DesignCon 2017

� Logic-Logic: – CNRZ-5 is an excellent choice given its NRZ-shaped eyes, high

native signal integrity, and low power.

– CNRZ-5’s eyes are ~ 70% wider and 95% taller than those of PAM-4 in the included simulations. Implementations need much less equalization.

� Logic-Driver:– PAM-4 is a good choice to allow a driver device to have the

same modulation on both of its sides.

� NRZ is not likely to be viable for either given its excessive switching speed, which looks to exceed what is possible in silicon

Modulation technique: CNRZ-5 or PAM-4

KANDOU BUS

Page 76: OIF 112G Panel at DesignCon 2017

• 5 bits on 6 wires• 5 comparators

• Ideal for shorter connections including die-to-die interconnect inside a package

• Delivers NRZ shaped eyes at the decision point

• 69.6 GBaud delivers 116 Gb/s equivalent

• 50 GBaud delivers a useful product

CNRZ-5

KANDOU BUS

Page 77: OIF 112G Panel at DesignCon 2017

� Channel– Real 1.2 cm MCM channel, extended

to 100GHz– Uses GX13 substrate (er = 3.1, tandD =

0.019 @ 10GHz)– Assumed 0.4ps skew between wires

• For example, PAM4 uses 2 wires. The skew between the 2 wires is 0.4ps. CNRZ5 uses 6 wires. The skew between every wire and its neighbor is 0.4ps. The total skew across the 6 wires is 2ps

� Noise and jitter– Gaussian noise with std dev of 2mV– Rj (1 sigma) = 1% UI– Dj p-p = 10% UI

Simulation setup

KANDOU BUS

Page 78: OIF 112G Panel at DesignCon 2017

� Baud rate– CNRZ5 (EE-DR variant): 69.6GBd (50 GBd also shown)– PAM4: 58GBd

� Impedance– 50 ohms throughout

� Tx– Trise/Tfall = 5 ps– Tx peak-to-peak single ended = 0.3V– No FIR filter

� Rx– Auto-adaptive CTLE used– Ranges from 0 to 12 dB (1.2 cm only needed 2 dB)

Simulation configuration

KANDOU BUS

Page 79: OIF 112G Panel at DesignCon 2017

Robust, even at high speed

CNRZ-5 at 69.6 GBd

KANDOU BUS

Page 80: OIF 112G Panel at DesignCon 2017

Eyes are much smaller, both horizontally and vertically

PAM-4 at 58 GBd

KANDOU BUS

Page 81: OIF 112G Panel at DesignCon 2017

Works even better at 50 GBd,

Which is a good match to networking needs

CNRZ-5 at 50 GBd

KANDOU BUS

Page 82: OIF 112G Panel at DesignCon 2017

CEI-112G in MCM 1.2 cm simulation results

Code BaudGBd

UI(ps)

CTLE

Min EH15(mV)

Min EW15(ps)

Min EW15 (UI)

EyeHeights (mV)

Eye Width (ps)

CNRZ5 50 20 0 122.1 11.1 55.5 145.6, 132.4, 123.0, 130.1, 122.1

11.7, 11.3, 11.1, 11.5, 10.7

CNRZ5 69.6 14.37 1 101.6 7.1 49.5 120.4, 107.7, 101.6, 114.1, 102.3

8.7, 7.8, 7.1, 8.1, 7.4

PAM4 58 17.24 2 51.7 4.2 24.5 52.0, 52.1, 52.0,51.7, 52.0, 51.7

4.2, 4.4, 4.2, 4.2, 4.4, 4.2

KANDOU BUS

Page 83: OIF 112G Panel at DesignCon 2017

� This example bump map is runs at 25 GBdwith little equalization

� It gets 500 Gb/s per direction over 26 wires per direction in 2.4mm of chip beachfront

– Uses 150 um conventional bumps

Example bump map from 25 GBd implementation

KANDOU BUS

Page 84: OIF 112G Panel at DesignCon 2017

CNRZ-5 is the best choice for the Logic-Logic application.

Here is a good set of answers to our five key questions:� Logic-Logic:

1. DC coupling, silicon to silicon only2. Shared forwarded clock3. 1E-15 raw BER4. Not pair-oriented5. CNRZ-5

� Logic-Driver:1. DC coupled path, on-die AC coupling optional, silicon to SiGe (and III-V)2. Clock-Data Recovery (CDR)3. 1E-15 raw BER4. Pair oriented5. PAM-4

Conclusions

KANDOU BUS

Page 85: OIF 112G Panel at DesignCon 2017

---

QUESTIONS?

Thank you!


Recommended