4/25/2016
Higher Speed Fiber Transceiver & Cable Plant Roadmap
Robert ReidSr. Product Development ManagerFiber Product GroupPanduit Corp.
DISCLAIMER: TIA does not endorse or promote any product, service, company or provider
Agenda
Introduction
Transceiver Historical Perspective
Going Forward - Higher Speed Roadmap
Cable Plant Implications
Planning and Best Practices
Summary
Q & A
Introduction
• Newer network designs (e.g., Flat Networks) require more transmission media to enable scalable and higher density solutions
• Large Enterprise/Webscale DCs are challenged to deal with significant transitions in the market to higher speed and longer reach channels
• Seamless infrastructure migration plans are necessary as data center port speeds are increasing (10Gb to 40Gb to 100Gb/s)(16Gb to 32Gb to 128Gb/s)
New Global Market Conditions/Challenges
• Data center size continues to grow– Mega Data Centers will drive growth in market
• MMF/VCSEL solution will continue seeing pressure from competing SMF/Long wavelength laser solutions
• VCSEL/MMF solutions offer several advantages worth preserving– Lowest energy consumption– Dust/debris immunity at connections (robust operation, MACs)– Support break-out via parallel optics (vs. WDM breakout)– Large installed base (>80% of DC fiber media)
• Benefits preserved for the bulk of 100G/128GFC/400G/512G channels by supporting 100m reach (even in break-out implementations)– using 4x parallel solutions
New Global Market Conditions/Challenges
DC Market & Customer Requirements Need REACH !!!!
Fiber HD Distribution - Market Segmentation SuperNAP Las Vegas
Google Mayes County
Relative cost of 40G LR4 compared to SR4
ANSI FC Higher Speed EffortsMarket Data
• Mainstream/High Volume use of 16G HBAs & Targets with 16GFC
• Marriage of 16GFC capability with SSDs• Launch of Cisco MDS 97K Multilayer Directors• Less FC Edge switches being sold - More
Directors
• Support of Installed Base: Support for 16GFC, 32GFC, 40GbE, 100GbE, 128GFC and beyond on the existing installed MMF fiber plant
• Shortwave WDM (SWDM): The ability to multiplex multiple lanes onto a single fiber to reduce the fiber count and enable duplex-LC interfaces for 40GbE, 32GFC,100GbE, 128GFC and beyond
• Lane rates > 25 Gb/s: Developing and standardizing technology that enables multimode VCSELs to operate at 50 Gb/s and beyond, to enable future generations of both single-lane and multiple lane optical interfaces
• Wideband MMF: Support for the definition and standardization of wideband multimode fiber to enable WDM transmission over links that are greater than 300m
DC Market & Customer Requirements Transceiver OEM Macro Trends
Transceiver Historical Perspective
Functional Outline of a Transceiver (SFP)
Standards that control transceiver form-factors
A multi-source agreement (MSA) is an agreement between multiple manufacturers to make products which are compatible across vendors, acting as de facto standards, establishing a competitive market for interoperable products.
What is Defined by the MSA?
The major elements or characteristics defined in the MSAs are:
Mechanical Interface • Mechanical dimensions of the device• Transceiver edge connector to host PCB-mounted electrical connector mating• Host board mechanical layout (location/size of solder pads, etc.)• Insertion, Extraction and Retention forces• Labeling• Bezel design considerations for host systems• Electrical connector mechanical aspects• Cage assembly dimensions (hollow cage mounted in host system)
Electrical Interface • Pin definitions• Timing requirements and Status I/O• Module definition interface and data field description
Fiber Channel Transceiver
Data sheet for commercial 8G serial PMD with SFP+ form factor
Typical modular transceiver used for SAN switch ports
Supports 8G, 4G & 2G
QSFP Parallel Optics Modular Transceiver Form-Factor
• MSA for MM short wave 40Gb/s Ethernet (4 x 10 lanes of 10G) and/or 100Gb/s Ethernet (4 lanes of 25G)
CFP MSA
Railing System (host) With CFP MSA module inserted
Hot-pluggable optical transceiver form factor for high speed applications, including next-generation Ethernet (40GBASE-SR4, 100GBASE-SR10, 40GBASE-LR4, 40GBASE-LR & 100GBASE-LR4)
Profile allows access for 4 SFP+ or 2 QSFP modules
Low Cost
High VolumeTooling Intensive
High Cost
Low Risk DesignsLow Volume
Historical map of transceiver speeds/applications10G Ethernet Serial Transceiver Historical Migration
Low Cost
High VolumeTooling Intensive
High Cost
Low Risk DesignsLow Volume
Historical map of transceiver speeds/applications100G Transceiver Migration
CFP
CFP2
QSFP28CXP
Driver VCSEL
DriverTX Lane 0TX Lane 1TX Lane 2TX Lane 3
NC
VCSEL
MPO12
Driver VCSELDriver VCSEL
NCRX Lane 0RX Lane 1RX Lane 2RX Lane 3 TIA PIN
TIA PINTIA PINTIA PIN
NCNC
MMF 40G 850nm TxRx ArchitectureProposed 40G MM Transceiver - 40GBASE-SR4
• “Extended Reach” transceivers now available– Cisco/Dell
• Operates as 4 x 10G– QSFP+ has 2.5X edge-density for 10GBASE-S
• Operates as 1 x 40G– 300m (OM3) or 400m (OM4) vs. 100m/150m for Std SR4 device
• Lower cost alternative to SM (40GBASE-LR4 QSFP+)– Lower CAPEX – Estimated 75%– Lower OPEX – 50% of power dissipation (1.5W vs. 3.5W)
40GBASE-eSR4 QSFP+Emerging ‘Defacto’ Standard
TIA PINTIA PINTIA PIN
TIA PINTIA PINTIA PIN
Driver VCSEL
DriverNC
TX Lane 0TX Lane 1TX Lane 2TX Lane 3TX Lane 4TX Lane 5TX Lane 6TX Lane 7TX Lane 8TX Lane 9
NC
VCSEL
MPO24
Driver VCSELDriver VCSEL
NCRX Lane 0RX Lane 1RX Lane 2RX Lane 3RX Lane 4RX Lane 5RX Lane 6RX Lane 7RX Lane 8RX Lane 9
NC
TIA PIN
TIA PINTIA PINTIA PIN
Driver VCSEL
Driver VCSELDriver VCSELDriver VCSELDriver VCSEL
Driver VCSELDriver VCSELDriver VCSEL
Driver VCSELDriver VCSEL
MMF 100G 850nm TxRx ArchitectureProposed 100G MM Transceiver - 100GBASE-SR10
Driver VCSEL
DriverTX Lane 0TX Lane 1TX Lane 2TX Lane 3
NC
VCSEL
MPO12
Driver VCSELDriver VCSEL
NCRX Lane 0RX Lane 1RX Lane 2RX Lane 3 TIA PIN
TIA PINTIA PINTIA PIN
NCNC
MMF 100G 850nm TxRx ArchitectureProposed 40G MM Transceiver - 40GBASE-SR4
SMF 4 x 10Gb/s TxRx Front-endProposed 40G SM Transceiver - 40GBASE-LR4
1x4deMUX
4 x 10GWDM ROSAw/ TIA/AGC
&CDR
RX Lane 3
RX Lane 2
RX Lane 1
RX Lane 0
LC Conn.
1x4MUX
LaserDrivers
&4 x 10G
WDM TOSA
TX Lane 3
TX Lane 2
TX Lane 1
TX Lane 0
LC Conn.
4 x 10Gb/sWDM
SMF 4 x 25Gb/s TxRx Front-end SM Transceiver - 100GBASE-LR4
1x4deMUX
4 x 25GWDM ROSAw/TIA/AGC
&CDR
RX Lane 3
RX Lane 2
RX Lane 1
RX Lane 0
LC Conn.
1x4MUX
LaserDrivers
&4 x 25G
WDM TOSA
TX Lane 3
TX Lane 2
TX Lane 1
TX Lane 0
LC Conn.
4 x 25Gb/sWDM
Going ForwardHigher Speed Roadmap
From http://www.ethernetalliance.org/roadmap/
Ethernet Roadmap
SM
Fiber Channel Roadmap
From http://www.ethernetalliance.org/roadmap/
Ethernet Roadmap(beyond 100G)
Ethernet Roadmap(beyond 16G)
Technology and Standards UpdatesFiber Media Devices
802.3 Media Device Interface (MDI)
Application 10GBASE-SR
25GBASE-SR
40GBASE-SR4
100GBASE-SR10
100GBASE-SR4**
Data Rate 10 Gbps 25 Gbps 40 Gbps 100 Gbps 100 Gbps
IEEE Std 802.3ae TBD 802.3ba 802.3ba 802.3bm
Form Factor SFP+ TBD QSFP+ CFP, CXP QSFP28, CFP4
Fiber Type OM3/4 OM3/4 OM3/4 OM3/4 OM3/4
Reach* 300/400m 70/100m? 100/150m 100/150m 70/100m
# of Fibers 2 2 12 (8 used) 24 (20 used) 12 (8 used)
Connectors
Duplex LC Duplex LC 12f MPO 24f MPO (2 x 12) 12f MPO
Schematic
*1.5 dB Link Budget **IEEE P802.3bm approved May 10, 2015
Technology and Standards UpdatesIEEE 802.3bs - Optical Proposals for 400GbE
Technology and Standards UpdatesEnabling Technology - PAM-4 Multilevel Encoding
• 4 distinct pulse amplitudes used• Amplitude represented by two bits 00,
01, 11, and 10 (a ‘symbol’)• One of the four amplitudes is transmitted
in a symbol period, there are two bits transmitted in parallel (data rate doubled)
• PAM-4 modulation is twice as bandwidth-efficient as binary modulation
100G SR4 Enabler
Extension of VCSEL emitter done in support of 16Gb/s Fiber Channel short wave transceivers (Finisar work in early 2008)
Several companies have research focused on direct modulation of VCSEL devices to support the serial short wave channel (Avago, Merge Optics, VIS, NEC, etc.)
Direct impact direction of future 100G MM implementations (4x25G instead of 10x10G)
Technology and Standards UpdatesEnabling Technology - VCSEL Technology
TIA TR-42.11 Optical Fiber SystemsNew Project: Wide Band Multimode (WBMMF)
• Support 4 or more wavelengths• Possibly transmit 40G or 100G over a pair of fibers instead of four or ten
pairs today
TIA TR-42.11 Optical Fiber SystemsNew Project: Wide Band Multimode (WBMMF)
• Standards Implementation - 1.5 years (Products in same time frame)• Proposal in TIA standards group for OM? Specification• Will be backwardly compatible (more expensive than OM4)• Will likely become the de facto standard• Will provide customer with more capacity without resorting to 32-fiber
channels - at least for now…(Only 4 wavelengths for now)• Will allow other CWDM solutions for intermediate speeds
– (2 or 4) x 20 Gb = 40 Gb or 80Gb Ethernet– 2 x 50 Gb = 100 Gb Ethernet
What lies ahead for 100G Enet/128GFC and beyond?
• ASIC Integration
San Jose, CA USAFebruary 2012 36
Chip Scale Photonics
• Gigabit Ethernet & 4/8G FC still workhorses of the typical data center but this is quickly changing to 10G & 16GFC
• 10G Ethernet is the largest revenue generator application in the data center for the next five years
• SFP+ 10GBASE-SR• OM3 and OM4 Fiber• SFP+ DAC 10GBASE-CR• RJ45 10GBASE-T• CAT6A/7 Twisted Pair
• 40G Ethernet is becoming mainstream for access to aggregation/distribution and core connections
• 100G Ethernet is the fastest growing networking technology in the data center over the next five years
Ethernet/FC SummaryHigher Speed Market View
Non-standards based transceivers
• Bidi Technology over two fiber SFP
• “Universal” Optics – For SM & Legacy Cable Plant
• Parallel Singlemode Optics - 40GBASE-PLR4
• Embedded Optics Multispeed Ports
Bidirectional Duplex SFPs
• BiDi – short for bidirectional• 40G Ethernet over two fibers• Allows use of existing LC infrastructure• Uses Wavelength Division Multiplexing – 2 x 20 Gbps signals
‘Universal’ Optics - 40GBASE-UNIV
• Addresses customer concerns around the reduced distances with 40GBASE-SR4
• Migrations from existing 10 to 40GbE networking without requiring a redesign or
expansion of the fiber network
• Supports operation over a full 150 m of OM3 or OM4
• Can be used for up to 500 m and interconnected with both 40GBASE-LR4 and
40GBASE-LRL4
40GBASE-UNIVQSFP
Parallel Singlemode Optics - 40GBASE-PLR4
• Parallel LR4 (PLRL4) supports distances compatible with 10GBASE-LR, (10km on SM
fiber). ‘Lite’ Parallel LR4 (PLRL4) supports distances compatible with 10GBASE-LRL,
(1km on SM fiber)
• Both modules can support 4 individual 10G-LR connections using a 4x10G mode
and fiber breakout cables or cassettes for single mode fiber
• PLR4 and PLRL4 use an MTP-12 connector, and require an APC (Angle polished
connector) single-mode MTP-12 cable
40GBASE-PLR4QSFP
Embedded Optics Multispeed Ports (‘SR12’)
Cable Plant Implications
Leaf-SpineLeaf/Spine Breakout Use Case - Network scaling example
Should you construct a Leaf/Spine fabric with 10G or 40G?
Let’s suppose you’ve decided to build a Leaf/Spine fabric for your data center network with the current crop of 10G/40G switches today that have QSFP ports. Each QSFP port can be configured as a single 1 x 40G interface, or 4 x 10G interfaces (using a breakout cable).
Simple example - build a data center fabric with primary goal of having, say, 1200 10G ports in one fabric. I also want the ability to seamlessly expand this fabric to over 5000 10G servers as necessary without increasing latency or oversubscription.
Leaf-Spine40G-Based Architecture
If I configure each QSFP port in the fabric as a single 40G interface, how wide will I be able to scale in terms of ports?
SPINE (x4 switches)
LEAF (x32 switches)40 ports x 10G(8 ports/switch unused)
1280, 10G ports
Port Mapping(Ethx/1 - ports 0,1,2,3)
Ethx/1 Ethx/2 Ethx/2
Port Grouping on switch
‘Aggregation’ Breakout (1-MPO to 4-LC)Application to 40G Switch
Leaf-Spine10G-Based Architecture
If I configure each QSFP port in the fabric as 4 x 10G interfaces and using an optical breakout cable, how wide will I be able to scale in terms of servers?
SPINE (x16 switches)
LEAF (x128 switches)40 ports x 10G(8 ports/switch unused)
5120, 10G ports
How do we accomplish 100G Breakout to 40G for SR10?
With the SR10 transceiver we are in the unfortunate position of only being able to support 2 40G ports…..
Unlike the ‘SR12’ which can support 3, 40 G ports (because it is really a 120G transceiver in disguise!!)
Looking into the CPAK MPO the top leftmost Rx is dark, therefore 8 of the 10 Rxs in this row can support 2 40G breakouts
Same goes for the bottom Tx row (leftmost Tx is dark), therefore 8 of the TXs in the bottom row can support the Tx side of 2 40G channels
How do we accomplish 100G Breakout to 10G & 40G for ‘SR 12’?
With the ‘SR12’ transceiver we can support 12, 10G breakouts
With the ‘SR12’ transceiver we can support 3, 40G breakouts
SAN Fiber Channel exampleHigh Performance, Storage-Centric ‘Flat’ Network N Architecture
• Design adequate for moderate-size SANs, difficult to scale beyond 1000 ports. Three 256-port directors on each of the A and B sides, for example, would provide 768 ports for direct server and storage connections
• Adding a fourth or fifth director to each side would increase costs, complicate the cable plant, and increase the complexity of the SAN and its management
New FC Blade Technology64G Blades
4-channel QSFP (Quad Small Form-factor Pluggable) optic on the FC16-64 blade enables high density port configuration
Support individual SFP+ 8/16G breakouts
FC16-64 Blade
How Do We Make Really BIG SAN Switches?Old School vs. ‘New’ School
Fiber ICL PortUp to 4 x 16 Gb/s per ICL port
Math for fully populated UltraScale Mesh12 fibers/QSFP x 16/blade x 2 blades = 384 fibers/chassis
&2/blade x 2 (A/B) x 36(mesh) = 144 cables
Max Configuration = 9 x DCX(4608 16 Gb/s per cluster)
UltraScale Mesh with DCX 8510Do We Make Really BIG SAN Switches?
Planning & Best Practices
TIA 568C.0 Single Row Parallel Transmission with array cables - Method ‘B’
Parallel Optics Cable PlantParallel Transmission Example - Ethernet/Fiber Channel
Serial Duplex Cable PlantCassette Model with Cross Connect
Three Different Cords!!!
Migration to Parallel Optics Cable PlantMPO Adapter Panels Model with Cross Connect
One Cord Type
Migration to Parallel Optics Cable PlantMPO Adapter Panels Model with Cross Connect - Gender Changing MPO Trunks
Trade-off between SCS ‘wants’ and IEEE requirements
100 meter OM3 channel with two 0.75dB (Max.) connectors (1.5dB connector insertion loss total)
150 meter OM4 channel with two 0.50dB (Max.) connectors (1.0dB connector insertion loss total)
“Engineered Link”
0.100.200.300.400.500.600.700.800.901.001.101.201.301.401.501.60
100 110 120 130 140 150 160 170 180 190 200
Tota
l Con
nect
or L
oss (
dB)
Maximum Reach (m)
OM3OM4
Source: Panduit extrapolation from IEEE model
MPO-Based 40/100G CablingLink Power Budgeting for Cabling
Migration to Parallel Optics Cable PlantDark Fiber Mitigation - Interconnect Channel
Center four fibers from trunk assemblies routed to 3rd (new) MPO/MTP on cassette front. 4 transmit from trunk ‘A’ and 4 receive from trunk ‘B’
Existing “Day One” 12 fiber MPO/MTP trunks
“Day Two” 8 fiber MPO/MTP patch cords
Migration to Parallel Optics Cable Plant24 Fiber MPO-Based Cable Plant
24 fiber MPO permanent infrastructure can remain constant through cable plant migration from 10G to 40G/100G
But the real beauty in going back to 24 fiber cassettes is the ability to support multiple breakout scenarios in a small form-factor footprint and minimization of the volume (number of legs) of trunk cabling entering the fiber distribution system
24 fiber cassettes(RED – 24 Fiber MPO)
Summary
Highlights and Take-Aways
• Standards Implementation of transceivers – multivendor support and interoperability for non-standards-based PMDs
• Wideband fiber, WDM, Bidi etc. are enablers of “customer-friendly” solutions that extend the lifetime of existing cable plant
• Breakout solutions for transceivers (BASE-4 ports) are most important for network scale in flat architectures (for both LAN & SAN)
• Network-ready status of cable plant for higher speeds and dark fiber can be mitigated by transceiver selection and the choice of cable plant
