Dynamic Changes in 40/100G Transceivers and the Impact to Structured Fiber Cabling

Rick Dallmann, CABLExpress

36 Years of Experience

CABLExpress is a manufacturer of custom built structured cabling solutions focused on high-end data center, network and SAN environments.

RCDD (R i t d C i ti Di t ib ti D i & DCDC)• RCDDs (Registered Communications Distribution Designer & DCDC); CDCD on staff

• Subject matter expert presenters at: AFCOM, BICSI, DatacenterDynamics(DCD) conferences and FOSE( )

• Expert structured connectivity data center design, spec and implementation according to TIA-942-A standards and IEEE 802.3 standards

d h d l d f d d h• OEM and Authorized Solution Provider for Brocade, Arista, HP and others

• ISO 9001:2008 process and design certified—qualification documented and verified by third party

Agenda

• Data Center Technology Trends

• The Optics and History• The Optics and History

• Standards and Solutions– Ethernet (25/40/100/400 Gigabit)

– Fibre Channel (16/32/64/128 Gigabit)

• Cabling Design Considerations and Challenges– Duplex/Serialp /

– Parallel Optics

• Questions

The Need for Speed

• Rapid growth of server, network, and internet traffic

• Higher density requirements in the data center• Higher density requirements in the data center

• Low cost optical fiber Ethernet/SAN solutions

• Advances in technologies now allow the specification of the new 40/100/400G physical layer types with reduced lane count and complexity… Lower cost results!

Next Generation Data Rates - Prepare Today, Avoid Upgrades Later

• IEEE 802.3ba Standards– OM3 glass support 40/100 gig speeds to 100 meters– OM3 glass support 40/100 gig speeds to 100 meters

– OM4 glass support 40/100 gig speeds to 150 meters

• IEEE 802.3bm Standards (Planned Release CY2015)100GB SR d 400G– 100GBase-SR and 400G

– Transceiver speeds will jump from 10G per fiber or lane to 25G

– Duplex LC: Bi-Di and universal 40G Ethernet

Fib Ch l• Fibre Channel– 32/64/128 Gigabit Fibre Channel

A Look Back in History

Historically, fiber connectors had discrete ferrules for each fiber. Now, 12 fiber MT ferrules are standard in high

LCDuplex fiber. Now, 12 fiber MT ferrules are standard in high

density connectors.Duplex

Connector

MTP® connectors can have

12 fiber MT ferrule

12 to 72 fibers in one MT () ferrule. Mass termination.

24 fiberMT ferrules

The Optics

EMB (Effective Modal Bandwidth)• VCSEL’s• Power distributed in a narrow

region

OFL (Overfilled Launch)• LEDs, not lasers• Power distributed over 100% of the fiber core• PRE 1 GIG Transceivers region

• More accurate indication of performance in high-speed laser-based systems

Diff t VCELS fill diff t t f d i h fib hi h ff t l diDifferent VCELS fill a different set of modes in each fiber, which can affect pulse spreading.

Data Center Trends: Network Data Rates

Rack Servers peak for 10G in 2019 while cloud servers peaked in 2014. Cloud customers deploying 25G and above in 2015p y g

Data Center Trends: Network Data Rates

Significant growth of 25G rack and 40/100G blades

802.3ba Standards Document

802 3ba Standards Document802.3ba Standards Document• Ratified 06/17/2010

• Coverso Physical Layer

o OM3 glass support 40/100 gig speeds to 100 meters

l /o OM4 glass support 40/100 gig speeds to 150 meters

The MTP® ferrule: 40 and 100G

For data rates of 40G and 100G Ethernet over OM3 fiber and OM4, parallel optics will be the transmission method

12 & 24 fiberMT ferrules

Infrastructure must be SKEW tested

Demand DocumentationDemand Documentation

Serial vs. Parallel Transmission

Typical SERIAL Transceiver

Serial vs. Parallel Transmission

Serial vs. Parallel Transmission

RX TX

Typical SERIAL Transceiver

Serial vs. Parallel Transmissionskew

40/100/400G Transceivers Roadmap

Foot Print Distance 40G 100G 400GDuplex Fiber OM3/OM4 100-150m BiDi/WDM over MM

Serial 40G UniversalBiDi WDM ??

Serial 40G Universal

Parallel OM3/OM4 100-150m SR4/eSR44x10G

Gen1: SR1010x10GGen2: SR4 4x25G

Gen1: SR1616x25GGen2: SR8 8x50GGen3: SR4 4x100GGen3: SR4 4x100G

Duplex Fiber SM 2-10km LR4(10km)IR4(2km)WDM(500m)

LR4(10km)CLR4(2km)CWDM4(2km)

?

Parallel SM 300-1000m PLR4 PSM4 Gen1: PSMGen2: PSM44x100G

• 40G solutions have SM/MM duplex and // solutions and development for 100G• 400G shows similar path

40/100/400G Transceivers Roadmap

• eSR4 Parallel Optics Extended Reach– Testing showing over 1000m +Testing showing over 1000m +

– Branded cSR4(Cisco) and xSR4(Arista)

– Sr4/eSR4 capable to 4x10G breakouts

• BiDi: 40G qsfp + (Cisco)– OM3 to 100M w/ 1.5db link loss budget

– OM4 to 150M w/ 1.0db link loss budgetOM4 to 150M w/ 1.0db link loss budget

– Cannot breakout into 10G links

40/100/400G Transceivers Roadmap

• 40G “Universal” cWDMUtili 40G(4 10G) di i i lti l i i l d l h– Utilizes a 40G(4x10G) coarse wave division multiplexing single-mode launch

– The four 10G wavelengths operate in the 1310nm region and comply to the IEEE802.3ba 40GBASE-LR4 standard

– Branded QSFP-40G-UNIV (Arista)• OM3/OM4 to 150m with 2.1 db(1300nm) link loss budget

– Branded QSFP-40GE-LX4 (Juniper)• OM3 to 100M w/ 1.9db(1300nm) link loss budget

• OM4 to 150M w/2.1db(1300nm) link loss budget

IEEE 802.3bm: 100G(revised) & 400G Ethernet

• Publication Planned for CY 2015

• 100GBase-SR4 to be released

• 400G to be released

T i d ill j f 10G• Transceiver speeds will jump from 10G per fiber to 25G on multimode fiber.

• 4 fibers X 25G = 100G• 4 fibers X 25G = 100G

Ethernet alliance.org

IEEE 802.3bm: 100G(revised) & 400G Ethernet

Projected 400G Ethernet connectivity:25G fib t ti ll 50 56G fib• 25G per fiber or potentially 50-56G per fiber

• 25G per fiber will use a 16 fiber MPO/MTP 16 fibers X 25G = 400G

Ethernet alliance.org

Cabling for 40G and 100G Considerations

Cabling for High-Speed Fibre Channel

• For lowest cost, 128G-FC recommends same metrics as for 100GBASE SR4 modules to avoid multipleas for 100GBASE-SR4 modules, to avoid multiple tests of similar/related attributes.

• In addition, more component and module data is needed to guide which attributes can be upgraded to meet a 100m OM4 reach objective for 128G-FC.j

Fibre Channel 128GFCp

16GFC or32GFC SFP+

16GFC or32GFC SFP+

QSFP 28

32GFC SFP

16GFC or32GFC SFP+

8 fiber/ 4 port MTP®-LCHarness

QSFP 28

16GFC or32GFC SFP+

Cabling Standards

Cabling design should be simple and effective by following the applicable standards and also be flexible enough tothe applicable standards and also be flexible enough to easily adapt to the dynamic footprint of each end-user.

Cabling for 40G and 100G Considerations

• Deployment of 40G being planned

– True 40G link to 4x10G link

– sR4; Parallel Optics MM

– cSR4/xSR4: extended reach Parallel Optics MM

– 40G BiDi, WDM, or Universal

Cabling for 40G and 100G Considerations

• Migration of existing cabling infrastructure or new install?

• How will DC infrastructure look or perform? – Direct Connect ISLs

– Point to Point Cabling

Structured Cabling; Port replication– Structured Cabling; Port replication

Cabling Methodologies

“65% of system outages arerelated to cabling”

AFCOM: Data Center Institute / Gartner estimate

“85% SAN Trouble Tickets are caused by cabling”

Brocade

Cabling Methodologies

Point-to-Point…Point to Point…

The wrong way…g y

Let’s look…

Point-to-Point: No Structure

Design, Installation and Products

History has shown (hundreds of visited data centers) that loss is based on the design, installation, and product choices

Loss That NEEDS to be Avoided

Lets Examine The Results

dB Link Loss for Transmission

Loss budget… Scary?Year Application Data Rate Standard Loss Budget (dB)Year Application Data Rate Standard Loss Budget (dB)

1982 Ethernet 10 Mbps IEEE 802.3 12.5

1991 Fast Ethernet 100 Mbps IEEE 802.3 11.0

1998 Short Wavelength Fast Ethernet 10/100 Mbps TIA/EIA-785 4.0

2000 1G Ethernet 1,000 Mbps IEEE 802.3z 3.56

2004 8&10G FC &10G Ethernet 10,000 Mbps IEEE 802.3ae 2.60

2010 16G FC & 40G Ethernet 40,000 Mbps IEEE 802.3ba 1.9

2010 100G Ethernet 100,000 Mbps IEEE 802.3ba 1.5

dB Link Loss for Transmission

Insertion loss is a critical performance parameter in current data center cabling deployments.

• Total connector loss impacts the ability to operate over the maximum supportable distance.

• As total connector loss increases, the supportable distance at that data rate decreases.

16/40/100G standard

• OM3; 100m distance; maximum channel loss of 1.9 dB, which includes1.5 dB total connector loss budget.

• OM4; 150m distance maximum channel loss of 1.5 dB, which includes1.0 dB total connector loss budget.

The maximum cabled fiber attenuation is 3.5 dB/km at 850 nm. Thus, the insertion loss specifications of connectivity components should be evaluated when designing data center cabling infrastructures With low-loss connectivity componentscabling infrastructures. With low-loss connectivity components.

IEEE 802.3bm - OM3 versus OM4 Glass

1G 10G 40G 100G 100G2 2 8 20 8

OM1 275 m 26 m ‐ ‐ ‐OM2 550 m 82 m ‐ ‐ ‐OM3 800 m 300 m 100 m 100 m 20m*OM3 800 m 300 m 100 m 100 m 20mOM4 1040 m 550 m 150 m 150 m 106m*

*Distances in red are specified by manufacturers but not in IEEE standardsDistances in red are specified by manufacturers but not in IEEE standards.*IEEE 802.3bm standard for 4‐lane 100G target 1Q2015

In 2005, Infrastructure Standards for Data Centers…

TIA 942 ATIA-942-Ao Site and space layoutso Site and space layoutso Tiered reliabilityo Environmental considerationso Cabling infrastructure

TIA-942: Distributed Data Center Topology

Entrance Room

• Analogy: “Entrance Facility”gy y

Main Distribution Area (MDA)

• Analogy: “Equipment Room”

Horizontal Distribution Area (HDA)

• Analogy: “Telecom Room”

Zone Distribution Area (ZDA)

• Analogy: “Consolidation Point”

Equipment Distribution Area (EDA)

• Analogy: “Work Area”

• Tier 3 Data Center!!!

Leaf and Spine

TIA-942: Distributed Data Center Topology

Interconnect dB Loss

The Industry Standard

EIA/TIA 568-C 3EIA/TIA 568-C.3

d / dLC 0.5dB/mated pair

SC 0.5dB/mated pair

MTP® 0.75dB/mated pair

Proper Product Selection Is Key

MTP® to LC conversion harness

MTP® cassette modules Various density enclosures

MTP® conversion modules

MTP®-MTP® multi-fiber trunks

Loss and Attenuation Performance

Standard Product : Max/Mated Pair

• LC 0.15dB/mated pair

MPO/MTP® 12F 0 20db/ t d i• MPO/MTP®- 12F 0.20db/mated pair

• MPO/MTP®- 24F 0.35db/mated pair

Demand Premium Performance

10G and 40G Together!

Realized Cost Savings!

40G Link Options

Properly planned, seamless transition

100G Link Options

Properly planned, seamless transition

100/40/10G Migrations

Properly planned, seamless transition

Cabling for 40G and 100G Considerations

• 40/100/400G will operate over both duplex and parallel optics

• The majority of 40/100G initial implementations will be 4x10G or 4x25G breakouts

– Fibre Channel data rates of 64G and 128G will also be supported by 4x16G and 4x32G breakouts

• MTP®-based cabling is the most effective way to future-proof structured cabling for future technologies and implementationsstructured cabling for future technologies and implementations

Questions?

www.cablexpress.com