Cross-Connect Design for Spine and Leaf Architectures
David KozischekMarketing Manager Enterprise NetworksCorning Cable Systems
October 2013
Telecom ©2013 Corning Incorporated 2
Cross-Connect Design for Spine and Leaf Architectures Outline
Spine and Leaf Design Key areas
Background • Based on Cisco’s MSDC Guide• Cisco Products used For 10G Spine and Leaf
What is Port Mapping• Mapping Logical Architecture to Physical Cabling• Mapping Physical Switch Ports to Patch Panels• Understanding key Corning Products Port Maps
Port Mapping Cisco Switches
• Cisco Nexus 3064 Switch to Corning Product• Cisco Nexus 7010 Switch to Corning Product• Cisco Nexus 7018 Switch to Corning Product• Cross-Connect Patching
Design Example
• Cisco Architectures with Key Design Parameters• Corning Cable Systems Products • Port Mapping Design and BOM – 4-Way Spine• Port Mapping Design and BOM – 8-Way Spine• Port Mapping Design and BOM – 16-Way Spine
Appendix: Leaf Interconnect Area
• Design and BOM – 4-Way Spine• Design and BOM – 8-Way Spine• Design and BOM – 16-Way Spine
Telecom ©2013 Corning Incorporated 3
Cross-Connect Design for Spine and Leaf Architectures Outline
Spine and Leaf Design Key areas
Background • Based on Cisco’s MSDC Guide• Cisco Products used For 10G Spine and Leaf
What is Port Mapping• Mapping Logical Architecture to Physical Cabling• Mapping Physical Switch Ports to Patch Panels• Understanding key Corning Products Port Maps
Port Mapping Cisco Switches
• Cisco Nexus 3064 Switch to Corning Product• Cisco Nexus 7010 Switch to Corning Product• Cisco Nexus 7018 Switch to Corning Product• Cross-Connect Patching
Design Example
• Cisco Architectures with Key Design Parameters• Corning Cable Systems Products • Port Mapping Design and BOM – 4 Way Spine• Port Mapping Design and BOM – 8 Way Spine• Port Mapping Design and BOM – 16 Way Spine
Appendix: Leaf Interconnect Area
• Design and BOM – 4-Way Spine• Design and BOM – 8-Way Spine• Design and BOM – 16-Way Spine
Telecom ©2013 Corning Incorporated 4
Cross-Connect Design for Spine and Leaf Architectures Based on Cisco’s Massively Scalable Data Center Overview
http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/MSDC/1.0/MSDC_Overview_1.pdf
Telecom ©2013 Corning Incorporated 5
Area Corning Cable Systems Product Other Components
Spine • Cisco 7000 Series • 48 Port 1/10Gig Ethernet Blade
Leaf
• Cisco 3064 • 48 SFP+ ports• 4 QSFP Ports
Cross-Connect Design for Spine and Leaf Architectures Cisco Products used For 10G Spine and Leaf
Telecom ©2013 Corning Incorporated 6
Cross-Connect Design for Spine and Leaf Architectures Outline
Spine and Leaf Design Key areas
Background • Based on Cisco’s MSDC Guide• Cisco Products used For 10G Spine and Leaf
What is Port Mapping• Mapping Logical Architecture to Physical Cabling• Mapping Physical Switch Ports to Patch Panels• Understanding key Corning Products Port Maps
Port Mapping Cisco Switches
• Cisco Nexus 3064 Switch to Corning Product• Cisco Nexus 7010 Switch to Corning Product• Cisco Nexus 7018 Switch to Corning Product• Cross-Connect Patching
Design Example
• Cisco Architectures with Key Design Parameters• Corning Cable Systems Products • Port Mapping Design and BOM – 4 Way Spine• Port Mapping Design and BOM – 8 Way Spine• Port Mapping Design and BOM – 16 Way Spine
Appendix: Leaf Interconnect Area
• Design and BOM – 4-Way Spine• Design and BOM – 8-Way Spine• Design and BOM – 16-Way Spine
Telecom ©2013 Corning Incorporated 7
Logical Architecture Physical Cabling
Cross-Connect Design for Spine and Leaf Architectures Mapping Logical Architecture to Physical Cabling
Cisco Nexus 3064
32 Servers
Cisco Nexus 7000
Cisco Nexus 3064
32 Servers
Cisco Nexus 7000
Main Distribution Area (Main Cross-Connect)
Telecom ©2013 Corning Incorporated 8
Logical Architecture
Cross-Connect Design for Spine and Leaf Architectures Mapping Physical Switch Ports to Patch Panel
Cisco Nexus3064
32 Servers
Cisco Nexus 7000
Main Distribution Area (Main Cross-Connect)
Spine Switch
Leaf Switch
Telecom ©2013 Corning Incorporated 9
Cross-Connect Design for Spine and Leaf Architectures Port Mapping – Understanding Key Corning Products
Cross-Connect Building Blocks Corning Cable Systems Product Port Map
Module• Pretium
EDGE® OM4 Module
• 6 Duplex LC Ports per Module
Housing• Pretium
EDGE 4UHousing
• 12 Shelves• 4 Modules/24 ports per Shelf• 288 Duplex ports
1 2 3 4 5 6
Telecom ©2013 Corning Incorporated 10
Cross-Connect Design for Spine and Leaf Architectures Outline
Spine and Leaf Design Key areas
Background • Based on Cisco’s MSDC Guide• Cisco Products used For 10G Spine and Leaf
What is Port Mapping• Mapping Logical Architecture to Physical Cabling• Mapping Physical Switch Ports to Patch Panels• Understanding key Corning Products Port Maps
Port Mapping Cisco Switches
• Cisco 3064 Switch to Corning Product• Cisco 7010 Switch to Corning Product• Cisco 7018 Switch to Corning Product• Cross-Connect Patching
Design Example
• Cisco Architectures with Key Design Parameters• Corning Cable Systems Products • Port Mapping Design and BOM – 4 Way Spine• Port Mapping Design and BOM – 8 Way Spine• Port Mapping Design and BOM – 16 Way Spine
Appendix: Leaf Interconnect Area
• Design and BOM – 4-Way Spine• Design and BOM – 8-Way Spine• Design and BOM – 16-Way Spine
Telecom ©2013 Corning Incorporated 11
Cross-Connect Design for Spine and Leaf Architectures Port Mapping – Cisco Nexus 3064 Switch to Corning Product
Cisco Switch Total Ports Port Map to Pretium EDGE® Housing BOM for 100% Population
Cisco Nexus 3064
• 64• 32 ports
facing fabric
• 8 Leaf Switches per 4U Pretium EDGE Housing
• (1) Pretium EDGE 4U Housing
• EDGE-04U
• (48) Pretium EDGE Modules
• ECM-UM12-05-93Q
• (8) 72-Fiber OM4 Pretium EDGE MTP® Trunk Cables
• G757572QPNDDUxxxF
• (256) OM4 Pretium EDGE Reverse Polarity Jumpers
• 797902QD120003M
Leaf 1
Leaf 2
Leaf 3
Leaf 4
Leaf 5
Leaf 6
Leaf 7
Leaf 8
Telecom ©2013 Corning Incorporated 12
Cross-Connect Design for Spine and Leaf Architectures Port Mapping – Cisco Nexus 7010 Switch to Corning Product
Cisco Switch
Total Ports(48-Port
Line Cards)Port Map to Pretium EDGE® Housing BOM for 100% Population
Cisco Nexus 7010
• 384 ports• 8 Line
Cards
• (6) 48-Port Line cards per EDGE-04U
• (2) Pretium EDGE 4U Housings
• EDGE-04U
• (64) Pretium EDGE Modules
• ECM-UM12-05-93Q
• (64) OM4 Pretium EDGE Module Harnesses
• H757912QPH-3W010F
• (384) OM4 Pretium EDGE Reverse Polarity Jumpers
• 797902QD120003M
Card 1
Card 2
Card 3
Card 4
Card 5
Card 6
Card 7
Card 8
Telecom ©2013 Corning Incorporated 13
Cross-Connect Design for Spine and Leaf Architectures Port Mapping – Cisco Nexus 7018 Switch to Corning Product
Cisco Switch
Total Ports(48-Port
Line Cards)Port Map to Pretium EDGE® Housing BOM for 100% Population
Cisco Nexus7018
• 768 ports• 16 Line
Cards
• (6) 48-Port Line cards per EDGE-04U
• (3) Pretium EDGE Housings
• EDGE 04U
• (128) Pretium EDGE Modules
• ECM-UM12-05-93Q
• (128) OM4 Pretium EDGE Module Harness
• H757912QPH-3W010F
• (768) OM4 Pretium EDGE Reverse Polarity Jumpers
• 797902QD120003M
Card 1
Card 2
Card 3
Card 4
Card 5
Card 6
Card 7
Card 8
Card 9
Card 10
Card 11
Card 12
Card 13
Card 14
Card 15
Card 16
Telecom ©2013 Corning Incorporated 14
Cross-connect Design for Spine and Leaf Architectures Port Mapping – Cross Connect Patching
Leaf 1
Main Dist Area (Main Cross Connect)
Leaf 2
Leaf 3
Leaf 4
Leaf 5
Leaf 6
Leaf 7
Leaf 8
Spine + Leaf
Cabling Architecture
Spine Switch
Max Dist 150M (OM4)Co-locate Spine at MDA
Spine + Leaf
Cross Connect Patching
72F72F72F72F72F72F72F72F
Telecom ©2013 Corning Incorporated 15
Cross-Connect Design for Spine and Leaf Architectures Outline
Spine and Leaf Design Key areas
Background • Based on Cisco’s MSDC Guide• Cisco Products used For 10G Spine and Leaf
What is Port Mapping• Mapping Logical Architecture to Physical Cabling• Mapping Physical Switch Ports to Patch Panels• Understanding key Corning Products Port Maps
Port Mapping Cisco Switches
• Cisco Nexus3064 Switch to Corning Product• Cisco Nexus 7010 Switch to Corning Product• Cisco Nexus 7018 Switch to Corning Product• Cross-Connect Patching
Design Example
• Cisco Architectures with Key Design Parameters• Corning Cable Systems Products• Port Mapping Design and BOM – 4-Way Spine• Port Mapping Design and BOM – 8-Way Spine• Port Mapping Design and BOM – 16-Way Spine
Appendix: Leaf Interconnect Area
• Design and BOM – 4-Way Spine• Design and BOM – 8-Way Spine• Design and BOM – 16-Way Spine
Telecom ©2013 Corning Incorporated 16
Cross-Connect Design for Spine and Leaf Architectures Cisco Architectures with Key Design Parameters
Key Design Parameters:• All 10G Ethernet, no 40G• Spine Switches: Cisco Nexus 7000 series with 48-port blades• Leaf Switches Cisco Nexus 3064
• 32 Ports Facing Fabric, 32 Ports Facing Servers
Spine Configuration/Device Count 4-way Spine 8-way Spine 16-way Spine
Number of Nexus 70xx SpineSwitches
4x 7009 or4x 7010
8x 7009 or 8x 7010 or8x 7018
16x 7018
Number of N7K-F248XP-25 Blades per Chassis
7009: 77010: 8
7009: 77010: 87018: 16
7018: 16
Number of Ports Used for Leaf Switches per Chassis
7009: 3367010: 384
7009: 3367010: 3847018: 768
7018: 768
Number of Nexus 3064 Leaf Switches 48 96 192
Number of Nexus 3064 Ports Facing Fabric 32 32 32
Number of Nexus 3064 Ports Facing Servers 32 32 32
Telecom ©2013 Corning Incorporated 17
Cross-Connect Design for Spine and Leaf Architectures Cisco Architectures with Key Design Parameters
Cisco Nexus 3064
32 Servers
Cisco Nexus 7000
Main Distribution Area (Main Cross-Connect)
Fabric A
Fabric B
72F Cables
12F Harnesses
Telecom ©2013 Corning Incorporated 18
Cross-Connect Design for Spine and Leaf Architectures Corning Cable Systems Products
Cross-Connect Building Blocks
Corning Cable SystemsProduct Product Picture Part Number
Cable • 72-Fiber OM4 Pretium EDGE® Trunk Cable • G757572QPNDDUxxxF
Hardware • Pretium EDGE Housings• EDGE-04U• EDGE-02U• EDGE-01U-SP
Module • OM4 Pretium EDGE Module • ECM-UM12-05-93Q
Harness
• 12-Fiber OM4 Pretium EDGE Module Harness for Cisco 7000 48-port Blades
• H757912QPH-3W010F
Harness• 8-Fiber OM4 Pretium
EDGE QSFP+ to SFP+ Switch Harness
• H757908QPH-KB007F
Jumper • OM4 Reverse Polarity LC Duplex Jumper • 797902QD120003M
Telecom ©2013 Corning Incorporated 19
Cross-Connect Design for Spine and Leaf Architectures Port Mapping Design and BOM – 4-Way Spine
Trunks to TOR Cisco Nexus 3064
Harnesses to Cisco Nexus 70xx
Spine BOM for 100% Population
• Housings: (5) EDGE-04U(1) EDGE-02U
• Modules:(256) ECM-UM12-05-93Q
• Module Harnesses: (256) H757912QPH-3W010
• Jumpers:(1728) 797902QD120003M
LEAF BOM for 100% Population
• Housings: (6) EDGE-04U
• Modules: (288) ECM-UM12-05-93Q
• Cables: (48) G757572QPNDDUxxxF
4 Spine Switches
Cisco Nexus 7010
48 Leaf Switches
Cisco Nexus 3064
Spine FrameLeaf Frame
(48) 72F Cables (256) 12F Harnesses
Port
Map
Port
Map
Telecom ©2013 Corning Incorporated 20
Cross-Connect Design for Spine and Leaf Architectures Port Mapping Design and BOM – 8-Way Spine
Spine BOM for100% Population
• Housings: (11) EDGE-04U
• Modules: (512) ECM-UM12-05-93Q
• Module Harnesses:(512) H757912QPH-3W010
• Jumpers(3456) 797902QD120003M
LEAF BOM for100% Population
• Housings: (12) EDGE-04U
• Modules: (576) ECM-UM12-05-93Q
• Cables: (96) G757572QPNDDUxxxF
8 Spine Switches
Cisco Nexus 7010
96 Leaf Switches
Cisco Nexus 3064
Trunks to TORCisco Nexus 3064
Harnesses to Cisco Nexus 70xx
Spine FrameLeaf FrameLeaf Frame
(96) 72F Cables
(512) 12F Harnesses
Port
Map
Port
Map
Telecom ©2013 Corning Incorporated 21
Cross-Connect Design for Spine and Leaf Architectures Port Mapping Design and BOM – 16-Way Spine
Trunks to TOR Cisco Nexus 3064 Harnesses to Cisco Nexus 70xx
16 Spine Switches
Cisco Nexus 7010
192 Leaf Switches
Cisco Nexus 3064
Leaf BOM for 100% Population
• Housings: (24) EDGE-04U
• Modules: (1,152) ECM-UM12-05-93Q
• Cables: (192) G757572QPNDDUxxxF
Spine FrameLeaf Frame Spine FrameLeaf Frame
Spine BOM for 100% Population
• Housings:(22) EDGE-04U
• Modules: (1,024) ECM-UM12-05-93Q
• Module Harnesses: (1,024) H757912QPH-3W010
• Jumpers:(6912) 797902QD120003M
Port Map
Port
Map
(192) 72F Cables (1024) 12F Harnesses
Telecom ©2013 Corning Incorporated 22
Cross-Connect Design for Spine and Leaf Architectures Outline
Spine and Leaf Design Key areas
Background • Based on Cisco’s MSDC Guide• Cisco Products used For 10G Spine and Leaf
What is Port Mapping• Mapping Logical Architecture to Physical Cabling• Mapping Physical Switch Ports to Patch Panels• Understanding key Corning Products Port Maps
Port Mapping Cisco Switches
• Cisco Nexus 3064 Switch to Corning Product• Cisco Nexus 7010 Switch to Corning Product• Cisco Nexus 7018 Switch to Corning Product• Cross-Connect Patching
Design Example
• Cisco Architectures with Key Design Parameters• Corning Cable Systems Products • Port Mapping Design and BOM – 4 Way Spine• Port Mapping Design and BOM – 8 Way Spine• Port Mapping Design and BOM – 16 Way Spine
Appendix: Leaf Interconnect Area
• Design and BOM – 4-Way Spine• Design and BOM – 8-Way Spine• Design and BOM – 16-Way Spine
Telecom ©2013 Corning Incorporated 23
Interconnect Design Leaf ArchitecturePort Mapping Design and BOM – Cisco Nexus 3064 Interconnect Map
1 Cis
co N
exus
30
64
Inte
rcon
nect
EDG
E-01
U-S
P
24
36
32 Servers
32 Fabric Ports
Telecom ©2013 Corning Incorporated 24
Interconnect Design Leaf ArchitecturePort Mapping Design and BOM per Rack
(2) 72F Trunks To MDA Cross-Connect
Spine Switches
Cisco Nexus7000
Leaf BOM for 100% Population per rack
• Housings: (1) EDGE-01U-SP
• Modules: (12) ECM-UM12-05-93Q
• Jumpers:(96) 797902QD120003M
• Harnesses:(8) H757908QPH-KB007F
• Trunks: (2) G757572QPNDDUxxxF
Fabric A
Fabric B
(32) 2-F Jumpers(32) 2-F Jumpers
Leaf 1 Fabric A
Leaf 2 Fabric B
Telecom ©2013 Corning Incorporated 25
Total BOM Assuming All Spine Switches are Cisco Nexus 7010
Product Product Description 4-way Spine
8-way Spine
16-way Spine
EDGE-04U 4U Pretium EDGE® Housing, Holds 48 Modules 11 23 46
EDGE-02U 2U Pretium EDGE Housing, Holds 24 Modules 1
EDGE-01U-SP 1U Pretium EDGE Housing, Holds 12 Modules 24 48 96
ECM-UM12-05-93Q 12-Fiber/6-Port Pretium EDGE OM4 Module 832 1,664 3,328
H757912QPH-3W010F12-Fiber/6-Port Pretium EDGE OM4 Module Harness with Cisco Nexus 7k Leg Stagger
48 96 192
H757908QPH-KB007F 8-Fiber/4-Port Pretium EDGE OM4 QSFP+ to SFP+ Switch Harness 192 384 768
G757572QPNDDUxxxF 72-Fiber Pretium EDGE MTP® to MTP Trunk 48 96 192
797902QD120003M 2-Fiber OM4 Reverse Polarity LC Duplex Uniboot Jumper, 3 meters 4,032 8,064 16,128