Date post: | 16-Jul-2015 |
Category: |
Technology |
Upload: | jacyn1 |
View: | 409 times |
Download: | 0 times |
Whataburger Network Stabilization Single Point Of Failure Optimization
Objective
• To improve overall performance of the Whataburger network - Three areas are being addressed:
– Sprint Network
– Network Connectivity at the Units
– Internal Network Design at ADC
History
• Over the last 7 months…– Two major business impacts due to Fortigate firmware
– Six major Sprint MPLS outages
– Two major local fiber cuts (one was a single fiber cut, the other was both main network connections to ADC)
– Outage with Cisco core equipment at ADC
Whataburger Network Design Overview
Corpus Christi, TX(Corporate HQ)
Irving, TX(East Division)
Austin, TX(Data Center)
OC-3
OC-3
OC-3
SPRINT GLOBAL MPLS VPN
(Fully MeshedAny to Any connectivity ))
OC-3
San Antonio,TX(West Division)
OC-3
7206VXR 7206VXR
OC-3
7206VXR
7206VXR
SBC/AT&T Frame
Approx. 100 restaurants
Approx. 450+ restaurants
CDMA Data LinkWWAN
Sprint Network
• Whataburger Network Single Points of Failure– Austin Data Center
• Two local circuits from Sprint’s Point Of Presence (POP) to ADC follow common terrestrial path and enter ADC at single entry point
– Corpus Christi Data Center• Two local circuits from the CCDC and Sprint’s Point Of
Presence (POP) follow common terrestrial path
– Remote Offices (Irving & San Antonio) have only one network connection
Sprint Network
• Austin Data Center– Two local circuits from Sprint’s Point Of Presence (POP)
to ADC follow common terrestrial path and enter ADC at single entry point
• Plan – One of the two Sprint local circuits will be re-engineered using a
separate path to the ADC
– The new local circuit will enter the Austin Data Center (Data Foundry) at a physically separate location of facility
– ADC network connections will be terminated at two geographically separate Sprint MPLS switch sites, insuring that a catastrophic switch failure at one site would not result in a total network outage
Sprint Network
• Corpus Christi Data Center– Two network connections between CCDC and Sprint
follow common terrestrial path• Plan
– CCDC network connections will be terminated at two geographically separate Sprint MPLS switch sites, insuring that a catastrophic switch failure at one site would not result in a total network outage
• Issues– There is only one local circuit carrier in the city of Corpus Christi
(AT&T)
– Single local circuit terrestrial route and single entry point into headquarters could be cost prohibitive
Sprint Network
• Remote Offices (Irving & San Antonio) have only one network connection– Plan
• If the business feels that redundancy is required at both locations, a 2nd circuit can be installed
• Additional network connection will be terminated at geographically separate Sprint MPLS switch site, insuring that a catastrophic switch failure at the site would not result in a total site outage
Network Connectivity at Units
• Fortigates– Configured to point to only one router at ADC
• Plan – With new network design for Restaurant Technology, Cisco 1841
routers will replace Fortigates and be “dual-addressed” to both routers at ADC
• Network– Single point of failure with one network connection
• Plan – A second line is being installed at each Unit to improve
performance and add redundancy. The new DSL circuit will be the primary network connection and the wireless network will be the backup
Internal Network Design at ADC
10.200 .200.1/30
G0/310.200.200 .2
G0/3
G0/110.200.201.248/24
G0/110.200.201.247/24
G0/2G0/2
2/11
2/12
2/13
2/14
2/15
2/16
2/11
2/12
2/13
2/14
2/15
2/16
1/48 1/47 1/481/47
1
2
3
4
5
6
FANSTATUS
Power Supply 1 Power Supply 2
Catalyst 6500 SERIES
WB-ADC-6506-1
200-240 V23 A
60/50 Hz
INPUTOK
FANOK
OUTPUTFAIL
RUNINSTA
LL
200-240 V23 A
60/50 Hz
INPUTOK
FANOK
OUTPUTFAIL
RUNINSTALL
WS-SUP720
SUPERVISOR 720 WI TH INTEGRATED SWITCH FABRI C
SYSTE
M
STATUS
ACTIVE
PWR
MG
MT
EJECT
DI SK 0
EJECT
DISK 1
CONSOLE PORT 2
PORT 1
LINK
LINK
LINK
RESET
STATUS
WS-X6748-GE-TX47
48
37
3 8
35
36
25
26
2 3
24
13
14
11
12
1
2
4 8 P O R T
12119 107 85 63 41 2 242321 2219 2017 1815 1613 14 363533 3431 3229 3027 2825 26 484745 4643 4441 4239 4037 38
10/100/1000GE MOD
FABRIC ENABLEDRJ45
STATUS
WS-X6748-GE-TX47
48
37
3 8
35
36
25
26
2 3
24
13
14
11
12
1
2
4 8 P O R T
12119 107 85 63 41 2 242321 2219 2017 1815 1613 14 363533 3431 3229 3027 2825 26 484745 4643 4441 4239 4037 38
10/100/1000GE MOD
FABRIC ENABLEDRJ45
STATUS
WS-X6516A-GBI C
LINK
1 2LIN
KLIN
K
3 4LIN
KLIN
K
5 6LIN
KLIN
K
7 8LIN
KLIN
K
9 10LIN
KLIN
K
11 12LIN
KLIN
K
13 14LIN
KLIN
K
15 16LIN
K
3
4
1
2
5
6
7
8
9
10
11
12
13
14
15
16
S TATUS PHONE
WS-X6548-GE-TX 4748
373 8
3536
2526
2 324
1314
1112
12
4 8 P O R T
12119 107 85 63 41 2 242321 2219 2017 1815 1613 14 363533 3431 3229 3027 2825 26 484745 4643 4441 4239 4037 38
10/100/1000 BASE-TETHERNET
SWITCHING MODULE
STATU
S
F IREWAL L SERVICES MODULE
DO NOT REMOVE CARD WHI LESTATUS L ED IS GREEN OR
DISK CORRUPTION MAY OCCUR
WS-SVC -FWM-1
SHUT DOWN
1
2
3
4
5
6
FANSTATUS
Power Supply 1 Power Supply 2
Catalyst 6500 SERIES
WB-ADC-6506-2
200-240 V23 A
60/50 Hz
INPUTOK
FANOK
OUTPUTFAIL
RUNINSTALL
200-240 V23 A
60/50 Hz
INPUTOK
FANOK
OUTPUTFAIL
RUNINSTALL
WS-SUP720
SUPERVI SOR 720 WITH INTEGRATED SWITCH FABRI C
SYSTEM
STATUS
ACTIVE
PWR
MG
MT EJECT
DI SK 0
EJECT
DISK 1
CONSOLE PORT 2
PORT 1
LINK
LINK
LINK
RESET
STATUS
WS-X 6748-GE-TX47
48
37
3 8
35
36
25
26
2 3
24
13
1 4
11
12
1
2
4 8 P O R T
12119 107 85 63 41 2 242321 2219 2017 1815 1613 14 363533 3431 3229 3027 2825 26 484745 4643 4441 4239 4037 38
10/100/1000GE MOD
FABRIC ENABLEDRJ45
STATUS
WS-X 6748-GE-TX47
48
37
3 8
35
36
25
26
2 3
24
13
1 4
11
12
1
2
4 8 P O R T
12119 107 85 63 41 2 242321 2219 2017 1815 1613 14 363533 3431 3229 3027 2825 26 48
4745 4643 4441 4239 4037 38
10/100/1000GE MOD
FABRIC ENABLEDRJ45
STATUS
WS-X6516A-GBI C
LINK
1 2LIN
KLIN
K
3 4LIN
KLIN
K
5 6LIN
KLIN
K
7 8LIN
KLIN
K
9 10LIN
KLIN
K
11 12LIN
KLIN
K
13 14LIN
KLIN
K
15 16LIN
K
3
4
1
2
5
6
7
8
9
10
11
12
13
14
15
16
STATUS PHONE
WS-X 6548-GE-TX 4748
373 8
3536
2526
2 324
131 4
1112
12
4 8 P O R T
12119 107 85 63 41 2 242321 2219 2017 1815 1613 14 363533 3431 3229 3027 2825 26 48
4745 4643 4441 4239 4037 38
10/100/1000 BASE- TET HERNET
SWITCHING MODULE
STATUS
F IREWAL L SERVICES MODULE
DO NOT REMOVE CARD WHI LEST ATUS LED IS GREEN OR
DISK CORRUPTION MAY OCCUR
WS-SVC-FWM-1
SHUT DOWN
0
7206
2
4
6
1
5
3
PACKET OVER SONET/S DH
ENAB
LED
RX C
ARRI
ER
RX
PACK
ETS
BN
155-
MM
RX TX
CONSOLE
LIN K
TX GBICRX
EN
GIGABIT ETH ERN ET 0/1
RJ 45
LINK
TX GB ICRX
EN
GIGAB IT ETHER NET 0/2
R J45
LINK
TX GBICR X
EN
GIGA BIT ETH ERN ET 0/3
RJ45 COMPAC T FLASH
C PUR ESET
NETWOR K PROCESSING EN GINE – G1 SLOTA CTIVE
POWERON
A UXNPE-G1
0
7206
2
4
6
1
5
3
P ACKET OVER SONET/SDH
ENAB
LED
RX C
ARRI
ER
RX
PACK
ETS
BN
155-M
M
RX TX
CONSOLE
LIN K
TX GBICRX
EN
GIGAB IT ETH ERN ET 0/1
RJ45
LINK
TX GB ICR X
EN
GIGA BIT ETHER NET 0/2
R J45
LINK
TX GBICRX
EN
GIGAB IT ETH ERN ET 0/3
RJ45 COMPA CT FLA SH
C PUR ESET
N ETWORK PROC ESSING ENGINE – G1 SLOTA CTIVE
POWERON
AU XNPE -G1
WB-ADC-7206-1 WB-ADC-7206-2
10.200.200.253/2510.200.200.254/2
5
Switch X-Connects
Servers
Sup 720
Servers
• Internal network design review to be conducted to ensure 100% redundancy (to address Cisco Core architecture)– Plan
• Cisco Engineering is currently performing analysis and will develop recommendations, if required
Next Steps
• Initiate Network Diversity Order Request- For CCDC - Email request to Sprint account team
- For ADC - Sign diversity request document (PDF)
• Submit Project Work Plan• Develop WBS• Plan/Schedule ADC Site Walk Thru• Start Project Execution• Monitor Project Milestones/Deliverables• Report Project Status/Progress
Detailed Network Configuration
Current / Proposed Design
San Antonio & Irving OC-3 Connections
Plano MAN1262.003
MAN 3203
MAN node19232
IrvingPL 542692
IrvingPOP (4485)
Atlanta 2140 IP Node
SPRINT GLOBAL MPLS
VPN
MAN1040.006
SONETRing 60
MAN node17131
San AntonioPOP (5133 )
SONETRing 27.5
HoustonIP Node 2290SL-PE10-HOU
Austin POP4440
SPRINT GLOBAL MPLS
VPN
San AntonioPL 542700
San Antonio & Irving OC-3 connections
Ft. Worth 2040IP Node
SL-PE30-FW
San AntonioPOP (4540 ) Ft. Worth
2040
San Antonio/Irving Details / Current Design
• San Antonio, TX (West Division)
- Fiber cable run thru conduit terminating into OC-3 line card on Cisco router
- OC-3 connection terminating in Houston, TX via MAN ring 1040.006 & SONET rings 60/27.5 with single point of failure termination in Ft. Worth, TX
• Irving, TX (East Division)
- Fiber cable run thru conduit terminating into OC-3 line card on Cisco router
- OC-3 connection terminating in Ft. Worth, TX via MAN rings 1262.003 & 3203 with single point of failure termination in Ft. Worth, TX
San Antonio/Irving Details/Proposed Design
• San Antonio, TX (West Division)
- Fiber cable run thru separate conduit (possibly at opposite building entrance) terminating into OC-3 line card on separate Cisco router
- Additional network connection will be terminated at geographically separate Sprint MPLS switch site, insuring that a catastrophic switch failure at the site would not result in a total site outage
• Irving, TX (East Division)
- Fiber cable run thru separate conduit (possibly at opposite building entrance) terminating into OC-3 line card on separate Cisco router
- Additional network connection will be terminated at geographically separate Sprint MPLS switch site, insuring that a catastrophic switch failure at the site would not result in a total site outage
Austin Data Center OC-3 Connections
Austin MAN1090.002
SONETRing 60.7
MAN node17030
Austin Data center OC3
PL 537791Austin
POP (4889 )
Atlanta 2140 IP Node
SPRINT GLOBAL MPLS
VPN
Austin MAN1090.002
SONETRing 60.7
MAN node17030
AustinPOP (4889)
SONETRing 11.17
Atlanta 2140 IP NodeFt. Worth 2040
IP Node SPRINT GLOBAL MPLS
VPN
Austin Data center OC3
PL 538183
Austin OC-3 connections
Ft. Worth 2040IP Node
Known Common points : MAN node/ring, Austin POP, SONET Ring from Austin to
Ft. Worth
Austin Data Center Details / Current Design
• Austin Data Center, Austin, TX- Fiber cable runs thru same conduit terminating into OC-3 line
card on two separate Cisco routers
- First OC-3 connection terminating in Ft. Worth, TX via MAN ring 1090.002 & SONET ring 60.7 with single point of failure termination in Ft. Worth, TX
- Second OC-3 connection terminating in Atlanta, GA via MAN ring 1090.002 & SONET rings 60.7/11.17 with single point of failure termination in Ft. Worth, TX
Austin Data Center Details/Proposed Design
• Austin Data Center, Austin, TX<True Redundancy to operate in parallel with current
implementation>
- Aerial fiber cable run terminating into OC-3 line card on first Cisco router thru different OC-3 carrier (Time Warner) terminating in Ft. Worth, TX via MAN ring 1090.002 & SONET ring 60.7 with no single point of failure
<Time Warner has existing aerial setup that WB could utilize at no additional cost>
Corpus Christi HQ OC-3 Connections
SONETRing 60.4
OC-3 local access(AT&T)
WBHQ Copus OC3PL 540029
Corpus ChristiPOP (4480)
SONETRing 60.4
Corpus ChristiPOP 4480
AustinPOP (4440)
SONETRing 27.4
HoustonIP Node 2290
PE 11
Ft. Worth 2040IP Node
WBHQ Corpus OC3PL 537702
Corpus OC-3 connections
SPRINT GLOBAL MPLS
VPN
Corpus ChristiPOP (4480)
HoustonIP Node 2290
PE 11SPRINT
GLOBAL MPLS VPN
Known Common points : Corpus POP, SONET ring
(different paths ), Houston IP Node router
Corpus Christi HQ Details / Current Design
• Corpus Christi HQ, Corpus Christi, TX- Fiber cable runs thru same conduit terminating into OC-3 line
card on two separate Cisco routers
- First OC-3 connection terminating in Houston, TX via SONET ring 60.4 with single point of failure termination in Houston, TX
- Second OC-3 connection terminating in Houston, TX via SONET rings 60.4/27.4 with single point of failure termination in Houston, TX
Corpus Christi HQ Details / Proposed Design
• Corpus Christi HQ, Corpus Christi, TX<There is only one local carrier (SBC/AT&T) for the Corpus
region>
- Eliminate SONET ring path 27.4 in second OC-3 path terminating in Houston, TX, thereby eliminating single point of failure in Houston, TX
- Since there is only the option of one local carrier for the Corpus region, having the OC-3 path termination points in Houston and
in Ft. Worth will optimize network diversity
-Sprint charges a one time network diversity fee of $600.00 and a recurring fee of $300.00 per month