28 NGN Bearer Network Fault Handling

NGN Bearer NetworkFault Handling

Version 1.0

ZTE CORPORATIONZTE Plaza, Keji Road South,Hi-Tech Industrial Park,Nanshan District, Shenzhen,P. R. China518057Tel: (86) 755 26771900Fax: (86) 755 26770801URL: http://ensupport.zte.com.cnE-mail: [email protected]

LEGAL INFORMATION

Copyright © 2006 ZTE CORPORATION.

The contents of this document are protected by copyright laws and international treaties. Any reproduction or distribution ofthis document or any portion of this document, in any form by any means, without the prior written consent of ZTE CORPO-RATION is prohibited. Additionally, the contents of this document are protected by contractual confidentiality obligations.

All company, brand and product names are trade or service marks, or registered trade or service marks, of ZTE CORPORATIONor of their respective owners.

This document is provided “as is”, and all express, implied, or statutory warranties, representations or conditions are dis-claimed, including without limitation any implied warranty of merchantability, fitness for a particular purpose, title or non-in-fringement. ZTE CORPORATION and its licensors shall not be liable for damages resulting from the use of or reliance on theinformation contained herein.

ZTE CORPORATION or its licensors may have current or pending intellectual property rights or applications covering the subjectmatter of this document. Except as expressly provided in any written license between ZTE CORPORATION and its licensee,the user of this document shall not acquire any license to the subject matter herein.

ZTE CORPORATION reserves the right to upgrade or make technical change to this product without further notice.

Users may visit ZTE technical support website http://ensupport.zte.com.cn to inquire related information.

The ultimate right to interpret this product resides in ZTE CORPORATION.

Revision History

Revision No. Revision Date Revision Reason

R1.0 20090415 First edition

Serial Number: sjzl20091899

Contents

Preface............................................................... i

Common Processing Flow for NGN Bearer

Network Fault ...................................................1

NGN (Fixed Network 3G) Introduction ..............3Fixed Network 3G ........................................................... 3

Softswitch ..................................................................... 3

MG Category.................................................................. 4

Bearer Network Structure ................................................ 4

Networking Diagram ....................................................... 5

NGN Bearer Network Fault Handling Steps

and Commands..................................................7NGN Bearer Network Basic Configuration Commands........... 7

NGN Bearer Network Maintenance and Diagnosis ................ 9

NGN Bearer Network Troubleshooting...............................11

NGN Bearer Network Processing Cases ..........19Packets Loss in NGN Bearer Network................................19

Disconnected Communication Between MSC and

MGW ...................................................................22

NGN Bearer Network Loop ..............................................26

Figures ............................................................31

Preface

In implementation and maintenance of data products, variousfaults may be present, some of which are easy to solve while somecan take a lot time for troubleshooting. Maintenance personnelare bothered by these faults. Fault Handling Special collectssome fault handling instances, gives analyses to these faults, andconcludes ideas and methods for troubleshooting. It can do helpto readers.

Classified according to product functions, thirty pieces of FaultHandling Special are available for users as shown in table below:

TABLE 1 FAULT HANDLING SPECIAL

Name Content

ACL Fault Handling It describes ACL-related fault handlingprocess, steps and case analysis.

ARP Fault Handling It describes ARP-related fault handlingprocess, steps and case analysis.

DHCP Fault Handling It describes DHCP-related faulthandling process, steps and caseanalysis.

MAC Fault Handling It describes MAC-related faulthandling process, steps and caseanalysis.

QinQ Fault Handling It describes QinQ-related faulthandling process, steps and caseanalysis.

VRRP Fault Handling It describes VRRP-related faulthandling process, steps and caseanalysis.

MPLS L3VPN Fault Handling It describes MPLS L3VPN-related faulthandling process, steps and caseanalysis.

VPLS/VPWS Fault Handling It describes VPLS/VPWS-related faulthandling process, steps and caseanalysis.

BGP Fault Handling It describes BGP-related fault handlingprocess, steps and case analysis.

Multicast Fault Handling It describes multicast-related faulthandling process, steps and caseanalysis.

Port Rate Limit Fault Handling It describes port rate limit-relatedfault handling process, steps and caseanalysis.

Confidential and Proprietary Information of ZTE CORPORATION i

NGN Bearer Network Fault Handling

Name Content

Direct Route, Static Route andDefault Route Fault Handling

It describes direct route, staticroute and default route-related faulthandling process, steps and caseanalysis.

Device Card Fault Handling It describes device card-related faulthandling process, steps and caseanalysis.

Device Hardware FaultHandling

It describes device hardware-relatedfault handling process, steps and caseanalysis.

Device Interface FaultHandling

It describes device interface-relatedfault handling process, steps and caseanalysis.

MAN Fault Handling It describes MAN-related faulthandling process, steps and caseanalysis.

Layer 3 Route Fault Handling It describes layer 3 route-related faulthandling process, steps and caseanalysis.

Ethernet Packet Capture It describes Ethernet packet capturetools.

QoS Fault Handling It describes QoS-related fault handlingprocess, steps and case analysis.

Policy Route Fault Handling It describes policy route-related faulthandling process, steps and caseanalysis.

Switch Fault Handling It describes switch-related faulthandling process, steps and caseanalysis.

Network Fault Diagnosis It describes network-related faulthandling process, steps and caseanalysis.

NGN Bearer Network FaultHandling

It describes NGN bearernetwork-related fault handlingprocess, steps and case analysis.

Network Fault DiagnosisMethods

It describes network fault DiagnosisMethods.

ZXR10 Products Maintenanceand Diagnosis

It describes ZXR10 productsmaintenance and diagnosis.

Network Fault Theory Analysis It describes network fault theoryanalysis.

ZXR10 Data Products andSolution

It describes ZXR10 data products andsolution.

IPTV Fault Handling It describes IPTV-related faulthandling process, steps and caseanalysis.

ii Confidential and Proprietary Information of ZTE CORPORATION

Preface

Name Content

SmartGroup Fault Handling It describes SmartGroup-related faulthandling process, steps and caseanalysis.

OSPF Fault Handling It describes OSPF-related faulthandling process, steps and caseanalysis.

Confidential and Proprietary Information of ZTE CORPORATION iii


This page is intentionally blank.

iv Confidential and Proprietary Information of ZTE CORPORATION

C h a p t e r 1

Common ProcessingFlow for NGN BearerNetwork Fault

Common faults occurred in NGN Bearer Network includes: makinga call unsuccessfully due to softswitch device registration failure;voice quality downgrades rapidly. When one of the above phe-nomena appears, refer to processing flow, as shown in Figure 1.

FIGURE 1 GENERAL PROCESS OF NGN BEARER NETWORK FAULTHANDLING

Confidential and Proprietary Information of ZTE CORPORATION 1



2 Confidential and Proprietary Information of ZTE CORPORATION

C h a p t e r 2

NGN (Fixed Network 3G)Introduction

Table of ContentsFixed Network 3G............................................................... 3Softswitch ......................................................................... 3MG Category...................................................................... 4Bearer Network Structure .................................................... 4Networking Diagram ........................................................... 5

Fixed Network 3GFixed network 3G is applied in softswitch tandem office throughPSTN local network (softswitch control device(SS1b)+ high ca-pacity trunk gateway (MSG9000). All calls from PSTN end areforwarded to softswitch tandem office, and softswich controls thewhole-network calls and triggers service.

Additionally, using mobile network design theory, fixed network 3Gintroduces HLR(Host location regiester), which is used for storinguser information. In the call, calling and callee information are ob-tained through the interactive action between softswich and HLR.Interactive protocol from softswitch to HLR uses MAP which is com-monly used in mobile network.

Uniform service information platform is applied in fixed network3, which provides service to whole-network users and performsuniform management on services.

SoftswitchSoftswitch technology consists in separating service accessingmodule from telephone switch and forming an independentphysical unit, which is called media gateway (MG). MG completesthe data format and protocol conversion and transform all mediainformation flow to IP protocol-applied data packets to send insoftswitch network.


xy

附注

应该是softswitch

xy

附注

应该是softswitch


MG Category� Trunk Media Gateway

It is used to complete trunk connection for PSTN/PLMN tele-phone switch by transforming 64kbit/s voice signaling in PCMtrunk to IP packets.

� Signaling Gateway

It is used to complete signaling connection for PSTN/PLMN tele-phone switch by transforming No.7 signaling based on TNM cir-cuit to IP packets.

TG and SG complet the connetion between softswitch networkand PSTN/PLMN telephone network using TDMA circuit switchand coonect the ordinary telephone users and services inPSTN/PLMN to softswitch network.

� Access Gateway

It provides simulating subscriber line interface, which is usedto connect ordinary telephone user to softswich network. Sub-scriber can be provided with all PSTN services, such as: tele-phone services, dialing network service. It encapsulates sub-scriber data and subscriber line signaling in IP packets.

Bearer Network StructureFixed network 3G bearer network contains the following four func-tion modules:

� PE

� Backbone Bearer Network

� TG

� SS

PE PE interconnects the whole fixed-network 3G bearer network todevices on carrier’s IP network, which is provided by customer. Itis recommended to use vpn for guaranteeing the simplicity andreliability of NGN bearer network.

Backbone BearerNetwork

It is isolated from operator’s bearer network and is specialized forbearer. The network converging NGN service flow generally hassimple architecture, which composed of high-performance routerand layer 3 switch. The uplink is connected to operator’s PE device.

TG For data side, RF module is mainly used in bearing media flow.When using 9000, a few of signaling flow will pass. In general, twoL3 switch or routers with higher density port perform this function.It is recommended to use L3 switch, providing traffic convergence,load balance, address convergence and master/slave redundancyprotection. Uplink is accessed to backbone bearing network.

SS SS mainly acts as device for converging signaling traffic. In somecases it is neccesary to connect SG，SHLR，9000-SIGIPI andUP10. According to working environment, L2 swithyc, L3 switchand router (paired up with L2 router) can be used. In some


xy

附注

错误

xy

附注

错误

xy

附注

错误

Chapter 2 NGN (Fixed Network 3G) Introduction

cases, it is necessary to use firewall to protect them. Generallyspeaking, it is recommendeded to use L3 switch.

Networking DiagramNGN networking diagram is shown in Figure 2 .

FIGURE 2 NETWORKING DIAGRAM


xy

附注

错误




C h a p t e r 3

NGN Bearer NetworkFault Handling Stepsand Commands

Table of ContentsNGN Bearer Network Basic Configuration Commands............... 7NGN Bearer Network Maintenance and Diagnosis .................... 9NGN Bearer Network Troubleshooting...................................11

NGN Bearer Network BasicConfiguration Commands.

Configurationon NGN BearerNetwork Access

Layer

To configure NGN bearer network access layer, perform the follow-ing commands:

TABLE 2 INTRODUCTION TO CONFIGURATION COMMANDS ON NGN BEARERNETWORK ACCESS LAYER

Command Description

1 vrrp <group> ip <ip-address>[secondary]

This configures VRRP virtual IPaddress on the interface, run vrrpprotocol.

Several virtual addresses can beconfigured in one VRRP group,the connected host can useany gateway as gateway forcommunication, the virtual addressacts as the gateway of NGN device.

2 vrrp <group> priority<priority>

This configures VRRP priority oninterface, the priority on interfaceis in the range of 1－254.This makes a certain access devicealways act as active gateway inNGN bearer network.



Command Description

3 vrrp <group> preempt[delay <milliseconds>]

This configures whether the virtualdevice in standby status canpreempt. This function is enabled indevices provided by ZTE by default.

As a result, when a device,which had higher priority andacted as active router before it isdisconnected from the network ,isreconnected to the network, it canpreempt the prevenient function.

4 vrrp <group> advertise [msec]<interval>

This configures the interval forsending VRRP notification.

Configurationon NGN Bearer

NetworkConvergence

Layer

To configure NGN bearer network convergence layer, perform thefollowing commands:

TABLE 3 INTRODUCTION TO CONFIGURATION COMMANDS ON NGN BEARERNETWORK CONVERGENCE LAYER

Command Description

1 router ospf 1 This runs OSPF protocolin global configurationmode.

2 network <ip-address><wildcard-mask> area <area-id>

This definesOSPF-running interface,and defines area ID onthese interfaces.

3 ip ospf cost <cost> This configuresoverheads

Configurationon NGN BearerNetwork Core

Layer

To configure NGN bearer network core layer, perform the followingcommands:

TABLE 4 INTRODUCTION TO CONFIGURATION COMMANDS ON NGN BEARERNETWORK CORE LAYER

Command Description

1 router bgp <as-number> This activates BGPprocess in globalconfiguration mode.

2 neighbor <ip-address>remote-as <number>

This configures BGPneighbor.

3 network <ip-address><net-mask>

This uses BGPnotification on a network.


Chapter 3 NGN Bearer Network Fault Handling Steps and Commands

NGN Bearer NetworkMaintenance and Diagnosis

NGN BearerNetwork

Access LayerMaintenance

To maintain NGN bearer network access layer, perform the follow-ing commands:

Command Description

show vrrp [<group>|brief|interface <interface-name>]

This displays all configurationinformation on VRRP group.

ZXR10#debug vrrp (no debug all) This activates VRRP debugswitch.

show interface XX This displays interface statusinformation.

NGN Bearer Net-work Convergence

Layer Mainte-nance

1. To view details of OSPF process, perform the following com-mand:

Command Description

show ip ospf [<process-id>] This views details in OSPFprocess.

2. To view OSPF interface current configuration and status, per-form the following command:

Command Description

show ip ospf interface[<interface-name>][process<process-id>]

This views OSPF interfacecurrent configuration andstatus.

3. To view OSPF neighbor information, perform the following com-mand:

Command Description

show ip ospf neighbor[interface <interface-name>][neighbor-id<neighbor>][process<process-id>]

This views OSPF neighborinformation. That routinginformation between tworouters can not communicateeach other might be causedby that there is no adjacencybetween these two routers. Thisviews whether the neighborrelationship between tworouters is FULL. FULL statusindicates the normal running ofOSPF protocol.

4. To view part or all information in the link status database, per-form the following command:



Command Description

show ip ospf database This views part or all informationin the link status database.

Link status database is the source of all OSPF routes in IP rout-ing table. Many route faults are caused by incorrect informa-tion in link status database.

ZXR10 provides debug commands to debug OSPF protocol andtrace related information. The debug command can be used asfollows:

Command Description

debug ip ospf adj This activates the switch fordebugging loopback OSPFneighbor events.

debug ip ospf packet This activates switch fordebugging loopback OSPFpacket receiving and sendingevents and monitors all OSPFpackets’ sending and receiving.

debug ip ospf lsa-generation This activates switchfor debugging loopbackOSPF-generated link statusnotification event.

debug ip ospf events This activates the switch fordebugging loopback OSPFimportant events.

NGN BearerNetwork Access

Core Maintenance

When there are BGP routing faults, debugging commands can beused for locating and clearing faults. The shown command is usedmost. show command can be used to view current BGP neighborstatus and router can learn BGP routing information, as follows:

Command Description

show ip bgp protocol This displays configuration on BGPprotocol module.

show ip bgp neighbor This views BGP neighborhood anddisplays current neighborhoodstatus.

show ip bgp route This displays entries in BGProuting table.

show ip bgp summary This displays all BGP neighborconnection status.

Besides show command, debug command can be used to ob-serve the BGP neighbor establishment process and route upgrad-ing process, etc. These commands are used as follows:



Command Description

debug ip bgp in This traces and displaysnotification packets receivedby BGP and lists error No. andsub-error No.

debug ip bgp out This traces and displaysnotification packets sent from BGPand lists error No. and sub-errorNo.

debug ip bgp events This traces and displays the BGPstate transition.

Note:

The debug command might affect router performance, it is rec-ommended to shutdown it after using it.

NGN Bearer NetworkTroubleshooting

Context When there are faults occurred in the running of NGN bearer net-work, perform to perform the following steps:

Steps 1. Check whether connectivity among bearer network devices andthe direction of service traffic are normal.

1. Inspecting Interconnections

Log into adjacent softswitch data devices on which faultsoccur. Ping remote softswitch address, it ping unsuccess-fully or packets are lost. Inspect links step by steps andlocate the devices between which ping is unsuccessful orpackets are lost. Note: when packets are lost, it is neces-sary to contact operator to find whether transmission linkswork well.

It should be noted that bearer network services are gen-erally deployed in MPLS/VPN. If devices located separatelyin VPN can not communicate, it is necessary to check theconfiguration of MPLS/VPN and label forwarding tables.

2. Inspecting service traffic direction

Log into the data device in softswitch. Trace remotesoftswitch device address. It the host hop is unreachableor is not the designed one, it is necessary to log into therouter before the unreachable hop to check whether therouting table and interfacee are correct.

3. Related Commands


xy

附注

错误

xy

附注

末尾多了一个e


ZXR10#ping <Target IP address> option< Repeat count><Datagram size>< Timeout in seconds> extcom< Sourceaddress>< Type of service>< Time to live>< Set DONTFRAG>

ZXR10#trace <Destination IP address>

2. Inspecting Routing Table

� Use show ip route to view router, switch, routing table onbackbone bearer network router, whether there are routessent from various places, whether there PE from MAN orCN2. For example, Check whether there is route from Bei-jing is routed from PE in CN2 in Shanghai node of DC1.

� If OSPF or BGP routing protocol run, use show ip ospfnei，show ip bgp nei to check whether neighbor status isnormal. OSPF neighbor status should be FULL, BGP neigh-bor status should be Established.

� NMS devices generally contain firewall or default router,please check related device configuration and routes existnormally.

� Additionally, check whether backup channels betweenrouters and switches run normally, i.e. whether OSPFbetween them are normal and check whether externalroutes are distributed to OSPF processes normally.

� Use traceroute command to view whether uplink packetsare sent from MAN/CN2 PE.

Note:

If OSPF and BGP protocols can not be used to establish neigh-bors normally and can not learn routes normally, refer to re-lated fault handling modules.

3. Check whether active/standby devices are switched over nor-mally.

Many faults occurred in NGN bearer network are caused byincorrect switchover between active/standby devices. Thus,the inspection on switchover is indispensable step on bearernetwork.

1. Select two nodes, node A and node B. From node A pingnode B, record ping packet delay; traceroute from node A tonode B, record data packet direction in normal conditions.

2. Test the switchover of redundancy devices or link in thebearer network from node A to node B and view whetherthe ping and traceroute paths are in accordance with net-work design.

3. If ping unsuccessfully or packets are lost after theswitchover, it is necessary to check whether the redun-dancy configuration of switched over devices and routingprotocols are converged normally.

4. As to switchover fault, the firewall is easily to be ignored.The firewall access modes in NGN bearer network are de-scribed as follows:

5. VRRP check should be performed on access side in ac-tive/backup switchover.



It is necessary to check whether VRRP runs normally ondevice configured with vrrp. Especially in two-node outlets,faults on single node will not affect services which are easilyto be ignored. It is necessary to check whether both outletswork normally.ZXR10_1#show run int vlan 2Building configuration...interface vlan 2ip address 10.10.10.2 255.255.255.0vrrp 1 ip 10.10.10.1vrrp 1 priority 130

ZXR10_2#show run int vlan 3Building configuration...interface vlan 3ip address 10.10.10.3 255.255.255.0vrrp 1 ip 10.10.10.1

VRRP check is performed on two devices, one should be inmaster status and the other should be in standup status.AllSS uplink flows are forworded outwards through the masterdevice.ZXR10_1#show vrrpvlan2 - Group 1State is MasterVirtual IP address is 10.10.10.1Virtual MAC address is 0000.5e00.0101Advertisement interval is 1.000 secPreemption is enabledmin delay is 0.000 secPriority is 100 (config 100)Authentication is disabledMaster Router is 10.10.10.2 (local), priority is 100Master Advertisement interval is 1.000 secMaster Down interval is 3.609 sec

ZXR10_2#show vrrpvlan3 - Group 1State is BackupVirtual IP address is 10.10.10.1Virtual MAC address is 0000.5e00.0101Advertisement interval is 1.000 secPreemption is enabledmin delay is 0.000 secPriority is 100 (config 100)Authentication is disabledMaster Router is 10.10.10.2 , priority is 100Master Advertisement interval is 1.000 secMaster Down interval is 3.609 sec (expires in 3.029 sec)

When data are configured normally and services run nor-mally, ARP entries of downlinked 9000 IPI and SS NICshould be viewed on master router and vrrp master switch;at the same time the backup router port should be in downstatus, there isn’t related ARP entries on backup router andswitch.ZXR10#show arpArp protect whole is disabled . The count is 3.Address Age(min) Hardware Addr Interface10.10.10.1 - 0000.5e00.0101 vlan210.10.10.2 - 00d0.d0c2.9f83 vlan220.20.20.2 - 00d0.d0c2.9f83 vlan12

4. Viewing Device Alarms and Logs

Viewing alarm messages can help us to have an idea abouthidden problems and corresponding reasons.

� Viewning Alarm Record


xy

附注

错误


ZXR10#show logging alarm

� Viewing Time

ZXR10#show clock

� Viewing CPU Usage

ZXR10#show process

CPU utilization shall not be large (＜50%) with fluctuationwithin the range of 15%

� View Operation Logs

ZXR10#show logfile

Viewing whether there are incorrect operations and incor-rect configurations

� Viewing Fault Files

ZXR10#dir data

This mainly checks whether there is xxx.zte file andwhether related time setting is normal in data directory.

If system halted files are generated, it is necessary sendthem to technologies for analysis.

5. Checking Interconnected Port Status

Use show interface command to guarantee that interconnectedports on data devices work normally.ZXR10#show interface fei_1/3fei_1/3 is up, line protocol is up―physical status is UPDescription is to_qd-gnKeepalive set:10 secMAC address is 00d0.d0d5.60c0The port is electricDuplex full――To ensure working status is FULLMdi type:autoARP Timeout:00:10:00Internet address is 60.209.80.161/30MTU 1500 bytesMRU 1500 bytes BW 100000 Kbits―To ensure work bandwidth and interface type are in accordance.

Sample rate is 0Last clearing of "show interface" counters never120 seconds input rate: 0 Bps, 0 pps120 seconds output rate: 0 Bps, 0 pps

Interface peak rate :input 1180 Bps, output 1180 BpsInterface utilization:input 0%,output 0%

―traffic statistics is in normal rangeInput:

Packets : 2803 Bytes: 232685Unicasts : 2803 Multicasts: 0 Broadcasts: 164B : 2129 65-127B : 258 128-255B : 416256-511B : 0 512-1023B : 0 1024-1518B: 0Undersize: 0 Oversize : 0 CRC-ERROR : 0

－No CRC errorOutput:Packets : 2247 Bytes: 150180Unicasts : 118 Multicasts: 0 Broadcasts: 212964B : 2129 65-127B : 118 128-255B : 0256-511B : 0 512-1023B : 0 1024-1518B: 0Oversize : 0

1. When the port is down or in half-duplex status, it is neces-sary to check working status of devices on both sides andnetwork cable quality.


xy

附注

超出页面，需要换行


2. When port bandwidth usage exceeds 70%, it is necessaryto expand device link capacity to ensure packet forwardingsaddition, it is recommended to connect a PC (simulating9000/SS) to router/switch and ping 9000 or SS on othernodes, view whether packets are lost, whether jitter andtime delay meet the requirements. Use IAD to dial a testnumber, and use service to confirm whether time delay inbearer network meet the requirements.

3. When CRC-ERROR counting statistics is increasing withdata transmission, it is necessary to check port and net-work cable quality. After changing connector and networkcable, it is recommended to observe after clear counter byusing clear counter command.

4. Refer to preliminary instructions to interconnection be-tween NGN bearer network data and softswitch devices.

� SS

SS is core device of softswitch, which is connected todata device SW through NIC for transmitting signalingflow.

NIC boards contains two boards: active board andstandby board. In normal time, both ports are in upstatus, but only one board is working, SS only needsone IP address. In active/standby changeover, packetsare not lost.

The interconnected data devices are switched(L2/L3).Interconnecting interfaces word mode is the defaultconfiguration.

� SG

SG is core device of softswitch, which is connected todata device SW through SGIPI for transmitting signalingflow.

SGIPI contains two boards, one port is up, the other isdown. SG only needs one IP address. In active/standbyswitchover, it is normal to loss less than three packets.

Interconnected data devices are switches(L2/L3), andthe interconnection word mode are described as fol-lows: if SG activates L3 check function, the port in-terconnected to switch uses default configuration; ifSG activates backup check function, the port intercon-nected to switch uses force+configuration line sequencemode(MIDX-parallel).

� TG

SG is main device of softswitch, which is connected todata device SW through SGIPI for transmitting signalingflow.

SGIPI contains two boards, one port is up, the other isdown. TG only needs one IP address. In active/standbyswitchover, it is normal to loss less than three packets.

Interconnected data devices are switches(L2/L3), andthe interconnection word mode are described as fol-lows: if SG activates L3 check function, the port in-terconnected to switch uses default configuration; ifSG activates backup check function, the port intercon-



nected to switch uses force+configuration line sequencemode(MIDX-parallel).

There are two kinds of TG media: NIPI和MGEB

NIPI：electrical Interface card. As different IPI portsare not allowed to configure in the same network seg-ment, main physical port is up/down. Thus, as gatewayL3 device, it is recommended to configure it same to theaddress downlinked to active/standby interface and addit to OSPF area, which protects 9000 switchover. Inter-connection switch port has the same work mode withSG.

NIPI：optical Interface card. As different MGEB portsare not allowed to configure in the same network seg-ment, main physical port is up/down. Thus, as gatewayL3 device, it is recommended to configure it same to theaddress downlinked to active/standby interface and addit to OSPF area, which protects 9000 switchover. Inter-connection switch port has the default work mode.

� MSG7200

7200 is optional device for softswitch, with independentports. These ports dot not forward data and can beconfigured in the same network segment. Thus, sev-eral ports can be converged to the same L3 gateway. Ifdouble device protection is needed, activate vrrp pro-tocol in the same network segment on L3 device actingas gateway. To configure 7200 address, it is necessaryto view configured RF board, one RF board needs oneIP address. Those IP addresses can belong to the samenetwork segment.

Note:

In commissioning, telnet to Rf board for configuring adefault route.

6. Viewing Device Board Running

� Make sure no alarm indicator is on devices on bearer net-work board.

� Use show version to view whether each board works nor-mally.ZXR10#show versionZXR10 Router Operating System Software, ZTE CorporationZXR10 ROS Version V4.6.02aGER Software, Version V2.6.02a43, RELEASE SOFTWARECopyright (c) 2000-2005 by ZTE CorporationCompiled Dec 8 2006, 15:36:22System image files are flash:<//flash/img/zxr10ger.zar>System uptime is 14 days, 18 hours, 34 minutes

[SMP](Master, slot 9)Main processor: RM7000 with 512M bytes of memorySystem with multiple processors (2 Network processors)Every network processor with 256M bytes of memory64K bytes of non-volatile configuration memory64M bytes of processor board System flash (Read/Write)ROM: System Bootstrap, Version:MP:V2.0-05-19,DP:V2.0-05-19,RELEASE SOFTWARESystem serial: 11014


xy

附注

错误


FPGA Version :V36CPLD Version :V1

[SMP](Slave, slot 10)Main processor: RM7000 with 512M bytes of memorySystem with multiple processors (2 Network processors)Every network processor with 256M bytes of memory64K bytes of non-volatile configuration memory64M bytes of processor board System flash (Read/Write)ROM: System Bootstrap, Version:MP:V2.0-05-19,DP:V2.0-05-19,RELEASE SOFTWARESystem serial: 11014FPGA Version :V36CPLD Version :V1

[NP](slot 1)NP in slot NPSLOT1 is 2000(64KB)

[NP](slot 2)NP in slot NPSLOT2 is 2000(64KB)

[LIC](slot 1)FPGA Version :V23CPLD Version :V12



7. In case it still fails to locate fault, please contact 800 for furtherprocessing.

END OF STEPS





C h a p t e r 4

NGN Bearer NetworkProcessing Cases

Table of ContentsPackets Loss in NGN Bearer Network....................................19Disconnected Communication Between MSC and MGW............22NGN Bearer Network Loop ..................................................26

Packets Loss in NGN BearerNetwork

NetworkingDiagram

1. The overall network topology is shown in Figure 3:



FIGURE 3 NGN BEARER NETWORK PACKET LOSS NETWORKING DIAGRAM

2. The partial fault topology is shown in Figure 4:


Chapter 4 NGN Bearer Network Processing Cases

FIGURE 4 PARTIAL TOPOLOGY OF PACKET LOSS IN NGN BEARER NETWORK

Networking Illustration

� Core Layer: This layer contains 3 T128 and T64G devices

� Access layer: this layer contains 23 T64E and T40G devices inprovince’s equipment room.

� OSPF dynamic routing protocol: in order to implement fastroute switch, automatic and fast convergence when route flap-ping, run OSPF dynamic route protocol among three T128 andthree T64G devices in core layer to establish OSPF protocolbackbone area (area 0).

� VRRP (Virtual Router Redundancy Protocol): in order to avoidsingle-point failure and enhance network reliability, activateVRRP protocol on core layer router T64G and use floating IP ofdouble-backup router as terminal gateway.

Fault Description On-site technical of representative office reflect that calls can notbe made successfully in LHR area, although the dialing succeeds,the voice quality is poor.

Fault Location � From PC–1 ping PC–2 on site, it is found packets are lost seri-ously (>30%)



� From PC–1 tracert PC–2 address, it is found datagram directionis correction.

� From PC–1 ping next-hop addresses, it is found when pingto the two Loopback addresses of two T64G devices of LHR,pacekets are lost severely.

� From PC–1 ping STM interface and Loopback addresses of T128in LHR, no packet is lost. The fault can be located betweenT128 and T64G in LHR. In the next step we should locatewhether fault is caused by link or the device itself, whetherit is hardware or software.

� Logging into T128 and T64G on LHR to check the configuration,routing table and interface parameter, there is no fault. Whenviewing alarms, one important alarm for offline line card isfound. According to the network diagram, it is found two T64Gdevices are connected to the same line card of T128. Whenthe line card became offline, it generates off-line alarm. Atthis moment, it can be concluded that the fault exits in thecommunication of T128 card.

Fault Handling Extract and insert the faulty card, no packet is lost in severalmonths. Afterwards, it is notified that client had moved the de-vice. So the card might become loose in the movement.

Fault Summary This fault has typicality and universality in NGN bearer network,the troubleshooting process is rather clear.

Disconnected Communication Between MSC and MGW

Network Topology 1. The overall network topology is shown Figure 5:


xy

附注

错误


FIGURE 5 MOBILE SOFTSWITCH PROJECT DIAGRAM

2. The partial fault topology is shown in Figure 6:

FIGURE 6 PARTIAL TOPOLOGY DIAGRAM OF MOBILE SOFTSWITCHPROJECT

3. Networking Illustration

� Core layer: devices on this layer contain: two T64E devicesin province’s equipment room, two T64E devices in city B,two GER devices in equipment room C, two GER devices inequipment room in city D.

� Access layer: devices on this layer contain: two 3928 de-vices in province’s equipment room, two 3928 devices in



city B, two 3928 devices in equipment room C, two 3928devices in equipment room in city D.

� OSPF dynamic routing protocol: in order to implement fastroute switch, automatic and fast convergence when routeflapping, run OSPF dynamic route protocol among fourT64E and four GER devices in core layer to establish OSPFprotocol backbone area (area 0 domain).

� VRRP (Virtual Router Redundancy Protocol): in order toavoid single-point failure and enhance network reliability,activate VRRP protocol on core layer router T64E, GER anduse floating IP of double-backup router as gateways of ter-minal MGW, MSC Server.

Fault Description The restarting T64E router with VRRP cause that the communica-tion between MSCS and MGW has been interrupted for 4 minutes.As a result, all users connected MSC can not make call.

Fault Location Configure VRRP on the active T64E, which provides MSC with in-terface address, as follows:interface fei_1/2encapsulation dot1Q 2ip address 10.0.154.130 255.255.255.240description To-MSCSvrrp 1 ip 10.0.154.129vrrp 1 priority 254

Configure VRRP on the standby T64E, which provides MSC withinterface address, as follows:interface fei_1/2encapsulation dot1Q 2ip address 10.0.154.131 255.255.255.240description To-MSCSvrrp 1 ip 10.0.154.129vrrp 1 priority 150

MSG interface address:10.0.154.132 ； virtual gateway:10.0.154.129.

Configure VRRP on the active T64E, which provides MGW with in-terface address, as follows:interface fei_1/3encapsulation dot1Q 3ip address 10.0.155.130 255.255.255.240description To- MGWvrrp 2 ip 10.0.155.129vrrp 2 priority 254

Configure VRRP on the standby T64E, which provides MGW withinterface address, as follows:interface fei_1/3encapsulation dot1Q 3ip address 10.0.155.131 255.255.255.240description To-MGWvrrp 2 ip 10.0.155.129vrrp 2 priority 150

MGW interface address:10.0.155.132 ； virtual gateway:10.0.155.129.

It can be seen from the above network structure and data config-uration that core router redundancy is fully considered in networkdesign. That is to say when there is fault on any T64 core router orpower off, the communication between MSC and MGW will not beinterrupted (One or two packets are lost if there is interruption).



However, in actual practice when restarting active T64E router itis found the communication between MSC and MGW have beeninterrupted for four minutes (as the network between MSC andMGW is disconnected).

By viewing T64E logs, it is found faults are caused by that VRRPpreemption time is faster than OSPF route convergence time.When the original master T64E router restarts and VRRP becomesmaster, OSPF has not finished route convergence, packets sentfrom MSG can not find route and reach MSG, the communicationbetween MSC and MGW is disconnected. Why is VRRP preemptiontime faster than OSPF route convergence time? After severalexperiments, it is found there is competitive relation betweenboards on T64E, some boards start faster than others. If 100Mbpscard down-linked to slot 1(on which VRRP virtual gateway isconfigured on it) finished to start, VRRP status on master T64Ehas become to master; while the card uplinked to slot 2(OSPFinterface address) has not started, master T64E can not completeroute convergence, the above phenomenon will occur. In the testit is found OSPF route convergence is slower than VRRP eventhough they are located in the same card.

Fault Handling There are two ways to avoid above problem by modifying VRRPconfiguration:

Take the master T64E router providing MSC with interface addressas example:

1. To deactivate vrrp preemption--when the device with higherpriority restarts, it will not become master device but keepsas backup device, no matter how high priority it has. PerformVRRP preemption manually.interface fei_1/2encapsulation dot1Q 2ip address 10.0.154.130 255.255.255.240description To-MSCSno vrrp 1 preemptvrrp 1 ip 10.0.154.129vrrp 1 priority 254

At this moment, if T64E finish starting, to make active T64Eto be master, it is necessary to configure vrrp 1 preempt, i.e.preempt VRRP manually (active T64E has higher priority)

2. Delayed vrrp preemption: when vrrp can preempt master, seta delay time, in the range of 1-3600 seconds. For example:interface fei_1/2encapsulation dot1Q 2ip address 10.0.154.130 255.255.255.240description To-MSCSvrrp 1 preempt delay 1200vrrp 1 ip 10.0.154.129vrrp 1 priority 254

If T64E-1 downlink interface finish starting, it will not preemptas master until 20 minutes (1200 seconds) later. At this mo-ment, the uplink interface should have started and OSPF routeconvergence is completed.

Fault Summary With the coming of 3G time, data bearer network plays an rolemore and more important. Once a serious fault occurs, eventhough lasts for several minutes, millions of users will be affected.In this way data network design and maintenance should be paidspecial attention.



NGN Bearer Network LoopNetworking

DiagramThe overall network topology is shown in Figure 7:

FIGURE 7 NETWORKING DIAGRAM OF NGN BEARER NETWORK LOOP FAULT

The partial fault topology is shown in Figure 8:

FIGURE 8 PARTIAL NETWORKING DIAGRAM OF NGN BEARER NETWORKLOOP FAULT

Networking Illustration

� Core Layer: This layer contains 2 T128 and PE devices

� Access layer: this layer contains four 3928 and new T64G de-vice.



� OSPF dynamic routing protocol: in order to implement fastroute switch, automatic and fast convergence when route flap-ping, run OSPF dynamic route protocol among two T128 anddownlinked L3 switch in core layer to establish OSPF protocolbackbone area (area 0 domain).

Fault Description NMS finds that there is fault in dual-subscriber number function.View two core T128 router status, fault is found in PE router, whichis uplinked to emplacement T128. The link between junction T128and uplinked PE is normal, flow should be switched over to junctionT128. From emplacement T128 ping value-added service platformAS address of province network, it is found the response is T. Soloop exists in network. After that trace the value-added serviceplatform AS address of province network, if is found there is routeloop between emplacement T128 and new junction T64G (used forAG access). Disconnect the link between emplacement T128 andjunction T64G, the service becomes normal.

Fault Analysis 1. In normal conditions, registration on SS10 is sent to value-added service platform AS from T128 through CN2. The flowdirection diagram is shown in Figure 9.

FIGURE 9 LOOP FAULT ANALYSIS DIAGRAM I

2. When there is fault in the link between emplacement T128and uplinked PE, default flow will be sent to province value-added platform AS through junction. The actual flow directionis shown in Figure 10.



FIGURE 10 LOOP FAULT ANALYSIS DIAGRAM II

3. In above diagram, it can be seen there is loop between T128and junction T64G. The loop might be caused by the followingreason:

� Emplacement T128 receives CN2 route, which BGP learnsfrom T128. Find the next route hop by IGP protocol.There are two paths: direct network segment and theone through junction T64G. According to OSPF routingprinciple, the path with less cost has higher priority. Thepath through junction 3 is less than direct link 10, soemplacement T128 forwards traffic with destination toprovince value-added platform to junction T64G.

� There is no detailed route of province value-added platformon junction T64G. There is only the default OSPF routefrom emplacement T128 and junction T128. Accordingto OSPF routing priority principle, select the default routelearnt from T128. So the data with destination to provincevalue-added platform are forwarded to emplacement T128.

Fault Location In above conclusion, it can be concluded that the fault has no rela-tionship with the device itself. In special conditions (there is faultin the link between PE and T128)it occurs dut to special networkstatus (mainly because cost value is not optimized).

Fault Handling Re-plan cost value of OSPF in on-site AG accesses network to avoidsuch fault. The cost value of the network device after the plan isshown in Figure 11 .


xy

附注

错误


FIGURE 11 LOOP FAULT ANALYSIS DIAGRAM III

Fault Summary Analyzing the overall network topology is helpful for finding out un-reasonable factors. Good understanding of route protocol is usefulfor troubleshooting.





Figures

Figure 1 GENERAL PROCESS OF NGN Bearer Network FAULT

HANDLING......................................................... 1

Figure 2 Networking Diagram............................................... 5

Figure 3 NGN Bearer Network Packet Loss Networking

Diagram...........................................................20

Figure 4 Partial Topology of Packet Loss in NGN Bearer

Network ..........................................................21

Figure 5 Mobile Softswitch Project Diagram...........................23

Figure 6 Partial Topology Diagram of Mobile Softswitch

Project .............................................................23

Figure 7 Networking Diagram of NGN Bearer Network Loop

Fault ................................................................26

Figure 8 Partial Networking Diagram of NGN Bearer Network

Loop Fault ........................................................26

Figure 9 Loop Fault Analysis Diagram I .................................27

Figure 10 Loop Fault Analysis Diagram II ..............................28

Figure 11 Loop Fault Analysis Diagram III .............................29


Date post:	21-Apr-2015
Category:	Documents
Upload:	khangvd
View:	53 times
Download:	1 times

28 NGN Bearer Network Fault Handling

Documents