Chapter TwelveChapter TwelveTroubleshooting Network Troubleshooting Network

ProblemsProblems

ObjectivesObjectives

Describe the elements of an effective Describe the elements of an effective troubleshooting methodologytroubleshooting methodology

Follow a systematic troubleshooting Follow a systematic troubleshooting process to solve networking problemsprocess to solve networking problems

Use a variety software and hardware tools Use a variety software and hardware tools to diagnose problemsto diagnose problems

Discuss practical issues related to Discuss practical issues related to troubleshootingtroubleshooting

Troubleshooting MethodologyTroubleshooting Methodology

Identify the symptomsIdentify the symptoms

Identify the scope of the problemIdentify the scope of the problem

Establish what has changed on the Establish what has changed on the networknetwork

Determine the most probable cause of the Determine the most probable cause of the problemproblem

Troubleshooting MethodologyTroubleshooting Methodology

Determine the most probable cause of the Determine the most probable cause of the problem (cont.)problem (cont.) Verify user competencyVerify user competency Re-create the problemRe-create the problem Verify the physical integrity of the network Verify the physical integrity of the network

connectionconnection Verify the logical integrity of the network Verify the logical integrity of the network

connectionconnection

Troubleshooting MethodologyTroubleshooting Methodology

Implement a solutionImplement a solution

Test the solutionTest the solution

Recognize the potential effects of the Recognize the potential effects of the solutionsolution

Document the solutionDocument the solution

Troubleshooting MethodologyTroubleshooting Methodology

Figure 12-1: Simple flow

chart of trouble-

shooting steps

Identify the SymptomsIdentify the Symptoms

Questions to help you identify symptoms Questions to help you identify symptoms of a network problemof a network problem Is access to the network affected?Is access to the network affected? Is network performance affected?Is network performance affected? Are data or programs (or both) affected?Are data or programs (or both) affected? Are only certain network services affected?Are only certain network services affected?

Identify the SymptomsIdentify the Symptoms

Questions to help you identify symptoms of a Questions to help you identify symptoms of a network problem (cont.)network problem (cont.) If programs are affected, does the problem If programs are affected, does the problem

include one local application, one networked include one local application, one networked application, or multiple networked applications?application, or multiple networked applications?

What specific error messages do users report?What specific error messages do users report? Is one or multiple users affected?Is one or multiple users affected? Do the symptoms manifest themselves Do the symptoms manifest themselves

consistently?consistently?

Identify the Scope of the Identify the Scope of the ProblemProblem

Identify the scope of the problemIdentify the scope of the problem How many users or network segments are How many users or network segments are

affected?affected? When did the problem begin?When did the problem begin?

If a problem is universal—if it affects the If a problem is universal—if it affects the entire LAN or WAN—you will naturally want entire LAN or WAN—you will naturally want to answer these questions very quicklyto answer these questions very quickly

Identify the Scope of the Identify the Scope of the ProblemProblem

Figure 12-2: Troubleshooting

while identifying the demographic scope

of a problem

Identify the Scope of the Identify the Scope of the ProblemProblem

FIGURE 12-3: Troubleshooting while identifying the chronological

scope of a problem

Establish What Has ChangedEstablish What Has Changed

Did the operating system or configuration on Did the operating system or configuration on a server, workstation, or connectivity device a server, workstation, or connectivity device change?change?

Were new components added to a server, Were new components added to a server, workstation, or connectivity device?workstation, or connectivity device?

Were old components removed from a Were old components removed from a server, workstation, or connectivity device?server, workstation, or connectivity device?

Establish What Has ChangedEstablish What Has Changed

Was a server, workstation, or connectivity Was a server, workstation, or connectivity device moved from its previous location to a device moved from its previous location to a new location?new location?

Was a server, workstation, or connectivity Was a server, workstation, or connectivity device replaced?device replaced?

Was new software installed on a server, Was new software installed on a server, workstations, or connectivity device?workstations, or connectivity device?

Was old software removed from a server, Was old software removed from a server, workstation, or connectivity device?workstation, or connectivity device?

Select the Most Probable CauseSelect the Most Probable Cause

Verify user competencyVerify user competency

Re-create the problemRe-create the problem Can you make the symptoms recur every time?Can you make the symptoms recur every time? Can you make the symptoms recur some of the Can you make the symptoms recur some of the

time?time? Do the symptoms happen only under certain Do the symptoms happen only under certain

circumstances?circumstances? Do the symptoms ever happen when you try to Do the symptoms ever happen when you try to

repeat them?repeat them?

Select the Most Probable CauseSelect the Most Probable Cause

Verify physical connectivityVerify physical connectivity Is the device turned on?Is the device turned on? Is the NIC properly inserted?Is the NIC properly inserted? Is a device’s network cable properly connected to Is a device’s network cable properly connected to

both its NIC and the wall jack?both its NIC and the wall jack? Do patch cables properly connect punch-down Do patch cables properly connect punch-down

blocks to patch panels and patch panels to hubs blocks to patch panels and patch panels to hubs or switches?or switches?

Select the Most Probable CauseSelect the Most Probable Cause

Verify physical connectivity (cont.)Verify physical connectivity (cont.) Is the hub, router, or switch properly connected to the Is the hub, router, or switch properly connected to the

backbone?backbone? Are all cables in good condition?Are all cables in good condition? Are all connectors in good condition and properly Are all connectors in good condition and properly

seated?seated? Do network segment lengths conform to the IEEE 802 Do network segment lengths conform to the IEEE 802

specifications?specifications?

Select the Most Probable CauseSelect the Most Probable Cause

Figure 12-4: Troubleshooting while verifying

physical connectivity

Verify Logical ConnectivityVerify Logical Connectivity

Questions to help you identify a problem Questions to help you identify a problem with logical connectivitywith logical connectivity Do error messages reference damaged or Do error messages reference damaged or

missing files or device drivers?missing files or device drivers? Do error messages reference malfunctioning Do error messages reference malfunctioning

or insufficient resources?or insufficient resources? Has an operating system, configuration, or Has an operating system, configuration, or

application been recently changed, application been recently changed, introduced, or deleted?introduced, or deleted?

Verify Logical ConnectivityVerify Logical Connectivity

Questions to help you identify a problem Questions to help you identify a problem with logical connectivity (cont.)with logical connectivity (cont.) Does the problem occur with only one Does the problem occur with only one

application or a few, similar applications?application or a few, similar applications? Does the problem happen consistently?Does the problem happen consistently? Does the problem affect a single user or one Does the problem affect a single user or one

group of users?group of users?

Implement a SolutionImplement a Solution

Steps to help you implement a safe and Steps to help you implement a safe and reliable solutionreliable solution Collect all the documentation you have about a Collect all the documentation you have about a

problem’s symptomsproblem’s symptoms If you are reinstalling software on a device, make If you are reinstalling software on a device, make

a backup of the device’s existing software a backup of the device’s existing software installationinstallation

Perform the change, replacement, move or add Perform the change, replacement, move or add that you believe will solve the problemthat you believe will solve the problem

Test your solutionTest your solution

Implement a SolutionImplement a Solution

Steps to help you implement a safe and reliable Steps to help you implement a safe and reliable solution (cont.)solution (cont.) Before leaving the area in which you were working, Before leaving the area in which you were working,

clean it upclean it up If the solution fixes the problem, record the details If the solution fixes the problem, record the details

collected about the symptoms, the problem, and the collected about the symptoms, the problem, and the solution in your organization’s call tracking databasesolution in your organization’s call tracking database

If your solution solved a significant change or If your solution solved a significant change or addressed a significant problem, revisit the solution a addressed a significant problem, revisit the solution a day or two later to verify the problem is fixed and day or two later to verify the problem is fixed and hasn’t caused other problemshasn’t caused other problems

Test the SolutionTest the Solution

After implementing the solution, it must be After implementing the solution, it must be tested to verify that it works properlytested to verify that it works properly

Type of testing performed depends on the Type of testing performed depends on the solutionsolution

Often good to enlist the user who reported Often good to enlist the user who reported the problem in testing your solution, toothe problem in testing your solution, too

Recognize the Potential Effects Recognize the Potential Effects of the Solutionof the Solution

When implementing a solutions, consider When implementing a solutions, consider the:the: ScopeScope TradeoffsTradeoffs SecuritySecurity ScalabilityScalability CostCost

Document the Problems and Document the Problems and SolutionsSolutions

Name, department, and phone number of Name, department, and phone number of the problem originatorthe problem originator

Information regarding whether the problem Information regarding whether the problem is software- or hardware-relatedis software- or hardware-related

If software-related, the package to which it If software-related, the package to which it pertainspertains

If hardware related, the device or If hardware related, the device or components to which it pertainscomponents to which it pertains

Document the Problems and Document the Problems and SolutionsSolutions

Symptom of the problem, including when it Symptom of the problem, including when it was first noticedwas first noticed

Name and telephone number of the Name and telephone number of the network support contactnetwork support contact

Amount of time spent troubleshooting the Amount of time spent troubleshooting the problemproblem

Resolution of the problemResolution of the problem

Hardware Troubleshooting Hardware Troubleshooting ToolsTools

Crossover cableCrossover cable Cable in which the transmit and receive wire pairs in Cable in which the transmit and receive wire pairs in

one of the connectors are reversedone of the connectors are reversed

Tone generator and tone locatorTone generator and tone locator A tone generator is a small electronic device that A tone generator is a small electronic device that

issues a signal on a wire pairissues a signal on a wire pair A tone locator is a device that emits a tone when it A tone locator is a device that emits a tone when it

detects electrical activity on a wire pairdetects electrical activity on a wire pair Combination of these devices is known as a Combination of these devices is known as a fox and fox and

houndhound

Tone Generator and Tone Tone Generator and Tone LocatorLocator

Figure 12-5: Use of a tone generator and tone locator

MultimeterMultimeter

MultimeterMultimeter Simple instrument that can measure multiple Simple instrument that can measure multiple

characteristics of an electric circuitcharacteristics of an electric circuit

VoltmeterVoltmeter Device used to measure voltage of an electric Device used to measure voltage of an electric

currentcurrent

MultimeterMultimeter

ResistanceResistance Opposition to electric currentOpposition to electric current Resistance is measured in ohmsResistance is measured in ohms The device used to measure resistance is an The device used to measure resistance is an ohmmeterohmmeter

Figure 12-6: A multimeter

Uses for a MultimeterUses for a Multimeter

Verify that a cable is properly conducting Verify that a cable is properly conducting electricityelectricity

Check for the presence of noise on a wireCheck for the presence of noise on a wire

Verify that amount of resistance generated by Verify that amount of resistance generated by terminators on coaxial cable networks is terminators on coaxial cable networks is appropriate or whether terminators are appropriate or whether terminators are actually present and functionalactually present and functional

Test for short or open circuitsTest for short or open circuits

Cable CheckersCable Checkers

Cable checkersCable checkers Determine Determine

whether your whether your cabling can cabling can provide provide connectivityconnectivity

Figure 12-7: Basic cable checker

Cable TestersCable Testers

A cable tester performs the same A cable tester performs the same continuity and fault tests as a cable continuity and fault tests as a cable checker, but also provides the following checker, but also provides the following functions:functions: Ensures that cable is not too longEnsures that cable is not too long Measure the distance to a cable faultMeasure the distance to a cable fault Measures attenuation along a cableMeasures attenuation along a cable

Cable TestersCable Testers

Additional functions provided by a cable Additional functions provided by a cable tester (cont.)tester (cont.) Measures near-end crosstalk between wiresMeasures near-end crosstalk between wires Measures termination resistance and impedance Measures termination resistance and impedance

for Thinnet cablingfor Thinnet cabling Issues pass/fail ratings for CAT3, CAT5, CAT6, or Issues pass/fail ratings for CAT3, CAT5, CAT6, or

even CAT7 standardseven CAT7 standards Stores and prints cable testing resultsStores and prints cable testing results

Cable TestersCable Testers

Figure 12-8: A high-end cable

tester

Time Domain Reflectors (TDRs)Time Domain Reflectors (TDRs)

High-end instrument for testing qualities of High-end instrument for testing qualities of a cablea cable

Optical time domain reflectors (OTDRs)Optical time domain reflectors (OTDRs) TDRs made of fiber-optic cableTDRs made of fiber-optic cable

BaselineBaseline Record of how well a network operates under Record of how well a network operates under

normal conditionsnormal conditions

Software Troubleshooting ToolsSoftware Troubleshooting Tools

Network monitorsNetwork monitors Software-based tools that continually monitor Software-based tools that continually monitor

traffic on the network from a server or traffic on the network from a server or workstation attached to the networkworkstation attached to the network

To take advantage of software-based To take advantage of software-based monitoring and analyzing tools, the network monitoring and analyzing tools, the network adapter installed in your machine must adapter installed in your machine must support support promiscuous modepromiscuous mode

Abnormal Data Patterns and Abnormal Data Patterns and PacketsPackets

Local collisionsLocal collisions occur when two or more occur when two or more stations are transmitting simultaneouslystations are transmitting simultaneously

Late collisionsLate collisions take place outside the window take place outside the window of time in which they would normally be detected of time in which they would normally be detected by the network and redressedby the network and redressed

RuntsRunts are smaller than the medium’s minimum are smaller than the medium’s minimum packet sizepacket size

GiantsGiants exceed the medium’s maximum packet exceed the medium’s maximum packet sizesize

Abnormal Data Patterns and Abnormal Data Patterns and PacketsPackets

JabberJabber is a device that handles electrical is a device that handles electrical signals improperlysignals improperly

Negative frame sequence checksNegative frame sequence checks are the are the result of the cyclic redundancy checksum (CRC) result of the cyclic redundancy checksum (CRC) generated by the originating node not matching generated by the originating node not matching the checksum calculated from the data receivedthe checksum calculated from the data received

GhostsGhosts are frames that are not actually data are frames that are not actually data frames, but aberrations caused by a repeater frames, but aberrations caused by a repeater misinterpreting stray voltage on the wiremisinterpreting stray voltage on the wire

Capabilities of Microsoft’s Network Capabilities of Microsoft’s Network Monitor (NetMon)Monitor (NetMon)

Capturing network data traveling from one or Capturing network data traveling from one or many segmentsmany segments

Capturing frames sent by or to a specified nodeCapturing frames sent by or to a specified node

Reproducing network conditions by Reproducing network conditions by transmitting a selected amount and type of transmitting a selected amount and type of datadata

Detecting any other running copies of Detecting any other running copies of NetMon on the networkNetMon on the network

Generating statistics about network activityGenerating statistics about network activity

Novell’s LANalyzerNovell’s LANalyzer

LANalyzer performs the following LANalyzer performs the following functions:functions: Initially discovering all network nodes on a Initially discovering all network nodes on a

segmentsegment Continuously monitoring network trafficContinuously monitoring network traffic Tripping alarms when traffic conditions Tripping alarms when traffic conditions

meet preconfigured thresholdsmeet preconfigured thresholds Capturing traffic to and from all or selected Capturing traffic to and from all or selected

nodesnodes

Network AnalyzersNetwork Analyzers

Tool that can capture traffic and analyze Tool that can capture traffic and analyze packetspackets

Also known as Also known as protocol analyzersprotocol analyzers

Sniffer PortableSniffer Portable Network analyzer from Network AssociatesNetwork analyzer from Network Associates

SnifferSniffer Laptop equipped with special network adapter Laptop equipped with special network adapter

and software that performs network analysisand software that performs network analysis

Physical Layer Problems and Physical Layer Problems and SymptomsSymptoms

Figure 12-9: Services and devices in the

OSI Model

Physical Layer Problems and Physical Layer Problems and SymptomsSymptoms

Segment problemsSegment problems

Signal noiseSignal noise

Cable damageCable damage

Connector flawsConnector flaws

Adapter flawsAdapter flaws

Staff Involved in TroubleshootingStaff Involved in Troubleshooting

Help desk staffed with help analystsHelp desk staffed with help analysts Organizations may group their help desk Organizations may group their help desk

analysts into teams based on their expertiseanalysts into teams based on their expertise

Most help desks have a help desk Most help desks have a help desk coordinatorcoordinator

Most organizations have an operations Most organizations have an operations manager, who supervises the help desk manager, who supervises the help desk coordinatorcoordinator

Swapping EquipmentSwapping Equipment

If a suspected problems lies with a network If a suspected problems lies with a network component, an easy way to test the theory is component, an easy way to test the theory is to exchange that component for a functional to exchange that component for a functional oneone

A better alternative to swapping parts is to A better alternative to swapping parts is to have redundancy built into your networkhave redundancy built into your network

Before swapping any network component, Before swapping any network component, make sure the replacement has exactly the make sure the replacement has exactly the same specifications as the original partsame specifications as the original part

Using Vendor InformationUsing Vendor Information

Table 12-1: Links for troubleshooting resources on the Web

Notify Others of ChangeNotify Others of Change

Change management systemChange management system Process or program that provides support Process or program that provides support

personnel with a centralized means of personnel with a centralized means of documenting changes to the networkdocumenting changes to the network

Type of changes that should be recorded in a Type of changes that should be recorded in a change management systemchange management system Adding or upgrading software on network servers Adding or upgrading software on network servers

or other devicesor other devices Adding or upgrading hardware components on Adding or upgrading hardware components on

network servers or other devicesnetwork servers or other devices

Notify Others of ChangeNotify Others of Change

Type of changes that should be recorded Type of changes that should be recorded in a change management system (cont.)in a change management system (cont.) Adding new hardware on the networkAdding new hardware on the network Changing the network properties of a network Changing the network properties of a network

devicedevice Increasing or decreasing rights for a group of Increasing or decreasing rights for a group of

usersusers Physically moving networked devicesPhysically moving networked devices

Notify Others of ChangeNotify Others of Change

Type of changes that should be recorded in a Type of changes that should be recorded in a change management system (cont.)change management system (cont.) Moving user IDs and their files/directories from Moving user IDs and their files/directories from

one server to anotherone server to another Making changes in processesMaking changes in processes Making changes in vendor policies or Making changes in vendor policies or

relationshipsrelationships

Preventing future problemsPreventing future problems Though not all network problems are preventable, Though not all network problems are preventable,

many are avoidablemany are avoidable

Chapter SummaryChapter Summary

Before you can resolve a network Before you can resolve a network problem, you must determine its causeproblem, you must determine its cause

The key to solving network problems is to The key to solving network problems is to approach them methodically and logicallyapproach them methodically and logically

When assessing a network problem, act When assessing a network problem, act like a doctor diagnosing a patientlike a doctor diagnosing a patient

After implementing your solution, test it to After implementing your solution, test it to ensure that it works correctlyensure that it works correctly

Chapter SummaryChapter Summary

Be familiar with the different hardware-Be familiar with the different hardware-based troubleshooting toolsbased troubleshooting toolsBe familiar with the different software-Be familiar with the different software-based troubleshooting toolsbased troubleshooting toolsMost organizations operate a help deskMost organizations operate a help deskIf you suspect a problem lies with a If you suspect a problem lies with a network component, one of the easiest network component, one of the easiest ways to test your theory is to exchange ways to test your theory is to exchange that component for a functional onethat component for a functional one

Chapter SummaryChapter Summary

Keep a list of hardware and software vendors for Keep a list of hardware and software vendors for your networking equipmentyour networking equipmentSome organizations use a call tracking systemSome organizations use a call tracking systemIn addition to communicating problems and In addition to communicating problems and solutions to your peers whenever you work on a solutions to your peers whenever you work on a network problem, you should follow up with the network problem, you should follow up with the person who reported the problemperson who reported the problemOrganizations often implement change Organizations often implement change management systems to methodically track management systems to methodically track changes on the networkchanges on the network