The Network Affandi Singaren, Singapore Doug Pearson Indiana University Internet2 Commons Site...

The Network

Affandi Singaren, Singapore

Doug PearsonIndiana University

Internet2 Commons Site Coordinator TrainingDecember 3, 2003National University of Singapore

2

A Sample University Network

LAN

Building network

Campus backbone

WAN intranet

Internet

Internet2

3

Sample Network

UNIVERSITY3rd floor

2nd floor

1st floor

Ethernet switch horizontal wiring station cable device

router

WAN

Campus Backbone

datajack

iMac

risers

CommercialInternet

LAN and Building Network

intranet

Internet2

4

Wiring and Station Cables

3rd floor

Ethernet switch horizontal wiring station cable devicedatajack

iMac

Common practice is unshielded twisted-pair (UTP) according to the specs:

• CAT 3 [old] supports 10 Mbps Ethernet (10base-T)• CAT 5 [modern] supports 10base-T, 100 Mbps

(100base-TX) and 1000 Mbps (1000base-T) Ethernet • CAT 6 [new] supports CAT 5 applications +

5

Wiring and Station Cables

Actual wire used and quality of installation may vary widely – know your wiring!

Important to consider the station cables • Don’t use sub-CAT 5 station cables for 100 Mbps connections.

6

Ethernet LAN

3rd floor

2nd floor

1st floor


router

datajack

iMac

7

Ethernet LAN

10 / 100 / 1000 Mbps

Full- and Half-Duplex• Half-duplex: send or receive, one at a time.• Full-duplex: send and receive simultaneously.• 10 Mbps Ethernet supports half-duplex; full-duplex is not consistently implemented.

• 100 Mbps supports half- and full-duplex.

8

LAN: Switches vs. Repeaters

Repeaters (hubs) are old technology. A repeater sends (repeats) packets

that are incoming on one port, out all other ports (I know you’re out there somewhere!).

Can only operate in half-duplex mode. Bandwidth and jitter provided to any

single device is highly dependent on the LAN traffic.

9

LAN: Switches vs. Repeaters

A switch learns the MAC addresses of the devices connected to it, and sends packets directly and only to the target end-point.

Provides much more consistent bandwidth and latency (low jitter).

A well-designed switched LAN is important for videoconferencing. Repeater-based LANs should be upgraded to switched for videoconferencing!

10

LAN: Ethernet Duplex Mismatch

“One of the most common causes of performance issues on 10/100Mb Ethernet links is when one port on the link is operating at half-duplex while the other port is operating at full-duplex.”• http://www.cisco.com/warp/public/473/3.html

11


“There is a silent performance-killer out there, one so inconspicuous that it is hardly ever looked for or even suspected. You could suffer from it and never know it, as it robs a site of performance but not connectivity. This performance-killer has a name: Ethernet duplex mismatch.”

• http://www.hostingtech.com/nm/01_01_mismatch.html

12


If one end of a connection (device or Ethernet switch) is set for auto-negotiation, and fails to see auto-negotiation at the other end, the former sets itself to the default, half-duplex.

Auto-negotiation can sometimes fail, even when both sides are set to auto (although this isn’t as prevalent as in the past).

13

LAN: Duplex Mismatch – Detection

Microsoft Windows doesn’t display the auto-negotiated duplex setting.

Some routers re-negotiate Auto-duplex, which introduces jitter.

14


switch device switch device

auto auto auto autohalf half half halffull full full full

auto full BAD! half fullfull auto BAD! full half

auto half half halfhalf auto half half

SETTINGS RESULTS

15

LAN: Duplex Mismatch – Detection

Show port statistics on the Ethernet switch. When mismatched, the full-duplex end will report a high level of CRC or alignment errors; the half-duplex end will report a high number of late collisions.

Port Align FCS Xmit Rcv UnderSize

Err Err Err Err

2/11 - 0 0 3077 0

Port Single- Multi- Late- Excess- CarriSen Runts Giants

Coll Coll Coll Coll

2/11 3233 0 2588 0 0 2489 0

16

LAN: Duplex Mismatch- Prevention

Always configure switches and devices according to your local policy. An example policy is:• If building wiring is sub-CAT 5, then set switch ports to 10/half

• If building wiring is CAT 5 or better, then set switch ports and devices to Auto.

Monitor switch port stats and logs

17

Router

3rd floor

2nd floor

1st floor

Ethernet switch

router

Segments LANs into distinct networks and subnetworks, e.g., the distinct red, green, and blue LANs with distinct network numbers.

Segments LANs into broadcast domains

18

Router

WAN

Campus Backbone

intranet Internet2intranetInternet2

CommercialInternet

Provides interface to the WAN.• Intranet, commercial

Internet, and Internet2 connections.

• Typically, every networked device at an Internet2-connected institution has connectivity to Internet2.

19

VLAN

A single, physical LAN can be logically segmented into multiple logical LANs; and,

Physically separate LANs can be made to behave and appear as a single LAN.

20

VLAN

Packets are tagged according to LAN membership, e.g., green LAN, red LAN, and blue LAN.

Ethernet switches establish broadcast domains according to the defined VLAN boundaries.

Routers establish multiple VLANs on a single interface.

21

VLAN

3rd floor

2nd floor

1st floor

Campus Backbone

3rd floor

2nd floor

1st floor

Building A Building B

Router

22

VLAN

Modern campus network architectures are tending to move away from traditional router-for-a-building design, to VLAN designs.

23

Old Design Included a lot of Routers

UNIVERSITY3rd floor

2nd floor

1st floor


router

WAN

Campus Backbone

datajack

iMac

risers

CommercialInternet


intranet

Internet2routers

24

New Design Includes VLAN Router

router

UNIVERSITY3rd floor

2nd floor

1st floor


WAN

Campus Backbone

datajack

iMac

risers

CommercialInternet


intranet

Internet2

25

WAN Segments

Service Speed Sample Uses

T1 1.5 Mbps remote building; extension center

DS3 45 Mbps inter-campus; Internet (I1) connection

OC3 155 Mbps inter-campus; I1 & Internet2 connection

OC12 622 Mbps I1 backbones; Internet2 connection

OC48 2.4 Gbps I1 and Internet2 backbones

GigabitEthernet

1 Gbps advanced inter-campus connections when have access to dark fiber

26

Indiana University Abilene NOC Weathermap

27

High Performance Research and Education Networks

Internet2 / Abilene• http://www.internet2.edu• http://www.abilene.iu.edu

STARTAP and International Networks• http://www.startap.net

US Government-Sponsored Networks• http://www.startap.net/NETWORKS

28

Traffic on the Network

Typical university today:• IP

– TCP– UDP

• IPX [diminishing]• Appletalk [diminishing]

29

Traffic on the LAN

Unicast : one-to-one

Multicast: one-to-many

Broadcast: one-to-every

30

Unicast

Most common traffic

Common applications: mail, Web browsing, file transfer, etc.

31

IP Multicast

A one-to-many mode of transmission

Network numbers 224.0.0.0 through 239.255.255.255 are reserved for multicast.

Examples of multicast applications:• Vic/rat videoconferencing• Centralized PC software administration tools such as Symantec Ghost

32

IP Multicast – Leak Problems

Beware: high rates of unpruned multicast can adversely affect videoconference performance.

Use a network traffic and protocol analyzer to identify this problem.

33

Broadcast

A one-to-every mode of transmission Used by network protocols including ARP

and IPX, NetBIOS system discovery, and name resolution.

All devices on the network must process every broadcast packet; high broadcast rates can divert processing capacity.

If the broadcast domain is too large or unusually active, the activity required at the end-point to deal with the broadcasts could diminish performance.

34

Broadcast

A healthy network should have less than 100 broadcast packets per second.

Check using a network traffic and protocol analyzer tool.

35

Firewalls

A firewall is a network node that acts to enforce an access control policy between two networks, e.g., between a university intranet and the commercial Internet.

Used to secure IT resources against external attacks and break-ins.

Network-layer firewalls typically make their decisions based upon port numbers and source/destination addresses.

Application-layer firewalls act as proxies.

36

Firewalls

H.323 uses the IP ports:• Statically-assigned TCP ports 1718 – 1720 and 1731 for call setup and control.

• Dynamically-assigned UDP ports in the range of 1024 – 65535 for video and audio data streams.

Firewalls don’t allow unrestricted ports. Typical modern firewalls and H.323 don’t get along so well.

37

Firewalls – Solutions for H.323

[bad; non-scaleable] Allow unrestricted ports for specific, known, external IP-addresses.

[better, but still not so good] Use feature of some videoconferencing clients to confine dynamic ports to a specific, narrow range.

[OK, but extra admin work and cost] Use an H.323 application proxy.

[best] Use a firewall that snoops on the H.323 call set-up channels (static ports) and opens ports for the audio/video (dynamic ports) as needed.

38

NATs

Allows multiple computers behind the NAT to share one external network address.

Uses:• Alleviate shortage of IP addresses• Security – obscures view of the network from outside

• Flexible network administration

Not commonly used at universities on the campus level. Used somewhat in corporations. Common in small offices and at home – behind DSL, cable modem, or ISDN network service.

39

NATs

Difficult to use H.323 behind NATs.

Some videoconferencing terminals provide features to work with NAT – refer to videoconferencing terminal documentation.

40

Latency

Latency is the time required for a packet to traverse a network from source to destination.

Components of latency include:• Propagation delay: the time it takes to traverse the distance of the transmission line; controlled by the speed of light in the media; rule-of-thumb: 20ms San Francisco to New York.

41

Latency

Transmission delay: the time it takes for the source to put a packet on the network. Rule-of-thumb: < 1ms.

Store-and-forward delay: the cumulative length of time it takes the internetworking devices along the path to receive, process, and resend the packets. Rule-of-thumb: variable, and depends upon network load.

42

Latency

Rule of thumb:• A one-way delay of:

• 0 – 150 ms provides excellent interactivity• 150 – 300 ms is OK• 300 – 400 ms is bad• 400+ ms is unacceptable

43

Jitter

Jitter is variation in latency over time.If the endpoints are on switched LANs,

then the primary source of jitter is variation in the store-and-forward time, resulting from network load.

H.323, particularly audio, is adversely affected by high levels of jitter.

What is high? Rule of thumb?

44

Packet Loss

Packet loss is typically due to congested links and routers.

• 1% is noticeable

• 5% becomes intolerable

45

QoS

Not currently feasible on commercial Internet and Internet2 networks for production, regular use. Internet2 is working on QoS plans, but the current over-provisioned Internet2 network doesn’t dictate need.

Is useful on over-utilized intranet WAN links.

46

QoS

How:•Some videoconferencing terminals can set the IP precedence bits. Use that for marking and priority queuing on the WAN.

Or:

•Use a H.323 Proxy for consolidation of traffic to a single address, router access list for marking, and priority queuing on the WAN.

47

QoS

Caution!•The wrong implementation could result in unwanted tradeoffs, e.g., packet loss improves but jitter gets worse.

48

The End-to-End Performance Problem

Scenario• Users on two different campuses of a university are experiencing poor video and audio in a conference.

• Each user is supported by a different group of videoconferencing engineers.

• Each campus is supported by a different group of network engineers.

• The wide-area network is supported by a third group of network engineers.

49


Problem• How do the users get timely, useful assistance?• How is network problem resolution coordinated?

50


Obstacles• Different groups, schedules, and priorities.• No one engineer has a complete understanding of the entire network path.

• No one engineer can gain access to all the network nodes (routers, switches) along the path to inspect for trouble.

• Communications are inconsistent from engineer to engineer.

51


Solutions• Articulate the E-2-E problem to network management and engineers on all campuses.

• Establish reliable communication tools, and insist that engineers utilize the tools.

• Hold regular meetings; bring all engineers together in one place and time to share information.

• Have good network documentation for all networks.

52

H.323 is Network Sensitive!

The big problems are:• Half/Full-duplex mismatches• Packet loss• Jitter• Substandard horizontal wiring or station cables• Multicast leaks• High broadcast rates

53

Tools

Ping – availability, loss, roundtrip time Traceroute – path discovery Pingplot – graphical traceroute/ping MRTG – graph link/port utilization & errors Iperf – bandwidth, loss and jitter Gnuplotping – visualize jitter Sniffer – inspect traffic on the LAN VideNet Scout – bandwidth, loss and jitter Internet2 Detective – detect I2 connection H.323. Beacon –protocol-specific tests

54

Tools: Ping

Test for availability, loss, and roundtrip time

ICMP Echo Request• Plus optional dummy payload – only in the direction of the ping, i.e., source destination

55

Tools: Sample Ping from Windows

C:\WINDOWS>ping 10.1.1.1 Pinging 10.1.1.1 with 32 bytes of data: Reply from 10.1.1.1: bytes=32 time=88ms TTL=112 Reply from 10.1.1.1: bytes=32 time=72ms TTL=112 Reply from 10.1.1.1: bytes=32 time=69ms TTL=112 Reply from 10.1.1.1: bytes=32 time=69ms TTL=112 Ping statistics for 10.1.1.1: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds: Minimum = 69ms, Maximum = 88ms, Average = 74ms C:\WINDOWS>

56

Tools: Sample Ping from Windows

C:\WINDOWS>ping -l 40000 10.1.1.1

Pinging 10.1.1.1 with 40000 bytes of data:

Reply from 10.1.1.1: bytes=40000 time=2412ms TTL=112




Ping statistics for 10.1.1.1:

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 2412ms, Maximum = 2761ms, Average = 2652ms

C:\WINDOWS>

57

Tools: Ping Big and Fast on Linux

ping -c2000 -i.03 -s1470 -q [destination]• Count of 2000 packets• Interval of .03 seconds between packet starts• Packet size of 1470 bytes• (2000)(.03) = 60 second long test• (1/(.03 sec/packet))(1470 bytes/packet)(8 bits/byte) = 392 Kbps

58

Tools: Ping Big and Fast on Linux

$ ping -c2000 -i.03 -s1470 -q [hostname]

PING [hostname] ([hostaddr]) from [hostaddr2] : 1470(1498) bytes of data.

--- [hostname] ping statistics ---2000 packets transmitted, 2000 packets received, 0% packet loss

round-trip min/avg/max = 4.8/5.1/13.2 ms

59

Tools: Traceroute

Used to discover the layer-3 network path (routers) between the two endpoints

Doesn’t identify layer-2 devices (switches)

Must run from one of the discovery endpoints – it can’t act as a third party.

Take baselines – know what your path should be in advance of trouble

60

Tools: Traceroute

Microsoft tracert uses ICMP packets, whereas Unix traceroute uses UDP; may be of importance in networks where routers are configured to not respond to ICMP; or if ICMP is blocked.

61

Tools: Traceroute; Sample Output

[dodpears@huck dodpears]$ traceroute www.internet2.edu traceroute to www.internet2.edu (209.211.239.208), 30 hops max, 38 byte

packets

1 wcc-sub5-hp1 (129.79.5.253) 11.726 ms 0.627 ms 0.571 ms 2 iub-gw (129.79.8.10) 3.133 ms 0.717 ms 0.651 ms 3 156.56.249.22 (156.56.249.22) 2.544 ms 3.138 ms 2.538 ms 4 abilene-iupui.abilene.ucaid.edu (198.32.11.13) 5.245 ms 3.402 ms 3.493

ms 5 clev-ipls.abilene.ucaid.edu (198.32.8.26) 9.381 ms 9.586 ms 9.244 ms 6 nycm-clev.abilene.ucaid.edu (198.32.8.30) 23.198 ms 21.963 ms 21.775

ms 7 border-abilene-oc3.advanced.org (209.211.237.97) 23.448 ms 23.268 ms

23.052 ms 8 www.internet2.edu (209.211.239.208) 23.559 ms 23.478 ms 23.234 ms

62

Tools: Traceroute

8 www.internet2.edu (209.211.239.208) 23.559 ms 23.478 ms 23.234 ms

Hop

Router/host name

Router/host address

Round-trip times of each of three probes

63

Tools: Traceroute – Example Uses

Identify the path, and then perform pings along the path segments to isolate troublesome segments.

Insure that Internet2 is being used for a connection, rather than commercial Internet.

64

Tools: Ping Plotter

Shareware tool; $15• http://www.pingplotter.com

Performs a visual traceroute and ping tests along the entire path

Permits identification of bottlenecks along a path

65

Tools: Ping Plotter

66

Tools: MRTG

Multi Router Traffic Grapher

Collect and graph scalar, time-based data, e.g., router and link performance data.

67

Tools: MRTG

68

Tools: MRTG

69

Tools: Iperf

http://dast.nlanr.net/Projects/Iperf/

Client/server application that• Measures maximum TCP bandwidth• Facilitates tuning of TCP and UDP parameters• Reports bandwidth, jitter, and packet loss

70

Tools: IperfExample on Intercampus DS3

At server, invoke:

iperf -fk -i30 -u -s(f)ormat reports in kbps

(i)nterval for reporting = 30 seconds

(u)dp

(s)erver mode

71


At client, invoke:

iperf -u -b800k -t3600 -c [hostname-server](u)dp

(b)andwidth = 800kbps

(t)ime of run = 3600 seconds

(c)lient mode

[hostname-server] = server to target

72


[dodpears@vc-iperf iperf]$ iperf -fk -i30 -u -s

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1470 byte datagrams

UDP buffer size: 64.0 KByte (default)

------------------------------------------------------------

[ 3] local 149.166.197.80 port 5001 connected with 129.79.92.230 port 1031

[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams

[ 3] 0.0-30.0 sec 3000 KBytes 819 Kbits/sec 0.300 ms 0/ 2090 (0%)



[...]


73

Iperf- Example on Intercampus DS3

MRTG utilization graph showed bandwidth peaking at capacity ~ 10:00a – 2:00p

As utilization peaked on the DS3, jitter measured by Iperf rose to unacceptable level

Iperf also reported periodic high packet loss, with no apparent correlation to the low-resolution MRTG utilization reports

74


Second day, utilization as reported by MRTG is staying reasonable.

Jitter measured by Iperf is staying low.

The periodic high packet loss remains, until noon when network engineer adjusted the QoS settings.

75

Third day, utilization peaking at 3:00p.

As utilization peaked jitter measured by Iperf also rising.

Still no packet loss.QoS fixed the packet loss

problem, but still not certain about the jitter – more analysis needed.


76

Tools: Gnuplotping

Pings multiple hosts in parallel with graphical display (gnuplot) of the delay distribution.

Runs on Unix/X-Windows

77

Tools: gnuplotping

78

Tools: Network Traffic Analyzer

Reveals the traffic on a LAN

Protocol analysis

Reports such as utilization, protocols, conversations, nodes, etc.

Network General Sniffer

WildPackets EtherPeek

79

Tools: ViDeNet Scout

Scout is a web-based, distributed network performance analysis tool developed at the University of North Carolina at Chapel Hill.

Scout makes use of the Chariot performance testing engine developed by NetIQ.• http://scout.video.unc.edu/

80

Tools: Internet2 Detective

I2 Detective is a small application.

Detects Internet2 connection.

Measures connection bandwidth (using Iperf).

Detects multicast connection.• http://detective.internet2.edu/

81

Tools: H.323 Beacon

Used to measure, monitor and qualify the performance of an H.323 Videoconference session.

Provides H.323-protocol specific evidence and other information necessary to troubleshoot H.323 application performance problems in the network and at the host (end-to-end) • http://www.itecohio.org/beacon/

82

Recommendations

Develop a close relationship with the network engineers and NOC. Make sure they understand what’s being done with videoconferencing and the network sensitivity of IP-based video.

Articulate the End-to-End Performance Problem to network engineering and operations management. Champion ways to reduce the problem.

Be sure to open trouble tickets with your NOC so that a problem history is maintained.

83

Recommendations

Use switched Ethernet.

Watch out for duplex mismatches.

Keep an eye on utilization of WAN links, packet loss, and jitter.

Make sure you don’t have broadcast or multicast leaking problems.

Make sure wiring is up to the task.

84

Recommendations

Have engineers in the videoconferencing support group trained to understand networking issues and tools.

Date post:	23-Dec-2015
Category:	Documents
Upload:	primrose-greene
View:	219 times
Download:	0 times

The Network Affandi Singaren, Singapore Doug Pearson Indiana University Internet2 Commons Site...

Documents