Licentiate Seminar:

On Measurement and Analysis of Internet Backbone Traffic

Wolfgang John Department of Computer Science and Engineering

Chalmers University of TechnologyGöteborg, Sweden

2008-02-29Licentiate Seminar Wolfgang John

Internet, 1983Internet, 2005

Why measure Internet traffic? (1)

The Internet is changing in size

ARPANET, 1969

2008-02-29Licentiate Seminar Wolfgang John

The Internet is changing in application

Why measure Internet traffic? (2)

2008-02-29Licentiate Seminar Wolfgang John

• The Internet– is constantly developing– is used differently in different locations– is heterogeneous

The Internet is not understood in its entirety!

INTERconnected NETworks

Why measure Internet traffic? (3)

INTER NET

2008-02-29Licentiate Seminar Wolfgang John

• Operational purpose– Troubleshooting, provisioning, planning ….

• Scientific purpose– Protocols, infrastructure and services

– Performance properties

– Internet simulation models

– Security measures

Why measure Internet traffic? (4)

2008-02-29Licentiate Seminar Wolfgang John

Thesis Objectives

1. Guidelines for Internet measurement

2. Current traffic characteristics

3. Traffic decomposition

4. Inconsistent behavior

2008-02-29Licentiate Seminar Wolfgang John

Outline

• Measurement approaches

• Internet measurement challenges

• The MonNet project

• Scientific contribution

• Results– Four studies included

• Conclusions

Measurement

Analysis

2008-02-29Licentiate Seminar Wolfgang John

Measurement approaches

Network traffic measurement

Active Passive

Software Hardware

Online Offline

Flows Packets

Complete Headers

Different protocol levels

Statistical summaries

Transport layer

2008-02-29Licentiate Seminar Wolfgang John

Internet measurement challenges (1)

• Legal considerations

• Ethical and moral considerations

• Operational considerations

• Technical considerations

2008-02-29Licentiate Seminar Wolfgang John

Measurement challenges (3)

Technical considerations

• Data amount– Exhausting I/O and storage access speeds

• Data reduction techniques– Filtering, sampling, packet truncation

• Timing– Clock synchronization

2008-02-29Licentiate Seminar Wolfgang John

The MonNet Project (1)

Technical Solution

10 GbpsGöteborg

splitterBorås

10 Gbps

Processing Platform and Storage

Measurement Node 2

Measurement Node 1

2008-02-29Licentiate Seminar Wolfgang John

The MonNet Project (2)

Internet

Internet

Regiona

l ISPsRegiona

l ISPs

Göteborg

Stockholm

Other smaller Univ. and Institutes

Göteborgs Univ.

Student-Net

Chalmers Univ.

Measurement location

Borås

•April 2006 148 traces (20 minutes) 11 billion packets, 7.6 TB of data

•Sept. – Nov. 2006 554 traces (10 minutes) 28 billion packets, 19.5 TB of data

2008-02-29Licentiate Seminar Wolfgang John

Scientific ContributionLevel of

complexity

Quantification of inconsistent

behavior

Traffic characterization

Packet level Flow level Traffic classes

Stu

dy I

Stu

dy II

Stu

dy IV

Stu

dy III

Upcoming

2008-02-29Licentiate Seminar Wolfgang John

Study I: Packet Level Analysis

• Updated packet-level characteristics of Internet traffic

• Inconsistencies in headers will appear

– Network attacks and malicious traffic– Active OS fingerprinting– Buggy applications or protocol stacks

2008-02-29Licentiate Seminar Wolfgang John

• High level analysis does not necessarily show differences → detailed analysis does!

• 2 main reasons for directional differences: – Malicious traffic

• the Internet is “unfriendly”

– P2P• Göteborg is a P2P source• P2P is changing traffic characteristics

e.g. packet sizes, TCP termination, TCP option usage

Study II: Flow level analysis

2008-02-29Licentiate Seminar Wolfgang John

Study III: Classification Method (1)

• Classification of flow traffic without payload

• Heuristics to identify nature of endpoints

• Rules based on connection patterns and port numbers– 5 rules for P2P traffic

– 10 rules to classify other types of traffic• remove ‘false positives’ from P2P

2008-02-29Licentiate Seminar Wolfgang John

Study III: Classification Method (2)

# connections in 106 Amount of data in TB

Comparison of classification methods for P2P traffic

2008-02-29Licentiate Seminar Wolfgang John

Study III: Classification Method (3)

• Previous classification methods on packet header traces don’t work well on backbone data

• Proposal of refined and updated heuristics– Simple and fast method to decompose traffic– No payload required– Effectively used even on short traces (10 min)

• 0.2% of the data left unclassified

2008-02-29Licentiate Seminar Wolfgang John

Study IV: Classification Results (1)

Tuesday, 18.04.2006

2008-02-29Licentiate Seminar Wolfgang John

Study IV: Classification Results (2)

Application breakdown April till Nov. 2006

2008-02-29Licentiate Seminar Wolfgang John

Study IV: Classification Results (3)

Connection establishment for traffic classes

2008-02-29Licentiate Seminar Wolfgang John

Study IV: Classification Results (4)

• Behavior of P2P traffic– Unsuccessful TCP connection attempts increasing

– Serving peers terminate with FIN and RSTDecreased from 20% to 8%

– UDP overlay traffic doubled

• TCP options deployment differs– P2P behaves as expected

– Web traffic shows artifacts of client-server pattere.g. popular web-servers neglecting SACK option

2008-02-29Licentiate Seminar Wolfgang John

Summary

1. Guidelines for Internet measurement• Experiences of the MonNet project

2. Current traffic characteristics• Packet and flow level

3. Traffic decomposition• Traffic classification method

4. Inconsistent behavior• Packet header anomalies

• Malicious traffic flows

2008-02-29Licentiate Seminar Wolfgang John

General remarks

• Internet today is essential, but still not understood entirely• Large-scale traffic measurements uncommon

– A lot of analysis is done on outdated datasets

• Each study generated as much questions as answers• Reconsider measurement process (duration, payload…)• A lot of open questions …

…get more answers in two years…