2006/12/04 NTU/nslab 1

Jigsaw: Solving the Puzzle of Enterprise 802.11 Analysis

Yu Chung Cheng, John Bellardo, P´eter Benk¨oAlex C. Snoeren, Geoffrey M. Voelker and Stefan Savage

Dep. CS and Eng., UCSD

SIGCOMM 2006

Jeffrey Hsiao

2006/12/04 NTU/nslab 2

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 3

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 4

Introduction

• Wireless networks based on the 802.11 have become ubiquitous

• Developed a large-scale monitor infrastructure – overlays a building-scale production

802.11b/g network– with over 150 passive radio monitors

2006/12/04 NTU/nslab 5

Jigsaw

• These monitors in turn feed a centralized system, called Jigsaw

• Jigsaw uses this data to produce a precisely synchronized global picture– Contains all physical, link-layer, network-layer

and transport-layer activity

2006/12/04 NTU/nslab 6

Principal Contributions• Large-scale Synchronization

– designed and implemented a passive synchronization algorithm

– can accurately synchronize over 150 simultaneous traces

– down to microsecond granularity• Frame Unification

– combine the contents of all traces– merge duplicates and constructing a synchronized sin

gle trace of all frame transmissions• Multi-layer Reconstruction

– Reconstruct a complete description of all link and transport-layer conversations

2006/12/04 NTU/nslab 7

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 8

Background and related work

• Operation of the 802.11 protocol

• Previous 802.11 measurement research

2006/12/04 NTU/nslab 9

Operation of the 802.11 protocol

• MAC– a CSMA/CA variant that uses virtual carrier sense– support an RTS/CTS capability– protect multi-frame exchanges

• 802.11b – CCK modulation with coded rates up to 11 Mbps

• 802.11g – OFDM, coded up to 54 Mbps

• Legacy 802.11b radios are unable to decode the OFDM encoding of an 802.11g frame– can incorrectly sense the medium as idle

2006/12/04 NTU/nslab 10

802.11g protection mode

• 802.11g access points determine if they have any 802.11b stations as clients

• If so, they enable 802.11g protection mode– each 802.11g frame is preceded by a low-rate

CCK-coded CTS frame (CTS-to-self)– reserves the channel for the time needed to c

omplete the 802.11g transaction

2006/12/04 NTU/nslab 11

Previous 802.11 measurement research

• Small studies focused on low-level channel behavior between pairs of nodes

• Over larger environments– university campuses– industrial factories– corporate networks– conference and professional meetings

2006/12/04 NTU/nslab 12

Over larger environments

• Treat wireless networks as a black box

• Base their analyses on – wired distribution network traffic– polled SNMP management data from APs

• (O) what user behavior and network performance wireless LANs provide

• (X) why applications and users experience such behavior and performance

2006/12/04 NTU/nslab 13

More related work

• Passively capture and analyze link-level characteristics

• Yeo et al.– the first to explore the feasibility of using separate mo

nitors for passive wireless network measurement– use beacon frames to merge traces of a single flow o

bserved from three wireless monitors– demonstrate the utility of merging observations to imp

rove monitoring accuracy

2006/12/04 NTU/nslab 14

More related work

• Jardosh et al.– analyze the link-level behavior of traffic from a large I

ETF meeting– using three monitors capturing traffic on orthogonal ch

annels

• Rodrig et al. and Mahajan et al.– use five distributed wireless monitors to capture netw

ork events in a large conference venue– analyze various performance characteristics of the 80

2.11 MAC protocol

2006/12/04 NTU/nslab 15

Proposed approach

• Scale– over 150 monitors– four floors of a 150,000-square-foot building

• Performance– extensive spatial and channel coverage– extensive on-line monitoring

• Methodology– globally synchronizing events in time across subsets

of monitors as well as across channels• Analysis

– observe a large wireless network from a global perspective

2006/12/04 NTU/nslab 16

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 17

Data collection

• Environment

• Hardware

• Software

2006/12/04 NTU/nslab 18

Environment

2006/12/04 NTU/nslab 19

Environment

2006/12/04 NTU/nslab 20

Hardware

• each sensor pod consists of a pair of monitors set a meter apart

2006/12/04 NTU/nslab 21

Hardware

Monitors

2006/12/04 NTU/nslab 22

Hardware

Monitors

Pod

2006/12/04 NTU/nslab 23

Specifications

• Each monitor consists of – a modified Soekris Engineering net4826 syste

m board– a 266-MHz AMD Geode CPU– 128 MB of DRAM– 64 MB of Flash RAM– a 100-Mbps Ethernet interface– two Wistron CM9 miniPCI 802.11a/b/g interfac

es based on the Atheros 5004 chipset

2006/12/04 NTU/nslab 24

Specifications

• Each monitor receives wired connectivity and power through a port on an HP 2626-PWR switch (seven in total)

• Trace data from all radios is sent via NFS to a single 2.8-GHz Pentium server– hosting 2 GB of memory and 2 TB of storage– four 500-MB SATA disks in a RAID-0 configur

ation

2006/12/04 NTU/nslab 25

Software

• Each monitor runs a version of Pebble Linux– using the MadWifi driver to drive the Atheros-

based wireless interfaces

• Have made significant modifications to the driver – to support additional transparency to the physi

cal layer– to improve capture efficiency

2006/12/04 NTU/nslab 26

Driver modifications

• Standard madwifi driver only delivers valid 802.11 frames

• Proposed version captures all available physical layer events– including corrupted frames and physical errors

• Atheros hardware uses a 1s resolution clock to timestamp each packet as it is received

• Proposed driver slaves this timestamp facility to the clock of a single radio– thereby recording frames at both radios using the sam

e time reference

2006/12/04 NTU/nslab 27

Jigdump

• A specialized user-level application called jigdump manages data capture

• Each monitor executes two jigdump processes– one per radio– responsible for

• putting the wireless interface into monitor mode• pulling physical event records from the kernel• transferring this data via NFS to a central repository

2006/12/04 NTU/nslab 28

Jigdump

• Jigdump reads data records 64 KB at a time via a standard PF PACKET socket– compresses them using the LZO algorithm to minimiz

e storage and I/O overhead • the two bottlenecks on our monitor platform

– generates a metadata index record to facilitate subsequent accesses

• Data and metadata are written to separate files via NFS, creating a new file pair each hour

• In steady state, the NFS traffic across all 156 simultaneous feeds averages 2.10 MB/s

2006/12/04 NTU/nslab 29

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 30

Trace merging

• Combine traces from all the radios into a single coherent description – To construct a global viewpoint it is necessary

• Must satisfy three key requirements– Unification– Synchronization– Efficiency

2006/12/04 NTU/nslab 31

Trace merging

• Exploits the broadcast nature of wireless• In an indoor environment, propagation delay is effe

ctively instantaneous– less than 1 microsecond to cover 500 meters at 2.4 GHz– can treat the time at which a given frame is received by

multiple monitors as a simultaneous event for all potential interactions

– can use frames heard by multiple monitors as a common reference point to synchronize the clocks at each monitor and globally order subsequent events between traces

2006/12/04 NTU/nslab 32

Bootstrap synchronization

• Find reference points to synchronize the radios of a set of individual monitors

• Then synchronizes among sets until it establishes a single coordinated time standard

2006/12/04 NTU/nslab 33

Frame unification

• After bootstrap synchronization, Jigsaw processes all traces in time order

• Unifies duplicate frames, called instances, into a single data structure called a jframe.

2006/12/04 NTU/nslab 34

Frame unification

2006/12/04 NTU/nslab 35

Basic unification

• For each radio trace Jigsaw maintains an instance queue sorted in time order

• The simplest unification approach– linearly scan the head of all radio queues and– group the instances with the same timestamp

s and contents

2006/12/04 NTU/nslab 36

To minimize overhead

• Jigsaw instead populates a single priority queue sorted by time with the earliest instance from each trace

• To create a jframe, Jigsaw simply – pops this queue until the timestamp of the nex

t instance differs by a significant amount– groups the popped instances according to thei

r content

2006/12/04 NTU/nslab 37

Each radio's clock skews over time

2006/12/04 NTU/nslab 38

Current accuracy of our algorithm

• using a 10-ms search window

2006/12/04 NTU/nslab 39

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 40

Link and transport reconstruction

• In principle– this reconstruction is straightforward– Jigsaw provides a time-ordered list of all frames– each frame contains up to 200 bytes of payload

• MAC addresses, IP addresses and TCP port numbers.

• In practice– missing data and vantage point ambiguities complicat

e this reconstruction process– Jigsaw must use inference to help reconstruct these h

igher-layer descriptions

2006/12/04 NTU/nslab 41

Link-layer inference• Assemble individual jframes into transmission att

empts– Identifies each transmission attempt from a sender– a CTS-to-self packet– a subsequent DATA frame – the trailing ACK response

• Compose transmission attempts into complete frame exchanges– complete sets of transmission attempts (including retr

ansmissions) – that end in a link-layer frame being successfully delive

red or not

2006/12/04 NTU/nslab 42

Link-layer inference

2006/12/04 NTU/nslab 43

Transport inference

• Takes frame exchanges as input

• Reconstructs individual TCP flows based on the network and transport headers

• Then infer connection characteristics – e.g., RTT, RTO, fast retransmissions, segmen

t losses

2006/12/04 NTU/nslab 44

Two ambiguities

• Passive wireless context has two ambiguities that differ from the wired environment

• First– we may process frame exchanges in which it is u

nclear if the frame was actually delivered

• Second – existing analyses assume that monitors are lossle

ss

2006/12/04 NTU/nslab 45

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 46

Coverage

2006/12/04 NTU/nslab 47

Sensitivity of coverage to the number of sensor pods

2006/12/04 NTU/nslab 48

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 49

Analyses

• Trace summary

• Interference

• 802.11g protection mode

• TCP loss rate inference

2006/12/04 NTU/nslab 50

Trace summary

• Entire day of Tuesday, January 24, 2006

2006/12/04 NTU/nslab 51

Time series of network activity

2006/12/04 NTU/nslab 52

Interference

• Analyze the extent of transmission interference experienced by nodes in our trace

• I: event that interference causes a lost transmission from s to r

• L: event that the transmission from s to r was a background loss due to some other cause

• S: event that there is a simultaneous transmission from at least one other device i when s transmits to r

2006/12/04 NTU/nslab 53

Interference

• For a given (s, r) pair– n: the number of transmissions from s to r

– n0 n: the number of transmissions from s to r without a simultaneous transmission from another node

– : the number of n0 transmissions lost

– nx: the number of transmissions from s to r with a simultaneous transmission

– : the number of nx transmissions lost

2006/12/04 NTU/nslab 54

:

2006/12/04 NTU/nslab 55

Interference loss rate

2006/12/04 NTU/nslab 56

802.11g protection mode

• find that the protection policy by APs is overly conservative– potentially reducing performance for 802.11g

clients

2006/12/04 NTU/nslab 57

802.11g protection mode

2006/12/04 NTU/nslab 58

A more practical protection policy

• Would provide two benefits to clients in the network

• First– the 802.11g clients associated with overprotective AP

s could potentially improve their throughput substantially

• Second– reducing the use of CTS-to-self reduces the possibility

of exposed terminals in the network, • which could improve the performance of the network

2006/12/04 NTU/nslab 59

TCP loss rate inference

2006/12/04 NTU/nslab 60

Outline

• Introduction• Background and related work• Data collection• Trace merging• Link and transport reconstruction• Coverage• Analyses• Conclusion

2006/12/04 NTU/nslab 61

Conclusion

• Network research comes to understand the artifacts it has created slowly– by careful instrumentation, monitoring and an

alysis

• Production 802.11 wireless networks have so far escaped the level of detailed analysis experienced on the wired network– largely because of the difficulty in monitoring t

he wireless environment

2006/12/04 NTU/nslab 62

Jigsaw

• Unifies traces from multiple passive wireless monitors– to reconstruct a global view of network activity

in a production 802.11 network

• Used to – scalably synchronize traces– unify common frames– reconstruct the link- and transport-layer conve

rsations embedded in those frames

2006/12/04 NTU/nslab 63

Comments

• Strength– Real, large scale, detailed measurement of

802.11 network

• Weakness– Can do more analysis with so much detailed

data

• Relevance to our research– Maybe SY can build a similar kind of monitor

infrastructure for BL-Live

2006/12/04 NTU/nslab 64

Thank you very much for your attention!