1

TCP over Wireless Links

ECE 256Spring 2009

2

Today’s Discussions

Setting up the context (recap) MAC (radios, HTP, antennas, rate/power/CS control), routing

Motivating a Transport Layer The e2e argument The basics of flow control and congestion control From wireline to wireless

Peek into the large gamut of research in wireless TCP Snoop, ELN, Vegas, Reno, Paris, Hollywood (just kidding !!) Performance over wireless links

What’s hot now ?

3

Recall 1: PHY and MAC

MAC MAC

PHY PHY

• Spread Spectrum radios (DS and FH)• RTS/CTS and Carrier Sensing for Hidden Terminals• Directional antennas to reduce interference• Rate control to extract max capacity from available SINR• Power control for spatial reuse & energy savings – topology control• TDMA scheduling, multi-channel use, encryption security

… and many more

4

Recall 2: Network Layer

Routing

• The first view of the network

• Coping up with (uncontrolled) user mobility-Flooding the network reactively, or proactive updation

• Mobile IP, coping with handoffs, etc.

• Ad hoc routing – discovery, optimal metric, maintenance, caching• Secure routing – Routes bypassing malicious nodes

Routing

Routing

Routing

Routing

5

We have a route, now what ?

TCP

• Transport packets quickly and reliably over this network

• Network properties often unknown (or difficult to track)- Where is the congestion ? How much cross traffic ?- What is the bottleneck bandwidth ?- How much buffers at intermediate nodes ?

Motivation for end to end TCP

TCP

NETWORK

Data

Data

Data

6

The End to End Argument [Reed,Clark]

“The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible.”

In other words, as a middle man, don’t attempt to do it, if you cannot do it completely.

Let the end point handle it completely !!

Caveat: Middle-man involvment OK for performance optimization

Question is: Who is the end point ? TCP ? Application layer ? Human user ?… the debate goes on

7

Some transmission methods

Stop & Wait Pipelined

Go Back N Selective Repeat

8

Stop-and-wait operation

first packet bit transmitted, t = 0

sender receiver

RTT

last packet bit transmitted, t = L / R

first packet bit arriveslast packet bit arrives, send ACK

ACK arrives, send next packet, t = RTT + L / R

U sender

= .008

30.008 = 0.00027

microseconds

L / R

RTT + L / R =

9

Pipelined protocols

Pipelining: sender allows multiple, “in-flight”, yet-to-be-acknowledged pkts range of sequence numbers must be increased buffering at sender and/or receiver

Two generic forms of pipelined protocols: go-Back-N, selective repeat

10

Pipelining: increased utilization

first packet bit transmitted, t = 0

sender receiver

RTT

last bit transmitted, t = L / R

first packet bit arriveslast packet bit arrives, send ACK

ACK arrives, send next packet, t = RTT + L / R

last bit of 2nd packet arrives, send ACKlast bit of 3rd packet arrives, send ACK

U sender

= .024

30.008 = 0.0008

microseconds

3 * L / R

RTT + L / R =

Increase utilizationby a factor of 3!

11

Go-Back-N

Sender: k-bit seq # in pkt header “window” of up to N, consecutive unack’ed pkts allowed

ACK(n): ACKs all pkts up to, including seq # n - “cumulative ACK” may receive duplicate ACKs

timer for each in-flight pkt timeout(n): retransmit pkt n and all higher seq # pkts in window

12

GBN inaction

13

Selective Repeat

receiver individually acknowledges all correctly received pkts buffers pkts, as needed, for eventual in-order delivery to

upper layer sender only resends pkts for which ACK not received

sender timer for each unACKed pkt sender window

N consecutive seq #’s again limits seq #s of sent, unACKed pkts

14

Selective Request

Makes sense to transmit only the lost packets But this is true under what assumption ?

Can you say a case in which Go-BACK-N might be better

15

Selective repeat: sender, receiver windows

16

Selective repeat

data from above : if next available seq # in window,

send pkt

timeout(n): resend pkt n, restart timer

ACK(n) in [sendbase,sendbase+N]:

mark pkt n as received if n smallest unACKed pkt,

advance window base to next unACKed seq #

sender

pkt n in [rcvbase, rcvbase+N-1]

send ACK(n) out-of-order: buffer in-order: deliver (also

deliver buffered, in-order pkts), advance window to next not-yet-received pkt

pkt n in [rcvbase-N,rcvbase-1]

ACK(n)

otherwise: ignore

receiver

17

Selective repeat in action

18

Selective repeat: dilemma

Example: seq #’s: 0, 1, 2, 3 window size=3

receiver sees no difference in two scenarios!

incorrectly passes duplicate data as new in (a)

Q: what relationship between seq # size and window size?

What if pkts go out of orderWhat if pkts go out of order

19

TCP

20

TCP Congestion Control

Problem Definition How much data should I pump into the network to

ensure• Intermediate router queues not filling up• Fairness achieved among multiple TCP flows

Why is this problem difficult? TCP cannot have information about the network Only TCP receiver can give some feedbacks

21

The TCP Intuition

Pourwater

Collectwater

22

The Control Problem

Two main components in TCP Flow Control and Congestion Control

Flow Control If receiver’s bucket filling up, pour less water

Congestion Control Don’t pour too much if there are leaks in intermediate

pipes Regulate your flow based on how much is leaking out

• Aggressive pouring calls for retransmission of lost packets

• Conservative pouring lower e2e capacity

Challenge: At what rate(t) should you pour ?Why ?

23

The TCP Protocol (in a nutshell)

T transmits few packets, waits for ACK Called slow start

R acknowledges all packet till seq #i by ACK i (optimizations possible) ACK sent out only on receiving a packet Can be Duplicate ACK if expected packet not received

ACK reaches T indicator of more capacity T transmits larger burst of packets (self clocking) … so on Burst size increased until packet drops (i.e., DupACK)

When T gets DupACK or waits for longer than RTO Assumes congestion reduces burst size (congestion

window)

24

TCP Timeline

Host A

one segment

RTT

Host B

time

two segments

four segments

Think of a blindperson trying tostand up in a lowceiling room

Objective:Don’t bang yourhead, but standup quickly

Think of a blindperson trying tostand up in a lowceiling room

Objective:Don’t bang yourhead, but standup quickly

25

When waited for > RTO

0

5

10

15

20

25

0 3 6 9 12 15 20 22 25

Time (round trips)

Congestion window (segments)

ssthresh = 8 ssthresh = 10

cwnd = 20

After RTO timeout

26

The TCP Protocol (in a nutshell)

DupACK not necessarily indicator of congestion Can happen due to out of order (OOO) delivery of packets

If 3 OOO pkts, then CW need not be cut drastically The DupACK packet retransmitted Continue with same pace of transmission as before

(fast recovery)

R advertizes its receiver window in ACKs If filling up, T reduces congestion window

Why ?

27

Fast Recovery on 3 OOO DupACKs

0

2

4

6

8

10

0 2 4 6 8 10 12 14

Time (round trips)

Window size (segments)

Receiver’s advertized window

After fast recovery

28

TCP Round Trip Time and Timeout

EstimatedRTT = (1- )*EstimatedRTT + *SampleRTTEstimatedRTT = (1- )*EstimatedRTT + *SampleRTT

Exponential weighted moving average influence of past sample decreases exponentially fast typical value: = 0.125

29

Example RTT estimation:

RTT: gaia.cs.umass.edu to fantasia.eurecom.fr

100

150

200

250

300

350

1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106

time (seconnds)

RTT (milliseconds)

SampleRTT Estimated RTT

30

TCP Round Trip Time and Timeout

Setting the timeout EstimtedRTT plus “safety margin”

large variation in EstimatedRTT -> larger safety margin first estimate of how much SampleRTT deviates from EstimatedRTT:

TimeoutInterval = EstimatedRTT + 4*DevRTT

DevRTT = (1-)*DevRTT + *|SampleRTT-EstimatedRTT|

(typically, = 0.25)

DevRTT = (1-)*DevRTT + *|SampleRTT-EstimatedRTT|

(typically, = 0.25)

Then set timeout interval:

31

Several flavors of TCP: combines options / optimizations

Reno, Vegas, Eifel, Westwood …

Overall TCP has worked well – proven on the internet

Then why study it again for wireless networks ?

32

The TCP Intuition

Pourwater

Collectwater

33

Renewed Challenge

Key assumption in TCP A packet loss is indicative of network congestion Source needs to regulate flow by reducing CW

Assumption closely true for wired networks BER ~ 10 -6

With wireless, errors due to fading, fluctuations Need not reduce CW in response … But, TCP is e2e CANNOT see the network Thus, TCP cannot classify the cause of loss

CHALLENGE

34

The Problem Model

wireless

physical

link

network

transport

application

physical

link

network

transport

application

physical

link

network

transport

application

rxmt

TCP connection

Wireline

35

Impact of Misclassification

0.0E+00

5.0E+05

1.0E+06

1.5E+06

2.0E+06

0 10 20 30 40 50 60

Time (s)

Seq

uen

ce n

um

ber

(byte

s)

TCP Reno(280 Kbps)

Best possible TCP with no errors(1.30 Mbps)

2 MB wide-area TCP transfer over 2 Mbps WaveLAN

36

The Solution Space

Much research on TCP over wireless

Difficult to cover complete ground

We peek into some of the key ideas Link layer mechanisms Split connection approach TCP-Aware link layer TCP-Unaware approximation of TCP-aware link layer Explicit notification Receiver-based discrimination Sender-based discrimination

37

Link Layer Mechanisms

38

Link Layer Mechanisms

Forward error corrections Add redundancy in the packets to correct bit-errors TCP retransmissions can be alleviated

Link layer retransmissions MAC layer ACKnowledgments Overhead only when errors occur (unlike FEC)

Such mechanisms require no change in TCPIs that breaking e2e argument ??

39

Issues with Link Layer Mechanisms

Link layer cannot guarantee reliability Have to drop packets after some finite limit What is the retransmission limit (??)

Retransmission can take quite long Can be significant fraction of RTT TCP can timeout and retransmit the same packet again

• Increasing RTO can avoid this• But that impacts TCP’s recovery from congestion

Head of the line blocking Link layer has to keep retransmitting even if bad

channel Blocks other streams

Why?

40

Findings

Link layer retransmission good When channel errors infrequent When retransmit time << RTO When modifying TCP is not an acceptable solution

41

Split Connection Approach

42

1 TCP = ½ TCP + ½ (TCP or XXX)

wireless

physical

link

network

transport

application

physical

link

network

transport

application

physical

link

network

transport

application rxmt

Per-TCP connection state

TCP connection TCP connection

Base Station

43

Splitting Approaches

Indirect TCP [Baker97] Fixed host (FH) to base station (BS) uses TCP BS to mobile host (MH) uses another TCP connection

Selective Repeat [Yavatkar94] Over FH to BS: Use TCP Over BS to MH: Use selective repeat on top of UDP

No congestion control over wireless [Haas97] Also use less headers over wireless

• Header compression

44

Issues with Splitting

E2E totally broken 2 separate connections

BS maintains hard state for each connection What if MH disconnected from BS ? Huge buffer requirements at BS What if BS fails ?

Handoff between BS requires state transfer

What if Data and ACK travel on different routes ? BS will not see the ACK at all – splitting not feasible

Whats the Problem ?

45

TCP-Aware Link Layer

46

Snoop

Link layer at BS buffers un-acknowledged packets

Now, BS peeks into every returning TCP ACK from MH If DupACK

• Retransmits the necessary packet• Drops the DupACK

DupACK does not reach sender Prevents fast retransmit

47

Snoop : Example

FH MHBS

40 39 3738

3634

Example assumes delayed ack - every other packet ack’d

36

37

38

35 TCP statemaintained at

link layer

48

Snoop : Example

41 40 3839

3634

36

37

38

35 39

49

Snoop : Example

42 41 3940

36

Duplicate acks are not delayed

36

dupack

37

38

39

40

50

Snoop : Example

40

363636

Duplicate acks

4143 42

37

38

39

40

41

51

Snoop : Example

FH MHBS

41

3636

3744 43

36

37

38

39

40

41

42

Discarddupack

Dupack triggers retransmissionof packet 37 from base station

BS needs to be TCP-aware to

be able to interpret TCP headers

52

Snoop : Example

37

36

36

4245 44

36

37

38

39

40

41

42

43

36

53

Snoop : Example

42

36

36

4346 45

36

37

38

39

40

41

42

43

41

36

44

TCP sender does notfast retransmit

54

Snoop : Example

43

3636

4447 46

36

37

38

39

40

41

42

43

41

36

44

TCP sender does notfast retransmit

45

55

Snoop : Example

FH MHBS

44

3636

4548 47

36

42

43

41

36

44

45

43

46

56

Snoop [Balakrishnan95acm]

0

400000

800000

1200000

1600000

2000000

16K32K64K 128K256Kno error

1/error rate (in bytes)

bits/sec

base TCP

Snoop

2 Mbps Wireless link

57

Issues with Snoop

Link layer needs to be TCP aware Smelling cross layer Link layer needs to buffer and perform sliding window

Not useful when TCP headers encrypted

Not feasible when Data and ACK travel different routes

RTT estimates can still go up due to link layer retransmission Affects performance of Snoop

58

Wireless TCP

WTCP attempts to nullify RTT estimation problem When data packets are lost due to errors

Link layer includes own time stamp in ACK packet ACK packets that have BS time stamps indicate a

wireless loss RTT of these packets not considered for RTO calculation

• But then, what if wireless hop is also congested !!!!!!• Time stamping cannot take care of that

59

Quick look at other schemesTCP-unaware schemes

Explicit notificationReceiver-based

60

TCP-Unaware, ELN

Delayed DupACKs Receiver waits for sometime before sending DupACK If link retransmission solves problem

• Then TCP sender does not send redundant packet

Explicit Loss Notification (ELN) BS remembers only packet’s sequence numbers When DupACKs return through them, they check If packet was received by BS, then colors the DupACK Sender realizes that packet lost on wireless link

• Does not cut down CW, just retransmits that packet

61

Receiver-Based TCP

Receiver estimates cause of packet loss If estimate == wireless loss, sets ELN bit

FH MHBS1012 11

FH MHBS

11

1012T

Congestion loss

FH MHBS

101112Error loss

2 T

When is thisMechanism a Big problem ?

62

Closing Thoughts

Reliable and in-order packet delivery important TCP aims to support these features Implements congestion control and flow control

TCP widely tuned for wireline networks Proven to be efficient on the internet

When network periphery has wireless “last mile” TCP exhibits myriad problems Mainly because of

“misclassification between congestion and channel errors”

Several solution approaches but many open problems

63

Questions ?

64

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

65

Marathon Presentations

7 candidates

1 Slot on Apr 22 3 slots on Apr 24 3 more candidates: We need to decide on a time.

66

What’s Hot Now ??

TCP over wireless multihop (mesh) Each hop has contention-based MAC

• Unpredictable delays and congestion Fairness between TCP e2e flows a very challenging

problem Mobility can significantly affect TCP(Very difficult set of open problems)

More fundamental: Is TCP the way to go for wireless Strong ongoing debate in community

Useful queuing solutions in ad hoc networks Neighborhood RED solution

… and many many more …

67

If all that did not make sense, or too heavy for “total recall”, then here is a visual example

Thanks to Nitin Vaidya for tutorial slides(Highly recommended for those interested in TCP)

68

40 39 3738

3533

Cumulative Acknowledgements

A new cumulative acknowledgement is generated only on receipt of a new in-sequence packet

41 40 3839

35 37

3634

3634

i data acki

69

Delayed Acknowledgements

An ack is delayed until another packet is received, or delayed ack timer expires (200 ms typical)

Reduces ack traffic

40 39 3738

3533

41 40 3839

35 37

New ack not producedon receipt of packet 36,

but on receipt of 37

70

Duplicate Acknowledgements

A dupack is generated whenever an out-of-order segment arrives at the receiver

40 39 3738

3634

42 41 3940

36 36

Dupack

(Above example assumes delayed acks)On receipt of 38

71

Duplicate Acknowledgements

Duplicate acks are not delayed Duplicate acks may be generated when

a packet is lost, or a packet is delivered out-of-order (OOO)

40 39 3837

3634

41 40 3739

36 36

DupackOn receipt of 38

72

Number of dupacks depends on how much OOO a packet is

40 39 3837

3634

41 40 3739

36 36

Dupack

42 41 3940

36 36 38

New Ack

New AckNew Ack

New Ack

34

New Ack

DupackNew Ack