Internet congestion control

Damon Wischik, UCL

DJW

200

6 2

/24 What is queueing theory for?

• Given the arrival process, what is

• We often seek limit theorems to describe the arrival process, and to approximate this probability

• Typical motivation: “this lets us choose buffer size etc. to ensure that queue overflow is rare”

• This does not make sense for Internet queues, since TCP (which governs most Internet traffic) deliberately causes the queue to overflow

arrival processqueue Example: for a queue with N independent

flows, each self-similar with Hurst parameter H, then for some , >0

DJW

200

6 3

/24 Some Internet history

• 1974: First draft of TCP/IP[“A protocol for packet network interconnection”, Vint Cerf and Robert Kahn]

• 1983: ARPANET switches on TCP/IP

• 1986: Congestion collapse• 1988: Congestion control for TCP

[“Congestion avoidance and control”, Van Jacobson]

DJW

200

6 4

/24 TCP transmission control protocol

• Internet congestion is controlled by the end-systems• The TCP algorithm, running on the server, decides how fast to send data

– It sends data packets, and waits for ACKnowledgement packets

– It sets a target window size wt

the number of packets that have been sent but not yet acknowledged (here 5+3)

– The average transmission rate is xt ≈ wt/RTT where RTT is the round trip time

– It adapts wt based on the ACKs it receives

UserServer(TCP)

data

acknowledgements

DJW

200

6 5

/24

if (seqno > _last_acked) {

if (!_in_fast_recovery) {

_last_acked = seqno;

_dupacks = 0;

inflate_window();

send_packets(now);

_last_sent_time = now;

return;

}

if (seqno < _recover) {

uint32_t new_data = seqno - _last_acked;

_last_acked = seqno;

if (new_data < _cwnd) _cwnd -= new_data; else _cwnd=0;

_cwnd += _mss;

retransmit_packet(now);

send_packets(now);

return;

}

uint32_t flightsize = _highest_sent - seqno;

_cwnd = min(_ssthresh, flightsize + _mss);

_last_acked = seqno;

_dupacks = 0;

_in_fast_recovery = false;

send_packets(now);

return;

}

if (_in_fast_recovery) {

_cwnd += _mss;

send_packets(now);

return;

}

_dupacks++;

if (_dupacks!=3) {

send_packets(now);

return;

}

_ssthresh = max(_cwnd/2, (uint32_t)(2 * _mss));

retransmit_packet(now);

_cwnd = _ssthresh + 3 * _mss;

_in_fast_recovery = true;

_recover = _highest_sent;

}

TCPtr

ansm

issi

on r

ate

[0–1

00 k

B/s

ec]

time [0–8 sec]

DJW

200

6 6

/24 How TCP shares capacity

individual flow rates

aggregate flow rate

time

available capacity

DJW

200

6 7

/24 How to describe a network

Some networks admit models at three different levels:

• Microscopic rules of behaviour for the individual elements– e.g. rules of motion of electrons

• Macroscopic laws that describe the behaviour of aggregates– e.g. Kirchhoff’s law, Ohm’s law

• Teleological principles that describe the operating point to which the system settles down– e.g. Thomson’s principle

DJW

200

6 8

/24 TCP model I: the sawtooth

• Consider a single link, and a single flow with round trip time RTT

• When the link is not congested, transmission rate xt increases by 1/RTT i2 pkt/sec every sec

• When the link becomes congested, transmission rate is cut by 50%

queue becomes full and packets are dropped

TCP realizes there is congestion, and cuts its rate

transmissionrate x(t)

queuesize

time

***

service capacity

buffer size

DJW

200

6 9

/24 TCP model II: billiards

• Consider a single link, and many flows with round trip times RTT(1), RTT(2),...

• When the link is not saturated, flow r increases its transmission rate xt(r) by 1/RTT(r) pkt/sec2

• When the link becomes saturated, some flows experience packet drops and they cut their rates

• Assume some probabilistic model that says which flows experience drops

[Baccelli+Hong 2002]

+

+

=* * * * * **

DJW

200

6 10

/24 TCP model III: fluid

• Individual TCP flows follow a sawtooth

++

=

++

=

individual flow rates

aggregate flow rate

time

DJW

200

6 11

/24 TCP model III: fluid

• Individual TCP flows follow a sawtooth• ... but many TCP flows added together are smoother

– the average rate xt varies according to a time-delayed differential equation

– eqn involves pt, the packet drop probability experienced by packets sent at time t[Misra+Gong+Towsley 2000, Baccelli+McDonald+Reynier 2002, after Kelly et al. 1998]

++

=

++

=

individual flow rates

aggregate flow rate

time

dxt

dt=

1RTT 2

¡ pt ¡ RTT xt ¡ RTT

xt

2

DJW

200

6 12

/24 TCP model III: fluid

• There are three possible limit models for describing the behaviour of the queue and calculating the drop probability

• TCP selects a limiting model, depending on the size of the buffer

N TCP flows

Service rate NCBuffer size B(N)

Round trip time RTT

DJW

200

6 13

/24 Three buffer-sizing regimes

• These three regimes are qualitatively different; they are described by different dynamical systems, operating over different timescales– The relationship between buffer size and timescales was first observed by

[Deb+Srikant, 2004]

queuesize

[0–B pkt]

dropprobability

[0–.3]

trafficintensity

[0–1]

queue size[(B-25)–B pkt]

time [0–5s]

time [50ms]

small buffersB(N)=B

medium buffersB(N)=B√N

large buffersB(N)=BN

DJW

200

6 14

/24 Instability / synchronization

• Sometimes the dynamical system for the network is unstable, and the queue size flip-flops

– This means there are short periods of concentrated packet loss

– This might manifest itself as a burst of static in an Internet telephone call

– In fact, this corresponds to the billiards model for TCP

• Engineers can use the dynamical system description of TCP to help them design the network so that it is stable

queuesize

[0–619pkt]

TCPwindow

sizes[0-25pkt]

time [0–5s]

DJW

200

6 15

/24 Equilibrium analysis

• A well-designed network should be stable, i.e. it should reach some equilibrium state

• How can we characterize the equilibrium state?

DJW

200

6 16

/24 Teleological description

• The equilibrium state is the solution to the optimization problem

x

U(x

)

N TCP flows;flow r has rate xr Service rate C

Mediumbuffer size

Round trip time RTT

DJW

200

6 17

/24 Teleological description

• The equilibrium state is the solution to the optimization problem

x

U(x

)

N TCP flows;flow r has rate xr Service rate C

Buffer sizemedium

Round trip time RTT

little extra value attached to high-throughput flows

little extra value attached to high-throughput flows

severe penalty for allocating too little throughput

severe penalty for allocating too little throughput

DJW

200

6 18

/24 Teleological description

• The equilibrium state is the solution to the optimization problem

N TCP flows;flow r has rate xr Service rate C

Buffer sizemedium

Round trip time RTT

x

U(x

)

flows with large RTT are satisfied with just a little

throughput

flows with large RTT are satisfied with just a little

throughput

flows with small RTT demand

higher throughput

flows with small RTT demand

higher throughput

DJW

200

6 19

/24 Teleological description

• The equilibrium state is the solution to the optimization problem

N TCP flows;flow r has rate xr Service rate C

Buffer sizemedium

Round trip time RTT

x

U(x

)

flows with large RTT are satisfied with just a little

throughput

flows with large RTT are satisfied with just a little

throughput

flows with small RTT demand

higher throughput

flows with small RTT demand

higher throughput

We should thus set up caches very

close to the user, even if that means putting them in a congested part of

the network

We should thus set up caches very

close to the user, even if that means putting them in a congested part of

the network

DJW

200

6 20

/24 Teleological description

• The equilibrium state is the solution to the optimization problem

x

U(x

)

N TCP flows;flow r has rate xr Service rate C

Buffer sizemedium

Round trip time RTT

there is no penalty until the

link is overloaded, i.e.

y>C

there is no penalty until the

link is overloaded, i.e.

y>C

The network seeks to be overloaded,

leading to poor quality for Internet telephone

calls

The network seeks to be overloaded,

leading to poor quality for Internet telephone

calls

DJW

200

6 21

/24 Maths conclusion

• TCP chooses its own limit regime

• TCP, through its adaptive feedback mechanism, induces a fluid limit over the timescale of a round trip time

• The queue dynamics change fluidly over the timescale of a round trip time, and stochastically over much shorter timescales

• The choice of buffer size determines which of three queueing models is relevant

DJW

200

6 22

/24 Design conclusion

• The TCP code admits description at two higher levels– a dynamical system model at the macroscopic level– a welfare optimization problem at the teleological level

• This gives us insight into how to extend TCP– make sure the dynamical system is stable– choose a more desirable optimization problem

• These insights have led to new ideas about routing, load balancing, choice of buffer size in core routers, quality of service, high-speed TCP