Game Theory and the Internet

Flow and Congestion Control

2

“Differential QoS and Pricing in Networks: where flow-control meets game theory.”

Peter Key and Derek McAuley, 1999

3

Notation

r R user / network routej J network resource (eg link)

xr resources demanded by user r (eg bandwidth per unit time)

A = (Ajr)1 if resource j is consumed by user r0 otherwise(assume fixed routing)

Ax is the (offered) load to the resource vector

4

Notation cont.

Ur(xr) utility function for user r, relating bandwitdth to worth

Cj(y) cost of loading y on resource j

Assume Ur(x) is concave:

f((x1 + x2)/2) (f(x1) + f(x2))/2

Assume Cj(y) is increasing, differentiable.

5

“Social Optimum”

Maximize

U(x)=r Ur(xr) - j Cj(r Ajrxr)

over x 0.

6

If Cj(y) are convex functions, the optimum for nonzero xr satisfies

Ur’(xr)=j jAjr

where the j are associated with the shadow prices:

j = pj (j Ajr xr)

with

pj(y) = dCj(y)/dy

(Shadow price: marginal increment in cost at the resource for a marginal increment in load.)

7

User optimum

Assume users are charged ar per amount of data sent.

Maximize:

Ur(xr) – arxr

over xr 0

8

User optimum cont.Solution:

x*, where Ur’(x*) = ar, if Ur’(0) > ar

xr = 0, otherwiseThe user accepts any amount of bandwidth offered

up to x*, if the prices are “rational”, i.e. Ur’(0) > ar

User optimum coincides with social optimum if the prices are “right”:

ar = jr j

9

TCP congestion control (in brief)

Congestion is controlled by changing the rate of sending packets.

When a resource in the network becomes congested, some packets will be lost.

Loss of a packet is interpreted by the destination as an indication of congestion.

The source is notified to slow down transmission.The transmission rate is gradually increased until a

new notification of congestion.This allows the utilization of available bandwidth.

10

Framework

Users generate load x(t) and receive feedback signals f(t), which are related to shadow prices.

For example certain packets receive “marks”.

A packet is marked if and only if it contributed to a loss event, i.e. the removal of the packet arriving at some resource would result in one less packet loss at that resource.

So, mark all packets arriving in an overloaded resource.

11

More specific…

Consider single unbuffered resource, capacity C.

The load Y is a random variable with mean y, following Poisson or Gaussian distribution.

The expected loss is E[(Y - C)+]

The shadow price can be shown to be equal to the probability of resource saturation Psat = P{Y C}

12

So marking scheme for unbuffered resource:

Mark all the packets that contribute to the load when the resource is overloaded, otherwise mark none.

The probability of loss: Ploss = E[(Y – C)+]/E[Y]

Typically Ploss << Psat, so many more packets are marked than lost.

13

Flow control

Optimality depends on the utility functions of the users, but these functions are unknown to the network.

So: Let the network determine the prices and let the

users place demands on the network, which the network will try to meet.

The users can behave as they wish, possibly performing some user optimization.

14

Redefinition of flow control algorithm

Implement a means to feed back information to the user: mark a packet

Every resource used for the transmission of the packet has the ability to mark it.

But the probability of multiple markings is very small, so one bit can be used to approximate a “mark”.

This bit can be incorporated in most existing protocols easily.

15

Algorithms for flow control

“Willingness to pay” scheme:

xt+1 = xt + k(w – ft)

w represents “willingness to pay”,

k affects the rate of convergence.

“TCP-like” scheme:

xt+1 = xt + k(w – ft xt/2)

16

Experimentation

Run a distributed emulation of a network, where users are free to invent their own algorithms or behave as they wish.

It is a distributed multi-user game, where the users try to optimize their own objective.

17

Proposed framework

Users play against a network representing the resource system:

• The game consists of a distributed set of users and a Network, emulated by a PC

• A protocol interfaces between the users and the Network

• Users can interact (connect) across the Internet• Users can interact in real time or through

programs• Game time can be slowed down or speeded up

18

• Users select a destination and generate load (packets), which are sent to the Network.

• The Network decides which resources to allocate to the user and sends back a set of marks, depending on the state of the resources.

• The Network also sends to the user signals about the start/end of the game and its relative speed compared to real time.

19

Protocol for a distributed game<packet> : <pkt>=<destination><token><size>

<token> is generated by the user and used to detect loss. <destination> is used to identify a route in the Network

<ACK> : <ACK>=<token><mark><token> can be void or corrupted. <mark> gives the number of marks in the packet

<time> : <time>=<sec>sends heartbeats back to the user giving time in the game world

20

Typical sequence of eventsUser requests to join the gameReceives acknowledgement and relative speedReceives notice of start of gameRepeat

Generate tokenCalculate load (packet size)Send <packet>=<destination><token><size>Receive <ACK>=<token><mark>Receive <time>=<sec>

Until End of gameUser ends session

21

Typical objectives

We consider optimality from the perspective of the user, not the network.

Reminder (and some slight changes):

User r generates load xt = x(t) at time t.

Number of marked packets is ft = f(t).

Cost factor a, so cost incurred is: aft.

22

Typical objectives cont.• Control cost: Keep the cost constant and equal

to w (“willingness to pay”). L is some loss function

minimize L(w – aft)

• Throughput: Maximize throughput minus the cost

maximize Tt=1(xt-aft)/T

• Constraint throughput: Same as above given a cost constraint

maximize Tt=1(xt-aft)/T, such that T

t=1aft

23

Typical objectives cont.• Maximise net utility: Given a utility function U

maximize t(U(xt) - aft)

• File transfer: Transfer an amount of data F at minimum cost

minimize taft, such that txt = F

• Fixed time file transfer: Same as above given a time constraint T

minimize Tt=1aft, such that T

t=1xt = F

24

Conclusions

It is possible to provide differential quality of service to users by means of a simple pricing scheme, where the network sends back the correct congestion prices to the user and the users choose how to react.

Use the presented framework and play the defined distributed game, to assess the effectiveness and stability of various flow control schemes.

25

A modest proposal for preventing Internet congestion

Andrew Odlyzko, 1997

26

The problem of congestion

There is a demand for improved Quality of Service on the Internet, especially for reducing congestion.

Proposed schemes are usually complicated and have high cost in development and operation.

The basic problem is one of allocating a limited resource.

A solution will have to involve some pricing mechanism.

27

The Paris Metro (or Subway)

Until some years ago, the Paris Metro had two different classes of cars.

1st and 2nd class cars were identical in the number and quality of seats.

1st class cars were more expensive and therefore less congested. Used by people who wanted a seat, silence etc.

The system was self-regulated: if 1st class cars became very popular, some people would decide they were not worth the extra cost and move to 2nd class, thereby reducing congestion in 1st class.

28

PMP

PMP = Paris Metro Protocol

Basic idea: partition the network into several logically separate networks.

Each network has a fraction of the capacity of the entire network.

All networks route packets using existing protocols on a “best effort” basis.

Every network charges a different price for the same transfer.

29

The pricing mechanism is expected to affect user

demand and thus modify the traffic loads.

Networks with higher prices will be less congested, offering better QoS.

PMP will not provide QoS guarantees, only expected levels of service.

Not necessarily bad. QoS demands can be complicated and widely varying, impossible to satisfy all of them.

PMP will offer few classes of service satisfying the majority of applications.

30

PMP advantagesApproaches that don’t use any network scheme

and just increase capacity fail. With no pricing mechanism all packets travel at the highest priority and traffic grows to exhaust all available capacity.

Policies trying to control bandwidth and provide QoS guarantees require the coordination of network nodes along the transmission path, causing delays and “reinventing telephone technologies”.

31

Details

The number of subnetworks in PMP should be small (<5). This minimizes losses from not aggregating all the traffic and fits consumer preferences for simple schemes.

Every subnetwork need not be assigned a fixed fraction of the total capacity. But this assignment, as well as the prices, should stay constant for extended periods of time, to allow usage patterns to stabilize.

32

Problems and Solutions

To provide a higher QoS than the current Internet, the low-traffic networks will have to be lightly loaded. Will the capacity utilization have to be so low that it proves to be uneconomic?

We don’t know. There is not enough information about the operation and performance of the Internet. The user requirements are not precisely defined.

33

Problems and Solutions

How to set the network prices?

Use the variations in traffic patterns with the time of day. (Also customer surveys, complaints, etc.)

34

Problems and solutions cont.

The average performance of the networks can be unstable.

For example, an extended congestion period on the lowest-priced channel might lead some users to decide to pay extra and send their packets to the higher-priced networks.

Degradation of service on one channel will thus propagate to other channels.

35

Problems and solutions cont.

So, how to keep the average performance of the networks stable?

Segregate the different types of services on different networks. E.g. introduce artificial packet delays in the lowest-priced network, making it unusable for certain applications. (“damaged goods” approach)

This can be used to induce a more even load on the separate networks.

36

PMP implementation

Use current protocols:IPv4 packets have an unused 3 bit priority field.Protocols without priority fields can operate at the lowest cost (and priority) network.

Set the cost of the lowest-priority network to zero (at least initially).

Use software to handle all traffic on a user’s machine according to user set rules, until applications are rewritten to take the multiple network choices into account.

37

PMP implementation cont.

Also the service providers will have to:

Change the router software to maintain separate queues for the different channels.

Install software to count the packets sent by each user.

Special provisions needed for multicasting.

38

The obstacle of the users and how to overcome it.

[Disclaimer: The following proposals represent only the opinions of the author of the paper.]

The need for usage-sensitive pricing appears irresistible. It has impeccable economic logic and practical evidence behind it.

But it collides with the user’s preference for flat-rate pricing.

39

Flat rates

Flat rate pricing can cause significant problems.

Example: America Online

When it switched to flat-rate pricing, usage increased considerably.

Users had trouble getting a new connection and so left their connections open even when not using them.

40

Flat rates cont.

Perverse incentives exist on the Internet to increase data transfers[Note: The paper is from 1997]

To improve performance in Web surfing users use programs that prefetch linked pages when the connection is idle. With flat-rate pricing there is an incentive for the users to use them indiscriminately, even for minor benefits.

41

Flat rates cont.

The survival of the Internet owes much to altruism and ignorance.

TCP slows down transmissions when congestion occurs.

Rogue versions of TCP can be used to speed up transmission of packets when delays or losses occur. No extra cost is incurred to their users under flat-rate pricing. Their extensive use can cause the network to collapse.

42

So why are flat rates used?• Users know how much they will have to

pay.

• Consumers overestimate their actual usage of a service.

• Hassle factor: consumers don’t have to worry if every use of the service is worth its money.

• No need for traffic measurement and charging infrastructure.

43

How to introduce pricing without user complaints.

Users have accepted usage-sensitive rates in a variety of services (e.g. long distance calls)

It should be possible to persuade users to accept usage-sensitive pricing.

PMP should make the transition easier that with most other schemes, since the lowest-priced channel could be offered initially at zero cost per packet, thus behaving just like today’s internet.

44

PMP offers a simple and easily understood

pricing plan, which fits consumer desires.It does not offer service guarantees. No

problem: most purchases are made on the basis of expectations (magazine subscriptions, clothes, meals etc.) Consumers are willing to accept occasional deviations from the expected quality of service, as long as the average quality is predictable enough.

45

Auctions and consumersConsumer and business behavior can be

economically irrational.Free enterprise companies prefer the socialist

method of rationing by queue to that of rationing by price.

Laws against ticket scalping are common and widely supported. But to most economists scalpers fulfill a socially useful role: they provide tickets to those that are willing to pay the most for them. Why don’t theaters and sports arenas adjust ticket prices to clear the market and appropriate some of the gain the scalpers obtain?

46

Auctions and consumers cont.

It seems that the cause lies in the consumers’ irrationall economic behavior.

The challenge is to design pricing schemes that approach the efficiency of auction mechanism, but respect the consumer aversion to the auction.

It is important to convince consumers of the fairness of pricing schemes.

47

Other pricing schemes

• Charge packets when they contribute to congestion.– Complicated implementation

• Use a set of service classes and priorities– Substantial overhead– Low priority classes can be denied any

bandwidth if traffic from higher priority classes exceeds some threshold.

48

Gains from network segmentation

We will examine simple models of the gains obtained by logically separate networks working at different utilization levels.

49

Two types of demand for data tranfer.Type A users are insensitive to delays (e.g. bulk

file transfers)When the price per byte is x, type A users wish to

send ax-1e-x

bytes (per hour).

They generate network revenues:ae-x

50

The cost of operating a network that carries w

bytes is cw3/4

for some constant c > 0.

This is a conservative assumption, since it corresponds to less than a 16% reduction in costs when the network doubles in size (23/4 = 1.681…). In practice the reductions are higher, corresponding to even smaller exponents.

51

If there are only type A users, the cost of the

network will equal the revenues:ae-x = c(ax-1e-x)3/4

x3e-x = a-1c4

The unique maximum of x3e-x occurs at x = 3 and equals 1.344…

For c4 > 27ae-3 there is no price x that will recover costs.

For c4 < 27ae-3 there are two solutions for x and we prefer the smaller one xA, that corresponds to higher traffic and revenue.

52

Type B users use a network only when its

utilization rate is at most half of that acceptable to type A users.

This assumption is conservative because even small reductions in network loads produce substantial improvements in service.

When the price per byte is x, type B users wish to send

bx-1e-x bytes (per hour).

They generate network revenues:be-x

53

Constructing a separate network for type B users

will cost

c(2bx-1e-x)3/4

The 2 is because of the lower utilization rate.

A price x that will equalize revenue and costs is found by solving:

x3e-x = 8b-1c4

and exists when 27b 8c4e3. Let xB denote the minimal solution (if more than one exists).

54

Single network for bothCase 1: Single network with single price for both

types of users.

At price x would have revenue

(a+b)e-x

and will cost

c(2(a+b)x-1e-x)3/4

because its average utilization would have to be half of a network meant for type A users alone.

55

The price x that equalizes revenue and costs is found by solving:

x3e-x = 8(a+b)-1c4

and exists when 27(a+b) 8c4e3. Let xAB denote the minimal solution.

If b > 7a (demand from type B users is larger), type A users benefit from lower prices: xAB < xA

If b << a, then xAB > xA and type A users pay more (and also get better service but they don’t need it)

Type B users benefit always from having type A users on their network, because prices are always reduced.

56

Separate logical networks

Case 2: Same physical network, logically separated into two for each type of user.

Price for type A users is y, for type B is z.

Revenue:

ae-y + be-z

Cost of the network:

c(ay-1e-y + 2bz-1e-z)-3/4

57

y and z must satisfy:

ae-y + be-z = c(ay-1e-y + 2bz-1e-z)-3/4

Great freedom of choices.

If y or z we reduce to networks exclusively used by type A and type B users respectively.

58

Example 1

a = b = 3, c = 1

xA = 0.952

xB does not exist

xAB = 2.784

y = 0.9z = 1.338

Type A users gain about 5.5%.Type B gain in that they are offered a service.

network traffic revenue

A only 1.215 1.157

B only - -

A + B on single network

0.133 0.37

A + B on logically separate networks

1.942 2.006

59

Example 2

a = 20, b = 10, c = 1

xA = 0.424

xB = 1.563

xAB = 0.856

y = 0.42z = 0.606

Type A users experience once more only a slight gain. Type B users pay a lot less.

network traffic revenue

A only 30.829 13.083

B only 1.34 2.095

A + B on single network

14.88 12.74

A + B on logically separate networks

40.27 18.59

60

Example 3a = 10, b = 20, c = 1

xA = 0.559

xB = 1.043

xAB = 0.856

y = 0.53

z = 0.693

The network provider is almost indifferent between having two separate networks and one giving all traffic the quality demanded by type B users.

network traffic revenue

A only 10.22 5.71

B only 6.75 7.04

A + B on single network

14.88 12.74

A + B on logically separate networks

25.51 15.88

61

In all the examples, gains to type A users are small.

This might explain why there has not been much pressure from users of the Internet (whose application are designed to work well in the presence of congestion) for higher quality of service.

In the examples, a and b are comparable, corresponding to comparable traffic from type A and type B users.

This might not be true now, but will most probably be the case, if the quality of service on the Internet is improved.