Raj Jain is now at [email protected], ...jain/papers/ftp/mpt2mpt.pdffairness. Multipt-to-pt oin oin ABR...

Fair Flow Control for ATM-ABR Multipoint

Connections�

Sonia Fahmy, Raj Jain, Rohit Goyal, and Bobby Vandalore

Purdue University

Department of Computer Sciences

E-mail: [email protected]

Abstract: Multipoint-to-multipoint communication can be implemented by combining the point-

to-multipoint and multipoint-to-point connection operations. In a multipoint-to-point connection,

multiple sources send data to the same destination on a shared tree. Tra�c from multiple branches

is merged into a single stream after every merge point. It is impossible for the network to determine

any source-speci�c characteristics since all sources in the multipoint connection may use the same

connection identi�ers. The challenge is to develop a fair rate allocation algorithm without per-source

operations, as these are no longer equivalent to per-connection or per- ow operations.

We give fairness de�nitions for multipoint connections, and we design and simulate an O(1) fair ATM-

ABR rate allocation scheme for point-to-point and multipoint connections. Simulation results show

that the algorithm performs well and exhibits desirable properties. We discuss the main modi�cations

necessary for any ATM-ABR rate allocation scheme to accommodate multiple sources.

1 Introduction

Multipoint communication is the exchange of information among multiple senders and multiple re-

ceivers. Multipoint support in Asynchronous Transfer Mode (ATM) networks is essential for e�cient

duplication and synchronization of data. Examples of multipoint applications include audio and

video conferencing, and server and replicated database synchronization (see �gure 1). Multipoint-

to-point connections are especially important for overlaying Internet (IP) networks and simplifying

end systems and edge devices [15]. In multipoint-to-point connections, only one connection needs to

be set up even if there are multiple data sources.

�This research was sponsored in part by Rome Laboratory/C3BC Contract # F30602-96-C-0156. This paper and

all our papers and contributions are available through http://www.cis.ohio-state.edu/~ jain/

1Computer Communications, Vol. 25, No. 8, pp. 741-755, May 15, 2002

Raj Jain is now at [email protected], http://www.cse.wustl.edu/~jain

Figure1:ATM

multipointcommunication

Ane�cientand exibleATM

multipointserviceisakeyfactortothesuccessofATM

networks.

SeveralissuesneedtobeaddressedintheATM

multipointservicede�nition,suchasrouting,sig-

naling,andtra�cmanagement.Inthispaper,wefocusontra�cmanagementissuesinthecaseof

multiplesources.Speci�cally,wetacklethede�nitionoffairness,andthecongestionandfeedback

controlproblem

formultipoint-to-pointconnections.Theproblem

ofconsolidatingcontrolcellsis

nottackledinthispaper(see[4]forananalysisofthesolutionstofeedbackconsolidation).

ATM

networkscurrentlyo�ertwoservicecategoriesfordatatra�c:theavailablebitrate(ABR)

andtheunspeci�edbitrate(UBR)services.Capacityleftoverbyreal-timetra�cisfairlydivided

amongtheactiveABRsourcesandindicatedtothesourcesthroughclosed-loopfeedbackcontrol[5].

Themostcommonlyadoptedfairnessde�nitionismax-minfairness[7].Intuitively,thismeansthat

allsourcesbottleneckedatthesamenodeareallocatedequalrates(orproportionalratesusing

weights).Thisde�nitionwasdevelopedforpoint-to-pointconnections,andinthispaper,weextend

itformultipointconnections,anddiscussthedevelopmentofadistributedalgorithm

toachieve

fairness.

Multipoint-to-pointABRconnectionsrequirefeedbacktobereturnedtotheappropriatesourcesat

theappropriatetimes.Thebandwidthrequirementsforavirtualconnection(VC)afteramerge

pointisthesum

ofthebandwidthsusedbyallsourceswhosetra�cismerged(see�gure2).This

isbecausetheaggregatedatarateafteramergepointisthesum

ofallincomingdataratestothe

mergepoint[8].Consolidatingcontrolcellsintheforwarddirectionisnotnecessarysincetheratio

ofcontrolcellstodatacellsaftermergingremainsthesame.

Wehavede�nedseveraltypesoffairnessformultipoint-to-pointVCsimplementedassharedtrees[3].

Amongthese,webelievethatweightedsource-basedfairnessisthemostpreferredbecauseitisa

simpleandlogicalextensionofpoint-to-pointfairnessde�nitions.Tocomputesource-basedfairallo-

cations,asingleN-to-oneconnectionistreatedasN

one-to-oneconnections(intermsofbandwidth

allocation),regardlessofwhichVCeachsourcebelongsto.

2

Figure2:Multipoint-to-pointconnections

Asource-basedfairalgorithm

mustgivethesame(orproportional)allocationtoallsourcesbottle-

neckedonthesamelink.Source-basedfairnessinsomeswitchimplementationsposesdi�culties,

sincesourcesinthesameVCcannotbedistinguished(theyhavethesameconnectionidenti�er).

Thechallengesforrateallocationalgorithmsinthiscaseincludeavoidingper-sourceaccountingand

avoidingestimatingthenumberofactivesources.Thishastobedonewithoutadverselya�ecting

thetransientresponseorincreasingtherateoscillations.

Theremainderofthispaperisorganizedasfollows.First,wegivesomede�nitionsanddiscussthe

VCmergetechniqueforavoidingcellinterleavinginmultipointconnections.Then,wesummarize

relatedworkonmultipoint-to-pointalgorithms.Wede�nefairnessusinganexample.Insection4

wedeveloptherateallocationandmergepointalgorithmsformultipointconnections,andexamine

theirfeatures.Weanalyzetheperformanceofthealgorithminsection5,andconcludewithasetof

recommendationsforrateallocationschemestosupportmultiplesources.

2

PreliminariesandRelatedWork

Inthissection,wegivesomebackgroundontheproblem

ofABR

multipoint ow

control.We

distinguishconnections,sourcesand ows,discussVCmergingandABR owcontrol,anddiscuss

previousworkonmultipoint-to-pointalgorithms.

2.1

Connections,SourcesandFlows

De�nition:Acomponentc jissaidtobedownstreamofanothercomponentc iinacertainconnection

ifc jisonthepathfromc itothedestination.Inthiscase,c iissaidtobeupstreamofc j.

2

Figure3showsacon�gurationwithtwovirtualconnections(VCs).OneoftheVCsisapoint-

to-pointVC,whiletheotherisamultipoint-to-pointVC.Thesourcesinthemultipoint-to-point

3

Figure3:SourceversusVCversus ow

VCareindicatedbydark-coloredcircles,whilethesourceinthepoint-to-pointVCisdenotedby

thelight-coloredcircle.Atthesecondswitch,tra�cfrom

foursources,butonlytwoVCs,isbeing

switchedtotheoutputport.Note,however,thatthesecondswitchcandistinguishthreeinput ows

(thepoint-to-pointconnection,andtwo owsofthemultipoint-to-pointconnectioncomingthrough

di�erentinterfaces).Thetwosourceswhosetra�cwasmergedatthe�rstswitchconstituteasingle

owatthesecondswitch,assumingthattheycannotbedistinguisheddownstream

oftheirmerge

point.Thus,twooftheinput owsthatcanbedistinguishedatthesecondswitchbelongtothe

sameVC,whilethethird owbelongstoadi�erentVC.Thesecondswitchmergesthetwo owsof

thesameVCintoasingle ow.

2.2

VC

Merging

InATM

networks,thevirtualpathidenti�er(VPI)andvirtualconnectionidenti�er(VCI)�elds

inthecellheaderareusedtoswitchATM

cells.TheATM

adaptationlayer(AAL)atthesource

segmentspacketsintoATM

cells,markingthelastcellofeachpacket.TheAALatthedestination

usestheVPI/VCI�eldsandtheendofpacketmarkertoreassemblethedatafromthecellsreceived.

ATM

adaptationlayer5(AAL5),whichisusedformostdatatra�c,doesnotintroduceanymul-

tiplexingidenti�erorsequencenumberinATM

cells.Ifcellsfrom

di�erentsourcesaremergedand

interleavedonthelinksofamultipointconnection(implementedasasharedtree),theAAL5at

thedestinationcannotassemblethedata.Thisisbecausealltra�cwithinthegroupusesthesame

VPI/VCI,andtheidentityofthesourceisnotindicatedineachcell.TheAAL5layerusesthe

end-of-messagebittodeterminetheendofeachpacket,butsincethecellsofdi�erentpacketsare

interleaved,allthepacketsmaygetcorrupted,asillustratedin�gure4(theend-of-messagebitvalue

isshownaboveeachATM

cellinthe�gure).

OneofthesolutionsproposedtothisproblemistheVCmergeapproach.Thisapproachbu�ersthe

cellsofpacketscomingthroughotherinterfacesattheswitchuntilallcellsofthecurrentpacket

gothrough(see[6]foradescriptionofthetechniqueand[17]forananalysisofitsperformance).

Thus,apacket-basedschedulingalgorithmisimplementedatthemergepoint,andseparatequeues

aremaintainedforeach ow

(wherea ow

isde�nedasthecellsofaVC

comingonaninput

4

Figure4:Thecellinterleavingproblem

link).Theend-of-messagebitsignalstotheswitchthatapacketfrom

adi�erentportcannow

beforwarded.Inthispaper,wefocusonVCmergeimplementations,wherecellsfrom

di�erent

sourcesareindistinguishable,sincethisisthemostdi�cultcaseforbandwidthallocationalgorithms

tohandle.

2.3

ABR

Flow

Control

Theavailablebitrate(ABR)serviceperiodicallyindicatestosourcestherateatwhichtheyshould

betransmitting.Thefeedbackfromtheswitchestothesourcesisindicatedinresourcemanagement

(RM)cellsgeneratedbythesourcesandturnedaroundbythedestinations(�gure5).TheRM

cells

owingfromthesourcetothedestinationarecalledforwardRM

cells(FRMs)whilethosereturning

fromthedestinationtothesourcearecalledbackwardRM

cells(BRMs).

Figure5:ResourcemanagementcellsinanATM

network

TheRM

cellscontainthesourcecurrentcellrate(CCR),inadditiontoseveral�eldsthatcanbeused

bytheswitchestoprovidefeedbacktothesources.Feedbackcanbejustoneortwobits,oritcanbe

therateatwhichthesourceshouldtransmit,calledtheexplicitrate(ER).Whenasourcereceives

aBRM

cell,itcomputesitsallowedcellrate(ACR)usingitscurrentACRvalue,thecongestion

indicationbits,andtheexplicitrate�eldoftheRM

cell.

2.4

PreviouslyProposedABR

Multipoint-to-PointAlgorithms

Tra�cmanagementrulesformultipoint-to-pointconnectionsarestillintheirearlyphasesofdef-

inition[14,13,1,12].RenandSiu[14,13]havedescribedanalgorithm

formultipoint-to-point

congestioncontrol,whichassumesthatVCmergeisemployed.Thealgorithm

operatesasfollows.

5

A bit is maintained at the merge point for each of the ows being merged. The bit indicates that an

FRM has been received from this ow after a BRM had been sent to it. Therefore, when an FRM is

received at the merge point, it is forwarded to the root and the bit is set. When a BRM is received

at the merge point, it is duplicated and sent to the branches that have their bit set, and then the

bits are reset. We implement this algorithm as explained in section 4.2, and show simulation results

in section 5.

In their papers [14] and [13], Ren and Siu only show simulation results for simple LAN con�gura-

tions. We discuss more complex problems and many general algorithm design issues that arise in all

multipoint algorithms, and show more simulation results for our proposed solutions. Furthermore,

Ren and Siu's work does not clearly state which types of rate allocation algorithms the proposed

multipoint extension works for. In fact, the extension does not work for many popular ABR schemes

that perform per VC accounting, since this is no longer equivalent to per-source accounting.

Recently more complex algorithms have been developed [1, 12] for multipoint-to-point and multipoint-

to-multipoint connections respectively. The algorithm in [1] aims at fairness among the sources as

in [14]. The algorithm in [12] adds a weight in RM cells to allow scaling of the rates to give the

appropriate allocations to sources. The throughput of a unicast source is given a pre-determined

weight with respect to that of a sender in a multicast session. This technique adds more exibility

at the expense of complexity in RM cells and processing. Weight assignment is also very di�cult.

3 Fairness De�nition

Throughout the rest of the paper, we use max-min fairness as the underlying fairness de�nition.

However, the de�nition we give apply for any underlying de�nition, e.g., general weighted fairness

with minimum rate guarantees [16]. Max-min fairness means that no connection can be allocated

a higher rate without hurting another connection having an equal or lower rate. We de�ne a net-

work con�guration as a set of sources, destinations and switches, interconnected with links of given

distances and bandwidths, and a set of virtual connections. We use the following notation:

n denotes the number of sources in a given con�guration

xi denotes the allocation given to the ith source in a given con�guration

Source-based. Source-based fairness allocates bandwidth fairly among all sources, regardless of

which VC each source belongs to. Each N -to-one connection is treated the same as N one-to-one

connections.

6

De�nition:Source-basedfairnessdividesbandwidthfairlyamongactivesourcesasiftheywere

sourcesinpoint-to-pointconnections,ignoringgroupmemberships.

Theallocationvectorfx1;x2;:::;xngisdeterminedbyapplyingtheunderlyingfairnessde�nition

forallactivesourcesxi.

2

Note,however,thatthebandwidthallocatedtoamultipoint-to-pointVCwithN

concurrentsources

allbottleneckedonacertainlinkwouldbeN

timesthebandwidthforapoint-to-pointVCbottle-

neckedonthatsamelink,andN=K

timesthatforaK-sourcemultipoint-to-pointVCbottlenecked

onthesamelink.

Figure6:Examplemultipoint-to-pointcon�gurationwithanupstreambottleneck

Thefairnessde�nitioncanbeexplainedusingthefollowingexample.Figure6illustratesacon�gura-

tionwithtwoVCs:oneoftheVCsisamultipoint-to-pointVCwithfoursourcesandonedestination,

andtheotherisapoint-to-pointVC.SourcesS1,S2,S3

andS4

aresendingtodestinationdS1,and

sourceSA

issendingtodestinationdSA.Alllinksareapproximately150Mbps(afterSONETover-

headisaccountedfor),exceptforthelinkbetweenSwitch1

andSwitch2

(LINK1)whichisonly

50Mbps.Clearly,sourcesS1,S2

andSA

arebottleneckedatLINK1,whilesourcesS3

andS4

are

bottleneckedatLINK3.Theaim

ofthisexampleistoillustratetheallocationofthecapacityleft

overbysourcesbottleneckedonLINK1

tothesourcesbottleneckedonLINK3.

Theallocationvectoraccordingtothesource-basedde�nitionis:

fS1;S2;S3;S4;SAg f16.67,16.67,58.33,58.33,16.67g

ThisisbecauseeachofsourcesS1,S2

andSA

isallocatedonethirdofthebandwidthofLINK1.

AtLINK3,the50�

2 3

=33:33MbpsusedbysourcesS1

andS2

issubtractedfrom

theavailable

bandwidth,andtheremainingcapacity(116.67Mbps)isequallydividedamongsourcesS3

andS4.

Otherfairnessde�nitions,e.g.,thosein[3],arealsopossible.Thechoiceofthetypeoffairnessto

adoptreliesontheapplicationtype,andpricingmethodsused.Weprefersource-basedfairness,

asitisanaturalextensionofpoint-to-pointfairnessde�nitions.Inaddition,itisthesimplestto

implement,anddoesnotsu�erfrom

beat-downproblemsaswith ow-basedsolutions.Theinter-

connectionunfairnessdiscussedaboveisnotamajorproblemsincethenumberofconcurrentsenders

inthemultipointconnectionisusuallysmallintypicalapplications(e.g.,onespeakeratatimein

7

an audio conference). Weights can also be used to eliminate any unfairness. Pricing can be based

on sources in this case. In the remainder of this paper, we discuss and analyze the development of

an algorithm to achieve source-based fairness.

4 The Algorithm

We �rst discuss the rate allocation algorithm, and then the merge point algorithm. Then we discuss

some design issues.

4.1 Rate Allocation Algorithm

Rate allocation algorithms are employed at every network switch to compute and indicate the appro-

priate feedback to the sources. The algorithm we discuss is based upon the ERICA+ rate allocation

algorithm [10]. However, we eliminate all the steps that required per-VC accounting in ERICA+.

The reason for this is that rate algorithms perform per-VC accounting as if it were per-source ac-

counting. Per-source accounting must be avoided for compatibility with VC merge switches and for

scalability.

The algorithm uses a measurement interval to measure the quantities required for computing the

rate allocation. At the end of every interval, the algorithm averages some of the quantities measured,

and uses these quantities to give the appropriate feedback to the sources in the following interval.

The algorithm measures: (1) the ABR input rate to each port, and (2) the available capacity on

each link, subtracting the capacity used by higher priority classes such as VBR. It also computes a

function of the queueing delay and uses its value to scale the available capacity (in order to leave

some of the capacity for the queues to drain). The ratio of the (average) measured input rate to the

(average) mesaured target capacity is called the overload factor.

The algorithm also uses the current cell rate (CCR) of the sources, as indicated in the FRM cells.

In addition, it keeps track of the maximum explicit rate indicated to all sources sending to this port

during each interval.

The overload is compared to 1+� (usually � is set to 0.1). If the overload is greater than 1.1, which

means there is high overload, the algorithm scales down the current cell rate of the connection by

the overload factor. Otherwise, if there is underload (overload is � 1+ �), the algorithm also uses an

additional quantity. This quantity is the maximum allocation allocated during the previous interval.

Bringing up all allocations to this quantity ensures that all connections get fair rates according to

the speci�ed weights.

8

In the pseudocode below, there are two options that are not necessary for the algorithm, but help

reduce rate uctuations in some cases (especially when the measurement interval value is very small).

The �rst option (which we label option 1) does not use the most current CCR value from FRM cells,

but uses the maximum of the CCR values seen in FRMs in the current interval. This option is useful

when there are multiple sources in the same VC, as explained in the next subsection. The second

option (option 2) uses exponential averaging for the maximum ER given in the previous interval to

smooth out variations.

The algorithm executes for each output port: when an FRM cell is received, when a BRM cell is

received, and at the end of each measurement interval. The algorithm is O(1) and its complexity is

independent of the number of connections and the number of sources. Since the calculations of the

input rate, target capacity and overload factor are the same as in the ERICA+ algorithm, we only

brie y outline these here.

FRM cell is received for VC j:

(current cell rate)j CCR �eld from the FRM cell

Or as an option (option 1: maximum CCR option):

IF (�rst FRM in interval)j = TRUE THEN

(current cell rate)j CCR �eld from the FRM cell

(�rst FRM in interval)j FALSE

ELSE

(current cell rate)j maximum (CCR �eld from the FRM cell, (current cell rate)j)

END

BRM cell is to be sent out for VC j:

IF (overload factor > 1+�) THEN

ER (current cell rate)j/overload factor

ELSE

ER maximum ((current cell rate)j/overload factor, maximum ER in previous interval)

END

ER minimum (target capacity, ER)

maximum ER in current interval maximum (ER, maximum ER in current interval)

ER in BRM cell minimum (ER, ER in BRM cell)

End of measurement interval:

target capacity exponential average of (across intervals) of link capacity minus CBR and VBR

9

capacity, scaled for queues to drain by using a fractional function (refer to [10])

input rate exponential average (across intervals) of total ABR input cells being switched to this

output port

overload factor input rate/target capacity

8j (�rst FRM in interval)j TRUE

maximum ER in previous interval maximum ER in current interval

Or as an option (option 2: averaging the maximum ER in previous interval option):

maximum ER in previous interval

(1-�) � maximum ER in current interval + � � maximum ER in previous interval

maximum ER in current interval 0

Notes:

1. The input rate, target capacity, overload factor, maximum ER in current interval and maximum

ER in previous interval are computed and stored for each output port. The \�rst FRM in

interval" (if used) and the \current cell rate" are stored for each VC for each output port.

2. In our simulations, the parameter � is set to 0.1, and the parameter � is also set to 0.1. These

are the recommended values for these parameters.

3. The \averaging of maximum ER in previous interval" option (option 2) slightly reduces rate

oscillations in some cases. It is not essential if its implementation complexity is high.

4. The maximum CCR option (option 1) also reduces rate oscillations in cases of extremely small

averaging interval values (< 200 �s for rates about 10 Mbps per source). It is also unnecessary.

Exponentially averaging the maximum CCR values across intervals might further improve the

performance. The next subsection discusses the usage of CCR in more detail.

Reference [10] gives a proof that this algorithm converges to the max-min fair rates for a single

bottleneck case. The main idea of the proof is that the algorithm is fair because it allocates all

sources bottlenecked at the same link the exact same rates. In addition, the algorithm converges to

rates that result in an overload factor value close to one, because the rates are scaled by the overload

factor.

10

4.2 Merge Point Algorithm

This algorithm is the same as the multipoint-to-point algorithm developed by Ren and Siu in [13].

The algorithm is employed at every merge point where cells from di�erent sources in the same

multipoint-to-point VC are being merged and follow the same path to the destination. We �rst give

the pseudocode for the algorithm, and then discuss some properties of the algorithm.

A ag (can be one bit) called Ready is maintained for each of the ows being merged. The ag

indicates that an FRM cell has been received from this ow after a BRM cell had been sent to it.

Upon the receipt of an FRM cell from branch i:

1. Forward FRM cell to the outgoing link

2. Let Readyi = TRUE

Upon the receipt of a BRM cell from the root:

FOR ALL upstream branches DO

IF Readyi = TRUE THEN

Send a copy of the BRM to branch i

Let Readyi = FALSE

END

END

When a BRM cell is about to be scheduled:

Perform the rate allocation algorithm as described in the previous section

Reference [13] gives a proof by induction on the number of levels of the multipoint tree to show that

this algorithm gives fair allocations for multiple sources if the rate allocation algorithm employed

gives max-min fair allocations.

4.3 Rate Allocation Design Issues

As previously mentioned, rate allocation algorithms for multipoint-to-point (or multipoint-to-multipoint)

connections may not be able to distinguish cells from di�erent sources in the same VC. Thus they

cannot: (1) use the number of established connections as an indication of the number of sources, (2)

measure or estimate the rate of each source, (3) distinguish between overloading and underloading

sources, or compute the number of overloading sources, (4) estimate the e�ective number of active

sources. Such techniques are used in many of the popular point-to-point switch schemes, such as the

MIT scheme [2] and the UCSC scheme [9].

11

Most switch schemes also use the current cell rate of the sources in the computation of the explicit

rate. Algorithms which use the CCR values noted from backward RM cells are not fair

for multipoint-to-point connections. This is because it may be impossible to determine which

source the RM cell belongs to. The CCR value in the BRM cells at the merge point may not capture

upstream bottleneck information for any of the ows whose tra�c is being merged, since it may

actually be the CCR of a downstream source whose bottleneck rate is high. We explain this next.

Lemma 1: Algorithms which use the CCR values noted from backward RM cells are not fair for

multipoint-to-point connections: ER = f(CCRBRM ) is not necessarily max-min fair.

Proof Sketch: The proof is by counter-example. We give a case where an algorithm using CCRBRM

gives unfair allocations. Suppose a multipoint-to-point VC has two sources, one of which has a

bottleneck rate of 58 Mbps, and the other has a bottleneck rate of 16 Mbps, and the two sources are

being merged at a switch. Figure 6 shows an example where at Switch2, S1 (and S2) of rate 16 Mbps

and S3 of rate 58 Mbps are being merged (we will simulate this case in sections 5.2.2 to 5.2.4). The

source which is bottlenecked at 16 Mbps (say S1) shares its bottleneck link with a point-to-point

connection (SA to dSA). At the merge point, BRM cells of the higher rate source (the 58 Mbps

source) are more frequently sent to all the sources in this VC being merged with a high ER value

(since the CCR is assumed to be 58 Mbps). This can result in over-allocation to the lower rate

source(s) being merged, and unfairness to the point-to-point connection. 2

Hence, algorithms that use the CCR value for rate computation must use the value of the CCR

indicated in FRM cells for computation when a BRM cell is received. This is the most up-to-date

value of CCR, since the CCR in the BRMs may be stale after traveling all the way to the destination

and back. The CCR value in the FRM cells at the merge point captures upstream bottleneck

information for one of the ows whose tra�c is being merged. The FRM cells of the sources being

merged, however, may still be indistinguishable at the merge point. In the remainder of this section,

we argue that this does not a�ect the convergence and steady state behavior of the algorithm.

Lemma 2: Algorithms which use the CCR values noted from forward RM cells can compute statis-

tically fair allocations for multipoint-to-point connections.

Proof Sketch:

Since the guaranteed fairness is statistical, the proof is also statistical. Assume that there are two

ows Slow and Shigh being merged. We will brie y examine the situation when the forward CCR

used to compute the ER for a ow is not the CCR corresponding to that ow.

CASE 1:

When computing the ER for Slow, if the CCR of Shigh is used, then the ER computed for Slow will

12

be too high. But Slow is bottlenecked upstream of the merge point (otherwise its bottleneck rate will

not be less than that for Shigh, since Slow and Shigh merge at the merge point and never split after

that), so the ER given to Slow at the merge point will be overwritten by upstream switches.

CASE 2:

For the case when the CCR of Slow is used to compute the ER for Shigh, �rst consider the algorithm

with the maximum CCR option. The only situation when the ER for Shigh is calculated based upon

the CCR for Slow is when only FRM cells of Slow have been seen since the beginning of the current

interval. (Note that if no FRM cells have been seen at all, the CCR value used is the maximum seen

in the previous interval, which will be the CCR of the higher rate source Shigh unless Shigh is sending

at a very low rate, in which case the scheme should not allocate it high rates: see the discussion

in [11] for more details on handling low rate sources.) Since Shigh has a higher rate, it has a higher

frequency of FRM cells, so it becomes highly improbable for this to hold.

This argument can be extended for the algorithm without the maximum CCR option. In this case,

instead of the smaller CCR being used when only FRM cells from the lower rate source have been

seen so far in this interval, it is the last FRM cell received that determines the CCR used. But,

again, since the higher rate source has a higher FRM rate, it is statistically unlikely for the smaller

CCR to be used. The maximum ER in the previous interval term ensures that if the small CCR is

in fact used, the source is allocated at least as much as other VCs going to the same output port,

which ensures fairness and fast convergence. 2

4.4 Merge Point Design Issues

There are a number of ways to implement multipoint-to-point merge point algorithms. Each method

o�ers a tradeo� in complexity, scalability, overhead, response time and steady state behavior.

In the above algorithm, a BRM cell is returned to a source for every one or more FRM cells it sends.

Thus the BRM to FRM cell ratio at the source is less than or equal to one. In steady state, the ratio

is likely to approach one, since the FRM rate and BRM rate will be similar. This is an important

property of ABR ow control that should be maintained for multipoint-to-point connections. The

BRM to FRM ratio in the network is also one in this case. (If FRM cells are turned around at merge

points as in [14], the same FRMs can be turned around at another merge point or the destination,

creating BRM cells that eventually get discarded in the network.)

Also observe that in this scheme, since the merge point does not need to turn around every FRM

cell, the overhead of the algorithm is reduced. However, the scheme needs to duplicate BRM cells.

With the new advances in multicast ATM switch architectures, this operation can be quite e�cient.

13

The algorithm we use returns a BRM cell received from the root to the branches which have sent

FRM cells to the merge point since the last BRM cell had been passed. This makes the scheme less

sensitive to the number of levels of merge points, as compared to those schemes which turn around

FRM cells (such as the scheme in [14]). This is because schemes turning around FRMs have to wait

for an FRM to be received at every merge point, so their response time increases with the number of

levels in the tree. In addition, the ER value returned by such schemes may be incorrect if no BRM

cells have been received since the last one was sent, leading to rate oscillations and possibly large

queue lengths.

5 Performance Analysis

This section provides a simulation analysis of the multipoint algorithm described in the previous two

sections. Only a few simple experiments are shown here; more stringent tests have been conducted,

and the preliminary results are consistent with those presented next.

The results are presented in the form of four graphs for each con�guration:

(a) Graph of allowed cell rate (ACR) in Mbps versus time for each source

(b) Graph of ABR queue lengths in cells versus time at the bottleneck port of each switch

(c) Graph of link utilization versus time for each of the main (backbone) links (those that connect

two switches to each other)

(d) Graph of number of cells received versus time for each destination

5.1 Parameter Settings

Throughout our experiments, the following parameter values are used:

1. Except where otherwise indicated (in sections 5.2.2 to 5.2.4), all links have a bandwidth of

155.52 Mbps (149.76 Mbps after SONET overhead is accounted for).

2. All multipoint-to-point tra�c ows from the leaves to the root of the tree. No tra�c ows from

the root to the leaves, except for RM cells. Point-to-point connections are also unidirectional.

3. Except in section 5.2.3 where we experiment with the source parameter rate increase factor

(RIF), we have set RIF to 1/32 in our simulations. We do not, however, expect the performance

of the algorithm to be signi�cantly in uenced by the value of RIF, as seen in section 5.2.3.

14

4. The source parameter transient bu�er exposure (TBE) is set to large values to prevent rate

decreases due to the triggering of the source open-loop congestion control mechanism. This

was done to isolate the rate reductions due to the switch congestion control scheme from the

rate reductions due to TBE.

5. All other ABR parameters are set to their default values [5].

6. A dynamic queue control function is used to scale the available capacity and achieve a constant

queuing delay in steady state [10]. The \target delay" parameter speci�es the desired queuing

delay. A value of 1.5 ms was used. An inverse hyperbolic function is used. The hyperbolic

function curve parameters used were a = 1:15 and b = 1. The queue drain limit factor is set

to 0.5 (which means that up to 50% of the link capacity can be used to drain queues).

7. A �xed time measurement interval is used to measure and average the input rate and available

capacity, and to note the maximum allocation given (and possibly the maximum CCR value

in FRM cells). The interval is set to 5 ms in all experiments except those in section 5.2.4.

8. Since we do not implement VC merge in our switches, we only use one cell long packets. Our

next study will implement VC merge and examine its e�ect.

9. All sources are deterministic, i.e., their start/stop times and their transmission rates are known.

10. Simulation time is two seconds.

11. The simulations use both the maximum CCR option and exponentially averaging the maximum

ER option as discussed in section 4.1. We have simulated all our con�gurations without using

either option, and with each option separately, and the di�erences were insigni�cant. We do

not show these results here for space considerations. In particular, the results when neither

of the two options is enabled, and with extremely small measurement intervals (as with the

simulations in section 5.2.4) showed that the algorithm still rapidly converges to the optimal

allocations, and that the oscillations (though they do slightly increase) were not signi�cantly

more than the results we show in section 5.2.4.

5.2 Simulation Results

In this section, we discuss a sample of our simulation results. We mainly use two con�gurations, and

experiment with di�erent link lengths, initial cell rates of the sources, rate increase factor values,

and lengths of the measurement interval.

15

5.2.1

Downstream

BottleneckCon�guration

Figure7:Examplemultipoint-to-pointcon�gurationwithadownstreambottleneck

Figure7illustratesacon�gurationwithtwoVCs:oneoftheVCsisamultipoint-to-pointVCwith

threesourcesandonedestination,andtheotherisapoint-to-pointVC.SourcesS1,S2,andS3

are

sendingtodestinationdS1,andsourceSA

issendingtodestinationdSA.Alllinksare149.76Mbps

(OC-3linksafterSONEToverheadisaccountedfor),andtheirlengthsareasshowninthe�gure.

Clearly,allfoursourcesaresharingabottlenecklink(LINK3)betweenSwitch3

andSwitch4.This

experimentshowsthedivisionofthecapacityofthisbottlenecklinkamongallsourcesinbothtypes

ofconnections.

Applyingthefairnessde�nitionamongsources,theoptimalallocationsshouldbe:

fS1;S2;S3;SAg f37.5,37.5,37.5,37.5g

Eachofthefoursourcesisallocated

1 4

�150=37:5.

Figure8illustratestheresultsofsimulatingtheabovecon�guration.ThesourcesstartwithanICR

valueof25Mbps,whichisbelowtheiroptimalallocation.Clearly,allsourcesrisetotheiroptimal

ratesquickly(�gure8(a)),andthequeuesaresmall(�gure8(b)).Thebottlenecklink(LINK3)

isfullyutilized(�gure8(c)).LINK2

is50%

utilized(sinceonly2ofthe4sourcesutilizeit)and

LINK1

isonly25%

utilized(1outof4sources).

Observethatwithsource-basedfairness,VCsthathavealargernumberofconcurrentlyactivesources

getmorebandwidththanVCswithlessconcurrentsourcesonthesamelink.Thiscanbeclearly

seenfromtheslopeofthecellsreceivedgraph(�gure8(d))fordS1

anddSA.ClearlydS1

hasaslope

thatisthreetimesaslargeasthatfordSA.After2seconds,theratioofcellsreceivedatdSA

todS1

isaround175000to520000,whichisexactly1to3.Thustheresourceallocationisnotfairamong

theVCs.

Figure9illustratestheresultsofsimulatingthesamecon�guration(�gure7)whenallsourcesstart

atahighICRvalue.TheICRforallsourceshereis100Mbps.Thiscreatesaninitialoverload

onLINK3

of

400

150

=2

2 3.Thealgorithmrecoversfrom

thissituationandallsourcesconvergetothe

correctvalueofapproximately37.5Mbps(�gure9(a)).ThequeuesatSwitch3

startdroppingafter

approximatelyoneroundtrip(�gure9(b)).

16

0

20

40

60

80

100

120

140

160

180

0 200 400 600 800 1000 1200 1400 1600 1800 2000

AC

Rs

Time in milliseconds

WAN 3-leaf and downstream bottlenck: ACRs

ACR for S1 ACR for S2 ACR for S3 ACR for SA

(a) Allowed Cell Rate

0

200

400

600

800

1000

1200

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Que

ue L

engt

hsTime in milliseconds

WAN 3-leaf and downstream bottlenck: Queue Lengths

Queue Length for Switch 1 Queue Length for Switch 2 Queue Length for Switch 3

(b) Queue Length

0

20

40

60

80

100

120

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Util

izat

ion


WAN 3-leaf and downstream bottlenck: Utilization

Utilization for Link 1 Utilization for Link 2 Utilization for Link 3

(c) Link Utilization

0

100000

200000

300000

400000

500000

600000

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Cel

ls R

ecei

ved


WAN 3-leaf and downstream bottlenck: Cells Received

Cells Received at dS1 Cells Received at dSA

(d) Cells Received

Figure 8: Results for a WAN multipoint-to-point con�guration with a downstream bottleneck (long

LINK3, low ICR)

17

0

20

40

60

80

100

120

140

160

180

0 200 400 600 800 1000 1200 1400 1600 1800 2000

AC

Rs





0

5000

10000

15000

20000

25000

30000

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Que

ue L

engt




(b) Queue Length

0

20

40

60

80

100

120

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Util

izat

ion





0

100000

200000

300000

400000

500000

600000

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Cel

ls R

ecei

ved




(d) Cells Received


LINK3, high ICR)

18

0

20

40

60

80

100

120

140

160

180

0 200 400 600 800 1000 1200 1400 1600 1800 2000

AC

Rs





0

200

400

600

800

1000

1200

1400

1600

1800

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Que

ue L

engt

hs




(b) Queue Length

0

20

40

60

80

100

120

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Util

izat

ion





0

100000

200000

300000

400000

500000

600000

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Cel

ls R

ecei

ved




(d) Cells Received


LINK3, di�erent ICRs)

Figure 10 shows the results for the same con�guration, but di�erent sources start at di�erent ICR

values. Sources S1 and S3 start at an ICR of 65 Mbps, while sources S2 and SA start at 10 Mbps.

Notice that the sum of the source rates for all sources is 150 Mbps, so the initial load value is close

to 1. The rates for sources S1 and S3 are quickly reduced, while those of sources S2 and SA quickly

rise, as seen in �gure 10(a). The queues are also quite small (�gure 10(b)).

5.2.2 Upstream Bottleneck with Heterogenous Links Con�guration

Figure 11: Example multipoint-to-point con�guration with an upstream bottleneck

19

Figure 11 illustrates the same con�guration as in �gure 6, where all links are approximately 150 Mbps,

except for the link between Switch1 and Switch2 (LINK1) which is only 50 Mbps. The link lengths

are as shown in the �gure.

Recall that the allocation vector according to the source-based fairness de�nition is:

fS1; S2; S3; S4; SAg f16.67, 16.67, 58.33, 58.33, 16.67g

Figure 12 illustrates the results for this con�guration. Sources S1 and S2 start at an ICR of 20 Mbps.

Source S3 starts at 30 Mbps and source S4 starts at 80 Mbps. Source SA starts at 10 Mbps.

As seen in �gure 12(a), sources S1, S2 and SA converge to about 16.67 Mbps, while sources S3 and

S4 converge to about 58.33 Mbps. The queues are bounded to reasonable values (�gure 12(b)) and

utilization of the bottleneck links (LINK1 and LINK3) are close to 100% (�gure 12(c)). Destination

dSA gets much less throughput than dS1 (�gure 12(d)), since source SA is bottlenecked on a 50 Mbps

link with 2 other sources. After 2 seconds, the ratio of the throughputs for destinations dSA to dS1

is approximately 80000 to 700000 which is 0.11. The slopes of the two lines also have the same ratio.

This is close to the optimal value since 16.67/149.76 = 0.11.

5.2.3 E�ect of Large Rate Increase Factor Values

The rate increase factor determines the maximum increase when a BRM cell indicating underload

is received. If the RIF is set to a fraction less than one, the maximum increase at each step is

limited to RIF � the peak cell rate for the VC. Setting RIF to small values is a more conservative

strategy that controls queue growth and oscillations, especially during transient periods. It, however,

may slow down the response of the system when capacity suddenly becomes available leading to

underutilization.

Figure 13 illustrates the results for the con�guration of �gure 11 when the rate increase factor (RIF)

is set to its maximum possible value, which is 1. Part (a) of the �gure shows that the rates do not

oscillate more than the corresponding �gure with a small RIF value (�gure 12(a)). The queues in

�gure 13(b) are also similar to those in �gure 12(b).

5.2.4 E�ect of Extremely Short Measurement Intervals

As discussed in section 4.1, extremely short measurement intervals can cause the algorithm to su�er

from oscillations. To examine this e�ect, we have simulated the algorithm with a measurement

interval of 200 �s. Recall that in the upstream bottleneck con�guration (shown in �gure 11), the

optimal rates for sources S1, S2 and SA are 16.67 Mbps, and those for sources S3 and S4 are

20

0

20

40

60

80

100

120

140

160

180

0 200 400 600 800 1000 1200 1400 1600 1800 2000

AC

Rs


WAN 4-leaf with upstream bottleneck: ACRs

ACR for S1 ACR for S2 ACR for S3 ACR for S4 ACR for SA


0

200

400

600

800

1000

1200

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Que

ue L

engt


WAN 4-leaf with upstream bottleneck: Queue Lengths


(b) Queue Length

0

20

40

60

80

100

120

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Util

izat

ion


WAN 4-leaf with upstream bottleneck: Utilization



0

100000

200000

300000

400000

500000

600000

700000

800000

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Cel

ls R

ecei

ved


WAN 4-leaf with upstream bottleneck: Cells Received


(d) Cells Received

Figure 12: Results for a WAN multipoint-to-point con�guration with an upstream bottleneck (long

LINK3)

21

0

20

40

60

80

100

120

140

160

180

0 200 400 600 800 1000 1200 1400 1600 1800 2000

AC

Rs





0

200

400

600

800

1000

1200

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Que

ue L

engt




(b) Queue Length

0

20

40

60

80

100

120

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Util

izat

ion





0

100000

200000

300000

400000

500000

600000

700000

800000

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Cel

ls R

ecei

ved




(d) Cells Received


LINK3, large RIF)

22

58.33 Mbps. This implies that, in steady state, RM cells for sources S1, S2 and SA arrive every:

Nrm� bits=cell

ACR=

32� 53� 8

16:67 M= 813:92 �s

For sources S3 and S4, RM cells arrive every:

Nrm� bits=cell

ACR=

32� 53� 8

58:33 M= 232:61 �s

where a source sends an FRM cell every Nrm cells, and the default value of Nrm is 32.

Setting the measurement interval to 200 �s means that RM cells for S3 and S4 might not be received

every measurement interval, and that RM cells for S1, S2 and SA might not be received for 4

consecutive measurement intervals.

In order to receive at least one FRM cell from the highest rate source in a certain interval, the interval

length should be > NrmACR maximum

. This condition is likely to hold for reasonably long intervals, unless

all sources are sending at very low rates, in which case the overload factor will be low and their rates

will increase if they have data to send.

Figure 14 illustrates the results for the con�guration of �gure 11. Clearly, the short averaging interval

causes more oscillations, but the rates of the sources still converge to their fair rates. Also observe

that the number of cells received for both connections is the same as in �gure 12(d). Increasing the

value of the parameter � (in section 4.1) can reduce the oscillations.

6 Conclusions and Recommendations for Switch Schemes

All source-based switch algorithms operating in VC merge switches need to avoid distinguishing

among sources in the same VC. Key lessons learned from this study include (refer to section 4.3 for

supporting arguments):

1. Source-level accounting should not be performed in multipoint rate allocation algorithms. For

example, measuring the rates for each source, or distinguishing overloading and underloading

sources cannot be performed. If such accounting is performed at the VC level or the ow level,

an additional mechanism to divide VC or ow bandwidth among sources is necessary.

2. Estimating the e�ective number of active sources in order to divide the available capacity

among them is very di�cult in multipoint connections, since it is impossible to distinguish

among sources in the same multipoint VC with VC merge implementations.

3. The only information a multipoint rate allocation algorithm can use is the information supplied

in RM cells, in addition to aggregate measurements of load, capacity and queuing delays.

23

0

20

40

60

80

100

120

140

160

180

0 200 400 600 800 1000 1200 1400 1600 1800 2000

AC

Rs





0

100

200

300

400

500

600

700

800

900

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Que

ue L

engt




(b) Queue Length

0

20

40

60

80

100

120

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Util

izat

ion





0

100000

200000

300000

400000

500000

600000

700000

800000

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Cel

ls R

ecei

ved




(d) Cells Received


LINK3, short interval)

24

4. CCR values from BRM cells should not be used in computing rate allocations for sources

in multipoint connections, since the CCR value can be that of another source that does not

go through the switch performing the computation. That source may have a much higher

bottleneck rate, and using its CCR can result in unfairness.

5. CCR values from FRM cells can be used to compute rate allocations for sources in multipoint

connections, even though the CCR used to compute the rate for a source may not actually be

the CCR value of the source. This does not create problems due to the properties of the merged

ow (see section 4.1 for a more detailed explanation). The maximum CCR value seen in an

interval can be used instead of the CCR of the source. Exponential averaging of the maximum

CCR seen or maximum ER given may further improve the performance of the algorithm.

6. Merge point algorithms should avoid changing the BRM to FRM ratio at the source or inside

the network, to maintain the rate of feedback that the source requires, and avoid excessive

overhead in the network. Scalability of the scheme is also a�ected by these ratios. Excessive

complexity, noise, and response time can also be avoided by returning the BRM cells coming

from the root, instead of turning around the RM cells at the merge points (refer to section 4.4

for supporting arguments).

We have given and simulated an O(1) algorithm for computing source-based fair allocations for

multipoint-to-point and point-to-point connections. The algorithm uses simple aggregate measure-

ments and maximum CCR values from FRM cells during successive intervals to perform rate com-

putation. The algorithm exhibited very good behavior for the con�gurations tested. More extensive

performance analysis is crucial to examine the fairness, complexity, overhead, transient response, de-

lays, and scalability tradeo�s in multipoint algorithm design. Extending multipoint-to-point schemes

for multipoint-to-multipoint connections can be performed by combining point-to-multipoint algo-

rithms (such as those developed in [4]) with the multipoint-to-point algorithm.

References

[1] D. Cavendish and M. Gerla. Rate based congestion control for many-to-many multicast ABR

tra�c. In Proceedings of SPIE '98 Conference on Performance and Control of network systems

II, volume 3530, pages 122{130, November 1998.

[2] A. Charny, D. Clark, and R. Jain. Congestion Control with Explicit Rate Indication. In

Proceedings of ICC, June 1995.

25

[3] S. Fahmy, R. Jain, R. Goyal, and B. Vandalore. Fairness for ABR multipoint-to-point connec-

tions. In Proceedings of SPIE '98 Conference on Performance and Control of network systems

II, volume 3530, pages 131{142, November 1998.

[4] Sonia Fahmy, Raj Jain, Rohit Goyal, Bobby Vandalore, Shivkumar Kalyanaraman, Sastri Kota,

and Pradeep Samudra. Feedback consolidation algorithms for ABR point-to-multipoint connec-

tions. In Proceedings of the IEEE INFOCOM, volume 3, pages 1004{1013, March 1998.

[5] The ATM Forum. The ATM forum tra�c management speci�cation version 4.0.

ftp://ftp.atmforum.com/pub/approved-specs/af-tm-0056.000.ps, April 1996.

[6] Matthias Grossglauser and K. K. Ramakrishnan. SEAM: Scalable and e�cient ATM multipoint-

to-multipoint multicasting. In Proceedings of the IEEE INFOCOM, April 1997.

[7] Je�rey M. Ja�e. Bottleneck ow control. IEEE Transactions on Communications, COM-

29(7):954{962, July 1981.

[8] Mark Je�rey. Scope, concepts and issues for the new multiway BOF. ATM Forum/96-0628,

June 1996.

[9] L. Kalampoukas, A. Varma, and K. K. Ramakrishnan. An e�cient rate allocation algorithm

for ATM networks providing max-min fairness. In Proceedings of the 6th IFIP International

Conference on High Performance Networking, September 1995.

[10] S. Kalyanaraman, Raj Jain, Sonia Fahmy, Rohit Goyal, and Bobby Vandalore. The ERICA

switch algorithm for ABR tra�c management in ATM networks. IEEE/ACM Transactions on

Networking, 1998. Submitted. Available through our web page.

[11] Shivkumar Kalyanaraman, Raj Jain, Rohit Goyal, Sonia Fahmy, and Seong-Cheol Kim. Use-it

or lose-it policies for the available bit rate (ABR) service in ATM networks. Journal of Computer

Networks and ISDN Systems, 1998.

[12] W. M. Moh and Y. Chen. Design and evaluation of multipoint-to-point multicast ow control. In

Proceedings of SPIE '98 Conference on Performance and Control of network systems II, volume

3530, pages 143{154, November 1998.

[13] W. Ren, K-Y Siu, H. Suzuki, and M. Shinohara. Multipoint-to-multipoint ABR service in ATM.

Journal of Computer Networks and ISDN Systems, 30(19):1793{1810, October 1998.

[14] Wenge Ren, Kai-Yeung Siu, and Hiroshi Suzuki. Multipoint-to-point ABR service in ATM net-

works. In Proceedings of the International Conference on Communications, ICC'97, Montreal,

June 1997.

26

[15] E. M. Spiegel. Multipoint connection support in PNNI 2.0. ATM Forum Perspectives, IEEE

Network, Nov/Dec 1997.

[16] B. Vandalore, S. Fahmy, R. Jain, R. Goyal, and M. Goyal. A de�nition of general weighted

fairness and its support in explicit rate switch algorithms. In Proceedings of the Sixth In-

ternational Conference on Network Protocols 1998 (ICNP '98), pages 22{30, October 1998.

http://www.cis.ohio-state.edu/~jain/papers/icnp98 bv.htm.

[17] Indra Widjaja and Anwar I. Elwalid. Performance issues in VC-merge capable switches for IP

over ATM networks. In Proceedings of the IEEE INFOCOM, volume 1, pages 372{380, March

1998.

All our papers and ATM Forum contributions are available through http://www.cis.ohio-state.edu/~jain/

27

Date post:	21-May-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Raj Jain is now at [email protected], ...jain/papers/ftp/mpt2mpt.pdffairness. Multipt-to-pt oin oin ABR...

Documents