ConcurrentDisjointSetUnion

RobertE.TarjanPrincetonUniversity&IntertrustTechnologiesjointworkwithSiddharthaJayanB,Princeton

KeymessagesIdeasandresultsfromsequenBalalgorithmscancarryovertoconcurrentalgorithms,butnewideasareneeded:thedesignspaceisdifferent

ininteresBngwaysDesignandanalysisofconcurrentdata

structuresandalgorithmsissBllarichareatoexplore

ConcurrencycreatesnewandsubtlecomplicaBons!

ThisTalk

•  Disjointsetunionproblem•  MoBvaBngapplicaBon:

Strongcomponentsformodelchecking•  SequenBalsetunion•  Concurrencymodel•  Previouswork•  Ourresults

a

b

same-set(a,b)

unite(a,b)

same-set(a,b) True

b

Disjointsetunion

False

DeviseadatastructurewiththefollowingoperaBons:make-set(x) same-set(x,y):trueifx,y

insameset,elsefalseunite(x,y):combinesets

containingx,yEachelementisinoneset(setsaredisjoint)

ApplicaBons

FORTRANcompilers:COMMONandEQUIVALENCEstatements

Kruskal’sminimumspanningtreealgorithmIncrementalconnectedcomponentsingraphs

PercolaBonFindingdominatorsinflowgraphs

Findingstrongcomponentsindigraphs

StrongComponentsStrongComponent:AmaximalsetofmutuallyreachableverBcesinadirectedgraph.Strongcomponentsarevertex-disjointandcanbetopologicallyordered:numberedsonoarcleadsfromalargertoasmallercomponent.Goal:Findcomponents,andatopologicalorder.

Two Topological Orders

Model-checkingforprogramverificaBon

Inapossiblyhuge,implicitlydefineddigraph,discoverwhethercertainsetsofstatescanbevisitedinfinitelyo\en.Equivalently,arethesestatesinacommonstrongcomponent?EventhoughstrongcomponentscanbefoundinlinearBmesequenBally(viaaspecialcaseofdisjointsetunion)thismaynotbefastenough.

Canconcurrencyhelp?

DisjointSets:Compressed-treeImplementaBon

Representeachsetbyarootedtree•  Eachsetelementisatreenode•  Eachnodexhasaparentx.parent•  SetinformaBon(suchasvalue)isstoredintheroot•  Thetreeshapeisarbitrary

c

f g

b

da e

Auxiliary(internal)operaBons

find(x):Returntherootofthetreecontainingnodex.

NaïveimplementaBonoffind:returnifx.parent=xthenxelsereturnfind(x.p)link(x,y):Unitethetreeswithrootsxandy.

ImplementaBonoflink:makeytheparentofx(orxtheparentofy):

x.parent←y(ory.parent←x)

AlinktakesBmeO(1),afindtakesBmeO(depthofnodefound).

SequenBalimplementaBonofoperaBons

make-set(x):x.parent←xsame-set(x,y):returnfind(x)=find(y)unite(x,y):iffind(x)≠find(y)thenlink(find(x),find(y))AsetoperaBontakesO(1)Bmeplustwofinds.

EachlinktakesBmeO(1),eachfindtakesBmeO(depthofnodefound).

Goal:reducethe(amorBzed)Bmeperfindbyreducingnodedepths

Improvelinks:linkbysizeorrankorrandomindex

Improvefinds:compactfindpaths

Linkingbysize:maintainthenumberofnodesineachtree(storeinroot).Linkrootofsmallertreetolarger.BreakaBearbitrarily.

Linkingbyrank:Maintainanintegerrankforeachroot,iniBally0.Linkrootofsmallerranktorootoflargerrank.IfBe,increaserankofnewrootby1.

Linkingbyrandomindex:Giveeachelementauniquenumericindexchosenuniformlyandindependentlyatrandom.Linkrootofsmallerindextorootoflargerindex.

EachoftheselinkingrulesreducesthemaximumpathlengthtoO(logn),wheren=#nodes:atmostn/2knodesofheightkorgreater.

PathCompacBon

Compression:duringeachfind,maketheroottheparentofeachnodeonthefindpath.

Splihng:Duringeachfind,replacetheparentofeachnodeonthefindpathbyitsgrandparent.

Compressiontakestwopassesoverthefindpath,

splihngonlyone.

10

8

5

2

1

7

3 4

Compressionoffind(1)path

10

8

5

2

1

7

3 4

Splihngoffind(1)path

Original

A\erCompression A\erSplihng

RunningTime

Assumen>1make-setoperaBonsaredonefirst,followedbym≥n/2intermixedsame-setanduniteoperaBons.

Defined,thefinddensity,tobe⎡m/n⎤.

Withlinkingbysize,rank,orrandomindex,andcompressionorsplihng,totalBmeis

O(mα(n,d))

Ackermann’sfuncBon(Péter&Robinson)

A0(n)=n+1Ak(0)=Ak–1(1)ifk>0Ak(n)=Ak–1(Ak(n–1))ifk>0,n>0A1(n)=n+2,A2(n)=2n+3,A3(n)>2n,A4(n)>towerofn2’s,A4(2)has19,729decimaldigits

Ak(n)isstrictlyincreasinginbothargumentsα(n,d)=min{k>0|Ak(d)>n}

Computermodel:SharedmemorymulBprocessor(APRAM)

p1 p2 p3 …. pi pk-1 pk….

SharedMemory

LocalMemory

LocalMemory

LocalMemory

LocalMemory

LocalMemory

LocalMemory

Asynchronous–ArbitrarydelaysbetweenoperaBons

ConcurrencyAnderson&Woll,1994

EachsetoperaBonisdonebyoneprocessSeveraloperaBonscanrunconcurrently(differentprocesses).

Correctness

Linearizable:EachoperaBoncanbeassignedalineariza<on<meduringitsexecuBon,differentforeachoperaBon,suchthattheoutcomeofalltheoperaBonsisthesameasiftheywereexecutedinstantaneouslyattheirlinearizaBonBmes.Wait-free:EachprocesscompleteseachofitsoperaBonsinaboundednumberofitsownsteps.

Linearizability[Herlihy, Wing 1990] p1

p2

p3

Time

Union(x, y)

Union(z, x)

SameSet(x, y)

Union(w, z)

SameSet(x, w) SameSet(x, y)

Union(w, y) SameSet(z, y)

SameSet(w, y)

Correctness

Efficiency

Totalwork:totalnumberofstepstakenbyallprocesses,asaworst-casefuncBonofn,m,andp(thenumberofprocesses).Goal:TotalworknottoomuchbiggerthanthesequenBalBmeboundandsublinearinp:thenconcurrencymayhelp.

SynchronizaBonPrimiBvesforwait-freedom

Compare&SwapCAS(x,y,z):ifx=ythen{x←z;returntrue}elsereturnfalse

DoubleCompare&SwapDCAS(x,y,z,u,v,w):ifx=yandu=vthen{x←z;u←w;returntrue}elsereturnfalse

Previouswork:Anderson&Woll1994

ConcurrentversionoflinkingbyrankwithsplihngusingCAS.Bigproblem:CASseemstooweak:linkingbyrankrequireschangingarankinonenodeandapointerinanother.TheiralgorithmdoesnotavoidrankBes.WorkboundisO(m(α(n,1)+p)):notsogood,and“proof”isbuggy:theydidnotaccountforinteracBonsbetweendifferentprocessorsdoingsplihngalongoverlappingpaths.

Ourgoal

Simplealgorithmswithgoodworkbounds,sublinearinpifpossibleAnderson&Wollgaveasimplewait-freeimplementaBonoffindwithsplihngusingCAS,buttheiranalysisisnotcorrect.

Splihng:candoshortcutsviaCASbutconcurrentthreadscaninterfere

5

4

3

2

1

ACAScanfailbecauseanotherCASdoesalessfavorablechange:Avisits1,2,3thenstalls;Bvisits2,3,4,changesparentof2to4;Cvisits1,2,4,triestochangeparentof1to4butAwakesup,changesparentof1to3.

A C

B

OurResultsTwosplihngalgorithms:•  1-trysplihng:tryeachparentchangeoncebeforemovingtothenextnode

•  2-trysplihng:tryeachparentchangetwicebeforemovingtothenextnode

Wegetslightlybe~erboundsfor2-trysplihng

Linking

CandounrankedlinksusingCASbutDCASneededforrankedlinks

k

k

LinkingAlgorithms

Fourconcurrentlinkingalgorithms:•  Linkingbyrank(DCAS)•  Linkingbyrandomindex(CAS)•  Hybridlinkingbyrankviacoin-flipping(CAS)•  HybridlinkingbyrankviadeterminisBccoin-flipping(CAS)

OurBoundsWorst-caseBmeperoperaBon

O(logn)withorwithoutsplit/compressTotalworkwith“1-try”splihng

O(m(α(n,⎡m/(np2)⎤)+log(np2/m+1)))Totalworkwith“2-try”splihng

O(m(α(n,⎡m/(np)⎤)+log(np/m+1)))

Boundsareworst-casefordeterminisBclinking,Expectedforrandomizedlinking

Hybridlinking

link(v,w):ifvandwhaveequalrank,firstchangetheparentofv,ortherankofw?Flipafaircointodecide:whentryingtolinktwoequal-ranknodesvandw,withv<winsome(arbitrary)totalorder,ifheadstrytomakewtheparentofv;iftails,trytoadd1totherankofv

AnalysisTreedepthislogarithmic(worst-caseforlinkingbyrank,high-probabilityforlinkingbyrandomizedindexorhybrid}.ThisgivesO(logn)BmeperoperaBon.Workbound:Letd=m/(np)(2-trysplihng)ord=m/(np2)(1-trysplihng).Ifd≥1,thesequenBalanalysisextendstogiveaboundofO(α(n,d))workperfindplusO(pd)orO(p2d)workpernode(porp2BmesthesequenBalbound).ThetotalworkisO(mα(n,d)).

WorkboundconBnued

Theothercaseisd<1.Thenumberofnodesofrankatleastrisatmostn/2r.WeapplythesequenBalargumenttothenodesofrankatleastlg(1/d),ofwhichthereareatmostnd.ThesenodesaccountfortotalworkO(mα(n,1)+nkd)=O(mα(n,1)).Thelow-ranknodesonfindpathsaccountforanaddiBonalO(log(1/d))workperfind.

CurrentWork

ImplementsetswithvaluesandsetswithiteraBonwhilepreservingefficiency:seemstorequirerelaxinglinearizaBon.Idea:AllowsubsetsofuniteoperaBonstobereplacedbyequivalentsubsets,implementusingabifurcaBngqueue.

QuesBonsforthefuture

IsouramorBzedupperboundBghtfortheproblem?

Concurrentstrongcomponents?Otherconcurrentdatastructures:binarysearch

trees?

Thanks!