SHUFFLING CARDS AND STOPPING TIMES DAVID ALDOUS* Depurtment ofStutistics, University ofCalifornia, Berkeley,CA 94720 PERSI DIACONIS** Department ofStutistics,StanfordUniversity, Stanford,CA 94305 1.Introduction.How many times musta deck ofcards beshuffled until it is close to random? Thereisanelementarytechniquewhichoftenyieldssharpestimatesinsuchproblems.The methodis bestunderstoodthrough a simple example. EXAMPLE1.Topinatrandomshuffle.Considerthefollowingmethodofmixingadeckof cards: the topcard is removedand inserted into the deck at a random position. This procedure is repeatedanumberoftimes.Thefollowingargumentshouldconvincethereaderthatabout nlog nshuffles suffice to mix upncards. The argument depends on following the bottom card of thedeck.Thiscardstaysatthebottomuntilthefirsttime(TI) acardisinsertedbelowit. Standard calculations, reviewed below, imply this takes aboutnshuffles. As the shuffles continue, eventuallyasecond card is inserted below the original bottomcard (this takes aboutn/ 2further shuffles). Consider the instant (T,)that a second card is inserted below the original bottom card. The two cards under the original bottom card are equally likely to be in relative order low-high or high-low. Similarly, thefirst timeah r dcardis insertedbelowthe original bottomcard, each ofthe6 possibleordersofthe 3 bottomcards isequally likely. Nowconsider the first timeT,-,that the original bottomcard comes up to the top. Byan inductive argument, all (n - l ) !arrangements of thelowercards are equally likely. Whentheoriginal bottomcardisinsertedat random,attime T = q,-, + 1, thenalln!possible arrangements ofthe deck are equally likely. abccahuadc bcuahaddcd cahbddccau ddddcchbhh FIG. I. Exampleofrepeatedtopin at randomshuffles ofa4-card deck. Whentheoriginal bottomcardisatpositionkfromthe bottom,thewaitingtimeforanew card to be inserted below it is aboutn/ k.Thus the waiting timeT for the bottom card to come to I DavidAldousconfessestoaconventionalcareer,goingfromaPh.D.andResearchFellowshipatCambridge UniversitytotheStatisticsDepartmentatBerkeley,whereheisnowAssociateProfessor.Hedoesresearchin theoreticalandapplied probabilitytheory,andfor recreationhe plays volleyball (well), bridge(badly) and watches MontyPythonreruns. Persi Diaconis left High School at an earlyage to earna living asa magician andgambler,only later to become interestedinmathematicsandearnaPh.D.atHaward.AfteraspellatBellLabs,heisnowProfessorinthe Statistics Departmentat Stanford. He was an early recipientofa MacArthurFoundation award, and his wide range ofmathematicalinterestsispartlyreflectedinhisfirstbookGroupTheoryinStutistics.Heretainsaninterestin magicandtheexposure offraudulent psychics. *Research supportedby NationalScience FoundationGrant MCS80-02698. **Research supportedby NationalScience Foundation Grant MCS80-24649. ofG.Definethe, . . .[,, &' , , &' , , dent picksaccordingtothesame scheme yieldrandomelements 334 DAVIDALDOUS AND PERSIDIACONIS thetopand be insertedisabout This paperpresentsarigorous version oftheargumentandillustratesitsuseinavarietyof randomwalkproblems.Thenextsectionintroducesthebasicmathematicalsetup.Section3 detailsanumberofexamplesdrawnfromapplicationssuchascomputergeneratedpseudo randomnumbers.Section4treatsordinaryriffleshuffling,analyzingamodelintroducedby Gilbert, Shannon,andReeds. Section 5 explains asense in whichthe methodofstoppingtimes always worksandcomparesthistotwoothertechniques (Fourieranalysis andcoupling).Some open problemsare listed. 2.TheBasicSet-Up. Repeatedshuffling is besttreatedasrandomwalkonthepermutation groupS,,.Forlaterapplications,wetreatanarbitraryfinitegroupG.Givensomescheme for randomlypickingelementsofG,letQ(g)betheprobabilitythatgispicked.Thenumbers { Q(g) : gEG)area(probability)distribution:Q(g) > 0andZQ(g) = 1. Repeatedindepen- products Xo= identity Xl= 61 Xk = 6kXk-1= [ k t , - 1 . . .61. The randomvariablesXo, XI, X2, . . . , are therandomwalkon G withstep distribution Q. Thlnk ofXkas the position at timekofa randomly-moving particle. The distribution ofX2, that is the setofprobabilitiesP(X2 = g),gE G,is given by convolution ForQ(h)Q(ghV' ) is the chance that element hwas picked first andgh-'was picked second; for anyh,thismakestheproductequaltog.Similarly,P(Xk = g) =whereQk* isthe repeatedconvolution In modellingshuffling ofann-card deck, the state ofthe deck is representedas a permutation mE S,,, meaning that the card originally at positioniis now at positionm(i). In Example 1, G = S,,, and using cycle notation for permutationsm,Q(m)= 0, else. Here6,is a randomly chosen cycle,Xkis the state ofthe deck afterkshuffles, andQk*(m)is the chance thatthe state afterkshuffles ism. In Fig. 1, 6,= (3,2, I), [,= (3,2,1) andX2 = &':[,= (1,2, 3). Weshallstudy the distributionQk*.NotethatQk* can bedefined by (2.1) withoutusingthe richerstructureoftherandomwalk(X,);however,thisricherstructureisessentialforour methodofanalysis. Afundamental result isthat repeatedconvolutions converge tothe uniformdistribution U:unlessQisconcentratedonsomecosetofsomesubgroup.ThiswasfirstprovedbyMarkov (1906)-seeFeller (1968), Section 15.10 for a clear discussion-andcan nowadays be regarded as aspecialcaseofthebasic limittheoryoffiniteMarkovchains.Poincari. (1912)gaveaFourier 19861 SHUFFLING CARDSAND STOPPING TIMES335analytic proof, and subsequent workers have extended (2.2) to general compact groups-seeGrenander(1963),Heyer(1977),Diaconis(1982)forsurveys.A versionof thisresultisgivenhereasTheorem3of Section3.Despitethisworkonabstracting theasymptoticresult(2.2),littleattentionhasbeenpaid untilrecently tothekind of non-asymptoticquestionswhcharethesubjectof t hs paper.A naturalwaytomeasurethedifferencebetweentwoprobability distributionsQ,,Q, onG isby variationdistanceThere areequivalent definitionswhere Q(A)= C,,,Q(g), Q( f ) = Cf(g)Q(g),and I l f I I = maxlf(g)l. Thestringof equalitiesisproved by noting that the maxima occur for A = {g : Q,(g) > Q,(g)) and for f= 1, - 1,;Thus,twodistributionsarecloseinvariationdistanceif andonlyif theyareuniformlycloseonallsubsets.Plainly 0< llQ, - Q211< 1.An examplemaybeuseful.Suppose,afterwell-shufflingadeckof n cards,thatyouhappentosee thebottom card, c.Thenyour distribution Q on S,,isuniform onthe setof permutations mfor whch m(c)= n, and llQ - UII = 1- l / n. This shows the variation distance can be very"unforgiving" of smalldeviationsfromuniformity.Given adistribution Qonagroup G, (2.2)says def (2.4) d,(k) = 1lek*- UII+ 0 a s k + co. Where Q models arandom shuffle,d( k) measureshowclose k repeated shufflesgetthedecktobeingperfectly(uniformly)shuffled.Onemightsupposed( k) decreasessmoothlyfrom(near)1to0;anditisnothard to showd( k) isdecreasing.However,THEOREM 1.For the"top in atrandom" shuffle, Example1, (a) d( nlogn + cn)< ep'; allc> 0, n > 2.(b) d ( n 1 o g n - c , n ) + l a s n + w; al l c, , + w. Th s gives a sharpsensetothe assertionthat nlogn shufflesareenough.Ths is aparticularcase of a general cut-off phenomenon,which occursin all shufflingmodelswe havebeen able toanalyze;thereisacriticalnumber k, of shufflessuchthat d(k, + o(k,)) - 0but d(k, - o(k,))= 1.(SeeFig. 2.)336DAVID ALDOUS AND PERSIDIACONIS[May Our aim is to determine k,in particularcases. This is quite different from looking at asymptotics in(2.4):itiselementarythatd( k)-0geometrically fast,andPerron-Frobeniustheorysays d( k)- ahk,wherea,hhaveeigenvalue/eigenvectorinterpretation, buttheseasymptoticsmiss the cut-off phenomenon.For card players, the question is not "exactly how close to uniform is the deckafter a million riffleshuffles?", but"is7 shuffles enough?". The main purpose ofthis paperis to show how upperbounds ond(k),like (a) in Theorem 1, can beobtained using thenotionofstrong uniformtimes, whlch wenow define in two steps. DEFINITIONthesetofallG-valuedinfinitesequences1.LetGbeafinitegroup,andG" g= (g,, g,,. . .). Astopping rulef is a functionf :G"{1, 2, 3,. . ; m) such that iff(g)=j , -thenT(g)=j for all g with2,= g,,i k) ,all k2 0. Proof.For any Ac G and so 19861SHUFFLING CARDSAND STOPPING TIMES337 WeconcludethissectionbyusingLemma1 andelementaryprobabilityconceptstoprove Theorem 1.Here is one elementary result weshall usein several examples. LEMMA2.Sample uniformly withreplacement fromanurnwithn balls.LetV bethenumber ofdraws required until eachball hasbeendrawnat least once.Then Proof.Letm = n log n+ cn.Foreach ballbletA,betheevent"ballbnotdrawn inthe firstmdraws".Then REMARK.Thisisthefamous"coupon-collector'sproblem",discussedinFeller(1968).The asymptotics areP( V > nlog n+ cn) + 1- e ~ p ( - e - ~ ) asn+ oo,cfixed. So forcnotsmall the boundin Lemma2 is close to sharp. Proofof Theorem 1.Recall wehave argued thatT, the first time thatthe original bottomcard hascometothetopandbeeninsertedinto thedeck, isastronguniformtimeforthisshuffling scheme. We shall prove thatThas the same distribution asV in Lemma 2;then assertion (a) is a consequence ofLemmas 1and 2. Wecan write (2.5) T =TI+ (T2 - TI)+. . .+( T, - , - T,-2)+ ( T - T,-,), whereq. isthetime untiltheith cardis placedunderthe original bottomcard. When exactlyi cards are under the original bottom cardb,the chance that the current top card is inserted below i + 1bis-, and hence the random variableq+, - has geometric distribution n The random variableV inLemma 2 can be writtenas (2.7) v =( v - K-,)+ (v, -l- K-,)+ . . .+( v2 - vl)+vl, whereisthenumberofdraws requireduntilidistinctballshavebeendrawnatleastonce. Afteridistinctballs have beendrawn, thechance thatadraw producesanot-previously-drawn n - iballis-. So v - v-,has distribution n P ( y- v-,=j ) = -n - i1 n) - lj-1 ;j > l .nI Comparing with(2.6), wesee that corresponding terms ( T+l- ?I)and ( K-, - K -,-,)have the same distribution; since thesummands within each of(2.5) and(2.7) are independent, itfollows that the sumsTandV have the same distribution, as required. To prove(b),fixj and letA, bethe set ofconfigurations ofthe deck such thatthe bottomj original cards remain in their original relative order. Plainly U(A,)= l / j ! Letk= k(n) be ofthe formnlog n- c,n,c,+ oo.We shall show (2.8) Q~*( A, )+ 1a s n + oo;jfixed.Thend( k)2max {ek*(A,) - U(A,))+ 1asn+ oo,establishing part(b). jTo prove(2.8), observe thatek*(A,) 2 P( T - ?;-, > k).ForT - I ; - , isdistributed asthe 338DAVID ALDOUS AND PERSIDIACONIS[May timefor the card initially j t h from bottomto come to thetopand be inserted; and ifthis has not occurred by timek,then the original bottomj cards must still be in their original relative order at timek. Thus itsuffices toshow (2.9)~ ( T - ? ; - , < k ) + o a s n + w ; jfixed. Weshall provethis usingChebyshev'sinequality: var( z )P( I Z - EZI > a ),< ---- ,wherea> 0, andZ is any random variable. a 2 From (2.6), and so from (2.5) n - 1E( T- T, )=C7= nl ogn+ O( n) ,i =j ' + 1 andChebyshev's inequality applied toZ = T - ?;_, readily yields (2.9). REMARK. Note that the "strong uniform time"property ofT played no role in establishing the lowerbound(b). Essentially, weget lower bounds byguessing some setAfor whichlQk*(A)-U(A) Ishould be large, and using the obvious (from (2.3)) inequality d( k)= 1lQk*- UII> I ~ ~ * ( A ) - U(A)I. 3.Examples.We present constructions ofstrong uniform times for a variety ofrandom walks: simplerandomwalkonthecircle, general randomwalksonfinitegroups,andarandomwalk arising in random numbergeneration. Sometimes our arguments give the optimal rate, oftenthey give the correctorder ofmagnitude. EXAMPLE2.Simple randomwalkonthe integers mod n.Letn be a positive odd integer. LetZ,, be the integers mod n, thought ofasn points on a circle. Imagine a particle which moves by steps, eachstep beingequally likely to move 1tothe rightor1 to the left. Thls randomwalk hasstep distribution QonZ,,; 1 (3.1)Q(1)= Q(-1)=- .2 Thefollowing theorem shows thatthenumber ofstepskrequiredto become uniformis slightly morethann2. THEOREM 2.Let n> 3 be an odd integer.For simple randomwalkonthe integers mod n defined by(3.1), fork> n2, d( k) G6e-ak/112 witha= 4.rr2/3. Proof.First considern = 5and the following 5 patterns Asequenceofsuccessive steps ofthe walkonZ,yieldsasequence off symbols. Consider the sequence in disjoint blocks of4.Stop the first timeTthata block of4 equals one ofthe above 5 patterns. Thus, ifthe sequence starts+ + - +, + + + -, + + - -, T = 12. 19861SHUFFLINGCARDSAND STOPPING TIMES339 This stopping time is clearly a strong uniformtime; given thatT = 12, all 5final positions in Z,are equally likely. Such sets ofk-tuples can bechosen forany oddn. Itturns out that to get thecorrectrateofconvergence,kshouldbechosenasalargemultipleofn2. Herearesome details. Forfixedintegersnandk, withnodd,letB, bethesetofbinaryk-tupleswithjpluses (mod M). Letj *betheindexcorrespondingtothesmallestIB,,1. Partitionthesetofbinaryk-tuples intongroupsofsizeIB,*l,thej t hgroupbeingchosenarbitrarilyfromB,.Therandomwalk generates a sequence ofsymbols. Consider these in disjoint blocks oflengthk. DefineTas the firsttimeablockequals one ofthechosen group. This clearly yieldsastrong uniformtime. The following lemma gives an explicit upperboundford(k).LEMMA3.LetT beas defined above.For n 2 3and k2 n2, P ( T> k)g6 e - 0 ~ 1 " ~witha= 4.rr2/3. Proof.The number ofelements inB,is thisbeingaclassical identityduetoC.Ramus (see Knuth(1973, p.70)). The chance ofagiven blockfalling in the chosen group equals Now x Straightforwardcalculususingquadraticapproximationstocosinesuchascos x< 1- -,