SHUFFLING CARDS AND STOPPING TIMES DAVID ALDOUS* Depurtment
ofStutistics, University ofCalifornia, Berkeley,CA 94720 PERSI
DIACONIS** Department ofStutistics,StanfordUniversity, Stanford,CA
94305 1.Introduction.How many times musta deck ofcards beshuffled
until it is close to random?
Thereisanelementarytechniquewhichoftenyieldssharpestimatesinsuchproblems.The
methodis bestunderstoodthrough a simple example.
EXAMPLE1.Topinatrandomshuffle.Considerthefollowingmethodofmixingadeckof
cards: the topcard is removedand inserted into the deck at a random
position. This procedure is
repeatedanumberoftimes.Thefollowingargumentshouldconvincethereaderthatabout
nlog nshuffles suffice to mix upncards. The argument depends on
following the bottom card of
thedeck.Thiscardstaysatthebottomuntilthefirsttime(TI)
acardisinsertedbelowit. Standard calculations, reviewed below,
imply this takes aboutnshuffles. As the shuffles continue,
eventuallyasecond card is inserted below the original bottomcard
(this takes aboutn/ 2further shuffles). Consider the instant
(T,)that a second card is inserted below the original bottom card.
The two cards under the original bottom card are equally likely to
be in relative order low-high or high-low. Similarly, thefirst
timeah r dcardis insertedbelowthe original bottomcard, each ofthe6
possibleordersofthe 3 bottomcards isequally likely. Nowconsider the
first timeT,-,that the original bottomcard comes up to the top.
Byan inductive argument, all (n - l ) !arrangements of
thelowercards are equally likely. Whentheoriginal
bottomcardisinsertedat random,attime T = q,-, + 1,
thenalln!possible arrangements ofthe deck are equally likely.
abccahuadc bcuahaddcd cahbddccau ddddcchbhh FIG. I.
Exampleofrepeatedtopin at randomshuffles ofa4-card deck.
Whentheoriginal bottomcardisatpositionkfromthe
bottom,thewaitingtimeforanew card to be inserted below it is
aboutn/ k.Thus the waiting timeT for the bottom card to come to I
DavidAldousconfessestoaconventionalcareer,goingfromaPh.D.andResearchFellowshipatCambridge
UniversitytotheStatisticsDepartmentatBerkeley,whereheisnowAssociateProfessor.Hedoesresearchin
theoreticalandapplied probabilitytheory,andfor recreationhe plays
volleyball (well), bridge(badly) and watches MontyPythonreruns.
Persi Diaconis left High School at an earlyage to earna living asa
magician andgambler,only later to become
interestedinmathematicsandearnaPh.D.atHaward.AfteraspellatBellLabs,heisnowProfessorinthe
Statistics Departmentat Stanford. He was an early recipientofa
MacArthurFoundation award, and his wide range
ofmathematicalinterestsispartlyreflectedinhisfirstbookGroupTheoryinStutistics.Heretainsaninterestin
magicandtheexposure offraudulent psychics. *Research supportedby
NationalScience FoundationGrant MCS80-02698. **Research supportedby
NationalScience Foundation Grant MCS80-24649. ofG.Definethe, . .
.[,, &' , , &' , , dent picksaccordingtothesame scheme
yieldrandomelements 334 DAVIDALDOUS AND PERSIDIACONIS thetopand be
insertedisabout This paperpresentsarigorous version
oftheargumentandillustratesitsuseinavarietyof
randomwalkproblems.Thenextsectionintroducesthebasicmathematicalsetup.Section3
detailsanumberofexamplesdrawnfromapplicationssuchascomputergeneratedpseudo
randomnumbers.Section4treatsordinaryriffleshuffling,analyzingamodelintroducedby
Gilbert, Shannon,andReeds. Section 5 explains asense in whichthe
methodofstoppingtimes always
worksandcomparesthistotwoothertechniques (Fourieranalysis
andcoupling).Some open problemsare listed. 2.TheBasicSet-Up.
Repeatedshuffling is besttreatedasrandomwalkonthepermutation
groupS,,.Forlaterapplications,wetreatanarbitraryfinitegroupG.Givensomescheme
for
randomlypickingelementsofG,letQ(g)betheprobabilitythatgispicked.Thenumbers
{ Q(g) : gEG)area(probability)distribution:Q(g) > 0andZQ(g) = 1.
Repeatedindepen- products Xo= identity Xl= 61 Xk = 6kXk-1= [ k t ,
- 1 . . .61. The randomvariablesXo, XI, X2, . . . , are
therandomwalkon G withstep distribution Q. Thlnk ofXkas the
position at timekofa randomly-moving particle. The distribution
ofX2, that is the setofprobabilitiesP(X2 = g),gE G,is given by
convolution ForQ(h)Q(ghV' ) is the chance that element hwas picked
first andgh-'was picked second; for
anyh,thismakestheproductequaltog.Similarly,P(Xk = g) =whereQk*
isthe repeatedconvolution In modellingshuffling ofann-card deck,
the state ofthe deck is representedas a permutation mE S,,, meaning
that the card originally at positioniis now at positionm(i). In
Example 1, G = S,,, and using cycle notation for
permutationsm,Q(m)= 0, else. Here6,is a randomly chosen cycle,Xkis
the state ofthe deck afterkshuffles, andQk*(m)is the chance thatthe
state afterkshuffles ism. In Fig. 1, 6,= (3,2, I), [,= (3,2,1)
andX2 = &':[,= (1,2, 3). Weshallstudy the
distributionQk*.NotethatQk* can bedefined by (2.1) withoutusingthe
richerstructureoftherandomwalk(X,);however,thisricherstructureisessentialforour
methodofanalysis. Afundamental result isthat repeatedconvolutions
converge tothe uniformdistribution
U:unlessQisconcentratedonsomecosetofsomesubgroup.ThiswasfirstprovedbyMarkov
(1906)-seeFeller (1968), Section 15.10 for a clear
discussion-andcan nowadays be regarded as aspecialcaseofthebasic
limittheoryoffiniteMarkovchains.Poincari. (1912)gaveaFourier 19861
SHUFFLING CARDSAND STOPPING TIMES335analytic proof, and subsequent
workers have extended (2.2) to general compact
groups-seeGrenander(1963),Heyer(1977),Diaconis(1982)forsurveys.A
versionof thisresultisgivenhereasTheorem3of
Section3.Despitethisworkonabstracting
theasymptoticresult(2.2),littleattentionhasbeenpaid untilrecently
tothekind of non-asymptoticquestionswhcharethesubjectof t hs
paper.A naturalwaytomeasurethedifferencebetweentwoprobability
distributionsQ,,Q, onG isby variationdistanceThere areequivalent
definitionswhere Q(A)= C,,,Q(g), Q( f ) = Cf(g)Q(g),and I l f I I =
maxlf(g)l. Thestringof equalitiesisproved by noting that the maxima
occur for A = {g : Q,(g) > Q,(g)) and for f= 1, -
1,;Thus,twodistributionsarecloseinvariationdistanceif andonlyif
theyareuniformlycloseonallsubsets.Plainly 0< llQ, - Q211<
1.An examplemaybeuseful.Suppose,afterwell-shufflingadeckof n
cards,thatyouhappentosee thebottom card, c.Thenyour distribution Q
on S,,isuniform onthe setof permutations mfor whch m(c)= n, and llQ
- UII = 1- l / n. This shows the variation distance can be
very"unforgiving" of smalldeviationsfromuniformity.Given
adistribution Qonagroup G, (2.2)says def (2.4) d,(k) = 1lek*- UII+
0 a s k + co. Where Q models arandom shuffle,d( k) measureshowclose
k repeated
shufflesgetthedecktobeingperfectly(uniformly)shuffled.Onemightsupposed(
k) decreasessmoothlyfrom(near)1to0;anditisnothard to showd( k)
isdecreasing.However,THEOREM 1.For the"top in atrandom" shuffle,
Example1, (a) d( nlogn + cn)< ep'; allc> 0, n > 2.(b) d (
n 1 o g n - c , n ) + l a s n + w; al l c, , + w. Th s gives a
sharpsensetothe assertionthat nlogn shufflesareenough.Ths is
aparticularcase of a general cut-off phenomenon,which occursin all
shufflingmodelswe havebeen able toanalyze;thereisacriticalnumber k,
of shufflessuchthat d(k, + o(k,)) - 0but d(k, - o(k,))= 1.(SeeFig.
2.)336DAVID ALDOUS AND PERSIDIACONIS[May Our aim is to determine
k,in particularcases. This is quite different from looking at
asymptotics in(2.4):itiselementarythatd( k)-0geometrically
fast,andPerron-Frobeniustheorysays d( k)-
ahk,wherea,hhaveeigenvalue/eigenvectorinterpretation,
buttheseasymptoticsmiss the cut-off phenomenon.For card players,
the question is not "exactly how close to uniform is the deckafter
a million riffleshuffles?", but"is7 shuffles enough?". The main
purpose ofthis paperis to show how upperbounds ond(k),like (a) in
Theorem 1, can beobtained using thenotionofstrong uniformtimes,
whlch wenow define in two steps.
DEFINITIONthesetofallG-valuedinfinitesequences1.LetGbeafinitegroup,andG"
g= (g,, g,,. . .). Astopping rulef is a functionf :G"{1, 2, 3,. . ;
m) such that iff(g)=j , -thenT(g)=j for all g with2,= g,,i k) ,all
k2 0. Proof.For any Ac G and so 19861SHUFFLING CARDSAND STOPPING
TIMES337 WeconcludethissectionbyusingLemma1
andelementaryprobabilityconceptstoprove Theorem 1.Here is one
elementary result weshall usein several examples. LEMMA2.Sample
uniformly withreplacement fromanurnwithn balls.LetV bethenumber
ofdraws required until eachball hasbeendrawnat least once.Then
Proof.Letm = n log n+ cn.Foreach ballbletA,betheevent"ballbnotdrawn
inthe firstmdraws".Then
REMARK.Thisisthefamous"coupon-collector'sproblem",discussedinFeller(1968).The
asymptotics areP( V > nlog n+ cn) + 1- e ~ p ( - e - ~ ) asn+
oo,cfixed. So forcnotsmall the boundin Lemma2 is close to sharp.
Proofof Theorem 1.Recall wehave argued thatT, the first time
thatthe original bottomcard hascometothetopandbeeninsertedinto
thedeck, isastronguniformtimeforthisshuffling scheme. We shall
prove thatThas the same distribution asV in Lemma 2;then assertion
(a) is a consequence ofLemmas 1and 2. Wecan write (2.5) T =TI+ (T2
- TI)+. . .+( T, - , - T,-2)+ ( T - T,-,), whereq. isthetime
untiltheith cardis placedunderthe original bottomcard. When
exactlyi cards are under the original bottom cardb,the chance that
the current top card is inserted below i + 1bis-, and hence the
random variableq+, - has geometric distribution n The random
variableV inLemma 2 can be writtenas (2.7) v =( v - K-,)+ (v, -l-
K-,)+ . . .+( v2 - vl)+vl, whereisthenumberofdraws
requireduntilidistinctballshavebeendrawnatleastonce.
Afteridistinctballs have beendrawn, thechance thatadraw
producesanot-previously-drawn n - iballis-. So v - v-,has
distribution n P ( y- v-,=j ) = -n - i1 n) - lj-1 ;j > l .nI
Comparing with(2.6), wesee that corresponding terms ( T+l- ?I)and (
K-, - K -,-,)have the same distribution; since thesummands within
each of(2.5) and(2.7) are independent, itfollows that the sumsTandV
have the same distribution, as required. To prove(b),fixj and letA,
bethe set ofconfigurations ofthe deck such thatthe bottomj original
cards remain in their original relative order. Plainly U(A,)= l / j
! Letk= k(n) be ofthe formnlog n- c,n,c,+ oo.We shall show (2.8)
Q~*( A, )+ 1a s n + oo;jfixed.Thend( k)2max {ek*(A,) - U(A,))+
1asn+ oo,establishing part(b). jTo prove(2.8), observe thatek*(A,)
2 P( T - ?;-, > k).ForT - I ; - , isdistributed asthe 338DAVID
ALDOUS AND PERSIDIACONIS[May timefor the card initially j t h from
bottomto come to thetopand be inserted; and ifthis has not occurred
by timek,then the original bottomj cards must still be in their
original relative order at timek. Thus itsuffices toshow (2.9)~ ( T
- ? ; - , < k ) + o a s n + w ; jfixed. Weshall provethis
usingChebyshev'sinequality: var( z )P( I Z - EZI > a ),< ----
,wherea> 0, andZ is any random variable. a 2 From (2.6), and so
from (2.5) n - 1E( T- T, )=C7= nl ogn+ O( n) ,i =j ' + 1
andChebyshev's inequality applied toZ = T - ?;_, readily yields
(2.9). REMARK. Note that the "strong uniform time"property ofT
played no role in establishing the lowerbound(b). Essentially,
weget lower bounds byguessing some setAfor whichlQk*(A)-U(A)
Ishould be large, and using the obvious (from (2.3)) inequality d(
k)= 1lQk*- UII> I ~ ~ * ( A ) - U(A)I. 3.Examples.We present
constructions ofstrong uniform times for a variety ofrandom walks:
simplerandomwalkonthecircle, general
randomwalksonfinitegroups,andarandomwalk arising in random
numbergeneration. Sometimes our arguments give the optimal rate,
oftenthey give the correctorder ofmagnitude. EXAMPLE2.Simple
randomwalkonthe integers mod n.Letn be a positive odd integer.
LetZ,, be the integers mod n, thought ofasn points on a circle.
Imagine a particle which moves by steps, eachstep beingequally
likely to move 1tothe rightor1 to the left. Thls randomwalk hasstep
distribution QonZ,,; 1 (3.1)Q(1)= Q(-1)=- .2 Thefollowing theorem
shows thatthenumber ofstepskrequiredto become uniformis slightly
morethann2. THEOREM 2.Let n> 3 be an odd integer.For simple
randomwalkonthe integers mod n defined by(3.1), fork> n2, d( k)
G6e-ak/112 witha= 4.rr2/3. Proof.First considern = 5and the
following 5 patterns Asequenceofsuccessive steps ofthe
walkonZ,yieldsasequence off symbols. Consider the sequence in
disjoint blocks of4.Stop the first timeTthata block of4 equals one
ofthe above 5 patterns. Thus, ifthe sequence starts+ + - +, + + +
-, + + - -, T = 12. 19861SHUFFLINGCARDSAND STOPPING TIMES339 This
stopping time is clearly a strong uniformtime; given thatT = 12,
all 5final positions in Z,are equally likely. Such sets ofk-tuples
can bechosen forany oddn. Itturns out that to get
thecorrectrateofconvergence,kshouldbechosenasalargemultipleofn2.
Herearesome details. Forfixedintegersnandk, withnodd,letB,
bethesetofbinaryk-tupleswithjpluses (mod M). Letj
*betheindexcorrespondingtothesmallestIB,,1.
Partitionthesetofbinaryk-tuples intongroupsofsizeIB,*l,thej t
hgroupbeingchosenarbitrarilyfromB,.Therandomwalk generates a
sequence ofsymbols. Consider these in disjoint blocks oflengthk.
DefineTas the firsttimeablockequals one ofthechosen group. This
clearly yieldsastrong uniformtime. The following lemma gives an
explicit upperboundford(k).LEMMA3.LetT beas defined above.For n 2
3and k2 n2, P ( T> k)g6 e - 0 ~ 1 " ~witha= 4.rr2/3. Proof.The
number ofelements inB,is thisbeingaclassical identityduetoC.Ramus
(see Knuth(1973, p.70)). The chance ofagiven blockfalling in the
chosen group equals Now x
Straightforwardcalculususingquadraticapproximationstocosinesuchascos
x< 1- -,
