Peer Pressure: Distributed Recovery in GnutellaPedram KeyaniBrian LarsonMuthukumar Senthil

Computer Science DepartmentStanford University

IntroductionGnutella is a P2P file sharing protocolThe issue we are addressing is distributed recovery from malicious attacks in GnutellaOur solution is a mechanism for proactive failure detection and recoveryOur experimental process and modelsThe fruits of our labor: RESULTS!

Failure in GnutellaFailure of nodes in Gnutella can be caused by any number of reasonsFailure of 4% of the most highly connected nodes in Gnutella fragments the network to the point where it is unusable by anyone The exact details of this are outlined in work done by Stefan Saroiu

Scale Free Networks (Gnutella, Internet)Abide by power law where# of nodes of degree N is proportional to N -lambdaLambda is observed to be roughly 2.3Scale Free networks are highly resilient to large scale random failures but weak for malicious attacks on the most highly connected well known nodes

Chart1

5

1.0153154954

0.3995683852

0.2061731111

0.2061731111

0.1234067725

0.0811375946

0.0569173291

0.0418661509

0.0319309789

0.0250593617

Number of Neighbors

Number of Nodes

Power Law Distribution K=5, Lambda = 2.3

Sheet1

15

21.0153154954

30.3995683852

40.2061731111

40.2061731111

50.1234067725

60.0811375946

70.0569173291

80.0418661509

90.0319309789

100.0250593617

110.0201264338

120.0164760514

130.0137056707

140.0115578092

150.009861889

160.0085014703

170.0073949882

180.0064840035

190.0057258072

200.0050886316

210.0045484731

220.004086936

230.0036897418

240.0033456781

Sheet1

0

0

0

0

0

0

0

0

0

0

0

Number of Neighbors

Number of Nodes

Power Law Distribution K=5, Lambda = 2.3

Sheet2

Sheet3

Exponential NetworksConnections between nodes are randomNo preferential connections ensures no node holds the entire network togetherThey react the same way to malicious attacks and random failures

Scale Free and Exponential

Our HypothesisIn order to allow Gnutella to recover from malicious attacks nodes must plan for failures by discovering and maintaining backup connections to form an exponential network. These backups will be used to replace dead neighbors in the case of a malicious attack.

Recovery MethodBuild and maintain a virtual exponential network connecting all the nodesAccomplish this through random node discoveryDetect malicious attacks on active networkSwitch over to exponential network

Random Node DiscoveryProblem: no centralized name authority to give a truly random nodeSolution: use random walks through the network to arrive at random nodeRandom Discovery Ping (RDP) is forwarded to only one of a nodes neighbors, selected in such a way to give a random distributionRDPs use a hop count of 20, roughly equal to the network diameter

Maintenance of Virtual Exponential NetworkEach node discovers N random nodes, where N is the minimum number of connections the node wants to maintainThen periodically ping these nodes to make sure they are aliveDiscover new neighbors to replace them should they die

Failure DetectionRandom failures result in loss of 1st degree neighborsMalicious attacks result in greater loss of 2nd degree neighbors than 1st degreeKeep a history (30 seconds) of 1st and 2nd degree neighbor lossIf 2nd degree loss exceeds 1st degree loss and a threshold (50%), mark as malicious

Reacting to FailuresFor each neighbor lost, replace it with a node from the virtual exponential networkOnly nodes local to an attack will switch, preserving the rest of the network structureDo not attempt to discover additional random nodes during an attackWhen attack is deemed to be over, return to normal operations

P2P SimulatorGeneralized P2P network simulatorHandles message routing, time managementSupport for bringing nodes up or down, injecting failures, loggingAlso created a compatible Gnutella client, and our enhanced Gnutella clientAbout 5k lines of Java

Modeling GnutellaNo standard way to do thisProtocol only specifies message formatsClients free to implement other aspectsSome degree of standardizationWe used the most common client in our simulation model - Bearshare

BootstrappingHow do nodes connect in our simulation?Defunct www.gnutellahosts.comMaintain list of highly-available, well-connected nodesClients connect by receiving one of these nodesBearshare clients do something similarConnect to service pubic.bearshare.netKeep a range of neighbors (3-10)

Uptime DistributionHow long do nodes stay up in our simulation?Modeled by a power law functionMost nodes are up for a short period of time, few are up for a long periodMany users just sign off after getting their contentMost users are dialup usersWithin a reasonable time slice, nodes have uptimes following the power law distribution

Our ExperimentsRan with recovery method and without No failures just ran our simulator without removing any nodes (control)Malicious attack on most highly connected nodes

Malicious AttackRan the experiment for 10 minutesWe removed 5% of the most highly connected nodes over a 5 minute interval in the middleRepresentative of a coordinated distributed attack on the network

MetricsLarge number of metrics that we could have usedWe picked metrics that measureHow partitioned the network isHow useful the network is in sending queries

Size of Largest Connected ComponentLargest set of nodes V, where any vm and vn V have a path between each otherMeasures the number of nodes that can potentially communicate with each otherCan get any data from any other node

# of Connected ComponentsNumber of separate pieces of the networkIf number of CCs is large then the network is heavily partitionedNot possible to retrieve content between CCsWant to monitor this number to make sure it is not increasing

Nodes Reachable Within 6 HopsSum of number of 1st, 2nd . . ., 6th degree neighbors of a nodeEnd to end measurement of how many nodes you can reach with a queryTypically queries are forwarded about 6 nodesRough estimate of the number of nodes a user can search.

Results Largest CC

Chart1

0.99983459820.00005494960.00005494960.99983497560.0000385410.000038541

0.99985973030.00002429940.00002429940.99982977260.00005331970.0000533197

0.99982982270.00006040870.00006040870.99980469770.00007255430.0000725543

0.99974460040.00006412060.00006412060.99982975740.00004682460.0000468246

0.99986959150.00005910450.00005910450.99981503340.00006493090.0000649309

0.99980460850.00006779960.00006779960.99986483990.0000389090.000038909

0.99987468520.00004216830.00004216830.99984495430.00004471040.0000447104

0.9998395520.00004605320.00004605320.99981468880.00008295720.0000829572

0.99982450610.000067720.000067720.99980479990.00005150880.0000515088

0.99980444990.00005774810.00005774810.99987479340.00003945010.0000394501

0.99982977240.0000605650.0000605650.99982474960.00004680490.0000468049

0.9998544820.0000665350.0000665350.99980992440.00006143540.0000614354

0.99981964220.00003643980.00003643980.99982973510.00006267530.0000626753

0.99320055340.00308886160.00308886160.33273752650.03531952740.0353195274

0.9965893140.00158189120.00158189120.23730294030.0191217420.019121742

0.99760707750.00186783290.00186783290.18582906440.03519528060.0351952806

0.99888470970.00056877830.00056877830.14214210730.032063460.03206346

0.99939845240.00042094510.00042094510.10021101760.02190499350.0219049935

0.99985329250.00006168910.00006168910.08397148570.02205775160.0220577516

0.99972614630.00007590190.00007590190.07378161880.01385080340.0138508034

0.99971568020.00008410.00008410.06690181110.01349456210.0134945621

0.99966434550.00007144250.00007144250.06387852140.01292318370.0129231837

0.99947431350.0004561040.0004561040.05362399410.01249720370.0124972037

0.99965293480.00004407720.00004407720.05309430110.01263263960.0126326396

0.99967286820.00008311890.00008311890.04995975960.01260103540.0126010354

0.99965671840.00009263040.00009263040.04751102180.01160186670.0116018667

0.99968700460.00006421610.00006421610.04271776390.01067691940.0106769194

0.99969182010.00006678530.00006678530.04108789520.01158211420.0115821142

0.99964456710.00009238820.00009238820.03707467940.01000440270.0100044027

0.99968543790.00006443140.00006443140.03342026490.00990901020.0099090102

0.99969003210.00005237990.00005237990.03209062720.0101978670.010197867

0.9997515370.00004494540.00004494540.03082146880.00877820410.0087782041

0.99965796420.00007854240.00007854240.02923052250.00802548920.0080254892

0.99967803250.00010798730.00010798730.02613284360.00654260110.0065426011

0.99961498890.0000484940.0000484940.02566191750.0063174990.006317499

0.99976536460.00006342530.00006342530.02569520510.00631271860.0063127186

0.99957751250.00009843070.00009843070.02507311580.00589930960.0058993096

0.999612950.0000632710.0000632710.02419786180.00543531830.0054353183

0.99952319560.00006905520.00006905520.0209883860.00317637540.0031763754

0.99956934020.00011081840.00011081840.01985000650.00246800860.0024680086

0.99965297450.00007704670.00007704670.01930733550.00255300190.0025530019

0.99960475440.00007624450.00007624450.01929850410.00254827620.0025482762

0.9995567420.0000913430.0000913430.01901165340.00278219690.0027821969

0.99987845480.00005586970.00005586970.0190351950.00278128040.0027812804

0.9998523940.00003702460.00003702460.0190261110.00277924420.0027792442

0.99985234370.00004568880.00004568880.01901628820.00276554730.0027655473

0.99985237410.00006126540.00006126540.01903540750.00276971480.0027697148

0.99985206850.00004842750.00004842750.01889918230.00294349560.0029434956

0.99985719480.00005390460.00005390460.01871769080.00300231210.0030023121

0.99985759270.00004874830.00004874830.01872692960.00300478340.0030047834

0.99985745080.00007435920.00007435920.01856325760.00282633960.0028263396

0.99987349190.00003482940.00003482940.01858259890.00283983760.0028398376

0.9998259180.00005122710.00005122710.01857468940.00284721480.0028472148

0.99989423120.00004643670.00004643670.01843247150.00292928840.0029292884

0.99984695750.00006430290.00006430290.01841725560.00292719950.0029271995

0.99985225130.00005502460.00005502460.01840999830.00291571640.0029157164

0.99981511860.0000792340.0000792340.01839787070.00289405460.0028940546

0.99979933130.00008141110.00008141110.01837324280.00290302830.0029030283

0.99981015650.00005765830.00005765830.01835556110.00291343810.0029134381

0.99985757490.00004870490.00004870490.01760818470.00291868080.0029186808

0.99980464220.00006907850.00006907850.0173186090.00295944910.0029594491

Smart Recovery

No Recovery

Time (minutes)

Percentage of Nodes

% of nodes in the largest connected component

Sheet1

0:00

0:10

0:20

0:30

0:40

0:50

1:00

1:10

1:20

1:30

1:40

1:50

start

2:10

2:20

2:30

2:40

2:50

3:00

3:10

3:20

3:30

3:40

3:50

4:00

4:10

4:20

4:30

4:40

4:50

5:00

5:10

5:20

5:30

5:40

5:50

6:00

6:10

6:20

6:30

6:40

6:50

end

7:10

7:20

7:30

7:40

7:50

8:00

8:10

8:20

8:30

8:40

8:50

9:00

9:10

9:20

9:30

9:40

9:50

10:00

Sheet1

0.99983459820.00005494960.00005494960.99983497560.0000385410.000038541

0.99985973030.00002429940.00002429940.99982977260.00005331970.0000533197

0.99982982270.00006040870.00006040870.99980469770.00007255430.0000725543

0.99974460040.00006412060.00006412060.99982975740.00004682460.0000468246

0.99986959150.00005910450.00005910450.99981503340.00006493090.0000649309

0.99980460850.00006779960.00006779960.99986483990.0000389090.000038909

0.99987468520.00004216830.00004216830.99984495430.00004471040.0000447104

0.9998395520.00004605320.00004605320.99981468880.00008295720.0000829572

0.99982450610.000067720.000067720.99980479990.00005150880.0000515088

0.99980444990.00005774810.00005774810.99987479340.00003945010.0000394501

0.99982977240.0000605650.0000605650.99982474960.00004680490.0000468049

0.9998544820.0000665350.0000665350.99980992440.00006143540.0000614354

0.99981964220.00003643980.00003643980.99982973510.00006267530.0000626753

0.99320055340.00308886160.00308886160.33273752650.03531952740.0353195274

0.9965893140.00158189120.00158189120.23730294030.0191217420.019121742

0.99760707750.00186783290.00186783290.18582906440.03519528060.0351952806

0.99888470970.00056877830.00056877830.14214210730.032063460.03206346

0.99939845240.00042094510.00042094510.10021101760.02190499350.0219049935

0.99985329250.00006168910.00006168910.08397148570.02205775160.0220577516

0.99972614630.00007590190.00007590190.07378161880.01385080340.0138508034

0.99971568020.00008410.00008410.06690181110.01349456210.0134945621

0.99966434550.00007144250.00007144250.06387852140.01292318370.0129231837

0.99947431350.0004561040.0004561040.05362399410.01249720370.0124972037

0.99965293480.00004407720.00004407720.05309430110.01263263960.0126326396

0.99967286820.00008311890.00008311890.04995975960.01260103540.0126010354

0.99965671840.00009263040.00009263040.04751102180.01160186670.0116018667

0.99968700460.00006421610.00006421610.04271776390.01067691940.0106769194

0.99969182010.00006678530.00006678530.04108789520.01158211420.0115821142

0.99964456710.00009238820.00009238820.03707467940.01000440270.0100044027

0.99968543790.00006443140.00006443140.03342026490.00990901020.0099090102

0.99969003210.00005237990.00005237990.03209062720.0101978670.010197867

0.9997515370.00004494540.00004494540.03082146880.00877820410.0087782041

0.99965796420.00007854240.00007854240.02923052250.00802548920.0080254892

0.99967803250.00010798730.00010798730.02613284360.00654260110.0065426011

0.99961498890.0000484940.0000484940.02566191750.0063174990.006317499

0.99976536460.00006342530.00006342530.02569520510.00631271860.0063127186

0.99957751250.00009843070.00009843070.02507311580.00589930960.0058993096

0.999612950.0000632710.0000632710.02419786180.00543531830.0054353183

0.99952319560.00006905520.00006905520.0209883860.00317637540.0031763754

0.99956934020.00011081840.00011081840.01985000650.00246800860.0024680086

0.99965297450.00007704670.00007704670.01930733550.00255300190.0025530019

0.99960475440.00007624450.00007624450.01929850410.00254827620.0025482762

0.9995567420.0000913430.0000913430.01901165340.00278219690.0027821969

0.99987845480.00005586970.00005586970.0190351950.00278128040.0027812804

0.9998523940.00003702460.00003702460.0190261110.00277924420.0027792442

0.99985234370.00004568880.00004568880.01901628820.00276554730.0027655473

0.99985237410.00006126540.00006126540.01903540750.00276971480.0027697148

0.99985206850.00004842750.00004842750.01889918230.00294349560.0029434956

0.99985719480.00005390460.00005390460.01871769080.00300231210.0030023121

0.99985759270.00004874830.00004874830.01872692960.00300478340.0030047834

0.99985745080.00007435920.00007435920.01856325760.00282633960.0028263396

0.99987349190.00003482940.00003482940.01858259890.00283983760.0028398376

0.9998259180.00005122710.00005122710.01857468940.00284721480.0028472148

0.99989423120.00004643670.00004643670.01843247150.00292928840.0029292884

0.99984695750.00006430290.00006430290.01841725560.00292719950.0029271995

0.99985225130.00005502460.00005502460.01840999830.00291571640.0029157164

0.99981511860.0000792340.0000792340.01839787070.00289405460.0028940546

0.99979933130.00008141110.00008141110.01837324280.00290302830.0029030283

0.99981015650.00005765830.00005765830.01835556110.00291343810.0029134381

0.99985757490.00004870490.00004870490.01760818470.00291868080.0029186808

0.99980464220.00006907850.00006907850.0173186090.00295944910.0029594491

Smart Recovery

No Recovery

Time (minutes)

Percentage of Nodes

% of nodes in the largest connected component

Sheet2

4.30.77577494560.77577494564.31.09516448881.0951644888

4.41.06151994031.06151994033.80.48890019080.4889001908

4.91.44470981331.44470981334.41.21172875621.2117287562

4.40.93312811950.93312811956.11.28854916511.2885491651

4.71.30827300291.30827300293.61.17597664941.1759766494

3.70.77577494560.77577494564.91.35317772241.3531777224

4.10.89816700190.89816700193.50.83920519830.8392051983

4.71.65407503921.65407503924.20.91464850510.9146485051

4.91.03092214041.03092214044.51.34685442741.3468544274

3.50.78670189150.78670189154.91.14843239061.1484323906

4.50.93541240230.93541240234.41.21172875621.2117287562

4.81.23268239131.23268239133.91.32125866111.3212586611

4.41.24645580651.24645580654.60.72750676440.7275067644

309.542.335209788642.335209788611.22.62631534662.6263153466

332.944.870744396944.87074439698.71.62806594971.6280659497

350.746.64181212946.6418121297.51.43878884651.4387888465

366.748.816672681248.816672681282.04521623292.0452162329

382.450.422789715350.42278971537.50.83920519830.8392051983

395.552.553318855952.55331885593.91.22050315581.2205031558

408.552.831970002452.83197000246.41.49551866171.4955186617

416.452.116950339252.11695033926.61.65794121131.6579412113

423.353.956549537553.95654953757.61.40729512581.4072951258

434.555.961188902655.96118890266.81.04531257721.0453125772

443.956.43073289256.4307328927.80.86672740180.8667274018

452.556.401831997556.40183199757.41.63199373811.6319937381

462.857.01267423357.0126742337.71.80226447221.8022644722

469.158.154650535158.154650535171.23958822911.2395882291

477.959.155949025959.155949025971.30664072151.3066407215

490.559.129102044659.12910204467.91.78799830221.7879983022

498.859.696323489459.69632348947.11.25498743111.2549874311

50658.523794658758.523794658771.01211955081.0121195508

515.857.789722004957.78972200495.80.86672740180.8667274018

525.157.331197556357.33119755637.61.52379183821.5237918382

532.258.319812191158.31981219117.22.08244263722.0824426372

540.858.446289234658.44628923468.40.93312811950.9331281195

547.760.537138436360.53713843635.41.21172875621.2117287562

557.460.634465229860.63446522989.11.90361903291.9036190329

567.959.869566362459.86956636248.41.21172875621.2117287562

57859.796482179759.796482179710.11.32125866111.3212586611

586.459.578518587259.57851858729.22.1028392322.102839232

595.159.134299185859.13429918587.61.46670057231.4667005723

604.559.329438875359.32943887538.51.43878884651.4387888465

614.759.33792741759.3379274179.41.73345480371.7334548037

611.859.948898444559.94889844453.31.05547134441.0554713444

614.460.335965789360.33596578933.80.70364756290.7036475629

616.360.724396684760.72439668473.80.86672740180.8667274018

618.561.085453732361.08545373233.81.16136767671.1613676767

620.560.796189902860.79618990283.80.91464850510.9146485051

623.361.247904011461.24790401143.71.01422594131.0142259413

625.861.034109822361.03410982233.70.9262414710.926241471

627.960.527548763160.52754876313.71.40273830291.4027383029

630.460.699192742560.69919274253.40.66625865360.6662586536

632.959.766816416859.76681641684.30.97123053970.9712305397

636.259.495069820959.495069820930.87652124270.8765212427

637.660.14891749760.1489174973.91.22050315581.2205031558

640.359.634980897359.63498089733.81.04531257721.0453125772

642.660.204242250560.20424225054.51.49694500321.4969450032

646.860.525680010260.52568001024.81.54050669121.5405066912

648.960.28023087860.2802308784.61.10099504191.1009950419

652.359.809364888259.80936488823.70.9262414710.926241471

655.159.474727583359.47472758334.71.30827300291.3082730029

No Recovery

Smart Recovery

Time (minutes)

Number of Components

Number of Connected Components

Sheet3

0.23464913310.00436961790.00436961790.23786524430.00617432790.0061743279

0.23320761040.00552288760.00552288760.23845619190.00609876010.0060987601

0.23395703480.00549518090.00549518090.23708527650.0059843580.005984358

0.23532182650.0065614030.0065614030.23844513820.00477545910.0047754591

0.23298287660.00455405430.00455405430.23610178820.00577121590.0057712159

0.2327222120.00527436090.00527436090.23579978510.00589376020.0058937602

0.23345910920.00574364360.00574364360.23670566880.00559587440.0055958744

0.23258378850.00599213090.00599213090.23782254030.00503472870.0050347287

0.23304705760.00573576110.00573576110.23844829340.00613534160.0061353416

0.23414483790.0057432410.0057432410.23860783460.00633934060.0063393406

0.23433175080.00470021890.00470021890.23631635090.00659342380.0065934238

0.23192524310.00582480430.00582480430.23890288330.00571359210.0057135921

0.2327932170.00554433910.00554433910.23733490040.006396690.00639669

0.00732213250.00035416090.00035416090.0274158420.00200416350.0020041635

0.00538154670.00023268550.00023268550.03392621970.00268352610.0026835261

0.00462740320.00025040610.00025040610.03899277050.00304748480.0030474848

0.00397102250.00023462590.00023462590.04339794870.00340409990.0034040999

0.00359712040.0002529290.0002529290.04735230120.00367983030.0036798303

0.00327394160.00022706190.00022706190.05251950510.00388120.0038812

0.00309432460.00020858620.00020858620.05594072050.00432043820.0043204382

0.00293251720.00018709580.00018709580.06023060490.00469466760.0046946676

0.00280173080.00020608280.00020608280.06391433640.00519573580.0051957358

0.00268292540.00021112630.00021112630.06812735810.0051397360.005139736

0.00256604710.00019947340.00019947340.07104753290.00534900190.0053490019

0.00251088870.00019083330.00019083330.07434040180.00584590170.0058459017

0.00239682390.00019380040.00019380040.07766227210.00566017280.0056601728

0.00232263170.00018939480.00018939480.08057586570.00585087370.0058508737

0.00225107170.00017811860.00017811860.08424126480.00624007680.0062400768

0.00218891140.00017403830.00017403830.087267830.00626123440.0062612344

0.002135410.00016445610.00016445610.08940411680.00661556950.0066155695

0.00207890810.00015885530.00015885530.09321777090.00646182410.0064618241

0.00202333730.00015917870.00015917870.09580321180.00623768670.0062376867

0.00198132910.00014796260.00014796260.09833825920.00653992210.0065399221

0.00193448580.0001435530.0001435530.10108017690.00642283360.0064228336

0.00190147370.00014407930.00014407930.10337931670.00688711830.0068871183

0.00187275670.00014225870.00014225870.10531985610.00664382590.0066438259

0.00182121870.00014218640.00014218640.10869955510.00703809450.0070380945

0.00178127650.00012201590.00012201590.11039636390.00691479030.0069147903

0.0017408370.00012145980.00012145980.11264702480.00705092790.0070509279

0.00171455140.0001133480.0001133480.11403451470.00705636840.0070563684

0.00167378950.00011183660.00011183660.11639122970.00711883770.0071188377

0.00166281110.00010869340.00010869340.11832621440.00694433520.0069443352

0.00163258010.00011355530.00011355530.12019569090.00658196010.0065819601

0.00162483130.00011052230.00011052230.123276880.00706198950.0070619895

0.00161902850.00011813050.00011813050.1222289180.00709439940.0070943994

0.00161265340.00010773280.00010773280.1216049160.00706678840.0070667884

0.00162344040.00011254130.00011254130.12003288160.0067621360.006762136

0.00161234630.00010795330.00010795330.11962683770.00653108340.0065310834

0.00160633990.00011298150.00011298150.11923385410.00671601930.0067160193

0.00161198030.00011354690.00011354690.11835598180.00679794830.0067979483

0.00160738380.00010487710.00010487710.11779428320.00684153460.0068415346

0.00159460610.00010806050.00010806050.11593353890.00645363010.0064536301

0.00160228460.00011107090.00011107090.11580086550.00652481060.0065248106

0.00159881260.00011796420.00011796420.11502062420.00664953480.0066495348

0.0016001840.00010782210.00010782210.1135189620.00644398070.0064439807

0.00158873920.00011183150.00011183150.11362617820.00639389660.0063938966

0.00159870730.00011304710.00011304710.11226382060.00653775890.0065377589

0.00159058490.00011659850.00011659850.11155994120.0063347660.006334766

0.00158937110.00010974680.00010974680.11077910590.00669510250.0066951025

0.00158623230.00011110480.00011110480.11046136920.00629453630.0062945363

0.00158769690.00011172360.00011172360.10951868510.00587700220.0058770022

No Recovery

Smart Recovery

Time (minutes)

Percentage of Nodes

Average % of nodes within 6 hops

1.950.1037092210.1037092212.30.08121132680.0812113268

1.952325840.01577391680.01577391682.301531310.0130040910.013004091

1.953678870.01496655880.01496655882.301357390.0151458140.015145814

1.953590520.01548581340.01548581342.299936020.01439586730.0143958673

1.953659860.01554898810.01554898812.301190710.01471179830.0147117983

1.95328670.01584233240.01584233242.30206940.0129134980.012913498

1.953829570.0150692160.0150692162.300477490.0133495080.013349508

1.954788830.01495036340.01495036342.301931530.01507990210.0150799021

1.954610480.01533573290.01533573292.301422760.01228129710.0122812971

1.955093470.0153484970.0153484972.301902860.01212360020.0121236002

1.955146470.01500929550.01500929552.302704810.01264109870.0126410987

1.95624110.01527774920.01527774922.30515580.01327696820.0132769682

1.955672610.01536275910.01536275912.299374460.01334313260.0133431326

1.949407030.0163621560.0163621562.385655710.01208466920.0120846692

1.924790560.01623475630.01623475632.434574660.01399855730.0139985573

1.899458830.0146119190.0146119192.434213970.00972090840.0097209084

1.879404410.01464699150.01464699152.751365420.01192319760.0119231976

1.861478410.01414329210.01414329212.895688740.01404199670.0140419967

1.845404570.01427567850.01427567852.771901820.01558512940.0155851294

1.830077590.01388990820.01388990822.744917120.02043762790.0204376279

1.815186670.01354669880.01354669882.723360530.01826282540.0182628254

1.801629210.01295176480.01295176482.709334920.0154022520.015402252

1.787863530.01221522910.01221522912.704945960.01850340250.0185034025

1.775780710.01222362170.01222362172.690584370.0133624750.013362475

1.763506470.01191443460.01191443462.673471640.01276703280.0127670328

1.751881410.01156059570.01156059572.670401190.01355871690.0135587169

1.740899590.01217064010.01217064012.669586360.01587006370.0158700637

1.729868460.01152238990.01152238992.659622250.01293701650.0129370165

1.718395040.01119737910.01119737912.645290560.01328720740.0132872074

1.708831410.01086792990.01086792992.643449090.01192074630.0119207463

1.698740910.01044465320.01044465322.638378990.01394955690.0139495569

1.688730150.01039113220.01039113222.630514430.0179193070.017919307

1.679823840.00992315180.00992315182.626762040.015752020.01575202

1.670274860.00919599520.00919599522.623787740.01556958850.0155695885

1.660993660.00983735950.00983735952.615220930.01277432180.0127743218

1.652730590.0091033310.0091033312.608704750.013884860.01388486

1.644599590.00895068180.00895068182.599120710.01427321530.0142732153

1.63518380.00827392140.00827392142.595478540.01388250310.0138825031

1.627237120.00796672440.00796672442.591830480.00745244130.0074524413

1.61929370.00766958260.00766958262.590371930.01252318210.0125231821

1.610430760.00715308960.00715308962.59129970.01373700580.0137370058

1.602827430.00660019760.00660019762.581863850.0097470320.009747032

1.595842430.00623951140.00623951142.575454150.01527021490.0152702149

1.591012370.00622176530.00622176532.555467830.01274074670.0127407467

1.588599810.00613759670.00613759672.504121510.01103129350.0110312935

1.589525510.0062860220.0062860222.493408130.0110184530.011018453

1.591006360.00614508790.00614508792.41715380.01048041810.0104804181

1.592711880.00652243280.00652243282.280763280.0068162440.006816244

1.594062610.00611043860.00611043862.277970850.00746328670.0074632867

1.595470740.00588707870.00588707872.277886130.00853262940.0085326294

1.596853390.00610127940.00610127942.277024950.00827989730.0082798973

1.598685110.00620038810.00620038812.276530060.00957547590.0095754759

1.599986880.0062153160.0062153162.270571720.00612057240.0061205724

1.601840290.00576958720.00576958722.271857760.00815321520.0081532152

1.60256870.00653324890.00653324892.268769550.00934328810.0093432881

1.604181340.00629765550.00629765552.269248850.00704313060.0070431306

1.605882370.00626224890.00626224892.267515610.00844547220.0084454722

1.607175170.00590808680.00590808682.266604270.00677518430.0067751843

1.608501670.00593929190.00593929192.267836780.00873346690.0087334669

1.610523120.00607246050.00607246052.266808740.00679198090.0067919809

1.611604280.00614966230.00614966232.264537690.0058815260.005881526

No Recovery

Smart Recovery

Time (minutes)

Pings

Average Number of Pings

Results Number of CCs

Results - % of nodes within 6 hops

Chart2

0.23464913310.00436961790.00436961790.23786524430.00617432790.0061743279

0.23320761040.00552288760.00552288760.23845619190.00609876010.0060987601

0.23395703480.00549518090.00549518090.23708527650.0059843580.005984358

0.23532182650.0065614030.0065614030.23844513820.00477545910.0047754591

0.23298287660.00455405430.00455405430.23610178820.00577121590.0057712159

0.2327222120.00527436090.00527436090.23579978510.00589376020.0058937602

0.23345910920.00574364360.00574364360.23670566880.00559587440.0055958744

0.23258378850.00599213090.00599213090.23782254030.00503472870.0050347287

0.23304705760.00573576110.00573576110.23844829340.00613534160.0061353416

0.23414483790.0057432410.0057432410.23860783460.00633934060.0063393406

0.23433175080.00470021890.00470021890.23631635090.00659342380.0065934238

0.23192524310.00582480430.00582480430.23890288330.00571359210.0057135921

0.2327932170.00554433910.00554433910.23733490040.006396690.00639669

0.00732213250.00035416090.00035416090.0274158420.00200416350.0020041635

0.00538154670.00023268550.00023268550.03392621970.00268352610.0026835261

0.00462740320.00025040610.00025040610.03899277050.00304748480.0030474848

0.00397102250.00023462590.00023462590.04339794870.00340409990.0034040999

0.00359712040.0002529290.0002529290.04735230120.00367983030.0036798303

0.00327394160.00022706190.00022706190.05251950510.00388120.0038812

0.00309432460.00020858620.00020858620.05594072050.00432043820.0043204382

0.00293251720.00018709580.00018709580.06023060490.00469466760.0046946676

0.00280173080.00020608280.00020608280.06391433640.00519573580.0051957358

0.00268292540.00021112630.00021112630.06812735810.0051397360.005139736

0.00256604710.00019947340.00019947340.07104753290.00534900190.0053490019

0.00251088870.00019083330.00019083330.07434040180.00584590170.0058459017

0.00239682390.00019380040.00019380040.07766227210.00566017280.0056601728

0.00232263170.00018939480.00018939480.08057586570.00585087370.0058508737

0.00225107170.00017811860.00017811860.08424126480.00624007680.0062400768

0.00218891140.00017403830.00017403830.087267830.00626123440.0062612344

0.002135410.00016445610.00016445610.08940411680.00661556950.0066155695

0.00207890810.00015885530.00015885530.09321777090.00646182410.0064618241

0.00202333730.00015917870.00015917870.09580321180.00623768670.0062376867

0.00198132910.00014796260.00014796260.09833825920.00653992210.0065399221

0.00193448580.0001435530.0001435530.10108017690.00642283360.0064228336

0.00190147370.00014407930.00014407930.10337931670.00688711830.0068871183

0.00187275670.00014225870.00014225870.10531985610.00664382590.0066438259

0.00182121870.00014218640.00014218640.10869955510.00703809450.0070380945

0.00178127650.00012201590.00012201590.11039636390.00691479030.0069147903

0.0017408370.00012145980.00012145980.11264702480.00705092790.0070509279

0.00171455140.0001133480.0001133480.11403451470.00705636840.0070563684

0.00167378950.00011183660.00011183660.11639122970.00711883770.0071188377

0.00166281110.00010869340.00010869340.11832621440.00694433520.0069443352

0.00163258010.00011355530.00011355530.12019569090.00658196010.0065819601

0.00162483130.00011052230.00011052230.123276880.00706198950.0070619895

0.00161902850.00011813050.00011813050.1222289180.00709439940.0070943994

0.00161265340.00010773280.00010773280.1216049160.00706678840.0070667884

0.00162344040.00011254130.00011254130.12003288160.0067621360.006762136

0.00161234630.00010795330.00010795330.11962683770.00653108340.0065310834

0.00160633990.00011298150.00011298150.11923385410.00671601930.0067160193

0.00161198030.00011354690.00011354690.11835598180.00679794830.0067979483

0.00160738380.00010487710.00010487710.11779428320.00684153460.0068415346

0.00159460610.00010806050.00010806050.11593353890.00645363010.0064536301

0.00160228460.00011107090.00011107090.11580086550.00652481060.0065248106

0.00159881260.00011796420.00011796420.11502062420.00664953480.0066495348

0.0016001840.00010782210.00010782210.1135189620.00644398070.0064439807

0.00158873920.00011183150.00011183150.11362617820.00639389660.0063938966

0.00159870730.00011304710.00011304710.11226382060.00653775890.0065377589

0.00159058490.00011659850.00011659850.11155994120.0063347660.006334766

0.00158937110.00010974680.00010974680.11077910590.00669510250.0066951025

0.00158623230.00011110480.00011110480.11046136920.00629453630.0062945363

0.00158769690.00011172360.00011172360.10951868510.00587700220.0058770022

No Recovery

Smart Recovery

Time (minutes)

Percentage of Nodes

Average % of nodes within 6 hops

Sheet1

0:00

0:10

0:20

0:30

0:40

0:50

1:00

1:10

1:20

1:30

1:40

1:50

start

2:10

2:20

2:30

2:40

2:50

3:00

3:10

3:20

3:30

3:40

3:50

4:00

4:10

4:20

4:30

4:40

4:50

5:00

5:10

5:20

5:30

5:40

5:50

6:00

6:10

6:20

6:30

6:40

6:50

end

7:10

7:20

7:30

7:40

7:50

8:00

8:10

8:20

8:30

8:40

8:50

9:00

9:10

9:20

9:30

9:40

9:50

10:00

Sheet1

0.99983459820.00005494960.00005494960.99983497560.0000385410.000038541

0.99985973030.00002429940.00002429940.99982977260.00005331970.0000533197

0.99982982270.00006040870.00006040870.99980469770.00007255430.0000725543

0.99974460040.00006412060.00006412060.99982975740.00004682460.0000468246

0.99986959150.00005910450.00005910450.99981503340.00006493090.0000649309

0.99980460850.00006779960.00006779960.99986483990.0000389090.000038909

0.99987468520.00004216830.00004216830.99984495430.00004471040.0000447104

0.9998395520.00004605320.00004605320.99981468880.00008295720.0000829572

0.99982450610.000067720.000067720.99980479990.00005150880.0000515088

0.99980444990.00005774810.00005774810.99987479340.00003945010.0000394501

0.99982977240.0000605650.0000605650.99982474960.00004680490.0000468049

0.9998544820.0000665350.0000665350.99980992440.00006143540.0000614354

0.99981964220.00003643980.00003643980.99982973510.00006267530.0000626753

0.99320055340.00308886160.00308886160.33273752650.03531952740.0353195274

0.9965893140.00158189120.00158189120.23730294030.0191217420.019121742

0.99760707750.00186783290.00186783290.18582906440.03519528060.0351952806

0.99888470970.00056877830.00056877830.14214210730.032063460.03206346

0.99939845240.00042094510.00042094510.10021101760.02190499350.0219049935

0.99985329250.00006168910.00006168910.08397148570.02205775160.0220577516

0.99972614630.00007590190.00007590190.07378161880.01385080340.0138508034

0.99971568020.00008410.00008410.06690181110.01349456210.0134945621

0.99966434550.00007144250.00007144250.06387852140.01292318370.0129231837

0.99947431350.0004561040.0004561040.05362399410.01249720370.0124972037

0.99965293480.00004407720.00004407720.05309430110.01263263960.0126326396

0.99967286820.00008311890.00008311890.04995975960.01260103540.0126010354

0.99965671840.00009263040.00009263040.04751102180.01160186670.0116018667

0.99968700460.00006421610.00006421610.04271776390.01067691940.0106769194

0.99969182010.00006678530.00006678530.04108789520.01158211420.0115821142

0.99964456710.00009238820.00009238820.03707467940.01000440270.0100044027

0.99968543790.00006443140.00006443140.03342026490.00990901020.0099090102

0.99969003210.00005237990.00005237990.03209062720.0101978670.010197867

0.9997515370.00004494540.00004494540.03082146880.00877820410.0087782041

0.99965796420.00007854240.00007854240.02923052250.00802548920.0080254892

0.99967803250.00010798730.00010798730.02613284360.00654260110.0065426011

0.99961498890.0000484940.0000484940.02566191750.0063174990.006317499

0.99976536460.00006342530.00006342530.02569520510.00631271860.0063127186

0.99957751250.00009843070.00009843070.02507311580.00589930960.0058993096

0.999612950.0000632710.0000632710.02419786180.00543531830.0054353183

0.99952319560.00006905520.00006905520.0209883860.00317637540.0031763754

0.99956934020.00011081840.00011081840.01985000650.00246800860.0024680086

0.99965297450.00007704670.00007704670.01930733550.00255300190.0025530019

0.99960475440.00007624450.00007624450.01929850410.00254827620.0025482762

0.9995567420.0000913430.0000913430.01901165340.00278219690.0027821969

0.99987845480.00005586970.00005586970.0190351950.00278128040.0027812804

0.9998523940.00003702460.00003702460.0190261110.00277924420.0027792442

0.99985234370.00004568880.00004568880.01901628820.00276554730.0027655473

0.99985237410.00006126540.00006126540.01903540750.00276971480.0027697148

0.99985206850.00004842750.00004842750.01889918230.00294349560.0029434956

0.99985719480.00005390460.00005390460.01871769080.00300231210.0030023121

0.99985759270.00004874830.00004874830.01872692960.00300478340.0030047834

0.99985745080.00007435920.00007435920.01856325760.00282633960.0028263396

0.99987349190.00003482940.00003482940.01858259890.00283983760.0028398376

0.9998259180.00005122710.00005122710.01857468940.00284721480.0028472148

0.99989423120.00004643670.00004643670.01843247150.00292928840.0029292884

0.99984695750.00006430290.00006430290.01841725560.00292719950.0029271995

0.99985225130.00005502460.00005502460.01840999830.00291571640.0029157164

0.99981511860.0000792340.0000792340.01839787070.00289405460.0028940546

0.99979933130.00008141110.00008141110.01837324280.00290302830.0029030283

0.99981015650.00005765830.00005765830.01835556110.00291343810.0029134381

0.99985757490.00004870490.00004870490.01760818470.00291868080.0029186808

0.99980464220.00006907850.00006907850.0173186090.00295944910.0029594491

Smart Recovery

No Recovery

Time (minutes)

Percentage of Nodes

% of nodes in the largest connected component

Sheet2

4.30.77577494560.77577494564.31.09516448881.0951644888

4.41.06151994031.06151994033.80.48890019080.4889001908

4.91.44470981331.44470981334.41.21172875621.2117287562

4.40.93312811950.93312811956.11.28854916511.2885491651

4.71.30827300291.30827300293.61.17597664941.1759766494

3.70.77577494560.77577494564.91.35317772241.3531777224

4.10.89816700190.89816700193.50.83920519830.8392051983

4.71.65407503921.65407503924.20.91464850510.9146485051

4.91.03092214041.03092214044.51.34685442741.3468544274

3.50.78670189150.78670189154.91.14843239061.1484323906

4.50.93541240230.93541240234.41.21172875621.2117287562

4.81.23268239131.23268239133.91.32125866111.3212586611

4.41.24645580651.24645580654.60.72750676440.7275067644

309.542.335209788642.335209788611.22.62631534662.6263153466

332.944.870744396944.87074439698.71.62806594971.6280659497

350.746.64181212946.6418121297.51.43878884651.4387888465

366.748.816672681248.816672681282.04521623292.0452162329

382.450.422789715350.42278971537.50.83920519830.8392051983

395.552.553318855952.55331885593.91.22050315581.2205031558

408.552.831970002452.83197000246.41.49551866171.4955186617

416.452.116950339252.11695033926.61.65794121131.6579412113

423.353.956549537553.95654953757.61.40729512581.4072951258

434.555.961188902655.96118890266.81.04531257721.0453125772

443.956.43073289256.4307328927.80.86672740180.8667274018

452.556.401831997556.40183199757.41.63199373811.6319937381

462.857.01267423357.0126742337.71.80226447221.8022644722

469.158.154650535158.154650535171.23958822911.2395882291

477.959.155949025959.155949025971.30664072151.3066407215

490.559.129102044659.12910204467.91.78799830221.7879983022

498.859.696323489459.69632348947.11.25498743111.2549874311

50658.523794658758.523794658771.01211955081.0121195508

515.857.789722004957.78972200495.80.86672740180.8667274018

525.157.331197556357.33119755637.61.52379183821.5237918382

532.258.319812191158.31981219117.22.08244263722.0824426372

540.858.446289234658.44628923468.40.93312811950.9331281195

547.760.537138436360.53713843635.41.21172875621.2117287562

557.460.634465229860.63446522989.11.90361903291.9036190329

567.959.869566362459.86956636248.41.21172875621.2117287562

57859.796482179759.796482179710.11.32125866111.3212586611

586.459.578518587259.57851858729.22.1028392322.102839232

595.159.134299185859.13429918587.61.46670057231.4667005723

604.559.329438875359.32943887538.51.43878884651.4387888465

614.759.33792741759.3379274179.41.73345480371.7334548037

611.859.948898444559.94889844453.31.05547134441.0554713444

614.460.335965789360.33596578933.80.70364756290.7036475629

616.360.724396684760.72439668473.80.86672740180.8667274018

618.561.085453732361.08545373233.81.16136767671.1613676767

620.560.796189902860.79618990283.80.91464850510.9146485051

623.361.247904011461.24790401143.71.01422594131.0142259413

625.861.034109822361.03410982233.70.9262414710.926241471

627.960.527548763160.52754876313.71.40273830291.4027383029

630.460.699192742560.69919274253.40.66625865360.6662586536

632.959.766816416859.76681641684.30.97123053970.9712305397

636.259.495069820959.495069820930.87652124270.8765212427

637.660.14891749760.1489174973.91.22050315581.2205031558

640.359.634980897359.63498089733.81.04531257721.0453125772

642.660.204242250560.20424225054.51.49694500321.4969450032

646.860.525680010260.52568001024.81.54050669121.5405066912

648.960.28023087860.2802308784.61.10099504191.1009950419

652.359.809364888259.80936488823.70.9262414710.926241471

655.159.474727583359.47472758334.71.30827300291.3082730029

No Recovery

Smart Recovery

Time (minutes)

Number of Components

Number of Connected Components

Sheet3

0.23464913310.00436961790.00436961790.23786524430.00617432790.0061743279

0.23320761040.00552288760.00552288760.23845619190.00609876010.0060987601

0.23395703480.00549518090.00549518090.23708527650.0059843580.005984358

0.23532182650.0065614030.0065614030.23844513820.00477545910.0047754591

0.23298287660.00455405430.00455405430.23610178820.00577121590.0057712159

0.2327222120.00527436090.00527436090.23579978510.00589376020.0058937602

0.23345910920.00574364360.00574364360.23670566880.00559587440.0055958744

0.23258378850.00599213090.00599213090.23782254030.00503472870.0050347287

0.23304705760.00573576110.00573576110.23844829340.00613534160.0061353416

0.23414483790.0057432410.0057432410.23860783460.00633934060.0063393406

0.23433175080.00470021890.00470021890.23631635090.00659342380.0065934238

0.23192524310.00582480430.00582480430.23890288330.00571359210.0057135921

0.2327932170.00554433910.00554433910.23733490040.006396690.00639669

0.00732213250.00035416090.00035416090.0274158420.00200416350.0020041635

0.00538154670.00023268550.00023268550.03392621970.00268352610.0026835261

0.00462740320.00025040610.00025040610.03899277050.00304748480.0030474848

0.00397102250.00023462590.00023462590.04339794870.00340409990.0034040999

0.00359712040.0002529290.0002529290.04735230120.00367983030.0036798303

0.00327394160.00022706190.00022706190.05251950510.00388120.0038812

0.00309432460.00020858620.00020858620.05594072050.00432043820.0043204382

0.00293251720.00018709580.00018709580.06023060490.00469466760.0046946676

0.00280173080.00020608280.00020608280.06391433640.00519573580.0051957358

0.00268292540.00021112630.00021112630.06812735810.0051397360.005139736

0.00256604710.00019947340.00019947340.07104753290.00534900190.0053490019

0.00251088870.00019083330.00019083330.07434040180.00584590170.0058459017

0.00239682390.00019380040.00019380040.07766227210.00566017280.0056601728

0.00232263170.00018939480.00018939480.08057586570.00585087370.0058508737

0.00225107170.00017811860.00017811860.08424126480.00624007680.0062400768

0.00218891140.00017403830.00017403830.087267830.00626123440.0062612344

0.002135410.00016445610.00016445610.08940411680.00661556950.0066155695

0.00207890810.00015885530.00015885530.09321777090.00646182410.0064618241

0.00202333730.00015917870.00015917870.09580321180.00623768670.0062376867

0.00198132910.00014796260.00014796260.09833825920.00653992210.0065399221

0.00193448580.0001435530.0001435530.10108017690.00642283360.0064228336

0.00190147370.00014407930.00014407930.10337931670.00688711830.0068871183

0.00187275670.00014225870.00014225870.10531985610.00664382590.0066438259

0.00182121870.00014218640.00014218640.10869955510.00703809450.0070380945

0.00178127650.00012201590.00012201590.11039636390.00691479030.0069147903

0.0017408370.00012145980.00012145980.11264702480.00705092790.0070509279

0.00171455140.0001133480.0001133480.11403451470.00705636840.0070563684

0.00167378950.00011183660.00011183660.11639122970.00711883770.0071188377

0.00166281110.00010869340.00010869340.11832621440.00694433520.0069443352

0.00163258010.00011355530.00011355530.12019569090.00658196010.0065819601

0.00162483130.00011052230.00011052230.123276880.00706198950.0070619895

0.00161902850.00011813050.00011813050.1222289180.00709439940.0070943994

0.00161265340.00010773280.00010773280.1216049160.00706678840.0070667884

0.00162344040.00011254130.00011254130.12003288160.0067621360.006762136

0.00161234630.00010795330.00010795330.11962683770.00653108340.0065310834

0.00160633990.00011298150.00011298150.11923385410.00671601930.0067160193

0.00161198030.00011354690.00011354690.11835598180.00679794830.0067979483

0.00160738380.00010487710.00010487710.11779428320.00684153460.0068415346

0.00159460610.00010806050.00010806050.11593353890.00645363010.0064536301

0.00160228460.00011107090.00011107090.11580086550.00652481060.0065248106

0.00159881260.00011796420.00011796420.11502062420.00664953480.0066495348

0.0016001840.00010782210.00010782210.1135189620.00644398070.0064439807

0.00158873920.00011183150.00011183150.11362617820.00639389660.0063938966

0.00159870730.00011304710.00011304710.11226382060.00653775890.0065377589

0.00159058490.00011659850.00011659850.11155994120.0063347660.006334766

0.00158937110.00010974680.00010974680.11077910590.00669510250.0066951025

0.00158623230.00011110480.00011110480.11046136920.00629453630.0062945363

0.00158769690.00011172360.00011172360.10951868510.00587700220.0058770022

No Recovery

Smart Recovery

Time (minutes)

Percentage of Nodes

Average % of nodes within 6 hops

1.950.1037092210.1037092212.30.08121132680.0812113268

1.952325840.01577391680.01577391682.301531310.0130040910.013004091

1.953678870.01496655880.01496655882.301357390.0151458140.015145814

1.953590520.01548581340.01548581342.299936020.01439586730.0143958673

1.953659860.01554898810.01554898812.301190710.01471179830.0147117983

1.95328670.01584233240.01584233242.30206940.0129134980.012913498

1.953829570.0150692160.0150692162.300477490.0133495080.013349508

1.954788830.01495036340.01495036342.301931530.01507990210.0150799021

1.954610480.01533573290.01533573292.301422760.01228129710.0122812971

1.955093470.0153484970.0153484972.301902860.01212360020.0121236002

1.955146470.01500929550.01500929552.302704810.01264109870.0126410987

1.95624110.01527774920.01527774922.30515580.01327696820.0132769682

1.955672610.01536275910.01536275912.299374460.01334313260.0133431326

1.949407030.0163621560.0163621562.385655710.01208466920.0120846692

1.924790560.01623475630.01623475632.434574660.01399855730.0139985573

1.899458830.0146119190.0146119192.434213970.00972090840.0097209084

1.879404410.01464699150.01464699152.751365420.01192319760.0119231976

1.861478410.01414329210.01414329212.895688740.01404199670.0140419967

1.845404570.01427567850.01427567852.771901820.01558512940.0155851294

1.830077590.01388990820.01388990822.744917120.02043762790.0204376279

1.815186670.01354669880.01354669882.723360530.01826282540.0182628254

1.801629210.01295176480.01295176482.709334920.0154022520.015402252

1.787863530.01221522910.01221522912.704945960.01850340250.0185034025

1.775780710.01222362170.01222362172.690584370.0133624750.013362475

1.763506470.01191443460.01191443462.673471640.01276703280.0127670328

1.751881410.01156059570.01156059572.670401190.01355871690.0135587169

1.740899590.01217064010.01217064012.669586360.01587006370.0158700637

1.729868460.01152238990.01152238992.659622250.01293701650.0129370165

1.718395040.01119737910.01119737912.645290560.01328720740.0132872074

1.708831410.01086792990.01086792992.643449090.01192074630.0119207463

1.698740910.01044465320.01044465322.638378990.01394955690.0139495569

1.688730150.01039113220.01039113222.630514430.0179193070.017919307

1.679823840.00992315180.00992315182.626762040.015752020.01575202

1.670274860.00919599520.00919599522.623787740.01556958850.0155695885

1.660993660.00983735950.00983735952.615220930.01277432180.0127743218

1.652730590.0091033310.0091033312.608704750.013884860.01388486

1.644599590.00895068180.00895068182.599120710.01427321530.0142732153

1.63518380.00827392140.00827392142.595478540.01388250310.0138825031

1.627237120.00796672440.00796672442.591830480.00745244130.0074524413

1.61929370.00766958260.00766958262.590371930.01252318210.0125231821

1.610430760.00715308960.00715308962.59129970.01373700580.0137370058

1.602827430.00660019760.00660019762.581863850.0097470320.009747032

1.595842430.00623951140.00623951142.575454150.01527021490.0152702149

1.591012370.00622176530.00622176532.555467830.01274074670.0127407467

1.588599810.00613759670.00613759672.504121510.01103129350.0110312935

1.589525510.0062860220.0062860222.493408130.0110184530.011018453

1.591006360.00614508790.00614508792.41715380.01048041810.0104804181

1.592711880.00652243280.00652243282.280763280.0068162440.006816244

1.594062610.00611043860.00611043862.277970850.00746328670.0074632867

1.595470740.00588707870.00588707872.277886130.00853262940.0085326294

1.596853390.00610127940.00610127942.277024950.00827989730.0082798973

1.598685110.00620038810.00620038812.276530060.00957547590.0095754759

1.599986880.0062153160.0062153162.270571720.00612057240.0061205724

1.601840290.00576958720.00576958722.271857760.00815321520.0081532152

1.60256870.00653324890.00653324892.268769550.00934328810.0093432881

1.604181340.00629765550.00629765552.269248850.00704313060.0070431306

1.605882370.00626224890.00626224892.267515610.00844547220.0084454722

1.607175170.00590808680.00590808682.266604270.00677518430.0067751843

1.608501670.00593929190.00593929192.267836780.00873346690.0087334669

1.610523120.00607246050.00607246052.266808740.00679198090.0067919809

1.611604280.00614966230.00614966232.264537690.0058815260.005881526

No Recovery

Smart Recovery

Time (minutes)

Pings

Average Number of Pings

Failure Detection Results

Chart1

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

0.00131666710.0600372580.0005423320.0005423320.00038132750.0003813275

0.0028870670.1040909940.0007575850.0007575850.00089601210.0008960121

0.00503910940.1411671540.0012169570.0012169570.00118738080.0011873808

0.00448099740.1182532110.0015168390.0015168390.00073391580.0007339158

0.00498171860.1070576650.0013093460.0013093460.00099260720.0009926072

0.00514949010.1011128320.0012694170.0012694170.00176009260.0017600926

0.00496743380.0965244510.0013185590.0013185590.00152504040.0015250404

0.00480805850.0941231460.0015413970.0015413970.00172832450.0017283245

0.00526148760.0914465030.0016576230.0016576230.00177157220.0017715722

0.00579270470.0885591880.0016935770.0016935770.00088540850.0008854085

0.00509324380.0861684160.001479130.001479130.00145816420.0014581642

0.00512661410.0850345860.0016607890.0016607890.00156919370.0015691937

0.00545373770.0842780990.0015188520.0015188520.002176010.00217601

0.00650877880.0823011890.0013519810.0013519810.00070479620.0007047962

0.00569852570.0806435410.0011964770.0011964770.00109792840.0010979284

0.0064804670.0797249910.0012825430.0012825430.00090137450.0009013745

0.00559904230.0787842420.0018507420.0018507420.00128599430.0012859943

0.00543097610.0785099550.0017091330.0017091330.00150700820.0015070082

0.00557424830.0769625720.0014754990.0014754990.00177728120.0017772812

0.00599003690.0761246780.0016141980.0016141980.00180048570.0018004857

0.0059356390.0761871210.0021566290.0021566290.00133462080.0013346208

0.00530911070.0750163920.0023886690.0023886690.00084396230.0008439623

0.00524305650.0742018020.0020820190.0020820190.00135880670.0013588067

0.00566260430.0742137380.0011609130.0011609130.00267140810.0026714081

0.00555614880.0742111460.0015506690.0015506690.00170878920.0017087892

0.0056596310.0734114520.0016095320.0016095320.00125181650.0012518165

0.00403507160.0727923710.0017350440.0017350440.00084356620.0008435662

0.00476985630.0721603960.0017554250.0017554250.00087119360.0008711936

0.00405265150.0715980310.0016211380.0016211380.00180316280.0018031628

0.00424520730.0712104260.0013517440.0013517440.0019868520.001986852

0.00356325980.0567623850.0011964030.0011964030.00198522640.0019852264

0.00234859560.0332197650.0005661720.0005661720.00120865630.0012086563

0.0009965380.0090888030.0002765660.0002765660.00072484830.0007248483

000000

000000

000000

000000

00.00000523070.00001025190.000010251900

00.0000052290.00001024870.000010248700

000000

000000

00.00000526730.00001032370.000010323700

000000

000000

000000

000000

000000

000000

Random

Malicious

Time (minutes)

Percent

Percentage of Nodes Detecting Attack

rand1

This is percentage of nodes detecting a faiulre

for 1000 nodes removed at random over 5 minute period

Can you make the confidence levels for this and plot against percentage of nodes that detect failure using smart with malicious attack

Time(sec)RandomMaliciousConfidence

000000000:00

1000000000:10

2000000000:20

3000000000:30

4000000000:40

5000000000:50

6000000001:00

7000000001:10

8000000001:20

9000000001:30

10000000001:40

11000000001:50

1200000000Start

1300.00171172530.00154953510.00115566270.00084974510.00131666710.00038132750.0600372580.0005423322:10

1400.00398205550.00315267980.00181132080.00260221190.0028870670.00089601210.1040909940.0007575852:20

1500.00666094770.00485996290.00372570740.00490981960.00503910940.00118738080.1411671540.0012169572:30

1600.00530624620.00351282180.00438817710.00471674440.00448099740.00073391580.1182532110.0015168392:40

1700.00496076940.00376770820.00495074510.00624765160.00498171860.00099260720.1070576650.0013093462:50

1800.00294072910.00447686120.00627784530.00690252490.00514949010.00176009260.1011128320.0012694173:00

1900.00385806390.00478734130.00400425770.00722007260.00496743380.00152504040.0965244510.0013185593:10

2000.00508336720.00550366070.00228542410.0063597820.00480805850.00172832450.0941231460.0015413973:20

2100.00626591950.00460107190.00305234780.00712661110.00526148760.00177157220.0914465030.0016576233:30

2200.00627454980.00531968790.0047900530.00678652820.00579270470.00088540850.0885591880.0016935773:40

2300.00572567860.00299522790.00520700390.00644506470.00509324380.00145816420.0861684160.001479133:50

2400.00573594180.00289766660.00521499050.00665785730.00512661410.00156919370.0850345860.0016607894:00

2500.00605656210.0031071720.00440190410.00824931260.00545373770.002176010.0842780990.0015188524:10

2600.00642508350.0061199510.00594445010.00754563070.00650877880.00070479620.0823011890.0013519814:20

2700.00602068650.00699693560.00431255780.00546392280.00569852570.00109792840.0806435410.0011964774:30

2800.00680552690.00762538380.00560209690.00588886050.0064804670.00090137450.0797249910.0012825434:40

2900.0063918570.00558573330.00375437150.00666420740.00559904230.00128599430.0787842420.0018507424:50

3000.00741756420.00420901340.0042233210.00587400560.00543097610.00150700820.0785099550.0017091335:00

3100.00789064690.00349344980.00572489560.00518800080.00557424830.00177728120.0769625720.0014754995:10

3200.00757594510.00339453790.00692220270.00606746190.00599003690.00180048570.0761246780.0016141985:20

3300.00624869820.00396580140.00709073030.00643732620.0059356390.00133462080.0761871210.0021566295:30

3400.00448711260.00603808640.00606752060.00464372320.00530911070.00084396230.0750163920.0023886695:40

3500.00466541240.00677177570.00591746690.00361757110.00524305650.00135880670.0742018020.0020820195:50

3600.00320381030.0077643770.00826661120.00341561870.00566260430.00267140810.0742137380.0011609136:00

3700.00404799830.00700280110.00712836260.00404543330.00555614880.00170878920.0742111460.0015506696:10

3800.0051522310.00514660010.0075579880.00478170480.0056596310.00125181650.0734114520.0016095326:20

3900.00404939570.00291560370.00501122310.00416406410.00403507160.00084356620.0727923710.0017350446:30

4000.00379123660.005893090.00496316810.00443193080.00476985630.00087119360.0721603960.0017554256:40

4100.00258711720.00588800080.00235577430.00537971380.00405265150.00180316280.0715980310.0016211386:50

4200.00322580650.00605987330.00188758390.00580756550.00424520730.0019868520.0712104260.001351744End

4300.00285487710.0045445130.00110201510.0057516340.00356325980.00198522640.0567623850.0011964037:10

4400.00248480040.00298382450.00057655010.00334920720.00234859560.00120865630.0332197650.0005661727:20

4500.00079285370.00172783920.00005236420.00141309470.0009965380.00072484830.0090888030.0002765667:30

46000000007:40

47000000007:50

48000000008:00

49000000008:10

5000000005.23E-061.03E-058:20

5100000005.23E-061.02E-058:30

52000000008:40

53000000008:50

5400000005.27E-061.03E-059:00

55000000009:10

56000000009:20

57000000009:30

58000000009:40

59000000009:50

600000000010:00

rand1

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

000000

000.0005423320.0005423320.00038132750.0003813275

000.0007575850.0007575850.00089601210.0008960121

000.0012169570.0012169570.00118738080.0011873808

000.0015168390.0015168390.00073391580.0007339158

000.0013093460.0013093460.00099260720.0009926072

000.0012694170.0012694170.00176009260.0017600926

000.0013185590.0013185590.00152504040.0015250404

000.0015413970.0015413970.00172832450.0017283245

000.0016576230.0016576230.00177157220.0017715722

000.0016935770.0016935770.00088540850.0008854085

000.001479130.001479130.00145816420.0014581642

000.0016607890.0016607890.00156919370.0015691937

000.0015188520.0015188520.002176010.00217601

000.0013519810.0013519810.00070479620.0007047962

000.0011964770.0011964770.00109792840.0010979284

000.0012825430.0012825430.00090137450.0009013745

000.0018507420.0018507420.00128599430.0012859943

000.0017091330.0017091330.00150700820.0015070082

000.0014754990.0014754990.00177728120.0017772812

000.0016141980.0016141980.00180048570.0018004857

000.0021566290.0021566290.00133462080.0013346208

000.0023886690.0023886690.00084396230.0008439623

000.0020820190.0020820190.00135880670.0013588067

000.0011609130.0011609130.00267140810.0026714081

000.0015506690.0015506690.00170878920.0017087892

000.0016095320.0016095320.00125181650.0012518165

000.0017350440.0017350440.00084356620.0008435662

000.0017554250.0017554250.00087119360.0008711936

000.0016211380.0016211380.00180316280.0018031628

000.0013517440.0013517440.0019868520.001986852

000.0011964030.0011964030.00198522640.0019852264

000.0005661720.0005661720.00120865630.0012086563

000.0002765660.0002765660.00072484830.0007248483

000000

000000

000000

000000

000.00001025190.000010251900

000.00001024870.000010248700

000000

000000

000.00001032370.000010323700

000000

000000

000000

000000

000000

000000

Random

Malicious

Time (minutes)

Percent

Percentage of Nodes Detecting Attack

Random Node Distribution

Chart1

0.04978333440.06411744490.00022217250.0002221725

0.14935747090.15834381010.00041407250.0004140725

0.22404367460.21864879370.00042025680.0004202568

0.22404740890.21242240430.00063243020.0006324302

0.16803555670.14945113830.0006695930.000669593

0.10081965360.098474190.00051456290.0005145629

0.05040814640.05390112020.00040362690.0004036269

0.02160241110.02547436520.0002646610.000264661

0.00810036410.01162080740.0001784320.000178432

0.00269989630.00505912020.00010708660.0001070866

0.00080988790.00192042240.00005849260.0000584926

0.00022085270.00056638330.00007352970.0000735297

Binomial

RDP

Frequency

Percent of Nodes

Distribution of Random Nodes Found

Sheet1

Can you plot these 4 distributions (confidence levels) against the binomial distribution for me?

BinomialOur distributionsRDPCONF

00.04978333440.0645287830.06342349420.06481160570.06370589690.06411744490.0002221725

10.14935747090.15940963030.159000540.15834034760.15662472240.15834381010.0004140725

20.22404367460.21741688730.21830052530.21849779670.22037996550.21864879370.0004202568

30.22404740890.20976757870.21265524520.2141407140.21312607940.21242240430.0006324302

40.16803555670.1507306070.15144077360.14715056690.14848260550.14945113830.000669593

50.10081965360.09998038640.0977369790.09669752930.09948186530.098474190.0005145629

60.05040814640.05398646660.05218202350.05471109570.05472489510.05390112020.0004036269

70.02160241110.02451701480.02528103680.02639005790.02570935110.02547436520.000264661

80.00810036410.01176816710.01217416920.01163539140.01090550210.01162080740.000178432

90.00269989630.00549181130.00505620740.00495123040.00473723170.00505912020.0001070866

100.00080988790.00205942920.00191448630.00203000450.00167776960.00192042240.0000584926

110.00022085270.00034323820.00083451970.00064365990.00044411550.00056638330.0000735297

Sheet1

000.00022217250.0002221725

000.00041407250.0004140725

000.00042025680.0004202568

000.00063243020.0006324302

000.0006695930.000669593

000.00051456290.0005145629

000.00040362690.0004036269

000.0002646610.000264661

000.0001784320.000178432

000.00010708660.0001070866

000.00005849260.0000584926

000.00007352970.0000735297

Binomial

RDP

Frequency

Percent of Nodes

Distribution of Random Nodes Found

Sheet2

Sheet3

Messages Per Node Results

ConclusionsBy planning for and detecting failures our recovery method can drastically increase the likelihood that the network will not become partitionedIt lessens the impact of malicious attacks on the querying capability of the network

Further WorkInvestigating other techniques for random node discoveryRestoring network to a scale free topology immediately following failuresHow the Gnutella network has changed over time

ThanksStefan Saroiu and Steven Gribble for letting us use their data and giving us adviceArmando Fox, George Candea, Dave Patterson, Aaron BrownBling-Bling Industries, 2001

40 sec

The basis for our distributed recovery method is that each node will help build and maintain a virtual exponential network. What I mean by virtual is that connections between the nodes in the network will not be actually made, and no traffic will be sent through the network.An exponential network will be build through random node discovery, each node maintaining a list of randomly chosen nodes as its neighbors on the virtual network. The random selection of neighbors makes this network exponential, and makes it very robust to malicious attacks.

Our recovery method also requires nodes to detect malicious attacks on the active network.

When attacks are detected, nodes will begin switching over to the exponential network.

40 minutesRandom node discovery would be easy if there was a centralized name authority which could provide a truly randomly selected node in the network. Because Gnutella is decentralized, we need another way to find a random node.

Our solution is built around the idea of taking random walks through a graph and selecting the node you end up at as a random node.

To do these random walks, we propose a message called a random discovery ping. A random discovery ping is originated by a node trying to discover a random neighbor, and it is forwarded through the network for a certain number of hops, after which point the node that ended up with the ping responds to the originator.

Each node receiving the RDP forwards it to only one of its neighbors, selected at random. Our RDPs use a hop count of 20, which is approximately equal to the diameter of Gnutella.

30 secMaintenance of the virtual exponential network is quite simple. Each node discovers N random nodes, where N is the minimum number of connections the node wants to maintain. These will be the nodes neighbors on the exponential network

The node periodically pings these neighbors to see if they are alive

If a neighbor is found to be dead, discover a new random node to replace it.40 secWe believe that malicious failures are far more likely to result in the loss of a nodes second degree neighbors than its first degree neighbors. If a node with 50 neighbors is removed, then all its neighbors will lose only 1 1st degree neighbor but 50 2nd degree neighbors. This unbalanced loss should tell a node that something malicious has happened

In order to detect failures in this way, nodes keep a history of 1st and 2nd degree neighbor loss over 30 seconds.

If this loss exceeds 50% of their 2nd degree neighbors and it exceeds the percentage of 1st degree neighbors lost, the failure is marked as malicious.

50 secondsReacting to failures is quite simple. When a failure has been detected, replace each 1st degree neighbor lost with a neighbor from the exponential network.

This has the advantage that only nodes local to the attack will switch, thus preserving the rest of the network if the attack is small scale.

The only other rule is that nodes should not attempt to discover new random nodes during an attack, in order to cut down the traffic on an already stressed network.

When the attack is over, return to normal operations.

30 secondsTo run our experiments we build a generalized P2P network simulator.

This simulator handles time management, routing messages between hosts.

It also has support for connecting new hosts and disconnecting current hosts from the network, as well as support for injecting failures into the network, which in our case means killing hosts.

We used this to simulate Gnutella by building a simple Gnutella client that would interface with the simulator, as well as an enhanced client with the recovery mechanism built into it.

We also wrote logging features and metric calculation algorithms. This was written in about 5000 lines of java.

40 secondsThis graph shows the percentage of nodes that detect a malicious failure when 1000 nodes are removed from the system over five minutes. The red line shows the results when we remove nodes maliciously, and the blue line shows the results when we remove nodes randomly.

As you can see, less than 1 percent of the nodes detect a malicious attack when we remove nodes randomly, but somewhere around 20% of the nodes detect a malicious attack when we remove nodes maliciously. Also, notice that no attack is detected outside of the failure range, even though nodes are entering and leaving the system.

20 secondsThis graph shows the distribution of nodes selected using the random ping discovery technique, plotted against the binomial distribution showing perfectly random results. As you can see, there is a little starvation and a little favoritism, but the distribution is very close to random.

This graph shows the average number of messages per node per second during a malicious attack.

The top line is with our recovery method in place, and the bottom line is without it. This only counts pings and pongs and does not count query messages.

This shows that our method only increases the number of messages sent to maintain the network by about 20%. This increase is negligible compared to query traffic.

There is an increase in messages during the beginning of the attack and this lasts throughout the attack

The first increase is due to nodes connecting to new neighbors and detecting failures, while the second increase is due to nodes searching for new random neighbors.