GIA: Making Gnutella-like P2P GIA: Making Gnutella-like P2P Systems ScalableSystems Scalable

Yatin ChawatheYatin ChawatheSylvia Ratnasamy, Scott Shenker, Sylvia Ratnasamy, Scott Shenker, Nick Lanham, Lee BreslauNick Lanham, Lee Breslau

The Peer-to-peer PhenomenonThe Peer-to-peer Phenomenon

Internet-scale distributed system– Distributed file-sharing applications

– E.g., Napster, Gnutella, KaZaA

File sharing is the dominant P2P app

Mass-market– Mostly music, some video, software

The ProblemThe Problem

Potentially millions of users– Wide range of heterogeneity

– Large transient user population

Existing search solutions cannot scale– Flooding-based solutions limit capacity

– Distributed Hash Tables (DHTs) not necessarily appropriate

Our Solution: GIAOur Solution: GIA

Scalable Gnutella-like P2P system

Design principles:– Explicitly account for node heterogeneity

– Query load proportional to node capacity

Results:– Gia outperforms Gnutella by 3–5 orders of

magnitude

OutlineOutline

Existing approaches

GIA: Scalable Gnutella

Results: Simulations & Experiments

Conclusion

GnutellaGnutella

Distributed search and download

Unstructured: ad-hoc topology– Peers connect to random nodes

Random search– Flood queries across network

Scaling problems– As network grows, search

overhead increases

P1

P2

P4

P3

who has“madonna”

P 4 has “

madonna-

american-life.mp3”P5

P6

P2 has “madonna-ray-of-light.mp3”

Distributed Hash Tables (DHTs)Distributed Hash Tables (DHTs)

Structured solution– Given a filename, find its location

Can DHTs do file sharing?– Probably, but with lots of extra work:

Caching, keyword searching

Do we need DHTs?– Not necessarily: Great at finding rare files,

but most queries are for popular files

Note: Not questioning the utility of DHTs in general, merely for mass-market file sharing

Other SolutionsOther Solutions

Supernodes [KaZaA]– Classify nodes as low- or high-capacity– Only pushes the problem to a bigger scale

Random Walks [Lv et al]– Forwarding is blind– Queries can get stuck in overloaded nodes

Biased Random Walks [Adamic et al]– Right idea, but exacerbates overloaded-node

problem

OutlineOutline

Existing approaches

GIA: Scalable Gnutella

Results: Simulations & Experiments

Conclusion

GIA: 10,000-foot viewGIA: 10,000-foot view

Unstructured, but take node capacity into account– High-capacity nodes have room for more

queries: so, send most queries to them

Will work only if high-capacity nodes:– Have correspondingly more answers, and

– Are easily reachable from other nodes

Make high-capacity nodes easily reachable– Dynamic topology adaptation

Make high-capacity nodes have more answers– One-hop replication

Search efficiently– Biased random walks

Prevent overloaded nodes– Active flow control

Make high-capacity nodes easily reachable– Dynamic topology adaptation

Make high-capacity nodes have more answers– One-hop replication

Search efficiently– Biased random walks

Prevent overloaded nodes– Active flow control

GIA DesignGIA Design

Query

Dynamic Topology AdaptationDynamic Topology Adaptation

Make high-capacity nodes have high degree (i.e., more neighbors)

Per-node level of satisfaction, S:– 0 no neighbors, 1 enough neighbors

– Function of: Node’s capacity ● Neighbors’ capacities

Neighbors’ degrees ● Their age

– When S << 1, look for neighbors aggressively

Active Flow ControlActive Flow Control

Accept queries based on capacity– Actively allocation “tokens” to neighbors

– Send query to neighbor only if we have received token from it

Incentives for advertising true capacity– High capacity neighbors get more tokens

– Allocate tokens with weighted fair queuing

Practical ConsiderationsPractical Considerations

Query resilience: node death– Periodic keep-alive messages

– Query responses are implicit keep-alives

Determining node capacity– Function of bandwidth and “age” of node

Finding rare items– Bifurcate the random walk every 10 hops…

OutlineOutline

Existing approaches

GIA: Scalable Gnutella

Results: Simulations & Experiments

Conclusion

Simulation ResultsSimulation Results

Compare four systems– FLOOD: TTL-scoped, random topologies

– RWRT: Random walks, random topologies

– SUPER: Supernode-based search

– GIA: search using GIA protocol suite

Metric:– Collapse point: aggregate throughput that

the system can sustain

QuestionsQuestions

What is the relative performance of the four algorithms?

Which of the GIA components matters the most?

How does the system behave in the face of transient nodes?

System PerformanceSystem Performance

0.00001

0.001

0.1

10

1000

0.01 0.1 1Replication Rate (percentage)

Co

lla

ps

e P

oin

t (q

ps

/no

de

)

GIA: N=10,000

SUPER: N=10,000

RW RT: N=10,000

FLOOD: N=10,000

GIA outperforms SUPER, RWRT & FLOOD by many orders of magnitude in terms of aggregate query load

Factor AnalysisFactor Analysis

Algorithm Collapse point

RWRT 0.0005

RWRT+OHR 0.005RWRT+BIAS 0.0015

RWRT+TADAPT 0.001RWRT+FLWCTL 0.0006

Algorithm Collapse point

GIA 7GIA – OHR 0.004GIA – BIAS 6

GIA – TADAPT 0.2

GIA – FLWCTL 2

No single component is useful by itself; the combinationof them all is what makes GIA scalable

Transient BehaviorTransient Behavior

0.001

0.01

0.1

1

10

100

1000

10 100 1000 10000Per-node max-lifetime (seconds)

Co

llap

se

po

int

(qp

s/n

od

e)

replication rate = 1.0%

replication rate = 0.5%

replication rate = 0.1%

Static SUPER

Static RWRT (1% repl)

Even under heavy churn, GIA outperforms the otheralgorithms by many orders of magnitude

DeploymentDeployment

Prototype client implementation using C++

Deployed on PlanetLab:– 100 machines spread across 4 continents

Measured the progress of topology adaptation…

Progress of Topology AdaptationProgress of Topology Adaptation

0

10

20

30

40

50

0 20 40 60 80 100Time (seconds)

Nu

mb

er

of

ne

igh

bo

rs

C=1xC=10xC=100xC=1000x

Nodes quickly discover each other and soon reach their target “satisfaction level”

OutlineOutline

Existing approaches

GIA: Scalable Gnutella

Results: Simulations & Experiments

Conclusion

SummarySummary

GIA: scalable Gnutella– 3–5 orders of magnitude improvement in

system capacity

Unstructured approach is good enough!– DHTs may be overkill– Incremental changes to deployed systems

Status: Prototype implementation deployed on PlanetLab