Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | kyle-turnley |
View: | 219 times |
Download: | 1 times |
IntelIntel Research Research Internet Coordinate Systems - 03/03/2004
Internet Coordinate SystemsInternet Coordinate Systems
Marcelo PiasMarcelo Pias
Intel Research CambridgeIntel Research Cambridge
[email protected]@intel.com
2Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
OutlineOutline MotivationMotivation ProblemsProblems General ApproachGeneral Approach TechniquesTechniques
Global Network Positioning (GNP)Global Network Positioning (GNP) Practical Internet Coordinates (PIC)Practical Internet Coordinates (PIC) LighthousesLighthouses PCA-based techniques (Virtual Landmark and ICS)PCA-based techniques (Virtual Landmark and ICS)
ConclusionsConclusions Open Issues/Future work (PII projects)Open Issues/Future work (PII projects)
3Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
MotivationMotivation
Source: planet-lab.org Source: planet-lab.org
What’s the closest server to a client in Brazil ?What’s the closest server to a client in Brazil ?
Geographical distancesGeographical distances--------------------------------------------------------------server1 -> 4500 milesserver1 -> 4500 milesserver2 -> 6000 milesserver2 -> 6000 miles
……… …
ClientClient
ServerServer
4Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
MotivationMotivation Difficulties:Difficulties:
Geographical distances Geographical distances ≠ network distances≠ network distances Routing policies/ConnectivityRouting policies/Connectivity GPS not availableGPS not available
Client needs ‘N’ distances to select the closest serverClient needs ‘N’ distances to select the closest server
5Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
MotivationMotivation
Source: planet-lab.orgSource: planet-lab.org
Network Latency (time)
Network Latency (time)
Network LatencyNetwork Latency--------------------------------------------------------------
server1 -> 120 msserver1 -> 120 msserver2 -> 130 msserver2 -> 130 ms
……… …
6Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
MotivationMotivation Network latency = network distanceNetwork latency = network distance
E.g. ping measurementsE.g. ping measurements
Still have the issue of ‘N’ distances…Still have the issue of ‘N’ distances… Need ‘N’ measurements (high overhead)Need ‘N’ measurements (high overhead) Update list of network distancesUpdate list of network distances How do we solve this problem ?How do we solve this problem ?
7Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
Internet Coordinate SystemInternet Coordinate System
(x1,y1)(x1,y1)
(x2,y2)(x2,y2)
xx
yy
Difference between coordinates ≈network distance
Source: planet-lab.org Source: planet-lab.org
8Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
OutlineOutline MotivationMotivation ProblemsProblems General ApproachGeneral Approach TechniquesTechniques
Global Network Positioning (GNP)Global Network Positioning (GNP) Practical Internet Coordinates (PIC)Practical Internet Coordinates (PIC) LighthousesLighthouses PCA-based techniques (Virtual Landmark and ICS)PCA-based techniques (Virtual Landmark and ICS)
ConclusionsConclusions Open Issues/Future work (PII projects)Open Issues/Future work (PII projects)
9Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
ProblemProblem
Problem statement (graph embedding into a vector space):Problem statement (graph embedding into a vector space): Find a scalable mapping Find a scalable mapping αα : H → V : H → Vkk, such that , such that d(hi,hj) d(hi,hj) ≈ D≈ D(vi,vj)(vi,vj), where:, where: H is the original space (Internet graph)H is the original space (Internet graph) V is the target vector space of dimensionality KV is the target vector space of dimensionality K Example: H = {h1,h2,h3}; V = {v1,v2,v3}; d(h1,h2) Example: H = {h1,h2,h3}; V = {v1,v2,v3}; d(h1,h2) ≈ D(v1,v2) ≈ D(v1,v2)
MappingMappingh1h1
h2h2
20 ms20 ms
70 ms70 ms
80 ms80 ms
h3h3
v1 (5,3)v1 (5,3)
v2 (15,23)v2 (15,23)
22.36
ms
22.36
ms
80 ms80 ms
v3 (85,3)v3 (85,3)
XX
YY
72.80 ms
72.80 ms
InternetInternet
10Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
OutlineOutline MotivationMotivation ProblemsProblems General ApproachGeneral Approach TechniquesTechniques
Global Network Positioning (GNP)Global Network Positioning (GNP) Practical Internet Coordinates (PIC)Practical Internet Coordinates (PIC) LighthousesLighthouses PCA-based techniques (Virtual Landmark and ICS)PCA-based techniques (Virtual Landmark and ICS)
ConclusionsConclusions Open Issues/Future work (PII projects)Open Issues/Future work (PII projects)
11Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
General ApproachGeneral Approach Tasks:Tasks:
1) Select a subset of hosts for ‘reference points’ (RP)1) Select a subset of hosts for ‘reference points’ (RP) Create the origin of the coordinate systemCreate the origin of the coordinate system
2) Measure distances between RFs2) Measure distances between RFs 3) Calculate coordinates for each RP3) Calculate coordinates for each RP 4) Measure distance between host and RFs4) Measure distance between host and RFs 5) Calculate coordinates for the host 5) Calculate coordinates for the host
Different proposed techniques for tasks 1,3 and 5Different proposed techniques for tasks 1,3 and 5 Reference points = landmarks, lighthouses, beaconsReference points = landmarks, lighthouses, beacons
12Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
OutlineOutline MotivationMotivation ProblemsProblems General ApproachGeneral Approach TechniquesTechniques
Global Network Positioning (GNP)Global Network Positioning (GNP) Practical Internet Coordinates (PIC)Practical Internet Coordinates (PIC) LighthousesLighthouses PCA-based techniques (Virtual Landmark and ICS)PCA-based techniques (Virtual Landmark and ICS)
ConclusionsConclusions Open Issues/Future work (PII projects)Open Issues/Future work (PII projects)
13Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
Global Network Positioning (GNP)Global Network Positioning (GNP) T.S.E. Ng, H. Zhang [ACM IMW’01]T.S.E. Ng, H. Zhang [ACM IMW’01] Landmark coordinatesLandmark coordinates
L1
L2
L3
1) Landmark Selection (fixed set)1) Landmark Selection (fixed set) 2) ‘L’ landmarks measure mutual network distances (ping)2) ‘L’ landmarks measure mutual network distances (ping) 3) Landmarks computes coordinates by minimizing the overall error between the measured and 3) Landmarks computes coordinates by minimizing the overall error between the measured and
the estimated distancesthe estimated distances Multi-dimensional global minimisation problemMulti-dimensional global minimisation problem
minimise: error(dminimise: error(d ijij,D,Dijij))
L1
L2
L3
x
y
(x1,y1)
(x2,y2)
(x3,y3)
d 12 D 12
14Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
Host coordinatesHost coordinates
InternetInternet
L1
L2
L3
H
1) Host measures its network distances to the ‘L’ landmarks1) Host measures its network distances to the ‘L’ landmarks 2) Host computes its own coordinate relative to the Landmarks2) Host computes its own coordinate relative to the Landmarks 3) Multi-dimensional global minimisation problem3) Multi-dimensional global minimisation problem
minimise: error(dminimise: error(dijij,D,Dijij))
x
y
(x1,y1)
(x2,y2)
(x3,y3)L1
L2
L3
H
(x4,y4)
Global Network Positioning (GNP)Global Network Positioning (GNP)
15Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
Global Network Positioning (GNP)Global Network Positioning (GNP)
Issues:Issues: Landmark selectionLandmark selection
Fixed setFixed set Issues with landmark failures, landmark overloadIssues with landmark failures, landmark overload What’s the optimal selection ?What’s the optimal selection ?
Technique (Simplex downhill)Technique (Simplex downhill) Unique coordinates are not guaranteedUnique coordinates are not guaranteed Depends on the starting point for the algorithmDepends on the starting point for the algorithm
16Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
Practical Internet Coordinates (PIC)Practical Internet Coordinates (PIC)
Costa, M. et al [IEEE ICDCS’04]Costa, M. et al [IEEE ICDCS’04] Host coordinatesHost coordinates
1) Any host who has already a computed coordinate can be a landmark1) Any host who has already a computed coordinate can be a landmark 2) Host measures its network distances to the ‘L’ landmarks2) Host measures its network distances to the ‘L’ landmarks 3) Host computes its own coordinate relative to the Landmarks3) Host computes its own coordinate relative to the Landmarks 4) Multi-dimensional global minimisation problem4) Multi-dimensional global minimisation problem
minimise: error(dminimise: error(dijij,D,Dijij))
H
x
y(x1,y1)
(x2,y2)
(x3,y3)
H
(x4,y4)
HL2
L2
L2
17Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
Practical Internet Coordinates (PIC)Practical Internet Coordinates (PIC)
PIC was tested in Pastry (Structured P2P system):PIC was tested in Pastry (Structured P2P system): Each node maintains a routing table with distances to Each node maintains a routing table with distances to
closest nodesclosest nodes For a system with 20,000 nodes, a joining node needs to For a system with 20,000 nodes, a joining node needs to
measure 297 distancesmeasure 297 distances Using a coordinate system reduces measurements to 32 Using a coordinate system reduces measurements to 32
probesprobes Selection strategySelection strategy
Random: pick landmarks randomlyRandom: pick landmarks randomly Closest: pick landmarks ‘closest’ to the hostClosest: pick landmarks ‘closest’ to the host Hybrid: pick landmarks as in random and others as in Hybrid: pick landmarks as in random and others as in
closestclosest
18Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
Practical Internet Coordinates (PIC)Practical Internet Coordinates (PIC)
Source: Costa, M. et al [ICDCS’04]Source: Costa, M. et al [ICDCS’04]
19Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
LighthousesLighthouses Pias, M. et al [IPTPS’03]Pias, M. et al [IPTPS’03] Host selects the reference points (lighthouses)Host selects the reference points (lighthouses) Coordinates computed using linear transformationsCoordinates computed using linear transformations Similarly to PIC, Lighthouses is scalableSimilarly to PIC, Lighthouses is scalable
L1
L2H1
L3 H2
L4
L5
L6
20Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
LighthousesLighthouses Lighthouses coordinatesLighthouses coordinates
L1
L2
L3
Derive a distance matrix D = Derive a distance matrix D =
Computes an orthogonal basis (Gram-Schmidt) using QR decomposition:Computes an orthogonal basis (Gram-Schmidt) using QR decomposition: D = Q . RD = Q . R Q is the orthogonal basis that creates the coordinate systemQ is the orthogonal basis that creates the coordinate system Each lighthouse is assigned a column vector of QEach lighthouse is assigned a column vector of Q
21Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
LighthousesLighthouses Host coordinatesHost coordinates
1) Host measures its network distances to the ‘L’ lighthouses1) Host measures its network distances to the ‘L’ lighthouses 2) Distances of the host are projected onto the orthogonal basis:2) Distances of the host are projected onto the orthogonal basis: 3) Host coordinates H = Q . D, where D is the matrix with distances between the 3) Host coordinates H = Q . D, where D is the matrix with distances between the
host and lighthouseshost and lighthouses
H1
L1
L2
L3
d
dx
dy
22Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
PCA-based techniques (Virtual Landmarks and ICS)PCA-based techniques (Virtual Landmarks and ICS)
Tang, L, Crovella, M. [ACM IMC’03]: “Virtual Landmarks”Tang, L, Crovella, M. [ACM IMC’03]: “Virtual Landmarks” Lim, H, Hou, J.C, Choi, C-H [ACM IMC’03]: “ICS”Lim, H, Hou, J.C, Choi, C-H [ACM IMC’03]: “ICS”
Larger number of landmarks/beacons (m)Larger number of landmarks/beacons (m) Derive a landmark distance matrix m x mDerive a landmark distance matrix m x m Use Principal Component Analysis to derive an optimal basisUse Principal Component Analysis to derive an optimal basis
23Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
PCA-based techniques (Virtual Landmarks and ICS)PCA-based techniques (Virtual Landmarks and ICS)
Optimal basis using Singular Value Decomposition: D = U . W . V Optimal basis using Singular Value Decomposition: D = U . W . V TT Where columns of U are the principal components and form an Where columns of U are the principal components and form an
orthogonal basisorthogonal basis U has ‘m’ columns (components)U has ‘m’ columns (components) Use the first ‘k’ principal components that allow ‘good’ projectionsUse the first ‘k’ principal components that allow ‘good’ projections
Pc1
Pc2
24Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
PCA-based techniques (Virtual Landmarks and ICS)PCA-based techniques (Virtual Landmarks and ICS)
Host CoordinatesHost Coordinates Linear projections on the first ‘k’ principal componentsLinear projections on the first ‘k’ principal components HHi = i = UUT T . d. dii
Pc1
Pc2
H
d
25Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
OutlineOutline MotivationMotivation ProblemsProblems General ApproachGeneral Approach TechniquesTechniques
Global Network Positioning (GNP)Global Network Positioning (GNP) Practical Internet Coordinates (PIC)Practical Internet Coordinates (PIC) LighthousesLighthouses PCA-based techniques (Virtual Landmark and ICS)PCA-based techniques (Virtual Landmark and ICS)
ConclusionsConclusions Open Issues/Future work (PII projects)Open Issues/Future work (PII projects)
26Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
ConclusionsConclusions Techniques explored:Techniques explored:
Error minimisation: GNP and PICError minimisation: GNP and PIC Linear projections: Lighthouses, Virtual Landmarks and Linear projections: Lighthouses, Virtual Landmarks and
ICSICS
ApplicationsApplications Closest server selection (e.g. distributed network games)Closest server selection (e.g. distributed network games) Network-aware construction of peer-to-peer systemsNetwork-aware construction of peer-to-peer systems Routing in mobile ad-hoc networksRouting in mobile ad-hoc networks Network distance estimationNetwork distance estimation
Internet Coordinate System is promising but …Internet Coordinate System is promising but …
27Internet Coordinate Systems – 03/03/2004IntelIntel Research Research
Open Issues/Future workOpen Issues/Future work Landmark placement Landmark placement How many dimensions do we need to create an How many dimensions do we need to create an
“Internet Coordinate System” ?“Internet Coordinate System” ? Some of the research suggested 6-9 dimensionsSome of the research suggested 6-9 dimensions However, different datasets give different valuesHowever, different datasets give different values
Routing policies x dimensionality x errorRouting policies x dimensionality x error PII projects (implementation):PII projects (implementation):
A distance estimation service on PlanetLabA distance estimation service on PlanetLab Closest node service on PlanetLabClosest node service on PlanetLab