Semidefinite Facial Reduction forEuclidean Distance Matrix Completion
Nathan Krislock, Henry Wolkowicz
Department of Combinatorics & OptimizationUniversity of Waterloo
First Alpen-Adria Workshop on OptimizationUniversity of Klagenfurt, Austria
June 3-6, 2010
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 1 / 41
Sensor Network Localization
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
8
9
10
n = 100, m = 9, R = 2
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 2 / 41
Molecular Conformation
50
510
1520
1520
2530
3540
4550
0
5
10
15
20
25
30
35
40
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 3 / 41
Previous Work I
Abdo Y. Alfakih, Amir Khandani, and Henry Wolkowicz.Solving Euclidean distance matrix completion problems via semidefinite programming.Comput. Optim. Appl., 12(1-3):13–30, 1999.
L. Doherty, K. S. J. Pister, and L. El Ghaoui.Convex position estimation in wireless sensor networks.In INFOCOM 2001. Twentieth Annual Joint Conference of the IEEE Computer andCommunications Societies. Proceedings. IEEE, volume 3, pages 1655–1663 vol.3, 2001.
Pratik Biswas and Yinyu Ye.Semidefinite programming for ad hoc wireless sensor network localization.In IPSN ’04: Proceedings of the 3rd international symposium on Information processing insensor networks, pages 46–54, New York, NY, USA, 2004. ACM.
Pratik Biswas, Tzu-Chen Liang, Kim-Chuan Toh, Ta-Chung Wang, and Yinyu Ye.Semidefinite programming approaches for sensor network localization with noisy distancemeasurements.IEEE Transactions on Automation Science and Engineering, 3:360–371, 2006.
Pratik Biswas, Kim-Chuan Toh, and Yinyu Ye.A distributed SDP approach for large-scale noisy anchor-free graph realization withapplications to molecular conformation.SIAM Journal on Scientific Computing, 30(3):1251–1277, 2008.
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 4 / 41
Previous Work II
Zizhuo Wang, Song Zheng, Yinyu Ye, and Stephen Boyd.Further relaxations of the semidefinite programming approach to sensor networklocalization.SIAM Journal on Optimization, 19(2):655–673, 2008.
Sunyoung Kim, Masakazu Kojima, and Hayato Waki.Exploiting sparsity in SDP relaxation for sensor network localization.SIAM Journal on Optimization, 20(1):192–215, 2009.
Ting Pong and Paul Tseng.(Robust) Edge-based semidefinite programming relaxation of sensor network localization.Mathematical Programming, 2010.Published online.
Nathan Krislock and Henry Wolkowicz.Explicit sensor network localization using semidefinite representations and facialreductions.To appear in SIAM Journal on Optimization, 2010.
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 5 / 41
Summary
We developed a theory of semidefinite facial reduction for theEDM completion problemUsing this theory, we can transform the SDP relaxations into muchsmaller equivalent problemsWe developed a highly efficient algorithm for EDM completion:
SDP solver not requiredcan solve problems with up to 100,000 sensors in a few minutes ona laptop computerobtained very high accuracy for noiseless problemsour running times are highly competitive with other SDP-basedcodes
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 6 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 7 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 7 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 7 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 8 / 41
Introduction
Euclidean Distance MatricesAn n × n matrix D is an EDM if ∃p1, . . . ,pn ∈ Rr
Dij = ‖pi − pj‖2, for all i , j = 1, . . . ,n
En := set of all n × n EDMs
Embedding Dimension of D ∈ En
embdim(D) := min{
r : ∃p1, . . . ,pn ∈ Rr s.t. Dij = ‖pi − pj‖2, for all i , j}
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 9 / 41
Introduction
Partial Euclidean Distance MatricesD is a partial EDM in Rr if
every entry Dij is either “specified” or “unspecified”, diag(D) = 0for all α ⊆ {1, . . . ,n}, if the principal submatrix D[α] is fullyspecified, then D[α] is an EDM with embdim(D[α]) ≤ r
Problem: determine if there is a completion D ∈ En with embdim(D) = r
Graph of a Partial EDMG = (N,E , ω) weighted graph with
nodes N := {1, . . . ,n}edges E :=
{ij : i 6= j ,and Dij is specified
}weights ω ∈ RE
+ with ωij :=√
Dij
Problem: determine if there is a realization p : N → Rr of G
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 10 / 41
Introduction
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
8
9
10
n = 100, m = 9, R = 2
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 11 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 12 / 41
EDMs and Semidefinite Matrices
Linear Transformation KLet D ∈ En be given by the points p1, . . . ,pn ∈ Rr (or P ∈ Rn×r )Let Y := PPT = (pT
i pj) ∈ Sn+
Dij = ‖pi − pj‖2
= pTi pi + pT
j pj − 2pTi pj
= Yii + Yjj − 2Yij
Thus D = K(Y ), where:
K(Y ) := diag(Y )eT + ediag(Y )T − 2Y
K(Sn+) = En (but not one-to-one)
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 13 / 41
EDMs and Semidefinite Matrices
Moore-Penrose Pseudoinverse of K
K†(D) = −12
J [offDiag(D)] J where J := I − 1n
eeT
Theorem: (Schoenberg, 1935)A matrix D with diag(D) = 0 is a Euclidean distance matrix if and only if
K†(D) is positive semidefinite.
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 14 / 41
EDMs and Semidefinite Matrices
Properties of K and K†
SnC := {Y ∈ Sn : Ye = 0} and Sn
H := {D ∈ Sn : diag(D) = 0}
K (SnC) = Sn
H and K† (SnH) = Sn
C
K(Sn
+ ∩ SnC)
= En and K† (En) = Sn+ ∩ Sn
C
embdim(D) = rank(K†(D)
), for all D ∈ En
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 15 / 41
EDMs and Semidefinite Matrices
Vector FormulationFind p1, . . . ,pn ∈ Rr such that ‖pi − pj‖2 = Dij , ∀ij ∈ E
Matrix Formulation using KFind P ∈ Rn×r such that H ◦ K(Y ) = H ◦ D, Y = PPT
Semidefinite Programming (SDP) RelaxationFind Y ∈ Sn
+ ∩ SnC such that H ◦ K(Y ) = H ◦ D
Vector/Matrix Formulation is non-convex and NP-hardSDP Relaxation is tractable, but only problems of limited size canbe directly handled by an SDP solverTo solve this SDP, we use facial reduction to obtain a muchsmaller equivalent problem
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 16 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 17 / 41
Semidefinite Facial Reduction
An LP Exampleminimize 2x1 + 6x2 − x3 − 2x4 + 7x5subject to x1 + x2 + x3 + x4 = 1
x1 − x2 − x3 + x5 = -1x1 , x2 , x3 , x4 , x5 ≥ 0
Summing the constraints:
2x1 + x4 + x5 = 0 ⇒ x1 = x4 = x5 = 0
Restrict LP to the face{
x ∈ R5+ : x1 = x4 = x5 = 0
}E R5
+
minimize 6x2 − x3subject to x2 + x3 = 1
x2 , x3 ≥ 0
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 18 / 41
Semidefinite Facial Reduction
The Minimal FaceIf K ⊆ E is a convex cone and S ⊆ K , then
face(S) :=⋂
S⊆FEK
F
Proposition:Let K ⊆ E be a convex cone and let S ⊆ F E K .If S 6= ∅ and convex, then:
face(S) = F if and only if S ∩ relint(F ) 6= ∅.
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 19 / 41
Semidefinite Facial Reduction
Representing Faces of Sn+
If F E Sn+ and X ∈ relint(F ) with rank(X ) = t , then
F = US t+UT and relint(F ) = US t
++UT ,
where U ∈ Rn×t has full column rank and range(U) = range(X ).
Semidefinite Facial ReductionIf:
face({
X ∈ Sn+ : 〈Ai ,X 〉 = bi , ∀i
})= US t
+UT
Then:
minimize 〈C,X 〉subject to 〈Ai ,X 〉 = bi , ∀i
X ∈ Sn+
=⇒minimize 〈C,UZUT 〉subject to 〈Ai ,UZUT 〉 = bi , ∀i
Z ∈ S t+
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 20 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 21 / 41
Facial Reduction Theory for EDM Completion
Theorem: Clique Facial ReductionLet:
D ∈ Ek
t := embdim(D)
F :={
Y ∈ Sk+ : K(Y ) = D
}Then:
face(F) = US t+1+ UT
whereU :=
[UC e
]UC ∈ Rk×t full column rank and
range(UC) = range(K†(D))
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 22 / 41
Facial Reduction Theory for EDM Completion
Theorem: Extending the Facial ReductionLet D be a partial EDM, α ⊆ {1, . . . ,n}, and:F :=
{Y ∈ Sn
+ ∩ SnC : H ◦ K(Y ) = H ◦ D
}Fα :=
{Y ∈ Sn
+ ∩ SnC : H[α] ◦ K(Y [α]) = H[α] ◦ D[α]
}Fα :=
{Y ∈ S |α|+ : H[α] ◦ K(Y ) = H[α] ◦ D[α]
}If:
face(Fα) E US t+1+ UT
Then:
face(Fα) E(
USn−|α|+t+1+ UT
)∩ Sn
C
where U :=
[U 00 I
]∈ Rn×(n−|α|+t+1)
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 23 / 41
Facial Reduction Theory for EDM Completion
Theorem: Distance Constraint ReductionLet D be a partial EDM, α ⊆ {1, . . . ,n}, and:Fα :=
{Y ∈ Sn
+ ∩ SnC : H[α] ◦ K(Y [α]) = H[α] ◦ D[α]
}Fα :=
{Y ∈ S |α|+ : H[α] ◦ K(Y ) = H[α] ◦ D[α]
}face(Fα) E US t+1
+ UT
If ∃Y ∈ Fα and β ⊆ α is a clique with embdim(D[β]) = t , then:
Fα ={
Y ∈(
USn−|α|+t+1+ UT
)∩ Sn
C : K(Y [β]) = D[β]}
where U :=
[U 00 I
]∈ Rn×(n−|α|+t+1)
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 24 / 41
Facial Reduction Theory for EDM Completion
Theorem: Disjoint Subproblems
Let F be as above. Let {αi}`i=1 be disjoint subsets, α := ∪αi , and :Fi :=
{Y ∈ Sn
+ ∩ SnC : H[αi ] ◦ K(Y [αi ]) = H[αi ] ◦ D[αi ]
}Fi :=
{Y ∈ S |αi |
+ : H[αi ] ◦ K(Y ) = H[αi ] ◦ D[αi ]}
If face(Fi) E UiS ti+1+ UT
i , then
face(F) E(
USn−|α|+t+1+ UT
)∩ Sn
C
where U :=
U1 · · · 0 0...
. . ....
...0 · · · U` 00 · · · 0 I
∈ Rn×(n−|α|+t+1)
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 25 / 41
Facial Reduction Theory for EDM Completion
Facial Reduction AlgorithmFor each node i = 1, . . . ,n, find a clique Ci containing iLet Cn+1 be the clique of anchors
Let Fi :=(
UiSn−|Ci |+ti+1+ UT
i
)∩ Sn
C be the corresponding faces
Compute U ∈ Rn×(t+1) full column rank such that
range(U) =n+1⋂i=1
range(Ui)
Then:
face({
Y ∈ Sn+ ∩ Sn
C : H ◦ K(Y ) = H ◦ D})
E(
US t+1+ UT
)∩ Sn
C
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 26 / 41
Facial Reduction Theory for EDM Completion
Rigid Intersection
Ci
Cj
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 27 / 41
Facial Reduction Theory for EDM Completion
Lemma: Rigid Face IntersectionSuppose:
U1 =
U ′1 0U ′′1 00 I
and U2 =
I 00 U ′′20 U ′2
and U ′′1 ,U
′′2 ∈ Rk×(t+1) full column rank with range(U ′′1 ) = range(U ′′2 )
Then:
U :=
U ′1U ′′1
U ′2(U ′′2 )†U ′′1
or U :=
U ′1(U ′′1 )†U ′′2U ′′2U ′2
Satisfies:
range(U) = range(U1) ∩ range(U2)
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 28 / 41
Facial Reduction Theory for EDM Completion
Clique Union Node Absorption
Rigid
Ci
Cj
Ci
j
Non-rigid
Cj
Ci
Ci
j
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 29 / 41
Facial Reduction Theory for EDM Completion
Theorem: Euclidean Distance Matrix CompletionLet D be a partial EDM and :F :=
{Y ∈ Sn
+ ∩ SnC : H ◦ K(Y ) = H ◦ D
}face(F) E
(US t+1
+ UT)∩ Sn
C = (UV )S t+(UV )T
If ∃Y ∈ F and β is a clique with embdim(D[β]) = t , then:Y = (UV )Z (UV )T , where Z is the unique solution of
(JU[β, :]V )Z (JU[β, :]V )T = K†(D[β]) (1)
F ={
Y}
D := K(PPT ) ∈ En is the unique completion of D, where
P := UVZ 1/2 ∈ Rn×t
Note: In this case, an SDP solver is not required.Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 30 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 31 / 41
Numerical Results
Random noiseless problemsDimension r = 2Square region: [0,1]× [0,1]
m = 4 anchorsR = radio rangeUsing only Rigid Clique Union and Rigid Node AbsorptionError measure: Root Mean Square Deviation
RMSD :=
(1
# positioned
∑i positioned
‖pi − ptruei ‖
2
) 12
.
Results averaged over 10 instancesUsed MATLAB on a 2.16 GHz Intel Core 2 Duo with 2 GB of RAMSource code SNLSDPclique available on author’s website,released under a GNU General Public License
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 32 / 41
Numerical Results
Face Representation Approach# # Sensors CPU
sensors R Positioned Time RMSD2000 .07 2000.0 1 s 2e-132000 .06 1999.9 1 s 3e-132000 .05 1996.7 1 s 2e-132000 .04 1273.8 3 s 4e-126000 .07 6000.0 4 s 8e-146000 .06 6000.0 4 s 7e-146000 .05 6000.0 3 s 1e-136000 .04 5999.4 3 s 3e-13
10000 .07 10000.0 9 s 7e-1410000 .06 10000.0 8 s 1e-1310000 .05 10000.0 7 s 2e-1310000 .04 10000.0 6 s 1e-1320000 .030 20000.0 17 s 2e-1360000 .015 60000.0 1 m 53 s 7e-13100000 .011 100000.0 5 m 46 s 9e-11
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 33 / 41
Numerical Results
Point Representation Approach# # Sensors CPU
sensors R Positioned Time RMSD2000 .07 2000.0 1 s 5e-162000 .06 1999.9 1 s 6e-162000 .05 1996.7 1 s 7e-162000 .04 1274.4 2 s 7e-166000 .07 6000.0 3 s 5e-166000 .06 6000.0 3 s 5e-166000 .05 6000.0 3 s 8e-166000 .04 5999.4 3 s 6e-16
10000 .07 10000.0 7 s 9e-1610000 .06 10000.0 6 s 7e-1610000 .05 10000.0 6 s 6e-1610000 .04 10000.0 5 s 1e-1520000 .030 20000.0 14 s 8e-1660000 .015 60000.0 1 m 27 s 9e-16100000 .011 100000.0 3 m 55 s 1e-15
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 34 / 41
Outline
1 Euclidean Distance MatricesIntroductionEuclidean Distance Matrices and Semidefinite Matrices
2 Facial ReductionSemidefinite Facial Reduction
3 ResultsFacial Reduction Theory for EDM CompletionNumerical ResultsNoisy Problems
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 35 / 41
Noisy Problems
Multiplicative Noise Model
d2ij = ‖pi − pj‖2(1 + σεij)
2, for all ij ∈ E
εij is normally distributed with mean 0 and standard deviation 1σ ≥ 0 is the noise factor
Least Squares Problem
minimize∑ij∈E
v2ij
subject to ‖pi − pj‖2(1 + vij)2 = d2
ij , for all ij ∈ E∑ni=1 pi = 0
p1, . . . ,pn ∈ Rr
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 36 / 41
Noisy Problems
Point Representation Approach# CPU
σ sensors R Time RMSD0 2000 .08 1 s 5e-16
1e-6 2000 .08 1 s 1e-061e-4 2000 .08 1 s 1e-041e-2 2000 .08 1 s 7e-02
0 6000 .06 3 s 5e-161e-6 6000 .06 3 s 1e-061e-4 6000 .06 3 s 1e-041e-2 6000 .06 3 s 2e-01
0 10000 .04 5 s 1e-151e-6 10000 .04 5 s 1e-061e-4 10000 .04 5 s 1e-041e-2 10000 .04 5 s 2e-01
Note: No refinement technique is used in our numerical tests
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 37 / 41
Noisy Problems
1LFBnf = 0.01%, RMSD = 0.002
1LFBnf = 0.1%, RMSD = 0.023
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 38 / 41
Noisy Problems
1LFBnf = 1%, RMSD = 1.402
1LFBnf = 10%, RMSD = 9.919
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 39 / 41
Summary
We developed a theory of semidefinite facial reduction for theEDM completion problemUsing this theory, we can transform the SDP relaxations into muchsmaller equivalent problemsWe developed a highly efficient algorithm for EDM completion:
SDP solver not requiredcan solve problems with up to 100,000 sensors in a few minutes ona laptop computerobtained very high accuracy for noiseless problemsour running times are highly competitive with other SDP-basedcodes
Nathan Krislock (University of Waterloo) SDP Facial Reduction for EDM Completion WO 2010 40 / 41