Outline
Matrix Completion Problems
William Âáóßëåéïò Êáñáãåþñãïò
ÌÐËÁ, ÅÊÐÁ
Áëãüñéèìïé óôçí ÄïìéêÞ ÂéïðëçñïöïñéêÞ8 Áðñéëßïõ, 2008
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
Outline
Outline
1 Introduction
2 Completion ProblemsPositive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
3 Summary
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Introduction
Some De�nitions
Partial matrixAny matrix A ∈ Fm×n with unspeci�ed entries, i.e. only a subset
S ⊆ A of its positions is speci�ed
Matrix completion & completion problemsA matrix completion is the complement CAS of S over A
Completion problems deal with whether it is possible to �nd such a
complement, which meets certaint criteria, in the form of matrix
properties
Patternbipartite graph G(V1 + V2;E ) : V1 = {i |i = 1; :::;m};V2 = {j |j =
1; :::;n};E = {{i ; j}|aij ∈ S ⊆ A}
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Introduction
Some De�nitions
Matrix properties (criteria) we will be concerned with:
Positive (de�nite) matrices(∀x ∈ Fm(〈x |Ax 〉 ≥ 0) ⇒ A � 0) ⇔ (A =
∑i ëi |i〉〈i | = BBT )
Distance matrices (Euclidian)D ∈ Rn×n : ∃u1; :::; uk∀i ; j ≤ n(dij = ‖ui − uj‖2)Completely positive matrices A = BBT , where B is nonnegative
i.e. bij ≥ 0
Contraction matrices〈Ax |Ax 〉 = ë〈x |x 〉 ⇒ ë ≤ 1
Matrix rankThe maximum number of linearly independant columns/rows
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Introduction
Remarks
For positive (de�nite) and distance n × n matrices, we usuallyassume aii ∈ S
A � 0⇒ A = A† ∵ A = B + iC for any A, whereB = B†;C = C †
Inheritance structureAny principal submatrix of A must share the same propertieswith AGives rise to combinatorial aspect of completion problems
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Outline
1 Introduction
2 Completion ProblemsPositive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
3 Summary
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Positive (semi)de�nite Completion Problem (PSD)
The (PSD) problem is an instance of the more generalSemide�nite Programming Problem (P):
Given Hermitian matrices Ai and scalars bi ,
where i = 1; :::;m, decide whether
X � 0; 〈Aj |X 〉 = bj
is feasible
which in turn can be viewed as a generalization of the LinearProgramming problem (LP)
maximize cT x ;Ax ≤ b; x � 0
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Complexity of (P)
Still open question, but if NP 6= coNP , then neitherNP-complete nor coNP-complete
Can be solved in P with arbitrary precision by usingcombinatorial means such as the ellipsoid and interior-pointmethods, for �xed m
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
PSD Solvability Methods
PSD actually a convex optimization problem
Ellipsoid methodEnclose the minimizer of a convex function in a sequence of volume
decreasing ellipsoids
Interior-point methodDe�ne a self-concordant fuction f : R → R : |f ′′′(x )| ≤ 2f ′′(x )
3
2 to
encode the convex set and traverse the feasibility region
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
PSD Solvability Preconditions I
To be positive (de�nite), matrix A must also be partialpositive (de�nite)
Partial positive matrix A has positive completion i� pattern Gis a chordal graph (C4 �ind G)
If G is not chordal, then the minimum �ll-in is the number ofedges that need to be added in order to make it so
Finding the minimum �ll-in is NP-hard, but if it is known to beequal to m, then (P) can be solved in P (m linear constraints)
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
PSD Solvability Preconditions II
If a partial m × n matrix A with diagonal entries equal to 1has a positive (de�nite) completion, then the associated vector
x :=1
ðarccos aij
satis�es ∑e∈F
xe −∑
e∈C\F
xe ≤ |F | − 1
where F ⊆ C ;C cycle in G ; |F | oddConversely, if a pattern G satis�es the above relation, then theaccording matrix has a positive (de�nite) complement i� Gdoes not contain a homeomorph of K4 as an induced subgraph
Polynomial complexity for x ∈ Q, but generally unknown
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
PSD Solvability Preconditions III
Partial matrix A has a positive (de�nite) completion i�∑i∈V
aiixii +∑
i 6=j ;ij∈Eaij xij ≥ 0;∀X ∈ PG
where PG is the cone graph associated to G
PG := {X = [xij ]|X � 0; xij = 0;∀i 6= j ; ij ∈ E}
One needs only check those X's extremal in PG
If we de�ne order(G) := min{rank(X )|X extremal in PG},then rank 1 graphs are exactly the chordal graphs
Complexity unknown for higher orders
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Outline
1 Introduction
2 Completion ProblemsPositive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
3 Summary
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Euclidian Distance Matrix Completion Problem (EDM)
(EDM): Given partial symmetric matrix A with pattern G
A ∈ Rn×n : aii = 0
G = {V ;E};V = {i |i = 1; :::;n}
determine whether there exist a realization of A, i.e. vectors
u1; :::; un ∈ Rk; k ≥ 1 : ∀{i ; j} ∈ E (aij = ‖ui − uj ‖2)
(EDMk): Graph Realization Problem. Variation of (EDM) for�xed k
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
EDM Complexity
(EDM) can be reduced to (PSD)
Given symmetric n × n matrix D = [aij ]; aii = 0, D is adistance matrix i� n − 1× n − 1 matrix X
X = [xij ]; xij :=1
2(din + djn − dij )
is positiveD has a realization in kd-space i� rank(X ) ≤ k
Therefore, (EDM) solved with arbitrary precision in P
Exact complexity of (EDM) unknown, but (EDMk) isNP-complete for k = 1 and NP-hard for k ≥ 2
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
EDM Solvability
The same preconditions that apply to (PSD) also apply
(EDMk) for k ≤ 3, can be reformulated into an optimizationproblem, by minimizing a cost function such as
f : Rn×k → R; f (w) :=∑i ;j
(‖ui − uj ‖2 − aij )2
where w := (u1; :::; un); ui ∈ Rk
Since at least NP-hard, special programming techniques suchas \tabu", \pattern searching" and of course \D&C" are used
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Outline
1 Introduction
2 Completion ProblemsPositive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
3 Summary
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Completely Positive Matrice Completion Problem
Matrix A is doubly nonnegative if A � 0 and A is nonnegative
A completely positive ⇒ A doubly nonnegative
Actually a con�nement of (PSD) over R+
Problem solvable if K4 �ind G
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Outline
1 Introduction
2 Completion ProblemsPositive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
3 Summary
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Contraction Matrice Completion Problem
Reduction to (PSD)Matrix A is a contraction matrix i� A � 0, where
A :=
(In A
A† Im
)Solvable i� one of the below preconditions hold
G does not contain an induced matching of size 2(nonseparable bipartite)~G is chordal, where ~G is the pattern of ~A
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Outline
1 Introduction
2 Completion ProblemsPositive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
3 Summary
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Determining the rank of a completion
If minimum and maximum ranks are known then we canconstruct completions of any rank in between, by starting o�one of the former and changing entries one at a time
Maximum rank MR(A) computed in P
Minimum rank mr(A) hard to compute
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Maximum rank
Equivalent to computing generic rank of A, where unspeci�edvalues are concidered variables
For X ⊆ V1 ∪ V2, X is a cover of G i�AX := [aij ]; i ∈ V1 \ X ; j ∈ V2 \ X is a fully specifedsubmatrix
MR(A) = rank(AX ) + |X |, where X in the minimum cover ofG
Solvable in P by greedy algorithm which perturbs an arbitrarycompletion
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Minimum rank
Generally depends on the actual values aij
Which are the graphs G for which mr(A) depends only on therank of submatrices, like with MR(A)? (rank determinedgraphs)
Conjecture: bipartite chordal graphs (C6 �ind G) are rankdetermined. Proven for nonseparable bipartite
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions
Minimum rank
Matrix A has nonseparable pattern G i� it is of the\triangular" form
A =
(B ?C D
)Then,
mr(A) = rank
(B
C
)+ rank
(C D
)− rank(C )
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
Summary
Summary
All problems are closely related to each other
Graph theory is an important tool
Complexity dependant upon the �eld over which the matricesare de�ned
Generally hard problems, but in most cases approximatelysolvable in P
Constructive algorithms for approximate solutions
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
IntroductionCompletion Problems
SummaryReferences
References
M. Laurent.Matrix Completion Problems.The Encyclopedia of Optimization, vol. III, pages 221{229.
M. Laurent.Polynomial instances of the positive semide�nite and euclideandistance matrix completion problems.SIAM Journal on Matrix Analysis and its Applications,22:874{894, 2000.
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
AppendixNotationPartial Matrices
Appendix Outline
4 AppendixNotationPartial Matrices
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
AppendixNotationPartial Matrices
Dirac Notation: bra & ket
Ket: A \ket" |x 〉 is an element of a Hilbert space(V ;F ;+; ·) ≡ VBra: A \bra" 〈x | is an element of the dual spaceV∗ ≡ (V ∗;L(V ;F );+; ·) of a Hilbert space V
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
AppendixNotationPartial Matrices
Dirac Notation: braket
Inner product (\braket"): For x ; y ∈ V , x · y ≡ 〈x |y〉 ∈ Fe.g.
For V an in�nite dimensional Hilbert space over a �eld F offunctions öi , 〈ök |ö`〉 ≡
∫ö∗k (t)ö`(t)dt
For V a �nite dimensional Hilbert space over a �eld Fn×1 ofvectors
〈x |y〉 ≡nXi=1
x∗i yi =
ˆx∗1 : : : x∗n
˜ 264 y1.
.
.
yn
375 = (X ∗)TY ≡ X†Y
For V a �nite dimensional Hilbert space over the �eld Fn×n ofmatrices
〈x |y〉 ≡ tr(X †Y ) =n∑i=1
(X †Y )ii =n∑i=1
n∑j=1
x∗ij yji
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
AppendixNotationPartial Matrices
Dirac Notation: braket viewpoints
Analytical viewpoint: The braket is properly de�ned as abilinear mapping 〈·|·〉 : V∗ × V → F
Geometric viewpoint: If |x 〉 and |y〉 reside on di�erentsubspaces of space V, and |x 〉 is unitary, then the braket 〈x |y〉consists of a projection of |y〉 onto the subspace spanned by|x 〉
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
AppendixNotationPartial Matrices
Dirac Notation: linear operators
If A is a linear operator acting on a Hilbert space V, then
〈x |A|y〉 ≡ 〈x |Ay〉 ∈ F
〈x |A†|y〉 ≡ 〈Ax |y〉 ∈ F
where |Ay〉 ∈ V is a new ket resulting from operating on |y〉with A, and mutatis mutandis
Outer product: Any linear operator A acting on V can beexpressed as A = |x 〉〈y | in terms of the mapping
|·〉〈·| : (V ∗;L(F ;V ))× (V ;F ) → F × F
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
AppendixNotationPartial Matrices
Dirac Notation: linear operator viewpoints
Matrix viewpoint: When working with a �eld F n×1 of vectors,then A = |x 〉〈x | is the matrix product of an n × 1 matrix witha 1× n matrix, thus A is an n × n matrix
Geometric viewpoint: Operators of the form |x 〉〈x | consist ofprojection operators onto the subspace spanned by |x 〉
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems
AppendixNotationPartial Matrices
Partial Matrices as Linear Partial Functions
Any matrix A ∈ Fm×n can be viewed of as a linear function
fv ;u : (V ;Fm) → (U ;F n)
uniquely de�ned by the choise of bases v ; u for spaces V;Uaccordingly
Similarily, a partial matrix can be viewed of as a partialfunction
öv ;u : (V ;Fm) * (U ;F n) ≡ öv ;u : (V ;Fm) → (U ;F n∪{⊥})
where \bottom" ⊥ is an abstract value devoid of meaning
William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems