Jon Dattorro

convexoptimization.com

prototypical cardinality problemprototypical cardinality problem

kx

x

bAx

x

card

0

subject to

find

Perspectives:Combinatorial

Geometric

2

Euclidean bodiesEuclidean bodiesPermutation Polyhedron

T{ , , }n nX X X X R 1 1 1 1 0P• n! permutation matrices are vertices in (n-1)2 dimensions.• permutaton matrices are minimum cardinality doubly stochastic matrices.

Hyperplane

}1{ T 1R xx nH

H

3

Geometrical perspective

Compressed SensingCompressed Sensing

bAx

x

subject to

||||minimize 1

1-norm ball: 2n vertices, 2n facets

4 Candes/Donoho (2004)

Candes demo %Emmanuel Candes, California Institute of Technology, June 6 2007, IMA Summerschool.

clear all, close all n = 512; % Size of signal m = 64; % Number of samples (undersample by a factor 8)

k = 0:n-1; t = 0:n-1; F = exp(-i*2*pi*k'*t/n)/sqrt(n); % Fourier matrix freq = randsample(n,m); A = [real(F(freq,:)); imag(F(freq,:))]; % Incomplete Fourier matrix

S = 28; support = randsample(n,S); x0 = zeros(n,1); x0(support) = randn(S,1); b = A*x0;

% Solve l1 using CVX cvx_quiet(true); cvx_begin variable x(n); minimize(norm(x,1)); A*x == b; cvx_end

norm(x - x0)/norm(x0) figure, plot(1:n,x0,'b*',1:n,x,'ro'), legend('original','decoded')

5wikimization.org

6

Candes demo

0

find

subject to

x

Fx Fx

binary mask

Fourier matrix

is sparse

F

x

0

64 2

512

28

m n

m

A F

b Fx

m

n

k

R

R

k-sparse sampling theoremk-sparse sampling theorem

nmA R• Donoho/Tanner (2005)

7

two geometrical two geometrical interpretationsinterpretations

0

subject to

find

xbAx

x

}0{ xAxK 8

motivation to study conesmotivation to study conesconvex cones generalize orthogonal

subspaces

Projection on K determinable from projection on -K* and vice versa. (Moreau)

Dual cone:

nKK R *

}0{ T* KxyxyK

9

application - LP application - LP presolverpresolverDelete rows and columns of matrix Acolumns: smallest face F of cone K containing

b

A holds generators for K

If feasible, throw A(: , i) away

}0{ xAxK

10

}{)( * bKaKabKF

1),(:

0

0tosubject

find

T

T

T

ziA

zA

zb

z

application - application - CartographyCartography

11

list reconstruction from distance list reconstruction from distance DD

metric multidimensional scalingprincipal component analysisKarhunen-Loeve transform

cartography: projection on semidefinite cone

12

a.k.a

13

0tosubject

||)(||minimize

nT

n

FnT

n

VDV

VHDVhD

S

projection on semidefinite cone because

nT

n VV hSS subspace of symmetric matrices is isomorphic with subspace of symmetric hollow matrices

is convex problem (Eckart & Young) (§7.1.4 CO&EDG)

optimal list X from (§5.12 CO&EDG)

14

nT

n VDV

nT

n

nT

n

FnT

n

rank

0tosubject

||)(||minimize

VDV

VDV

VHDVhD

S

(EY)

ordinal reconstructionordinal reconstruction

15

M

D

KD

VDV

VDV

VODVh

vect

rank

0tosubject

||)(||minimize

nT

n

nT

n

FnT

n

S

• nonconvex• strategy: break into two problems: (EY) and convex problem

• fast projection on monotone nonnegative cone KM+ (Nemeth, 2009) •

MK

D

tosubject

||vect||minimize

25020R

Cardinality heuristics

16

y

*minimize vec ,

subject to vec

y

E t

0

0minimize || ||

subject to vecE t

4M 4MR

0

Rank heuristicstrace is convex envelope of rank on PSD

matrices

rank function is quasiconcave

17

Idea behind convex iteration

18

IGG ,tr (vector inner product)

Convex Iteration

19

application -

20

(Recht, Fazel, Parrilo, 2007)

(Rice University 2005)

one-pixel camera - MIT

21

22

one-pixel camera - MIT

application - MRI phantom

23

Candes, Romberg, Tao 2004

• Led directly to sparse sampling theorem

MATLAB>> phantom(256)

24

application - MRI phantom• MRI raw data called k-space

• Raw data in Fourier domain• aliasing at 4% subsampling

1vect f

25

application - MRI phantom

P16 162 × 2C

(projection matrix)

y is direction vector from convex iteration

• hard to compute

26

application - MRI phantom

27

application - MRI phantom

reconstruction error: -103dB