Compressive Sensing in Imaging - mathe.tu-freiberg.debernstei/Web5/MOIMA2016-Rauhut.pdf ·...

Compressive Sensing in Imaging

Holger RauhutLehrstuhl C fur Mathematik (Analysis)

RWTH Aachen

MOIMA HannoverJune 23, 2016

1 / 49

Overview

I Compressive SensingI Imaging (Structured Random Matrices)

I Subsampled Random Convolutions (Coded Aperture Imaging)I Random Fourier Sampling

I Phase Retrieval

2 / 49

Compressive sensing

Reconstruction of signals from minimal amount of measured data

Key ingredients

I Compressibility / Sparsity (small complexity of relevantinformation)

I Efficient algorithms (convex optimization)

I Randomness (random matrices), incoherence

3 / 49

Fourier-Coefficients

Time-Domain Signal with 30

Samples

Traditional Reconstruction

(`2-minimization)

Compressive sensing

(`1-minimization)

4 / 49

Fourier-Coefficients Time-Domain Signal with 30

Samples


(`2-minimization)

Compressive sensing

(`1-minimization)

4 / 49


Samples


(`2-minimization)

Compressive sensing

(`1-minimization)

4 / 49


Samples


(`2-minimization)

Compressive sensing

(`1-minimization) 4 / 49

Mathematical formulation

Recover a vector x ∈ CN from underdetermined linearmeasurements

y = Ax, A ∈ Cm×N ,

where m� N.

Key finding of compressive sensing:Recovery is possible if x belongs to a set of low complexity.

I Standard compressive sensing: Sparsity (small number ofnonzero coefficients)

I Low rank matrix recovery

I Low rank tensor recovery (only partial results so far)

5 / 49



y = Ax, A ∈ Cm×N ,

where m� N.





5 / 49



y = Ax, A ∈ Cm×N ,

where m� N.





5 / 49

Sparsity and Compressibility

I coefficient vector: x ∈ CN , N ∈ NI support of x: supp x := {j , xj 6= 0}I s- sparse vectors: ‖x‖0 := |supp x| ≤ s.

s-term approximation error

σs(x)q := inf{‖x− z‖q, z is s-sparse}, 0 < q ≤ ∞.

x is called compressible if σs(x)q decays quickly in s.

6 / 49

Sparsity and Compressibility

I coefficient vector: x ∈ CN , N ∈ NI support of x: supp x := {j , xj 6= 0}I s- sparse vectors: ‖x‖0 := |supp x| ≤ s.

s-term approximation error

σs(x)q := inf{‖x− z‖q, z is s-sparse}, 0 < q ≤ ∞.

x is called compressible if σs(x)q decays quickly in s.

6 / 49

Compressive Sensing Problem

Reconstruct an s-sparse vector x ∈ CN (or a compressible vector)from its vector y of m measurements

y = Ax, A ∈ Cm×N .

Interesting case: s < m� N.

Preferably fast reconstruction algorithm!

7 / 49

`1-minimization

`0-minimization is NP-hard:

minx∈CN

‖x‖0 subject to Ax = y.

`1 minimization

minx‖x‖1 subject to Ax = y

Convex relaxation of `0-minimization problem.

Efficient minimization methods available.

Alternatives:Greedy Algorithms (Matching Pursuits)Iterative hard thresholdingIteratively reweighted least squares

8 / 49

`1-minimization


minx∈CN


`1 minimization





8 / 49

`1-minimization


minx∈CN


`1 minimization





8 / 49

`1-minimization


minx∈CN


`1 minimization





8 / 49

Null space property

Null space property (NSP): Necessary and sufficient for exactrecovery of all s-sparse vectors via `1-minimization with A,

‖vS‖1 ≤ ρ‖vSc‖1 for all v ∈ kerA,S ⊂ {1, . . .N}, |S | = s

for some 0 < ρ < 1.(Here vS denotes the restriction of v to index set S .)

Implies also stability of reconstruction:

‖x − x ]‖1 ≤2(1 + ρ)

1− ρσs(x)1

Version for robustness under noise on measurements available

9 / 49

Matrices satisfying the NSP

Open problem: Give explicit matrices A ∈ Cm×N satisfying theNSP of order s in the parameter regime

m ≥ Cs lnα(N),

Deterministic matrices known, where m ≥ Cks2 suffices if N ≤ mk .

Small improvement by Bourgain et al. (2010): m ≥ Cs2−ε, ε > 0,under additional assumptions on m,N.

Way out: consider random matrices.

10 / 49



m ≥ Cs lnα(N),




10 / 49



m ≥ Cs lnα(N),




10 / 49

NSP for Gaussian random matrix

TheoremA Gaussian random matrix A ∈ Rm×N satisfies the null spaceproperty of order s and constant 0 < ρ < 1 with high probability if

m ≥ Cρ−2s ln(eN/s)

C = 5e ≈ 5.43 in case ρ = 1 [Donoho, Tanner 2006]C = 8: general ρ; [Foucart, R 2013], [Kabanava, R 2014]

Implies uniform stable reconstruction of all s-sparse vectors with asingle random draw of a Gaussian matrix via `1-minimization.

Bound optimal: Any algorithm that provides stable recovery (inthe sense of approximate sparsity) requires at leastm ≥ Cs ln(eN/s) samples.

Distribution of (independent) entries of A can significantly berelaxed [Lecue, Mendelson 2014]; [Dirksen, Lecue, R 2015]:Mean-zero, variance one and log(N) finite moments are sufficient.Proof via Mendelson’s small ball method (2013)

11 / 49







Distribution of (independent) entries of A can significantly berelaxed [Lecue, Mendelson 2014]; [Dirksen, Lecue, R 2015]:Mean-zero, variance one and log(N) finite moments are sufficient.Proof via Mendelson’s small ball method (2013)

11 / 49







Distribution of (independent) entries of A can significantly berelaxed [Lecue, Mendelson 2014]; [Dirksen, Lecue, R 2015]:Mean-zero, variance one and log(N) finite moments are sufficient.Proof via Mendelson’s small ball method (2013) 11 / 49

Restricted Isometry Property (RIP)

DefinitionThe restricted isometry constant δs of a matrix A ∈ Cm×N isdefined as the smallest δs such that

(1− δs)‖x‖22 ≤ ‖Ax‖2

2 ≤ (1 + δs)‖x‖22

for all s-sparse x ∈ CN .

Requires that all s-column submatrices of A are well-conditioned.

12 / 49

Restricted Isometry Property (RIP)

DefinitionThe restricted isometry constant δs of a matrix A ∈ Cm×N isdefined as the smallest δs such that

(1− δs)‖x‖22 ≤ ‖Ax‖2

2 ≤ (1 + δs)‖x‖22

for all s-sparse x ∈ CN .

Requires that all s-column submatrices of A are well-conditioned.

12 / 49

Stable and robust recoveryTheorem (Candes, Romberg, Tao ’06 – Candes ’08 – Foucart,Lai ’09 – Foucart ’09/’12 – Li, Mo ’11 – Andersson,Stromberg ’12 – Cai, Zhang ’13)

Let A ∈ Cm×N with δ2s < 1/√

2 ≈ 0.7071. Let x ∈ CN , andassume that noisy data are observed, y = Ax + η with ‖η‖2 ≤ σ.Let x# by a solution of

minz‖z‖1 such that ‖Az− y‖2 ≤ σ.

Then

‖x− x#‖2 ≤ Cσs(x)1√

s+ Dσ

and‖x− x#‖1 ≤ Cσs(x)1 + D

√sσ

for constants C ,D > 0, that depend only on δ2s .

Implies exact recovery in the s-sparse and noiseless case.In other words: RIP implies NSP (but the converse is not true)

13 / 49

RIP for Gaussian and Bernoulli matrices

TheoremLet A ∈ Rm×N be a Gaussian or Bernoulli random matrix andassume

m ≥ Cδ−2(s ln(eN/s) + ln(2ε−1))

for a universal constant C > 0. Then with probability at least1− ε the restricted isometry constant of 1√

mA satisfies δs ≤ δ.

Consequence: Recovery via `1-minimization with probabilityexceeding 1− e−cm provided

m ≥ Cs ln(eN/s).

Generalizes to subgaussian random matrices

14 / 49

Structured Random Matrices

15 / 49

Structured Random MeasurementsWhy structure?

I Applications such as imaging impose structure due to physicalconstraints,limited freedom to inject randomness

I Fast matrix vector multiplies (FFT) in recovery algorithms,unstructured random matrices impracticable for large scaleapplications.

Here:I Subsampled random convolutionsI Random Fourier subsampling (Candes, Romberg, Tao 2006;

Rudelson, Vershynin 2008; R 2007, 2010, ...)

Other options:I Time-Frequency structured random matrices (Pfander, R,

Tropp 2012; Krahmer, Mendelson, R 2014)I Radar: Antenna arrays with random antenna positions

(Friedlander, Strohmer 2013; Hugel, R, Strohmer 2014)I ...

16 / 49

Structured Random MeasurementsWhy structure?

I Applications such as imaging impose structure due to physicalconstraints,limited freedom to inject randomness

I Fast matrix vector multiplies (FFT) in recovery algorithms,unstructured random matrices impracticable for large scaleapplications.

Here:I Subsampled random convolutionsI Random Fourier subsampling (Candes, Romberg, Tao 2006;

Rudelson, Vershynin 2008; R 2007, 2010, ...)

Other options:I Time-Frequency structured random matrices (Pfander, R,

Tropp 2012; Krahmer, Mendelson, R 2014)I Radar: Antenna arrays with random antenna positions

(Friedlander, Strohmer 2013; Hugel, R, Strohmer 2014)I ...

16 / 49

Subsampled random convolutions

Cyclic Convolution: (b ∗ x)` =∑N

j=1 b`−j modNxjSubsampling

Ax = ΦΘ(b)x = RΘ(b ∗ x)

RΘ : CN → Cm: restriction to entries in Θ ⊂ {1, . . . ,N},#Θ = m.Example: Θ = {1, 2, . . . ,m}

Task: Recovery of sparse (compressible) x from subsampledconvolution y = ΦΘ(b)x!

Choice of b = ξ as subgaussian random vector – independent,mean-zero, variance one, subgaussian entries, P(|ξj | ≥ t) ≤ 2e−ct

2

Examples

I Rademacher b = ε: independent ±1 entries

I Gaussian b = g: standard Gaussian random vector,g ∼ N (0, Id)

17 / 49

Subsampled random convolutions

Cyclic Convolution: (b ∗ x)` =∑N

j=1 b`−j modNxjSubsampling

Ax = ΦΘ(b)x = RΘ(b ∗ x)

RΘ : CN → Cm: restriction to entries in Θ ⊂ {1, . . . ,N},#Θ = m.Example: Θ = {1, 2, . . . ,m}

Task: Recovery of sparse (compressible) x from subsampledconvolution y = ΦΘ(b)x!

Choice of b = ξ as subgaussian random vector – independent,mean-zero, variance one, subgaussian entries, P(|ξj | ≥ t) ≤ 2e−ct

2

Examples

I Rademacher b = ε: independent ±1 entries

I Gaussian b = g: standard Gaussian random vector,g ∼ N (0, Id)

17 / 49

Partial random circulant matrices

Circulant matrix Φ = Φ(b) ∈ CN×N with entries Φi ,j = bj−i mod N

Φx = Φ(b)x = b ∗ x

Subsampled convolution with random vector ξ corresponds topartial random circulant matrix ΦΘ(ξ) = RΘΦ(ξ).

Fast matrix-vector multiplication via FFT

Variationspartial random Toeplitz matrices (noncyclic convolution),subsampled 2D-convolutions, ...

18 / 49

Motivation: Compressive Coded Aperture Imaging

Super-resolution based on compressive sensing

I Use coded mask insteadof pinhole (or lense)

I Observed coded apertureimage is subsampled2D-convolution of imagex with point-spreadfunction b

Marcia, Willett 2009 – Romberg 2009

Further application: Radar

19 / 49

Numerical experimentsSparse recovery via `1-minimization with partial random circulantmatrix A ∈ Rm×N , N = 500, m = 100.

0 5 10 15 20 25 30 35 400

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1Empirical Recovery Rate

Sparsity

20 / 49

RIP estimate for partial random circulant matrices

Theorem (Krahmer, Mendelson, R 2012)

Let Θ ⊂ [N] be an arbitrary (deterministic) set of cardinality m.Let ξ be a subgaussian (e.g. Gaussian or Rademacher) randomvector in CN . Assume

m ≥ Cδ−2s max{ln2(s) ln2(N), ln(ε−1)}.

Then with probability at least 1− ε the restricted isometryconstants of 1√

mΦΘ(ξ) satisfy δs ≤ δ.

Previous bounds:Haupt, Bajwa, Raz (2008): m ≥ Cδs

2 lnNR, Romberg, Tropp (2010): m ≥ Cδs

3/2 ln3/2(N)

Random sets Θ, Romberg (2009): m ≥ Cδs ln6 N

Mendelson, Paouris, R, Ward (2016): m & s ln(N) for s .√

Nlog(N)

Nonuniform recovery, R 2009; James, R 2013: m ≥ Cs ln(N)

21 / 49

Mathematical Analysis: Chaos processesRecall that δs is smallest constant such that

(1− δs)‖x‖22 ≤ ‖Ax‖2

2 ≤ (1 + δs)‖x‖22 for all s-sparse x.

Equivalently, with Ts = {x ∈ CN : ‖x‖2 ≤ 1, ‖x‖0 ≤ s}

δs = supx∈Ts

∣∣‖Ax‖22 − ‖x‖2

2

∣∣ .

For partial random circulant matrices generated by random vector ξ

Ax =1√mRΘ(ξ ∗ x) =: Vxξ

with appropriate Vx ∈ Rm×N . Furthermore, E‖Vxξ‖22 = ‖x‖2

2.Therefore, δs is supremum of a chaos process,

δs = supx∈Ts

∣∣‖Vxξ‖22 − E‖Vxξ‖2

2

∣∣.

22 / 49

Mathematical Analysis: Chaos processesRecall that δs is smallest constant such that

(1− δs)‖x‖22 ≤ ‖Ax‖2

2 ≤ (1 + δs)‖x‖22 for all s-sparse x.

Equivalently, with Ts = {x ∈ CN : ‖x‖2 ≤ 1, ‖x‖0 ≤ s}

δs = supx∈Ts

∣∣‖Ax‖22 − ‖x‖2

2

∣∣ .For partial random circulant matrices generated by random vector ξ

Ax =1√mRΘ(ξ ∗ x) =: Vxξ

with appropriate Vx ∈ Rm×N . Furthermore, E‖Vxξ‖22 = ‖x‖2

2.Therefore, δs is supremum of a chaos process,

δs = supx∈Ts

∣∣‖Vxξ‖22 − E‖Vxξ‖2

2

∣∣.22 / 49

Generic Chaining (Talagrand)Let B be a subset of a vector space with norm ‖ · ‖.diameter: d‖·‖(B) = supB∈B ‖B‖.

A sequence of subsets Tr ⊂ A, r ∈ N0, is called admissible if|T0| = 1, |Tr | ≤ 22r , r ≥ 1.For α > 0 define the γα-functional (most important: α = 2)

γα(B, ‖ · ‖) = inf supB∈B

∞∑r=0

2r/αd(B,Tr ), d(B,Tr ) = infBr∈Tr

‖B−Br‖,

where the infimum is over all admissible sequences (Tr ).

Estimate by Dudley-type integral

γα(B, ‖ · ‖) ≤ C

∫ d‖·‖(B)

0(lnN(B, ‖ · ‖, u))1/α du,

where N(B, ‖ · ‖, u) denotes the smallest number of balls of radiusu in the norm ‖ · ‖ required to cover B.

23 / 49




∞∑r=0


‖B−Br‖,



γα(B, ‖ · ‖) ≤ C

∫ d‖·‖(B)

0(lnN(B, ‖ · ‖, u))1/α du,


23 / 49




∞∑r=0


‖B−Br‖,



γα(B, ‖ · ‖) ≤ C

∫ d‖·‖(B)

0(lnN(B, ‖ · ‖, u))1/α du,


23 / 49

Generic Chaining for Chaos Processes

Theorem (Krahmer, Mendelson, R 2012)

Let B = −B ⊂ Cm×N be a symmetric set of matrices and ξ ∈ CN

be a subgaussian random vector. Then

E supB∈B

∣∣‖Bξ‖22 − E‖Bξ‖2

2

∣∣≤ C1γ2(B, ‖ · ‖2→2)2 + C2∆‖·‖F (B)γ2(B, ‖ · ‖2→2).

Here, ‖B‖F =√

tr(B∗B) denotes the Frobenius norm.

Symmetry assumption B = −B can be dropped at the cost ofslightly more complicated bound.

24 / 49

Tail bound

Theorem (Krahmer, Mendelson, R ’12 – Dirksen ’13)

Let B = −B ⊂ Cm×N and ξ ∈ CN be a subgaussian randomvector. Then

P(

supB∈B

∣∣‖Bξ‖22 − E‖Bξ‖2

2

∣∣ ≥ C1E + t

)≤ 2 exp

(−C2 min

{t2

V 2,t

U

}),

where

E := ∆‖·‖F (B)γ2(B, ‖ · ‖2→2) + γ2(B, ‖ · ‖2→2)2,

V := ∆‖·‖2→2∆‖·‖F (B),

U := ∆2‖·‖2→2

(B).

25 / 49

Relation to previous estimate of Talagrand

In the Rademacher case ξ = ε, rewrite process as

supB∈B

∣∣‖Bε‖22 − E‖Bε‖2

2

∣∣ = supB∈B

∣∣∣∣∣∣∑j 6=k

εjεk(B∗B)j ,k

∣∣∣∣∣∣Chaos process indexed by D = {B∗B : B ∈ B}.General estimate (Talagrand, 1993)

E supD∈D|∑j 6=k

εjεkDj ,k | ≤ C1γ2(D, ‖ · ‖F ) + C2γ1(D, ‖ · ‖2→2).

This bound was used in the previous RIP estimate due toR, Romberg, Tropp (2010). The appearance of the γ1-functionalresults in the exponent 3/2.

26 / 49

Application to RIP estimate

For B = {Vx : x ∈ Ts} with Ts = {x ∈ CN : ‖x‖2 ≤ 1, ‖x‖0 ≤ s}and Vxξ = 1√

mRΘ(x ∗ ξ) we have

∆‖·‖F (B) = 1, ∆‖·‖2→2(B) ≤

√s

m.

Covering number estimates are similar to the ones for randomFourier sampling and lead to

γ2(B, ‖ · ‖2→2) .∫ ∆‖·‖2→2

0

√ln(N(B, ‖ · ‖2→2, u))du

.

√s log2 s log2 N

m.

27 / 49

Random Sampling

28 / 49

Bounded orthonormal systems (BOS)

D ⊂ Rd endowed with probability measure ν.φ1, . . . , φN : D → C function system on D.

Orthonormality∫Dφj(t)φk(t)dν(t) = δj ,k =

{0 if j 6= k,1 if j = k.

Uniform bound in L∞:

‖φj‖∞ = supt∈D|φj(t)| ≤ K for all j ∈ [N].

29 / 49


D ⊂ Rd endowed with probability measure ν.φ1, . . . , φN : D → C function system on D.Orthonormality∫

Dφj(t)φk(t)dν(t) = δj ,k =

{0 if j 6= k,1 if j = k.



29 / 49


D ⊂ Rd endowed with probability measure ν.φ1, . . . , φN : D → C function system on D.Orthonormality∫

Dφj(t)φk(t)dν(t) = δj ,k =

{0 if j 6= k,1 if j = k.



29 / 49

Fourier system

D = [0, 1]d endowed with Lebesgue measure

φk(t) = e2πik·t , t ∈ [0, 1]d , k ∈ Zd .

The Fourier system is a bounded orthonormal system withconstant K = 1.

30 / 49

SamplingConsider functions

f (t) =N∑

k=1

xkφk(t), t ∈ D.

f is called s-sparse if x is s-sparse.Sampling points t1, . . . , tm ∈ D. Sample values:

y` = f (t`) =N∑

k=1

xkφk(t`) , ` ∈ [m].

Sampling matrix A ∈ Cm×N with entries

A`,k = φk(t`), ` ∈ [m], k ∈ [N].

Theny = Ax .

Choose sampling points t1, . . . , tm independently at randomaccording to ν. Then A is structured random matrix.

31 / 49


f (t) =N∑

k=1

xkφk(t), t ∈ D.

f is called s-sparse if x is s-sparse.

Sampling points t1, . . . , tm ∈ D. Sample values:

y` = f (t`) =N∑

k=1

xkφk(t`) , ` ∈ [m].


A`,k = φk(t`), ` ∈ [m], k ∈ [N].

Theny = Ax .


31 / 49


f (t) =N∑

k=1

xkφk(t), t ∈ D.


y` = f (t`) =N∑

k=1

xkφk(t`) , ` ∈ [m].


A`,k = φk(t`), ` ∈ [m], k ∈ [N].

Theny = Ax .


31 / 49


f (t) =N∑

k=1

xkφk(t), t ∈ D.


y` = f (t`) =N∑

k=1

xkφk(t`) , ` ∈ [m].


A`,k = φk(t`), ` ∈ [m], k ∈ [N].

Theny = Ax .


31 / 49


f (t) =N∑

k=1

xkφk(t), t ∈ D.


y` = f (t`) =N∑

k=1

xkφk(t`) , ` ∈ [m].


A`,k = φk(t`), ` ∈ [m], k ∈ [N].

Theny = Ax .


31 / 49


f (t) =N∑

k=1

xkφk(t), t ∈ D.


y` = f (t`) =N∑

k=1

xkφk(t`) , ` ∈ [m].


A`,k = φk(t`), ` ∈ [m], k ∈ [N].

Theny = Ax .


31 / 49

Restricted isometry property

Theorem (Candes, Tao ’06 – Rudelson, Vershynin ’06 – R’08/’10 – Bourgain ’14 – Haviv, Regev ’15; Webster et al. ’16)

Let A ∈ Cm×N be the random sampling matrix associated to aBOS with constant K ≥ 1 generated from independent randomsampling points. If

m ≥ CδK2s max{ln2(s) ln(N), ln(ε−1)},

then the restricted isometry constant of 1√mA satisfies δs ≤ δ with

probability at least 1− ε. The constant C > 0 is universal.

Implies stable recovery of all s-sparse trigonometric polynomialsfrom m ≥ Cs ln2(s) ln(N) random samples via `1-minimization.

Simplified condition: m ≥ Cδs ln3(N) for RIP to hold with

probability at least 1− N ln3(N)

32 / 49

Nonuniform recovery

Theorem (Candes, Romberg, Tao ’06 – R’07)

Let x ∈ CN be s-sparse and A ∈ Cm×N be the random Fouriermatrix generated with independent and uniformly distributedsampling points. If

m ≥ Cs ln(N/ε)

then `1-minimization reconstructs x from y = Ax exactly withprobability at least 1− ε.

Same result holds for row subsampled discrete Fourier matrix.

Generalizes to sampling in orthonormal systems with uniformlybounded L∞-norm.

33 / 49

Numerical ExampleThe real part of a sparse trigonometric polynomial with sparsityk = 6, N = 81 (maximal degree 40) and n = 25 random samplingpoints. Reconstruction by `1-minimization is exact!

34 / 49

Legendre Polynomials

Consider D = [−1, 1] with normalized Lebesgue measure andorthonormal system of Legendre polynomials φj = Pj ,j = 0, . . . ,N − 1.

It holds ‖Pj‖∞ =√

2j + 1, so K =√

2N − 1.

The previous result yields the (almost) trivial bound

m ≥ CNs log2(s) log(m) log(N) > N.

Can we do better?

35 / 49




2j + 1, so K =√

2N − 1.



Can we do better?

35 / 49




2j + 1, so K =√

2N − 1.



Can we do better?

35 / 49




2j + 1, so K =√

2N − 1.



Can we do better?

35 / 49

Preconditioning (R, Ward 2012)For an orthonormal system {φj} on D w.r.t. prob. measure ν choose a

positive weight function w on [−1, 1] with∫ 1

−11

w2(t)dν(t) = 1. Then

dµ(t) := 1w2(t)dν(t) defines prob. measure.

Define ψj(t) = w(t)φj(t). Then∫Dψj(t)ψk(t)dµ(t) =

∫Dφj(t)w(t)φk(t)w(t)

1

w2(t)dν(t) = δj,k ,

so that {ψj} is ONS w.r.t. µ.If maxj ‖ψjw‖∞ ≤ K , then random sampling matrix with sampling pointchosen according to µ satisfies RIP with high probability if

m ≥ Cδ−2K 2s ln2 ln(N).

Samples with respect to preconditioned system

y ′` =N∑j=1

xkψk(t`) =N∑j=1

xkw(t`)φk(t`) = w(t`)y`.

Legendre polynomials: Choose Chebyshev weightw(x) =

√π2 (1− x2)1/4. Then

‖wPj‖∞ ≤√

3 for all j ∈ N0.

36 / 49



−11





1





y ′` =N∑j=1

xkψk(t`) =N∑j=1



√π2 (1− x2)1/4. Then

‖wPj‖∞ ≤√

3 for all j ∈ N0.

36 / 49



−11





1





y ′` =N∑j=1

xkψk(t`) =N∑j=1



√π2 (1− x2)1/4. Then

‖wPj‖∞ ≤√

3 for all j ∈ N0.

36 / 49



−11





1





y ′` =N∑j=1

xkψk(t`) =N∑j=1



√π2 (1− x2)1/4. Then

‖wPj‖∞ ≤√

3 for all j ∈ N0. 36 / 49

Towards tomography: Sparsity in WaveletsSimple image model: sparsity in (discrete) wavelet basisW ∈ RN×N , z = Wx for sparse x .Measurements w.r.t. randomly selected Fourier coefficientsONS: φj(t) = e2πij ·t/N , t ∈ ZN , columns of Fourier matrix F

(Uniform) random selection operator R : CN → Cm

Samples: y = RF ∗z = RF ∗Wx = RUx with new ONS U = F ∗WEntries Uk,j = 〈wk , φj〉 satisfy maxj ,k |Uj ,k | �

√N (after correct

normalization)

Preconditioning (Krahmer, Ward ’14): Use variable densitysampling of Fouier coefficients!Probabilities in 2D:

p(j , k) =c

1 + j2 + k2.

Preconditioned matrix satisfies RIP with high probability if

m ≥ Cδ−2s ln3(s) log2(N).

37 / 49

Towards tomography: Sparsity in WaveletsSimple image model: sparsity in (discrete) wavelet basisW ∈ RN×N , z = Wx for sparse x .Measurements w.r.t. randomly selected Fourier coefficientsONS: φj(t) = e2πij ·t/N , t ∈ ZN , columns of Fourier matrix F(Uniform) random selection operator R : CN → Cm


√N (after correct

normalization)


p(j , k) =c

1 + j2 + k2.



37 / 49


Samples: y = RF ∗z = RF ∗Wx = RUx with new ONS U = F ∗W

Entries Uk,j = 〈wk , φj〉 satisfy maxj ,k |Uj ,k | �√N (after correct

normalization)


p(j , k) =c

1 + j2 + k2.



37 / 49



√N (after correct

normalization)


p(j , k) =c

1 + j2 + k2.



37 / 49

Fourier recovery via total variation minimization

Relation of total variation semi-norm to wavelet coefficients(Needell, Ward ’13) leads to

Theorem (Krahmer, Ward ’13)

Randomly choose m Fourier coefficients of a vector x indexed by{−N, . . . ,N} × {−N, . . . ,N} according to the probabilitydistribution p(j , k) = c

1+j2+k2 , j , k ∈ {−N, . . . ,N}. If

m ≥ Cs log3(s) log5(N)

then with high probability 2D-TV minimization recovers x up tothe error

‖x − x ]‖2 ≤ C‖∇x − (∇x)s‖1√

s,

where (∇x)s is the best s-term approximation to the gradient ∇x .

38 / 49

Extension to weighted `1-minimization (R, Ward ’14)

Weighted sparsity: ‖x‖0,ω =∑

j∈supp(x) ω2j

Recovery of weighted s-sparse via weighted `1-minimization:

minN∑j=1

|zj |ωj subject to Az = y .

May promote smoothness in addition to sparsity!Recovery guaranteed under weighted version of RIP

Weighted RIP holds with high probability if ONS {φj} satisfies

‖φj‖∞ ≤ ωj for all j

andm ≥ Cs log3(s) log(N).

39 / 49



j∈supp(x) ω2j


minN∑j=1


May promote smoothness in addition to sparsity!

Recovery guaranteed under weighted version of RIP




39 / 49



j∈supp(x) ω2j


minN∑j=1


May promote smoothness in addition to sparsity!Recovery guaranteed under weighted version of RIP




39 / 49

Interpolation via weighted `1-minimization

−1 −0.5 0 0.5 10

0.2

0.4

0.6

0.8

1Original function f (x) = |x |

Weights: wj = 1 + |j |.20 Interpolation points chosen uniformlyat random from [−1, 1].

1 0.5 0 0.5 10

0.2

0.4

0.6

0.8

1Least squares solution

1 0.5 0 0.5 10.5

0

0.5Residual error

1 0.5 0 0.5 10

0.2

0.4

0.6

0.8

1Unweighted l1 minimizer

1 0.5 0 0.5 10.5

0

0.5Residual error

1 0.5 0 0.5 10

0.2

0.4

0.6

0.8

1Weighted l1 minimizer

1 0.5 0 0.5 10.5

0

0.5Residual error

40 / 49

Low rank matrix recovery and phase retrieval

41 / 49

Low rank matrix recovery

Recover X ∈ Rn1×n2 of low rank from

y = A(X ) ∈ Rm

where A : Rn1×n2 → Rm is linear, m� n1n2!

42 / 49

Low rank matrix recovery

Recover X ∈ Rn1×n2 of low rank from

y = A(X ) ∈ Rm

where A : Rn1×n2 → Rm is linear, m� n1n2!

42 / 49

Nuclear norm minimization

Rank minimization problem minZ :A(Z)=y rank(X ) is NP-hard.

Observation: rank(X ) = ‖σ(X )‖0 where σ(X ) is vector of singularvalues of X

Nuclear norm minimization (Fazel, 2001)

min ‖X‖∗ subject to A(X ) = y

with ‖X‖∗ =∑

` σ`(X ).

Alternatives

I Iteratively reweighted least squares (Fornaser, R, Ward 2011;Fazel, Mohan 2012)

I Iterative hard thresholding (Tanner, Wei 2013)

I ADMiRA (Lee, Bresler 2010)

I ...

43 / 49







` σ`(X ).

Alternatives




I ...

43 / 49







` σ`(X ).

Alternatives




I ...

43 / 49

Recovery from subgaussian random mapsWrite A : Rn1×n2 → Rm as

A(X )j =n1∑k=1

n2∑`=1

Aj,k,`Xk,`

Assume Aj,k,` are i.i.d. mean-zero, variance one subgaussian random

variables, i.e., P(|Aj,k,`| ≥ t) ≤ 2e−ct2

.

Examples: Gaussian, Rademacher ±1

Theorem (Fazel, Parillo, Recht 2010; Candes, Plan 2011)

Let A : Rn1×n2 → Rm be a subgaussian random measurementmap. If

m ≥ Cr(n1 + n2)

then with probability at least 1− e−cm, every matrix X ∈ Rn1×n2

of rank at most r can be recovered from A(X ) via nuclear normminimization.

Based on an analysis of the rank restricted isometry property.Gaussian maps: C ≈ 3 (nonuniform; Chandrasekaran et al. 2012);C ≈ 10 (uniform; Kabanava, Kueng, R, Terstiege 2015)

44 / 49

Four finite moments are sufficient

Theorem (Kabanava, Kueng, R, Terstiege 2015)Let A : Rn1×n2 → Rm a random linear map with independent mean-zeroand variance one entries Aj,k,` such that

E|Aj,k,`|4 ≤ C for all j , k , `.

Ifm ≥ Cr(n1 + n2)

then with probability at least 1− e−cm, every X ∈ Rn1×n2 is recoveredfrom y = A(X ) + n with ‖n‖ ≤ η via

X ] = argmin ‖Z‖∗ subject to A(Z ) = y

with error

‖X − X ]‖F ≤ C1‖X − Xr‖∗√

r+ C2η,

where ‖X − Xr‖∗ =∑min{n1,n2}

j=r+1 σj(X ).

Analysis based on Mendelson’s small ball method45 / 49

Recovery from rank one measurementsRank one measurements of X ∈ Hn = {X ∈ Cn×n,X = X ∗}:

A(X )j = tr(Xaja∗j ) = a∗j Xaj , aj ∈ Cn, j = 1, . . . ,m.

Applications

I Phaseless (quadratic) measurements of x ∈ Cn,

yj = |〈x , aj〉|2 = tr(xx∗aja∗j ) = tr(Xaja

∗j ) with X = xx∗.

Recovery via nuclear norm minimization:PhaseLift (Candes, Strohmer, Voroninski 2013)

I Quantum state tomography: State is modeled asX ∈ Sn = {X ∈ Hn,X < 0} with tr(X ) = 1Pure state: rank(X ) = 1, mixed state: rank(X ) small.Quantum measurements:

yj = tr(Xaja∗j ) = a∗j Xaj

46 / 49



Applications


yj = |〈x , aj〉|2

= tr(xx∗aja∗j ) = tr(Xaja





46 / 49



Applications







46 / 49



Applications







46 / 49

Recovery from Gaussian measurement vectorsTheorem (Kueng, R, Terstiege 2014; Kabanava, K, R, T 2015)Let a1, . . . , am be independent standard complex Gaussian randomvectors in Cn. If m ≥ Cnr then with probability at least 1− e−cm forevery X ∈ Sn and

y = A(X ) + n =(a∗j Xaj

)mj=1

+ n

the solution X ] ofminZ<0‖A(Z )− y‖2

satisfies

‖X − X ]‖F ≤ C1‖X − Xr‖∗√

r+ C2‖n‖2.

Generalizes PhaseLift estimate of Candes, Li (2012) from rank r = 1 toarbitrary rank and includes stability for approximately low rank matrices.

Positive semidefinite constraint allows to replace nuclear norm

minimization by least squares (result also holds for nuclear norm

minimization in Hn).47 / 49

Complex projective designs

Previous result can be extended for rank-one measurements takenwith respect to randomly chosen elements from an (approximate)complex projective t-design with t = 4.Requires (Kueng, R, Terstiege 2014)

m ≥ Cnr log(n)

Improves and generalizes previous result for PhaseLift due toGross, Krahmer, Kung (2013), where m ≥ Ctn1+2/t log2(n) forrank r = 1.

Applies to some approximate 4-designs that may be implementedin a real quantum tomography experiment.(Quantum computer?)

48 / 49

Complex projective designs

Previous result can be extended for rank-one measurements takenwith respect to randomly chosen elements from an (approximate)complex projective t-design with t = 4.Requires (Kueng, R, Terstiege 2014)

m ≥ Cnr log(n)

Improves and generalizes previous result for PhaseLift due toGross, Krahmer, Kung (2013), where m ≥ Ctn1+2/t log2(n) forrank r = 1.

Applies to some approximate 4-designs that may be implementedin a real quantum tomography experiment.(Quantum computer?)

48 / 49

The End

49 / 49

Date post:	18-Oct-2020
Category:	Documents
Upload:	others
View:	5 times
Download:	0 times

Compressive Sensing in Imaging - mathe.tu-freiberg.debernstei/Web5/MOIMA2016-Rauhut.pdf ·...

Documents