Cs: Cs: compressecompressed sensingd sensing

Jialin pengJialin peng

• Introduction• Exact/Stable Recovery Conditions

– -norm based recovery– OMP based recovery

Some related recovery algorithmsSparse RepresentationApplications

p

OutlineOutline

Introduction

high-density sensorhigh speed sampling……A large amount of

sampled data will be discarded

A certain minimum number of samples is required in order to perfectly c

apture an arbitrary bandlimited signal

Data Storage

Receiving & Storage

Sparse PropertySparse Property• Important classes of signals have naturally spars

e representations with respect to fixed bases (i.e., Fourier, Wavelet), or concatenations of such bases.

• Audio, images …• Although the images (or their features) are natur

ally very high dimensional, in many applications images belonging to the same class exhibit degenerate structure.

• Low dimensional subspaces, submanifolds• representative samples—sparse representation

Transform coding: JPEG, JPEG2000, MPEG, and MP3

The GoalThe GoalDevelop an end-to-end system • Sampling• processing • reconstruction• All operations are performed at a low rate:

below the Nyquist-rate of the input (too costly, or even physically impossible)

• Relying on structure in the input

Sparse: the simplest choice is the best oneSparse: the simplest choice is the best one

• Signals can often be well approximated as a linear combination of just a few elements from a known basis or dictionary.

• When this representation is exact ,we say that the signal is sparse.

Remark:

In many cases these high-dimensional signals contain relatively little information com

pared to their ambient dimension.

Introduction

high-density sensorhigh speed sampling……A large amount of

sampled data will be discarded

A certain minimum number of samples is required in order to perfectly c

apture an arbitrary bandlimited signal

Data Storage

Receiving & Storage

IntroductionSparse priors of sig

nalNonuniform sampli

ngImaging algorithm:

optimization

Alleviated sensorReduced data……

modified sensor

Data Storage

Receiving & Storageoptimization

Introduction

x×1N ×1N

M N

Φ y• =

×1N

×1MSensing Matrix

×N N

×M N

compressioncompressionFind the most concise representation:

Compressed sensing: sparse or compressible representation • A finite-dimensional signal having a sparse or compressible repr

esentation can be recovered from a small set of linear, nonadaptive measurements

• how should we design the sensing matrix A to ensure that it preserves the information in the signal x?.

• how can we recover the original signal x from measurements y?• Nonlinear:1. Unknown nonzero locations results in a nonlinear model:the choice of which dictionary elements are used can change from signal to

signal . 2. Nonlinear recovering algorithmsthe signal is well-approximated by a signal with only k nonzerocoefficients

IntroductionLet be a matrix of size with .For a –sparse signal , let b

e the measurement vector.Our goal is to exact/stable recovery the unknow

n signal from measurement.The problem is under-determined.Thanks for the sparsity, we can reconstruct the s

ignal via .

Φ M N M NK Nx M y Φx

How can we recovery the unknown signal:

Exact/Stable Recovery Condition

0min , s.t. x y Φx

Exact/stable recovery conditions• The spark of a given matrix A• Null space property (NSP) of order k• The restricted isometry propertyRemark: verifying that a general matrix A satisfi

es any of these properties has a combinatorial computational complexity

Exact/stable recovery conditions

The restricted isometry constant (RIC) is defined as the smallest constant which satisfy:

The restricted orthogonality condition (ROC)is the smallest number such that:

2 2 2

2 2 21 1K K x Φx x

K

,K K ,K K

, 2 2, K K Φu Φv u v

Restricted Isometry Property

Exact/stable recovery conditions

Solving minimization is NP-hard, we usually relax it to the or minimization.

01 , 0 1p p

Exact/stable recovery conditionsFor the inaccurate measurement ,

the stable reconstruction model is

1

1 2min , s. t. x y Φx

y Φx e

Exact/stable recovery conditionsSome other Exact/Stable Recovery

Conditions:

Exact/stable recovery conditionsBraniuk et al. have proved that for some random

matrices, such as Gaussian, Bernoulli, ……

we can exactly/stably reconstruct unknown signal with overwhelming high probability.

Exact/stable recovery conditions

cf: minimization1

Exact/stable recovery conditionsSome evidences have indicated that

with , can exactly/stably recovery signal with fewer measurements.

min , s. t.p

x y Φx

0 1p

Quicklook InterpretationQuicklook Interpretation• Dimensionality-reducing projection.• Approximately isometric embeddings, i.e., pairwise

Euclidean distances are nearly preserved in the reduced space

RIP

Quicklook Interpretation Quicklook Interpretation

Quicklook InterpretationQuicklook Interpretation

•the ℓ2 norm penalizes large coefficients heavily, therefore solutions tend to have many smaller coefficients.•In the ℓ1 norm, many small coefficients tend to carry alarger penalty than a few large coefficients.

AlgorithmsAlgorithms• L1 minimization algorithms iterative soft thresholding iteratively reweighted least squares …• Greedy algorithms Orthogonal Matching Pursuit iterative thresholding• Combinatorial algorithms

CS builds upon the fundamental fact thatCS builds upon the fundamental fact that

• we can represent many signals using only a few non-zero coefficients in a suitable basis or

dictionary.

• Nonlinear optimization can then enable recovery

of such signals from very few measurements.

• Sparse property• The basis for representing the data• incoherent->task-specific (often overco

mplete) dictionary or redundant one

MRI ReconstructionMRI Reconstruction

MR images are usually sparse in certain transform domains, such as finite difference and wavelet.

Sparse RepresentationConsider a family of images, representing natural and typical image content:•Such images are very diverse vectors in•They occupy the entire space?•Spatially smooth images occur much more often than highly non-smooth and disorganized images •L1-norm measure leads to an enforcement of sparsity of the signal/image derivatives.•Sparse representation

Matrix completion algorithmsRecovering a unknown (approximate) low-

rank matrix from a sampling set of its entries.

min rank : , ,ij ijX

X X M i j NP-hard

Convex relaxation

*min : , ,ij ijX

X X M i j

*min , ,ij ij FX

X X M i j Unconstraint