DICTIONARY LEARNING IN THE ANALYSIS SPARSE REPRESENTATION WITH OPTIMIZATION ON STIEFEL MANIFOLDYujie Li1, Shuxue Ding2, Zhenni Li2, Xiang Li2, Benying Tan2

1National Institute of Advanced Industrial Science and Technology (AIST) 2School of Computer Science and Engineering, The University of Aizu, Fukushima, Japan

INTRODUCTIONSparse representation has been proven to be a powerful tool forsignals and images processing. This paper addresses sparserepresentation with the so-called analysis model. We pose theproblem as to learn an analysis dictionary from signals using anoptimization formulation with an orthogonal constraint.

PROPOSED METHOD

EXPERIMENTS AND DISCUSSION

CONCLUSION

➢Image Denoising with Learned Analysis Dictionary.

➢Exact recovery of analysis operators.

➢we introduce manifold constraint for the dictionary learningalgorithm for analysis sparse representation, which is parallelto the synthesis model in its rationale and structure.

➢To efficiently solve the orthogonality constraint in analysisdictionary learning formulation, project to Stiefel Manifold isan efficient method to solve this optimization problem.

➢Numerical experiments on recovery of analysis dictionaryshow the effectiveness of the proposed algorithm. In addition,for realistic applications, the proposed algorithms show goodperformances in signal denoising and classification.

➢However, with a wealth of mathematical and computer toolsalready developed, much work remains to be done. In thefuture, we will apply the proposed algorithm to moreapplications such as inpainting and deblurring.

Dictionary learning with

orthogonal constraint

Approximation

by projection

Optimization

on manifold

Not an accurate solution

The optimization problem

with constraints,

M can be any constraints

In our situation:

min ( )M

fX

X

TM X X X I

2 2

1

e.g. 1 , 1N

ijFj

X X X X

The standard

(Euclidean) gradient

The gradient in

the tangent space

The projection PX

The retraction

operator RX

➢Dictionary update with manifold method.

➢Problem formulation.

Given an observed signals set Y, which is a noisy versionof a signal X. We formulate the sparse representation inanalysis model with ℓ1 norm as,

where L means the constraints on the dictionary .

Use an alternative method.

Update

2

1,min , s.t. , ,

FL

Ω XΩX Y X Ω

Ω2 2

1, ,

2

min ,

s.t. ; .

F F

T

ii c

Ω X ZZ Y X ΩX Z

Ω Ω I

The uniformly normalized constraintΩ Z X

The orthogonal constraint

We only consider the terms with dictionary ,

Use nonmonotone line search with Barzilai-Borwein (BB)steps size to solve the problem with orthogonality constraint.

2

2min s.t. ; .T

iFi c

ΩΩX Z Ω Ω I

Ω

1

1

(2 2 ) (2 2 ),

( ) ( ) .2 2

T T T T T

k k

A ΩXX ZX Ω Ω ΩXX ZX

Ω I A I A Ω

➢Algorithm 1 : Manifold based Analysis Dictionary Learning

Algorithm (MADL).

We compared our algorithm MADL with the state-of-the-artalgorithm Analysis operator learning (AOL) and the results isrepresented in Fig.3. We can see our algorithm can reach to abetter recovery ratio even when the cosparsity is lower comparedwith AOL.

➢Classification with Learned Analysis Dictionary.

We applied ourproposed algorithmto the classification,which is conductedon the database ofUSPS handwrittendigits.

Rec

ove

ry r

ate