Multilevel approach for signal restoration problems with Toeplitz matrices

Malena Español, Tufts UniversityMisha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

2

Outline Background Multilevel Method Algorithm Implementation Numerical Example Conclusion and Future Work

10th Copper Mountain Conference on Iterative Methods

3

Ill-posed Problem

A problem is ill-posed if its solution is not unique, or its solution does not depend continuously

on the data

)()( ,

kindfirst ofequation integral Fredholm :Example

sgdttft)K(s

10th Copper Mountain Conference on Iterative Methods

4

Discrete Ill-Posed problem

holds condition Picard Discrete

yoscillator more become ectorssingular v

noise (white)unknown is

gap without aluessingular v Decaying

:Properties

e

matrix dconditione-ill large, a is where

,

model theand ,given , Find

nm

truetrue

true

RA

ebbAx

bAx

10th Copper Mountain Conference on Iterative Methods

5

Need for regularization

)(

1

)(

1ii

)(

1i

†

)(

1i

†

)(

)(

:are solutions squareleast Then the

. of SVD theLet

Arank

i

Arank

itrue

i

T

i

trueT

Arank

i i

trueT

true

Arank

i i

trueT

truetrue

errorxeubu

ebuebAx

bubAx

AVUA

ii

i

i

10th Copper Mountain Conference on Iterative Methods

6

Regularization

small is if 0

large is if 1

where

solution dRegularize

1i

i

ii

n

i i

T

ireg

bux i

10th Copper Mountain Conference on Iterative Methods

7

Regularization methods

CGLS.or LSQR, eg.

methods, Krylov of iterationsk :Methods Iterative

min

:tionRegulariza Tikhonov

: (TSVD) SVD Truncated

2

2

22

2

1i22

2

1i

LxbAx

bux

bux

x

n

i i

T

i

iTik

k

i i

T

TSVD

i

i

10th Copper Mountain Conference on Iterative Methods

8

Two-Level Method

cii

ic

i

iiii

iiii

iii

iiii

iii

iiii

iii

xxx

rxA

xAbr

xPxx

rxA

PARA

rRr

xAbr

bxA

Solve""

Solve""

Solve""

1

111

1

1

Pre-Smoothing

Post-Smoothing

Coarse-Grid Correction

Restriction

Prolongation

10th Copper Mountain Conference on Iterative Methods

9

Multilevel Method

If End

Solve

,

Solve

Else

Solve

gridcoarsest If

function

1

111

11

cii

ic

i

iiii

iiii

iii

iiiiiii

iiii

iii

iii

iii

xxx

rxA

xAbr

xPxx

),bMGM(Ax

PARArRr

xAbr

bxA

bxA

),bMGM(Ax

10th Copper Mountain Conference on Iterative Methods

10

Restriction and Prolongation Operators

11000000

00110000

00001100

00000011

11000000

00110000

00001100

00000011

2

2TW

Haar wavelet transform

W2W1

10th Copper Mountain Conference on Iterative Methods

11

Wavelet Domain

2

1

2

1

43

21

ˆ

ˆ

ˆ

ˆˆˆ

ˆˆ

.ˆ and ˆ ,ˆ where

ˆˆˆ

becomes domain In wavelet

b

b

x

x

AA

AA

bWbxWxAWWA

bxA

bAx

TTT

10th Copper Mountain Conference on Iterative Methods

12

Coarse-scale equation

x

1x̂111

22111

2

1

2

1

43

21

ˆˆˆ

ˆˆˆˆˆ

ˆ

ˆ

ˆ

ˆˆˆ

ˆˆ

bxA

xAbxA

b

bx

x

AA

AA

10th Copper Mountain Conference on Iterative Methods

13

.ˆˆˆ

Then,

ˆˆ

ˆˆˆ

ˆ

ˆˆ

ˆ

ˆˆ

ˆˆˆˆˆˆˆˆˆ

111

)ˆ(

1i

2211

221122111

1

bxA

xAueubu

xAebxAbxA

Arank

i i

T

i

T

i

trueT

true

iii

Coarse-scale equation

10th Copper Mountain Conference on Iterative Methods

14

p

p

p

xxLbxA

bAx

bxA

1

2

2111ˆ

111

111

ˆˆˆˆmin

3)or 2 ,ˆ,ˆLSQR(ˆ

solve we

ˆˆˆ

ofsolution dregularize aget To

1

Coarse-Grid Correction

Pre-smoothing

Pre-smoothing

Coarse-Grid Correction

10th Copper Mountain Conference on Iterative Methods

15

1

3

1

2

12

4

2 ˆˆ

ˆ

ˆ

ˆˆ

ˆ

ˆx

A

A

b

bx

A

A

Post-Smoothing

p

pprep

newx

xWxWxLrxA

A)ˆˆ(ˆ

ˆ

ˆmin 2211

2

2

2

4

2

ˆ2

newr

10th Copper Mountain Conference on Iterative Methods

16

Multilevel Method

If End

ˆ

),,,(Newtonˆ

ˆ

ˆ

;

If End

,3),(

gridfinest If

Else

)(Newtonor solvedirect

gridcoarsest If

function

122

21

2

111

1111

111

11

iinew

i

inew

inew

Tii

inew

iiinew

iiinew

iii

iTii

Ti

iiii

iii

iii

iii

xWxx

xLWrWWAx

xAbr

xWxx

),bMGM(Ax

WAWArWb

xAbr

bALSQRx

non

,bAx

),bMGM(Ax

10th Copper Mountain Conference on Iterative Methods

17

Toeplitz Matrices

),,,,,(: 1101)1(

0321

)3(012

)2(101

)1(210

mm

mmm

m

m

m

ttttttvectorToeplitz

tttt

tttt

tttt

tttt

A

10th Copper Mountain Conference on Iterative Methods

18

Toeplitz structure inheritance

A

43

21

ˆˆ

ˆˆˆ

AA

AAA

.ˆeach ofvector -Toeplitz theknow weMoreover,

Toeplitz. are ˆ and ˆ,ˆ,ˆ then Toeplitz, is If

:Theorem

4321

iA

AAAAA

10th Copper Mountain Conference on Iterative Methods

19

Pre-smoothing

productvector -matrixFast structure Toeplitz

LSQR. of iterations 3or 2 applyingby

ˆˆˆ

solve We

111

bxA

10th Copper Mountain Conference on Iterative Methods

20

Coarse-Grid Correction

.ˆon depend and where,ˆ

ˆmin

Method sNewton'by ˆˆˆˆmin

or

solvedirect aby ˆˆˆ

solve We

11

2

21

11

1

2

2111ˆ

111

1

xrDxDL

rd

DL

A

xLbxA

bxA

d

p

p

p

x

10th Copper Mountain Conference on Iterative Methods

21

Post-Smoothing

.ˆon depend ~ and ~, where,~

~ˆ

ˆ

min

Method sNewton'by

)ˆˆ(ˆˆ

ˆmin

solve We

221

2

2

2

1

2

4

2

2211

2

2

2

4

2

ˆ2

xrrDrcD

rd

DLW

A

A

xWxWxLrxA

A

d

p

pprep

newx

10th Copper Mountain Conference on Iterative Methods

22

Numerical Example

1.1 operator, derivative

)01.0,0( with

solution edgy :

matrix symmetric Toeplitz, Gaussian, :

pL

bNeeAxb

x

A

true

true

10th Copper Mountain Conference on Iterative Methods

23

Numerical Example

truex b

MGMxLSQRx

10th Copper Mountain Conference on Iterative Methods

24

Conclusions and Future work

General Cases – Non Structured Matrices

Parameter Selection Adaptive p-norm Extension to 2D, 3D