Total Variation and Geometric Regularization for Inverse Problems

Regularization in StatisticsSeptember 7-11, 2003

BIRS, Banff, Canada

Tony ChanDepartment of Mathematics, UCLA

Outline

• TV & Geometric Regularization (related concepts)• PDE and Functional/Analytic based• Geometric Regularization via Level Sets Techniques• Applications (this talk):

– Image restoration

– Image segmentation

– Elliptic Inverse problems

– Medical tomography: PET, EIT

Regularization: Analytical vs Statistical

• Analytical: – Controls “smoothness” of continuous functions– Function spaces (e.g. Sobolov, Besov, BV)– Variational models -> PDE algorithms

• Statistical:– Data driven priors– Stochastic/probabilistic frameworks– Variational models -> EM, Monte Carlo

Taking the Best from Each?

• Concepts are fundamentally related: – e.g. Brownian motion Diffusion Equation

• Statistical frameworks advantages: – General models

– Adapt to specific data

• Analytical frameworks advantages:– Direct control on smoothness/discontinuities, geometry

– Fast algorithms when applicable

Total Variation Regularization

dxuuTV ||)(

• Measures “variation” of u, w/o penalizing discontinuities.

• |.| similar to Huber function in robust statistics.

• 1D: If u is monotonic in [a,b], then TV(u) = |u(b) – u(a)|, regardless of whether u is discontinuous or not.

• nD: If u(D) = char fcn of D, then TV(u) = “surface area” of D.

• (Coarea formula)

• Thus TV controls both size of jumps and geometry of boundaries.

• Extensions to vector-valued functions

• Color TV: Blomgren-C 98; Ringach-Sapiro, Kimmel-Sochen

drdsfdxufnR

ru

)(||}{

The Image Restoration ProblemA given Observed image z

Related to True Image u

Through Blur K

And Noise n

Blur+NoiseInitial Blur

Inverse Problem: restore u, given K and statistics for n.

Keeping edges sharp and in the correct location is a key problem !

nuKz

Total Variation Restoration

2||||2

1)()(min zKuuTVuf

u

0n

uGradient flow:

)(||

)( *zKKuKu

uugut

anisotropic diffusion data fidelity

dxuuTV ||)(

* First proposed by Rudin-Osher-Fatemi ’92.

* Allows for edge capturing (discontinuities along curves).

* TVD schemes popular for shock capturing.

Regularization:

Variational Model:

Comparison of different methods for signal denoising & reconstruction

Image Inpainting (Masnou-Morel; Sapiro et al 99) Disocclusion

Graffiti Removal

Unified TV Restoration & Inpainting model

EDE

dxdyuudxdyuuJ ,||2

||][ 20

,0)(||

0

uuu

ue

.0; ,,DzEz

e

(C- J. Shen 2000)

TV Inpaintings: disocclusion

Examples of TV Inpaintings

Where is the Inpainting Region?

TV Zoom-in

Inpaint Region: high-res points that are not low-res pts

Edge Inpainting

edge tube T

No extra data are needed. Just inpaint!

Inpaint region: points away from Edge Tubes

Extensions

• Color (S.H. Kang thesis 02)• “Euler’s Elastica” Inpainting (C-Kang-Shen 01)

– Minimizing TV + Boundary Curvature

• “Mumford-Shah” Inpainting (Esedoglu-Shen 01)– Minimizing boundary + interior smoothness:

S S

SudSu 2

,||(min

Geometric Regularization

• Minimizing surface area of boundaries and/or volume of objects

• Well-studied in differential geometry: curvature-driven flows

• Crucial: representation of surface & volume• Need to allow merging and pinching-off of

surfaces• Powerful technique: level set methodology

(Osher/Sethian 86)

Level Set Representation (S. Osher - J. Sethian ‘87)

Inside C

Outside C

Outside C0

0

0

0C

nn

n

||

,||

Normal

divKCurvaturen

Example: mean curvature motion

* Allows automatic topology changes, cusps, merging and breaking.

• Originally developed for tracking fluid interfaces.

0),(|),( yxyxC

C= boundary of an open domain

Application: “active contour”

Initial Curve Evolutions Detected Objects

objects theof boundaries on the stop tohas curve the

in objectsdetect to curve a evolve

: imagean giving

0

0

uC

u

Basic idea in classical active contours

Curve evolution and deformation (internal forces):

Min Length(C)+Area(inside(C)) Boundary detection: stopping edge-function (external forces)

Example:

0)(lim , ,0

tgggt

puGug

||1

1|)(|

00

Snake model (Kass, Witkin, Terzopoulos 88)

1

0

1

0

2 |)))(((||)('|)(inf dssCIgdssCCFC

1

0

|)))(((||)('|2)(inf dssCIgsCCFC

Geodesic model (Caselles, Kimmel, Sapiro 95)

Limitations - detects only objects with sharp edges defined by gradients

- the curve can pass through the edge

- smoothing may miss edges in presence of noise

- not all can handle automatic change of topology

Examples

A fitting term “without edges”

)(

220

2

)(

10 ||||CoutsideCinside

dxdycudxdycu

where Cuaveragec

Cuaveragec

outside )(

inside )(

02

01

Fit > 0 Fit > 0 Fit > 0 Fit ~ 0

Minimize: (Fitting +Regularization)

Fitting not depending on gradient detects “contours without gradient”

)( )(

220

210

21,,

||||

))((||),,(inf21

Cinside Coutside

Ccc

dxdycudxdycu

CinsideAreaCCccF

An active contour model “without edges”

Fitting + Regularization terms (length, area)

C = boundary of an open and bounded domain

|C| = the length of the boundary-curve C

(C. + Vese 98)

Mumford-Shah Segmentation 89

S S

SudSuuu ))(||(min 22

,0

S=“edges”

MS reg: min boundary + interior smoothness

CV model = p.w. constant MS

Variational Formulations and Level Sets(Following Zhao, Chan, Merriman and Osher ’96)

The Heaviside function

The level set formulation of the active contour model

0),( :),( yxyxC

0 if ,0

0 if ,1)(

H

dxdyHCinsideArea

HC

)())((

|)(|||Length

dxdyHcyxudxdyHcyxu

dxdyHHccF

ccFcc

))(1(|),(|)(|),(|

)(|)(|),,(

),,(inf

220

210

21

21,, 21

))),((1()),((),( 21 yxHcyxHcyxu

The Euler-Lagrange equations

),,(inf 21, ,21

ccFcc

dxdyH

dxdyHu

cdxdyH

dxdyHu

c))(1(

))(1(

)( ,)(

)(

)(0

2

0

1

),,(for Equation yxt

),(),,0(

)()(||

)(

0

220

210

yxyx

cucudivt

2 1 andfor Equations cc

Using smooth approximations for the Heaviside and Delta functions

Advantages

Automatically detects interior contours!

Works very well for concave objects

Robust w.r.t. noise

Detects blurred contours

The initial curve can be placed anywhere!

Allows for automatical change of topolgy

Experimental ResultsC of Evolution ),( Averages 21 cc

A plane in a noisy environment

Europe nightlights

0

0

2

1

0

0

2

1

0

0

2

1

0

0

2

1

0

0

4-phase segmentation2 level set functions

2-phase segmentation1 level set function

}0{

:Curves

}0{}0{

:Curves

21

Multiphase level set representations and partitions allows for triple junctions, with no vacuum and no overlap of phases phases 2 ),...,( 1

nn

Example: two level set functions and four phases

|)(||)(|

))(1))((1(||)())(1(||

))(1)((||)()(||),(

Energy

))(1))((1()())(1(

))(1)(()()(

vectorConstant ),,,(

functionsset level The ),(

21

212

000212

010

212

100212

110),(

21002101

21102111

00011011

21

HH

dxdyHHcudxdyHHcu

dxdyHHcudxdyHHcucFInf

HHcHHc

HHcHHcu

ccccc

c

0,0 ,0,0 ,0,0 ,0,0

:segmentsor phases 4 ),( functionsset level 2

21212121

21

Phase 11 Phase 10 Phase 01 Phase 00

mean(11)=45 mean(10)=159 mean(01)=9 mean(00)=103

An MRI brain image

S S

SudSfuu ))(||(min 22

,

References for PDE & Level Sets in Imaging

* IEEE Tran. Image Proc. 3/98, Special Issue on PDE Imaging

* J. Weickert 98: Anisotropic Diffusion in Image Processing

* G. Sapiro 01: Geometric PDE’s in Image Processing

• Aubert-Kornprost 02: Mathematical Aspects of Imaging Processing

• Osher & Fedkiw 02: “Bible on Level Sets”

• Chan, Shen & Vese Jan 03, Notices of AMS