Martin Burger Total Variation 1 Cetraro, September 2008 Numerical Schemes Wrap up approximate...

Total Variation 1

Cetraro, September 2008Martin Burger

Numerical SchemesWrap up approximate formulations of subgradient relation

Total Variation 2


Numerical SchemesPrimal Approximation

Primal Fixed Point

Dual Approximation

Dual Fixed Point

Dual Fixed Point for Primal Relation

Total Variation 3


1. Fixed point methods Matrix form

Total Variation 4


Fixed Point Schemes IPrimal Gradient Method Based on approximation of F:Fixed-point approach for first optimality equation

Total Variation 5


Fixed Point Schemes IPrimal Gradient Method Based on Approximation, Rudin-Osher-Fatemi 89

+ easy to implement, efficient iteration steps+ global convergence (descent method for variational problem)

- dependent on approximation- slow convergence- severe step size restrictions (explicit approximation of differential operator)

Total Variation 6



Special case of fixed point methods with choice

Total Variation 7



+ easy to implement, efficient iteration steps+ global convergence (descent method for variational problem)

- dependent on approximation- slow convergence- severe step size restrictions (explicit approximation of differential operator)

Total Variation 8


Fixed Point Schemes IIDual Gradient Projection MethodDual methods eliminate also u

Total Variation 9


Fixed Point Schemes IIDual Gradient Projection MethodDo gradient step on the quadrativc functional and project back to constraint set M

Note: can be interpreted as a scheme where the first equation is always satisfied, i.e. special case with

Total Variation 10


Fixed Point Schemes IIDual Gradient Projection Method, Chambolle 05, Chan et al 08, Aujol 08 + easy to implement, efficient iteration steps+ global convergence (descent method for dual problem)+ no approximation necessary- slow convergence- needs inversion of A*A, hence good for ROF, bad for inverse problems

Obvious generalization for last point: use preconditioning of A*A

Total Variation 11


Fixed Point Schemes IIChambolle‘s MethodDual method, again eliminates u

Total Variation 12


Fixed Point Schemes IIIChambolle‘s MethodComplicated derivation from dual minimization problem in original paper

Note: can be interpreted as a scheme where the first equation is always satisfied, in addition using dual fixed point form for the primal subgradient relation, i.e. special case with

Total Variation 13


Fixed Point Schemes IIIChambolle‘s Method, Chambolle 04 + easy to implement, efficient iteration steps+ global convergence (descent method for dual problem)+ no approximation necessary- slow convergence- needs inversion of A*A, hence good for ROF, bad for inverse problems

Obvious generalization for last point: use preconditioning of A*A

Total Variation 14


Fixed Point Schemes IVInexact Uzawa method, Zhu and Chan 08

Primal gradient descent, dual projected gradient ascent in the reduced Lagrangian (for u and w)

Coincides with dual gradient projection if A = I and appropriate choice of damping parameter

Total Variation 15


Fixed Point Schemes VPrimal Lagged Diffusivity with Approximation, Vogel et al 95-97

Approximate smoothed primal optimality condition

Semi-implicit treatment of differential operator

Total Variation 16


Fixed Point Schemes VSpecial case with choice

Total Variation 17


Fixed Point Schemes VPrimal Lagged Diffusivity with Approximation+ acceptable step-size restrictions+ global convergence (descent method for variational problem)

- dependent on approximation- still slow convergence- differential equation with changing parameter to be solved in each step

Total Variation 18


2. Thesholding methods

C is damping matrix, possible perturbation T is thresholding operator

Total Variation 19


Thresholding Methods IPrimal Thresholding Method, Daubechies-Defrise-DeMol 03

Only used for D = -I

Introduce

Total Variation 20


Thresholding Methods IPrimal Thresholding Method, Daubechies-Defrise-DeMol 03

Hence, special case with

Total Variation 21


Thresholding Scheme I Primal Thresholding Method+ easy to implement, efficient iteration steps if D= - I - slow convergence- cannot be generalized to cases where D is not invertible

Total Variation 22


Thresholding Methods IIAlternating Minimization, Yin et al 08, Amat-Pedregal 08

Use quadratic penalty for the gradient constraing (Moreau-Yosida regularization)

Alternate minimization with respect to the variables

Total Variation 23



Use quadratic penalty for the gradient constraing (Moreau-Yosida regularization)

Alternate minimization with respect to the variables

Total Variation 24



Introduce

Hence, special case with

Total Variation 25


Thresholding Scheme IIPrimal Thresholding Method+ efficient iteration steps if D*D and A*A can be jointly inverted easily (e.g. by FFT) + treats differential operator implicitely, no severe stability bounds

-Linear convergence- Smoothes the regularization functional

Total Variation 26


Thresholding Methods IIISplit Bregman, Goldstein-Osher 08

Original motivation from Bregman iteration, can be rewritten as

Total Variation 27


Thresholding Methods IIISplit Bregman, Goldstein-Osher 08

Difficult to solve directly, hence subiteration with thresholdingAfter renumbering

Total Variation 28


Thresholding Scheme IIISplit Bregman, Goldstein-Osher 08

+ efficient iteration steps if D*D and A*A can be jointly inverted easily (e.g. by FFT) + treats differential operator implicitely, no severe stability bounds+ does not need smoothing

- Linear convergence

Total Variation 29


2. Newton type methods Matrix form

Total Variation 30


Newton-type MethodsPrimal or dual + fast local convergence - global convergence difficult - dependent on approximation (Newton-matrix degenerates)- needs inversion of large Newton matrix

Good choice with efficient preconditioning for linear system in each iteration step

Date post:	21-Dec-2015
Category:	Documents
View:	213 times
Download:	0 times

Martin Burger Total Variation 1 Cetraro, September 2008 Numerical Schemes Wrap up approximate...

Documents