Nonlocal Total Variation Based Speckle

Denoising Model

Arundhati Bagchi Misra and Hyeona Lim

Abstract A large range of methods covering various fields of mathematics are

available for denoising an image. The initial denoising models are derived from

energy minimization using nonlinear partial differential equations (PDEs). The

filtering models based on smoothing operators have also been used for denoising.

Among them the recently developed nonlocal means method proposed by Buades,

Coll and Morel in 2005 is quite successful. Though the method is very accurate, it

is very slow and hence quite impractical. In 2008, Gilboa and Osher extended

some known PDE and variational techniques in image processing to the nonlocal

framework and proposed the nonlocal total variation method for Gaussian noise.

We used this idea to develop a nonlocal model for speckle noise. Here we have

extended the speckle model introduced by Krissian et al. in 2005 to the nonlocal

framework. The Split Bregman scheme is used solve this new model.

Keywords Image denoising � Speckle denoising models � Nonlocal means �Nonlocal PDE � Nonlocal TV

A. B. Misra (&)

Department of Mathematical Sciences, Saginaw Valley State University, University Center,

MI 48710, USA

e-mail: abmisra@svsu.edu

H. Lim

Department of Mathematics and Statistics, Mississippi State University, Mississippi,

MS 39762, USA

e-mail: hlim@math.msstate.edu

Mohan S. and S. S. Kumar (eds.), Proceedings of the Fourth International Conference

on Signal and Image Processing 2012 (ICSIP 2012), Lecture Notes in Electrical

Engineering 221, DOI: 10.1007/978-81-322-0997-3_46, Ó Springer India 2013

517

1 Introduction

Image restoration is a very important process and is often necessary as a pre-

processing for other imaging techniques such as segmentation and compression. In

general, an observed image f is represented by the equation

f ¼ uþ n; ð1Þ

where u is the original noise free image, f is the observed noisy image and n is the

Gaussian noise. Here u; f : X � I R2 ! I R. For any denoising model, the main

objective is to reconstruct u from an observed image f .

In 1992 Rudin et al. [8] proposed the total variation (TV) denoising model as

the minimization problem:

minu

Z

X

ruj jdx ð2Þ

subject to the constraints,

Z

X

fd x ¼Z

X

u dx ð3Þ

and

Z

X

1

2f ÿ uð Þ2dx ¼ r2; ð4Þ

where r is the standard deviation of the noise n. These constraints ensure that the

resulting image and the observed images are quite close to each other.

Combining the above constraints the TV functional is obtained by

FðuÞ ¼Z

X

ruj jdxþ k

2

Z

X

f ÿ uð Þ2dx: ð5Þ

Here k is a constraint parameter. The equivalent Euler–Lagrange equation gives

the TV denoising model as

ou

otÿr � ru

jruj

� �

¼ kðf ÿ uÞ: ð6Þ

To avoid singularities, it was regularized by using jruj � jreuj ¼ ðu2x þ u2y þ e2Þ1=2.

518 A. B. Misra and H. Lim

2 Preliminaries

Here we present the initial speckle denoising model and the filtering based non-

local means method.

2.1 Speckle Denoising Model

In 2005 Krissian et al. [6] considered the speckle noise model by

f ¼ uþffiffiffi

up

n; ð7Þ

where u is the desired image to find, n is Gaussian noise and f denotes the observed

image. Thus,we have n ¼ f ÿ uffiffiffi

up . Then replacing f ÿ u in (5)with the new expression

for n, the minimization functional for speckle denoising is given by

FðuÞ ¼Z

X

jruj þ k

2

f ÿ uffiffiffi

up

� �2" #

dx: ð8Þ

From energy minimization of this functional, the TV-based speckle denoising

model can be derived as:

ou

otÿ u2

f þ ujreuj r � ru

jreuj

� �

¼ k jreuj ðf ÿ uÞ: ð9Þ

2.2 Nonlocal Means Method

In 2005 Buades et al. proposed the new state of art image denoising algorithm

known as nonlocal means algorithm [2]. The algorithm is given by the formula

NL½u�ðxÞ ¼ 1

CðxÞ

Z

X

eÿðGaIjuðxþ�Þÿuðyþ�Þj2Þð0Þ

h2 uðyÞdy; ð10Þ

where CðxÞ ¼Z

X

eÿðGaIjuðxþ�Þÿuðzþ�Þj2Þð0Þ

h2 dz. Here Ga is the Gaussian kernel with stan-

dard deviation a, h is a filtering parameter and uðxþ �Þ denotes the neighborhoodof the pixel x. Here each pixel value is denoised using the weighted average of all

the pixels in the image. Thus for a given discrete noisy image u ¼ fuðiÞ : i 2 Ig,the estimated value NL½u�ðiÞ, for a pixel i, is computed as

Nonlocal Total Variation Based Speckle Denoising Model 519

NL½u�ðiÞ ¼X

j2Iwði; jÞuðjÞ; ð11Þ

where the weight wði; jÞ depends on the similarity of the i and j pixels and satisfies

the conditions 0�wði; jÞ� 1 andX

jwði; jÞ ¼ 1. In this paper the authors mea-

sured the similarity of a square neighborhoodNi of a fixed size at each pixel i as a

decreasing function of the weighted Euclidean distance, jjuðNiÞ ÿ uðNjÞjj22;a,where j 2 I and a[ 0 is the standard deviation of the Gaussian kernel. The weight

function is defined as

wði; jÞ ¼ 1

CðiÞ eÿ

jjuðNiÞÿuðNjÞjj22;ah2 ; ð12Þ

where CðiÞ is the normalizing constant given by

CðiÞ ¼X

j

eÿ

jjuðNiÞÿuðNjÞjj22;ah2 : ð13Þ

The similar neighborhoods have a very small Euclidean distance which in turn

results in larger weight. Since the nonlocal means uses the similarity neighborhood

for denoising, it works really well for periodic or patterned textured cases. But

though the method is very good in removing noise, it is very slow and hence quite

impractical.

2.3 Nonlocal TV Model

In 2008, Gilboa and Osher extended some known PDE and variational techniques

in image processing to the nonlocal framework [3]. The motivation behind this

was to make any point interact with any other point in the image. Since the

classical derivatives were local operators, they had to redefine the required

operators in [3] following the ideas of Zhou and Scholkopf [10, 11]. Gilboa and

Osher proposed the nonlocal TV (NLTV) model based on the NLTV operator

(R

XjrNLuj), as follows:

ou

ot¼ jNLðuÞ ÿ kðuÿ f Þ

¼Z

X

wðx; yÞðuðyÞ ÿ uðxÞÞðjrNLujÿ1ðxÞ þ jrNLujÿ1ðyÞÞdyÿ kðuÿ f Þ;ð14Þ

where rNLð�Þ ¼ nonlocal gradient, divNLð�Þ ¼ nonlocal divergence and jNLð�Þ¼ divNL

rNL

jrNLj denotes the nonlocal curvature. Since the steepest descent scheme

(14) is very slow, a faster numerical scheme was introduced by Bresson in [1]. The

520 A. B. Misra and H. Lim

scheme is based on the Split Bregman method introduced in [4]. It was proved to

be very fast for regular TV.

3 Nonlocal TV Based Speckle Denoising Model

We now present our new model for speckle denoising. We extend the idea pre-

sented in Sect. 2.3 and develop the following nonlocal TV based speckle denoising

model:

minu

FðuÞ; FðuÞ ¼Z

X

jrNLuj þ kf ÿ uffiffiffi

up

� �2" #

dx ð15Þ

where rNL is a nonlocal gradient defined in [3]. For faster computation, we adopt

Split Bregman scheme [4] for finding a solution to our new model (15). We first

develop in Sect. 3.1 the Split Bregman scheme for Krissian et al. [6].

3.1 Split Bregman Scheme for Krissian et al. Model

As discussed in Sect. 2.1, the minimization functional was given by (8). For Split

Bregman scheme, we introduce d ¼ ru and construct the unconstrained functional

as

minu

FðuÞ; FðuÞ ¼Z

X

jdj þ kðf ÿ uÞ2

udxþ b

2kd ÿruÿ bk22: ð16Þ

Here b is a penalty parameter. We can now split this in two subproblems of u and d

given as:

minu

Z

X

kðf ÿ uÞ2

udxþ b

2kd ÿruÿ bk22; ð17Þ

mind

Z

X

jdjdxþ b

2kd ÿruÿ bk22: ð18Þ

Now,

Nonlocal Total Variation Based Speckle Denoising Model 521

o

oukðf ÿ uÞ2

u

!

¼ kÿ2uðf ÿ uÞ ÿ ðf ÿ uÞ2

u2

" #

¼ kðuÿ f Þð2uÿ uþ f Þ

u2

� �

¼ k

u2ðuÿ f Þðuþ f Þ:

ð19Þ

Therefore the optimality condition for u in (17) gives us

kðuþ f Þu2

ðuÿ f Þ þ b � ðd ÿruÿ bÞ ¼ 0

) kÿ bu2

uþ fD

� �

u ¼ kf ÿ bu2

uþ f� ðd ÿ bÞ:

ð20Þ

The optimality condition can be discretized using the definition of discrete gra-

dient, divergence and Laplacian as discussed in [3]. This gives

kþ 4bu2ij

uij þ fij

!

uij ¼ kfij þ bu2ij

uij þ fij

�

uiþ1;j þ uiÿ1;j þ ui;jþ1 þ ui;jÿ1

þ dx;iÿ1;j ÿ dx;ij þ dy;i;jÿ1 ÿ dy;ij

ÿ bx;iÿ1;j þ bx;ij ÿ by;i;jÿ1 þ by;ij

�

:

ð21Þ

Thus, we get

ukþ1ij ¼ 1

kþ 4bukij

2

ukijþfij

� �

h

kfij þ bukij

2

ukij þ fij

�

ukiþ1;j þ ukiÿ1;j þ uki;jþ1 þ uki;jÿ1

þ dkx;iÿ1;j ÿ dkx;ij þ dky;i;jÿ1 ÿ dky;ij ÿ bkx;iÿ1;j þ bkx;ij ÿ bky;i;jÿ1 þ bky;ij

�i

:

ð22Þ

Since there is no coupling between elements of d, the optimal value of d is

computed using the shrinkage operator described in [7, 9]:

dkþ1ij ¼

rukþ1ij þ bkij

jrukþ1ij þ bkijj

maxfjrukþ1ij þ bkijj ÿ 1=b; 0g: ð23Þ

The variable b is initialized to zero and is updated after each Bregman iteration as:

522 A. B. Misra and H. Lim

bkþ1ij ¼ bkij þrukþ1

ij ÿ dkþ1ij : ð24Þ

The scheme (22)–(24) provides a much faster solution for the TV based speckle

denoising model introduced by Krissian et al. [6].

3.2 Split Bregman scheme for nonlocal TV speckle model

For our new model (15) the Split Bregman functional will be of the form

minu

FðuÞ; FðuÞ ¼Z

X

jdj þ kðf ÿ uÞ2

uþ b

2kd ÿrNLuÿ bk22: ð25Þ

Then the optimality condition for u gives us the equation

kÿ bu2

uþ fDNL

� �

u ¼ kf ÿ bu2

uþ fdivNLðd ÿ bÞ: ð26Þ

Denoting the discretized points x; y 2 X� X by i and j, and using the discrete

definition for divNL and Dbadhbox [1, 3], the discrete minimization scheme is given

as:

kui ÿ bu2i

ui þ fi

X

j

wijðuj ÿ uiÞ !

¼ kfi ÿ bu2i

ui þ fi

X

j

ffiffiffiffiffiffi

wijp ðdij ÿ dji ÿ bij þ bjiÞ

!

:

ð27Þ

Hence, the iteration steps are:

ukþ1i ¼ 1

kþ bu2i

uiþfi

P

j wij

"

bu2i

ui þ fi

X

j

wijukj þ kfi

ÿ bu2i

ui þ fi

X

j

ffiffiffiffiffiffi

wij

p ðdij ÿ dji ÿ bij þ bjiÞ!#

;

ð28Þ

Table 1 Summary of numerical results

Krissian et al. Nonlocal means NLTV

Images Time (s) PSNR Time (s) PSNR Time (s) PSNR

Lenna (PSNR ¼ 24:51) 0:72 26:39 11:46 28:21 4:67 29:10

Gallstones 0:14 ÿ 11:92 ÿ 4:62 ÿ

Nonlocal Total Variation Based Speckle Denoising Model 523

dkþ1ij ¼

ffiffiffiffiffiffi

wijp ðukþ1

j ÿ ukþ1i Þþ bkij

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

P

jwijðukþ1j ÿ ukþ1

i Þ2þ bkij2

q max

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

X

j

wijðukþ1j ÿ ukþ1

i Þ2þ bkij2

s

ÿ 1

b;0

0

@

1

A;

ð29Þ

bkþ1ij ¼ bkij þ

ffiffiffiffiffiffi

wij

p ðukþ1j ÿ ukþ1

i Þ ÿ dkþ1ij : ð30Þ

(28–30) provides a fast and accurate nonlocal scheme for TV based speckle

denoising model.

4 Numerical Results

The numerical results are displayed in the next pages. We have considered the

images of size 128� 128 and the summary of results are in Table 1. Other than the

visual results, we have used peak-signal-to-noise ratio (PSNR) for synthetic

images, which was mentioned in [5], to measure the efficiency of the models.A

cleaner image provides a higher PSNR value. We have provided the absolute

residual ðjf ÿ ujÞ and the speckle noise residual�

n ¼ fÿuffiffi

up�

for all images. It is

evident that the new nonlocal speckle model works best among the models

compared here. For this model only 1–2 iterations were enough for all images. The

Krissian et al. is not denoising very well compared to the new one. The residual

shows presence of edges and fine textures due to the blurring effect of TV. But the

Krissian et al. is certainly the fastest one. The nonlocal means is not working well

for fine textures and slower than the new model. It is also the slowest of all.

Nonlocal speckle model maintains texture and removes noise very well. The noise

residuals show that nonlocal speckle model has picked up more noise compared to

the other ones.

524 A. B. Misra and H. Lim

Synthetic image result (Lenna)

Nonlocal Total Variation Based Speckle Denoising Model 525

5 Conclusion

In this paper we have incorporated the positive aspects of both PDE and filtering

models. The PDE based TV model is very fast but has a blurring effect. Nonlocal

Ultrasound image result (Gallstones)

526 A. B. Misra and H. Lim

means has very effective denoising properties but it is very slow. Therefore, have

developed an accurate model using both of them. We have extended the existing

Krissian et al. speckle denoising model [6] to a nonlocal framework, and provided

an efficient nonlocal TV based speckle denoising model. We have also developed a

numerical scheme for solving this new model. The numerical scheme is based on

the Split Bregman method introduced by Gilboa and Osher. The results show the

new model is more accurate than both the Krissian et al and nonlocal means

model. It is also much faster than nonlocal means.

References

1. Bresson X (2009) A short note for nonlocal tv minimization. Technical report

2. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: IEEE

computer society conference on computer vision and pattern recognition, CVPR 2005, vol 2,

pp 60–65. doi:10.1109/CVPR.2005.38

3. Gilboa G, Osher S (2008) Nonlocal operators with applications to image processing.

Multiscale Model Simul 7(3):1005–1028. http://dblp.uni-trier.de/db/journals/mmas/

mmas7.html#GilboaO08

4. Goldstein T, Osher S (2009) The split bregman method for l1-regularized problems. SIAM J

Imaging Sci 2:323–343. doi:10.1137/080725891, http://portal.acm.org/

citation.cfm?id=1658384.1658386

5. Kim S, Lim H (2007) A non-convex diffusion model for simultaneous image denoising and

edge enhancement. Electron J Differ Equ Conference 15:175–192

6. Krissian K, Kikinis R, Westin CF, Vosburgh K (2005) Speckle-constrained filtering of

ultrasound images. In: CVPR ’05: Proceedings of the 2005 IEEE computer society

conference on computer vision and pattern recognition (CVPR’05), vol 2. IEEE Computer

Society, Washington, DC, pp 547–552. doi:10.1109/CVPR.2005.331

7. Li C (2003) An efficient algorithm for total variation regularization with applications to the

single pixel camera and compressive sensing. Master’s thesis, Rice University, Houston,

Texas

8. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal

algorithms. Physica D: Nonlinear Phenom 60(1–4):259–268. doi:10.1016/0167-

2789(92)90242-F

9. Wang Y, Yin W, Zhang Y (2007) A fast algorithm for image deblurring with total variation

regularization. Technical report, Department of Computational and Applied Mathematics,

Rice University, Houston, Texas

10. Zhou D, Schlkopf B (2004) A regularization framework for learning from graph data. In:

ICML workshop on statistical relational learning, pp 132–137

11. Zhou D, Schlkopf B (2005) Regularization on discrete spaces. In: Pattern recognition.

Springer, Heidelberg, pp 361–368

Nonlocal Total Variation Based Speckle Denoising Model 527