Smoothing Techniques in Image Processing[1]

7/27/2019 Smoothing Techniques in Image Processing[1]

1/59

Smoothing Techniques in ImageProcessing

Prof. PhD. Vasile Gui

Polytechnic University

of Timisoara


2/59


3/59

Introduction

Why do we need image

smoothing?

What is image and what

is noise?

Frequency spectrum

Statistical properties


4/59

Brief Review of Linear Operators

[Pratt 1991]

Generalized 2D linear operator

Separable linear operator:

Space invariant operator:

Convolution sum

1

0

1

0

),(),;,(),(M

j

N

k

kjfnmkjOnmg

);();(),;,( nkOmjOnmkjO CR

1

0

1

0

),();();(),(M

j

N

k

CR kjfnkOmjOnmg

),(),(),;,( knjmHknjmOnmkjO

1

0

1

0

),(),(),(M

j

N

k

kjfknjmhnmg


5/59


Geometrical

interpretationof 2D

convolution

h0

0

h01h02

h10

h20h21h22

h11h12

h0

0

h01h02

h10

h20h21h22

h11h12

h0h01h02

h10

h20

h21

h22

h11h12h0

0

h01h02

h10

h20h21h22

h11

h12

N+L-1

N

N-

L+1

h0

0

h01h02h10

h20h21h22

h11h12

L-1

2L-1

2

L-1

2 L-1

2H

G

F


6/59

Matrix formulation

Unitary transform

Separable transform

Transform is invertible


TRCFAAT

IfAfAtAf*T*T

*T1AA

Aft


7/59


Basis vectors are orthogonal

Inner product and energy are conserved

Unitary transform: a rotation in N-dimensional vector

space

lk

lk

lk ,0

,1

),(lT*

k aa

2T*T*T*T*

*T*T2

||f||ffIffAfAf

(Af)(Af)tt||t||


8/59


2D vector space interpretation

of a unitary transform

x1

x2

12


9/59


DFT unitary matrix

A

1

0 0 0 0

0 1 2 1

0 2 4 2 1

0 2 1 1 2

N

W W W W W W W W

W W W W

W W W W

N

N

N N N

( )

( ) ( )

)2

sin()2

cos(}2

exp{

N

i

NN

iWN


10/59


1D DFT

2D DFT

1,...,1,0,}2exp{)(1)(1

0

NuunNinf

Nut

N

n

1,...,1,0,1,...,1,0

)}(2exp{),(

1

),(

1

0

1

0

NvMu

nN

v

mM

u

inmfMNvut

M

m

N

n


11/59


Circular convolution theorem

Periodicity off,h and g with Nare assumed

t u vN

t u v t u vg f h( , ) ( , ) ( , )1

1

0

1

0),(),(),(

N

j

N

kkjfknjmhnmg


12/59

Linear Image smoothing techniques

Box filters. Arithmetic mean LL operator

1

1

1

1*111

1

111

1

111

111

12

LLLh


13/59



Separable

Can be computed recursively, resulting inroughly 4 operations per pixel

L pixels

+

m,n

m,n+1


14/59



Optimality properties

Signal and additive white noise

Noise variance is reduced Ntimes

g f n f n

1 1 1

1 1 1N N Nk k

k

N

kk

N

kk

N

( ) .

z n

1

1N kk

N

.

2

1 1

2

2

1 12

1 12

T2

1

),(

1

}E{1

}E{1

}E{

N=klN

nnN

nnN

zz

N

k

N

l

N

k

N

l

kl

N

k

N

l

klz


15/59



Unknown constant signal plus noise

Minimize MSE of the estimation g:

N

k

k gfg1

22)()(

0)(2

g

g

N

k

kfN

g1

1


16/59



i.i.d. Gaussian signal with unknown mean.

Given the observed samples, maximize

Optimal solution: arithmetic mean

}2

)(exp{.)|( 2

2

fctfp

N

k

N

k

kk fctfp1 1

2

2})(

2

1exp{.)|(


17/59



Frequency response:

Non-monotonically decreasing with frequency1

9

1 1 1

1 1 1

1 1 1

1

3111

1

3

1

1

1

[ ] h hx y

t u h h n iN

unx xn N

N

( ) ( ) ( ) exp{ }( )/

( )/

0 21 2

1 2

)2

cos(3

2

3

1)( u

Nut


18/59



An example

8 5 8 8 5 8

8 5 8 8 5 8

8 5 8 8 5 8

1

9

1 1 1

1 1 1

1 1 1

7 7 7 7 7 7

7 7 7 7 7 7

7 7 7 7 7 7

8 5 8 5 8 5

8 5 8 5 8 5

8 5 8 5 8 5

1

9

1 1 1

1 1 1

1 1 1

6 7 6 7 6 7

6 7 6 7 6 7

6 7 6 7 6 7


19/59



Image smoothed with

33, 55, 99

and 11 11 box filters


20/59



Original Lena imageLena image filtered with

5x5 box filter


21/59


Binomial filters [Jahne 1995]

Computes a weighted average of pixels in

the window Less blurring, less noise cleaning for the

same size

The family of binomial filters can be defined

recursively

The coefficients can be found from (1+x)n


22/59


Binomial filters. 1D versions

112

11b

11 bbb *1214

12

1111 bbbbb ***1464116

14

111111 bbbbbbb *****161520156164

16

As size increases, the shape of the filter is closer to a Gaussian one


23/59


Binomial filters. 2D versions

121

242

121

16

1

1

2

1

4

1*121

4

12b

14641

41624164

62436246

41624164

14641

2561

1

4

6

4

1

161*14641

1614b


24/59


Binomial filters. Frequency response

121

4

12

b )

2cos(

2

1

2

1)( u

N

ut

1464116

14

b 2)]2

cos(2

1

2

1[)( u

Nut

Monotonically decreasing

with frequency


25/59


Binomial filters. Example

Original Lena image Lena image filtered

with binomial 5x5 kernel

Lena image filtered

with box filter 5x5


26/59


Binomial and box filters. Edge blurring comparison

Linear filters have to compromise smoothing

with edge blurring

Step edge

Result of size L

box filter

L

Size L binomial


27/59

Nonlinear image smoothingThe median filter [Pratt 1991]

Block diagram

2

1

Nm

f1

f2

fN

.

.

.

Order

samples

f(1)f(2)

f(N)

.

.

.

selectf(m)

median

N is odd

)()3()2()1( Nffff


28/59

Nonlinear image smoothingThe median filter

Numerical example

67

3 7 8

2 3

4 6 7

2, 3, 3, 4, 6, 7, 7, 7, 8


29/59

Nonlinearity


median{f1 + f2} median{ f1} + median{ f2}.

However:

median{ c f } = c median{ f},

median{ c + f } = c + median{ f}.

The filterselects a sample from the window, doesnot average

Edges are better preserved than with liner filters

Best suited for salt and pepper noise


30/59


Noisy image 5x5 median filtered 5x5 box filter


31/59


Optimality

Grey level plateau plus noise. Minimize sum of

absolute differences:

Result:

If 51% of samples are correct and 49% outliers,

the median still finds the right level!

N

k

k gfg1

||)(

)(321 },,,,{ mN fffffmediang


32/59


Caution: points, thin lines and corners are erased by the median filter

Test images

Results of 33 pixel median filter


33/59


Cross shaped window can correct some of the problems

Results of the 9 pixel cross shaped window median filter


34/59


Implementing the median filter

Sorting needs O(N2

) comparisons Bubble sort

Quick sort

Huang algorithm (based on histogram)

VLSI median


35/59


VLSI median block diagram

a

b

Min(a,b)

Max(a,b)

x(1)

x(2)

x(3)

x(4)

x(5)

x(6)

x(7)

x1

x2

x3

x4

x5

x6

x7


36/59


Color median filter There is no natural ordering in 3D (RGB) color

space Separate filtering on R,G and B components does

not guarantee that the median selects a truesample from the input window

Vector median filter, defined as the sampleminimizing the sum of absolute deviations from allthe samples

Computing the vector median is very timeconsuming, although several fast algorithms exist


37/59


Example of colormedian filtering

5x5 pixels window

Up: original image

Down: filtered image


38/59

Nonlinear image smoothingThe weighted median filter

The basic idea is to give higher weight to some samples, according to

their position with respect to the center of the window

Each sample is given a weight according to its spatial position in the

window.

Weights are defined by a weighting mask

Weighted samples are ordered as usually

The weighted median is the sample in the ordered array such that

neither all smaller samples nor all higher samples can cumulate more

than 50% of weights.

If weights are integers, they specify how many times a sample is

replicated in the ordered array


39/59

Nonlinear image smoothingThe weighted median filter

Numerical example for the weighted median filter

67

3 7 8

2 3

4 6 7

2, 2, 3, 3, 3, 3, 4, 6, 6, 7, 7, 7, 7, 7, 8

3

1 1

1 1

2 2

2

2


40/59

Nonlinear image smoothingThe multi-stage median filter

Better detail preservation

mi = median( Ri ), i =1,2,3,4.

Result = median( m1,m2,m3,m4,m5 )

R1

R2

R3

R4

m5


41/59

Nonlinear image smoothingRank-order filters (L filters)

Block diagram

sum

y

f(1)f1

f2

fN

.

.

.

ordering

f(2)

f(N)

.

.

.

a1

a2

aN

weights

y= a1f(1) + a2f(2)+...+ aNf(N)akk

N

1

1


42/59


43/59


Optimality

( ) | |( )f f fk rk

N

1

Minkovski distance

Case r= 1: best estimator is median.

Case r= 2, best estimator is arithmetic mean.

Case r, best estimator is mid-range.


44/59


Alpha-trimmed mean filter

otherwiseQmkQmforQak

,0

),12/(1

= (2Q + 1) / N, defines de degree of averaging

= 1 corresponds to arithmetic mean

= 1 / N corresponds to the median filter

Properties: in between mean and median


45/59


Median of absolute differences trimmed mean Better smoothing than the median filter and good edge

preservation

},,,,{ 3211 NffffmedianM

NiMfmedianM i ,...,2,1|},{| 12

}||:{ 21 MMffaverageoutput ii M2 is a robust estimator of the variance


46/59


k nearest neighbour (kNN) median filter

Median of the k grey values nearest by rankto the central pixel

Aim: same as above


47/59

Kuwahara type filtering Form regions Ri

Compute mean, mi and variance,Si for each region.

Result = mi : mi < mj for all j

different from i, i,e. themean of

the most homogeneous region

Matsujama & Nagao

Nonlinear image smoothingSelected area filtering [Nagao 1980]

reg 1 reg 2

reg 3

reg 4

reg 5 reg 6

reg 7reg 8


48/59

Nonlinear image smoothingConditional mean

Pixels in a neighbourhood are averaged only if they

differ from the central pixel by less than a given

threshold:

otherwise

thnmflnkmfiflkh

lnkmflkhnmgL

Lk

L

Ll

,0

|),(),(|,1),(

),,(),(),(

L is a spacescale parameter and th is a range scale parameter


49/59

Nonlinear image smoothingConditional mean

Example with L=3,

th=32


50/59

Nonlinear image smoothingBilateral filter [Tomasi 1998]

Space and range are treated in a similar way

Space and range similarity is required for the averaged pixels

Tomasi and Manduchi [1998] introduced soft weights to penalize the

space and range dissimilarity.

)),(),((),(),( nmflnkmfrlkslkh

k l

k l

lkhK

lnkmflkhK

nmg

),(

,),(),(1

),(

s() and r() are space and range similarity functions (Gaussian

functions of the Euclidian distance between their arguments).


51/59

Nonlinear image smoothingBilateral filter

The filter can be seen as weighted averaging in the

joint space-range space (3D for monochromatic

images and 5D x,y,R,G,B - for colour images) The vector components are supposed to be properly

normalized (divide by variance for example)

The weights are given by:

));K(d()(

}||||exp{)(

2

sh

sh

c

c

xxx

xxx


52/59

Nonlinear image smoothingBilateral filter

Example of Bilateral filtering

Low contrast texture has been removed

Yet edges are well preserved


53/59

Nonlinear image smoothingMean shift filtering [Comaniciu 1999, 2002]

Mean shift filtering replaces each pixels value with the most probable local value, found

by a nonparametric probability density estimation method.

The multivariate kernel density estimate obtained in the point x with the kernel K(x) and

window radius r is:

For the Epanechnikov kernel, the estimated normalized density gradient is proportional

to the mean shift:

d

inii R xx ...1}{

n

id r

Knr

f1

1)( i

xxx

)(

21

)()(

)(

2 xxi

x

xxxx

x

ri S

rn

Mf

f

d

r

S is a sphere of radius r, centered on x and nx is the number of samples inside the sphere


54/59

Nonlinear image smoothingMean shift filtering

The mean shift procedure is

a gradient ascent method to

find local modes (maxima)

of the probability density

and is guaranteed to

converge.

Step1: computation of the

mean shift vector Mr(x). Step2: translation of the

window Sr(x) by Mr(x).

Iterations start for each pixel

(5D point) and tipically

converge in 2-3 steps.


55/59

Nonlinear image smoothingMean shift filtering

Example1.


56/59

Nonlinear image smoothing

Mean shift filtering

Detail of a 24x40

window from the

cameraman imagea) Original data

b) Mean shift paths for

some points

c) Filtered data

d) Segmented data


57/59



Example 2


58/59



Comparison to bilateral filtering Both methods based on simultaneous processing of both the

spatial and range domains

While the bilateral filtering uses a static window, the mean shift

window is dynamic, moving in the direction of the maximum

increase of the density gradient.


59/59

REFERENCES D. Comaniciu, P. Meer: Mean shift analysis and applications. 7th

International Conference on Computer Vision, Kerkyra, Greece, Sept.1999, 1197-1203.

D. Comaniciu, P. Meer: Mean shift: a robust approach toward featurespace analysis. IEEE Trans. on PAMI Vol. 24, No. 5, May 2002, 1-18.

M. Elad: On the origin of bilateral filter and ways to improve it. IEEEtrans. on Image Processing Vol. 11, No. 10, October 2002, 1141-1151.

B. Jahne: Digital image processing. Springer Verlag, Berlin 1995.

M. Nagao, T. Matsuiama: A structural analysis of complex aerialphotographs. Plenum Press, New York, 1980.

W.K. Pratt: Digital image processing. John Wiley and sons, New York1991

C. Tomasi, R. Manduchi: Bilateral filtering for gray and color images.Proc. Sixth Intl. Conf. Computer Vision , Bombay, 839-846.

Date post:	14-Apr-2018
Category:	Documents
Upload:	shajehan
View:	222 times
Download:	0 times

Smoothing Techniques in Image Processing[1]

Documents