Image Enhancement II: Neighborhood Operations · PDF file1 EE 264: Image Processing and...

1

EE 264: Image Processing and Reconstruction Peyman Milanfar

UCSC EE Dept. 1

Image Enhancement II:

Neighborhood Operations


UCSC EE Dept. 2

Image Enhancement:Spatial Filtering Operation

• Idea: Use a “mask” to alter pixel values

according to local operation

• Aim: (De)-Emphasize some spatial frequencies

in the image.

2


UCSC EE Dept. 3

Overview of Spatial Filtering

• Local linear oprations on an image

)1,1()1,1(

),1()1,1(),(

98

21

yxfwyxfw

yxfwyxfwyxg

Input: f(x,y), Output: g(x,y)


UCSC EE Dept.

• We replace each pixel with a weighted

average of its neighborhood

• The weights are called the filter kernel

• What are the weights for the average of a

3x3 neighborhood?

Moving average

1 1 1

1 1 1

1 1 1

“box filter” Source: D. Lowe

3


UCSC EE Dept. 5

Spatial Filtering: Blurring

• Example

1 1 1

1 1 1

1 1 1

1/9

Averaging Mask:


UCSC EE Dept. 6


• Local linear oprations on an image

)1,1()1,1(

),1()1,1(),(

98

11

yxfwyxfw

yxfwyxfwyxg Input: f(x,y), Output: g(x,y):

Input Image

Mask

Output Image

Usually odd

4


UCSC EE Dept.

Defining convolution

lk

lkglnkmfnmgf,

],[],[],)[(

f

• Let f be the image and g be the kernel. The

output of convolving f with g is denoted f * g.

Source: F. Durand

• Convention: kernel is “flipped”

• MATLAB: conv2 vs. filter2 (also imfilter)


UCSC EE Dept.

Key properties

• Linearity: filter(f1 + f2 ) = filter(f1) + filter(f2)

• Shift invariance: same behavior

regardless of pixel location: filter(shift(f)) =

shift(filter(f))

• Theoretical result: any linear shift-invariant

operator can be represented as a

convolution

5


UCSC EE Dept.

Important details • What is the size of the output?

• MATLAB: filter2(g, f, shape)

– shape = ‘full’: output size is sum of sizes of f and g

– shape = ‘same’: output size is same as f

– shape = ‘valid’: output size is difference of sizes of f and g

f

g g

g g

f

g g

g g

f

g g

g g

full same valid


UCSC EE Dept. 10


• An important point: Edge Effects

– To compute all pixel values in the output image, we need to fill in a “border”

Mask dimension = 2M+1

Border dimension = M

6


UCSC EE Dept. 11


• An important point: Edge Effects (Ex.: 5x5 Mask)

– How to fill in a “border”

• Zeros (Ringing)

• Replication (Better)

• Reflection (“Best”) a b

c d

a b

a b c d

a

a

c

c

b

b

d

d

• Procedure:

– Replicate row-wise

– Replicate column-wise

– Apply filtering

– Remove borders


UCSC EE Dept.

Implementation

• What about near the edge?

– the filter window falls off the edge of the

image

– need to extrapolate

– methods (MATLAB):

• clip filter (black): imfilter(f, g, 0)

• wrap around: imfilter(f, g, ‘circular’)

• copy edge: imfilter(f, g, ‘replicate’)

• reflect across edge: imfilter(f, g, ‘symmetric’)

Source: S. Marschner

7


UCSC EE Dept.

• What about near the edge?

– the filter window falls off the edge of the

image

– need to extrapolate

– methods:

• clip filter (black)

• wrap around

• copy edge

• reflect across edge

Source: S. Marschner


UCSC EE Dept. 14

Image Enhancement:Spatial Filtering Operation 5x5 Blurring with 0-padding 5x5 Blurring with reflected padding

8


UCSC EE Dept.

Examples of linear filters

0 0 0

0 1 0

0 0 0

Original

?

Source: D. Lowe


UCSC EE Dept.

Practice with linear filters

0 0 0

0 1 0

0 0 0

Original Filtered

(no change)

Source: D. Lowe

9


UCSC EE Dept.


0 0 0

1 0 0

0 0 0

Original

?

Source: D. Lowe


UCSC EE Dept.


0 0 0

1 0 0

0 0 0

Original Shifted left

By 1 pixel

Source: D. Lowe

10


UCSC EE Dept.


Original

? 1 1 1

1 1 1

1 1 1

Source: D. Lowe


UCSC EE Dept.


Original

1 1 1

1 1 1

1 1 1

Blur (with a

box filter)

Source: D. Lowe

11


UCSC EE Dept.


Original

1 1 1

1 1 1

1 1 1

0 0 0

0 2 0

0 0 0

- ?

(Note that filter sums to 1)

Source: D. Lowe


UCSC EE Dept.


Original

1 1 1

1 1 1

1 1 1

0 0 0

0 2 0

0 0 0

-

Sharpening filter

- Accentuates differences

with local average

Source: D. Lowe

12


UCSC EE Dept.

Sharpening

Source: D. Lowe


UCSC EE Dept. 24

Examples of some other smoothing or “low-pass” filters:

00000

00000

11111

00000

00000

5

1

00100

00100

00100

00100

00100

5

1

10000

01000

00100

00010

00001

5

1

00001

00010

00100

01000

10000

5

1

13


UCSC EE Dept.

Gaussian Kernel

• Constant factor at front makes volume sum to 1 (can be

ignored, as we should re-normalize weights to sum to 1 in any case)

0.003 0.013 0.022 0.013 0.003 0.013 0.059 0.097 0.059 0.013 0.022 0.097 0.159 0.097 0.022 0.013 0.059 0.097 0.059 0.013 0.003 0.013 0.022 0.013 0.003

5 x 5, = 1

Source: C. Rasmussen

fspecial(‘gauss’,5,1)


UCSC EE Dept.

Choosing kernel width

• Gaussian filters have infinite support, but

discrete filters use finite kernels

Source: K. Grauman

14


UCSC EE Dept.

Choosing kernel width

• Rule of thumb: set filter half-width to about

3 σ


UCSC EE Dept.

Example: Smoothing with a Gaussian

15


UCSC EE Dept.

Mean vs. Gaussian filtering


UCSC EE Dept.

Gaussian filters

• Remove “high-frequency” components from the image (low-pass filter)

• Convolution with self is another Gaussian

– So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have

– Convolving two times with Gaussian kernel of width σ is same as convolving once with kernel of width σ√2

• Separable kernel

– Factors into product of two 1D Gaussians

16


UCSC EE Dept.

Separability of the Gaussian filter

Source: D. Lowe


UCSC EE Dept.

Sharpening revisited • What does blurring take away?

original smoothed (5x5)

–

detail

=

sharpened

=

Let’s add it back:

original detail

+ α

17


UCSC EE Dept. 33

More on Linear Operations: Sharpening Filters

• Sharpening filters use masks that typically have

+ and – numbers in them.

• They are useful for highlighting or enhancing

details and high-frequency information (e.g.

edges)

• They can (and often are) based on derivative-

type operations in the image (whereas

smoothing operations were based on “integral”

type operations)


UCSC EE Dept.

Differentiation and convolution

• Recall, for 2D function,

f(x,y):

• This is linear and shift

invariant, so must be

the result of a

convolution.

• We could approximate

this as

• which is obviously a

convolution with kernel

f

x lim

0

f x , y

f x,y

f

xf xn1,y f xn , y

x

-1 1

Source: D. Forsyth, D. Lowe

18


UCSC EE Dept. 35

Derivative-type Filters

)1,(),(2)1,(y

),1(),(2),1(x

),()1,(y

),(),1(x

2

2

2

2

yxfyxfyxff

yxfyxfyxff

yxfyxff

yxfyxff 11

1

1

121

1

2

1

Laplacian:

2

2

2

22

yx

fff 121

010

141

010

1

2

1


UCSC EE Dept. 36

Variations of the Laplacian Filter

Laplacian:

2

2

2

22

yx

fff 121

010

141

010

1

2

1

Same response in row/column directions

101

040

101

001

020

100

100

020

001

Consider: Same response in

diagonal directions

Together:

111

181

111

010

141

010

101

040

101

“Isotropic” filter

19


UCSC EE Dept. 37


UCSC EE Dept. 38

Sharpening Using the Laplacian Filter

),( ),( ),( 2 yxfyxfAyxg

111

181

111

A

Boosting High

Frequencies

20


UCSC EE Dept.

Gaussian Unsharp Mask Filter

Gaussian unit impulse

Laplacian of Gaussian

))1(()1()( gefgffgfff

image blurred

image

unit impulse

(identity)


UCSC EE Dept.

Edge detection

• Goal: Identify sudden changes (discontinuities) in an image – Intuitively, most semantic

and shape information from the image can be encoded in the edges

– More compact than pixels

• Ideal: artist’s line drawing (but artist is also using object-level knowledge)

Source: D. Lowe

21


UCSC EE Dept.

Characterizing edges

• An edge is a place of rapid change in the

image intensity function image

intensity function

(along horizontal scanline) first derivative

edges correspond to

extrema of derivative


UCSC EE Dept.

The gradient points in the direction of most rapid increase in intensity

Image gradient • The gradient of an image:

•

The gradient direction is given by

Source: Steve Seitz

The edge strength is given by the gradient magnitude

• How does this direction relate to the direction of the edge?

22


UCSC EE Dept. 43

Using the first derivative for enhancement:

yfxfyfxffyf

xff

2/122

• Sobel Edge Detector:

• Given image f apply :

101

202

101

121

000

121

xf

y

f

Compute Magnitudes and Add

Edge Image


UCSC EE Dept. 44

Application of Sobel gradient:

Many other edge detecting filters exist. Which is best?

23


UCSC EE Dept. 45

• Linear filters of the type we have seen (with all positive coefficients) will blur the image and reduce certain kinds of noise.

• Nonlinear smoothing filters can also be considered – Instead of computing a weighted average over the

masked area, perform an operation on the sorted list

of pixels in the area.

• Order statistic filters are useful for removing

certain “impulsive” types of noise.

Order-statistics Filters


UCSC EE Dept. 46

Order-statistics Filters

9821 ffff

9821 wwww

Inner product

fwfwgT

i

ii

Linear Filter

9821 ffff

9821 zzzz

sort

9821 wwww

Inner product

zwzwgT

i

ii

Nonlinear (OS) Filter

Nonlinear operator

24


UCSC EE Dept. 47

Examples of Order-statistics Filters

9821 ffff

9821 zzzz

sort

9821 wwww

Inner product

zwzwgT

i

ii

Nonlinear (OS) Filter

000010000

000000001

100000000

000

4

1

2

1

4

1000

Median Filter

Min Filter

Max Filter

Trimmed Mean Filter


UCSC EE Dept. 48

• For Gaussian noise removal: Use linear smoothing filter

• For impulsive (Salt+Pepper, heavytailed,..): Use order statistic filter

Date post:	31-Jan-2018
Category:	Documents
Upload:	dangthu
View:	223 times
Download:	0 times

Image Enhancement II: Neighborhood Operations · PDF file1 EE 264: Image Processing and...

Documents