Filtering and Edge Detection

Scott WehrweinCSCI 497/597P: Computer Vision

Reading

• Szeliski, Chapter 4.2

Announcements

Goals

• Understand practical considerations involved in filtering:• Handling the boundaries• Preserving total brightness

• Learn the basics numpy for array and image processing.• Understand the basics of edge detection:• The sobel operator as an approximation of the image gradient in the presence

of noise.

Last time: Convolution and cross-correlation

• Cross correlation

• Convolution

S[f ](m,n) =kX

i=�k

kX

j=�k

w(i, j)f(m+ i, n+ j)

S[f ](m,n) =kX

i=�k

kX

j=�k

w(i, j)f(m� i, n� j)

S[f ] = w ⌦ f

S[f ] = w ⇤ f

Last time: Convolution and cross-correlation

• Properties• Shift-invariant: a sensible thing to require• Linearity: convenient

• Can be used for smoothing, sharpening• Also main component of CNNs

Boundary conditions

• What if m-i <0?• What if m-i > image size• Assume f is defined for [−∞,∞] in both directions,

just 0 everywhere else• Same for w

(w ⇤ f)(m,n) =kX

i=�k

kX

j=�k

w(i, j)f(m� i, n� j)

(w ⇤ f)(m,n) =1X

i=�1

1X

j=�1w(i, j)f(m� i, n� j)

Boundary conditions

90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0

Boundary conditions

90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0

Boundary conditions

90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0

Boundary conditions

90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0

Boundary conditions

90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0

Boundary conditions in practice

• “Full convolution”: compute if any part of kernel intersects with image• requires padding• Output size = m+k-1

• “Same convolution”: compute if center of kernel is in image• requires padding• output size = m

• “Valid convolution”: compute only if all of kernel is in image• no padding• output size = m-k+1

“valid”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0

“same”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0

“full”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0

Padding Modes (OpenCV)

• BORDER_REPLICATE: aaaaaa|abcdefgh|hhhhhhh• BORDER_REFLECT: fedcba|abcdefgh|hgfedcb• BORDER_REFLECT_101: gfedcb|abcdefgh|gfedcba• BORDER_WRAP: cdefgh|abcdefgh|abcdefg• BORDER_CONSTANT: iiiiii|abcdefgh|iiiiiii

(with specified constant ‘i’; default = 0)

(image)(L pad) (R pad)

Padding Modes (OpenCV)

• BORDER_REPLICATE: aaaaaa|abcdefgh|hhhhhhh• BORDER_REFLECT: fedcba|abcdefgh|hgfedcb• BORDER_REFLECT_101: gfedcb|abcdefgh|gfedcba• BORDER_WRAP: cdefgh|abcdefgh|abcdefg• BORDER_CONSTANT: iiiiii|abcdefgh|iiiiiii

(with specified constant ‘i’; default = 0)

(image)(L pad) (R pad)

The sum of the kernel• The sum of all kernel weights is important.

*1 1 1

1 1 11 1 1

=

The sum of the kernel• The sum of all kernel weights is important.

*1 1 1

1 1 11 1 1

=1

9

Usually want it to sum to either 1 or 0.

Edge detection

• Convert a 2D image into a set of curves– Extracts salient features of the scene– More compact than pixels

Origin of Edges

• Edges are caused by a variety of factors

depth discontinuity

surface color discontinuity

illumination discontinuity

surface normal discontinuity

Images as functions…

• Edges look like steep cliffs

Characterizing edges• An edge is a place of rapid change in the image

intensity function

imageintensity function

(along horizontal scanline) first derivative

edges correspond toextrema of derivativeSource: L. Lazebnik

• How can we differentiate a digital image F[x,y]?– Option 1: reconstruct a continuous image, f, then

compute the derivative– Option 2: take discrete derivative (finite difference)

1 -1

How would you implement this as a linear filter?

Image derivatives

-1

1: :

Source: S. Seitz