CSC508

What You Should Be Doing

• Code, code, code– Programming

• Gaussian Convolution

• Sobel Edge Operator

CSC508

Effects of Threshold Selection

CSC508

Effects of Threshold Selection

• The preceding images were edge-detected with a Sobel operator then binarized with various thresholds

• The overall trend is predictable but the individual results are not

• This threshold dependency is the curse of computer vision

• We need smarter algorithms

CSC508

Scale-Space Processing

• One method of dealing with the threshold dependency is through scale-space processing– An image is processed at various scales revealing

different sets of features at each scale– Features common to all [or many] sets are deemed

“important”

CSC508

Gaussian Pyramid

• Selection of the “appropriate” parameters (sigma and width) of a Gaussian filter is analogous to setting a binarization threshold

• The Gaussian Pyramid helps in such a situation

CSC508

Gaussian Pyramid Formation

1. Select a sigma and width

2. Filter the image

3. Subsample the image by removing every other line and sample– Resultant image is ¼ the size of the original

4. Repeat on the subsampled image

CSC508

Gaussian Pyramid Formation

• You end up with a series of images (a pyramid) each exposing different features– Coarse features at the top of the pyramid– Fine features at the bottom

• You can now use the coarse features to select where to concentrate your processing of the fine features

CSC508

Effects of Threshold Selection

• Another method of dealing with the threshold dependency is through more complex algorithms

CSC508

A Better Kernel

• The Sobel mask is a derivative mask (computes the weighted difference of neighborhood pixels)

• So is the Laplacian

• From calculus: the 2nd derivative of a signal is zero when the derivative magnitude is extremal

CSC508

A Better Edge Detector

• Combining these observations we can derive the Laplacian-Gaussian filter due to Marr and Hildreth– Also known as a DoG (Difference of Gaussians)

because it can be approximated by subtracting two Gaussian kernels of same width and differing sigmas

CSC508

Laplacian-Gaussian Filter

• Bypassing the math and jumping to the results of applying the rules of calculus…

eyx yx

2

22

22

22

42

11

• Similar in form to the Gaussian kernel

CSC508

Laplacian-Gaussian Filter

• In 1 dimension it looks like this…

Convolution Filter (1 Line)

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 2 4 6 8 10 12 14 16

Laplacian-Gaussian Filter

• In 2 dimensions it looks like this…

CSC508

CSC508

Laplacian-Gaussian Filter

• Convolution with this kernel results in edge magnitudes of both positive and negative values

• We’re interested in where the sign changes occur

• These are where the edges are (recall the calculus result)

Zero-Crossing Detection

• For each pixel, look at it’s 4 previous (in scan line order) neighbors

• If the sign is different in the edge detected image, there is an edge at pixel I(i, j) in the original image

• Why don’t we look at the other 4 neighbors?

CSC508

+-- - -

i

j

-++ + +

i

j

CSC508

Width = 15, Sigma = 1.6

CSC508

Comparable Sobel vs. Laplacian-Gaussian

CSC508

Width = 15, Sigma = 2.6

CSC508

Comparable Sobel vs. Laplacian-Gaussian

CSC508

Improvement

• Results of the Marr-Hildreth method are better than those of the Sobel/Binarization method– Edge placement accuracy– Removal of edges due to noise

• But, there’s still something missing…

CSC508

What Else Can We Do?

• What are we ignoring in the following edge detection processes?

1.Gaussian filter

2.Sobel edge detection

3.Threshold based binarization of Sobel magnitude

And1.Laplacian-Gaussian filter

2.Zero crossing detection

CSC508

Context!

• We merely selected each edge based on its magnitude and zero-crossing

• We completely ignored its immediate surroundings• Why is this bad?

– Detects edges we don’t want• Prone to noise

• Prone to clutter

– Doesn’t detect edges we do want• Weak magnitude edges are thresholded (thresheld?) away

• We need a smarter algorithm!

CSC508

Canny Edge Detector

• John Canny – MIT 1986• Developed a process (series of steps) for

extracting edges from an image– Gaussian smoothing of input image– Basic derivative operator to detect edge candidates– Removal of edges that are clustered around a

single location– Adaptive thresholding algorithm for final edge

selection

CSC508

Canny Edge Detector

• Gaussian smooth original image– Reduce the effects of noise– In the code I posted the implementation is

somewhat strange• Broken up based on filter size

• Some odd scaling goes on

– This is due to constraints of the original system on which the implementation was created

CSC508

Canny Edge Detector

• Select two thresholds– The high threshold is used to identify large

magnitude (strong) edges• This is applied first

– The low threshold is used to identify small (weak) magnitude edges that are spatially connected to large (strong) magnitude edges

– My code accepts these as percentages of the total range of edge magnitudes

CSC508

Canny Edge Detector

• Non-maximal Suppression– This process thins out clusters of edge points– Ideal step edges rarely occur in natural images

CSC508

Non-Maximal Suppression

• Life would be grand if edges looked like this (1 dimension)

• But, in reality they usually look like this (1 dimension)

• This is why our Sobel magnitude images look so awful

CSC508

Non-Maximal Suppression

• We need to employ an algorithm that eliminates all the weak edges (due to noise) and retains the strong edges (due to signal)

• The basic algorithm is given on page 180 of the textbook (this actually combines two steps)– The implementation in the posted code is

somewhat cryptic but follows the basic philosophy which is

CSC508

Non-Maximal Suppression

• Find an intermediate or strong edge• Search a neighborhood along the direction of the

gradient (perpendicular to the edge) for stronger edges

• Eliminate those that are of lesser magnitude than the strongest found

• The result is the removal of [relatively] weak edges in the neighborhood of a [relatively] strong edge

• Notice that we have not applied any thresholds to this point in the process (other than Gaussian kernel parameters)

Non-Maximal Suppression

• Assume the “boldness” of the arrow represents the magnitude of the edge

• Assume the direction of the arrow represents the direction of the edge (perpendicular to the direction of the gradient)

CSC508

CSC508

Non-Maximal Suppression

CSC508

Contour Following

• We now have a set of “reasonable” candidate edges– We used some local context to eliminate some

candidates (non-maximal suppression)

• Next we use global context to eliminate additional candidates

CSC508

Contour Following

• Find a strong edge– Apply the high threshold– This isn’t so bad as we can set it very high with the

expectation that some real edges will meet this criteria

CSC508

Strong Edges

CSC508

Contour Following

• Compute the gradient direction of the edge– Similar to the computation done in Sobel

• Search for a neighboring edge in the direction of the edge (perpendicular to the gradient) that is above the weak threshold– We’re using a strong edge as an anchor point for a contour

of weak and strong edges – known as Hysteresis• Repeat the process until we have no more connected

weak edges• Find the next disconnected strong edge and repeat the

entire contour following process

CSC508

Final Thinning

• An additional thinning step is performed

• This is similar to the non-maximal suppression step

• Just one last clean-up operation

CSC508

Final Results

CSC508

Canny Summary

All edges Weak edges

Strong edges Contours

CSC508

Canny Summary

• Through more complex processing we arrive at a set of edges in which we have confidence– Minimal reliance on thresholds

• The cost is complex (slow, memory intensive) processing

• We must trade resources for accuracy!

CSC508

Things To Do

• Reading for Next (few) Week(s)– Chapter 9 – Texture

• 9.1 – 9.3 describe one approach

• I’ll describe a second approach

– Chapter 14 – Segmentation by Clustering• We’ll consider various clustering methods

– Chapter 15 – Segmentation by Models• We’ll analyze the Hough Transform

CSC508

Things To Do

• Additions to current programming assignment– Use your Gaussian filter code to create a Gaussian Pyramid

of Lenna of 4 levels• Resultant image sizes: (512x512), (256x256), (128x128), (64x64)

– Use your Sobel filter code to create edge images of the Gaussian Pyramid images

– Turn in your resultant image files– Comment on the results