CS223b, Jana Kosecka Image Features Local, meaningful, detectable parts of the image. Line detection...

CS223b, Jana Kosecka

Image Features

Local, meaningful, detectable parts of the image.• Line detection • Corner detectionMotivation• Information content high• Invariant to change of view point, illumination• Reduces computational burden• Uniqueness • Can be tuned to a task at hand


Canne Edge Detector

• Edge detection involves 3 steps:– Noise smoothing– Edge enhancement– Edge localization

• J. Canny formalized these steps to design an optimal edge detector

• How to go from derivatives to edges ?

Horizontal edges

Before:


• Compute image derivatives • if gradient magnitude > and the value is a local maximum along gradient direction – pixel is an edge candidate

Canny edge detector

gradient magnitudeoriginal image

Edge Detection


Algorithm Canny Edge detector

• The input is image I; G is a zero mean Gaussian filter (std = )

– J = I * G (smoothing)– For each pixel (i,j): (edge enhancement)

– Compute the image gradient 1. J(i,j) = (Jx(i,j),Jy(i,j))’

1. Estimate edge strength » es(i,j) = (Jx

2(i,j)+ Jy2(i,j))1/2

– Estimate edge orientation 1. eo(i,j) = arctan(Jx(i,j)/Jy(i,j))

• The output are images Es - Edge Strength - Magnitude• and Edge Orientation Eo

-


• Es has large values at edges: Find local maxima

• … but it also may have wide ridges around the local maxima (large values around the edges)

ThTh


NONMAX_SUPRESSION

• The inputs are Es & Eo (outputs of CANNY_ENHANCER)• Consider 4 directions D={ 0,45,90,135} wrt x

• For each pixel (i,j) do:• Find the direction dD s.t. d Eo(i,j) (normal to

the edge)

• If {Es(i,j) is smaller than at least one of its neigh. along d} • IN(i,j)=0

• Otherwise, IN(i,j)= Es(i,j)

1. The output is the thinned edge image IN

Edge orientationEdge orientation


Graphical Interpretation

xx xx


Thresholding

• Edges are found by thresholding the output of NONMAX_SUPRESSION

• If the threshold is too high:– Very few (none) edges

•High MISDETECTIONS, many gaps• If the threshold is too low:

– Too many (all pixels) edges•High FALSE POSITIVES, many extra edges


SOLUTION: Hysteresis Thresholding

Es(i,j)Es(i,j)> H> H

Es(i,j)<Es(i,j)<HH

Es(i,j)Es(i,j)>L>L

Es(i,j)<LEs(i,j)<LEs(i,j)Es(i,j)>L>L


Canny Edge Detection (Example)

courtesy of G. Loy

gap is gone

Originalimage

Strongedgesonly

Strong +connectedweak edges

Weakedges


Other Edge Detectors

(2nd order derivative filters)


First-order derivative filters (1D)

• Sharp changes in gray level of the input image correspond to “peaks” of the first-derivative of the input signal.

F(x)F(x)F ’(x)F ’(x)

xx


Second-order derivative filters (1D)

• Peaks of the first-derivative of the input signal, correspond to “zero-crossings” of the second-derivative of the input signal.

F(x)F(x) F ’(x)F ’(x)

xx

F’’(x)F’’(x)


NOTE:

• F’’(x)=0 is not enough!– F’(x) = c has F’’(x) = 0, but there is no edge

• The second-derivative must change sign, -- i.e. from (+) to (-) or from (-) to (+)

• The sign transition depends on the intensity change of the image – i.e. from dark to bright or vice versa.


Edge Detection (2D)

1D1D 2D2D

I(x)I(x) I(x,y)I(x,y)

dd22I(x)I(x)dxdx22

= 0= 0

xx

yy

||I(x,y)| =(II(x,y)| =(Ix x 22(x,y) + I(x,y) + Iyy

22(x,y))(x,y))1/2 1/2 > Th> Th

tan tan = I = Ixx(x,y)/ I(x,y)/ Iyy(x,y) (x,y)

F(x)F(x)

xx

dI(x)dI(x)dxdx

> Th> Th

22I(x,y) =II(x,y) =Ix x x x (x,y) + I(x,y) + Iyy yy (x,y)=0(x,y)=0

LaplaciaLaplacia

nn


Notes about the Laplacian:

• 22I(x,y) is a SCALARI(x,y) is a SCALAR Can be found using a SINGLE maskCan be found using a SINGLE mask Orientation information is lostOrientation information is lost

22I(x,y) is the sum of SECOND-order derivativesI(x,y) is the sum of SECOND-order derivatives– But taking derivatives increases noiseBut taking derivatives increases noise– Very noise sensitive!Very noise sensitive!

• It is always combined with a smoothing It is always combined with a smoothing operation:operation:


LOG Filter

• First smooth (Gaussian filter),• Then, find zero-crossings (Laplacian filter):– O(x,y) = 22(I(x,y) * G(x,y))(I(x,y) * G(x,y))

• Using linearity:– O(x,y) = 22G(x,y) * I(x,y)G(x,y) * I(x,y)– This filter is called: “Laplacian of the This filter is called: “Laplacian of the Gaussian” (LOG)Gaussian” (LOG)


1D Gaussian

2

2

2)( x

exg−

=

2

2

2

2

22

222

2

1)('

xx

ex

xexg−−

−=−=


First Derivative of a Gaussian

2

2

2

2

22

222

2

1)('

xx

ex

xexg−−

−=−=

PositivPositivee NegativeNegative

As a mask, it is also computing a difference (derivative)As a mask, it is also computing a difference (derivative)


Second Derivative of a Gaussian

2

2

23

2

)1

()(''

x

ex

xg−

−=

2D2D

““Mexican Hat”Mexican Hat”


An edge is not a line...

• How can we detect lines ?


Finding lines in an image

• Option 1:– Search for the line at every possible position/orientation

– What is the cost of this operation?

• Option 2:– Use a voting scheme: Hough transform



• Connection between image (x,y) and Hough (m,b) spaces– A line in the image corresponds to a point in Hough space

– To go from image space to Hough space:• given a set of points (x,y), find all (m,b) such that y = mx + b

x

y

m

b

m0

b0

image space Hough space



• Connection between image (x,y) and Hough (m,b) spaces– A line in the image corresponds to a point in Hough space– To go from image space to Hough space:

• given a set of points (x,y), find all (m,b) such that y = mx + b

– What does a point (x0, y0) in the image space map to?

x

y

m

b

image space Hough space

• A: the solutions of b = -x0m + y0

• this is a line in Hough space

x0

y0


Hough transform algorithm• Typically use a different

parameterization

– d is the perpendicular distance from the line to the origin

is the angle this perpendicular makes with the x axis

– Why?Idea – keep an accumulator array (Hough space)and let each edge pixel contribute to it Line candidates are the maxima in the accumulatorarray


Typical Hough Transform

Basic Hough transform algorithm

1. Initialize H[d, q]=02. For each edge point I[x,y] in the image3. For q = 0 to 180 H[d, q] += 1 where point is now a sinusoid in Hough space Find the value(s) of (d, q) where H[d, q] is maximum

The detected line in the image is given b

What’s the running time (measured in # votes)?


Radon transform


Hough Transform for Curves

• The H.T. can be generalized to detect any curve that can be expressed in parametric form:– Y = f(x, a1,a2,…ap)– a1, a2, … ap are the parameters– The parameter space is p-dimensional– The accumulating array is LARGE!


x

y

• Edge detection, non-maximum suppression (traditionally Hough Transform – issues of resolution, threshold selection and search for peaks in Hough space)• Connected components on edge pixels with similar orientation - group pixels with common orientation

Non-max suppressed gradient magnitude

Line fitting


• Line fitting lines determined from eigenvalues and eigenvectors of A• Candidate line segments - associated line quality

second moment matrixassociated with eachconnected componentv1 - eigenvector of A

Line Fitting


Corners contain more edges than lines.

• A point on a line is hard to match.

Corner detection


Finding Corners

Intuition:

• Right at corner, gradient is ill defined.

• Near corner, gradient has two different values.


Formula for Finding Corners

⎥⎥⎦

⎤

⎢⎢⎣

⎡=

∑∑∑∑

2

2

yyx

yxx

III

IIIC

We look at matrix:

Sum over a small region, the hypothetical corner

Gradient with respect to x, times gradient with respect to y

Matrix is symmetric


⎥⎦

⎤⎢⎣

⎡=

⎥⎥⎦

⎤

⎢⎢⎣

⎡=

∑∑∑∑

2

12

2

0

0

λ

λ

yyx

yxx

III

IIIC

First, consider case where:

This means all gradients in neighborhood are:

(k,0) or (0, c) or (0, 0) (or off-diagonals cancel).

What is region like if:

1. l1 = 0?

2. l2 = 0?

3. l1 = 0 and l2 = 0?

4. l1 > 0 and l2 > 0?


General Case:

From Linear Algebra, it follows that because C is symmetric:

RRC ⎥⎦

⎤⎢⎣

⎡= −

2

11

00λ

λ

With R a rotation matrix.

So every case is like one on last slide.


So, to detect corners

• Filter image.• Compute magnitude of the gradient everywhere.

• We construct C in a window.• Use Linear Algebra to find λ1and λ2.• If they are both big, we have a corner.


• Compute eigenvalues of G• If smalest eigenvalue of G is bigger than - mark pixel as candidate feature point

• Alternatively feature quality function (Harris Corner Detector)

Point Feature Extraction


% Harris Corner detector - by Kashif Shahzadsigma=2; thresh=0.1; sze=11; disp=0;

% Derivative masksdy = [-1 0 1; -1 0 1; -1 0 1];dx = dy'; %dx is the transpose matrix of dy % Ix and Iy are the horizontal and vertical edges of imageIx = conv2(bw, dx, 'same');Iy = conv2(bw, dy, 'same'); % Calculating the gradient of the image Ix and Iyg = fspecial('gaussian',max(1,fix(6*sigma)), sigma);Ix2 = conv2(Ix.^2, g, 'same'); % Smoothed squared image derivativesIy2 = conv2(Iy.^2, g, 'same');Ixy = conv2(Ix.*Iy, g, 'same'); % My preferred measure according to research papercornerness = (Ix2.*Iy2 - Ixy.^2)./(Ix2 + Iy2 + eps); % We should perform nonmaximal suppression and thresholdmx = ordfilt2(cornerness,sze^2,ones(sze)); % Grey-scale dilatecornerness = (cornerness==mx)&(cornerness>thresh); % Find maxima[rws,cols] = find(cornerness);

clf ; imshow(bw); hold on;p=[cols rws];plot(p(:,1),p(:,2),'or'); title('\bf Harris Corners')


Example (=0.1)


Example (=0.01)


Example (=0.001)


Harris Corner Detector - Example

Date post:	20-Dec-2015
Category:	Documents
View:	213 times
Download:	0 times

CS223b, Jana Kosecka Image Features Local, meaningful, detectable parts of the image. Line detection...

Documents