Morphological Image ProcessingSegmentation algorithms for monochrome images generally are based on of two basic properties

of image intensity values: discontinuity and similarity. In the firest category, the approach is to

partition an image based on abrupt change in intensity, such as edge. The principal approaches in

the second category are based on portioning an image into regions that are similar according to a

set of predefined criteria.

Discontinuities detection: point, line, edge.

Thresholding techniques, region-oriented approaches, watershed segmentation (generally involve

a prior knowledge).

1. point detection

principle: the response of a mask must be the strongest when the mask is centered on an isolated

point, and that the response be in areas of constant intensity.

Typical mask:

Matlab code:

g = abs(imfilter(double(f),w)) >= T; where w is a appropriate point detection mask which satisfies

the above condition.

For example: the detection of isolated bright point in the dark gray area of the northeast quadrant.

(image size: 675*675)

the original image the isolated point detected (only a part displayed)

code:

f=imread('Fig1002(a)(test_pattern_with_single_pixel).tif');

w = [-1 -1 -1; -1 8 -1; -1 -1 -1];

g = abs(imfilter(double(f), w));

T = max(g(:));

g = g>= T;

subplot(121);imshow(f);

title('the original image');

subplot(122);imshow(g(1:end-400, 400:end));

title('the isolated point detected (only a part displayed)');

======================================================================

2. line (commonly one pixel thick) detection

Typical mask:

for example: -450 line (one pixel thick) detection.

the original image results from the -45 detecting Zoomed view of the top, left region of (b)

Zoomed view of the bottom, right region of (b) absolute value of (b) -45 line detected

f = imread('Fig1004(a)(wirebond_mask).tif');

w = [2 -1 -1; -1 2 -1; -1 -1 2];

g = imfilter(double(f), w);

gtop = g(1:120, 1:120);

gtop = pixeldup(gtop, 4); % zoomed the top, left region of g;

gbot = g(end-119:end, end-119:end);

gbot = pixeldup(gbot, 4);

g1 = abs(g);

T = max(g1(:));

g2 = g>= T;

subplot(231); imshow(f, []);

title(' the original image');

subplot(232); imshow(g, []);

title(' results from the -45 detecting');

subplot(233); imshow(gtop, []);

title(' Zoomed view of the top, left region of (b)');

subplot(234); imshow(gbot, []);

title(' Zoomed view of the bottom, right region of (b)');

subplot(235); imshow(g1, []);

title(' absolute value of (b)');

subplot(236); imshow(g2, []); %note the isolated point (why) detected simultaneously which must

be deleted by other ways;

title(' -45 line detected');

======================================================================

3. edge detection

Methods: using the first- (gradient) and second-order derivatives.

For the first-order derivative, one of the key issues is how to estimate the derivatives Gx and Gy.

Second-order derivatives are generally computed using the Laplacian operator. But the Laplacian

is seldom used by itself for edge detection, it is unacceptably sensitive to noise, its magnitude

produces double edges, and it is unable to detect the direction of edge. However, the Laplacian can

be a powerful complement when combined with other edge-detection algorithm, such as edge

location.

Two general criteria for determining the rapid changes of intensity in an image:

(1) Find places where the first derivative of the intensity is greater in magnitude than a specified

threshold;

(2) Find places where the second derivative of the intensity has a zero-crossing.

IPT function edge provides several derivative estimators based on criteria just discussed.

Sytax of edge: [g, t] = edge(f, ‘method’, parameters);

Where ‘method’ parameter includes various types of edge detectors: Sobel, Prewitt, Roberts,

Laplacian of a Gaussian (LoG), Zero crossings and Canny.

Canny Edge Detector (Canny [1986]):

It is the most powerful edge detector provided by function Edge, which can summarized as

follows:

(1) First, the image is smoothed using a Gaussian filter with a specified standard deviation (2) The local gradient, g(x, y) = [Gx

2+Gy2]1/2, and edge direction, (x, y) = tan-1(Gy /Gx), are

computed at each point. Any of the first three techniques can be used to computer the Gx and

Gy. An edge point is defined to be a point whose strength is locally maximum in the direction

of the gradient.

(3) The edge points give rise to ridges in the gradient magnitude image. The algorithm then tracks

along the top of these ridges and sets to zero all pixels that are not actually on the ridge top so

as to give a thin line, a process known as nonmaximal suppression. The ridge pixels are the

thresholded using thresholds, T1 and T2, with T1 < T2. Ridge pixels with values greater than

T2 are said to be “strong” edge pixels and pixels between T1 and T2 “weak” edge pixels.

(4) Finally, the algorithm performs edge linking by incorporating the weak pixels that are 8-

connected to strong pixels.

Notation: Edge function does not compute edges at ±450. To compute edges we need to specify

the mask and use imfilter.

Example of edge function:

f = imread('Fig1006(a)(building).tif');

[gv, t] = edge(f, 'sobel', 'vertical'); % using threshold computed automatically, here t = 0.0516

subplot(231);imshow(f, []);

title('the original image');

subplot(232);imshow(gv, []);

title('vertical edge with threshold determined automatically');

gv1 = edge(f, 'sobel', 0.15, 'vertical'); % using a specified threshold.

subplot(233);imshow(gv1, []);

title('vertical edge with a specified threshold');

gboth = edge(f, 'sobel', 0.15); % edge detection of two directions

subplot(234);imshow(gboth, []);

title('horizontal and vertical edge');

% edge detection of 450 direction using imfilter function

w45 = [-2 -1 0; -1 0 1; 0 1 2];

g45 =imfilter(double(f), w45, 'replicate');

T = 0.3*max(abs(g45(:)));

g45 = g45 >= T;

subplot(235);imshow(g45, []);

title('edge at 45 with imfilter');

wm45 = [ 0 1 2; -1 0 1; -2 -1 0];

g45 =imfilter(double(f), wm45, 'replicate');

T = 0.3*max(abs(g45(:)));

g45 = g45 >= T;

subplot(236);imshow(g45, []);

title('edge at -45 with imfilter');

the original image vertical edge with threshold determined automatically vertical edge with a specified threshold

horizontal and vertical edge edge at 45 with imfilter edge at -45 with imfilter

Another example of edge detection: comparison the relative performance of the Sobel, LoG and

Canny edge detectors.

% using the default threshold

f = imread('Fig1006(a)(building).tif');

[gs_default, ts] = edge(f, 'sobel'); % ts = 0.074

[gl_default, tl] = edge(f, 'log'); % tl = 0.002 and the default sigma = 0.2

[gc_default, tc] = edge(f, 'canny'); % tc = [0.0189 0.047] and the default sigma = 0.1

% using the optimal threshold acquired by manual test

gs_best = edge(f, 'sobel', 0.05);

gl_best = edge(f, 'log', 0.003, 2.25);

gc_best = edge(f, 'canny', [0.04 0.1], 1.5);

Sobel operator with deafult threshold

LoG operator with deafult threshold

canny operator with deafult threshold

The left column in above figure shows the edge images obtained using the default syntax for the

‘sobel’, ‘log’ and ‘canny’ operator respectively, whereas the right column are the results using

optimal threshold and sigma values obtained by try.

4. Hough transform

In practice, the resulting pixels produced by the methods discussed in the previous sections

seldom characterize an edge completely because of noise, breaks from nonuniform illumination,

and other effects that introduce spurious discontinuities. Hough Transform is one type of linking

procedure to find and link line segments for assembling edge pixels into meaningful edges.

About the principle of Hough transform, please refer to page 586 in textbook.

Instance of Hough transform:

% constructing an image containing 5 isolated foreground pixels in several locaitons:

f = zeros(101, 101);

f(1, 1) = 1, f(101, 1) = 1, f(1, 101) = 1, f(101, 101) = 1, f(51, 51) = 1;

[H, theta, rho] = hough(f); % hough transform

imshow(theta, rho, H, [], 'notruesize');

axis on, axis normal;

xlabel('\theta'), ylabel('\rho');

-80 -60 -40 -20 0 20 40 60 80

-100

-50

0

50

100

4.1 Hough transform for peak detection

Peak detection is the first step in using Hough transform for line detection and linking. However,

finding a meaningful set of distinct peaks in a Hough transform can be challenging. Because of the

quantization in space of the digital image and in parameter space of the Hough transform, as well

as the fact that edges in typical images are not perfectly straight, Hough transform peaks tend to

lie in more than one Hough transform cell. One strategy to overcome this problem is following:

(1) find the HT cell containing the highest value and record its location;

(2) suppress (set to zero) HT cells in the immediate neighborhood of the maximum;

(3) repeat until the desired number of peaks has been found, or until a specified threshold has

been reached.

function [r, c, hnew] = houghpeaks(h, numpeaks, threshold, nhood)

%HOUGHPEAKS Detect peaks in Hough transform.

% [R, C, HNEW] = HOUGHPEAKS(H, NUMPEAKS, THRESHOLD, NHOOD) detects

% peaks in the Hough transform matrix H. NUMPEAKS specifies the

% maximum number of peak locations to look for. Values of H below

% THRESHOLD will not be considered to be peaks. NHOOD is a

% two-element vector specifying the size of the suppression

% neighborhood. This is the neighborhood around each peak that is

% set to zero after the peak is identified. The elements of NHOOD

% must be positive, odd integers. R and C are the row and column

% coordinates of the identified peaks. HNEW is the Hough transform

% with peak neighborhood suppressed.

%

% If NHOOD is omitted, it defaults to the smallest odd values >=

% size(H)/50. If THRESHOLD is omitted, it defaults to

% 0.5*max(H(:)). If NUMPEAKS is omitted, it defaults to 1.

======================================================================

4.2 HT for line detection and linking

For each peak, the first step is to find the location of all nonzero pixels in the image that

contributed to that peak. This purpose can be implemented by the following function:

function [r, c] = houghpixels(f, theta, rho, rbin, cbin)

%HOUGHPIXELS Compute image pixels belonging to Hough transform bin.

% [R, C] = HOUGHPIXELS(F, THETA, RHO, RBIN, CBIN) computes the

% row-column indices (R, C) for nonzero pixels in image F that map

% to a particular Hough transform bin, (RBIN, CBIN). RBIN and CBIN

% are scalars indicating the row-column bin location in the Hough

% transform matrix returned by function HOUGH. THETA and RHO are

% the second and third output arguments from the HOUGH function.

[x, y, val] = find(f);

x = x - 1; y = y - 1;

theta_c = theta(cbin) * pi / 180;

rho_xy = x*cos(theta_c) + y*sin(theta_c);

nrho = length(rho);

slope = (nrho - 1)/(rho(end) - rho(1));

rho_bin_index = round(slope*(rho_xy - rho(1)) + 1);

idx = find(rho_bin_index == rbin);

r = x(idx) + 1; c = y(idx) + 1;

The pixels associated with the locations found using houghpixles must be grouped into line

segments, which is programmed into the following function:

function lines = houghlines(f,theta,rho,rr,cc,fillgap,minlength)

%HOUGHLINES Extract line segments based on the Hough transform.

% LINES = HOUGHLINES(F, THETA, RHO, RR, CC, FILLGAP, MINLENGTH)

% extracts line segments in the image F associated with particular

% bins in a Hough transform. THETA and RHO are vectors returned by

% function HOUGH. Vectors RR and CC specify the rows and columns

% of the Hough transform bins to use in searching for line

% segments. If HOUGHLINES finds two line segments associated with

% the same Hough transform bin that are separated by less than

% FILLGAP pixels, HOUGHLINES merges them into a single line

% segment. FILLGAP defaults to 20 if omitted. Merged line

% segments less than MINLENGTH pixels long are discarded.

% MINLENGTH defaults to 40 if omitted.

%

% LINES is a structure array whose length equals the number of

% merged line segments found. Each element of the structure array

% has these fields:

%

% point1 End-point of the line segment; two-element vector

% point2 End-point of the line segment; two-element vector

% length Distance between point1 and point2

% theta Angle (in degrees) of the Hough transform bin

% rho Rho-axis position of the Hough transform bin

Instance:

% first compute and display the Hough transform using a finer spacing than the default.

f = imread('Fig1006(a)(building).tif');

gc_best = edge(f, 'canny', [0.04 0.1], 1.5);

[H, theta, rho] = hough(gc_best, 0.5);

imshow(theta, rho, H, [], 'notruesize');

axis on, axis normal;

xlabel('\theta'), ylabel('\rho')

% next use function houghpeaks to find five HT peaks that are likely to be significant.

[r, c] = houghpeaks(H, 5);

hold on;

plot(theta(c), rho(r), 'linestyle', 'none', 'marker', 's', 'color', 'w');

title('Hough transform with the peak locations superimposed');

% finally, use function houghlines to find and link line segments, and superimpose the line

segments on the original binary image.

lines = houghlines(gc_best, theta, rho, r, c);

figure, imshow(gc_best), hold on;

for k = 1:length(lines)

xy = [lines(k).point1; lines(k).point2];

plot(xy(:, 2), xy(:,1), 'linewidth', 4, 'color', [.6 .6 .6]);

end

title('line segment correspoding to 5 peaks');

Hough transform with the peak locations superimposed

-80 -60 -40 -20 0 20 40 60 80

-800

-600

-400

-200

0

200

400

600

800

line segment correspoding to 5 peaks

5. thresholding

Two kind of methods for determining thresholding automatically:

(1) iterative procedure suggested by Gonzalez and Woods [2002];

Steps:

a. Select an initial estimate for T;

b. Segment the image using T. This will produce two groups of pixels: G1 consisting of all pixels

with gray level value > T and G2 with values £ T;

c. Compute the average gray level values m1 and m2 for the pixels in region G1 and G2

d. Compute a new threshold value: T = 0.5(m1 + m2 );

e. Repeat the steps 2 through 4 until the difference in T in successive iterations is smaller than a

predefined parameter T0.

(2) Ostu’s method: maximizes the between-class variance (matlab function: graythresh)

Example: thresholding text

clear, close all;

f = imread('Fig1013(a)(scanned-text-grayscale).tif');

T = 0.5*(double(min(f(:))) + double(max(f(:))));

done = false;

% computing threshold by Gonzalez's method

while ~done

g = f >= T;

Tnext = 0.5*(mean(f(g)) + mean(f(~g)));

done = abs(T - Tnext) < 0.5;

T = Tnext;

end

subplot(221); imshow(f);

title('original image');

subplot(222); imhist(f);

title('histogram of orginal image');

fout = im2bw(f, T/255);

subplot(223); imshow(fout, []);

title('thresholed text by method of Gonzalez');

% computing threshold by Ostu's method

T2 = graythresh(f);

fout = im2bw(f, T2);

subplot(224); imshow(fout, []);

title('thresholed text by method of Otus');

deltaT = T2*255 - T;

original image

0 50 100 150 200 250

0

1

2

3

4

x 104 histogram of orginal image

thresholed text by method of Gonzalez thresholed text by method of Otus

6. region-based segmentation

6.1 Basic formulation:

Let R represent the entire image region. Segmentation may be viewed as a process that partitions

R into subregions, R1, R2, … , Rn, such that:

Where P(.) is a logical predicate which deals with the properties that must be satisfied by the pixel

in a segmented region.

6.2 Matlab implement for region growing:

function [g, NR, SI, TI] = regiongrow(f, S, T)

%REGIONGROW Perform segmentation by region growing.

% [G, NR, SI, TI] = REGIONGROW(F, SR, T). S can be an array (the

% same size as F) with a 1 at the coordinates of every seed point

% and 0s elsewhere. S can also be a single seed value. Similarly,

% T can be an array (the same size as F) containing a threshold

% value for each pixel in F. T can also be a scalar, in which

% case it becomes a global threshold.

%

% On the output, G is the result of region growing, with each

% region labeled by a different integer, NR is the number of

% regions, SI is the final seed image used by the algorithm, and TI

% is the image consisting of the pixels in F that satisfied the

% threshold test.

f = double(f);

% If S is a scalar, obtain the seed image.

if numel(S) == 1

SI = f == S;

S1 = S;

else

% S is an array. Eliminate duplicate, connected seed locations

% to reduce the number of loop executions in the following

% sections of code.

SI = bwmorph(S, 'shrink', Inf);

J = find(SI);

S1 = f(J); % Array of seed values.

end

TI = false(size(f));

for K = 1:length(S1)

seedvalue = S1(K);

S = abs(f - seedvalue) <= T;

TI = TI | S;

end

% Use function imreconstruct with SI as the marker image to

% obtain the regions corresponding to each seed in S. Function

% bwlabel assigns a different integer to each connected region.

[g, NR] = bwlabel(imreconstruct(SI, TI));

application instance of region growing to weld porosity detection:

f = imread('Fig1014(a)(defective_weld).tif');

[g, NR, SI, TI] = regiongrow(f, 255, 65);

subplot(221); imshow(f, []);

title('original image');

subplot(222); imshow(SI, []);

title('seed region image');

subplot(223); imshow(TI, []);

title('binary image passing threshold test');

subplot(224); imshow(g);

title('result after 8-connectivity considered to seed point');original image seed region image

binary image passing threshold test result after 8-connectivity considered to seed point

0 50 100 150 200 250

0

500

1000

1500

2000

2500

3000

3500

4000

4500

6.3 region splitting and merging

This approach is to subdivide an image initially into a set of arbitrary, disjointed regions and then

merge and/or split the regions in an attempt to satisfy the conditions (a)~(e) mentioned in the

starting of this section.

After finishing the splitting, for any two adjacent regions Ri and Rj, if P(RiÈRj)=True, the two

regions must be merged.

7. segmentation using watershed transform

This method comes from the idea in geography. Relative several concepts: watershed, catchment

basin, watershed ridge line, topological surface. The key of this method is to change the starting

image into another image whose catchment basins are the objects or regions we want to identify.

7.1 watershed segmentation using the distance transform

Euclidean distance transform of a binary image use IPT function bwdist, for example:

bw =

0 0 0 0 0

0 1 0 0 0

0 0 0 0 0

0 0 0 1 0

0 0 0 0 0

D = bwdist(bw)

D =

1.4142 1.0000 1.4142 2.2361 3.1623

1.0000 0 1.0000 2.0000 2.2361

1.4142 1.0000 1.4142 1.0000 1.4142

2.2361 2.0000 1.0000 0 1.0000

3.1623 2.2361 1.4142 1.0000 1.4142

example:

f = imread('Fig0925(a)(dowels).tif');

% preprocessing opening-by-reconstruction

se = strel('disk', 5);

fe = imerode(f, se);

fobr = imreconstruct(fe, f);

% preprocessing closing-by-reconstruction

fobrc = imcomplement(fobr);

fobrce = imerode(fobrc, se);

forbrcbr = imcomplement(imreconstruct(fobrce, fobrc));

% transform into binary image by adaptive global thresholding

fp = forbrcbr;

g = im2bw(fp, graythresh(fp));

subplot(231); imshow(f, []);

title('a original image of dowel');

subplot(232); imshow(fp, []);

title('preprocessed image by morphological operation');

subplot(233); imshow(g, []);

title('binary processing image');

gc = imcomplement(g); % or gc = ~g;

D = bwdist(gc); % distant tranform

L = watershed(-D); % L is a label matrix, whose positive integers corresponds to

catchment

% basins, and zero values indicate watershed ridge pixels.

w = L ==0; % find the ridge pixels.

subplot(234); imshow(gc, []);

title('complement of binary image');

subplot(235); imshow(D, []);

title('distance tranform');

subplot(236); imshow(w);

title('image of watershed ridge line');

g2 = g & ~w; % superimposed image of ridge lines and original binary image, note the

oversegmentation

figure;

imshow(g2, []);

title('superimposed image');

a original image of dowel preprocessed image by morphological operation binary processing image

complement of binary image distance tranform image of watershed ridge line

superimposed image

7.2 watershed segmentation using gradients

clear; close all;

f = imread('Fig1021(a)(small-blobs).tif');

h = fspecial('Sobel');

fd = double(f);

g = sqrt(imfilter(fd, h, 'replicate').^2 + ...

imfilter(fd, h', 'replicate').^2); %computing the gradient by Sobel operator

subplot(221); imshow(f, []);

title(' the original image');

subplot(222); imshow(g, []);

title(' the gradient image');

L = watershed(g);

wr = L==0; % get the watershed ridge lines;

subplot(223); imshow(wr); % note the servere oversegmentation

title('ridge lines image');

%an approach to oversegmentation is to smooth the gradient image before

%its watershed transform using a close-opening operation

g2 = imclose(imopen(g, ones(3,3)), ones(3,3));

L2 = watershed(g2);

wr2 = L2==0;

f2 = f;

f2(wr2) = 255; % superimpose the ridge lines and original, note

% the small oversegmentation is still evident.

subplot(224); imshow(f2, []);

title(' the segmentation with some smoothing operation');

======================================================================

the original image the gradient image

ridge lines image the segmentation with some smoothing operation

7.3 marker-controlled watershed segmentation

A practical solution to control over-segmentation is to limit the number of allowable regions by

incorporating a preprocessing stage designed to bring additional knowledge (such an marker) into

the segmentation procedure.

Example: marker-controlled watershed segmentation to eh electrophoresis gel image.

clear; close all;

f = imread('Fig1022(a)(gel-image).tif');

h = fspecial('Sobel');

fd = double(f);

g = sqrt(imfilter(fd, h, 'replicate').^2 + ...

imfilter(fd, h', 'replicate').^2); %computing the gradient by Sobel operator directly

subplot(231); imshow(f, []);

title(' the original image');

L = watershed(g);

wr = L == 0; % get the watershed ridge lines;

subplot(232); imshow(wr); % note the servere oversegmentation

title('oversegmentation result'); %oversegmentation resulting fom applying the watershed

% tranform directly to the gradient magnitude image g, which due in part

% to the large number of regional minima.

%rm = imregionalmin(f); % compute the location of all regional minima.

rm = imregionalmin(g) % compute the location of all regional minima of gradient image

subplot(233); imshow(rm);

title(' regional minima of gradient magnitude image');

% To eliminate these extraneous minima, IPT function imextendedmim can be used

% this function uses the extended minima transform suggested by Soille [2003]

im = imextendedmin(f, 2); % obtain the internal markers

fim = f;

fim(im) = 175; % superimpose the extended minima locations as gray blobs

subplot(234); imshow(fim, []);

title(' internal markers');

% next external markers must be found

Lim = watershed(bwdist(im));

em = Lim == 0;

subplot(235); imshow(em);

title('external markers');

% then, a so-called technique of minima imposition is used to modify the

% original gray-scale image.

% mp = imiposemin(f, mask);

% modify the gradient image by imposing regional minima at the location of

% both the internal and external markers

g2 = imimposemin(g, im|em);

subplot(236); imshow(g2, []);

title(' modified gradient image');

%at last, compute the watershed transform of the marker-modified gradient image

L2 = watershed(g2);

f2 = f;

f2(L2 == 0) = 255;

figure; imshow(f2, []);

title('improved segmentation results');

the original image oversegmentation result regional minima of gradient magnitude image

internal markers external markers modified gradient image

improved segmentation results