10.1 Detection of Discontinuities10.2 Edge Linking and Boundary Detection10.3 Thresholding10.4 Region-Based Segmentation10.5 Segmentation by Morphological Watersheds (x)10.6 The Use of Motion in Segmentation
Digital Image Processing, 2nd ed.Digital Image Processing, 2nd ed.www.imageprocessingbook.com
10.2 Edge Linking and Boundary Detection 58510.2.1 Local Processing 58510.2.2 Global Processing via the Hough Transform 58710.2.3 Global Processing via Graph-Theoretic Techniques 591
Digital Image Processing, 2nd ed.Digital Image Processing, 2nd ed.www.imageprocessingbook.com
10.3 Thresholding 59510.3.1 Foundation 59510.3.2 The Role of Illumination 59610.3.3 Basic Global Thresholding 59810.3.4 Basic Adaptive Thresholding 60010.3.5 Optimal Global and Adaptive Thresholding 60210.3.6 Use of Boundary Characteristics for Histogram Improvementand Local Thresholding 60810.3.7 Thresholds Based on Several Variables 611
Digital Image Processing, 2nd ed.Digital Image Processing, 2nd ed.www.imageprocessingbook.com
10.5 Segmentation by Morphological Watersheds 61710.5.1 Basic Concepts 61710.5.2 Dam Construction 62010.5.3 Watershed Segmentation Algorithm 62210.5.4 The Use of Markers 624
Digital Image Processing, 2nd ed.Digital Image Processing, 2nd ed.www.imageprocessingbook.com
Goal: Segmentation subdivides an image into its constituent regions or objects that have similar features according to a set of predefined criteriafeatures?
intensityhistogrammean, varianceenergytexture….
It depends on applications or the problems to solve
Image Processing StepsImage Processing Steps
KnowledgeBase
Preprocessing
ImageAcquisition
ImageAnalysis
Recognition& Interpretation
enhancement restoration, etc
segmentation feature extraction, etc
Detection of DiscontinuitiesDetection of Discontinuities
3 basic types of gray-level discontinuities: points, lines, edgesWe segment the image along the discontinuitiesThe most common way to look for discontinuities is to run a mask through the imageSpatial filter(mask)
ex) R = w1 z1+ w2 z2+…+ w9 z9
w9w8w7
w6w5w4
w3w2w1
mask
Point DetectionPoint Detection
We say that a (isolated) point has been detected at the location on which the mask is centered if
The formulation measures the weighted differences between the
center point and its neighbors
This filter is a highpass spatial filter(Laplacian)
The filter detects a point whose gray level is significantly different from its background
The result depends on the threshold T
-1-1-1
-18-1
-1-1-1
Point DetectionPoint Detection
(a) original image (b) highpass filtered image
Thresholding of (b), T=100 Thresholding of (b), T=200
Point DetectionPoint Detection
X-ray image of a jet-engine turbine blade with a porosity in the upper, right quadrant of the image
The threshold T is set to 90% of the highest absolute pixel value of the image in Fig. 10.2(c)
Line DetectionLine Detection
These filter masks would respond more strongly to linesNote that the coefficients in each mask sum to zero, indicating a zero response from the masks in areas of constant gray levelIf we are interested in detecting lines in a specified direction, we could use the mask associated with that direction and threshold its outputNote: these filters respond strongly to lines of one pixel thick
Line DetectionLine Detection
A binary image of a wire-bond mask for an electronic circuit
We are interested in finding all the lines that are one pixel thick and are oriented at -45°
The line in the top, left quadrant is not detected because it is not one pixel thick
Edge DetectionEdge Detection
Edge detection is the most common approach for detecting meaningful discontinuitiesEdge
An edge is a set of connected pixels that lie on the boundary between two regionsAn edge is a “local” concept whereas a region boundary is a more global ideaA reasonable definition of “edge” requires the ability to measure gray level(intensity) transitions in a meaningful way
An ideal edge has the properties of the model shown in Fig. 10.5(a)An ideal edge according to this model is a set of connected pixels, each of which is located at an orthogonal step transition in gray level
Edge DetectionEdge DetectionIn practice, optics, sampling, and other image acquisition imperfections yeild edges that are blurredAs a result, edges are more closely modeled as a “ramp”-like profileThe degree of blurring is determined by factors such as the quality of the image acquisition system, the sampling rate, and illumination conditionsAn edge point is any point contained in the ramp, and an edge would be a set of such points that are connectedThe “thickness” of the edge is determined by the length of the rampBlurred edges tend to be thick and sharp edges tend to be thin
Edge DetectionEdge DetectionThe magnitude of the first derivative can be used to detect the presence of an edge The sign of the second derivative can be used to determine whether an edge pixel lies on the dark or light side of an edgeAn imaginary straight line joining the extreme positive and negative values of the second derivative would cross zero near the midpoint of the edge (zero-crossing property)
Edge DetectionEdge Detection
Derivative is very sensitive to noise
Image smoothing is needed
Edge DetectionEdge Detection
We define a point in an image as being an edge point if its two-dimensional first-order derivative is greater than a specified thresholdA set of such points that are connected according to a predefined criterion of connectedness is defined as an edgeThe term edge segment generally is used if the edge is short in relation to the dimensions of the imageA key problem in segmentation is to assemble edge segments into longer edges
First-order derivatives of a digital image are based on various approximations of the 2-D gradient
The gradient is defined as the two-dimensional column vector
The magnitude of the gradient vector often is referred to as thegradient, too
We usually approximate the magnitude by absolute values instead of squares and square roots
Gradient OperatorsGradient Operators
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
∂∂∂∂
=⎥⎦
⎤⎢⎣
⎡=∇
y
fx
f
G
Gf
y
x
)( mag ff ∇=∇ 2/122 ][ Yx GG +=2/122
⎥⎥⎦
⎤
⎢⎢⎣
⎡⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+⎟⎠⎞
⎜⎝⎛∂∂
=y
f
x
f
yx GGf +≈∇
The direction of the gradient vector
The direction of an edge at (x,y) is perpendicular to the direction of the gradient vector at the point
Gradient OperatorsGradient Operators
⎟⎟⎠
⎞⎜⎜⎝
⎛= −
x
y
G
Gyx 1tan),(α
Robert cross-gradient operators
Sobel Operator
The GradientThe Gradient
)z (=)z (= 6859 zGandzG yx −−
[ ] 2/1268
259 )()( zzzzf −+−=∇
6859 zzzzf −+−≈∇
)2()2( 321987 zzzzzzf ++−++≈∇
)2()2( 741963 zzzzzz ++−+++
Gradient OperatorGradient Operator
⎥⎦
⎤⎢⎣
⎡−−
≅⎥⎦
⎤⎢⎣
⎡=∇
)(
)(
58
56
zz
zz
G
G
y
xf
z8
z5
z2
z9
z6
z3
z4
z7
z1
[ ] 2/1258
256 )()( zzzzf −+−≅∇
|||| 5856 zzzz −+−≅
Gradient OperatorGradient Operator
Roberts cross-gradient
⎥⎦
⎤⎢⎣
⎡−−
≅⎥⎦
⎤⎢⎣
⎡=∇
)(
)(
68
59
zz
zz
G
G
y
xf
z8
z5
z2
z9
z6
z3
z4
z7
z1
[ ] 2/1268
259 )()( zzzzf −+−≅∇
|||| 6859 zzzz −+−≅
Gradient OperatorGradient Operator
Robert cross-gradient filter masks
⎥⎦
⎤⎢⎣
⎡−−
≅⎥⎦
⎤⎢⎣
⎡=∇
)(
)(
68
59
zz
zz
G
G
y
xf
1
0
0
-1
[ ] 2/1268
259 )()( zzzzf −+−≅∇
|||| 6859 zzzz −+−≅0
-1
1
0
Prewitt OperatorPrewitt Operator
⎥⎦
⎤⎢⎣
⎡++−++++−++
≅⎥⎦
⎤⎢⎣
⎡=∇
)()(
)()(
321987
741963
zzzzzz
zzzzzz
G
G
y
xf
z8
z5
z2
z9
z6
z3
z4
z7
z1
|||| yx GGf +≅∇
1
0
-1
1
0
-1
0
1
-1
0
0
0
1
1
1
-1
-1
-1
xG yG
SobelSobel OperatorOperator
⎥⎦
⎤⎢⎣
⎡++−++++−++
≅⎥⎦
⎤⎢⎣
⎡=∇
)2()2(
)2()2(
321987
741963
zzzzzz
zzzzzz
G
G
y
xf
z8
z5
z2
z9
z6
z3
z4
z7
z1
|||| yx GGf +≅∇
2
0
-2
1
0
-1
0
1
-1
0
0
0
1
2
1
-2
-1
-1
xG yG
Various filter masksVarious filter masks
ExamplesExamples
edges by wall bricks, too detail !
ExamplesExamples
Note that averaging caused the response of all edges to be weaker
Diagonal Edge MasksDiagonal Edge Masks
Diagonal EdgeDiagonal Edge
LaplacianLaplacian
The Laplacian is a second-order derivative
Digital approximation
2
2
2
22
y
f
x
ff
∂∂
+∂∂
=∇
)(4 864252 zzzzzf +++−=∇
)(8 9876432152 zzzzzzzzzf +++++++−=∇
-1-1-1
-18-1
-1-1-1
0-10
-14-1
0-10
LaplacianLaplacian
The Laplacian generally is not used in its original form for edge detection for several reasons
It is unacceptably sensitive to noise
The magnitude of the Laplacian produces double edges
It is unable to detect edge direction
The role of the Laplacian in segmentationfinding the location of edge using its zero crossing property
judging whether a pixel is on the dark or light side of an edge
LaplacianLaplacian of Gaussianof Gaussian
The Laplacian is combined with smoothing to find edges
Gaussian function
LoG(Laplacian of Gaussian) : Mexican hat
⎟⎟⎠
⎞⎜⎜⎝
⎛−−=
2
2
2exp)(
σr
rh
⎟⎟⎠
⎞⎜⎜⎝
⎛−⎥
⎦
⎤⎢⎣
⎡ −−=∇
2
2
4
222
2exp)(
σσσ rr
rh
222 yxr +=
LoGLoG : : LaplacianLaplacian of Gaussianof Gaussian
LaplacianLaplacian of Gaussianof Gaussian
MeaningGaussian function : smoothing, lowpass filter, noise reduction
It is of interest to note that neurophysiologicalexperiments carried out in the early 1980s provide evidence that certain aspects of human vision can be modeled mathematically in the basic form of LoG function
LaplacianLaplacian of Gaussianof GaussianGaussian function with a standard deviation of five pixels
27x27 spatial smoothing maskThe mask was obtained by sampling the Gaussian function at equal intervalsAfter smoothing we apply the Laplacian maskComparison
The edges in the zero-crossing image are thinner than the gradient edgesThe edges determined by zero crossings form numerous closed loops (spaghetti effect: drawback)The computation of zero crossings presents a challenge
Edge LinkingEdge LinkingEdge detection edge linkingOne of the simplest approaches for linking edge points is to analyze the characteristics of pixels in a small neighborhoodAll points that are similar according to a set of predefined criteria are linkedTwo principal properties
The strength of the gradient The direction of the gradient vector
The direction of the edge at (x,y) is perpendicular to the direction of the gradient vector at that point
Eyxfyxf ≤∇−∇ ),(),( 00
Ayxyx ≤− ),(),( 00αα
Edge LinkingEdge Linking
Given n points in an image, suppose that we want to find subsets of these points that lie on straight linesOne possible solution is to first find all lines determined by ebery pair of points,then find all subsets of points that are close to particular linesIt involves finding n(n-1)/2 lines and then performing n x n(n-1)/2 comparisons !Hough transform[1962]
Hough Transform[1962]Hough Transform[1962]
Parameter space considerationInfinitely many lines pass through a point are represented a line in the parameter space(ab-plane)
When the line associated with intersects the line associated with at , is the slope and
the intercept of the line determined by the two points
baxy ii += ii yaxb +−=),( ii yx
),( jj yx )','( ba 'a
'b
Hough transformHough transformThe Hough transform subdivides the parameter space into so-called accumulator cellsThese cells are set to zeroFor every point (xk, yk) , we let the parameter a equal each of the allowed subdivision values on the a-axis and solve for the corresponding bThe resulting b’s are then rounded off to the nearest allowed value in the b-axisIf a choice of ap results in solution bq, we let A(p,q)=A(p,q)+1At the end, a value of Q in A(i,j) corresponds to Q points in the xy-plane lying on the line y=aix+bj
kikj yaxb +−=
Hough transformHough transform
If there are K increments in the a axis, K computation is needed for every point nK computation for n image points (linear)There is a problem when the slope approaches infinity (vertical line)
Hough transformHough transformOne solution: to use the following representation
The loci are sinusoidal curves in the -planeQ collinear points lying on a lineyield Q sinusoidal curves that intersect at
ρθθ =+ sincos yx
ijj yx ρθθ =+ sincos),( ji θρ
ρθ
Hough transformHough transformAerial infrared imageThresholded gradient image using Sobel operatorsPixels are linked
they belongs to one of the three accumulator cells with highest countNo gaps were longer than five pixels
ThresholdingThresholding
Global ThresholdingApply the same threshold to the whole image
Local ThresholdingThe threshold depends on local property such as local average
Dynamic(adaptive) thresholdingThe threshold depends on the spatial coordinates
The problem is how to select the threshold automatically!
Global Global ThresholdingThresholding
It is successful in highly controlled environmentOne of the areas is in industrial inspection application, where control of the illumination usually is feasible
Global Global ThresholdingThresholding
Automatic threshold selection
1. Select an initial estimate for T
2. Segment the image using T, which produce two groups, G1, G2
3. Compute the average gray level values and for the pixels in regions G1 and G2
4. Compute a new threshold value:
5. Repeat step 2 through 4 until the difference in T in successive iterations is smaller than a predefined parameter T0
Optimal Global and Adaptive Optimal Global and Adaptive ThresholdingThresholding
Method for estimating thresholds that produce the minimum average segmentation error Suppose that an image contains only two principal gray-level regionsThe histogram may be considered an estimate of the probability density function(PDF) of gray level valuesThis overall density function is the sum or mixture of two densities, one for the light and the other for the dark regionsThe mixture parameters are proportional to the relative areas of the dark and light regionsIf the form of the densities is known or assumed, it is possible to determine an optimal threshold
Optimal Global and Adaptive Optimal Global and Adaptive ThresholdingThresholding
The mixture probability density function is
and are the probabilities of occurrence of the two classes of pixels
is the probability (a number) that a random pixel with value Z is an object pixelAny given pixel belongs either to an object or to the background
The main objective is to select the value of T that minimizes the average error in making the decisions that a given pixel belongs to an object or to the background
)()()( 2211 zpPzpPzp +=
121 =+ PP
1P 2P
1P
Optimal Global and Adaptive Optimal Global and Adaptive ThresholdingThresholding
The probability of errorneously classifying a background point as an object point is
Similarly,
Then the overall probability of error is
To find the threshold value for minimal error requires differentiating E(T) with respect to T and equating the result to 0The result is
Note that if then the optimum threshold is where the curves for and intersect
)()()( 2112 TEPTEPTE +=
∫ ∞−=
TdzzpTE )()( 21
∫∞
=T
dzzpTE )()( 12
)()( 2211 TpPTpP =21 PP =
)(2 zp)(1 zp
Optimal Global and Adaptive Optimal Global and Adaptive ThresholdingThresholding
If the PDFs are Gaussians such as
and the variances are equal,
then the threshold is given
If , the optimal threshold is the average of the means
),,( 11 σμG
⎟⎟⎠
⎞⎜⎜⎝
⎛−
++
=1
2
21
221 ln
2 P
PT
μμσμμ
21 PP =
),( 22 σμG22
21
2 σσσ ==
Use of Boundary CharacteristicsUse of Boundary Characteristics
The chance of selecting a “good” threshold are enhanced considerably if the histogram peaks are tall, narrow, symmetric, and separated by deep valleysOne approach for improving the shape of histograms is to consider only those pixels that lie on or near the edges between objects and the backgroundSeparate an object and the background around a boundary usinggradient and Laplacian
(…) (-,+) (0 or +) (+,-) (…) structure
0
0
0
),(2
2
<∇≥∇≥∇≥∇
<∇
⎪⎩
⎪⎨
⎧
−+=
fandTff
fandTfif
Tfif
yxs
0 01 Labeling
Use of Boundary CharacteristicsUse of Boundary Characteristics
Segmentation is a process that partitions R into nsubregions, R1, R2,..., Rn, such that
is a logical predicate which represents criterion that segments the regions
(complete))(1
RRa i
n
i=
=U
region connected a is)( iRb
(disjoint), and allfor )( jijiRRc ji ≠=φI
iRPd i allfor TRUE)()( =
jiRRPe ji ≠= for FALSE)()( U
)( iRP
Region GrowingRegion GrowingRegion growing is a procedure that groups pixels or subregions into larger regions based on predefined criteria
The basic approach is to start with a set of ‘Seed’ points and from these grow regions by appending to each seed those neighboring pixels that have properties similar to the seed
Selections of ‘Seed’ point and similarity criteria are the primary problem
Another problem in region growing is the formulation of a stopping rule
ex) P: diff<3 P: diff<8
66702
56510
77610
78511
76500
bbbaa
bbbaa
bbbaa
bbbaa
bbbaa
aaaaa
aaaaa
aaaaa
aaaaa
aaaaa
Region GrowingRegion Growing
Region Splitting and MergingRegion Splitting and Merging
An alternative is to subdivide an image initially into a set of arbitrary, disjointed regions and then merge and/or split the regions to satisfy the conditions
Algorithm(1) We start with the entire region R
(2) If P(Ri)=FALSE, split into four disjoint quadrants
(3) Merge any adjacent regions Rj , Rk for which P(Rj U Rk)=TRUE
(4) Stop when no further merging or splitting is possible
Region Splitting and MergingRegion Splitting and Merging
Region Splitting and MergingRegion Splitting and MergingP(Rj)=TRUE
if at least 80% of the pixels in Rj have the property
where is the gray level of the jth pixel, is the mean gray level of region Rj, and is the standard deviation of the gray levels
iij mz σ2≤−
jz imiσ
The Use of MotionThe Use of Motion
Motion is a powerful cue used by humans and many animals to extract objects of interest from a background
One of the simplest approaches for detecting changes between two image frames is to compare them pixel by pixel
A difference image
It resulted from object motion
This approach is applicable only if the two images are registered spatially and if the illumination is relatively constant within the bounds established by T
Practically 1-valued entries often arise as a result of noise
They are usually isolated points ignore small regions, accumulation, filtering
⎩⎨⎧ >−
=otherwise
Ttyxftyxfifyxd ji
ij0
),,(),,(1),(
Building a Static Reference ImageBuilding a Static Reference Image
Digital Image Processing
1-Segmentation (Cont.)
2-Color Image Processing
Region-Oriented Segmentation
• Basic formulation
is a connected region
Region Growing by Pixel Aggregation
• Region growing is a procedure that groups
pixels or sub-regions into larger regions.
• Simplest approach is pixel aggregation
• Pixel aggregation needs a seed point
Region Growing Problems
• Selecting initial seed
• Selecting suitable properties for including
points
– Example: In military applications using infra red
images, the target of interest is slightly hotter than
its environment
Example : Weld defects
Region Splitting and Merging
• Divide the image into a set arbitrary disjoint
regions.
• Merge/split the regions
Quad-Tree
The Use of Motion in Segmentation
• Compare two image taken at time t1 and t2
pixel by pixel (difference image)
• Non-zero parts of the difference image
corresponds to the non-stationary objects
• dij(x,y)= 1 if |f(x,y,t1) – f(x,y,t2)| > θ
0 otherwise
Accumulating Differences
• A difference image may contain isolated
entries that are the result of the noise
• Thresholded connectivity analysis can remove
these points
• Accumulating difference images can also
remove the isolated points
Color Image Processing
• Color fundamentals
Visible Wavelength
Absorption of light by human eye
Red, Green, Blue Color Cube
RGB Color Cube
Composing Color Components
YIQ Color Model
• YIQ is the color space used by the NTSC color TV
system
• The Y component represents the luma
information, and is the only component used by
black-and-white television receivers
• I and Q represent the chrominance information
• For example, applying a histogram equalization to
a color image is done by Y component only
YIQ Color Model
HSI Color Model
Conversion from RGB to HSI
Pseudo-Color Image Processing
Pseudo-Color Image Processing
Pseudo-Color Image Processing
Enhancement using HSI Model
Enhancement using HSI Model
Assignment
An image is composed of small non-overlapping
blobs of mean gray level m=150 and variance
σ2=400. All the blobs occupy approximately
20% of the image area. Propose a technique
based on thresholding fro segmenting the
blobs out of the image.
Potential Term Project Topics
• Segmentation using Watershed algorithm
Potential Term Project Topics
• Segmentation Using Level Sets
Image Processing : Feature
Extraction
CS-293
Guide: Prof G. Sivakumar
Compiled by:
Siddharth Dhakad
Chirag Sethi
Nisarg Shah
Problem Definition
• Image processing is a field of signal processing in which input and output signals are both images (2D matrices).
• Feature extraction involves simplifying the amount of resources required to describe a large set of data accurately.
• Our project studies and implements various algorithms on feature extractions.
The following Feature Extraction techniques were implemented
• Edge detection is a terminology which aim at identifying points in a digital image at which the image brightness changes sharply or more formally has discontinuities.
• Edge Detection algorithms are broadly classified as
– First order
– Higher order
Original (Input) Image
Sobel Edge Detector
Input Image
Gray-Scaled Image
Masking
Edge Detected
Threshold
Canny Edge Detector
Input Image Gray-Scaled Image
Noise Reduction
Edge DetectedThreshold
Masking
Non-maxima
Suppression
• For reducing noise, we used Gaussian Filter (Blurring Algorithm).
• On edge detected by first order detector, we used an algorithm called Non-maximal Suppression to have single/thin edges.
• After that, we used Stack data structure for path traversal for finding continuous edges.
Manual Threshold Automatic Threshold
Original Image Sobel
Canny Manual Canny Automatic
Canny v/s Sobel
• Thin Edges
• Noise reduction
• Complete path
• High adjustability
• Thick Edges
• No blurring
• Discrete pixels
• Low adjustability
Line & Circle Detection
Input Image Edge Detection Hough Transform
DetectedLines/Circles
ThresholdPath Traversal
In Circle Detection, we also included taking range of radius of circles
to be detected as input to make the algorithm run faster.
Original Image Line Hough Transform
Circle Hough TransformOriginal Image
Corner Detection
Input Image
Cornerness of each pixel
Corner Detected
Threshold
Original Image
Corner
Detected
Threshold Selection
• Manual:
– User can set high-low thresholds for each operation performed during the algorithm.
• Automatic:
– By experimentation, we have found a heuristic to select a suitable threshold for any input image.
The Hard Part
• Selecting suitable threshold for any input
image
• Deciding the proper class hierarchy for
modular implementation
• Implementing path traversal and non-
maximal suppression algorithms
Results & Analysis
(Applied to standard
test image)
Original Image Standard Canny Output Our Canny Output
