Lecture 8:Image Segmentation -...

transcript

Prof. YingLi TianNov. 7, 2018

Department of Electrical EngineeringThe City College of New York

The City University of New York (CUNY)

Lecture 8: Image Segmentation

I2200: Digital Image processing

Thanks to G&W website, Dr. Shahram Ebadollahi for slide materials

Outline Introduction Thresholding Edge Segmentation and Linking

Hough Transform Region-based Approach Using Morphology for Segmentation

Watershed Algorithm

Image Analysis and Segmentation Low-level image processing: inputs and

outputs are all images Mid-/High-level image processing: inputs

are images; outputs are information or attributes of the images

Image analysis: extract useful information from the result of low-level image processing

Image segmentation: divides an image into meaningful parts or objects 3

Image Segmentation Image segmentation -- decompose image into

“meaningful” regions or objects Discontinuity: the abrupt change in graylevels -- point,

line, edge detection… Similarity: predefined criteria of similarity --

thresholding, region growing, region splitting and merging…

Approaches: Thresholding edge-based vs. region-based global vs. local Feature-based – texture, motion, color

Image Segmentation

Detection of discontinuities Points, Lines, Edges

Point Detection

MATLAB:w = [-1 -1 -1; -1 8 -1; -1 -1 -1];

g = abs(imfilter(double(f), w) >= T;

Point Detection Example

Line Detection

Line Detection Example

Edge Detection Common method to detect discontinuities

because isolated points or one-pixel wide lines are rare

Edge: the borders of regions with different graylevels

Digital edge model:

Ramp Edge More clear edge: width of the ramp is narrow;

fuzzy edge: width of the ramp is wider All pixels in the ramp are edge points First derivatives:

Ramp Edge

Second derivatives:

Ramp Edge

Second derivatives:

Sign of second derivative determines if an edge point is on the bright or dark side There are positive and negative values on a ramp

edge Zero-crossing: intersection of the 0-value line and

the line connecting the positive and negative points position of edge

Image Derivatives

Image Derivatives 1st order derivative produces “thick” edges 2nd order derivative has stronger response

to fine edges 2nd order derivative produces double edge

response at ramp and step transitions in intensity

2nd order derivative’s sign can be used to find out if going from dark to light or vice versa

Edge Detection Edge point: 2nd order derivative is greater

than a threshold Edge: set of connected edge points (need to

predefine “connection”) Edge segment: short edges

Edge Detection Implementation

2nd order derivative: Laplacian or LoG operator

Edge Detection

After thresholding

Averaged prior to edge detection

Edge detection: Marr-Hildreth method

Edge detection operator should be “tunable” to detect edges at different “scales”

Edge detection: Marr-Hildreth method

In practice- sample Gaussian function [nxn samples]- convolve with image f(x,y): image smoothing- convolve result with Laplacian mask- find zero-crossings of g(x,y) How?

Low zero-crossingthreshold

High zero-crossingthreshold

g(x, y)

Edge detection: Canny method Goal: Satisfy the following 3 criteria:

Detection: should not miss important edges Localization: distance between the actual and located

position of the edge should be minimal One response: only one response (edge) to a single

actual edge Algorithm

Step 1: Smooth input image with a Gaussian filter Step 2: Compute the gradient magnitude and angle

images

Canny Edge detection – Step 3: nonmaxima suppression

Canny Edge detection – Step 4: Reducing false edge points (hysteresis thresholding)

Canny Edge detection

Thresholding Goal: setting threshold T in histogram to

segment the image Global Thresholding: T is constant

everywhere in the image Variable Thresholding: T varies

Local Thresholding: T changes based on the neighborhood properties

Adaptive Thresholding: T changes based on the coordinates

Thresholding

Thresholding Multimodal

T related to f(x, y) only: global thresholding

T related to f(x, y) and p(x, y) only: local thresholding

T related to f(x, y), p(x, y), and (x, y): dynamic (adaptive) thresholding

Thresholding - pitfalls

Effect of Illumination

Effect of Noise

Global Thresholding

1. T=T02. Segment using T3. Get average gray-level for region G1 (f > T) and region G2 (f <= T)4. T_new = average of average of gray-levels5. Repeat until convergence

Global Thresholding

Optimal Threshold

Pose problem as: minimizing error of assigning pixels in image to two or more groups

Optimal Threshold

Optimal Threshold – Otsu’s method Otsu defines the measure of “goodness” of

a threshold based on how well it can separate two (or more) classes (regions)

Goal: Measure of goodness

Optimal Threshold – Otsu’s method

Thresholding in MATLAB using Otsu’s method fordetermining the threshold:

Adaptive Thresholding – Image Partitioning

Adaptive Thresholding – Moving Average

40http://en.wikipedia.org/wiki/Thresholding_(image_processing)

Hough Method for Line Detection

Hough Method: Normal line representation

Hough Method: Matlab

Edge-based segmentation: Problems Spurious edges due to noise and low

quality image. Difficult to identify spurious edges.

Dependent on local neighborhood information

No notion of higher order organization of the image

Gaps and discontinuities in the linked edges

Region-based segmentation

Region Growing Goal: group pixels or subregions into larger

regions based on predefined criteria for growth

Method:1. Start with a set of “seed” points2. Grow regions by appending to each seed

those neighboring pixels that have predefined properties similar to the seed (e.g., graylevel, texture…)

Region Growing

Use points (3, 2) and (3, 4) are seeds; property: grayleveldifference < 3:

Region splitting and merging Goal: Partition image into disjoint

subregions and then perform splitting and merging

Method: subdivide R until subregion P(Ri) = TRUE

1. Splitting: if P(R) = FALSE, divide R into quadrants (Ri, i = 1, 2, …, 4); if P(Ri) = FALSE, divide Ri into subquadrants, and so on Quadtree of quadregions

49MATLAB quadtree decomposition: qtdecomp( )

Region splitting and merging

Morphological Watersheds “Topographic” points:

Points belong to regional minimum Points at which drop of water, if placed at the

location of any of those points, would fall with certainty to a single minimum (catchment basin or watershed)

Points at which water would be equally likely to fall to more than one such minimum (divide lines or watershed lines)

Morphological Watersheds

Goal: Find watershed lines

Build dams to prevent water flow from one catchment basin to other

Boundaries between flooded catchment basins

Segmentation using Watershed

Moving Object Detection Main goal: detecting moving objects from a

video sequence of a fixed camera Background – static scene Foreground – moving objects Approach: detect the moving objects as the

difference between the current frame and the image of the scene background.

Current imagefrom C. Stauffer and W. Grimson

Background image Foreground pixels

Challenges How to automatically obtain the background

image? Foreground processing The background image must adapt to:

Illumination changes (gradual and sudden) Distracting motions (camera shake, swaying

tree, moving elevators, ocean waves…) Scene changes (parked car)

Others: Shadows Black/blue screen Bad weathers Foreground fragment

Basic BGS – basic idea (1) Pixels belongs to foreground if | Current

Frame – BG Image| > Threshold BG image can be just the previous frame or

the average image of a number of frames Works only in particular conditions of

objects’ speed and frame rate Very sensitive to the threshold

Basic BGS – example (2)

Current Frame

Difference

High threshold

Lowthreshold

Basic BGS -- average (3) Median/average: use the average or the

median of the previous n frames Quick but memory consuming

Running average Alpha is adapt rate (0.05).

)()1(1 iii FBB

Basic BGS – selectivity (4)

BGS Examples

Background Subtraction and Foreground Analysis

Moving Object Detection with Lighting Changes

Improved BGS Results

Recent Deep Learning-based Image Segmentation

Image Segmentation (PSPnet):https://www.youtube.com/watch?v=HYghTzmbv6Q&t=27s

https://www.youtube.com/watch?v=rB1BmBOkKTw&t=23s

Announcement Reading

G&W 8.1-8.2 (exclude 8.2.2) McKay book chapter 1, 5

http://www.inference.phy.cam.ac.uk/mackay/itila/

Final project is finalized. The projects and report template can be found from course website.

Peter presents HW3 today.

Final Project -- Due: 12/11/2018 Each Project,

final report (doc file)

presentation (ppt file) 12/11/2018 source code

Final Report Introduction (Problem statement, Motivation) Approach Work Performed (Experiments, software programming, etc.) Results Discussion Reference (cited papers or website) A table listing tasks performed in the project and the percentage

contributions by each team member of individual tasks, as well as overall project. All team members must sign this table.

Final Project -- Due: 12/11/2018 Steps to do final project,

Understand the problem to be solved Literature review to understand state-of-the-art Decide which methods you want to try Method implementation Method testing and evaluation Analyze results Write final report Prepare presentation slides

Total 15 minutes (12 mins talk + 3 mins questions)

Final Project -- Due: 12/11/2018 Project Presentation on 12/12/2018,

Project 1: Peter Vertenten Project 2: Jinglun Feng Project 3: Rezwon Bhuiyan Project 4: Dingdong Li Project 5: Md Rashiduzzaman and SM Asraful

Alam Project 6: Subash Shrestha Project 7: Sandeep kumar and Azita darvishi Project 8: Tahiya Tazreen and Md. Minhaz Zaman

Lasker

Final Project -- Due: 12/11/2018 Project Presentation on 12/19/2018,

Project 9: Muhammad Bhatti Project 10: Ching Chan and Tamanna Tanjil Project 11: Munib Ahsan Project 12: Mollie Murray Project 13: Keshav Teeluck Project 14: Michael Ousseinov Project 15: Ashfaq A Khan

HW3 Presentation

Lecture 8:Image Segmentation -...

Documents