Robust Object SegmentationUsing Adaptive Thresholding
Xiaxi Huang and
Nikolaos V. Boulgouris
International Conference on Image Processing 2007
Outline
Introduction Proposed algorithm Experimental results Conclusions
Introduction(1/2) The extraction of moving objects from video se
quences is important!! Object detection methods
SGM ( Single Guassian Model ) MGM ( Mixed Guassian Model ) BG substaction combined color and edge inform
ation. (Aug. 2000)
Serious drawbacks
Poor performance for indoor shadow, light reflection, and high similarity of FG and BG.
Introduction(2/2)
Proposed algorithm Adaptive thresholding detection Shadow removal method
Proposed AlgorithmProcedures Of Algorithm
UpdatingBG Image
Initial MaskEstimate FG Areas
EstimateBG Areas
ConfidenceMap From Detection
Of RGB
ConfidenceMap From Detection Of Edge
Maximum Of The
ConfidenceMaps
Minimum Of The
ConfidenceMaps
Combined Confidence
Map
HysteresisThresholding
FG Map(Object andShadows)
Edge Map(BoundaryOf objects)
Post-processing
Final ObjectMap
Next Frame
BackgroundSubstraction
ShadowRemoval
Proposed AlgorithmBackground Updating
Why? In many background substraction method up
date all pixels in a frame. A serious drawback
To avoid this condition
Misclassfied a stop moving object.
11 1 ttt BIB
Proposed AlgorithmInitial Block-size Mask(1/5)
Roughly determine the foreground areas. Lower threshold
Calculate average different between the current frame and background frame in a block
Threshold it with T
BIdavg
1
Proposed AlgorithmInitial Block-size Mask(2/5)
Divide the blocks with larger difference which are assumed to contain foreground pixels into smaller size
Apply a higher threshold and detect sub-blocks
41 /nn
addednn TTT 1
Proposed AlgorithmInitial Block-size Mask(3/5)
The block with larger difference
Using a higher threshold
…
Proposed AlgorithmInitial Block-size Mask(4/5)
How to get a foreground blocks map? Median filter
Edge pixels of the objects might be lost Apply two initial block-size, and combine
their foreground map
' MapMapMapblock
Proposed AlgorithmInitial Block-size Mask(5/5)
Proposed AlgorithmColor Change Detection With Adaptive Threshold(1/2)
Adaptive threshold
D : difference frame ( D = | I - B|)
202
2
002
0
22
21
1
1
1
/
currcurr
diffdiff
diff
diffdiffcurradapt
I
D
D
kkT
Local variance in the current frame
Local variance in the difference frame
Local mean value in the difference frame
Proposed AlgorithmColor Change Detection With Adaptive Threshold(2/2)
Get a threshold T by setting k1, k2
Create confidence maps in three color channels respectively Maximum the confidence map CMapcolor
Proposed AlgorithmEdge Detection(1/2)
In order to - Extraction of the foreground Removal of shadow
Compute edge magnitude
Gx and Gy are the horizontal and vertical difference in the difference frame D.
Sobel mask
22yxmagnitude GGG
121
000
121
101
202
101
yx HH
Proposed AlgorithmEdge Detection(2/2)
Proposed AlgorithmCombination(1/3)
Combine two confidence maps. Estimate foreground area
Maximum the confidence map
Estimate background area Minimum the confidence map
blockedgecolor MapCMapCMapCMap ,max1
blockedgecolor MapCMapCMapCMap ,min2
Proposed AlgorithmCombination(2/3)
Combination
21 CMapCMapCMap
Proposed AlgorithmCombination(3/3)
(a) (b)
(c) (d)
Fig. (a) Original image, (b) Confidence map Of RGB change Detection with adaptive threshold, (c) Confidence map of Sobel edge detection, (d) Combined confidence Map.
Proposed AlgorithmHysteresis thresholding
Remove false positive Set two thresholds T0, T1 ; T1/T0 is about 2
or 3 C(x) > T1 : High confidence region T0 < C(x) < T1 : Check neighbors
Hysteresisthresholding
Proposed AlgorithmShadow Removal And Post-Processing(1/4)
Indoor environment Soft colored illumination, light-reflect
floor, shadow Hard to distinguish shadows from objects
by using color information. How to solve this problem?
Combine FG and edge confidence map
Proposed AlgorithmShadow Removal And Post-Processing(2/4)
Apply hysteresis thresholding to edge confidence map
Set bounding boxes Remove pixels out of bounding boxes
Proposed AlgorithmShadow Removal And Post-Processing(3/4)
(a) (b)
(d)(c)
Fig. (a) Foreground map before shadow removal, (b) Hysteresis thresholdingresult of edge confidence Map, (c) Foreground mapafter shadow removal, (d) Binary map of extracted objects.
Proposed AlgorithmShadow Removal And Post-Processing(4/4)
Some temporal filters of offline detection
To achieve above -
11
11
tttt
TF
ttttTF
MapMapMapMap
MapMapMapMap
11 ,, ttttTF MapMapMapmedianMap
Eliminate spurious points
Retrieve missed FG pixels
Experimental ResultsFig. (a) Origin image, (b) Foreground maps createdby MGM plus HSV method, (c) Foreground maps created by mixture GaussianModel, (d) Foreground mapscreated by proposed algori-thm.
(a) (b)
(d)(c)
Experimental ResultsFig. (a) Origin image, (b) Foreground maps createdby MGM plus HSV method, (c) Foreground maps created by mixture GaussianModel, (d) Foreground mapscreated by proposed algori-thm.
(a) (b)
(d)(c)
Experimental ResultsFig. (a) Origin image, (b) Foreground maps createdby MGM plus HSV method, (c) Foreground maps created by mixture GaussianModel, (d) Foreground mapscreated by proposed algori-thm.
(a) (b)
(d)(c)
Experimental ResultsFig. (a) Origin image, (b) Foreground maps createdby MGM plus HSV method, (c) Foreground maps created by mixture GaussianModel, (d) Foreground mapscreated by proposed algori-thm.
(a) (b)
(d)(c)
Conclusions Compared with the popular MGM object
segmentation method and the HSV shadow removal method, proposed method achieves more robust performance
Considering that the proposed algorithm does not involve future frames, it can be used in real-time processing applications.
Furthermore, if it is used offline, a temporal filter can be applied to further improve the performance of the algorithm.