Post on 03-Apr-2018
transcript
7/28/2019 Data Clustering for Image Segmentation
1/51
DataData clusteringclustering forfor imageimage
segmentationsegmentation
Domenico Bloisibloisi@dis.uniroma1.it
7/28/2019 Data Clustering for Image Segmentation
2/51
27/10/2008Data clustering for image segmentation Page 2
Summary
ARGOS Project overview
Image Segmentation and Optical Flow (OF) K-means based OF data clustering
Rek-means vs k-means Examples and experimental evaluation
7/28/2019 Data Clustering for Image Segmentation
3/51
27/10/2008Data clustering for image segmentation Page 3
ARGOS project overview
TheARGOS system controls a waterway of about 6
km length, 80 to 150 meters width, through 14observation posts (Survey Cells).
Automatic Remote Grand Canal Observation System
7/28/2019 Data Clustering for Image Segmentation
4/51
27/10/2008Data clustering for image segmentation Page 4
Rai Tg1 video
7/28/2019 Data Clustering for Image Segmentation
5/51
27/10/2008Data clustering for image segmentation Page 5
ARGOS objectives
management and evaluation of
navigation rulestraffic statistics and analysis
securitypreservation of historical heritage
(reduction of wave motion)
7/28/2019 Data Clustering for Image Segmentation
6/51
27/10/2008Data clustering for image segmentation Page 6
Motivation
Human Surveillance
Until 2006 the Municipal Administration of Venice paid people forboats counting:
9 Error prone
9 Una tantum
Automatic Surveillance
FromNovember 2007 Venice has an automatic video surveillancesystem
9 24 hours/day 7 days/week
9 recorded traffic violations
9 Certified error
7/28/2019 Data Clustering for Image Segmentation
7/51
27/10/2008Data clustering for image segmentation Page 7
ARGOS functions
optical detection and tracking of moving targets
computing position, speed and heading of targetsevent detection (speed limits, access control, )recording 24/7 video and track information (post-
analysis)rectifying camera frames and stitching them into acomposite view
automatic PTZ trackingon line traffic information http://www.argos.venezia.it
7/28/2019 Data Clustering for Image Segmentation
8/51
27/10/2008Data clustering for image segmentation Page 8
www.argos.venezia.it
7/28/2019 Data Clustering for Image Segmentation
9/51
27/10/2008Data clustering for image segmentation Page 9
Examples
Detect boats docking inthe highlighted area
Speed limit control
7/28/2019 Data Clustering for Image Segmentation
10/51
27/10/2008Data clustering for image segmentation Page 10
ARGOS features
Dynamic background (water)
Multi object tracking(up to 10 boats in the same view)
Multi camera for large areas(56 cameras x 6 km length)
Third party extensive evaluation(2.2107 frames analyzed)
7/28/2019 Data Clustering for Image Segmentation
11/51
27/10/2008Data clustering for image segmentation Page 11
General architecture
7/28/2019 Data Clustering for Image Segmentation
12/51
27/10/2008Data clustering for image segmentation Page 12
Survey cell
3 high resolution network cameras, a PTZ camera for zoomand tracking of the selected target, and 2 computers running
the image processing and tracking software.
The survey cells are installedon the top of severalbuildings leaning over theGrand Canal
7/28/2019 Data Clustering for Image Segmentation
13/51
27/10/2008Data clustering for image segmentation Page 13
Survey cells
7/28/2019 Data Clustering for Image Segmentation
14/51
27/10/2008Data clustering for image segmentation Page 14
Segmentation module
Rek-meansclustering algorithm
S t ti
7/28/2019 Data Clustering for Image Segmentation
15/51
27/10/2008Data clustering for image segmentation Page 15
Segmentation(detecting objects of interest)
Background
Formation
CurrentFrame
I
BackgroundModel
BForegroundComputation
ListofDetected Objects
continuously
updatedF
7/28/2019 Data Clustering for Image Segmentation
16/51
27/10/2008Data clustering for image segmentation Page 16
Background formation
Problems:
- gradual illumination changes and sudden ones (clouds)- motion changes (camera oscillations)- high frequency noise (waves in our case)- changes in the background geometry (parked boats).
Approach:- computation of color distribution of a set of frames
- highest component formthe background
7/28/2019 Data Clustering for Image Segmentation
17/51
27/10/2008Data clustering for image segmentation Page 17
Background modeling
Background Image
computed fromS(the image displayonly the higher
gaussian values)
SetS ofn images froma camera
Natural images
Artificial image
Foreground computation
7/28/2019 Data Clustering for Image Segmentation
18/51
27/10/2008Data clustering for image segmentation Page 18
Foreground computation(background subtraction technique)
current frame
background image
foreground image
THRESHOLD T(based on illumination conditions)
blobs (Binay Large OBjectS)
>
T
7/28/2019 Data Clustering for Image Segmentation
19/51
27/10/2008Data clustering for image segmentation Page 19
Tracking module
Single-hypothesis Tracking
We use a set of Kalman Filters (one for each tracked boat).
Data Association: Nearest Neighbor ruleTrack formation: unassociated observationsTrack deletion: high covariance in the filter
Multi-hypotheses Tracking
Track splitting: in ambiguous cases (data association has
multiple solutions)Track merging: high correlation between tracks
7/28/2019 Data Clustering for Image Segmentation
20/51
27/10/2008Data clustering for image segmentation Page 20
Multi hypothesis tracking
3 tracks (240, 247, 285)only 1 actual observation (285)
240285247
R tifi ti
7/28/2019 Data Clustering for Image Segmentation
21/51
27/10/2008Data clustering for image segmentation Page 21
Rectification
7/28/2019 Data Clustering for Image Segmentation
22/51
27/10/2008Data clustering for image segmentation Page 22
Unified Views
7/28/2019 Data Clustering for Image Segmentation
23/51
27/10/2008Data clustering for image segmentation Page 23
Panoramic view
PTZ Camera
DENSITA DI TRAFFICO TEMPO REALE
7/28/2019 Data Clustering for Image Segmentation
24/51
27/10/2008Data clustering for image segmentation Page 24
DENSITA DI TRAFFICO TEMPO REALE
1
2
4
3
8
5
7
6
10
9
12
11
13
88Bacino S.MarcoSalute13
1814SaluteAccademia12
98AccademiaCa' Rezzonico11
2521Ca' RezzonicoSan Tom10
2614San TomS.Silvestro9
85San SilvestroRialto8
1612RialtoCa D'oro7
44Ca D'oroSanta Fosca6
186Santa FoscaCannaregio5
1810CannaregioFerrovia4
108FerroviaPonte Calatrava3
126Ponte CalatravaScomensera2
85ScomenseraPonte Libert1
Densit
maxDensit mediaADaTratto
Numero Totale Imbarcazioni in Canal Grande: 121
02/11/2006 o re 11,30
DENSITA' MEDIE E MASSIME DEL TRAFFICO
E l
7/28/2019 Data Clustering for Image Segmentation
25/51
27/10/2008Data clustering for image segmentation Page 25
Example
B k d bt ti bl
7/28/2019 Data Clustering for Image Segmentation
26/51
27/10/2008Data clustering for image segmentation Page 26
Background subtraction problems
Background subtraction is a fast and effectivetechnique, but it presents a series of problems:
How to compute a correct background?
How to manage gradual and sudden illumination
changes? How to manage high-frequencies background objects(such as tree branches, sea waves, and similar)?
How to manage partially occluded objects (for example,two cars crossing in a street)?
How to manage partially occluded
7/28/2019 Data Clustering for Image Segmentation
27/51
27/10/2008Data clustering for image segmentation Page 27
How to manage partially occluded
objects?
UNDER SEGMENTATION1 blob 2 boats
OVER SEGMENTATION3 blobs 2 boats
P d l ti
7/28/2019 Data Clustering for Image Segmentation
28/51
27/10/2008Data clustering for image segmentation Page 28
Proposed solution
Background Subtraction
+Optical Flow+
Rek-means
[Bloisi and Iocchi 2008]
O ti l fl
7/28/2019 Data Clustering for Image Segmentation
29/51
27/10/2008Data clustering for image segmentation Page 29
Optical flow
We use a sparse iterative version of Lucas-Kanade optical flow inpyramids [Bouget2000]. It calculates coordinates of the featurepoints on the currentvideo frame given their coordinates on the
previous frame. The function finds the coordinates with sub-pixelaccuracy. Every feature point is classiefied into one of four principaldirections.
previous frame current frameoptical flow image
(a particular)
NW direction
Under segm solution using OF
7/28/2019 Data Clustering for Image Segmentation
30/51
27/10/2008Data clustering for image segmentation Page 30
Under segm. solution using OF
Solution (non trivial):clustering the OF sparse map
outliersnoisek=?
Over segm solution using OF
7/28/2019 Data Clustering for Image Segmentation
31/51
27/10/2008Data clustering for image segmentation Page 31
Over segm. solution using OF
K means clustering
7/28/2019 Data Clustering for Image Segmentation
32/51
27/10/2008Data clustering for image segmentation Page 32
K-means clustering
+k-means centroidsx truecentroids
randominitialization
K means limitations
7/28/2019 Data Clustering for Image Segmentation
33/51
27/10/2008Data clustering for image segmentation Page 33
K-means limitations
the user may knowthe exact number of clusters (k)beforehand
k-means is notguaranteed to return a global optimum:The quality of the final solution depends largely on theinitial set of centroids, and may, in practice, be much
poorer than the global optimum.
If we choose the k initial clusters at random, k-means
can converge to the wrong answer (in the sense thatadifferent and optimal solution to the minimization functionexists).
Rek means features
7/28/2019 Data Clustering for Image Segmentation
34/51
27/10/2008Data clustering for image segmentation Page 34
Rek-means features
Improved clustering using Rek-means
7/28/2019 Data Clustering for Image Segmentation
35/51
27/10/2008Data clustering for image segmentation Page 35
Improved clustering using Rek-means
Rek means algorithm
7/28/2019 Data Clustering for Image Segmentation
36/51
27/10/2008Data clustering for image segmentation Page 36
Rek-means algorithm
Rek means steps
7/28/2019 Data Clustering for Image Segmentation
37/51
27/10/2008Data clustering for image segmentation Page 37
Rek-means steps
a) Initial data setb) Over-clustering
c) Rectangles mergingd) Final centroids
Rectangle distance
7/28/2019 Data Clustering for Image Segmentation
38/51
27/10/2008Data clustering for image segmentation Page 38
Rectangle distance
Validating step
7/28/2019 Data Clustering for Image Segmentation
39/51
27/10/2008Data clustering for image segmentation Page 39
Validating step
Anderson Darling test
7/28/2019 Data Clustering for Image Segmentation
40/51
27/10/2008Data clustering for image segmentation Page 40
Anderson Darling test
if A2 > CV the test is negative
else if A2 < CV the test is positivewhere CV is the critical value
Validating step example
7/28/2019 Data Clustering for Image Segmentation
41/51
27/10/2008Data clustering for image segmentation Page 41
Validating step example
The associatingstep finds 1
cluster
The validatingstep correctsthe error and
finds 2 clusters
d >>
Rek-means real data example
7/28/2019 Data Clustering for Image Segmentation
42/51
27/10/2008Data clustering for image segmentation Page 42
Rek means real data example
Current frame and foreground image Rek-means output
Rek-means evaluation
7/28/2019 Data Clustering for Image Segmentation
43/51
27/10/2008Data clustering for image segmentation Page 43
Rek means evaluation
Time complexity O(nlogn)
100 images (with resolution 2000x2000)
ARGOS experimental evaluation
7/28/2019 Data Clustering for Image Segmentation
44/51
27/10/2008Data clustering for image segmentation Page 44
ARGOS experimental evaluation
on-line, evaluation is performed during the actual operationof the system;recorded on-line evaluation is performed on a video
recording the output of the system running on-line;off-line evaluation is performed on the system running off-line on recorded input videos.
Online evaluation
7/28/2019 Data Clustering for Image Segmentation
45/51
27/10/2008Data clustering for image segmentation Page 45
Online evaluation
FN: False negatives, i.e. boats not tracked
FP-R: False positives due to reflections (wrong track with a random direction)FP-W: False positives due to wakes (wrong track following the correctone)
Counting evaluation
7/28/2019 Data Clustering for Image Segmentation
46/51
27/10/2008Data clustering for image segmentation Page 46
Counting evaluation
COUNTING EVALUATION TEST
A virtual line has been put across the Canal in the field of viewof a surveycell, the number of boats passing this line has been counted automatically
by the systemnSys, and the same value is manually calculated by visuallyinspection n, the average percentage error is then computed as
= | nSys n | / n
An additional error measure is calculated byconsidering the probability of makingan error in counting a single boat passing the line
where () is 0 when the argument is 0 and 1 otherwise.
Counting evaluation (2)
7/28/2019 Data Clustering for Image Segmentation
47/51
27/10/2008Data clustering for image segmentation Page 47
Cou g e a ua o ( )
New features
7/28/2019 Data Clustering for Image Segmentation
48/51
27/10/2008Data clustering for image segmentation Page 48
It is crucial to know who does what
Behavior analysis based on theidentity of boats observed is verydifficult through Computer Vision only
(plate recognition impossible withocclusion)
Add sensors such as GPS or RadioFrequency Identifiers (RFID)
References
7/28/2019 Data Clustering for Image Segmentation
49/51
27/10/2008Data clustering for image segmentation Page 49
- D. Bloisi, L. Iocchi, G.R. Leone, R. Pigliacampo, L. Tombolini, L. Novelli. ADistributed Vision System for Boat Traffic Monitoring in the Venice Grand Canal
(VISAPP), 2007.
- D. Bloisi and L. Iocchi. Rek-Means: A k-Means Based Clustering Algorithm. In Proc.of 6th Int. Conf. on Computer Vision Systems (ICVS). 2008. pp. 109 - 118
- J .Y. Bouguet. Pyramidal Implementation of the Lucas Kanade Feature Tracker.2000
Homework
7/28/2019 Data Clustering for Image Segmentation
50/51
27/10/2008Data clustering for image segmentation Page 50
Generate a data set froma distribution formedby a mixture of k Gaussians with the same
circular variance (i.e., = diag {, }) andwith | i - j | > 3 for each pair i,j
Apply k-means on this data (with the correctvalue of k) several times starting fromdifferent
initial values and compute the percentage ofcorrect clustering.
DataData clusteringclustering forfor imageimage
7/28/2019 Data Clustering for Image Segmentation
51/51
DataData clusteringclustering forfor imageimage
segmentationsegmentation
Domenico Bloisibloisi@dis.uniroma1.it