Sapienza Università di Roma

Dipartimento di Informatica eSistemistica

A DISTRIBUTED VISION SYSTEMFOR BOAT TRAFFIC

MONITORINGIN THE VENICE GRAND CANAL

D. Bloisi, L. Iocchi

2

ARGOS Project Overview

The ARGOS system is going to control a waterway ofabout 6 km length, 80 to 150 meters width, through14 observation posts (Survey Cells).

Automatic Remote Grand Canal Observation System

3

ARGOS Objectives

• management and evaluation of navigation rules• traffic statistics and analysis• security • preservation of historical heritage (reduction of wave motion)

4

ARGOS Functions

• optical detection and tracking of moving targets• computing position, speed and heading of targets• event detection (speed limits, access control, …)• recording 24/7 video and track information (post- analysis)• rectifying camera frames and stitching them into a composite view • automatic PTZ tracking• …

5

Survey Cell

3 high resolution network cameras, a PTZ camera for zoom and tracking of the selected target, and 2 computers running the image processing and tracking software.

The survey cells are installed on the top of severalbuildings leaning over the Grand Canal

6

Survey Cells

7

SC Software Architecture

Backgroundestimation

Backgroundsubtraction

Optical Flow

Foreground BlobsAnalysis

Segmentation

Centercamera

Rightcamera

Segmentation

Segmentation

List ofobservations

TrackingModule Boat

IDs

Leftcamera

8

Background Estimation

Problems:- gradual illumination changes and sudden ones (clouds)- motion changes (camera oscillations) - high frequency noise (waves in our case)- changes in the background geometry (parked boats).

Approach:- computation of color distribution of a set of frames- highest component form the background

9

Background Estimation (2)

Background Imagecomputed from S(the image display

only the higher gaussian values)

Set S of 20 images from a camera

Mask for cuttting offbuildings from computation

10

Background Subtraction

current frame

background image

foreground image

THRESHOLD (based onillumination conditions)

blobs (Binay Large OBjectS)

>

11

Optical Flow Computation

We use a sparse iterative version of Lucas-Kanade optical flow in pyramids ([Bouget00]). It calculates coordinates of the feature points on the current video frame given their coordinates on the previous frame. The function finds the coordinates with sub-pixel accuracy. Every feature point is classiefied into one of the four principal directions NE, NW, SE, SW.

[Bouguet00] Jean-Yves Bouguet. Pyramidal Implementation of the Lucas Kanade Feature Tracker.

previous frame current frameoptical flow image

(a particular)

NW direction

12

Segmentation

Exploiting the foreground image and the optical flow image,for every blob we obtain its centroid (that is (x, y) position into the current frame) its direction (and consequentely the probability of under segmentation if the blob is classified into more than one of the principal directions) its ellipse approximation (and consequentely its dimensions in meters through homography matrices)

Blob filtering: If a blob is too small according to the minimal dimension a boat must be in order to navigate the Gran Canal)

Under segmentation: If a blob has two or more directions we compute the center of mass and the variance for every of the four predetermined

principal direction.

13

Segmentation (2)

blue → NW directionred → NE directiongreen → SE direction

centroid

ellipsecenter of

mass

14

Rek-means

15

Rek-means (2)

16

Tracking module

Single-hypothesis TrackingWe use a set of Kalman Filters (one for each tracked boat).

Data Association: Nearest Neighbor ruleTrack formation: unassociated observationsTrack deletion: high covariance in the filter

Multi-hypothesis TrackingTrack splitting: in ambiguous cases (data association has multiple solutions)Track merging: high correlation between tracks

17

Multi hypothesis tracking (2)

3 tracks (240, 247, 285)only 1 actual observation (285)

240285247

18

Rectification

19

Unified Views

20

Panoramic view

PTZ Camera

21

Example

22

DENSITA’ DI TRAFFICO – TEMPO REALE

1

2

4

3

8

5

7

6

10

9

12

1113

DENSITA' MEDIE E MASSIME DEL TRAFFICO

02/11/2006 ore 11,30

Numero Totale Imbarcazioni in Canal Grande: 121

Tratto Da A Densità mediaDensità

max

1 Ponte Libertà Scomensera  5  8

2 Scomensera Ponte Calatrava 6  12 

3 Ponte Calatrava Ferrovia 8   10

4 Ferrovia Cannaregio 10  18

5 Cannaregio Santa Fosca 6   18

6 Santa Fosca Ca D'oro 4   4

7 Ca D'oro Rialto 12   16

8 Rialto San Silvestro 5 8

9 S.Silvestro San Tomà 14   26

10 San Tomà Ca' Rezzonico 21   25

11 Ca' Rezzonico Accademia 8   9

12 Accademia Salute  14  18

13 Salute Bacino S.Marco 8  8 

23

Example

24

Experimental Evaluation

on-line, evaluation is performed during the actual operation of the system;recorded on-line evaluation is performed on a video recording the output of the system running on-line;off-line evaluation is performed on the system running off-line on recorded input videos.

25

Online Evaluation

FN: False negatives, i.e. boats not trackedFP-R: False positives due to reflections (wrong track with a random direction)FP-W: False positives due to wakes (wrong track following the correct one)

26

Counting Evaluation

COUNTING EVALUATION TEST

A virtual line has been put across the Canal in the field of view of a surveycell, the number of boats passing this line has been counted automaticallyby the system nSys, and the same value is manually calculated by visuallyinspection n, the average percentage error is then computed as

ε = | nSys – n | / n

An additional error measure is calculated by considering the probability of makingan error in counting a single boat passing the line

where δ(·) is 0 when the argument is 0 and 1 otherwise.

27

Counting Evaluation (2)

28

Speed and velocity tests