ConstrainedOpticalFlowForAerialImageChangeDetection.pdf

CONSTRAINED OPTICAL FLOW FOR AERIAL IMAGE CHANGE DETECTION

- IGARSS 2011 -

Speaker: Nicolas BOURDIS – EADS France

Other Authors: Denis MARRAUD – EADS France

Hichem SAHBI – CNRS Télécom ParisTech

2

Context:

– Aerial observation

– Change detection

Introduction

2

Context:



Operationnal difficulties:

– Huge volumes of data (videos)

– Confusing image contents

– Poor image quality

Applications: video surveillance, database update, …

Introduction

2

Context:



→ Need for automatic approaches addressing attention focusing

Operationnal difficulties:

– Huge volumes of data (videos)

– Confusing image contents

– Poor image quality

Applications: video surveillance, database update, …

Introduction

Introduction to Change Detection problem • Terminology • Nature of Changes

Contributions

• Change Detection using Unconstrained Optical Flow • Change Detection using Constrained Optical Flow • Likelihood Ratio Test

Experimentations

• Novel Public Database • Evaluation Criterion • Approach Evaluation Results

3

Presentation Outline

4

Data: ‐ Test data, on which changes are to be detected ‐ Reference data, with which test data is compared

Problem Changes Baseline

Method

Optimized

Method

Likelihood

Ratio Test Database Criterion Results

General problem Contributions Experimentations

Change Detection Problem (1/2)

4


Objectives:

‐ Detection of the presence of changes ‐ Change localization and estimation of the change mask,

which associates binary labels to the test data.


Method

Optimized

Method

Likelihood





Objectives:

‐ Detection of the presence of changes ‐ Change localization and estimation of the change mask,

which associates binary labels to the test data.

Sub-problems involved:

‐ Search of a candidate reference data ‐ Registration of the considered data ‐ Comparison of the data

4


Method

Optimized

Method

Likelihood




5

Sources of differences between images: Acquisition: artifacts, sensibilities… Appearance : specularities, color changes, … Movement: water, tree leaves, … Viewpoints: parallax, occlusions, … Illumination: shadows, nature of lighting, … Displacements: cars, pedestrians, … Modifications: appearing/disappearing objects, deformation, …


Method

Optimized

Method

Likelihood




4

Test image Difference > Threshold Reference image

Sources of differences between images: Acquisition: artifacts, sensibilities… Appearance : specularities, color changes, … Movement: water, tree leaves, … Viewpoints: parallax, occlusions, … Illumination: shadows, nature of lighting, … Displacements: cars, pedestrians, … Modifications: appearing/disappearing objects, deformation, …


Method

Optimized

Method

Likelihood




How to recognize true changes from parallax effects?

6


Method

Optimized

Method

Likelihood



Parallax Effects

6

P

C1

C2

p2

p1



Method

Optimized

Method

Likelihood



Parallax Effects

→ Residual vectors:

Collinear to epipolar lines, after registration with an arbitrary surface [1].

[1] Kumar, Anandan & Hanna. ICPR94. « Shape recovery from multiple views: A parallax based approach ».

6

d H

P

C1

C2

p2

p1





Method

Optimized

Method

Likelihood



Parallax Effects


6

d H

P

C1

p1 h1

C2

p2

e1



Method

Optimized

Method

Likelihood



Parallax Effects




Estimation of residual vectors using optical flow:

7

HSV color encoding of 2D vectors (Hue↔Angle, Saturation↔Norm, Value=Cst)


Method

Optimized

Method

Likelihood



Unconstrained Optical Flow

First Changed/Unchanged decision criterion at pixel x:

7

v

e

ε x

Estimation of residual vectors using optical flow:


Method

Optimized

Method

Likelihood



HSV color encoding of 2D vectors (Hue↔Angle, Saturation↔Norm, Value=Cst)

Unconstrained Optical Flow

Further reduction of false alarms:

• Integration of the epipolar constraint in optical flow algorithm

• New decision criterion based on local matching error

8


Method

Optimized

Method

Likelihood



Constrained Optical Flow

8

1. Polynomial expansion on a neighborhood P1(x) = xT.A1.x + b1

T.x + c1 P2(x) = xT.A2.x + b2

T.x + c2

Image 2

x0

Image 1

x0





Method

Optimized

Method

Likelihood




Farneback. IA03. « Two-frame motion estimation based on polynomial expansion ».

8 Image 2

x0

Image 1

x0





Method

Optimized

Method

Likelihood





T.x + c1 P2(x) = xT.A2.x + b2

T.x + c2

2. Ideally: P2(x) = P1(x-d) Hence A1=A2, b2=b1-2.A1.d, c2=dT.A1.d-b1

T.d+c1 d is found from A.d = Δb

With A=(A1+A2)/2 and Δb=-1/2.(b2-b1)


8 Image 2

x0

Image 1

x0





Method

Optimized

Method

Likelihood





T.x + c1 P2(x) = xT.A2.x + b2

T.x + c2



With A=(A1+A2)/2 and Δb=-1/2.(b2-b1)

3. Noise sensitivity → average on a neighborhood Least squares minimization of ∑i||Ai.d- Δbi||2 Hence d = (∑iAi

T.Ai)-1.∑iAi

T. Δbi And e = ∑iΔbi

T.Δbi - d.∑iAiT.Δbi


8


T.x + c1 P2(x) = xT.A2.x + b2

T.x + c2

Image 2

x0

Image 1

x0



With A=(A1+A2)/2 and Δb=-1/2.(b2-b1)

3. Noise sensitivity → average on a neighborhood Least squares minimization of ∑i||Ai.d.ue- Δbi||2 Hence d = (∑iue

TAiT.Aiue)-1.∑iue

TAiT. Δbi

And e = ∑iΔbiT.Δbi - d.∑iue

TAiT.Δbi





Method

Optimized

Method

Likelihood




9

Likelihood ratio test:

• Hypothesis H0 = absence of change

• Hypothesis H1 = presence of change


Method

Optimized

Method

Likelihood



Likelihood Ratio Test

9

→ Exponential distribution

→ Rayleigh distribution

Likelihood ratio test:

• Hypothesis H0 = absence of change

• Hypothesis H1 = presence of change


Method

Optimized

Method

Likelihood



Likelihood Ratio Test

10

Difficulties of real data acquisition campaigns:

• Heaviness of acquisition logistics

• Difficulty of implementing interesting changes

• Heaviness and subjectivity of ground truth annotation


Method

Optimized

Method

Likelihood



Benchmark Dataset

10

→ Public datasets for image change detection algorithms are difficult to find

Benchmark dataset of synthetical images: “AICD Dataset”

• Realistic aerial images with complex illumination / relief

• Automatic extraction of objective ground truth

Difficulties of real data acquisition campaigns:

• Heaviness of acquisition logistics

• Difficulty of implementing interesting changes

• Heaviness and subjectivity of ground truth annotation


Method

Optimized

Method

Likelihood



Benchmark Dataset

11

Contents of our “AICD Dataset”: • 100 different scenes acquired from 5 viewpoints

• 1000 pairs of 800x600 images plus ground-truth

Sample images:


Method

Optimized

Method

Likelihood



AICD Dataset (1/2)

12

For each scene and each viewpoint: • Reference & Test image (different illumination) • Ground-truth mask

Sample images:


Method

Optimized

Method

Likelihood



AICD Dataset (2/2)

12 → Freely downloadable

Sample images: For each scene and each viewpoint: • Reference & Test image (different illumination) • Ground-truth mask


Method

Optimized

Method

Likelihood



AICD Dataset (2/2)

13

Objective :

– Attention focusing on areas of interest

Pixel-based criterion = too restrictive

Image-based criterion = too global


Method

Optimized

Method

Likelihood



Evaluation Criterion (1/2)

13

« Connected component » criterion: Ground-Truth

Estimation

Valid case Intermediary case Invalid case

Objective :

– Attention focusing on areas of interest

Pixel-based criterion = too restrictive

Image-based criterion = too global


Method

Optimized

Method

Likelihood




14

« Connected Components » criterion


Method

Optimized

Method

Likelihood




14



Method

Optimized

Method

Likelihood




14



Method

Optimized

Method

Likelihood




15

Visual Inspection

Majority of accurate detections

Results Sample


Method

Optimized

Method

Likelihood



Processing time ≈ 5s per 800x600 image (monothread CPU)

Results Sample

15

Visual Inspection


Method

Optimized

Method

Likelihood



Some mis-detections and false alarms

Processing time ≈ 5s per 800x600 image (monothread CPU)

16

Comparison of the two Change Detection criteria


Method

Optimized

Method

Likelihood



Evaluation Results (1/3)

17

Comparison of the robustness to viewpoint difference


Method

Optimized

Method

Likelihood




18

Comparison of the robustness to relief


Method

Optimized

Method

Likelihood




19

Summary

Efficient & effective change detection approach:

• Optical-flow based→ fast and well suited to GPU porting

• Good performance in the context of attention focusing

Novel benchmark dataset for change detection:

• Publicly available: www.enst.fr/~bourdis/Work/ChangeDetectionDataset.html

• Objective dense ground truth

Summary

19

Efficient & effective change detection approach:

• Optical-flow based→ fast and well suited to GPU porting

• Good performance in the context of attention focusing

Thank you for your attention

Novel benchmark dataset for change detection:

• Publicly available: www.enst.fr/~bourdis/Work/ChangeDetectionDataset.html

• Objective dense ground truth

Date post:	12-Jul-2015
Category:	Business
Upload:	grssieee
View:	418 times
Download:	0 times

ConstrainedOpticalFlowForAerialImageChangeDetection.pdf

Business