Object Detection by Matching

Longin Jan Latecki

Contour-based object detection

?

Database shapes: …..

3

Easy for Human Eyes

• Everybody can find the swan in these images.

4

Recall humans can draw a swan

• We always draw edges.

5

Computer can also capture edges

Humans can detect shapes given only edges

7

Problem: separating noise

• Too much noise, and the computer can’t tell which edges belong to object of interest.

8

Object recognition process:

Source:2D image of a 3D object

Matching to database shapes

Contour Segmentation

Contour Extraction

Object Segmentation

Contour Cleaning, e.g., Evolution

Object detection as matching database shapes to image edge segments

Database shapes: …..

Contour groupingEdge detection Edge linking

?matching

Main challenges

2 . Part of the true contour of the target object may be wrongly connected to part of a background contour resulting in a single edge fragment

1 . The contour of the desired object is typically fragmented over several pieces.

How to find the true contours of the target shape in the edge image?

Problem formulation

?

Key idea:Given a minimal required coverage of the model contour,we want to select non overlapping model fragments that maximizethe configuration similarity to the corresponding image fragments.

All relevant edge fragments are mapped to their corresponding model fragments.

Key idea:1. Build an association graph.2. Find maximum weight

subgraph

Construction of Affinity Matrix

Each vertex of the graph corresponds to a partial match

The affinity between node i and node j is based on their shape similarity.

jv

iv

( , )A i j

The weighted affinity graph is denoted as G = (V, A).

Construction of Affinity Matrix

High affinity:

Low affinity:

Problem with Affinity Matrix

Wrong matches may also have high affinity:

We use model and image location constraints to sparsify the Affinity Matrix

Maximal Cliques in a Weighted GraphA maximal clique is a subset of V with maximal average affinity between all pairs of its vertices.

In this example, the maximal clique has 4 nodes selected from over 500 nodes. Therefore, most clustering based approach may not succeed.

[ M. Pavan and M. Pelillo. PAMI 2007]

In order to solve this combinational problem, we relax it to

A vertex is selected as belonging to a MWS iff v V 0vx

Each MWS corresponds to a local maximum of:

Each local solution is not a final solution but a detection hypothesis.

Indicator = selected maximal clique of vertices of V.

Computing Maximum Weight Subgraphs

{0,1}NX

1: 0 and 1NX R X X

( ) Tf X X AX

Tianyang Ma and Longin Jan Latecki. From Partial Shape Matching through Local Deformation to Robust Global Shape Similarity for Object Detection.CVPR 2011.

Object detection examples