Date post: | 05-Jul-2015 |
Category: |
Documents |
Upload: | riccardo-satta |
View: | 994 times |
Download: | 1 times |
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 20111
Università degli Studi di Cagliari
Exploiting Multiple Component Representationsfor Person Re-Identification
Ph.D. candidate: Riccardo SattaAnnual report, Year I
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 2
Outline
• Introduction to Person Re-Identification
• Problem formulation
• The proposed Multiple Component Matching (MCM) framework
• Implementation of MCM, and experimental results
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 3
Introduction
In video surveillance, it is often desirable to determine if an individualhas already been observed over a network of cameras.
This issue is called Person Re-Identification
In general, we can’t apply face recognition algorithms (low resolution!),therefore we must consider the global “appearance” of the individual
short term problem
Scenarios: Tracking of the movements of an individual in large public places
monitored by several non-overlapping cameras Use in conjunction with other common identification techniques
(RFID, biometric systems)
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 4
Main issues
- Low resolution of the frames (all figs)Less data availableMore subject to noise
- Changing lighting conditions (fig. 1)Brightness changesContrast changes
- Different sensor responses (fig. 2)Colour temperatureWhite balance
- Partial occlusions (fig. 3)
- Pose variations (fig. 1, 4)
figure 1
figure 4
figure 2
figure 3
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 5
Problem formulation
We can model the problem as follows:given a gallery set of templates T = {T1,…,Tn}
and a probe Q, find the most similar template T* T with respect to a similarity measure D(·, ·):
T*= arg min D(Ti, Q)
Descriptor generation
MATCHING SCORE(similarity)
Descriptor generation
TEMPLATE
PROBE
T=
QTi
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 6
My contribution to this field
• A general framework to for people re-identification problems– Inspired to common paradigms in Machine Learning and Computer Vision– Includes and generalizes ideas partly embedded in previous works– Aims at providing a common foundation for existing and future methods
• A method for person re-identification– Simple, yet effective, implementation of the framework
• … and some interesting ideas for further developments
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 7
Towards a Multiple Component Representation of the Human Body
Human body peculiarities– non-rigid object– complex kinematics– composed of many quasi-rigid parts
Possible approach: COMPONENT SUBDIVISION
We can take into account an arbitrary m-component subdivision by fusing matching scores of every part
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 8
Towards a Multiple Component Representation of the Human Body
How to represent every single component?
Multiple Instance Learning (MIL) paradigm:the object of interest is represented by a bag of instances
• MIL framework adapted to a matching problem• Every component is described by a series of instances
– Patches, point of interest…
{x1 , … , xp}
{x1 , … , xp}
{x1 , … , xp} {x1 , … , xp} {x1 , … , xp}
{x1 , … , xp}
{x1 , … , xp}
Object
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 9
Multiple Component Matching Framework
The proposed Multiple Component Matching framework integrates multiple instance representations with component subdivision in a matching paradigm
– every individual Xi is decomposed
into an ordered sequence of parts X j
i
– every part X j
i is represented
by a set of instances X j
i,n
– matching is performed at set level
– global matching distance is a combination of set distances
Example (4 parts)
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 10
Multiple Component Matching Framework
Multiple Component Matching (MCM) framework– Inspired to MIL statistical learning framework– Embeds two concepts (part subdivision and multiple instance representation)
that can be found, even if partly and implicitly, in the main part of the existing works
– Aims at providing a solid foundation for existing and future works
Well, nice theory… but in practice?
Implementation of MCM:– choose part subdivision– choose what to treat as an “instance”– choose an appropriate distance between sets d( ), and how to combine set
distances
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 11
MCM Implementation
Part subdivisionAnti-symmetry properties of the human silhouette [1]
Let we first define two operators:
chromatic bilateral operator
spatial covering operator
[1] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani. Person re-identification by symmetry-driven accumulation of local features. In Proc. IEEE Conf. on Comp. Vision and Patt. Rec., 2010
[ ]∑
+−
=δδ
δiiB
ii ppdiC,
),(),( 2
[ ]( ) [ ]( )[ ]δδδδ +− −= iiii BABA
JiS ,,
1),(
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 12
MCM Implementation
Part subdivision
head-torso axis is given by
separating regions that strongly differ in area
torso-legs axis is given by
separating regions with strongly different appearance and similar area
Two parts: [torso] and [legs], discarding the head
( )( ) ( )δδ ,,1minarg iSiCii
TL +−=
( )( )δ,minarg iSii
HT −=
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 13
MCM Implementation
Instances– For each part, we extract random overlapping patches of [25% - 75%] width in
respect of the part width and [25% - 75%] height in respect of the part height– Each instance is described by an HSV colour histogram (24-12-4 bins)
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 14
MCM Implementation
Instances - Simulation– To asset lighting and contrast variations we generate synthetic patches for
templates– Each RGB pixel is multiplied by different values
• we chose [1.4 1.2 1.0 0.8 0.6]• This changes both brightness (mean) and contrast (variance) of the patches
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 15
MCM Implementation
Distance between sets d( )– k-th Haussdorff distance [2]
Metric:
distB Bhattacharyya distance between the HSV histograms of the patches
ypos,a, ypos,b mean vertical position of the patch
Set distance combination– Average distance between sets:
[2] Wang and J.-D. Zucker. Solving the multiple-instance problem: A lazy learning approach. In Proc. Int. Conf. Mach. Learn., 2000.
( ) ( ) ( )( )ABhBAhBAdH ,,,max, =
( ) ( )bakthBAhk −= min, BbAa ∈∈ ,
( ) ( ) bposaposbaB yyHSVHSVba ,,1,dist −+⋅=− β
( ) ( )∑=i
i BAdM
D ,1
, BA
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 16
Evaluation
Evaluation on the ViPER dataset– 632 individuals, 1 template and 1 probe image per individual, SvsS scenario– random selection of 316 individuals, average performance on 10 runs
SDALF refers to the state-of-the art on this dataset [1]
[1] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani. Person re-identification by symmetry-driven accumulation of local features. In Proc. IEEE Conf. on Comp. Vision and Patt. Rec., 2010
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 17
Prototypes
AmI LAB demo applicationA prototype is being realized at the Ambient Intelligence Lab of Sardegna Ricerche
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 18
Further developments
• Research issues– Dissimilarity-based representations
– Frame quality
– Multiple frames accumulation
Prototype 1
Prototype 2
Prototype N
d1
d2
dN
Dissimilarity representation
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 19
Further developments
• New applications– Personal Photo Re-Tagging
ID 1 ID 1
ID 2
ID 2 ID 3 ID 3
Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 20
Thanks!
Questions?