Post on 19-Jan-2016
description
transcript
Object detection, tracking and event recognition: the ETISEO experience
Andrea Cavallaro
Multimedia and Vision Lab
Queen Mary, University of London
andrea.cavallaro@elec.qmul.ac.uk
Outline
• QMUL’s object tracking and event recognition• Change detection and object tracking• Event recognition
• ETISEO• Evaluation: protocol, data, ground truth• Impact• Improvements of future evaluation campaigns
• Conclusions
• … and an advert
Outline
• QMUL’s object tracking and event recognition• Change detection and object tracking• Event recognition
• ETISEO• Evaluation: protocol, data, ground truth• Impact• Improvements of future evaluation campaigns
• Conclusions
• … and an advert
Prior system for event detection
http://www.elec.qmul.ac.uk/staffinfo/andrea/CREDS-help.html
RATP/CREDS
Introduction
• QMUL Detection, Tracking, Event Recognition (Q-DTE)• initially designed for Event Detection and Tracking in metro
stations• modified to respond to ETISEO • components:
• Moving object detection
• Background subtraction with noise modeling
• Object tracking
• Graph matching
• Composite target distance based on multiple object features
• Event recognition
M. Taj, E. Maggio, A. Cavallaro“Multi-feature graph-based object tracking”Proc. of CLEAR Workshop - LNCS 4122, 2006
Object detection and tracking
• Change detection• Statistical change detection
• Gaussians on colour components
• Noise filtering• Contrast enhancement
• Problem: data association after object detection• Appearance/disappearance of objects• False detections due to clutter and noisy observations
Moving object segmentation
• Motion detection through frame difference
• Problem• D = {d k}, dk 0 even if there is no structural change in k
current frame reference frame difference frame D
02 / HtP k
0Hdp k
2
2
2 2exp
2
1
kd
Adaptive threshold for change detection
• Noise modelling
• Test statistics
• Significance test
Hyp. H0: “no changes in k”, camera noise N(0, )
kw
kk d 22
02 Hp k ondistributi2
Tracking
• Graph matching using weighted features• Data association verified throughout several frames
to validate the correctness of the tracks • Support track recovery in occlusion scenarios • Features
• centre of mass
• velocity
• bounding box
• colour
).1(),(.),(.),(.),(.),(
H]h,w, , y,[x, = X
4321
..
ijXXgXXgXXgXXgXXg
yx
bj
ai
bj
ai
bj
ai
bj
ai
bj
ai
velocity
appearance
size
position
Graph matching: full graph
v(x11)
v(x21)
v(x31)
v(x41)
v(x13)
v(x23)
v(x33)
v(x43)
v(x12)
v(x22)
v(x32)
V1 V3
v(x11)
v(x21)
v(x31)
v(x41)
v(x13)
v(x23)
v(x33)
v(x43)
V2
v(x12)
v(x22)
v(x32)
v(x11)
v(x21)
v(x31)
v(x41)
v(x13)
v(x23)
v(x33)
v(x43)
v(x12)
v(x22)
v(x32)
Graph matching: max path cover
V1 V3V2
Experimental framework
• Key parameters• noise variance: 1.8• kernel size: 3x3 • feature weights
• position α = 0.40
• velocity β = 0.30
• appearance γ = 0.15
• size δ = 0.15
• Determined using CLEAR dataset/metrics• Moving object detection accuracy / precision (MODA / MODP)• Moving object tracking accuracy / precision (MOTA / MOTP)
Event recognition
Event recognition
Event recognition
Event recognition
Outline
• QMUL’s object tracking and event recognition• Change detection and object tracking• Event recognition
• ETISEO• Evaluation: protocol, data, ground truth• Impact• Improvements of future evaluation campaigns
• Conclusions
• … and an advert
ETISEO
• Impact• Promote evaluation
• Formal and objective evaluation is (urgently) needed
• Data collection and distribution• time consuming!
• common ground for research
• Priority sequences • Use of an existing XML schema • Discussion forum
• Choice of performance measures and experimental data is not obvious
Improvements
• Involve stakeholders at earlier stages• More input from end users
• what do they want / need?
• costs / weights of errors
• Involve (more) researchers from the beginning • Facilitate understanding of the protocol
• Fix errors / ambiguities early
• Use training/testing dataset• see i-Lids and CLEAR
• Maybe private dataset too
• Give meaning to measures • what is the “value” of these numbers?
• e.g., compare with a naïve result
• what is the “value” of a difference of (e.g.) 0.1?
• Improvements of future evaluation campaigns• Are we evaluating too many things simultaneously?
• Too many variables
• Do we need so many measures?• remove redundant measures
• Is the ground truth really “truth”?• statistical analysis / more annotators / confidence level
• Should we distribute the evaluation tool / ground-truth earlier? • Are we happy with the current demarcation of regions / definition of
events?• Do we want to evaluate all the event types together?
• should we focus on subsets of events and move on progressively
• Is the dataset too heterogeneous? • Can we generalize the results obtained so far?
Questions
Conclusions
• Conclusions• QMUL submission
• Statistical colour change detection
• Multi-feature weighted graph matching
• Event recognition module: evolution from CREDS 2005.
• Next: extend to 3D
• Feedback on ETISEO• Evaluation + discussion
• Extend the community / do not duplicate efforts
• Metrics
More information
http://www.elec.qmul.ac.uk/staffinfo/andrea
… and an advert
IEEE International Conference on
Advanced Video and Signal based Surveillance
IEEE AVSS 2007London (UK)
5-7 September 2007
Paper submission: 28 February 2007
• Acknowledgments• Murtaza Taj• Emilio Maggio
Evaluation metric
http://www.elec.qmul.ac.uk/staffinfo/andrea/CREDS-help.html
S
DAt
B
Maximum Delay
Maximum Score
Accepted anticipation
Unaccepted anticipation