Date post: | 15-Jan-2016 |
Category: |
Documents |
View: | 216 times |
Download: | 0 times |
1
Presentation of Vista team(VIsion Spatio-Temporelle et Apprentissage)
Patrick Bouthemy
UR Rennes, IRISA
(http://www.irisa.fr/vista/)
2
INRIAP. Bouthemy, F. Cao (2001), I. Laptev (2005),
P. Pérez (2004),
C. Kervrann (2003, on Inria secondment, Inra)
CNRS J. P. Le Cadre
University of Rennes 1 E. Mémin
J. Yao (scientific collaborator, Maths dept., Irmar)
Secretary :: H. Béchu (Inria, with Temics team)
VISTA-2005 research staff
3
Ph-D students (11)
Post-docs (2) I. Laptev (until August 2005),
Venkatesh Babu (until March 2005)
Temporary technical staff (3) N. Gengembre (FT-RD contract), P. Heas (IST-FET Fluid),
B. Fauvet (RIAM Feria project, until June 2005)
VISTA-2005 research staff – continued
G. Piriou, T. Bréhard, T. Veit (tbd, Dec. 2005)
V. Auvray, A. Cuzol, J. Boulanger (3rd year)
A. Bugeau, N. Papadakis (2nd year)
T. Crivelli, C. Simonin, A. Hervieu (1st year)
4
Research topics (1) Overall objective: Analyzing dynamic scenes or physical
phenomena from image sequences (video, IR, satellite images, video-microscopy)
Involved topics:Restoring image sequencesDetecting, segmenting, tracking moving entitiesMeasuring and representing image motionModeling, learning and recognizing spatio-temporal contents
Problem formulation based on visual (2D) motion models and measurements
5
Research topics (2)
Applied maths foundations: mostly statistical approaches for image analysis
Probabilistic models (Markov models, probabilistic mixed-state models)
Bayesian inference, robust estimation, non-parametric estimation
A contrario decision methodsParticle filteringRobust classification
6
Application domains
Content-aware video applications (multimedia and TV, sports videos, enriched video, video summarization, video indexing, video re-purposing) main partners: INA, Thomson, FT-RD projects: IST Lava (XRCE, Lear team), RNTL Domus Videum (Atlas, Metiss
and Temics teams), RIAM Feria (Texmex team), IST-NoE Muscle (Ariana, Imedia and Texmex teams), FT-RD contract, ACI Behaviour (UTC, PSA)
Experimental fluid mechanics and Meteorological imagery partners: Cemagref, LMD projects: Eumetsat, IST-FET Fluid, ACI Assimage (Clime and Idopt teams)
Biological imagery (video-microscopy) partners: INRA, Curie Institute, Univ. Rennes 1 (Dept of Biology) projects: ACI-IMPBio MoDynCell5D, AC DRAB
7
Main evolutions 2001- 2005
Important changes in the team composition Creation of Texmex team (2002, P. Gros, 2 people from Vista, multimedia
indexing, as announced in 2001) Creation of Lagadic team (2004, F. Chaumette, E. Marchand, F. Spindler,
11 people all from Vista, as announced in 2001) Creation of Visages team (2004, C. Barillot, P. Hellier, S. Prima, 6 people
from Vista, medical imaging) Arrival of 4 research scientists over 2001-2005 (3 Inria, 1 Inra)
Given the present Vista team composition and the corresponding re-focusing (in 2004) on dynamic scene analysis: Achievement of (or at least investigation of) all the objectives listed in
2001 (apart from a minor point, NDC of structures under strain) Addition of two new topics (a contrario motion analysis, video-microscopy)
and reinforcement of a third one (video analysis and tracking).
8
Main contributions (1) Adaptive non-parametric estimation for image sequence denoising
Efficient method based on locally adaptive kernel regression while
preserving space-time discontinuities without motion estimation Data-driven adaptive estimation window based on bias-variance trade-off
criterion Point-wise and patch-based versions
Experimental comparative evaluation on usual video image sequences: our method outperforms other recent methods (joint Kalman-Wiener, wavelet-based, and PDE-based methods)
Left: original sequence; middle: noisy sequence; right: denoised sequence with our method
9
Main contributions (1bis) Application to video-microscopy
Goal: detection and tracking of moving intra-cell protein vesicles to model and interprete their dynamics for biological studies
First issue addressed: 3D image sequence denoising
Positioning (national and international context): Pasteur Institute (J.-C. Olivo-Marin), ENS Paris, EPFL (M. Unser), Heidelberg Univ.Image sequence provided Denoised sequence
by Curie Institute, with our methodstudy of Rab6 protein
One 3D imageof the sequence:noisy (up) anddenoised (bottom)
10
Main contributions (2) A contrario methods for motion detection and matching
Decision is performed a contrario: large deviation (very small probability, NFA computation) to a random background model (independence assumption), no prior required on the searched entities, importance of the measurement choice.
General, simple and parameter-free (automatic threshold setting) method while supplying confidence values.
Pos.: ENS Cachan (J.-M. Morel, a contrario framework for static image analysis, collaboration on shape matching and recognition), Ceremade (F. Dibos), UPF Barcelona (V. Caselles)
11
Main contributions (2bis)
Addressed problems A contrario instantaneous detection of moving regions A contrario coherent motion detection (clustering in an appropriate
space, initialization for tracking)
A contrario image comparison (global and local criteria) and retrieval in video streams
12
Main contributions (3) Mixed-state probabilistic models for motion modeling and recognition
A unique representation for information which can be both discrete (symbolic) and continuous. No supplementary (hidden) label field requiring an inference stage.
Definition of specific mixture distributions (Dirac + exponential family) to model (global or local) occurrence statistics of 2D motion measurements for motion classification, supervised motion recognition and event detection (in
sports videos). Recognition of serve shots
Definition of mixed-state auto-models (extension of Besag’s framework to multi-parameter cases) and first application to the modeling of temporal (motion) textures.
Pos.: UCF (M. Shah), Weizmann Institute (M. Irani), Microsoft Research (H.-J. Zhang), Univ. Rochester (M. Tekalp), UCLA (S. Soatto, linear models for dynamic textures)
(false detection)
13
Main contributions (4)
Fluid motion analysis
Estimation of dense velocity fields for fluid flowsSpecific data model (continuity equation) and second-order div-curl regularization
Parsimonious decomposition on adapted basis functions (vortex and source particles)Estimation of the motion of multiple layers (exploiting a prior image segmentation)Intensive evaluation on typical fluid flow experiments (in collaboration with Cemagref)
Pos.: Correlation-based methods in applied meteorology (wind field computation), PIV (Particle Image Velocimetry) techniques in experimental fluid mechanics
(Image sequence provided by Onera)
14
Main contributions (4bis) Fluid motion analysis (continued)
Structuration of the fluid flow fieldEstimation of the potential functions associated with the irrotational and solenoidal components of the flow field (Fourier and variational approaches)
Tracking of high-dimensional flow structures (e.g., motion fields)Stochastic filtering relying on the vorticity-velocity form of Navier-Stokes equation and then Itô diffusion processDeterministic filtering exploiting a variational data assimilation technique
Tracking of a cycloneby stochastic filtering
16 vortex particles1000 filtering particles
15
Main contributions (5) Non-linear tracking methods
Robust tracking in low-dimensional state-space and complex likelihoodsMulti-object tracking with particle filters (novel SMC techniques, data-target association issue) New conditional formulation of classical filters with state equation estimated from the image data (point and planar structure trackers) Tracking with auxiliary variables (handling of visibility, occlusion events)
Left: our method (item 2), all the points are correctly tracked,Right: STK tracker (several points are lost) Tracking with auxiliary variables
of visibility
16
Main contributions (5bis) Non-linear tracking methods (continued)
High-dimensional state spaces and detailed dynamicsSee previous slide on fluid motion analysis
Target tracking for partially observed non-linear systemsInitialization of particle filtering based on a hierarchical approachDerivation of the PCRB closed-form (Posterior Cramer-Rao Bound) for performance analysis and sensor managementDesign of decentralized particle filters
Pos.: Georgia Tech (F. Delleart), EPFL (P. Fua), ETHZ (L. van Gool), Brown University (M. Black), Siemens Princeton (D. Comaniciu), Microsoft Research (A. Blake), DSTO Adelaide (N. Gordon), Qinetiq UK (S. Maskell)
Distributed target tracking with two observers (in blue) True trajectory in green, estimated trajectories obtained by each observer in red. Confidence areas for each observer in red are large. Distributed target tracking in black. Confidence area in black is very small.
17
Some highlights (1)
Academic dissemination and visibility 16 Ph-D theses, 62 journal papers (2001-2005), 2 books, 2 best paper awards Co-general chair of ICCV’07, organization of CBMI’03 (of Miccai’04, Visages
team), PC co-chair of RFIA’06, Associate editors IEEE-IP, Area editor JAIF.
Involvements in industrial partnership, applications and transfer Consortium (GIS Aeternum Multimedia) formed by Thomson, INA and Irisa
(Metiss, Texmex and Vista teams)
Cross-modal video summarization tool validated on football and rugby TV programs (Thomson)
FT-RD contract on team sports video processing (Rugby world cup’2007)
Head of European FLUID project (LaVision PIV company); transfer of estimation tool of dense fluid flow fields to meteorological domain (Eumetsat) and experimental fluid mechanics one (Cemagref, Onera)
18
Some highlights (2) International collaborations
Associate Inria team FIM (University of Buenos-Aires, on fluid motion analysis and on dynamic texture analysis)
Univ. of Mannheim (C. Schnörr), Univ.of Cambridge (J. Vermaak), Univ. of Las Palmas (L. Alvarez), UCSD (S. Belongie), European NoE Muscle
Software Motion-2D (source code for estimating parametric motion models, publically
available under QPL): about 550 downloads (explicitly identified) since August 2003 (http://www.irisa.fr/vista/Motion2D/)
Tracking application (on-going development)
Platform Multimedia platform (with Texmex and Metiss teams, collaboration with
INA, starting 2005): capture of TV programs, content server, metadata server, multimedia processing, content-based multimedia indexing, experimental evaluation,…
19
Perspectives (1)
Video analysis and understanding
Analysis of motion textures with mixed-state models (segmentation, classification)
Modeling and recognition of dynamic video contents from bags of trajectories
Video alignment (dynamic multi-view constraints) for content recognition and retrieval
Combined motion and object categorization and recognition Learning issues: semi-supervised learning with tracking for video
contents; dynamic kernel-based methods Specific focus on sports TV programs (cross-modal analysis, self-
structuration, video summarization)
20
Perspectives (2)
Tracking problems
Visual tracking with no prior (on-line learning and updating of appearance model, cluttered background, combination of complementary representations, conditional Markov models)
Camera network (partially observed dynamic system, low-level measurements, polyedral separation methods, trajectory reconstruction, situation analysis for groups of moving entities)
High dimensionality in visual tracking problems (variational assimilation techniques and non-linear Bayesian filters)
Trajectory optimization for terrain-aided navigation (collaboration with Onera)
21
Perspectives (3) Image sequence analysis for environmental, fluid mechanics
and biological studies
Further incorporation of physical laws 3D motion estimation from 3D volume data or from multi-view 2D
data (in experimental fluid mechanics and meteorological applications) New involvement in oceanography applications
Continuation of our work in video-microscopy on the detection and tracking of small moving intra-cell entities
Interpretation of analysed dynamic contents in relation with biological studies (partnership with biologists)
Envisaged proposition (in 2-3 years) of a new joint Inria-Inra team devoted to biological image sequence analysis and related applications