Scene-independent group
profiling in crowd
Jing Shao, Chen Change Loy, and Xiaogang Wang
The Chinese University of Hong Kong
Hillsborough Disaster, England, 96 deaths, 766 injuries
The inquiry into the disaster ... the main reason …
the failure of police crowd control.
How to characterize and compare dynamics
across so many different crowded scenes?
Scene-specific Crowd flow field
R. Mehran, et al., ECCV, 2010
S. Ali, et al., CVPR, 2007
Topic model
B. Zhou, et al., CVPR, 2011
T. Hospedales, et al., ICCV, 2009
X. Wang, et al., CVPR, 2008
Social forces
B. Zhou, et al., CVPR, 2012
R. Mehran, et al., CVPR, 2009
P. Scovanner, et al., ICCV, 2009
Scene
Activity models learned from a specific-scene cannot be ap-plied to other scenes
Problem
Micro-level Multi-target tracking & individual
behavior analysis
W. Choi, et al., ECCV, 2012
S. Pellegrini, et al., ICCV, 2009
M. Andriluka, et al., CVPR, 2008
T. Zhao, et al., CVPR, 2004
Macro-level
Scale
It’s hard to detect and track in-dividuals precisely in different crowded scenes
How to characterize and compare dynamics
across so many different crowded scenes?
Micro-level
Macro-level Optical flow codewords
D. Kuettel, et al., CVPR, 2010
C. C. Loy, et al., BMVC, 2009
Spatio-temporal grids
R. Mehran, et al., CVPR, 2009
L. Kratz, et al., CVPR, 2009
S. Ali, et al., CVPR, 2007
Dynamic texture
A. B. Chan, et al., TPAMI, 2008
Finer group motion pattern cannot be captured by holistic methods
Scale
How to characterize and compare dynamics
across so many different crowded scenes?
Scene-specific
Scene-independent
Micro-level
Macro-level
Group-level
Scene Scale
Are there any common group properties to
characterize different crowded scenes?
How to characterize and compare dynamics
across so many different crowded scenes?
Biological and human crowd systems share the
similar properties
Our Goals
Collectiveness Stability Uniformity Conflict
Low
Hig
h
Group property quantification from vision view
GasSolid Pure Fluid Impure Fluid
Our Goals
Group state analysis
Gas
Solid
Fluid
Videos will be deleted after the presentation
Our Goals
Scene-independent crowd video classification
Contributions
A robust group detector
A set of scene-independent visual descriptors to
quantify group properties
Applications
Intrinsic group state identification
Scene-independent crowd video classification
474 crowd video clips
Most comprehensive crowd dataset to date
With 215 crowd scenes
Group are exhaustively labeled
Labeled with classes based on group activities
A New Crowd Dataset
Framework
Input video Group detection
11
21
2
3
3
2 Gas
Solid
1
3
Group state analysis & Crowd scene understanding
Fluid2
Crowd video classification
11
21
2
3
3
2 Gas
Solid
1
3
Group state analysis & Crowd scene understanding
Fluid2
Crowd video classification Group descriptors
Dynamic Group Detection
Collective Transition priorreveals collective motion of all
members in a groupCoherent filtering cluster Anchor tracklet
Seeding tracklets Final groups
[1]
[1] B. Zhou, et al.. In ECCV, 2012.
√×
Dynamic Group Detection
Group descriptors
Framework
Group descriptors
11
21
2
3
3
2 Gas
Solid
1
3
Group state analysis & Crowd scene understanding
Fluid2
Crowd video classification
Input video Group detection
Descriptors
Group properties
Intra-group Inter-group
Graph-driven group quantifications
Geometric
structure
Topological
structure
Collective
Transition
prior,
Collectiveness Stability Uniformity Conflict
Characterize the degree of individuals acting
as a union
A AA
Low High
Collectiveness
4 2 3 1 4 2 3 1
Stability
Characterize whether a group keeps internal
topological structure over time
Low
1 2 45
3
6
6
1 3 45 2
4
1 235
6
4
3
6
1 25
KNN KNN
High
12
3
4
4
213
1 23
4
4
21 3
KNN KNN
6 5 4 1 2 36 2 3 4 1 5
Edit distance: 5 Edit distance: 0
Cut number5
Cut number1
Uniformity
Characterize homogeneity of a group in terms
of spatial distribution
Low
High
high
low
high
low
Conflict
Characterize interaction/friction between
groups when they approach each other
Framework
Group detectionInput video
11
21
2
3
3
2 Gas
Solid
1
3
Group state analysis & Crowd scene understanding
Fluid2
Crowd video classification
11
21
2
3
3
2 Gas
Solid
1
3
Group state analysis & Crowd scene understanding
Fluid2
Crowd video classification Group descriptors
Application I: Group State Analysis
Confusion matrix Accuracy of using each
descriptor and combining them
1
0.5
0
Gas
Solid
Pure
fluid
Impure
fluid0
0.2
0.4
0.6
0.8
Collectiveness Stability Uniformity
Conflict Combination
Application I: Group State Analysis
Application II: Crowd Video Classification
(a) Holistic features (44%) (b) Our group descriptors (70%)
Confusion matrix
[3]
[3] L. Kratz, et al.. In CVPR, 2009.
1
0.8
0
0.6
0.4
0.2
1
0.8
0
0.6
0.4
0.2
Application II: Crowd Video Classification
Conclusion
Groups can
characterize and compare dynamics across different crowded scenes
be quantified from vision point of view
Scene independent crowd analysis
group state analysis
large-scale crowd video classification
Future work
crowd video retrieval
crowd scene understanding
crowd modeling, simulation, and behavior analysis.
Any Questions?Datasets and code are available. Project page is
http://www.ee.cuhk.edu.hk/~jshao/CUHKcrowd.html