Robust Video Surveillance for Fall Detection Based on Human Shape Deformation
Caroline Rougier, Jean Meunier, Alain St-Arnaud, and Jacqueline RousseauIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 5, MAY 2011
outline
Introduction Our System and data set Falls Characteristics
Shape deformation▪ mean matching cost▪ full Procrustes distance
Fall Detection Using GMM Experimental Results Conclusion
Introduction (1/2) Establish new healthcare systems to
ensure the safety of elderly people at home. Falls are one of the major risks for old
people living alone. Fall detection wearable sensor:
Accelerometers or help buttonsProblem:-forget to wear-unconscious after the fall-recharged regularly
Introduction (2/2) Computer vision systems has overcome
these problems. A camera provides a vast amount of
information on his/her environment▪ Monocular Systems▪ Bounding box[8]▪ Only placed sideways▪ Occluding objects
▪ Multi-Camera Systems▪ Auvinet et al.[17] reconstructed 3-D silhouette of an
elderly person ▪ Need to be calibrated▪ The video sequences need to be synchronize
Our System and data set (1/2)
Uncalibrated multi-camera system
Low-cost IP cameras, 30 frames/s, 720 × 480 pixels
Wide angle to cover all the room
Our System
and data set
Total of 75 different events , more than 12 min
Falls Characteristics
1. Lack of significative movement2. A lying position3. A person lying on the ground4. Vertical speed5. An impact shock6. Body shape change
Silhouette Edge Point Extraction The silhouette is extracted by a background
subtraction
N = 250 landmarks * Canny edge detector[25]
Matching Using Shape Context (1/2) Shape context[20] is a way of
describing shapes.
Matching cost for pair (pi, qj):
, K=5*12 bins
Matching Using Shape Context (2/2) Minimizing the total matching cost given a
permutation π (i)
Use the Hungarian algorithm[27] for bipartite matching Time complexity: O(n^3) Bad landmarks due to segmentation errors or partial occlusions ▪ Add dummy points (not easy to choose).▪ Match only the most reliable points in our implement (mini Cij = minj Cij)
mean matching cost: i j
bipartite graph
N∗: the total number of best matching points.
Procrustes analysis
Procrustes analysis [21] has been widely used to compare shapes. Detect abnormal shape deformation for fall
detection▪ Step1 : image registration(one translation, no rotation,
no scaling)▪ Step2: Compute full Procrustes distance for compare.
centered landmarks Zc :
1
11
kl
Z
Zc
two centered vectors : v = (v1, · · · , vk)w = (w1, · · ·,wk).
full Procrustes distance :
Fall feature
mean matching cost full Procrustes distance Consider 2 feature (F1, F2)
CfD
1) F1 representing the fall : F1 will high in case of fall
2) F2 representing the lack of significative movement after the fall : A period (t+1s to 5s) will low
Fall Detection Using GMM Model normal activity data with a Gaussian Mixture
Model(GMM). GMM: weighted sum of Gaussian(normal) distributions
M : the number of components in the mixture P (j) : the mixing coefficients The jth Gaussian probability density function p (x | j)
▪ d is the dimensionality of the input space
expectation-maximization (EM)algorithm by maximizing the data likelihood
GMM Classifier : only tell normal or abnormal!
Training and test the dataset Leave-One-Out Cross-Validation
1. Divided the dataset into N video sequences2. One sequence is removed3. Training using the N − 1 remaining sequences
(falls are deleted)4. This sequence is classified with the resulting
GMM.5. Repeat N times6. Count the number of errors, classification
error rate
GMM Classifier Analysis
1. True Positives (TP): falls correctly detected;2. False Negatives (FN): falls not detected;3. False Positives (FP): normal activities detected
as a fall;4. True Negatives (TN): normal activities not
detected as a fall;5. Sensitivity: Se = TP/ (TP + FN);6. Specificity: Sp = TN/ (TN + FP);7. Accuracy: Ac = (TP+TN) / (TP+TN+FP+FN) ;8. Classification error rate: Er = (FN+FP) /
(TP+TN+FP+FN) .
Experimental Results Shape matching : C++ using the OpenCV
library [33] Fall detection : MATLAB using the NETLAB
toolbox [32] to perform the GMM classification.
The original video sequences frame : 30 frames/s 5 frames/s was sufficient to detect a fall Intel Core 2 Duo processor (2.4 GHz) The computational time of the shape matching
step is about 200 ms
Number of GMM Components
train a GMM with three components for our experiment.
Classification Results Normalize training data.
Detection threshold depends on the sensitivity.
Receiver operating characteristic (ROC) analysis
false positives
true positives
Ensemble Classifier
Simply majority vote on all cameras (>= 3 vote) In fig. 9 : error rate 10%2.7%
Comparative Study with Other 2-D Features (1/2)
Comparative Study with Other 2-D Features (2/2)
Occlusions and Other Difficulties
Conclusion
We presented a new GMM classification method to detect falls By analyzing human shape deformation
Robust to large occlusions and other segmentation difficulties
