Real Time Motion CaptureUsing a Single Time-Of-Flight Camera
Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun
CVPR 2010
Q36981123 邱碁森
Outline
• Introduction• Probabilistic Model• Inference• Experiments• Conclusions
Introduction
• Motion capture is used to human-machine interaction, smart surveillance and so on.
• Time-of-flight sensors offers rich sensory information, not sensitive to changes in lighting, shadows, and some other problems.
• This paper propose an efficient filtering algorithm for tracking human pose for fast operation at video frame.
What is Probabilistic Model?
• A tree-shaped kinematic chain (skeleton)– Human body is modeled as 15 body parts– The transformations of the body Xt at time t is a
set: Xt= {Xi}, i = 1~15
– X1: the root of tree → the pelvis part• root(pelvis): could freely rotate and translate • other parts: connected to the their parent,
allow to rotate (not to translate)
What is Probabilistic Model? (cont.)
• The absolute orientation of a body part i: Wi(X)– multiplying the transformations of its ancestors in
the kinematic chain– Wi(X) = X1 X∙ 2 · ...· Xparent(i) X∙ i
Why need the Probabilistic Model?
• Determine the most likely state at at time t– the pose set Xt
– the first discrete-time derivative set Vt (velocities)– zt: the recorded range measurements
• The system is modeled as a dynamic Bayesian network (DBN)
Probabilistic Model
• The measured range scan is denoted by z = {zk} k=1
M • where zk gives the measured depth of the
pixel at coordinate k.
Probabilistic Model
• Assumption: the accelerations in our system are drawn from a Gaussian distribution with zero mean
Inference
• How to perform efficient inference at each frame?– Model Based Hill Climbing Search (HC)• A component locally optimizes the likelihood function
– Evidence Propagation (EP)• An inference procedure generate likely states which are
used to initialize the HC
Model Based Hill Climbing Search
• coarse-to-fine– The procedure can then potentially be applied to a smaller
interval about the value chosen at the coarser level
• hill-climbing– Start from the base of kinematic chain which includes the
largest body parts, and proceed toward the limbs
1
23
optimize the X axis
0.50.450.4...-0.35-0.4-0.45-0.5
sample:
then chose the best one
Evidence Propagation
• Problem: – fast motion cause motion blur– occlusion cause the estimate of the state of
hidden parts to drift– the likelihood function has ridges (difficult to
navigate)• This procedure that identifies promising
locations for body parts to find likely poses
Evidence Propagation
• Steps in this procedure:1. Body Part Detection: identify possible body part
locations from the current range image2. Probabilistic Inverse Kinematics: update the body
configuration X given possible correspondences between mesh vertices and part detections
3. Data Association and Inference: determine the best subset of such correspondences
Body Part Detection• Five body parts: head, left hand, right hand, left foot and right
foot are found from the current range image.• Interest Point(AGEX) Detection
– start on the geodesic centroid of the mesh: AGEX1(M)
– recursively find the vertex AGEXk(M) which has max geodesic distance to AGEXk-1(M)
• Identification of Parts– points are classified as body part by training these data using a
marker-based motion capture system( LED mark)
Evidence Propagation
Experiments
• Using a Swissranger SR4000 Time-of-Flight camera
• Tracking results on real-world test sequences, sorted from most complex (left) to least complex (right).
Experiments
• A Tennis sequence
Only use Model-Based search
Our combined tracker
Conclusions
• A novel algorithm for combining part detections with local hill-climbing for marker less tracking of human pose.
• With the hybrid, GPU-accelerated filtering approach