Post on 25-Feb-2016
description
transcript
Computer Vision GroupUniversity of California Berkeley
Estimating Human Body Configurations using Shape Context Matching
Greg Mori and Jitendra Malik
Computer Vision GroupUniversity of California Berkeley
Problem
Computer Vision GroupUniversity of California Berkeley
Approach: Exemplar-based Matching
• Set of stored exemplars with hand-labeled keypoints
• Obtain sample points
• Deformable matching to exemplars:– Shape context matching to get correspondences– Kinematic chain deformation model
• Estimate 3D body configuration
Computer Vision GroupUniversity of California Berkeley
Comparing Pointsets
Computer Vision GroupUniversity of California Berkeley
Shape ContextCount the number of points inside each bin, e.g.:
Count = 4
Count = 10
...
Compact representation of distribution of points relative to each point
Computer Vision GroupUniversity of California Berkeley
Shape Context
Computer Vision GroupUniversity of California Berkeley
Comparing Shape ContextsCompute matching costs using Chi Squared distance:
Recover correspondences by solving linear assignment problem with costs Cij
[Jonker & Volgenant 1987]
Computer Vision GroupUniversity of California Berkeley
Deformable Matching
• Kinematic chain-based deformation model
• Use iterations of correspondence and deformation
• Keypoints on exemplars are deformed to locations on query image
Computer Vision GroupUniversity of California Berkeley
Computer Vision GroupUniversity of California Berkeley
Problem
Computer Vision GroupUniversity of California Berkeley
Estimate 3D Body Configuration [Taylor ’00]
• Known:– Relative lengths of body segments– (x,y) Image locations of keypoints– “closer endpoint” labels for each segment– Scaled orthographic camera model
• Solve for 3D locations of keypoints up to some scale factor– Scale factor can be estimated automatically
Computer Vision GroupUniversity of California Berkeley
Solving for Foreshortening
)()()( 2121212222 ZZYYXXl
2221
221
2 /))()(( suuuuldZ
Computer Vision GroupUniversity of California Berkeley
Choosing Scale
))()(( 221
221 vvuu
ls
Computer Vision GroupUniversity of California Berkeley
Results
Computer Vision GroupUniversity of California Berkeley
Computer Vision GroupUniversity of California Berkeley
Multiple Exemplars
• Parts-based approach– Use a combination of keypoints/whole limbs from different
exemplars– Reduces the number of exemplars needed
• Compute a matching cost for each limb from every exemplar
• Compute pairwise “consistency” costs for neighbouring limbs
• Use dynamic programming to find best K configurations
Computer Vision GroupUniversity of California Berkeley
Parts-based Approach
Computer Vision GroupUniversity of California Berkeley
Tracking by Repeated Finding