Date post: | 19-Dec-2015 |
Category: |
Documents |
View: | 224 times |
Download: | 5 times |
Recognition of Human Recognition of Human GaitsGaits
Alessandro BissaccoAlessandro BissaccoComputer ScienceComputer Science
UCLAUCLA
Yi MaYi MaElectrical and Computer Electrical and Computer
EngineeringEngineeringUniversity of Illinois at Urbana-University of Illinois at Urbana-
ChampaignChampaign
Alessandro ChiusoAlessandro ChiusoDipartimento di Elettronica e Dipartimento di Elettronica e
InformaticaInformaticaUniversita di PadovaUniversita di Padova
Stefano SoattoStefano SoattoComputer ScienceComputer Science
UCLAUCLA
Presented by Peter Schwer
SummarySummary
Problem DescriptionProblem Description Related WorkRelated Work Overview of Gait Recognition ProcessOverview of Gait Recognition Process
TrackingTracking Learning the ModelLearning the Model RecognitionRecognition
ResultsResults
The Human GaitThe Human Gait
Observable from great distancesObservable from great distances Discernable independent of skew (relative Discernable independent of skew (relative
direction of subject’s gait)direction of subject’s gait) Communicates:Communicates:
IdentityIdentity MoodMood
Good Recognition TraitsGood Recognition Traits
Invariant to:Invariant to: Photometric Factors:Photometric Factors:
Illumination, clothingIllumination, clothing Geometric Factors:Geometric Factors:
Camera PositionCamera PositionLength of subject’s limbsLength of subject’s limbsRelative direction of gaitRelative direction of gait
Good Recognition TraitsGood Recognition Traits
Capable of discerning:Capable of discerning: Different types of gaits:Different types of gaits:
RunningRunningWalkingWalkingSkipping, etc.Skipping, etc.
MoodMoodNot addressed, but dynamical models Not addressed, but dynamical models
can be extended to this area.can be extended to this area.
Related WorkRelated Work
Analyzing gait with spatiotemporal surfaces Analyzing gait with spatiotemporal surfaces (1994) Sourabh A. Niyogi, Edward H. (1994) Sourabh A. Niyogi, Edward H. AdelsonAdelson
University of SouthamptonCarnegie Mellon University (CMU)Massachusetts Insitute of Technology (MIT)Georgia Tech (GATech)University of MarylandUniversity of South Florida
The Recognition ProblemThe Recognition Problem
The Sub-problems:The Sub-problems:TrackingTrackingLearning the modelLearning the modelRecognizingRecognizing
TrackingTracking
““The emphasis of our work is The emphasis of our work is notnot tracking.” tracking.” Images Images → Skeletons→ Skeletons
Results:Results:
y(t), t = 1 … y(t), t = 1 … ττ
(Sequence of joint positions)(Sequence of joint positions)
Learning the ModelLearning the Model
The Right ModelThe Right Model
Ideally,Ideally, Unique model for each datasetUnique model for each dataset Method for calculating similarity (distance) between Method for calculating similarity (distance) between
modelsmodels
Catch:Catch: ““The set of dynamical models in canonical form is The set of dynamical models in canonical form is notnot
a linear space (even if the model itself is linear!)”a linear space (even if the model itself is linear!)” Distances between dynamical models are not trivially Distances between dynamical models are not trivially
computed.computed. Data, as usual, is inherently noisy.Data, as usual, is inherently noisy.
Creating the Dynamical ModelCreating the Dynamical Model
Input: Input: y(t)y(t) – Sequence of Joint Positions – Sequence of Joint Positions What are some of the statistical properties of What are some of the statistical properties of
this sequence of joint positions?this sequence of joint positions? Stochastic Stationary Second-order processStochastic Stationary Second-order process How is noise distributed in the data?How is noise distributed in the data?
We use a Gauss-Markov Model as a We use a Gauss-Markov Model as a representative of each class.representative of each class.
The ARMA ModelThe ARMA Modelx(t+1) = Ax(t) + v(t), v(t) ~ N(0,Q); x(0) = x0x(t+1) = Ax(t) + v(t), v(t) ~ N(0,Q); x(0) = x0y(t) = C(x(t) + w(t); w(t) ~ N(0,R)y(t) = C(x(t) + w(t); w(t) ~ N(0,R)
The ARMA ModelThe ARMA Model
Other Applications:Other Applications: Speech RecognitionSpeech Recognition TrackingTracking Geophysics: Forecasting and SimulationGeophysics: Forecasting and Simulation Pattern RecognitionPattern Recognition Dynamic Texture RecognitionDynamic Texture Recognition
The ARMA modelThe ARMA model
x(t) is a process describing the “state” at time t (vector)x(t) is a process describing the “state” at time t (vector)y(t) is the measured joint positions (vector)y(t) is the measured joint positions (vector)Q,R are input and output noise covariances (matrices)Q,R are input and output noise covariances (matrices)
x(t+1) = Ax(t) + v(t), v(t) ~ x(t+1) = Ax(t) + v(t), v(t) ~ NN(0,Q); x(0) = x(0,Q); x(0) = x00
y(t) = C(x(t) + w(t); w(t) ~ y(t) = C(x(t) + w(t); w(t) ~ NN(0,R)(0,R)
((AA:[n :[n ×× n], n], B B:[m :[m × n])× n])
PROBLEM: Non-uniqueness of A, C, Q, R, S
Innovation/Canonical Innovation/Canonical RepresentationRepresentation
Identify stochastic subspace models Identify stochastic subspace models without forming the covariance matrix.without forming the covariance matrix.
Principal Angles and DirectionsPrincipal Angles and Directions Source:Source:
P. Van Overschee and B. De MoorP. Van Overschee and B. De Moor
Subspace Algorithms for the stochastic Subspace Algorithms for the stochastic identification problem. identification problem. AutomaticaAutomatica, 29:649-, 29:649-660, 1993.660, 1993.
A Canonical RepresentationA Canonical Representation
x(t) is a process describing the “state” (vector)x(t) is a process describing the “state” (vector)y(t) is the measured joint positions (vector)y(t) is the measured joint positions (vector)Q,R are input and output noise covariances (matrices)Q,R are input and output noise covariances (matrices)
x(t+1) = Ax(t) + v(t), v(t) ~ x(t+1) = Ax(t) + v(t), v(t) ~ NN(0,Q); x(0) = x(0,Q); x(0) = x00
y(t) = C(x(t) + w(t); w(t) ~ y(t) = C(x(t) + w(t); w(t) ~ NN(0,R)(0,R)
From Overschee and De Moor we have {A,C,Q,R}From Overschee and De Moor we have {A,C,Q,R}
A and C describe the gait.A and C describe the gait.
RecognizingRecognizing
Our Dynamic ModelOur Dynamic Model
A, C, Q, R providedA, C, Q, R provided What are they?What are they? How might they be useful?How might they be useful?
How to Describe GaitHow to Describe Gait
Tracking produces a “waveform” that describes Tracking produces a “waveform” that describes a series of joint positions/angles:a series of joint positions/angles:
Gait 1
Gait 2
What is the best way to compare
these to waveforms?
Distance Between ModelsDistance Between Models
Subspace/principal anglesSubspace/principal angles Start with:Start with: O(M)- infinite observability matrix O(M)- infinite observability matrix
0 ),,...,,(
:tuple-n by the denoted are and subspaces thebetween angles principal The
)(cos),...,(cos :are
matrix theof aluessingular v ordered-n The
ization)orthogonalSchmidt -Gramby (Obtain
. as subspace same thespans that basis lorthonorma thebe Let
subspace. ldimensiona- an spanning columns hmatrix wita be Let
)(
12121
21
21
2
2
21
iin
n
nnH
TH
H
H
nTTTTT
HH
HH
Q
HQ
nH
CACACM
O
Form Form O(m)O(m)
O(m)O(m) is a subspace is a subspace We want to compare the observability We want to compare the observability
subspaces of two modelssubspaces of two models MM11 and and MM22..
Form Form O(m)O(m) for each model: for each model:
nTTTTT CACACM 2)(O
Call the results Call the results HH11 and and HH22, , respectively.respectively.
Form Orthonormal MatricesForm Orthonormal Matrices
Gram-Schmidt Orthogonalization on Gram-Schmidt Orthogonalization on HH11 and and HH22 Described: Described:
http://lagrange.la.asu.edu/VirtualClass/Algebra/Gramhttp://lagrange.la.asu.edu/VirtualClass/Algebra/GramSchm.htmlSchm.html
Applet: Applet: http://http://www.mste.uiuc.edu/exner/ncsa/orthogonal/#simulatiowww.mste.uiuc.edu/exner/ncsa/orthogonal/#simulationn
Call the result orthonormal matrices Call the result orthonormal matrices QQH1H1 and and QQH2H2
Calculate Principal AnglesCalculate Principal Angles
Compute n-ordered Singular Values of Compute n-ordered Singular Values of QQTT
H1H1 QQH2H2
n-ordered singular values can be denoted: n-ordered singular values can be denoted: coscos22((θθ11), … , ), … , coscos22((θθnn))
Principal Angles between the subspaces Principal Angles between the subspaces HH11 and and HH22 are denoted by the n-tuple: are denoted by the n-tuple: ((θθ1 1 , , θθ22 ,…, ,…, θθn n ) sorted in descending order.) sorted in descending order.
Calculate the DistanceCalculate the Distance First measure is not necessarily non-negative (less fit)First measure is not necessarily non-negative (less fit) Second Measure is the largest principal angle (more Second Measure is the largest principal angle (more
fit)fit)
1
222
:Norm) (
)(cosln :Norm) (
F
iiM
dL
or
dL
The Martin and Finsler Distance, respectively.The Martin and Finsler Distance, respectively.
OrOr
Just use their matlab code:Just use their matlab code:function theta = subspace_angles(A1,K1,C1,A2,K2,C2)n = size(A1,1);m = size(C1,1);A = [ A1 zeros(n,3*n); zeros(n) A2-K2*C2 zeros(n,2*n);zeros(n,2*n) A2 zeros(n); zeros(n,3*n) A1-K1*C1];C = [ C1 -C2 C2 -C1 ];Q = dlyap(A’,C’*C);E = eig([zeros(2*n) pinv(Q(1:2*n,1:2*n))*Q(1:2*n,2*n+1:4*n);pinv(Q(2*n+1:4*n,2*n+1:4*n))*Q(2*n+1:4*n,1:2*n)zeros(2*n)]);E = max(-ones(size(E)),E);E = min(ones(size(E)),E);theta = acos(E(1:2*n));
ResultsResults
Example GaitsExample Gaits
Name that GaitName that GaitPensive and determined.
Obligatory Gait Video 2Obligatory Gait Video 2Not in a rush.
Obligatory Gait Video 3Obligatory Gait Video 3Sad walking.
Obligatory Gait Video 4Obligatory Gait Video 4There is no stopping me.
ResultsResults“The pairwise distance between each sequence in the dataset is displayed in this plot. Each row/column of a matrix represents a sequence, and sequences corresponding to similar gaits are grouped in block rows/columns. Dark indicates a small distance, light a large distance. The minimum distance is of course along the diagonal, and for each column the next closest sequence is indicated by a circle, while the second nearest is indicated by a cross.”
ResultsResults How accurate are How accurate are
the results?the results? How does this How does this
compare with compare with human ability?human ability?
Questions?Questions?
From Discussion BoardFrom Discussion Board
Diem: How are the ARMA model and Kalman Diem: How are the ARMA model and Kalman filters related?filters related?
Mei-fang: Is there any method to find the best Mei-fang: Is there any method to find the best size training set, and the best k for classifying size training set, and the best k for classifying the new sample?the new sample?
Matt Clothier: What is the advantage of the Matt Clothier: What is the advantage of the Martin distance over the Finsler distance?Martin distance over the Finsler distance?
Neil and Mike: What do they mean by “Transient Neil and Mike: What do they mean by “Transient Actions” and why does the previous assumption Actions” and why does the previous assumption not hold for them?:not hold for them?: