Hidden Markov ModelNov 11, 2008

Sung-Bae Cho

Hidden Markov Model

Inference of Hidden Markov Model

Path Tracking of HMM

Learning of Hidden Markov Model

Hidden Markov Model Applications

Summary & ReviewAgenda

Temporal Pattern RecognitionThe world is constantly changing.

Temporal data sequence = , X2, X1, X0, X1, X2,

Observed vs. real valueReal value: XObservation: Y*~

Hidden Concept and Actual Realizationrealityidea*

Hidden Markov ModelDefinition: A statistical model in which the system being modeled is assumed to be a Markov process with unknown parameterschallenge is to determine the hidden parameters from the observable parametersExtracted model parameters can be used to perform further analysisExpression:

A hidden random variable Xt that conditions another random variable Yt Xt S = {1, 2, , N}*

Random ProcessesXt1 Xt or Xt | Xt1 => Markov processDescription: {P(Xt|Xt1)}

Yt | Xt => Random process (often a Gaussian process)Description: {P(Yt|Xt)}

Combination: {P(Xt|Xt1)P(Yt|Xt)}Doubly stochastic process*

Why HMM?A good model for highly variable discrete-time sequenceoften noisy, uncertain and incomplete

Generalization of DTW matching template

Rigorous and theoretical foundationthe model can be optimized

Models spatiotemporal variabilities elegantlygreater variability-modeling power than M-Chain

Efficient inference/computation algorithms

Theoretical and robust learning algorithm

Can be combined to model complex patterns (composition and extension)

*

What is an HMM - NotationThree sets of parameters = (, A, B)Initial state probabilities : = {i : i = Pr(X1= i)}constraints:

Transition probabilities A: A = {aij : aij = Pr(Xt+1= j|Xt= i)}constraints:

Observation probabilities B: B = {bj(v) : bj(v) = Pr(ot= v|Xt= j)}constraints:*

Model ParametersState space/alphabet:Matrices:

S = { 1, 2, 3 }, N = 3V = { 1, 2, 3, 4 }N : num. of hidden stateQ : state set M : num. of observation symbolS : observation symbol set A : transition probabilitiesB : observation probabilities : initial state probabilities : HMM model=(A, B, )

*

Markov Model RuleObservation sequence

Chain rule

Markov assumptionObservation oi is affected by only observation oi-1

Markov chain rule *

Hidden Markov Model

Inference of Hidden Markov Model

Path Tracking of HMM

Learning of Hidden Markov Model

Hidden Markov Model Applications

Summary & ReviewAgenda

Three Basic ProblemsEvaluation (Estimation) Problemgiven an HMM given an observationcompute the probability of the observation Solution: Forward Algorithm, Backward AlgorithmDecoding Problemgiven an HMMgiven an observation compute the most likely state sequencei.e.

Solution: Viterbi AlgorithmLearning / optimization problemgiven an HMM given an observationfind an HMM such that Solution: Baum-Welch Algorithm

The Evaluation ProblemWe know :

=

From this :

=

Obvious:for sufficiently large values of T, it is infeasible to compute the above term for all possible state sequences need other solution

The Forward AlgorithmAt time t and state i, probability of partial observation sequence : array

As a result at the last time T

Forward AlgorithmDefinition

AlgorithmInitialization

Induction

End condition

*

Backward AlgorithmDefinition

AlgorithmInitialization

Induction

End condition

*

Hidden Markov Model

Inference of Hidden Markov Model

Path Tracking of HMM

Learning of Hidden Markov Model

Hidden Markov Model Applications

Summary & ReviewAgenda

The Decoding ProblemFinding the optimal state sequence associated with the given observation sequence

Forward-BackwardOptimality criterion : to choose the states that are individually most likely at each time t

The probability of being in state i at time t

: accounts for partial observation sequence : account for remainder

Viterbi AlgorithmSolution to model decoding problemGiven Y = O = o1 o2 o3 oT,What is the best among all possible state sequences that might have produced O?The best? Be evaluated in probabilistic terms1. A sequence of the most likely states at each time? (Greedy fashion)2. The most likely complete state sequence (from any one of start states to any one final states): P(X, O|)*

Viterbi Path is the path whose joint probability with the observation is the most likely:N T possible paths of X O(TN T) multiplications with exhaustive enumeration Simplistic rewriting:(Let X = X1,T = x1 x2 xT )*

Viterbi Path LikelihoodPartial Viterbi path likelihood: (for X1,t, tT)

Back pointer to the prev best state*

Viterbi Algorithm123statesInitialization

Recursion

Termination

Backtracing

*

Viterbi Algorithm: ExampleViterbi trellis constructionP(O, X*|) = Pr(RRGB, X= 1123|) = 0.01008*

Hidden Markov Model

Inference of Hidden Markov Model

Path Tracking of HMM

Learning of Hidden Markov Model

Hidden Markov Model Applications

Summary & ReviewAgenda

The Learning / Optimization problemHow do we adjust the model parameters to maximize ??

Parameter EstimationBaum-Welch Algorithm ( EM : Expectation Maximization )Iterative Procedure

Parameter EstimationProbability of being in state i at time t, and state j at time t+1

Probability of being in state i at time t, given the entire observation sequence and the model

We can relate these by summing over j

Parameter Estimation (3)By summing over time index t expected number of times that state i visitedexpected number of transitions made from state i

That is = expected number of times that state i in O

= expected number of transitions made from state i to j in O

Update using & : expected frequency (number of times) in state i at time (t=1)

Parameter Estimation (5)New Transition Probability

expected number of transitions from state i to j

expected number of transitions from state I

=

Parameter Estimation (6)New Observation Probability

expected number of times in state j and observing symbol

expected number of times in j

=

Parameter Estimation (7)From , if we define new

New model is more likely than old model in the sense that

The observation sequence is more likely to be produced by new modelhas been proved by Baum & his colleaguesiteratively use new model in place of old model, and repeat the reestimation calculation ML estimation

Baum-Welch Algorithm (1)Definition

Calculation

Definition

Calculation

*

Baum-Welch Algorithm (2)AlgorithmSet initial model (0)Estimate next model Calculate: ,

Maximization : finding

If P(O|)-P(O|0) < threshold then stop

Else = 0, move to 2 (repetition)

*

Classification AlgorithmClassification

Viterbi algorithmDomain/linguistic knowledgeMarkov source model for character probabilityP(W) = P(w1 w2 wn) = P(w1) P(w2|w1) P(wn|wn-1)P(123) = P(1) P(2|1) P(3| 2)*

Hidden Markov Model

Inference of Hidden Markov Model

Path Tracking of HMM

Learning of Hidden Markov Model

Hidden Markov Model Applications

Summary & ReviewAgenda

University of AlbertaNational ICT Australia project, University of Alberta, Canada

Object Motion/gesture recognition of human

sensorsactive, magnetic field, acoustic, laser, camera sensor

methodCoupled hidden Markov model (CHMM)Coupled HMMs provide an efficient way to resolve many complex problems, and offer superior training speeds, model likelihoods, and robustness to initial conditions.Proposed by M. Brand (1997)[M. Brand, N. Oliver, and A. Pentland, Coupled Hidden Markov Models for complex action recognition, in IEEE Intl. Conf. Comp. Vis. Pat. Rec., 1997, pp. 994.999.]*

University of BolognaMicrel, University of Bologna, Lab Italy, (2004)

Research Setup ubiquitous environmentsSensory data processingGesture recognition

SensorsDevelop: Wireless MOCA (Motion capture with integrated accelerometers)Accelerometer, gyroscopeSmall size, small consumption, wirelessWearing on body

Recognition methodHidden Markov Model

*

MIT Media LABMedia Laboratory, Massachusetts Institute of TechnologyArea: Visual Contextual Awareness in Wearable Computing (1998)Sensor: Vision

MethodProbabilistic object recognitionBased on observed diverse feature vectorUsing probabilistic relations (O: object, M: measurement)

Task recognition with HMM

*

eWatch Sensor PlatformCMU Computer Science Lab, 2005Activity Recognition + improving power consumption

HardwareLCD, LED, vibration motor, speaker, Bluetooth for wireless communicationLi-Ion battery with a capacity of 700mAhSensorsa two-axis accelerometer (ADXL202; +/- 2g)Microphone, light & temperature sensorsMethodmulti-class SVMs + HMM based Selective Sampling

*

SummaryHidden Markov Model introduction

HMM inference method (estimation)

HMM path tracking (decoding)

HMM learning

HMM application

**

************************************