Date post: | 27-Dec-2015 |
Category: |
Documents |
Upload: | silvia-harper |
View: | 213 times |
Download: | 0 times |
Isolated-Word Speech Recognition Using
Hidden Markov Models
6.962 Week 10 Presentation
Irina Medvedev
Massachusetts Institute of Technology
April 19, 2001
Outline
• Markov Processes, Chains, Models
• Isolated-Word Speech Recognition¤ Feature Analysis
¤ Unit Matching
¤ Training
¤ Recognition
• Conclusions
Markov Process
• The process, x(t), is first-order Markov if, for any set of ordered times, ,
• The current value of a Markov process depends all of the memory necessary to predict the future
• The past does not add any additional information about the future
Nttt 21
Markov Process
• The transition probability density provides an important statistical description of a Markov process and is defined as
• A complete specification of a Markov process consists of
• first-order density :
• transition density :
tszxp sxtx ,|)(|)(
zxp sxtx |)(|)(
xp tx )(
Markov Chains
1S
2S 3S
12a
21a
23a
32a
13a31a
11a
33a22a
Fully Connected Markov Model
• A Markov chain can be used to describe a system which, at any time, belongs to one of N distinct states,
• At regularly spaced times, the system may stay in the same state or transition to a different state
• State at time t is denoted by qt
• The state transition probabilities are made according to a set of probabilities associated with each state
• These probabilities are stored in the state transition matrix
where N is the number of states in the Markov chain.
• The state transition probabilities are
and have the properties and
Markov Chains
NNNN
N
N
aaa
aaa
aaa
11
22221
11211
A
Hidden Markov Models
• Hidden Markov Models (HMMs) are used when the states are not observable events.
• Instead, the observation is a probabilistic function of the state rather than the state itself
• The states are described by a probability model
• The HMM is a doubly embedded stochastic process
HMM Example: Coin Toss
H H T H T How do we build an HMM to explain the observed sequence of head and tails?
Choose a 2-state model
Several possibilities exist
1-coin Model: Observable
H T
)H(P )H(1 P
)H(1 P
)H(P1 2
11a 22a
111 a
221 a1
1
1)T(
)H(
PP
PP
2
2
1)T(
)H(
PP
PP
2-coin Model: States are Hidden
Hidden Markov Models
Hidden Markov Models are characterized by
• N, the number of states in the model
• A, the state transition matrix
• Observation probability distribution state
• Initial state distribution,
Model Parameter Set:
jbBj ,
Left-Right HMM
1S 2S 3S 4S
11a 22a 33a 44a
23a 34a12a
44
3433
2322
1211
000
00
00
00
a
aa
aa
aa
A
• Can only transition to a higher state or stay in the same state
• No-skip constraint allows states to transition only to the next state or remain the same state
• Zeros in the state transition matrix represent illegal state transitions
4-state left-right HMM with no skip transitions
Isolated-Word Speech Recognition
• Recognize one word at a time
• Assume incoming signal is of the form:
silence – speech – silence
• Feature Analysis • Training
• Unit Matching • Recognition
Feature Analysis
• We perform feature analysis to extract observation vectors upon which all processing will be performed
• The discrete-time speech signal is with discrete Fourier transform
• To reduce the dimensionality of the V-dim speech vector, we use cepstral coefficients, which serve as the feature observation vector for all future processing
TVyyy 110 y
TKYYY 110 Y
Cepstral Coefficients
• Feature vectors are cepstral coefficients obtained from the sampled speech vector
where is the periodogram estimate of the power spectral density of the speech
• We eliminate the zeroth component and keep cepstral coefficients 1 through L-1
• Dimensionality reduction = LV
Properties of Cepstral Coefficients
• Serve to undo the convolution between the pitch and the vocal tract
• High-order cepstral components carry speaker dependent pitch information, which is not relevant for speech recognition
• Cepstral coefficients are well approximated by a Gaussian probability density function (pdf)
• Correlation values of cepstral coefficients are very low
Modeling of Cepstral Coefficients
• HMM assumes that the Markovian states generate the cepstral vectors
• Each state represents a Gaussian source with mean vector and covariance matrix
• Each feature vector of cepstral coefficients can be modeled as a sample vector of an L-dim Gaussian random vector with mean vector and diagonal covariance matrix
Formulation of the Feature Vectors
Title:/afs/athena.mit.edu/user/i/r/irinam/SPEECH/cepstral1.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Unit Matching
• Initial Goal: obtain an HMM for each speech recognition unit
• Large vocabulary (300 words): recognition units are phonemes
• Small-vocabulary (10 words): recognition units are words
We will consider an isolated-word speech recognition system for a small vocabulary of M words
Notation
• Observation vector is , where each is a cepstral feature vector and is the number of feature vectors in an observation
• State Sequence is , where each
• State index
• Word index
• Time index
• The term model will be used for both the HMM and the parameter set describing the HMM,
Training
• We need to obtain an HMM for each of the M words
• The process of building the HMMs is called training
• Each HMM is characterized by the number of states, N, and the model parameter set,
• Each cepstral feature vector, , in state, , can be modeled by an L-dim Gaussian pdf
where is the mean vector and is the covariance matrix in state
Training
• A Gaussian pdf is completely characterized by the mean vector and covariance matrix
• The model parameter set can be modified to
• The training procedure is the same for each word. For convenience, we will drop the subscript from
Building the HMM
• To build the HMM, we need to determine the parameter set that maximizes the likelihood of the observation for that word.
• Objective:
• The double maximization can be performed by optimizing over the state sequence and the model individually
Uniform Segmentation
50 segments 8 states
7 7 6 6 6 6 6 6
1S 2S 3S 4S 5S 6S 7S 8S
Determining the initial state sequence
• Given the initial state sequence, we maximize over the model
• The maximization entails estimating the model parameters from the observation given the state sequence
• Estimation is performed using the Baum-Welch re-estimation formulas
Maximization over the Model
Re-estimation Formulas
Initial state distribution
Covariance matrix per state
Mean vector per state
State transition matrix
it Sqt
Titit
ii diagN :
ˆˆ1ˆ xx
Ni 1
is the number of feature vectors in state
Model Estimation
1S2S3S4S5S6S7S8S9S
10S
1t 2t 3t 4t 5t 6t 7t 8t 9t 10t 11t 12t 13t 14t
100000
0000
010000
0000
0000
0000
31
32
21
21
31
32
31
32
A
• Given the model, we maximize over the state sequence
• The probability expression can be rewritten as
Maximization over the state sequence
• Applying the logarithm transforms the maximization of a product into a maximization of a sum
• We are still looking for the state sequence that maximizes the expression
• The optimal state sequence can be determined using the Viterbi algorithm
Maximization over the state sequence
Trellis Structure of HMMs
1S2S3S4S5S
1t 2t 3t 4t 5t
• Redrawing the HMM as a trellis makes it easy to see the state sequence as a path through the trellis
• The optimal state sequence is determined by the Viterbi algorithm as the single best path that maximizes
Training Procedure
CepstralCalculation
Uniform Segmentation
Estimation of
(Baum-Welch)
πA ,,, ii
)(ny x
State Sequence
Segmentation (Viterbi)
Converged?No
Yes
qx |,max P
1|,max qxqP
Recognition
• We have a set of HMMs, one for each word
• Objective: Choose the word model that maximizes the probability of the observation given the model (Maximum Likelihood detection rule)
• Classifier for observation is
• The likelihood can be written as a summation over all state sequences
Recognition
• Replace the full likelihood by an approximation that takes into account only the most probable state sequence capable of producing the observation
• Treating the most probable state sequence as the best path in the HMM trellis allows us to use the Viterbi algorithm to maximize the above probability
• The best-path classifier for observation is
Recognition
CepstralCalculation
)(ny x SelectMaximum
1| xP
2| xP
NP |x
Index of recognized
word
Conclusion
• Introduced hidden Markov models
• Described process of isolated-word speech recognition
¤ Feature vectors ¤ Unit matching
¤ Unit matching Training ¤ Recognition
• Other considerations
¤ Artificial Neural Networks (ANNs) for speech recognition
¤ Hybrid HMM/ANN models
¤ Minimum classification error HMM design