CSCE 478/878 Lecture 9: Hidden Markov...

transcript

CSCE478/878

Lecture 9:HiddenMarkovModels

Stephen Scott

Introduction

Outline

MarkovChains

HiddenMarkovModels

CSCE 478/878 Lecture 9:Hidden Markov Models

Stephen Scott

sscott@cse.unl.edu

1 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

HiddenMarkovModels

Introduction

Useful for modeling/making predictions on sequentialdataE.g., biological sequences, text, series ofsounds/spoken wordsWill return to graphical models that are generative

2 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

HiddenMarkovModels

Outline

Markov chainsHidden Markov models (HMMs)

Formal definitionFinding most probable state path (Viterbi algorithm)Forward and backward algorithms

Specifying an HMM

3 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChainsThe Markov Property

Begin and EndStates

Discrimination

HiddenMarkovModels

Markov ChainsBioinformatics Example: CpG Islands

Focus on nucleotide sequences: Sequences ofsymbols from alphabet {A, C, G, T}The sequence “CG” (written “CpG”) tends to appearmore frequently in some places than in othersSuch CpG islands are usually 102–103 bases longQuestions:

1 Given a short segment, is it from a CpG island?2 Given a long segment, where are its islands?

4 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

Begin and EndStates

Discrimination

HiddenMarkovModels

Markov ChainsModeling CpG Islands

Model will be a CpG generatorWant probability of next symbol to depend on currentsymbolWill use a standard (non-hidden) Markov model

Probabilistic state machineEach state emits a symbol

5 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

Begin and EndStates

Discrimination

HiddenMarkovModels

Markov ChainsModeling CpG Islands (cont’d)

P(A | T)

6 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

Begin and EndStates

Discrimination

HiddenMarkovModels

Markov ChainsThe Markov Property

A first-order Markov model (what we study) has theproperty that observing symbol xi while in state πi

depends only on the previous state πi−1 (whichgenerated xi−1)Standard model has 1-1 correspondence betweensymbols and states, thus

P(xi | xi−1, . . . , x1) = P(xi | xi−1)

P(x1, . . . , xL) = P(x1)

L∏i=2

P(xi | xi−1)

7 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

Begin and EndStates

Discrimination

HiddenMarkovModels

Markov ChainsBegin and End States

For convenience, can add special “begin” (B) and “end”(E) states to clarify equations and define a distributionover sequence lengthsEmit empty (null) symbols x0 and xL+1 to mark ends ofsequence

P(x1, . . . , xL) =L+1∏i=1

P(xi | xi−1)

Will represent both with single state named 0

8 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

Begin and EndStates

Discrimination

HiddenMarkovModels

Markov ChainsMarkov Chains for Discrimination

How do we use this to differentiate islands fromnon-islands?Define two Markov models: islands (“+”) andnon-islands (“−”)

Each model gets 4 states (A, C, G, T)Take training set of known islands and non-islandsLet c+st = number of times symbol t followed symbol s inan island:

P̂+(t | s) = c+st∑t′ c+st′

Now score a sequence X = 〈x1, . . . , xL〉 by summing thelog-odds ratios:

(P̂(X | +)

P̂(X | −)

L+1∑i=1

(P̂+(xi | xi−1)

P̂−(xi | xi−1)

)9 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

HiddenMarkovModelsExample

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov Models

Second CpG question: Given a long sequence, where areits islands?

Could use tools just presented by passing a fixed-widthwindow over the sequence and computing scoresTrouble if islands’ lengths varyPrefer single, unified model for islands vs. non-islands

A+ C T G+ + +

A C T G- - - -

between all pairs][complete connectivity

Within the + group, transition probabilities similar tothose for the separate + model, but there is a smallchance of switching to a state in the − group

10 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsWhat’s Hidden?

No longer have one-to-one correspondence betweenstates and emitted characters

E.g., was C emitted by C+ or C−?Must differentiate the symbol sequence X from thestate sequence π = 〈π1, . . . , πL〉

State transition probabilities same as before:P(πi = ` | πi−1 = j) (i.e., P(` | j))Now each state has a prob. of emitting any value:P(xi = x | πi = j) (i.e., P(x | j))

11 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsWhat’s Hidden? (cont’d)

[In CpG HMM, emission probs discrete and = 0 or 1]

12 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsExample: The Occasionally Dishonest Casino

Assume casino is typically fair, but with prob. 0.05 itswitches to loaded die, and switches back with prob. 0.1

Example: The Occasionally Dishonest Casino

• Assume that a casino is typically fair, but with probability 0.05 it switchesto a loaded die, and switches back with probability 0.1

1: 1/62: 1/63: 1/64: 1/65: 1/66: 1/6

LoadedFair1: 1/102: 1/103: 1/104: 1/105: 1/106: 1/2

0.90.95

• Given a sequence of rolls, what’s hidden?13

Given a sequence of rolls, what’s hidden?13 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsThe Viterbi Algorithm

Probability of seeing symbol sequence X and statesequence π is

P(X, π) = P(π1 | 0)L∏

P(xi | πi)P(πi+1 | πi)

Can use this to find most likely path:

π∗ = argmaxπ

P(X, π)

and trace it to identify islands (paths through “+” states)There are an exponential number of paths throughchain, so how do we find the most likely one?

14 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsThe Viterbi Algorithm (cont’d)

Assume that we know (for all k) vk(i) = probability ofmost likely path ending in state k with observation xi

v`(i + 1) = P(xi+1 | `) maxk{vk(i)P(` | k)}

All states at

State at

15 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsThe Viterbi Algorithm (cont’d)

Given the formula, can fill in table with dynamicprogramming:

v0(0) = 1, vk(0) = 0 for k > 0For i = 1 to L; for ` = 1 to M (# states)

v`(i) = P(xi | `)maxk{vk(i− 1)P(` | k)}ptri(`) = argmaxk{vk(i− 1)P(` | k)}

P(X, π∗) = maxk{vk(L)P(0 | k)}π∗L = argmaxk{vk(L)P(0 | k)}For i = L to 1

π∗i−1 = ptri(π

∗i )

To avoid underflow, use log(v`(i)) and add

16 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsThe Forward Algorithm

Given a sequence X, find P(X) =∑

π P(X, π)

Use dynamic programming like Viterbi, replacing max withsum, and vk(i) with fk(i) = P(x1, . . . , xi, πi = k) (= prob. ofobserved sequence through xi, stopping in state k)

f0(0) = 1, fk(0) = 0 for k > 0For i = 1 to L; for ` = 1 to M (# states)

f`(i) = P(xi | `)∑

k fk(i− 1)P(` | k)P(X) =

∑k fk(L)P(0 | k)

To avoid underflow, can again use logs, though exactness ofresults compromised

17 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsThe Backward Algorithm

Given a sequence X, find the probability that xi was emittedby state k, i.e.,

P(πi = k | X) = P(πi = k,X)P(X)

fk(i)︷︸︸︷P(x1, . . . , xi, πi = k)

bk(i)︷︸︸︷P(xi+1, . . . , xL | πi = k)

P(X)︸︷︷︸computed by forward alg

Algorithm:

bk(L) = P(0 | k) for all kFor i = L− 1 to 1; for k = 1 to M (# states)

bk(i) =∑

` P(` | k)P(xi+1 | `) b`(i + 1)18 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsExample Use of Forward/Backward Algorithm

Define g(k) = 1 if k ∈ {A+, C+, G+, T+} and 0otherwiseThen G(i | X) =

∑k P(πi = k | X) g(k) = probability that

xi is in an islandFor each state k, compute P(πi = k | X) withforward/backward algorithmTechnique applicable to any HMM where set of states ispartitioned into classes

Use to label individual parts of a sequence

19 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsSpecifying an HMM

Two problems: defining structure (set of states) andparameters (transition and emission probabilities)Start with latter problem, i.e., given a training setX1, . . . ,XN of independently generated sequences,learn a good set of parameters θGoal is to maximize the (log) likelihood of seeing thetraining set given that θ is the set of parameters for theHMM generating them:

N∑j=1

log(P(Xj; θ))

20 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsSpecifying an HMM: State Sequence Known

Estimating parameters when e.g., islands alreadyidentified in training setLet Ak` = number of k→ ` transitions and Ek(b) =number of emissions of b in state k

P(` | k) = Ak` /

(∑`′

Ak`′

P(b | k) = Ek(b) /

(∑b′

Ek(b′)

21 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsSpecifying an HMM: State Sequence Known (cont’d)

Be careful if little training data available

E.g., an unused state k will have undefined parametersWorkaround: Add pseudocounts rk` to Ak` and rk(b) toEk(b) that reflect prior biases about parobabilitiesIncreased training data decreases prior’s influence

22 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsSpecifying an HMM: The Baum-Welch Algorithm

Used for estimating params when state seq unknownSpecial case of expectation maximization (EM)Start with arbitrary P(` | k) and P(b | k), and use toestimate Ak` and Ek(b) as expected number ofoccurrences given the training set1:

N∑j=1

1P(Xj)

L∑i=1

f jk(i)P(` | k)P(xj

i+1 | `) bj`(i + 1)

(Prob. of transition from k to ` at position i of sequencej, summed over all positions of all sequences)

1Superscript j corresponds to jth train example23 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsSpecifying an HMM: The Baum-Welch Algorithm (cont’d)

Ek(b) =N∑

∑i:xj

P(πi = k | Xj) =

N∑j=1

1P(Xj)

∑i:xj

f jk(i) bj

Use these (& pseudocounts) to recompute P(` | k) andP(b | k)After each iteration, compute log likelihood and halt ifno improvement

24 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsSpecifying an HMM: Structure

How to specify HMM states and connections?States come from background knowledge on problem,e.g., size-4 alphabet, +/−,⇒ 8 statesConnections:

Tempting to specify complete connectivity and letBaum-Welch sort it outProblem: Huge number of parameters could lead tolocal maxBetter to use background knowledge to invalidate someconnections by initializing P(` | k) = 0

Baum-Welch will respect this

25 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsSpecifying an HMM: Silent States

May want to allow model to generate sequences withcertain parts deleted

E.g., when aligning DNA or protein sequences againsta fixed model or matching a sequence of spoken wordsagainst a fixed model, some parts of the input might beomitted

Problem: Huge number of connections, slow training,local maxima

26 / 27

CSCE478/878

Stephen Scott

Introduction

Outline

MarkovChains

Viterbi

Forward Algorithm

Backward Algorithm

HMM Learning

Baum-Welch

Structure

Hidden Markov ModelsSpecifying an HMM: Silent States (cont’d)

Silent states (like begin and end states) don’t emitsymbols, so they can “bypass” a regular state

If there are no purely silent loops, can update Viterbi,forward, and backward algorithms to work with silentstatesUsed extensively in profile HMMs for modelingsequences of protein families (aka multiple alignments)

27 / 27

CSCE 478/878 Lecture 9: Hidden Markov...

Documents