QuantitativeUnderstandinginBiologyConclusion:IntroductiontoMarkovChainsandHiddenMarkovModelsDualitybetweenKineticModelsandMarkovModelsWe’llbeginbyconsideringthecanonicalmodelofahypotheticalionchannelthatcanexistineitheranopenstateoraclosedstate.Wemightdescribethesystemintermsofchemicalspeciesandrateconstantsusingasimplerepresentationofthereactionlike…

𝑂𝑘!↔𝑘!!

𝐶

Weshouldbeonfamiliarground,andcanproceedtowritedifferentialequationsbasedontheprincipleofmassaction.

𝑑𝑥!𝑑𝑡 = −𝑘!𝑥! + 𝑘!𝑥!𝑑𝑥!𝑑𝑡

= 𝑘!𝑥! − 𝑘!!𝑥!

Herewehavewrittenthedifferentialsintermsofthefractionsofopenandclosedchannels,denotedasxOandxC.Youmightbeusedtowritingthesetypesofequationsintermsofconcentrations;however,itiseasytoshowthat,foraconstantvolumesystem,thepopulationfractionisproportionaltotheconcentration.

𝑂 =𝑛!𝑉=𝑛!𝑛!∙𝑛!𝑉= 𝑥! ∙ [𝑇]

Hereweseethattheconcentrationofopenchannelsisjustthenumberofopenchannels,nO,dividedbythevolumeofthesystem,V.Wecanintroducethetotalnumberofchannelsinthesystem,nt,andstillhaveavalidmathematicalrelations.Thisleadstothe(unsurprising)resultthattheconcentrationofopenchannelsisequaltothetotalconcentrationofchannelstimesthefactionofchannelsthatareopen.

Weshouldnotethatinexperimentsinvolvingchannels,theconcentrationofchannelsmaybemeasuredinchannelsperunitsurfaceareaofmembrane.Thisdoesnotchangetheresult,whichallowsustowritethedifferentialequationsabove,withtheunderstandingthattherateconstantshavetheappropriateunits.

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page2

Inthislecture,we’llbeconsideringonlydiscreettimeMarkovchains,soitwillbenaturalforustotransformourdifferentialequationsintoadiscretetimerepresentation.Theprocessshouldbefamiliar;representadiscreetchangeinthestatevariablesintermsoffinitebutsufficientlysmallΔt.

𝑥!,!!! − 𝑥!,! = −𝑘!𝑥!,!∆𝑡 + 𝑘!!𝑥!,!∆𝑡𝑥!,!!! − 𝑥!,! = 𝑘!𝑥!,!∆𝑡 − 𝑘!!𝑥!,!∆𝑡

Thiscanberewrittenas…

𝑥!,!!! = 1 − 𝑘!∆𝑡 𝑥!,! + 𝑘!!∆𝑡 𝑥!,!𝑥!,!!! = 𝑘!∆𝑡 𝑥!,! + 1 − 𝑘!!∆𝑡 𝑥!,!

…andexpressedinmatrixformas…

𝑥!𝑥! !!!

= 1 − 𝑘!∆𝑡 𝑘!!∆𝑡𝑘!∆𝑡 1 − 𝑘!!∆𝑡

𝑥!𝑥! !

Analternativeviewofthissystemistoconsiderasingleionchannel.Thisisasystemthatcanbeinoneoftwostates,labeledasOandC,asinthediagrambelow.Ifweimaginethechanneliscurrentlyintheclosedstate,thenwereasonthatinasmalltimeinterval,Δt,thesystemhasaprobability,α,oftransitioningintotheopenstate.Similarly,ifthechannelisintheopenstate,thereisaprobability,β,thatitwilltransitionintotheclosedstate.

Implicitinthisdiagramisthefactthattheprobabilityoftheionchannelremainingintheclosedstategiventhatitstartsinthatstateis1–α,andsimilarlythattheprobabilityofremainingintheopenstategiventhatasthestartingpointis1–β.Insomeofthemoreelementaryliterature,youwouldseethediagramfortheMarkovchaindrawnasshownbelow.Fortheremainderourdiscussion,we’llleaveoutthesecases,andunderstandthattheyareimplicitbecausealloftheprobabilitiesleavingastatemustsumtoone.

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page3

IfxOandxCrepresenttheprobabilityofthechannelbeingintheopenandclosedstate,thenwecanwriteamatrixequationthatderivesdirectlyfromthesediagrams.

𝑥!𝑥! !!!

= 1 − 𝛽 𝛼𝛽 1 − 𝛼

𝑥!𝑥! !

Thismatrixequationisverysimilartotheonewederivedfromthekineticviewofthesystem.Akeydifferenceisthatinthekineticview,wetookxOandxCtobethefractionofchannelsinapopulationthatwereopenorclosed,whereasinthesingle-channelview,wesaidxOandxCweretheprobabilitiesthataparticularchannelofinterestisopenorclosed.Itturnsoutthatwhenallofthestatesofasystemareaccessiblefromallotherstates,thesequantitiesareequal.Suchsystemsarecalledergodic,andtheequalityofthetimeaverageofasingleentityandthebulkaverageofanensembleofentitiesiscalledtheergodichypothesis.NotethatforaMarkovchaintobeegrodic,theremustbeawaytoreacheverystatefromeverystate,butnotnecessarilyinonestep.1

Aslightlymorecomplexmodelofanionchannelisonethatincorporatesaninactivestate…

𝐼𝑘!↔𝑘!!

𝑂𝑘!↔𝑘!!

𝐶

Thediscretizedversionsofthedifferentialequationsthatdescribethissystemare(youshouldbeabletoderivethisonyouown):

𝑥!𝑥!𝑥! !!!

=1 − 𝑘!Δ𝑡 𝑘!!Δ𝑡 0𝑘!Δ𝑡 1 − 𝑘!Δ𝑡 − 𝑘!!Δ𝑡 𝑘!Δ𝑡0 𝑘!Δ𝑡 1 − 𝑘!!Δ𝑡

𝑥!𝑥!𝑥! !

ItsMarkovchaindiagramwouldlooklikethis:

Thissystemisergodicbecausethereisapathfromeverystatetoeveryotherstate(e.g.,youcangetfromItoCthroughO.Thematrixequationthatdescribesthissystemis…

1Theconditionisactuallyalittlemorestrict.Afinite-stateMarkovchainisergodicifthereisafinitenumberNsuchthatanystatecanbereachedbyanyotherstateinexactlyNsteps.ThechaintotherightisnotergodicbecausethereisnosinglenumberofstepsfromwhichyoucangetfromAtoBandfromAtoC.

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page4

𝑥!𝑥!𝑥! !!!

=1 − 𝛿 𝛾 0𝛿 1 − 𝛽 − 𝛾 𝛼0 𝛽 1 − 𝛼

𝑥!𝑥!𝑥! !

Forboththetwoandthreestatesystems,youcanseethereisadirectcorrespondencebetweenthekineticrateconstantsandthetransitionprobabilities.IfyouknowthetransitionprobabilitiesforaspecificΔt,thenyouknowtherateconstants,andvice-versa.

AsignificantmotivationforusingMarkovchainsisthatitgivesustoolsforsystematicallydeterminingthetransitionprobabilitiesforcomplexsystems.Forthetwocanonicalionchannelsystemsabove,itisprettyeasytodesignvoltageclampexperimentsthatallowforthedeterminationoftherates.However,formorerealisticandcomplexmodels,thisbecomesincreasinglydifficult.Forexample,ClancyandRudy(1999)proposedamodelthatincludedthreeclosedstatesandbothafastandaslowinactivestate:

ThekeycharacteristicofMarkovprocessesisthattheprobabilityoftransitingtothenextstateisdependentONLYonthecurrentstate.Inotherwords,thesystemhasnomemory.ThisisknownastheMarkovproperty.

Weshouldalsopointoutthatformanyionchannelsofphysiologicalimportance,thetransitionprobabilitiesarevoltagedependent.Intheotherwords,α,β,γ,andδareallfunctionsofvoltage.Wewillonlyconsiderprocessweretheelementsofthetransitionmatrixareconstantovertime(asinanexperimentwiththevoltageclamped).

AbsorbingMarkovchainsNotallMarkovprocessesareergodic.Animportantclassofnon-ergodicMarkovchainsistheabsorbingMarkovchains.Theseareprocesseswherethereisatleastonestatethatcan’tbetransitionedoutof;youcanthinkifthisstateasatrap.Someprocesseshavemorethanonesuchabsorbingstate.

OneverycommonexampleofaMarkovchainisknownatthedrunkard’swalk.Inourvariationofthisclassictoyexample,weimagineadrunkpersonwanderingaone-dimensionalstreet.Therearefivelocationsalongthestreet,asshowninthediagrambelow.

LocationAistheman’shome,andlocationEisabar.Thedrunkwillstaggerfromonelocationtothenextwhileheisbetweenthebarandhishome,butoncehereachesoneofthetwolocations,hewill

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page5

staytherepermanently(oratleastforthenight).Thedrunkard’swalkisusuallypresentedwheretheprobabilityofmovingtotheleftorrightisequal,butinourscenario,we’llsaythattheprobabilityofmovingtotherightis2/3becausethepleasantodorsemanatingfromthekabobshopnexttothebarserveasanattractant.

Wecanwriteamatrixequationtomodelourdrunkard’swanderings:

𝑝! 𝑝! 𝑝! 𝑝! 𝑝! !!! = 𝑝! 𝑝! 𝑝! 𝑝! 𝑝! !

1 0 0 0 013

023

0 0

013

023

0

0 013

023

0 0 0 0 1

Notethatinthiscasewe’vewrittenthestatevariablesofoursystem(theprobabilitiesthatourdrunkardisatanyofthefivepositionsalongthestreet)asarowvector.ThisfollowstheconventionusedinmostoftheliteratureonMarkovmodels,sowe’veadoptedithere,andwe’lluseitfortherestofthislecture.Asaconsequence,ourequationstodescribethetimeevolutionmultiplythetransitionmatrixontheleft.Also,thematrixinthisrepresentationisthetransposeofthematrixwe’dhavewrittenifwewereusingcolumnvectors.Whenwrittenlikethis,alloftherowsofthetransitionmatrixmustsumtoone.

WecanuseresultsfromthetheoryofMarkovchainstoquicklyanswersomeinterestingquestionsaboutaparticularabsorbingchain.We’lllookatthreesuchquestions:howtocomputethenumberoftimeseachparticularstateislikelytobevisited,howlongasystemislikelytolastbeforebeingabsorbed,andwhattheprobabilityofbeingabsorbedintoeachoftheabsorbingstatesis.Ingeneral,theanswertoeachofthesequestionsisdependentonwhereyoustartonthechain.

Beforeweansweranyofthesequestions,we’llneedtore-arrangeourtransitionmatrixintoacanonicalform.Allweneedtodoisreorderthestatessothatthetransientonesarefirst,andtheabsorbingonescomelast.Forourexampleproblem,wehave…

𝑝! 𝑝! 𝑝! 𝑝! 𝑝! !!! = 𝑝! 𝑝! 𝑝! 𝑝! 𝑝! !

𝐵 𝐶 𝐷 𝐴 𝐸0 !

! 0 !! 0

!! 0 !

! 0 0

0 !! 0 0 !

!

0 0 0 1 0

0 0 0 0 1

𝐵

𝐶

𝐷

𝐴

𝐸

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page6

Herewehaveannotatedtherowsandcolumnswiththenamesofthestatethateachcorrespondsto.Thiscanonicalmatrix,T,canbesubdivideintofourpartsandwrittenas…

𝑇 = 𝑄 𝑅0 𝐼

…where…

𝑄 =

0 !!

0!!

0 !!

0 !!

0

𝑅 =

!!

00 00 !

!

…and0representsamatrixofallzeros,andIrepresentsanidentitymatrix(onesonthediagonal,zeroselsewhere).

Exercise:Showthat𝑻𝒏 = 𝑸𝒏 ∗𝟎 𝑰 ,wherethe*representsacomplicatedmatrix.

Itisusefultodefineonemorematrix…

𝑁 = 𝐼 − 𝑄 !!

Thematrixwe’vedenotedasNisknownasthefundamentalmatrixoftheoriginalMarkovchain.ItturnsoutthatseveralpracticalquestionscanbereadilyansweredaboutanabsorbingMarkovchainonceNandRandknown.Infact,itcanbeshownthatifyoustartinstatei,theexpectednumberoftimesthatyou’llvisitstatejisgivenbetheelementNi,jofthefundamentalmatrix(wewon’tprovethishere).

TocomputethisinMATLABforourdrunkardswalk…

>> Q = [ 0, 2/3, 0; 1/3, 0, 2/3; 0, 1/3, 0] Q = 0 0.6667 0 0.3333 0 0.6667 0 0.3333 0 >> N = (eye(size(Q)) - Q)^-1 N = 1.4000 1.2000 0.8000 0.6000 1.8000 1.2000 0.2000 0.6000 1.4000

Thistellus,forexample,thatifourdrunkardstartsoutinthemiddleoftheroad(stateC),onaveragehewillvisitstateB0.6times,stateC1.8times,andstateD1.2times(rememberthatthefirstrowofthis

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page7

matrixcorrespondstostateB,thesecondtostateC,andthelasttostateD).NotethatthequantityforCincludesthestartingvisit.

Wecanalsocomputetheaverageabsorptiontimeforeachstartingstate.ThisistrivialifweknowN.Forexample,considerstartinginstateC.WejustsawthatthequantitiesinthesecondrowofNgivetheaveragenumberoftimeseachtransientstatewillbevisited,sothesumofalloftheseentriesmustbetheaveragenumberofvisitstoalltransientstates.Butthenumberoftransientstatesonevisitsbeforebeingabsorbedistheabsorptiontime.

Acolumnvector,t,thatrepresentsthesumofrowentriesinthefundamentalmatrixcanbewrittenas..

𝑡 = 𝑁 ∙ 𝑐

…wherecisacolumnvectorofones.ContinuingtheexampleaboveinMATLAB…

>>t=N*ones(size(N,1),1)t=3.40003.60002.2000

ThisshowsthatifourdrunkardstartsatstateB,onaveragethedrunkardwillstaggeraroundfor3.4stepsbeforefinishinguptheeveningathomeoratthebar.

Wecanalsocomputetheabsorptionprobabilitymatrix,B,simplybymultiplyingNontherightbyR.TheelementsofBgivetheprobabilityofultimatelybeingabsorbedintoeachoftheabsorbingstatesforeachstartingstate.Inourexample…

𝐵 =

𝐵 𝐶 𝐷 𝐴 𝐸𝐵

𝐶

𝐷

1.4 1.2 0.8

0.6 1.8 1.2

0.2 0.6 1.4

!! 0

0 0

0 !!

=

𝐴 𝐸𝐵

𝐶

𝐷

0.4667 0.5333

0.2000 0.8000

0.0667 0.9333

TheseresultstellusthatifwestartatstateB,forexample,thereisa46.7%chancethatwe’lleventuallyabsorbintostateA,anda53.3%changethatwe’llabsorbintostateE.

Thedrunkard’swalkisclearlyahypotheticalproblem,butexamplesjustlikeitappearinmanyothercontexts.Forexample,consideratennisgameatdeuce(e.g.,tiedat40-40).Therulesoftennisrequirethatagamebewonbytwoserves.IfplayerIwinsthenextserve,thescoreissaidtobe“AdvantageI”,

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page8

andthatplayermustwinthenextservetowinthegame.Ifshedoesnot,thescoregoesbacktodeuce.IfweimagingthatplayerIhasaprobabilityαofwinninganyserve,thentherelevantMarkovchainis…

ComparingthiswiththeMarkovchainforthedrunkard’swalkshouldmakeitapparentthatthesediagramsinfacthavethesamestructure.Wecanthereforconclude,forexample,thatifoneplayerhastwicethechanceofwinningaserveasanother(α=2/3;1-α=1/3),thentheprobabilityofthelesserplayerwinningagamethathasreacheddeuceis20%(seecomputationoftheBmatrix,above).

HiddenMarkovModelsSofar,we’veconsideredMarkovmodelswhereweknowthevaluesofthetransitionmatrix,andwentontocomputepropertiesofthesystem.We’llnowinverttheproblem,andaddressthecasewherewehaveobservationsofasystem,andwanttoestimatethevaluesinthetransitionmatrix.Inthiscase,we’llassumethatweknowhowmanystatesthereare,andneedtoestimatetheprobabilitiesoftransitioningfromeachstatetoeveryotherstate.

Asaworkingexample,we’llconsiderthisinthecontextofionchannels,continuingthediscussionatthebeginningofthislecture.Asshownpreviously,theproblemistantamounttodeterminingtherateconstantsforthestatetransitions.

We’llbeginbygeneratingsomerepresentativedatafromamodelwithknownparameters,andthenpretendwedon’tknowtheparametervalues,andseehowwecanestimatethemfromthedata.Westartwiththesimple,two-statechannel:

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page9

Beforewebegin,we’llneedtosayafewwordsabouthiddenMarkovmodels(orHMMs).InahiddenMarkovmodel,thestatesofthesystemarenotdirectlyobservable;instead,weareonlyabletoobserveandrecordsomequantitythatcorrelateswiththeinternalstateofthesystem.Inthecaseofoursimpleionchannel,theobservedquantityisthemeasuredcurrent.Thereisnothingreallyhiddenaboutthismodelbecausethesystemhastwostates,andthestateofthesystemisobviousfromthecurrent–ifyouseeahighcurrent,thechannelisintheopenstate,andalow(nearzero)currentimpliesthatthechannelisintheclosedstate.Fornow,we’llchalkthisuptosemantics,butinashortwhilewe’llseeamodelwheresomeofthestatesarebetterhidden.

TheobservablequantitiesinanHMMarecalledemissions(sinceeachstate‘emits’suchaquantity),andanHMMalwayshasanemissionmatrixalongwithitstransitionmatrix.Forourfirstsystem,thetransitionmatrix(usingnotationconsistentwitharowvectorofstatevariables)is

𝑇 =𝐶 𝑂

1 − 𝛼 𝛼

𝛽 1 − 𝛽𝐶

𝑂

…andtheemissionmatrixis…

𝐸 =𝐿 𝐻1 0

0 1

𝐶

𝑂

Herewehaveannotatedtheemissionmatrixtoclarifywhichstateeachrowcorrespondsto,andwhichemissioneachcolumncorrespondsto.Theelementsoftheemissionmatrixareprobabilitiesandgenerallydon’thavetobezerosorones;theyjusthappentobeinthiscase.

WearenowreadytodescribethissysteminMATLAB,andgeneratesomerepresentativedataforthissystem.

>> alpha = 0.01; beta = 0.06; >> T = [1 - alpha, alpha; beta, 1 - beta]; >> E = [ 1 0; 0 1]; >> [seq, states] = hmmgenerate(100000, T, E);

HerewashaveaskedMATLABtogenerate100,000iterationsofourmodel.Wearegiventheemissionsintheseqvector,andthe‘unobservable’internalstatesofthesysteminthestatesvector.

Plottingthedatarequiresthatwespecifythescaleoftheaxes,sinceallthedatacan’tmeaningfullybeshown.

>> plot(seq); >> axis([0 1000 0 3]);

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page10

Youcanpantotherighttoseemoreofthetrace.Asasidenotehere,youshouldbeawarethatwhenmakeaplotthathasmoredatapointsthanyourscreenhaspixels,yourplottingprogrammayentirelymissartifactsinyourdata(thisdependsonhowyoursoftwaresamplesthedataforplotting).

Notethatintheplotproduced,therearesomelongperiodsoflowcurrentandsomeshortperiodsoflowcurrent,andsimilarlyfortheperiodsofhighcurrent.Wesaysthatthedwelltimesforeachstatefollowsomedistribution.Itshouldbeclearfromtheplotthattheclosedstatestendtolastlongerthantheopenstates.Thisimpliesthatβ>α,whichisinfactthecase.

NotethatMATLABnumbersthestatesandemissionsfrom1,soanemissionvalueof1correspondstothelowcurrentemission,andanemissionvalueof2representsahighcurrentemission.

Formally,anHMMconsistsofatransitionmatrix,anemissionsmatrix,andastartingstate.WhenusingMATLAB’sHMMfunctions,itisassumedthatthesystemalwaysstartsinstate1.Youcanalwaysrearrangethestatestoaccommodate(orseetheMATLABhelpforothertricks).

Nowthatwehavegeneratedagoodamountoftracedata(representativeofasinglechannelrecording),outgoalistorecovertheαandβparametersfromthetracedataalone.Forthissimplemodel,thereareseveralapproaches.Onewaytogoaboutthisistoanalyzethedistributionsofthedwelltimesintheopenandclosedstates.Tothisend,wecanwriteafunctionwhichreportsthedwelltimesforaparticularemission.Thesourcecodeiskeptinafilenameddwell_times.m:

function [ times ] = dwell_times(seq,s) %DWELL_TIMES Given a sequence of states (seq), % compute the dwell times state s in the sequence. imax = length(seq); times = []; previous_state = seq(1); counter = 0; for idx = 2:imax if(previous_state == seq(idx)) counter = counter + 1; else if (previous_state == s) times = [times ; counter]; end counter = 1; previous_state = seq(idx); end end end

Wecannowcomputethedwelltimesforthelowcurrentstate.

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page11

>> dt_closed = dwell_times(seq, 1); >> mean(dt_closed) ans = 96.6472

Thisistellingusthattheaveragedwelltimeinthelowcurrentstateis96.6.Thereciprocalofthisisourestimateofthetimeconstant,ortransitionprobability,toleavetheclosedtotheopenstate.Wethereforeestimatethatα=1/96.6472=0.0103.Similarly,inthisexample,weestimatetheβ=0.0642fromthedistributionofdwelltimesintheopenstate.

Inthisexample,wewereabletomaketheseestimatesbecausewecantriviallyinferthestateofthechannelfromitsemissionmeasurement.We’llnowconsideraslightlymorecomplicatedchannelmodelwhere(someof)thestatesarehidden.

We’llreturntoourthreestatechannelmode,assigningvaluestoallofthetransitionprobabilities.

Ourtransitionmatrixis…

𝑇 =

𝐶 𝐼 𝑂1 − 𝛼 0 𝛼

0 1 − 𝛿 𝛿

𝛽 𝛾 1 − 𝛽 − 𝛾

𝐶

𝐼

𝑂

…andouremissionmatrixis…

𝐸 =

𝐿 𝐻1 0

1 0

0 1

𝐶

𝐼

𝑂

Wecannowproceedtosimulatethissystemandmakeaplotofthesimulatedcurrent:

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page12

>> alpha = 0.1; beta = 0.01; gamma = 0.005; delta = 0.005; >> T = [1 - alpha, 0, alpha; 0, 1 - delta, delta; beta, gamma, 1 - beta - gamma]; >> E = [1, 0; 1, 0; 0, 1]; >> [seq, states] = hmmgenerate(500000, T, E); >> plot(seq); >> axis([0 1000 0 3]);

Ifyoupanthroughthecurrenttrace,you’llshouldobservethattherearealargenumberofshort-livedlow-currentstates,andasmallernumberoflong-livedlow-currentstates.Lookingatourparameters,weseethattheclosedstatesareexpectedtobeshortlivedrelativetotheinactivestates(becauseα>δ),andtheweshouldhavemoreoccurrencesoftheclosedstatethantheopenstate(becauseβ>γ).

NowwecanstarttoseewhathiddenMarkovmodelsareallabout.Ifyouobservealowcurrentemission,youdon’tknowifthechannelisintheclosedorinactivestate.However,youcanmakeguessbasedonthedwelltimeofthestate.Short-livedintervalsofzero-currentimplythatthechannelisintheclosedstate,andlongeronesimplyitisintheinactivestate.Butsincestatetransitionsarestochastic,youcanneverbesureifyourassignmentiscorrect.

OneoftheverypowertoolsintheworldofHMMsistheVitirbialgorithm.Thisalgorithmtakesasequenceofemissions(i.e.,yourobservations)andthespecificationofyourHMM(i.e.,yourTandEmatrices),andcomputesthemostlikelyhistoryofstatesthatwouldgiverisetotheemissionsthatyousupplied.Herewecomputethepredictedstates,andcomparethemgraphicallytotheactualstates(whichthealgorithmdidn’tknow).

>> predstates = hmmviterbi(seq,T,E); >> plot(states, 'color', 'blue', 'linewidth', 3); >> hold on >> plot(predstates, 'color', 'red', 'linewidth', 1); >> hold off >> axis([1 1000 0 4]);

Ifyouscrollthroughtheplot,you’llseethealgorithmdoesaprettygoodjob,butonceinawhilewillmiss-classifyastate.You’lllikelyneedtopanquiteabittofindthemistakes.YoucanquicklysummarizetheperformanceoftheVitirbialgorithmforthiscase:

>> sum(states == predstates)/length(states) ans = 0.9866

Hereweseethatforour(notsohiddenafterall)model,wecanassignthecorrectstate98.7%ofthetime.Ofcourse,ifthemeandwelltimesoftheinactiveandclosedstateswerecloser,itwouldbehardertotellthemapart,andtheerrorratewouldincrease.

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page13

OurlastapplicationofHMMswillbetodemonstrateatrainingalgorithm.Thejobofthetrainingalgorithmistodeterminethevaluesinthestatetransitionmatrixandtheemissionmatrixgivenatraceofsequences.Thisisaniterativeprocedure,soitwillrequireaninitialguessforthesetwomatrices.Inourexample,theemissionmatrixissimpleandtheguessisobvious.However,guessingthetransitionprobabilitiesfromonlythesimulateddataisabittrickier.

We’llbeginbycomputingthedwell-timesasbefore.

>> dt_low = dwell_times(seq, 1); >> dt_high = dwell_times(seq,2);

Nowthestructureofourmodeltellsusthatweshouldexpecttwoclassesofnon-conductingstateswithdifferentdwell-timedistributions.Wemightsuspectthatahistogramofthedwelltimeswouldhavetwopeaks,buttryingoutafewplotsdoesn’tyieldanythingobvious.However,ahistogramofthelogarithmofthedwelltimeisinformative.

>> hist(log(dt_low), 0:0.1:10)

Youshouldseeapeakveryroughlyaround2.5andapeakaround5.Thisimpliesthattherearetwo(somewhathidden)peaksate2.5=12.2ande5=148.Soourveryroughguessesofthetransitionprobabilitiesareα=1/12.2=0.08andδ=1/148=0.007.Wealsoneedtoguessβandγ;sincethetrainingalgorithmwilldotheheavylifting,we’llassumethatβ=γ.Nowtheaveragedwelltimefortheconductingstateis…

>> mean(dt_high) ans = 67.3133

…sowehavethatβ+γ(thetwowayswecanleavetheopenstate)=1/67.3=0.0149soweguessβ=γ=0.0149/2=0.0074.Wearenowreadytorunthetrainingalgorithm:

>> alphag = 0.08; betag = 0.0074; gammag = 0.0074; deltag = 0.007; >> Tg = [ 1 - alphag, 0, alphag; 0, 1 - deltag, deltag; betag, gammag, 1 - betag - gammag]; >> Eg = E; >> [T_estimate, E_estimate] = hmmtrain(seq, Tg, Eg);

Thetrainingalgorithmmaytakealittlewhiletofinish,butwhenitdoes,wecancomparetheestimatedtransitionmatrixwiththeonefromwhichthesampledatawasderived:

MarkovChainsandHiddenMarkovModels

©Copyright2008,2016–JBanfelder,WeillCornellMedicalCollege Page14

>> T_estimate T_estimate = 0.9001 0 0.0999 0 0.9950 0.0050 0.0098 0.0050 0.9852 >> T T = 0.9000 0 0.1000 0 0.9950 0.0050 0.0100 0.0050 0.9850

HerewecanseethattheHMMhasdoneanearlyperfectjobofextractingtheunderlyingparametersfromthedata.

Weshouldnoteherethatforbothcases,itispossibletodesignexperimentsthatestimatethetransitionprobabilities(orrateconstants)byothermeansthatsomemightconsidermoredirect.However,asmodelcomplexityincreases,thedifficultyofdesigningandexecutingsuchexperimentsbecomesincreasinglydifficult;considerhowonemightestimatetherateconstantsintheClancyandRudymodeldiscussedabove.OneofthegreatutilitiesofHMMsisthattheygiveaconsistentframeworkforthinkingaboutandsolvingproblemsofarbitrarycomplexity.