SequenceLabeling,Contd
Prof.SameerSinghCS295:STATISTICALNLP
WINTER2017
February2,2017
BasedonslidesfromDanKlein,andeveryoneelsehecopiedfrom.
Outline
CS295:STATISTICALNLP(WINTER2017) 2
MarginalInferenceinHMMs:F/Balgorithm
MaximumEntropyMarkovModels
ConditionalRandomFields
NeuralSequenceTagging
Outline
CS295:STATISTICALNLP(WINTER2017) 3
MarginalInferenceinHMMs:F/Balgorithm
MaximumEntropyMarkovModels
ConditionalRandomFields
NeuralSequenceTagging
ExpectationMaximization
CS295:STATISTICALNLP(WINTER2017) 4
K-Means
InitializationPickKrandomcentroids
LabelDatafromtheModelClusterallthepoints
UpdatetheModelfromData
Updatecentroids
LabelDatafromtheModel
CS295:STATISTICALNLP(WINTER2017) 5
Hard-EM
Soft-EM
DynamicProgramming
CS295:STATISTICALNLP(WINTER2017) 6
ForwardBackwardAlgorithm
CS295:STATISTICALNLP(WINTER2017) 7
UpdatingtheModelfromData
CS295:STATISTICALNLP(WINTER2017) 8
Hard-EM
Soft-EM
Outline
CS295:STATISTICALNLP(WINTER2017) 9
MarginalInferenceinHMMs:F/Balgorithm
MaximumEntropyMarkovModels
ConditionalRandomFields
NeuralSequenceTagging
NamedEntityRecognition
CS295:STATISTICALNLP(WINTER2017) 10
GeorgeW.BushspokefromtheWhiteHousetoday.
PERPER PEROOOLOCLOCOO
B-PERI-PERI-PEROOOB-LOCI-LOCOO
B-PERI-PERE-PEROOOB-LOCE-LOCOO
Max.EntropyMarkovModels
CS295:STATISTICALNLP(WINTER2017) 11
GraphicalModelNotation
CS295:STATISTICALNLP(WINTER2017) 12
S E
HMMs
S E
MEMMs
AddingFeatures(forPOS)
CS295:STATISTICALNLP(WINTER2017) 13
CurrentWord
WindowWords
AddingFeatures
CS295:STATISTICALNLP(WINTER2017) 14
S E
PredictionsUsingMEMMs
CS295:STATISTICALNLP(WINTER2017) 15
Greedy
ViterbiDecoding
TrainingMEMMs
CS295:STATISTICALNLP(WINTER2017) 16
Outline
CS295:STATISTICALNLP(WINTER2017) 17
MarginalInferenceinHMMs:F/Balgorithm
MaximumEntropyMarkovModels
ConditionalRandomFields
NeuralSequenceTagging
LabelBiasProblem
CS295:STATISTICALNLP(WINTER2017) 18
S
O O O
I I I
B B B
E
B B
ConditionalRandomFields
CS295:STATISTICALNLP(WINTER2017) 19
GraphicalModelNotation
CS295:STATISTICALNLP(WINTER2017) 20
S E
HMMs/MEMMs
S E
CRFs
PredictionsUsingCRFs
CS295:STATISTICALNLP(WINTER2017) 21
LikelihoodTrainingofCRFs
CS295:STATISTICALNLP(WINTER2017) 22
LikelihoodTrainingofCRFs
CS295:STATISTICALNLP(WINTER2017) 23
Forward-BackwardAlgorithm
CS295:STATISTICALNLP(WINTER2017) 24
Outline
CS295:STATISTICALNLP(WINTER2017) 25
MarginalInferenceinHMMs:F/Balgorithm
MaximumEntropyMarkovModels
ConditionalRandomFields
NeuralSequenceTagging
SimpleNeuralTagger
CS295:STATISTICALNLP(WINTER2017) 26
PRP
I
VB
love
NN
foodStart End
MEMM-ish NeuralTagger
CS295:STATISTICALNLP(WINTER2017) 27
PRP
I
VB
love
NN
foodStart End
S E
RecurrentNeuralTagger
CS295:STATISTICALNLP(WINTER2017) 28
PRP
I
VB
love
NN
food
BidirectionalRNNTagger
CS295:STATISTICALNLP(WINTER2017) 29
PRP
I
VB
love
NN
food
Upcoming…
CS295:STATISTICALNLP(WINTER2017) 30
• Homework2isdue(~10days):February13,2017• Write-up,data,andcodeforHomework2isup• Askquestionsearly!
Homework
• ProposalisdueonTuesday:February7,2017• Only2pagesProject
• Papersummaries:February17,February28,March14• Only1 pageeachSummaries