50
Towards Understanding Long Short Term Memory Networks HMI 1/28/2019 Jordan Rodu Department of Statistics University of Virginia
Towards Understanding Long Short Term Memory Networks
HMI
1/28/2019
Jordan Rodu
Department of Statistics
University of Virginia
Towards Understanding Long Short Term Memory Networks
HMI
1/28/2019
Jordan Rodu
Department of Statistics
University of Virginia with Joao SedocUniversity of PennsylvaniaDepartment of Computer Science
Mapping LSTMs to state space models
• Goal is not to interpret results on specific data
• Rather, map LSTM onto reasonable models- understand the space of sequences captured by LSTMs
• Preliminary work, basic ideas
Hidden Markov Models
t t+1 t+2
Hidden Markov Models
t t+1 t+2
Hidden Markov Models
t t+1 t+2
T T
Hidden Markov Models
t t+1 t+2
T T
𝑝 𝑥𝑡+1 𝑥1:𝑡) = 𝑝 𝑥𝑡+1 𝑥𝑡)
Hidden Markov Models
t t+1 t+2
O O O
Hidden Markov Models
t t+1 t+2
O O O
𝑝 𝑦𝑡 𝑥𝑡)
Hidden Markov Models- flavors
• Output• Discrete• Continuous• Low dimensional• High dimensional
• States• Discrete• Continuous
• Low dimensional• High dimensional
• Time• Discrete• Continuous
Hidden Markov Models- flavors
• Output• Discrete• Continuous• Low dimensional• High dimensional
• States• Discrete• Continuous
• Low dimensional• High dimensional
• Time• Discrete• Continuous
Hidden Markov Models1⋮0
0⋮1
0⋮0
Hidden Markov Models
𝑏1⋮𝑏𝑘
Hidden Markov Models
𝑏1⋮𝑏𝑘
Hidden Markov Models
𝑏1⋮𝑏𝑘
𝑝𝑥1|𝑦⋮
𝑝𝑥𝑘|𝑦
Hidden Markov Models
𝑏1⋮𝑏𝑘
𝑝𝑥1|𝑦⋮
𝑝𝑥𝑘|𝑦
෨𝑏1⋮෨𝑏𝑘
Hidden Markov Model- belief states
Hidden Markov Model- belief states
Hidden Markov Model- belief states
A few related architectures
⋮ ⋮ ⋮
A few related architectures
⋮ ⋮ ⋮
A few related architectures
⋮ ⋮ ⋮
A few related architectures
⋮ ⋮ ⋮
A few related architectures
⋮ ⋮ ⋮
A few related architectures
Τ𝑡 Τ𝑡+1 Τ𝑡+2
A few related architectures
Τ𝑡 Τ𝑡+1 Τ𝑡+2
𝑂𝑡 𝑂𝑡+1 𝑂𝑡+2
LSTM
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
LSTM
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
LSTM
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
LSTM
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
Hidden Markov Models-reminder
𝑏1⋮𝑏𝑘
𝑝𝑥1|𝑦⋮
𝑝𝑥𝑘|𝑦
෨𝑏1⋮෨𝑏𝑘
LSTM
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
LSTM
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
prior (T)
LSTM
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
posterior (T and O)
LSTM
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
Hidden states, 2𝑘 states for k hidden nodes
⋮ ⋮ ⋮
Dependencies specified by weights from 𝑊ℎ
⋮ ⋮ ⋮
Incorporating memory cell
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
Incorporating memory cell
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡𝐶𝑡 = 𝑓𝑡 ∗ 𝐶𝑡−1 + 𝑖𝑡 ∗ ෩𝐶𝑡
Incorporating memory cell
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡𝐶𝑡 = 𝑓𝑡 ∗ 𝐶𝑡−1 + 𝑖𝑡 ∗ ෩𝐶𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
⋮ ⋮ ⋮
⋮ ⋮ ⋮−1, 0, 1
⋮ ⋮ ⋮−1, 0, 1
State space size 3𝑘 with special symmetry thatmodulates excitation andinhibition
Partially Observable Markov Decision Process
t t+1 t+2T T
*relaxed visualization
O O O
T
𝑎𝑡 𝑎𝑡+1 𝑎𝑡+2
POMDP representation
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
POMDP representation
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
POMDP representation
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
policy
POMDP representation
𝜎 𝜎 tanh 𝜎
x
x
+
x
tanh
𝑦𝑡
ℎ𝑡
𝜎(𝑊ℎℎ𝑡−1 +𝑊𝑦𝑦𝑡 + 𝑏)
policy
Recall that ourgoal here is notto learn a POMDPor to approximatea POMDP usingLSTMs, rather towrap LSTMs inmodels for which we have a morerobust understanding.
Thanks!
