CS11-747 Neural Networks for NLP
Adversarial MethodsGraham Neubig
Sitehttps://phontron.com/class/nn4nlp2019/
With many slides by Zihang Dai & Qizhe Xie
Generative Models
P (X) =X
Z
P (X | Z)P (Z)<latexit sha1_base64="gzwAYFR/DfB073PQH17WZuyNuCg=">AAACBnicbZBPS8MwGMbT+W/Of1WPggSHsF1GK4J6EIZePE6wbmwtJU3TLSxpS5IKo+zmxa/ixYOKVz+DN7+N2daDbj4Q+OV535fkfYKUUaks69soLS2vrK6V1ysbm1vbO+bu3r1MMoGJgxOWiE6AJGE0Jo6iipFOKgjiASPtYHg9qbcfiJA0ie/UKCUeR/2YRhQjpS3fPGzVOnV4CV2Zcb8L9Q26nIawW9fcrftm1WpYU8FFsAuogkIt3/xywwRnnMQKMyRlz7ZS5eVIKIoZGVfcTJIU4SHqk57GGHEivXy6xxgeayeEUSL0iRWcur8ncsSlHPFAd3KkBnK+NjH/q/UyFZ17OY3TTJEYzx6KMgZVAiehwJAKghUbaUBYUP1XiAdIIKx0dBUdgj2/8iI4J42Lhn17Wm1eFWmUwQE4AjVggzPQBDegBRyAwSN4Bq/gzXgyXox342PWWjKKmX3wR8bnD0Lile8=</latexit><latexit sha1_base64="gzwAYFR/DfB073PQH17WZuyNuCg=">AAACBnicbZBPS8MwGMbT+W/Of1WPggSHsF1GK4J6EIZePE6wbmwtJU3TLSxpS5IKo+zmxa/ixYOKVz+DN7+N2daDbj4Q+OV535fkfYKUUaks69soLS2vrK6V1ysbm1vbO+bu3r1MMoGJgxOWiE6AJGE0Jo6iipFOKgjiASPtYHg9qbcfiJA0ie/UKCUeR/2YRhQjpS3fPGzVOnV4CV2Zcb8L9Q26nIawW9fcrftm1WpYU8FFsAuogkIt3/xywwRnnMQKMyRlz7ZS5eVIKIoZGVfcTJIU4SHqk57GGHEivXy6xxgeayeEUSL0iRWcur8ncsSlHPFAd3KkBnK+NjH/q/UyFZ17OY3TTJEYzx6KMgZVAiehwJAKghUbaUBYUP1XiAdIIKx0dBUdgj2/8iI4J42Lhn17Wm1eFWmUwQE4AjVggzPQBDegBRyAwSN4Bq/gzXgyXox342PWWjKKmX3wR8bnD0Lile8=</latexit><latexit sha1_base64="gzwAYFR/DfB073PQH17WZuyNuCg=">AAACBnicbZBPS8MwGMbT+W/Of1WPggSHsF1GK4J6EIZePE6wbmwtJU3TLSxpS5IKo+zmxa/ixYOKVz+DN7+N2daDbj4Q+OV535fkfYKUUaks69soLS2vrK6V1ysbm1vbO+bu3r1MMoGJgxOWiE6AJGE0Jo6iipFOKgjiASPtYHg9qbcfiJA0ie/UKCUeR/2YRhQjpS3fPGzVOnV4CV2Zcb8L9Q26nIawW9fcrftm1WpYU8FFsAuogkIt3/xywwRnnMQKMyRlz7ZS5eVIKIoZGVfcTJIU4SHqk57GGHEivXy6xxgeayeEUSL0iRWcur8ncsSlHPFAd3KkBnK+NjH/q/UyFZ17OY3TTJEYzx6KMgZVAiehwJAKghUbaUBYUP1XiAdIIKx0dBUdgj2/8iI4J42Lhn17Wm1eFWmUwQE4AjVggzPQBDegBRyAwSN4Bq/gzXgyXox342PWWjKKmX3wR8bnD0Lile8=</latexit><latexit sha1_base64="gzwAYFR/DfB073PQH17WZuyNuCg=">AAACBnicbZBPS8MwGMbT+W/Of1WPggSHsF1GK4J6EIZePE6wbmwtJU3TLSxpS5IKo+zmxa/ixYOKVz+DN7+N2daDbj4Q+OV535fkfYKUUaks69soLS2vrK6V1ysbm1vbO+bu3r1MMoGJgxOWiE6AJGE0Jo6iipFOKgjiASPtYHg9qbcfiJA0ie/UKCUeR/2YRhQjpS3fPGzVOnV4CV2Zcb8L9Q26nIawW9fcrftm1WpYU8FFsAuogkIt3/xywwRnnMQKMyRlz7ZS5eVIKIoZGVfcTJIU4SHqk57GGHEivXy6xxgeayeEUSL0iRWcur8ncsSlHPFAd3KkBnK+NjH/q/UyFZ17OY3TTJEYzx6KMgZVAiehwJAKghUbaUBYUP1XiAdIIKx0dBUdgj2/8iI4J42Lhn17Wm1eFWmUwQE4AjVggzPQBDegBRyAwSN4Bq/gzXgyXox342PWWjKKmX3wR8bnD0Lile8=</latexit>
Generative Models
• Model a data distribution P(X) or a conditional one P(X|Y)
• Latent variable models: introduce another variable Z, and model
P (X) =X
Z
P (X | Z)P (Z)<latexit sha1_base64="gzwAYFR/DfB073PQH17WZuyNuCg=">AAACBnicbZBPS8MwGMbT+W/Of1WPggSHsF1GK4J6EIZePE6wbmwtJU3TLSxpS5IKo+zmxa/ixYOKVz+DN7+N2daDbj4Q+OV535fkfYKUUaks69soLS2vrK6V1ysbm1vbO+bu3r1MMoGJgxOWiE6AJGE0Jo6iipFOKgjiASPtYHg9qbcfiJA0ie/UKCUeR/2YRhQjpS3fPGzVOnV4CV2Zcb8L9Q26nIawW9fcrftm1WpYU8FFsAuogkIt3/xywwRnnMQKMyRlz7ZS5eVIKIoZGVfcTJIU4SHqk57GGHEivXy6xxgeayeEUSL0iRWcur8ncsSlHPFAd3KkBnK+NjH/q/UyFZ17OY3TTJEYzx6KMgZVAiehwJAKghUbaUBYUP1XiAdIIKx0dBUdgj2/8iI4J42Lhn17Wm1eFWmUwQE4AjVggzPQBDegBRyAwSN4Bq/gzXgyXox342PWWjKKmX3wR8bnD0Lile8=</latexit><latexit sha1_base64="gzwAYFR/DfB073PQH17WZuyNuCg=">AAACBnicbZBPS8MwGMbT+W/Of1WPggSHsF1GK4J6EIZePE6wbmwtJU3TLSxpS5IKo+zmxa/ixYOKVz+DN7+N2daDbj4Q+OV535fkfYKUUaks69soLS2vrK6V1ysbm1vbO+bu3r1MMoGJgxOWiE6AJGE0Jo6iipFOKgjiASPtYHg9qbcfiJA0ie/UKCUeR/2YRhQjpS3fPGzVOnV4CV2Zcb8L9Q26nIawW9fcrftm1WpYU8FFsAuogkIt3/xywwRnnMQKMyRlz7ZS5eVIKIoZGVfcTJIU4SHqk57GGHEivXy6xxgeayeEUSL0iRWcur8ncsSlHPFAd3KkBnK+NjH/q/UyFZ17OY3TTJEYzx6KMgZVAiehwJAKghUbaUBYUP1XiAdIIKx0dBUdgj2/8iI4J42Lhn17Wm1eFWmUwQE4AjVggzPQBDegBRyAwSN4Bq/gzXgyXox342PWWjKKmX3wR8bnD0Lile8=</latexit><latexit sha1_base64="gzwAYFR/DfB073PQH17WZuyNuCg=">AAACBnicbZBPS8MwGMbT+W/Of1WPggSHsF1GK4J6EIZePE6wbmwtJU3TLSxpS5IKo+zmxa/ixYOKVz+DN7+N2daDbj4Q+OV535fkfYKUUaks69soLS2vrK6V1ysbm1vbO+bu3r1MMoGJgxOWiE6AJGE0Jo6iipFOKgjiASPtYHg9qbcfiJA0ie/UKCUeR/2YRhQjpS3fPGzVOnV4CV2Zcb8L9Q26nIawW9fcrftm1WpYU8FFsAuogkIt3/xywwRnnMQKMyRlz7ZS5eVIKIoZGVfcTJIU4SHqk57GGHEivXy6xxgeayeEUSL0iRWcur8ncsSlHPFAd3KkBnK+NjH/q/UyFZ17OY3TTJEYzx6KMgZVAiehwJAKghUbaUBYUP1XiAdIIKx0dBUdgj2/8iI4J42Lhn17Wm1eFWmUwQE4AjVggzPQBDegBRyAwSN4Bq/gzXgyXox342PWWjKKmX3wR8bnD0Lile8=</latexit><latexit sha1_base64="gzwAYFR/DfB073PQH17WZuyNuCg=">AAACBnicbZBPS8MwGMbT+W/Of1WPggSHsF1GK4J6EIZePE6wbmwtJU3TLSxpS5IKo+zmxa/ixYOKVz+DN7+N2daDbj4Q+OV535fkfYKUUaks69soLS2vrK6V1ysbm1vbO+bu3r1MMoGJgxOWiE6AJGE0Jo6iipFOKgjiASPtYHg9qbcfiJA0ie/UKCUeR/2YRhQjpS3fPGzVOnV4CV2Zcb8L9Q26nIawW9fcrftm1WpYU8FFsAuogkIt3/xywwRnnMQKMyRlz7ZS5eVIKIoZGVfcTJIU4SHqk57GGHEivXy6xxgeayeEUSL0iRWcur8ncsSlHPFAd3KkBnK+NjH/q/UyFZ17OY3TTJEYzx6KMgZVAiehwJAKghUbaUBYUP1XiAdIIKx0dBUdgj2/8iI4J42Lhn17Wm1eFWmUwQE4AjVggzPQBDegBRyAwSN4Bq/gzXgyXox342PWWjKKmX3wR8bnD0Lile8=</latexit>
What do we want from generative models?
• A “perfect” generative model
• Evaluate likelihood: P(x)
• e.g. Perplexity in language modeling
• Generate samples: x ~ P(X)
• e.g. Generate a sentence randomly from P(X) or conditioned on some other information using P(X|Y)
• Infer latent attributes: P(Z|X)
• e.g. Infer the “topic” of a sentence in topic models
No Generative Model is Perfect (so far)
Likelihood
Generation (image)
Inference
Non-Latent VAE GAN
• Mostly rely on MLE (Lower bound) based training
• GANs are particularly good at generating continuous samples
MLE vs. GAN
Image Credit: Lotter et al. 2015
MLE vs. GAN• Over-emphasis of common outputs, fuzziness
Image Credit: Lotter et al. 2015
MLE vs. GAN• Over-emphasis of common outputs, fuzziness
Real MLE Adversarial
Image Credit: Lotter et al. 2015
MLE vs. GAN• Over-emphasis of common outputs, fuzziness
• Note: this is probably a good idea if you are doing maximum likelihood!
Real MLE Adversarial
Image Credit: Lotter et al. 2015
Adversarial Training
Adversarial Training
• Basic idea: create a “discriminator” that criticizes some aspect of the generated output
Adversarial Training
• Basic idea: create a “discriminator” that criticizes some aspect of the generated output
• Generative adversarial networks: criticize the generated output
Adversarial Training
• Basic idea: create a “discriminator” that criticizes some aspect of the generated output
• Generative adversarial networks: criticize the generated output
• Adversarial feature learning: criticize the generated features to find some trait
Generative Adversarial Networks
Basic Paradigm• Two players: generator and discriminator
• Discriminator: given an image, try to tell whether it is real or not → P(image is real)
• Generator: try to generate an image that fools the discriminator into answering “real”
• Desired result at convergence
• Generator: generate perfect image
• Discriminator: cannot tell the difference
Training Method
xreal
sample minibatch
sample latent vars.
z
xfake
convert w/ generator
yreal
discriminator loss (higher if fail predictions)
generator loss (higher if correct predictions)
predict w/ discriminator
yfake
D gradientG gradient
In Equations• Discriminator loss function:
• Make generated data “less fake” → Zero sum loss:
• Make generated data “more real” → Heuristic non-saturating loss:
• Latter gives better gradients when discriminator accurate
`D(✓D, ✓G) = �1
2Ex⇠Pdata logD(x)� 1
2Ez log(1�D(G(z)))
Predict fake for fake data
`G(✓D, ✓G) = �1
2Ez logD(G(z))
`G(✓D, ✓G) = �`D(✓D, ✓G)
• Generator loss function:
P(fake) = 1 - P(real)
Predict real for real data
Interpretation: Distribution Matching
Process
• [Step1] Z ~ P(Z), P(Z) can be any distribution
• [Step2] X = F(Z), F is a deterministic function
Result
• X is a random variable with an implicit distribution P(X), which decided by both P(Z) and F
• The process can produce any complicated distribution P(X) with a reasonable P(Z) and a powerful enough F
Image Credit: He et al. 2018
z1<latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit><latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit><latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit>
z2<latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit><latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit><latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit>
z3<latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit><latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit><latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit>
x3<latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit>
x2<latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit>
x1<latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit>
{<latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit><latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit><latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit>
Symbo
lic
Mod
el
{<latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit><latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit><latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit>
Neu
ral
Projec
tor
e1<latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit><latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit><latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit>
e2<latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit>
e3<latexit sha1_base64="LgiizHRmhHOKEEbwqVR4tWBS3lw=">AAACDXicZVDLTgIxFL2DL8QXauLGzURi4oKQQRfqjujGJSQiJECwUzrQ0E4n7R0TMuEfjFv9DlfGrd/gxn/wD+wAC4GbNPf03HOb0+NHghv0vG8ns7K6tr6R3cxtbe/s7uX3Dx6MijVldaqE0k2fGCZ4yOrIUbBmpBmRvmANf3ibzhtPTBuuwnscRawjST/kAacELdVs+zJh4+5FN1/wSt6k3GVQnoFC5aj28wgA1W7+t91TNJYsRCqIMa2yF2EnIRo5FWyca8eGRYQOSZ+1LAyJZKaTTPyO3VPL9NxAaXtCdCfs/42ESCMJDqwybWZuljKolDBFq8KBTFv6zORuRtIv+rKYirQJzIIRDK46CQ+jGFlIpz6CWLio3DQbt8c1oyhGFhCquf2KSwdEE4o2wZzNqLyYyDKon5euS17NZnUD08rCMZzAGZThEipwB1WoAwUBL/AKb86z8+58OJ9TacaZ7RzCXDlff9YqntM=</latexit><latexit sha1_base64="RMNkU799kfoHwXHS6vi9XiRQ2tw=">AAACDXicZVDLSgMxFM3UV62vquDGzWARXJQy1YW6K3XjsgXHFtqhZNJMG5pMhuSOUIb+g7jVX3DrSlwJfoMb/8E/MDPtwrYXwj0599xwcvyIMw2O823lVlbX1jfym4Wt7Z3dveL+wb2WsSLUJZJL1faxppyF1AUGnLYjRbHwOW35o5t03nqgSjMZ3sE4op7Ag5AFjGAwVLvri4ROehe9YsmpOFnZy6A6A6XaUfOHvdY/G73ib7cvSSxoCIRjrTtVJwIvwQoY4XRS6MaaRpiM8IB2DAyxoNpLMr8T+9QwfTuQypwQ7Iz9v5FgoQWGoVGmTc/NUgak5LpsVDAUaUufye56LPyyL8qpSOlALxiB4MpLWBjFQEMy9RHE3AZpp9nYfaYoAT42ABPFzFdsMsQKEzAJFkxG1cVEloF7XrmuOE2TVR1NK4+O0Qk6Q1V0iWroFjWQiwji6Ak9oxfr0Xqz3q2PqTRnzXYO0VxZX39bgqCP</latexit><latexit sha1_base64="RMNkU799kfoHwXHS6vi9XiRQ2tw=">AAACDXicZVDLSgMxFM3UV62vquDGzWARXJQy1YW6K3XjsgXHFtqhZNJMG5pMhuSOUIb+g7jVX3DrSlwJfoMb/8E/MDPtwrYXwj0599xwcvyIMw2O823lVlbX1jfym4Wt7Z3dveL+wb2WsSLUJZJL1faxppyF1AUGnLYjRbHwOW35o5t03nqgSjMZ3sE4op7Ag5AFjGAwVLvri4ROehe9YsmpOFnZy6A6A6XaUfOHvdY/G73ib7cvSSxoCIRjrTtVJwIvwQoY4XRS6MaaRpiM8IB2DAyxoNpLMr8T+9QwfTuQypwQ7Iz9v5FgoQWGoVGmTc/NUgak5LpsVDAUaUufye56LPyyL8qpSOlALxiB4MpLWBjFQEMy9RHE3AZpp9nYfaYoAT42ABPFzFdsMsQKEzAJFkxG1cVEloF7XrmuOE2TVR1NK4+O0Qk6Q1V0iWroFjWQiwji6Ak9oxfr0Xqz3q2PqTRnzXYO0VxZX39bgqCP</latexit>
f�(e)<latexit sha1_base64="8oxgnabmEamXCM6ELWdhWTrrt8c=">AAACHHicZVA9SwNBEJ3z2/gVFSubwyhECOFio3ZBG0sFzwgmhL3NnlncvT1254Rw3G8RW+2tbazEVvBnWNi7d0lh4sAyb9+8Wd6+IBbcoOd9OVPTM7Nz8wuLpaXlldW18vrGlVGJpsynSih9HRDDBI+YjxwFu441IzIQrBXcnebz1j3ThqvoEgcx60hyG/GQU4KW6pa3wm7aDmTajvs8y6o5ZNl+t1zx6l5R7n/QGIFKc/fn5RUAzrvl73ZP0USyCKkgxtw0vBg7KdHIqWBZqZ0YFhN6R27ZjYURkcx00sJ+5u5ZpueGStsToVuwfzdSIo0k2LfKvJmxWc6gUsLUrAr7Mm/5M8XdDGRQC2QtF2kTmgkjGB51Uh7FCbKIDn2EiXBRuXlUbo9rRlEMLCBUc/sVl/aJJhRtoCWbUWMykf/AP6gf170Lm9UJDGsBtmEHqtCAQ2jCGZyDDxRSeIQneHYenDfn3fkYSqec0c4mjJXz+QtiiKWs</latexit><latexit sha1_base64="JbCPJ2fHAGZa8Ebw6DTj/vvFjWE=">AAACHHicZVDLSgMxFM34rK2PqrhyM1iFCqXMuFF3RTcuK1hbaEvJpJk2NJkMyZ1CGeZbxK3u/QFxJW4FP8OFrs20Xdj2Qrgn554bTo4XcqbBcb6speWV1bX1zEY2t7m1vZPf3bvXMlKE1ojkUjU8rClnAa0BA04boaJYeJzWvcF1Oq8PqdJMBncwCmlb4F7AfEYwGKqTP/A7ccsTcSvssyQpppAmp518wSk747IXgTsFhcrxz8vrMPdb7eS/W11JIkEDIBxr3XSdENoxVsAIp0m2FWkaYjLAPdo0MMCC6nY8tp/YJ4bp2r5U5gRgj9n/GzEWWmDoG2Xa9MwsZUBKrktGBX2RtvSZ8V2PhFfyRCkVKe3rOSPgX7RjFoQR0IBMfPgRt0HaaVR2lylKgI8MwEQx8xWb9LHCBEygWZORO5/IIqidlS/Lzq3J6gpNKoMO0REqIhedowq6QVVUQwTF6BE9oWfrwXqz3q2PiXTJmu7so5myPv8Ah3SnJg==</latexit><latexit sha1_base64="JbCPJ2fHAGZa8Ebw6DTj/vvFjWE=">AAACHHicZVDLSgMxFM34rK2PqrhyM1iFCqXMuFF3RTcuK1hbaEvJpJk2NJkMyZ1CGeZbxK3u/QFxJW4FP8OFrs20Xdj2Qrgn554bTo4XcqbBcb6speWV1bX1zEY2t7m1vZPf3bvXMlKE1ojkUjU8rClnAa0BA04boaJYeJzWvcF1Oq8PqdJMBncwCmlb4F7AfEYwGKqTP/A7ccsTcSvssyQpppAmp518wSk747IXgTsFhcrxz8vrMPdb7eS/W11JIkEDIBxr3XSdENoxVsAIp0m2FWkaYjLAPdo0MMCC6nY8tp/YJ4bp2r5U5gRgj9n/GzEWWmDoG2Xa9MwsZUBKrktGBX2RtvSZ8V2PhFfyRCkVKe3rOSPgX7RjFoQR0IBMfPgRt0HaaVR2lylKgI8MwEQx8xWb9LHCBEygWZORO5/IIqidlS/Lzq3J6gpNKoMO0REqIhedowq6QVVUQwTF6BE9oWfrwXqz3q2PiXTJmu7so5myPv8Ah3SnJg==</latexit>
zi ⇠ Symbolic Model<latexit sha1_base64="qRL/AxTFpgdIbBxTWc/weaeer2U=">AAACJXicZVBNaxsxEJ112sZ1P+Imx4Qiago9GLPOJenNpJdeCgmNY4NtjFaetYWl1SLNljqLb/0xIdf2kF+RUwj01P6F/oNo7Rxqe0DM05s3w8yLUiUdheHvoLT15Omz7fLzyouXr17vVN/sXjiTWYFtYZSx3Yg7VDLBNklS2E0tch0p7ETTT0W98w2tkyY5p1mKA83HiYyl4OSpYfXt5VCyvpOa9Qm/U/51piOjpGBfzAjVfFithY1wEWwTNB9BrXVwc/YXAE6H1X/9kRGZxoSE4s71mmFKg5xbkkLhvNLPHKZcTPkYex4mXKMb5Is75uy9Z0YsNta/hNiC/b8j59ppThOvLJJbqRUMGaNc3atoootUjFn8nT+rHul6IbIudmuLUHw8yGWSZoSJWO4RZ4qRYYVnbCQtClIzD7iw0p/CxIRbLsg7W/EeNdcd2QTtw8bHRnjmvTqBZZRhH97BB2jCEbTgM5xCGwT8gGv4Cb+Cq+A2uAvul9JS8NizBysR/HkAgemouQ==</latexit><latexit sha1_base64="Dnc6JOWSxrJAoiQOjwnmnyt3/Tw=">AAACJXicZVA9SwNBEN3zM8avqKUii0GwCOFio3ZBGxsholHBhLC3mTOLu7fH7pwYj3T6X8RWC3+FlQhW/gAr/4F7iYXRgWXevnkzzLwglsKi7797I6Nj4xOTuan89Mzs3HxhYfHE6sRwqHMttTkLmAUpIqijQAlnsQGmAgmnweVeVj+9AmOFjo6xG0NTsYtIhIIzdFSrsHrTErRhhaINhGtMj7oq0FJweqDbIHutQtEv+/2g/0HlBxSrK8+Hn3erz7VW4avR1jxRECGXzNrzih9jM2UGBZfQyzcSCzHjl+wCzh2MmALbTPt39Oi6Y9o01Ma9CGmf/d2RMmUVw45TZskO1TIGtZa25FTYUVnKxvT/1p1VClQpExkb2j+LYLjdTEUUJwgRH+wRJpKipplntC0McJRdBxg3wp1CeYcZxtE5m3ceVf468h/UN8s7Zf/QebVLBpEjy2SNbJAK2SJVsk9qpE44uSUP5JE8effei/fqvQ2kI95PzxIZCu/jG4oiqh8=</latexit><latexit sha1_base64="Dnc6JOWSxrJAoiQOjwnmnyt3/Tw=">AAACJXicZVA9SwNBEN3zM8avqKUii0GwCOFio3ZBGxsholHBhLC3mTOLu7fH7pwYj3T6X8RWC3+FlQhW/gAr/4F7iYXRgWXevnkzzLwglsKi7797I6Nj4xOTuan89Mzs3HxhYfHE6sRwqHMttTkLmAUpIqijQAlnsQGmAgmnweVeVj+9AmOFjo6xG0NTsYtIhIIzdFSrsHrTErRhhaINhGtMj7oq0FJweqDbIHutQtEv+/2g/0HlBxSrK8+Hn3erz7VW4avR1jxRECGXzNrzih9jM2UGBZfQyzcSCzHjl+wCzh2MmALbTPt39Oi6Y9o01Ma9CGmf/d2RMmUVw45TZskO1TIGtZa25FTYUVnKxvT/1p1VClQpExkb2j+LYLjdTEUUJwgRH+wRJpKipplntC0McJRdBxg3wp1CeYcZxtE5m3ceVf468h/UN8s7Zf/QebVLBpEjy2SNbJAK2SJVsk9qpE44uSUP5JE8effei/fqvQ2kI95PzxIZCu/jG4oiqh8=</latexit>
ei ⇠ N (µzi ,⌃zi)<latexit sha1_base64="8CLMr+o0dZvNiJKf88rFwAP//jQ=">AAACQ3icZVBNa9tAEB0laZO4H3GaYy9LQiGlxki5JIUcQnvJKaS0bgyWEKv1yl6yqxW7o4Ir9F9Kfkzptb31P+TS0Guhu7IPiT2wzJs3b5aZl5VSWAzD38Ha+sajx5tb250nT5893+nuvvhsdWUYHzAttRlm1HIpCj5AgZIPS8OpyiS/yq7f+/7VF26s0MUnnJU8UXRSiFwwio5Ku6dxpmrepILEVigSK4pTRmV90Rz6TqyqJq2/pqLpkbb+KCaKLqjXafcg7IdtkFUQLcDB2X785hsAXKbdP/FYs0rxApmk1o6isMSkpgYFk7zpxJXlJWXXdMJHDhZUcZvU7ZUNeeWYMcm1ca9A0rL3J2qqrF/fKX2yD3qeQa2l7TkVTpVP/pu2tjOV9TLV8yJjc7u0COYnSS2KskJesPkeeSUJauIdJWNhOEM5c4AyI9wphE2poQyd7x3nUbTsyCoYHPXf9sMPzqt3MI8teAn7cAgRHMMZnMMlDIDBDfyAn/Ar+B7cBnfB37l0LVjM7MGDCP79BwcMtOc=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit>
xi = f�(ei)<latexit sha1_base64="270VJLD7gt0vH7AtNf4cVJpk6VA=">AAACKXicZVC7TisxEJ3lTXgFKGksHhJIUbShgVtcKYKGEiQCSCRaeR0vsbDXK3v26kar7fkYRAmUfAMV0PIFFPTYCQWPkaw5c+aMNXPiTAqLYfgcjIyOjU9MTk1XZmbn5heqi0snVueG8RbTUpuzmFouRcpbKFDys8xwqmLJT+PLfd8//ceNFTo9xn7GO4pepCIRjKKjoupqO1bF/zIS5C9JosJX7awnynLTQ+4aW1F1LayHgyC/QeMTrDXX328fAOAwqr61u5rliqfIJLX2vBFm2CmoQcEkLyvt3PKMskt6wc8dTKnitlMMbinJhmO6JNHGvRTJgP06UVBlFcWeU/pkv/U8g1pLW3Mq7Cmf/DeD2vZVXItVzYuMTeyPRTDZ7RQizXLkKRvukeSSoCbeN9IVhjOUfQcoM8KdQliPGsrQuVtxHjV+OvIbtLbrf+rhkfNqD4YxBSuwCpvQgB1owgEcQgsYXMEN3MF9cB08Bk/By1A6EnzOLMO3CF4/AGOkqtY=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit>
xi ⇠ Point mass at f�(ei)<latexit sha1_base64="bmmUzqeCYB8Tl/BJoP0d5EVDo5U=">AAACQnicZVCxThtBEJ2DhIAJxJCSZhWCBJJlnWkSRIOShtJIcUDirNPeeg6v2L097c4hW6f7mPxE6jSINpT5hlQookuRXZsiwEirefvmzWjmZaWSjuL4V7Sw+OLl0qvlldbq67X1N+2Nza/OVFbgQBhl7FnGHSpZ4IAkKTwrLXKdKTzNLj+H+ukVWidN8YWmJQ41vyhkLgUnT6XtwyTT9aRJJUuc1CwhnFDdN7IgprlzjBNrWJ7WQZaUY9k0uwGi79hL29txN54Few56D2D76P399x8A0E/bd8nIiEpjQUL56ee9uKRhzS1JobBpJZXDkotLfoHnHhZcoxvWsyMbtuOZEcuN9c9vN2P/76i5dprT2CtDco9qgSFjlOt4FY11SGHM7O+mOutkuhNE1uXuySKUfxzWsigrwkLM98grxciwYCgbSYuC1NQDLqz0pzAx5pYL8ra3vEe9p448B4P97kE3PvFefYJ5LMMWvINd6MEHOIJj6MMABHyDG/gJt9F19Du6i/7MpQvRQ89beBTR339stbVk</latexit><latexit sha1_base64="ZRpvVrSQojZgWGWUaf4g+64Qgc4=">AAACQnicZVBNTxsxEPUCbdOkHykcuVjQSlSKok0vUPUS0QvHIDUQiY1WXmc2sbDXK3sWEa32x/An+AOo6rUc+Q2cItSeeqid5ABkJGue37wZzbwkl8JiGN4Fa+sbL16+qr2uN968ffe++WHzxOrCcOhzLbUZJMyCFBn0UaCEQW6AqUTCaXL+3ddPL8BYobMfOM1hqNg4E6ngDB0VN79FiSovq1jQyApFI4RLLHtaZEgVs5YypBVN49LLonwiqmrPQ3Adn+PmbtgO50FXQWcJdrsf/1zfXDT+9uLmLBppXijIkEs3/awT5jgsmUHBJVT1qLCQM37OxnDmYMYU2GE5P7Kinxwzoqk27rnt5uzjjpIpqxhOnNIn+6TmGdRa2pZT4UT55MfM/3aqklaiWl5kbGqfLYLpwbAUWV4gZHyxR1pIipp6Q+lIGOAopw4wboQ7hfIJM4yjs73uPOo8d2QV9L+0v7bDY+fVIVlEjWyTHbJHOmSfdMkR6ZE+4eSK/CK/yW3wM7gPZsHDQroWLHu2yJMI/v0HkaG23g==</latexit><latexit sha1_base64="ZRpvVrSQojZgWGWUaf4g+64Qgc4=">AAACQnicZVBNTxsxEPUCbdOkHykcuVjQSlSKok0vUPUS0QvHIDUQiY1WXmc2sbDXK3sWEa32x/An+AOo6rUc+Q2cItSeeqid5ABkJGue37wZzbwkl8JiGN4Fa+sbL16+qr2uN968ffe++WHzxOrCcOhzLbUZJMyCFBn0UaCEQW6AqUTCaXL+3ddPL8BYobMfOM1hqNg4E6ngDB0VN79FiSovq1jQyApFI4RLLHtaZEgVs5YypBVN49LLonwiqmrPQ3Adn+PmbtgO50FXQWcJdrsf/1zfXDT+9uLmLBppXijIkEs3/awT5jgsmUHBJVT1qLCQM37OxnDmYMYU2GE5P7Kinxwzoqk27rnt5uzjjpIpqxhOnNIn+6TmGdRa2pZT4UT55MfM/3aqklaiWl5kbGqfLYLpwbAUWV4gZHyxR1pIipp6Q+lIGOAopw4wboQ7hfIJM4yjs73uPOo8d2QV9L+0v7bDY+fVIVlEjWyTHbJHOmSfdMkR6ZE+4eSK/CK/yW3wM7gPZsHDQroWLHu2yJMI/v0HkaG23g==</latexit>
z1<latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit><latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit><latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit>
z2<latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit><latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit><latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit>
z3<latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit><latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit><latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit>
x3<latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit>
x2<latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit>
x1<latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit>
{<latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit><latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit><latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit>
Symbo
lic
Mod
el
{<latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit><latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit><latexit sha1_base64="/RVgraER0C9MMfFvKD5SjE6r/Oo=">AAACB3icZVDLSgMxFM34rPVVdelmsAguSpmKoO6KblxWcWyhHUomzbSheQzJHaEM/QFxq9/hStz6GX6Gf2BmOgvbXgj35Nxzw8kJY84MeN6Ps7K6tr6xWdoqb+/s7u1XDg6fjEo0oT5RXOlOiA3lTFIfGHDaiTXFIuS0HY5vs3n7mWrDlHyESUwDgYeSRYxgsNRDL+1Xql7dy8tdBo0CVFFRrX7ltzdQJBFUAuHYmG7DiyFIsQZGOJ2We4mhMSZjPKRdCyUW1ARp7nTqnlpm4EZK2yPBzdn/GykWRmAYWWXWzNwsY0ApbmpWBSORteyZ/G4mIqyFopaJtInMghGIroKUyTgBKsnMR5RwF5SbpeIOmKYE+MQCTDSzX3HJCGtMwGZXthk1FhNZBv55/bru3V9UmzdFWCV0jE7QGWqgS9REd6iFfERQhF7RG3p3XpwP59P5mklXnGLnCM2V8/0HdQyacw==</latexit>
Neu
ral
Projec
tor
e1<latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit><latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit><latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit>
e2<latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit>
e3<latexit sha1_base64="LgiizHRmhHOKEEbwqVR4tWBS3lw=">AAACDXicZVDLTgIxFL2DL8QXauLGzURi4oKQQRfqjujGJSQiJECwUzrQ0E4n7R0TMuEfjFv9DlfGrd/gxn/wD+wAC4GbNPf03HOb0+NHghv0vG8ns7K6tr6R3cxtbe/s7uX3Dx6MijVldaqE0k2fGCZ4yOrIUbBmpBmRvmANf3ibzhtPTBuuwnscRawjST/kAacELdVs+zJh4+5FN1/wSt6k3GVQnoFC5aj28wgA1W7+t91TNJYsRCqIMa2yF2EnIRo5FWyca8eGRYQOSZ+1LAyJZKaTTPyO3VPL9NxAaXtCdCfs/42ESCMJDqwybWZuljKolDBFq8KBTFv6zORuRtIv+rKYirQJzIIRDK46CQ+jGFlIpz6CWLio3DQbt8c1oyhGFhCquf2KSwdEE4o2wZzNqLyYyDKon5euS17NZnUD08rCMZzAGZThEipwB1WoAwUBL/AKb86z8+58OJ9TacaZ7RzCXDlff9YqntM=</latexit><latexit sha1_base64="RMNkU799kfoHwXHS6vi9XiRQ2tw=">AAACDXicZVDLSgMxFM3UV62vquDGzWARXJQy1YW6K3XjsgXHFtqhZNJMG5pMhuSOUIb+g7jVX3DrSlwJfoMb/8E/MDPtwrYXwj0599xwcvyIMw2O823lVlbX1jfym4Wt7Z3dveL+wb2WsSLUJZJL1faxppyF1AUGnLYjRbHwOW35o5t03nqgSjMZ3sE4op7Ag5AFjGAwVLvri4ROehe9YsmpOFnZy6A6A6XaUfOHvdY/G73ib7cvSSxoCIRjrTtVJwIvwQoY4XRS6MaaRpiM8IB2DAyxoNpLMr8T+9QwfTuQypwQ7Iz9v5FgoQWGoVGmTc/NUgak5LpsVDAUaUufye56LPyyL8qpSOlALxiB4MpLWBjFQEMy9RHE3AZpp9nYfaYoAT42ABPFzFdsMsQKEzAJFkxG1cVEloF7XrmuOE2TVR1NK4+O0Qk6Q1V0iWroFjWQiwji6Ak9oxfr0Xqz3q2PqTRnzXYO0VxZX39bgqCP</latexit><latexit sha1_base64="RMNkU799kfoHwXHS6vi9XiRQ2tw=">AAACDXicZVDLSgMxFM3UV62vquDGzWARXJQy1YW6K3XjsgXHFtqhZNJMG5pMhuSOUIb+g7jVX3DrSlwJfoMb/8E/MDPtwrYXwj0599xwcvyIMw2O823lVlbX1jfym4Wt7Z3dveL+wb2WsSLUJZJL1faxppyF1AUGnLYjRbHwOW35o5t03nqgSjMZ3sE4op7Ag5AFjGAwVLvri4ROehe9YsmpOFnZy6A6A6XaUfOHvdY/G73ib7cvSSxoCIRjrTtVJwIvwQoY4XRS6MaaRpiM8IB2DAyxoNpLMr8T+9QwfTuQypwQ7Iz9v5FgoQWGoVGmTc/NUgak5LpsVDAUaUufye56LPyyL8qpSOlALxiB4MpLWBjFQEMy9RHE3AZpp9nYfaYoAT42ABPFzFdsMsQKEzAJFkxG1cVEloF7XrmuOE2TVR1NK4+O0Qk6Q1V0iWroFjWQiwji6Ak9oxfr0Xqz3q2PqTRnzXYO0VxZX39bgqCP</latexit>
f�(e)<latexit sha1_base64="8oxgnabmEamXCM6ELWdhWTrrt8c=">AAACHHicZVA9SwNBEJ3z2/gVFSubwyhECOFio3ZBG0sFzwgmhL3NnlncvT1254Rw3G8RW+2tbazEVvBnWNi7d0lh4sAyb9+8Wd6+IBbcoOd9OVPTM7Nz8wuLpaXlldW18vrGlVGJpsynSih9HRDDBI+YjxwFu441IzIQrBXcnebz1j3ThqvoEgcx60hyG/GQU4KW6pa3wm7aDmTajvs8y6o5ZNl+t1zx6l5R7n/QGIFKc/fn5RUAzrvl73ZP0USyCKkgxtw0vBg7KdHIqWBZqZ0YFhN6R27ZjYURkcx00sJ+5u5ZpueGStsToVuwfzdSIo0k2LfKvJmxWc6gUsLUrAr7Mm/5M8XdDGRQC2QtF2kTmgkjGB51Uh7FCbKIDn2EiXBRuXlUbo9rRlEMLCBUc/sVl/aJJhRtoCWbUWMykf/AP6gf170Lm9UJDGsBtmEHqtCAQ2jCGZyDDxRSeIQneHYenDfn3fkYSqec0c4mjJXz+QtiiKWs</latexit><latexit sha1_base64="JbCPJ2fHAGZa8Ebw6DTj/vvFjWE=">AAACHHicZVDLSgMxFM34rK2PqrhyM1iFCqXMuFF3RTcuK1hbaEvJpJk2NJkMyZ1CGeZbxK3u/QFxJW4FP8OFrs20Xdj2Qrgn554bTo4XcqbBcb6speWV1bX1zEY2t7m1vZPf3bvXMlKE1ojkUjU8rClnAa0BA04boaJYeJzWvcF1Oq8PqdJMBncwCmlb4F7AfEYwGKqTP/A7ccsTcSvssyQpppAmp518wSk747IXgTsFhcrxz8vrMPdb7eS/W11JIkEDIBxr3XSdENoxVsAIp0m2FWkaYjLAPdo0MMCC6nY8tp/YJ4bp2r5U5gRgj9n/GzEWWmDoG2Xa9MwsZUBKrktGBX2RtvSZ8V2PhFfyRCkVKe3rOSPgX7RjFoQR0IBMfPgRt0HaaVR2lylKgI8MwEQx8xWb9LHCBEygWZORO5/IIqidlS/Lzq3J6gpNKoMO0REqIhedowq6QVVUQwTF6BE9oWfrwXqz3q2PiXTJmu7so5myPv8Ah3SnJg==</latexit><latexit sha1_base64="JbCPJ2fHAGZa8Ebw6DTj/vvFjWE=">AAACHHicZVDLSgMxFM34rK2PqrhyM1iFCqXMuFF3RTcuK1hbaEvJpJk2NJkMyZ1CGeZbxK3u/QFxJW4FP8OFrs20Xdj2Qrgn554bTo4XcqbBcb6speWV1bX1zEY2t7m1vZPf3bvXMlKE1ojkUjU8rClnAa0BA04boaJYeJzWvcF1Oq8PqdJMBncwCmlb4F7AfEYwGKqTP/A7ccsTcSvssyQpppAmp518wSk747IXgTsFhcrxz8vrMPdb7eS/W11JIkEDIBxr3XSdENoxVsAIp0m2FWkaYjLAPdo0MMCC6nY8tp/YJ4bp2r5U5gRgj9n/GzEWWmDoG2Xa9MwsZUBKrktGBX2RtvSZ8V2PhFfyRCkVKe3rOSPgX7RjFoQR0IBMfPgRt0HaaVR2lylKgI8MwEQx8xWb9LHCBEygWZORO5/IIqidlS/Lzq3J6gpNKoMO0REqIhedowq6QVVUQwTF6BE9oWfrwXqz3q2PiXTJmu7so5myPv8Ah3SnJg==</latexit>
zi ⇠ Symbolic Model<latexit sha1_base64="qRL/AxTFpgdIbBxTWc/weaeer2U=">AAACJXicZVBNaxsxEJ112sZ1P+Imx4Qiago9GLPOJenNpJdeCgmNY4NtjFaetYWl1SLNljqLb/0xIdf2kF+RUwj01P6F/oNo7Rxqe0DM05s3w8yLUiUdheHvoLT15Omz7fLzyouXr17vVN/sXjiTWYFtYZSx3Yg7VDLBNklS2E0tch0p7ETTT0W98w2tkyY5p1mKA83HiYyl4OSpYfXt5VCyvpOa9Qm/U/51piOjpGBfzAjVfFithY1wEWwTNB9BrXVwc/YXAE6H1X/9kRGZxoSE4s71mmFKg5xbkkLhvNLPHKZcTPkYex4mXKMb5Is75uy9Z0YsNta/hNiC/b8j59ppThOvLJJbqRUMGaNc3atoootUjFn8nT+rHul6IbIudmuLUHw8yGWSZoSJWO4RZ4qRYYVnbCQtClIzD7iw0p/CxIRbLsg7W/EeNdcd2QTtw8bHRnjmvTqBZZRhH97BB2jCEbTgM5xCGwT8gGv4Cb+Cq+A2uAvul9JS8NizBysR/HkAgemouQ==</latexit><latexit sha1_base64="Dnc6JOWSxrJAoiQOjwnmnyt3/Tw=">AAACJXicZVA9SwNBEN3zM8avqKUii0GwCOFio3ZBGxsholHBhLC3mTOLu7fH7pwYj3T6X8RWC3+FlQhW/gAr/4F7iYXRgWXevnkzzLwglsKi7797I6Nj4xOTuan89Mzs3HxhYfHE6sRwqHMttTkLmAUpIqijQAlnsQGmAgmnweVeVj+9AmOFjo6xG0NTsYtIhIIzdFSrsHrTErRhhaINhGtMj7oq0FJweqDbIHutQtEv+/2g/0HlBxSrK8+Hn3erz7VW4avR1jxRECGXzNrzih9jM2UGBZfQyzcSCzHjl+wCzh2MmALbTPt39Oi6Y9o01Ma9CGmf/d2RMmUVw45TZskO1TIGtZa25FTYUVnKxvT/1p1VClQpExkb2j+LYLjdTEUUJwgRH+wRJpKipplntC0McJRdBxg3wp1CeYcZxtE5m3ceVf468h/UN8s7Zf/QebVLBpEjy2SNbJAK2SJVsk9qpE44uSUP5JE8effei/fqvQ2kI95PzxIZCu/jG4oiqh8=</latexit><latexit sha1_base64="Dnc6JOWSxrJAoiQOjwnmnyt3/Tw=">AAACJXicZVA9SwNBEN3zM8avqKUii0GwCOFio3ZBGxsholHBhLC3mTOLu7fH7pwYj3T6X8RWC3+FlQhW/gAr/4F7iYXRgWXevnkzzLwglsKi7797I6Nj4xOTuan89Mzs3HxhYfHE6sRwqHMttTkLmAUpIqijQAlnsQGmAgmnweVeVj+9AmOFjo6xG0NTsYtIhIIzdFSrsHrTErRhhaINhGtMj7oq0FJweqDbIHutQtEv+/2g/0HlBxSrK8+Hn3erz7VW4avR1jxRECGXzNrzih9jM2UGBZfQyzcSCzHjl+wCzh2MmALbTPt39Oi6Y9o01Ma9CGmf/d2RMmUVw45TZskO1TIGtZa25FTYUVnKxvT/1p1VClQpExkb2j+LYLjdTEUUJwgRH+wRJpKipplntC0McJRdBxg3wp1CeYcZxtE5m3ceVf468h/UN8s7Zf/QebVLBpEjy2SNbJAK2SJVsk9qpE44uSUP5JE8effei/fqvQ2kI95PzxIZCu/jG4oiqh8=</latexit>
ei ⇠ N (µzi ,⌃zi)<latexit sha1_base64="8CLMr+o0dZvNiJKf88rFwAP//jQ=">AAACQ3icZVBNa9tAEB0laZO4H3GaYy9LQiGlxki5JIUcQnvJKaS0bgyWEKv1yl6yqxW7o4Ir9F9Kfkzptb31P+TS0Guhu7IPiT2wzJs3b5aZl5VSWAzD38Ha+sajx5tb250nT5893+nuvvhsdWUYHzAttRlm1HIpCj5AgZIPS8OpyiS/yq7f+/7VF26s0MUnnJU8UXRSiFwwio5Ku6dxpmrepILEVigSK4pTRmV90Rz6TqyqJq2/pqLpkbb+KCaKLqjXafcg7IdtkFUQLcDB2X785hsAXKbdP/FYs0rxApmk1o6isMSkpgYFk7zpxJXlJWXXdMJHDhZUcZvU7ZUNeeWYMcm1ca9A0rL3J2qqrF/fKX2yD3qeQa2l7TkVTpVP/pu2tjOV9TLV8yJjc7u0COYnSS2KskJesPkeeSUJauIdJWNhOEM5c4AyI9wphE2poQyd7x3nUbTsyCoYHPXf9sMPzqt3MI8teAn7cAgRHMMZnMMlDIDBDfyAn/Ar+B7cBnfB37l0LVjM7MGDCP79BwcMtOc=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit>
xi = f�(ei)<latexit sha1_base64="270VJLD7gt0vH7AtNf4cVJpk6VA=">AAACKXicZVC7TisxEJ3lTXgFKGksHhJIUbShgVtcKYKGEiQCSCRaeR0vsbDXK3v26kar7fkYRAmUfAMV0PIFFPTYCQWPkaw5c+aMNXPiTAqLYfgcjIyOjU9MTk1XZmbn5heqi0snVueG8RbTUpuzmFouRcpbKFDys8xwqmLJT+PLfd8//ceNFTo9xn7GO4pepCIRjKKjoupqO1bF/zIS5C9JosJX7awnynLTQ+4aW1F1LayHgyC/QeMTrDXX328fAOAwqr61u5rliqfIJLX2vBFm2CmoQcEkLyvt3PKMskt6wc8dTKnitlMMbinJhmO6JNHGvRTJgP06UVBlFcWeU/pkv/U8g1pLW3Mq7Cmf/DeD2vZVXItVzYuMTeyPRTDZ7RQizXLkKRvukeSSoCbeN9IVhjOUfQcoM8KdQliPGsrQuVtxHjV+OvIbtLbrf+rhkfNqD4YxBSuwCpvQgB1owgEcQgsYXMEN3MF9cB08Bk/By1A6EnzOLMO3CF4/AGOkqtY=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit>
xi ⇠ Point mass at f�(ei)<latexit sha1_base64="bmmUzqeCYB8Tl/BJoP0d5EVDo5U=">AAACQnicZVCxThtBEJ2DhIAJxJCSZhWCBJJlnWkSRIOShtJIcUDirNPeeg6v2L097c4hW6f7mPxE6jSINpT5hlQookuRXZsiwEirefvmzWjmZaWSjuL4V7Sw+OLl0qvlldbq67X1N+2Nza/OVFbgQBhl7FnGHSpZ4IAkKTwrLXKdKTzNLj+H+ukVWidN8YWmJQ41vyhkLgUnT6XtwyTT9aRJJUuc1CwhnFDdN7IgprlzjBNrWJ7WQZaUY9k0uwGi79hL29txN54Few56D2D76P399x8A0E/bd8nIiEpjQUL56ee9uKRhzS1JobBpJZXDkotLfoHnHhZcoxvWsyMbtuOZEcuN9c9vN2P/76i5dprT2CtDco9qgSFjlOt4FY11SGHM7O+mOutkuhNE1uXuySKUfxzWsigrwkLM98grxciwYCgbSYuC1NQDLqz0pzAx5pYL8ra3vEe9p448B4P97kE3PvFefYJ5LMMWvINd6MEHOIJj6MMABHyDG/gJt9F19Du6i/7MpQvRQ89beBTR339stbVk</latexit><latexit sha1_base64="ZRpvVrSQojZgWGWUaf4g+64Qgc4=">AAACQnicZVBNTxsxEPUCbdOkHykcuVjQSlSKok0vUPUS0QvHIDUQiY1WXmc2sbDXK3sWEa32x/An+AOo6rUc+Q2cItSeeqid5ABkJGue37wZzbwkl8JiGN4Fa+sbL16+qr2uN968ffe++WHzxOrCcOhzLbUZJMyCFBn0UaCEQW6AqUTCaXL+3ddPL8BYobMfOM1hqNg4E6ngDB0VN79FiSovq1jQyApFI4RLLHtaZEgVs5YypBVN49LLonwiqmrPQ3Adn+PmbtgO50FXQWcJdrsf/1zfXDT+9uLmLBppXijIkEs3/awT5jgsmUHBJVT1qLCQM37OxnDmYMYU2GE5P7Kinxwzoqk27rnt5uzjjpIpqxhOnNIn+6TmGdRa2pZT4UT55MfM/3aqklaiWl5kbGqfLYLpwbAUWV4gZHyxR1pIipp6Q+lIGOAopw4wboQ7hfIJM4yjs73uPOo8d2QV9L+0v7bDY+fVIVlEjWyTHbJHOmSfdMkR6ZE+4eSK/CK/yW3wM7gPZsHDQroWLHu2yJMI/v0HkaG23g==</latexit><latexit sha1_base64="ZRpvVrSQojZgWGWUaf4g+64Qgc4=">AAACQnicZVBNTxsxEPUCbdOkHykcuVjQSlSKok0vUPUS0QvHIDUQiY1WXmc2sbDXK3sWEa32x/An+AOo6rUc+Q2cItSeeqid5ABkJGue37wZzbwkl8JiGN4Fa+sbL16+qr2uN968ffe++WHzxOrCcOhzLbUZJMyCFBn0UaCEQW6AqUTCaXL+3ddPL8BYobMfOM1hqNg4E6ngDB0VN79FiSovq1jQyApFI4RLLHtaZEgVs5YypBVN49LLonwiqmrPQ3Adn+PmbtgO50FXQWcJdrsf/1zfXDT+9uLmLBppXijIkEs3/awT5jgsmUHBJVT1qLCQM37OxnDmYMYU2GE5P7Kinxwzoqk27rnt5uzjjpIpqxhOnNIn+6TmGdRa2pZT4UT55MfM/3aqklaiWl5kbGqfLYLpwbAUWV4gZHyxR1pIipp6Q+lIGOAopw4wboQ7hfIJM4yjs73uPOo8d2QV9L+0v7bDY+fVIVlEjWyTHbJHOmSfdMkR6ZE+4eSK/CK/yW3wM7gPZsHDQroWLHu2yJMI/v0HkaG23g==</latexit>
P(Z)
x = F(z)
P(X)
In Pseudo-Code• xreal ~ Training data
• z ~ P(Z) → Normal(0, 1) or Uniform(-1, 1)
• xfake = G(z)
• yreal = D(xreal) → P(xreal is real)
• yfake = D(xfake) → P(xfake is real)
• Train D: minD - log yreal - log (1 - yfake)
• Train G: minG - log yfake → non-saturating loss
Why are GANs good?
• Discriminator is a “learned metric” parameterized by powerful neural networks
• Can easily pick up any kind of discrepancy, e.g. blurriness, global inconsistency
• Generator has fine-grained (gradient) signals to inform it what and how to improve
Problems in GAN Training• GANs are great, but training is notoriously difficult
• Known problems • Convergence & Stability:
• WGAN (Arjovsky et al., 2017) • Gradient-Based Regularization (Roth et al., 2017)
• Mode collapse/dropping: • Mini-batch Discrimination (Salimans et al. 2016) • Unrolled GAN (Metz et al. 2016)
• Overconfident discriminator: • One-side label smoothing (Salimans et al. 2016)
Applying GANs to Text
Applications of GAN Objectives to Language
Applications of GAN Objectives to Language
• GANs for Language Generation (Yu et al. 2017)
Applications of GAN Objectives to Language
• GANs for Language Generation (Yu et al. 2017)
• GANs for MT (Yang et al. 2017, Wu et al. 2017, Gu et al. 2017)
Applications of GAN Objectives to Language
• GANs for Language Generation (Yu et al. 2017)
• GANs for MT (Yang et al. 2017, Wu et al. 2017, Gu et al. 2017)
• GANs for Dialogue Generation (Li et al. 2016)
Problem! Can’t Backprop through Sampling
xreal
sample minibatch
sample latent vars.
z
xfake
convert w/ generator
y
predict w/ discriminator
Problem! Can’t Backprop through Sampling
xreal
sample minibatch
sample latent vars.
z
xfake
convert w/ generator
y
predict w/ discriminator Discrete! Can’t backprop
Solution: Use Learning Methods for Latent Variables
Solution: Use Learning Methods for Latent Variables
• Policy gradient reinforcement learning methods (e.g. Yu et al. 2016)
Solution: Use Learning Methods for Latent Variables
• Policy gradient reinforcement learning methods (e.g. Yu et al. 2016)
• Reparameterization trick for latent variables using Gumbel softmax (Gu et al. 2017)
Discriminators for Sequences
Discriminators for Sequences
• Decide whether a particular generated output is true or not
Discriminators for Sequences
• Decide whether a particular generated output is true or not
• Commonly use CNNs as discriminators, either on sentences (e.g. Yu et al. 2017), or pairs of sentences (e.g. Wu et al. 2017)
Discriminators for Sequences
• Decide whether a particular generated output is true or not
• Commonly use CNNs as discriminators, either on sentences (e.g. Yu et al. 2017), or pairs of sentences (e.g. Wu et al. 2017)
GANs for Text are Hard! (Yang et al. 2017)
GANs for Text are Hard! (Yang et al. 2017)
Type of Discriminator
GANs for Text are Hard! (Yang et al. 2017)
Type of Discriminator
Strength of Discriminator
GANs for Text are Hard! (Wu et al. 2017)
GANs for Text are Hard! (Wu et al. 2017)
Learning Rate for Generator
GANs for Text are Hard! (Wu et al. 2017)
Learning Rate for GeneratorLearning Rate for Discriminator
Stabilization Trick: Assigning Reward to Specific Actions
Stabilization Trick: Assigning Reward to Specific Actions• Getting a reward at the end of the sentence gives a
credit assignment problem
Stabilization Trick: Assigning Reward to Specific Actions• Getting a reward at the end of the sentence gives a
credit assignment problem
• Solution: assign reward for partial sequences (Yu et al. 2016, Li et al. 2017)
Stabilization Trick: Assigning Reward to Specific Actions• Getting a reward at the end of the sentence gives a
credit assignment problem
• Solution: assign reward for partial sequences (Yu et al. 2016, Li et al. 2017)
D(this)
Stabilization Trick: Assigning Reward to Specific Actions• Getting a reward at the end of the sentence gives a
credit assignment problem
• Solution: assign reward for partial sequences (Yu et al. 2016, Li et al. 2017)
D(this)D(this is)
Stabilization Trick: Assigning Reward to Specific Actions• Getting a reward at the end of the sentence gives a
credit assignment problem
• Solution: assign reward for partial sequences (Yu et al. 2016, Li et al. 2017)
D(this)D(this is)
D(this is a)
Stabilization Trick: Assigning Reward to Specific Actions• Getting a reward at the end of the sentence gives a
credit assignment problem
• Solution: assign reward for partial sequences (Yu et al. 2016, Li et al. 2017)
D(this)D(this is)
D(this is a)D(this is a fake)
Stabilization Trick: Assigning Reward to Specific Actions• Getting a reward at the end of the sentence gives a
credit assignment problem
• Solution: assign reward for partial sequences (Yu et al. 2016, Li et al. 2017)
D(this)D(this is)
D(this is a)D(this is a fake)
D(this is a fake sentence)
Stabilization Tricks: Performing Multiple Rollouts
Stabilization Tricks: Performing Multiple Rollouts
• Like other methods using discrete samples, instability is a problem
Stabilization Tricks: Performing Multiple Rollouts
• Like other methods using discrete samples, instability is a problem
• This can be helped somewhat by doing multiple rollouts (Yu et al. 2016)
Stabilization Tricks: Performing Multiple Rollouts
• Like other methods using discrete samples, instability is a problem
• This can be helped somewhat by doing multiple rollouts (Yu et al. 2016)
Discrimination over Softmax Results (Hu et al. 2017)
• Attempt to generate outputs with a specific trait (e.g. tense, sentiment)
• Discriminator over the softmax results
x h yP(y)Adversary!
Adversarial Feature Learning
Adversaries over Features vs. Over Outputs
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
x h y
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
x h yAdversary!
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
• Adversarial feature learning
x h yAdversary!
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
• Adversarial feature learning
x h y
x h y
Adversary!
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
• Adversarial feature learning
x h y
x h y
Adversary!
Adversary!
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
• Adversarial feature learning
x h y
x h y
Adversary!
Adversary!
• Why adversaries over features?
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
• Adversarial feature learning
x h y
x h y
Adversary!
Adversary!
• Why adversaries over features?• Non-generative tasks
Adversaries over Features vs. Over Outputs
• Generative adversarial networks
• Adversarial feature learning
x h y
x h y
Adversary!
Adversary!
• Why adversaries over features?• Non-generative tasks• Continuous features easier than discrete outputs
Learning Domain-invariant Representations (Ganin et al. 2016)
Learning Domain-invariant Representations (Ganin et al. 2016)
• Learn features that cannot be distinguished by domain
Learning Domain-invariant Representations (Ganin et al. 2016)
• Learn features that cannot be distinguished by domain
Learning Domain-invariant Representations (Ganin et al. 2016)
• Learn features that cannot be distinguished by domain
• Interesting application to synthetically generated or stale data (Kim et al. 2017)
Learning Language-invariant Representations
Learning Language-invariant Representations
• Chen et al. (2016) learn language-invariant representations for text classification
Learning Language-invariant Representations
• Chen et al. (2016) learn language-invariant representations for text classification
Learning Language-invariant Representations
• Chen et al. (2016) learn language-invariant representations for text classification
• Also on multi-lingual machine translation (Xie et al. 2017)
Adversarial Multi-task Learning (Liu et al. 2017)
Adversarial Multi-task Learning (Liu et al. 2017)
• Basic idea: want some features in a shared space across tasks, others separate
Adversarial Multi-task Learning (Liu et al. 2017)
• Basic idea: want some features in a shared space across tasks, others separate
Adversarial Multi-task Learning (Liu et al. 2017)
• Basic idea: want some features in a shared space across tasks, others separate
• Method: adversarial discriminator on shared features, orthogonality constraints on separate features
Implicit Discourse Connection Classification w/ Adversarial Objective
(Qin et al. 2017)
Implicit Discourse Connection Classification w/ Adversarial Objective
(Qin et al. 2017)• Idea: implicit discourse relations are not explicitly
marked, but would like to detect them if they are
Implicit Discourse Connection Classification w/ Adversarial Objective
(Qin et al. 2017)• Idea: implicit discourse relations are not explicitly
marked, but would like to detect them if they are
• Text with explicit discourse connectives should be the same as text without!
Implicit Discourse Connection Classification w/ Adversarial Objective
(Qin et al. 2017)• Idea: implicit discourse relations are not explicitly
marked, but would like to detect them if they are
• Text with explicit discourse connectives should be the same as text without!
Professor Forcing (Lamb et al. 2016)
Professor Forcing (Lamb et al. 2016)
• Halfway in between a discriminator on discrete outputs and feature learning
Professor Forcing (Lamb et al. 2016)
• Halfway in between a discriminator on discrete outputs and feature learning
• Generate output sequence according to model
Professor Forcing (Lamb et al. 2016)
• Halfway in between a discriminator on discrete outputs and feature learning
• Generate output sequence according to model
• But train discriminator on hidden states
Professor Forcing (Lamb et al. 2016)
• Halfway in between a discriminator on discrete outputs and feature learning
• Generate output sequence according to model
• But train discriminator on hidden states
x h y
(sampled or true output sequence)
Professor Forcing (Lamb et al. 2016)
• Halfway in between a discriminator on discrete outputs and feature learning
• Generate output sequence according to model
• But train discriminator on hidden states
Adversary!x h y
(sampled or true output sequence)
Unsupervised Distribution Matching
Unsupervised Style Transfer for Text (Shen et al. 2017)
• Task: transfer sentences with one style to another style • Decipherment: Translate ciphered sentences to natural sentences (A simpler case of
unsupervised MT) • Transfer sentences with positive sentiment to negative sentiment. • Word reordering
• Impressive performance on decipherment
Unsupervised Alignment of Word Embeddings (Lample et al. 2018)• We have two word embedding spaces (A) in different languages
• Define a function (e.g. orthogonal transform) to map between the spaces
• Use adversarial loss to try to align (B), further find closest words (C), use supervised objective (D)
Unsupervised Machine Translation (Lample et al. 2017, Artetxe et al. 2017)
• Methods:
• Cycle consistency (dual learning) (He et al. 2016, Zhu et al. 2017)
• Employing denoising auto-encoder to refine translated sentence
Adversarial Robustness
Problem!Networks Sensitive to Small Perturbations
(e.g. Belinkov et al. 2018)
Adversarial Noise: Noise Specifically Designed to Break
Systems• Relatively simple to perform attacks on image
classification systems: calculate gradient to maximize loss
• More difficult for text because input is discrete, but still some success (e.g. Ebrahimi et al. 2018)
What is an Adversarial Example? (Michel et al. 2019)
• It should be "meaning preserving" on the source side, and "meaning destroying" on the target side
• Meaning defined by semantic similarity (whatever that means)
Adversarial Training
• We'd like to train our models to be robust to attacks!
• Simplest idea: sample adversarial examples at training time and make sure that they are also classified correctly
• Lots of theory, but little for NLP taskshttps://adversarial-ml-tutorial.org
Questions?