Date post: | 17-Aug-2015 |
Category: |
Data & Analytics |
Upload: | nhathai-phan |
View: | 160 times |
Download: | 1 times |
1
Tutorial on Deep Learning and Applications
Hai PhanAIM Lab, University of Oregon
Includes slide material sourced from Yoshua Bengio, Geoff Hinton, Yann LeCun, Andrew Ng, Honglak Lee, and Marc’Aurelio Ranzato
2
Outline
• Deep learning– Greedy level-wise training (supervised learning)– Restricted Boltzmann machine (RBM)– Deep belief networks– Stacked autoassociators– Deep Boltzmann machines
• Applications– Human motion modeling– Vision– Language
3
Motivation: why go deep?
• Deep Architectures can be representationally efficient
• Deep representation might allow for a hierarchy of representations– Comprehensibility
• Multiple levels of latent variables allow combinatorial sharing of statistical strength
• Deep architectures work well (vision, audio, NLP, etc.)!
11
Outline
• Deep learning– Greedy level-wise training (supervised learning)– Restricted Boltzmann machine (RBM)– Deep belief networks– Stacked autoassociators– Deep Boltzmann machines
• Applications– Human motion modeling– Vision– Language
12
Restricted Boltzmann Machine (RBM)
• Binary visible units– Pixels
• Binary hidden units – Feature detectors
14
Different Types of Unit
• Gaussian visible units
• Gaussian visible and hidden units
• Softmax units
15
Outline
• Deep learning– Greedy level-wise training (supervised learning)– Restricted Boltzmann machine (RBM)– Deep belief networks– Stacked autoassociators– Deep Boltzmann machines
• Applications– Human motion modeling– Vision– Language
21
Outline
• Deep learning– Greedy level-wise training (supervised learning)– Restricted Boltzmann machine (RBM)– Deep belief networks– Stacked autoassociators– Deep Boltzmann machines
• Applications– Human motion modeling– Vision– Language
26
Outline
• Deep learning– Greedy level-wise training (supervised learning)– Restricted Boltzmann machine (RBM)– Deep belief networks– Stacked autoassociators– Deep Boltzmann machines
• Applications– Human motion modeling– Vision– Language
35
Outline
• Deep learning– Greedy level-wise training (supervised learning)– Restricted Boltzmann machine (RBM)– Deep belief networks– Stacked autoassociators– Deep Boltzmann machines
• Applications– Human motion modeling– Vision– Language
37
Conditional Restricted Boltzmann Machines (CRBM)
• http://www.uoguelph.ca/~gwtaylor/thesis/4/
38
Outline
• Deep learning– Greedy level-wise training (supervised learning)– Restricted Boltzmann machine (RBM)– Deep belief networks– Stacked autoassociators– Deep Boltzmann machines
• Applications– Human motion modeling– Vision– Language
42
Outline
• Deep learning– Greedy level-wise training (supervised learning)– Restricted Boltzmann machine (RBM)– Deep belief networks– Stacked autoassociators– Deep Boltzmann machines
• Applications– Human motion modeling– Vision– Language