Date post: | 21-Jan-2018 |
Category: |
Technology |
Upload: | embedded-vision-alliance |
View: | 79 times |
Download: | 0 times |
Methods for Understanding How
Deep Neural Networks Work
Dr. Wojciech Samek
Head of Machine Learning Group
Fraunhofer Heinrich Hertz Institute
Vision Industry and Technology Forum 6th Sep 2017, Hamburg, Germany
Unbeatable AI Systems
AlphaGo beats Go
human champ
Deep Net outperforms humans
in image classification
Deep Net beats human at
recognizing traffic signs
DeepStack beats
professional poker players
Computer out-plays
humans in "doom"
Autonomous search-and-rescue
drones outperform humans
IBM's Watson destroys
humans in jeopardy
Elon Musk (2017): “AI will be able to beat humans at
EVERYTHING by 2030.”
Unbeatable AI Systems
You may heard such promises before
H. A. Simon (1965): "Machines will be capable, within
twenty years, of doing any work a man can do."
M. Minsky (1970): "In from three to eight years we will
have a machine with the general intelligence of an
average human being."
Computing power
Very large deep neural networks Information (implicit)
Solve task
Huge volumes of data
Past promises did not become reality, but today …
650,000 neurons, 60 million parameters
Deep Neural Networks
Why are these models so successful ?
- “end-to-end” training
- feature hierarchy
- distributed / reusable representation
Deep Neural Networks: “End-to-End” Training
“Slippery road”
Traditional Approaches
input feature extraction classification output
Deep Learning
“Slippery road”
input feature extraction + classification output
(Source: Yann LeCun
Deep Neural Networks: Feature Hierarchy
Low level
features
Mid level
features
High level
featuresClassifier
“Car”
Hierarchical information processing in the brain
Deep Neural Networks: Feature Hierarchy
(Source: Simon Thorpe)
Deep Neural Networks: Reusable Representation
Features extracted from “car” images can be used for
other tasks / classification of other objects.
objects with
roundish shape
Very large deep neural networks Information (implicit)
What did the neural network learn ?
How does it solve the problem ?
Can we extract human interpretable information ?
Do we understand the AI ?
Can we trust these “end-to-end” trained black box algorithms ?
Black-Box Systems
(More information: https://neil.fraser.name/writing/tank/)
Already in the 1980s neural networks have been used for
classification tasks, e.g., identify tanks in the forest.
Training Dataset:
- 100 photos of tanks hiding behind trees
- 100 photos of trees with no tanks.
First results were promising, but did
network solve the task correctly ?
Understand “weaknesses” of classifier
Detect biases / bring in human intuition.
Learn from the learning machine
“I've never seen a human play this move.” (Fan Hui)
Wrong decisions can be harmful and costly.
Verify that system works as expected
Interpretability in the sciences
The “why” often more important than the prediction.
Compliance to legislation
“right to explanation”, retain human decision …
We need to “open” Black-Box Systems
“rooster”
We developed a general method to explain
individual classification decisions.
Main idea:
Opening the Black-Box
Layer-wise Relevance Propagation (LRP)(Bach et al., PLOS ONE, 2015)
“measure how much each pixel
contributes to the overall prediction”
Opening the Black-Box
Explanation
cat
rooster
dog
?
Intuition: Redistribute relevance proportionally
- a neuron gets more relevance if it’s more activated
- more relevance flows over strong connections
Opening the Black-Box
Explanation
cat
rooster
dog
Theoretical interpretation: Deep Taylor Decomposition(Montavon et al., Pattern Recognition, 2017)
[number]: explanation target class
red color: evidence for prediction
blue color: evidence against prediction
what speaks for / against
classification as “3”
what speaks for / against
classification as “9”
Opening the Black-Box
Application: Compare Classifiers
Two classifiers
- similar classification accuracy on horse class
- but do they solve the problem similarly ?
(Lapuschkin et al., IEEE CVPR, 2016)
Application: Compare Classifiers
Images from
PASCAL VOC
2007 dataset
Interpretability helps to
- understand biases / flaws in the data and weaknesses of the classifier
- verify that system works as expected
Application: Compare Classifiers
GoogleNet focuses on
faces of animal.
—> suppresses background noise
(Binder et al., ICML Visualization
Workshop, 2016)
Interpretability helps to
- compare and select models / architectures
Application: Measure Context Use
classifier
how important
is context ?
how important
is context ?
relevance outside bbox
relevance inside bbox
importance
of context=
(Lapuschkin et al., IEEE CVPR, 2016)
Image Fisher Vector DNN
boat
boat
Co
nte
xt u
se
airplane
airplane
sofachair chair sofa
Application: Measure Context Use
Interpretability helps to
- extract additional classification-related information
Application: Video Analysis
Motion vectors can be extracted
from the compressed video
-> allows very efficient analysis
Application: Video Analysis
Interpretability helps to
- extract additional classification-related information
- compare features
Other Applications
Identifying age-related features
(Arbabzadah et al., GCPR, 2016)
(Sturm et al., J Neuroscience Methods, 2016)
Brain-Computer Interfacing
Identifying relevant words in text
(Arras et al., PLOS ONE, 2017)
Detection of morphing attacks
(Seibold et al., IWDW, 2017)
In many problems interpretability as important as prediction
(trusting a black-box system may not be an option).
Use in practice
- verify predictions, detect biases and flaws, debug models
- compare and select architectures, understand and improve models
- extract additional information, perform further tasks
We have a powerful, mathematically well-founded method to explain
individual predictions of complex machine learning models.
More research needed on how to compare and evaluate all the different
aspects of interpretability.
Summary
Thank you for your attention
For more information, check out our tutorial paper:
Montavon et al. “Methods for Interpreting and Understanding Deep Neural Networks”
https://arxiv.org/abs/1706.07979