Visual Intelligence
Prof. Rita Cucchiara
AimageLab, Dipartimento di Ingegneria «Enzo Ferrari»Università di Modena e Reggio Emilia, ItalyDirector of the National CINI Lab AIIS
“Like the steam engine or electricity in the past, AI is transforming our world, our society and our industry.
Growth in computing power, availability of data and progress in algorithms have turned AI into one of the most strategic technologies of the 21st century.”
Artificial Intelligence for Europe - Brussels, 25.4.2018
Artificial Intelligence
“AI refers to systems that display intelligent behaviour by analysing their environment and taking actions – -with a certain degree of autonomy- to achieve a specific goal.”
Artificial Intelligence for Europe - Brussels, 25.4.20182018
Artificial Intelligence
Machine Learning
Deep Learning
Game theory
Knowledge representation
Automated Reasoning
Logics
Computer Vision
Pattern Recognition
Natural Language Processing
Cognitive Robotics
IntelligentIoT
Speech Recognition
Multi-Agents
Fuzzy systems
…systems that display intelligent behaviour by analysing their environment and taking actions
National Lab CINI AIIS:
51 nodes ( 47 universities, CNR, IIT, FBK)910 members
>100 Labs>700 projects>80 spinoff
National CINI Lab AIISArtificial intelligence andIntelligent systems
UNIMORE Modena , Italy
AImageLab Dipartimento di Ingegneria «Enzo Ferrari» & Modena Technopole
AIRI AI Research & Innovation Center; 36 People working in AI, ML and CV
ComputerVision
PatternRecognition
MachineLearning
Deep Learning with (Artificial)
Neural NetworksIntelligentInference
Action
Display intelligent behaviour
Analyse the environment
Take actions
Visual Intelligence
Where do you put your attention?
What do you predict while driving?
Saliency or task-driven attention?
Visual intelligence
Visual Intelligence for a better human and machine mutual Comprehension
..l’arte suprema di saper vedere..
Visual intelligencecan helpmachines
Interacting with AI
Viual Intelligence
Interacting with AI
by collaborative robotics
Interacting with AI
AI, data understanding and visual Intelligence
can helphumans in controlling Machines in cyber
Visual Intelligencefor secuirty
Imagine to haveImagination
Fakes from Art
Art2Realby GANs
[M. Tomei, M. Cornia, L. Baraldi, R. Cucchiara. “Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation”. CVPR 2019]
AI recognizes
YOU and your
ancestors!
Visual Intelligence is Imagination and Hallucination
DeepFakes
https://www.cnn.com/interactive/2019/01/business/pentagons-race-against-deepfakes/
REAL FAKE
AI can helpdesigners
AI can helpall of usin security and smart cities
PrEVUE
..l’arte suprema di saper vedere..
Saliency
SALIENCY [Itti Koch PAMI 1989]
[ Itti and Koch PAMI ’89, Nature Reviews 2001]“Saliency map”: an image map representing areas of saliency
Saliency: data-driven, perceptual or semantics driven?
• SAM
Saliency Attentive Model (SAM) @ AImageLab
M.Cornia, L.Baraldi, G.Serra, R.Cucchiara
Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model
IEEE Transactions on Image Processing, 2018Ranked #1 at LSUN Competition CVPR2017
SAM
Refine with an iterative (LSTM-based) model the saliency detection
Trained with generic images
(SALICON MIT300).. Now
SAM can explore the world
Saliency and Attention
[A.Palazzi,D. Abati, Davide S. Calderara, and R.Cucchiara Predicting the Driver's Focus of Attention: the DR(eye)VE Project IEEE Transactions on Pattern Analysis and Machine Intelligence 2018]
Tell me what you see
Saliency and Attentionin image captioningfrom Vision to Language
Saliency and Captioning
[M Cornia, L. Baraldi, G. Serra, R. Cucchiara Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention ACM TOMM 2018 ]
ResNet -50, trained with Imagenet;
SAM Saliency /context detectionSoft-attentive LSTM
Text generation LSTM
Toward an explainable AI: What the Machine pays attention of when is describing the scene
Attention to details
Where people are, what people are doing, what the people see.
Ball in hand?
Learning to put Attention in details
People detection
People Join detection
People Join detection
[M. Fabbri, F. Lanzi , S. Calderara, R. Cucchiara, Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World ECCV2018]
Hallucinating occluded joints
Hallucinating third dimension of (occluded) joints
For Machine and Human Mutual Comprehension
Human Behavior Understanding
Future HMI
ControllableExplainableCorrectable
Captioning for Explainable Reasoning
A penny for your thoughts
Captioning for Explainable Reasoning
A penny for your thoughts
Captioning for Explainable Reasoning
Imagine to understandwhat the robot sees
Captioning for Explainable Reasoning
L. Baraldi, R.CucchiaraExplainable Robot-World interactionArxiv 2019.
AI ALGORITHMS & ARCHITECTURESAI DATAAI HARDWARE
What do you need?
What you don’t need..
1 ignorance2 negligence3 malevolency4 skepticism
Ma c’e’ una magia che e’ opera divinaLa’ dove la scienza di Dio si manifestaattraverso la scienza dell’uomo…
(U.Eco 1984)
THANKS
Thanks to all AImageLab UNIMORE