Institute of Electrical Measurement and Measurement Signal Processing 1 Axel Pinz WS 2017/18 Image and Video Understanding 3 To Conclude: Vision in terms of … • …Neurophysiology Receptive fields Left/right hemisphere Visual pathway, packing problem, columns, complementary features • …Cognitive psychology – Perceptual grouping – Bottom-up vs. top-down processes – Optical illusions – Hemispheres, motion perception, … • …Information processing The “Marr Paradigm”
Transcript
Institute of Electrical Measurement and Measurement Signal Processing
1
Axel Pinz WS 2017/18 Image and Video Understanding 3
To Conclude: Vision in terms of …
• …Neurophysiology Receptive fields
Left/right hemisphere
Visual pathway, packing problem, columns, complementary features
• …Cognitive psychology– Perceptual grouping
– Bottom-up vs. top-down processes
– Optical illusions
– Hemispheres, motion perception, …
• …Information processing
The “Marr Paradigm”
Institute of Electrical Measurement and Measurement Signal Processing
2
Axel Pinz WS 2017/18 Image and Video Understanding 3
David Marr [1945-1980] – Vision [1982]
VISION
A Computational Investigation into the Human
Representation and Processing of Visual Information
Freeman Co., 1982NEUROBIOLOGY
Institute of Electrical Measurement and Measurement Signal Processing
3
Axel Pinz WS 2017/18 Image and Video Understanding 3
David Marr – Vision
Institute of Electrical Measurement and Measurement Signal Processing
4
Axel Pinz WS 2017/18 Image and Video Understanding 3
David Marr – Vision
Institute of Electrical Measurement and Measurement Signal Processing
5
Axel Pinz WS 2017/18 Image and Video Understanding 3
David Marr – Vision
“What does it mean, to see? The plain man’s answer
(and Aristotle’s, too) would be, to know what is where by
looking. In other words, vision is the process of
discovering from images what is present in the world, and
where it is.” (p.3, 1st paragraph of General Introduction)
Vision Image Understanding: To know what is where.
D. Marr
Video Understanding: What is where and when?
(borrowed from D. Marr) in space and time
3D 2D, Reconstruction vs. recognition
4D 3D
Institute of Electrical Measurement and Measurement Signal Processing
6
Axel Pinz WS 2017/18 Image and Video Understanding 3
Marr – Vision: Emphasis on Reconstruction
Institute of Electrical Measurement and Measurement Signal Processing
7
Axel Pinz WS 2017/18 Image and Video Understanding 3
Marr – Vision:
Emphasis on
Reconstruction
Institute of Electrical Measurement and Measurement Signal Processing
8
Axel Pinz WS 2017/18 Image and Video Understanding 3
The “Marr Paradigm” – “Computational Framework”
Stone (Vision and Brain, 2012): “computational framework” suggests:
“Vision works like a computer”. Better: “informational framework” …“… because Marr was keen to emphasize the nature of information being processed
without necessarily referring to the particular machinery (e.g., neurons or chips)…”
Institute of Electrical Measurement and Measurement Signal Processing
9
Axel Pinz WS 2017/18 Image and Video Understanding 3
The “Marr Paradigm” – Analogy with Flying
Marr (p.27): “Importance of Computational Theory”
“… an algorithm is likely to be understood more readily by understanding
the nature of the problem being solved than by examining the mechanism
(and the hardware) in which it is embodied.”
“… trying to understand perception by studying only neurons is like trying to
understand bird flight by studying only feathers: It just cannot be done.”
First understand aerodynamics, then think about structures of feathers,
shape of wings etc.
Wright brothers 1902
(from [Stone, 2012])
Institute of Electrical Measurement and Measurement Signal Processing
10
Axel Pinz WS 2017/18 Image and Video Understanding 3
The “Marr Paradigm” – “Computational Framework”
3D surface shape by
finding surface normals
from shading information
SfS
0 90 180
Greylevels
Surface
normals
Neurons
A single CPU
Multicore CPUs
GPUs
“…frogs passing cupcakes.” [Stone]
Institute of Electrical Measurement and Measurement Signal Processing
11
Axel Pinz WS 2017/18 Image and Video Understanding 3
David Marr – Vision
Representational Framework
Primal sketch
2-1/2D sketch
3D model
Institute of Electrical Measurement and Measurement Signal Processing
12
Axel Pinz WS 2017/18 Image and Video Understanding 3
David Marr – Vision
Representational Framework
Primal sketch
2-1/2D sketch
3D model
“viewer centered”
“object centered”
Institute of Electrical Measurement and Measurement Signal Processing
13
Axel Pinz WS 2017/18 Image and Video Understanding 3
Marr – Primal Sketch
Compare today’s interest point, line, edge detection, etc.
