91
PART-WHOLE RELATIONSHIPS IN GESTALT PSYCHOLOGY AND MODERN VISION SCIENCE JOHAN WAGEMANS LABORATORY OF EXPERIMENTAL PSYCHOLOGY UNIVERSITY OF LEUVEN IEA PARIS, FEBRUARY 29, 2012
PART-WHOLE RELATIONSHIPS IN GESTALT PSYCHOLOGY AND MODERN
VISION SCIENCE
JOHAN WAGEMANS
LABORATORY OF EXPERIMENTAL PSYCHOLOGY UNIVERSITY OF LEUVEN
IEA PARIS, FEBRUARY 29, 2012
PSYCHOLOGY AND VISION SCIENCE
FOREWORD
(Experimental) Psychology
• Central position in humanities by subject matter
• Peripheral position in humanities by methododology
• … and other aspects:
– research style – publication style
• Central position as a hub scientific discipline:
– life sciences, biomedical sciences – natural sciences – engineering – arts
Visual perception
• Our window on the world – we are visual animals – 1/3 of our brain areas respond to visual
stimuli
• Our window on the mind – mind/brain problem – consciousness – … one of the last remaining mysteries in
science
The power of visual perception
• Some striking visual phenomena – apparent motion – aftereffects – visual illusions – impossible figures – pareidolia
• http://www.michaelbach.de/ot/
Vision science
• Understanding/explaining visual perception
• Different levels: – phenomenology – psychological processes and factors – computational mechanisms – neural mechanisms
• Central position as a hub scientific discipline:
– life sciences, biomedical sciences: ophthalmology, visual neuroscience
– natural sciences: geometry, optics – engineering: computer vision, robotics, computer
graphics – arts: visual arts
Central question
• “Why do things look the way they do?” (Koffka, 1935) – because of the way things are – because of the way we are
– objectivism-subjectivism – empiricism-rationalism – mentalism-materialism
Central project
• “Perceptual organization in the context of the dynamic, hierarchical visual brain” – Methusalem funding (2008-2015) – research group of ± 20 people
• Overall aims
• Overall themes
Central issue
• Part-whole relationships – in visual perception (Gestalt psychology) – in visual cortex (visual neuroscience)
Some examples
Some examples
Some examples
Overview
• Part 1: historical and conceptual background
• Part 2: some recent empirical studies – bistable diamonds – motion silencing – configural-superiority effect
• Part 3: general discussion
HISTORICAL AND CONCEPTUAL BACKGROUND
PART 1
How Gestalt psychology started
Wertheimer, M. (1912). Experimentelle Studien über das Sehen von Bewegung. Zeitschrift für Psychologie, 61, 161-265. phi motion
Steinman, R. M., Pizlo, Z., & Pizlo, F. J. (2000). Phi is not beta, and why Wertheimer’s discovery launched the Gestalt revolution. Vision Research, 40, 2257-2264.
http//psych.purdue.edu/magniphi/ key role
phi as pure motion, not a displacement between two objects
phi as a process, “an across in itself”, that cannot be composed from the usual optical contents
Max Wertheimer (1880-1943)
A radical vision
emerging Gestalt theory not Gestalt qualities added to the primary sensations not Gestalts as more than the sum of the parts but Gestalts as different from the sum of the parts often the whole is grasped even before the individual parts
enter consciousness a structured unit emerges as a whole
Berlin school versus Graz school (Meinong, von
Ehrenfels, Benussi)
psychological facts and physiological hypotheses went hand-in-hand
Some early Gestalt history
Koffka, K. (1915). Beitrage zur Psychologie der Gestalt. III. Zur Grundlegung der Wahrnehmungspsychologie. Eine Auseinandersetzung mit V. Benussi. Zeitschrift für Psychologie, 73, 11-90. implications of this view primary relations
no longer stimulus ~ sensation but stimulus pattern ~ perceived whole
perceived wholes not constructed in the mind from elementary sensations but direct experience-correlates emerging in the brain
Kurt Koffka (1886-1941)
Some early Gestalt history
Köhler, W. (1920). Die physischen Gestalten in Ruhe und im stationären Zustand. Eine natur-philosophische Untersuchung. Braunschweig, Germany: Vieweg.
decisive step: real physical Gestalts in the brain
strong Gestalts the mutual dependence among the parts is so great that
no displacement or change of state is possible without influencing all the other parts of the system
in fact: there are no parts at all, only interacting moments of structure that carry one another
psychophysical isomorphism psychological facts and the brain events that underlie
them are similar in all of their structural characteristics in fact: the brain described as a self-organizing physical
system
Wolfgang Köhler
(1887-1967)
Historical “coincidence”
• many important Gestalt psychologists were Jew (Wertheimer, Koffka, Arnheim, Lewin, …) and had to move out of Germany (Nazi regime from 1933)
• in the U.S. – they did not have many students – they were faced with a different intellectual climate
(behaviorism)
• as a result: Gestalt psychology declined naturally
From speculation to facts
Köhler, W., & Held, R. (1949). The cortical correlate of pattern vision. Science, 110, 414-419.
first recordings of visual currents, picked up by an electrode at the scalp of human observers
“electrical field theory”
Lashley, K. S., Chow, K. L., & Semmes, J. (1951). An examination of the electrical field theory of cerebral integration. Psychological Review, 58, 123-136.
more direct test of electrical field theory rationale: insulate part of a cortical field and test for consequent
disturbances of function metallic strips and pins inserted in macaque cortex almost no effect on post-operative retention of object discrimination devastating blow to
electrical field theory basic isomorphism postulate of Gestalt theory
Single-neuron doctrine
Hubel & Wiesel: big success
Barlow, H. (1972): Single units and sensation: A neuron doctrine for perceptual psychology. Perception, 1, 371-394.
our perceptions are caused by the activity of a small number of neurons the activity of a single neuron is related quite simply to our subjective
experience
reductionist, elementalist approach which Gestalt theorists had banished
mapping of responses of single neurons in LGN, striate and extrastriate cortex in cat and monkey
tuning to simple stimulus attributes (e.g., orientation)
single neurons interpreted as “feature detectors” (e.g., line detectors, edge detectors)
Further developments
single-unit recording flourished tuning properties of different types of cells in different
areas of the brain functional specialization hierarchical organization
confirmed in human fMRI (modules, maps) standard view
Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1, 1-47.
Grill-Spector, K., & Malach, R. (2004). The human visual cortex. Annual Review of Neuroscience, 27, 649-677.
Serre, T., Oliva, A., & Poggio, T. (2007). A feedforward architecture accounts for rapid categorization. Proceedings of the National Academy of Science of the USA, 104, 6424-6429.
Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1, 1-47.
Grill-Spector, K., & Malach, R. (2004). The human visual cortex. Annual Review of Neuroscience, 27, 649-677.
Serre, T. et al. (2007). A feedforward architecture accounts for rapid categorization. PNAS, 104, 6424-6429.
Re-emergence of Gestalt issues
surround influences from outside classic receptive field (cRF) Allman, J., Miezin, F., & McGuinness, E. (1985). Direction- and
velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception, 14, 105-126.
neural responses to illusory contours
von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260-1262.
neural responses to figure-ground organization
Zhou, H., Friedman, H. S., & von der Heydt, R. (2000). Coding of border-ownership in monkey visual cortex. Journal of Neuroscience, 20, 6594-6611.
Allman, J. et al. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT).
Perception, 14, 105-126.
Allman, J. et al. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT).
Perception, 14, 105-126.
von der Heydt, R. et al. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260-1262.
Zhou, H. et al. (2000). Coding of border-ownership in monkey visual cortex. Journal of Neuroscience, 20, 6594-6611.
A modern view on a radical vision
more general models Hochstein, S., & Ahissar, M. (2002). View from the top:
Hierarchies and reverse hierarchies in the visual system. Neuron, 36, 791-804.
Bar, M. et al. (2006). Top-down facilitation of visual recognition. Proceedings of the National Academy of Science of the USA, 103, 449-454.
interesting characteristics from viewpoint of Gestalt theory “wholes” come first highly interactive highly dynamic not limited to visual areas
Hochstein, S., & Ahissar, M. (2002). View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron, 36, 791-804.
Bar, M. et al. (2006). Top-down facilitation of visual recognition. PNAS, 103, 449-454.
Interim conclusions
• “parts” versus “wholes” constitutes a prominent theme in vision science, from Gestalt psychology up until today
• the pendulum seems to have swung back-and-forth
• new attempts towards a synthesis are emerging
• new research inspired by these views
Some conceptual points
• “parts” and “wholes” are used generically (not
just parts of objects)
• parts: features, components, constituents • wholes: objects, compositions, configurations,
Gestalts
• parts are generally smaller than wholes
• hierarchical relationship: – parts belong to wholes – wholes consist of parts
BISTABLE DIAMONDS: MURRAY ET AL. (2002) FANG ET AL. (2008)
PART 2A
Murray et al.: Key papers
• Murray, S. O., Kersten, D., Olshausen, B. A.,
Schrater, P., & Woods, D. L. (2002). Shape perception reduces activity in human primary visual cortex. Proceedings of the National Academy of Sciences, 99, 15164-15169.
• Fang, F., Kersten, D., & Murray, S. O. (2008).
Perceptual grouping and inverse fMRI activity patterns in human visual cortex. Journal of Vision, 8(7):2, 2-9. doi: 10.1167/8.7.2
Demonstration
Demonstration
Demonstration
Nice features
• perceptual bi-stability: – “parts” seen to move vertically – “whole” seen to move horizontally
• switching relatively slow, perceptual states rather clear
• stable individual differences
• studied rather extensively at psychophysical level, e.g. – Lorenceau, J. & Shiffrar, M. (1992). The influence of
terminators on motion integration across space. Vision Research, 32, 263-273.
Murray et al.: Design
• present bistable diamonds
• ask observers to indicate perception of “parts”
(line segments) or “whole” (diamond)
• record BOLD responses (fMRI) in different areas and relate these to the reported percepts
Murray et al.: Results
Murray et al.: Results
Interim conclusions
• convincing demonstration of inverse activity patterns in V1 and LOC
• interpretation: perception of “parts” suppressed by perception of “whole”
MOTION SILENCING: SUCHOW & ALVAREZ (2011) POLJAC, DE-WIT, & WAGEMANS (2012)
PART 2B
Suchow & Alvarez (2011)
• Suchow, J. W., & Alvarez, G. A. (2011). Motion silences awareness of visual change. Current Biology, 21(2), 140-143. doi:10.1016/j.cub.2010.12.019
• “Best Illusion of the Year 2011”
Demonstration
Demonstration
More demonstrations
Methods
• Stimuli: – 100 dots – first stationary, then rotating back and forth for 30° – 2 phases alternating every 3 s
• Task: – observers had to adjust the rate of change during the
stationary phase to match the apparent rate of change in the moving phase
– rate of change (“silencing factor”) between 0.1 (static perceived as changing slower) and 10 (static perceived as changing faster)
Results
Interpretation
• Suchow & Alvarez:
– local mechanisms with small receptive fields – because a fast-moving object spends little time at any
one location, a local detector is afforded only a brief window in which to assess the changing object
• alternative interpretation: – objecthood – when a good “whole” is formed, the details of the
“parts” are fundamentally less accessible to conscious perception
Our study
• motivation: to test this alternative interpretation and more general idea with biological motion
• Poljac, E., de-Wit, L., & Wagemans, J. (2012). Perceptual wholes can reduce the awareness of their changing parts. Cognition, in press. doi:10.1016/j.cognition.2012.01.001
Why biological motion?
• a prototypical case of a complex hierarchical stimulus (Johansson, 1973; Cutting & Proffitt, 1982) – multiple elements, each with their own spatio-temporal
trajectories – organized quickly and efficiently in a hierarchical
configuration, in which the motion of the local elements are coded relative to a more global structural description
• the perceptual Gestalt is constructed automatically
by the visual system (Thornton & Vuong, 2004)
• inversion allows control over low-level motion trajectories (Sumi, 1984; Pavlova & Sokolov, 2000)
Demonstrations
Demonstrations
Demonstrations
Methods
• Stimuli:
– motion captured point-light treadmill walkers (Vanrie & Verfaillie, 2004)
– 70 colored dots (“confetti walker”) – upright, inverted, phase-scrambled
• Task: adjust rate of change in the test figure until it matches the range of change in the comparison figure
• Experiment 1: all static – comparison figure: scrambled – test figures: upright, inverted or scrambled
• Experiment 2: dynamic and static – comparison figure: scrambled – test figures: upright or inverted
Results: Experiment 1
Results: Experiment 2
Discussion
• on top of the effect of static vs moving, there is
a clear effect of configurality (“goodness” of the whole percept)
• cost of objecthood: the more strongly the parts are integrated into the perception of a whole object, the less accessible the changing features of the parts are (e.g., also embedded figures)
Interim conclusions
• two sides of the same coin:
– the rapid extraction of a perceptual Gestalt – the inaccessibility of the parts that make up
that Gestalt
• general principle: – human vision provides only the most useful
level of abstraction to conscious awareness
CONFIGURAL-SUPERIORITY EFFECT KUBILIUS ET AL. (2011)
PART 2C
Point of departure
• configural-superiority effect
– Pomerantz, J. R., Sager, L. C., & Stoever, R. J. (1977). Perception of wholes and their component parts: Some configural superiority effects. Journal of Experimental Psychology: Human Perception and Performance, 3, 422-435.
– Pomerantz, J. R., & Portillo, M. C. (2011). Grouping and emergent features in vision: Toward a theory of basic Gestalts. Journal of Experimental Psychology: Human Perception and Performance, 37, 1331-1349.
Kubilius et al. (2011)
• Kubilius, J., Wagemans, J., & Op de Beeck, H. P. (2011). Emergence of perceptual Gestalts in the human visual cortex: The case of the configural superiority effect. Psychological Science, 22(10), 1296-1303.
• behavioral results
• fMRI decoding results
Behavioral results
parts corner whole
Scanning protocol
fMRI results: Retinotopic mapping
fMRI results: decoding
Interim conclusions
• behavioral configural-superiority effect
• neural configural-superiority effect: – better coding of “wholes” than “parts” in higher shape-
selective regions – better coding of “parts” than “wholes” in lower-level
retinotopic regions
• general conclusions: – at least some Gestalts emerge only at higher stages of
visual information processing – feedforward processing may be sufficient to produce
some Gestalts
GENERAL DISCUSSION
PART 3
Some tentative conclusions
• first set of studies (bistable diamond, motion silencing): – “wholes” dominate and “parts” disappear from experience – “wholes” emerge in higher areas of the brain and encoding of “parts” is
then suppressed
• second set of studies (configural superiority effect): – functional “wholes” also arise spontaneously and “parts” become less
functional – but still, the encoding of these “wholes” at higher levels of the cortical
hierarchy does not suppress the encoding of the “parts”
• more generally: – not all Gestalts are equal – we seem to get better handles to understand the relations between
“parts” and “wholes” in visual perception – a lot of work remains to be done
SOME IEA-PARIS ACTIVITIES
AFTERWORD
Overview
• Writing projects
• Invited lectures and symposia
Writing projects
• Wagemans, J., Elder, J. H., Kubovy, M., Palmer, S. E., Peterson, M. A., Singh,
M., & von der Heydt, R. (2012). A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychological Bulletin. Revision submitted.
• Wagemans, J., Feldman, J., Gepshtein, S., Kimchi, R., Pomerantz, J. R., van der Helm, P., & van Leeuwen, C. (2012). A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations. Psychological Bulletin. Revision submitted.
• Wagemans, J. (2012a). Contours of outline shapes. In S. Gepshtein & L. T. Maloney (Eds.),The Oxford Handbook of Computational Perceptual Organization. New York: Oxford University Press. Manuscript submitted.
• Wagemans, J. (2012b). Shape similarity and categorization. In S. Gepshtein & L. T. Maloney (Eds.),The Oxford Handbook of Computational Perceptual Organization. New York: Oxford University Press. Manuscript submitted.
• Wagemans, J. (2012c). Perceptual multi-stability in figure-ground organization. In S. Gepshtein & L. T. Maloney (Eds.),The Oxford Handbook of Computational Perceptual Organization. New York: Oxford University Press. Manuscript submitted.
• Wagemans, J. (Ed.), (2013). The Oxford Handbook of Perceptual Organization. Oxford, U.K.: Oxford University Press. Book proposal approved.
Symposia 2012
• Rovereto, Italy (Feb 19-23): “The science of experiential qualities and spaces”
• Leuven, Belgium (March 14-16): “Understanding intrinsic simplicity and statistical likelihood in visual Gestalts”
• Mannheim, Germany (April 1-4, TEAP): “Experimental methods in perceptual organization”
• Naples, Florida, USA (May 11, VSS): “Part-whole relationships in visual cortex”
• ??? Paris, France (May 28-29): “Visual perception and visual arts”
THANK YOU
WWW.GESTALTREVISION.BE
Part-Whole Relationships �in Gestalt psychology and modern vision science�Psychology and vision science(Experimental) PsychologyVisual perceptionThe power of visual perceptionSlide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Vision scienceCentral questionCentral projectSlide Number 20Slide Number 21Central issueSome examplesSome examplesSome examplesOverviewhistorical and conceptual backgroundHow Gestalt psychology startedA radical visionSome early Gestalt historySome early Gestalt historyHistorical “coincidence”From speculation to factsSingle-neuron doctrineFurther developmentsFelleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1, 1-47.Grill-Spector, K., & Malach, R. (2004). The human visual cortex. �Annual Review of Neuroscience, 27, 649-677.Serre, T. et al. (2007). A feedforward architecture accounts for rapid categorization. PNAS, 104, 6424-6429.Re-emergence of Gestalt issuesAllman, J. et al. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception, 14, 105-126.Allman, J. et al. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception, 14, 105-126.von der Heydt, R. et al. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260-1262.Zhou, H. et al. (2000). Coding of border-ownership in monkey visual cortex. Journal of Neuroscience, 20, 6594-6611.A modern view on a radical visionHochstein, S., & Ahissar, M. (2002). View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron, 36, 791-804.Bar, M. et al. (2006). Top-down facilitation of visual recognition. �PNAS, 103, 449-454.Interim conclusionsSome conceptual pointsBistable diamonds:�Murray et al. (2002)�Fang et al. (2008)Murray et al.: Key papersDemonstrationDemonstrationDemonstrationNice featuresMurray et al.: DesignMurray et al.: ResultsMurray et al.: ResultsInterim conclusionsMotion silencing:�Suchow & alvarez (2011)�Poljac, De-wit, & wagemans (2012)Suchow & Alvarez (2011)DemonstrationDemonstrationMore demonstrationsMethodsResultsInterpretationOur studyWhy biological motion?DemonstrationsDemonstrationsDemonstrationsMethodsResults: Experiment 1Results: Experiment 2DiscussionInterim conclusionsConfigural-superiority effect�Kubilius et al. (2011)Point of departureKubilius et al. (2011)Behavioral resultsScanning protocolfMRI results: Retinotopic mappingfMRI results: decodingInterim conclusionsGeneral discussionSome tentative conclusionsSome IEA-Paris activities OverviewWriting projectsSymposia 2012Thank You
