Category Cohesion and DistinctivenessFavor Different Taxonomic Levels
Across Human Visual Cortex
Attention and Perception Lab
Marius Cătălin Iordan Michelle R. Greene Diane M. Beck Li Fei-Fei
SfN 2014
2Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
How do category representations change acrosstaxonomic levels in human visual cortex ?
2Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
dog
border collie
natural object
SUBORDINATE
SUPERORDINATE
General
Specific
BASIC
Rosch et al. (1976)
3Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
1. Experimental Setup
2. Similarity of Neural Patterns
3. Category Information at Each Taxonomic Level
How do category representations change acrosstaxonomic levels in human visual cortex ?
Is there a privileged taxonomic level ?
dog
border collie
natural object
SUBORDINATE
SUPERORDINATE
General
Specific
BASIC
4Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
FURNITURE
VEHICLE
INSTRUMENT
big, inanimate objectssubset of Rosch et al. (1976)
5Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
FURNITURE
VEHICLE
INSTRUMENT
big, inanimate objectssubset of Rosch et al. (1976)
FURNITURE
VEHICLE
INSTRUMENT
6Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
40
40
40
7Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Match-to-Category Behavioral Experiment
is this a “fighter plane” / “plane” / “vehicle” ?
8Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Match-to-Category Behavioral Experiment
9Taxonomic Levels in Human Visual Cortex
fMRI Experiment: Example “Fighter Plane” Block
+
8 images per block ⋇ 5 blocks per category ⋇ 27 categories
Iordan, Greene, Beck, Fei-Fei
FURNITURE
VEHICLE
INSTRUMENT
10Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
27 categories x
40 images each
passive viewing fMRI
experiment
3 taxonomic
levels
FURNITURE
VEHICLE
INSTRUMENT
10Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
27 categories x
40 images each
passive viewing fMRI
experiment
SUBORDINATE
3 taxonomic
levels
FURNITURE
VEHICLE
INSTRUMENT
10Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
27 categories x
40 images each
passive viewing fMRI
experiment
SUBORDINATE
Connolly et al. (2012), Konkle & Caramazza (2013)
3 taxonomic
levels
FURNITURE
VEHICLE
INSTRUMENT
SUPER-ORDINATE
FURNITURE
VEHICLE
INSTRUMENT
10Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
27 categories x
40 images each
passive viewing fMRI
experiment
SUBORDINATE
Connolly et al. (2012), Konkle & Caramazza (2013)
3 taxonomic
levelsRosch et al. (1976)
SUPER-ORDINATE
BASIC
11Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
1. Experimental Setup• stimulus sets & fMRI scanning
2. Similarity of Neural Patterns• within- and between-category similarity
3. Category Information at Each Taxonomic Level• MVPA decoding
How category representations change acrosstaxonomic levels in human visual cortex
A mean( )mean( )
Within-Category Similarity
A
Category Boundary Effect =
B_
Cohesion
Category Boundary Effect: Basic Level Example
Kriegeskorte et al. (2008)
12Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Between-Category Similarity
B
Distinctiveness
computed separately for each
taxonomic level
***
***
V1
***
***
***
V2
***
***
V3v
***
***
***
hV4
******
***
TOS
***
**
***
PPA
*****
RSC
******
**
LOC
0.01
0.02
0.03
0.04
0.00
CATE
GO
RY B
OUN
DARY
EFF
ECT
0.01
0.02
0.03
0.04
0.00
Subordinate Basic Superordinate
0.05 0.05
0.060.06
******
**
* n.s. *** *** **n.s. *n.s.*** *** ** n.s. *** n.s. n.s.
Category Boundary Effect: Results
13Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Early Visual Objects Scenes*** p < 0.001
* p < 0.05** p < 0.01
n = 17
***
***
V1
***
***
***
V2
***
***
V3v
***
***
***
hV4
******
***
TOS
***
**
***
PPA
*****
RSC
******
**
LOC
0.01
0.02
0.03
0.04
0.00
CATE
GO
RY B
OUN
DARY
EFF
ECT
0.01
0.02
0.03
0.04
0.00
Subordinate Basic Superordinate
0.05 0.05
0.060.06
******
**
* n.s. *** *** **n.s. *n.s.*** *** ** n.s. *** n.s. n.s.
Category Boundary Effect: Results
14Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Early Visual Objects Scenes*** p < 0.001
* p < 0.05** p < 0.01
n = 17
Gradual trade-off in favor of the basic level
***
***
V1
***
***
***
V2
***
***
V3v
***
***
***
hV4
******
***
TOS
***
**
***
PPA
*****
RSC
******
**
LOC
0.01
0.02
0.03
0.04
0.00
CATE
GO
RY B
OUN
DARY
EFF
ECT
0.01
0.02
0.03
0.04
0.00
Subordinate Basic Superordinate
0.05 0.05
0.060.06
******
**
* n.s. *** *** **n.s. *n.s.*** *** ** n.s. *** n.s. n.s.
Category Boundary Effect: Results
14Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Early Visual Objects Scenes*** p < 0.001
* p < 0.05** p < 0.01
n = 17
Gradual trade-off in favor of the basic level
0.00
0.02
0.04
0.06
-0.02
-0.04
-0.06
Basic Level ⎯ Subordinate Level
V1 V2 hV4V3v LOC
Cate
gory
Bou
ndar
y Ef
fect
Diff
eren
ce p = 0.0001, Friedman test
*** *** **
15Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Gradual trade-off in favor of the basic level
0.00
0.01
0.02
0.03
-0.01
-0.02
-0.03
Basic Level ⎯ Superordinate Level
V1 V2 hV4V3v LOC
Cate
gory
Bou
ndar
y Ef
fect
Diff
eren
ce p < 0.0001, Friedman test
** *
***
16Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Gradual trade-off in favor of the basic level
17Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
1. Experimental Setup• stimulus sets & fMRI scanning
2. Similarity of Neural Patterns• within- and between-category similarity
3. Category Information at Each Taxonomic Level• MVPA decoding
How category representations change acrosstaxonomic levels in human visual cortex
ns*ns
******
***
V1
*
******
***
V2
*
******
***
V3v
ns
***
***
***
hV4
* ns
***
**
***
TOS
* ns
***
**
***
PPA
*
******
**
RSC
*
***
******
*
LOC
0.03
0.06
0.09
0.12
Chance
PRO
PORT
ION
CORR
ECT
ABO
VE C
HANC
E
0.03
0.06
0.09
0.12
Chance
Subordinate Basic Superordinate
* ns ns
MVPA Decoding: Results
18Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Early Visual Objects Scenes*** p < 0.001
* p < 0.05** p < 0.01
n = 17
decode category at each taxonomic level independently
ns*ns
******
***
V1
*
******
***
V2
*
******
***
V3v
ns
***
***
***
hV4
* ns
***
**
***
TOS
* ns
***
**
***
PPA
*
******
**
RSC
*
***
******
*
LOC
0.03
0.06
0.09
0.12
Chance
PRO
PORT
ION
CORR
ECT
ABO
VE C
HANC
E
0.03
0.06
0.09
0.12
Chance
Subordinate Basic Superordinate
* ns ns
MVPA Decoding: Results
18Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
Early Visual Objects Scenes*** p < 0.001
* p < 0.05** p < 0.01
n = 17
decode category at each taxonomic level independently
Basic level is the optimal level of specificity in LOC
19Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
How category representations change acrosstaxonomic levels in visual cortex
FURNITURE
VEHICLE
INSTRUMENT
real-world object taxonomy with behavioral basic level advantage
0.00
0.01
0.02
0.03
-0.01
-0.02
-0.03
Basic Level ⎯ Superordinate Level
V1 V2 hV4V3v LOC
Cate
gory
Bou
ndar
y Ef
fect
Diff
eren
ce p = 0.0001
** *
***
gradual trade-off between basic and other levels in favor of the former
basic level is optimal level of specificity decodable from LOC patterns
0.00
0.01
0.02
0.03
-0.01
-0.02
-0.03
Basic Level ⎯ Superordinate Level
V1 V2 hV4V3v LOC
Cate
gory
Bou
ndar
y Ef
fect
Diff
eren
ce p = 0.0001
** *
***
FURNITURE
VEHICLE
INSTRUMENT
20Taxonomic Levels in Human Visual Cortex Iordan, Greene, Beck, Fei-Fei
real-world object taxonomy with behavioral basic level advantage
gradual trade-off between subordinate and basic levels in favor of the latter
basic level is optimal level of specificity decodable from LOC patterns
basic level representation may be anemergent property of the visual system
How category representations change acrosstaxonomic levels in visual cortex
high-level visual areas may share computationsgeared at specifically separating categories
categorization may be part of visual processing
Category Cohesion and DistinctivenessFavor Different Taxonomic Levels
Across Human Visual Cortex
Attention and Perception Lab
Marius Cătălin Iordan Michelle R. Greene Diane M. Beck Li Fei-Fei
