Date post: | 26-Dec-2015 |
Category: |
Documents |
Upload: | brice-carson |
View: | 227 times |
Download: | 0 times |
USING BRAIN DATA TO IDENTIFY CATEGORY DISTINCTIONS
• Studies of brain-damaged patients have been shown to provide useful insights in the organization of conceptual knowledge in the brain
• Some patients are unable to identify or name man made objects and others may not be able to identify or name natural kinds (like animals)– Warrington and Shallice 1984, Caramazza & Shilton 1998
• fMRI has been used to identify these distinctions in healthy patients as well– E.g., Haxby et al 2000, Martin & Chao 2003
• See, e.g., Mahon & Caramazza 2011, Martin 2007 for review
Warrington & Shallice 1984
• Warrington and Shallice (1984) reported a patient called JBR who following an acute lesion to the left temporal lobe (as a result of herpes encephalitis) had a selective deficit when asked to name pictures from just one semantic category – living things.
• By contrast JBR was able to name non-living objects very well including those with low frequency names such as ‘accordion’ that were matched for the number of letters in the name and the visual complexity of the object.
• Other patients have shown opposite pattern
Evidence from semantic category deficits
• Modality-specific deficits – Patients are unable to name visually presented objects, but can name
them from other modalities and can access other semantic information about visually presented stimuli (Beauvois, 1982)
– Other visual processing is fine.
• Category-specific deficits (e.g., Warrington & McCarthy, 1983, 1987; Warrington & Shallice, 1984; Gainotti & Silveri, 1996)
– Patients show impairments in processing living things vs. man-made objects and vice versa.
– Interesting exceptions: fruits, vegetables & other foods; musical instruments
Study
• 16 adults (8M, 8F) participated in a PET (positron emission tomography) study.– Involves injecting subject with a positron emitting radioactive
substance (dye) – Regions with more metabolic activity will absorb more of the
substance and thus emit more positrons– Positron-electron collisions yield gamma rays, which are detected
• Increased rCBF (regional changes in cerebral blood flow) was measured– When subjects viewed line drawings of animals and tools.
The experiment
• Subjects looked at pictures of animals and tools and named them silently.
• They also looked at noise patterns (baseline 1)• And novel nonsense objects (baseline 2)• Each stimulus was presented for 180ms followed by a
fixation cross of 1820 ms.• Drawings were controlled for name frequency and
category typicality
Conclusions• Both animal and tool naming activate the ventral
temporal lobe region.• Tools differentially activate the ACC, pre-motor and
left middle temporal region (known to be related to processing action words).
• Naming animals differentially activated left medial occipital lobe (early visual processing)
• The object categories appear to be in a distributed circuit that involves activating different salient aspects of the category.
REPRESENTATION OF CONCEPTS IN THE BRAIN: COMPETING HYPOTHESES
• Unitary Content Hypothesis– Semantic information is stored in an abstract, amodal
format organized by category.
• Multiple Semantics Hypothesis– Semantic information is stored in many modality-specific
semantic subsystems. Information in each subsystem is stored in a modality specific format.
– Our intuitive sense of information being organized by categories is based on strong connections between related parts of these modality-specific semantic systems.
Unitary Content Hypotheses (UCH)
(Caramazza et al., 1990; Caramazza & Shelton, 1998; Riddoch et al., 1988; Pylyshyn, 1973)
Multiple Semantics Hypotheses (MSH)
(Paivio, 1971; Beauvois, 1982; Shallice, 1987, 1988; McCarthy & Warrington, 1988)
Representation of words in semantic memory: the Functional Web hypothesis
• A word is represented in the cortex as a functional web
• Spread over a wide area of cortex– Includes perceptual information– As well as specifically conceptual information
• For nominal concepts, mainly in– Angular gyrus– (?) For some, middle temporal gyrus– (?) For some, supramarginal gyrus
– Plus phonological information
Example: The concept DOG
• We know what a dog looks like– Visual information, in occipital lobe
• We know what its bark sounds like– Auditory information, in temporal lobe
• We know what its fur feels like– Somatosensory information, in parietal lobe
• All of the above..– constitute perceptual information– are subwebs with many nodes each– have to be interconnected into a larger web– along with further web structure for conceptual
information
Building a model of a functional web:First steps
V
C
Each node in this diagramrepresents the cardinal node* of a subweb of properties
For exampleM
T
*to be defined in a moment!
Add phonological recognition
V
M
C
For example, FORK
Labels for Properties:C – ConceptualM – Motor P – Phonological imageT – TactileV – Visual
T
P
The phonological image of the spoken form [fork] (in Wernicke’s area)
These are allcardinal nodes –each is supportedby a subweb
Add node in primary auditory area
V
M
CT
P
PA
Primary Auditory: the cortical structures in the primary auditory cortex that are activated when the ears receive the vibrations of the spoken form [fork]
For example, FORK
Labels for Properties:C – ConceptualM – Motor P – Phonological imagePA – Primary AuditoryT – TactileV – Visual
Add node for phonological production
V
M
CT
P
PA
PP
For example, FORK
Labels for Properties: C – Conceptual M – Motor P – Phonological image PA – Primary Auditory PP – Phonological Production T – Tactile V – Visual
Arcuate fasciculus
Part of the functional web for DOG(showing cardinal nodes only)
V
MC
T
P
PA
PP
Each node shown here is the cardinal node of a subweb
For example, the cardinal node of the visual subweb
An activated functional web(with two subwebs partly shown)
V
PRPA
M
C
PP
T
Visual features
C – Cardinal concept nodeM – MemoriesPA – Primary auditoryPP – Phonological productionPR – Phonological recognitionT – TactileV – Visual
FROM WORDNET TO BRAINNET
• Neural evidence, unlike the evidence used to compile dictionaries and WordNet, and like the evidence one gathers from corpora and certain behavioral experiments, is entirely objective (although it can be subjective in the sense of differing from subject to subject)
• The objective of our research is to combine evidence from brain data, from corpora, and from behavioral experiments (all of which is rather noisy) to develop a new architecture for conceptual knowledge: BrainNet
A CASE STUDY: ABSTRACT CONCEPTS
• Until recently, most work on concepts in CL / neuroscience / psychology focused on concrete concepts
• But the type of conceptual knowledge that really challenges traditional assumptions about its organization are `abstract concepts’ – or to be more precise, the set of categories of non-concrete concepts– Events / actions– States– ‘Urabstract’ concepts: LAW, JUSTICE, ART
• We are carrying out explorations of abstract knowledge using fMRI
Anderson et al 2012a, 2012b, 2013, submitted
THEORIES OF ABSTRACT CONCEPTS IN AI AND COGNITIVE (NEURO)SCIENCE
• In CL/AI: TAXONOMIC organization for both abstract and concrete concepts– ‘UPPER ONTOLOGIES’, e.g., DOLCE
• In psychology: ‘concreteness’ scale• Best known Cognitive Neuroscience: Paivio’s DUAL CODE
theory (Paivio, 1986)– CONCRETE: verbal system & visual system– ABSTRACT: verbal system only
• Schwanenflugel & Akin 1994: CONTEXT AVAILABILITY• Barsalou’s SCENARIO-BASED MODEL (Barsalou, 1999):
– Abstract knowledge organized around SCENARIOS
The DOLCE UPPER ONTOLOGY
QQualit
y
PQPhysicalQuality
AQAbstractQuality
TQTemporalQuality
PDPerdurant
EVEvent
STVStative
ACHAchievement
ACCAccomplishment
STState
PROProcess
PTParticular
RRegion
PRPhysicalRegion
ARAbstractRegion
TRTemporalRegion
TTime
Interval
SSpaceRegion
ABAbstrac
t
SetFact…
… … …
TLTemporalLocation
SLSpatial
Location
… … …
ASOAgentive
Social Object
NASONon-agentive Social Object
SCSociety
MOBMental Object
SOBSocial Object
FFeature
POBPhysicalObject
NPOBNon-physical
Object
PEDPhysicalEndurant
NPEDNon-physical
Endurant
EDEndurant
SAGSocial Agent
APOAgentive Physical
Object
NAPONon-agentive
Physical Object
…
ASArbitrary
Sum
MAmount of
Matter
… … … …
THE OBJECTIVES OF OUR EXPERIMENT
• Identify the representation in the brain of a variety of WordNet categories exemplifying both concrete and abstract concepts (abstract words chosen by inspecting the words rated as most abstract in the De Rosa et al norms 2005)– Really abstract: ATTRIBUTE, COMMUNICATION, EVENT, LOCATION,
‘URABSTRACT’ – A category of concrete objects: TOOLS– A complex category: SOCIAL-ROLE
• Comparing two types of classification:– TAXONOMIC (as in WordNet)– DOMAIN (cfr. Barsalou’s hypothesis about abstract concepts being ‘situated’)
• Two domains: LAW and MUSIC– Using WordNet Domain
STIMULI
CATEGORY LAW (English) MUSIC (English)
attributegiurisdizione jurisdiction sonorita' sonority
cittadinanza citizenship ritmo rhythm
impunita' impunity melodia melody
legalita' legality tonalita' tonality
illegalita' illegality intonazione pitchcommunication divieto prohibition canzone song
verdetto verdict pentagramma stave
ordinanza decree ballata ballad
addebito accusation ritornello refrain
ingiunzione injunction sinfonia symphony
STIMULI, 2: URABSTRACTS
CATEGORYurabstracts giustizia justice musica music
liberta' liberty blues blues
legge law jazz jazz
corruzione corruption canto singing
refurtiva loot punk punk
STIMULI, 3: SOCIAL ROLES
Social-role giudice judge musicista musician
ladro thief cantante singer
imputato defendantcompositore composer
testimone witness chitarrista guitarist
avvocato lawyer tenore tenor
THE OBJECTIVES OF OUR EXPERIMENT
• Identify the representation in the brain of a variety of WordNet categories exemplifying both concrete and abstract concepts (abstract words chosen by inspecting the words rated as most abstract in the De Rosa et al norms 2005)– Really abstract: ATTRIBUTE, COMMUNICATION, EVENT, LOCATION,
‘URABSTRACT’ – A category of concrete objects: TOOLS– A complex category: SOCIAL-ROLE
• Comparing two types of classification:– TAXONOMIC (as in WordNet)– DOMAIN (cfr. Barsalou’s hypothesis about abstract concepts being ‘situated’)
• Two domains: LAW and MUSIC– Using WordNet Domain
ABSTRACT CONCEPTS: DATA COLLECTION AND ANALYSIS
• 7 right-handed native speakers of Italian• Task:
– Words presented in white on grey screen for 10 sec– Cross in between, 7 sec– Subjects had to think of a situation in which the word applied
• Scanner: 4T Bruker MedSpec MRI scanner, EPI pulse sequence
– TR=1000ms, TE=33ms, 26° flip angle. – Voxel dimensions 3mm*3mm*5mm
• Preprocessing: using UCL’s Statistical Parameter Mapping Software– Data corrected for head motion
• Classification: using a single layer NN
MAIN QUESTIONS
• Can the taxonomic and domain classes be distinguished from the fMRI data?
• Is there a difference in classification accuracy between taxonomy and domain?
• Can the taxonomic and domain classes be predicted across participants?
RESULTS WITHIN PARTICIPANTS (CATEGORY DISTINCTIONS)
ALL CATEGORICAL DISTINCTIONS CAN BE PREDICTED ABOVE CHANCE
THERE ARE SIGNIFICANT DIFFERENCES BETWEEN CATEGORIES
WITHIN PARTICIPANTS RESULTS SUMMARY
• Can discriminate with accuracy well above chance both taxonomic and domain distinctions
• Easiest categories to recognize: TOOL, ATTRIBUTE, LOCATION, – Then SOCIAL ROLE, COMMUNICATION– Main confusions: communication / event
Red: AttributeBlue: ToolGreen: Location
R+G=YellowG+B=CyanR+B=PinkR+G+B=White
CATEGORY LOCALIZATION IN THE BRAIN
Red: Social-roleGreen: AttributeBlue: Urabstract
Red: Social-roleGreen: CommunicationBlue: Event
R+G=YellowG+B=CyanR+B=PinkR+G+B=White
• Concrete categories TOOL and LOCATION can be predicted across participant; ATTRIBUTE can also be significantly classified; but less concrete classes become conflated with ATTRIBUTE.
• In general DOMAIN can be predicted across participants, however domain membership is much better classified in the most abstract taxonomic classes (attribute, communication and urabstract)
CROSS PARTICIPANTS RESULTS SUMMARY
LAW MUSICAttribute giurisdizione jurisdiction sonorita' sonority
cittadinanza citizenship ritmo rhythmimpunita' impunity melodia melodylegalita' legality tonalita’ tonalityillegalita’ illegality intonazione pitch
communication divieto prohibition canzone songverdetto verdict pentagramma staveordinanza decree ballata balladaddebitoaccusation ritornello refrainingiunzione injunction sinfonia symphony
event arresto arrest concertoconcert
processotrial recital recitalreato crime assolo solofurto theft festival festivalassoluzione acquittal spettacolo show
social-role giudice judge musicista musicianladro thief cantantesingerimputato defendant compositore composertestimone witness chitarrista guitaristavvocatolawyer tenore tenor
tool manette handcuffs violino violin
toga robe tamburo drummanganello truncheon tromba trumpetcappio noose metronomo metronomegrimaldello skeleton key radio radio
Location tribunale court/tribunal palco stagecarcere prison auditorium auditoriumquesturapolice station discoteca discopenitenziario penitentiary conservatorio conservatorypatibolo gallows teatro theatre
urabstracts giustizia justice musica musicliberta' liberty blues blueslegge law jazz jazzcorruzione corruption canto singingrefurtiva loot punk punk
TAXONOMIC / DOMAIN ORGANIZATION