Annotating affective neuroscience data with the Emotion Ontology

Janna Hastings1,2

Werner Ceusters3

Kevin Mulligan2

Barry Smith3

1 Cheminformatics and Metabolism, European Bioinformatics Institute, UK2 Swiss Center for Affective Sciences, University of Geneva, Switzerland

3 National Center for Ontological Research, University at Buffalo, USA

ICBO MFO Workshop, 22 July 2012

Wednesday, April 12, 2023 2

Affective science

The interdisciplinary study of:

emotional functioning, regulation, expression, and physiological markers

affective disorders such as bipolar, depression and schizoaffective disorder

PHILOSOPHY | PSYCHOLOGY | NEUROSCIENCE | PSYCHIATRY | LINGUISTICS

The Emotion Ontology (MFO-EM)

Wednesday, April 12, 2023 5

To define the characteristics of different emotions start with canonical emotions

Emotion types (such as fear) show enormous variance across instancesJust as do anatomical types, e.g. human bodies

Ontology expresses what is always true… But also aims to say something useful for representation of domain knowledge.

Solution: encode such knowledge in ‘canonical’ types

canonical fear

appraisal process

Appraisal of dangerousness

Has part Has output

Canonical fear results from an appraisal of dangerousness

Wednesday, April 12, 2023 6

Canonical fear

canonical fear

fear

EMOTION COMPONENT CHARACTERISTIC FOR FEARAction tendency Fight-or-flightSubjective emotional feeling Negative, tense, powerlessBehavioural response Characteristic fearful facial

expressionCharacteristic appraisal Something is dangerous to me

subtype

Affective neuroscience

Cognitive neuroscience of emotion and other affective phenomena: studying the brain correlates of emotional functioning in humans using functional brain imaging technology (fMRI, EEG)

Meta-analysis: comparing the results of multiple different experiments in order to arrive at a consistent view of brain activation for a process in a larger population

Brain imaging databases

• BrainMap, http://www.brainmap.org/: 2226 papers, 10606 experiments (~400 related to emotion)

• SumsDB, http://sumsdb.wustl.edu/sums/index.jsp, ~800 papers• AMAT, http://www.antoniahamilton.com/amat.html, 212

papers, 675 contrasts• Brede,

http://hendrix.imm.dtu.dk/services/jerne/brede/brede.html, 186 papers, 586 experiments

• ** fMRI Data Center, http://www.fmridc.org/f/fmridc/index.html, 107 “datasets”

• ** openfMRI, http://www.openfmri.org/, 12 “datasets”

Size of the neuroimaging literature

“By the end of the year 2008 approximately 9400 fMRI studies investigating human cognition and action will have been published in English language journals” and 2012?

Derfuss and Mar, Lost in localization: The need for a universal coordinate database, NeuroImage 48, 2009

NeuroSynth

Automatically assembled by text mining neuroimaging journals and extracting

a) brain localization coordinates reported in tables, and b) words that appear in the papers with activation reported in or near specific coordinates, sorted by frequency.

But, the trouble with words is…

NeuroSynth

Cognitive Atlas

The Cognitive Atlas is a collaborative knowledge building project (using a Wiki platform) that aims to develop a knowledge base that characterizes the state of current thought in cognitive science.

• But: the trouble with cognitive scientists’ thoughts is…

Using the Emotion Ontology for annotation of data from Cognitive Neuroscience of Emotion

Wednesday, April 12, 2023 17

Study Task Annotation class in MFO/MFOEMRecognition of gender in emotional facial expressions

Visual perception of emotional facial expressions (subClassOf perception)

Recall of personal emotional memories with instructions to try re-create feeling

Memory of emotional episodes (subClassOf memory)

Listening to emotional sounds (e.g. grunts of disgust)

Auditory perception of emotional stimuli (subClassOf perception)

Viewing emotional film extracts Visual and auditory perception of emotional stimuli (subClassOf perception)

Visual perception of an angry face

MF:visual perception andhas-participant some (

IAO:picture and is-about someMFOEM:characteristic angry facial

expression )

What is the subject feeling as they perceive?What does the experimenter think they are studying (and what they report on)?

Wednesday, April 12, 2023 20

Acknowledgements Thanks!

Emotion Researchers in GenevaDavid Sander, Julien Deonna

Chemistry, Biology, NeuroscienceChristoph Steinbeck, Nicolas le Novère, Colin Batchelor,

David Osumi-Sutherland, Jane Lomax, Gwen Frishkoff, Jessica Turner, Angela

Laird