Language, Linguistics and the Brain

Sharon Armon-Lotem

From Neuron to Mind Session 5: Language as a window to the mind

June 17, 2004

How does the brain manipulate and process language?

Language is a system of pairing of forms (often sounds) and meanings which is used to (formulate) and transfer information.

Language manipulation appears to be a rule-governed activity involving symbol manipulation involving the pairing of symbols and 'meaning‘

How the brain manipulates language involves answering how the brain performs these three functions.

If we can understand how the brain manipulates language, we have a window on to how the hardware is connected to the higher cognitive functions = mind.

What is language?

Language includes:

a phonological system- which determines which sounds are part of the language and how they are strung together in words.

a set of words assigned to categories – lexicon. a syntactic system - which determines how

words can be strung together in utterances. a semantics system – which determines how

meaning is assigned to words and strings of words.

a pragmatics module – which determines how context affects the interpretation of utterances.

Language as a window to the mind

There are at least three approaches to language as a window to the mind:

What are the properties of the language that the brain is processing? (Theoretical Linguistics) Susan Rothstein, Yael Greenberg, Gabi Danon, Galit Adam

What are the properties of language in use, which will tell us something about language as it is processed? (Psycholinguistics) Jonathan Fine, Joel Walters, Sharon Armon-Lotem, Elinor Saiegh-Haddad

How does the brain process language?(Psychology and Language) Miriam Faust, Michal Raveh, Elisheva Ben-Artzi, Gil Diesdendruk

Theoretical Linguistics

Susan Rothstein specializes in formal semantics and the question of how our use of language reflects basic cognitive capacities; e.g. how the use of count/mass expressions reflects the way in which we individuate and count entities.

Yael Greenberg specializes in formal semantics, and, among

other things, how we unconsciously use minimally distinct constructions to express different meanings e.g. rina yafa / rina hi yafa.

Gabi Danon specializes in formal syntax: what is the relation between syntactic representation and the meaning of a linguistic expression. Why are there environments where indefinites but not definites are allowed e.g. ba li glida but not *ba li ha-glida ha-zot? (but: ba li al ha-glida ha-zot is OK)

Galit Adam specializes in phonological theory and the acquistion of phonological competence. Why are there disparities between child and adult production? Why do children say babuk for bakbuk ‘bottle’, dropping the k at the middle but not in the end?

Psycholinguistics Jonathan Fine works on how language is a window onto the mind

by examining the language of speakers with psychiatric disorders such as schizophrenia, autism and hyperactivity. The language reflects and, in fact, partly defines the disorders. Changes in the disorder by medication can then be studied in the language.

Joel Walters studies phenomena representing cognitive processes by examining the special characteristics of bilinguals. Such bilingual speakers switch languages and acquire languages in ways that give a unique view onto processing of language.

Sharon Armon-Lotem specializes in first language acquisition of syntax and semantics in normal and special populations; e.g., how children learn to use the inflectional system, the definite marker, relative clauses and the mass count distinction; and what makes some aspect of the system more penetrable than others.

Elinor Saiegh-Haddad works on language and reading acquisition in bi-dialectal and bilingual contexts. She studies the effect of linguistic distance between the oral language of children and the dominant/written language on acquiring basic linguistic and literacy skills.

Psychology and Language Miriam Faust studies the representation of language in the human brain

using behavioral, electrophysiological (ERPs) and brain imaging (fMRI) techniques. A special focus is given to the processing of high level linguistic information by the left and right cerebral hemispheres, language organization in the brain in different clinical populations and word retrieval processes in typically developing and clinical populations

Elisheva Ben-Artzi works on Dyslexia (with Harvey Babkoff) focusing on the processes underlying dyslexia; on false memories for words, studying the nature of false memories for words; and on aging and speech comprehension (with Harvey Babkoff), focusing on the interrelationships between auditory temporal processes and speech comprehension in the elderly.

Gil Diesdendruk focuses on ways in which language and thought relate. He investigates to what extent children rely on their understanding of the mental states of speakers in order to infer the meanings of words, arguing that while children might rely on the species-specific capacity to "mind-read" when acquiring words, the capacity is not language-specific.

Michal Raveh examines word recognition abilities of children and adults,

normal and dyslexic to understand how the structure of words affects their recognition. She conducts computerized simulations of neural net model of language processing to develop theories of reading.

Looking through the three windows

The semantics of the count-mass distinction: Theory and Acquisition

Language Use and Processing

Brain and Creativity: The Effects of Semantically Convergent and Semantically Divergent Priming on Lexical Processing by the Two Cerebral Hemispheres

The semantics of the count-mass distinction: Theory and Acquisition

Susan Rothstein and Sharon Armon-Lotem

The Department of English and

The Gonda Multidisciplinary Brain Research Center Bar-Ilan University

Semantic definitions of count and mass (Link 1983)

Mass expressions are homogeneous. Count expressions are not.

Upward homogeneity (cumulativity): wine + wine = wine

a dog + a dog = dogs/*a dog Downward homogeneity:

If I split wine into two parts each part is still wine.If I split a dog into two parts, each part is not a dog.

Homogeneity is not a linguistic distinction (Rothstein 2004(

Homogeneity is a real-world distinction. The fact that half a dog is not a dog while

half a quantity of wine is wine depends on what counts as a dog or as wine in the world.

So is the count mass distinction formal, or does it reflect the way things are in the world?

Count nouns can be homogeneous too (Rothstein 2004)

Fence, wall, hedge are all homogeneous and count. A fence can be broken into two

fences: Two fences can be put together

into a single fence. Rothstein 2004: homogeneous count

nouns are a systematic part of the language and not odd exceptions (in contradistinction to Krifka 1992, Zucchi and White).

So [±homogeneous] is not at the basis of the count mass distinction.

What are children sensitive too?

Are children sensitive to

[± homogeneous] = world distinction

[± count] = formal distinction both

Methodology

Syntactic information: plural vs. singular 20 children, age: 3 -5 + 10 Adults Methodology: elicitation task with real words

and objects. Three semantic categories, which the two

features yield. Presentation: Mom went shopping. She

put an apple in her basket, then another apple, and then another apple. What’s in the basket? What do we have here?

When is plural used?

Figure 1 – The use of plural forms by different age groups (by percentage)

100%100%94%

0%6%0%10%

34%

16%

0%

20%

40%

60%

80%

100%

AdultsFivesThrees

Apples

Sugars

Fences

Findings

The Threes like the adults made a binary distinction based on world distinction.

Three of the Fives made a binary distinction based on the formal distinction.

Seven of the Fives followed the world distinction, but used other linguistic means to make the formal distinction, making a three-fold distinction.

How did the Five’s make the three-fold distinction?

Half of their responses included classifiers in order to make the formal distinction between ‘sugar’ and ‘fence’:

Shalosh peamim xol three times sandshtey kosot mayimtwo cups water

Half of their responses included an adjective indicating the length of the row of objects in order to make the formal distinction between ‘sugar’ and ‘fence’:

srox aroxShoelace longsharsheret arukaChain long

Conclusion

The Threes go by the world distinction between things which are homogeneous and things which are not.

The Fives show awareness of the tension between the linguistic indicators and the non-linguistic ones.

The Fives form a syntactic distinction between the three semantic categories, in line with the semantic distinctions.

From mental representations to language use and processing

The work just described on the formal semantics of the mass/count distinction and studies investigating the acquisition of that distinction in early child language is an attempt to get at mental representation, one, if not THE, central concern of theoretical linguistics.

This too will be the concern in the presentation of Naama Friedmann on syntactic theory and Broca’s aphasia.

Another line of work in our linguistics group focuses on LANGUAGE USE AND PROCESSING.

By way of example, we tell a tale of two frameworks; one on the language of psychiatric conditions and one on the languages of bilingualism.

Using and Processing Language

The Department of English and

The Gonda Multidisciplinary Brain Research Center Bar-Ilan University

Jonathan Fine and Joel Walters

Two approaches to language use and processing

Fine starts with the resources for making meaning (from culture and genre, to discourse and syntax) and analyses texts from different kinds of speakers. These techniques of discourse analysis are used by both Fine and Walters. They are illustrated briefly here and elaborated in greater detail in Obler’s talk on Alzheimers patients.

Walters’ information components (identity, context, genre, intentions, formulation, and articulation) bridge the sociopragmatic-psycholinguistic divide. His processing mechanisms - imitation, variation, integration, and control - are grounded in psychology, and as Cziko shows later in his talk on Perceptual Control Theory, they make more explicit claims than conventional constructs.

Both approaches aim to capture levels of language processing which characterize special populations

Psychiatric disorders are largely assessed through clinical interviews that depend almost entirely on the language of the patient and clinician. In fact, many disorders are defined by how speakers use language. •Attention deficit hyperactivity disorder (ADHD)

•butts into conversations•answers questions before they are finished•difficulty playing quietly

•Schizophrenia •loose associations•tangential talk•talks too much

•Pervasive developmental disorders (PDD/Autism) •pedantic speech•one-sided social interactions•lack of spontaneous seeking to share

Linguistic characteristics in Psychiatric Disorders

Methodologies of discourse analysis (ADHD)

1) OverlappingC1: can I go to the washroom? T: um yes you may go if you've got those < puzzles pieces> C2: < Miss he's going to play with his yo-yo>

2) Topic structureon topicon topic (pronouns, lexical repetition, conjunction, ellipsis)T: okay, there you go, so what you want to do is figure out maybe you put the first three on the bottom and then work up. Just put them in the first three <at the top>. C1: <all of them>

off topicoff topic  T: and Paula, you can whole punch that so we'll put this

into the detective section, so it was our last one. There you go, so the green section’s okay and it's almost finished.

C2: glue glue glue glue. Glue glue. Glue. Glue. Glue.T: so . C1: who likes Pokeman here?  

Percent of initiations that are off-topic

0

10

20

30

40

50

60

Inattentive (P ) Hyperactive (M)

Percent of initiations that are off-topic and also overlap

0

5

10

15

20

25

30

Inattentive (P ) Hyperactive (M)

Methodologies of discourse analysis (Bilingualism)

Codeswitching and pragmatic markers(5) Secretary: What are you doing with our lists. Yo!! ((very

agitated))(6) Student: No, they gave it to me.(7) Secretary: Who gave it to you?(8) Student: AA, eh (.) I was advised by her. I spoke to her.(9) Secretary: AA gave it to you?(10) Student: Of course.(11) Secretary: To take home?(12) Student: Sure.(13) Secretary: This is not-at-all meant for students. It is not

allowed to be given to anyone.

Codswitching(1) Do you have חבילות עוגיות שלימות? (whole cookie boxes)(2) Do you know to לעטוף ספרים? (to wrap books)(3) muzi [move + zuzi](4) noce [no + roce]

Summary

Functional linguistic features define psychiatric disorders and describe cognitive processes in bilingualism

Precise linguistic formulations allow psychiatric disorders and different forms of bilingualism to be distinguished and track changes in the course of treatment or development

From language use and processing to imaging language in the brain

The understanding of functional linguistic categories in psychiatry and bilingualism broadens our insights about language in use ands offers different methodologies for studying language processing.

The next line of work described hereby takes us into experimental laboratories of psychology and language, in the study of language processing in the hemispheres.

 

 

 

Brain and Creativity:

The Effects of Semantically Convergent and Semantically Divergent Priming on Lexical

Processing by the Two Cerebral Hemispheres

Miriam Faust

Department of PsychologyAnd

The Gonda Multidisciplinary Brain Research Center Bar-Ilan University

The LH strongly activates a narrow semantic field, only including the interpretations that are dominant or most relevant to the immediate context. This process is very effective for most straightforward language processing

The RH weakly activates a broad semantic field, including many interpretations or meanings that may seem less relevant. This process is ineffective for rapid interpretation or selection but increases semantic overlap among multiple semantic fields.

This asymmetry can be described as fine semantic coding in the LH and coarse semantic coding in the RH (Beeman, 1994).

Comprehension of semantic information by the LH and RH

Nonselective RH semantics seems particularly well-suited for:

Comprehending words/messages where several levels of meaning must be simultaneously considered

Revising an initial interpretation

Continued activation of discourse level meaning

Processing unusual, nonsalient semantic relations (poetry)

Insight problem solving

Right Hemisphere (RH) contribution to language processing

The Mednick Test of Creative Cognition (1962)

REPORT, CREDIT, PLAYING - CARD

Performance on insight problems is associated with creative thinking (Schooler & Melcher, 1997)

There are some striking parallels between the properties of insight problem solving and the cognitive characteristics of the RH(Fiore & Schooler, 1998)

The RH may have unique abilities to avoid the inhibitory processes that promote fixedness and that reduce access to nondominant interpretations(Fiore & Schooler, 1998)

Insight problem solving and the RH

If the activation of word meanings in the RH is

broader, especially sensitive to overlapping from distantly related words, and less differentiated, than the RH should take better advantage of semantically divergent primes

Four divided visual field experiments examined the

ability of the LH and the RH to process semantically convergent (either dominant or subordinate) versus semantically divergent (combinations of dominant + subordinate) primes for facilitating the recognition of ambiguous target words

The present research

Participants: Sixty-four right handed, native English speaking students

Stimuli: Targets: 128 ambiguous words (“novel”) 128 nonwords (“nohel”)

Primes: 256 words related to the dominant meaning 256 words related to the subordinate meaning 256 unrelated words

Lexical decision and semantic judgment(Faust & Lavidor, 2003)

Target: Novel

Unrelated: Apple, Fruit

Dominant (convergent): New, Fresh

Dominant + Subordinate/Subordinate+ Dominant (divergent):

New, Story

Subordinate (convergent): Story, Book

Four types of word pairs were constructed from the word primes:

Focusing signalX

Primes(500 ms)

Focusing signal (300 ms + 150 ms)

X

Target word Target word (150 ms) (150 ms)

Procedure

FIGURE 2 : RT PRIMING FOR TARGET WORDS PRESENTED TO THE RVF & LVF FOLLOWING THREE TYPES OF PRIMES (LEXICAL

DECISION)

0

10

20

30

40

50

60

70

80

TWO DOM ONE DOM ONE SUB TWO SUB

RT

PR

IMIN

G

RVF/LH

LVF/RH

The overall pattern of priming indicates that the combinatorial influence of multiple related primes is different within each hemisphere

RVF/LH Large benefit from dominant primes No benefit from subordinate primes Facilitation from divergent primes is smaller than the

facilitation from dominant primes Facilitation from 2 dominant primes is significantly

larger than from one dominant prime

LVF/RH No benefit from either dominant or subordinate

convergent primes Only divergent primes facilitate target word recognition

Findings - Lexical decision

FIGURE 3 : RT PRIMING FOR TARGET WORDS PRESENTED TO THE RVF & LVF FOLLOWING THREE TYPES OF PRIMES (SEMANTIC JUDGEMENT)

-20

0

20

40

60

80

100

120

140

TWO DOM ONE DOM ONE SUB TWO SUB

RT

PRIM

ING RVF/LH

LVF/RH

The overall pattern of priming indicates that the combinatorial influence of multiple related primes is different within each hemisphere

RVF/LH Large facilitation from both dominant and subordinate

primes No facilitation from divergent primes (underadditive

effect, inhibition)

LVF/RH Facilitation from all types of primes (largest for or

divergent primes)divergent primes)

Absolute LVF/RH advantage for divergent primes (65 ms)

Findings - semantic judgment

The findings point to an important implication of RH propensity to activate and maintain many potential word meanings: The ability to converge on a target word from different, including incompatible, word meanings without being disrupted by seemingly semantic conflicts

RH ability to utilize two different incompatible concepts may serve specific linguistic functions

Maintaining simultaneous activation for alternate word meanings could be essential in cases where the ability to recognize alternative interpretations of problem elements represents a critical component, e.g., in creative, insight problem solving

Conclusions

Concluding remarks

The three presentations focused on different windows to meaning construction in the mind:

Rothstein and Armon-Lotem show how semantic categories are built on the interaction between formal and 'real world' criteria.

Fine and Walters focus on special populations in an attempt to show how we construct meanings from rich context

Faust shows how our two hemispheres process and access meanings, highlighting the role of the RH in these processes.

These different approaches to language research complement each other and lead to an understanding of what language is and what the mind must be in order to use language.

Thank you!