Embodiment & Compositionality Two fundamental, but unreconciled, aspects of human language:...

transcript

Embodiment & Compositionality

• Two fundamental, but unreconciled, aspects of human language: embodiment and compositionality

• Embodiment – the realization that scientific understanding of mind and language entails detailed modeling of the human brain

• Compositionality - only people can express and understand an essentially unbounded range of messages

Compositionality

Anything that deserves to be called a language must contain meaningful expressions built up from other meaningful expressions” [4]

strong compositionality states that the meaning of a expression is totally determined by its form and is independent of context

“In its strict version, this claim is clearly wrong” [5].

Truth-conditional SemanticsThe existence of these forms of context-dependence

would thus appear to spell the doom for truth-conditional semantics, understood as the project of assigning properties to syntactic items (partly by finding principles for determining properties of complex syntactic items on the basis of their parts) such that we await only the specification of something like an index to know what the truth-conditions of a given utterance of a given assertive sentence would be.- Jason Bridges

Wittgenstein’s Philosophical Investigations: A Critical Guide, A. M. Ahmed (ed.), Cambridge University Press, 2009.

Simulation Semantics

• Lectures 1 and 3 => embodied

• But what about public language, text?

• Skeletal shared meaning => schemas

• Language Community (LC)– Shares grammar, schemas, analysis rules

Compositionality

• Put the parts together to create the meaning of the whole.

• Questions:– what is the nature of the parts?– How and why do they combine with one another?– What meaning is associated with this composition?

Short NTL answers

• Parts = constructions, schemas

• Combination = binding, unification

• Meaning of the whole :– Public (LC) ~ Semantic Specification = SemSpec– Private ~ Enactment/Simulation of SemSpec

ECG Lattices

• Schemas – Image, X-schemas, Frames

• Constructions – Lexical, Grammatical

• Language Communities

• Situations ~ Mental Spaces

• Maps ~ Metaphor, etc.

LecturesI. Overview2. Simulation Semantics3. ECG and Best-fit Analysis4. Compositionality5. Simulation, Counterfactuals, and Inference

Constructions

Simulation

Utterance Discourse & Situational Context

Semantic Specification:

image schemas, bindings, action schemas

Analyzer:

incremental,competition-based,

psychologically plausibleA

Conceptual Structure

• Embodied

• Schematic

• (Partially) language-independent

Conceptual Composition• Highly interconnected

• Primitives

• Conceptual composition

• Metaphor

Image Schemasparameters of spatial cognitionaction schemas - controllergoals, force-dynamics (causation)parameters of parts & boundaries Time

Social Worldyoung/ mature/ oldauthority, approval, helpvalue , exchange, obligationtheory of mind, perception and intention, Communication speaker/ hearer, direct/ indirecttrue/ falsequestion, command, etc.

Grammaticalized conceptsperson, gender, age, agent, speakerpossession, mass/count, reflexives, Primal scenes/ event types – transitive, tense, aspect General Logicconnectives, numberssimilarity, inference, uncertaintypart/ whole, scales, magnitudebinding, variables, indefinites, generalization Mental Operations Learning, matching, bindingmental spaces, mappingssimulation, displacement

Some Conceptual Primitives

Conceptual CompositionEllen Dodge Thesis

1. Conjunction – horse with stripes2. Modification – camel without hump3. Abstraction - vehicle4. Mapping a. Metonymy – London thinks b. Metaphor – causes are forces5. Blending – apple bus6. Relational – horses eat grass

First example

• He bit the apple.

schema MotorControl subcase of Process roles Actor ↔ Protagonist

Effector Effort

Routine constraints Actor ← animate

Schemas

schema ForceApplication subcase of MotorControl evokes ForceTransfer as FT roles

Actor ↔ FT.Supplier ↔ Protagonist Acted Upon↔ FT.Recipient Effector

Routine Effort ↔ FT.Force.amount

schema ForceTransfer evokes Conact as C roles

Supplier ↔ C.entity1 Recipient ↔ C.entity2 Force

schema MotorControl subcase of Process roles Actor ↔ Protagonist

Effector Effort

Routine constraints Actor ← animate

schema Contact subcase of SpatialRelation roles Entity1: entity Entity2: entity

Schema Lattice

MotorControl

Motion

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

ContactSpatiallyDirectedAction

CauseEffect

Contact

Agentive Impact

SelfMotion

SelfMotionPath

MotionPath

Construction BITE1 subcase of Verb form: bite meaning: ForceApplication constraints: Effector ← teeth Routine ← bite // close mouth

Verb Constructions

schema ForceApplication subcase of MotorControl evokes ForceTransfer as FT roles Actor ↔ FT.Supplier ↔ Protagonist Acted Upon ↔ FT.Recipient Effector Routine Effort ↔ FT.Force.amount

Verb Constructions

schema ForceApplication subcase of MotorControl

schema Agentive Impact subcase of ForceApplication

cxn BITE meaning: ForceApplication

schema MotorControl

cxn GRASP meaning: ForceApplicationcxn PUSH meaning: ForceApplicationcxn SLAP meaning: AgentiveImpactcxn KICK meaning: AgentiveImpactcxn HIT meaning: AgentiveImpact

Argument Structure Construction

construction ActiveTransitiveAction2 subcase of VP constituents: V : verb NP: NP form constraints: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

{Vm ↔ ED.ProfiledProcess}

Causer ↔ ED.ProfiledParticipant

FA ↔ Vm

Causer ↔ FA.Actor Affected ↔ FA.ActedUpon Affected ↔ NPm

FA ↔ Vm

CauseEffect schema

schema CauseEffect subcase of ForceApplication; Process roles

Causer ↔ Actor Affected ↔ ActedUpon ↔ Process.Protagonist Instrument ↔ Effector

MotorControl

Motion

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

CauseEffect

Contact

SelfMotion

SelfMotionPath

MotionPath

Agentive Impact

Process

Schema Lattice

meaning: CauseEffect evokes: EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

FA ↔ Vm

Important points

Compositionality does not require that each component contain different information.

Shared semantic structure is not viewed as an undesirable redundancy

Causer ↔ ED.ProfiledParticipant FA ↔ Vm

schema EventDescriptor

EventType: Process

ProfiledProcess: Process

ProfiledParticipant: Entity

ProfiledState(s): State

SpatialSetting

TemporalSetting

Event Descriptor schema

Preconditions, resources, fine control structure are important aspects of events

Construction ActiveTransitiveAction2 subcase of VP constituents: V : verb NP: NP form constraints: VF before NPF

Causer ↔ ED.ProfiledParticipant FA ↔ Vm

construction NPVP1 constituents: Subj: NP VP : VPform Constraints Subj f before VPf

meaning: EventDescriptor ProfiledParticipant ↔ Subjm

Bindings with other cxnsconstruction ActiveTransitiveAction2 subcase of VP constituents: V ; NP form: VF before NPF

meaning: CauseEffect evokes; EventDescriptor as ED constraints: {Selfm ↔ ED.EventType}

Affected ↔ NPm

Construction NPVP1 constituents: Subj: NP VP : VPform constraints Subj f before VPf

Bindings with other cxnsconstruction ActiveTransitiveAction2 subcase of VP constituents: V ; NP form: VF before NPF

Affected ↔ NPm

schema EventDescriptor roles EventType ProfiledProcess ProfiledParticipant ProfiledState(s) SpatialSetting TemporalSetting

Bindings with other cxns

schema EventDescriptor roles EventType ProfiledProcess ProfiledParticipant ProfiledState(s) SpatialSetting TemporalSetting

construction NPVP1 constituents: Subj: NP VP : VPform Constraints Subj f before VPf

construction ActiveTransitiveAction2 subcase of VP constituents: V ; NP form: VF before NPF

Affected ↔ NPm

Unification

CauseEffect causer affected

ForceApplication actor actedupon

EventDescriptor EventType ProfiledProcess ProfiledParticipant

TransitiveAction2

THE APPLE

NP2ReferentDescriptor

ReferentDescriptor

Meaning Constructions

Unification

EventDescriptor EventType ProfiledProcess ProfiledParticipant

TransitiveAction2

THE APPLE

ReferentDescriptor resolved referent

Unification

EventDescriptor eventtype ProfiledProcess ProfiledParticipant

TransitiveAction2 Verb

THE APPLE

ReferentDescriptor resolved referent

Unification

TransitiveAction2

NPVP1 subj

THE APPLE

ReferentDescriptor

Unification

TransitiveAction2 NP

THE APPLE

ReferentDescriptor

Semantic SpecificationHe bit the apple

ForceApplication actor actedupon routine bite effector teeth

RD55category

Person

RD27category

Argument Structure ConstructionHe was bitten (by a toddler)

construction PassiveTransitiveAction2 subcase of VP constituents: V : PassiveVerb (PP: agentivePP) form constraints: VF before PPF

meaning: CauseEffectAction evokes; EventDescriptor as ED; ForceApplication as FA constraints: {Selfm ↔ ED.EventType}

Affected ↔ ED.ProfiledParticipant FA ↔ Vm

Causer ↔ FA.Actor Affected ↔ FA.ActedUpon Causer ↔ PP.NPm

Semantic SpecificationHe was bitten (by a toddler)

ForceApplication actor actedupon routine bite effector teeth

RD48category

Person

RD27category

Variations on a theme

• He shattered the window

• The window was shattered

• The window shattered

Construction SHATTER1 subcase of Verb form: shatter meaning: StateChange constraints: Initial :: Undergoer.state ← whole Final :: Undergoer.state ← shards

Verb Construction -- shatter

schema StateChange subcase of Process roles Undergoer ↔ Protagonist

Argument Structure ConstructionHe shattered the window

meaning: CauseEffect evokes: EventDescriptor as ED; StateChange as SC constraints: {Selfm ↔ ED.EventType}

SC ↔ Vm

Affected ↔ SC.Undergoer Affected ↔ NPm

Semantic SpecificationHe shattered the window

StateChange Undergoer state “wholeness”

RD189category

Person

Window

RD27category

Argument Structure ConstructionThe window was shattered

construction PassiveTransitiveAction3 subcase of VP constituents: V : PassiveVerb (PP: agentivePP) form constraints: VF before NPF

meaning: CauseEffect evokes: EventDescriptor as ED; StateChange as SC constraints: {Selfm ↔ ED.EventType}

Affected ↔ ED.ProfiledParticipant

SC ↔ Vm

Affected ↔ SC.Undergoer Causer ↔ PP.NPm

Semantic SpecificationThe window was shattered

RD175category

Window

Argument Structure ConstructionThe window shattered

construction ActiveIntransitiveAction1 subcase of VP constituents: V : verb form meaning: Process evokes: EventDescriptor as ED; StateChange as SC constraints: {Selfm ↔ ED.EventType}

Protagonist ↔ ED.ProfiledParticipant

SC ↔ Vm

Protagonist ↔ SC.Undergoer

Semantic SpecificationThe window shattered

Process protagonist

RD177categoryWindow

Summary

• Small set of constructions and schemas

• Composed in different ways

• Unification produces specification of simulation

• Sentence understanding is simulation

• Different meanings => different simulations

NTL Compositionality

• Language understanding is simulation• Simulation activates conceptual structures• Conceptual Compositionality is basic• Grammatical Compositionality is inherently

constructional, not surface• SemSpec ~ Semantic Specification

– Skeletal Meaning– Captures shared understanding of an LC– Site of Compositionality

Argument Structure ConstructionHis white teeth bit into the apple

construction ActiveEffectorMotionPath3 subcase of VP constituents: V : verb PP: Spatial-PP form constraints:

VF before PPF

meaning: EffectorMotionPath evokes; EventDescriptor as ED; ForceApplication as FA constraints:

{Selfm ↔ ED.EventType}

Effector ↔ ED.ProfiledParticipant FA ↔ Vm

Actor ↔ FA.Actor // INIEffector ↔ FA.EffectorTarget ↔ FA.ActedUpon SPG ↔ PPm

Target ↔ PPm .Prep.LM

Simulation: His white teeth bit into the apple

Action

Source

Path Goal

Effector Motion

Protagonist = Actor

Protagonist = Effector

Non-agentive biting

• He landed on his feet, hitting the narrow pavement outside the yard with such jarring impact that his teeth bit into the edge of his tongue. [BNC]

• The studs bit into Trent's hand. [BNC]

• His chest burned savagely as the ropes bit into his skin. [BNC]

MotorControl

Motion

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

CauseEffect

Contact

SelfMotion

SelfMotionPath

MotionPath

Agentive Impact

Process

Schema Network

Simulation: His teeth bit his tongue

Source

Path Goal

MotionProtagonist = Mover

Key assumptions of NTL

• Language understanding is simulation

• Simulation involves activation of neural structures

Constructions

Construction Grammar• Constructions are form-meaning pairings • A given utterance instantiates many different constructions

Embodied Construction Grammar• Construction meaning is represented using schemas• Meaning is embodied

LecturesI. Overview2. Simulation Semantics3. ECG and Best-fit Analysis4. Compositionality5. Simulation, Counterfactuals, and Inference

Constructions

Simulation

Analyzer:

psychologically plausible

Constructions

Simulation

Analyzer:

psychologically plausible

Image Schemasparameters of spatial cognitionaction schemas - controllergoals, force-dynamics (causation)parameters of parts & boundaries Time

Social Worldyoung/ mature/ oldauthority, approval, helpvalue , exchange, obligationtheory of mind, perception and intention, Communication speaker/ hearer, direct/ indirecttrue/ falsequestion, command, etc.

Grammaticalized conceptsperson, gender, age, agent, speakerpossession, mass/count, reflexives, Primal scenes/ event types – transitive, tense, aspect General Logicconnectives, numberssimilarity, inference, uncertaintypart/ whole, scales, magnitudebinding, variables, indefinites, generalization Mental Operations Learning, matching, bindingmental spaces, mappingssimulation, displacement

Some Conceptual Primitives

Conceptual Composition

1. Conjunction – horse with stripes2. Modification – camel without hump3. Abstraction - vehicle4. Mapping a. Metonymy – London thinks b. Metaphor – causes are forces5. Blending – trash can basketball

Putting the parts together

• Bindings

• Unification

“Pre-existing” structure

schema

schema schema

schema

Unification

schema

schema schema

schema

Process

Simulation - He bit the apple

CauseEffect

ForceApplication

Protagonist = Causer ↔ Actor

Protagonist = Affected ↔ ActedUpon

Process

Simulation - He bit the apple

CauseEffect

ForceApplication

Protagonist = Causer ↔ Actor

Protagonist = Affected ↔ ActedUpon

MotorControl

Motion

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

CauseEffect

Contact

SelfMotion

SelfMotionPath

MotionPath

Agentive Impact

Process

Schema Lattice

Prototypes and extensions?

CauseMotion Path:

• He threw the ball across the room

• He kicked the ball over the table

• He sneezed the napkin off the table

• [He coughed the water out of his lungs]

Key points

• In prototypical verb-argument structure construction combinations, verb meaning is very similar to argument structure meaning.

• Verbs whose meaning partially overlaps that of a given argument structure constructions may also co-occur with that argument structure construction

• These less prototypical combinations may motivate extensions to the central argument structure constructions

Some more variations on a theme

• He bit the apple

• He bit into the apple

• His white teeth bit into the apple.

Argument Structure ConstructionHe bit into the apple

VF before PPF

meaning: EffectorMotionPath evokes; EventDescriptor as ED; ForceApplication as FA constraints:

Actor ↔ ED.ProfiledParticipant FA ↔ Vm

Actor ↔ FA.ActorEffector ↔ FA.Effector // INITarget ↔ FA.ActedUpon SPG ↔ PPm

Schema

schema EffectorMotionPath subcase of EffectorMotion subcase of SPG // or evokes SPG roles Actor ↔ MotorControl.protagonist

Effector ↔ SPG.Tr ↔ M.Mover ↔ Motion.protagonistTarget ↔ SPG.Lm

MotorControl

Motion

EffectorMotion

EffectorMotionPath

ForceTransfer

ForceApplication

CauseEffect

Contact

SelfMotion

SelfMotionPath

MotionPath

Agentive Impact

Process

Schema Lattice

Argument Structure Construction He bit into the apple

VF before PPF

meaning: EffectorMotionPath evokes: EventDescriptor as ED; ForceApplication as FA constraints:

Actor ↔ ED.ProfiledParticipant FA ↔ Vm

Actor ↔ FA.ActorEffector ↔ FA.Effector // INITarget ↔ FA.ActedUpon SPG ↔ PPm

EffectorMotionPath

Action

Source

Path Goal

Effector Motion

Protagonist = Actor

Protagonist = Effector

The ‘subject’ NP (She) is bound to these roles:

• profiledParticipant of EventDescriptor• causer of CauseMotionAction• actor of ForceApplication• supplier of ForceTransfer• [actor of ForcefulMotionAction]• [actor of EffectorMotionAlongAPath]• protagonist of CauseMotionAction and

ForceApplication• [protagonist of ForcefulMotionAction and

EffectorMotionAlongAPath]

The ‘direct object’ NP (his hand) is bound to:

•affected of CauseMotionAction•protagonist2 of CauseMotionAction•actedUpon of ForceApplication•recipient of ForceTransfer•mover of MotionAlongAPath•trajector of SPG•[actedUpon < -- > Target of ForcefulMotionAction]•[target of EffectorMotionAlongAPath] (landmark of a different SPG schema)

construction CauseMotion4 subcase of CauseMotion1 constructional constituents v : Verb np: NP

pp: Path-PP form constraints v.f before np.f np.f before pp.f meaning: CauseMotionAction evokes EventDescriptor as ed evokes ForcefulMotionAction as fma constraints ignore self.m <--> v.m self.m.process1 <--> v.m.process1 v.m <--> fma self.m<--> ed.eventType v.m<--> ed.profiledPorcess self.m.affected <--> np.m self.m.causer <--> ed.profiledParticipant self.m.process2.spg <--> pp.m.spg

Embodiment & Compositionality Two fundamental, but unreconciled, aspects of human language:...

Documents