Four Receipes for Social RoboticsFour Receipes for Social Robotics

Günther Günther KnoblichKnoblichSchool of School of PsychologyPsychology, University of Birmingham, University of Birmingham

((Previously Previously at Rutgers University)at Rutgers University)

Actor2

Object

Actor3Actor1

Intention Intention Neglect Neglect and Intention Inflationand Intention Inflation

Intention Intention neglect neglect in in mainstream Cognitive mainstream Cognitive Science andScience andCognitive NeuroscienceCognitive Neuroscience, , Social PsychologySocial Psychology::

controlled controlled = = conscious conscious = intentional= intentionalautomatic automatic = = unconscious unconscious = unintentional= unintentional

Intention Intention inflation inflation ((for higher level intentionsfor higher level intentions) in) inphilosophy philosophy of of action action ((social social interaction)interaction)

Both conceptual frameworks donBoth conceptual frameworks don‘‘t t seem seem to to be be optimal tooptimal toguide current research guide current research in in social cognitive neurosciencesocial cognitive neuroscience

AimAim

AimAim:: Sketch Sketch approaches approaches to to basic social interaction basic social interaction andandattempt integrationattempt integration

- - Scenario Scenario 1: 1: EcologicalEcological- - Scenario Scenario 2: 2: Mirroring/action simulationMirroring/action simulation- - Scenario Scenario 3: 3: Shared perceptionsShared perceptions-- Scenario Scenario 4: Joint 4: Joint intentionalityintentionality-- Embedding Embedding 1-3 in 41-3 in 4-- Interfaces Interfaces for tools for tools and and languagelanguage

Scenario Scenario 1: 1: Ecological Ecological ApproachApproach

Intentions are not Intentions are not in in the organism but arise the organism but arise in in the the DYNAMICDYNAMICinteraction between organisms interaction between organisms and and their environmenttheir environment..This is This is a a radically interactionist notion radically interactionist notion of of intentionintention(Shaw, 2001)(Shaw, 2001)

A1 O

Scenario Scenario 1: Interpersonal 1: Interpersonal SynchronizationSynchronization

Other organisms are Other organisms are just just like other objects except theirlike other objects except theirmovementsmovements. . The combined movements create particularThe combined movements create particulardynamics between otherwise dynamics between otherwise „„socially socially blindblind““ organismsorganisms((fish swarmfish swarm, , rocking chairsrocking chairs))Particular dynamics may provide cues Particular dynamics may provide cues as to as to whether one iswhether one isinteracting with interacting with an an animate or inanimate animate or inanimate „„objectobject““..

A1 A2

ExampleExample: : Rocking together Rocking together ((Idealized resultsIdealized results))Richardson Richardson et al.(in press), JEP:HPPet al.(in press), JEP:HPP

People in rocking chairswho look at each otherspontaneously act againstthe chairs‘ „naturalfrequency“ in order to bein synchrony with oneanother.

Scenario Scenario 1: 1: Simultaneous AffordanceSimultaneous Affordance

Organisms who perceive the same Organisms who perceive the same „„affordancesaffordances““ will willapproach approach and and act upon the same objectact upon the same object. . Interesting objectsInteresting objectsbecome become „„magnetsmagnets““ for social encounters for social encounters and and may mediatemay mediateinterpersonal interpersonal dynamics dynamics ((feeding fishfeeding fish, , buffetbuffet))

A1 A2O

Simultaneous AffordanceSimultaneous Affordance

Scenario Scenario 2: 2: Mirroring/action simulationMirroring/action simulation

Internal representations Internal representations of of intentions assumedintentions assumed. Match of. Match ofperceived actions perceived actions to to own action repertoire leads leads own action repertoire leads leads to anto an„„intuitiveintuitive““ understanding understanding of of the action the action and and can be used can be used ininprediction prediction ((RizzolattiRizzolatti, , FadigaFadiga, Prinz, , Prinz, ShipleyShipley, Heyes, Heyes……))

A2

O

A1

ActionSimulation

Action Affects Perception:Action Affects Perception:Tritone Tritone ParadoxParadoxRepp Repp & & KnoblichKnoblich, 2007, Psychological Science, submitted, 2007, Psychological Science, submitted

Shepard scales (1964): The sameShepard scales (1964): The sametwelve tones on the chromatictwelve tones on the chromaticscale are perceived as continuouslyscale are perceived as continuouslygoing up or down in pitchgoing up or down in pitch (Shepard Demo 18-32)(Shepard Demo 18-32)

Action Affects Perception: Action Affects Perception: Tritone Tritone ParadoxParadox

Does perception of the sameDoes perception of the sameambiguous tone pair depend onambiguous tone pair depend onmovement direction?movement direction?

Pianist and non-pianists produce thePianist and non-pianists produce thetritones tritones moving from right to left ormoving from right to left orfrom left to right on a keyboard.from left to right on a keyboard.

Tritones Tritones derived from Shepardderived from Shepardscales are ambiguous with regard toscales are ambiguous with regard totheir pitch (Deutsch, 1987)their pitch (Deutsch, 1987)

Tritone-ParadoxTritone-ParadoxRepp Repp & & KnoblichKnoblich, 2007, Psychological Science, 2007, Psychological Science

Experts in an Active and Passive (Observation) ConditionExperts in an Active and Passive (Observation) ConditionRepp Repp & & KnoblichKnoblich, submitted, submitted

Scenario Scenario 2: 2: Mirroring/action simulationMirroring/action simulation

Observation of Observation of two actors two actors ((one acting upon anotherone acting upon another) ) hardlyhardlyexploredexplored. Problem: . Problem: Which actor should be simulatedWhich actor should be simulated??Asymmetry between agens Asymmetry between agens and and patiens could contribute patiens could contribute totodevelopment development of of self-other distinction needed for self-other distinction needed for 3 and 43 and 4

A2A1

ActionSimulation

A3

Scenario Scenario 3: 3: Shared PerceptionsShared Perceptions

Self-other distinction needed Self-other distinction needed to to jointly attend the samejointly attend the sameobjectobject. . This ability is assumed This ability is assumed to to play play a a crucial role crucial role inindevelopment development ((cfcf. . Tomasello Tomasello et al., 2005).et al., 2005).

A1

A2

O

Scenario Scenario 3: 3: Shared PerceptionsShared Perceptions

Attention Attention ((gazegaze) ) followingfollowingdoes not presuppose self-does not presuppose self-other distinction for perceptsother distinction for percepts

Declarative pointingDeclarative pointingpresupposes distinctionpresupposes distinctionbetween onebetween one‘‘s s own own andandothersothers‘‘ perceptspercepts

Scenario Scenario 4: Joint 4: Joint IntentionalityIntentionalitySelf-other distinction applied Self-other distinction applied to to intentions implying intentions implying aadifferent form of different form of understanding than understanding than in in Scenario Scenario 22((understanding otherunderstanding other‘‘s s intentions intentions as as being being different different fromfromoneone‘‘s s ownown, , Frith Frith & & FrithFrith). In ). In additionaddition, , ability ability to to relaterelateindividual parts individual parts to a to a joint intention joint intention ((not depicted not depicted in in figurefigure).).

A1

A2

O

Task co-representationTask co-representation((Sebanz Sebanz et al., 2003-7)et al., 2003-7)

People People represent eachrepresent eachotherother‘‘s s task even if theytask even if theycould be more effective ifcould be more effective ifthey ignored each otherthey ignored each other

They need They need to to suppresssuppressaction tendencies for tasksaction tendencies for tasksthey never perform they never perform (EEG)(EEG)

FMRI FMRI study shows increasedstudy shows increasedactivation activation in in areas for self-areas for self-related processing related processing ((when iswhen isit my it my turn?)turn?)

Flanker TaskFlanker Task

Targets: H, K Distractors: H, K, S, C, and U

HHHHH

KKHKK

UUHUU

SSHSS

Conditions

Identical

Compatible

Neutral

Incompatible

Examples Typical two-choice result

Two-choice

HHHHH

S, CH, K

3HHHHH

H, K

Neutral

UUHUUFlankers donot affordresponse

Incompatible

SSHSSFlankers afford

other’s response

≠ neutral?

Incompatible

SSHSSFlankers do not afford

response

= neutral!

ResultsResultsAtmacaAtmaca, , SebanzSebanz, & , & KnoblichKnoblich, submitted, submitted

F (1, 31) = 5.55, p < .05

Intentionality Intentionality of of co-actor should co-actor should mattermatter……AtmacaAtmaca, , SebanzSebanz, & , & KnoblichKnoblich, in , in prepprep..

Intentional Co-actor Unintentional Co-actor

ResultsResultsAtmacaAtmaca, , SebanzSebanz, & , & KnoblichKnoblich, in prep., in prep.

Embedding Entrainment Embedding Entrainment in Joint in Joint IntentionalityIntentionality

Intentional Intentional control control of of synchronization with others recruitssynchronization with others recruitsentrainment mechanismsentrainment mechanisms

Embedding Embedding Action SimulationAction Simulationin Joint in Joint IntentionalityIntentionality

Parallel Parallel simulation simulation of of contributions contributions of of self self and and other other ininrelation relation to a to a common goal common goal ((Knoblich Knoblich & Jordan, 2003, Jordan & & Jordan, 2003, Jordan & KnoblichKnoblich, 2004), 2004)

ExampleExample::Jazz Jazz musicians improvising togethermusicians improvising togetherPiano DuettPiano Duett

Do Do pianists duet betterpianists duet betterwhen they play with themselveswhen they play with themselves??

Keller, Keller, Knoblich Knoblich & & ReppRepp, , Consciousness Consciousness and and CognitionCognition, 2007, 2007

•• Nine pianists of the Yale school of music recorded one partNine pianists of the Yale school of music recorded one partof several piano duets.of several piano duets.

•• Two months later they performed the second part, either toTwo months later they performed the second part, either toa recording of their own performance or to a recording ofa recording of their own performance or to a recording ofanother pianistanother pianist’’s performance.s performance.

•• Better synchronization accuracy for self?Better synchronization accuracy for self?

AlignmentAlignment

•• Are synchronous notes in the score also synchronous in theAre synchronous notes in the score also synchronous in theperformances of the two parts of the duet?performances of the two parts of the duet?

ResultsResults

34

35

36

37

38

39

40

41

42

43

44

Take 1 Take 2 Take 3

Take Number

SD

of

As

yn

ch

ron

ies

(m

s) .

20

22

24

26

28

30

32

34

36

38

40

Me

dia

n A

bs

olu

te A

sy

nc

hro

ny

(m

s) .

Other

Self

•• Significantly lower SD of asynchronies for self. The SD ofSignificantly lower SD of asynchronies for self. The SD ofasynchronies describes the average temporal deviationasynchronies describes the average temporal deviationbetween two notes that should be synchronous.between two notes that should be synchronous.

Embedding Shared PerceptionsEmbedding Shared Perceptionsin Joint in Joint IntentionalityIntentionality

TomaselloTomasello, , WarnekenWarneken……

Guide Guide otherother‘‘s s attentions attentions to to spots they should be looking spots they should be looking atatin order to in order to complete their part complete their part of of the taskthe task

E.g.: E.g.: Pointing Pointing out an out an object the other is looking forobject the other is looking for..

Interface Interface for for Tool Tool UseUse

Organism with the Scenario Organism with the Scenario 2 2 functionality functionality ((actionactionsimulationsimulation) will ) will sometimes use sometimes use simple simple toolstools, e.g. , e.g. stickssticks..

Simulation Simulation could lead could lead to an intuitive to an intuitive understandingunderstandingbetween actors using the same toolbetween actors using the same tool

HoweverHowever, , in in Scenario Scenario 2 2 each actor each actor has to has to discover the tooldiscover the toolfor him/herselffor him/herself.. Action Action simulation simulation per se per se does not allowdoes not allowcultural transmission cultural transmission of of tool use through imitationtool use through imitation. Joint. Jointintentionality is neededintentionality is needed

Maps Maps -> -> shared perception embedded shared perception embedded in in jointjointintentionalityintentionality

Interface Interface for for Verbal Verbal CommunicationCommunication((cfcf. H. Clark). H. Clark)

Language is Language is ((nothing butnothing but) a ) a coordination device for jointcoordination device for jointactionaction. . Joint Joint intentionality needs intentionality needs to to be be in in place forplace forlanguage language to to be usefulbe useful

Language remains grounded Language remains grounded in in mechanisms for basic socialmechanisms for basic socialinteraction interaction (e.g., (e.g., synchronization synchronization of of postural sway duringpostural sway duringconversationconversation))

Mechanisms for basic social interactions may be adaptedMechanisms for basic social interactions may be adaptedfor reuse with symbolic informationfor reuse with symbolic information, e.g., , e.g., simulation forsimulation forphonologyphonology, , semanticsemantic, , syntax syntax ((cfcf. Pickering & . Pickering & GarrodGarrod, 2006), 2006)

Entrainment Entrainment of Body Postures of Body Postures through Speechthrough SpeechShockley Shockley et al., JEP:HPP 2003et al., JEP:HPP 2003

Task: Find the difference between the pictures(only one can be seen by each participant)

Partner visible Partner invisible

Entrainment Entrainment of Body Postures of Body Postures through Speechthrough SpeechShockley Shockley et al., JEP:HPP 2003et al., JEP:HPP 2003

Confederate (control):

Joint Joint attention between speaker attention between speaker and and listenerlistenerRichardson Richardson & Dale, 2005, & Dale, 2005, Cognitive Cognitive ScienceScience

Does joint attention affect how Does joint attention affect how well a well a listener understandslistener understandsa a speakerspeaker??

A A speaker talks about speaker talks about a a video while she is watching itvideo while she is watching it. . AAlistener watches the same video while hearing the speakerlistener watches the same video while hearing the speaker..

The eye movements The eye movements of of speaker speaker and and listener are trackedlistener are tracked..Listeners answer questions about what the speaker saidListeners answer questions about what the speaker said..The are categorized into high/low accuray listenersThe are categorized into high/low accuray listeners..

Is there Is there a a higher overlap higher overlap in in eye movements between theeye movements between thespeaker speaker and and high-accuracy listenershigh-accuracy listeners??

High High accuracy listeners show more overlap withaccuracy listeners show more overlap withspeakerspeaker‘‘s s eye movementseye movements..Good Good listeners are very likely listeners are very likely to to look look at at objects thatobjects thatwill will be/have been looked be/have been looked at at by the speakerby the speaker..

ConclusionsConclusions

Embedding entrainment, action simulation, sharedEmbedding entrainment, action simulation, sharedperception in joint intentionality allows one to explainperception in joint intentionality allows one to explainmany recent findings on basic social interactionsmany recent findings on basic social interactions

Any form of cultural transmission (including imitationAny form of cultural transmission (including imitationof tool use) requires joint intentionalityof tool use) requires joint intentionality

Language may not be necessary for culturalLanguage may not be necessary for culturaltransmission, but once in place it remains grounded intransmission, but once in place it remains grounded inmechanisms for basic social interaction (and maymechanisms for basic social interaction (and mayreshape this patterns)reshape this patterns)