New MNS

New Grasp

New Concept

New Anatomy

Fig 1

Fig 2

Fig 3

Fig 3

AIP

DorsalStream:

Affordances

IT

Ways to grab

“It’s a mug” F5

canonicalF1

Hand Control

Task Constraints (F6)Working Memory (46)Instruction Stimuli (F2)

PFC

cIPSVentralStream:

Recognition

PF

IT

Hand-objecttrajectories

“It’s a mug” F1

Hand ControlFeedback Correction

Task Constraints (F6)Working Memory (46)Instruction Stimuli (F2)

PFC

STS

HandShape &Motion

F5mirror

F5 canonical

Command Actions

Observe Execute

Interpretation

F5 mirror

Recognize Actions

Planning

VisualInput AIP F5 canonical

PF F5 mirror

Choosing an Action

Recognizing an Action

PF extended

MultimodalBroca’s area

(vocal, manual, oro-

facial) Protosign (basis for sign language)

Tpt Protospeech (basis for speech)

Wernicke’s From Imitation to Describing Actions, Objects, Relationships

AuditoryInput

STSa

DPLF

IT

RecognizingActions, Objects, and

Relationships

Working Memory forActions, Objects, and

Relationships

Working Memory forPerception and Production of

Utterances

Arbib, 2001 updated by FARS modificato

STS 7a

Handshape

recognition

Hand-Objectspatial relation

analysis

Object affordance-hand stateassociation

Object affordanceextraction

Actionrecognition(MirrorNeurons)

Motor program(Grasp)

Motorexecution

MirrorFeedback

Integratetemporalassociation

7b: PF/PGF5canonical

AIP

M1

F5mirror

Object features

Objectlocation

Motorprogram(Reach)F4

Vis

ual C

orte

xcIPS

Handmotiondetection

The Mirror Neuron

System (MNS) Model

(Oztop & Arbib)

•Activity of F5 canonical neurons is part of the code forCommand: Grasp-A(Object) The full neural representation of the “Cognitive

Form” (CF): Grasp-A(Object) requires not only the regions AIP and F5canonical shown in the MNS diagram, but also inferotemporal cortex (IT) which holds the identity of the object. How are these representations bound together?

•NOTE: This is only the “Cognitive Form”. There are no “Linguistic Forms” in the monkey.

• Activity of F5 mirror neurons is part of the code forDeclarative: Grasp-A(Agent, Object) The full neural representation of the “Cognitive Form” (CF): Grasp-A(Agent, Object) requires not only the regions AIP, STS, 7a, 7b and F5miirror shown in the MNS diagram, but also inferotemporal cortex (IT) which holds the identity of the object and regions of STS (?) not included in MNS which hold the identity of the agent.

• How are these representations bound together?

STS7a

Handshape

recognition

Hand-Objectspatial relationanalysis

Object affordance-hand stateassociation

Object affordanceextraction

Motor program(Grasp)

Motorexecution

MirrorFeedback

Integratetemporalassociation

7b: PF/PG

F5canonicalAIP

M1

F5mirror

Object features

Objectlocation

Motorprogram(Reach)

F4

Vis

ual

Cor

tex

cIPS

Handmotiondetection

The Mirror Neuron System (MNS) Model

(Oztop & Arbib)

Actionrecognitio

n

• Agent Action Object Analysis of the Mirror System makes• Grasp (Raisin) primary, where the Agent is the self and thus unspecified, • and then makes• Grasp (Leo, Raisin) secondary• The mirror system expands the representational capacity of the canonical system to allow the Agent role to be played by others as well.

• But note that the full power of the system is “Beyond the Mirror” embedding MNS in a larger system:

F5 canonical

Executed Action Object

IT

F5 mirror

Observed ActionAgent

STS

view of object

AIP

grasp of object

gesture recognition

STS (and 7b?)

commandENN 7b or F5?

viewof gesture

F5

MPG

corollary discharge

MP: Gesture Motor Program

Learn by Imitation "Social

Learning"

Learn by DoingTry to Grasp

Object

Mirror neurons

Non-Mirror Neurons

gesture description

expectation