Date post: | 18-Jan-2016 |
Category: |
Documents |
Upload: | maude-gaines |
View: | 217 times |
Download: | 0 times |
1
Visually-guided Motor Actions in Humans
Ken Nakayama and Joo-Hyun SongHarvard University
VISION SCIENCESLABORATORY
2
Actions: Interaction with The External World
3
.
• Discrete responses • Continuous interaction
Actions: Read Out Cognitive Processing
• Visually-guided actions: Instantaneously read out internal processes
Yet, the importance of motor actions is overlooked
4
Cognitive Theories in Human Behaviors
• Perception Cognitive decision Action
• Issue of motor control
- Limited to post-cognitive decision-making - Mere reflection of completed cognitive decisions
5
Target without distractors Target among distractors
Visually-Guided Manual-Pointing Task
+
Single targetEasy Task
+
Odd-color target Difficult Task
6
.
Hand movement recording: 120Hz with Polhemus Fastrak
Visually-Guided Manual-Pointing Task
QuickTime™ and aSorenson Video 3 decompressorare needed to see this picture.
7
Outline
• Part 1: Role of focal attention on latencies and trajectories of manual pointing
• Part 2: Concurrent processing of manual pointing to competing stimuli
• Part 3: Automatic adjustment of visuo-motor readiness
8
Outline
• Part 1: Role of focal attention on latencies and trajectories of manual pointing
• Part 2: Concurrent processing of manual pointing to competing stimuli
• Part 3: Automatic adjustment of visuo-motor readiness
9
Visual Attention: Selection for Perception & Action
10
• Broadly distributed attention for entire displays is sufficient to detect and localize the target
Distributed vs. Focused Attention
• Focused attention needs to be allocated to the target for feature discrimination
( Atkinson & Braddick, 1989; Folk & Egeth, 1989; Johnson & Pashler, 1990; Sagi & Julesz, 1985 a, b; Green, 1992)
11
Diagnostic Visual Search Paradigm
Mixed Condition
Target: Red Green Red
+ + +
Blocked Condition
Red Red RedTarget:
+ + +
(Bravo & Nakayama, 1992) Number of distractors
React
ion
Tim
es
Detection(distributed)
MixedBlocked
Discrimination (focused)
MixedBlocked
Saccadic tasks: No subtle discrimination required
(McPeek et al., 1999)
12
Role of Focused Attention on Latencies & Trajectories of Manual Pointing
Number of distractorsR
eact
ion
Tim
es
Detection (distributed)
MixedBlocked
Discrimination (focused)
MixedBlocked
+ + +
B. Blocked Condition
+ + +
Green GreenGreen
A. Mixed Condition
Green GreenRed
• Full Movement trajectories over the time course: Interaction between cognitive-decision making for the correct target & motor control
13
Reaction Time Indexes
(Initiation)Latency
MovementDuration
Total Time
+
Stimulus onset
Finger lift-off
Target touch
Time
14
Total Time (ms)
7 Subjects
Exp 1: Effect of Perceptual Grouping of Distractors
680
700
720
740
760
2 5 8 11
Number of distractors
Mixed
Blocked
280
300
320
340
2 5 8 11Number of distractors
MixedBlocked
Initial latency (ms)
380
400
420
440
460
2 5 8 11Number of distractors
MixedBlocked
Movement duration (ms)
15
Exp 1: Effect of Perceptual Grouping of Distractors
-5 0 50
2
4
6
8
10
12
-5 0 50
2
4
6
8
10
12
-5 0 50
2
4
6
8
10
12
-5 0 50
2
4
6
8
10
12 # Distractor=0
Vert
ical M
ovem
en
t
Horizontal Movement (Inches)
Mixed condition
Baseline condition
-5 0 50
2
4
6
8
10
12# Distractors=5
0
2
4
6
8
10
12
-5 0 5-5 0 50
2
4
6
8
10
12# Distractors=2 # Distractors=11
C.
Blocked condition
16
Exp 1: Effect of Perceptual Grouping of Distractors
-5 0 50
6
12
Horizontal movement (inches)
Ver
tical
mov
emen
t (in
ches
)
• Initially toward distractors & corrected to the target
• Size & frequency of curved trajectories are correlated with the strength of target-distractor competition (McPeek, Han & Keller, 2003).
17
Max Curvature=Max(D)/L
-50
5
05
1002468
1012
Forward
Left Right
D
L
.
0.07
0.08
0.09
0.1
0.11
0.12
0 2 4 6 8 10 12
Number of distractors
Baseline
Mixed
Blocked
Exp 1: Effect of Perceptual Grouping of Distractors
18
680
700
720
740
760
2 5 8 11
Number of distractors
Mixed
Blocked
Position within Same Color Sequence
. . .1 8. . .
RT
5
Target:
(Maljkovic & Nakayama, 1994; Bichot & Schall, 1999; McPeek et al., 1999)
Exp 2: Effect of Perceptual Priming
Total Time Attended target feature repetition
19
Exp 2: Effect of Perceptual Priming
650
675
700
725
750
775
0 1 2 3 4 5 6 7
Order in sequence
Blocked Baseline
7 Subjects
Total time (ms)
Order in sequence400
425
450
475
500
0 1 2 3 4 5 6 7
Blocked Baseline
Movement duration
Order in sequence
240
245
250
255
260
265
270
0 1 2 3 4 5 6 7
Blocked Baseline
Initial latency
20
Maximum curvature
Exp 2: Effect of Perceptual Priming
0.05
0.075
0.1
0.125
0.15
0.175
0 1 2 3 4 5 6 7
Order in sequence
Max
imum
cur
vatu
re
Blocked Baseline
21
Discussion
• Focused attention is closely associated with manual-pointing process though subtle discrimination is not required. • Trajectory curvature is reduced as decreasing competition between target and distractors by perceptual grouping and cumulative priming
• Interactive processes between target selection and motor control may challenge assumption of serial processing in traditional cognitive theories
22
Outline
• Part 1: Role of focal attention on latencies and trajectories of manual pointing
• Part 2: Concurrent processing of manual pointing to competing stimuli
• Part 3: Automatic adjustment of visuo-motor readiness
23
Assumption of Serial Processing & Optimal Planning
Perception
Cognitive decision
Action• A single movement is planned & executed serially at a time
• An optimal trajectory is planned• Straight trajectories• Bell-shaped velocity profile
( Flash & Hogan, 1985; Uno, Kawato, & Suzuki, 1989; Engelbrecht, 2001)
24
• Reaching trajectories swerves away from distractors ( Tipper, Lortie, & Baylis, 1992; Tipper, Howard, & Houghton, 1998, 2000)
• Dorsal premotor areas encode multiple action plans concurrently, even before cognitive decision is made (Cisek & Kalaska, 2002, 2005)
Multiple Reaching Plans For potential Targets
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
26
Are they concurrently planned?
• Do we execute every hand movement after target selection is finalized? • Or, do we sometimes initiate hand movements before completing target selection? • Can movements to competing stimuli be concurrently processed?
When selecting a target among multiple distractors…
+
Odd-color target
27
Concurrent Processing of Saccades
Target
InitialsaccadeCorrective
saccade
(McPeek, Skavenski, & Nakayama, 2000)
28
Concurrent Processing of Visually-Guided Manual Pointing
vs.
Tiny mass/inertia
High mass/inertia
Hand/arm
29
+
Trial 1: Odd-color target
Main Experiment
+
Trial 2: Single target (Baseline)
Trial 3
Trial 4+
+
. .
30
Down
Left Right
Up
Single target trials
Movement Trajectories:Single Target
31
Down
Left Right
Up
Movement Trajectories:Single Target
Single target trials
32
Down
Left Right
Up
Movement Trajectories:Single Target
Single target trials
33
Odd-color target trials
Down
Left Right
Up
Movement Trajectories:Odd-Color Target
Single target trials
34
Down
Left Right
Up
Movement Trajectories:Odd-Color Target
Odd-color target trialsSingle target trials
35
Down
Left Right
Up
Movement Trajectories:Odd-Color Target
Odd-color target trialsSingle target trials
36
Down
Left Right
Up
Odd-color target trialsSingle target trials
Movement Trajectories:Odd-Color Target
37
Are Initial & Corrective Movements Planned Concurrently ?
• Curved trajectories- Influences of competing distractors on planning & execution of pointing movements
• Are initial & corrective movements concurrently planned, overlapping in time?
• Difficulty of decomposing curved trajectories into discrete two movements
38
Estimate of Corrective Movement Onset
Average Baseline Trajectory
Time (ms)
Pos
ition
(inch
es)
0 40 80 120 160 200 240 280 320 360-5
-4
-3
-2
-1
0
1
2
3
4
5
Left
Cen
ter
Rig
ht
-1.5SD
+1.5SD
InitialMovement
CorrectiveMovement
StimulusOnset
+
Initial latency
Corrective latency
39
Corrective Latency vs. Initial Latency
8 subjects
243 Trials
0
10
20
30
40
50
60
0 50 100 150 200 250 300 350 400 450 500
Latency(ms)
Number of trials
Corrective latencyInitial latency
40
Difference Between Corrective & Initial Latencies
243 Trials
0
5
10
15
20
25
30
35
-400 -300 -200 -100 0 100 200 300 400
Differece betweencorrective and initial latencies (ms)
Number of trials
8 subjects
41
Time Cost Of Executing two Concurrently Planned Movements
Vel
ocity
alo
ng t
he c
urve
500 10000
10
20
30
40
Time(ms)
Curved Straight
Straight
Curved
Odd-Color
+
Non-optimal properties
42
Time Cost Of Executing two Concurrently Planned Movements
8 subjects
0
100
200
300
400
500
600
700
800
Initial latency Movementduration
Total time
Time (ms)
Curved trajectory
Straight trajectory
43
Displaced target
Initial target
~150-250 ms
+
++
Fixation
Time
35% Trials: Displaced
65% Trials: Baseline
Time
+
+
Fixation
Control Experiment:Double-Step Task
44
Control Experiment:Double-Step Task
931 Trials
0
50
100
150
200
250
300
0 50 100 150 200 250 300 350 400 450 500
Latency(ms)
Number of trials
Corrective latencyInitial latency
45
Discussion
• The cost of planning corrective movements may be negligible because these can be planned in parallel with initial movements.
• Like saccades, pointing movements to competing stimuli can be concurrently processed.
• Hand movements are sometimes initiated before cognitive decision-making for the correct target is completed.
• The visuo-motor system is less concerned with planning a single optimal trajectory as long as the ultimate talk goal is achieved
46
Outline
• Part 1: Role of focal attention on latencies and trajectories of manual pointing
• Part 2: Concurrent processing of manual pointing to competing stimuli
• Part 3: Automatic adjustment of visuo-motor readiness
47
Down
Left Right
Up
Single Target
• Fast initiation & straight trajectory
Influence of Trial Difficulty on Manual Pointing Strategies
+
Single targetEasy Task
48
Single Target
Down
Left Right
Up
• Fast initiation & straight trajectory
Influence of Trial Difficulty on Manual Pointing Strategies
+
Single targetEasy Task
49
Single Target
Down
Left Right
Up
• Fast initiation & straight trajectory
Influence of Trial Difficulty on Manual Pointing Strategies
+
Single targetEasy Task
50
Odd-Color Target
• Slow initiation & straight trajectory
• Fast initiation but curved trajectory
Influence of Trial Difficulty on Manual Pointing Strategies
+
Odd-color target Difficult Task
51
Odd-Color Target
• Slow initiation & straight trajectory
• Fast initiation but curved trajectory
Influence of Trial Difficulty on Manual Pointing Strategies
+
Odd-color target Difficult Task
52
Odd-Color Target
• Slow initiation & straight trajectory
• Fast initiation but curved trajectory
Influence of Trial Difficulty on Manual Pointing Strategies
+
Odd-color target Difficult Task
53
• How does intermixing trials of varying difficulty influence latency & accuracy criteria of visually-guided manual pointing?
• What mechanism adjusts latency & accuracy criteria in visually-guided manual-pointing tasks?
• How does the time course of latency & accuracy criteria adjust?
Questions
54
• Predictability Knowing upcoming trial type may lead to initiation
latency & accuracy adjustment
• Trial Type RepetitionContinuous repetition of the same type trials may lead
to initiation latency & accuracy adjustment
Initiation latency & accuracy criteria for trials of
varying difficulty may be influenced by either …
Predictability vs. Repetition
55
Alternating(Odd-color)
Mixed
++
+
+
. .
.
+
. .
.
+
+
+
(Single)
+
+
+
++
+
+
+
. .
. .
. .
Exp1. Dissociation of Predictability & Past Experience
To compare influences of predictability and past trial type repetition on initiation latency & accuracy
adjustment
Blocked
56
Exp1. Dissociation of Predictability & Past Experience
Prediction for Adjustment
Blocked Mixed Alternating
Predictability Yes No Yes
Trial Repetition
Yes No No
Cognitive knowledge of the next trial but frequent trial type switch
57
Blocked Mixed Alternating
Predictability Yes No Yes
Trial Repetition
Yes No No
200
250
300
350
400
Blocked Mixed Alternated
Single targetOdd color target
N=9
Initiation Latency
√
Adjustment of Initiation Latency Criteria
Alternating
Initi
atio
n la
tenc
y(m
s)Not simply due to task
switchingNo difference in error
rates ( over 98%)
58
Blocked Mixed Alternating
Predictability Yes No Yes
Trial Repetition
Yes No No
Movement duration
√
Adjustment of Initiation Latency Criteria
300
350
400
450
500
Blocked Mixed Alternating
Movement duration(ms)
59
.
200
250
300
350
400
Blocked Mixed Alternated
Single targetOdd color target
N=9
Adjustment of Initiation Latency Criteria
Alternating
• Homogenization: Initiation latency difference vanished in the mixed & alternating conditions
• No initiation latency differentiation by cognitive knowledge in the alternating condition
• Adjustment of initiation latency criteria by trial type repetitions
+ +++ +Trial sequence
0 0 1 0 1
+
2
Initiation Latency
60
-50
-25
0
25
50
75
100
1 over 2
Number of the same type trial repetition
0
.
+ +++ +Trial sequence
0 0 1 0 1
+
2
Trial-by-Trial Initiation Latency Adjustment
Moment-to-Moment cumulative learning
61
.
200
250
300
350
400
Blocked Mixed Alternated
Single targetOdd color target
N=9
Dissociation of Predictability & Past Experience in Initiation Latency
Alternating
-50
-25
0
25
50
75
100
1 over 2
Number of the same type trial repetition
0
.
No initiation latency differentiation by cognitive knowledge
Cumulative learning from recent experience
Dissociation of the two sources Motor initiation latency is
adjusted by recent experience not by cognitive knowledge
62
Adjustment of Accuracy Criteria
Cost of sub-optimal adjustment of latency criteria?
.
200
250
300
350
400
Blocked Mixed Alternated
Single targetOdd color target
N=9
Initiation Latency
Alternating
63
Movement Trajectories
Left Right (Inch)
Up
(Inc
h)D
own
Single target
Odd-color target
Blocked Mixed Alternating
-5 0 50
2
4
6
8
10
12
-5 0 50
2
4
6
8
10
12
-5 0 50
2
4
6
8
10
12
-5 0 50
2
4
6
8
10
12
-5 0 50
2
4
6
8
10
12
-5 0 50
2
4
6
8
10
12
64
Maximum curvatures.
0.00
0.05
0.10
0.15
0.20
Blocked Mixed Alternated
Single targetOdd color target
N=9
Alternating
More curved trajectory trials in mixed & alternated conditions
65
.
200
250
300
350
400
Blocked Mixed Alternated
Single targetOdd color target
N=9
Exp2: Cumulative Learning in Predictable Sequence
Alternating
-50
-25
0
25
50
75
100
1 over 2
Number of the same type trial repetition
0
.
Cumulative learning from recent experience
Questions• Time course with longer sequence?• Predictable sequence?
66
Exp2: Cumulative Learning in Predictable Sequence
+ +
++ +
0
+
+
+
++
1 2 3 4 0 1 2 3 4
Odd-color
200
250
300
350
400
0 1 2 3 4
N=9
Number of the same type trial repetition
Singe target
Odd-color target
switching
• 5 single/odd-color target alternation• Alternation informed in advance & Visual cue
67
• Intermixing trials of varying difficulty eliminates differences between initiation latencies of easy and difficult trials
Discussion
.
200
250
300
350
400
Blocked Mixed Alternated
Single targetOdd color target
N=9
Alternating
• It is Not task switching : both easy & difficult trials slowed by intermixing
(Allport et al., 1994)
Initiation latency in Exp1
68
• Repetition of trials with the same difficulty gradually differentiates initiation latencies by cumulative learning
Discussion
200
250
300
350
400
0 1 2 3 4
N=9
Number of the same type trial repetition
Singe target
Odd-color target
• It is not perceptual priming : both easy and difficult trials facilitated by repetition
(Maljkovic & Nakayama, 1994)
Initiation latency in Exp 2
69
• It is not specific to input or output systems: Also, no difference between easy and difficult trials in reading aloud tasks (Lupker et al., 1997)
Discussion
70
Fast initiation
• Central mechanism?: Determine sensorimotor readiness by very recent experience: Adjustable gain or threshold setting
Discussion
Time
Threshold
Deci
sion(A
ctiv
ati
on)
Slow initiation
71
• Part 1: Role of focal attention on latencies and trajectories of manual pointing
• Part 2: Concurrent processing of manual pointing to competing stimuli
• Part 3: Automatic adjustment of visuo-motor readiness
72
• Simple visually-guided action requires the allocation of focused attention to the target • Multiple planning & execution of manual pointing toward competing stimuli can occur
• Visuo-motor action & target selection process are interacting continuously even after motor execution
Conclusion
73
• Visuo-motor system is less concerned with planning a single optimal trajectory as long as the ultimate task goal is successfully achieved
• Application: Instantaneous read-out Perceptual/cognitive processing
Conclusion
74
Acknowledgements
• Ken Nakayama• Patrick Cavanagh• Yuhong Jiang• Anne Grossetete• Harvard Vision Lab
75
Thank you!