Date post: | 15-Jan-2016 |
Category: |
Documents |
View: | 216 times |
Download: | 0 times |
IMPRINT models of training: Digit Data Entry and RADAR
MURI Annual Meeting
September 7, 2007
Carolyn Buck-GenglerDepartment of Psychology and
Center for Research on Training
University of Colorado at Boulder
September 7, 2007
Overview: Modeling effort
• Digit Data Entry (DDE; with Bill Raymond):– Completed model– Did goodness of fit evaluations (also Bengt Fornberg)– Reported at BRIMS (Buck-Gengler, Raymond, Healy,
& Bourne, March, 2007)– Bengt and Bill: Model comparisons
• RADAR: First start in February– Beginning structure of model created; old IMPRINT– Resumed when new IMPRINT Pro available in
April/May
September 7, 2007
Summary of DDE modelwhen different, last year’s model in blue italics
• Modeled both experiments of Healy, Kole, Buck-Gengler, & Bourne (2004) in one model (only Experiment 2)
• Model based on cognitive model of DDE– Cognitive and physical components assigned to different keystrokes
– Chunking
• Difference between hands (R faster than L) and hand switching
• Learning (RT improvement) due to repetition of numbers and due to general practice (only unique numbers; improvements made to how learning modeled)– Differences in cognitive and physical learning, assigned to different
keystrokes
• Cognitive Fatigue (not in previous model)
• Accuracy over the course of the experiment– Overall, and by output length
September 7, 2007
Picture of RADAR taskone frame
September 7, 2007
Shift type
• 4 different shift types, created by crossing mapping type and load– Mapping: Consistent (CM) vs. Varied (VM)
• CM: target(s), distractors from DIFFERENT character sets (letters, digits)
• VM: target(s), distractors from SAME set
– Load: light (1-1) or heavy (4-4)• 1-1: one possible target; one blip has a character• 4-4: four possible targets; all 4 blips have
characters
September 7, 2007
Other details
• Particular target(s) different every shift• 2 sessions (training, test)
– Each session 8 blocks of 20 shifts of one type• 15 shifts have a target, 5 have only distractors• IF target, in only one of the frames, rest only distractors
– Block order each session:CM1-1 CM4-4 VM1-1 VM4-4 VM4-4 VM1-1 CM4-4 CM1-1
• Half the subjects also have secondary tone-counting task (tone/no tone crossed between sessions)
September 7, 2007
Picture of RADAR taskCM 4-4 shift example – one frame
target
September 7, 2007
Subject response
• GO-NO GO (at frame level): Press space bar if see target character in the frame, otherwise ignore
• Scored on:– Response time to correctly hitting space bar
(when target) from time target frame starts– Accuracy in correctly identifying target– Correct rejection of non-target frames (by
non-response)
September 7, 2007
Scoring
Non-target shiftEvery frame non-target
Target shift
No response Correct reject Miss
Response
- any frame False alarm NA
- before target frame NA False alarm
- target frame NA Hit
- after target frame NA Miss
September 7, 2007
Results to be modeled• RT:
– CM faster than VM, 1 target faster than 4
– CM 1-1 and VM 1-1 about same RT; CM 4-4 a bit slower, VM 4-4 much slower
• Hit rate:– CM more accurate than VM, 1 target more accurate than 4
– VM 4-4 is only shift type to show accuracy degradation
– Small improvement over time
• False alarm rate:– sharp learning curve between first and second blocks
– some penalty for VM and more for VM 4-4
– lots of learning between 1st and 2nd block of VM 4-4 (blocks 4 and 5) and 1st and 2nd blocks of CM 1-1 (blocks 1 and 8)
• Tone counting hurts both accuracy and RT– If done during training, hurts performance at test regardless of whether
tone counting at test
September 7, 2007
Current modeling approach• Similar approach to how things done in digit data
entry model– Simplest subset of data modeled first, then after that
works, add new ones• CM/VM x 1-1/4-4; no tone counting• Tone counting• Learning• Add test session to training session
– RT and accuracy• Statistical Ss will vary around the mean S RT and accuracy
for the experiment• RTs vary for each trial around the base time for the S• Response results from random selection of trial as response
or no response based on hit and false alarm rates
September 7, 2007
Model
• Two networks– Computer:
• presents frames within shifts; 15 of 20 shifts per block contain target; target randomly assigned to one of frames 2-6
• “presents” tones to tone-counting “subjects” (will be half of subjects in each session)
– Subject goal network: wait until a frame is presented, then respond (or not) to that frame
• Third network planned– Probable second subject goal network: count tones
that are different from the reference tone
September 7, 2007
Main network
Do One Shift subnetwork
Subject goal network
September 7, 2007
Do one shift subnetworkSubject network responds here
Parallel paths forpresenting targets
and presenting tones
Parallel pathsrejoin here
September 7, 2007
CM vs. VM
• CM: Targets from different character set than distractors very easy and fast discrimination for CM
• VM: Targets from same character set as distractors slower decision
• MODEL: Two separate paths; decision time longer on VM path than CM path; time accrued for each blip looked at
September 7, 2007
1 target/blip filled invs. 4
• Eye movement: Occurs for each blip looked at• Decision:
– 1: once there, just have to make the decision– 4: assumption: sequential search/decision process
because blips start outside fovea• Target frames: Target occurs equally in any of the 4
locations, and subjects could focus first equally on any of the 4 locations. So 1/4 of the time, should need to look at just 1, 1/4 of time look at 2, etc. time to make decision accrues for each looked at
• Non-target frames: Subject should have to look at all 4; when they don’t false alarm, they don’t respond and time will be maximum frame time
September 7, 2007
Subject goal network Respond to frame
Decide on whether to respond
Respond (or not)
Response time (positive response) =N*(eye movement + decision time) + time to press space bar
N = number of blips looked at to make decision
CM
VM
September 7, 2007
Tone counting
• Making the go-no go target detection decision (primary task) incurs cognitive workload
• Keeping track of tones (secondary task) adds cognitive workload
• Assumption: when workload added, if workload threshold reached, something suffers; accuracy, RT, or both (on primary task)
September 7, 2007
Learning
• Learning is our term for the observation of improved RT or accuracy over the sessions
• Assumption: Learning only occurs on correct trials
• Data:– No consistent pattern of improvement for RTs – Small improvement in hit rates (by shift type) (silent
group)– More complex pattern in false alarm rates
• Tone counting, to the extent it decreases (detection) accuracy, will decrease improvement due to learning
September 7, 2007
Current state of model
• Only silent group, only first (training) session
• RT, Hit rate, False Alarm rate– Incorporates both hit rate and false alarm rate
into decision making
• Trial-level variability of RTs
• NOT doing tone counting or learning yet• Not yet incorporating subject-level
variability
September 7, 2007
RTs for correct shifts
0
200
400
600
800
1000
1200
1400
1600
CM 1-1 CM 4-4 VM 1-1 VM 4-4
Shift type
RT (ms)
DataModel 24 Ss
September 7, 2007
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
CM 1-1 CM 4-4 VM 1-1 VM 4-4
Shift type
Hit rate (proportion of targets correctly
identified)
DataModel 24 Ss
Hit rate
September 7, 2007
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
CM 1-1 CM 4-4 VM 1-1 VM 4-4
Shift type
False Alarm Rate
DataModel 24 Ss
False Alarm Rate
September 7, 2007
Planned
• Learning/training– Program it similarly to method used in DDE
model– Explore possibility of using new ARL training
plug-in
• Incorporate IMPRINT workload– Tone counting impact on performance
• Add second (test) session– Tone vs. non-tone manipulation– Effects of training over longer period
September 7, 2007
Contributions of IMPRINT modeling of DDE and RADAR• Translatable to MATLAB – allows using MATLAB for model comparison and parameter optimization
• Allows exploration of feasibility of modeling basic low-level cognitive tasks in IMPRINT
• Leads to insights concerning underlying cognitive processes
• Leads to predictions concerning training effects and suggestions for further experiments