Date post: | 15-Jan-2016 |
Category: |
Documents |
View: | 214 times |
Download: | 0 times |
The final resting place The final resting place for all this research…for all this research…
Ron Laughery, Ph.D.
University of Colorado
Items to be covered…• What is the problem this
research is trying to solve from an operational perspective?
• What is the basic human performance modeling and simulation approach that this research will feed?
• What is the specific tool and architecture that we are working to advance?
• What are the issues in moving this research into practice?
What is the problem this research is trying to solve from an operational perspective?
• The £5,000,000,000 question…– In about 1995, Robin Miller, an operational
analyst with the MoD asked us this question and made this statement at a meeting:
• “A question I get all the time can be summed up as this – should we invest £5B in new kit, or should we instead invest that £5B in training? If your models can’t help me answer that question, you’re not doing your job.”
• We are trying to ensure that we are doing our job in Mr. Miller’s eyes
What is the basic human performance modeling and simulation approach that
this research will feed?• In military and civilian systems, decisions are
increasingly being made on the basis of model based analyses– System effectiveness depends upon…
SoftwareSoftware
HumansHumansHumans
HardwareHardware
Two basic approaches to modeling human/system performance
• Reductionist– Breaking human activity and interaction
with the system into discrete activities
Advantages/disadvantages of reductionist modeling approach
• Advantages– Intuitive– Level of detail determined by need– Basic data are usually available or easily obtained– Consistent with many military systems and
operational analysis models
• Disadvantages– Often requires extensive subject-matter expert
input
Second approach to modeling human/system performance
• First principled/cognitive models– Based on theories of the underlying
mechanisms that facilitate human behavior
perception
Workingmemory
Long-termmemory
Iconicstorage
Centralprocessing
Responsemechanisms
Example: ACT-R Representation and Equations
RetrievalGoal
ManualVisual
Productions
Intentions Memory
MotorVision
World
Ai =Bi + Wj ⋅Sji +σAj∑
Bi =ln tj−d
j∑
Ui =Pi ⋅G−Ci +σU
Pi =Succi
Succi +Faili
Ti =F ⋅e−Ai
Activation
Learning
Latency
Utility
Learning
IF the goal is to categorize new stimulus and visual holds stimulus info S, F, TTHEN start retrieval of chunk S, F, T and start manual mouse movement
S 20 1Size Fuel Turb Dec
L 20 3 Y
Stimulus
ChunkBi
SSL S13
Advantages/disadvantages of the first principle approach
• Advantages– Requires less data input from either
experiments or subject matter experts– More first-principle based and, if component
models are valid, easier to defend
• Disadvantages– Model construction can be quite cumbersome
for simple tasks– We don’t have enough real first-principle
models of human performance
A strategy that has worked- a hybrid approach
• The flexibility of reductionist models combined with the power of first principles of human behavior is the formula for success
perception
Workingmemory
Long-termmemory
Iconicstorage
Centralprocessing
Responsemechanisms
Reductionist modeling with Task Network Modeling
• Largely involves the extension of a task analysis into a network defining sequencing
Going from a task network to a running computer model
• Add timing information and task/system interdependencies
Add human decision making strategies
• Any defined branch point represents a need for a decision
• Logic and rule sets, goal seeking, naturalistic
?
Then, develop a scenario, equipment model and/or links to other simulations
Run the model to collect human/system performance data
Combining First Principles of human behavior with Task Network Models
• For the past 16 years, we have been embedding and linking first principle models of human performance into our tools including– Cognitive workload and human response– Micro models of human time and accuracy– Human error and system response to error– Performance shaping factor effects– Linkage to anthropometric, biomechanical models– Goal driven task scheduling– Naturalistic Decision Making– Situation awareness modeling– Integration of cognitive engineering models such as ACT/R
• Predicting training effects is still the Predicting training effects is still the weakest link!weakest link!
Improved Performance Research Improved Performance Research Integration Tool (IMPRINT): Integration Tool (IMPRINT): Capability and ApplicationCapability and Application
What Does IMPRINT Do?What Does IMPRINT Do? It helps you... Set realistic system requirements Identify future manpower & personnel constraints Evaluate operator & crew workload Test alternate system-crew function allocations Assess required maintenance manhours Assess performance during extreme conditions Examine performance as a function of personnel
characteristics, training frequency & recency Identify areas to focus test and evaluation resources
IMPRINT Architecture - IMPRINT Architecture - Operations ModelingOperations Modeling
StochasticDiscrete Event
Simulation
StochasticDiscrete Event
Simulation
Task Time & Error Data -Estimates & Requirements
Unit &ForceUnit &Force
System MeasuresSystem
Measures
TaskLibraries
TaskLibraries
WorkloadFuture Manpower
Performance Shaping Functions- personnel
- training- stressors
WorkloadFuture Manpower
Performance Shaping Functions- personnel
- training- stressors
WorkloadFuture Manpower
Performance Shaping Functions- personnel
- training- stressors
Improved Performance Research Improved Performance Research Integration Tool (IMPRINT)Integration Tool (IMPRINT)
IMPRINT Architecture - IMPRINT Architecture - Maintenance ModelingMaintenance Modeling
Send systems on missions as defined by scenarioSimulate need for maintenance
Systems Readyfor Next Mission
Repair systems
Manpower Pool
corrective& continue
mission
combatdamage
corrective& stop
mission
preventive
Repair Parts
Who Has IMPRINT?Who Has IMPRINT?
Army Navy Air Force Other Government Contractors University
108
23
9
12
108
19
279 and growing
Mental WorkloadMental Workload
Degree of Resource Use?Which Brain Resources Involved?
Mission Tasks
1. monitoralarms
2. decideresponseaction
3. pull trigger
.
.
.n. task n
Visual
Cognitive
Auditory
Psychomotor
Cognitive
0.0 - No Cognitive Activity1.0 - Automatic (simple association)1.2 - Alternative Selection3.7 - Sign/Signal Recognition4.6 - Evaluation/Judgment (consider
single aspect)5.3 - Encoding/Decoding, Recall6.8 - Evaluation/Judgment (consider
several aspects)7.0 - Estimation, Calculation,
Conversion
Psychomotor
Auditory
Visual
Cognitive
0.0 - No Cognitive Activity1.0 - Automatic (simple association)1.2 - Alternative Selection3.7 - Sign/Signal Recognition4.6 - Evaluation/Judgment (consider
single aspect)5.3 - Encoding/Decoding, Recall6.8 - Evaluation/Judgment (consider
several aspects)7.0 - Estimation, Calculation,
Conversion
Psychomotor
Auditory
Visual
Task type (Taxon*) MOPP Heat Cold Noise Sleepless Hours
Visual T A TNumerical A TACognitive A TAFine Motor Discrete T A TFine Motor ContinuousGross Motor Light T TGross Motor HeavyCommo. (Read & Write) ACommo. (Oral) T A A
T = affects task time, A = affects task accuracy, TA= affects both
Current IMPRINT Implementation: Current IMPRINT Implementation: Stressors by Task TypeStressors by Task Type
* O’Brien, L. H., Simon R. and Swaminathan, H. (1992). Development of the Personnel-Based System Evaluation Aid (PER-SEVAL) Performance Shaping Functions. ARI Research Note 92-50
Approach to modeling human response to stressors
Performance multipliersas a function of time since sleep
6 12 18 24 30 36 42 48 54 60 66 700
0.2
0.4
0.6
0.8
1
1.2
time since sleep
performance multiplier
attention
perception
cognition
psychomotor
physical
The general effects...
Detect ring - 50% attention, 50% perception
Select menu item using a mouse - 40% attention, 60% psychomotor
Interpret customer’s request for information - 100% cognitive
On a specific task….
• attention performance multiplier = .82
• perception performance multiplier = .808
• cognition performance multiplier = .856
• psychomotor performance multiplier = .784
- physical performance multiplier = .727
under specificconditions...
leads to this specificeffect at this time…..
task time = 112.3% of normalTime to Prepare to Engage
at 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Use task network models to study aggregate effects of PSFs
Net Performance Op 6
0
2
4
6
8
10
12
14
16
18
20
100500900130017002100250029003300370041004500490053005700610065006900730077008100
Time (sec)
Wo
rklo
ad0
20
40
60
80
100
120
Uti
lizat
ion
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
Time to Prepare to Engageat 20 Hours Since Sleep
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25
Time, sec
Relative Freq.
preptime
What view of training is in What view of training is in IMPRINT now?...IMPRINT now?...
What we really need for a reasonably What we really need for a reasonably accurate representation of training…accurate representation of training…
• We need these functional relationships…– For different task types (the taxonomy)– For different “types” of training
Performance
Amount of training received
Classroomtraining
Simulatortraining
Fieldtraining
No training Retraining
Big questions…Big questions…
• Purpose of models– Design of optimal training systems– Design of systems considering training
• Taxonomies– Training environment– Task type
• Scope/complexity of tasks studied– Do small tasks scale to large tasks?
• How do we treat Retention