Paul Mitchell – MIT Humans and Automation LabPaul Mitchell – MIT Humans and Automation Lab

Management of Multiple Dynamic Management of Multiple Dynamic Human Supervisory Control TasksHuman Supervisory Control Tasks

MIT-Boeing Research Project Review Meeting – Feb. 2, 2005 MIT-Boeing Research Project Review Meeting – Feb. 2, 2005

22

OutlineOutline

• MotivationMotivation

• Experiment ObjectiveExperiment Objective

• Experiment ScenarioExperiment Scenario

• Experimental DesignExperimental Design

• Human-Automation Interaction ModelingHuman-Automation Interaction Modeling

• Wait Time ModelingWait Time Modeling

• Key Experiment Display ElementsKey Experiment Display Elements

• Expected Research Results / BenefitsExpected Research Results / Benefits

• Multi Aerial Unmanned Vehicle Experiment (MAUVE) Multi Aerial Unmanned Vehicle Experiment (MAUVE) Program DemoProgram Demo

• HAL Lab TourHAL Lab Tour

33

MotivationMotivation

• Consequences of NCWConsequences of NCW

Volume of informationVolume of information

Number of information sourcesNumber of information sources

Operational tempo Operational tempo

• Greater attentional demands on operatorsGreater attentional demands on operators

• Efficient attention allocation becomes critical to human & Efficient attention allocation becomes critical to human & system performance!system performance!

44

Experiment ObjectivesExperiment Objectives

• To investigate:To investigate:

• How operators cope with managing multiple HSC processes How operators cope with managing multiple HSC processes simultaneouslysimultaneously

• What kinds of decision support can aid operators in these What kinds of decision support can aid operators in these situationssituations

• What effects human performance limitations have on the What effects human performance limitations have on the overall systemoverall system

55

Experiment ScenarioExperiment Scenario

• UAV domain has UAV domain has immediate military immediate military applicationsapplications

• NCW concept of swarmsNCW concept of swarms

• Subject is an operator Subject is an operator supervising four separate, supervising four separate, independent unmanned independent unmanned aerial vehicles (UAVs)aerial vehicles (UAVs)

• Objective:Objective:

• To destroy a set of To destroy a set of targets (which may targets (which may change) within a certain change) within a certain time period, while taking time period, while taking minimum damage from minimum damage from enemy air defensesenemy air defenses

Predator and Global Hawk, both UCAVs currently in use.

66

Experimental DesignExperimental Design – Independent Variables – Independent Variables

Level of Decision Support (Scheduling Assistance)Level of Decision Support (Scheduling Assistance)

• Between subjectsBetween subjects

• 4 levels4 levels

− Manual = LOA 1Manual = LOA 1

− Passive = LOA Passive = LOA

− Active = LOA 4Active = LOA 4

− Super Active = LOA 6Super Active = LOA 6

Amount of Schedule Re-PlanningAmount of Schedule Re-Planning

• Within subjectsWithin subjects

• 3 levels3 levels

− NoneNone

− InfrequentInfrequent

− FrequentFrequent

The Patriot missile battery, a prominent example of a high level of automation in use today.

77

Experimental DesignExperimental Design – Dependent Variables – Dependent Variables

• Primary task performance – Number/priority of deadlines Primary task performance – Number/priority of deadlines missedmissed

• Performance scorePerformance score

• Combines target and threat eventsCombines target and threat events

• To provide insight into overall test session performanceTo provide insight into overall test session performance

• Secondary task performance – Chat BoxSecondary task performance – Chat Box

• Percentage correct answers Situation awareness Percentage correct answers Situation awareness metricmetric

• Average time to respond Workload metricAverage time to respond Workload metric

• Wait TimesWait Times

• Result from deviations from “ideal” mission planResult from deviations from “ideal” mission plan

88

Human-Automation Interaction ModelingHuman-Automation Interaction Modeling

• First proposed by Olsen and Wood (2004) with regard to First proposed by Olsen and Wood (2004) with regard to traditional human-robot interactionstraditional human-robot interactions

Interaction Time (IT)Interaction Time (IT)

• The human operator is actively engaged in improving The human operator is actively engaged in improving the performance of the vehicle, allowing overall the performance of the vehicle, allowing overall mission accomplishment to occurmission accomplishment to occur

Neglect Time (NT)Neglect Time (NT)

• The vehicle is operating autonomously, needing no The vehicle is operating autonomously, needing no input from an operator to continue its missioninput from an operator to continue its mission

Wait Time (WT)Wait Time (WT)

• The vehicle needs input from the human to execute its The vehicle needs input from the human to execute its missionmission

99

Human-Automation Interaction ModelingHuman-Automation Interaction Modeling

Vehicle Effectiveness vs. Time

Interaction Time Neglect Time Wait Time Interaction Time Neglect Time

Performance Threshold

1010

Wait Time ModelingWait Time Modeling

• Wait times dramatically impact system performance Wait times dramatically impact system performance and risk ofand risk of failure in time-critical applications (eg. failure in time-critical applications (eg. CC22))

• Two main categoriesTwo main categories

• Workload Wait Times (W-WT)Workload Wait Times (W-WT)

−Result from operator overloadResult from operator overload

• Situation Awareness Wait Times (SA-WT)Situation Awareness Wait Times (SA-WT)

−Result from loss of situational awarenessResult from loss of situational awareness

• Work-in-progressWork-in-progress

• Need to accurately model then measure individual WT Need to accurately model then measure individual WT componentscomponents

• WT can be further broken downWT can be further broken down

1111

Key Experiment Display ElementsKey Experiment Display Elements – Mission Plan – Mission Plan

33:00 N

34:00 N

35:00 N

36:00 N

120:00 E 121:00 E 122:00 E 123:00 E

1

3

4

2

33:00 N

34:00 N

35:00 N

36:00 N

120:00 E 121:00 E 122:00 E 123:00 E

1

3

4

2

• Current mission Current mission plan for each UAV plan for each UAV is shownis shown

• UAV that operator UAV that operator is currently is currently interacting with interacting with highlighted in highlighted in greengreen

• Active targets = Active targets = red diamondsred diamonds

• Threat areas = Threat areas = yellow circlesyellow circles

• Way Points = Way Points = black trianglesblack triangles

• Loiter Points = Loiter Points = directed circlesdirected circles

1212

Key Experiment Display ElementsKey Experiment Display Elements - Decision Support - Decision Support

12:00:00ZCurrent Time

12:05:00Z5:00min

12:10:00Z10:00min

12:15:00Z15:00min

12:20:00Z20:00min

T-004-L

WP-2BLP-2A LP-2C

WP-1A

T-011-H

T-006-M

1

3

2

4

• Separate marching timeline for each UAVSeparate marching timeline for each UAV

• Represents mission plan as laid out on map displayRepresents mission plan as laid out on map display

• Current/future tasks color coded by actionCurrent/future tasks color coded by action

1313

Expected Research Results / BenefitsExpected Research Results / Benefits

• Validation of wait time modelsValidation of wait time models

• Conclusions on how different types of wait times influence Conclusions on how different types of wait times influence the overall cost function and fan outthe overall cost function and fan out

• Workload predictive model based on wait timesWorkload predictive model based on wait times

• Further results on the validity of an imbedded chat box as Further results on the validity of an imbedded chat box as a measure of secondary workloada measure of secondary workload

• An evaluation of the timeline decision support tool, with An evaluation of the timeline decision support tool, with comparisons across its various levels and effectiveness comparisons across its various levels and effectiveness under different re-planning conditionsunder different re-planning conditions

1414

MAUVE Demo & HAL Lab TourMAUVE Demo & HAL Lab Tour

• Questions?Questions?