Supporting European
Aviation
Interactive Performance DashboardsMaking sense of Big Human Performance Data in large-scale Real-Time SimulationsEUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’ Atlantic City, USA, 16/17 October 2019
Bernd LorenzHuman Performance & Validation Expert
Overview
How we do Real-Time Simulations (RTS)
The Challenge: Large-scale RTS
Examples of Interactive Performance Dashboards
The way forward
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 2
E-OCVM: Performance-driven Operational Concept Development
3
R&D
Full-scale RTS
PeopleSystem
Procedures
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019
Staff adaptation
Validation Activities in RTS Life-cycle
Kick-off MeetingState Concept and Assumptions
Validation StrategyIdentify stakeholders, issues, aims, objectives, claims, indicators, maturity criteria, cost-benefit mechanisms and expected outputs of the validation process
Validation Plan
2-stage Acceptance Testing
Simulation Conduct
Final ReportProvide a documented body of evidence in conclusion on the claimsInteractive Validation Data Repository
Core team Review
Core team Review
Core team Review
Final Review
Data Analysis&
Initial Reporting
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 4
Visualisation in Data Analysis and Claim Evaluation
Quantitative DataSubjective ratings
Platform recordings
Qualitative DataQuestionnaire comments
DebriefingsInterviews
SharedSense-Making
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 5
The challenge: Large-scale RTS
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019
165ATCOs
25Controller Working Positions
11Scenarios
88Exercises
1900Questionnaires
1335Debriefing log entries
616Interview
log entriesTMA re-organisation supporting Istanbul New Airport (2018)
6
Interactive Sector Performance Dashboard
(1) Sector SEN
7
Interactive Sector Performance Dashboard
(1) Sector SWF
8
Interactive Sector Performance Dashboard
(1) Sector SEL
9
Drilling into details … what exactly was the issue?
10
Drilling into further details … what exactly was the issue?
11
Dashboard: Targeted browsing through ATCO comments made in Post-Exercise-
Questionnaires
12
Dashboard: LOS analysis
13
13
Dashboard: LOS analysis
14
14
15
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019
Dashboard: Inspection of a more
complicated statistical evaluation of workload
16
17
Our experience: Interactive Performance Dashboards …
Provide an actionable insight into the collected data
Provide data visualisations at the right level of detail
Help to better grasp connections between different data-sets
Help to better communicate the data
Help to better prompt expert reasoning about operational issues
Support deeper human performance analysis
Make data integrity checks more efficient
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 18
The way forward
Data-base Management
Live-connect to the SIM platform to support online monitoring dashboards
Extending the Metric Toolbox
ATCO monitoring/information acquisition/intervention Eye-tracking Mouse-position/click tracking HMI/support tool usage
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 19
(2) Extending the metric tool box
20
Eye/mouse - tracking HMI input
Sector load
Fixed vs. Free Route
R/T activity
ISA –WL rating
Hilburn, 2004
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 21
Thank you for your attention
