ARTICLE IN PRESS Applied Ergonomics 38 (2007) 437–455 Human factors evaluations of Free Flight Issues solved and issues remaining Rob C.J. Ruigrok , Jacco M. Hoekstra National Aerospace Laboratory NLR, Anthony Fokkerweg 2, Amsterdam, The Netherlands Accepted 31 January 2007 Abstract The Dutch National Aerospace Laboratory (NLR) has conducted extensive human-in-the-loop simulation experiments in NLR’s Research Flight Simulator (RFS), focussed on human factors evaluation of Free Flight. Eight years of research, in co-operation with partners in the United States and Europe, has shown that Free Flight has the potential to increase airspace capacity by at least a factor of 3. Expected traffic loads and conflict rates for the year 2020 appear to be no major problem for professional airline crews participating in flight simulation experiments. Flight efficiency is significantly improved by user-preferred routings, including cruise climbs, while pilot workload is only slightly increased compared to today’s reference. Detailed results from three projects and six human-in-the-loop experiments in NLR’s Research Flight Simulator are reported. The main focus of these results is on human factors issues and particularly workload, measured both subjectively and objectively. An extensive discussion is included on many human factors issues resolved during the experiments, but also open issues are identified. An intent-based Conflict Detection and Resolution (CD&R) system provides ‘‘benefits’’ in terms of reduced pilot workload, but also ‘‘costs’’ in terms of complexity, need for priority rules, potential compatibility problems between different brands of Flight Management Systems and large bandwidth. Moreover, the intent-based system is not effective at solving multi-aircraft conflicts. A state-based CD&R system also provides ‘‘benefits’’ and ‘‘costs’’. Benefits compared to the full intent-based system are simplicity, low bandwidth requirements, easy to retrofit (no requirements to change avionics infrastructure) and the ability to solve multi-aircraft conflicts in parallel. The ‘‘costs’’ involve a somewhat higher pilot workload in similar circumstances, the smaller look-ahead time which results in less efficient resolution manoeuvres and the sometimes false/nuisance alerts due to missing intent information. www.elsevier.com/locate/apergo 0003-6870/$ - see front matter r 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.apergo.2007.01.006 Abbreviations: ACAS: Airborne Collision Avoidance System; ADS-B: Automatic Dependent Surveillance Broadcast; AENA: Spanish Air Traffic Service Provider; ALT: Altitude; ASAS: Airborne Separation Assurance System; ASOR: Allocation of Safety Objectives and Requirements; ATC: Air Traffic Control; ATCo: Air Traffic Controller; ATI: Air Traffic Information; ATM: Air Traffic Management; CAA: Civil Aviation Authority; CD: Conflict Detection; CD&R: Conflict Detection and Resolution system; CDTI: Cockpit Display of Traffic Information; CDU: Control & Display Unit; CNS: Communication, Navigation, Surveillance; DCP: Display Control Panel; DME: Distance Measuring Equipment; DNA: French Air Traffic Service Provider; ENAV: Italian Air Traffic Service Provider; FAA: Federal Aviation Authority (US); FFAS: Free Flight Airspace; FL: Flight Level; FLOS: Flight Level Orientation Scheme; FMS: Flight Management System; GCP: Generic Control Panel; GNSS: Global Satellite Navigation System; GRACE: Generic Research Aircraft Cockpit Environment; HCAA: Greek Civil Aviation Administration; HDG: Heading; HMI: Human–Machine Interface; INTENT: not an acronym, name of the project investigating the use of intent information; IRS: Inertial Reference System; ISA: Instantaneous Self- Assessment; LNAV: Lateral Navigation; LND: Lateral Navigation Display; MAS: Managed Airspace; MATS: Malta’s Air Traffic Service Provider; MCP: Mode Control Panel; MFF: Mediterranean Free Flight; NASA: National Aeronautics and Space Administration; NATS: Air Traffic Service Provider of the United Kingdom; ND: Navigation Display; NLR: National Lucht- en Ruimtevaartlaboratorium; nm: Nautical mile; OHA: Operational Hazard Assessment; PASAS: Predictive ASAS; PF: Pilot Flying; PFD: Primary Flight Display; PNF: Pilot Not-Flying; R/T: Radio Telephony; RFMS: Research Flight Management System; RFS: Research Flight Simulator (NLR); RNP: Required Navigation Performance; RSME: Rating Scale for Mental Effort; RTCA: Radio Technical Commission for Aeronautics; RTS: Real-time Simulation; SCAA: Scandinavian Air Traffic Service Provider (LFV in Swedish); SUA: Special Use Airspace; TIS-B: Traffic Information Service - Broadcast; TOPAZ: Traffic Organization and Perturbation AnalyZer; TRK: Track; TSA: Temporary Segregated Area; VHF: Very High Frequency; VND: Vertical Navigation Display; VOR: VHF Omni-directional Range; V/S: Vertical Speed. Corresponding author. Tel.: +31 20 511 3595; fax: +31 20 511 3210. E-mail address: [email protected] (R.C.J. Ruigrok).
Transcript
ARTICLE IN PRESS
0003-6870/$ - se
doi:10.1016/j.ap
Abbreviations
Service Provide
Traffic Control
Conflict Detect
CNS: Commun
Provider; ENA
Flight Level Or
Generic Resear
INTENT: not a
Assessment; LN
MCP: Mode C
Provider of the
Hazard Assessm
Research Flight
Effort; RTCA:
Swedish); SUA
Track; TSA: Te
Vertical Speed.�CorrespondE-mail addr
Applied Ergonomics 38 (2007) 437–455
www.elsevier.com/locate/apergo
Human factors evaluations of Free FlightIssues solved and issues remaining
Rob C.J. Ruigrok�, Jacco M. Hoekstra
National Aerospace Laboratory NLR, Anthony Fokkerweg 2, Amsterdam, The Netherlands
Accepted 31 January 2007
Abstract
The Dutch National Aerospace Laboratory (NLR) has conducted extensive human-in-the-loop simulation experiments in NLR’s
Research Flight Simulator (RFS), focussed on human factors evaluation of Free Flight. Eight years of research, in co-operation with
partners in the United States and Europe, has shown that Free Flight has the potential to increase airspace capacity by at least a factor of
3. Expected traffic loads and conflict rates for the year 2020 appear to be no major problem for professional airline crews participating in
flight simulation experiments. Flight efficiency is significantly improved by user-preferred routings, including cruise climbs, while pilot
workload is only slightly increased compared to today’s reference.
Detailed results from three projects and six human-in-the-loop experiments in NLR’s Research Flight Simulator are reported. The
main focus of these results is on human factors issues and particularly workload, measured both subjectively and objectively. An
extensive discussion is included on many human factors issues resolved during the experiments, but also open issues are identified.
An intent-based Conflict Detection and Resolution (CD&R) system provides ‘‘benefits’’ in terms of reduced pilot workload, but also
‘‘costs’’ in terms of complexity, need for priority rules, potential compatibility problems between different brands of Flight Management
Systems and large bandwidth. Moreover, the intent-based system is not effective at solving multi-aircraft conflicts. A state-based CD&R
system also provides ‘‘benefits’’ and ‘‘costs’’. Benefits compared to the full intent-based system are simplicity, low bandwidth
requirements, easy to retrofit (no requirements to change avionics infrastructure) and the ability to solve multi-aircraft conflicts in
parallel. The ‘‘costs’’ involve a somewhat higher pilot workload in similar circumstances, the smaller look-ahead time which results in less
efficient resolution manoeuvres and the sometimes false/nuisance alerts due to missing intent information.
e front matter r 2007 Elsevier Ltd. All rights reserved.
The optimal CD&R system (in terms of costs versus benefits) has been suggested to be state-based CD&R with the addition of
intended or target flight level. This combination of state-based CD&R with a limited amount of intent provides ‘‘the best of both
worlds’’. Studying this CD&R system is still an open issue.
r 2007 Elsevier Ltd. All rights reserved.
Keywords: Free Flight; Human factors; Real-time simulations
1. Introduction
1.1. Today’s problems
We experience the problems of today’s Air TrafficManagement (ATM) system every day. Airspace capacityis often insufficient in relation to traffic demand, resultingin delays. Flight efficiency is non-optimal since airspaceusers are unable to fly their preferred routes. Moreover, itappears that safety can be improved, given recent accidentsand incidents.
1.2. How things evolved
In the early days of flying, pilots navigated using groundfeatures such as roads and rail tracks. By looking outside,collisions with other aircraft were avoided. In those days,visual contact with the ground and other aircraft wasrequired. The introduction of radar allowed flying withoutoutside visual reference. Air Traffic Control (ATC) becameresponsible for separation, while navigation beaconsallowed navigation without ground visibility. The current,world-wide ATM system is still using these technologies.
Fig. 1. Rating scale of mental effort (RSME).
1.3. Definition and expectation
Modern navigation no longer relies on flying to and fromground-based navigation beacons, while new technologiesare being developed which allow pilots to ‘‘see’’ otheraircraft electronically. These developments have led to thedefinition of Free Flight by the Radio Technical Commis-sion for Aeronautics (RTCA) as (Anonymous, 1995): ‘‘ya
safe and efficient flight operating capability under instrument
flight rules in which the operators have the freedom to select
their path and speed in real timey’’. The definition of FreeFlight was from 1999 onwards included in the Europeanoperational concept (Anonymous, 1999a) for the future ofthe ATM system. The goal of Free Flight is to providemore flexibility for aircraft operators, while at the sametime improving safety and airspace capacity. Free Flightwill change the ATM system significantly: from a centrallycontrolled system to a distributed system, where pilots willreceive the task and responsibility for separation assurancewith other aircraft, as in the old days. In order to take thisresponsibility, the pilot will be supported by enhancementsto Communication, Navigation, and Surveillance (CNS)equipment as well as by airborne Conflict Detection andResolution (CD&R) systems.
2. Studies performed
Before discussing NLR’s human factors evaluations ofFree Flight in detail, the next two sections summarise thepilot workload measurements as used in the evaluationsand the validation platform used in the experiments,respectively.
2.1. Pilot workload measurements in studies
2.1.1. Pilot subjective workload, acceptability and safety
Pilot subjective workload, acceptability and safety aremeasured by questionnaires, during or after an experimentrun or experiment. For workload, the Rating Scale ofMental Effort (RSME) is used, see Fig. 1.The RSME is a validated scale for measuring
mental workload (Zijlstra and Doorn, 1985). Acceptabilityand safety were measured by questions using 5-pointscales for the answers, with headings: ‘‘perfect’’, ‘‘favour-able’’, ‘‘acceptable’’, ‘‘undesirable’’ and ‘‘completelyundesirable’’.
Another way to measure pilot workload is by means ofInstantaneous Self-Assessment (ISA) techniques. Eachpilot is asked to rate his/her own workload every 2min.This was done in the Research Flight Simulator (RFS) byselecting a button on the Control and Display Unit (CDU)of the Flight Management System (FMS) on which a‘‘Workload’’ page would pop-up every 2min (for aduration of 30 s), see Fig. 2.
For each experiment run, a series of ISA measurementscan be obtained per participant pilot. Workload evolutionand average workload over a run can be measured thisway.
The overall workload of a run is the mean of various ISAmeasurements. The participant does not rate the completerun in one rating, but it is an evolution of a series of moreor less independent ratings. This method is still subjective,but the resulting overall workload is not one simple ratingfor the participant.
2.1.3. Pilot objective workload
Visual scanning randomness—or entropy (as the term isused in thermodynamics, to describe the amount ofdisorder present in a system)—can be used to describethe randomness present in the visual scan of the participantpilots. The rationale behind the use of the entropy measureis that visual scan patterns become more stereotypical (less
Fig. 2. CDU workload pop-up page.
random) with mental loading (Tole et al, 1983; Harris et al,1986), so entropy should decrease with increased task load.Visual scan randomness can be measured using eye-point-of-gaze measurement equipment, see Fig. 3.
2.2. Validation platform
The validation platform used in the experimentsdescribed in this section was the RFS of NLR inAmsterdam, see Figs. 4 and 5.The RFS was a generic flight simulator and represented a
modern large airliner. The configuration during theexperiments was Boeing 747, with Boeing 747-400 en-hanced displays and a simulated Boeing 747-400 FMS. TheRFS has since the beginning of 2005 been replaced by aneven more flexible and versatile Generic Research AircraftCockpit Environment (GRACE) flight simulator. GRACEhas full interchangeable cockpits (Airbus, Boeing, Fokker,generic), an advanced visual system and an electrical 6degrees-of-freedom motion system, see Fig. 6.
2.3. NLR/NASA Free Flight
Free Flight has been studied at NLR since 1996. Theresearch started in co-operation with the National Aero-nautics and Space Administration (NASA) Ames andLangley research centres, the United States FederalAviation Administration (FAA) and the Dutch CivilAviation Administration (RLD) (Hoekstra, 2001). Theprimary focus of the research was to explore human factorsissues of the Free Flight concept. In order to identify thehuman factors issues, several sub-studies were performed:
�
Conceptual design and off-line validation � Safety analysis � 1997 Human-in-the-loop experiment—‘‘explore issues’’ � Cost/benefit analysis � Avionics requirements study � Critical conflict geometry study
ARTICLE IN PRESS
Fig. 4. NLR’s Research Flight Simulator, outside.
Fig. 5. NLR’s Research Flight Simulator, inside.
Fig. 6. NLR’s Generic Research Aircraft Cockpit Environment
1998 Human-in-the-loop experiment—‘‘mixed equi-page’’ � 2000 Human interaction experiment � 2001 Human-in-the-loop experiment—‘‘how low can
you go?’’
� 2001 Classroom experiment
This section will report on the results from the 1997, 1998and 2001 human-in-the-loop experiments. Before this, theAirborne Separation Assurance System (ASAS) will bedescribed.
2.3.1. Airborne Separation Assurance System
The ASAS in the NLR/NASA Free Flight experimentsconsisted of:
�
Automatic Dependent Surveillance—Broadcast (ADS-B) system � Cockpit Display of Traffic Information (CDTI) on
Navigation Display (ND)
� CD&R algorithms � Predictive ASAS (PASAS) � Alerting logic
The ADS-B system (Anonymous, 1998; Barhydt andWarren, 2002) is considered to be an element of the ASASequipment. ADS-B provides a digital data link betweenaircraft by which they can electronically ‘‘see’’ each otherwith high accuracy. Equipped aircraft will have ADS-B butare also able to receive Traffic Information Services—Broadcast (TIS-B) information. The TIS-B information isassumed to contain similar data as the ADS-B information,in the same format as ADS-B but generated by groundstations using radar data.The ADS-B traffic information is displayed on the ND,
roughly in the same way the Airborne Collision AvoidanceSystem (ACAS) displays traffic today. Since additionalinformation on the other traffic is available with ADS-B,the traffic symbology is slightly enhanced compared toACAS symbology. The following information is shown,both on the Horizontal Navigation Display as well as onthe Vertical Navigation Display (VND), which has beenintegrated below the Horizontal Navigation Display in thecockpit:
The traffic information on the displays can be controlledwith Generic Control Panels (GCP), located in the pedestalarea.
CD&R functionality of the ASAS system was based onaircraft state information only. This means CD&R usesposition plus three-dimensional speed vector informationfrom its own position and other traffic to detect and resolveconflicts. Conflicts are detected up to 6min ahead of theaircraft. The closest point of approach between ownship(own aircraft) and an intruder (other aircraft) is calculated,and if the closest point of approach is below the separationstandard, a conflict will be shown. The separation standardused is 5 nm horizontally and 950 ft vertically. This resultsin a so-called ‘‘protected zone’’ around each aircraft of5 nm and 7950 ft.
After thorough study in the concept design and offlinevalidation study (studying and testing CD&R algorithmsas listed by Kuchar and Yang, 1997), it was decided to basethe conflict resolution algorithm on the so-called ModifiedVoltage Potential theory (Eby, 1994; Hoekstra, 2001). Theconflict resolution function will give heading (HDG) adviceand vertical speed (V/S)+altitude (ALT) advice, the‘‘shortest way out of the conflict’’. Fig. 7 shows theownship and the position of the protected zone of theintruder at closest point of approach. The rotated speedvector to the edge of the intruder’s protected zone solvesthe conflict. This will require a heading change asindicated. Similarly, a vertical resolution is calculated.
Fig. 7. Conflict detection and resolution.
Both horizontal and vertical options will solve the conflict,so the crew can chose which option to implement. Ingeneral, a vertical change is preferred since the separationstandards are 5 nm horizontally and 950 ft vertically, sovertically takes a much smaller change than horizontally.Moreover, the autopilot Lateral Navigation mode (LNAV)can remain engaged if the vertical option is used.The conflict resolution algorithms aim at a separation of
5.5 nm and 1000 ft, so introducing a buffer in betweenconflict detection and conflict resolution logic. Within theNLR/NASA Free Flight experiments, it was assumed thatboth aircraft in the conflict would manoeuvre, therebycreating a so-called co-operative concept. Both aircraftmanoeuvre and solve (part of) the conflict. Moreover, incase of multiple conflicts (conflict with more than oneaircraft at the same time) within the look-ahead time, theresolution vectors are summed, thereby creating a CD&Rsystem capable of solving multiple conflicts. This methodfulfilled the following important requirements on theCD&R system as defined within the NLR/NASA FreeFlight project:
�
Safe, because initially both aircraft manoeuvre to avoidthe other as if the other will not react. The safety of theconcept was further investigated in an evaluation usingNLR’s Traffic Organization and Perturbation AnalyZer(TOPAZ), showing it to be safer than present day ATC(Daams et al., 1998), expressed in the chance of twoaircraft colliding. � Transparent, because the geometry of the conflict, if
shown on a display, leads indubitably to the resolutionalgorithm.
� Capable of handling multiple encounters.
Experience in the first experiment has revealed that anASAS system should also be capable of conflict prevention,i.e. not introduce conflicts while manoeuvring the aircraft.This resulted in the development of PASAS. PASAScalculates which headings and vertical speeds will resultin a conflict with another aircraft within the look-aheadtime (6min). The results of these calculations are shown as‘‘bands’’ on the Primary Flight Display (PFD) and ND. Infact, PASAS generates ‘‘no-go’’ bands on the vertical speedtape and the heading scale.A distinction is made between potential conflicts within
0–3min away from intrusion, displayed in amber (orange),and potential conflicts between 3 and 6min away fromintrusion, displayed in yellow. An extra rule for all pilots isintroduced: pilots are not allowed to turn, climb or descendinto an amber band. Moreover, pilots are not allowed tostay in a yellow band, but crossing a yellow band isallowed. The result is that intrusions due to sudden aircraftmanoeuvres nearby, like an aircraft reaching a top-of-descent, are prevented. Figures of a PFD and ND with‘‘no-go’’ bands are shown in Figs. 8 and 9. These figuresalso clearly show the CD&R symbology, in line with Fig. 7.
ARTICLE IN PRESS
Fig. 8. CD&R and PASAS symbology on PFD (PASAS no-go bands are
shown on the HDG scale, lower centre arc, and on the vertical speed tape,
most right tape. CD&R information is shown at the same places, as
magenta indicators).
Fig. 9. CD&R and PASAS symbology on ND (PASAS no-go bands are
shown on the HDG scale: upper centre arc. CD&R information is shown
at the display centre: the circles around ownship and around the projected
position of the intruder at closest point of approach, together with the
resolution advisory, the dashed heading advisory from ownship to the
The implementation defined by the conceptual design(Hoekstra, 2001) and the safety analysis (Daams et al.,1998) was tested in a human-in-the-loop simulationexperiment in 1997 (Hoekstra, 2001). The goal of thisexperiment was to determine the human factors issues ofoperating an aircraft in a Free Flight environment withairborne separation assurance. For this reason, an extremeversion of airborne separation assurance was chosen: noATC and full responsibility for traffic separation with theaircrew on board the aircraft.
In the 1997 human-in-the-loop experiment, the trafficdensity, the level of automation and nominal/non-nominalconditions were varied as independent variables. Nominalin this case means normal; in non-nominal conditionsspecial events occurred such as ASAS failures and pilotsnot obeying the rules. The traffic densities used in theexperiment were one, two and three times mean Western-European traffic density (mean daytime traffic density overBelgium in 1996). Three levels of automation were used forresolution activation via the aircraft autopilot:
�
Manual: no automation: use normal select mode forheading, vertical speed and/or speed manually. � Separate: two modes available: vertical or horizontal
manoeuvre.
� Combined: one mode performs combined horizontal
and vertical manoeuvre.
Combinations of traffic densities and levels of automationwere tested in nominal and non-nominal conditions. The
non-nominal conditions consisted of own and otheraircraft system failures and increased delay times inCD&R.With positive results obtained from the 1997 human-in-
the-loop study, a second human-in-the-loop study wasdefined to explore the human factors issues of severalsolutions of the future ATM system, covering bothtransitions in time (mixed equipage, i.e. aircraft equippedfor Free Flight operations and aircraft not equipped forthese operations) and transitions in space (Hoekstra, 2001).The transition to Free Flight AirSpace (FFAS) (in space)was studied using three different ATM operationalconcepts or scenarios, especially designed for this study:‘‘vertical split’’, ‘‘protected airways’’ and ‘‘full mix’’, seeFig. 10.In 2000, a follow-on experiment was conducted to find
the answer to a remaining research question of ‘‘humaninteraction’’. The previous human-in-the-loop experimentshad one ‘‘human’’ crew involved, while the other trafficwas controlled automatically. Since the behaviour of theautomatic controlled traffic was rather ‘‘ideal’’, thequestion was what will happen when more aircraft are
ARTICLE IN PRESS
Fig. 10. Free Flight Airspace definitions.
Fig. 11. Pilot subjective workload (x-axis shows factor of 1996 mean
Western European traffic density, y-axis shows mean and 95% confidence
controlled by humans. The ‘‘human interaction’’ experi-ment was born. In this experiment, up to 25 pilots flew anASAS equipped desktop flight simulator (Free FlightSimulator, FreeSIM, see http://www.nlr.nl/public/hosted-sites/freeflight/main.htm, download section) simulta-neously from their home personal computer, connectedto a central Traffic Manager located at NLR in Amster-dam. In 2001, this experiment was repeated in a morecontrolled pilot school classroom environment, in which allparticipants received the same training.
The experiments conducted so far focussed on FreeFlight in the en-route phase of flight. One of the openresearch questions from the experiments was until whichaltitude one could perform Free Flight. A new experimentwas born, questioning ‘‘how low can you go?’’ with FreeFlight. In this 2001 human-in-the-loop experiment, eightcrews were asked to fly a Boeing 747 into SanFrancisco International airport, Free Flight until the finalapproach fix.
Fig. 12. Questionnaire results of pilot workload.
2.3.3. Results
Pilot subjective workload appears to increase slightly,but remains well within acceptable levels, even in hightraffic densities. Fig. 11 shows pilot subjective workload asa function of traffic density, as derived from the 1997human-in-the-loop experiment (Hoekstra, 2001).
Fig. 12 shows the results obtained with workloadquestionnaires and Fig. 13 with scan randomness measure-ments during the 1998 human-in-the-loop experiment onFree Flight. Shown is a two-way interaction of ATCcondition and equipage, using ANOVA techniques (F-test).
As can be seen, the results are very similar and indicatethat the ‘‘full mix’’ concept was most appreciated by pilots.Note the reverse impact of scan randomness: lowerworkload is associated with more scanning randomness.However, experiments conducted with Air Traffic Con-trollers in the same project (Hilburn and Pekela, 1999) andwith the same concepts revealed that controller cannothandle the ‘‘full mix’’ concept, given the unexpected
behaviour of the Free Flight aircraft. Taking into accountthe feedback from the Air Traffic Controllers, the ‘‘verticalsplit’’ concept seems the most appropriate concept fortransitions.
Eye-point-of-gaze objective data in the same experimentshowed that participant pilots spent about 50% of theirtime looking at the Lateral Navigation Display (LND).This is higher than in today’s cruise operation and couldtherefore be of concern. However, it is not clear whetherthese high fixation times are required to operate in thisconcept. Further, the eye-point-of-gaze objective datasuggested that the VND was not used or only used on avery limited basis.
An interesting effect observed during the NLR/NASAFree Flight experiments was the behaviour of pilots withrespect to the co-operative state-based CD&R concept.Pilots learned during the experiment runs that the firstaircraft to manoeuvre during a conflict would often solvethe conflict completely. This sometimes resulted in waitingfor ‘‘the other’’ aircraft in the conflict to manoeuvre. Thiseffect of the co-operative, state-based CD&R concept isconsidered undesirable since it could potentially lead to aserious safety issue: both aircrew waiting for the otheraircrew to manoeuvre.
2.4. INTENT
Although the results of the experiments so far with state-based ASAS were very positive, many people argued thatintent-based ASAS (using the aircraft FMS generated flightplans) would significantly improve pilot workload andsafety, as compared to only using aircraft state informa-tion. The INTENT project was defined and co-funded bythe European Union and the INTENT consortium (NLR,QinetiQ, ONERA, Rockwell Collins, Smiths Aerospace,
AIRBUS, Eurocontrol, Delft University of Technology,Dutch Airline Pilot Association VNV, KLM Royal DutchAirlines, British Airways and Scandinavian AirlinesSystem). Within the INTENT project, two human-in-the-loop simulations were conducted to study intent-basedASAS (Ruigrok et al., 2003), as will be discussed in thissection.
2.4.1. ASAS
As part of the aircraft’s CD&R system, the conflictdetection module detects only conflicts with aircraft forwhich an intrusion of the protected zone takes place withina predetermined look-ahead time. This potential intrusion,or conflict, can be detected based on the aircraft stateinformation (ground speed, track and vertical speed) asused in the NLR/NASA Free Flight experiments, or basedon the aircraft intent information. This implies that thereare in fact two types of conflicts, a state-based conflict andan intent-based conflict. Hence, calculated resolutionadvisories must match the type of the detected conflict.The conflict resolution module therefore makes a distinc-tion when calculating and presenting resolution advisories.For state-based conflicts the resolution module uses the
state-based Modified Voltage Potential algorithms as usedin the NLR/NASA Free Flight experiments. For intent-based conflicts the resolution is calculated and presented asan amendment to the active route. Fig. 14 shows a conflictsituation in which the ownship has detected a conflict usingthe intent-based conflict detection method.It is clear from the highlighted loss of separation that the
method has taken into account the intent (flight plans) ofboth aircraft. Based on the conflict geometry and theaircraft flight plans, the resolution module can nowdetermine a route change that will resolve the conflict.The figure illustrates how the addition of a resolutionwaypoint resolves the conflict in the horizontal plane. Thishorizontal resolution includes automatically a recoverymanoeuvre, represented by the leg after passing theresolution waypoint.Another option to resolve this conflict would be an
altitude change in the flight plan of the ownship, as shownin Fig. 15.With intent-based CD&R priority rules were used
instead of a co-operative solution for conflict resolutions,due to discontinuous (not continuously updated) resolu-tions, as explained in Hoekstra et al. (1999). This meansthat one aircraft in the conflict was assigned priority, andthe other aircraft in the conflict was expected tomanoeuvre.The purpose of conflict prevention is to provide pilots
with additional situation awareness with respect topotential conflicts. The conflict prevention module deter-mines if manoeuvres are conflict free. In the INTENTsimulation set-up, conflict prevention indications wereprovided for both the state-based and the intent-basedconcept. The state-based conflict prevention indicationsconsisted of the PASAS ‘‘no-go’’ bands as also used in the
NLR/NASA Free Flight experiments. Intent-based conflictprevention consisted of an additional function to the FMS,which checked all route changes, either suggested by thepilot manually or by the CD&R system to solve a conflict.When the suggested route change did not generate (new)conflicts, the pilots were able to activate the route change.When the route change generated (new) conflicts within thelook-ahead time of the intent-based CD&R system, thepilots were not allowed to activate this route.
The conflict prevention strategy, which was also appliedfor the conflict resolution manoeuvres, resulted sometimesin the inability of the intent-based CD&R algorithms tofind a resolution manoeuvre within the look-ahead time ofthe CD&R system. This was due to a combination of trafficdensity and large look-ahead times. When the intent-basedCD&R system was unable to find a solution for theconflict, the pilots had the option to fall back on the state-based CD&R system. When using the state-based CD&Rsystem, the pilots would have to give way to intent-basedCD&R aircraft.
2.4.2. Experiments
Within the INTENT project, two human-in-the-loopexperiments were conducted in the RFS. The aim of thefirst INTENT human-in-the-loop experiment was to derivea human operator model for fast-time simulations based onmeasurements during the simulations and to get feedbackon the use of an ASAS incorporating aircraft intent
information, used in the en-route phase of flight. The aimof the second airborne experiment was to validate theoutcome from the fast-time simulations and the part-tasksimulations, extending the scope of the flight to departureand arrival phases of flight.The first INTENT experiment was a 4� 3 within-
subjects design, varying the following factors:
�
four operational concepts: state-based with 5min look-ahead time, intent-based with 5, 10 and 20min look-ahead time; � three traffic loads: 1� , 2� and 3� today’s traffic.
A total of 12 experiment sessions were run per participantcrew (captain and co-pilot). The pilots were asked to fly anASAS equipped aircraft on a pre-programmed FMS routeand to be responsible for the separation with other aircraft.The experiment consisted of a flight from London
Heathrow (EGLL) to Munich (EDDM). The flight startedat cruise altitude (FL280) just before the Belgian coast,with a planned route to Munich already implemented inthe FMS. The route was guidelined and determined by ‘‘theairline’’, but since the flight was performed in FFAS, thisroute could be freely altered by the crew. The onlyconstraints were active military areas, which were clearlydepicted on the CDTI. Traffic samples were created fortraffic density 1, 2 and 3. In these traffic samples, all trafficavoided the active military areas.
In the second INTENT experiment, it was decided toexpand the experiment domain by shifting focus from thecruise phase to the descent and climb phases. It wasexpected that workload ratings and acceptability wouldsignificantly rise and challenge the state-based and intent-based concepts. The first INTENT simulations clearlyindicated the effect of the traffic intent information level(see next section). It was therefore decided to only comparethe state-based system, using 5min look-ahead time, withthe intent-based using 10min look-ahead time. Lessonslearned from the previous experiment were incorporated inthe CD&R system, and enhancements to the Human–Ma-chine Interface (HMI) were made.
The second INTENT experiment was a 2� 2� 3 within-subjects design varying the following factors:
�
two operational concepts—state-based ASAS with 5minlook-ahead time, intent-based ASAS with 10min look-ahead time; � two flight phases—climb and descent; � three traffic loads—1� , 2� and 3� today’s traffic.
A total of 12 experiment sessions, divided over two days,were run per crew. Two experiment flights were flown: onefor the climb phase and another for the descent. The firstflight was from Frankfurt (EDDF) to New York (KJFK).It started in the climb phase at FL120, with a flight plan toNew York already implemented in the FMS. FFAS wasassumed over Europe above FL100. The flight included theclimb-out to cruise level (FL330) and part of the cruiseflight. The second experiment flight was from New York(KJFK) to Frankfurt (EDDF). This flight started in thecruise phase at FL390 about 10min before top-of-descent,
Fig. 16. Concept Effect on RSME for all traffic loads (on x-ax
with a flight plan to Frankfurt already implemented in theFMS. This second flight included a part of the cruise anddescent towards Managed Airspace (MAS) (FL100).
2.4.3. Results
Analysis of the measurements from the first INTENTexperiment shows that both the state-based and the intent-based concepts are considered workable and acceptable (orbetter) for all traffic loads. Fig. 16 illustrates that the intentwith 10min look-ahead time results in the lowest workload(using MANOVA techniques).The traffic loads had a significant effect on the RSME.
Nevertheless, ratings for all traffic loads were below 40(‘‘costing some effort’’). Debriefing results indicated theintent-based system as a preferred solution. Nevertheless,the option to fall back in the state-based mode, as wasavailable during the experiment, was considered as arequired back up (‘‘don’t leave home without it’’). TheISA measurements were used to derive workload modelsfor the aircrew, which were input to fast-time simulationexperiments (Magill and Platt, 2002).The second INTENT experiment, with focus on climb
and descent phases of flight, revealed that the traffic loadhas a significant effect on the pilot workload andacceptability. Moreover, it is confirmed that workloadsignificantly rises with respect to cruise flight. Nevertheless,traffic loads 1 and 2 are still considered to be workable,with RSME ratings below 50 (‘‘rather effortful’’) andacceptability ratings ‘‘acceptable’’ or better than ‘‘accep-table’’. Traffic load 3 seems to become a problem withRSME rating towards ‘‘60’’ and acceptability ratingsslightly less than ‘‘acceptable’’.
is: i ¼ intent, s ¼ state, value is minutes look-ahead time).
ARTICLE IN PRESS
Fig. 17. Flight phase effect on RSME (on x-axis: c ¼ climb phase, d ¼ descent phase).
Another significant result is shown in Fig. 17 (usingMANOVA techniques). The figure illustrates that theworkload for both concepts is lower in descent phase offlight. The same result is found for the acceptability. Thiseffect is surprising, since it was expected that descending,converging (horizontally) flights would generate moreworkload than climbing, diverging flights. However,climbing flights appear to be vertically converging towardsthe popular cruise levels, an effect that appears significant.
A surprising result is that there was no significantconcept effect on the workload. This is explained by thefact that intent-based CD&R ‘‘blocks’’ more airspace(combination of look-ahead time and conflict prevention),resulting often in fall-backs to state-based CD&R.
In conclusion, intent-based CD&R reduces pilot work-load in en-route phases of flight. In climb and descentphases of flight the ‘‘blocking’’ airspace effect of the longerlook-ahead time combined with the conflict preventionstrategy often prevents the use of intent-based CD&R.Together with the inability to handle multi-aircraftconflicts efficiently (processed serially in time), the requiredhigh bandwidth (up to 10 Trajectory Change Pointsrequired) and the overall complexity of the intent-basedCD&R system, the added value of the intent-based CD&Rsystem compared to the state-based CD&R system (whichis needed anyway) is low.
2.5. Mediterranean Free Flight (MFF)
The Mediterranean Free Flight (MFF) Program is aEuropean project led by ENAV in partnership withAENA (Spain), DNA (France), Eurocontrol, HCAA(Greece), MATS (Malta), NATS (UK), NLR (NL) and
SCAA-LFV (Sweden). The main objectives of MFF arefocused on CNS/ATM technologies, Free Route andASAS applications. Thanks to the foresight and fundingprovided by the European Commission, MFF is aneffective exercise in international co-operation that devel-oped European consensus on new innovative ATMtechniques. It enabled experts from all over Europe toco-operate in overcoming the operational and technicalhurdles to defining and evaluating future ATM conceptsand led to the development of advanced simulationfacilities and techniques. The first hurdle for MFF was tostructure its scope, and the partners agreed to focus on aseries of increasingly innovative and challenging applica-tions. Because of its wide scope, MFF has produceddetailed results concerning the feasibility of a large range ofapplications leading towards user-preferred flight trajec-tories, redistribution of tasks between controllers andaircrew and making use of the possibilities offered byADS-B technology.Within MFF, two human-in-the-loop simulations have
been conducted (Ruigrok, 2003; Ruigrok et al., 2004) onFree Flight (or ASAS Self-Separation, Anonymous, 2001)using NLR’s RFS. The first experiment specificallyinvestigated the procedures defined within MFF fortransitions between MAS and FFAS, using the ‘‘verticalsplit’’ as previously tested in the NLR/NASA Free Flightexperiment. For the second MFF human-in-the-loopsimulation on Free Flight, it was decided to investigatethe effect of weather, military activities and failures(system failures and pilots failing to comply with the FreeFlight rules) on the Free Flight concept. Further,training needs for ASAS have been assessed, as well asthe need for a VND.
Given the experience from the INTENT experimentswith intent-based CD&R, it was decided in the MFFproject to study state-based CD&R again as used in theNLR/NASA Free Flight experiments. In order to avoid theproblem of full co-operative state-based CD&R asobserved in the NLR/NASA Free Flight experiments, itwas decided to try to combine state-based CD&R andpriority rules. Priority rules would clearly assign oneaircraft to manoeuvre, thereby avoiding the problem ofaircrews waiting for the other aircraft to manoeuvre first.However, the good aspects of co-operative CD&R shouldremain: safety (if one aircraft does not manoeuvre, theconflict should still be solved), transparent (pilots shouldunderstand why and how to resolve conflicts) and theASAS system should be capable of handling multi-aircraftconflicts. This has resulted in the development of the state-based CD&R system with two phases: a priority rulesphase, and a co-operative phase.
Priority rules are used to indicate which of the aircraftinvolved in a conflict needs to manoeuvre. The followingpriority rules are used, in this order:
1.
Aircraft with priority switch ON has priority overaircraft with priority switch OFF. When Free Flightmode is selected, a ‘‘priority switch’’ is available on theCDU of the FMS. The priority switch is available incase of failures for which ownship would require priorityover other traffic.
2.
Most manoeuvrable aircraft has to give priority overless manoeuvrable, based on manoeuvre categories. Allair transport aircraft have the same category.
3.
Intent CD&R equipped aircraft has priority over astate+target state CD&R equipped aircraft, state+tar-get state CD&R equipped aircraft has priority over astate only CD&R equipped aircraft.
4.
Level aircraft has priority over climbing and descendingaircraft.
5.
Descending aircraft has priority over climbing aircraft. 6. If two aircraft are level, aircraft obeying Flight level
Orientation Scheme (FLOS) has priority over aircraftnot obeying FLOS.
7.
If two aircraft are in the same flight phase, theovertaking aircraft should give priority. Overtaking isdefined as an aircraft on an aspect angle from 150 to 210degrees of ownship.
8.
If two aircraft are in the same flight phase, the aircraftfrom the right has priority. From the right is defined asan aircraft on an aspect angle from 0 to 150 degrees ofownship.
9.
If none of the above apply, the aircraft callsign will beused to determine priority, based on a deterministic rule.
In case of the state-only or state+target state CD&Rsystem, additional rules are added for safety reasons, dueto the relatively short look-ahead time of 6min:
�
In between 6 and 3min from intrusion of the protectedzone of the other aircraft, the aircraft not receivingpriority shall manoeuvre. The aircraft receiving priorityis allowed to manoeuvre. � In between 3 and 0min from intrusion, both aircraft
shall manoeuvre.
In case of the state or state+target state CD&R, thedirection of the Resolution Advisory (RA) should alwaysbe respected, since the ASAS of the other aircraft will alsodetermine the conflict and will also receive resolutionsassociated with ‘‘the shortest way out’’ of the conflict.In the state-only CD&R system, four alert levels can be
identified:
�
6–3min, with priority: green � 6–3min, without priority: yellow � 3–0min: amber � o0min (intrusion): red
This means:
�
Crew action required at alert level yellow, amber andred. � Crew action allowed at alert level green.
The conflicts and resolutions are shown on the PFD andND and the crew is alerted by an aural and visual alert inthe glareshield. The displays use the above colour coding todepict the threat level. The aural alerts used vary dependingon the alerting level of the conflict (green/yellow/amberconflict). The CD&R symbology is shown on the PFD inFig. 8 and on the ND in Fig. 9, in this case an ‘‘amber’’alert (within 3min, both aircraft shall manoeuvre).Although the ASAS implementation in avionics is not
yet completely specified, it is possible to give a high levelfunctional architecture. The navigation system in theaircraft is fed with several independent sources forpositioning estimates, typically redundant Inertial Refer-ence Systems (IRS), Global Navigation Satellite System(GNSS) and Very High Frequency (VHF) Omni-direc-tional Range (VOR)/Distance Measuring Equipment(DME) sensors. ASAS receives input from and sendsoutput to the ADS-B transponder. ACAS plays more orless the same function in the Free Flight concept as it playsin today’s MAS: it is a safety-net, in case other means toprovide separation between aircraft apparently have failed.ACAS is provided with Mode-S data and can beconsidered functionally independent from ASAS. TheCDTI is fed with two sources of traffic information: ASASinformation derived from ADS-B on one hand and ACASinformation derived from Mode-S on the other hand. Nodata fusion takes place, only prioritisation, meaning thatACAS information is only shown on the CDTI in caseASAS surveillance information is unavailable and in caseof ACAS alerts.
The first MFF experiment specifically investigated theprocedures defined within MFF for transitions betweenMAS and FFAS, using the previously tested ‘‘verticalsplit’’. Four different transitions were tested:
For the MFF evaluations, FFAS was assumed to be locatedabove FL285, west of Italy. East of Italy was assumed to beMAS. The horizontal transitions took place at waypointANEDA, east of Rome. At ANEDA, transitions took placeboth ways (FFAS-MAS and MAS-FFAS). A singlealternate FLOS was applicable: westbound used ‘‘even’’flight levels (FL300, FL320, etc.) and eastbound used ‘‘odd’’flight levels (FL310, FL330, etc.). The vertical transitionswere tested inbound and outbound Rome Fiumicino airport(LIRF). FFAS is located above FL285, MAS below FL275.At FL280 there was a ‘‘transition’’ level. Traffic was onlyallowed to fly at FL280 after ATC clearance. This flight levelprovided a buffer in between FFAS and MAS and wasconsidered FFAS. In FFAS the crew could freely alter theroute. The cruise altitude was determined, based on theFLOS as applicable in MAS. In FFAS, the crew was free tochange the cruise altitude, but it was highly recommended tochoose a cruise altitude in line with the FLOS.
The following programme of experiment runs wasexecuted per crew:
�
Four reference scenarios:J two conventional horizontal transitions from MAS
to MAS;J two conventional vertical transitions from MAS to
MAS (1 climb, 1 descent).
� Eight Free Flight transition scenarios:
J two Free Flight cruise and horizontal transition (exitto MAS);
J two Free Flight cruise and vertical transition (exit toMAS);
J two Horizontal transition (entry from MAS) andFree Flight cruise;
J two Vertical transition (entry from MAS) and FreeFlight cruise.
Each run consisted of a 20–30min flight of part of a route to/from/over Rome, Italy. The runs were balanced over the crews,i.e. the order of the runs per crew was varied to prevent order-effects in the results. This balancing was done on the varioustransitions and the reference versus Free Flight runs. Thefollowing conditions were kept constant during the experiment:
�
Nominal operations. � All aircraft equipped. � No military operations.
�
No active restricted airspace (TSAs, SUAs). � No significant weather effects (ISA conditions), i.e. no
turbulence, no thunderstorms, no strong wind.
� Pseudo-Air Traffic Control, i.e. ATC and Radio
Telecommunication (R/T) performed by one person,and aircraft actions and R/T for all aircraft by oneother person.
Participating pilots received a briefing guide prior to theexperiment. On the first day of the experiment, the pilotswere briefed for about 1 h, and were trained in the RFScockpit by an instructor. Gradually, the crew was trainedand able to handle the RFS and the ASAS system.The original starting point of the second MFF experi-
ment was directly related to hazards identified in theintermediate Operational Hazard Analysis for the FreeFlight application (Klein Obbink, 2003) within MFF.Twelve hazards that were considered interesting and suitedfor simulation were selected, with the aim to observe theparticipants’ reaction and to obtain their feedback. Thesehazards were:
�
other ADS-B transmit failure, � own ASAS failure, � own ADS-B transmit failure, � rude pilot, � own aircraft RNP failure, � other ADS-B receive failure, � incorrect manoeuvre, � other aircraft ASAS failure, � ownship navigation shift, � own aircraft 2 engines flame out/right side cockpit
displays black,
� other aircraft explosive decompression, � own ADS-B receive failure.
The second set of MFF experiments also aimed todetermine the training needs for the Free Flight concept.Another interesting question, still not answered today, isthe necessity of a VND in Free Flight operations. In orderto study this, it was decided to make the availability of aVND a variable in the experimental design. Based uponcomments in previous Free Flight experiments, the FreeFlight concept should be tested in a challenging, realisticenvironment, including military operations (area activa-tion/de-activation), significant weather (thunderstorm,turbulence and wind) and failures. Further, the ACAS/ASAS integration should be taken into account in thisexperiment, since ACAS is the foreseen safety-net ofASAS, in case of failures.The validation area for the experiment was identical to
the area used in the first set of MFF experiments, so thetransitions to MAS were available in the scenario, althoughthe flight crossed the area differently. The experiment flightwas in each run a flight from Johannesburg to Amsterdam,starting over the Mediterranean in FFAS. FFAS was nowassumed to be located from FL290 and above, west of Italy
and south of France. East of Italy and North of Francewas MAS. Vertically a 1000 ft altitude layer was definedbetween MAS and FFAS. This meant that FL280 andbelow was part of MAS, FL290 and above was part ofFFAS (so no transition layer/level as in the first MFFexperiment). MAS entry point for the RFS was locatedover Genoa. In principle, the transition to MAS was notthe subject of study in this experiment. A single alternateFLOS was applicable in both MAS and FFAS. Aircraftobeying the FLOS would receive priority over aircraft notadhering to the FLOS.
The following conditions defined the experiment matrix:
�
Two training methods (note: between-subjects):J 3.5 h Computer Based Training (CBT) +30min Full
Flight Simulator;J 4 h Full Flight Simulator.
� Two Human-Machine Interfaces:
J without VND;J with VND.
There were scenarios with weather, with military activitiesand with both. Moreover, there were failure cases as listed,one for each run. Twelve runs were conducted per crew.Each run consisted of a 30min part of an en-route flight.
The following conditions were kept constant during theexperiment:
�
all aircraft equipped, � pseudo-Air Traffic Control.
Given that the training needs were subject of this secondMFF experiment, more attention was given to the way thepilots were trained for the experiment. Two crews werefully trained in the full flight simulator, with instructor.
Fig. 18. Computer Based
Their time on the full flight simulator was around 4 h. Twoother crews were trained using a CBT system (Windows,PC-based). These crews were provided with a self-learningguide, together with the CBT, based on a PC. These crewsdid also have an instructor available during the CBT, butthe instructor acted much more reactive (only answerquestions of subjects). At the end of the CBT, these crewsreceived a short familiarisation on the Full FlightSimulator, with instructor. Their time on the FullFlight Simulator was around 30min. CBT time was around3.5 h.To carefully manage pilot workload during training,
training modules were designed according to a number offactors that influence the complexity of the trainingscenarios. The identified complexity factors are trafficdensity, number of conflicts with other aircraft, conflictdifficulty level, severity of failures and presence of weatherand military areas. These complexity factors have beenused to create four modules. The first module emphasisesthe application of the priority rules in FFAS. The secondmodule adds transitions from FFAS to MAS. In the thirdmodule failures on CD&R systems are introduced,while module four addresses complex failures, like ‘‘UnableRNP’’ and a double engine flame out. Complexity overthe training scenarios is gradually increased over themodules.The CBT deployed in this experiment should be regarded
as a desktop simulator. All displays (PFD, ND, EICAS), aCDU and Autopilot Mode Control Panel (MCP), includ-ing the Display Control Panel (DCP) are displayed on asingle 20-in high-resolution screen, see also Fig. 18.A CDU and the MCP can be controlled using a mouse.
The pilot operating the mouse was appointed pilot-flying,while the other pilot was pilot non-flying. Communicationwith ATC and other traffic was included. Before the
Training (CBT) tool.
ARTICLE IN PRESS
Fig. 20. Results on which entity is responsible for separation (y-axis
shows total number of responses of the eight participating subjects).
training and the experiment the pilots were requested tostudy a briefing guide. In addition, a 60-min oral briefingwas provided just before the training to prepare the crewsand give them the opportunity to ask questions. Before,during and after a run, module, the training and theexperiment, the crews received various questionnaires.
To give the pilots guidance in the experiment, crewprocedures were defined in detail for normal operations inFFAS and for transitioning to and from MAS. For allpotential emergency conditions, emergency procedureswere defined. The procedures indicated to set or not setthe priority switch, to leave or not leave FFAS and toinform or not inform other traffic and Air TrafficInformation (ATI) about the failure, see Ruigrok et al.(2004).
2.5.3. Results
The results from the first MFF experiment show thatworkload in general was rated to be ‘‘low’’ (below 40) andmainly determined by communication with ATC (R/T) inthe reference flights. In the ASAS flights, workload wasmore associated with the presence of traffic information,with indications that the number of aircraft, the number ofconflicts and the conflict geometry caused the pilotworkload. Highest workload is found in the verticaltransitions between FFAS and MAS, see Fig. 19, showingmean and 95% confidence intervals.
It was found in the reference runs that workload isnormally already higher in descent than in the cruise andclimb phases of flight. While flying in FFAS, traffic belowcomplicated this.
Interestingly, there was no rating below ‘‘acceptable’’ inany of the experiment runs. Lowest ratings for accept-ability (but still better than ‘‘acceptable’’) are found in thevertical transition from FFAS to MAS (descent), confirm-ing the workload results. Responsibility figures show someinteresting results, which might explain the lower accept-ability and higher workload ratings for the verticaltransitions between FFAS and MAS. In MAS it is clearto the pilots that ATC is primarily responsible for
Fig. 19. Transition effect on workload.
separation. In FFAS, the pilots understand that ATC doesnot have a role in traffic separation. In the transition zone/layer, responsibility is quite unclear, with (almost) the samenumber of responses for ATC, aircrew and the ASASsystem, see Fig. 20.The order of entities to obey in case of conflicting
instructions confirms the responsibility issue. Althoughpilots responded with very different orders, the followingaverage orders were found:
The transition zone was conceptually considered to beFFAS, while the answers indicate that the crews consideredit more to be MAS, with equal sequence as in MAS. Thisshows that the concept of the transition layer or zone wasnot completely clear, as was also expressed during thedebriefings.On the ASAS system, it can be concluded that the HMI
was between favourable and perfect. The less thanfavourable ratings (in between favourable and acceptable)indicate that the options to customise the CDTI can befurther improved and the distinction between yellow andamber colours could be improved.The CD&R system was rated favourable, except for the
false/nuisance alerts and the use of priority rules. Pilotsstrongly suggested to broadcast the intended level-off flightlevel when descending or climbing and to include thisinformation in the CD&R algorithm such that nuisancealerts are suppressed. When priority rules were considered,half of the pilots preferred the priority rules, half preferredthe co-operative approach.The safety observations during the second set of MFF
experiments reveal that pilots tend to be keen on solvingconflicts in nominal situations; there seems to be sufficienttime and effort available for conflict resolution manage-ment, and pilots tend to have a corrupted and partial trafficsituation awareness (as tested during the experiment).
Further, the most important and interesting observationsrelate to explicit differences in pilot behaviour, limitedawareness of Free Flight logic, operational view ondetection and recovery means, severity indications byoperational experts and some new elementary causes forhazardous situations. The safety observations are used forupdating the OHA and for constructing the ASOR(Allocation of Safety Objectives and Requirements). Inaddition, some safety feedback is given to the designers ofthe operation. This all will accumulate into safety require-ments.
Pilot role appears to have an effect on workload andacceptability results, see Ruigrok et al. (2004). Workloadseems to be slightly higher for Pilot Flying (PF) andacceptability slightly less for PF compared to Pilot-NotFlying (PNF). Given that the role effect has not beenexperienced in the first MFF experiment and before in FreeFlight studies (Hoekstra, 2001; Ruigrok et al., 2003), it islikely that the role effect is caused by the failures, of whichthere was one in each run. The PF had to not only fly theaircraft in FFAS, but also had to take the decision on theaction after the failure.
The results for the training procedure (CBT or SIM)effect on workload and acceptability shows better work-load ratings of subjects trained on the Full FlightSimulator, both from post-run questionnaires and fromISA measurements, see Fig. 21, showing mean and 95%confidence intervals.
Given the small group of participants per trainingprocedure (four each for CBT and SIM) and the possible‘‘between subjects’’ effect, firm conclusions cannot bedrawn from this result, although it indicates that theCBT as given is not sufficient. The training specific resultsconfirm this. On the ‘‘ideal’’ training, participants suggeststarting with an extensive briefing guide like the one used inthe experiment, including more in-depth system knowl-edge. Next, a demonstration should be provided, repre-senting different scenarios and using for instance a videoincorporating all flight instruments and inter-crew and ATIand ATC communication. Afterwards, a static CBT
Fig. 21. Training procedure effect on workload and acceptability.
programme should be deployed to familiarise crew withthe Free Flight symbology and practise priority rules in anon-dynamic environment before applying Free Flight inpractice. Then a number of scenarios on a desk-topsimulator (the CBT in this experiment) should be provided,followed by a single simulator session (3.5–4 h), which maybe combined with recurrent training.The VND appears to have no effect on workload and
acceptability. The debriefing results revealed that generally,the VND is preferred in FFAS, but not needed in MAS.For ASAS, it is not required in MAS, nor in FFAS.However, a VND could be useful for terrain awareness.The effect of military activities and weather on workload
and acceptability seems limited, while the effect of failuresis significant, especially on workload. The ‘‘Own Naviga-tion Shift’’ and the ‘‘Own 2 engine/2 display failure’’appear to generate most workload, see Fig. 22, althoughstill rated acceptable on average.This is confirmed by the answers to the question which
asked participants to explain the reason for their workloadrating. Failures were the main reported reason for work-load, followed by the number of conflicts.The HMI was generally rated acceptable or better than
acceptable. The less than favourable but still acceptableratings concerned the VND presentation and the options tocustomise the CDTI. Further, the false alerts of the CD&Rwere rated less than favourable: pilots suggested theinclusion of the target flight level in the CD&R algorithm,as also suggested in the first MFF experiment.The ASAS/ACAS integration as implemented was
appreciated. ACAS should always remain next to ASAS,as a safety-net (100% response YES). In the current set-up,
Fig. 22. Failure effect on workload (showing mean and 95% confidence
PASAS information and CD&R information was removedin case of an ACAS Traffic Alert (TA) or RA. It is stronglysuggested to only remove ASAS information in case of anACAS RA. ASAS CD&R appears to have an overlappingalert area with ACAS. This should be changed in order toprevent ACAS warnings while clear of ASAS warnings.
The failure procedures as developed for the experimentreported in several procedures to ‘‘Leave FFAS’’. Mostsubject pilots argued about this instruction, since it was notclear to them if they had to leave FFAS ‘‘as soon aspossible’’ or ‘‘as soon as practical/suitable’’. Participantpilots therefore invented their own arguments to eitherleave FFAS or stay in FFAS. This was surprising and verydifferent per crew.
3. Discussion: issues solved and issues remaining
During the years, a lot of scepticism has been expressedon Free Flight. Opponents of Free Flight say that FreeFlight is unsafe; pilots do not have the skills nor the mentalresources to perform the extra cockpit task; it is technicallyimpossible; not efficient; and politically unacceptable. Theresults found in the experiments and studies referenced inthis paper have, however, not been able to refute the FreeFlight concept.
It is clear from research referenced in this paper thatFree Flight has a bright future. The comparison betweenthe results for controller workload and those for pilotworkload, as tested in the INTENT project, is interesting:whereas controllers become overloaded at about 1.5 timesthe summer 2000 traffic density, pilots are still notoverloaded at 3 times this density, as confirmed by theNLR/NASA Free Flight results. This comparison suggeststhat, in the long term, ATM systems based on conceptswhere aircrews have the primary responsibility for separa-tion are likely to offer several times the capacity of thosebased on ground control concepts.
Pilot workload and acceptability appear to be no issue,not even in conditions of several times current trafficdensity and in realistic environments with significantweather, military activities and failures. Part of this resulthas to do with the nature of the concept: distributed ratherthan centralised, as explained in Hoekstra et al. (2000).Part of the result is accounted for by the simple, ergonomicdesign of the Free Flight concept and symbology used. TheFree Flight HMI was in all experiments designed as muchas possible in line with the guidelines for human centreddesign as stated in the ICAO circular 249-AN/149:
1.
The human must be in command. 2. To command effectively, the human must be involved. 3. To be involved the human must be informed. 4. Functions must be automated only if there is a good
reason for doing so.
5. The human must be able to monitor the automated
system.
6. Automated systems must, therefore, be predictable.
7.
Automated systems must be able to monitor the humanoperator.
8.
Each element of the system must have knowledge of theother’s intent.
9.
Automation must be designed to be simple to learn andoperate.
These principles formed the guidelines for the conflictdetection, resolution and display in the design phase priorto the experiments. The part transferring most of the ASASinformation to the human is the ND. The considerationsthat led to the current display design, include the following:
�
No extra display with dedicated traffic and conflictfunction. � Absolute co-ordinates (latitude, longitude) frame for
conflicts to avoid a separate mode on ND.
� Colours should indicate urgency based on time to loss of
separation.
� Traffic symbols should present as much information as
possible without clutter (leads to directional aircraftsymbols instead of track vectors).
� Symbology should be ‘‘natural’’ and easy to understand.
The human centred design guidelines and the resultingchoices on the HMI also resulted in favourable to perfectratings for the HMI in most experiments. The HMI forFree Flight is therefore to be considered an issue resolved.State-based and intent-based CD&R has been studied
extensively. Intent-based CD&R provides ‘‘benefits’’ interms of reduced pilot workload, but also ‘‘costs’’ in termsof complexity, need for priority rules, potential compat-ibility problems between different brands of FMS and largebandwidth, see also Hoekstra et al. (1999). Moreover, theintent-based system is not very well capable of solvingmulti-aircraft conflicts. Due to the use of priority rules,multiple-aircraft conflicts are solved sequentially. Thestate-based CD&R system also provides ‘‘benefits’’ and‘‘costs’’. Benefits compared to the full intent-based systemare simplicity, low bandwidth requirements, easy to retrofit(no requirements to change avionics infrastructure) and theability to solve multi-aircraft conflicts in parallel. The‘‘costs’’ involve a somewhat higher pilot workload insimilar circumstances, the smaller look-ahead time whichresults in less efficient resolution manoeuvres and thesometimes false/nuisance alerts due to missing intentinformation. The most optimal CD&R system (in termsof costs versus benefits) is suggested to be state-basedCD&R as tested in the NLR/NASA Free Flight and MFFexperiments with the intended/target flight level added.This combination of state-based CD&R with a limitedamount of intent provides ‘‘the best of both worlds’’: it issimple, it is effective, it is easy to understand by the users, itoffers fail-safe options which enhance safety, requires lowbandwidth, is easy to retrofit (no major changes to theavionics infrastructure), it has the ability to solve multi-aircraft conflicts in parallel and it greatly reduces the false/
nuisance alerts. Studying this CD&R system is still an openissue.
A conflict prevention function (such as PASAS) added toCD&R is found to be required. It is further found that thePASAS system to a certain extent even takes away the needfor the use of intent information. By enhancing the ASASsystem with the PASAS system, the following rule-of-the-sky has been applied: ‘‘It is forbidden to manoeuvre (i.e.change the direction or magnitude of the speed vector) insuch a way that this causes a conflict within the look-aheadtime with another aircraft.’’ This rule is a way to relieve theneed for exchanging intent information. It is no longernecessary to know whether an aircraft will turn, because itwill not if that causes a conflict. Vertically it is slightlydifferent. In the experiments conducted (not taking intoaccount the intended level-off altitude), an aircraft levellingoff just below the ownship would also have to adjust thevertical speed or track angle because it is not allowed toaim its speed vector at the ownship. It was found, however,that this makes it difficult to reach (busy) cruise altitudewithout false/nuisance alerts. This is the main reason tostudy the inclusion of the intended level-off altitude also inPASAS. Note, however, that even if an aircraft is levellingoff below the ownship, it might still be relevant for the crewto know about the undesirable situation of their speedvector aiming at a short-term conflict. This is an open issueto be resolved.
The pilots in the MFF experiments were not inagreement on the use of priority rules versus co-operativeCD&R in the state-based CD&R system. Some likedpriority rules, some liked the co-operative approach. Froman analytical point of view, this issue is also not clear yet.The co-operative approach allows multi-aircraft conflictsto be solved in parallel, but the first aircraft to react is oftensolving the complete conflict. This results sometimes inwaiting for ‘‘the other’’ aircraft in the conflict tomanoeuvre, an effect which is considered undesired sinceit could potentially lead to a serious safety issue: bothaircrew waiting for the other aircrew to manoeuvre. Whenusing priority rules, multi-aircraft conflicts need to besolved sequentially, which could take too long for themulti-aircraft conflict to be solved on time. Further,priority rules introduce new problems such as the missingfail safe element if only one aircraft manoeuvres. The effectof ‘‘waiting for the other aircraft to manoeuvre’’ as withco-operative concept is, however, taken away since thepriority rules clearly indicate which of the aircraft haspriority and which aircraft needs to manoeuvre. Within theMFF project a possible solution was found combining thebest of both worlds. Two phases were defined,first a priority phase, then a co-operative phase.Given pilot feedback, this issue is still to be consideredunsolved. Further study is required on the combi-nation (or not) of priority rules and a co-operativephase, mainly in combination with an analytical/safetystudy on multi-aircraft conflict solutions using the state-based CD&R system.
The transitions between FFAS and MAS have beentested extensively within the MFF experiments. In the firstMFF experiment a transition layer was defined, with adedicated transition flight level (FL280), which was onlyavailable for transitioning aircraft between FFAS andMAS. The use of this transition layer between MAS andFFAS resulted in quite unclear situations in thisexperiment. In the second MFF experiment, in whichthe transition layer was removed, this was clearlyimproved. This suggests that the use of a dedicatedtransition layer and level in between MAS and FFAS isnot recommended.During the MFF experiments a FLOS was used in both
MAS (as applicable today) and also in FFAS. This easedthe horizontal transitions to MAS, was in line with today’soperations and reduced the amount of head-on type ofconflicts significantly. Especially the reduction of the fastevolving (head-on type of) conflict geometries seems aninteresting input for the Free Flight safety case. This issueneeds further attention in the Free Flight safety studies.During the various Free Flight experiments, scenarios
with significant weather, military areas and failures weresimulated. The debriefing and results from the experimentruns (workload, acceptability, safety) show no majorissues. For this reason, Free Flight operations in realisticscenarios are considered to be feasible.The MFF experiment has tested the training needs for a
concept such as Free Flight. The ‘‘ideal’’ training, asexpressed by participants, starts with an extensive briefingguide like the one used in the experiment, including morein-depth system knowledge. Next, a demonstration shouldbe provided, representing different scenarios and using forinstance a video incorporating all flight instruments andinter-crew and ATI/ATC communication. Afterwards, astatic CBT programme should be deployed to familiarisecrew with the Free Flight symbology and practise priorityrules in a non-dynamic environment before applying FreeFlight in practice. Then a number of scenarios on a desk-top simulator (the CBT in this experiment) should beprovided, followed by a single simulator session (3.5–4 h),which may be combined with recurrent training. Thetraining needs for a concept such as Free Flight is close tobe solved, but it needs at least one more validation activityto confirm the suggested training schedule.An interesting question, still not answered today, is the
necessity of a VND in Free Flight operations. In order tostudy this, it was decided to make its availability a variablein the MFF experiments. The VND appears to have noeffect on workload and acceptability. The debriefing resultsrevealed that, generally, the VND is preferred in FFAS,but not needed in MAS. ASAS is not required in MAS, norin FFAS. So for Free Flight operations, the VND is notrequired. However, a VND could in general be useful for,for example, terrain awareness.An important open issue is the integration of ASAS and
ACAS in the aircraft. The ACAS/ASAS integration astested in the MFF experiment provides important feedback
on this integration. ACAS should always remain next toASAS, as safety-net. ASAS (CD&R and PASAS) informa-tion should only be suppressed in case of an ACAS RA.However, ASAS CD&R appears to have an overlappingalert area with ACAS: if the conflict is solved according toASAS, sometimes ACAS warnings still appear. Soalthough the ASAS/ACAS integration issue is close to besolved, some more study is required to tune the ASASalgorithms to be more compatible with ACAS algorithms.
In summary, the results of the Free Flight experimentsare best illustrated by the words of one of the participatingpilots in the first experiment. Before the experiment thispilot, whose wife was an Air Traffic Controller, told theexperiment team: ‘‘I very much doubt this concept.’’ Afterthe experiment he was convinced but worried: ‘‘How do Iexplain it to my wife?’’ Scepticism had been considerablyreduced.
After solving the open issues, the next step is the actualintroduction of Free Flight. For this, action is requiredtoday. Standardisation, development and certification ofFree Flight equipment will take time. To accelerate thisprocess, NLR has developed and supports the plan to startimplementing Free Flight over the North Atlantic Ocean.This will meet the short-term return-on-investment require-ments of the airlines, while addressing existing short-comings of the North Atlantic System, see Anonymous(1999b, c).
References
Anonymous, 1995. Final Report of RTCA TF3: Free Flight Implementa-
tion, RTCA TF3, 1995. RTCA Inc., Washington (Chapter 3).
Anonymous, 1998. Minimum Aviation System Performance Standards
(MASPS) for Automatic Dependent Surveillance Broadcast (ADS-B).
RTCA/DO-242, 19 February 1998, prepared by RTCA SC-186.
