EUROCONTROL Experimental Centre

EUROCONTROL

Advanced Emission Model (AEM3) V1.5

Validation Exercise #2

EEC/SEE/2004/012

Sandrine CarlierJames Smith

ii

Advanced Emission Model (AEM3) v1.5

Validation Exercise #2 EEC/SEE/2004/012 Prepared for EUROCONTROL Experimental Center under Contract no C/1.060/CE/SS/02-TRSB156/03 by ISA Software Ltd – 38, rue des Gravilliers – 75003 PARIS ISA Software Project team Contributing Authors Sandrine Carlier

James Smith Project Management Ian Crook Contract Management Ian Crook EUROCONTROL Experimental Centre Project Manager Frank Jelinek – EUROCONTROL Experimental Centre SEE

frank.jelinek@eurocontrol.int For further information in regard to this project please contact EEC Project Manager Documents Revisions Revision 0.1 SC Oct 29 2004 Initial draft Revision 0.2 SC Jan 19 2005 Mods requested by EEC Project Mgr

© European Organisation for the Safety of Air Navigation EUROCONTROL July 2002

This document is published by EUROCONTROL in the interest of the exchange of information. It may be copied in whole or in part providing that the copyright notice and disclaimer are included.

The information contained in this document may not be modified without prior written permission from EUROCONTROL.

EUROCONTROL makes no warranty, either implied or express, for the information contained in this document, neither does it assume any legal liability or responsibility for the accuracy, completeness or usefulness of this

information.

iii

REPORT DOCUMENTATION PAGE

Reference: EEC/SEE/2004/012

Security Classification: Unclassified

Originator: Society, Environment, Economy

Originator (Corporate Author) Name/Location: EUROCONTROL Experimental Centre Centre de Bois des Bordes B.P.15 91222 BRETIGNY SUR ORGE CEDEX France Telephone: +33 1 69 88 75 00

Sponsor: EUROCONTROL project EEC/SEE/GAES

Sponsor (Contract Authority) Name/Location: EUROCONTROL Agency Rue de la Fusée, 96 B –1130 BRUXELLES Telephone: +32 2 729 90 11

TITLE: Advanced Emission Model (AEM3) v1.5 – Validation Exercise #2 Authors : Sandrine Carlier, James Smith

Date 10/04

Pages 40

Figures 10

Tables 8

Appendix 0

References 7

EATMP Task Specification -

Project AEM3v1.5 Validation

Task No. Sponsor -

Period 2004

Distribution Statement: (a) Controlled by: EUROCONTROL Project Manager (b) Special Limitations: None (c) Copy to NTIS: YES / NO Descriptors (keywords): Global Emissions – AEM – TEA – NOx – CO – HC – CO2 – H2O – SOx – Benefits – EEC - etc –– keyword2 – keyword3 – etc. Abstract: This report is an appendix to the AEM3 validation report EEC/SEE/2004/004 dealing with statistics and validation measures concerning three supplementary aircraft types.

v

TTaabbllee ooff CCoonntteennttss

REPORT DOCUMENTATION PAGE ........................................................................ 3

TABLE OF CONTENTS ............................................................................................ 5

LIST OF FIGURES .................................................................................................... 7

LIST OF TABLES ...................................................................................................... 7

LIST OF ABBREVIATIONS....................................................................................... 9

EXECUTIVE SUMMARY ......................................................................................... 11

INTRODUCTION...................................................................................................... 13

DATA COLLECTION AND PREPARATION ........................................................... 15 Data set..............................................................................................................................15 Obtaining AEM3 input data...............................................................................................16

Obtaining fuel statistics....................................................................................................16 Determination of the chronology inside each file..............................................................16 Separation of flights .........................................................................................................16 Data cleaning...................................................................................................................17 Determination of airports .................................................................................................17 Calculation of missing values for AEM3...........................................................................18 Resulting flight profiles.....................................................................................................19

Aircraft Types....................................................................................................................19 Available input files for AEM3 ..........................................................................................20 AEM3 execution ................................................................................................................20

OUTPUT DATA ANALYSIS AND RESULTS – FUEL BURN ESTIMATION WITH AEM3 ....................................................................................................................... 23 Fuel burn estimation with AEM3 ......................................................................................23

Actual fuel data quality.....................................................................................................23 Consequences of "no flight completion" option ................................................................24 Scope of the AEM3 validation exercise#2........................................................................24

vi

FDR#2 results versus FDR and SITA results...................................................................24 Fuel burn by aircraft type .................................................................................................25 Fuel burn by distance range ............................................................................................25

Fuel flow analysis .............................................................................................................26 Fuel flow evolution ...........................................................................................................26 Fuel flow limits .................................................................................................................30

Emissions estimation with AEM3.....................................................................................32 NOx, CO and HC distribution ...........................................................................................32 NOx average emission indices from ANCAT and NASA...................................................34

CONCLUSION......................................................................................................... 37

REFERENCES......................................................................................................... 39

vii

LLiisstt ooff FFiigguurreess

FIGURE 1: THE AEM3 FUEL BURN VS. OPERATIONAL FUEL BURN FOR 792 FLIGHTS 11 FIGURE 2: EVOLUTION OF FUEL FLOW – A310 27 FIGURE 3: EVOLUTION OF FUEL FLOW – B733 28 FIGURE 4: EVOLUTION OF FUEL FLOW – B737 28 FIGURE 5: FUEL FLOW LIMITS – A310 30 FIGURE 6: FUEL FLOW LIMITS – B733 31 FIGURE 7: FUEL FLOW LIMITS – B737 31 FIGURE 8: EMISSIONS COMPARISON OF 757-200 FOR A 750 KM AND 5500 KM MISSION [REF 5] 33 FIGURE 9: EMISSION DISTRIBUTION FOR THE WHOLE FDR#2 TRAFFIC SAMPLE 34 FIGURE 10: THE AEM3 FUEL BURN VS. OPERATIONAL FUEL BURN FOR 4642 FLIGHTS 37

LLiisstt ooff TTaabblleess

TABLE 1: AIRCRAFT TYPES AVAILABLE FOR AEM3 VALIDATION EXERCISE#2 .......................................................15 TABLE 2: FDR#2 FLEET..........................................................................................................................................20 TABLE 3: GENERAL FUEL RATIOS FOR AEM3 VALIDATION EXERCISE ....................................................................24 TABLE 4: FUEL RATIO PER AIRCRAFT TYPE .............................................................................................................25 TABLE 5: FUEL RATIO PER DISTANCE......................................................................................................................26 TABLE 6: EMISSION DISTRIBUTION FOR THE WHOLE FDR#2 TRAFFIC SAMPLE .......................................................33 TABLE 7: PUBLISHED AVERAGE EINOX (G/KG FUEL) OF REFERENCE PROJECTS [REF 7]..........................................35 TABLE 8: AEM3 ESTIMATED EINOX AVERAGES IN G/KG FUEL...............................................................................35

ix

LLiisstt ooff aabbbbrreevviiaattiioonnss

AEM Advanced Emission Model

AEM3 Advanced Emission Model, 3rd version

ANCAT Abatement of Nuisances Caused by Air Transport

BADA Base of Aircraft Data

BEN Benzene

BM2 The Boeing Method 2

EEC-BM2 The EUROCONTROL modified Boeing Method 2

CDRate Climb/Descent rate

CO Carbon Monoxide

CO2 Carbon Dioxide

EEC EUROCONTROL Experimental Center

EEC-BM2 EEC corrected BM2

EI Emission Index

FDR Flight Data Recordings

FL Flight Level

H2O Water

HC Hydrocarbon

ICAO International Civil Aviation Organisation

Lat Latitude

Long Longitude

LTO Landing- and Take-Off cycle

Max Maximum

Min Minimum

MS Microsoft

MTOW Maximum Take-Off Weight

NASA National Aeronautics and Space Administration

NOx Oxides of Nitrogen

SEE Society, Environment, Economy

SOx Oxides of Sulphur

TEA Toolset for Emission Analysis

TOG Total Organic Gases

TOW Take-Off Weight

VOC Volatile Organic Compounds

11

EExxeeccuuttiivvee SSuummmmaarryy

AEM3 validation exercise#2 builds on the AEM3 validation exercise#1 completed in early 2004. It addresses three new aircraft types: A310-325, B737-300 and B737-700. The data set used for this supplementary validation exercise consists of flight data recordings collected from a European airline. The data granularity was of high level, allowing a typical step of 4 seconds between two flight points after data preparation.

No completion of flight profile was required during AEM3 execution. Like the first AEM3 validation exercise, the current exercise indicates a very good ability to estimate fuel burn. The resulting average fuel ratio for the whole traffic sample indicates that AEM3 underestimates fuel burn by only 6%. This value would read closer to zero if actual MTOW were corresponding in a better extent to standard MTOW used to model aircrafts. The same is true of the influence of distance range on AEM3 results. Results obtained during AEM3 validation exercise#1 with FDR data are thus perfectly confirmed and the stability of AEM3 regarding granularity of data is proved.

This validation exercise also corroborated the influence of MTOW on the accuracy of fuel burn estimation through AEM3. Results seem to indicate AEM3 to slightly underestimate fuel burn for flights with a high take-off weight.

The results for the 792 flights of the data set are visualized on Figure 1.

Figure 1: The AEM3 fuel burn vs. operational fuel burn for 792 flights

The influence of flight attitude on fuel flow evolution for data set of high granularity was highlighted. During AEM3 validation exercise#1, significant differences were highlighted for low and high thrust fuel flows compared to fuel flows documented in the ICAO Engine

12

Exhaust Emissions Data Bank and BADA data. Similar differences were observed for the three supplementary aircraft types under study, leading to the same errors.

The influence of many parameters on emissions for the specific data set exploited for this second validation exercise led to an increase of the proportion of CO and HC regarding NOx. NOx emission indices lie 1.23% lower in the current exercise than in the first one.

As a conclusion of the two AEM3 validation exercises, AEM3 fuel burn estimation offers a high level of realism. Emissions estimation compares with the results published by NASA and ANCAT; the quality of the results depends on the granularity of the AEM3 input data.

13

IInnttrroodduuccttiioonn

The aim of the AEM3 validation exercise #2 is to build on the AEM3 validation work already carried out previously ([Ref 2]).

The result of the supplementary analysis therefore, is to produce a new set of validation tables presented in this appendix to the existing report ([Ref 2]) which deals only with statistics and validation measures concerning the specific aircraft types contained in new FDR data (A310-325, B737-300, B737-700).

Since the desire for a supplementary validation exercise is to focus on new FDR data and corresponding additional aircraft types it is not necessary to repeat many of the fundamental (non-aircraft specific) validation tests, focusing only on aircraft specific validation figures.

Unlike input data to the first AEM3 validation exercise, the available FDR data for this second validation exercise did not allow to distinguish and re-evaluate the different phases of the LTO cycle.

Results for CO2, H2O and SOx are not repeated either as they follow exactly the evolution of fuel burn. Similarly VOC/TOG are proportional to HC and thus do not need to be validated in the current validation exercise.

To avoid a confusion between FDR data used in the first validation exercise and the current validation exercise, FDR data concerning the current validation exercise are called "FDR#2" in the continuation of this report.

15

DDaattaa ccoolllleeccttiioonn aanndd pprreeppaarraattiioonn

The data preparation phase for this additional validation exercise represented a significant portion of the work. The data preparation phase involves analysis of the FDR#2 data, content, format and correctness followed by manipulation to produce suitable and sufficient data for input into AEM3.

The objective is twofold – one is to create suitable input data to use in AEM3 and the second is to produce fuel usage statistics from the actual FDR#2 data in order to permit comparison of AEM3 fuel use results with the actual recordings. In most cases we must assume that the FDR#2 data is complete in terms of fuel data, however, care must be taken that the final statistics are both complete, and correspond to the same flight operations as those being replicated in AEM3.

Data set

The current AEM3 validation exercise#2 concentrates on operational flight data recordings from a European airline.

Data was provided in the form of 5 CDs of 27 to 35 data files, each of which contained records for only one aircraft type. During the data preparation, the data files were grouped to create one single file for each CD and thus avoid the same manipulation to be performed hundreds of times on different files. Let us call "disk0" to "disk4" the 5 CDs in question.

Disk Aircraft type Disk0 B737-700 Disk1 B737-300 Disk2 B737-300 Disk3 A310-325 Disk4 ATR 42-500

Table 1: Aircraft types available for AEM3 validation exercise#2

Information available for the 5 CDs was different. Units were different and the files did not specifically hold the same type of data. The first step was thus to identify data available for each of the CDs.

It was concluded from this investigation that only the 4 first CDs were exploitable in the scope of this validation exercise, which deprives the validation exercise of the ATR 42-500. Actually neither fuel information nor even aircraft weight were available in the files on disk4.

In the same way disk1 and disk2 hold special files out of which no fuel statistics would have been extractable. The corresponding files were ignored.

Finally each CD holds between 26 and 34 exploitable files which consist in an almost continuous record lasting for one to three days with one flight point every one second. To reduce the size of the files, it was decided to keep only the lines in which GMT time was actually indicated. The impact of using four seconds steps instead of one second step is

16

negligible regarding the duration of a flight. As a result, the files used as AEM3 input hold around one point every four seconds. This step is slightly longer when erroneous points were detected and cancelled from the data set.

Obtaining AEM3 input data

Given the content and format of the data, and in recognition that much of the modification/cleaning process did inevitably involve manual intervention even if an automation was set up when possible, this part was the most time consuming part of the supplementary exercise.

Before being used in AEM3, data has to be cleaned and formatted to AEM3 format. The process is described below.

Obtaining fuel statistics

It was decided to keep fuel statistics data together with all other data available all along the data preparation process to assure that fuel statistics correspond exactly to the flight profiles used with AEM3. The basic values are thus in an easy to use format that only requires a simple formula to calculate actual fuel used for each engine (i.e. fuel flow in Pounds per Hour (PPH) or kg per second is recorded for each engine, and can be used in conjunction with the flight leg duration to calculate how much fuel was used per engine on each leg).

Determination of the chronology inside each file

Each flight point time was indicated in a 0-24 format but the data stretched sometimes over many days in the same file.

The day was not always indicated even when data recordings lasted for more than 24 hours.

The flight points were not always in a chronological order, which prevented a simple "order by time" command completing the chronology of a flight.

These items made it difficult to automate a process to order files. A manual check was necessary to correct exceptions and ensure that the flights were correctly reconstituted.

Then, flight point times were translated to allow AEM3 to deal with many days in the same file. Day1 was expressed as 00:00:00 to 23:59:59, day2 as 24:00:00 to 47:59:59, and so on.

Separation of flights

Once the chronology in the files was obtained, the flights were separated and a call sign was attributed to each single flight.

Flights point times, flight level, latitudes and longitudes were used to separate the flights. Some of the flights separated in this way were identified as long taxis in the same airport. The corresponding points were deleted from the data set.

17

Data cleaning

During preparation phase most of the data errors have to be removed to create a clean set of data and thus avoid AEM3 execution errors and imprecision in the results analysis.

Flight points presenting the following errors were deleted from the data set:

No event time indicated Event time values with an obvious error (eg. 18:25:62) Latitude and longitude equal to zero Latitude/longitude absolute value higher than 90 (resp. 180) Event time, latitude, longitude or altitude value equal to "RANGE" Event time, latitude, longitude or altitude value very different from values of the other

points of the flight. For example, previous and following altitude around 30000 ft, current altitude around 15000 ft.

The data set held sometimes many points supposed to correspond to exactly the same flight point but indicating different values for latitude, longitude, flight level, fuel flows, etc.

When AEM3 reads such "double points", it is not able to identify which point should be used. Therefore to ensure that AEM3 uses correct information, "double points" have to be deleted before AEM3 execution. Moreover deleting erroneous double points helps avoiding errors in the determination of leg attitudes.

In the majority of the cases, "double points" corresponded to recording errors and thus had to be eliminated. Some of these "double points" were obviously erroneous and thus directly deleted. Some others were apparently correct. In this case a deep manual investigation was carried on to determine which point was correct regarding the evolution of the other parameters of the flight.

A manual check flight point by flight point for all the data set was then inevitable to detect and delete erroneous points with no precise criteria. This step of the data preparation was crucial to ensure a reasonable level of quality to AEM3 input data.

Determination of airports

Since the FDR#2 data only contains latitude and longitude references for each data point (and not the airport name) whilst AEM3 requires the airport names (which are cross reference in an internal AEM3 database to the lat/long) it was necessary to ensure that all the airports used by the flights in the data are correctly identified.

To reconstitute as realistic flight profiles as possible, departure and arrival airports had to be determined out of latitudes and longitudes of aircrafts at ground. Even if the LTO cycle is not part of this exercise, attributing wrong airports to a flight would have lead to unrealistic flight profiles while fuel burn information is realistic. This could have serious consequences on the quality of results and above all would limit deep investigations when analysing the results.

RAMS airport database was used for that issue together with time tables available in the internet web site of the airline providing FDR#2 data.

18

A margin of about 10 km was accepted to determine airports. When many airports were available in this margin, a manual choice was performed.

Note that not all the airports found are supposed to be served by the airline providing FDR#2 data. However the choice has been kept since the time tables used may not be complete and up to date.

Calculation of missing values for AEM3

To deal with AEM3 input format, some values have to be determined, specifically ground speed, rate of climb / descent and attitude.

Ground speed

Even if the value was available for some of the flights, this value was recalculated using latitude, longitude (i.e. Great Circle distance) and leg duration.

Rate of Climb / Descent

This value was calculated using flight level at both ends of a leg and the leg duration.

Attitude

The attitude was determined by a comparison between the flight level at the beginning and the end of a leg.

FLFrom < FLTo Climb

FLFrom = FLTo Cruise

FLFrom > FLTo Descent

As the LTO cycle is not under study, it was not necessary to determine attitudes such as taxi, take-off or landing.

The method used to determine the attitude may lead to a tiny climb or descent during the main cruise phase to be considered as climb or descent phase. The consequence is a climbing or descending fuel burn rate used for the calculation of fuel burn for the leg in question. However the duration of legs in the files under study is around 4 seconds. The consequence of such tiny climb or descent phases thus reflects the reality of the aircraft attitude and has not to be considered as cruising phases.

Flight level

AEM3 does not accept negative flight levels. As a consequence, negative flight levels appearing in the data were adjusted to zero. This adjustment was performed after the allocation of attitudes.

19

Note that the negative flight levels in the files were originally very close to zero and probably due to a barometric measurement vagueness.

Resulting flight profiles

FDR#2 data holds measured values and is consequently submitted to measurement errors. For example it can be observed in the FDR#2 data set that flight level differs from one point to the following one even when the aircraft does obviously not move. This is a proof that the flight level values in the FDR#2 data set suffer from barometric errors.

All the measurements in the data set are subject to present a consequent level of imprecision regarding the precision required for the values to be used as a basis for LTO phase identification. The determination of LTO phases and especially taxi phases would thus require many complex and time consuming filters to be applied on data to ensure a correct identification of transitions between LTO phases.

The limited impact of supplementary flight points on fuel burn and emission final results led to keep flight profiles from engine start to engine off for the current validation exercise.

Indeed, the fuel consumption before taxi-out and after taxi-in (i.e. when the aircraft is not moving) is negligible regarding the total amount of fuel burn. Similarly, the impact of such additional "no-move phases" on NOx, CO and HC distribution (see section "The estimation of level of realism for the NOx, CO and HC emissions is based on ANCAT and NASA projects, as detailed during the AEM3 validation exercise#1 [Ref 2]. As a reminder, a brief summary of ANCAT and NASA findings can be found at the beginning of the following sections.

NOx, CO and HC distribution") represents less than 0.7 percent evolution of NOx. This can be considered as negligible regarding overestimations of CO and HC due to ICAO and BADA fuel flow limits (cf. section "Fuel flow limits").

Note that no figure using this "topping of flights" do appear in this report since the approximation to obtain the percentage of evolution presented above is very global. Not all the flights were verified to be cut exactly at the correct moment.

Aircraft Types

The unique aircraft registration number was used to get information on real FDR#2 fleet using JPFleets database ([Ref 4]). Useful information for this validation exercise is reported in the table below.

20

ACType Engine MTOW Max altitudeActual A310-325 2 X PW PW4156A 164.000 kg 38.135 ft

BADA/AEM3 A310 2 X CF6-80A3 Low: 96.000 kg Nominal: 120.000 kg High: 142.000 kg

41.000 ft

Actual B737-300 2 CFMI CFM56-3C1 56.472 kg to 61.235 kg 37.056 ft

BADA/AEM3 B733 2 X CFM56-3-B1 Low: 38.280 kg Nominal: 54.000 kg High: 62.800 kg

37.000 ft

Actual B737-700 2 CFMI CFM56-7B22 68175 kg 40.032 ft

BADA/AEM3 B737 2 X CFM56-7B24 Low: 45.936 kg Nominal: 60.000 kg High: 70.800 kg

41.000 ft

Table 2: FDR#2 fleet

"BADA/AEM3" rows indicate aircrafts and engines used to model the actual aircraft-engine couples.

Actual values enter BADA/AEM3 values' envelop, with the exception of MTOW for the A310 (red font in Table 2). A310-325 MTOW indicated in JPFleets database based on unique aircraft registration number is significantly higher than the highest BADA MTOW. This difference leads to a systematic underestimation of fuel burn and emissions between BADA data and real MTOW for the A310-325 by AEM3.

Available input files for AEM3

After data preparation, 182 movements were available for the A310-325, 483 movements for the B737-300 and 127 movements for the B737-700,

The very large number of flight points describing the flight profiles (1640 on average) offsets the small number of movements (792) regarding data sets used in the AEM3 validation exercise#1.

Aircrafts will be referred to using the BADA name (i.e. A310, B733 and B737) in the continuation of this report.

AEM3 execution

AEM3 available options were discussed during validation exercise#1 ([Ref 2]) and do not need to be repeated here. This section only presents AEM3 user options used for validation exercise#2.

As the LTO cycle is not in the scope of this exercise, flight profiles were not completed. Another reason why flight profiles were not completed is that the determination of airports

21

from latitudes and longitudes may not always be precise and could induce errors if flights are completed.

Traffic sample entry time was "off-block" since the value used in the traffic.txt file was the first available point of each flight.

Other options were chosen similarly to the AEM3 validation exercise#1 (for more details, see [Ref 2] section "AEM3 User Options for Validation").

23

OOuuttppuutt ddaattaa aannaallyyssiiss aanndd rreessuullttss –– FFuueell bbuurrnn eessttiimmaattiioonn wwiitthh AAEEMM33

Similarly to AEM3 validation exercise#1, results of the comparisons between airline and AEM3 fuel burn estimation and flight duration are presented as follows:

=

=

ninformatio burn fuel Airlineestimation burn fuel AEM3Ratio Fuel

duration flight Airlineduration flight AEM3Ratio Duration

Note that due to AEM3 options chosen (no flight completion), the duration ratio is always equal to 1 in this validation exercise. Therefore this value does not appear in the results tables. For the same reason, an analysis of the flight duration does not make sense in the context of this second validation exercise.

As stated above, LTO cycles are not in the scope of the AEM3 validation exercise#2.

Fuel burn estimation with AEM3

Actual fuel data quality

There were a substantial number of errors detected in the initial actual data. Especially actual fuel flows are obviously wrong for some particular flights. Two main types of errors were detected:

fuel flow equal to 0 when the aircraft is flying, constant fuel flow all along the flight.

The actual fuel consumption for each leg is based on fuel flow for each engine of the aircrafts under study. Indeed fuel flow was the only usable information since fields entitled "total fuel Q (kg)" and "gross weight (kg)" were empty in almost all the FDR#2 files.

As a consequence the quality of all the current validation exercise is strictly linked to the quality of the fuel flows indicated in the FDR#2 files.

Nevertheless as leg duration is most of the time equal to 4 seconds, the actual fuel consumption can be considered as a sum of instantaneous fuel consumptions. Most of data errors should be smoothed because of the large number of legs, which allows actual fuel burn to be compared with AEM3 outputs.

24

Consequences of "no flight completion" option

As no flight completion is required during AEM3 execution, no standard part of the profile replaces the actual flight profile below flight level 30. The comparison between AEM3 outputs and actual data takes into account exactly the same flight profile with the same number of flight legs.

As discussed during AEM3 validation exercise#1 ([Ref 2]), no flight completion means that BADA fuel flows are used all along the flight profiles, even below flight level 30. Consecutive AEM3 errors were studied during AEM3 validation exercise#1.

Scope of the AEM3 validation exercise#2

AEM3 validation exercise#1 presented a detailed study of fuel ratio depending on take-off weight. The actual take-off weight is not available in the FDR#2 data since the corresponding field is empty. As a consequence the influence of take-off weight on the fuel burn and emissions estimation can not be evaluated with FDR#2 data.

An analysis of fuel ratio per city pair is not in the scope of this validation exercise. Such an analysis had however been approximative since FDR#2 data do not indicate any departure or arrival airport.

AEM3 validation exercise#2 focuses on an analysis of fuel ratio per aircraft type, with a breakdown per distance range.

FDR#2 results versus FDR and SITA results

Table 3 below repeats average fuel ratios obtained for the whole AEM3 validation exercise.

AEM3 validation exercise Data set NoAdd

Fuel ratio AddAll

Fuel Ratio

AddAll corrected (*) Fuel Ratio

#2 FDR#2 0.94 - - FDR 0.97 1.01 0.99 #1 SITA - 1.24 1.08

Table 3: General fuel ratios for AEM3 validation exercise

The fuel ratio for the overall data sample is 0.94. This value is 3% lower than the fuel ratio obtained for FDR data with the same AEM3 options during the AEM3 validation exercise#1. If compared to "AddAll corrected" values, FDR#2 results lies 5% lower than FDR and 14% lower than SITA results. The tendency which seems to come out of Table 3 is that the less precise input data, the more AEM3 overestimates fuel burn.

* "AddAll corrected" column corresponds to the resulting fuel ratio after compensation of the error due to AEM3 duration estimation. For more details, see [Ref 2] section "Flight Profile Analysis: Fuel Burn".

25

As observed in Table 2, MTOW indicated in JPFleets database for the A310-325 is higher than modelled A310 MTOW. Fuel burn and emissions for this aircraft are thus expected to be globally underestimated by AEM3. This is confirmed by Table 4 and Table 5 below where A310 fuel ratios are lower than B733 and B737 fuel ratios. As a result, the fuel ratio for the overall FDR#2 data sample is actually higher that the value obtained during this study (0.94). This means that FDR and FDR#2 data produce practically the same results and confirmed in the meantime the constant quality of AEM3 results.

Fuel burn by aircraft type

Table 4 below shows fuel burn ratios as a function of aircraft type.

ACType Number of flights Fuel Ratio

A310 182 0.90 B733 483 0.98 B737 127 1.09

Table 4: Fuel ratio per aircraft type

As expected, AEM3 underestimates by 10% fuel burn for A310. The underestimation of fuel burn for B733 lies at 2% only while B737 highlights an overestimation by 9%.

The explanation comes from BADA/AEM3 aircraft/engine combinations differing from actual combinations. A comparison of fuel flows indicated by the ICAO engine exhaust emissions data bank for the actual and the modelled engines lead to the following observations:

For A310, actual fuel flows are higher than modelled fuel flows For B733, actual fuel flows are higher than modelled fuel flows For B737, actual fuel flows are lower than modelled fuel flows

These remarks are valid for all flight phases covered by ICAO engine exhaust emissions data bank.

It is thus logical to notice an underestimation of fuel burn for A310 and B733 while B737 fuel burn is overestimated.

The same tendency is observed when results are broken down per distance range.

Fuel burn by distance range

As discussed in [Ref 2], flight efficiency is strongly dependent on the type of missions range aircrafts are designed for. AEM3 uses mathematical nominal aircraft performance and fuel burn model based on the typical range assumption. As a consequence, AEM3 error increases when aircrafts are not flying optimised distances. (See [Ref 2] for more details.)

26

This statement is verified in Table 5 considering the fact that A310 are designed for long haul missions (around 8500 km maximum) while B733 and B737 are designed for medium haul missions (around 6000 km missions maximum).

In the following Table 5, distance ranges are comparable to distance range used for FDR data during AEM3 validation exercise#1. They are repeated below:

Short haul: less than 1000 km Medium haul: between 1000 and 5000 km Long haul: more than 5000 km

ACType Distance Range

Number of flights Fuel Ratio

Short Haul 52 1.07 Medium Haul 50 0.96 A310 Long Haul 80 0.88 Short Haul 177 1.16 B733 Medium Haul 306 0.94 Short Haul 40 1.30 B737 Medium Haul 87 1.05

Table 5: Fuel ratio per distance

Even if 12% underestimation for A310 long haul missions seems to indicate a consequent AEM3 error, it has to be kept in mind that the high MTOW of the actual aircrafts explains this value. Indeed the fuel burnt to carry a specific mass is not linear: a little additional mass induces a consequent additional need for fuel to carry this mass. As AEM3 can not take into account this feature, the result of 12% underestimation obtained is perfectly coherent.

The highest fuel ratio obtained in Table 5 is 30% fuel burn overestimation for short haul missions performed by a B737. This high value is also logical considering the low fuel flows of the actual engine compared to the modelled one in addition to an un-adapted mission length.

The tendency observed in Table 4 and Table 5 thus confirms findings from AEM3 validation exercise#1.

Fuel flow analysis

Fuel flow evolution

The evolution of AEM3 fuel flow versus time regarding actual FDR#2 fuel flow was plotted for the three aircraft types under study. Three characteristic flights each of 2000 km were plotted

27

for the analysis. This mission length was chosen because it corresponds to a standard medium range flight performed by the three aircraft types to be considered.

The plots are presented in Figure 2 to Figure 4. In each figure, the blue curve represents fuel flow estimated by BADA in AEM3; the pink curve represents the real fuel flow provided with FDR data; and the dotted curve shows the flight profile. The scale used for this third curve refers to the secondary axis on the right of the plot. For more readability, the three plots were represented using the same scale.

Figure 2: Evolution of fuel flow – A310

28

Figure 3: Evolution of fuel flow – B733

Figure 4: Evolution of fuel flow – B737

The first remark coming out of above figures is the variation of fuel flow calculated by AEM3. This phenomenon is due to the way attitudes were determined during data preparation phase

29

(see section "Calculation of missing values for AEM3" p.18). Indeed the high granularity of data allowed of determine flight phases based on real flight levels instead of using only "top of climb" and "top of descent" values as flight phase delimiters. Actual fuel flow observed on Figure 3 for B733 during the main cruising phase is not constant, which confirms that the real aircraft attitude during this flight phase is not permanently "cruise". The real main cruising phase is indeed constituted of a succession of tiny climbing, descending and cruising phases. This corroborates the fact that the choice made for attitude determination during data preparation is correct.

This flight phase determination results in a more precise description of flight profiles and a better estimation of total fuel burn and emissions calculated by AEM3 on a "by flight" scale than the use of "top of climb" and "top of descent" values. Nevertheless a flight leg by flight leg comparison between actual and AEM3-calculated fuel flows would suffer from discrepancies.

AEM3 fuel flows are globally higher than FDR#2 fuel flows during the LTO cycle. This is a logical result due to the fact that LTO fuel flows are calculated using BADA fuel flows even during LTO with the AEM3 options chosen. The result would be different if LTO phases had been added to the profiles.

The highest AEM3 fuel flow variations are found during the main cruise phase. It can be observed on Figure 3 for the B733 that three main values appear during the main cruise phase. This phenomenon is exactly the same on the other figures, even if it is less readable on the plots. As the flight level during this phase is almost constant, the three main values automatically correspond to the attitude indicated in AEM3 input files. AEM3 fuel flow is highest when the attitude of the leg is climb and lowest when the attitude of the leg is descent. When the attitude indicated in AEM3 input files is cruise, AEM3 and FDR#2 fuel flows are close to each other on the three graphs. A deeper investigation directly in AEM3 outputs confirms these statements.

"Per flight" fuel burn totals obtained through AEM3 remain close to actual fuel burn total, as discussed in section "Fuel burn estimation with AEM3". This observation used together with fuel flow analysis performed with DFR data during AEM3 validation exercise#1 confirms the correctness of AEM3 fuel flows.

The consequences from the phenomenon highlighted above appear on the estimation of emissions. As discussed in AEM3 validation exercise#1 ([Ref 2]), the estimation of emissions using AEM3 is very sensitive to the flight attitude indicated in the input flights. Fuel flow peaks due to short descent phases during the main cruise phase, even if lasting for a few seconds only, create a consequent increase of CO and HC emissions. The high number of peaks observed with the FDR#2 data produces a significant amount of CO and HC on a "per flight" basis with respect to NOx.

The impact of "climb peaks" on total NOx emission increase, even if not negligible, is of a lesser extent. One of the reasons is that the absolute figures are bigger for NOx than for CO and HC, which implies a lesser increase in percentage of emission.

As a consequence the evolution of fuel flows used by AEM3 with this precise FDR#2 data is expected to lead to a consequent proportion of CO and HC regarding NOx proportion. This tendency will be confirmed in section "Emissions estimation with AEM3".

30

To summarize, Figure 2 to Figure 4 stress once again the fact that care has to be put on the data preparation to obtain correct results. AEM3 users have to always keep in mind the influence of each parameter in the input files for AEM3. The granularity of data determines the way attitudes have to be attributed.

Fuel flow limits

Limits of actual fuel flow as absolute values were compared to the limits of modeled fuel flow determined with BADA and ICAO. Graphical representations of this comparison are shown on Figure 5 to Figure 7. The limit fuel flows tested in the ICAO engine exhaust emissions data bank for the real engine installed on the aircrafts under study appear on the plots as an indication.

Figure 5: Fuel flow limits – A310

31

Figure 6: Fuel flow limits – B733

Figure 7: Fuel flow limits – B737

32

Figure 5 to Figure 7 show that in keeping with AEM3 validation exercise#1 findings, actual fuel flows indicated in the FDR#2 files do not fit either BADA nor ICAO fuel flow envelopes. Discussion included in section "Fuel Flow Limits" of [Ref 2] remains valid for the three new aircraft types.

The worse correlation between actual fuel flows and BADA envelopes can be observed for A310 and B733. Actual fuel flow is up to twice higher than BADA highest value, which implies a consequent underestimation of NOx emission. This phenomenon considered together with an overestimation of CO and HC due to lower fuel flows (as discussed in section "Fuel Flow Limits" of [Ref 2]) and findings from section "Fuel flow evolution" allows to predict that the distributions between NOx, CO and HC will be significantly impacted.

As observed on Figure 7, the correlation between actual, BADA and ICAO fuel flow limits for the B737 of better quality. The emission distribution will be impacted in a lesser extent than A310 and B733 distributions.

Emissions estimation with AEM3

The estimation of level of realism for the NOx, CO and HC emissions is based on ANCAT and NASA projects, as detailed during the AEM3 validation exercise#1 [Ref 2]. As a reminder, a brief summary of ANCAT and NASA findings can be found at the beginning of the following sections.

NOx, CO and HC distribution

The NASA Scheduled Civil Aircraft Emission Inventories for 1992 [Ref 5] indicates that the internal distribution between the three above pollutants should vary between 72.5 and 90% for Oxides of Nitrogen, 25 and < 10% for Carbon Monoxide and <1 - 2.5% for Hydrocarbon, dependent on the mission length. The estimation is based on Boeing standard mission profiles, for a mission range between 750 and 5500 km (400 NM and 3000 NM) for a B757-200.

33

Figure 8: Emissions Comparison of 757-200 for a 750 km and 5500 km Mission [Ref 5]

The impact on emissions of fuel flow limits is obvious when observing Table 6, which presents the emission distribution for validation data. Indeed B737, which benefits from the best fuel flow correlation, presents a significantly better distribution than A310 and B737 regarding NASA findings ([Ref 5]). Similarly B733 fuel flow limits are worse than A310 limits since ICAO envelope is even smaller than BADA envelope.

Previous sections of the report foresaw a consequent amount of CO and HC regarding NOx emission. Table 6 shows that this prediction is respected.

ACType Number of flights % NOx % CO % HC

A310 182 58.26 34.34 7.40 B733 483 55.59 42.13 2.27 B737 127 66.37 30.58 3.05

Table 6: Emission distribution for the whole FDR#2 traffic sample

Figure 9 provides the same information in a graphic.

34

Figure 9: Emission distribution for the whole FDR#2 traffic sample

NOx proportion evolutes between 55.5 and 66%, while CO varies from 30.5 to 42% and HC from 2.3 to 7%.

The reason for such a high CO and HC percentage lies in the extrapolation of emissions indices outside the ICAO fuel flows, as observed on Figure 5 to Figure 7. This extrapolation to further low fuel flows leads to very high emission indices for CO and HC.

Nevertheless the reasons for obtaining such emissions distributions do not depend only on AEM3 calculations. They come also from FDR#2 data, in particular fuel flow variations due to the high granularity of data and the fact that flight profiles begin (end) before (after) taxi-out (taxi-in). Note that, as indicated in [Ref 2], a way of providing a better treatment to climb and descent phases during the main cruise phase and thus avoid extreme CO and HC peaks will be studied.

NOx average emission indices from ANCAT and NASA

NASA and ANCAT researchers have analyzed the NOx emission estimates for larger traffic inventories and calculated the amount of NOx emissions in relation to the estimated amount of fuel burn. These calculated values are called Average NOx Emission Indices (EINO).

This analysis led to the following estimations for average NOx Emissions Indices in g per kg fuel burn:

35

ANCAT 1A ANCAT2 NASA NASA 1992 1992 1990 1992

Horizontal resolution (°) 2.8 * 2.8 1 * 1 1 * 1 1 * 1

Vertical resolution (°) 1 1 1 1

EINOx (g/kg) 16.8 13.7 10.9 11.1 Table 7: Published average EINOx (g/kg fuel) of reference projects [Ref 7]†

NASA results are based on Boeing Method 1 and Method 2.

ANCAT 1A results have been obtained using a thermo-dynamic NOx emission model which has been replaced during ANCAT 2 by the DLR NOx estimation method. The ANCAT/EC2 inventory in the base year 1991/1992 (published in 1998) estimates that the civil subsonic fleet average emissions index (EINOx: g NOx/kg fuel) is 13.7. This estimation is significantly lower than the previous ANCAT/EC1A study published in 1995 which indicated an average of 16.8 g NOx/kg fuel. This difference is due to revisions to the movement data base and use of a different methodology for the prediction of NOx at cruise altitudes.

The average NOx emission indices for the 792 flights under study are 12.49 g/kg fuel burn. This result is about 12.5% higher than NASA results ([Ref 5]) from 1992 but 8.8% lower than ANCAT2 results ([Ref 6]).

Average EINOx for AEM3 validation exercise#2 are in accordance with ANCAT and NASA findings, but 1.23% lower than results obtained from FDR data with the same options during validation exercise#1.

A break down per aircraft type presented in Table 8 is necessary to identify the phenomenon.

ACType Number of flights

Average EINOx

A310 182 13.37 B733 483 11.53 B737 127 14.89

Table 8: AEM3 estimated EINOx averages in g/kg fuel

Bearing in mind the relatively low discrepancy between fuel flow rates for high power settings for the B737 (Figure 7), it is considered that the average EINOx obtained for the B737 does not experience fuel flow error influence.

† Source: Impact de la flotte aérienne sur l'environnement atmosphérique et le climat; Rapport no.40, Décembre 1997, Institut de France, Académie des sciences – Académie Nationale de l'air et de l'espace.

36

It is a different matter for the A310 and B733, which suffer from fuel flow underestimation for high power settings (Figure 5 and Figure 6). As a consequence, average EINOx in Table 8 for these two aircraft types might be lower than in reality.

A study of "emissions per flight phase" would not be significant with FDR#2 data because of the presence of many climb and descent phases during the main cruise phase.

A graphical representation of emission indices versus time were plotted for the three particular flights studied in section "Fuel flow evolution". But fuel flow variations induced emission indices variation which made plots un-exploitable.

37

CCoonncclluussiioonn

The AEM3 validation exercise#2 confirmed results obtained during exercise#1. As a conclusion of the two AEM3 validation exercises, AEM3 fuel burn estimation offers a high level of realism. AEM3 fuel burn is globally underestimated by only 3 to 6% on average when no flight completion is required. The percentage of underestimation depends on many factors mainly based on aircraft types and characteristics.

It is obvious that the better a real aircraft compares to the underlying aircraft model assumptions (BADA) in AEM3, the higher is the level of realism obtained by AEM3 for fuel burn and emissions estimation. The influence of take-off weight is significant as well as mission length for which aircrafts are designed.

The following figure compares AEM3 fuel burn estimation to actual FDR and FDR#2 operational data for "NoAdd" option, i.e. for both AEM3 validation exercises.

Figure 10: The AEM3 fuel burn vs. operational fuel burn for 4642 flights

The correlation between AEM3 fuel burn and actual fuel burn is of high level for more than 4600 flights.

Emissions can not be compared to actual data since such information was not available. Nevertheless they compare to NASA and ANCAT estimations which allows to conclude that emissions indicated by AEM3 are realistic.

38

The two different AEM3 validation exercises exploited different kinds of data granularity leading to comparable positive results. Based on the good results for the 14 aircraft types validated so far, it is assumed that the AEM3 fuel burn and emission estimation for other aircraft types is of similar quality. AEM3 validation for additional aircraft-types will continue with availability of FDR data.

39

RReeffeerreenncceess

[Ref 1] ICAO Engine Exhaust Emissions Data Bank; ICAO; Doc 9646-AN/943: First Edition – 1995; Internet Issue 1 (10/03/1998); Internet Issue 2 (08/02/1999), Internet Issue 10(19/05/2003)

[Ref 2] The Advanced Emission Model (AEM3) Version 1.5 – Validation Report; EUROCONTROL Experimental Centre; Society, Environment and Economics Business Area; Jelinek, Carlier, Smith; EEC/SEE/2004/004

[Ref 3] ICAO Engine Exhaust Emissions Data Bank; ICAO; Doc 9646-AN/943; First Edition – 1995; Internet Issue 1(3/10/1998); Internet Issue2(8/2/99)

[Ref 4] JP airline-fleets international Aviation Database – BUCHair UK ltd

[Ref 5] Scheduled Civil Aircraft Emission Inventories for 1992: Database Development and Analysis; April 1996; NASA LRC; Contractor Report 4700; Steven L. Baughcum, Terrance G. Tritz, Stephen C. Henderson, David C. Picket

[Ref 6] ANCAT/EC2: Global Aircraft Emissions Inventories for 1991/92 and 2015 – Report by the ECAC/ANCAT and EC working group – Editor R.M. Gardner

[Ref 7] Impact de la flotte aérienne sur l'environnement atmosphérique et le climat; Rapport no.40, Décembre 1997, Institut de France, Académie des sciences – Académie Nationale de l'air et de l'espace

For more information about the EEC Society, Environment and Economy Research Area please contact: Ted Elliff SEE Research Area Manager, EUROCONTROL Experimental Centre BP15, Centre de Bois des Bordes 91222 BRETIGNY SUR ORGE CEDEX France Tel: +33 1 69 88 73 36 Fax: +33 1 69 88 72 11 E-Mail: Ted.Elliff@eurocontrol.int or visit http://www.eurocontrol.fr/