ICAO EUR Region Performance Framework (ICAO EUR Doc … Meetings Seminars and Workshops... ·...

6 March 2014 Page 1

ICAO EUR Region Performance Framework

(ICAO EUR Doc 030)

World ATM Congress

Madrid, 5th March 2014

ICAO European and North Atlantic Office

6 March 2014 Page 2

- Present the context: the ICAO global framework and regional

initiatives

- Describe the ICAO EUR Region performance framework

- Raise awareness for non-regional stakeholders and provide

information to support implementation for regional

stakeholders

- Collect inputs and views from participants

2

Objectives

6 March 2014 Page 3

States in the ICAO EUR Region, responding to ICAO requirements, decided to

elaborate a Regional Performance Framework, to improve the performance of

the aviation system and indentify areas where improvements are possible

To prepare such framework a dedicated Task Force was created

The TF analysed what already in place in/outside the Region – to understand the

context, to make maximum use of what available and avoid duplications

The TF prepared a comprehensive framework with KPAs/KPIs and guidance

material to support implementation

We are at the beginning of the implementation phase

3

Introduction

6 March 2014 Page 4

ICAO Performance Based

Approach


6 March 2014 Page 5

• Endorsement of the global ATM operational concept already at 11th Air Navigation Conference in 2003 and ICAO is requested to:

» develop ATM system requirements

» address interoperability and seamlessness

» define requirements for global AIM

» publish the operational concept

» amend the Global Plan

» develop a performance framework

• States and PIRGs consider the Global Air Navigation Plan for CNS/ATM Systems as a catalyst for change, providing a global safety and interoperability framework while allowing regional or local adaptation to efficiently meet regional and local needs

• States and PIRGs agree on a set of metrics related to KPAs, incorporate these metrics into the performance monitoring process and review their results on a regular basis (EANPG,RASG)

Air Navigation Conference 11 in 2003 How did we start ?

6 March 2014 Page 6

• The Conference unanimously endorsed the Aviation System Block Upgrades (ASBUs) framework introduced by ICAO to set goals in terms of operational improvements on a consensus-driven basis.

• The Conference also endorsed the draft 4th edition of the Global Air Navigation Plan (Doc 9750), as unified planning mechanism, which was approved by the 38th ICAO Assembly in September 2013

• ICAO is tasked to include the key policy principles in the GANP, develop financial policies, SARPS development plan, define a stable and efficient process to update the GANP

• ICAO (through Regional Offices) will provide guidance and assistance on ASBUs implementation to States, establish a group and mechanism for inter-regional cooperation

Air Navigation Conference 12 in 2012 How do we continue

6 March 2014 Page 7

GANP and GASP Advantages

• A globally harmonized and time-wise synchronized set of ATM improvements to direct the work of States, ANSPS, AOs, airports, ICAO and other international and standard making organizations

• A globally agreed ATM baseline through commitment to implement Block 0

• A systematic approach to ATM improvements, a packaged solution that includes standards/guidance, technology, procedures, regulatory framework, business case, performance measurements

6 March 2014 Page 8

• Recommendation 1/15 – Performance monitoring and measurement of air navigation systems

The Committee noted that the implementation of “One Sky” would require the use of a common “language” regarding performance monitoring and measurement. This implies a coordinated approach across States and stakeholders for data provision, collection, storage, protection, dissemination, indicator calculation and use of the results to support the various ATM improvement processes. It was agreed that there was a need to develop a global methodology, to identify metrics and indicators which could be used to allow States and regions to measure and evaluate the effectiveness of their ATM performance initiatives. In doing so, particular attention should be given to avoiding duplication of efforts and to use, to the maximum extent possible, existing arrangements and solutions

That ICAO: a) establish a set of common air navigation service performance metrics supported by guidance material, building on existing ICAO documentation (e.g. Manual on Global Performance of the Air Navigation System (Doc 9883) and the Manual on Air Navigation Services Economics (Doc 9161)); b) promote the development and use of “leading safety indicators” to complement existing “lagging safety indicators” as an integral and key component to drive improvement in performance and in the achieved management of risk; and; c) encourage the early and close involvement of the regulator and oversight bodies in the development, proving of concepts and implementation of the aviation system block upgrades and regional programmes.

AN-Conf/12 Recommendations

6 March 2014 Page 9

• A wider (more regional, if not even global vision) perspective is needed to make even greater gains in capacity and efficiency

• A far-reaching cooperation is necessary in order to support the implementation of facilities and services over larger geographical areas

• Enhanced planning perspectives and a global framework for performance measurement is required to support the implementation activities

• States and all aviation stakeholders should – work together towards a common vision (global ATM Operational Concept)

– use a common planning framework

• Global Air Navigation Plan (GANP), the regional air navigation plans and several other documents and tools (e.g. European ATM Masterplan)

• Global Aviation Safety Plan (GASP)

– utilize harmonised performance objectives (targets, metrics, indicators)

• Interoperability and harmonization are key factors in order to ensure the required improvements for the future

The wider ATM perspective

6 March 2014 Page 10

ICAO EANPG

• Ensure that air navigation system development plans and actions within the EUR Region remain coherent and compatible with the ICAO Strategic Objectives, with the ICAO global plans (especially GANP and ASBUs, parts of GASP), with those of the adjacent ICAO Regions and with other world-wide provisions.

• Manage the EUR Air Navigation Plan (Performance directives and associated requirements for

facilities and services established through regional air navigation agreements, in support of the global air navigation infrastructure) and facilitate the implementation of the international operational requirements contained therein.

• Promote and facilitate the harmonisation and co-ordination of the air navigation related programmes (e.g. SESAR, State ATM Corporation modernisation program, NextGen) of other international organisations such as the European Commission, the European Civil Aviation Conference (ECAC), Eurocontrol and the Interstate Aviation Committee of the Commonwealth of Independent States (IAC/CIS).

• Ensure the interoperability of the EUR CNS/ATM systems with those of adjacent regions and in line with the ICAO GANP.

• Assist States in their planning and implementation efforts, if and when required.


Performance Based Approach

• ICAO Performance Based Approach is based on the following three principles:

– Strong focus on desired/required results

– Informed decision making, driven by the desired/required results

– Reliance on facts and data for decision making


• Instead of prescribing solutions, desired/required performance is specified.

• Management attention is shifted from a resource and solution centric view (how will we do it) towards a primary focus on desired/required (performance) results (what is the outcome we are expected to achieve).

• This implies finding out – what the current performance situation is, – what the most appropriate results should be, – clarifying who is accountable for achieving those

results.

Focus on desired/required results

6 March 2014 Page 13 ICAO EUR/NAT Office COG PERF TF Workshop 13

• Performance objectives: Specific, Measurable, Achievable, Relevant & Timely (SMART) objectives defined to satisfy ATM community expectations.

• Performance indicators: to measure achievement of performance

objective • Performance targets: value of performance indicators that need to be

reached/exceeded to fully achieve performance objective. • Performance gaps: Performance indicators compared against performance

targets identifying gaps • Performance assessment metrics: determine which data needs to be

collected to calculate values of performance indicators

Performance terms


Expectations

KPA

Objectives

Indicators Quantitative

Targets ... Objective is met when indicators meet or exceed targets


• Assess current performance: establish baseline & track progress towards performance objectives

• Identification/diagnosis performance gaps: Performance gaps denote current/anticipated mismatch (unfavorable) between current/planned performance and performance target.

• Addressing performance gaps: Operational improvements in terms of changes to ATM system that are on transition path towards Global ATM Operational Concept and result in a direct performance enhancement.

• Developing list of options for operational improvements: Operational improvements made possible by technical systems, human factors, procedure and industrial enablers in terms of feasibility, timing, cost and impact on operational change. Global Air Navigation Plan (Doc 9750) is one of the sources, which can be used.

• Building/updating transition roadmaps: Performance gaps identify the affected performance objectives. Categorization of operational improvements according to performance objectives leads to development of appropriate shortlist of candidate solutions. If the solution is not included in the transition roadmap, then it should be selected on the basis of maturity or research, if no mature solution is available.

Performance assessment


• Understanding Performance is key for successful transition from past and current to the future

• Working under one common umbrella – Performance based transition

– Continued operational improvements

– Toward a common vision

– Using the ICAO framework

• Using the 3-step transition approach – Set objectives/metrics

– Identify and understand performance gaps

– Select and plan Operational Improvements

• Supporting the goal of a more seamless and performance based Global Air Navigation System

In summary


Existing processes in the Region



SES II - Performance in RP2 (2015-2019) – Binding performance targets in 4 key

performance areas (Safety, Environment, Capacity and Cost-Efficiency)

– Mandatory implementation of a financial incentive scheme for the target achievement on the KPA Capacity

– Performance planning on FAB level

– 'gate-to-gate' approach covering the entire chain of air navigation services

• Target setting on terminal charges as of 2015 (Union-wide target setting foreseen as of 2017)

– Network Manager integral part of the scheme

– Clear link of new investments and major overhaul of systems to the EU ATM Master Plan

EU Performance scheme


Performance Review Report – analysis of the performance of the ATM systems in the KPA of Safety, Capacity, Environment and Cost-efficiency in Europe

US-Europe comparison reports – analysis of the operational and economic ATM performance

ATM Cost-Effectiveness Benchmarking Report – analysis of cost-effectiveness and productivity in 37 European ANSPs

- Comparison of Air Traffic Management-related Operational Performance 2008-2012

- Comparison of ANS cost-efficiency trends 2002-2011

- Joint publications influenced ICAO and CANSO work on global performance benchmarking

Performance reports


ICAO EUR Region Performance Framework



• Based on clear principles: • Produce improvements to the aviation system • Avoid duplication of efforts and make maximum use of existing arrangements • As far as possible be applicable also in the NAT Region • Consider impact on resources • Follow a pragmatic and stepwise approach

WHY • To elaborate a Performance Framework for the EUR Region, to improve

the performance of the aviation system and indentify areas where improvements are possible

WHEN • Decision at COG 47 (Rostov on Don, June 2010)

HOW

Should also represent

• Regional input to ICAO HQ activities in developing metrics for global use

• Support the future developments of EU Performance Scheme

ICAO EUR Performance Framework WHO ?


Pragmatic approach in developing the proposal

Input ICAO framework (ANC 12, GA38, etc.) EU-ECTL (Single European Sky and Performance Scheme) FAA Russian Federation ICAO EUR Workshops (Rome 2011, Bishkek 2013)

To assess the Regional performance and to identify areas where improvements are possible

Output

Development of a comprehensive framework Identification of 6 KPAs out of 11 ICAO KPAs Definition of focus areas, objectives and KPIs Definition of Processes, Roles and Responsibilities Guidance material

Safety

Access and equity

Efficiency

Capacity

Environment

CostEffectiveness

Participation byATM

community

Global Interoperability

Flexibility

Security

Predictability

ICAO EUR Performance Framework WHAT ?


• ICAO EUR Region Performance Framework Document (ICAO EUR Doc 030) was approved by EANPG and COG in 2013 and published in English and Russian language.

• Performance Framework document describes:

INTRODUCTION

– BACKGROUND INFORMATION

– RELATIONSHIPS WITH ICAO GLOBAL DEVELOPMENTS AND REGIONAL INITIATIVES

PART A

– SCOPE

– ROLES AND RESPONSIBILITIES

– KPA/KPI/METRICS

– MONITORING, REPORTING AND REVISION PROCESS AT REGIONAL/NATIONAL LEVEL

PART B

– KPAs

– TEMPLATES FOR THE REGIONAL PERFORMANCE REVIEW REPORT

ANNEX 1: TERMINOLOGY

ANNEX 2: REFERENCE MATERIAL

Performance Framework Document


KPA OBJECTIVES FOCUS AREAS INDICATORS

SAFETY

Ensure safety continuousimprovement through reductionof ATM related safetyoccurrences and implementation of uniformsafety standards

Effectiveness of Safety Management (SafetyMaturity Questionnaire)

Level of State Safety/Just culture (SafetyCulture Questionnaire)

Adoption of an harmonized occurrences severity classification methodology

CAPACITY Capacity meets demand for en-route and at airports

En-route ATFM Delay Average en-route ATFM delay generated by airspace volume

Airport ATFM Delay Average ATFM delay per flight in the main airports (to be identified by States)

EFFICIENCY Ensure users may use most efficient routes

HorizontalFlight Efficiency

Average horizontal en route flight efficiency (length of the en route part of the actual trajectory/last flight planned route vs great circle)

ENVIRONMENT Contribute to the protection of environment (fuel/CO2 emissions reduction)

CO2 emissions related to inefficiencies in route extension

COST EFFECTIVENESS

Contribute to optimization of costs for ANS

ATCO Productivity IFR Flights (en-route) per ATCO hour duty

IFR flight hours per ATCO hour on duty

IFR movements per ATCO hour on duty

PARTICIPATION BY ATM COMMUNITY

Ensure States' participation toRegional planning and implementation activities

Level of participation to meetings

Level of responses to planning activities

Level of provision of performance results

KPAs-KPIs summary table


KPA Safety

Objective Ensure the continuous improvement of safety through the reduction of ATM related safety occurrences and the implementation of uniform safety standards.

Indicators - Effectiveness of Safety Management (EoSM)

- Level of State Just Culture (JC)

- Application of a common methodology for classification of occurrences in terms of risk severity

Safety KPA – Overview


• Effectiveness of Safety Management (EoSM) – Based on the annual completion of two questionnaires

(State level and ANSP level)

• Level of State Just Culture (JC) – Based on the annual completion of two questionnaires

(State level and ANSP level)

• Application of RAT methodology for Severity Classification – Objective is to motivate States to use a common

methodology for the severity assessment of safety occurrences

– Benefit: harmonisation of occurrence statistics

Safety KPA – Nature of indicators


Safety KPA – Conclusions

• Three Indicators – These are leading indicators

• measuring precursors to improved safety – And not lagging indicators

• which are directly related to safety outcome

• Application – Have already been used by many States since 2012 during SES

Reference Period 1 (RP1) and now reinforced for RP2 – Under ICAO umbrella geographical extension to remaining States of the

EUR Region • Complex subject, but detailed guidance material is available

– Fully described in EASA documentation – ICAO guidance material summarises and explains the EASA

documentation


KPA Capacity

Objective Ensure that Air Navigation Service capacity meets demand in en-route airspace and at airports

Indicators - Average ATFM delay per flight generated by the airspace volume (en-route)

- Average ATFM delay per flight in the main airports (to be identified by States in advance and based on the regional relevance)

Capacity KPA – Overview


• Re-use of ATFM delay indicators as defined in SES Performance Scheme Regulation – Measures the delay generating

entities (airspace and airports which are bottlenecks), not the penalised entities (flights)

• Covers majority of States where capacity is an issue – ATFCM Area

• Relevance in remainder of EUR Region to be determined – Capacity often not (yet) an issue

or only a local issue

ATFM regulation

requested by ACC

Arrival

Airport Departure

Airport

ACC anticipating

capacity shortfall

NM

Aircraft held at

dep. airport

Capacity KPA – Application


• Indicators measure – The location where the problem is, not where the delay is

taken (departure airport) – Performance of airspace volumes and airports, not flights

• Despite the expression as a value “per flight”

– Within the Capacity KPA • Demand/capacity imbalance • Not capacity itself

• Limitations • Not designed to measure excess capacity • No data if airspace or airport does not participate in a centralised

ATFCM process

Capacity KPA – Conclusions


KPA Efficiency

Objective Ensure that users [can?] use the most efficient routes – focussing on the horizontal flight-efficiency

Indicator Average horizontal en route flight efficiency, defined as the difference between the length of the en route part of the actual trajectory (where available) or last flight planned route and the great circle. Note: specificities shall be considered for flights longer than 1000 nm where the optimum

could differ from the great circle (wind optimal routes, etc.).

Efficiency KPA – Overview


• Re-use of indicator as defined in SES Performance Scheme Regulation – The average horizontal en route flight efficiency of [(a) the actual

trajectory, or (b) the last filed flight plan trajectory], defined as follows: • (i) the indicator is the comparison between the length of the en route part

of [(a) the actual trajectory derived from surveillance data, or (b) the last filed flight plan trajectory] and the corresponding portion of the great circle distance, summed over all IFR flights within or traversing the European airspace;

• (ii) ‘en route’ refers to the distance flown outside a circle of 40NM around the airports;

• (iii) where a flight departs from or arrives at a place outside the European airspace, only the part inside European airspace is considered.

Note: the SES indicator is expressed as a percentage: Sum of extra distance divided by the sum of corresponding portions of great circle distances

Efficiency KPA – Origin of indicator


Indicator option B

requires surveillance

data

Indicator option A

City pair distance (Great circle distance)

Shortest Route

Shortest Available Route

Last Filed FPL

Actual Trajectory

Business need: “Get from A to B” = direct route

Planning Operations Leng

th o

f Tra

ject

ory

Route Network Design

Route & Airspace Availability

Awareness and Choice

ATC Separation Fragmentation

Wind-optimum Cost-optimum

Difference due to planning limitations

Ref

eren

ce d

ista

nce

Extr

a di

stan

ce

Ref

eren

ce d

ista

nce

Extr

a di

stan

ce

Based on information known in advance

Tactical decisions based on updated information

Efficiency KPA – Nature of indicator


• Indicator – Measures actual (flight plan data or radar data), achieved,

and additional distance flown within an airspace volume (eg within a State)

– Can cope with overflights

– Additional distance can be computed separately for flights shorter and longer than 1000 nm

– Results are geographically aggregatable

– Represents the overall outcome of several influencing factors, for which different Stakeholders are accountable

Efficiency KPA – Conclusions


KPA Environment

Objective Contribute to the protection of the environment – focussing on fuel savings and CO2 emission reductions

Indicator CO2 emissions deriving from inefficiencies in flight efficiency (conversion of additional distance into CO2 emissions based on standard values formula) .

Environment KPA – Overview


• Reference to excess fuel consumption: Total additional distance flown in the airspace volume (i.e. State) multiplied by a standard fuel consumption factor (value State-specific)

• Reference to excess CO2 emission: Reference to excess fuel consumption multiplied by 3.15 (net carbon content of kerosene)

Trajectory

Actual distance

Achieved distance

Additional distance

Standard fuel consumption

per km

Net carbon content of kerosene (3.15)

Reference excess fuel consumption

Reference excess CO2 emission

Environment KPA – Definitions


• Indicator – Has low data requirements and is therefore easy to

implement • Because mostly based on “additional distance” already computed

under the Efficiency KPA

– Publishes an approximation of excess CO2 emission, resulting from horizontal flight inefficiency

• The optimum indicator value is not equal to zero

• Hence the absolute value of the indicator should not be interpreted as representing the CO2 emissions caused by ANS.

• Indicator can be calculated with IFSET or other tools and should be used for trend analysis only

Environment KPA – Conclusions


KPA Cost effectiveness

Objective Contribute to optimize the cost for air navigation services

Indicators - IFR flights (en-route) per ATCO hour on duty

- IFR flight hours (en-route) per ATCO hour on duty

- IFR movements (airport) per ATCO hour on duty

Cost Effectiveness KPA – Overview


• SES indicators not re-usable because dependent on States’ participation in EUROCONTROL Route Charges System

• Asking for extensive disclosure of economic and financial information not deemed realistic for the initial EUR Region performance framework

• Focus is on controller (ATCO) productivity:

– Staff cost represents the main cost for service provision – ATCO staff cost represents the most important share in

staff cost – ATCO productivity can be expressed in non-financial terms – Existing data flow covers many EUR Region States already

Cost Effectiveness KPA – Origin of indicator


• Indicators – Focus on an important component of ANSP costs

• However with the limitation that working hours per ATCO, employment cost per ATCO and support cost are not covered

– Re-use data reporting already in place for many States • Also reuse existing data definitions and terminology

– Have relatively simple reporting requirements • Only traffic volume and ATCO hours on duty

– Avoid entering into financial information disclosure issues for the initial implementation of the framework

Cost Effectiveness KPA – Conclusions


KPA PARTICIPATION BY ATM COMMUNITY

Objective Ensure States participation to regional planning and implementation activities so that air navigation system development plans and action within the EUR/NAT Region remain coherent and compatible with those of the adjacent ICAO Regions and with the ICAO global plan and world-wide provisions.

Indicators - Level of active participation by States and international organisations

- Level of responses to State Letters

- Level of provision of requested information or performance results from aviation stakeholders

ATM Community Participation KPA – Overview


• ICAO Manual on Global Performance of the Air Navigation System (Doc 9883) KPA - Participation by the ATM community There are currently no common indicators obtained and some stakeholders are attempting to use the following indicators:

– number of yearly meetings covering planning, implementation and operation, and covering a significant estimated proportion (e.g. 90 per cent) of the whole of the regional aviation activity

– number of yearly meetings for planning

– number of yearly meetings for implementation

– number of yearly meetings for operations

• The ATM community is defined in Appendix A of the Global Air Traffic Management Operational Concept (Doc 9854) : It comprises (in alphabetical order) the aerodrome community, the airspace providers, the airspace users, the ATM service providers, the ATM support industry, the International Civil Aviation Organization (ICAO), the regulatory authorities and States

ATM Community Participation KPA – Origin


ATM Community Participation KPA – Conclusions • Three Indicators will be monitored and evaluated by the ICAO

EUR/NAT Office • Level of participation of States and international organisations to planning and

implementation meetings (e.g. EANPG or its contributory groups with a focus on regional coordination/implementation aspects like Aeronautical Fixed Services Group, Air Traffic Management Group, Frequency Management Group, Meteorology Group, Route Development Group, Regional Performance Framework implementation activities)

• Level of responses to State Letters asking for information on planning and implementation aspects (e.g. Completed Air Navigation Report Forms ARNF, State Letters which assess the status of planning and implementation of key ASBU modules, State Letters concerning the amendment to Regional Supplementary Procedures, Reports on the environmental benefits that were derived from the implementation of operational improvements)

• Level of provision of performance results from States for the Regional Performance Review Report (RPRR)

• Crucial support mechanism for PIRGs (EANPG) to monitor planning and implementation activities in the ANP

• Important for Air Navigation Report Forms (ANRFs) and ASBU monitoring


Implementation and next steps



Roles and Responsibilities in the functioning of the ICAO EUR Performance Framework – EANPG – COG and contributory Bodies – ICAO EUR Secretariat – States

Processes at Regional and National level

Roles and Responsibilities for implementation


Process (Regional – National)


• EANPG 55 (2013) concluded to launch the implementation phase – Collect data (2014)

– Monitor performance

– Fill the gaps in the system to improve

– Report performance results in 2015

TF is working to support States and Stakeholders in this implementation

– Circulation of a questionnaire

– Identification of focal points

– Awareness campaign - Meetings and workshops (especially in the Eastern part of the EUR Region)

Implementation of the Framework


Implementation of the Framework

All material is on the ICAO EUR/NAT Office, Paris website http://www.paris.icao.int/documents_open_meetings/subcategory.php?id=169

http://www.paris.icao.int/documents_open/files.php?subcategory_id=170

Next workshop in Baku (Azerbaijan) 10 - 11 April 2014 !!!

ICAO Regional Performance Framework Workshop

(State Letter invitation EUR/NAT 14-0109.TEC, from 10 February 2014)


Conclusions

• ICAO EUR Region has defined a performance framework aimed at monitoring/reporting performance

• Objective is to identify areas where improvements are possible

• Built on existing ICAO documents and regional initiatives

• Proposed as a good practice towards ICAO HQ and other Regions

• This presentation is available at: http://www.paris.icao.int/documents_open_meetings/subcategory.php?id=169


Thank You

Date post:	06-Jul-2018
Category:	Documents
Upload:	votram
View:	242 times
Download:	0 times

ICAO EUR Region Performance Framework (ICAO EUR Doc … Meetings Seminars and Workshops... ·...

Documents