International Civil Aviation Organization

ICAO EUR/NAT Office COG PERF TF Workshop 1

COG Performance Task ForceWorkshop

EASA Standardisation InspectionsVerification of Safety KPIs

Milen Dentchev, ATM/ANS Standardisation Manager

Rome 21-22 September 2011

ICAO EUR/NAT Office COG PERF TF Workshop 22

EASA Standardisation – Legal Basis

Where Member States act as Competent Authority the EASA Basic Regulation establishes a mechanism to monitor the correct implementation of applicable regulations.

EASA Basic Regulation (EC) No. 216/2008:

- Art. 24 (Monitoring the application of rules)

- Art. 54 (Inspections of Member States)

Working methods regulated by Commission Regulation (EC) No 736/2006

ICAO EUR/NAT Office COG PERF TF Workshop 333

Head ofStandardisation

T. Mickler

Head ofStandardisation

T. Mickler

Air OperationsG. Rebender

Air OperationsG. Rebender

Accreditation J. Müller

Accreditation J. Müller

Flight Crew LicensingG. De Crescenzo

Flight Crew LicensingG. De Crescenzo

Airworthiness M. Kieft

Airworthiness M. Kieft

Deputy HoDM. Kompare

Deputy HoDM. Kompare

SecretariatSecretariat

Organisational Structure

DevelopmentC. Trevisan

DevelopmentC. Trevisan

IAWIAWCAWCAW

ANSM. Dentchev

ANSM. Dentchev

LISLIS FSTDFSTD MEDMED ATMATM CNSCNS AISAIS METMET LISLISCFCF NCFNCFACCACC

OCCOCC

ICAO EUR/NAT Office COG PERF TF Workshop 4

Standardisation Department (S.1)

• Tasks

- Standardisation (“EASA Countries”)- Standardisation (“ECAA” and “WA” Countries)- International Standardisation (“Third Countries”)

- Accreditation (NAAs working for EASA)

ICAO EUR/NAT Office COG PERF TF Workshop 5

FL

IS

GEAM

AZ

EE

UA

MD

TR

CY

FR

LV

LT

BE

NL

DE

UK

IR BY

RO

AL

BG

GR

CH

IT

AU HU

ESPT

DK

NO SE

PL

CZ

MT

BA

MA DZ TN

SY IQ

IR

KZ

RU

LU SK

SL

RSM

MC

RU

LI

The playing field:27 EU3 EEA1 CH7 ECAA8 ECAC

ME KO

RS

FY

HR

Geographical Scope

ICAO EUR/NAT Office COG PERF TF Workshop 6

EASA Inspection process

6

Preliminary meeting

10 weeks min

Preparatory phase

Openingsession

Closure Phase

Follow-up phase

Final Report

16 weeks max

On Site

As necessary

ClosureActionPlan

Finding Classification Committee

Wrap-up meeting

Reporting phase

12 weeks max

Closingsession

As necessary

Preliminary Report

Visiting phase

Notice of inspection

Visit programme

Supplementary Report

ICAO EUR/NAT Office COG PERF TF Workshop 7

Standardisation Strategy

Standardisation Strategy

Successful standardisation cannot depend only on regulatory compliance verification– Performance verification is part of regulatory verification

EASA’s strategy is based on three pillars:

RegulatoryComplianceverification

Pro-active measures

Regulatoryfeedback

ICAO EUR/NAT Office COG PERF TF Workshop 8

NSA Standardisation Forums

NAA Partnership meetings

NSA Standardisation Coordination meetings

ANS National Coordination meetings

Post ATM/ANS Standardisation visit meetings

Internal EASA FCC meetings SES

NCP

ICAO EUR/NAT Office COG PERF TF Workshop 9 9

Increase added value

MoU between EU and ICAO

– Addressing aviation safety, aviation security, air traffic management and environmental protection

– New Working arrangements

– Transparency through mutual exchange of safety information and access to databases

– Reduce burden of multiple audits/inspections on Member States

ICAO EUR/NAT Office COG PERF TF Workshop 10 10

Second extension of EASA

Regulation (EC) No 1108/2009 amending EASA Basic Regulation

– EASA mandated to conduct ATM/ANS standardisation inspections

– Regulation 736/2006 will be amended accordingly

– Transition from EUROCONTROL ESIMS audits to EASA Standardisation inspections

ICAO EUR/NAT Office COG PERF TF Workshop 11 11

Scope of ANS according to ICAO

Annex 2 - Rules of the AirAnnex 3 - MeteorologyAnnex 4 - ChartingAnnex 5 - Units of measurementsAnnex 10 - Aeronautical telecommunicationsAnnex 11 - Air Traffic ServicesAnnex 12 - Search and RescueAnnex 13 - Accident & Incident InvestigationAnnex 15 - Aeronautical Information ServicesAnnex 16 – Environmental protection

PANEP meeting 17-18 May 2011

ICAO EUR/NAT Office COG PERF TF Workshop 12 12

Scope of EASA ANS standardisation inspections

Annex 1 - Personnel Licensing (ATCO + ATSEP)Annex 2 - Rules of the AirAnnex 3 - MeteorologyAnnex 4 - ChartingAnnex 5 - Units of measurementsAnnex 10 - Aeronautical telecommunicationsAnnex 11 - Air Traffic ServicesAnnex 12 - Search and RescueAnnex 13 - Accident & Incident Investigation (SOR)Annex 15 - Aeronautical Information ServicesAnnex 16 – Environmental protection

PANEP meeting 17-18 May 2011

ICAO EUR/NAT Office COG PERF TF Workshop 13

• EASA’s ATM/ANS standardisation section:

• Checklists and pre-visit questionnaires in preparation

• Standardisation inspection annual programme ready

• NSAs invited to confirm or nominate National Coordinator

• NSAs have been invited to identify and nominate qualified

inspectors to participate in EASA standardisation

inspections

• 1st training course for NAA inspectors in Jul, 2nd in Sep, next

in Jan 2012 (40 people trained)

• 6 EASA Team Leaders Q1 of 2012, 9 Team Leaders Q2 2012

• 1st inspection in Feb 2012 (1 TL + 4 TMs)

Transition from ESIMS to EASA inspections

ICAO EUR/NAT Office COG PERF TF Workshop 14 14

Amendment of WA

At the request of the European Commission, EASA has approached all WA countries (including all non EASA – EUROCONTROL countries) with a proposal to extend existing WAs to ATM/ANS.

All ECAA States responded positively

Non-ECAA States:Positive answers from Ukraine, Armenia, Georgia, Moldova, Monaco & San MarinoNo answer from Azerbaijan yetTurkey – no decision yet

14

ICAO EUR/NAT Office COG PERF TF Workshop 15 1515

Verification of safety performance KPIs

ICAO EUR/NAT Office COG PERF TF Workshop 16

ADDITIONAL FACTORS

08.2011 16

Confidence Level: the concept

TimeElapsed

NAA CORE INDICATORS

= ConfidenceLevel

Follow-upIndicators

AuthorityIndicators

Safety occIndicators

ActivityIndicators

GeneralIndicators

ICAO EUR/NAT Office COG PERF TF Workshop 17

The model evaluates firstly two «core indicators»:

08.2011 17

Core indicators: the NAA and its follow-up

Authority

Complexity (structure, ROs, ...)Funding / SalariesStaffingStabilityMaturityStaff qualification and training Internal guidance

External guidance Oversight methodologyManagement of findings Enforcement capabilitiesReaction to AIBs' rec’sICAO USOAP/CMA results

Follow-up

Reactivity IndexSupplementary Reports Repetitive key findings

This is, to a large extent, the current scope of our

standardisation activities

ICAO EUR/NAT Office COG PERF TF Workshop 18

The model then takes into account the following indicators:

08.2011 18

Additional indicators

Aircraft registeredOrganisations approved Personnel licences issuedTraffic indicators Domain-specific indicators

AccidentsSAFA Country ratioOperators in "high SAFA“ listImmediate Safety HazardsDomain-specific indicators

Activity

Safety

NAA IndependenceSafety cultureReporting cultureJust cultureCorruption indexFinancial indicatorsMajor gov’t changes

General

ICAO EUR/NAT Office COG PERF TF Workshop 19

Finally, the model considers the influence of time elapsed since the last inspection,

as the reliability of the information on the NAA’s capability and performance taken

at that stage will inevitably decrease with time:

08.2011 19

The influence of time

Timeelapsed

Time elapsed

Confi

denc

e le

vel

Threshold

Inspection due!

=

Decrease with time

ICAO EUR/NAT Office COG PERF TF Workshop 2008.2011 20

Vulnerability Index

- Confidence levels (CLs) are determined for each domain:

- The lowest CL would highlight the «most vulnerable» area

- A Vulnerability Index may therefore be defined as follows:

VI = 100 – min (CLs) Note: work in progress!