Abstract The purpose of this document is to provide guidance to Project Managers (Action Coordinators) of SESAR 2020 projects and their teams about how to conduct their project and comply with needs as defined by the SESAR 2020 programme execution framework. This guidance is applicable as of 01 March 2020 (e.g. ER4 projects launched into execution) This framework will support all projects and its application is mandatory with the aim to execute the European ATM Master Plan; for ATM Excellent Science and Outreach (Fundamental Oriented research) projects only sections 1 to 3 are fully applicable while for ATM Application-Oriented research projects all sections are applicable as necessary. Project Handbook of SESAR 2020 Exploratory Research Call H2020-SESAR-2019-2 (ER4) (Programme Execution Guidance) Edition date: 14 th March 2019 Edition: 03.00.00
Transcript
Abstract
The purpose of this document is to provide guidance to Project Managers (Action Coordinators) of SESAR 2020 projects and their teams about how to conduct their project and comply with needs as defined by the SESAR 2020 programme execution framework. This guidance is applicable as of 01 March 2020 (e.g. ER4 projects launched into execution)
This framework will support all projects and its application is mandatory with the aim to execute the European ATM Master Plan; for ATM Excellent Science and Outreach (Fundamental Oriented research) projects only sections 1 to 3 are fully applicable while for ATM Application-Oriented research projects all sections are applicable as necessary.
Project Handbook of SESAR
2020 Exploratory Research Call
H2020-SESAR-2019-2 (ER4)
(Programme Execution Guidance) Edition date: 14th March 2019 Edition: 03.00.00
3.10. Communication and Dissemination .......................................................................... 56 3.10.1. Objective ..................................................................................................................................... 56 3.10.2. Approach ..................................................................................................................................... 57 3.10.3. Principal roles and responsibilities ............................................................................................. 57 3.10.4. Applicable tooling ....................................................................................................................... 57
3.11. Involvement and co-ordination in external and international affair matters .............. 58 3.11.1 Objective ..................................................................................................................................... 58 3.11.2 Approach ..................................................................................................................................... 58 3.11.3 Principal roles and responsibilities ............................................................................................. 59
4. Project participation to content development to design of SESAR Solutions in function of Master Plan execution ................................................................................................ 61
4.1 Introduction ................................................................................................................ 61 4.1.1. Scope of general practices .......................................................................................................... 61 4.1.2. Some principal roles and responsibilities ................................................................................... 61 4.1.3. Purpose and role of Content Integration related assignments .................................................. 63 4.1.4. Community of Practices .............................................................................................................. 63
4.2 Concept development .................................................................................................. 65 4.2.1 Objective ......................................................................................................................................... 65 4.2.2 Overall approach ............................................................................................................................. 65 4.2.3 Reference material .......................................................................................................................... 67
5. Project participation to Performance assessment, cost benefit analysis and business cases in support to Master Plan execution ....................................................................... 78
5.1. Performance assessment overview .............................................................................. 78 5.1.1. Objective ..................................................................................................................................... 78 5.1.2. Applicability ................................................................................................................................ 78 5.1.3. Introduction to overall approach ................................................................................................ 78 5.1.4. Principal roles and responsibilities ............................................................................................. 83 5.1.5. Applicable Framework ................................................................................................................ 83 5.1.6. Reference material ..................................................................................................................... 84
The Project Handbook (also called Programme Execution Guidance) is a set of tightly coordinated processes that enable project management in the context of the global programme steering.
In the context of the SESAR R&D pipeline, the Project Handbook addresses the SESAR Solution development life cycle, the various processes to support the programme execution (such as Risk management, Deliverable assessment, Solution maturity gate or Release Management), the participation to the content development and the Performance assessment.
This updated edition (v2.1) is the result of the continuous evolution and maturity of the Programme Execution prepared by work done in SESAR 1 and gradually matured and streamlined during S2020 ER 1, 2 and 3 and Wave 1.
In this updated edition special attention has been taken for:
- The streamline of the project initiation thanks to consolidation of the Call and Gap activities;
- The simplification of the reporting activities while covering the necessary programme steering and reporting needs;
- The integration of the ER activity in the Release process.
This document is entitled ‘Project Handbook (Programme Execution Guidance)’ and provides guidance explaining how SESAR 2020 projects should be executed within the framework of the SESAR 2020 Research and Innovation (R&I) programme.
While the SESAR 2020 Programme Management Plan provides a high level overview of the management approach, the Project Handbook outlines for each applicable process its objectives, principles and steps to be taken.
The initial Project Handbook was developed by SJU in collaboration with mainly SESAR 1 transversal projects and reviewed in an iterative manner at DMSC level; it has then been further refined considering the lessons learnt during management of a first set of projects. Comments and feedback received steered the need to create the second version to adjust to the reality and the new needs of the programme. This update opens some processes to ER projects to secure the R&D pipeline (i.e. from Exploratory Research up to Very Large Scale Demonstration).
1.2 Handbook objectives
The main objective of the Handbook is to provide a single entry point into the SESAR 2020 development framework for any SESAR 2020 project. In doing so the Handbook aims to give an overview of what is expected from the Project Manager and his/her team when defining, validating and reporting upon the development of SESAR deliverables (such as for Solution/Demonstration, Transversal or Exploratory Research projects) in the context of SESAR 2020. The aim is to ensure an agreed and common approach across all projects to deliver the vision of the ATM MP, allowing a uniform way of monitoring project execution as part of the overall programme, as well as supporting programme-level decision making, while facilitating and securing the R&D pipeline.
To achieve these objectives the document is structured as follows:
It gives an overview and explains the details of the SESAR life cycle and the practices to validate SESAR Solutions in relation to pre-defined and standardised maturity levels (chapter 2).
It describes practices that projects should apply, specifically: - The project management practices that are in compliance with SESAR 2020 programme
management needs (chapter 3)1. - The practices that contribute to the identification of requirements and the design of SESAR
Solutions (chapter 4). - The practices that contribute to the performance assessment of SESAR Solutions, cost
benefit analysis and business cases (chapter 5).
1 It should be noted that this Handbook complements the Horizon 2020 Online Manual, which explains the basic principles and obligations for executing projects (actions) funded under the Horizon 2020 framework of the European Commission.
In addition to this Handbook, a series of SESAR 2020 document templates are available. Also for each practice or method, references to detailed and complementary guidance are included throughout this document.
1.3 Applicability
The practices outlined in this document are applicable to all projects of the SESAR 2020 programme. Any foreseen deviation of those practices will have to be documented in the project management plan (PMP) which is one of the first deliverables of the project after the kick off of the project. Any deviations to the application of practices should be agreed by the SJU.2
Since it is the purpose of the SESAR development life cycle (introduced in next chapter) to sequence activities in order to develop, validate and progressively increase SESAR Solution maturity, the Handbook is applicable to projects outside the scope of IR projects in the following manner:
Very large-scale demonstration projects (VLD):
Project management processes are executed in a similar way as for IR projects, with the exception that VLDs are subject to contractual deliverables only (see further);
Project use, base their activities, give feedback and report against the information gathered as part of the design of SESAR Solutions as documented in chapter 4;
The demonstration report is the main input for a maturity gate at the end of the project; it shall report on the Performance of the demonstration (e.g. solution(s)) in accordance with the scope and methods outlined in chapter 5.
Exploratory research (ER) projects:
The ER project management processes are compliant with SESAR Programme Management and Grant Requirements as presented in chapter 3. However, since ER projects are generally smaller than IR projects, ER projects may only require alignment with programme lifecycle milestones (such as the Extended Release Strategy update and Release Process milestones as presented in sections 2.2 and 2.6).
At the end of each ER project a self-maturity assessment is made and embedded in the project closure report (see section 3.13). For ER projects the self-maturity assessment uses the common SESAR maturity framework with the aim to assess whether the initial concept description provided can be continued as industrial research towards the next level i.e. V1 maturity (see sections 2.7.4.1 and 2.7.4.2).
Practices in the domains of content integration, performance management, architecture, concept development and system engineering including requirements data management are recommended best practices (see chapter 5) which applicability for application oriented ER project only may be tuned during the preparation of the Grant (e.g. GAP phase).
Projects use generic SESAR 2020 document and presentation templates and communication guidelines and may use SESAR 2020 specific document type templates when appropriate.
2 The Handbook does not supersede the contractual baseline (i.e. the project grant) but it complements it in further defining the way the projects have to operate under the S2020 Programme Execution Framework.
The Handbook is part of a layered documentation structure that describes the overall framework to develop and validate SESAR Solutions.
Figure 1 : SESAR 2020 documentation hierarchy
Each practice introduced in this Handbook is not worked out in detail in the Handbook itself. This is done in detailed specialised guidance (so called third-level documentation available in the SJU Collaboration Platform), which is to be used by the domain experts to understand and comply with the way of working. For example safety practices are detailed in the ‘Safety reference material - edition 4’.
All third-level level documentation, programme content deliverables and other reference material are available in the programme library of the SESAR 2020 collaborative platform.
1.5 Concepts, terms and definitions
The following is a list of the concepts, terms or definitions introduced or most commonly referred to in this Handbook:
ATM Master Plan [ATM MP]: The European ATM Master Plan defines the agreed roadmap to bring ATM Research & Innovation to the deployment, supporting the agreed vision for the future European ATM system. It provides the main direction and principles for SESAR 2020 R&I programme, as well as the deployment planning and an implementation view with agreed deployment objectives. Through the SESAR Key Features, the ATM MP identifies the Essential Operational Changes and key R&D activities that support the identified performance ambition. The ATM MP is updated on a regular basis in collaboration and consultation with the entire ATM community. Amendments are submitted to the SJU Administrative Board for adoption.
Concept of Operations [ConOps]: The ConOps is jointly elaborated by all ATM stakeholders, from the civil and military airspace users and service providers, to airports and the manufacturing industry to gain common understanding of the ATM system. It describes the operational targets, to move ATM towards trajectory-based operations whereby aircraft can fly their preferred trajectories, taking into account the matching between constraints and optimization. The ConOps allows all ATM stakeholders, from the civil and military airspace users and service providers, to airports and the manufacturing industry to gain common understanding of the ATM system. In this context, the ConOps is the operational answer to reach the ATM Performance improvements targeted by the ATM MP. Furthermore the ConOps is an important reference for global interoperability and harmonization, as it has been adapted for Europe from the ICAO Global Air Traffic Management Operational Concept.
Contractual deliverable: The material that is submitted by a project to the SJU via the Horizon 2020 portal (Funding and tender Portal) to meet the output of the action as agreed in the Grant Agreement (GA). The main contractual deliverables for IR projects are the SESAR Solution data packs while for VLD projects are the Demonstration Reports.
Exploratory Research: The exploratory research investigates relevant scientific subjects (during the ATM Excellent Science & Outreach phase) and conducts feasibility studies looking for potential application areas in ATM (during the ATM application-oriented research phase).
Horizon 2020 call for proposals: An invitation to submit proposals for one or more topics in the context of the EU’s Horizon 2020 SESAR 2020 R&I programme. The complete SJU work programme is be composed of projects that are organised through a number of different calls, each covering a different part of the programme.
Horizon 2020 Funding and tender Portal: The entry point for electronic administration of EU-funded research and innovation projects, managed by the European Commission and which supports the publication of calls, the submission of proposals, their evaluation, the preparation of grant agreements and the execution of grants.
Horizon 2020 process: A process that is defined by the European Commission services and that applies to Grants executed under Horizon 2020 rules.
Integrated validation: An integrated validation aims to go beyond the boundaries of one solution/project and validate solutions delivered within one project or by different projects.
Key Feature: These are the pillars of SESAR 2020 R&I programme, which are detailed in the ATM Master Plan and which describe the strategic orientation of SESAR 2020 vision.
Maturity level: The level of maturity achieved by a SESAR Solution is defined in accordance with the maturity phases (V-levels) of the European operational concept validation methodology (E-OCVM). These are the further mapped to the technology readiness level scheme (TRL), details of which can be found in the Appendix part of this document.
Performance ambitions: The performance ambition supported by SESAR is aspirational and refers to the performance capability that may be achieved if SESAR Solutions are made available through R&D activities, deployed in a timely and, when needed, synchronised way and used to their full potential.
Performance assessment: The assessment of solution performance based on validation results obtained during the solution development life cycle.
PMP deliverable: output produced by the projects that is submitted to the SJU via the SESAR 2020 collaborative platform and that is subject to quality assessment by the SJU. However, these deliverables do not appear in the grant agreement as contractual deliverables. The production of PMP deliverables is done in support of subsequent contractual deliverables and is described in the PMP.
Programme content integration activities: a one-of-a-kind activity organised at programme level aiming to coordinate, consolidate and integrate SESAR ATM and Technological Solutions, and as such to support and guide the processes to ensure their completeness, consistency and coherency from a holistic perspective as expressed in the SESAR ConOps. The content integration activities also cover the maintenance and support of the performance framework, as well as architecture development and maintenance activities and ensuring the applicability of the practices by the projects.
Programme snapshot: a SJU picture of the Programme Integrated Schedule available in the S2020 collaborative platform (updated by projects on a regular basis when changes occur) that presents the status of the progress of all projects at programme level. It will be used to analyse the status of the programme and propose steering with the governance bodies (e.g. DMSC, PC).
Project Management Plan: Formal, approved document, provided by each SESAR 2020 Solution Project, used to manage its execution. It defines how the project is executed, monitored, controlled, and closed.
Release: is the yearly delivery process of the SJU. It covers the related activities that aim to validate the SESAR Solutions (at any maturity level) and that require members’ commitment.
SESAR Solution: A term used when referring to both SESAR ATM Solution and SESAR Technological Solution. Programme output is defined and packed in the form of SESAR Solutions.
- SESAR ATM Solution: These contain outputs from R&D activities which relate to either an operational improvement (OI) step or group of OI steps and associated enablers which have been designed, developed and validated in response to validation targets that when implemented, will deliver performance improvements to European ATM.
- SESAR Technological Solution: New technology that enables future SESAR ATM Solutions, verified as feasible, safe and to support or enable ATM Performance Improvements.
SESAR Solution project: A term introduced to refer to a project that delivers one or more SESAR Solutions and that is either a SESAR Enabling project or a SESAR ATM Solution project
- SESAR Enabling project: A project delivering SESAR Technological Solutions is known as an ‘enabling project’.
SESAR Solution data pack: is provided as contractual deliverable at the end of the activities related to each of the maturity phases under the scope of SESAR.
In the case of SESAR ATM Solution, the typical data pack contains:
o Safety and Performance Requirements - INTEROPerability requirements (SPR3-INTEROP)/ Operational Service and Environment Definition (OSED);
o Technical specification (TS)/Interface requirements specification (IRS) (when relevant, this includes documentation (SDD) of the services which are enabled by the SESAR Solution or common services); the TS/IRS is not required for V1;
o Cost benefit analysis (CBA) (in V1 this is just an initial qualitative analysis); o Validation report (VALR); in some cases, VALR is not required for V1.
Solution data packs are used as an input for the next maturity phases or, in case of V3, for industrialisation and deployment (e.g. it may include material that will form the basis for standardisation assuming those needs are identified during the Solution’s development).
In the case of SESAR Technological Solutions, the typical data pack contains: o Functional requirements definition (FRD) for TRL2, final TS/IRS for TRL4 and TRL6; o A Cost benefit analysis (CBAT) (i.e. CBA tailored to the specifics of the Technological
Solution and in TRL2 this is limited to an initial qualitative analysis); o TVALR.
For VLD project, there is no SESAR solution data pack as such but the Demonstration Report and the Availability Note could be considered as the equivalent of the pack. For ER project, there is in general no SESAR solution data pack, unless when the project is an ATM Application Oriented Research (see section 2.7.4.2).
SESAR 2020 programme execution framework: a programme wide system consisting of defined life cycle, processes, responsibilities (people) supported by a series of tools with the aim to institutionalise the approach and general way of working.
Single European Sky High Level Goals: The SES High Level Goals are political targets set by the European Commission. Their scope is the full ATM performance outcome resulting from the combined implementation of the SES pillars and instruments, as well as industry developments not driven directly by the EU.
SJU’s collaboration platform: a tool to enable the SJU processes described in this document in order to steer and manage the entire programme; the platform provides general collaboration capabilities for individual project teams (i.e. Stellar).
SJU process: Processes defined by the SJU to support the management of the programme for areas not covered by H2020 processes, and enabled by SEAR 2020 collaborative platform.
Standalone contractual deliverable: a deliverable specified in the grant agreement that is not part of a Solution data pack.
Transversal projects: projects that support the SJU: - To apply top-down approach to facilitate the strategic steering of the SESAR project as a
whole in line with Policy priorities. The focus of this project is to deliver and report against the three levels of the Master Plan
3 In case industrial partners have multiple SPRs for one solution then these will all be included as sub-SPRs within the same SPR document.
- To support Programme execution and solution projects developments for delivering the SESAR Solutions in line with the ATM Master Plan. The focus of this project is to coordinate and integrate operational and technological solution content (architecture, system engineering) and as such will support and guide the execution of the transversal processes (e.g. safety, security assessment, CBA, etc.). This to ensure their completeness, consistency and coherency from a holistic perspective.
Validation strategy: The validation strategy provides projects a view to fulfil their objectives including validation performance targets by ensuring that there is a common understanding of the set of shared principles and practices, which are to be used to structure and organise activities throughout concept development and validation to progress the maturity of the concept through to industrialization and deployment. It includes (although not necessarily in detail) the overall R&D needs and high level validation objectives per Key Feature/SESAR Solution, priorities at programme level, a list of general assumptions about concept, scope and context, shared processes, tools, reference material or criteria, etc.
Validation target: Validation targets are the goals that focus the development of enhanced capabilities by Solution Projects. They aim to guide projects in achieving the required performance capabilities in order to contribute towards the reaching the performance ambitions outline in the ATM MP.
Very large-scale demonstration project (VLD): a project set-up to pave the way to deployment with the aim to de-risk the start date of the full-scale deployment with the aim:
to generate further confidence of and support buy-in from the main ATM stakeholders including airspace users, airports and regulators, ahead of the full-scale deployment of targeted SESAR Solutions;
to significantly reduce the business risks for both operational stakeholders and industry, in particular for SESAR Solutions requiring coordination at European/global level included in Common Projects, and to provide further inputs to related standardisation activities;
to raise awareness regarding SESAR activities related to ATM performance and their results;
to establish SESAR labelled end-user demonstration platforms and infrastructure;
To assess readiness for full-scale deployment.
1.6 Change and configuration management of reference material
All documents (guidance, templates, etc.) listed in the Handbook are subject to SESAR 2020 change management under SJU configuration control. The aim is to create a stable working environment, while amendments and improvements are possible but in controlled and agreed manner.
1.7 Acronyms
ADD Architecture Description Document
OSED Operational Service and Environment Definition
AIRM ATM Information Reference Model
PAR Performance Assessment Report
AO Application Oriented R&I Research and Innovation
ATM Air Traffic Management R&D Research and Development
The work undertaken in the programme is guided by two interwoven life cycles:
Figure 2 : SESAR 2020 interwoven life cycles
Programme and release delivery life cycle
Enable the coordination of content integration output on a continuously iterative basis, using defined milestones on work performed on SESAR Solutions, supporting development and validation activities undertaken by SESAR Solution projects;
Enable a release delivery mechanism, including validation roadmap maintenance processes aimed at ensuring the delivery of SESAR Solutions;
Enable coordination between SESAR Solution and transversal projects via an annual sequence of programme milestones that make available key programme output including information for decision making.
Solution development life cycle
This lifecycle takes place at project level and aims to progressively increase the maturity of SESAR Solutions, with the final objective of delivering a SESAR Solution data packs for industrialisation and deployment. The Solution development life cycle is a continuous process aligned to the Programme and Release delivery life cycle, meaning that when planning internal activities Solution projects have to take account its milestones.
The programme and release delivery life cycle aims at ensuring consistency and coherency of the content developed under the SESAR solutions, while steering the R&D work to achieve the ATM Master Plan vision and performance ambitions and the maturity targets documented in the Extended Release Strategy.
The update of the extended release strategy and the update of the ATM Master Plan trigger the update of the programme dataset. The programme dataset:
Represents the reference for building the key programme and release delivery life cycle deliverables including CONOPS, ADD, Release Plan and Report, etc.;
Constitutes the starting point to consolidate yearly performance results and business cases;
Represents an opportunity to re-align the content integration layer with the solution development and validation layer e.g. baselining the changes issued during maturity gates, perform consistency and coherency checks, and identify potential significant impacts on the ATM MP level 1.
2.3 Solution development life cycle – Overall view
The SESAR Solution development life cycle is a process executed at project level which includes a standard sequence of activities to develop, validate and progressively increase SESAR Solution maturity from exploratory research, to industrial research and very large demonstrations. Its final objective is to deliver a SESAR Solution for Industrialisation and deployment.
Figure 3 : R&D pipeline
The SESAR Solution development life cycle can be broken down in three major sub-activities (detailed in the remaining part of this chapter) dealing respectively with:
The development and validation of SESAR Solutions, which is the process that plans and executes the development activities and validation exercises that are required to ensure SESAR Solutions progress from one maturity level to the next;
The maturity assessment, which establishes the maturity achieved by the Solution;
The contribution to the release process, which ensures the yearly planning (release plan) and delivery of the V3/TRL6 mature SESAR Solutions and successful VLDs (release close-out report). It is important to note that the scope of a release is extended to those solutions that have achieved V1 and V2, and the demonstration activities that were performed in the release timeframe.
The SESAR Solution development life cycle is supported by a set of documentation called SESAR deliverables. These deliverables, that are progressively matured and consolidated through the successive V/TRL phases, describe the Solutions (e.g. operational requirements, safety and performance, technical requirements, standardisation needs, validation objectives and results) together with the associated cost-benefit analysis so that, at the end of the solution life cycle process, a final V3/TRL6 solution data pack can be delivered for further industrialisation and deployment.
2.4 Development and validation4
The SESAR Solution development and validation process defines and executes appropriate validation activities and assessments to prove that a SESAR Solution is operationally/technically feasible, applicable to relevant sub operating environments and capable of meeting expected performance benefits as defined by established validation targets.
To validate SESAR Solutions, projects/solutions should follow the guidance and conform to the framework provided by the validation strategy (VALS). The validation strategy enables the SESAR validation programme and SESAR projects/solutions to fulfil their objectives by ensuring that there is a common understanding of a set of shared principles and practices. These are used to structure and organise activities throughout the concept development and validation phases in order to progress the maturity of SESAR Solutions towards industrialisation and deployment.
It includes (although not necessarily in detail): the overall R&D needs and high level validation objectives per SESAR Solution, initial set of integrated validations needs and a list of general assumptions about concept, scope and context.
The VALS provides limited guidance for the SESAR Technological solutions: additional references are the SESAR Concept of Operations and the ATM Master Plan e.g. performance needs.
SESAR Solution projects are responsible for planning and executing the required and sufficient number of exercises and assessments to ensure a given SESAR Solution can progress from the initial maturity level towards the target maturity level in compliance with the corresponding Maturity Criteria. This is captured for each V (or TRL) phase in the SESAR Solution VALP (or TVALP). At the end of the V (or TRL) phase activities, SESAR Solution projects shall provide a VALR (or TVALR) where results are consolidated, capturing the relevant conclusions and recommendations.
After each validation cycle, the SESAR Solution projects, shall update the corresponding SPR-INTEROP/OSED and TS/IRS (supported by relevant information in EATMA) considering the validation results. Validation and assessments results shall also support the consolidation of the solution CBA. The result of these updates will then be consolidated by the Content Integration Project. In particular this will also allow identifying possible gaps in the VALS of the operational concept.
Development and validation deliverables include some system engineering (SE) structured information, such as operational requirements, performance and safety, system requirements, validation objectives and results. Solution projects shall manage this information ensuring a proper change management across successive maturity levels and the connection between requirements and architectural elements in EATMA.
A validation exercise executed by a project may address several SESAR Solutions, potentially including some which are under the responsibility of other projects. These are known as ‘integrated validations’, and cover the integrated validation needs identified at VALS level. These integrated validation activities, although led by one project/solution, also contribute to finalising the validation work of other projects/solutions.
Demonstration activities, executed by projects in the VLD domain, are in some cases justified in order to bridge the gap between pre-industrial development and validation, and industrialisation and deployment. They correspond to demonstrations using early versions of end-user systems in order to confirm deployment readiness of the targeted solution at a larger scale. When they take place, these activities ensure a step beyond V3/TRL6 maturity and they build on top of SESAR Solution(s) and their corresponding Solution data packs that have already been delivered. VLDs also represent an opportunity to perform integrated validation for a number of SESAR solutions that have been validated individually till V3/TRL6 as part of Industrial Research.
The development of the V&VI required to validate/verify the SESAR ATM Solutions, Technological Solutions and to perform demonstrations is under the responsibility of the SESAR Solution and demonstration projects (VLDs). The corresponding detailed documentation remains internal to projects and is not handed over to the SJU. Projects are only required to deliver to the SJU an availability note that documents the verification results of prototypes and V&VI, along with their integration to build the V&VP and their potential impacts on the validation objectives.
The “V&V platforms, V&VIs and Demonstration Platform Development Methodology” contains a collection of best practices on this domain as guidance for the SESAR projects to develop the required platforms for the validation. The application of the methodology aims to ensure the maintenance of the V&V Platforms and infrastructures, their integration with the system prototypes and V&V tools and the rest of the preparation activities needed for successful execution of a validation exercise.
2.5 Maturity assessment
At the initiation phase each project in SESAR 2020 shall perform an initial self-maturity assessment of each SESAR Solution. This initial self-assessment will constitute the starting point for the work at solution level, and shall either confirm or amend the maturity of the SESAR Solution as captured in the latest edition of the SESAR programme maturity report if the solution was addressed before. An update of the programme maturity baseline will result from this activity.
There are six maturity gates in the SESAR Solution development life cycle. These are decision points to assess achieved results and authorise further development and validation activities along the life cycle. Each gate is based on a set of SESAR maturity criteria that are further defined in the SESAR 2020 maturity assessment guidance documentation. A maturity gate doesn’t necessarily rely on the execution of a maturity gate meeting. For example, the Fundamental Oriented research (FO)/Applications Oriented research (AO) and the Application Oriented (AO)/Industrial Research (IR) gates are mainly performed through the assessment of the maturity criteria with their status
reported in the Final Project Report and during the project closure meeting. The six maturity gates are as follows:
Gate Fundamental Oriented research (FO)/Applications Oriented research (AO) - This gate verifies whether TRL1 maturity has been achieved, and if the topics under fundamental research can be applied to a concrete application to ATM. These are the required conditions to allow for the transition towards application oriented research (still in ER).
Gate AO/IR - This gate verifies whether V1 (TRL2) maturity has been achieved, and the potential of the Solution to deliver ATM benefits. These are the required conditions to allow for the transition from ER (applications oriented) to IR (phase V2 – TRL4), or to a complementary/extension of V1 (TRL2) validation, taking place within the IR part of the programme.
Gate V1 (TRL2) - V1 (TRL2) phase is focused on the SESAR Solution definition aspects. This gate has two objectives: (i) to verify where a full V1 (TRL2) maturity has been achieved and (ii) to accept the plan for future V2 validation activities. A successful gate authorises the transition to phase V2. In case the Gate ER/IR has concluded a full V1 maturity, the gate V1 can be skipped.
Gate V2 (TRL4) – V2 (TRL4) phase is focused on ensuring the SESAR Solution feasibility. This gate has two objectives: (i) to verify that a full V2 maturity has been achieved including the potential to deliver ATM benefits once mature and deployed and, (ii) to accept the plan for future V3 validation activities. A successful gate authorises the transition to phase V3o.
Gate V3 (TRL6) – V3 (TRL6) phase is focused on SESAR Solution pre-operational validation. This gate verifies that a full V3 (TRL6) maturity has been achieved. A successful gate authorises the transition to industrialisation and deployment. The V3 / TRL6 gate will also assess the necessity and/or convenience to execute a VLD after V3, in order to secure a successful transition to industrialisation and deployment.
Gate DEMO (TRL7) - When demonstration activities are justified this gate is used to verify the VLD contribution toward industrialization and deployment and the successful achievement of the demonstration objectives (TRL7).
Projects should plan SESAR Solution maturity gates in accordance with the progress of the solution and in line with the milestones identified in the grant agreement. When planning their activities, projects/solutions should consider the programme milestone calendar in order to deliver in due time the required data packs and to schedule the maturity gate so that the resulting SESAR Solutions are available by the release close-out.
Maturity assessments are based on SESAR maturity criteria, which are part of the programme reference material, and are supported by a maturity assessment tool (MAT) provided by the SJU. The information related to individual SESAR Solution maturity assessments is collected, baselined and stored by SJU. A consolidated maturity report is generated by the SJU on an annual basis.
Further details on the execution process of the maturity gates can be found in section 3.6.
The release process ensures the yearly delivery of SESAR Solutions and of successful VLDs results. It consists of three phases: definition, execution and delivery.
Release milestones do not impose deadlines to projects. Validation activities and maturity Gate are based on project’s lifecycle. Release milestones are opportunities for a project to contribute to a specific Release.
Figure 5 provides an overview of the phases of the release process, which are detailed in the following sections.
Figure 5 : Overview of the release process phases
2.6.1. Release definition
The objective of this phase, lasting from May to December of Year N-1, is to identify the list of SESAR ATM and Technological Solutions which, according to project planned activities, are expected to successfully pass their V3/TRL6 or DEMO gates between May of Year N and April of Year N+1 (both inclusive) and therefore will be ready for industrialisation.
Upon agreement at programme governance level, and with the aim to promote other than V3/TRL6 activities, in order to show SESAR Programme deliveries in progress (Solutions not yet ready of industrialisation), and to feed SJU and SESAR Annual Report and SPD, the release definition phase has been extended to cater for the execution of ER projects (with FO/AO, AO/IR maturity steps) as well as V1 or TRL2 and V2 orTRL4 solutions maturity, and the execution of VLDs (with V4 or TRL7).
During the release definition phase, the iterative update of validation and demonstration plans will help identifying the SESAR Solutions that will be finally considered as part of Release.
An initial list of maturity gate are already available in the project proposals and this already allows to have an initial release plan.
These dates can be changed during the SESAR solution development and validation phase. The release plan is iteratively updated based upon the latest available information from the ER and solution projects. Relevant demonstration activities will be considered in a similar way.
The release plan also includes the list of validation activities / assessments planned per solution to achieve a given maturity level. These activities are extracted from the V&V Roadmap that consists of a synthetic view of all Programme validation and demonstration activities (as available in the projects schedule) and is a key input to define the content of the release plan.
Once delivered, the release plan is endorsed by the relevant SJU governance body and provides the reference for the following phase: the release execution.
Figure 6 : Inputs contributing to the release plan
2.6.2. Release execution
Projects complete the validation activities/ maturity assessments leading to the maturity gate of the SESAR Solutions identified in the release plan. This typically includes:
Running the scheduled validation and verification exercises and demonstrations;
Analysing and consolidating results, producing validation reports and updating projects’ documents accordingly (e.g. SPR-INTEROP/OSED and TS/IRS);
Organising and participating in SESAR Solution maturity gates.
Contributing to the review of the maturity gate data packs (e.g. Content integration project).
2.6.3. Release delivery
As soon as possible, but by April of Year N+1 at the latest, SESAR solution maturity will have been confirmed through the Maturity Gate and the (updated if needed) Solution Data Pack will be ready for consideration in the release close-out report. Similarly, demo results will have been confirmed through the DEMO gate and results made available in DEMO report.
In June of Year N+1, the Release N close-out report is delivered by the SJU. It presents validated or demonstrated SESAR Solutions during the release execution phase and confirmed during the relevant gates. The close-out report also provides a summary of results from all the activities identified in the Release N Plan.
Figure 7 : Inputs contributing to the close-out report
At the same time release results are used at the content integration level to consolidate performance and maturity at programme level, updating the relevant performance and maturity reports issued respectively by the maturity and performance consolidation activities (please refer to section 3.6 and section 5.1).
Release results contribute to feed the annual gap analysis against the original performance and maturity targets in the extended release strategy, updating the performance and maturity baselines (i.e. from Exploratory Research up to Very Large Scale Demonstration).
2.7 SESAR solution deliverables
2.7.1. General
The development of a SESAR Solution is based on the production of one or more deliverables:
Short title Title
AN Availability note
CBA Cost benefit analysis report
CBAT Cost benefit analysis report for Technological Solutions
Concept Outline High level description of the concept with identification of potential benefits
EVALR Exploratory Validation report capturing the results of the validation activities
FRD Functional requirements definition
Initial Concept Description Consolidation of the Concept outline towards a SESAR Solution
SPR-INTEROP/OSED7 Safety and performance requirements/Interoperability requirements/ Operational service and environment definition. As appendix, it includes the transversal assessments and the performance assessment report.
These deliverables can be grouped per type of project in the following manner:
5 Can also be called Study Plan under specific contract framework (like CEF). 6 Can also be called Study Report under specific contract framework (like CEF). 7 In case industrial partners have multiple SPRs contributions for one solution then these will all be included as sub-SPRs within the same SPR document.
Deliverables from an earlier maturity phase may be used as input into subsequent phases.
For more details on which deliverables are associated to which maturity level, please refer to sections 2.7.4 till 2.7.11.
2.7.2. KPA plans and detailed reports
Certain methodologies (e.g. safety, human performance) use dedicated templates to plan and/or to report on detailed work related to each specific area. For example the safety methodology comes with a SESAR Solution safety assessment plan template and safety assessment report template, both of which are listed as reference material in this document and are like any other reference material subject to SESAR 2020 programme change management. The documents themselves have been already integrated in other SESAR templates e.g. safety assessment plan is part of the VALP.
To avoid duplication of effort and information, these specialised templates in no way master or repeat information that is already hosted in project deliverables i.e. those listed in the above table. For example the safety plan should not duplicate the description of the SESAR Solution as documented in the SPR-INTEROP/OSED.
2.7.3. Project internal deliverables
Internal deliverables will be produced as needed in the context of project activities but will not be formally handed over to the SJU and will not go through the deliverable quality assessment process. Therefore they will not be considered at programme level to monitor the progress of a SESAR Solution through the various maturity phases. This is the case, for instance, for technical documents related to the development and verification of V&V infrastructures and platforms.
2.7.4. SESAR Solution deliverables per detailed maturity phase (General principles)
In the following sub-sections we outline the deliverables that are required to be produced as per SESAR Solution and as per targeted maturity level. We distinguish between SESAR ATM Solutions and Technological Solutions which may encompass ER phase, and there is a specific section for VLDs. Please note that the following sections describe ‘normal’ situations, but there may be exceptions which require adapting these patterns to specific needs.
Each phase of the life cycle is decomposed into a standard sequence of activities associated to formal deliverables. Key points in the activity sequence are identified by Milestones. When a phase of a SESAR Solution development life cycle is planned, its milestones are assigned specific planning dates, providing visibility at programme level on the expected evolution of development and validation tasks.
As a general principle, deliverables produced at the end of a maturity phase are the main inputs for the following one. Operational and technical deliverables finalised at a maturity gate are the initial baseline to start development activities in the next maturity phase. The initial VALP/TVALP presented at a maturity gate (or at the beginning of the project) describes the approach to the validation of the next maturity level and provides the detailed exercise plan of the first exercise at minimum; further update of the VALP/TVALP shall be provided every time an exercise plan is available.
Relying on past validation activities, projects may start working in the SESAR 2020 Programme on SESAR Solutions having already previously achieved FO/AO, AO/IR, V1/TRL2 or V2/TRL4 maturity status. In such case, the results and deliverables from these previous validation activities are expected to be available as an input to the maturity cycle of the SESAR 2020 Solution.
2.7.4.1. Exploratory Research initial phase
A development and validation phase related to Exploratory Fundamental Oriented Research conducted up to ER initial maturity can be broken down in the following activities and is completed via a successful assessment of the following deliverables before Gate FO/AO:
Concept outline and identification of potential benefits and risks
ER Initial Delivery
Gate FO/AO
Figure 9 Initial Exploratory phase
2.7.4.2. Exploratory Research final phase
A development and validation phase related to Exploratory Applied Research conducted up to ER final maturity can be broken down in the following activities and is completed via a successful assessment of the following deliverables before Gate AO/IR:
A development and validation phase related to SESAR ATM Solutions conducted up to V1 maturity can be broken down in the following activities and is completed via a successful assessment of the following deliverables before Gate V1:
Activity Deliverable Milestone
ATM Excellent Science and Outreach
Concept outline and identification of potential benefits and risks
Initial Validation Plan (VALP) V2 defining the validation roadmap for phase V2
Initial CBA (when applicable)
Gate V1
Figure 11 : V1 validation phase for SESAR ATM Solutions
Note that the Gates FO/AO and AO/IR do not follow the full maturity gate process described in section 3.5 here after as these Gates will be performed with a self-maturity assessment provided by the project and a review/assessment done by SJU.
2.7.4.4. V2 phase for SESAR ATM Solutions
A development and validation phase related to SESAR ATM Solutions conducted up to V2 maturity can be broken down in the following activities and is completed via a successful assessment of the following deliverables before gate V2:
Initial Validation Plan (VALP) V3 defining the validation roadmap for phase V3
Requirements consolidation
Safety, Performance and Operational Requirements (SPR-INTEROP/OSED) V2
Technical Specification (TS/IRS) V2
(when applicable)
Gate V2
Figure 12 : V2 validation phase for SESAR ATM Solutions
2.7.4.5. V3 phase for SESAR ATM Solutions
A development and validation phase related to SESAR ATM Solutions conducted up to V3 maturity can be broken down in the following activities and is completed via a successful assessment of the following deliverables before gate V3:
Activity Deliverable Milestone
Validation Objectives definition Validation Plan (VALP) V3 OPS
System Requirements definition Initial Technical Requirements (TS/IRS) V3
Different documents may be produced for different systems
SYS
V3 Platform development Availability Note (AN) EXE START
Exercise EXE END
Validation Results consolidation Validation Report (VALR) V3 V3 Final Delivery
Requirements consolidation Safety, Performance and Operational Requirements (SPR-INTEROP/OSED)
V3
Technical Requirements (TS/IRS) V3
Gate V3
Figure 13 : V3 validation phase for SESAR ATM Solution
2.7.4.6. TRL2 phase for SESAR Technological Solutions
The development and technical validation phase related to Technological Solutions conducted up to TRL2 maturity can be broken down in the following activities and is completed via a successful assessment of the following deliverables before gates FO/AO and gate AO/IR and gate TRL2:
Activity Deliverable Milestone
ATM Excellent Science and Outreach
Concept outline and identification of potential benefits and risks
ER Initial Delivery
Gate FO/AO
ATM Application Oriented Research
Initial Concept Description (EFDD) ER Final Delivery
2.7.4.7. TRL4 phase for SESAR Technological Solutions
A development and technical validation phase related to Technological Solutions conducted up to TRL4 maturity can be broken down in the following activities and is completed via a successful assessment of the following deliverables before gate TRL4:
Activity Deliverable Milestone
Initial System Requirements definition
Initial Technical Specification (TS/IRS) SYS
Technical Validation Objectives definition
Technical Validation Plan (TVALP)
TRL4 Platform development
Availability Note (AN) EXE START
Exercise EXE END
Technical Validation Results consolidation
Technical Validation Report (TVALR) TRL4 V2(TRL4) Final Delivery
(solution datapack) Cost Benefit Analysis tailored for the specific Technological Solution (CBAT)
2.7.4.8. TRL6 phase for SESAR Technological Solutions
A development and technical validation phase related to Technological Solutions conducted up to TRL6 maturity can be broken down in the following activities and is completed via a successful assessment of the following deliverables before gate TRL6:
2.7.4.9. Demonstration phase for SESAR ATM and Technological Solution
A demonstration project can be broken down in the following activities and is completed via a successful assessment of the following deliverables before a demo’s final gate:
3. Project Management in relation to Programme Management and Grant Requirements
N.B .as mentioned in section 1.3 before, the practices outlined in this chapter are applicable to all projects unless mentioned explicitly.
3.1. Background
For all SESAR 2020 projects, the legal base is the grant agreement, signed with the SJU; they consequently need to apply the Grant processes, either Horizon 2020 or SJU procurement.
For Horizon 2020 grant the European Commission has put in place a series of processes supported by the Horizon 2020 Funding and tender Portal as established in the Grant Agreement, where applicable, they are cross referenced appropriately. However, the very nature of the SESAR programme imposes additional requirements than the ones only supported by the H2020 tools, which is why additional common programme management processes have been put in place.
Having commonality in the way SESAR Solutions are developed and in the way projects are managed enables SESAR staff to participate in multiple projects knowing that the same processes will be applied. The same applies to the management oversight of the whole programme, which can obtain the same type of information from each project.
Furthermore, projects research and develop SESAR Solutions that eventually all need to be integrated in the same overall ATM-system-of-systems as defined in the ATM MP. When these projects have dependencies between each other those dependencies need must be made visible, monitored, managed and synchronised in a common way.
Each of the programme management-related sections below, explain the extent to which each process is following Horizon 2020 processes and what may be SESAR 2020 specific.
3.2. Programme and project initiation
3.2.1. Objective
The initiation of the programme and the individual projects is an activity which is specific to SESAR 2020. While the standard requirement in the Grant is that each project starts on its own the first day after the Grant signature; however given the interrelated programmatic nature of SESAR 2020 projects, it is necessary to put in place a coordinated approach that allows the following:
Makes available the relevant common baseline material to the projects by the SJU; To do this, projects provide in the project PMP/Demo Plan the most up to date/realistic view of the project plan;
Elaborate and align project interdependencies (refine the relationships between projects in terms of input/output or coordination needs). While the notion of complementary grants may be recognized in some grants, it does not provide specific guidance on how to make projects work together in a programme;
Elaborate on the PMP/Demo Plan-level planning. Considering the contractual nature of the project planning that is fixed in the GA (Annex 1), it has been decided to harmonise for IR/VLD projects the definition of project contractual deliverables at the level of Solution data packs, without specifying these in detail;
Analyses and where necessary prepares the alignment of individual project schedules to the programme life cycle (introduced in section 2.2). The need for a synchronisation between projects and between projects and the programme level is necessary to manage the overall SESAR 2020 programme.
The objective of the ‘programme and project initiation’ process is thus to ensure the proper initiation of each SESAR 2020 projects.
During the GA preparation phase, the relevant project information is already captured in SESAR 2020 collaborative platform to prepare for project start. In addition to the contractual level mainly characterized by the delivery of SESAR Solution Data Packs for Solution projects, the delivery of individual documents has also to be defined concerning the way the projects are going to produce their output. The further detailing of project plans is therefore foreseen during the project initiation period and all projects need to deliver a PMP within 2 months from the start of the project, using a standard template8.
The PMP and demonstration plan templates provide more details on what information has to be provided as output of the initiation activities.
3.2.2. Approach
The programme and project initiation starts with programme briefing sessions organised by the SJU. In these briefing sessions the SJU, with assistance of the transversal projects, introduces the participating projects with the necessary details on the application of this Handbook, with particular attention to the material that needs to be consolidated as part of the PMP/Demo Plan.
In addition, each Project Manager is expected to engage their project team (in particular Solution Leaders/WP Leaders) in the initialisation of the PMP/Demo Plan during the Grant Agreement Preparation (GAP) phase.
It is expected that a project kick-off meeting is normally organised by the Project Manager that brings together the whole project team. Participation from SJU is recommended in order to de-risk the initiation of the PMP. It also enables the Project Manager and his team to be further briefed on the details of the Project Handbook, to set the coordination approach with SJU and to ensure a common understanding on the content of the project (e.g. SESAR Solution to be validated/matured).
Within two months from the start of the project, the Project Manager delivers the PMP/Demo Plan to the SJU. The PMP/Demo Plan is then assessed as a standalone contractual deliverable.
8 VLD Projects on the other hand will need to deliver a Demonstration Plan, which is more focussed on the specific nature of these projects.
SJU Ensures that Project Managers have enough information and guidance to allow them to understand how to initiate and execute the project.
Project Manager 0rganises the participation of their project management team at the kick-off meeting and subsequently organises the team work to prepare the project PMP/Demo Plan.
Submit the PMP/Demo Plan deliverable to the SJU
3.2.4. Supporting tools
The project team uses the SJU Collaboration Platform and the H2020 Participants Portal (for H2020 Grants).
3.2.5. Reference material
PMP and DEMO PLAN templates are provided in the template part of the programme library of the SESAR 2020 collaborative platform.
Besides various training and detailed guidance stored in the programme library of the SESAR 2020 collaborative platform that provide insight to understand and develop PMP/Demo Plan, some specific guidance has been made available in the programme library of the SESAR 2020 collaborative platform such as:
SESAR 2020 Project scheduled data management guidelines;
Guidance Material;
Communication guidelines;
Programme FAQ.
3.3. Risk Management
3.3.1 Objective
While in each Grant it is required that the project identifies its Risks, it is not specified in any detail how such Risk/Issue & Opportunities (RIOs) need to be managed since the programme covers a very wide range of disciplines which makes it challenging to specify a unique method for defining and assessing the RIO which are globally managed at programme level when necessary. The SJU has therefore elaborated its own process and criteria for managing RIOs. This section describes how the project shall contribute to it.
3.3.2 Approach
Within SESAR 2020, the management of risk and issues at project level, is performed according to the following principles: every Project Manager shall identify and manage risks and issues affecting the project’s objectives by:
monitoring and tracking the status of action implementation;
Updating regularly the list of risks;
Escalating risks and issues as necessary (e.g. External risks, risks impacting the programme delivery, interdependencies with other projects);
Communicate about risk and issue status and actions undertaken within and outside the project as necessary.
3.3.3 Principal roles and responsibilities
Role Responsibility
Project Manager Responsible for identifying and reporting risks and issues; define appropriate mitigation actions; monitor and track the status of their implementation; communicate about Risk and Issue Status and actions undertaken.
SJU Programme Manager
Supervises the Risk and issues; supports the Project Manager in reporting and undertaking of corrective actions; support the identification of risks to be escalated; ensure proper coordination for risks escalated at programme level.
3.3.4 Supporting tools
The Project Manager will use SESAR collaborative platform to continuously provide more details and visibility on mitigation actions, while for H2020 Grants, principal risks and issues will be recorded and reported against the H2020 Participants Portal at least before end of each reporting period.
In addition, in order to ensure an efficient risk management at programme level, it is requested from the Project Manager to update the list of risks and the associated actions continuously in the S2020 collaborative platform.
This is in line with SESAR JU risk management policy and allows the SJU Programme Manager to identify risks which require an escalation.
3.4. Quality assessment of deliverables
3.4.1 Objective
In order to secure a common programme approach to the quality of the output of each project’s action, SJU has elaborated its own process and criteria for assessing deliverables as defined in this section.
3.4.2 Approach
Within SESAR 2020, the quality of the deliverables is managed according to the following principles:
9 As requested in the proposal template of each call (see table 3.2b)
Ensuring the quality of the deliverable is an integral part of the project’s responsibility. Submission of a deliverable to the SJU involves all project members and implies that the deliverable meets relevant standards and programme expectations;
The SJU assumes that projects correctly apply internal quality controls prior to the submission of a deliverable to the SJU;
It is recommended that projects ensure the close involvement of the SJU throughout the development of a deliverable so that the submitted deliverable meets the required quality level.
Several types of deliverables are subject to a quality assessment by the SJU:
Standalone contractual deliverables that, while being part of the GA, are not part of any solution data pack.
In the ‘quality assessment of deliverables’ process, the initial task of the Project Manager is the submission of the deliverable itself.
Before receiving the outcome of the quality assessment the Project Manager may receive a request for clarification from the SJU; responses to these clarifications are necessary for SJU to finalise the deliverable quality assessment.
Once the SJU has completed its quality assessment it will inform the Project Manager of the outcome. The Project Manager may need to take action to make the necessary changes to the deliverable in case comments are received from the SJU and the deliverable is re-opened for submission. Best practice recommends to resubmit the deliverable with track changes.
Figure 18 : Overview of the ‘quality assessment of deliverables’ process
The ‘quality assessment of deliverables’ process has three possible outcomes:
Outcome Description
Acceptance of the deliverable There are neither significant comments nor a need for the project to produce an improved version of the deliverable.
The status in the SESAR 2020 collaborative platform will show “Approved” (when reviewed by SJU) or “No Reservation (P)10” (when only reviewed by the project internal quality assessment).
Request a new version of the deliverable The deliverable could be accepted if some changes are applied to it and an improved version were submitted.
The Project Manager updates the document as requested by the SJU and resubmits the deliverable with track changes before the date mentioned in the assessment notification.
Permanent rejection The quality assessment is made in a collaborative way between SJU and the projects, and several cycles may occur until the deliverable reaches the required quality. If after several iterations SJU doesn’t see any progress on the deliverable quality, it can be permanently rejected:
The deliverable is considered with insufficient quality and/or not in line with the objectives foreseen.
Table 2: Possible outcomes in the quality assessment of deliverables
3.4.3 Principal roles and responsibilities
Role Responsibility
Project Manager Responsible for submitting the deliverables agreed in the GA and in the project PMP/Demo Plan in line with the baseline schedule and the applicable quality requirements.
Is also responsible to follow up on the quality assessment outcome in case corrections are required.
SJU Manages the quality assessment of deliverables. To do this he/she replies on subject matter experts either within or outside the SJU
3.4.4 Supporting tools
For PMP deliverables, the Project Manager will use SESAR 2020 collaborative platform to upload the deliverable.
10 Due to the high number of deliverable, SJU may not perform a formal assessment of all deliverables but will rely upon the project internal quality process and closed the deliverable assessment as ‘No Reservation (P)’.
For contractual deliverables of H2020 Grants, the Project Manager uses the SESAR 2020 collaborative platform for preparing the deliverable but the Horizon 2020 Funding and tender Portal for handing it over to the SJU; note that the assessment process in both cases will be done through the SESAR 2020 collaborative platform.
3.4.5 Reference material
For contractual deliverables of H2020 Grants, the Project Manager should consult the H2020 documentation to get a complete understanding of the process as outlined in H2020. For PMP deliverables, the Project Manager should use and consult SESAR 2020 collaborative platform guidance material.
3.5. Maturity gate
3.5.1. Objective
The SESAR 2020 maturity gate process aims to have a common approach for assessing the maturity of SESAR content (e.g. Solutions, VLD, AO results) that can be applied at the various levels foreseen along Maturity life cycle.
3.5.2. Approach
The extended release strategy defines the target releases of each of the maturity transitions for any SESAR content. To meet these targets, SESAR projects should plan the set of activities (e.g. validation exercises), which support the development and validation of their respective content (e.g. SESAR Solution) through the maturity life cycle as introduced in chapter 2. A high-level view of the validation path is given in the project’s PMP/Demo Plan, while more details are provided in the validation/demonstration plan.
When all the deliverables within a solution pack have been produced and agreed upon through the ‘quality assessment of PMP deliverables’ process, the Project Manager needs to upload the documentation needed to support the maturity gate session (e.g. Maturity self-assessment, contextual note…) on SESAR 2020 collaborative platform.
The maturity gate will be planned by the Project Manager/Solution Leader in coordination with SJU.
The Project Manager and Solution Leader have to prepare for the Maturity gate according to SJU guidance material available in the programme library of the SESAR 2020 collaborative platform.
The purpose of the maturity gate itself is threefold:
to assess the actual maturity of SESAR content based on delivered material e.g. Vx or TRLy data pack from the project responsible;
to assess/confirm the potential of the project outputs to deliver relevant ATM benefits once deployed;
to give recommendations and highlight issues11.
At the maturity gate Vx or TRLy, project results are evaluated against SESAR maturity criteria specific to the targeted maturity transition.
Only project deliverables already assessed and agreed by SJU (with no critical reservations) will be taken into account (no draft material).
The result of the maturity gate will be a SESAR maturity gate report containing the conclusions following the assessment of these criteria and the evidence offered. Conclusions may include recommendations that indicate additional research or development activities are needed later in the SESAR programme, either in further Vx or TRLy validation activities or in future Vx+1 orTRLy+1 validation activities12. These recommendations will be of a level of detail allowing an easy check in future maturity gate, e.g. making reference to the appropriate assessment thread of the SESAR maturity criteria.
After the maturity gate session, in which the Project Manager participates, SJU produces the maturity gate Report on SESAR 2020 collaborative platform.
It is also possible that the maturity gate concludes that further validation work is required to achieve the assessed maturity level. This implies that a grant amendment may be needed to change the baseline planned date.
Finally it is also possible that the Maturity Gate leads to the conclusion that all validation objectives have been properly addressed but that the solution does not work or is no longer expected to deliver the foreseen benefits. In that case the solution pack as contractual deliverable will be accepted, and will be considered during project review for payment, but the further R&D work foreseen for successive levels of maturity are no longer required and a grant amendment will be initiated to remove this work from the project.
11This does not apply to the assessment of Solutions at the V3 or TRL6 maturity phase nor for VLD project at V4 or TRL7. 12 These recommendations may trigger the need for a Grant Amendment.
In collaboration with the SJU must plan, prepares and follows up the maturity gate.
Transversal project Supports the maturity gate process by providing their view after assessment on the achievement of specific maturity criteria maturity.
SJU Chairs, moderates and coordinates activities related to the maturity gate including the gate meeting itself.
ATM Expert Support and possible participates to the maturity gate process and provides his assessment based on his expertise as subject matter expert.
3.5.4. Supporting tools and access rights
For this process, the Project Manager will need to apply the Maturity Assessment tool accessible on the SESAR 2020 collaborative platform and prepare the project presentation according to the guidance and template available in the SESAR 2020 collaborative platform.
3.6. Project review
3.6.1 Objective
The SESAR project review process considers as well the integrated way of work and partnership aspects.
The two main linked objectives of the project review are to assess the project execution relative to the description of action in the GA (Annex 1) and thereby to support the SJU in processing the payment request from the project. The project review is a mandatory process part of the ‘reporting and payment’ process (relationship between the reporting and payment process and the project review is presented in picture 25 section 3.8.2) and a tentative schedule has already been agreed when signing the GA (in each Annex I).
In addition, and mentioned in each GA, the SJU has the right to review a project at any required time. Such an ‘ad-hoc’ review could be organised when projects are facing difficulties to comply with the project plan or when the project is not performing as expected.
In both cases, the SJU will apply a standard process. A comprehensive evaluation of the project status at an agreed moment in time compared to the planning foreseen in the GA and complemented by the PMP/Demo Plan will be established.
3.6.2 Approach
Project reviews apply to all type of projects. Due to the supportive nature of project reviews to the reporting and payment process they have to be conducted at least once per reporting period (usually of 1 year), as established in the GA. Ad-hoc reviews may also be called in function of (lack of) progress made or in function of the way corrective action progress.
Within the project reviews, the SJU assesses the Project against management and content criteria. Projects are evaluated looking both backward (how has the project performed up to now?) and forward (is the plan for the future adequate?).
The preparation of the project review starts in parallel with the production of the periodic (or final) technical report and financial report. As soon as the reporting period starts the SJU plans the review meeting date and location in a way that the financial report and the technical Report from the project will be available beforehand.
The Project Manager ensures the submission of the technical report and financial report as foreseen in the reporting and payment process ahead of the review meeting, unless the Project Review is scheduled between reporting obligations, in which case the SJU Programme Snapshots serves as main input for the Review meeting.
The Project Manager also makes available to the SJU13 the most up to date Project PMP/Demo plan that provides sufficient details on how the next reporting period will be managed.
Finally, the Project Manager prepares and submits the Review meeting presentation, based on the template present on SESAR 2020 collaborative platform. On the day scheduled, the SJU chairs the review meeting, in which at least the Project Manager participates.
The possible outcomes14 of a review meeting are presented below.
13 The files shall be handed over to SJU through the SESAR 2020 collaborative platform. 14 These are coming from the H2020 project review definition.
Fully achieved Project has fully achieved its objectives and milestones for the period.
Largely achieved Project has achieved most of its objectives and milestones for the period with relatively minor deviations.
Any actions placed on the project must be completed by dates agreed between the Project Manager and the SJU. If any remain open longer, without compelling justification, the SJU shall change it to a ‘Partially achieved’.
Partially achieved Project has achieved some of its objectives and milestones; however, corrective actions will be required.
The project must complete the corrective actions defined within the period agreed with the SJU. If the corrective actions remain open longer, SJU shall change the project status to ‘Not achieved’.
Not achieved Project has failed to achieve critical objectives and/or milestones and/or is severely delayed.
The project requires suspension or termination. SJU escalates the problem internally to the Management. The SJU may then start the process that may lead to the suspension or termination of the project.
Table 3 Outcomes of the project review
Following the review meeting, the SJU formulates the project review report. This means:
Formulating the SJU conclusion on the extent to which the project has achieved the objectives and additional output (i.e. Deliverables) foreseen in the GA up to that point.
Defining any corrective actions that must be taken by the project in case of limited achievement; if necessary, that may trigger a Grant Amendment.
The project review report is sent to the project.
At this point, the Project Manager can review the project review report, and may, in case of disagreement with the outcome initiate a contradictory procedure (in line with grant agreement articles).
3.6.3 Principal roles and responsibilities
Role Responsibility
Project Manager15 Organises project contribution and input to the project review
SJU Organises the review on SJU side and involves required actors
15 Best practice recommends that Solution Leaders also participate to the review meeting
For this process the Project Manager needs access to both the SESAR 2020 collaborative platform and the H2020 Portal for H2020 grants.
3.7. Reporting and payment
3.7.1 Objective
The Reporting and Payment process is a process fully defined by SJU Annotated Model Grant Agreement. Its objective is to provide the SJU with a technical report and a financial report at defined intervals concerning the status of a project and includes the request for payment and the payment of eligible cost for the completed period.
3.7.2 Approach
In the preparation of the GAs the reporting periods have been fixed for each project. For each of these periods, the Project Manager will have to submit a technical report and a financial report. This has to happen within 60 calendar days of the end of the period16.
Each of the GA beneficiaries needs to prepare its own financial report while the Project Manager needs to assure that all contributions are available on time and meet the reporting requirements.
16 For H2020 grants, before the reporting period comes to an end, the Project Manager needs to make sure that the continuous reporting module on Funding and tender portal is up to date with the latest progress of the project. Once the reporting period has ended, the content of the continuous reporting module is frozen. Only the periodic reporting module can be completed to produce the technical and financial reports.
Figure 21 Relationship between project review and the reporting and payment process
Once the report has been submitted the SJU will review its compliance. If elements are missing or need clarification the report may be sent back to the Project Manager. When the SJU approves the eligible costs declared for the period in question, it will pay the interim amount due to the project within 90 days of receiving the report.
The project review meeting shall take place during this period (see 3.6). The SJU may carry out the review directly (using its own staff) or indirectly (using external experts appointed to do so). It will inform the Project Manager of the identity of the external experts and he/she will have the right to object to the appointment on grounds of commercial confidentiality.
At the end of this process, the SJU will elaborate a review report, including the contributions from the independent experts if they were part of the review process, and will initiate the payment process.
3.7.3 Principal roles and responsibilities
Role Responsibility
Project Manager Before the start of the reporting period, makes sure that the continuous reporting module on the portal is up to date.
Prepares and submits the Technical Report. Contributions may be requested from work package/solution leads. All beneficiaries need to agree with the report before it is submitted to the SJU.
Beneficiary administrative point of Contact
Prepares the cost statement for the reporting period
The administrative point of contact of the project coordinator assures that the other beneficiaries get paid their eligible co-financing once the SJU has paid the coordinator.
SJU Reviews the technical report (may engage ATM experts in doing so).
Uses the technical report, the latest SJU programme snapshot and the latest project PMP as main inputs to prepare and execute the project review.
3.7.4 Supporting tools and access rights
For H2020 grants, this Horizon 2020 process mainly requires access to the Horizon 2020 Funding and tender Portal, for all other grants, it requires access to SESAR 2020 collaborative platform. The Project team may need to access the SJU Collaboration platform for details at PMP/Demo Plan level and all supporting material for the Review meeting.
3.8. Grant Amendments
3.8.1 Objective
The objective of the ‘Grant Amendments’ process is to provide a controlled way for changing the Grant Agreement between the SJU and the beneficiaries of the Grant.
3.8.2 Approach
The process for Grant Amendments is defined in the Grant Agreement and documented in the Annotated Model Grant Agreement Article 55. When Project Coordinator identifies need to update/adjust/align its grant agreement, it shall first issue a Change Request (CR) in the SESAR 2020 collaborative platform. CR can be initiated by any beneficiary of the project addressing any part of the Grant Agreement (Part A and/or Part B Annex 1/2). An internal project workflow of maturity of the CR has to be followed before the project finalizes its CR that can then be reviewed by Contribution Managers. Once agree, the Project initiates the Amendment on the basis of the agreed CR using the Consult Officer phase. During this phase, the Project transform the CR in detailed impact on the Grant but in draft form for consultation and agreement with SJU. Once SJU has finalized the review, it delivers the Green light to the project to formally submit the amendment.
3.8.3 Principal Roles Applicable
Role Responsibility
Project Manager and other beneficiaries
Propose and consolidate by internal review Changes to the project grant.
Launch the formal Grant Amendment on the Change(s) agreed.
SJU Process the formal Amendment request
3.8.4 Supporting tools and access rights
This process is managed by the Grant Management tool (both the Horizon 2020 Funding and tender Portal for H2020 grants or SESAR 2020 collaborative platform for any other Grants).
The objective of Project Reporting process is to allow monitoring in a qualitative and quantitative manner the progress and the forecast of the Projects including the status of their risks and issues. It supports SJU in it is planning and steering of the entire programme with the governance bodies (DMSC and PC).
3.9.2. Approach
The reporting on the progress of project shall be done continuously while a formal report and an up to date project schedule have to be submitted once a year (end of month 10).
This approach aims to achieve the SJU steering and monitoring of the program delivery of the ATM Master Plan while it optimizes the project reporting activities.
To allow proper monitoring and control, the SJU performs SJU Programme snapshots to collect and consolidate information from all projects on their progress and reports it to governance bodies (DMSC and PC).
The content of the formal reporting can be summarised as follows:
A summary status that gives an executive statement on the progress made since the last report and on key issues;
Achievements made in the last reporting period, i.e. milestones that were passed, Reviews and their outcome, risks turning into issues, and key progress data on active tasks;
Status of milestones (achieved OR not achieved) and deliverables (completed and handed over OR not completed)
Issues that are being handled (including their recovery status);
Top 5 risks in order of criticality and/or priority;
3.9.3. Principal Roles Applicable
Role Responsibility
Project Manager / Solution Leader
The Project Manager/Solution Leader shall continuously report Progress in close collaboration with the rest of the project.
Once a year17, the Project Manager/Solution Leader shall prepare the formal Progress Report in close collaboration with its team and submit it to the SJU.
SJU Will assess the progress report and return feedback if any during a debrief meeting to be organized by the Project Manager within 2 weeks following submission of the Progress Report.
3.9.4. References and applicable tooling
Progress Reports will have to be produced and submitted to SESAR 2020 collaborative platform18 by using dedicated interface/template (note that Template could be totally integrated within the SESAR 2020 collaborative platform, consequently Project Manager/Solution Leader will interact and update directly their information in the SESAR 2020 collaborative platform).
3.10. Communication and Dissemination
3.10.1. Objective
According to the grant agreement, projects have certain obligations to perform communication and dissemination activities, and to reference the co-funding. In addition, the SJU has introduced a number of specific principles to:
Ensure that the project communication is aligned with the SESAR Programme communication managed by the SJU and to assure the fit for purpose of all communications across the programme.
Ensure that programme contributions to global coordination activities (e.g. EU-FAA) are aligned and co-ordinated across the programme.
The objectives of the communication and dissemination process are thus the following:
17 The first year, an additional report has to be produced at the end of the first quarter. 18 For H2020 Grants, PRs are contractual deliverables to be submitted through the Funding and tender Portal.
To create awareness and outreach about SESAR 2020 and its projects among stakeholders both inside and outside Europe, where applicable;
To showcase the research outcomes and benefits that SESAR solutions bring to real day-to-day air traffic management (ATM) operations as well as the performance of aviation both in Europe and within the broader global context;
To accelerate the operational stakeholder acceptance and subsequent deployment of SESAR solutions.
To demonstrate the value of public-private partnerships for European competitiveness and economic sustainability.(Refer to Grant obligations)
3.10.2. Approach
Projects are requested to plan and execute their communications and dissemination as follows:
- Define with project partners on key messages, communications activities, milestones, channels and metrics and record these in the project management plan
- Record and report on all communications milestones in SESAR 2020 collaborative platform, making use of available templates
When preparing external awareness-raising and outreach activities, projects are requested to liaise via appropriate template available in the SJU collaborative platform with the SJU Strategy and External Affairs team, in order to:
Ensure that project communications and outreach milestones are synchronised with broader SJU programme scheduling and planning;
Review strategies, key messages, targeted audiences and communications material on SESAR solutions to ensure consistency with the aforementioned communications objectives;
Develop joint outreach activities taking into account established cooperative arrangements by the SJU or with the European Commission within the context of SESAR;
Benefit from support of the SJU for various events and conferences;
Maximise outreach by using SJU communications channels and cooperative arrangements to further cascade relevant content.
3.10.3. Principal roles and responsibilities
Role Responsibility
Project communication Leader
Defines and manages the execution of the communication plan. He shall liaise both with the SJU Strategy and External Affairs team and the communications point of contacts of each beneficiary.
SJU Maintains SESAR communication guidelines
Supports and where required validates contributions to SJU programme communication or outreach (e.g. E-News…)
3.10.4. Applicable tooling
Each projects has to manage all its communication and dissemination activities and share material with SJU through the SESAR 2020 collaborative platform.
Each project is entitled to use SJU branding material (Logos, illustrations) and editorial content (Generic text, slides) to enhance their respective communications. This material is available through the SJU website, S2020 collaborative platform and upon request to the SJU Strategy and External Affairs team.
Each project can also take advantage of the following communications channels of the SJU:
- SJU website – as part of the project communications plan execution, each project is expected to provide relevant content to populate their pages through the SESAR 2020 collaborative tool via the Progress Report (publishable summary part) and by notifying SJU Strategy and External Affairs;
- SESAR LinkedIn Group (https://www.linkedin.com/groups/1828459/ ) – representatives from projects can post content in the Group feed (e.g. Event announcements and updates);
- E-news – the SJU publishes an e-newsletter on a monthly basis. Projects can request to have their news/results featured by contacting the SJU Strategy and External Affairs team.
3.11. Involvement and co-ordination in external and international affair matters
3.11.1 Objective
The SESAR Programme needs to coordinate well with ATM stakeholders worldwide in the context of pursuing global interoperability. The SJU has therefore introduced a number of additional principles to:
Ensure that the project’s external and international aspects are aligned with the SJU international affairs strategy and activities and with the EU’s aviation strategy, notably its external dimension.
Ensure that programme contributions to global coordination activities (e.g. EU-US, ICAO) are aligned and co-ordinated across the programme.
The objectives of the external and international affairs process are thus the following:
To create awareness and outreach about SESAR and its activities among stakeholders both inside and outside Europe, where applicable;
To ensure that SESAR solutions are globally interoperable and bring value to real day-to-day air traffic management (ATM) operations in the broader global context;
To accelerate the operational stakeholder acceptance and subsequent deployment of globally interoperable SESAR solutions.
To promote European competitiveness and economic sustainability by leveraging the public-private partnership.
3.11.2 Approach
To ensure consistency with the SESAR international affairs strategy, project consortia are requested to liaise with the SJU when preparing international activities or engagement, in order to:
Ensure that the activities are aligned with and integrated into broader SJU international coordination, scheduling and planning, notably taking into account established cooperative arrangements by the SJU or with the European Commission within the context of SESAR;
Review objectives and content of international activities to ensure consistency with SJU’s core international affairs objectives;
Benefit from support of the SJU for various events and conferences;
3.11.3 Principal roles and responsibilities
Role Responsibility
Project Manager The project Manager shall liaise with the SJU when planning any international activities including missions.
SJU Manages SESAR international affairs and cooperative arrangements with international partners
3.11.4 Reference Material
The following reference materials provide more detailed explanation or support the practice outlined before:
SJU Single Programming Document;
EU Aviation Strategy.
3.11.5 Applicable tooling
Not applicable.
3.12. Project Closure
3.12.1. Objective
The SESAR 2020 closure process aims to have a common approach for closing each project in the global context of the SJU Programme and in order to capture the key project output towards the ATM Master Plan.
3.12.2. Approach
Two months19 prior to the end of the Action, each project has to prepare and deliver a Final Project Report summarizing its key finding as well as it key achievement towards the European ATM Master Plan.
In particular, the project shall report on the achieved maturity steps but also report on the on-going work in progress, supported by a self-maturity assessment.
19 For Exploratory Research projects, the FPR shall be submitted as the final deliverable of the ‘production’ period of the project (i.e. prior to the final 6 months period dedicated to dissemination when defined).
4. Project participation to content development to design of SESAR Solutions in function of Master Plan execution
Practises in this chapter are mandatory only for IR projects while - ER projects do not need to consider this chapter; - VLD projects shall use, base their activities, give feedback and report against the information
gathered as part of the design of SESAR Solutions as documented in this chapter.
4.1 Introduction
4.1.1. Scope of general practices
While the core business of the project is to develop and validate SESAR Solution(s), this activity must be done in such a manner that SESAR Solutions are developed and documented and fit in the bigger picture as outlined in the ATM Master Plan.
In this chapter we introduce the applicable collaborative and iterative practices related to requirements identification and design of solutions that will be harmonised via the content integration process supported by the transversal projects. Specifically, these practices will contribute to the following aspects:
Concept development;
Systems development;
Services development;
Standards and regulations identification;
SE data identification;
Content Integration.
Each of these practices are briefly introduced hereafter in this Handbook. Further guidance and more specific details can be retrieved from the method & guidance materials and from the templates as made available in the S2020 Programme Library of the collaborative platform.
4.1.2. Some principal roles and responsibilities
Compared to the Grant articles where a few roles and responsibilities are defined in a firm and formal manner, this document does not define nor imposes any specific roles.
However, experience has shown that an institutionalised approach helps to ensure that project collaborates well with other Solution and Transversal projects. It is for this reason why some roles related to research, development and validation are listed in the table below and that needs to be considered when a project starts the work.
Project Manager & Solution Leader(s) Ensures that solution life cycle related activities are planned, resourced and performed according to applicable reference material AND that the activities contribute to the needs as set by the architectural and performance management framework.
Project/Solution Content Integration Team (PCIT) is a team lead by Content Integration Lead (PCIL) and composed of people from solution(s) (e.g. Solution Leaders and practionners) in charge of ensuring that the solution(s) content information, the information of dependent solutions are consistent and complete and are aligned with the needs as set from a transversal viewpoint.
- supports the Solution Leads in modelling the Solution Datapack and analyse/resolve the consistency and coherency checks produced during integration in the global model.
- ensures that the content of SE data provided by solution projects (e.g. requirements) is consistent at content integration level.
- supports to the Solution Lead to ensure that the performance assessment is performed and reported according to the Content Integration Reference Material.
- the PCIL is the focal point for interaction with Content Integration and Performance Projects and ensures that feedback information is disseminated internally within the project and in this function participates to programme transversal aspect co-ordination meetings and work sessions in order to:
• Ensure the technical and operational consistency and coherency between the different solutions developed in the project (i.e. identify and seek for solutions for any gaps, overlaps or conflicting choices between the solutions of the project),
• Identify gaps in the validation strategy (e.g. missing integrated validations);
• Ensure consistency with dependant Solutions within and/or with other projects,
• Coordinate interactions with Transversal needs and activities,
• Support the Solution Leader for the organisation of the maturity gates
services focal point SESAR Solution content and validate it
Performance experts21:
• Safety Management Focal Point
• [Cyber] Security Management Focal Point
• Environmental Management Focal Point
• Human Factors Management Focal Point
• Operational Performance Management Focal Point
Ensures that the requirements for the different areas (e.g. safety, security etc.) are considered in the SPR-INTEROP/OSED, matured and the results consolidated at solution level and its operational environment area (e.g. operation environment and/or Sub-operating environment).
Participates to programme transversal meetings or work session related to a specific area
Participates to the community of practice related to a specific area (see section below)
4.1.3. Purpose and role of Content Integration related assignments
A content integration team (PCIT) is a team that aims to:
• Ensure the technical and operational consistency and coherency between the different solutions developed in the project (i.e. identify and seek for solutions for any gaps, overlaps or conflicting choices between the solutions of the project),
• Identify gaps in the validation strategy (e.g. missing integrated validations);
• Ensure consistency with dependant Solutions within and/or with other projects,
• Coordinate interactions with Transversal needs and activities to
o Ensure common assumptions and methodology are used by the Solutions, in line with Transversal Projects guidance material;
o Facilitate the work of Solutions to resolve inconsistencies/gap inside and/or between them.
• Support the Solution Leader for the organisation of the maturity gates
The Content Integration Team is composed of representative and knowledgeable people (such as operational and technical experts, performance/CBA experts, integrators, etc.) working on ATM and Technological Solution within solution projects; people working on transversal activities within the dedicated TA projects could contribute to the Content Integration Team when requested by the project.
The content integration team is not producing deliverables in addition to the deliverables at solutions level.
4.1.4. Community of Practices
Community of Practices are small networks grouping of domain experts within the programme or partnership with the aim to stimulate collaboration and exchange of information for a series of specialised domains and as be a reference repository for SESAR Performance knowledge.
21 List of bullet is not exhaustive and may be adapted/adjusted to the project’s scope and programme’s need.
The SESAR operational concept is described at different levels of detail. The objectives of the SESAR operational concept development process is to drive, develop, refine and consolidate these descriptions in such a manner to keep a consistent and coherent operational concept and where needed expand or amend it. As such, it aims to illustrate the ATM MP vision and be the reference for global interoperability and harmonisation adapted for Europe from the ICAO Global Air Traffic Management operational concept.
The SESAR operational concept development and refinement is an iterative and collaborative process whereby SESAR Solution design and validated results are matched to, and aligned with, the concept of operations and validation strategy and can trigger amendments in both a top-down and a bottom-up manner.
4.2.2 Overall approach
The SESAR operational concept describes a high performing ATM system to match the ATM MP vision and its performance improvements, enabling airspace users to fly their optimum trajectories.
It allows all ATM stakeholders, from the civil and military airspace users and service providers, to airports and the manufacturing industry to gain common understanding of the ATM system and a clear view of the steps to achieve the target ATM concept.
The realisation of the SESAR target operational concept follows strategic orientations described by the SESAR four Key Features, which evolve through on-going R&D activities leading to deployment. The ConOps expands on the elements introduced in the integrated roadmap e.g. SESAR Solutions. It contains:
Description of the operational concept and method;
Description of the operational environments;
High-level operational scenarios;
High-level operational requirements;
Performance expectations;
Apportionment at SESAR Solution level and link to OI steps;
High-level operational process model in EATMA;
Any models related to the above when relevant.
The concept development practices are fully integrated in the EATMA framework and content integration approach as outlined in this chapter.
The ConOps is the result of collaborative work between content integration and SESAR Solution projects and represents the basis for a common agreement and understanding on operational concept aspects.
The ConOps is apportioned at SESAR Solution level: SESAR Solution projects develop and describe their respective concept elements in SPRs-INTEROPs/OSEDs. The SPRs-INTEROPs/OSEDs contain the analysis and refinement of the SESAR Solution(s) in order to capture formal operational,
performance, safety and interoperability requirements per SESAR Solution. The SPRs-INTEROPs/OSEDs contain:
Description of the operational environment;
Description of the previous operating method, new operating methods, and the difference brought in by the SESAR Solution;
Detailed operational processes and models (via architecture modelling, see further);
Detailed list and modelling of use cases (via architecture modelling, see further;
Detailed operational requirements (via SE data, see further)
Performance and safety requirements (via SE data, see further);
Information exchange requirements (IER), also captured in the SPR-INTEROP/OSED, to be analysed as candidates for the definition of information services (via SE data, see further).
Interoperability requirements (via SE data, see further);
Standardisation and regulatory needs (see further).
After each validation cycle, each solution project updates, if necessary, the content of both SPR-INTEROP/OSED (e.g. requirements by means of the system engineering data management framework) and EATMA models considering the validation results. The result of these updates will be used (if required) to revise the ConOps. Whenever there is an issue at ConOps level e.g. conflicting interpretations between two solution projects, a Content Integration Item management process will be activated to facilitate discussion and agreement.
Figure 22: Operational content integration process
The following reference materials (stored and made available in the programme library of the SESAR 2020 collaborative platform) - provide more a detailed explanation or support the practice outlined before:
- Latest edition of the Concept of Operations (ConOps) that describes the Air Traffic Management (ATM) operation envisaged in Europe.
- Latest edition of the Validation Strategy SESAR 2020 (VALS) provides projects a view to fulfil their objectives including validation performance targets by ensuring that there is a common understanding of the set of shared principles and practices
- Latest version of the SPR-INTEROP/OSED template - Latest edition EATMA guidance material Provide guidance to the structure of EATMA, its
development process and its link to deliverables.
4.3 Systems developments
4.3.1. Objective
The system development practices are required to guarantee a coherent and consistent outcome from the system design that provides the different stakeholders input for decision making, planning of deployment and investment.
4.3.2. Overall approach
The SESAR 2020 development cycle begins with the provision of a set of steering principles (including design principles and rules) and uses as a starting point the baseline system architecture and operational inputs. The ATM Solution and Enabling Solution projects should start their developments based on this input and their deliverables should make a clear reference or traceability to the applied steering principles and operational inputs.
The main activities to be undertaken in order to build a TS/IRS are the following:
Identification and description of the breakdown into functions of the system architecture. This is done following the aforementioned steering principles and starts from the impacted system/s identified in the impact table in the architecture description document (ADD) (i.e. the baseline SESAR architecture as generated from the previous yearly iteration) and the operational architecture and requirements.
Description of the functions that are needed in order to deliver the SESAR Solution and a view of how the technical systems, functional block(s), system ports and roles participate in realising the operational needs. Once done, functions are then assigned either directly to the system or to a functional block or to a role.
The choice of the level of functional decomposition of a technical system is a decision taken by the ATM Solution and Enabling Solution Projects following the steering principles. The functional breakdown has to be coordinated from an architectural perspective with content integration project and take into account system needs and to avoid functional overlaps and gaps with other systems and solutions.
Description of how the different resources will interact.
Description of the resource connectivity model: describes how the relevant resources interact in different operating environments to achieve the needed capabilities (using capability configurations, technical systems/functional blocks, services and system ports).
Definition of the services to exchange data between technical systems/function/capability configurations.
Definition and description of system ports of the technical systems under the system’s responsibility, the system-wide information management (SWIM) or legacy protocols/standards that are implemented by these systems ports, including the identification of the technical systems connected to the system port.
Identification of the data elements carried by the system port connectors. Where the data elements are defined in the AIRM consolidated logical data model (CLDM), this model should be used. Extensions to the AIRM could be proposed in coordination with content integration projects.
Elaboration of a service provisioning matrix describing the system context in which the services are deployed.
Identification of one or more common capabilities that could be consumed as a common service.
Allocation of enablers to functions/service/technical systems/system ports/roles (N to n relationship), in order to build a traceability matrix.
Assessment of the system/service changes and analysis of the impact of the architectural changes on the system/service architectures under their responsibility.
Documentation of architecture decisions impacting only a given solution. When the decision is impacting also other solutions this should be escalated to content integration project for assessment and coordination.
Definition of systems and interface requirements.
Confirmation and assessment of the requirements through prototyping.
Update of the TS/IRS considering results from verification and validation process.
The system development practices are fully integrated in the EATMA framework and content integration approach and with the Service Development methodology both introduced below.
Figure 23: System & Services content integration process
4.3.3. References and applicable tooling
The following reference materials (stored and made available in the programme library of the SESAR 2020 collaborative platform) provide more a detailed explanation or support the practice outlined before:
- Latest edition Technical Architecture Overview (ADD). - Latest edition TS/IRS Template. - Latest edition EATMA guidance material - Provide guidance to the structure of EATMA, its
The service development practices are required to guarantee a coherent and consistent outcome from the service oriented approach, which provides the different stakeholders with information input for decision making, planning of deployment and investment.
4.4.2. Overall approach
Service development is integrated with EATMA development and content integration processes. The service architecture is done according to the service working method [Ref. 1 & 2] and follows the three following steps:
Service identification: it is strongly embedded in the architecture because a service can be identified from operational architecting, system architecting and service architecting.
Service design: produces the SESAR 2020 service description document (SDD) and where needed (to be confirmed by the SJU) also the service technical design document (STDD) containing the technical payload definition and a mapping to the relevant SWIM profile. It contains additional models and machine readable files that are not in EATMA. STDD is only relevant for the prototype (so for the SWIM compliance self-assessment of the prototype).
Service prototyping: produces the service instance in a given prototype. This service instance is outside EATMA. Only the validation integration can interact with EATMA as represented through the blue arrow ‘validation integration’ in Figure 25: Architecting process overview.
The main activities to be undertaken in order to build an SDD will be to:
Analyse the business context and the operational requirements related to the given SESAR Solution. In the cases when no SPR-INTEROP/OSED exists as an input, the ATM MP and the ConOps should be checked for requirements or the issue should be raised to content integration project.
Provide an overview of the services provided or consumed by a solution’s elements.
Design the description of the services that are provided or consumed by a solution’s elements.
The development of services must be fully integrated into other areas of the project activities. These areas include performance (e.g. identification of options for common services), operational (e.g. identification of IER), systems (e.g. service provisioning matrix) and content integration (e.g. modelling the service in EATMA). The activities to be performed on each of these areas are included in their respective sections.
4.4.3. References
The following reference material (stored and made available in the SESAR 2020 collaborative platform) provide more a detailed explanation or support the aforementioned practice:
- Latest edition EATMA guidance material - Provide guidance to the structure of EATMA, its development process and its link to deliverables.
The modernisation of the European ATM system will be enabled by the evolution of European and global regulatory frameworks relying on performance-based regulations with the technical details delivered through standards and/or acceptable means of compliance.
In order to deploy SESAR Solutions resulting from SESAR development there is a need to capture standards and regulatory needs in order to ensure that necessary provisions are generated in a timely fashion. Solution projects should identify needs for standardisation and deliver inputs for standardisation work.
4.5.2. Overall approach
4.5.2.1. Standardisation
Standardisation needs are identified by solution projects as part of their R&D work. The potential need for a standard will be assessed by the master plan project through a yearly programme life cycle process against an identified need to harmonise in support of interoperability, performance, roles and/or responsibilities, as follows:
Technical and operational changes that involve physical interfaces, or the exchange of messages, between different systems or constituents, which require harmonising standards to ensure interoperability;
The introduction of changes at stakeholder level or across multiple stakeholders, which require common operating rules and procedures;
The need to support the allocation of specific performance requirements to different systems and constituents within and between stakeholder frameworks;
The need to review the allocation of frequencies in the aviation frequency-spectrum;
When a SESAR Solution is developed as an evolution of a function already supported by standards, it is necessary to assess the need for a modification of these standards.
Other possible objectives of harmonisation, identified in the EASA basic regulation, are associated with the need to ensure safety, free movement of goods, persons and services to achieve a cost-efficient regulation and certification process at a European level.
An initial identification of the needs for standards can be expressed by SESAR projects as early as V1. This need will be confirmed in subsequent V phases based on the results of the R&D work. The solution data packs will document this need as well as all the relevant data that will be later on used by standardisation bodies to develop them. The majority of the information required to support standardisation activities are produced by solution projects during V2 and V3 phases.
The SPR-INTEROP/OSED work can begin in V1 with the elements on service and environment definition necessary to develop the initial draft document and continue through to V4 to cover the pre-operational system. As the concept matures the process can start to focus on Technical Performance aspects (system requirements, system architecture descriptions and equipment specifications and standards).
It should be noted that, in parallel to this stream of work, the SJU may also identify other standardisation needs or challenge needs identified by a project. This may happen as a result of its
coordination tasks with relevant institutions and parties involved. Such identification may generate interactions between the SJU and the project(s) concerned, as well as the master plan project.
Once the need for a new standard is agreed, their development is performed by Standard Development Organisations (SDO) such as EUROCAE and EUROCONTROL in Europe and RTCA in the US. They provide technical aviation standards through voluntary support and technical expertise provided by a community of interest. When global provisions are required, ICAO is the organisation that leads this development and when relevant with the help of SDOs.
As part of the yearly programme life cycle update of the standardisation and regulatory roadmaps, the master plan project will include the standard needs in the standardisation roadmap as received from the solution projects. If necessary (e.g. if clarifications or explanations are needed, or if the need for standardisation is challenged) iterations will take place between the master plan project and the solution projects. The activities that have not been validated yet should be included in the roadmap specifying the nature of the need.
The roadmap is intended to be used to inform, in a timely manner and after appropriate coordination between SJU and the master plan project, the various standardisation organisations in order to help them to integrate potential new work items in their respective work programmes.
In parallel, the European ATM Standards Coordination Group (EASCG), established in 2015 by the European Commission, is in charge of continuously monitoring the standardisation activities required for the deployment of ATM technology, with a focus on the timely deployment of SESAR Solutions. This group is composed of EUROCAE (chair), EUROCONTROL, the European Commission, EASA, the SJU and the European Standardisation Organizations (CEN, CENELEC and ETSI). The European Defence Agency and the SESAR Deployment Manager attend the EASCG as observers while the Network Manager may participate for relevant items. The delivery of the yearly SESAR standardisation roadmap is an important input to their work.
4.5.2.2. Regulations
Regulatory needs are identified by solution projects as part of the R&D work, identifying the rationale for the development of regulations.
The initial provision of information supporting proposals for regulations – principles of operation, SPR-INTEROP/OSED, TS – required by the stakeholders should be documented at V2. Solution projects will document the final identification of regulatory needs in the V3 solution data pack. The date of availability of related enablers (i.e. IOCs, etc.) should be clearly linked with the development of regulations and the dates they should be ready in order to guide deployment.
The development of standards may lead to identifying the need for a regulatory activity, and therefore the information required at this point is based on the maturity of the standards.
The process for developing regulations begins with the gathering of operational information generated during the R&D work and captured in the V2 SPR-INTEROP/OSED as initial provision, and fully identified in V3 solution data pack.
The master plan project will then produce an initial regulation case. As part of the yearly cycle of update of the standardisation and regulatory roadmaps, MP project will include the regulatory need in the regulatory roadmap as received from the solution project. For activities that have not been validated yet, these should be included in the roadmap specifying the nature of the need. If
necessary (e.g. if clarifications or explanations are needed, or if the need for regulation is challenged) an iteration will take place between the master plan project and the solution project.
It should be noted that, in parallel to this stream of work, the SJU may also identify other regulatory needs or challenge needs identified by a project. This may happen as a result of its coordination tasks with relevant institutions and parties involved. Such identification may generate interactions between the SJU and the project(s) concerned, as well as the master plan project.
The roadmap is intended to be used to inform, in a timely manner, EASA in order to help it to integrate potential new work items in its Rulemaking programme.
4.6 System Engineering Data Management
4.6.1. Objective
SE is an interdisciplinary and iterative approach to enable the realisation of successful SESAR Solutions.
The purpose of SE management is to document, analyse, trace, prioritise and agree on SE data (e.g. requirements and sharing the information with the R&D community). It focuses first on defining stakeholder’s needs and required functionality early in the development cycle, documenting operational and system requirements, then proceeding with system design and verification and finally concept validation. It considers business, operational and technical needs of all stakeholders with the objective of providing a SESAR Solution that meets these needs. The definition and maintenance of the SE data model is under SJU responsibility and is part of the PIRM (Programme information reference model (PIRM): is a summary-level data model that is used as a single reference for the definition of information commonly used and referred to within the execution of the SESAR programme. This model is used as reference and basis to adapt any tools, templates, tools, processes etc. as described in the Handbook and more detailed guidance. The model and descriptions of information are made available in the programme library of the SESAR 2020 collaborative platform).
4.6.2. Overall approach
SE data management will be based on a data centric approach. The goal is to allow capturing SE data in a structured way ensuring consistency, coherence and coverage at programme level while facilitating and controlling change management across the SESAR Solution life cycle.
In SESAR 2020, SE data comprise SESAR Solution requirements (operational, performance, safety, interoperability, technical specifications, etc.) and SESAR Solution-related validation objectives and results.
Solution projects are responsible for formally capturing SE data produced in the different phases of their life cycle. This allows the mapping of content of different project deliverables (e.g. SPR-INTEROP/OSED covered by validation objectives, TS/IRS), which is essential to show that SESAR solution requirements are justified and achieved. The SJU retains full rights and ownership on the content of the SE data database.
At the end of each validation phase, in a perspective of one unique continuous solution development life cycle encompassing increasing maturity levels, an iteration of refinement of both operational and technical requirements with relevant documents is expected:
SESAR solution projects shall start formally capturing requirements at V2 (in the v2 solution data pack). The requirements will be further refined and updated based on the feedback received from validation activities performed by the project.
Validation objectives and results shall be managed for all the different maturity phases of the SESAR Solution life cycle. Traceability to requirements is mandatory for V2/TRL-4 and V3/TRL-6 maturity levels; requirements are linked to architectural elements in EATMA
Engineering artefacts related to the development, integration and verification of prototypes and/or V&VI will not be required or maintained at programme level and remain under the scope of solution projects only.
4.6.3. Reference and applicable tooling
The following reference materials (stored and made available in the programme library of the SESAR 2020 collaborative platform) provide more a detailed explanation or support the practice outlined before:
- Latest edition of SESAR requirements and validation guidelines - It gives Projects the rules to respect when writing requirements and Validation data
- Latest edition of System Engineering - Methodology for the V&VP, V&VI and Demonstration Platform development - System Engineering Methodology to be taken as a reference for the V&VP, V&VI and Demonstration Platform development processes (from the design phase to the validation). It is tailored to the SESAR Solutions life-cycle and includes information about phases and roles.
4.7 Integrating Content
4.7.1. Objective
Practices aim at facilitating the integration of content and at validating consistency and coherence of different developments in order to support decision making by facilitating:
Reporting (e.g. architecture of a solution, technical impact of a solution, etc.);
Impact analysis on operations, services, systems and interfaces;
Identification of gaps and overlaps;
The concept development and system design by providing a platform to support the tasks described in the previous sections.
4.7.2. Overall approach
The contribution to architecture is an iterative and collaborative process and uses the EATMA framework which assembles, in an organised fashion, the architectural information of the SESAR Solutions relevant to stakeholders and directly affecting the execution of ATM. Although sometimes recognised in the architecture, the scope does not cover purely internal stakeholder aspects.
The underlying EATMA structure comprises several interrelated elements which cover the extensive scope of the European ATM. This structure is better known at the European Master Plan LEVEL 1 and LEVEL 2.
For structuring purposes and understanding, elements are grouped in the following layers:
Master Plan and programme layer;
Performance layer (is also known as capability layer);
Operations layer;
Information/Services layer;
Systems/Resources layer;
Standards layer.
Figure 27 shows for each layer of information provided and maintained by the contributing projects and SJU:
Figure 24: Architecting layers and contributing projects
The architecture content Integration process starts by scoping the architecture and ends with deployment integration. However, the different processes will not need to be performed in sequence. In fact, all processes are executed simultaneously and in multiple iterations leading to a need to synchronise between these iterations and by undertaking a systematic series of data capture activities which are designed to ensure that the complete scope for a given aspect of the architectural process is brought together in a systematic and consistent fashion with the objective to identify inconsistencies, gaps etc. and to produce consistent and coherent quality output.
Content Integration is a transversal activity that supports other activities described in Chapter 4 and 5. It should be noted that the work undertaken by solution projects in the pursuit of their project goals (e.g. defining the operational content in the SPR-INTEROP/OSED, etc.) is used within the architecture content integration process to build and develop the architecture and as such the projects undertake architecting work.
The content produced by the projects is improved via the content integration process and its review period (where there is the possibility of cross-review of other project’s work), to avoid inconsistencies, duplications etc. with the work performed by other projects, and the resulting architecture views are included in the project deliverables (SPR-INTEROP/OSED, TS, etc.).
Specific architectural maturity criteria exist, that allow checking the architectural maturity at v1, v2 and v3 gates with the support of Content Integration Project. Corrective actions can be raised during these gates leading to an update of the project deliverables that will be then finalised and included into the Solution Pack.
The architectural content that has successfully passed through a maturity gate is considered mature and ready to be publication AND for which deliverables which are under SJU foreground and were been approved by the PC (or Admin Board) can be published in the EATMA via ATM MP Portal. It is also from this point that an architecture baseline for the next yearly life cycle will be extracted.
In this way the architecture becomes an integral element of the programme and provides comprehensive support to it.
Figure 26: Architecting process integrated in the Programme processes
During the content integration process, a series of sessions that focus on specific aspects of the EATMA content are to be performed. As input to the work the following artefacts are used:
SPR-INTEROP/OSED, TS/IRS and VALR from previous maturity phases and SESAR1;
These sessions consist in structured offline work undertaken by the projects with an interest in the areas of material in question. Guidance on how to provide this material is provided by Content Integration Project in their role as Central Architects and also included in the EATMA Guidance Material and EATMA Community Pages (see references)
4.7.3. Reference and applicable tooling
The following reference materials (stored and made available in the programme library of the SESAR 2020 collaborative platform provide a more detailed explanation or support the aforementioned practice:
- Latest EATMA guidance material - Provide guidance to the structure of EATMA, its development process and its link to deliverables.
- EATMA portal: https://www.eatmportal.eu - Gives access to Master Planning and EATMA information.
5. Project participation to Performance assessment, cost benefit analysis and business cases in support to Master Plan execution
Note: Practices in the domains presented in this chapter are recommended best practices which applicability for application oriented ER project only may be tuned during the preparation of the Grant (e.g. GAP phase).
5.1. Performance assessment overview
5.1.1. Objective
The objective of performance assessment is to use validation exercise results to quantify realistic benefits and impacts that are expected to be delivered by SESAR Solutions and then to present these in a suitable format for subsequent use by the content integration project for building consolidated performance assessments and business cases for deployment planning purposes.
While performance assessment addresses the benefits provided by SESAR Solutions, the performance requirements relate to detailed technical performance necessary for implementation and are addressed in SPRs.
5.1.2. Applicability
The performance assessment as introduced in this chapter and consistent with the description of the SESAR Solution Life cycle as provided in the previous chapter 2 is applicable to both SESAR ATM Solution as well as Technological Solutions, either in Industrial Research or VLD22. The application of the methodology itself may vary and is outlined in more detail in the specific guidance for each KPA and with support by Transversal activities/framework.
5.1.3. Introduction to overall approach
Solution projects assess the performance of their SESAR Solutions using data obtained from exercises conducted using analytical models, fast-time models, real-time simulations and live trials or from past study and/or expert judgment if no results are yet available
They assess performance using specific Indicators related to KPAs for which any impact is expected to be provided by the SESAR Solution, in line with the benefit impact mechanisms (BIMs) as described in the Exploratory Validation Report for ER project at qualitative level, SPR-INTEROP/OSED for IR projects or DEMOR for VLD project. These indicators are described as KPIs if they have been assigned validation targets or simply performance indicators (PIs) if they have no validation target.
The solution projects analyse their exercise data to provide quantitative performance assessments. In doing so they identify and record the limitations of the exercise scope, assumptions, applicability, representativeness, validation techniques and other experimental parameters as recorded in the VALR. Considering these limitations, the solution projects extrapolate and aggregate the results to
22 For exploratory research, only qualitative approach to performance will be required.
provide an estimation of the overall performance of the Solution (see next section) and use these results to develop CBA. These results and limitations are reported in a performance assessment report (PAR) for the solution (dedicated section in the SPR) and also recorded in EATMA for use by the content integration team to develop a consolidated performance assessment at programme level, which will subsequently be used to design deployment scenarios for solutions.
Comparison between solution-level validation targets and performance assessed will be used to guide the programme, e.g. by identifying where additional concept elements (or ultimately SESAR Solutions) are needed to respond to anticipated shortfalls in particular KPAs or to allow for comparison of different options/choices. This is why it is important to have quantified results already since the lowest maturity levels for a solution. There are several techniques to transform qualitative results into quantitative, e.g. by comparing the qualitative results with a known quantified baseline, using ranges to express the uncertainty related with the figure.
Technological Solutions are also covered in this approach. Besides their enabling character, it is acknowledged that they might bring benefits on their own in some KPAs (e.g. in Cost Efficiency (Technology Cost)), and can potentially have solution-level validation targets assigned to them when considered in the appropriate Environment with supporting Operational Services. The SESAR2020 performance-driven approach requires that all impacts in performance –either positive or negative- are captured, so Technological Solutions shall investigate opportunities to identify methods to assess on their own performance (as qualitative derivation of KPA).
5.1.3.1. Principles of measuring performance in SESAR 2020
Seven key principles are identified for the measurement of performance by SESAR solution IR projects in SESAR2020. These are detailed in the guidance documents available in the programme library of the SESAR 2020 collaborative platform and are as follows:
1. Principle 1: SESAR Solutions set-up, prepare and carry out quantified performance assessment starting in V1, when possible and evolving in V2 and V3 maturity phases;
2. Principle 2: Solutions to measure KPIs and PIs they impact at least in two scenarios, REFERENCE and SOLUTION;
3. Principle 3: SESAR Solutions projects to record all assumptions clearly;
4. Principle 4: SESAR Solutions projects should run sufficient and representative experiments that provide results with a significant level of confidence - application of validation techniques;
5. Principle 5: Recording data and traceability;
6. Principle 6: ECAC extrapolation of Results at solution level for content integration;
7. Principle 7: KPIs/PIs as described in the SESAR performance framework shall be used. This list is non-exhaustive in the sense that a project may identify other benefits and metrics that better represent the positive gains. These can also be measured and reported upon it but will not be used in function of master plan progress against ambition reporting.
8. Principle 8: Each SESAR Solution shall analyses the deployment relationships that exist with the other Solutions of the program which have been reported by the Solutions in the PAR. It is important to identify the relationships between Solutions, there are two types of possible relationships between two Solutions—they are either ‘compatible’ or ‘incompatible’. It is also
necessary to obtain the rational for the relations and the coefficient to be applied due to the fact that for aggregation in the Deployment scenarios the results of the combination of two Solutions together is not always a sum of their respective performances especially in case of cross effect of incompatibilities.
Some of these key principles are explored further hereafter.
5.1.3.2. Targets, KPIs, PIs and metrics and what to measure in exercises?
Solution projects should plan performance assessment in terms of collecting quantitative data on KPIs and PIs and for comparison with validation targets for the applicable KPI (Principle 1). The detailed definitions of KPIs, PIs and validation targets are provided in Ref. 1 and Ref. 3 identified in section 5.1.6.
The VALS and the BIMs will guide the solution project on the selection of KPIs and PIs to measure for a Solution, but in any case the Solution project must report on all KPIs and PIs, including when the performance impact is negative or zero (Principle 7).
Validation targets: Define the scope and magnitude of the desired progress towards the SESAR ambitions in terms of the different KPIs for each KPA. Validation targets are assigned to SESAR Solutions in a combined top-down/bottom-up process using allocation of performance ambitions (provided by the ATM-MP) and justification from performance expectations (provided by the ConOps and translated into objectives in the VALS). For instance: Solution X is expected to increase peak runway throughput by y %. The validation target is an orientation to be confirmed and assessed at the maturity gate. The target or value itself is not a maturity criteria (but the project or solution assessment is).
KPIs: Defined in the SESAR performance framework and relate to performance benefits in specific Key Performance Areas. Validation targets are assigned to KPIs. SESAR Solutions projects use the results of validation exercises to report performance assessment in terms of the KPIs, reporting the expected positive and negative impacts. All KPIs for which there is non-null target are mandatory for measurement, while reporting remains mandatory on all KPIs. An example of KPI is CAP3 - peak runway throughput in mixed-mode operations (%).
PIs: Defined in the SESAR performance framework and relate to performance benefits in specific KPAs. However, no validation targets are assigned to PIs. SESAR Solutions projects use the results of validation exercises to report performance assessment in terms of the PIs, reporting the expected positive and negative impacts. Certain PIs are mandatory for measurement and reporting by Solution projects, as detailed in [Ref. 3]. An example of a non-mandatory PI: CAP4 - Un-accommodated traffic reduction (flights/year).
Performance metrics: Sometimes proxies may be used in a validation exercise when it is not possible to measure an impact directly using the specified KPIs and PIs. In these cases, other metrics may be used provided the Solution project later converts the results into the reporting KPIs and PIs. An example of a performance metric: workload reduction using metrics related to specific controller tasks such as ‘number of instructions’, ‘number of phone calls’, ‘% of time speaking on frequency’. This can be later converted into the KPI through the formula: Increase in airspace capacity (%) = (1/ (1-workload reduction/2) -1) x 100. See [Ref. 3] for further details on conversion formulas. Performance Framework enables the use of this kind of complementary metrics.
The quantification of the expected performance impact implied by the SESAR Solution under assessment should be based on the validation results, complemented by any additional evidence and activity (analytical model, external inputs, expert judgement, previous work…), in order to guarantee the full coverage of the Benefit Impact Mechanisms as described in the SPR-INTEROP/OSED as produced for ATM Solution.
Four different levels of performance results are distinguishable across the Programme:
1. Exercise validation result: this refers to the KPIs, PIs and performance metrics which are measured during a validation exercise, taking into account that it may focus on just one part of the SESAR Solution under assessment and under the specific local circumstances in which the validation occurred. It is always referred to the sub-operating environment (sub-OE) in which the exercise occurred (see [Ref. 4] for the list of reference sub-OEs). This level of results by solution projects (i.e. at content development and validation level) and the results are recorded in the VALR.
2. Aggregated and Extrapolated validation results. This refers to the aggregated results (based on exercises results or expert judgements) of all the solutions exercises and extrapolated at ECAC level (by KPI/PI). This level of aggregation is produced by solutions in the SPR-INTEROP/OSED. In the PAR the solution should also provide the dependencies/relationships with other solutions.
3. Solution validation result: this refers to the KPIs and PIs calculated as an extrapolation of the results measured during the different validation exercises at a higher ‘SESAR Solution level’, i.e. an ‘aggregation’ of the complementary Exercise Validation Results to express the potential impact of the individual SESAR Solution at European Civil Aviation Conference (ECAC). It should cover all the sub-OEs relevant for that Solution, either by direct reference to the exercise or by extrapolation through modelling. This level of results is produced at Content Development and Validation level by Solution projects in the SPR-INTEROP/OSED and the results are recorded in EATMA along with supporting data as from the VALR (Principle 5).
4. Consolidated validation result: this refers to the (Key) performance indicators which are calculated as consolidation of the different solution validation results, considering the dependencies among Solutions in different sub-OEs, e.g. per Key Feature, Deployment Package or Capability. This level of results is produced at the content integration level (i.e. by transversal projects) and the results are consolidated on an annual basis using the current state of EATMA each year, in order to support programme monitoring and master planning activities.
5.1.3.4. Performance Results Extrapolation
A number of transformations are necessary to be applied by solution projects to aggregate the absolute exercise validation results obtained in the validation exercises into the more general performance impact analysis of the Solution validation results, extrapolating the local conditions typical of an exercise into more general ones required by the content integration project to consolidate results (Principles 2, 3 & 6). The aggregation activity considers hypothesis such as:
Scope: the expected performance impacts apportioned by the implementation of the full SESAR Solution could possibly differ from the ones observed during a specific validation exercise. This could be due to the limited scope of an exercise, covering just one feature or functionality of the entire solution and therefore not being fully representative for the full solution. For example, the full solution could foresee the complete automation of an air traffic controller task but the exercise is based on human intervention.
Time applicability: the number of hours per day and months per year during which the performance contribution is expected to be observed. For example, a concept may only be applicable at peak operational hours, and hence the annual aggregated performance at ECAC level would be pro-rata to the proportion of peak operational hours at the sub-OE across the year.
Traffic: the share of traffic to which the solution is beneficial, because of equipment or of type of operations. For example, a controlled time of arrival (CTA) concept could be of great benefit to improving the timeliness of equipped flights but if only usable by a subset of the flights in a terminal manoeuvring area (TMA) due to equipage requirements, the calculation of the aggregate benefit in the TMA as a whole has to also include the non-benefitting flights as well as the benefitting flights to get a true representation of the overall gain.
Geography: the specific sub-operating environment, or even specific sub-set of locations to which the SESAR Solution applies. For example, a concept may be applicable to Highly Complex airports, and even then, only to those airports subject to a high frequency of strong headwinds. In such a case the assessed benefit would be applicable to flights using those airports and at times when the adverse meteo conditions occur. Hence the aggregation of performance up to ECAC level would have to share the benefit for these limited numbers of flight across all flights using these airports.
Deployment: exercises will be planned with clear baseline, reference and solution scenarios (Principle 2). The baseline provides the absolute values of KPIs and PIs at the start of the timeframe covered by SESAR1 and SESAR2020 and are the starting point for the validation targets. Ideally the validation exercises will be run in environments (expected traffic, equipage, procedures…) comparable with the expected deployment date (e.g. 2035) but this will not often be feasible, notably for some techniques such as RTS and Live Trial. In these cases, argued assumptions must be made about the extrapolation of the benefits achievable in the expected deployment date compared to the actual assessed environment. Furthermore, the aggregation must take account of the changing environment (e.g. traffic growth) on the benefits since for example the benefits of a change in high complexity ACCs may apply to more ACCs since the number of high complexity ACCs will increase with time.
Critical to these aggregations is correctly identifying and recording the assumptions made when planning the exercises and then when carrying out the aggregation (Principle 3).
5.1.3.5. Confidence in results
The confidence in the performance results should be reported for each KPI or PI and depends on following a scientific approach to experimental design and planning (Principle 4). The approach to reporting confidence is qualitative but objective and the criteria used are as follows:
Results largely based on estimations and expert opinion => Confidence Level LOW;
Results come from few simulations and modelling with few data points and scenarios => Confidence Level MEDIUM;
Results come from complementary simulations and modelling with multiple data points and scenarios => Confidence Level HIGH.
5.1.3.6. Dedicated assessment for specific performance areas (e.g. safety)
This is specifically covered in following chapters. Results and evidences collected within these KPAs are to be appropriately reflected as part of normal validation reporting activity.
5.1.4. Principal roles and responsibilities
Please refer to the section 4.2 with the same name in the previous chapter 4.
5.1.5. Applicable Framework
A model of the performance entities is implemented in PIRM, as shown in the figure below:
Figure 27 – Main links among different elements directly related to Performance
It is expected that solution projects enter the following types of data per solution (Principles 3 & 5):
Assumptions underlying assessed performance (equipage, applicability, deployment data, dependencies…), which need to be consistent with the general assumptions provided by Content Integration;
Per KPI, mandatory PI and assessed non-mandatory PI: o Performance result per sub-OE; o Confidence per KPI and PI; o Underlying performance assessment and aggregation logic; o Caveats associated with results.
The following reference materials (stored and made available in the programme library of the SESAR 2020 collaborative platform within the SESAR 2020 collaborative platform) provide more a detailed explanation or support the practice outlined before:
- [Ref. 1] Latest edition of SESAR requirements and validation guidelines - It gives Projects the rules to respect when writing requirements and Validation data Latest edition of System Engineering - Methodology for the V&VP, V&VI and Demonstration Platform development - System Engineering Methodology to be taken as
- [Ref. 2] Latest edition of the Performance framework for SESAR 2020 - [Ref. 3] Latest edition SESAR 2020 transition validation targets - [Ref. 4]Latest edition Guidance on KPIs & data collection - [Ref. 5] Latest edition Validation Strategy SESAR 2020 (VALS) - provides projects a view to
fulfil their objectives including validation performance targets by ensuring that there is a common understanding of the set of shared principles and practices
5.2. CBA methods and practices overview
This section provides high-level guidance on CBA tasks to be undertaken by projects for each of the V1-V3 (or TRL2-TRL6) maturity phases. It is aimed at Project Managers and experts involved in:
Derivation of CBA(T) objectives and requirements;
Development of VALP and VALRs;
Cost benefit assessments for SESAR Solutions.
5.2.1. Purpose & Scope
The CBA is required to assess the affordability of a Solution with respect to its expected benefits for each SESAR Solution under the responsibility of Solution projects; it is structured around the two following activities:.
- Cost refers to the monetary value of the investment that is used up to produce or acquire the benefit. For example, the training of controllers to use a new automated support tool.
- Benefit refers to the positive value of the return on investment to (some or all) stakeholders. Examples of benefits are additional revenues or cost savings to stakeholders. Note that some impacts can be negative (e.g. increase in noise level) and should be identified and assessed to the possible extent
Considering that the benefits brought by a SESAR Technological Solution will depend on the ATM Solutions it will enable, a specific and if needed tailored CBA is be delivered with the Technological Solution Data-pack.
In V1, the CBA focuses more on scoping the solution, identifying its interdependencies on other solutions and implementation options, the stakeholders impacted, describing in qualitative terms the BIMs in the SPR-INTEROP/OSED, including the most impacted KPAs and KPIs. Cost drivers are also identified. This work should start jointly with validation activities so that the definition of scenarios and metrics used to capture benefits in validation takes account of the CBA needs. In V1, the output is in principle a qualitative description of the costs and benefits of the solution(s) to the different
impacted stakeholders with the aim to ensure understanding what the solution will bring – order of magnitude;
In V2, the feasibility phase, the CBA assesses the economic feasibility of the solution(s) and can help to compare different alternatives e.g. a system implemented with a centralised or local backups or whether a solution is deployed everywhere or only in most complex environments. In this phase there is a quantitative assessment of both costs and benefits (i.e. the performance assessment) of SESAR Solutions. In areas such as safety, security, environment and human performance the benefits are assessed only qualitatively but the costs (e.g. to implement associated requirements) need to be monetised. Critical variables to the economic value of the solution(s) are identified and recommendations for further research to reduce critical uncertainties and improve quality of inputs are made for V3/TRL6. In V2/TRL4, the output should already include a first order of magnitude of benefits and net present value (NPV) of the different options being compared.
In V3/TRL6, the final R&D CBA will include all the evidence gathered in terms of impacts, costs and benefits of a solution. By V3, the CBA should provide the NPV overall and per stakeholder group, a sensitivity analysis identifying most critical variables to the value of the project, a risk analysis, the CBA model, report and recommendations
When relevant, the demonstration (VLD) phase will also provide quantitative assessments to confirm and complement the R&D CBA.
Two principles should guide the CBA activity:
Principle 1 - Iteration: Proceeding by successive approximations, providing information to the Project Manager to select what really matters for further improvements, reducing uncertainty accordingly by collecting more information and/or gaining more control on the Solution dimensions.
Principle 2 - Interaction: Fostering a continuous dialogue between all main stakeholders impacted by the Solution(s), involving them as early as possible with the aim of obtaining their ownership and buy-in.
Principle 3 – Collaboration: partners needs to collaborate to provide credible costs and benefits that are accepted by all concerned stakeholders.
5.2.2. Overall approach
The CBAs to be developed in the programme and their interaction with performance are shown in Figure 31.
Solution CBAs will be used not only to show the economic value of each solution alone, but also as input to develop consolidated CBAs and business cases to support decisions on new implementation objectives and the master planning.
Figure 28 – Main links between performance assessment results at different levels
This Handbook provides guidance for Solution CBAs that are to be carried out in each project. However, considering the interdependencies between the different CBAs and that Solution CBAs are the basic brick to build the consolidated business case, solution CBAs need to:
1. Identify the technologies that enable the Solution. If there are technologies (e.g. system enablers) that enable more than one Solution, they need to be pinpointed and an agreement on how to share the cost of these technologies among the different solutions needs to be reached.
2. Identify interdependencies between solutions in terms of benefits and agree on how to address them to avoid double counting of benefits.
3. Use the official performance indicators as defined in the SESAR 2020 performance framework (See previous Section Ref.2), to capture the benefits of the Solution. If there are benefits for which there are no official indicators defined in the Programme, new metrics can be proposed to capture them and be clearly described and justified.
4. Use the common assumptions of the programme to build the Solution CBA scenarios. If there are deviations from these assumptions they need to be explained and justified. The CBA should consider a scenario where the solution is deployed as a common service/functionality.
5. Use the cost and benefit categories as detailed in [Ref.4].
6. Follow the guidance material listed in Section 5.2.5.
To prepare solution CBAs, the CBA experts in each project will have to liaise with the programme content integration (i.e. transversal projects) from the start of the CBA to ensure that the CBA is aligned with the most recent sets of assumptions, scenarios, guidance and data. Content Integration will also set up a process to ensure peer review of solution CBAs.
NB. In the following text we outline the different steps to be applied for ATM Solutions. In case the solution concerns a technological solution the same process and steps apply but limited and/or tailored to those elements that concern as per TRL 2/TRL 4 and TRL 6 (as opposed to V1, V2 and V3 for ATM solutions) .
The preparation of a CBA follows nine steps, as indicated in Figure 32. Each step is detailed in the sub-section below.
Stakeholders
identification
Benefits
assessment
Costs
assessment
Inputs
collectionModel
generation
Sensitivity &
risk analysis
Recommen
dations
Look back &
iterate
Solutions
& scope
CBA
methodology
Figure 29: The nine CBA steps
5.2.2.1.1. Step 1: Definition of the solution(s) and scope of the CBA
This step defines:
What is the solution subject to the CBA. What is to be deployed: new or modified technical system/infrastructure, human factors elements, procedures, standards or regulations to be deployed in a particular operational environment.
What relationship exist with other solutions (e.g. common enablers, cross effect, incompatibilities etc.) and agreement is reached on how to deal with them in the CBA.
What are the decisions to be supported by the CBA? (E.g. decision to move to next stage of life cycle, decision on which option to choose for further development)
What are the scenarios to be compared in the CBA, namely the solution scenario and the reference scenarios. They do not necessarily coincide with those used for the performance assessment, although they should be consistent with them (and possibly extended), to ensure that the benefit figures are applicable. The CBA can support comparisons between different options for a Solution, through the creation of high-level scenarios that consider when the Solution will be deployed. The reference scenario includes the situation at the start of implementation with assumptions about what enablers are deployed, what data items y and z are available from aircraft, and how deployment is likely to evolve without the solution under analysis, during the time horizon of the CBA.
From the common assumptions and scenarios maintained by transversal projects:
- Time horizon: development, deployment and operational timeframes of the solutions. Dates such as the implementation start date, IOC, FOC dates (e.g. in SESAR1 2013, 2017, 2020, 2025, 2030, other) are set.
- Geographical scope: sub-operating environment (Please refer to [Ref.7] for the list of Sub-OE) and applicability at ECAC level.
5.2.2.1.2. Step 2: Identification of stakeholders
In this Step, the stakeholders impacted by the Solutions are identified. Costs and benefits are identified from each stakeholder group’s perspective such that each one can assess how realistic the assessment is for them. Members of each impacted stakeholder group should be involved in the CBA from early on in the life cycle or, they should at least be consulted. The stakeholder groups typically are: Scheduled airlines (SA), general aviation (GA), business aviation (BA), rotorcraft, air navigation service providers (ANSPs) including civil, military and meteo service providers), airports (different sizes and types), Network Manager, the military (as airspace user, airport operator and service provider) and industry (airborne and ground industry).
5.2.2.1.3. Step 3: Benefits assessment
The assessment of benefits of solution(s) is part of the solution validation.
In V1, initial quantitative performance assessment is expected to be worked on to be prepared (see General Principle 1 in 5.1.3.1), but for CBA purposes a qualitative description of the benefits of the solution may be given, using Benefit Impact Mechanisms (BIMs) provided in the SPR-INTEROP/OSED. It is in this phase that the performance metrics are identified to assess the benefits during the solution’s validation.
In V2, the assessment of benefits should be quantified using validation results, comparing options and identifying areas of further research to reduce critical uncertainties and improve quality of CBA inputs in V3.
The V3 benefit assessment should cover all areas of benefit of the solutions providing more robust qualitative and quantitative assessments.
Developing the benefit mechanisms of a solution is a team work between operational experts/engineers and CBA experts in the project.
Once the benefits have been identified and validation results become available, they need to be monetised. This involves extrapolating results from local (validation exercise environment) to global (ECAC level) as required by the Performance Assessment and their further monetization as detailed in [Ref.1]. The unit for all benefits is euros.
It is important to point out that a SESAR Solution can result in a negative benefit in a certain KPA as a side effect of improving another KPA. This should also be monetised in order to feed a trade-off analysis.
Qualitative benefits should also be evaluated and included in the analysis. These results will be further taken into account within the consolidated business case. Examples of quantitative and qualitative benefits are available in the business cases for 10 SESAR 1 prioritised Solutions [Ref. 2].
The cost assessment of Solutions is also an iterative process. The accuracy of cost estimates evolves with the maturity of the solution, as more technical and operational aspects of the solution are defined. In V1, the costs of solutions are identified and described, in V2 they are quantified providing a first order of magnitude. In V3 cost estimates should be more robust and complete.
The costs to be assessed are the costs associated with the acquisition, implementation and operation of the solution by all stakeholders. Typical costs include purchasing, maintenance, operation, and training as well as any other direct or indirect costs.
The ATM cost breakdown structure consists of 3 main cost categories:
Pre-implementation costs (v4): All costs that need to be used up to define the needs, to develop solutions, to decide which solution best serves the needs. An important note in that respect is that the SESAR R&D costs (up to V3) should not be included as costs in any SESAR CBA as they are sunk costs. The CBA should be focussed on deployment, i.e. what the stakeholders will pay to put the solution in place.
Implementation costs (v5): All costs related to the acquisition and implementation of the solutions. Also decommissioning costs incurred to substitute the previous systems with the new ones should be estimated.
Operating costs: All costs related to the change in daily operations that is brought about by the solution. These can be additional costs due to the introduction of the new Solution, but also reduced ones stemming from decommissioning previous inefficient systems. For example an Airport deciding to decommission an instrument landing system (ILS) CAT II/III to substitute with ground-based augmentation system (GBAS) CAT II/III ground stations, would reduce its CAT II/III maintenance costs, especially the certification flights [Ref.2].
Costs should be referred to the deployment of the Solution into the reference Sub-OE in which it is validated. Indications should be provided on how to extend the values to other OEs.
Costs should be assessed by the stakeholders who are expected to incur them. However cost estimation is a challenging task due to the commercial-in-confidence information that has to be shared. This issue may be mitigated through methods as, but not limited to:
Use of cost groups: this approach consists in defining the cost group to which the Solution belongs, i.e. very low to very high depending on the type of investment it entails. This is especially applicable at V1 level when few validation results are available. Additional details are provided in [Ref.3].
Use of cost ranges instead of a single value: this approach consists in defining an upper and lower bound to reflect the uncertainty around the estimation or the different inputs available.
Establishment of groups of work client-provider, which provide only the final input to be used in the CBA model.
Grouping solutions to a higher level of granularity;
Solutions can be enabled by technologies that also enable other solutions. The allocation of these technologies and of their cost to the different solutions needs to be carefully analysed and agreed by the projects whose solutions share enablers.
Additionally, very often the implementation of solutions (and the associated cost) affects more than one stakeholder. In these cases, cost responsibility/sharing between the involved stakeholders should be analyzed and agreed.
For a non-exhaustive list of Costs categories to be included please see [Ref. 1].
5.2.2.1.5. Step 5: Collection of inputs
In this step all the inputs necessary to calculate the monetary values starting from validation results are collected from different sources, together with their associated ranges and confidence levels. A number of useful references for cost assessment are provided in [Ref. 4].
General inputs and common assumptions as provided by content integration project are always to be considered the valid reference in case of discrepancies among different sources. This is necessary in view of further consolidation of results at the content integration level.
5.2.2.1.6. Step 6: Model generation
In this step the CBA model is created. It can be developed from scratch or re-using existing models. Examples of existing models are: GBAS or remote tower services [Ref. 2] and the SESAR CBA integrated model [Ref. 8]
The model can be developed in excel or using other tools, as long as the tools provide all the required outputs and can be used widely.
All the assumptions made need to be described in the model. The Model should provide as outputs the monetised costs and benefits per stakeholders as required in the CBA template and in line with the CBA quality check list, [Ref. 6].
5.2.2.1.7. Step 7: Sensitivity and risk analyses
This step, using the ranges identified in the gathering of inputs and the assessment of costs and benefits, identifies the most critical variables to the economic value of the project. A possible way of doing a sensitivity analysis is to use the following ranges for uncertain inputs:
Base; 50% chance the cost/benefit/parameter is around this value;
Low; 10% chance the cost/benefit/parameter is below this value;
High; 10% chance cost/benefit/parameter is above this value.
A tornado diagram showing the uncertainties that have the highest impact on the variability of the NPV of the solution can then be calculated.
Using the above ranges, probability distributions for the NPV of the solution(s) can be built to assess the risk of for example having a negative NPV.
5.2.2.1.8. Step 8: Recommendation
In this step, the evidence gathered during the CBA is analysed to provide recommendations on the Solution(s). The recommendations are discussed and agreed involving all the stakeholders.
5.2.2.1.9. Step 9: Look back and iterate
This step is about taking stock of lessons learned and documenting the CBA, including directions for future improvements for the next iteration.
Across the SESAR 2020 programme, aggregated CBA data will be stored in EATMA. This will include assumptions, inputs and results. Solution projects are responsible to input CBA assessment data into EATMA as well as make available the underlying assessments, etc.
5.2.4. Reference material
The following reference materials (stored and made available in the programme library of the SESAR 2020 collaborative platform) provide more a detailed explanation or support the practice outlined before:
Ref 1. latest edition of Methods to assess costs and monetise benefits for CBAs
Ref 2. Latest edition of Method to assess cost of European ATM improvements and technologies
Ref 3. Latest standard inputs for EUROCONTROL cost benefit analyses
Ref 4. Latest edition Guidelines for producing benefit and impact mechanisms
Ref 5. Latest edition ATM CBA quality checklist
Ref 6. latest edition CBA model and methods
5.3. Safety methodology and assessment practices
This section provides high-level guidance on safety tasks undertaken by projects for each of the V1-V4 (or TRL 2 – TRL 7) maturity phases. It is aimed at Project Managers and experts involved in:
SESAR safety assessments
Derivation of safety objectives and safety requirements
Development of SPR-INTEROP/OSED, TS/IRS, VALP and VALRs.
Documenting VLD Safety Case
The SESAR 2020 safety assessment methodological framework is provided by the SESAR safety reference material (SRM) [Ref. 1] and companion guidance material [Ref. 2]. It is consistent with applicable common requirements as laid down in Commission Implementing Regulation (EU) No 1035/2011 [Ref. 7] and recent (EU) No 2017/373 repealing (EU) No 482/2008, 1034/2011 and & 1035/2011 [Ref. 8].
5.3.1. Objective
The purpose of safety assessment in SESAR2020 is to ensure that explicit consideration can be given at early definition and design stages to maximising the delivery of safety benefits and identifying /mitigating safety problems that could occur.
It provides evidence (e.g. to regulators) to demonstrate that safety assessment has been conducted in a systematic way so as to be able to argue that a Solution, or group of Solutions, is sufficiently safe to proceed to industrialisation and deployment.
In addition, at programme level, the purpose is to provide evidence that strategic safety targets are achievable. Eventually, it is to bring into a Solution safety assessment report (SAR) all the findings of the safety assessment to show, in a clear unambiguous way, that a tolerable level of safety has been achieved in design and that the Solution could, as a result, further proceed to industrialisation and deployment.
The SRM-based SESAR safety methodology continues to require that safety assessments examine internal system failures (termed ‘failure based approach’) but additionally requires the consideration of the ‘success based approach’ (SRM section 2.1). The success based approach determines the functionality and performance needed to be incorporated into the design to ensure that when the system is working as intended it is able to provide, at the very least, a tolerable level of safety but also ensures that the potential safety benefit of the design is maximised.
All safety assessments are undertaken at the SESAR solution level.
5.3.2.1. The safety processes
The SRM adopts a generic ‘safety-requirement’ hierarchy which is initiated in V1 by setting Safety Criteria (SAC) and further developed along the ATM/ANS functional system life cycle phases up to V3 .
First, the safety scoping and change assessment process provides an initial assessment of the safety implications of a SESAR solution. It should address, among other things, what the solution is seeking to achieve (e.g. to deliver benefits in capacity, efficiency and/or safety), the possible impact on safety, the safety criteria (SAC) for defining what is ‘safe’ in the context of the solution and, in broad terms, the strategy for demonstrating safety. The results of this process will allow to issue a Safety Plan [Ref. 4], aimed at specifying the safety assurance activities to be carried out by the Solution projects, whilst recording the relevant safety assessment information available at this initial stage. If it is concluded that the Solution has no safety implication, such conclusion has to be highlighted in the relevant SESAR Solution deliverables.
The SAC (what is meant by safe as design targets) are then allocated to safety objectives which set both the minimum positive and maximum negative safety contributions of the Solution at operational level and then safety requirements are derived for the elements of the ATM/ANS functional system (people, procedures, technical). Overall this is an iterative ‘requirements specification – requirements satisfaction’ exercise that is completed when it is demonstrated that the actual design is realistic, i.e. achievable in terms of the safety requirements it places on the human, procedure and technical elements of the system. This requirement hierarchy and its relationship to the SESAR deliverables are portrayed in Figure 33 below.
The safety process described above is mainly applicable to ATM operational solutions. The process to be applied by Technological solutions/Enablers is consistent with this process but has been tailored in a dedicated Guidance of the SRM.
For each solution, different versions of the Safety Assessment Reports (SARs) [Ref. 5] are prepared by the projects in accordance with the progress along the V1 to V3 phases, aggregating the results from the different safety assessment steps defined in sections below.
5.3.2.2. About Safety Criteria - SAC
SAC defines what is considered to be acceptably safe for the change being introduced by the solution. A safety criterion in SESAR is effectively taken as the safety ‘design hurdle’. In effect the Safety level that the ATM change should be designed to deliver.
SAC are derived (SRM section 2.3) at solution level in V1 through the application of an aviation risk model (termed accident-incident model AIM) and are documented in the Safety Plan appended to the Validation Plan. In order to ensure consistency, this activity is facilitated by the content integration project.
5.3.2.3. About safety objectives - SO
Safety assessments at V2 are undertaken at the operational level (air traffic services, ATM network services), i.e. Use Cases modelled through EATMA operational views. Safety objectives specify what the ATM Functional System has to provide across the interfaces between the service provider and the service user in order to satisfy the safety criteria. The primary outputs are success and failure case safety objectives, which are specified from three perspectives:
1 to capture what has to happen in order for the operational services to operate as required; 2 to mitigate the consequences of failure/degradation of the operational services, however
caused; 3 to limit the frequency with which such failures of the operational services may occur so as to
achieve an acceptable level for the associated risk, taking into account the above mitigations.
Safety objectives derived from perspectives 1 and 2 above address the success approach as they express the functionality and performance to be achieved. Safety objectives derived from perspective 3 above address the failure approach as they indicate the acceptable level of frequency to be achieved for such degradation. Safety objectives are presented in the Safety Assessment Report appended to the V2 Solution SPR-INTEROP/OSED.
5.3.2.4. About Safety requirements -SR
Safety requirements are the means by which safety objectives are achieved and they are allocated to technical, procedure and human elements. They are specified in V2 based on an initial design of the solution and in V3 based on a refined design of the solution, i.e. EATMA service and system views. The safety assessment activities at the design level will determine the safety requirements which satisfy the safety objectives at solution level. The initial safety requirements are presented in the V2 SPR-INTEROP/OSED and V2 TS/IRS23 (if applicable) and the refined safety requirements are presented in the V3 SPR-INTEROP/OSED and V3 TS/IRS23 (if applicable). The SPR-INTEROP/OSED will host safety requirements addressing what and how the human actors conduct the operations and what functionalities or information they require for that. The requirements regarding how the technical elements provide those functionalities and information will be hosted by the TS/IRS.
Traceability from the safety requirements to the safety criteria is explicit and presented in the SPR-INTEROP/OSED and TS/IRS, as appropriate. Consequently, the V3 safety assessment report will
23 Only for the safety requirements relative to technical elements
present the assurance that the whole safety requirements hierarchy is complete, correct and (from an implementation viewpoint) realistic.
5.3.2.5. Safety management of VLD (V4)
The VLD safety assurance activities are addressing the suitability of the SESAR Solution for the deployment and the safety Impact of the VLD on current operations.
In order to satisfy above objectives, the specific activities to be considered are the following:
Solution readiness for VLD by documenting the current safety assurance status in order to make a decision on approval to move a SESAR solution from a pre-industrialization stage to a ‘ready for VLD’ status.
Enabling significant levels of engagement and coordination of both the end-users (e.g. ANSPs, Network Manager, airports; airspace users, AOC; etc.) and appropriate regulatory authorities (National Authorities (NAAs; NSAs) and/or EASA) as per the guidance material to execute proof of concept (see dedicated guidance).
Determining and documenting in a VLD safety Plan the safety assurance to be conducted for the VLD encompassing the suitability of the SESAR Solution for the deployment and ensuring the control of the safety impact of the VLD on current operations; and finally;
Documenting the VLD Safety Case. The VLD Safety Case is here a means of structuring and documenting a summary of the results of a VLD Safety Assessment addressing the suitability of the SESAR Solution for the deployment and the safety impact of the VLD on current operations.
5.3.3. Applicable tools
In SESAR 2020 the management of all safety information (incl. SAC, SOs, SRs, assumptions, open issues, etc.) is to be done using the SE data repository, tracking progress and providing visibility of the status of the various safety assurance activities and deliverables (as per the Solution ‘x’ Safety Plan). The safety information (also including assumptions, safety issues, etc.) is input and maintained by the person in charge of the safety assessment within IR projects
5.3.4. Reference material
The following reference materials (stored and made available in the programme library of the SESAR 2020 collaborative platform Library) provide more a detailed explanation or support the practice outlined before:
Ref 1. Latest edition SESAR, SRM
Ref 2. Latest edition SESAR, Guidance to apply the SRM
Ref 3. Latest edition SESAR, Final guidance material to execute proof of concept
Ref 4. Latest edition SESAR, Solution safety plan template
Ref 7. (EU) No 1035/2011 of 17 October 2011 laying down common requirements for the provision of air navigation services and amending Regulations (EC) No 482/2008 and (EU) No 691/2010
Ref 8. (EU) No 2017/373 laying down common requirements for service providers and the oversight in air traffic management/air navigation services and other air traffic management network functions, repealing Regulation (EC) No 482/2008, Implementing Regulations (EU) No 1034/2011 and (EU) No 1035/2011 and amending Regulation (EU) No 677/2011
5.4. Security methodology and assessment practices
5.4.1. Objective
The objective is to deliver securable and cyber-resilient solutions. To demonstrate solutions being securable and cyber-resilient by the end of V3 or TRL6, it is recommended to perform a Security Risk Assessment that could be locally developed based on the SESAR Security Risk Assessment methodology. Security risks are identified, analysed and mitigated at solution level by identifying security requirements that are documented in the SPR-INTEROP/OSED and/or TS/IRS as appropriate.
5.4.2. Overall Approach
The Security Risk Assessment is recommended in particular for the Solutions considered as “security critical”. A solution is “security critical” if it connects multiple stakeholder systems and if a successful cyber-attack could lead to any of the following:
• An accident or serious incident (safety) • A high economic impact throughout Europe (economy)
5.4.3. Tools applicable
The final output of the Security Risk Assessment methodology is a list of security requirements. These need to be treated like any other requirement, and therefore shall be captured in the SE data repository and documented in the SPR-INTEROP/OSED and/or TS/IRS as appropriate. The justification for these requirements should be limited to a reference to the security risk assessment that shall not be included in the SPR-INTEROP/OSED and/or TS/IRS.
5.4.4. Reference material
The following Security related reference material is available in programme library of the SESAR 2020 collaborative platform, either under framework documents or in the templates section:
Ref 1. latest edition of Security Risk Assessment Methodology - describing the methodology
Ref 2. latest edition SecRAM catalogues - providing best practice checklists
5.5. Human performance methodology and practices
5.5.1. Objective
Implementing the ATM MP will only be successful if there is a detailed understanding of the impact that the human actors have on the successful operation of the system. The human element of the overall ATM system remains the most critical source of its performance, safety and resilience.
The purpose of the human performance (HP) assessment process is to ensure that HP aspects related to SESAR technical and operational developments are systematically identified and managed. This is done by incorporating the knowledge and understanding of how human and system actors’ work together, and to explicitly incorporate the requirements that enable all functions to work
collaboratively in managing performance in the SESAR programme. This is supported by a systematic analysis and management of Human Factor aspects of the design and validation of future operations.
5.5.2. Overall approach
HP is incorporated into the SESAR 2020 programme to ensure that sufficient account is taken of integrating the needs of the human centred design within the project environment. The HP reference material (HPRM) [Ref. 1] provides the means, the method and process by which the HP can be integrated into design and the outcomes assessed. The HPRM also provides a set of process based indicators that allow the maturity of the project to be assessed at the same time as supporting concept development and gathering data to support the project and validation activities
To achieve this, the HP assessment process:
describes arguments and necessary evidence to show that airborne and ground ATM actors will contribute to the SESAR expected performance benefits;
describes arguments and necessary evidence to show that the roles, responsibilities and tasks of airborne and ground ATM actors as developed in SESAR are within the scope of human capabilities and limitations;
defines the process to ensure HP proactively contributes to building the operational concept and system architecture and describes how results from HP activities should be used in the development process, with the aim of improving the concept and technology;
considers the HP transition criteria via the SESAR Solution gates from one V-phase to the next V-phase;
has a clear link with validation by (a) providing an input to the validation plan and (b) using the results of the validation activities in support of the HP arguments;
is aligned with the other Transversal Area (TA) assessment processes, by (a) using a shared description of the reference, the solution and the assumptions and (b) by identifying overlaps and synergies between HP and other TAs;
defines interactions and uses synergies with the other TA assessment processes, in particular, the safety assessment process;
Provides data that can feed the Consolidated Business Case.
HP assessment processes may be conducted in any V-phase. However, the scope of the SESAR HP assessment processes is the SESAR V-phases (V1-V3).
5.5.3. Human performance assessment process
5.5.3.1. HP activities in V1
Within the context of the V1 phase evidence is expected to be generated that consideration has been taken to explore the context of the human component of the system. The following activities should be performed to investigate and iteratively refine the Solution / operational concept(s) options:
Based on the argument structure, develop initial descriptive models of the human component of the system.
Make explicit the roles, tasks, constraints and needs of the human actors affected by the operational concept and how their performance contributes to the current system.
Support the development of the initial operational concept and system architecture.
Identify side effects that could compromise human performance and reveal show stoppers.
Influence emergent design ideas based on preliminary Human Factors assessment.
Relate human-centred models to other system models so as to ensure coherence.
Review the argument structure in the reference material and determine which top level arguments will be relevant – These should be expressed as requirements.
The outcome of the V1 work should be used to refine the concept and should be, in some form, reflected in a revised SPR-INTEROP/OSED. Transition criteria for exiting V1 are described in the HPRM [Ref 1].
5.5.3.2. HP Activities in V2
The V2 phase reflects the period within solution development that brings significant progress of prototypes and evolution in ways of working. The HP expert within each project team should be involved within the design process and supporting the development of the operational concept. The focus of the HP work during V2 will be to ensure that:
Roles, responsibilities, operating methods and human tasks are developed to ensure that roles and responsibilities of human actors are clear and cover all the foreseen situations the concept is intended to cover.
Operating methods are commensurate with the level of design and optimal controller / operator performance. It should also be demonstrated that operators (Controllers, ATSEPs and Pilots) can achieve their tasks in a timely and accurate way.
Technical support systems and human-machine Interface have to support an appropriate allocation of tasks between the people and equipment (i.e. the automation still has to support the operator in maintaining situational awareness). The HP support to the project has to ensure that the potential design of the human-machine interface supports the operators of the system in carrying out their tasks.
Proposed team structures and communication between team members must also be reviewed. It must be ensured that changes to the team composition are identified; that team tasks are allocated appropriately between team members. Task critical communication that ensures team performance must also assured in terms of content and delivery.
Although early in the design phase, any emerging transition factors should be recorded for subsequent analysis. These may include, but not restricted to; acceptance and job satisfaction, changes in competence requirements, impact on staff levels and shift organisation, and the need for re-location of the work force.
An issues log should be established that records HP design issues that arise. These should be documented using the high level structure of the arguments in the HPRM. Mitigation measures may be recorded against the issues for subsequent incorporation into design. Establishing the HP issues log ensure the issues are traceable from V2 to V3.
HP support to the design, and the assessment activity will focus on identifying human centred design issues related to each of the above areas. HP integration within the project will ensure that the system design takes into account the limitations and strengths of the people using the system, on the ground, as controllers and engineers, and as pilots. V2 validation activities should provide evidence that assumptions made about human performance in relation to the prototype can be demonstrated and substantiated effectively i.e. to gather evidence that the human component of the system works as effectively as the system being tested.
To suitably assess whether a Solution is sufficiently mature from a HP perspective, transition criteria for V2 to V3 are expressed in the HPRM.
The V3 phase represents the end of the design phase for operational concepts. At this point, although relatively untested in an operational environment the System should be at a point that is ready for deployment. Given the starting point ‘ready for industrialisation, and building on the design work established in V2, the following HP activities in V3 should be performed to develop a ‘body of evidence’ that the system has adequately accounted for its human component:
Demonstrate that roles and Responsibilities are recorded and are clear and exhaustive;
Demonstrate that (refined) operating methods are exhaustive;
Demonstrate system operators can achieve their tasks in a timely and accurate way;
Demonstrate that the automation (if instantiated) works as proposed and delivers the results expected;
Demonstrate that the technical support systems and HMI have incorporated a suitable allocation of tasks between the equipment and the person;
Demonstrate that the proposed design of the HMI supports task delivery in the manner expected by the project team and the controllers and the operational staff affected;
Demonstrate that team working structures are effective;
Demonstrate that team communication is not adversely affected, and that the communication required by the operational change can be effected as required;
Demonstrate the team composition can effectively discharge their Responsibilities;
Demonstrate that the allocation of tasks between team members is appropriate;
Use Argument 4 from the HPRM to document and mitigate the transition factors that should be addressed in moving from V3 to V4
Document all outstanding HP issues and communicate those considered to have a safety impact to the safety team.
5.5.4. Tools applicable
16.04.02, e-HP Repository [Ref. 2]
5.5.5. Reference Material
The following reference materials (stored and made available in the SESAR 2020 collaborative platform) provide more a detailed explanation or support the aforementioned practice:
Ref 1. Latest Human Performance Reference Material
Ref 2. Latest e-HP Repository
5.6. Environmental Methodology and Assessment Practices
5.6.1. Objective
The purpose of Environmental Impact Assessment in SESAR2020 is to provide evidence (e.g. to regulators) to demonstrate that the required Environmental Impact assessment has been performed and that the impacts have been identified and, where possible, mitigations implemented, if a Solution, or group of Solutions, is to proceed to industrialisation and deployment.
Each SESAR solution development and related validation activity should assess the likelihood of positive and negative impacts on the environment at two levels:
Locally: this assessment should concern noise and local air quality around airports,
Globally: this assessment should concern emissions of CO2 due to fuel burn in all phases of flight.
When potential environmental impacts are foreseen, an environmental impact assessment (EIA) should be conducted according to the principles recommended in the SESAR Environmental Reference Material (ERM) [Ref. 1]. As interdependencies might exist between noise levels and fuel burn/emissions produced, trade-offs might be exercised with regard to other performance measurements as well. However, no trade-offs should be made with safety.
Although fuel efficiency was targeted as a priority of the initial SESAR programme, with lot of political expectations, in line with SES political targets, aircraft noise in the vicinity of airports is a very important element to be considered in any airspace organisation, or implementation of an ATM change. Especially since, noise being a local issue, noise-related community opposition is increasingly becoming an obstacle to implementing ATM improvements at European airports. This means that it can cancel out other benefits, including fuel benefits, or even stop implementation altogether. Reinforcing environmental best practice and acknowledging airport neighbours' concerns requires making the consideration of noise impact mandatory. Furthermore, noise assessment should not be limited to the size of a noise footprint, issues of concentration and dispersion of noise, as well as concepts such as ‘new noise’, should also be examined.
Environment/Fuel efficiency focus area: Fuel efficiency is a pre-cursor to the reduction of exhaust emissions resulting from the combustion of jet fuel from flight movements in all phases of flight. Therefore, variation in fuel burnt is considered a good indicator for evaluating the contribution of aviation to global and local emissions. A good example of this is the direct link between fuel burn and the amount of CO2 produced (3.15 times the mass of fuel burnt); CO2 emissions are a major contributor to anthropogenic climate change, therefore their reduction, through improved fuel efficiency, is key to reducing aviation’s environmental impact. Noise focus area: Noise is defined as unwanted sound. Noise impact represents the adverse effect(s) of noise on its recipients (in this case, people living around airports). The noise focus area only covers aircraft noise source; other noise sources around the airport contributing to the background noise are not considered. This KPA focuses on the quantification of the number of people exposed to aircraft noise, using different types of metric, capturing different aspects of noise impact such as the notion of noise exposure (noise energy perceived on the ground), peaks in noise levels (maximum noise level perceived on the ground), and the frequency of ‘noisy’ events (number of flights/operations exceeding a given noise level threshold during a certain time period).
Local air quality focus area: Airport local air quality is a commonly used term to designate the condition of the ambient air to which humans and nature are typically exposed in the vicinity of an airport. In most cases, determining the quality of the air around an airport is based on the estimation of concentration of pollutants. These concentrations are compared with regulations and standards that are established to define acceptable levels of local air quality, including the necessary measures to achieve them. Many issues particular to the local air quality in and around airports are subject to these same regulations. Normally, airport environments comprise a complex mix of emission sources including aircraft, ground support equipment, terminal buildings and ground vehicular traffic (see
ICAO Doc 9889). In the context of SESAR, in most cases only exhaust emissions resulting from jet-fuel consumption can be estimated and only these are considered therefore.
All environmental impact assessments are undertaken at the validation exercise level of each SESAR Solution.
5.6.3. Environment impact assessment process
The SESAR environmental impact assessment process methodology is aligned with the high-level principles and guidance provided by ICAO Doc. 10031 entitled ‘Guidance on environmental assessment of proposed air traffic management operational changes’ [Ref 2].
The SESAR environmental impact assessment (EIA) process relies on five main EIA process steps:
Figure 31 Overview SESAR EAI method
5.6.3.1. EIA Step 1: Identify ATM changes and the scope of potential impacts on the environment
At the first step in an EIA, the Project Manager will identify the ATM changes and the scope of potential impacts on the environment with respect to climate change, noise and local air quality. Identification of ATM changes and the scope of potential impacts on the environment consist of:
Understanding the proposed concept and the scenarios for implementation;
Assessing any change to the impacts on the environment;
Identifying which environmental impacts should be assessed;
Identifying environmental performance drivers, targets and environmental benefits;
The conclusion of this step will be a ‘go/no-go’ decision on whether an environmental assessment will be carried out for this SESAR solution. Determining the ATM change can be achieved by consulting the supporting documentation. From this consultation, if the ATM change has an obvious environmental impact and/or the environmental scope of the ATM change is clear, it is expected that a ‘go’ decision will be taken.
If it is unclear whether the ATM change will have an environmental impact or if the scope of the potential impact (i.e. CO2, noise and/or local air quality) cannot be immediately identified, it may be pertinent to discuss with the environment practitioner and carry out an expert judgement process to determine whether the ATM change will or will not have an environmental impact, and the potential environmental scope of any impact identified. If the outcome of the expert judgement process determines that the ATM change will have an impact even though the level of potential impact or the extent (i.e. positive or negative) cannot be estimated, or the environmental scope is still not clear, it is recommended that a ‘go’ decision be taken. A ‘no-go’ decision should be taken only if the results of the scoping and expert judgement processes identify no apparent environmental impact.
Knowing the phase of flight and the flight levels covered by the operational concept, the Project Manager will determine which environmental impacts should be assessed. The geographical scale of the applicability of the operational concept will determine which regulations are to be considered.
In EIA step 2, the evidence collected in Step 1 is refined. The Project Manager must determine which validation exercises require an EIA and take the ‘go/no-go’ decision for further environmental impact assessment at an exercise level. This decision should be based on the validation strategy and a high level description of each validation exercise. The definition of environmental validation requirement activities consists of:
Developing a detailed understanding of the proposed concept and the alternative scenarios for implementation;
Determining which validation exercises should be subject to an environmental impact assessment and taking the exercise validation ‘go/no-go’ decision;
Identifying the contribution of the operational projects to environmental benefit expectations and targets;
Ensuring that requirements for environmental impact assessment are described.
5.6.3.3. EIA Step 3: Plan environmental impact assessment activities
At the beginning of step 3, the person in charge of conducting the validation exercise (validation exercise Leader) and the person in charge of conducting the environment impact assessment (environment practitioner) should already:
Have a very good understanding of the operational concept to be assessed, and understand the reference scenario and the solution scenario including the underlying matching criteria and system boundaries;
Have become familiar with the SESAR environment reference material (i.e. guidance documents, key performance parameters and tools) and should have followed (if necessary) the related environmental training.
The planning of environmental impact assessment activities consists of:
Understanding the specific operational environment of the validation exercise;
Ensuring assumptions, reference and solution scenarios are compliant with EIA requirements;
Reassessing which environmental impacts should be investigated and formulating hypotheses about the environmental impacts;
Identifying metrics and tools to measure GHG-emission, airport emissions, noise;
Data requirements for an EIA;
Identifying environmental regulation and local constraints when implementing ATM Updates.
The output of step 3, given by the Exercise Leader, is an EIA plan for the exercises describing the hypotheses to be tested, the metrics and tools to be used, the required input variables for these tools and the methodology used for analysing the results. The EIA plan is part of the VALP.
5.6.3.4. EIA Step 4: Conduct the environmental impact assessment exercise
The EIA plan is executed in EIA Step 4. After completing EIA Step 4 the Exercise Leader (EL) should be able to draw conclusions on the environmental impact of the proposed ATM change. The EL should know how to trade-off between the environmental KPA and other KPAs. Discrepancies between the VALP and the exercise, the methodology used to analyse the data, and the results are summarised in a VALR. The environmental impact assessment activities consist of:
Logging any event that might have affected the quality of the data of each run;
Collecting input data-files after each run, performing a quality check and transforming raw data into environmental toolset format;
Carrying out the environmental analysis;
Interpreting the results;
Consider trade-offs;
Writing or updating the ‘environmental impact assessment’ section of the validation report.
5.6.3.5. EIA Step 5: Scale the results up and aggregate with other ATM changes
Step 5 of the EIA process is about collating the results from all individual validation exercises and aggregating them at a Deployment Scenario level. Scaling the results up and aggregate with other ATM changes activities consists of:
Aggregating the validation exercise results;
Scaling the results up to the ECAC level;
Producing the corresponding performance assessment report;
Aggregating all contributions to fuel efficiency, noise and local air quality;
Aggregating results belonging to the same deployment package;
Producing the corresponding deployment package document.
5.6.4. Live/Flight trials considerations
Assessing fuel efficiency and other environmental aspects of flight trials based on aircraft derived data is a challenging task and, at the same time, a very important part of processing the results of the activities. There are many pitfalls in conducting a fuel efficiency analysis that could cause incorrect or directly misleading results if not avoided.
5.6.5. Applicable tools
The following tools are also recommended for other specific purposes:
The EUROCONTROL IMPACT tool implements state-of-the-art methodology for calculating noise footprints, while providing estimations of fuel consumption and emissions released using a common set of aircraft fleet composition and performance assumptions. Its use should be generalised in the SESAR 2020 programme.
Open-ALAQS - for detailed local air quality assessments that might or might not include sources of emissions other than flight movements.
AEM - for fuel and emissions assessments, which would require large amounts of data to be processed, otherwise, it is preferable to use IMPACT.
V-PAT - for vertical flight efficiency analysis and in particular Continuous Climb and Continuous Descent operations.
5.6.6. Reference material
The following reference materials (stored and made available in the SESAR 2020 collaborative platform) provide more a detailed explanation or support the aforementioned practice:
Ref 1. Latest SESAR, Environment reference material, alias, ‘Environmental impact assessment as part of the global SESAR validation
Ref 2. ICAO CAEP – ‘Guidance on environmental assessment of proposed air traffic management operational changes’ document, Doc 10031.
S2020 Programme Execution Framework specific documentation
The 3rd level guidance applicable is as much as possible referred to in this Handbook. The full list of guidance, templates, and any supporting materials can be retrieved from the programme library of the SESAR 2020 collaborative platform.
Appendix B Maturity phases EC Regulation 409/2013 calls for the maturity of ATM functionalities to be demonstrated, inter alia, on the basis of the results of validation activity carried out by the SJU, the status of standardisation and certification processes and an assessment of their interoperability, also in relation to the ICAO Global Air Navigation Plan and relevant ICAO material. Today the SJU has the established principle of managing research maturity using the European operational concept validation methodology (E-OCVM). While E-OCVM will continue to be applied in the Programme in terms of establishing the flow of ideas and results for Programme 2020, additionally the SJU is now required to communicate achievements externally in delivering increased maturity/readiness using technology readiness Level (TRLs). Details of the E-OCVM and TRLs are provided hereafter.
Figure 33 SESAR Maturity Phases (TRL versus E-OCVM)
The level of achievement and consequent maturity at each level is described along with the equivalence in E-OCVM maturity phase (V-levels) and mapped and communicated in terms of TRLs will be used in accordance with the following definition:
Exploratory research (V0 & V1) covers:
Pre-TRL1 Scientific research: Fundamental exploratory scientific research investigating relevant scientific subjects and conducting feasibility studies looking for potential application areas in ATM, concentrating both on out-reach to other disciplines as well as educating within.
TRL 1 Basic principles observed and reported: Exploring the transition from scientific research to applied research by bringing together a wide range of stakeholders to investigate the essential characteristics and behaviours of applications, systems and architectures. Descriptive tools are mathematical formulations or algorithms.
TRL 2 (V1) Technology concept and/or application formulated: Applied research. Theory and scientific principles are focused on very specific application area(s) to perform the analysis to define the concept. Characteristics of the application are described. Analytical tools are developed for simulation or analysis of the application.
Industrial research and validation (possible complement of V1, V2 & V3) covers:
TRL 3 Analytical and experimental critical function and/or characteristic proof-of concept: Proof of concept validation. Active Research and Development (R&D) is initiated with analytical and laboratory studies including verification of technical feasibility using early prototype implementations that are exercised with representative data.
TRL 4 (V2) Component/subsystem validation in laboratory environment: Standalone prototyping implementation and test with integration of technology elements and conducting experiments with full-scale problems or data sets.
TRL 5 System/subsystem/component validation in relevant environment: Thorough testing of prototyping in representative environment. Basic technology elements integrated with reasonably realistic supporting elements. Prototyping implementations conform to target environment and interfaces.
TRL 6 (V3) System/subsystem model or prototyping demonstration in a relevant end-to-end environment (ground or space): Prototyping implementations on full-scale realistic problems using partial integration with existing systems. While limited documentation is available, the Engineering feasibility is fully demonstrated in actual system application.
Very large-scale demonstration (V3+) covers:
TRL 7 System demonstration in an operational environment (ground, airborne or space): System demonstration in operational environment. System is at or near scale of the operational system, with most functions available for demonstration and test and with EASA proof of concept authorisation if necessary. Well integrated with collateral and ancillary systems, although limited documentation available
For the sake of completeness, beyond the scope SESAR 2020, the Industrialisation covers also TRL 8 and TRL 9 explained hereafter. It should be understood that this is not covered in the scope of the SESAR 2020 i.e. Reporting is limited to TRL2-V1, TRL4-V2, TRL6-V3, and where applicable TRL7 for the VLDs.
TRL 8: Actual system completed and ‘mission qualified’ through test and demonstration in an operational environment (ground or space): End of system development. Fully integrated with operational hardware and software systems, most user documentation, training documentation, and maintenance documentation completed. All functionality tested in simulated and operational scenarios. Verification, Validation (V&V) and Demonstration completed.
TRL 9: Actual system ‘mission proven’ through successful mission operations (ground or space): Fully integrated with operational hardware/software systems. Actual system has been thoroughly demonstrated and tested in its operational environment. All documentation completed and successful operational experience with sustaining engineering support in place. SESAR 2020 concentrates on the first seven levels and progresses towards TRL8. The SJU will establish gates at transition points between TRLs in order to ensure maturity achieved and results of investment in research can be reported. Note: The SJU Members individual assessments of TRL status for their own purposes may differ from the SJU communication on overall maturity.
