itpraktikud.eesti.ee · Web viewSubmitted to the EC on XXX/XX/20XX. COMPETITIVENESS AND INNOVATION...

Submitted to the EC on XXX/XX/20XX

COMPETITIVENESS AND INNOVATION FRAMEWORK PROGRAMMEICT Policy Support Programme (ICT PSP)

Project acronym: e-SENSProject full title: Electronic Simple European Networked Services

ICT PSP call identifier: CIP-ICT-PSP-2012-6ICT PSP main theme identifier: CIP-ICT-PSP-2012-6-4.1 Basic Cross Sector Services

Grant agreement n°: 325211

D3.1 Guidelines to the assessment of the sustainability and maturity of building blocks

Deliverable Id : D3.1Deliverable Name : Guidelines to the assessment of the sustainability and maturity of building blocks

Version : v0.7Status : Draft

Dissemination Level : Public Due date of deliverable : M6Actual submission date : Date of submission to EC

Work Package : WP3, Task 3.1Organisation name of lead partner for this deliverable : Ministry of Economic Affairs and Communications of Estonia

Author(s): Jaak TepandiPartner(s) contributing : Task 3.2, WP3, WP5, WP6

Abstract. This e-SENS Task 3.1 deliverable ("Guidelines") is intended for assessing various types of building blocks, including architecture and solution building blocks. The Guidelines are based on the e-SENS Technical Annex v 3.0, the CAMSS methodology, ADMS, suggestions from e-SENS stakeholders, and other sources. The assessment process comprises proposal, consideration, assessment, and recommendation steps. The current document presents a procedure for assessment, criteria to be used in the four assessment steps, organisational aspects of assessment, and research update on already existing assessments.

HISTORY

Version Date Changes made Modified by

D3.1 Guidelines to the assessment of the sustainability and maturity of building blocks 1

0.2 30.04.2013 D3.1 Guideline principles Jaak Tepandi

0.3 13.05.2013 Guideline principles extended and updated according to comments from Freek van Krevel,

Jack Verhoosel, Marijke Salters, and Uuno Vallner

Jaak Tepandi

0.5 28.05.2013 D3.1 draft Guideline. Elaborated criteria,

extended and updated according to suggestions from the WP3 Brussels meeting on 17.05 and

teleconference on 28.05

Jaak Tepandi

0.7 31.05.2013 Guideline extended and updated throughout the text according to suggestions from T3.2 and

other WP3 participants. New presentation of the assessment process. Assessment criteria

from different sources merged. New template for the report.

Jaak Tepandi


TABLE OF CONTENTS

EXECUTIVE SUMMARY.................................................................................................................................. 4

1INTRODUCTION .......................................................................................................................................... 5

1.1TIMELINE OF T3.1.................................................................................................................................................51.2CAMSS..............................................................................................................................................................51.3ADMS................................................................................................................................................................5

2THE PROCEDURE FOR THE ASSESSMENT OF EUROPEAN BUILDING BLOCKS ..................................................6

3PROPOSAL AND CONSIDERATION STEPS: DOCUMENTATION OF FORMAT.....................................................7

3.1THE OBJECTS TO BE ASSESSED .................................................................................................................................73.2THE PROPOSAL STEP...............................................................................................................................................83.3THE CONSIDERATION STEP.......................................................................................................................................8

4THE ASSESSMENT STEP.............................................................................................................................. 10

4.1INTEGRATION OF ASSESSMENT CRITERIA FROM VARIOUS SOURCES.................................................................................104.2STANDARDISATION CRITERIA...................................................................................................................................10

4.2.1Maturity.......................................................................................................................................................104.2.2Openness.....................................................................................................................................................114.2.3Intellectual property rights..........................................................................................................................114.2.4Life cycle, maintenance, service levels, security...........................................................................................11

4.3CRITERIA FOR ALIGNMENT WITH EXISTING POLICY FRAMEWORKS ..................................................................................194.3.1Basic alignment criteria...............................................................................................................................194.3.2Applicability ................................................................................................................................................194.3.3Potential......................................................................................................................................................20

4.4BUSINESS NEED CRITERIA......................................................................................................................................204.4.1Basic business need criteria.........................................................................................................................214.4.2Market support............................................................................................................................................21

5THE RECOMMENDATION STEP: CRITERIA AND CLASSIFICATION..................................................................22

5.1RECOMMENDATION CRITERIA.................................................................................................................................22

5.2PROPOSED CLASSIFICATION...................................................................................................................................22

6ORGANIZATIONAL ASPECTS OF ASSESSMENT ............................................................................................23

6.1A QUESTIONNAIRE FORM FOR THE ASSESSMENT OF EUROPEAN BUILDING BLOCKS............................................................236.2DEALING WITH CURRENT LSPS THAT DO NOT COMPLY TO THE CRITERIA UPFRONT.............................................................23


7RESEARCH UPDATE AND INPUT FOR THE ANALYSIS....................................................................................24

8APPENDIXES.............................................................................................................................................. 25

8.1ABBREVIATIONS AND GLOSSARY..............................................................................................................................258.2REFERENCES.......................................................................................................................................................25

8.3THE CAMSS ASSESSMENT CRITERIA........................................................................................................................26


1 EXECUTIVE SUMMARY

The aim of the e-SENS project is to consolidate the building blocks of the existing Large Scale Pilots,

focusing strongly on the core building blocks such as eID, eDocuments, e-Delivery, semantics and e-Signatures.

The e-SENS Work Package 3 aims to present proposals for sustainable building blocks that have emerged from the Large Scale Pilots relevant to the e-SENS project. This proposal should support

competitiveness, openness for future technologies, and interoperability.

Task T3.1 of the Work Package 3 supports presentation and assessment of building blocks. Its main

deliverable ‘Guidelines to the assessment of the sustainability and maturity of building blocks’ ("Guidelines") proposes a documentation of format and defining criteria for the maturity and

sustainability assessment of building blocks in close cooperation with Task 3.2, WP3, WP 5, WP 6. Task 3.1 will be active from M1 to M6.

The current version of the Guidelines builds upon existing work carried out in the European Interoperability Framework and is based on the e-SENS Technical Annex v 3.0, the CAMSS

methodology (Final Draft Revision of CAMSS, Version 1.0, March 2012), discussions with e-SENS project participants, and various other e-SENS project materials. As the CAMSS is relies on ADMS, the

Guidelines also make use of ADMS.

Following the e-SENS Technical Annex v 3.0 and the CAMSS methodology, the current document uses

the assessment process that comprises proposal, consideration, assessment, and recommendation steps as follows.

1. In the proposal step, an object of assessment (for example, an architecture or solution building block) is provided to Task 3.2 by WP6 in cooperation with WP5 or other relevant

stakeholders. The objects of assessment are classified under broader categories of Architecture Building Blocks and Solution Building Blocks. On the highest level, both

comprise core building blocks that may include eID, eDocuments, e-Signatures, e-Delivery, Privacy, Semantics, and others. The object of assessment is provided using the proposal

criteria. 2. In the consideration step, consideration criteria are used before the actual assessment, to

validate information received and relevance of the proposal.

3. In the assessment step, the criteria used are categorised under standardisation, alignment with existing policy frameworks, and business need.


4. In the recommendation step, recommendation criteria are applied to conclude with a classification (Discarded, Observed, Accepted, Recommended, Mandatory) of the object of

assessment. This classification will be reported back to WP6 for using by other Work Packages.

The Guidelines, including the sets of criteria provided, are intended for assessing various types of building blocks (including architecture and solution building blocks). During a specific assessment,

relevant criteria are selected depending on the object presented for evaluation.

The document starts with the background given in the introduction. The second section presents a

procedure for assessment, the third - an overview of the objects to be assessed and the criteria for the proposal and consideration steps. Section 4 is devoted to the assessment step and criteria,

section 5 - to the recommendation step. Sections 6 and 7 deal with organizational aspects of assessment and with research update on already existing assessments. Each section may include

subsections, questions, and comments (grouped under 4th level Headings) to be removed from the final text.

In case the terminology (eg, notion of a building block) depends on deliverables of other WPs, it is referred to these deliverables and may change with the feedback received. For presenting the

assessment guidelines, the notion of an object of assessment (an object, for example, an architecture or solution building block, submitted for assessment in the proposal step, to be assessed through the

consideration, assessment, and recommendation steps) is utilized.

The current version of the Guidelines has taken into account comments on the earlier versions from

Freek van Krevel, Jack Verhoosel, Marijke Salters, Uuno Vallner, Bertrand Grégoire, Cagatay Karabat, and other e-SENS project participants.

The current version of the Guidelines is still an early draft and work is in progress, in particular concerning the criteria and the research update.

Please address feedback to Jaak Tepandi ([email protected]), Uuno Vallner ([email protected]).


mailto:[email protected]

mailto:[email protected]

1 INTRODUCTION

The e-SENS project will consolidate the building blocks of the existing Large Scale Pilots, focusing

strongly on the core building blocks such as eID, eDocuments, e-Delivery, semantics and e-Signatures.

The e-SENS Work Package 3 aims to present proposals for sustainable building blocks that have emerged from the Large Scale Pilots relevant to the e-SENS project. This proposal should support

competitiveness, openness for future technologies, and interoperability.

Presentation and assessment of building blocks are supported by Task T3.1 of the Work Package 3.

The main deliverable ‘Guidelines to the assessment of the sustainability and maturity of building blocks’ ("Guidelines") of T3.1 proposes a documentation of format and defining criteria for the

maturity and sustainability assessment of building blocks in close cooperation with Task 3.2 and Work Packages 5 and 6.

The current version of the Guidelines builds upon existing work carried out in the European Interoperability Framework and is based on the e-SENS Technical Annex v 3.0, the CAMSS

methodology (Final Draft Revision of CAMSS, Version 1.0, March 2012), discussions with e-SENS project participants, and various other e-SENS project materials. As the CAMSS is relies on ADMS, the

Guidelines also make use of ADMS.

The Guidelines are intended for assessing various types of building blocks (including architecture and

solution building blocks).

The document is structured as follows. The second section presents a procedure for assessment, the

third - an overview of the objects to be assessed and the criteria for the proposal and consideration steps. Section 4 is devoted to the assessment step and of criteria, section 5 - to the recommendation

step. Sections 6 and 7 deal with organizational aspects of assessment and with research update on already existing assessments. Each section may include subsections, questions, and comments

(grouped under 4th level Headings) to be removed from the final text.

1.1 Timeline of T3.1

According to the Technical Annex it is expected that the much of the input for T3.1 will come from T3.2, WP5, WP6, and other WPs, which may have later delivery dates than T3.1 (month 6). Therefore

T3.1 needs to communicate with these tasks and WPs before they finish their work.


The task T3.1 will be active from M1 to M6. The result of this task is the Guidelines deliverable. A draft version will be ready in month 4 and the definitive version should be ready in month 6. The

indicative timeline includes the following deadlines and actions.

Deadline / duration Activities Deliverables1.04.2013 Start of the task T3.11.04 -15.05.2013 Development of main principles for

the Guidelines. Preliminary discussions with participants.

Main principles for the Guidelines

17.05. 2013 Discussion of the main principles with participants (Planned in Brussels)

Adjustment of main principles

May-June 2013 Preparation and discussion of the preliminary draft of the Guidelines (planned via e-mail and / or teleconference)

Preliminary draft of the Guidelines

Beginning of July 2013

Discussion of the preliminary draft of the Guidelines (planned in Tallinn)

Adjustment of the preliminary draft

01.07-31.07. 2013 Preparation and discussion of the draft version of the Guidelines for discussion

Draft version of the Guidelines

01.08-30.09. 2013 Discussion of the draft version Final version of the Guidelines

1.2 CAMSS

In this document, the CAMSS (Final Draft Revision of CAMSS. Version 1.0, March 2012) scenarios

"Assessment scenario 2 – An assessment of a formal specification for adoption by public administrations" and "Assessment scenario 3 – An assessment and selection of formal specifications

for specific business needs and requirements" are taken into account.

According to assessment scenario 2, a formal specification is assessed in order to evaluate and

provide a recommendation on the possible adoption of the formal specification by the public administrations. This assessment scenario can be triggered by a public administration or related

external stakeholders. The outcome for this assessment scenario will be the adoption of a certain formal specification by the public administrations.

According to assessment scenario 3, a proposed set of formal specifications are assessed and evaluated in order to select and adopt the most relevant formal specification for specific business

needs and requirements. If a certain business need arises and requires the adoption of a relevant formal specification, the business need should be examined to list the relevant requirements, and

based upon those requirements, a selection of relevant formal specifications may be established. The


outcome for this assessment scenario will be the selection and adoption of relevant formal specifications for the specific business needs and requirements.

The basic difference between scenarios 2 and 3 is availability of specific business needs and requirements in scenario 3. In case such needs and requirements are not available, preference is

given to scenario 2.

1.3 ADMS

The CAMSS make use of the ADMS (Asset Description Metadata Schema). Were possible, the current Guidelines use ADMS as well. In particular, the controlled vocabularies used during the assessment

process are based on ADMS (Asset Description Metadata Schema. Specification Version 1.00. Release date 18/04/2012).


2 THE PROCEDURE FOR THE ASSESSMENT OF EUROPEAN BUILDING BLOCKS

According to the e-SENS Technical Annex (p 60), T3.1 will develop a procedure for the assessment of

building blocks.

CAMSS scenarios 2 and 3, especially the assessment step (CAMSS, section 2.4) are considered for a

basis for organizational aspects of assessment, together with relevant frameworks such as BOMOS2i, ISO/IEC 152888, ISO/IEC 25000 series standards, and other sources as appropriate.

Following the e-SENS Technical Annex and the CAMSS methodology, the current document uses the assessment procedure that comprises proposal, consideration, assessment, and recommendation

steps (see Figure 1).

Figure 1. The procedure for assessment with main stakeholders

1. In the proposal step, an object of assessment (for example, an architecture or solution

building block) is provided to Task 3.2 by WP6 in cooperation with WP5 or other relevant stakeholders according to the Technical Annex, pp 21-24, 81, and 112. The object of

assessment is provided using the proposal criteria for presenting the object of assessment. The proposal criteria provide general information about the proposed object of assessment,

its status, items provided for assessment, and other.


2. In the consideration step, consideration criteria are used before the actual assessment, to validate information received and relevance of the proposal.

3. In the assessment step, the criteria used are categorised under standardisation, alignment

with existing policy frameworks, and business need. 4. In the recommendation step, recommendation criteria are applied to conclude with a

classification (Discarded, Observed, Accepted, Recommended, Mandatory) of the object of assessment. This classification will be reported back to WP6 for using by other Work

Packages.

The objects of assessment are classified under broader categories of Architecture Building Blocks and

Solution Building Blocks. On the highest level, both comprise core building blocks that may include eID, eDocuments, e-Signatures, e-Delivery, Privacy, Semantics, and others.


3 PROPOSAL AND CONSIDERATION STEPS: DOCUMENTATION OF FORMAT

The documentation of formats helps WP6 to provide building blocks for assessment and provides

consideration criteria for Task T3.2.

The documentation of formats is based on CAMSS sections 3.1 and 3.2 and, if applicable, takes into

account relevant standardization and asset description initiatives such as the Regulation on European standardisation, the Asset Description Metadata Schema (ADMS), the European Federated

Interoperability Repository (EFIR), the Joinup platform, and others.

Annex II of the Regulation on European standardisation comprises requirements for the identification

of ICT technical specifications. The Asset Description Metadata Schema (ADMS) is a vocabulary to describe interoperability assets. The EFIR contains interoperability assets of the Member States. The

Joinup repository currently covers the documentation of semantic and open-source software interoperability assets.

3.1 The objects to be assessed

In the proposal step, an object of assessment is provided to Task 3.2 by WP6 in cooperation with

WP5 or other relevant stakeholders. The objects of assessment are classified under broader categories of Architecture Building Blocks and Solution Building Blocks. On the highest level, both

comprise core building blocks.

Since the main topic for the current Guidelines is documentation of format and defining criteria for

assessment, only selected core building block examples are mentioned below to improve understanding of the assessment issues. Further selection of building blocks will be based on input

from WP6 during the assessment phase.

The core building blocks may include the following.

eIdentity eDocuments, Semantics

e-Signatures e-Delivery

Privacy

The Architecture Building Blocks provided for assessment by WP6 may include but are not

constrained to the following types.


Architecture strategy, frameworks, and methodologies Service contracts, service level agreements, policies

Documentation and guidelines, including software and deployment descriptions Specification of interfaces of components considering the building block as black box

Specification of the functionality of the building block Detailed design of the building block and its internal components

Other types

The Solution Building Blocks provided for assessment by WP6 may include but are not constrained to

the following types.

Implementation of the building block in terms of source files and its documentation

Implementation of the building block in terms of executable files Other types

These types of objects may occur in combination.

The complete list of building blocks and objects for assessment will be provided by WP6.

To deal with different types of building blocks, the current guideline proposes a wide set of criteria that should be applied selectively as follows.

Some criteria may be applicable to a specific type of building blocks. The criteria may be mandatory or optional.

There can be assessment levels defined for the criteria.

3.2 The proposal step

As input to the proposal step, an object of assessment is provided by WP6 in cooperation with WP5 or other stakeholders. The object of assessment is provided using the proposal criteria. Following

CAMSS (Section 3.1) and the ADMS specification v 0.8, the main categories of criteria for the documentation of format during the proposal step comprise asset description and relationship

(status) categories and criteria according to the following table.

Nr. Category Description Nr. Criteria Description1 Asset

description Providing the general information for the proposed object of assessment

P.1 Name Name of the asset


P.2 Date ofcreation

Creation date of this version of the asset

P.3 Date of last modification

Date of latest update of asset

P.4 Description Descriptive text for the asset P.5 ID URI for the assetP.6 Keyword Word or phrase to describe the

assetP.7 Alternative

nameAlternative name for the asset. Note: this information may be used to provide additional access points, e.g. allowing indexing of any acronyms, nicknames, shorthand notations or other identifying information under which a user might expect to find the asset

P.8 Version Version number or other designation of the asset

2 Relationship Providing the information on the status of the specific object of assessment being proposed

P.9 Asset type Classification of an asset according to a controlled vocabulary, e.g. code list, metadata schema

P.10 Current version Current or latest version of the asset

P.11 Documentation Document that further describes an asset or give guidelines for its use

P.12 Domain Government sector that an asset or repository applies to, e.g. “law” or“environment” according to a controlled vocabulary

P.13 Interoperability level

Level according to the European Interoperability Framework (EIF 2.0) for which an asset is relevant

P.14 Language Language of an asset if its contains textual information, e.g. the language of the terms in a controlledvocabulary or the language that a specification is written in

P.15 Previous version

Older version of the asset

P.16 Publisher Organisation responsible for a repository, asset or release

P.17 Release Implementation of the asset in a


particular formatP.18 Spatial

coverageGeographic region or jurisdiction to which the asset applies

P.19 Subject Theme or subject of an asset, e.g. “elections” or “immigration” according to a general or domain specific controlled vocabulary

P.20 Status Indication of the maturity of an asset or release

Further relevant criteria from ADMS and other sources can be included under this main classification. The actual items delivered with each object of assessment will be defined by deliverables of WP6.

According to the Technical Annex (p 60), the documentation of formats given in the Guidelines helps WP6 to provide building blocks for assessment. An alternative solution is that WP6 defines the

proposal criteria and agrees them with WP3.

3.3 The consideration step

In the consideration step, consideration criteria are used before the actual assessment, to validate

information received and relevance of the proposal. Following CAMSS (Section 3.2), the main

categories of criteria for the documentation of format during the consideration step are correctness, relevance, legislation and regulation, together with their corresponding criteria presented in the

table below.

These criteria check the validity of the information and the relevance of the proposed formal

specification. The criteria are defined as YES/NO questions, and the questions should be answered with YES in order to consider the formal specification for further assessment. The criteria could be

adapted by the assessment panel to fit their specific needs and requirements.

Nr. Category Description Nr. Criteria1 Correctness The proposal for the assessment is provided

correctly and the necessary documentation is provided

C.1 Is the proposal information correctly provided?

C.2 Is the complete documentation provided for performing the assessment?

2 Relevance The proposal for the assessment falls within the relevant areas of the public administration

C.3 Is the functional area of application for the object of assessment addressing e-SENS Building Blocks?


C.4 Is the object of assessment meeting the business needs?

3 Legislation and regulation

The proposal for the assessment does not impede any legislative or regulatory requirements.

C.5 Is the object of assessment not hampering the regulatory requirements for the area of application?

Relevant additional criteria from ADMS and other sources can be included under this main

classification.


4 THE ASSESSMENT STEP

This section proposes a method for integration of assessment criteria from different sources and

presents three sets of criteria based on the e-SENS Technical Annex classification.

4.1 Integration of assessment criteria from various sources

The assessment criteria originate from various sources. First, a number of sustainability assessment criteria have been provided in the e-SENS Technical Annex (pp 60-62) and in discussions with the e-

SENS project participants. Second, CAMSS (Section 3.3) presents the following main categories of criteria for the assessment step: applicability, maturity, openness, intellectual property rights, market

support, and potential. Third, there are other frameworks / standards (such as ADMS, D6.1, ISO/IEC 152888, ISO/IEC 25000 series standards, or other) potentially suitable for assessment; in particular,

as the CAMSS criteria mainly address specifications, they have to be complemented for solution building blocks, using ADMS.SW or other relevant sources.

There are several ways for integration of these criteria.

1. Taking the Technical Annex as the basis and integrating CAMSS criteria into the criteria of the

Technical Annex.2. Taking the CAMSS criteria as the basis and merging the criteria form the Technical Annex into

CAMSS.3. Taking some other frameworks / standards (such as ADMS, D6.1, ISO/IEC 152888, ISO/IEC

25000 series standards, or other) as the basis and integrating Technical Annex and CAMSS criteria into this framework.

This section presents a solution based mainly on the first option - integrated assessment criteria based on the e-SENS Technical Annex, the CAMSS assessment criteria, and other sources, taking into

account also other relevant e-SENS materials, such as deliverable D6.1.

According to the Technical Annex, analysis of sustainability of the building blocks will take into

account standards, policy frameworks, and business needs. Input for the analysis will be provided by WP5, WP6, and other relevant sources. The following sub-sections are devoted to these sets of

criteria.


To preserve compatibility with CAMSS, the CAMSS categories are fully integrated into the assessment criteria based on the e-SENS Technical Annex and complemented where necessary, clearly indicating

the source for the additional criteria.

The sets of criteria provided in this section are quite extensive and suitable to perform assessment of

different building blocks. During the specific assessment relevant criteria are selected depending on the object presented for evaluation.

4.2 Standardisation criteria

The set of standardisation criteria include technical, organisational and semantic criteria that can be

used to assess the maturity for standardizing the building blocks. Potential criteria include the following.

Change management and maintenance, licensing, common specifications of components, openness, support and manuals, security features, service levels (availability of the building

block, emergency helpdesk) as well as the deployment and running of these components in an organisation’s IT infrastructure.

Life cycle workflow over the project lifetime (development, revisions, updates, work in progress, and incremental version releases).

Maintaining building blocks beyond the project’s lifetime and who will be responsible for the different parts.

Interoperability between the LSPs and Building Blocks in Member States.

The following CAMSS categories of criteria may be included in this section.

Maturity Openness

Intellectual property rights

Relevant criteria from international standards, ISO/IEC 12207:2008 "Systems and software

engineering. Software life cycle processes" and ISO/IEC 25000:2005 "Software Engineering. Software product Quality Requirements and Evaluation (SQuaRE)" have been taken into account.

The set of standardisation criteria can be assessed within WP3 in close collaboration with WP6, WP5, the industry partners, and other relevant stakeholders.


4.2.1 Maturity

The following table presents maturity criteria based on CAMSS.

Nr Cate-gory

Description Nr Sub-Cate-gory

Description Nr Criteria

2 Ma-turity

An object of assessment should in itself be mature enough for adoption by public administrations. This category addresses the development status, the quality, guidelines and stability of the object of assessment.

2.1 Deve-lop-ment status

For the ‘development status’, the current development status of the object of assessment in the development cycle is addressed.

A.9 Has the object of assessment been sufficiently developed and in existence for a sufficient period to overcome most of its initial problems?

2.2 Quali-ty

For ‘quality’, the level of detail in the object of assessment and the conformance of implementations is addressed.

A.10 Are there existing or planned mechanisms to assess conformity of the implementations of the object of assessment (e.g. conformity tests, certifications)?

A.11 Has the object of assessment sufficient detail, consistency and completeness for the use and development of products?

2.3 Guide-lines

For the ‘guidelines’, the existence of implementation guidelines or reference implementations is addressed.

A.12 Does the object of assessment provide available implementation guidelines and documentation for the implementation of products?

A.13 Does the object of assessment provide a reference (or open source) implementation?

2.4 Stabi-lity

For ‘stability’, the level of change to the object of assessment and the stability of underlying technologies is addressed.

A.14 Does the object of assessment address backward compatibility with previous versions?

A.15 Have the underlying technologies for implementing the object of assessment been proven, stable and clearly defined?


4.2.2 Openness

The following table presents openness criteria based on CAMSS.

Nr Cate-gory



3 Openness

An object of assessment should be sufficiently open and available to be relevant for adoption by public administrations. This category addresses the openness of the organisation maintaining the object of assessment and decision-making process, and the openness of the documentation and accessibility of the object of assessment.

3.1 Orga-nisa-tion

For the ‘openness’ of the organisation, the level of openness for participating in the organisation maintaining the object of assessment is addressed.

A.16 Is information on the terms and policies for the establishment and operation of the organisation maintaining the object of assessment publicly available?

A.17 Is participation in the creation process of the object of assessment open to all relevant stakeholders (e.g. organisations, companies or individuals)?

3.2 Pro-cess

For the ‘process’, the level of openness regarding the development and decision-making process for the object of assessment is addressed.

A.18 Is information on the standardisation process publicly available?

A.19 Information on the decision making process for approving objects of assessment is publicly available?

A.20 Are the objects of assessment approved in a decision making process which aims at reaching consensus?

A.21 Are the objects of assessment reviewed using a formal review process with all relevant external stakeholders (e.g. public consultation)?

A.22 All relevant stakeholders can formally appeal or raise objections to the development and approval of objects of assessment?


Nr Cate-gory



3.3 Docu-men-tation

For the openness of the ‘documentation’, the accessibility and availability of the documentation of the object of assessment is addressed.

A.23 Relevant documentation of the development and approval process of objects of assessment is publicly available (e.g. preliminary results, committee meeting notes)?

A.24 Is the documentation of the object of assessment publicly available for implementation and use on reasonable terms?

4.2.3 Intellectual property rights

The following table presents intellectual property rights criteria based on CAMSS.

Nr Cate-gory



4 Intel-lec-tual pro-perty rights

An object of assessment should be licensed on (F)RAND terms or even on a royalty-free basis in a way that allows implementation in different products. This category addresses the availability of the documentation on the IPR and the licences for the implementation of the object of assessment.

4.1 IPR Docu-men-tation

For the ‘documentation of the intellectual property rights’, the availability of the information concerning the ownership rights of the object of assessment is addressed.

A.25 Is the documentation of the IPR for objects of assessment publicly available?

4.2 Licen-ces

For the ‘licences’ within the intellectual property rights, a (fair) reasonable and non-discriminatory ((F)RAND) or even royalty-free basis is addressed for the use and implementation of the object of assessment.

A.26 Is the object of assessment licensed on a (F)RAND basis?

A.27 Is the object of assessment licensed on a royalty-free basis?

4.2.4 Life cycle, maintenance, service levels, security

Depending on the object of assessment, criteria presented in the following tables may be relevant

for assessment. These criteria are selected according to the standardisation criteria in the Technical


Annex and are based on international standard ISO/IEC 12207:2008 "Systems and software engineering. Software life cycle processes", as well as on the ISO/IEC 20000, ISO/IEC 25000, and

ISO/IEC 27000 series of standards.

Criteria referring to interoperability are presented in the next section.

The range of possible criteria presented in the above standards is extremely wide. Only selected criteria most relevant to the requirements presented in the Technical Annex are presented below.

This selection may be expanded or restricted during a specific assessment.

Nr Cate-gory



Life Cycle ma-na-ge-ment

The life cycle management process provides life cycle policies, processes, and procedures.

There should exist a life cycle management process.

Is an organisation available for providing life cycle policies, processes, and procedures for the object of assessment?

Is life cycle workflow over the project lifetime (development, revisions, updates, work in progress, and incremental version releases) established?

Mainte-nan-ce

The maintenance process provides cost-effective support to the objects of assessment during their life-cycle, including change management.

Main-te-nance imp-le-men-tation

There should be plans and procedures for conducting the maintenance activities.

Does a maintaining organisation exist?

Has the maintaining organisation developed, documented, and executed plans and procedures for conducting the maintenance activities?

Prob-lem ana-lysis

The problem reports or modification requests should be analyzed for their impact.

Does the maintainer have procedures for analyzing the problem reports or modification requests for their impacts on the organization, the existing system, and its interfaces?


Nr Cate-gory



Modi-fica-tion imp-le-men-tation

It should be determined and documented which software items need to be modified.

Does the maintainer have procedures for determining which software units and versions need to be modified?

Migration

Migration of a system or software product (including data) should be planned, documented, and performed.

Are there procedures for developing, documenting, and executing migration plans, including the system, data, and users?

Disposal

Ending the existence of an object of assessment should be planned, documented, and performed.

Are there procedures for developing, documenting, and executing disposal plans for objects of assessment?

Ser-vice le-vels

The services related to the object of assessment should be agreed with the customers.

SLA If applicable, there should be service level agreements relating to the availability of the object of assessment.

Do SLAs relating to the availability of the object of assessment exist?

If applicable, there should exist emergency helpdesk for the object of assessment.

Is there an emergency helpdesk for the object of assessment?

Secu-rity

Systems, data, and resources should be protected from accidental or malicious acts.

ISMS The maintainer should have an information security management system.

Does the maintainer have a system based on a business risk approach, to establish, implement, operate, monitor, review, maintain and improve information security?

Iden-tifica-tion

Information security requirements should be understood.

Has the maintainer analysed and understood the information security requirements related to the object of assessment?

Risks Information security risks should be assessed.

Has the maintainer assessed the information security risks related to the object of assessment?


Nr Cate-gory



Cont-rols

Information security controls should be selected and implemented.

Has the maintainer selected and implemented information security controls related to the object of assessment?

Moni-tor

The effectiveness of the ISMS should be monitored, maintained, and improved.

Is the maintainer monitoring, maintaining, and improving the effectiveness of the ISMS?

4.3 Criteria for alignment with existing policy frameworks

This set of criteria addresses alignment of the individual building blocks with existing policy frameworks. Potential criteria include the following.

Criteria resulting from EIA and EIF. In particular, the criteria present in the EIF/EIA and missing in the current set of criteria will be considered for inclusion. Criteria include the EIF

conceptual model for public services (presented also in Technical Annex p. 36), as well as coverage of A2A services. Examples: user authorization and aggregation of services.

Alignment with the EIF and EIA. Requirements that the proposed solutions are compliant with the EU legal framework on

data protection and legislation on electronic signatures. Alignment with national frameworks of the participating countries.

Potential incompatibilities between Member States. Maintenance of the legal validity of information exchanged across borders.

Adherence to the data protection legislation in both originating and receiving countries.

The following CAMSS categories of criteria may be included in this section.

Applicability Potential

Alignment with existing policy frameworks will be assessed in collaboration with WP4, the e-SENS legal expertise centre.


4.3.1 Basic alignment criteria

The following table presents basic criteria for alignment with existing policy frameworks based on

requirements from the Technical Annex.

Nr Cate-gory



Interope-rabi-lity

The LSPs and Building Blocks in Member States should be interoperable.

EIA The object of assessment should confirm to the European Interoperability Architecture.

Are there any disagreements between the object of assessment and the EIA?

EIF The object of assessment should confirm to the European Interoperability Framework.

Are there any disagreements between the object of assessment and the EIF?

A2A ser-vices

The object of assessment should support A2A services, if applicable.

If applicable, does the object of assessment support A2A services, eg: user authorization and aggregation of services?

Com-pli-ance

The proposed solutions should be compliant with the EU legal framework on data protection and legislation on electronic signatures.

Data pro-tec-tion

The proposed solutions should be compliant with the EU legal framework on data protection.

Are the proposed solutions compliant with the EU legal framework on data protection?

Ele-ctro-nic signa-tures

The proposed solutions should be compliant with the EU legislation on electronic signatures.

Are the proposed solutions compliant with the EU legislation on electronic signatures?

Me-mber Sta-tes

Alignment with national frameworks of the participating countries and avoiding potential incompatibilities between Member States.

Nati-onal fra-me-works

Alignment with national frameworks of the participating countries.

Is the object of assessment aligned with national frameworks of the participating countries?

In-com-pati-bili-ties

Avoiding potential incompatibilities between Member States.

Are potential incompatibilities between Member States avoided?


Nr Cate-gory



Legal The legal validity of information exchanged must be maintained across borders.

Infor-ma-tion

Maintenance of the legal validity of information exchanged across borders.

Is the legal validity of information exchanged maintained across borders?

Pro-tec-tion

Data protection legislation in both originating and receiving countriesmust be respected.

Data pro-tec-tion

Adherence to the data protection legislation in both originating and receiving countries.

Is data protection legislation in both originating and receiving countries respected?

4.3.2 Applicability

The following table presents applicability criteria based on CAMSS.

1 Appli-ca-bility

An object of assessment should be usable and easy implementable in different products to be relevant for adoption by public administrations. This category addresses the definition of functional scope and area of application, the possible reusability in other areas, the possible alternative specifications, the compatibility and dependency on other specifications or technologies.

1.1 Area of appli-cation

For the ‘area of application’, the functionalities and intended use of the object of assessment are addressed within the context of interoperability and eGovernment.

A.1 Does the object of assessment address and facilitate interoperability between public administrations?

A.2 Does the object of assessment address and facilitate the development of eGovernment?

1.2 Requ-ire-ments

For the ‘requirements’, the functional and nonfunctional requirements for using and implementing the object of assessment are addressed. This criterion is related to the use of assessment scenario 3

A.3 Are the functional and nonfunctional requirements for the use and implementation of the object of assessment clearly defined?


1.3 Re-usabi-lity

For ‘reusability’, the level of reusability of the object of assessment in the same or other areas of application is addressed.

A.4 Is the object of assessment applicable and extensible for implementations in different domains?

1.4 Alter-nati-ves

For the ‘alternatives’, the degree to which the object of assessment adds value compared to alternative objects of assessment in the same area of application is addressed.

A.5 Does the object of assessment provide sufficient added value compared to alternative objects of assessment in the same area of application?

1.5 Com-pati-bility

For ‘compatibility’, the compatibility of the object of assessment with other objects of assessment in the same area of application is addressed.

A.6 Is the object of assessment largely compatible with related (not alternative) objects of assessment in the same area of application?

1.6 De-pen-den-cies

‘Dependencies’ addresses the degree of independence of the object of assessment from specific vendor products, platforms or technologies.

A.7 Is the object of assessment largely independent from specific vendor products?

A.8 Is the object of assessment largely independent from specific platforms or technologies?


4.3.3 Potential

The following table presents criteria for potential based on CAMSS.

Nr Cate-gory



6 Po-ten-tial

An object of assessment should have sufficient and positive future consequences, evolution and impact for being adopted by public administrations. This category addresses the consequences and impact of using or adopting the object of assessment, the advantages and risks, the maintenance and possible future developments.

6.1 Im-pact

For the ‘impact’, the minimisation of the consequences of using and adopting the object of assessment is addressed. The consequences can be evaluated and described in terms of different aspects.

A.33 Is there evidence that the adoption of the object of assessment positively impacts organisational processes?

A.34 Is there evidence that the adoption of the object of assessment positively impacts the migration of current systems?

A.35 Is there evidence that the adoption of the object of assessment positively impacts the environment?

A.36 Is there evidence that the adoption of the object of assessment positively impacts the financial costs?

A.37 Is there evidence that the adoption of the object of assessment positively impacts the security?

A.38 Is there evidence that the adoption of the object of assessment positively impacts the privacy?

A.39 Is there evidence that the adoption of the object of assessment positively impacts the administrative burden?

A.40 Is there evidence that the adoption of the object of assessment positively impacts the disability support?

6.2 Risks For the ‘risks’, the level of uncertainty is addressed for using and adopting the object of assessment

A.41 Are the risks related to the adoption of the object of assessment acceptable?


Nr Cate-gory



6.3 Main-te-nance and future deve-lop-ments

For the ‘maintenance’ and future developments, the support and the planned or existing actions to maintain, improve and develop the object of assessment in the long term are addressed.

A.42 Does the object of assessment have a defined maintenance organisation?

A.43 Does the maintenance organisation for the object of assessment have sufficient finances and resources for the long term?

A.44 Does the object of assessment have a defined maintenance and support process?

A.45 Does the object of assessment have a defined policy for version management?

4.4 Business need criteria

The set of business need criteria addresses the market validation and user aspects of the individual building blocks. Potential criteria include the following.

Need for the building block by end users. For example, potential change in the quality of the service delivered to the citizen/business by the administration before and after adopting the

building block. Opportunities for software/service providers to put the building block into use. For example,

availability of a commercially-oriented, robust Business Plan for investment, built upon an underlying ‘commercially sustainable’ business model. A business case should also take into

account how an object of assessment will help public partners in achieving their missions. Relevance of having the same components integrated as European (shared) building blocks

across different Use Cases and their usefulness in the development of eGovernment cross-border services.

Potential of the building block to be adopted by the market and be used in cross-border eGovernment services.

Where applicable the costs and benefits of adopting the building block, including the assessment of the Return on Investment.

Possibility for a broader geographic and sector usage.


The following CAMSS category of criteria may be included in this section.

Market support

This set of criteria will be assessed by involving stakeholders outside the project, e.g. by using questionnaires or other means to get the opinion of important end users and software/service

providers.

4.4.1 Basic business need criteria

The following table is based on the basic business need criteria based on the Technical Annex.

Nr Cate-gory



Busi-ness need

Need for the object of assessment by end users.

Chan-ge

Potential change in the quality of the service delivered to the citizen/business by the administration before and after adopting the object of assessment.

Are positive changes in the quality of the service delivered to the citizen/business by the administration before and after adopting the object of assessment foreseen?

Usa-ge

Opportunities for software/service providers to put the object of assessment into use.

Do opportunities exist for software/service providers to put the object of assessment into use?

Busi-ness plan

Availability of a commercially-oriented, robust Business Plan for investment, built upon an underlying ‘commercially sustainable’ business model.

Is the Business Plan for investment built upon an underlying ‘commercially sustainable’ business model?

Busi-ness case

A business case should take into account how an object of assessment will help public partners in achieving their missions.

Does the business case takes into account how an object of assessment will help public partners in achieving their missions?


Nr Cate-gory



Sha-ring

Relevance of having the same components integrated as European (shared) building blocks across different Use Cases.

Could the object of assessment be integrated as a European (shared) building block across different Use Cases?

Cross- boar-der

Usefulness of the object of assessment in the development of eGovernment cross-border services.

Could the object of assessment be useful in the development of eGovernment cross-border services?

Mar-ket

Potential of the object of assessment to be adopted by the market and be used in cross-border eGovernment services.

Does the object of assessment has potential to be adopted by the market and be used in cross-border eGovernment services?

ROI Where applicable the costs and benefits of adopting the object of assessment, including the assessment of the Return on Investment.

If applicable, is evaluation of the costs and benefits of adopting the object of assessment available?

Geo-gra-phic

Possibility for a broader geographic and sector usage.

Is there a possibility for a broader geographic and sector usage?


4.4.2 Market support

The following table presents market support criteria based on CAMSS.

Nr Cate-gory



5 Mar-ket sup-port

A object of assessment should have sufficient market acceptance and support in order to be adopted by public administrations. This category addresses the proven and operational implementations of the object of assessment, the market share and demand for the products, and the support from users and communities.

5.1 Imple-men-ta-tions

For the ‘implementations’, the existence of proven and best practice implementations for the object of assessment is addressed, in different domains and by different vendors.

A.28 Has the object of assessment been used for different implementations by different vendors/suppliers?

A.29 Has the object of assessment been used in different industries, business sectors or functions?

5.2 Mar-ket de-mand

For ‘market demand’, the penetration and acceptance of products implementing the object of assessment in the market is addressed.

A.30 Do the products that implement the object of assessment have a significant market share of adoption?

5.3 Users For the ‘users’, the diversity of the end-users of the products implementing the object of assessment is addressed.

A.31 Do the products that implement the object of assessment target a broad spectrum of end-uses?

5.4 Inte-rest gro-ups

For the ‘interest groups’, the degree of support from different interest groups is addressed.

A.32 Has the object of assessment a strong support from different interest groups?


5 THE RECOMMENDATION STEP: CRITERIA AND CLASSIFICATION

In the recommendation step, recommendation criteria are applied, concluding with a classification of

the object of assessment to be used by other Work Packages.

5.1 Recommendation criteria

Following CAMSS (Section 3.4), the main categories of criteria on the recommendation step are the same as used for grouping the assessment criteria: applicability, maturity, openness, intellectual

property rights, market support, and potential. For these main categories, a score is generated based on the number of ‘YES’ answers that indicates the level of meeting the criteria for the object of

assessment. The specified knock-out criteria are checked if they were all met and an answer is provided for the main categories. Based on the discussion in the assessment panel, an evaluation

and possible comments are provided for the main categories. A possible scale for the evaluation of the categories could include the following range: Very low, Low, Moderate, High, Very high.

The criteria are presented in the following table.

Nr. Category Automated score

Knock-out criteria met?

Evaluation Comments

R.1 ApplicabilityR.2 MaturityR.3 OpennessR.4 Intellectual

property rightsR.5 Market

supportR.6 Potential

5.2 Proposed Classification

The classification of the object of assessment is based on CAMSS (p 36), Technical Annex (pp 33-35),

PEPPOL, and other relevant sources. It uses the assessment and recommendation criteria and comprises the following classes.

Discarded: the object of assessment is not found relevant and should not be used in the public administration


Observed: the object of assessment is found relevant but should not be used in the current state or version

Accepted: the object of assessment is accepted and could be used in the public administrations (and for the specific business needs)

Recommended: the object of assessment is accepted and is preferred for the use in public administrations (compared to alternative objects of assessment)

Mandatory: the object of assessment is accepted and public administrations are obliged to use the object of assessment when applicable (or provide an explanation). Depending on the

e-SENS mandate, it may be agreed to avoid the "Mandatory" recommendation in the assessments.

The compliance levels of criteria for LSPs, necessary for the building block to be assessed as satisfying Accepted or higher levels, may be defined.

For the last three classes, maturity levels (Technical Annex pp 33-35 and PEPPOL) may be assessed in more detail if needed.


6 ORGANIZATIONAL ASPECTS OF ASSESSMENT

According to the Technical Annex (p 60), T3.1 will develop a questionnaire form for the assessment of

building blocks and discusses how to deal with current LSPs that do not comply to the criteria upfront.

6.1 A questionnaire form for the assessment of European Building Blocks

The questionnaire form follows closely the assessment framework, procedure, and criteria, and will

be provided in the subsequent versions of the Guidelines.

6.2 Dealing with current LSPs that do not comply to the criteria upfront

The solution for dealing with current LSPs that do not comply with the criteria upfront is based on a case-by-case analysis of the sources for non-compliance and application of appropriate solutions.

Possible sources and solutions are presented in the following table. In the course of assessment, additional sources for non-compliance and possible solutions will be considered as needed.

Source for non-compliance Possible solutionInsufficient presentation of the building block for assessment (for example, inadequate documentation of format)

Request for improvement of the presentation

A simple problem with reference to an otherwise acceptable building block

Request to correct the problem in the subsequent version of the building block

A complex problem with reference to an otherwise acceptable building block

Request to provide a plan to correct the problem in the subsequent versions of the building block

The project is classified as discarded or observed, with parallel building blocks providing adequate solutions in assessment

Selection of a parallel building block for assessment

The project is classified as discarded or observed, with no parallel building blocks providing adequate solutions in assessment

Providing a request to a relevant EU initiative for initiation of a new building block


7 RESEARCH UPDATE AND INPUT FOR THE ANALYSIS

Before assessing the building blocks, results of already existing assessments will be identified and re-

used (Task T3.1.2 of the Technical Annex). It is expected that the majority of the input for this subtask will come from WP5 through the work on D5.1 requirements framework, as well as the WP6

activities that are described in the e-SENS executable baseline architecture (D6.1). Other sources are the library http://joinup.ec.europa.eu/ as well as former LSP material. The assessment results from

standardisation bodies or other relevant bodies can be re-used and adapted for the assessment of an object of assessment of the building blocks.

The list of materials used for assessment is provided in the references section of this document.

The research update will be provided in the subsequent versions of the Guidelines.


8 APPENDIXES

8.1 Abbreviations and glossary

ABB - Architecture Building Block

ADMS - Asset Description Metadata Schema

ADMS.SW - Asset Description Metadata Schema for Software

BB - Building Block

BCSS - Basic Cross Sector Services

BOMOS - Management and Development Model for Open Standards

CAMSS - Common Assessment Method For Standards And Formal Specifications. Final Draft Revision of CAMSS. Version 1.0, March 2012

e-SENS - Electronic Simple European Networked Services

EIA - European Interoperability Architecture

EIF - European Interoperability Framework

(F)RAND - (fair,) reasonable, and non-discriminatory terms

Guidelines - Guidelines to the assessment of the sustainability and maturity of building blocks (the current document)

ISMS - information security management system

Object of assessment - An object (for example, an ABB or SBB) submitted for assessment in the

proposal step, to be assessed through the consideration, assessment, and recommendation steps.

SBB - Solution Building Block

SLA - Service Level Agreement

Technical Annex - e-SENS Technical Annex v3.0


8.2 References

1. Regulations

1.1. REGULATION (EU) No 1025/2012 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 25 October 2012 on European standardisation. OJ L 316, 14.11.2012, pp 12-33, http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:316:0012:0033:EN:PDF2. European wide scale frameworks, projects, and initiatives

2.1. European Interoperability Framework (EIF) for European public services, http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf

2.2. European Interoperability Architecture, http://ec.europa.eu/isa/documents/isa_2.1_eia-finalreport-commonvisionforaneia.pdf

2.3. EFIR, http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-4action_en.htm2.4. Joinup, https://joinup.ec.europa.eu/catalogue/all?current_checkbox=1

2.5. Data Catalog Vocabulary (DCAT), http://www.w3.org/2011/gld/wiki/Data_Catalog_Vocabulary

2.6. BOMOS (Management and Development Model for Open Standards), 2.7. BOMOS2i,

http://www.forumstandaardisatie.nl/fileadmin/os/publicaties/HR_BOMOS_English_translation_Jan2013.pdf

3. ADMS3.1. ADMS v 1.0. https://joinup.ec.europa.eu/asset/adms/release/100

3.2. ADMS. Asset Description Metadata Schema. Specification Version 1.00. Release date 18/04/2012

3.3. Asset Description Metadata Schema for Software, https://joinup.ec.europa.eu/asset/adms_foss/asset_release/admssw-100

4. Common Assessment Method For Standards And Formal Specifications (CAMSS). Final Draft Revision of CAMSS. Version 1.0, March 2012. See also the following sources.

4.1. CAMSS Project – Workshop 2. Agreement on Revision of CAMSS. Wednesday, March 7 th

2012 (fail Presentation of EC on CAMSS.ppt)

4.2. https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/ 5. International standards

5.1. ISO/IEC 12207:2008. Systems and software engineering. Software life cycle processes5.2. ISO/IEC 25000:2005. Software Engineering. Software product Quality Requirements and

Evaluation (SQuaRE). Guide to SQuaRE6. e-SENS project materials

6.1. e-SENS Technical Annex v 3.07. European specific area projects, building blocks, or assessments


https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/

https://joinup.ec.europa.eu/asset/adms_foss/asset_release/admssw-100

https://joinup.ec.europa.eu/asset/adms/release/100



http://www.w3.org/2011/gld/wiki/Data_Catalog_Vocabulary

https://joinup.ec.europa.eu/catalogue/all?current_checkbox=1

http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-4action_en.htm

http://ec.europa.eu/isa/documents/isa_2.1_eia-finalreport-commonvisionforaneia.pdf

http://ec.europa.eu/isa/documents/isa_2.1_eia-finalreport-commonvisionforaneia.pdf

http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:316:0012:0033:EN:PDF

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:316:0012:0033:EN:PDF

7.1. PEPPOL EIA Repository, http://www.peppol.eu/peppol_components/peppol-eia/eia 7.2. The feasibility and scenarios for the long-term sustainability of the Large Scale Pilots,

including 'ex-ante' evaluation. Task 1 Report. 11-03-2013 SMART 2012/0059, Deloitte.

8.3 The CAMSS assessment criteria

For reference, the CAMSS original categories, sub-categories, and criteria are presented in the

following table.

Nr Cate-gory



1 Applica-bility

A formal specification should be usable and easy implementable in different products to be relevant for adoption by public administrations. This category addresses the definition of functional scope and area of application, the possible reusability in other areas, the possible alternative specifications, the compatibility and dependency on other specifications or technologies.

1.1 Area of application

For the ‘area of application’, the functionalities and intended use of the formal specification are addressed within the context of interoperability and eGovernment.

A.1 Does the formal specification address and facilitate interoperability between public administrations?

A.2 Does the formal specification address and facilitate the development of eGovernment?

1.2 Requirements

For the ‘requirements’, the functional and nonfunctional requirements for using and implementing the formal specification are addressed. This criterion is related to the use of assessment scenario 3

A.3 Are the functional and nonfunctional requirements for the use and implementation of the formal specification clearly defined?

1.3 Reusability

For ‘reusability’, the level of reusability of the formal specification in the same or other areas of application is addressed.

A.4 Is the formal specification applicable and extensible for implementations in different domains?


http://www.peppol.eu/peppol_components/peppol-eia/eia

Nr Cate-gory



1.4 Alternatives

For the ‘alternatives’, the degree to which the formal specification adds value compared to alternative objects of assessment in the same area of application is addressed.

A.5 Does the formal specification provide sufficient added value compared to alternative objects of assessment in the same area of application?

1.5 Compatibility

For ‘compatibility’, the compatibility of the formal specification with other objects of assessment in the same area of application is addressed.

A.6 Is the formal specification largely compatible with related (not alternative) objects of assessment in the same area of application?

1.6 Dependencies

‘Dependencies’ addresses the degree of independence of the formal specification from specific vendor products, platforms or technologies.

A.7 Is the formal specification largely independent from specific vendor products?

A.8 Is the formal specification largely independent from specific platforms or technologies?

2 Maturity

A formal specification should in itself be mature enough for adoption by public administrations. This category addresses the development status, the quality, guidelines and stability of the formal specification.

2.1 Development status

For the ‘development status’, the current development status of the formal specification in the development cycle is addressed.

A.9 Has the formal specification been sufficiently developed and in existence for a sufficient period to overcome most of its initial problems?

2.2 Quality

For ‘quality’, the level of detail in the formal specification and the conformance of implementations is addressed.

A.10 Are there existing or planned mechanisms to assess conformity of the implementations of the formal specification (e.g. conformity tests, certifications)?

A.11 Has the formal specification sufficient detail, consistency and completeness for the use and development of products?

2.3 Guidelines

For the ‘guidelines’, the existence of implementation guidelines or reference implementations is addressed.

A.12 Does the formal specification provide available implementation guidelines and documentation for the implementation of products?

A.13 Does the formal specification provide a reference (or open source) implementation?


Nr Cate-gory



2.4 Stability

For ‘stability’, the level of change to the formal specification and the stability of underlying technologies is addressed.

A.14 Does the formal specification address backward compatibility with previous versions?

A.15 Have the underlying technologies for implementing the formal specification been proven, stable and clearly defined?

3 Openness

A formal specification should be sufficiently open and available to be relevant for adoption by public administrations. This category addresses the openness of the standardisation organisation and decision-making process, and the openness of the documentation and accessibility of the formal specification.

3.1 Organisation

For the ‘openness’ of the organisation, the level of openness for participating in the standardisation organisation is addressed.

A.16 Is information on the terms and policies for the establishment and operation of the standardisation organisation publicly available?

A.17 Is participation in the creation process of the formal specification open to all relevant stakeholders (e.g. organisations, companies or individuals)?

3.2 Process

For the ‘process’, the level of openness regarding the development and decision-making process for the formal specification is addressed.

A.18 Is information on the standardisation process publicly available?

A.19 Information on the decision making process for approving objects of assessment is publicly available?

A.20 Are the objects of assessment approved in a decision making process which aims at reaching consensus?

A.21 Are the objects of assessment reviewed using a formal review process with all relevant external stakeholders (e.g. public consultation)?


Nr Cate-gory



A.22 All relevant stakeholders can formally appeal or raise objections to the development and approval of objects of assessment?

3.3 Documentation

For the openness of the ‘documentation’, the accessibility and availability of the documentation of the formal specification is addressed.

A.23 Relevant documentation of the development and approval process of objects of assessment is publicly available (e.g. preliminary results, committee meeting notes)?

A.24 Is the documentation of the formal specification publicly available for implementation and use on reasonable terms?

4 Intellectual property rights

A formal specification should be licensed on (F)RAND terms or even on a royalty-free basis in a way that allows implementation in different products. This category addresses the availability of the documentation on the IPR and the licences for the implementation of the formal specification.

4.1 IPR Documentation

For the ‘documentation of the intellectual property rights’, the availability of the information concerning the ownership rights of the formal specification is addressed.

A.25 Is the documentation of the IPR for objects of assessment publicly available?

4.2 Licences

For the ‘licences’ within the intellectual property rights, a (fair) reasonable and non-discriminatory ((F)RAND) or even royalty-free basis is addressed for the use and implementation of the formal specification.

A.26 Is the formal specification licensed on a (F)RAND basis?

A.27 Is the formal specification licensed on a royalty-free basis?

5 Market support

A formal specification should have sufficient market acceptance and support in order to be adopted by public administrations. This category addresses the proven and operational implementations of the formal specification, the market share and demand for the products, and the support from users and communities.

5.1 Implementations

For the ‘implementations’, the existence of proven and best practice implementations for the formal specification is addressed, in different domains and by different vendors.

A.28 Has the formal specification been used for different implementations by different vendors/suppliers?


Nr Cate-gory



A.29 Has the formal specification been used in different industries, business sectors or functions?

5.2 Market demand

For ‘market demand’, the penetration and acceptance of products implementing the formal specification in the market is addressed.

A.30 Do the products that implement the formal specification have a significant market share of adoption?

5.3 Users For the ‘users’, the diversity of the end-users of the products implementing the formal specification is addressed.

A.31 Do the products that implement the formal specification target a broad spectrum of end-uses?

5.4 Interest groups

For the ‘interest groups’, the degree of support from different interest groups is addressed.

A.32 Has the formal specification a strong support from different interest groups?

6 Potential

A formal specification should have sufficient and positive future consequences, evolution and impact for being adopted by public administrations. This category addresses the consequences and impact of using or adopting the formal specification, the advantages and risks, the maintenance and possible future developments.

6.1 Impact

For the ‘impact’, the minimisation of the consequences of using and adopting the formal specification is addressed. The consequences can be evaluated and described in terms of different aspects.

A.33 Is there evidence that the adoption of the formal specification positively impacts organisational processes?

A.34 Is there evidence that the adoption of the formal specification positively impacts the migration of current systems?

A.35 Is there evidence that the adoption of the formal specification positively impacts the environment?

A.36 Is there evidence that the adoption of the formal specification positively impacts the financial costs?

A.37 Is there evidence that the adoption of the formal specification positively impacts the security?


Nr Cate-gory



A.38 Is there evidence that the adoption of the formal specification positively impacts the privacy?

A.39 Is there evidence that the adoption of the formal specification positively impacts the administrative burden?

A.40 Is there evidence that the adoption of the formal specification positively impacts the disability support?

6.2 Risks For the ‘risks’, the level of uncertainty is addressed for using and adopting the formal specification

A.41 Are the risks related to the adoption of the formal specification acceptable?

6.3 Maintenance and future developments

For the ‘maintenance’ and future developments, the support and the planned or existing actions to maintain, improve and develop the formal specification in the long term are addressed.

A.42 Does the formal specification have a defined maintenance organisation?

A.43 Does the maintenance organisation for the formal specification have sufficient finances and resources for the long term?

A.44 Does the formal specification have a defined maintenance and support process?

A.45 Does the formal specification have a defined policy for version management?


Date post:	16-Aug-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times