Introduction - Web viewICT Policy Support Programme ... focusing strongly on the core building...

D3.1 Guidelines to the assessment of the

sustainability and maturity of building blocks

Draft deliverable of theSubmitted to the EC on XXX/XX/20XX

COMPETITIVENESS AND INNOVATION FRAMEWORK PROGRAMMEICT Policy Support Programme (ICT PSP)

Project acronym: e-SENS Task T3.1e-SENSProject full title: Electronic Simple European Networked Services

D3.1 Guidelines to the assessment of the sustainability and maturity of building blocks 1

ICT PSP call identifier: CIP-ICT-PSP-2012-6ICT PSP main theme identifier: CIP-ICT-PSP-2012-6-4.1 Basic Cross Sector Services

Grant agreement n°: 325211

D3.1 Guidelines to the assessment of the sustainability and maturity of building blocks

Deliverable Id : D3.1

Deliverable Name : Guidelines to the assessment of the sustainability and maturity of building blocks

Version : v0.9Status : Draft

Dissemination Level : PublicDue date of deliverable : M6Actual submission date : Date of submission to EC

Work Package : WP3, Task 3.1

Organisation name of lead partner for this deliverable : Ministry of Economic Affairs and Communications of Estonia

Author(s): Jaak TepandiPartner(s) contributing : Task 3.2, WP3, WP5, WP6

Abstract. This e-SENS Task 3.1 deliverable ("Guidelines") is intended for assessing various types of building blocks. The Guidelines are based on the e-SENS Technical Annex v 3.0, the CAMSS methodology, ADMS, the e-SENS WP6 deliverable D6.1, suggestions from e-SENS stakeholders, standards, and other sources. The assessment procedure comprises proposal, consideration, assessment, and recommendation steps. The current document presents a procedure for assessment, criteria and information to be used in the four assessment steps, organisational aspects of assessment, and research update on already existing assessments.


History

Version Date Changes made Modified by

0.2 30.04.2013 D3.1 Guideline principles Jaak Tepandi

0.3 13.05.2013 Guideline principles are extended and updated to a draft Guideline, taking into account

comments from Freek van Krevel, Jack Verhoosel, Marijke Salters, and Uuno Vallner

Jaak Tepandi

0.5 28.05.2013 The draft Guideline criteria are elaborated,

extended and updated according to suggestions from the WP3 Brussels meeting on 17.05 and

teleconference on 28.05

Jaak Tepandi

0.7 31.05.2013 The draft Guideline is extended and updated throughout the text according to suggestions

from T3.2 and other WP3 participants. New presentation of the assessment procedure is

elaborated. Assessment criteria from different sources are merged. New template for the

report has been applied.

Jaak Tepandi

0.9 19.07.2013 Proposals and comments from Freek van Krevel, Jack Verhoosel, Marijke Salters, Uuno Vallner,

Bertrand Grégoire, Cagatay Karabat, Anni Buhr, Klaus Vilstrup Pedersen, Sven Rostgaard

Rasmussen, and other e-SENS project participants are taken into account. The

Guidelines are closer aligned with the e-SENS deliverable template. The proposal

information/criteria are aligned with WP6.

Jaak Tepandi


Table of Contents

EXECUTIVE SUMMARY .................................................................................................................................. 3

1 INTRODUCTION .................................................................................................................................... 5

1.1 TIMELINE OF T3.1 .................................................................................................................................... 5

1.2 CAMSS .................................................................................................................................................. 6

1.3 ADMS ................................................................................................................................................... 7

2 A PROCEDURE FOR THE ASSESSMENT OF EUROPEAN BUILDING BLOCKS ...............................................8

3 PROPOSAL AND CONSIDERATION STEPS: DOCUMENTATION OF FORMAT .............................................9

3.1 THE OBJECTS TO BE ASSESSED ...................................................................................................................... 9

3.2 THE PROPOSAL STEP ................................................................................................................................ 10

3.3 THE CONSIDERATION STEP ........................................................................................................................ 11

4 THE ASSESSMENT STEP ....................................................................................................................... 13

4.1.1.1 Merging the Technical Annex and the CAMSS assessment criteria .......................................................13

4.2 ASSESSMENT CRITERIA BASED ON THE E-SENS TECHNICAL ANNEX .................................................................... 13

4.2.1 Standardisation criteria .................................................................................................................13

4.2.2 Business need criteria ....................................................................................................................14

4.2.3 Alignment with existing policy frameworks ...................................................................................15

4.3 THE CAMSS ASSESSMENT CRITERIA ........................................................................................................... 15

5 THE RECOMMENDATION STEP: CRITERIA AND CLASSIFICATION ...........................................................21

6 ORGANIZATIONAL ASPECTS OF ASSESSMENT ......................................................................................23

6.1 A QUESTIONNAIRE FORM FOR THE ASSESSMENT OF EUROPEAN BUILDING BLOCKS ............................................... 23

6.2 DEALING WITH CURRENT LSPS THAT DO NOT COMPLY TO THE CRITERIA UPFRONT ............................................... 23

7 RESEARCH UPDATE AND INPUT FOR THE ANALYSIS (TASK T3.1.2) ........................................................24

8 APPENDIX ........................................................................................................................................... 25

8.1 ABBREVIATIONS AND GLOSSARY ................................................................................................................. 25

8.2 REFERENCES ........................................................................................................................................... 25

HISTORY ....................................................................................................................................................... 2

TABLE OF CONTENTS ..................................................................................................................................... 3


LIST OF FIGURES ........................................................................................................................................... 5

LIST OF TABLES ............................................................................................................................................. 6

LIST OF ABBREVIATIONS AND GLOSSARY ....................................................................................................... 7

EXECUTIVE SUMMARY .................................................................................................................................. 9

1 INTRODUCTION .................................................................................................................................. 11

1.1 SCOPE AND OBJECTIVE OF GUIDELINES ........................................................................................................ 11

1.2 METHODOLOGY OF WORK ........................................................................................................................ 11

1.2.1 CAMSS ............................................................................................................................................12

1.2.2 ADMS .............................................................................................................................................12

1.2.3 Other sources .................................................................................................................................12

1.3 RELATIONS TO INTERNAL E-SENS ENVIRONMENT .......................................................................................... 13

1.3.1 Reviewing and commenting on the Guidelines ..............................................................................13

1.3.2 Relations to e-SENS Work Packages ...............................................................................................13

1.3.3 Timeline of T3.1 .............................................................................................................................13

1.4 STRUCTURE OF THE DOCUMENT ................................................................................................................. 14

2 THE ASSESSMENT PROCEDURE AND TARGETS .....................................................................................15

2.1 THE ASSESSMENT PROCEDURE ................................................................................................................... 15

2.2 THE TARGETS OF ASSESSMENT ................................................................................................................... 16

3 PROPOSAL AND CONSIDERATION STEPS: DOCUMENTATION OF FORMAT ............................................18


3.1.1 Complete list of proposal information/criteria ...............................................................................19

3.1.2 The mandatory proposal information/criteria ...............................................................................23

3.2 THE CONSIDERATION STEP ........................................................................................................................ 24

4 THE ASSESSMENT STEP ....................................................................................................................... 26

4.1 INTEGRATION OF ASSESSMENT CRITERIA FROM VARIOUS SOURCES .................................................................... 26

4.2 STANDARDISATION CRITERIA ...................................................................................................................... 27

4.2.1 Maturity .........................................................................................................................................28

4.2.2 Openness .......................................................................................................................................29

4.2.3 Intellectual property rights ............................................................................................................31

4.2.4 Life cycle, maintenance, service levels, security .............................................................................31

4.3 CRITERIA FOR ALIGNMENT WITH EXISTING POLICY FRAMEWORKS ...................................................................... 34

4.3.1 Basic alignment criteria .................................................................................................................34

4.3.2 Applicability ...................................................................................................................................36

4.3.3 Potential ........................................................................................................................................37


4.4 BUSINESS NEED CRITERIA .......................................................................................................................... 39

4.4.1 Basic business need criteria ...........................................................................................................40

4.4.2 Market support ..............................................................................................................................41

5 THE RECOMMENDATION STEP: CRITERIA AND CLASSIFICATION ...........................................................43

5.1 RECOMMENDATION CRITERIA .................................................................................................................... 43

5.2 PROPOSED CLASSIFICATION ....................................................................................................................... 44

6 ORGANIZATIONAL ASPECTS OF ASSESSMENT ......................................................................................45

6.1 A QUESTIONNAIRE FORM FOR THE ASSESSMENT OF EUROPEAN BUILDING BLOCKS ............................................... 45

6.2 DEALING WITH CURRENT LSPS THAT DO NOT COMPLY TO THE CRITERIA UPFRONT ............................................... 45

7 RESEARCH UPDATE AND INPUT FOR THE ANALYSIS .............................................................................47

8 APPENDIX 1 ........................................................................................................................................ 48

8.1 REFERENCES ........................................................................................................................................... 48

8.2 PROPOSAL INFORMATION/CRITERIA BASED ON CAMMS ................................................................................ 49

8.3 PROPOSAL INFORMATION/CRITERIA BASED ON WP6 DELIVERABLE D6.1 ........................................................... 51

8.4 THE CAMSS ASSESSMENT CRITERIA ........................................................................................................... 52

9 APPENDIX 2 - TO BE REMOVED WHEN DELIVERABLE IS FINALISED .......................................................58


9.1.1 Proposal information/criteria based on WP6 deliverable D6.1 ......................................................58

9.1.2 Proposal information/criteria based on CAMMS ...........................................................................59

9.1.3 Proposal information/criteria based on ADMS ..............................................................................61

9.1.4 Proposal information/criteria based on ADMS.SW ........................................................................61


List of Figures

Figure 1. The procedure for assessment with main stakeholders .......................................................16


List of Tables

Table 1. Timeline of T3.1 .....................................................................................................................14Table 2. The complete list of proposal information/criteria based on CAMMS, D6.1, and ADMS .......23

Table 3. The mandatory proposal information/criteria in the recommended order of presentation ..24Table 4. Consideration criteria .............................................................................................................25

Table 5. Assessment criteria for standardization: maturity .................................................................29Table 6. Assessment criteria for standardization: openness ................................................................30

Table 7. Assessment criteria for standardization: intellectual property rights ....................................31Table 8. Assessment criteria for standardization: life cycle .................................................................34

Table 9. Assessment criteria for alignment: basic ................................................................................36Table 10. Assessment criteria for alignment: applicability ..................................................................37

Table 11. Assessment criteria for alignment: potential .......................................................................39Table 12. Assessment criteria for business need: basic .......................................................................41

Table 13. Assessment criteria for business need: market support ......................................................42Table 14. Recommendation criteria ....................................................................................................44

Table 15. Dealing with current LSPs that do not comply to the criteria upfront .................................46Table 16. Proposal information/criteria based on CAMMS .................................................................51

Table 17. Appendix: the CAMMS assessment criteria .........................................................................57Table 18. Appendix. Proposal information/criteria based on CAMMS ................................................61


List of Abbreviations and Glossary

A2A - Administration to Administration

ABB - Architecture Building Block

ADMS - Asset Description Metadata Schema

ADMS.SW - Asset Description Metadata Schema for Software

BB - Building Block, represents a (potentially re-usable) component of business, IT, or architectural capability that can be combined with other building blocks to deliver architectures and solutions (a

TOGAF 9 definition)

BCSS - Basic Cross Sector Services

BOMOS - Management and Development Model for Open Standards

CAMSS - Common Assessment Method For Standards And Formal Specifications. Final Draft Revision

of CAMSS. Version 1.0, March 2012

e-SENS - Electronic Simple European Networked Services

EIA - European Interoperability Architecture

EIF - European Interoperability Framework

(F)RAND - (fair,) reasonable, and non-discriminatory terms

Guidelines - Guidelines to the assessment of the sustainability and maturity of building blocks (the

current document)

HBB - High Level Building Block

ISMS - information security management system

OpenBRR - Open Software Business Readiness Rating

SBB - Solution Building Block

SLA - Service Level Agreement


Target of assessment - An artifact (typically in a TOGAF 9 sense: "Artifact - an architectural work product that describes an aspect of the architecture"), submitted for assessment in the proposal

step, to be assessed through the consideration, assessment, and recommendation steps

Technical Annex - e-SENS Technical Annex v3.0

TOA - See "target of assessment" (used in tables)

WP - Work Package


Executive Summary

The e-SENS project will consolidate the building blocks of the existing Large Scale Pilots, focusing strongly on the core building blocks such as eID, eDocuments, e-Delivery, semantics and e-

Signatures. eSignatures, eDocuments, eDelivery, and others. The e-SENS Work Package 3 aims to present proposals for sustainable building blocks that have emerged from the Large Scale Pilots

relevant to the e-SENS project. This proposal should support competitiveness, openness for future technologies, and interoperability.

The e-SENS Work Package 3 aims to present proposals for sustainable building blocks that have emerged from the Large Scale Pilots relevant to the e-SENS project. This proposal should support

competitiveness, openness for future technologies, and interoperability.Presentation

Task T3.1 of the Work Package 3 supports presentation and assessment of building blocks are

supported by Task T3.1 of the Work Package 3. It will be active from M1 to M6. The . Its main deliverable is ‘Guidelines to the assessment of the sustainability and maturity of building blocks’

("Guidelines")", the current document).

The objective of Task T3.1 proposesthe Guidelines is to propose a documentation of format and

defining criteria for the maturity and sustainability assessment of building blocks in close cooperation with Task 3.2, WP3, WP 5, WP 6. Task 3.1 will be active from M1 to M6.

The current version of the Guidelines builds upon existing work carried out in the European Interoperability Framework and is based on the e-SENS Technical Annex v 3.0, the CAMSS

methodology (Final Draft Revision of CAMSS, Version 1.0, March 2012), the ADMS (Asset Description Metadata Schema, Specification Version 1.00), discussions with e-SENS project participants, and

various other e-SENS project materials. As the CAMSS is relies on ADMS, the Guidelines also make use of ADMS.

The Guidelines are intended for assessing various types of building blocks (including architecture and solution building blocks).

Following the e-SENS Technical Annex v 3.0 and the CAMSS methodology, the current document uses the assessment processprocedure that comprises proposal, consideration, assessment, and

recommendation steps as follows.

[1.] In the proposal step, an objecta target of assessment (for example, an Architecture Building

Block or High level Building Block, including support artifacts such as guidelines) is provided


to Task 3.2 by WP 5, WP 6, or other relevant stakeholders.WP6 Architectural Board. The objects of assessment are classified under broader categories of core building blocks. These

blocks may include eID, eDocuments, e-Signatures, e-Delivery, Privacy, Semantics, and others. The objecttarget of assessment is provided using the proposal criteria. The proposal

criteria provide general information about the proposed target of assessment, its status, items provided for assessment, and other.

1.[2.] In the consideration step, consideration criteria are used before the actual assessment, to validate information received and relevance of the proposal.

[3.] In the assessment step, the criteria used are categorised under standardisation, business

need, and alignment with existing policy frameworks. , and business need. Additional information will be sought from other Work Packages and external stakeholders.

[4.] In the recommendation step, recommendation criteria are applied to conclude with a classification (Discarded, Observed, Accepted, Recommended, Mandatory) of the

objecttarget of assessment. This classification canwill be usedreported back to WP6 Architectural Board for using by other Work Packages.

The Guidelines, including the sets of criteria provided, are intended for assessing various types of building blocks. During a specific assessment, relevant criteria are selected depending on the target

submitted for evaluation.

The document starts with the background given in the introduction. The second section presents a

procedure for assessment and an overview of the targets to be assessed, the third - the criteria for the proposal and consideration steps. Section 4 is devoted to the assessment step and criteria,

section 5 - to the recommendation step. Sections 6 and 7 deal with organizational aspects of assessment and with research update on already existing assessments. The current version of the

Guidelines may include subsections, questions, and comments (grouped under Appendix 9 and the 4th level Headings) to be removed from the final text.

The current version of the Guidelines has taken into account comments on the earlier versionversions from Freek van Krevel, Jack Verhoosel, Marijke Salters, Uuno Vallner, Bertrand

Grégoire, Cagatay Karabat, Anni Buhr, Klaus Vilstrup Pedersen, Sven Rostgaard Rasmussen, and other e-SENS project participants. The terminology (eg, notions of a building block or an object of

assessment) and structure of this document are subject to change with the feedback received.

The structure and content of the document follow as much as possible the structure and content of

the final Guidelines. Each section may include subsections, questions, and comments (grouped under 4th level Headings) to be removed from the final text.

Please address feedback to Jaak Tepandi ([email protected] ) , Uuno Vallner ([email protected] ) .


mailto:[email protected]


The Guidelines is a living document that has to be updated after changes in underlying materials, due to experience gained in pilot assessments, as a result of new agreements, and other.

Please address feedback to Jaak Tepandi ([email protected] ) , Uuno Vallner ([email protected] ) .




1 Introduction

1.1 Scope and objective of Guidelines

The e-SENS project will consolidate the building blocks of the existing Large Scale Pilots, focusing

strongly on the core building blocks such as eID, eSignatures, eDocuments, e-Delivery, semanticseDelivery, and e-Signaturesothers.

The e-SENS Work Package 3 aims to present proposals for sustainable building blocks that have emerged from the Large Scale Pilots relevant to the e-SENS project. This proposal should support

competitiveness, openness for future technologies, and interoperability.

PresentationTask T3.1 of the Work Package 3 supports presentation and assessment of building

blocks are supported by Task T3.1 of the Work Package 3. The . Its main deliverable is ‘Guidelines to the assessment of the sustainability and maturity of building blocks’ ("Guidelines")", the current

document).

The objective of T3.1 proposesthe Guidelines is to propose a documentation of format and defining

criteria for the maturity and sustainability assessment of building blocks in close cooperation with Task 3.2 and Work Packages 5 and 6. , WP3, WP 5, WP 6. Task 3.1 will be active from M1 to M6.

The Guidelines are intended for assessing various types of building blocks. It is a living document that has to be updated after changes in underlying materials, due to experience gained in pilot

assessments, as a result of new agreements, and other.

1.2 Methodology of work

The current version of the Guidelines builds upon existing work carried out in the European Interoperability Framework and is based on the e-SENS Technical Annex v 3.0, the CAMSS

methodology (Final Draft Revision of CAMSS, Version 1.0, March 2012), the ADMS (Asset Description Metadata Schema, Specification Version 1.00), discussions with e-SENS project participants, and

various other e-SENS project materials. As the CAMSS is relies on ADMS,

In case the Guidelines also make use of ADMS. terminology (e.g. notion of a building block) depends

on deliverables of other WPs, it is referred to these deliverables. For presenting the assessment


guidelines, the notion of a "target of assessment" is utilized (please see the List of Abbreviations and Glossary section).

The Guidelines are intended for assessing various types of building blocks (including architecture and solution building blocks).

The structure and content of the document follow as much as possible the structure and content of the final Guidelines. Each section may include subsections, questions, and comments (grouped

under 4th level Headings) to be removed from the final text.

[1.1.1] Timeline of T3.1

According to the Technical Annex it is expected that the much of the input for T3.1 will come from

T3.2, WP5, WP6, and other WPs, which may have later delivery dates than T3.1 (month 6). Therefore T3.1 needs to communicate with these tasks and WPs before they finish their work.

The task T3.1 will be active from M1 to M6. The result of this task is the Guidelines deliverable. A draft version will be ready in month 4 and the definitive version should be ready in month 6. The

indicative timeline includes the following deadlines and actions.

Deadline / duration Activities Deliverables

1.04.2013 Start of the task T3.1

1.04 -15.05.2013 Development of main principles for

the Guidelines. Preliminary discussions with participants.

Main principles for the Guidelines

17.05. 2013 Discussion of the main principles

with participants (Planned in Brussels)

Adjustment of main principles

May-June 2013 Preparation and discussion of the

preliminary draft of the Guidelines (planned via e-mail and / or

teleconference)

Preliminary draft of the Guidelines

Beginning of July 2013

Discussion of the preliminary draft of the Guidelines (planned in

Tallinn)

Adjustment of the preliminary draft

01.07-31.07. 2013 Preparation and discussion of the draft version of the Guidelines for

discussion

Draft version of the Guidelines

01.08-30.09. 2013 Discussion of the draft version Final version of the Guidelines


The following sections give a brief overview of CAMMS, ADMS, and other sources used.

1.2.1 CAMSS

In this document, the CAMSS (Final Draft Revision of CAMSS., Version 1.0, March 2012) scenarios

"Assessment scenario 2 – An assessment of a formal specification for adoption by public administrations" and "Assessment scenario 3 – An assessment and selection of formal specifications

for specific business needs and requirements" are taken into account.

According to assessment scenario 2, a formal specification is assessed in order to evaluate and

provide a recommendation on the possible adoption of the formal specification by the public administrations. This assessment scenario can be triggered by a public administration or related

external stakeholders. The outcome for this assessment scenario will be the adoption of a certain formal specification by the public administrations.

According to assessment scenario 3, a proposed set of formal specifications are assessed and evaluated in order to select and adopt the most relevant formal specification for specific business

needs and requirements. If a certain business need arises and requires the adoption of a relevant formal specification, the business need should be examined to list the relevant requirements, and

based upon those requirements, a selection of relevant formal specifications may be established. The outcome for this assessment scenario will be the selection and adoption of relevant formal

specifications for the specific business needs and requirements.

The basic difference between scenarios 2 and 3 is availability of specific business needs and

requirements in scenario 3. In case such needs and requirements are not available, preference is given to scenario 2.

1.2.2[1.1.2] ADMS

The CAMMSCAMSS make use of the ADMS (Asset Description Metadata Schema). Were) - a vocabulary to describe interoperability assets. Where possible, the current Guidelines use ADMS as

well. In particular, the controlled vocabularies used during the assessment processprocedure are based on ADMS (Asset Description Metadata Schema. Specification Version 1.00. Release date

18/04/2012).

1.2.3[1.1.3] Other sources

A nnex II of the Regulation on European standardisation comprises requirements for the

identification of ICT technical specifications. The Joinup repository currently covers the


documentation of semantic and open-source software interoperability assets. The EFIR contains interoperability assets of the Member States.

1.3 Relations to internal e-SENS environment

1.3.1 Reviewing and commenting on the Guidelines

The current version of the Guidelines has taken into account comments on the earlier versions from Freek van Krevel, Jack Verhoosel, Marijke Salters, Uuno Vallner, Bertrand Grégoire, Cagatay Karabat,

Anni Buhr, Klaus Vilstrup Pedersen, Sven Rostgaard Rasmussen, and other e-SENS project participants.

The Guidelines is a living document that has to be updated after changes in underlying materials, due to experience gained in pilot assessments, as a result of new agreements, and other.

1.3.2 Relations to e-SENS Work Packages

e-SENS Task 3.2 will use the Guidelines for assessment. Task 3.4 will contribute to assessment of business need criteria.

WP3 has close connections with WP5, and WP6. In particular, the assessment activities are integrated with these WPs.

According to the Technical Annex (p 71), one of the Objectives of WP5 is to support WP3 in its mission to develop Long Term Sustainability activities for e-SENS services and building blocks, by

providing pilot-level and domain-level validation of requirements from a functional and organizational acceptance viewpoint.

One of the tasks of WP6 deliverable D6.1 is taking stock - creating an ICT Architectural Baseline for e-SENS (Technical Annex p 112). Each phase in the different WP6 Subgroups will have the main focus

area of Inception. This focus area comprises assessment of ICT maturity and together with WP3 and WP5 - the overall maturity in accordance with the maturity model (Technical Annex p 114).

1.3.3 Timeline of T3.1

According to the Technical Annex it is expected that much of the input for T3.1 will come from T3.2, WP5, WP6, and other WPs, which may have later delivery dates than T3.1 (month 6). Therefore T3.1

needs to communicate with these tasks and WPs before they finish their work.


The task T3.1 is active from M1 to M6. The result of this task is the Guidelines deliverable. A draft version will be ready in month 4 and the definitive version should be ready in month 6. The

indicative timeline includes the following deadlines and actions.

Deadline / duration Activities Deliverables

1.04.2013 Start of the task T3.1

1.04 -15.05.2013 Development of main principles for

the Guidelines. Preliminary discussions with participants.

Main principles for the Guidelines

17.05. 2013 Discussion of the main principles

with participants (in Brussels)

Adjustment of main principles

May-June 2013 Preparation and discussion of the preliminary draft of the Guidelines

(planned via e-mail and / or teleconference)

Preliminary draft of the Guidelines

11-12.07. 2013 Discussion of the preliminary draft

of the Guidelines (in Tallinn, 11-12 July 2013)

Adjustment of the preliminary draft

01.07-31.07. 2013 Preparation and discussion of the

draft version of the Guidelines for discussion

Draft version of the Guidelines

01.08-30.09. 2013 Discussion of the draft version Final version of the Guidelines

Table 1. Timeline of T3.1

1.4 Structure of the document

The document is structured as follows. The second section presents a procedure for assessment and

an overview of the targets to be assessed, the third - the criteria for the proposal and consideration steps. Section 4 is devoted to the assessment step and criteria, section 5 - to the recommendation

step. Sections 6 and 7 deal with organizational aspects of assessment and with research update on already existing assessments.

The current version of the Guidelines may include subsections, questions, and comments (grouped under Appendix 9 and the 4th level Headings) to be removed from the final text.

Please address feedback to Jaak Tepandi ([email protected]), Uuno Vallner ([email protected]).





[2] A procedure for the assessment of European Building BlocksThe Assessment Procedure and Targets

1.5 The assessment procedure

According to the e-SENS Technical Annex (p 60), T3.1 will develop a procedure for the assessment of

building blocks.

CAMSS scenarios 2 and 3, especially the assessment step (CAMSS, section 2.4) are considered for a

basis for organizational aspects of assessment, together with relevant frameworks such as BOMOS2i, ISO/IEC 152888, ISO/IEC 25000 series standards, and other sources as appropriate.

The assessment is conducted by the e-SENS WP3 Assessment Panel avoiding conflicts of interests and providing transparency of the assessment process.

Following the e-SENS Technical Annex and the CAMSS methodology, the current document uses the assessment processprocedure that comprises proposal, consideration, assessment, and

recommendation steps (see Figure 1).

Figure 1. A procedure for assessment

[1.] In the proposal step, an objecta target of assessment (for example, an Architecture Building Block or High level Building Block, including support artifacts such as guidelines) is provided


to Task 3.2 by WP 5, WP 6, or other relevant stakeholders.WP6 Architectural Board. The objects of assessment are classified under broader categories of core building blocks. These

blocks may include eID, eDocuments, e-Signatures, e-Delivery, Privacy, Semantics, and others. The objecttarget of assessment is provided using the proposal criteria. The proposal

criteria provide general information about the proposed target of assessment, its status, items provided for assessment, and other.

1.[2.] In the consideration step, consideration criteria are used before the actual assessment, to validate information received and relevance of the proposal.

[3.] In the assessment step, the criteria used are categorised under standardisation, business

need, and alignment with existing policy frameworks. , and business need. Additional information will be sought from other Work Packages and external stakeholders.

[4.] In the recommendation step, recommendation criteria are applied to conclude with a classification (Discarded, Observed, Accepted, Recommended, Mandatory) of the

objecttarget of assessment. This classification canwill be usedreported back to WP6 Architectural Board for using by other Work Packages.

Figure 1 captures the primary process flow. There can be many additional connections between the stakeholders. In particular, the Assessment Panel communicates with other Work Packages and

external stakeholders during the assessment step.


Figure 1. The procedure for assessment with main stakeholders


2 The targets of assessmentProposal and Consideration Steps: Documentation of Format

The documentation of formats helps WP6 to provide building blocks for assessment and provides consideration criteria for Task T3.2.

The documentation of formats is based on CAMSS sections 3.1 and 3.2 and, if applicable, takes into account relevant standardization and asset description initiatives such as the Regulation on

European standardisation, the Asset Description Metadata Schema (ADMS), the European Federated Interoperability Repository (EFIR), the Joinup platform, and others.

Annex II of the Regulation on European standardisation comprises requirements for the identification of ICT technical specifications. The Asset Description Metadata Schema (ADMS) is a

vocabulary to describe interoperability assets. The EFIR contains interoperability assets of the Member States. The Joinup repository currently covers the documentation of semantic and open-

source software interoperability assets.

[2.1] The objects to be assessed

[2.2] In the proposal step, an object of assessment is provided to Task 3.2 by WP 5, WP 6, or other relevant stakeholders. The objects of assessment are classified under broader categories of core building blocks.

Since the main topic for the current Guidelines is documentation of format and defining criteria for assessment, only selected core building block examples are mentioned below to improve

understanding of the assessment issues. Further selection of building blocks will be based on input from WP6 during the assessment phase.

The core building blocks may include the following.

eIdentity

eID eSignatures

eDocuments


eDelivery Other building blocks as applicable, such as eIdentity, Semantics, Circle of Trust

e-Signatures e-Delivery

Privacy

The objectstargets of assessment are classified under broader categories of Architecture Building

Blocks and Solution Building Blocks. The Architecture Building Blocks provided for assessment by WP6 may include but are not constrained to the following types.

Architecture strategy, frameworks, and methodologies Service contracts, service level agreements, policies

Documentation and guidelines, including software and deployment descriptions Specification of interfaces of components considering the building block as black box

Specification of the functionality of the building block Detailed design of the building block and its internal components

Other types

The targets of assessment may also comprise Solution Building Blocks that include but are not

constrained to the following types.

Implementation of the building block in terms of source files and its documentation

Implementation of the building block in terms of executable files Other types

Implementation of the building block in terms of source files and its documentation Implementation of the building block in terms of executable files

Other types

These types of objects may occur in combination.

The complete list of building blocks and objectstargets for assessment will be provided by WP6 Architectural Board.

To deal with different types of building blocks, the current guideline proposes a wide set of criteria that should be applied selectively as follows.

Some criteria may be applicable to a specific type of building blocksTo deal with different types of building blocks, the current guideline proposes a wide set of criteria that should be applied

selectively as follows.


Some criteria may be applicable to a specific type of building blocks. The criteria may be mandatory or optional.

There can be assessment levels defined for the criteria.


3 Proposal and Consideration Steps: Documentation of Format

The documentation of formats helps WP6 to provide building blocks for assessment and provides consideration criteria for Task T3.2.

The documentation of formats is based on CAMSS sections 3.1 and 3.2, on the WP6 deliverable "D6.1 Executable ICT Baseline Architecture", on the Asset Description Metadata Schema (ADMS), as

well as on other relevant standardization and asset description initiatives such as the Regulation on European standardisation, the Joinup platform, the European Federated Interoperability Repository

(EFIR), and others.

3.1 The proposal step

As input to the proposal step, an objecta target of assessment is provided by WP 5, WP 6 or other stakeholders.WP6 Architectural Board. The objecttarget of assessment is provided using the

proposal information/criteria. Following CAMSS (In the e-SENS reference materials, the following sets of proposal information/criteria are available.

Proposal information/criteria based on CAMMS Section 3.1) and the ADMS specification v 0.8, the. The main categories of criteria for the documentation of format during the proposal

step comprise asset description and relationship (status) categories. For reference, the CAMMS information/criteria are given in Appendix 1 (Section 8.2 of the current document).

Proposal information/criteria based on WP6 deliverable "D6.1 Executable ICT Baseline Architecture" (section 2.5, "Building Block Evaluation Model"). For reference, the D6.1

information/criteria are given in Appendix 1 (Section 8.3 of the current document). Proposal information/criteria based on the ADMS concept "Semantic Asset" (section 5.5.1 of

the document "ADMS. Asset Description Metadata Schema. Specification Version 1.00. Release date 18/04/2012").

In case of a software component, the proposal information/criteria may be based on the ADMS.SW main concepts: software project, software release, software package, software

repository (section 4.2 of the document "Asset Description Metadata Schema for Software 1.00. ADMS.SW 1.00. Release date 20/05/2012"). As WP3 will not evaluate SBB, this set is

here mentioned for the sake of completeness and in case of possible subsequent modifications.


These proposal information/criteria are integrated into one set according to the following tabletwo principles: (1) merging the information/criteria from CAMMS, ADMS, and the D6.1 deliverable and

(2) keeping ADMS as the central model for the merge.

The following sub-sections include (1) the complete list of proposal information/criteria and (2) for

convenience, the mandatory information/criteria in the recommended order of presentation.

Terminological note. Different sources refer differently to the proposal items: D6.1 - as "information

and criteria"; ADMS - as "property" and "relationship"; CAMMS - as "criteria". In the current document, the proposal items are referred to as "information/criteria".

3.1.1 Complete list of proposal information/criteria

The complete list for the proposal information/criteria given in the table below comprises the following fields.

Nr - category number. Category - CAMSS category or D6.1.

Description - category description. Reference - reference for the information/criteria (CAMSS number, ADMS, or D6.1). Note:

only CAMMS information/criteria that are included in the ADMS v 1.0 are given. Information/criteria - name of the information/criteria.

Description - description of the information/criteria, or reference to source information. M/O - mandatory or optional. The table indicates only mandatory (M) items; all other

information/criteria are optional. Mandatory information/criteria are: (1) those that are used to Identify an asset according to the table in ADMS Section 4; (2) those that are

included in the deliverable D6.1, but are not assessed by WP3. Note: in case of items assessed by WP3, there can still exist a preliminary optional assessment by WP6 included in

the proposal.


Nr.

Category DescriptionDescrip-tion

Nr.Refe-rence

CriteriaInformation/criteria

Description M/O

1 Asset description CAMMS asset descrip-tion

ProvidingPro-viding the general informationinfor-mation for the proposed formal specification or standardisaton organisation.TOA

CAMSS P.1

Alternative nameName

Alternative name for the asset. Note: this information may be used to provide additional access points, e.g. allowing indexing of any acronyms, nicknames, shorthand notations or other identifying information under which a user might expect to find the assetName of the asset

CAMSS P.2

Date ofcreation

Creation date of this version of the asset

M

CAMSS P.3

Date of last modification

Date of latest update of asset M

CAMSS P.4

Description Descriptive text for the asset , if applicable including its purpose and functions

M

CAMSS P.5

ID URI for the asset M

ADMS Identifier Any identifier for the asset M

CAMSS P.6

Keyword Word or phrase to describe the asset


Nr.


Nr.Refe-rence


Description M/O

P.7 ADMS

Metadata date Alternative name

Date of the most recent update of the metadata for the AssetAlternative name for the asset. Note: this information may be used to provide additional access points, e.g. allowing indexing of any acronyms, nicknames, shorthand notations or other identifying information under which a user might expect to find the asset

CAMSS P.7

Name Name of the asset. See also D6.1 Section 2.5.1

M

CAMSS P.8

Version Version number or other designation of the asset

M

ADMS Format Format in which an asset is distributed (e.g. PDF, XSD, RDF/XML, HTML, ZIP). See also ADMS Section 6.3

M

ADMS Version notes Description of changes between this version and the previous version of the Asset

2 Relationship CAMMS relation-ship

Providing the informationinformation on the status of the specific formal specificationTOA being proposed.

CAMSS P.9 , D6.1

Asset type Classification of an asset according to a controlled vocabulary, e.g. code list, metadata schema. See also ADMS Section 6.2, D 6.1 Section 2.5.4

M

ADMS Contact point Contact point for further information about an asset

CAMSS P.10

Current version Current or latest version of the asset


Nr.


Nr.Refe-rence


Description M/O

ADMS Home page A Web page that is fully dedicated to the asset

P.11 ADMS

Documentation Included asset

Document that further describes an asset or give guidelines for its useAn asset that is contained in the asset being described, e.g. when there are several vocabularies defined in a single document

P.12 ADMS

Domain Included item

Government sectoritem that an is contained in the asset or repository applies to, (e.g. “law” or“environment” according toa concept in a controlled vocabulary, an individual code in a code list or any other ‘atomic’ element)

CAMSS P.13

Interoperability level

Level according to the European Interoperability Framework (EIF 2.0) for which an asset is relevant

M

CAMSS P.14

Language Language of an asset if itsit contains textual information, e.g. the language of the terms in a controlledvocabulary or the language that a specification is written in

CAMSS P.11

Main documentation

The main documentation or specification of the asset

ADMS Metadata language

Language of the metadata for the asset

ADMS Metadata publisher

Organisation making the metadata for the asset available

ADMS Next version Newer version of the asset

CAMSS P.15

Previous version

Older version of the asset


Nr.


Nr.Refe-rence


Description M/O

CAMSS P.16

Publisher Organisation responsible for a repository,making the asset or releaseavailable

M

ADMS Related asset Assets related to the asset

ADMS Related documentation

Documentation that contains information related to the asset

ADMS Related web page

A Web page that contains information related to the asset

P.17ADMS

ReleaseDistribution

Implementation of the asset in a particular format

ADMS Repository origin

Repository that contains the primary description of the asset

ADMS Sample Sample of the asset

CAMSS P.18

Spatial coverage

Geographic region or jurisdiction to which the asset applies

CAMSS P.20

Status Status of the asset (Completed, Under development, Deprecated, Withdrawn), see ADMS sections 5.6.13 and and 6.10

ADMS Temporal coverage

Time period relevant to the asset, e.g. its validity

P.19ADMS

SubjectTheme Theme or subject of an asset, e.g. “elections” or “immigration” according to a general or domain specific controlled vocabularyTheme or sector to which the asset applies

P.20ADMS

StatusTranslation

IndicationTranslation of the maturity of an asset or release

3 D6.1 Items from D6.1 not included above

D6.1 Relationship to ICT Strategies in EC/MS

See D6.1 section 2.5.2

D6.1 Requirements See D6.1 section 2.5.3 M

D6.1 Generic/Specific See D6.1 section 2.5.5 M


Nr.


Nr.Refe-rence


Description M/O

D6.1 Architecture Framework / Component Specifications

See D6.1 section 2.5.6 M

D6.1 Standards See D6.1 section 2.5.7

D6.1 Open Source See D6.1 section 2.5.8 M

D6.1 Relationship and Couplings


D6.1 Related Artifacts and Solution Building Blocks


D6.1 Ownership See D6.1 section 2.5.11 M

D6.1 Life Cycle Management


D6.1 In use? See D6.1 section 2.5.13 M

D6.1 Technical Maturity


D6.1 Business Maturity


D6.1 Market Maturity


D6.1 Evaluation See D6.1 section 2.5.17 M

Table 2. The complete list of proposal information/criteria based on CAMMS, D6.1, and ADMS

Further relevant information/criteria from ADMS and other sources can be included under this main classification.


[3.1] The consideration step

[3.1.1] In the consideration step, consideration criteria are used before the actual assessment, to validatemandatory proposal information received and relevance of the proposal. Following CAMSS (Section 3.2), the main categories of criteria for the documentation of format during the consideration step are correctness, relevance, legislation and regulation, together with their corresponding criteria presented in the table below./criteria

These criteria check the validity of the information and the relevance of the proposed formal

specification. The criteria are defined as YES/NO questions, and the questions should be answered with YES in order to consider the formal specification for further assessment. The criteria could be

adapted by the assessment panel to fit their specific needs and requirements.

For convenience, the following table comprises the mandatory proposal information/criteria in the

recommended order of presentation, omitting the CAMMS category data and the M/O field.

Nr.Reference CategoryInformation/criteria

Description Nr.

Criteria

1CAMSS P.7 CorrectnessName The proposal for the assessment of formal specifications is provided correctly and the necessary documentation is providedName of the asset. See also D6.1 Section 2.5.1

C.1

Is the proposal information correctly provided?

C.2

Is the complete documentation provided for performing the assessment?



Description Nr.

Criteria

CAMSS P.2 RelevanceDate of creation The proposal for the assessment of formal specifications falls within the relevant areas of the public administrationCreation date of this version of the asset

C.3

Is the functional area of application for the formal specification addressing e-SENS Building Blocks?

CAMSS P.3 Date of last modification Date of latest update of asset

CCAMSS P.4

Description Is the formal specification meeting the business needs?Descriptive text for the asset (if applicable, including its purpose and functions)

CAMSS P.5 ID URI for the asset

ADMS Identifier Any identifier for the asset

3CAMSS P.8 Legislation and regulation Version

The proposal for the assessment of formal specifications does not impede any legislative or regulatory requirements.

C.5

Is the formal specification not hampering the regulatory requirements for the area of application?Version number or other designation of the asset

ADMS Format Format in which an asset is distributed (e.g. PDF, XSD, RDF/XML, HTML, ZIP). See also ADMS Section 6.3

CAMSS P.9, D6.1 Asset type Classification of an asset according to a controlled vocabulary, e.g. code list, metadata schema. See also ADMS Section 6.2, D 6.1 Section 2.5.4



Description Nr.

Criteria

CAMSS P.13 Interoperability level Level according to the European Interoperability Framework (EIF 2.0) for which an asset is relevant

CAMSS P.16 Publisher Organisation making the asset available

D6.1 Requirements See D6.1 section 2.5.3

D6.1 Generic/Specific See D6.1 section 2.5.5

D6.1 Architecture Framework / Component Specifications


D6.1 Open Source See D6.1 section 2.5.8

D6.1 Relationship and Couplings See D6.1 section 2.5.9

D6.1 Related Artifacts and Solution Building Blocks


D6.1 Ownership See D6.1 section 2.5.11

D6.1 Life Cycle Management See D6.1 section 2.5.12

D6.1 In use? See D6.1 section 2.5.13

D6.1 Technical Maturity See D6.1 section 2.5.14

D6.1 Business Maturity See D6.1 section 2.5.15

D6.1 Evaluation See D6.1 section 2.5.17

RelevantTable 3. The mandatory proposal information/criteria in the recommended order of presentation

3.2 The consideration step

In the consideration step, consideration criteria are used to preliminarily evaluate quality of

information/criteria received and relevance of the proposal. Following CAMSS (Section 3.2), the main categories of criteria for the consideration step are correctness, relevance, legislation and

regulation, as detailed in the table below.


The criteria are defined as YES/NO questions, and all answers should be YES in order to consider the formal specification for further assessment. The criteria may be adapted by the Assessment Panel to

fit their specific needs and requirements; in particular, relevant additional criteria from ADMS and other sources can be included under this main classification. The Assessment Panel also decides how

to make the checks needed. Note. All the checks made in the consideration step are preliminary and the conclusions may change in the following steps.

Nr. Category Description Nr. Criteria1 Correctness The proposal for the assessment is provided

correctly and the necessary documentation is provided

C.1 Is the proposal information correctly provided?

Is the proposal provided by an eligible entity?Is the proposal provided within the given time-frame?

C.2 Is the documentation provided for performing the assessment complete?

2 Relevance The proposal for the assessment falls within the relevant areas of the public administration

C.3 Is the purpose of the TOA related and of interest to the e-SENS community?

C.4 Is the TOA meeting its purpose?3 Legislation

and regulation

The proposal for the assessment does not impede any legislative or regulatory requirements.

C.5 Is the TOA not violating the regulatory requirements for the area of application?Is the TOA sufficiently is supported by the legislation implemented in respective jurisdictions?

Table 4. Consideration criteria


4 The Assessment Step

The set of criteria provided in thisThis section is quite broad to satisfy proposes a method for integration of assessment ofcriteria from different building blocks. During sources and presents

three sets of criteria based on the e-SENS Technical Annex, the CAMSS assessment relevant criteria are selected on depending on the object presented for evaluationcriteria, and other sources.

[3.2] AIntegration of assessment criteria from various sources

The assessment criteria originate from various sources. First, a number of evaluationsustainability

assessment criteria have been provided in the e-SENS Technical Annex (pp 60-62) and in discussions with the e-SENS project participants. Moreover, inSecond, CAMSS (Section 3.3), among) presents the

following main categories of criteria for the assessment step are the following: applicability, maturity, openness, intellectual property rights, market support, and potential.

The first part of this section presents the assessment criteria based on the e-SENS Technical Annex, the second part - the CAMSS assessment criteria.

[3.2.1.1] Merging different assessment criteria

In the next version of this document the sets of criteria presented below will be merged, taking into

account also other relevant e-SENS materials, Third, there are other frameworks / standards (such as Deliverable D6.1. There are several ways for merging.ADMS, WP6 deliverable D6.1, ISO/IEC

152888, ISO/IEC 25000 series standards, and other) potentially suitable for assessment. Fourth, the criteria should be aligned with the WP5 requirement model, containing scenarios, use cases, and

types of requirements (legal, business, technical and be functional and non-functional).

There are several ways for integration of these criteria.

1. Taking the Technical Annex as the basis and, integrating CAMSS criteria into the criteria of the Technical Annex, and aligning with the standards and the WP5 requirement model.

2.[1.] Taking the CAMSS criteria as the basis and merging the criteria form the Technical Annex into CAMSS.

[2.] Taking some other frameworks / standards (such as ADMS, D6.1, ISO/IEC 152888, ISO/IEC 25000 series standards, Open Software Business Readiness Rating, or other) as the basis and

integrating Technical Annex and CAMSS criteria into thisthe selected framework.


We would propose the secondAs proposed by WP3, this section presents a solution based on the first option - taking the CAMSS criteria as the basis and merging the criteria form the Technical

Annex into CAMSS. The reason is that the CAMSS criteria are already relatively well elaborated and it is easier to complement them than vice versa. The final decision depends on the parties who will use

the Guidelines.

One problem with the CAMSS criteria is that they mainly address specifications, so they have to be

complemented for solution building blocks, using for example ADMS.SW or other relevant sources.

Assessmentintegrated assessment criteria based on the e-SENS Technical Annex, the CAMSS

assessment criteria, and other sources, taking into account also other relevant e-SENS materials, such as deliverable D6.1 and the WP5 requirement model.

According to the Technical Annex, analysis of sustainability of the building blocks will take into account standards, business needs, and policy frameworks., and business needs. Input for the

analysis will be provided by WP5, WP6, and other relevant sources. The following sub-sections are devoted to these sets of criteria. In the tables, an acronym TOA is used for the notion of "target of

assessment".

To preserve compatibility with CAMSS, the CAMSS categories are fully integrated into the

assessment criteria based on the e-SENS Technical Annex and complemented where necessary, indicating the source for the additional criteria.

The sets of criteria provided in this section are quite extensive and suitable to perform assessment of different building blocks. During a specific assessment, relevant criteria are selected depending on

the target presented for evaluation. Additional criteria may be introduced by the Assessment Panel, for example longevity / experience of the owners, volatility (likelihood to change, ability to be

compatible with former versions), adaptability to other situations/contexts, adoption (past, short term, long term), and others.

Some of the terms in the criteria (e.g., “significant market share”) may be perceived differently by different respondents. Adjustment of these terms is left to the Assessment Panel, as it would be too

restrictive to specify these terms precisely in the current document, for example giving exact numerical threshold values.


4.1 Standardisation criteria

The set of standardisation criteria includeincludes technical, organisational and semantic criteria that

can be used to assess the maturity for standardizing the building blocks. Potential criteria include the following.

Criteria resulting from EIA and EIF. In particular, the criteria present in the EIF/EIA and missing in the current set of criteria will be considered for inclusion. Criteria include the EIF

conceptual model for public services (presented also in Technical Annex p. 36), as well as coverage of A2A services. Examples: user authorization and aggregation of services.

Interoperability between the LSPs and Building Blocks in Member States. Change management and maintenance, licensing, common specifications of components,

openness, support and manuals, security features, service levels (availability of the building block, emergency helpdesk) as well as the deployment and running of these components in

an organisation’s IT infrastructure. Life cycle workflow over the project lifetime (development, revisions, updates, work in

progress, and incremental version releases). Maintaining building blocks beyond the project’s lifetime and who will be responsible for the

different parts. Interoperability between the LSPs and Building Blocks in Member States.

The For reference, the following CAMSS assessment categories of criteria may beare included in this section.

Maturity Openness

Intellectual property rights

Relevant criteria from international standards, ISO/IEC 12207:2008 "Systems and software

engineering. Software life cycle processes" and ISO/IEC 25000:2005 "Software Engineering. Software product Quality Requirements and Evaluation (SQuaRE)" have been taken into account.

The set of standardisation criteria can be assessed within WP3 in close collaboration with WP6, WP5, industry partners, and other relevant stakeholders.


4.1.1[3.2.2] Maturity

The following table presents maturity criteria based on CAMSS.

Nr Cate-gory

Description Nr Sub-Cate-gory

Description Nr Criteria

2 Ma-turity

A TOA should in itself be mature enough for adoption by public administrations. This category addresses the development status, the quality, guidelines and stability of the TOA.

2.1 Deve-lop-ment status

For the ‘development status’, the current development status of the TOA in the development cycle is addressed.

A.9 Has the TOA been sufficiently developed and in existence for a sufficient period to overcome most of its initial problems?

2.2 Quali-ty

For ‘quality’, the level of detail in the TOA and the conformance of implementations is addressed.

A.10 Are there existing or planned mechanisms to assess conformity of the implementations of the TOA (e.g. conformity tests, certifications)?

A.11 Has the TOA sufficient detail, consistency and completeness for the use and development of products?

2.3 Guide-lines

For the ‘guidelines’, the existence of implementation guidelines or reference implementations is addressed.

A.12 Does the TOA provide available implementation guidelines and documentation for the implementation of products?

A.13 Does the TOA provide a reference (or open source) implementation?

2.4 Stabi-lity

For ‘stability’, the level of change to the TOA and the stability of underlying technologies is addressed.

A.14 Does the TOA address backward compatibility with previous versions?

A.15 Have the underlying technologies for implementing the TOA been proven?

A.15 Have the underlying technologies for implementing the TOA been stable?

A.15 Have the underlying technologies for implementing the TOA been clearly defined?If applicable, other indicators may be applied, such as volume of transitions, frequency of transactions, and so on.

Table 5. Assessment criteria for standardization: maturity


4.1.2 Openness

The following table presents openness criteria based on CAMSS.

Nr Cate-gory



3 Openness

A TOA should be sufficiently open and available, to be relevant for adoption by public administrations. This category addresses the openness of the organisation maintaining the TOA and its decision-making process, and openness of the documentation and accessibility of the TOA.

3.1 Orga-nisa-tion

For the ‘openness’ of the organisation maintaining the TOA, the level of openness for participating in this organisation is addressed.

A.16 Is information on the terms and policies for the establishment and operation of the organisation maintaining the TOA publicly available?

A.17 Is participation in the creation process of the TOA open to all relevant stakeholders (e.g. organisations, companies or individuals)?

3.2 Pro-cess

For the ‘process’, the level of openness regarding the development and decision-making process for the TOA is addressed.

A.18 Is information on the standardisation process publicly available?

A.19 Information on the decision making process for approving TOAs is publicly available?

A.20 Are the TOAs approved in a decision making process which aims at reaching consensus?

A.21 Are the TOAs reviewed using a formal review process with all relevant external stakeholders (e.g. public consultation)?

A.22 All relevant stakeholders can formally appeal or raise objections to the development and approval of TOAs?

3.3 Docu-men-tation

For the openness of the ‘documentation’, the accessibility and availability of the documentation of the TOA is addressed.

A.23 Relevant documentation of the development and approval process of TOAs is publicly available (e.g. preliminary results, committee meeting notes)?

A.24 Is the documentation of the TOA publicly available for implementation and use on reasonable terms?


Table 6. Assessment criteria for standardization: openness

4.1.3 Intellectual property rights

The following table presents intellectual property rights criteria based on CAMSS.

Nr Cate-gory



4 Intel-lec-tual pro-perty rights

A TOA should be licensed on (F)RAND terms or even on a royalty-free basis in a way that allows implementation in different products. This category addresses the availability of the documentation on the IPR and the licences for the implementation of the TOA.

4.1 IPR Docu-men-tation

For the ‘documentation of the intellectual property rights’, the availability of the information concerning the ownership rights of the TOA is addressed.

A.25 Is the documentation of the IPR for TOAs publicly available?

4.2 Licen-ces

For the ‘licences’ within the intellectual property rights, a (fair) reasonable and non-discriminatory ((F)RAND) or even royalty-free basis is addressed for the use and implementation of the TOA.

A.26 Is the TOA licensed on a (F)RAND basis?

A.27 Is the TOA licensed on a royalty-free basis?

Table 7. Assessment criteria for standardization: intellectual property rights

4.1.4 Life cycle, maintenance, service levels, security

Depending on the target of assessment, criteria presented in the following tables may be relevant for assessment. These criteria are selected according to the standardisation criteria in the Technical

Annex and are based on international standard ISO/IEC 12207:2008 "Systems and software engineering. Software life cycle processes", as well as on the ISO/IEC 20000, ISO/IEC 25000, and

ISO/IEC 27000 series of standards.

Criteria referring to interoperability are presented in the next section.


The range of possible criteria presented in the above standards is extremely wide. Only selected criteria most relevant to the requirements presented in the Technical Annex are presented below.

This selection may be expanded or restricted during a specific assessment.

Nr Cate-gory



Life Cycle ma-na-ge-ment

The life cycle management process provides life cycle policies, processes, and procedures.

There should exist a life cycle management process.

Is an organisation available for providing life cycle policies, processes, and procedures for the TOA?

Is life cycle workflow over the project lifetime (development, revisions, updates, work in progress, and incremental version releases) established?

Mainte-nan-ce

The maintenance process provides cost-effective support to the TOAs during their life-cycle, including change management.

Main-te-nance imp-le-men-tation

There should be plans and procedures for conducting the maintenance activities.

Does a maintaining organisation exist?

Has the maintaining organisation developed, documented, and executed plans and procedures for conducting the maintenance activities?

Prob-lem ana-lysis

The problem reports or modification requests should be analyzed for their impact.

Does the maintainer have procedures for analyzing the problem reports or modification requests for their impacts on the organization, the existing system, and its interfaces?

Modi-fica-tion imp-le-men-tation

It should be determined and documented which software items need to be modified.

Does the maintainer have procedures for determining which software units and versions need to be modified?


Nr Cate-gory



Mig-ration

Migration of a system or software product (including data) should be planned, documented, and performed.

Are there procedures for developing, documenting, and executing migration plans, including the system, data, and users?

Dis-posal

Ending the existence of a TOA should be planned, documented, and performed.

Are there procedures for developing, documenting, and executing disposal plans for TOAs?

Ser-vice le-vels

The services related to the TOA should be agreed with the customers.

SLA If applicable, there should be service level agreements relating to the availability of the TOA.

Do SLAs relating to the availability of the TOA exist?

If applicable, there should exist emergency helpdesk for the TOA.

Is there an emergency helpdesk for the TOA?

Secu-rity

Systems, data, and resources should be protected from accidental or malicious acts.

ISMS The maintainer should have an information security management system.

Does the maintainer have a system based on a business risk approach, to establish, implement, operate, monitor, review, maintain and improve information security?

Iden-tifica-tion

Information security requirements should be understood.

Has the maintainer analysed and understood the information security requirements related to the TOA?

Risks Information security risks should be assessed.

Has the maintainer assessed the information security risks related to the TOA?

Cont-rols

Information security controls should be selected and implemented.

Has the maintainer selected and implemented information security controls related to the TOA?

Moni-tor

The effectiveness of the ISMS should be monitored, maintained, and improved.

Is the maintainer monitoring, maintaining, and improving the effectiveness of the ISMS?

Table 8. Assessment criteria for standardization: life cycle


4.2 Criteria for alignment with existing policy frameworks

This set of criteria addresses alignment of the individual building blocks with existing policy

frameworks. Potential criteria include the following.

Criteria resulting from EIA and EIF. In particular, the criteria present in the EIF/EIA and

missing in the current set of criteria will be considered for inclusion. Criteria include the EIF conceptual model for public services (presented also in Technical Annex p. 36), as well as

coverage of A2A services. Examples: user authorization and aggregation of services. Alignment with the EIF and EIA.

The proposed solutions must be compliant with the EU legal framework on data protection and legislation on electronic signatures according to the WP5 Requirement model.

Alignment with national frameworks of the participating countries. Potential incompatibilities between Member States.

Maintenance of the legal validity of information exchanged across borders. Adherence to the data protection legislation in both originating and receiving countries.

For reference, the following CAMSS categories of criteria are included in this section.

Applicability

Potential

Alignment with existing policy frameworks will be assessed in collaboration with other Work

Packages, including WP4, the e-SENS legal expertise centre.

4.2.1 Basic alignment criteria

The following table presents basic criteria for alignment with existing policy frameworks based on

requirements from the Technical Annex.

Nr Cate-gory



Interope-rabi-lity

The LSPs and Building Blocks in Member States should be interoperable.

EIA The TOA should confirm to the European Interoperability Architecture.

Are there any disagreements between the TOA and the EIA?

EIF The TOA should confirm to the European Interoperability Framework.

Are there any disagreements between the TOA and the EIF?


Nr Cate-gory



A2A ser-vices

The TOA should support A2A services, if applicable.

If applicable, does the TOA support A2A services, eg user authorization and aggregation of services?

Com-pli-ance

The proposed solutions should be compliant with the EU legal framework on data protection and legislation on electronic signatures.

Data pro-tec-tion

The proposed solutions should be compliant with the EU legal framework on data protection.

Are the proposed solutions compliant with the EU legal framework on data protection?

Ele-ctro-nic signa-tures

The proposed solutions should be compliant with the EU legislation on electronic signatures.

Are the proposed solutions compliant with the EU legislation on electronic signatures?

Me-mber Sta-tes

Alignment with national frameworks of the participating countries and avoiding potential incompatibilities between Member States.

Nati-onal fra-me-works

Alignment with national frameworks of the participating countries.

Is the TOA aligned with national frameworks of the participating countries?

In-com-pati-bili-ties

Avoiding potential incompatibilities between Member States.

Are potential incompatibilities between Member States avoided?

Legal The legal validity of information exchanged must be maintained across borders.

Infor-ma-tion

Maintenance of the legal validity of information exchanged across borders.

Is the legal validity of information exchanged maintained across borders?

Pro-tec-tion

Data protection legislation in both originating and receiving countriesmust be respected.

Data pro-tec-tion

Adherence to the data protection legislation in both originating and receiving countries.

Is data protection legislation in both originating and receiving countries respected?

Table 9. Assessment criteria for alignment: basic


4.2.2 Applicability

The following table presents applicability criteria based on CAMSS.

Nr Cate-gory



1 Appli-ca-bility

A TOA should be usable and easy implementable in different products to be relevant for adoption by public administrations. This category addresses the definition of functional scope and area of application, the possible reusability in other areas, the possible alternative specifications, the compatibility and dependency on other specifications or technologies.

1.1 Area of appli-cation

For the ‘area of application’, the functionalities and intended use of the TOA are addressed within the context of interoperability and eGovernment.

A.1 Does the TOA address and facilitate interoperability between public administrations?

A.2 Does the TOA address and facilitate the development of eGovernment?

1.2 Requ-ire-ments

For the ‘requirements’, the functional and nonfunctional requirements for using and implementing the TOA are addressed. This criterion is related to the use of assessment scenario 3

A.3 Are the functional and nonfunctional requirements for the use and implementation of the TOA clearly defined?

1.3 Re-usabi-lity

For ‘reusability’, the level of reusability of the TOA in the same or other areas of application is addressed.

A.4 Is the TOA applicable and extensible for implementations in different domains?

1.4 Alter-nati-ves

For the ‘alternatives’, the degree to which the TOA adds value compared to alternative TOAs in the same area of application is addressed.

A.5 Does the TOA provide sufficient added value compared to alternative TOAs in the same area of application?

1.5 Com-pati-bility

For ‘compatibility’, the compatibility of the TOA with other TOAs in the same area of application is addressed.

A.6 Is the TOA largely compatible with related (not alternative) TOAs in the same area of application?


Nr Cate-gory



1.6 De-pen-den-cies

‘Dependencies’ addresses the degree of independence of the TOA from specific vendor products, platforms or technologies.

A.7 Is the TOA largely independent from specific vendor products?

A.8 Is the TOA largely independent from specific platforms or technologies?

Table 10. Assessment criteria for alignment: applicability

4.2.3 Potential

The following table presents criteria for potential based on CAMSS.

Nr Cate-gory



6 Po-ten-tial

A TOA should have sufficient and positive future consequences, evolution and impact for being adopted by public administrations. This category addresses the consequences and impact of using or adopting the TOA, the advantages and risks, the maintenance and possible future developments.

6.1 Im-pact

For the ‘impact’, the minimisation of the consequences of using and adopting the TOA is addressed. The consequences can be evaluated and described in terms of different aspects.

A.33 Is there evidence that the adoption of the TOA positively impacts organisational processes?

Is there somebody who directly benefits from the specification?

A.34 Is there evidence that the adoption of the TOA positively impacts the migration of current systems?

A.35 Is there evidence that the adoption of the TOA positively impacts the environment?

A.36 Is there evidence that the adoption of the TOA positively impacts the financial costs?

A.37 Is there evidence that the adoption of the TOA positively impacts the security?

A.38 Is there evidence that the adoption of the TOA positively impacts the privacy?


Nr Cate-gory



A.39 Is there evidence that the adoption of the TOA positively impacts the administrative burden?

A.40 Is there evidence that the adoption of the TOA positively impacts the disability support?Is there evidence that the adoption of the TOA advances or is supported by the emerging technologies such as cloud computing or Internet of Things?

6.2 Risks For the ‘risks’, the level of uncertainty is addressed for using and adopting the TOA

A.41 What are the risks? What is the probability of their emergence? Are they related to the adoption of the TOA?

6.3 Main-te-nance and future deve-lop-ments

For the ‘maintenance’ and future developments, the support and the planned or existing actions to maintain, improve and develop the TOA in the long term are addressed.

A.42 Does the TOA have a defined maintenance organisation?

A.43 Does the maintenance organisation for the TOA have sufficient finances and resources for the long term?

A.44 Does the TOA have a defined maintenance and support process?

A.45 Does the TOA have a defined policy for version management?

Table 11. Assessment criteria for alignment: potential

4.3 Business need criteria

The set of business need criteria addresses the interest of the market for the target of assessment, its validation and user aspects of the individual building blocks., and user-related aspects. The

business need criteria should be aligned with the WP5 requirement model, containing scenarios, use cases, and types of requirements (legal, business, technical and be functional and non-functional).

Potential criteria include the following.


Need for the building block by end users. For example, this need could be characterized by potential change in the quality of the service delivered to the citizen/business by the

administration before and after adopting the building block. Where applicable, SWOT analysis may be used.

Opportunities for software/service providers to put the building block into use. For example, availability of a commercially-oriented, robust Business Plan for investment, built upon an

underlying ‘commercially sustainable’sustainable business model. A business case should also take into account how a building block target of assessment will help public partners in

achieving their missions. Market/business assessment at national level, indicating whether national governments can

easily comply with them. Relevance of having the same components integrated as European (shared) building blocks

across different Use Cases and their usefulness in the development of eGovernment cross-border services.

Potential of the building block to be adopted by the market and be used in cross-border eGovernment services.

Existing implementations of specifications or other objects of assessment. Where applicable the costs and benefits of adopting the building block, including the

assessment of the Return on Investment. Cost and benefit analysis can be performed per each stakeholder group involved (e.g. government, private companies, users, etc). It is also

possible to consider different time frames (e.g. short, medium and long run), as outcomes of the costs and benefits will change depending on the time frame considered.

Possibility for a broader geographic and sector usage. TheWhere applicable, the confidence level of the Assessment Panel in the relevance of the

business need/purpose may be indicated (e.g. strong intuition / validated by technical experts / validated by administrations-users / reflected in national-international regulation).

For reference, the following CAMSS assessmentcategory of criteria may beare included in this section.

Intellectual property rights Market support

This set of criteria will be assessed by involving stakeholders both inside and outside the e-SENS project, e.g. by using questionnaires, semi-structured interviews, or other means to get the opinion

of important end users and software/service providers.


[3.3] Alignment with existing policy frameworks

This set of criteria addresses alignment of the individual building blocks with existing policy

frameworks. Potential criteria include the following.

Alignment with the EIF and EIA.

Requirements that the proposed solutions are compliant with the EU legal framework on data protection and legislation on electronic signatures.

Alignment with national frameworks of the participating countries. Potential incompatibilities between Member States.

Maintenance of the legal validity of information exchanged across borders. Adherence to the data protection legislation in both originating and receiving countries.

4.3.1[3.3.1] Basic business need criteria

The following CAMSS assessmenttable is based on the basic business need criteria based on the Technical Annex.

Nr Cate-gory



Busi-ness need

Need for the TOA by end users.

Chan-ge

Potential change in the quality of the service delivered to the citizen/business by the administration before and after adopting the TOA.

Are positive changes in the quality of the service delivered to the citizen/business by the administration before and after adopting the TOA foreseen?

Usa-ge

Opportunities for software/service providers to put the TOA into use.

Do opportunities exist for software/service providers to put the TOA into use?

Busi-ness plan

Availability of a commercially-oriented, robust Business Plan for investment, built upon an underlying ‘commercially sustainable’ business model.

Is the Business Plan for investment built upon an underlying ‘commercially sustainable’ business model?


Nr Cate-gory



Busi-ness case

A business case should take into account how a TOA will help public partners in achieving their missions.

Does the business case takes into account how a TOA will help public partners in achieving their missions?

Sha-ring

Relevance of having the same components integrated as European (shared) building blocks across different Use Cases.

Could the TOA be integrated as a European (shared) building block across different Use Cases?

Cross- boar-der

Usefulness of the TOA in the development of eGovernment cross-border services.

Could the TOA be useful in the development of eGovernment cross-border services?

Mar-ket

Potential of the TOA to be adopted by the market and be used in cross-border eGovernment services.

Does the TOA has potential to be adopted by the market and be used in cross-border eGovernment services?

ROI Where applicable the costs and benefits of adopting the TOA, including the assessment of the Return on Investment.

If applicable, is evaluation of the costs and benefits of adopting the TOA available?

Geo-gra-phic

Possibility for a broader geographic and sector usage.

Is there a possibility for a broader geographic and sector usage?


Table 12. Assessment criteria may be included in this section.for business need: basic

4.3.2 Market support

The following table presents market support criteria based on CAMSS.

Nr Cate-gory



5 Mar-ket sup-port

A TOA should have sufficient market acceptance and support in order to be adopted by public administrations. This category addresses the proven and operational implementations of the TOA, the market share and demand for the products, and the support from users and communities.

5.1 Imple-men-ta-tions

For the ‘implementations’, the existence of proven and best practice implementations for the TOA is addressed, in different domains and by different vendors.

A.28 Has the TOA been used for different implementations by different vendors/suppliers?

A.29 Has the TOA been used in different industries, business sectors or functions?

5.2 Mar-ket de-mand

For ‘market demand’, the penetration and acceptance of products implementing the TOA in the market is addressed.

A.30 Do the products that implement the TOA have a significant market share of adoption?

5.3 Users For the ‘users’, the diversity of the end-users of the products implementing the TOA is addressed.

A.31 Do the products that implement the TOA target a broad spectrum of end-uses?

5.4 Inte-rest gro-ups

For the ‘interest groups’, the degree of support from different interest groups is addressed.

A.32 Has the TOA a strong support from different interest groups?

Payer For the 'Payer' the existence of groups ready to pay for the service is addressed.

Who is willing to pay for the service?

Com-peti-tion

For the 'Competition' the existence of competing solutions is addressed.

To what extent the TOA competes with other solutions available in member countries?

Sup-port

For the 'Support' the existence of support for the market is addressed.

Is there any support available for the market in using the TOA?

Table 13. Assessment criteria for business need: market support



5[4] The Recommendation Step: Criteria and Classification

In the recommendation step, recommendation criteria are applied, concluding with a classification of the target of assessment to be reported back to WP6 Architectural Board for using by other Work

Packages.

5.1 Recommendation criteria

Following CAMSS (Section 3.4), the main categories of criteria on the recommendation step are the same as used for grouping the assessment criteria. These categories comprise those from CAMMS

(applicability; maturity; openness; intellectual property rights; market support; potential) and those from the Technical Annex and other sources (life cycle, maintenance, service levels, security; basic

alignment criteria; basic business need criteria). For these main categories, a score is generated based on the number of ‘YES’ answers that indicates the level of meeting the criteria for the target

of assessment. Note 1. A more complicated scale for scores may be used by the Assessment Panel, for example comprising four possible values. Note 2. The Assessment Panel may decide to consider

additional recommendation criteria, such as usability, backward compatibility, and others.

The specified knock-out criteria are checked if they were all met and an answer is provided for the

main categories. The CAMSS Assessment Library and Tools, including the tools for Scenarios 2 and 3, may be used for this purpose

(https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/index.php/CAMSS_Assessment_Library_an

d_Tools). Both tools propose the following Knock-out criteria. Additional criteria may be stated by the Assessment Panel.


https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/index.php/CAMSS_Assessment_Library_and_Tools


Nr Cate-gory



1 Applica-bility

A formal specification should be usable and easy implementable in different products to be relevant for adoption by public administrations....

1.1 Area of application

For the ‘area of application’, the functionalities and intended use of the formal specification are addressed within the context of interoperability and eGovernment.

A.1 Does the formal specification address and facilitate interoperability between public administrations?

(6) (Po-ten-tial)

A.43 Does the maintenance organisation for the formal specification have sufficient finances and resources for the long term?

Based on the discussion in the Assessment Panel, an evaluation and possible comments are provided

for the main categories. A possible scale for the evaluation of the categories could include the following range: Very low, Low, Moderate, High, Very high. The Assessment Panel may assign

different weights to each category to obtain the final evaluation.

The criteria are presented in the following table.

Nr. Category Automated score

Knock-out criteria met?

Evaluation Comments

R.2 (CAMMS) Maturity

R.3 (CAMMS) Openness

R.4 (CAMMS) Intellectual property rights

Technical Annex and other sources

Life cycle, maintenance, service levels, security


Basic alignment criteria

R.1 (CAMMS) Applicability

R.6 (CAMMS) Potential


Basic business need criteria

R.5 (CAMMS) Market support


Table 14. Recommendation criteria

5.2 Proposed Classification

The classification of the target of assessment is based on CAMSS (p 36), Technical Annex (pp 33-35), PEPPOL, and other relevant sources. It uses the assessment and recommendation criteria and

comprises the following classes.

Discarded: the target of assessment is not found relevant and should not be used in the

public administration Observed: the target of assessment is found relevant but should not be used in the current

state or version Accepted: the target of assessment is accepted and could be used in the public

administrations (and for the specific business needs) Recommended: the target of assessment is accepted and is preferred for the use in public

administrations (compared to alternative targets of assessment) Mandatory: the target of assessment is accepted and public administrations are obliged to

use the target of assessment when applicable (or provide an explanation). Note. Depending on the e-SENS mandate, it may be agreed to avoid the "Mandatory" recommendation in the

assessments.

The compliance levels of criteria for LSPs, necessary for a building block to be assessed as satisfying

Accepted or higher levels, may be defined.

For the last three classes, maturity levels (Technical Annex pp 33-35 and PEPPOL) may be assessed in

more detail if needed.


6[5] Organizational aspects of assessment

According to the Technical Annex (p 60), T3.1 will develop a questionnaire form for the assessment of building blocks and discusses how to deal with current LSPs that do not comply to the criteria

upfront.

6.1[5.1] A questionnaire form for the assessment of European Building Blocks

The questionnaire form follows closely the assessment framework, procedure, and criteria, and will

be provided in the subsequent versions of the Guidelines.

6.2 Dealing with current LSPs that do not comply to the criteria upfront

The solution for dealing with current LSPs that do not comply with the criteria upfront is based on a

case-by-case analysis of the sources for non-compliance and application of appropriate solutions. Possible sources and solutions are presented in the following table. The requests indicated in

possible solutions below may be provided by WP3 or other Work Packages. In the course of assessment, additional sources for non-compliance and possible solutions will be considered as

needed.

Source for non-compliance Possible solutionInsufficient presentation of the building block for

assessment (for example, inadequate documentation of format)

Request for improvement of the presentation

A simple problem with reference to an otherwise

acceptable building block

Request to correct the problem in the

subsequent version of the building block

A complex problem with reference to an otherwise acceptable building block

Request to provide a plan to correct the problem in the subsequent versions of the building block

The project is classified as discarded or observed,

with parallel building blocks providing adequate solutions in assessment

Providing a parallel building block for assessment


Source for non-compliance Possible solutionThe project is classified as discarded or observed, with no parallel building blocks providing

adequate solutions in assessment

Providing a request to a relevant EU initiative for initiation of a new building block

Table 15. Dealing with current LSPs that do not comply to the criteria upfront


7 Research update and input for the analysis

Before assessing the building blocks, results of already existing assessments will be identified and re-used (Task T3.1.2 of the Technical Annex). It is expected that the majority of the input for this

subtask will come from WP5 through the work on D5.1 requirements framework, as well as the WP6 activities that are described in the e-SENS executable baseline architecture (D6.1). Other sources are

the library http://joinup.ec.europa.eu/ as well as former LSP material. The assessment results from standardisation bodies or other relevant bodies can be re-used and adapted for the assessment of a

target of assessment of the building blocks.

According to the Technical Annex, this subtask involves identification and re-use of already existing

assessments, but not of the assessment tools and frameworks. Moreover, the research update focuses on assessment of building blocks relevant for e-SENS. The list of materials used for

assessment is provided in the references section of this document.

The following existing assessments are considered.

The feasibility and scenarios for the long-term sustainability of the Large Scale Pilots, including 'ex-ante' evaluation. Task 1,2,3 Report. 11-03-2013, SMART 2012/0059, Deloitte.

Interim evaluation of the ISA programme. Specific contract n°4 under framework contract n°DI/06693, 31 October 2012, Final Report.

http://ec.europa.eu/isa/documents/interim_evaluation_of_the_isa_programme.pdf

The detailed research update will be provided in the subsequent versions of the Guidelines.

Additions to the above list are welcome, please submit them to [email protected].




8 Appendix 1

8.1 References

1. Regulations

1.1. REGULATION (EU) No 1025/2012 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 25 October 2012 on European standardisation. OJ L 316, 14.11.2012, pp 12-33, http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:316:0012:0033:EN:PDF1.2. Amended proposal for a regulation of the European Parliament and of the Council on

guidelines for trans-European telecommunications networks and repealing Decision No 1336/97/EC. COM(2013) 329 final. Brussels, 28.5.2013,

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2013:0329:FIN:EN:PDF2. European wide scale frameworks, projects, and initiatives

2.1.[1.3.] European Interoperability Framework (EIF) for European public services, http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf

2.2.[1.4.] European Interoperability Architecture, http://ec.europa.eu/isa/documents/isa_2.1_eia-finalreport-commonvisionforaneia.pdf

2.3.[1.5.] EFIR, http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-4action_en.htm2.4.[1.6.] Joinup, https://joinup.ec.europa.eu/catalogue/all?current_checkbox=1

2.5.[1.7.] Data Catalog Vocabulary (DCAT), http://www.w3.org/2011/gld/wiki/Data_Catalog_Vocabulary

2.6.[1.8.] BOMOS (Management and Development Model for Open Standards), 2.7.[1.9.] BOMOS2i,

http://www.forumstandaardisatie.nl/fileadmin/os/publicaties/HR_BOMOS_English_translation_Jan2013.pdf

2.8.[1.10.] Open Software Business Readiness Rating (OpenBRR), http://www.oss-watch.ac.uk/resources/archived/brr

3. e-SENS project materials and deliverables3.1. e-SENS Technical Annex v 3.0

3.2. WP6 deliverable "D6.1 Executable ICT Baseline Architecture"4. ADMS and ADMS.SW

4.1. ADMS v 1.0. https://joinup.ec.europa.eu/asset/adms/release/1004.2.[1.11.] ADMS. Asset Description Metadata Schema. Specification Version 1.00. Release date

18/04/2012


https://joinup.ec.europa.eu/asset/adms/release/100

http://www.oss-watch.ac.uk/resources/archived/brr



http://www.w3.org/2011/gld/wiki/Data_Catalog_Vocabulary

https://joinup.ec.europa.eu/catalogue/all?current_checkbox=1

http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-4action_en.htm

http://ec.europa.eu/isa/documents/isa_2.1_eia-finalreport-commonvisionforaneia.pdf

http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2013:0329:FIN:EN:PDF

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:316:0012:0033:EN:PDF


4.3.[1.12.] Asset Description Metadata Schema for Software 1.00. ADMS.SW 1.00. Release date 20/05/2012, https://joinup.ec.europa.eu/asset/adms_foss/asset_release/admssw-100

5.[2.] Common Assessment Method For Standards And Formal Specifications (CAMSS). Final Draft Revision of CAMSS. Version 1.0, March 2012. See also the following sources.

5.1.[2.1.] CAMSS Project – Workshop 2. Agreement on Revision of CAMSS. Wednesday, March 7th 2012 (fail Presentation of EC on CAMSS.ppt)

5.2.[2.2.] CAMSS Assessment Library and Tools, including the tool for Scenario 2, https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/index.php/CAMSS_Assessment_Li

brary_and_Tools5.3. https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/

6. European specific area projects and building blocks6.1. PEPPOL EIA Repository, http://www.peppol.eu/peppol_components/peppol-eia/eia

Applicability

[5.1.1] Potential

Alignment with existing policy frameworks will be assessed in collaboration with WP4, the e-SENS

legal expertise centre.

[5.2] The CAMSS assessment criteria

[3.] TheEuropean project assessments6.2. The feasibility and scenarios for the long-term sustainability of the Large Scale Pilots,

including 'ex-ante' evaluation. Task 1,2,3 Report. 11-03-2013, SMART 2012/0059, Deloitte.6.3. Interim evaluation of the ISA programme. Specific contract n°4 under framework contract

n°DI/06693, 31 October 2012, Final Report. http://ec.europa.eu/isa/documents/interim_evaluation_of_the_isa_programme.pdf

7. International standards and frameworks7.1. TOGAF 9.1, http://pubs.opengroup.org/architecture/togaf9-doc/arch/toc.html

7.2. TOGAF 9.1, Definitions, http://pubs.opengroup.org/architecture/togaf9-doc/arch/chap03.html

7.3. ISO/IEC 12207:2008. Systems and software engineering. Software life cycle processes7.4. ISO/IEC 25000:2005. Software Engineering. Software product Quality Requirements and

Evaluation (SQuaRE). Guide to SQuaRE


http://pubs.opengroup.org/architecture/togaf9-doc/arch/chap03.html

http://pubs.opengroup.org/architecture/togaf9-doc/arch/toc.html


http://www.peppol.eu/peppol_components/peppol-eia/eia

https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/



https://joinup.ec.europa.eu/asset/adms_foss/asset_release/admssw-100

8.2 Proposal information/criteria based on CAMMS

Following CAMSS (Section 3.1) and the ADMS specification v 0.8, the main categories of criteria for the documentation of format during the proposal step comprise asset description and relationship

(status) categories and criteria according to the following table. The table is included for reference.

Nr. Category Description Nr. Criteria Description1 Asset

descriptionProviding the general information for the proposed TOA

P.1 Name Name of the asset

P.2 Date ofcreation


P.3 Date of last modification

Date of latest update of asset

P.4 Description Descriptive text for the asset

P.5 ID URI for the asset

P.6 Keyword Word or phrase to describe the asset

P.7 Alternative name

Alternative name for the asset. Note: this information may be used to provide additional access points, e.g. allowing indexing of any acronyms, nicknames, shorthand notations or other identifying information under which a user might expect to find the asset

P.8 Version Version number or other designation of the asset

2 Relationship Providing the information on the status of the specific TOA being proposed

P.9 Asset type Classification of an asset according to a controlled vocabulary, e.g. code list, metadata schema

P.10 Current version Current or latest version of the asset

P.11 Documentation Document that further describes an asset or give guidelines for its use

P.12 Domain Government sector that an asset or repository applies to, e.g. “law” or“environment” according to a controlled vocabulary


Nr. Category Description Nr. Criteria DescriptionP.13 Interoperability

levelLevel according to the European Interoperability Framework (EIF 2.0) for which an asset is relevant

P.14 Language Language of an asset if its contains textual information, e.g. the language of the terms in a controlledvocabulary or the language that a specification is written in

P.15 Previous version


P.16 Publisher Organisation responsible for a repository, asset or release

P.17 Release Implementation of the asset in a particular format

P.18 Spatialcoverage


P.19 Subject Theme or subject of an asset, e.g. “elections” or “immigration” according to a general or domain specific controlled vocabulary

P.20 Status Indication of the maturity of an asset or release

Table 16. Proposal information/criteria based on CAMMS

8.3 Proposal information/criteria based on WP6 deliverable D6.1

Following the WP6 deliverable "D6.1 Executable ICT Baseline Architecture" (section 2.5, "Building Block Evaluation Model"), a Building Block will be described using the following information/criteria.

The list is included for reference.

Name Relationship to ICT Strategies in EC/MS (in cooperation with WP3) Requirements (in cooperation with WP5) Type Generic/Specific Architecture Framework / Component Specifications Standards Open Source


Relationship and Couplings Related Artifacts and Solution Building Blocks Ownership Life Cycle Management In use? Technical Maturity Business Maturity (WP5) Market Maturity (WP3) Evaluation

For description of individual items from the above list we refer to the "D6.1 Executable ICT Baseline

Architecture" (section 2.5, "Building Block Evaluation Model").

8.4 The CAMSS assessment criteria

The CAMSS original categories, sub-categories, and criteria are presented in the following table. The table is included for reference.

Nr Cate-gory



1 Applica-bility

A formal specification should be usable and easy implementable in different products to be relevant for adoption by public administrations. This category addresses the definition of functional scope and area of application, the possible reusability in other areas, the possible alternative specifications, the compatibility and dependency on other specifications or technologies.

1.1 Area of application

For the ‘area of application’, the functionalities and intended use of the formal specification are addressed within the context of interoperability and eGovernment.

A.1 Does the formal specification address and facilitate interoperability between public administrations?

A.2 Does the formal specification address and facilitate the development of eGovernment?

1.2 Requirements

For the ‘requirements’, the functional and nonfunctional requirements for using and implementing the formal specification are addressed. This criterion is related to the use of assessment scenario 3

A.3 Are the functional and nonfunctional requirements for the use and implementation of the formal specification clearly defined?


Nr Cate-gory



1.3 Reusability

For ‘reusability’, the level of reusability of the formal specification in the same or other areas of application is addressed.

A.4 Is the formal specification applicable and extensible for implementations in different domains?

1.4 Alternatives

For the ‘alternatives’, the degree to which the formal specification adds value compared to alternative formal specificationsTOAs in the same area of application is addressed.

A.5 Does the formal specification provide sufficient added value compared to alternative formal specificationsTOAs in the same area of application?

1.5 Compatibility

For ‘compatibility’, the compatibility of the formal specification with other formal specificationsTOAs in the same area of application is addressed.

A.6 Is the formal specification largely compatible with related (not alternative) formal specificationsTOAs in the same area of application?

1.6 Dependencies

‘Dependencies’ addresses the degree of independence of the formal specification from specific vendor products, platforms or technologies.

A.7 Is the formal specification largely independent from specific vendor products?

A.8 Is the formal specification largely independent from specific platforms or technologies?

2 Maturity

A formal specification should in itself be mature enough for adoption by public administrations. This category addresses the development status, the quality, guidelines and stability of the formal specification.

2.1 Development status

For the ‘development status’, the current development status of the formal specification in the development cycle is addressed.

A.9 Has the formal specification been sufficiently developed and in existence for a sufficient period to overcome most of its initial problems?

2.2 Quality

For ‘quality’, the level of detail in the formal specification and the conformance of implementations is addressed.

A.10 Are there existing or planned mechanisms to assess conformity of the implementations of the formal specification (e.g. conformity tests, certifications)?

A.11 Has the formal specification sufficient detail, consistency and completeness for the use and development of products?


Nr Cate-gory



2.3 Guidelines

For the ‘guidelines’, the existence of implementation guidelines or reference implementations is addressed.

A.12 Does the formal specification provide available implementation guidelines and documentation for the implementation of products?

A.13 Does the formal specification provide a reference (or open source) implementation?

2.4 Stability

For ‘stability’, the level of change to the formal specification and the stability of underlying technologies is addressed.

A.14 Does the formal specification address backward compatibility with previous versions?

A.15 Have the underlying technologies for implementing the formal specification been proven, stable and clearly defined?

3 Openness

A formal specification should be sufficiently open and available to be relevant for adoption by public administrations. This category addresses the openness of the standardisation organisation and decision-making process, and the openness of the documentation and accessibility of the formal specification.

3.1 Organisation

For the ‘openness’ of the organisation, the level of openness for participating in the standardisation organisation is addressed.

A.16 Is information on the terms and policies for the establishment and operation of the standardisation organisation publicly available?

A.17 Is participation in the creation process of the formal specification open to all relevant stakeholders (e.g. organisations, companies or individuals)?

3.2 Process

For the ‘process’, the level of openness regarding the development and decision-making process for the formal specification is addressed.

A.18 Is information on the standardisation process publicly available?

A.19 Information on the decision making process for approving formal specificationsTOAs is publicly available?


Nr Cate-gory



A.20 Are the formal specificationsTOAs approved in a decision making process which aims at reaching consensus?

A.21 Are the formal specificationsTOAs reviewed using a formal review process with all relevant external stakeholders (e.g. public consultation)?

A.22 All relevant stakeholders can formally appeal or raise objections to the development and approval of formal specifications?TOAs?

3.3 Documentation

For the openness of the ‘documentation’, the accessibility and availability of the documentation of the formal specification is addressed.

A.23 Relevant documentation of the development and approval process of formal specificationsTOAs is publicly available (e.g. preliminary results, committee meeting notes)?

A.24 Is the documentation of the formal specification publicly available for implementation and use on reasonable terms?

4 Intellectual property rights

A formal specification should be licensed on (F)RAND terms or even on a royalty-free basis in a way that allows implementation in different products. This category addresses the availability of the documentation on the IPR and the licences for the implementation of the formal specification.

4.1 IPR Documentation

For the ‘documentation of the intellectual property rights’, the availability of the information concerning the ownership rights of the formal specification is addressed.

A.25 Is the documentation of the IPR for formal specificationsTOAs publicly available?

4.2 Licences

For the ‘licences’ within the intellectual property rights, a (fair) reasonable and non-discriminatory ((F)RAND) or even royalty-free basis is addressed for the use and implementation of the formal specification.

A.26 Is the formal specification licensed on a (F)RAND basis?

A.27 Is the formal specification licensed on a royalty-free basis?


Nr Cate-gory



5 Market support

A formal specification should have sufficient market acceptance and support in order to be adopted by public administrations. This category addresses the proven and operational implementations of the formal specification, the market share and demand for the products, and the support from users and communities.

5.1 Implementations

For the ‘implementations’, the existence of proven and best practice implementations for the formal specification is addressed, in different domains and by different vendors.

A.28 Has the formal specification been used for different implementations by different vendors/suppliers?

A.29 Has the formal specification been used in different industries, business sectors or functions?

5.2 Market demand

For ‘market demand’, the penetration and acceptance of products implementing the formal specification in the market is addressed.

A.30 Do the products that implement the formal specification have a significant market share of adoption?

5.3 Users For the ‘users’, the diversity of the end-users of the products implementing the formal specification is addressed.

A.31 Do the products that implement the formal specification target a broad spectrum of end-uses?

5.4 Interest groups

For the ‘interest groups’, the degree of support from different interest groups is addressed.

A.32 Has the formal specification a strong support from different interest groups?

6 Potential

A formal specification should have sufficient and positive future consequences, evolution and impact for being adopted by public administrations. This category addresses the consequences and impact of using or adopting the formal specification, the advantages and risks, the maintenance and possible future developments.

6.1 Impact

For the ‘impact’, the minimisation of the consequences of using and adopting the formal specification is addressed. The consequences can be evaluated and described in terms of different aspects.

A.33 Is there evidence that the adoption of the formal specification positively impacts organisational processes?

A.34 Is there evidence that the adoption of the formal specification positively impacts the migration of current systems?


Nr Cate-gory



A.35 Is there evidence that the adoption of the formal specification positively impacts the environment?

A.36 Is there evidence that the adoption of the formal specification positively impacts the financial costs?

A.37 Is there evidence that the adoption of the formal specification positively impacts the security?

A.38 Is there evidence that the adoption of the formal specification positively impacts the privacy?

A.39 Is there evidence that the adoption of the formal specification positively impacts the administrative burden?

A.40 Is there evidence that the adoption of the formal specification positively impacts the disability support?

6.2 Risks For the ‘risks’, the level of uncertainty is addressed for using and adopting the formal specification

A.41 Are the risks related to the adoption of the formal specification acceptable?

6.3 Maintenance and future developments

For the ‘maintenance’ and future developments, the support and the planned or existing actions to maintain, improve and develop the formal specification in the long term are addressed.

A.42 Does the formal specification have a defined maintenance organisation?

A.43 Does the maintenance organisation for the formal specification have sufficient finances and resources for the long term?

A.44 Does the formal specification have a defined maintenance and support process?

A.45 Does the formal specification have a defined policy for version management?


[6] The Recommendation Step: Criteria and Classification

In the recommendation step, recommendation criteria are applied, concluding with a classification of the object of Table 17. Appendix: the CAMMS assessment criteria


9 Appendix 2 - to be usedremoved when Deliverable is finalised

This appendix includes the following alternative options of earlier sections to be removed from the final version.

The proposal step - a version with different sets of proposal information/criteria.

9.1 The proposal step with different sets of information/criteria

As input to the proposal step, a target of assessment is provided by other Work Packages.WP6

Architectural Board. The target of assessment is provided using the proposal information/criteria. According to the Technical Annex (p 60), the documentation of formats given in the Guidelines helps

WP6 to provide building blocks for assessment.

Following CAMSS (Section 3.4), the main categories of criteria on the recommendation step are the

same as used for grouping the assessment criteria: applicability, maturity, openness, intellectual property rights, market support, and potential. For these main categories, a score is generated based

on the number of ‘YES’ answers that indicates the level of meeting the criteria for the formal specification. The specified knock-out criteria are checked if they were all met and an answer is

provided for the main categories. BasedTerminological remark. Different sources refer differently to the proposal items: D6.1 - "information and criteria"; ADMS - "property" and "relationship"; CAMMS

- "criteria". We will refer to the proposal items as "information/criteria".

The proposal information/criteria may depend on the discussion specific situation and type of the

asset. The following sets of proposal information/criteria are available.

Proposal information/criteria based on WP6 deliverable "D6.1 Executable ICT Baseline

Architecture" Proposal information/criteria based on CAMMS

Proposal information/criteria based on ADMS Proposal information/criteria based on ADMS.SW

The proposal must identify which set of information/criteria is used.


9.1.1 Proposal information/criteria based on WP6 deliverable D6.1

The proposal information/criteria may be based on WP6 deliverable "D6.1 Executable ICT Baseline Architecture" (section 2.5, "Building Block Evaluation Model"). This must be identified in the assessment panel, an evaluation and possible comments are provided for the proposal.

Following D6.1, a Building Block will Described using the following information/criteria:

Name Relationship to ICT Strategies in EC/MS (in cooperation with WP3) Requirements (in cooperation with WP5) Type Generic/Specific Architecture Framework / Component Specifications Standards Open Source Relationship and Couplings Related Artifacts and Solution Building Blocks Ownership Life Cycle Management In use? Technical Maturity Business Maturity (WP5) Market Maturity (WP3) Evaluation

For description of individual items from the above list we refer to the "D6.1 Executable ICT Baseline

Architecture" (section 2.5, "Building Block Evaluation Model").

9.1.2 Proposal information/criteria based on CAMMS

The proposal information/criteria may be based on CAMSS. This must be identified in the proposal.

Following CAMSS (Section 3.1) and the ADMS specification v 0.8, the main categories. A possible scale for the evaluation of the of criteria for the documentation of format during the proposal step

comprise asset description and relationship (status) categories could includeand criteria according to the following range: Very low, Low, Moderate, High, Very high.table.

The criteria are presented in the following table.

Nr. Category Automated score

Knock-out criteria met?

Evaluation Comments


R.1 Applicability

R.2 Maturity

R.3 Openness

R.4 Intellectual property rights

R.5 Market support

R.6 Potential

Proposed Classification:

The classification of the formal specification is based on CAMSS (p 36), Technical Annex (pp 33-35), PEPPOL, and other relevant sources. It uses the assessment and recommendation criteria and

comprises the following classes.

Discarded: the formal specification is not found

Nr. Category Description Nr. Criteria Description1 Asset

descriptionProviding the general information for the proposed TOA

P.1 Name Name of the asset

P.2 Date ofcreation


P.3 Date of last modification

Date of latest update of asset

P.4 Description Descriptive text for the asset

P.5 ID URI for the asset

P.6 Keyword Word or phrase to describe the asset

P.7 Alternative name

Alternative name for the asset. Note: this information may be used to provide additional access points, e.g. allowing indexing of any acronyms, nicknames, shorthand notations or other identifying information under which a user might expect to find the asset

P.8 Version Version number or other designation of the asset


Nr. Category Description Nr. Criteria Description2 Relationship Providing the

information on the status of the specific TOA being proposed

P.9 Asset type Classification of an asset according to a controlled vocabulary, e.g. code list, metadata schema

P.10 Current version Current or latest version of the asset

P.11 Documentation Document that further describes an asset or give guidelines for its use

P.12 Domain Government sector that an asset or repository applies to, e.g. “law” or“environment” according to a controlled vocabulary

P.13 Interoperability level

Level according to the European Interoperability Framework (EIF 2.0) for which an asset is relevant

P.14 Language Language of an asset if its contains textual information, e.g. the language of the terms in a controlledvocabulary or the language that a specification is written in

P.15 Previous version


P.16 Publisher Organisation responsible for a repository, asset or release

P.17 Release Implementation of the asset in a particular format

P.18 Spatialcoverage


P.19 Subject Theme or subject of an asset, e.g. “elections” or “immigration” according to a general or domain specific controlled vocabulary

P.20 Status Indication of the maturity of an asset or release

Table 18. Appendix. Proposal information/criteria based on CAMMS

Further relevant and should not be used in the public administration

Observed: the formal specification is found relevant but should not be used in the current state or version

Accepted: the formal specification is accepted and could be used in the public administrations (and for the specific business needs)


Recommended: the formal specification is accepted and is preferred for the use in public administrations (compared to alternative formal specifications)

Mandatory: the formal specification is accepted and public administrations are obliged to use the formal specification when applicable (or provide an explanation)

The compliance levels of criteria for LSPs, necessary for the building block to be assessed as satisfying Accepted or higher levels, may be defined.

For the last three classes, maturity levels (Technical Annex pp 33-35 and PEPPOL) may be assessed in more detail if needed.


[7] Organizational aspects of assessment

According to the Technical Annex (p 60), T3.1 will develop a questionnaire form for the assessment of building blocks and discusses how to deal with current LSPs that do not comply to the criteria

upfront.

[7.1] A questionnaire form for the assessment of European Building Blocks

The questionnaire form follows closely the assessment framework, procedure, and criteria.

[7.2] Dealing with current LSPs that do not comply to the criteria upfront

The solution for dealing with current LSPs that do not comply with the criteria upfront is based on a case-by-case analysis of the sources for non-compliance and application of appropriate solutions.

Possible sources and solutions are presented in the following table. In the course of assessment, additional sources for non-compliance and possible solutions will be considered as needed.

Source for non-compliance Possible solutionInsufficient presentation of the building block for assessment (for example, inadequate

documentation of format)

Request for improvement of the presentation

A simple problem with reference to an otherwise acceptable building block

Request to correct the problem in the subsequent version of the building block

A complex problem with reference to an

otherwise acceptable building block

Request to provide a plan to correct the problem

in the subsequent versions of the building block

The project is classified as discarded or observed, with parallel building blocks providing adequate

solutions in assessment

Selection of a parallel building block for assessment

The project is classified as discarded or observed, with no parallel building blocks providing

adequate solutions in assessment

Providing a request to a relevant EU initiative for initiation of a new building block


[8] Research update and input for the analysis (Task T3.1.2)

Before assessing the building blocks, results of already existing assessments will be identified and re-used. It is expected that the majority of the input for this subtask will come from WP5 through the

work on D5.1 requirements framework, as well as the WP6 activities that are described in the e-SENS executable baseline architecture (D6.1). Other sources are the library

http://joinup.ec.europa.eu/ as well as former LSP material. The assessment results from standardisation bodies or other relevant bodies can be re-used and adapted for the assessment of a

formal specification of the building blocks.

The list of materials used for assessment is provided in the references section of this from ADMS and

other sources can be included under this main classification.

9.1.3 Proposal information/criteria based on ADMS

The proposal information/criteria may be based on the ADMS concept "Semantic Asset" ( section

5.5.1 of the document.


[9] Appendix

[9.1] Abbreviations and glossary

ADMS - Asset Description Metadata Schema

ADMS.SW - Asset Description Metadata Schema for Software

BOMOS - Management and Development Model for Open Standards

(F)RAND - (fair,) reasonable, and non-discriminatory terms

Guidelines - Guidelines to the assessment of the sustainability and maturity of building blocks (the

current document)

Technical Annex - e-SENS Technical Annex v3.0

[9.2] References

[4.] Regulations

[4.1.] REGULATION (EU) No 1025/2012 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 25 October 2012 on European standardisation. OJ L 316, 14.11.2012, pp 12-33,

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:316:0012:0033:EN:PDF[5.] European wide scale frameworks, projects, and initiatives

[5.1.] European Interoperability Framework (EIF) for European public services, http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf

[5.2.] European Interoperability Architecture, http://ec.europa.eu/isa/documents/isa_2.1_eia-finalreport-commonvisionforaneia.pdf

[5.3.] EFIR, http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-4action_en.htm[5.4.] Joinup, https://joinup.ec.europa.eu/catalogue/all?current_checkbox=1

[5.5.] Data Catalog Vocabulary (DCAT), http://www.w3.org/2011/gld/wiki/Data_Catalog_Vocabulary

[5.6.] BOMOS (Management and Development Model for Open Standards), [5.7.] BOMOS2i,


8.[6.] "ADMS




http://www.w3.org/2011/gld/wiki/Data_Catalog_Vocabulary

https://joinup.ec.europa.eu/catalogue/all?current_checkbox=1

http://ec.europa.eu/isa/actions/04-accompanying-measures/4-2-4action_en.htm



http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf


[6.1.] ADMS v 1.0. https://joinup.ec.europa.eu/asset/adms/release/100[6.2.] ADMS. Asset Description Metadata Schema. Specification Version 1.00. Release date

18/04/2012[6.3.] . Asset Description Metadata Schema for Software,

https://joinup.ec.europa.eu/asset/adms_foss/asset_release/admssw-100[7.] Common Assessment Method For Standards And Formal Specifications (CAMSS). Final Draft

Revision of CAMSS. Version 1.0, March 2012. See also the following sources.[7.1.] CAMSS Project – Workshop 2. Agreement on Revision of CAMSS. Wednesday, March 7 th

2012 (fail Presentation of EC on CAMSS.ppt) [7.2.] https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/

[8.] e-SENS project materials[8.1.] e-SENS Technical Annex v 3.0

[9.] European specific area projects, building blocks, or assessments[9.1.] PEPPOL EIA Repository, http://www.peppol.eu/peppol_components/peppol-eia/eia

[9.2.] The feasibility and scenarios for the long-term sustainability of the Large Scale Pilots, including 'ex-ante' evaluation. Task 1 Report. 11-03-2013 SMART 2012/0059, Deloitte.

. Specification Version 1.00. Release date 18/04/2012"). This must be identified in the proposal.

For description of individual items from the above list we refer to the "D6.1 Executable ICT Baseline Architecture" (section 2.5, "Building Block Evaluation Model").

9.1.4 Proposal information/criteria based on ADMS.SW

In case of a software component, the proposal information/criteria may be based on the ADMS.SW

main concepts: software project, software release, software package, software repository (section 4.2 of the document "Asset Description Metadata Schema for Software 1.00. ADMS.SW 1.00.

Release date 20/05/2012"). This must be identified in the proposal.


http://www.peppol.eu/peppol_components/peppol-eia/eia

https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/

https://joinup.ec.europa.eu/asset/adms_foss/asset_release/admssw-100

https://joinup.ec.europa.eu/asset/adms/release/100

Date post:	05-Mar-2018
Category:	Documents
Upload:	truonghanh
View:	214 times
Download:	1 times