Post on 19-Jul-2018
transcript
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria
PACT - Public perception of security and privacy: Assessing knowledge, Collecting evidence, Translating research into action Project co-funded by the European Commission within the 7th Framework Programme – SECURITY theme
2nd Reporting period WP6 Decision Support System (DSS) Responsible Partner: NCSRD Contributing partners: ATOS, CSSC, CIES, KEMEA, HUJI, RAND, PRIO, UOW Due date of the deliverable: February 28th 2014 Actual submission date: October 16th 2014
Dissemination level: PU
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
2
Document Management
D6.1 “DSS Architecture, Technical Requirements and Definition of Validation Criteria” Task: 6.1 Leader: NCSRD – Other contributors: ATOS, CSSC, CIES, KEMEA, HUJI, RAND, PRIO, UoW Authors: NCSRD: Dimitris M. Kyriazanos, Olga E. Segou, Anastassios Bravakis, Constantinos Rizogiannis, Stelios C. A. Thomopoulos. ATOS: Jaime Martin Perez, Kaan Dulgar, Adem Yasar Mulayim. CIES: Daniel Deering.
PROJECT FULL TITLE
Public Perception of Security and Privacy: Assessing Knowledge, Collecting Evidence, Translating Research into Action
PROJECT ACRONYM PACT
Collaborative Project funded under Theme SEC-2011.6.5-2 “The relationship between human privacy and security”
GRANT AGREEMENT 285635
STARTING DATE 01/02/2012
DURATION 36 months
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
3
Executive summary
The current report is the output of PACT task T6.1 “DSS Architecture Technical Specifications and System
Validation definition” and the first deliverable of WP6 “Decision Support System (DSS)”. Following
assessment of theoretical knowledge and collection of empirical evidence, PACT project proceeds with
analysing the main factors that affect public assessment of the security and privacy implications of given
security technology. By integrating the results of this analysis into a software tool, WP6 is responsible of
developing a prototype DSS, which may help end users to evaluate pros and cons of specific security
investments also on the basis of the societal perception of privacy and liberty. T6.1 begins by collecting user
requirements, defining the DSS architecture, functional and technical requirements and corresponding
validation methodology and criteria.
The PACT DSS is a tool for assessing the impact of future security technology investments and deployments
in terms of privacy, ethics, social acceptance and public perception. It aims at assisting the decision process
of experts through efficient visualisation and user-DSS interaction powered by the software
implementation of PACT’s theoretical and empirical findings. The PACT DSS is a core tangible output of
PACT research activities and it implements as a software tool the Privacy Reference Framework for Security
Technologies (PRFST) toolbox described in D5.2 “PACT Privacy Reference Framework for Security
Technology”. The PACT DSS is context-specific and it is based on the three contexts specified in the PACT
survey, namely the Travel, Healthcare and Internet. However, the PACT DSS is modular and extensible,
having the ability to integrate future empirical studies in other contexts, or enrich the existing contexts
without overhead or need for extended offline time for the PACT DSS system. This is achieved through
optimised database design, an extensible data model and a modular architecture, which allow quick
inclusion of additional datasets and knowledge input.
Task 6.1 was responsible for kick starting WP6 activities during the joint PRFST-DSS workshop held in
Madrid in October 2013. A key event in the WP6 timeline, the workshop confirmed the alignment of
research activities between WP5 and WP6 and established the smooth inclusion of PACT’s PRFST into the
DSS design. Moreover, during the workshop end user perspective was integrated into the DSS design
through round table discussions, dedicated presentations and questionnaires. Following the workshop,
user requirements were extracted and documented in this report, based on feedback and in collaboration
with all PACT experts groups (Stakeholders Advisory Group - SAG, Privacy Advocate Panel – PAP, and High
Level Policy Panel - HLPP) and PACT End User Partners. In parallel, T6.1 also reviewed and assessed relevant
work in the field, establishing connections and organising online meetings with relevant EU research
projects such as PRISMS1, PARIS2 and VALUESEC3.
Building on the results of these activities, T6.1 defined the preliminary PACT DSS architecture based on user
needs, state of the art techniques and best practises. The PACT DSS architecture follows a three-tier
approach and is broken down into the following components: (i) User Layer, Human Machine Interface
(HMI), Dialogue Management & Visualisation Module, (ii) Decision Trees & Engine Module and (iii)
Knowledge Management Module & Database Server. Based on the user requirements and the design
considerations expressed through the DSS architecture, T6.1 proceeded with defining the functional and
1 PRISMS website: http://prismsproject.eu/ 2 PARIS website: http://www.paris-project.org/ 3 VALUESEC website: http://www.valuesec.eu/
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
4
technical requirements of the PACT DSS. The requirements were documented in a template and format
convenient for use and reference by WP6 activities that follow, building the consolidated Requirements
Traceability Matrix which will feed the forthcoming software engineering and prototype development
activities.
T6.1 concludes its design activities by defining the validation criteria for the target PACT DSS. They include
user requirement based validation and acceptance as well as performance and technical success measures
for the system and individual components. Once completed and delivered, the successful operation of the
PACT DSS software tool will be validated against the criteria defined in this document, including user
perspective criteria, success measures for each component, functionality and performance validation
criteria.
Finally, the legal, technical and ethical scope of the PRFST and the DSS have been considered by consortium
partners and PACT’s external actors for a prolonged period of time. It is important again to reiterate that
the PRFST tool and the DSS – just as with any decision support tool – are designed to support and not
displace legitimate, democratic decision-making and validation processes. Decisions are to be taken by the
users themselves, with the system supporting the process through multiple modalities and presenting
quantitative and qualitative information regarding the impact of each decision path.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
5
Table of contents Executive summary ------------------------------------------------------------------------------------------------------3
Table of contents ----------------------------------------------------------------------------------------------------------5
Table of figures ------------------------------------------------------------------------------------------------------------6
List of acronyms used in the document -------------------------------------------------------------------------------7
1. Introduction ---------------------------------------------------------------------------------------------------------8
1.1 Scope ----------------------------------------------------------------------------------------------------------------8
1.2 Deliverable Structure ---------------------------------------------------------------------------------------------9
1.3 Methodology -------------------------------------------------------------------------------------------------------9
2. Decision Support Systems: State of the Art and relevant work carried on in EU projects ----------- 12
2.1 DESSI ---------------------------------------------------------------------------------------------------------- 12
2.2 PARIS ---------------------------------------------------------------------------------------------------------- 13
2.3 PRISMS -------------------------------------------------------------------------------------------------------- 15
2.4 SURPRISE ----------------------------------------------------------------------------------------------------- 16
2.5 VALUESEC ---------------------------------------------------------------------------------------------------- 17
2.6 Summary ----------------------------------------------------------------------------------------------------- 20
3 PACT DSS Stakeholders Report and User Requirements -------------------------------------------------- 21
3.1 PRFST-DSS Workshop report --------------------------------------------------------------------------------- 21
3.2 DSS User Requirements ---------------------------------------------------------------------------------------- 25
4 DSS Architecture, Functional and Technical Requirements ----------------------------------------------- 29
4.1 DSS Architecture ------------------------------------------------------------------------------------------------ 29
4.2 Functional and Technical Requirements -------------------------------------------------------------------- 36
5 Requirements Traceability Matrix ----------------------------------------------------------------------------- 42
6 DSS Validation Criteria ------------------------------------------------------------------------------------------- 48
6.1 User Perspective Criteria -------------------------------------------------------------------------------------- 48
6.2 Component Specific Validation Criteria and Success Measures ---------------------------------------- 48
6.3 Functionality Validation Criteria ----------------------------------------------------------------------------- 50
6.4 Performance Validation Criteria ------------------------------------------------------------------------------ 50
7 Conclusions -------------------------------------------------------------------------------------------------------- 51
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
6
Table of figures Figure 1 A screenshot of the DESSI web-based application. ..................................................................................... 14
Figure 2 Structure of PRISMS DSS. ....................................................................................................................................... 16
Figure 3 Building blocks of the PRISMS DSS .................................................................................................................... 16
Figure 4 ValueSec Tool screenshot ...................................................................................................................................... 18
Figure 5 ValueSec DSS technical architecture. ................................................................................................................ 19
Figure 6: PACT DSS Context .................................................................................................................................................... 22
Figure 7 Mean and Median User Scores for each HMI design criterion .................................................................... 25
Figure 8 PACT Preliminary DSS Architecture .................................................................................................................. 30
Figure 9 Sample PACT bar chart ........................................................................................................................................... 32
Figure 10 Sample PACT spider diagram ............................................................................................................................ 33
Figure 11 Privacy Threat Index, Colour Scheme representation ............................................................................ 33
Figure 12: Sample decision tree ............................................................................................................................................ 34
Figure 13 Survey Data Table Sample for Travel context ............................................................................................ 35
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
7
List of acronyms used in the document
DSS: Decision Support System
E-R: Entity-Relationship
HLPP: High Level Policy Panel (external experts’ advisory group of PACT)
HMI: Human Machine Interface
PRFST: Privacy Reference Framework for Security Technologies (described in D5.1, D5.2)
PTI: Privacy Threat Index (described in D5.1, D5.2)
UI: User Interface
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
8
1. Introduction
The current document reports the activities of Task T6.1 “DSS Architecture Technical Specifications and
System Validation definition” and it is the first deliverable of WP6 “Decision Support System (DSS)”. The
report feeds into T6.2 “DSS Knowledge Base, Decision Context and Engine Functional Design” which
initiates software engineering activities and provides the DSS tool specifications to T6.3 for DSS prototype
development and implementation. T6.4 will finally perform the “Validation and End User Evaluation”,
basing the validation on defined criteria and methodology delivered by this document. therefore, this
deliverable D6.1 is the foundation stone of the overall WP6 work plan. For this reason T6.1 followed a
clearly specified step-by-step methodology (as described in brief in section 1.2 below) in order to achieve
the necessary milestones and the needed outputs, while engaging the PACT experts community and
collecting valuable end user perspective and experience4.
1.1 Scope
The main scope of this deliverable is to:
1. Define the user requirements by engaging and collecting feedback from PACT End User partners
and all PACT external experts groups.
2. Define the preliminary DSS architecture and subsequently define the technical and functional
requirements providing the functionality and technical aspects necessary to meet the user
requirements and DSS architecture design needs, taking also into account the state of the art and
relevant research work.
3. Define the measurable criteria which will validate the successful operation of the PACT DSS.
4. Report and document all relevant definitions in a template and format convenient for use and
reference by WP6 activities that follow (i.e. software engineering and prototype implementation).
It is important to remind readers that this deliverable intends to define the technical architecture,
requirements and validation criteria of the DSS model. As the scope of this deliverable sits within a
uniquely technical context, thematic considerations around fundamental rights, ethics, and data
protection are to be considered in the deliverables to follow; primarily in D6.2 where the functional
specifications will be set, in D6.3 embedded in the DSS prototype and in D6.4 which includes the
evaluation and user acceptance report.
Relevant previous PACT deliverables and results referenced throughout this document include:
D1.2 “Report on the review of DSS for decision making on security issues” referred to as PACT
D1.25
D1.3 “Report on security technology taxonomy and mapping” referred to as PACT D1.36
D1.6 “Use Case definition report” referred to as PACT D1.67
4 Key events: Joint PRFST-DSS workshop, 17-18 October 2013, Madrid, Spain and 2nd HLPP meeting, 18 November 2013, Brussels, Belgium. 5 Confidential, executive lay report to be made available soon on PACT website: http://www.projectpact.eu/deliverables/wp1-root-branch-review. 6 http://www.projectpact.eu/deliverables/wp1-root-branch-review/d1.3 7 http://www.projectpact.eu/deliverables/wp1-root-branch-review/d1.6
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
9
D4.2 “Survey Report” referred to as PACT D4.28
D5.1 “Report on the Definition and Design of the PRFST” referred as PACT D5.19
D5.2 “PACT PRFST” referred to as PACT D5.210
D7.5 “Updated plan for user and dissemination of foreground” referred to as PACT D7.511
1.2 Deliverable Structure
This document is organized in seven chapters.
Chapter 1 - Introduction: the current chapter presents the scope and objectives of the deliverable, briefly
explaining the methodology adopted in Task 6.1.
Chapter 2 – DSS State of the Art and connection with relevant work includes a review and an assessment
of the relevant work, collecting input from other relevant EU research projects PRISMS, PARIS, SURPRISE,
VALUSEC and DESSI.
Chapter 3 – PACT DSS Stakeholders report and User Requirements presents the collected and extracted
user requirements based on the results of the PRFST-DSS joint workshop as well as on the feedback
gathered from all PACT stakeholder groups.
Chapter 4 – DSS Architecture, Functional and Technical Requirements includes the description of the
preliminary DSS architecture along with the relevant functional and technical requirements.
Chapter 5 – Requirements Traceability Matrix: including for reference and convenience of future WP6
activities the consolidated, numbered table of Requirements reported in this document.
Chapter 6 – DSS Validation Criteria introduces the validation criteria against which the successful operation
of the PACT DSS software tool will be evaluated and validated.
Chapter 7 - Conclusions summarises the work reported in this document and highlights the more salient
PACT DSS features.
1.3 Methodology Task T6.1 methodology starts with the extraction of PACT DSS user requirements, which is followed by the
specification of the functional and technical requirements with respect to the PACT DSS Architecture as this
was defined and broken down into the following components: User Layer, HMI, Dialogue Management &
Visualisation Module, Decision Trees & Engine Module and Knowledge Management Module & Database
Server. Requirements are documented based on the template provided for each type of requirements and
consolidated in a Requirements Traceability Matrix, for reference and convenience of future activities and
deliverables of WP6. By analysing user requirements and technical aspects of the system design,
measurable success criteria are extracted, forming the validation approach for PACT DSS. These include the
8 http://www.projectpact.eu/deliverables/wp4-data-analysis/d4.2 9 http://www.projectpact.eu/deliverables/wp5-new-conceptualization-and-framework/d5.1 10 http://www.projectpact.eu/deliverables/wp5-new-conceptualization-and-framework/d5.2 11 To be published on PACT Website on: http://www.projectpact.eu/deliverables/wp7-dissemination-and-stakeholder-involvement
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
10
user perspective criteria along with the success measures for each DSS component, functionality and
performance validation. We present here the template used for each type of requirement.
User Requirements
User Requirements are extracted based on feedback from all PACT experts groups: the Stakeholder
Advisory Group (SAG), the Privacy Advocates Panel (PAP) and the High Level Policy Panel (HLPP). The
feedback was collected through questionnaires, round-table discussions and during dedicated presentation
& workshop sessions at the joint PRFST-DSS workshop (Madrid on 17th and 18th, October 2013, in the
premises of Atos Spain) as well as during the 2nd HLPP meeting in Brussels (18th November 2013).
In this context, user requirements extracted follow the specific table-row format, including the following
fields:
ID (ID): A unique ID for the specific user requirement, to be used for reference by following WP6 deliverables. The ID will follow the format common in all PACT WP deliverables: <document code>.<Type U/B/F/T (User/Business/Functional/Technical)>-<increasing number>, e.g. for this document D61.U-15.
Brief Summary: a summary, title-like of the requirement, so as to easily identify what this requirement is about.
Requirement: a full description of the requirement together with explanatory text and rationale, if necessary.
Source: the Experts Group responsible for contributing this requirement, i.e. Stakeholder Advisory Group, the Privacy Advocates Panel and the High Level Policy Panel.
Functional / Non-Functional: Whether the requirement is functional or non -functional.
Domain: General / Security/ Privacy / Ethics / etc…
Template:
ID Brief Summary
Requirement Source Functional / Non-Functional
Domain
D61.U-1
Functional and technical requirements
Functional and technical requirements are extracted taking into account a) the end user's perspective and
needs as encapsulated inside the user requirements, and b) the state of the art, best practices and market
trends. In this context, requirements extracted follow the specific table-row format, including the following
fields:
ID: A unique ID for the specific requirement, to be used for reference by following deliverables. The ID will follow the format common in all PACT WP deliverables: <document code>.<Type U/B/F/T (User/Business/Functional/Technical)>-<increasing number>, e.g. for this document D61.F-15.
Brief Summary: a summary, title-like of the requirement, so as to easily identify what this requirement is about.
Requirement: a full description of the requirement together with explanatory text and rationale, if necessary.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
11
Requirement Source Ref: A reference to the user requirement ID or market analysis requirement ID that leads to this requirement –if applicable. The identification code corresponding to identifies used in the respective deliverables, e.g. D61.B-12 for a market analysis requirement and D61.U-9 for a user requirement.
Mandatory/Optional: The corresponding box is checked if the requirement is mandatory or optional in the actual PACT solution being implemented during the project. Mandatory requirements are essential for the PACT implementation while optional are usually reserved for future extensions and design considerations.
Finally, after all requirements have been extracted, they are also systematically grouped and inserted inside
the Requirements Traceability Matrix.
Template:
ID Brief Summary Requirement
Require
ment
Source
Ref.
Man
dator
y
Optio
nal
D61.F-1 X
D61.F-2 X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
12
2. Decision Support Systems: State of the Art and relevant work carried on in EU
projects
In the context of PACT, the main aim of the Decision Support System (DSS) is to function as a tool
contributing to the assessment of public perceptions of security and privacy by end users and policy makers
when they deal with specific security investment issues. This chapter provides a state of the art of Decision
Support Systems as currently developed in relevant European projects. Each examined project is briefly
analysed to understand its goal and approach, the main design aspects and the possible contribution(s) or
input(s) for the design of the PACT DSS system itself. This brief comparative review is based on publicly
available information (websites, deliverables, etc.). The following projects are examined while collaboration
was established in case of ongoing projects:
DESSI: Decision Support on Security Investment)
PARIS: PrivAcy pReserving Infrastructure for Surveillance)
PRISMS: PRIvacy and Security MirrorS: Towards a European Framework for Integrated Decision
Making)
SurPRISE: Surveillance, Privacy and Security: A large scale participatory assessment of criteria and
factors determining acceptability and acceptance of security technologies in Europe)
VALUESEC: Mastering the Value Function of Security Measures)
2.1 DESSI12 DESSI project aims at developing a DSS system which will contribute to a transparent and participatory
decision making to specific security investment issues. The main goal is to overcome the exclusively
technology-driven solutions and to offer a versatile approach which takes into account the political, ethical
and social dimensions of society while at the same time offers alternative ways of addressing a specific
threat.
The DSS system that has been developed is a web-based application which guides the user in the analysis of
the security related issue through predefined questions and organisational steps. Specifically, the following
three phases are included (Fig. 1): i) Security problem ii) Security investment and alternatives, and iii)
Multi-criteria assessment workshop. Finally, a report that collects all phases of DESSI DSS system can be
produced.
Input for PACT
A relevant input for PACT DSS could be the architecture of the developed DESSI DSS, the technologies
adopted for the web application development and the knowledge database definition, as well as the
reasoning process of the decision-making system.
12 http://securitydecisions.org/, https://dessitool.org/en/
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
13
2.2 PARIS13 In the surveillance infrastructure design area PARIS project aims at the development of a methodological
approach which by design enforces the right of citizens for privacy, justice and freedom. At the same time,
the ethical and social aspects of such rights are taken into account, since the perception of such rights
varies between countries. Special consideration is also given to the adjustability of the proposed
methodology to any future changes in the legal framework.
The methodological approach will be based on two pillars; a theoretical framework which balances
surveillance with privacy, and a design process where privacy and accountability are considered by design.
13 http://www.paris-project.org/
Figure 1 A screenshot of the DESSI web-based application.
Specialized conceptual frameworks, called SALT frameworks, will also be defined and a framework
management tool will be developed to help surveillance system designers. The tool allows the creation,
digital reference and representation, and updating of a SALT framework. It can also allow for reasoning by
integrating information (such as rules, constraints), and guidelines on the relation between surveillance,
privacy, and accountability capability.
Input for PACT
An insight into the analysis, design and development phase of SALT framework management tool could be
of special interest for the PACT DSS design. Especially, the technologies used, the architecture, the updating
and representation of the theoretical framework and the integration of information and guidelines could
offer useful knowledge for the implementation of PACT DSS system.
2.3 PRISMS14 A close collaboration was established with PRISMS given the parallel activities, similar nature and objectives
with PACT. This was achieved through teleconferences, exchange of information and views on important
design aspects concerning the DSS and overall methodology.
The objectives of PRISMS project are to analyse the trade-off model relation between privacy and security,
investigate how surveillance technologies usage may infringe citizens’ privacy and fundamental rights, and
determine the factors that affect the public assessment of the security and privacy implications of a specific
security technology.
The outcomes of the aforementioned analysis will be used for the development of a DSS system which will
provide all stakeholders with an insight into the advantages, disadvantages, constraints, and conflicts of a
specific security investment as opposed to other alternatives.
In conceptual level, the DSS system of PRISMS project (Fig. 2) will take input from a knowledge database
which can comprise all the evidence and heuristics as well as input from a model-based management
system which contains possible scenarios and the necessary decision rules and hierarchy while the output is
depicted in the user interface. The building blocks of the PRISMS DSS are depicted in Fig. 3 where only the
impact assessment is to be implemented as a software tool.
Input for PACT
A source of useful input for PACT could be any information on the software implementation and the
architecture adopted but even any conceptual framework regarding the other building blocks of the
PRISMS DSS.
14 http://prismsproject.eu/
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
16
Figure 2 Structure of PRISMS DSS.
Figure 3 Building blocks of the PRISMS DSS
2.4 SURPRISE15
Another European project which has similar objectives of PRISMS and PACT is SurPRISE. This project will
deliver a large scale participatory assessment of criteria and determinants of acceptability of security
technologies in Europe.
In that context user requirements (factors influencing acceptance of these security technologies),
functional requirements (technical design, legal options, non-technical alternatives, and models about
15 http://surprise-project.eu/
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
17
privacy and security relationships) and non-functional requirements (empirical findings and testing of
models) will be identified for decision support on security technologies.
Input for PACT
SurPRISE can constitute a source of useful information for PACT in terms of identification and analysis of
the aforementioned requirements, models development, and technologies used as well as validation and
testing on DSS systems.
2.5 VALUESEC16
With ATOS being a common and key partner in both PACT and VALUESEC, relevant knowledge and useful
insight was provided:
“The vast majority of decisions within the security sector are characterized by a complex set of parameters
and uncertainties. They range from quantitative factors like costs, investment budgets and benefits such as
the intended reduction of damages to highly uncertain and qualitative factors such as political objectives or
societal acceptance. Funded by the European Commission’s 7th Framework Program, the ValueSec project
is intended to support and enhance the decision making process with a toolset so that security measures
reflect the interests of all stakeholders.
Throughout the last three years this project aimed to develop the means to make more transparent and
systematic assessments of the wide array of costs incurred by security decisions as well as the potential and
expected benefits linked to such decisions. It will thus facilitate better and more effective decision making.
Decisions must be based on transparent criteria and a rigorous cost-benefit assessment, incorporating
quantitative and qualitative factors such as social, cultural, ethical and economic implications in the overall
analysis.
Thus, the toolset functionalities are grouped by three pillars:
Risk Reduction Assessment, to evaluate how good a security measure is in terms of mitigating a threat and its corresponding impact on the organization assets.
Cost-Benefit Assessment, to calculate quantitative implications of decisions on security measures, focusing on monetary costs and benefits.
Qualitative Criteria Assessment, to integrate different non-tangible decision parameters into an evaluation process, such as societal or ethical factors.
The toolset aims to support rational decision-making by establishing methodologies to increase the
transparency of the decisions-making process and of the driving decision parameters”.17
16 http://www.valuesec.eu/ 17 http://www.valuesec.eu/sites/default/files/2014.02.27._VALUESEC_D7.5_Exploitation_Plan_FINAL.pdf
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
18
Figure 4 ValueSec Tool screenshot
Considering the system architecture (Fig. 5), the toolset comprises a set of centralized components and
tools as well as external components. Centralized components consist of the methodological framework
functions (Qualitative Criteria Assessment, Cost-Benefit Assessment), the knowledge database and other
common functionalities. They are all incorporated in the same web application and hosted in the same
server. The Risk Reduction Assessment functionality is handled by components placed on external servers
communicating with the central server using dedicated API.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
19
Figure 5 ValueSec DSS technical architecture.
Contexts and use cases
The methodology and toolset were assessed in the context of realistic use cases based on explicit
requirements of end users that represent pre-defined decision-making contexts and processes. Its target
group of stakeholders is that of public decision makers and the project aims to understand their needs and
under which framework conditions they make decisions regarding security measures.
The selected decision making contexts for ValueSec are: Public mass events, Public transportation, Airport
security, Communal security planning and Cyber security. These are the use cases where the project was
tested for each of the contexts:
- Use Case 1: Mass Event - Formula One Race Track, Valencia´s Street Circuit: Improved surveillance
and detection systems
- Use Case 2: Public Transportation - Train Infrastructure: Improved Intrusion Detection and Damage
Prevention for Passenger Trains in a Depot
- Use Case 3: Aviation - Airport Security: New generation Liquid Aerosol Gel (LAG) Scanners
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
20
- Use Case 4: Communal Security - Flood Protection: Improvement of infrastructure and water
management
- Use Case 5: Cyber Security - Smart Grid attack: Improving Security of Energy Smart Grids from
Targeted Viruses Attacks
Input for PACT
Some of the questions and approaches of ValueSec, in which PACT partners Atos and PRIO also
participated, are relevant for PACT too i.e. how is the decision making process structured? What are
current parameters that influence the decision-making process and which ones are necessary to make
suitable decisions? What kind of decision support is needed to achieve trade-offs? ValueSec findings are to
be evaluated from the different perspectives of security policy makers and the perspectives of policy,
economics and society.
During the first phase of PACT a collaboration was already established i.e. results from ValueSec on the
analysis of available methodologies and tools for decision analysis and support have been considered for
the review of DSS for decision making on security issues.
In later phases of PACT, especially for the DSS design, aspects addressed by ValueSec can prove to be of
relevance, for instance for the definition of a decision support tool that satisfies stakeholders’ requirements
in a security context and the corresponding technical architecture. The fact that “decisions must be based
on transparent criteria and a rigorous cost-benefit analysis, incorporating quantitative and qualitative
factors such as social, cultural, ethical and economic implications in the overall analysis” matches well the
objective approach of PACT’s PRFST and highlights the heterogeneous mixture of dimensions and factors
both projects consider.
Moreover, information on the definition of decision support tool, the corresponding technical architecture,
the knowledge database schema definition, and the technologies adopted for central server and web
application development can be relevant for PACT DSS system as well.
2.6 Summary
A number of ongoing European projects, related with PACT, have been examined in the context of their
potential use as input for the PACT’s DSS design. Summarizing the findings of the above study, the main
points from each project where special attention should be paid to, are the theoretical framework and the
decision-making and reasoning process, the models development process, the technical system
architecture or the architecture of any relevant component (e.g. SALT framework management tool), the
technologies adopted for server, database and tools development, as well as validation and testing of DSS.
Exploiting the existing repository of knowledge, PACT can accomplish its goals to go beyond a mere risk and
impact assessment analysis but deliver an innovative DSS which is context-aware and adaptive, supports
multi-criteria decision making and collaborative decision, as well as considering crowdsourcing and social
media empirical knowledge collection. However, concerns with regard to crowdsourcing and social media
perceptions need to be taken into account (issues explained in more detail in section 4.1 “The User Layer”).
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
21
3 PACT DSS Stakeholders Report and User Requirements
User requirements for the PACT DSS have been extracted from the feedback collected from PACT End User
partners participating in the task and from all PACT stakeholder groups: the Stakeholder Advisory Group
(SAG)(PAP) and the High Level Policy Panel (HLPP). The key event to achieve this was the joint PRFST-DSS
workshop, which took place in Madrid on 17th and 18th October 2013, in the premises of Atos Spain. The
DSS concept was also presented and discussed during the 2nd HLPP meeting in Brussels (18th November
2013). This chapter offers a review of the PRFST-DSS workshop, and a consolidated table of extracted User
Requirements.
3.1 PRFST-DSS Workshop report
As explained in D5.2, under the prism of the PRFST and DSS objectives, the joint PRFST-DSS Workshop was
considered an excellent opportunity to present the key design elements of these key PACT contributions to
the PACT expert groups (including the Stakeholder Advisory Group, Privacy Advocates Panel and High Level
Policy Panel). The main goal of the PRFST-DSS workshop was to collect and compile valuable expert
feedback and translate it into user requirements for the PACT DSS. Active contributions during the
workshop were provided from all participants. The multidisciplinary expertise represented in the workshop,
by the participation of the different expert groups, was instrumental to help the consortium translate the
theoretical and empirical approaches into a useful tool for the project’s target stakeholders.
Discussion on Context and DSS functionalities
During the workshop, dedicated presentations were given concerning the PACT DSS, its concept of
operations and the consistency/relationship with the work carried on in WP5 (in particular the
development of the PRFST). The workshop was indeed an opportunity to harmonize the efforts within WP5
and WP6, given also the input and feedback from stakeholders at the audience and in accordance to the
PRFST-DSS connection well established and presented in PACT D5.1.
Figure 6 explains the Context framework which feeds PACT DSS engine and functionalities, which was
discussed during the workshop.
The key discussion points during the workshop focused on analysing the security decision making context
and on the PACT DSS functionalities. In detail, the session on analysing context included the following
questions towards the potential DSS users:
Based in your experience, how can intangible elements be valuated? (such as the reputation, the
corporative image, the corporative deontological code, and others)
Please rate the contextual elements for their usefulness and importance during security decision
making and with respect to privacy issues.
Suggest additional contextual elements
Please rate the identified key choice points in terms of how challenging they can be from your
experience.
Suggest challenging key choice points
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
22
Figure 6: PACT DSS Context
With regards to DSS functionalities, participants were asked to respond to the following questions:
Given the presentation, picture your “ideal” PACT DSS. Please write here up to five (5) example
questions that would help your decision making and you would really like to “ask” PACT DSS. Feel
free to write either simple or more complex questions.
What kind of historical data would you find useful in the context of PACT DSS?
What kind of projected figures would you expect from PACT DSS? What kind of figures or charts
would you find useful in terms of estimating public acceptance and trust?
Please describe a typical decision cycle in your organisation for privacy related decision making,
describing in a few words the role of each person in the cycle.
In these sessions, context and scope were important points discussed during the workshop, expressing the
requirement that the DSS should be context-aware and context-driven, adapting to particular scenario, use
case and application. The output for the users was also a key issue, where users would expect a Data
Protection or Privacy Impact Assessment as a provided output to users.
Discussion on Human-Machine Interaction (HMI) design
A session dedicated to the HMI design was also held during the PACT Workshop which included the
collection and analysis of HMI design questionnaires. The main goals of the HMI session were to provide an
understanding of the basic concepts of efficient HMI design. This subsection provides an overview of the
discussion that took place during the HMI session of the PACT Workshop.
The basic concept of Human-Machine18 Interaction was first introduced, as a multidisciplinary field
combining computer science, psychology, behavioural science, design and ergonomics. Efficient HMI design
is necessary in order to provide an easy-to-use user interface, which will minimize the time and effort
needed to complete a task. Interaction Styles 19 are defined as ways of communication between the user
18 Stuart K. Card, Thomas P. Moran, Allen Newell (1983): The Psychology of Human–Computer Interaction. Erlbaum, Hillsdale 1983 ISBN 0-89859-243-7 19 Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., & Carey, T. (1994). Human-computer interaction. Wokingham,UK.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
23
and the computer system. The session briefly discussed the most common interaction styles, enabling
dialogue between the PACT consortium and the end-users, towards an efficient design of the user interface
for the PACT DSS.
The pros and cons of the most common Interaction Styles were discussed, including:
The basic Command Line Interface: it provides a powerful environment but has a steep learning curve.
The Window, Icon, Menu, Pointer (WIMP) style: the most common style of interaction and one of the easiest to use, since most users are already familiar with it.
Form fill-in: Easy to use, but considered to be tedious.
Direct Manipulation: The user interacts with icons that map real-world objects to user actions.
Crossing Interfaces: Gestures are integrated with conventional click-drag-drop functionalities
Zooming: Allows zooming in and out of a focus area.
Focus plus Context: Gives an overview of the whole system and allows user to focus to a specific area of the screen.
Brushing and Linking: Only subsets of information are shown the user, by applying appropriate filters to control visibility.
Various examples of Decision Support Systems were then discussed, focusing on the Interaction Styles that
their respective user interface implements, in order to provide a more clear understanding of the way
interaction styles are mixed in a real world implementation of a system. This “exercise” also served to
illustrate which interaction styles are more common in implementations of Decision Support Systems.
The discussion also gravitated towards the design of efficient and usable User Interfaces, listing the
Nielsen20 usability heuristics, which include:
Visibility of System Status
Match Between System and the Real World
User Control and Freedom
Consistency and Standards
Error Prevention
Recognition Rather Than Recall
Flexibility and Efficiency of Use
Aesthetic and Minimalist Design
Easy Recognition, Diagnosis, and Recovery from Errors
Help and Documentation.
Following the HMI session, a discussion on the heuristic criteria was held and a questionnaire was handed
to the session participants. The questionnaire was compiled by NCSRD and illustrated some examples of
DSS User Interface design and asking for participants’ input. Specifically, it featured:
Examples of DSS UI design
Examples of 2D data visualisations
The Weinschenk/Barker21 design evaluation criteria
The participants were then asked to complete the questionnaire, indicating their personal opinion on the
20 Nielsen, J. and Mack, R.L. (eds) (1994). Usability Inspection Methods, John Wiley & Sons Inc. 21
Weinschenk, Susan, and Dean T. Barker. Designing Effective Speech Interfaces. New York: Wiley, 2000.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
24
presented UIs and ranking the Weinschenk/Barker criteria according to their importance (as viewed by
each participant). When the HMI session concluded, sixteen (16) answered questionnaires were collected.
The results of this activity provided some insight on the user requirements for the DSS user interface, which
will be discussed in a following section.
The collection of the HMI design questionnaires during the PACT Workshop, provided some useful insight
on the user requirements for the efficient and usable UI design of the PACT DSS. According to participants’
input and their comments on the presented DSS UIs the required characteristic and functionalities for the
user interface can be summarized in the following list:
The user interface should not be cluttered.
Data visualizations should not be complex.
The HMI should give a minimalistic overview of the data also allowing the user to expand upon the visualizations when necessary, in order to obtain more information.
The HMI could resemble a Wizard-like process and provide the ability to process the data in an appropriate number of steps.
Condensation of graphics should be avoided.
Abbreviations should be avoided.
Cluttered Columns, lists and menus should be avoided.
Icons need to be meaningful and map to specific contexts.
The participants were also asked to rank the following design criteria22 according to their perceived
importance in UI design, by assigning a score (from 1-10) to each one:
1. User Control: Amount of user control available through the interface design. 2. Human Limitations: Taking into account human limitations, cognitive and sensorial, to avoid
overloading the user. 3. Modal Integrity: use of the most suitable modality for each task: auditory, visual, or
motor/kinesthetic. 4. Accommodation: the design is adequate to fulfil the needs of each user group. 5. Linguistic Clarity: the language used to communicate with the user is understandable, efficient and
adequate. 6. Aesthetic Integrity: the design is visually appealing to the user. 7. Simplicity: the design should not be unnecessarily complex. 8. Predictability: users will be able to form a mental model of how the system will respond to taken
actions and predict their outcomes. 9. Interpretation: the definition of a set of codified rules that interpret user actions to system
functions. 10. Accuracy: taken actions have the expected outcome and no errors are made 11. Technical Clarity: the concepts represented in the interface are clearly mapped to the domain they
are modeling. 12. Flexibility: the design can be adjusted to the needs of its user. 13. Fulfillment: the user has a fulfilling experience in using the system. 14. Cultural Propriety: user's cultural and social expectations are met. 15. Suitable Tempo: the pace at which users works with the system is adequate. 16. Consistency: different parts of the system have the same style, so that there are no different ways
to represent the same information or behaviour. 17. User Support: the design will support learning and provide the required assistance to usage. 18. Precision: the steps and results of a task will be what the user wants. 19. Forgiveness: the user will be able to recover to an adequate state after an error.
22 Weinschenk, Susan, and Dean T. Barker. Designing Effective Speech Interfaces. New York: Wiley, 2000.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
25
20. Responsiveness: the interface provides enough feedback information about the system status and the task completion.
Figure 7 shows the results of the end-user evaluation.
Figure 7 Mean and Median User Scores for each HMI design criterion.
According to participants’ input, the criteria that received the highest ranking are:
Simplicity,
Responsiveness,
Forgiveness,
Linguistic Clarity, and
Aesthetic Integrity.
Therefore, it is safe to conclude that PACT Decision Support System is expected to provide:
Minimalistic visualizations in a simple, non-cluttered environment,
Dialogue with the user and useful feedback about its status,
Easy tracking of and recovery from errors,
Clarity of language, with no abbreviations and unnecessary jargon, and
An aesthetically pleasing and cohesive user interface, without cluttered menus, lists etc.
3.2 DSS User Requirements
Following the discussions and feedback collected and analysed from all PACT expert groups and as
described in the previous section, a set of user requirements were extracted. The following table
consolidates the user requirements in the format defined in Chapter 1.
Each requirement has a unique ID and is briefly summarized within the table. Requirements are either
marked as functional or non-functional, depending on their scope. Functional requirements define
functionalities of a given system, including the inputs and outputs. Therefore, functional requirements
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
26
focus on describing what tasks a system should perform. Non-functional requirements set constraints on
the behaviour of a system and the resources that it utilizes. In essence, non-functional requirements focus
on how a system performs a specific task. The combination of functional and non-functional requirements
can be utilized to distil the system architecture and functional design.
The key points that were raised by the workshop participants, and that permeate the collected user
requirements are summarized as follows:
The main functionality of the PACT DSS is to provide an impact assessment that is context-
sensitive and flexible in its definition: The PACT DSS should provide an impact assessment taking
into account the user’s choice of scenario. This may include utilizing data from multiple sources, as
well as taking into account the legal, ethical implications of the scenario. This is particularly
apparent in D61.U-1, D61.U-3, D61.U-4, and D61.U-6 to D61.U-10 (see table below).
The PACT DSS should be easy-to-use and non-confusing to the user: This key point was raised
several times, regarding not only the HMI design but also the use of technical language etc. The use
of simple language is necessary and the system should not require a high level of ICT experience.
This is evident within D61.U-5, D61.U-7 and D61.U-11 to D61.U-17 (see table below).
The PACT DSS should allow the user to provide feedback in every stage of its use, while showing
fault-tolerant behaviour towards the user and ensuring the reliability of the presented data: The
users require a high level of customization to be available as well as the ability to provide feedback
to the system in every part of the process. The user must then be able to retrace their steps and
easily recover from errors. The information retrieved and processed by the PACT DSS should always
be reliable. This is a common theme, apparent in D61.U-2, D61.U-15, D61.U-17 and D61.U-18 (see
table below).
The user requirements can then be used to extrapolate the technical requirements and functional design of
the system, both essential parts of a robust engineering process. The Requirements Traceability Matrix that
follows in Chapter 5, provides further insight on how the collected user requirements are linked to the
individual technical requirements.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
27
ID Brief Summary Requirement Source Functional / Non-
Functional
Domain
D61.U-1 Privacy Impact Assessment
The system should provide a tangible report in the form of a privacy impact assessment for security technologies
Security Stakeholders/ Privacy Advocate
Functional General, Privacy
D61.U-2 Impact assessment assistance
Given that 1/3 impact assessment reports are sent back to be re-worked, the PACT DSS should offer personalised support, assistance and informal validation for impact assessment procedures.
High Level Policy Panel
Functional General, Security, Privacy
D61.U-3 Context-specific
The PACT DSS should focus in the particular context and scope of the scenario/technology under investigation.
Security Stakeholders/ High Level Policy Panel
Functional General, Security
D61.U-4 Consolidated knowledge
The system should consolidate theoretical and empirical data from multiple sources.
Security Stakeholders/ High Level Policy Panel
Non Functional
General, Security
D61.U-5 Security Language
The system should avoid negative connotations connected with specific terms, and should “talk” the same language with the users.
High Level Policy Panel
Functional General, Security, Privacy
D61.U-6 Laws and Ethics
The system should highlight case-specific connected legal and ethical issues, along with any potential tension between those.
High Level Policy Panel
Functional Legal, Ethical
D61.U-7 Measurable Index
The system should use whenever possible measurable index, applied e.g. to PTI.
High Level Policy Panel
Functional General, Security
D61.U-8 Intangible assets
The system should take into account intangible assets
Security Stakeholders
Functional Privacy, Security
D61.U-9 Compliance The system should check if proposed security solution complies with current standards, practices and legislations
Security Stakeholders/ High Level Policy Panel
Functional Legal, Privacy, Security
D61.U-10 Update Knowledge Base
Ability to update the knowledge base including results and standards taken into account etc.
High Level Policy Panel
Functional General
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
28
D61.U-11 Data Visualisations
The system should provide useful data visualizations to the user.
Security Stakeholders
Functional General
D61.U-12 Data Input The HMI must provide a way to input scenario-specific data
Security Stakeholders/ Privacy Advocate
Functional General
D61.U-13 Non-cluttered visualizations
The system’s HMI should not be cluttered with too much data to visualize simultaneously.
Security Stakeholders
Non-Functional
General
D61.U-14 Simple visualizations
Data visualizations should be simple, non-complex with the ability to expand into more complex views if necessary
Security Stakeholders
Non-Functional
General
D61.U-15 Wizard-like interface
The HMI should allow the data input and data processing in clearly defined steps, making it easy for the user to track back their steps if required.
Security Stakeholders
Non-Functional
General
D61.U-16 Direct Manipulation Icons
Use of icons in HMI design, with clear mapping to real-world concepts
Security Stakeholders
Non-Functional
General
D61.U-17 Easy Access Access to the PACT DSS should be easy and shouldn’t require too much ICT skills on behalf of the users
Security Stakeholders
Non-Functional
General
D61.U-18 Reliability PACT DSS should provide reliable information at all times.
Security Stakeholders, High Level Policy Panel
Non-Functional
General
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
29
4 DSS Architecture, Functional and Technical Requirements
In this chapter the core output of T6.1 is presented: the DSS Architecture and the Functional and Technical
Requirements for the system and individual components. These include namely the requirements
describing the behaviour and functions of the system and the technical aspects that the system must fulfil
respectively, such as performance related issues, reliability and availability issues.
The architecture is defined on the backdrop of a study of the state of the art (cf. PACT D.1.2), of relevant
ongoing research (as presented in Chapter 2) and of optimal design patterns suitable for the general needs
of the PACT DSS. Functional and technical requirements are extracted from the user requirements
presented in chapter 3 and the expressed needs of the wide range of stakeholder groups engaged in PACT
project.
4.1 DSS Architecture
Having gone through the State of the Art for relevant systems and given the user requirements and joint
WP5/WP6 activities, T6.1 produced a preliminary architecture for the PACT DSS. The preliminary
architecture sets the frame in which functional and technical requirements for the DSS will be populated
and will be finalised with the specifications definition in D6.2. The architecture (depicted in Figure 8) is
optimized for a web-based application and thus follows the design pattern for a three-tier architecture,
with distinct tiers for each of: data, application logic and front-end/user side. A three-tier architecture is
favoured for web applications due to its modularity and flexibility, as it creates a logical separation
between the presentation layer, the business logic layer, and the database layer. The separation of data
from the application (or business) rules and presentation creates a flexible environment where changes in
each layer leave other layers unaffected. The modular design enables the possibility of updating data
seamlessly into the system, or creating a multi-user environment with changing application rules without
disrupting presentation or causing extra development overhead. An analysis of each architecture layer
follows.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
30
Figure 8 PACT Preliminary DSS Architecture
The User Layer
Being a web-based application, the user layer includes a so-called web “thin client”. Namely the user can
access the application through this web-browser without the need of deploying or running a specific client
application on his computer. This comes with several advantages:
1. Convenience and accessibility: The user can access the PACT DSS anywhere, anytime. The
application is not bound physically to specific computer or device while as long as the server is
up and running, the user can access the application on a 24-7-365 basis.
2. No installation and maintenance effort: The web application does not come with extra effort or
skills required from the user to install it or maintain it, such as e.g. installing additional batches and
updates. Instead the application is deployed and maintained on the server-side, leaving this effort
to the particular application/service provider.
3. Interoperability: As a web application PACT DSS will run across all operating systems and platforms
equipped with a web browser and compliant with relevant internet presentation standards.
4. Cost-effective: Web applications are in general more cost-effective than desktop applications, given
the amount of users they can hold without the costs of individual maintenance fees.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
31
5. User Training: With the application being online and easily supervised remotely or connected with
an eLearning module, user training can be done far more efficiently, anywhere, anytime and at the
user’s pace of convenience.
Besides the general advantages that come with opting for a web-based approach for the PACT DSS, the
PACT application will also benefit from Web 2.0 state of the art techniques and patterns such as social
media, online collaboration, interactive interfaces and dynamic, user driven and user responsive content.
Typical UIs and DSS “look and feel” can be found in PACT D1.2, with PACT DSS aiming to provide a context
optimised and state-of-the-art UI. Visualisation components for the PACT DSS are managed and generated
on the server side, via the HMI, Dialogue Management & Visualisation module.
A small, but not insignificant, caveat here about social media and crowdsourcing perceptions must be
considered when the functional specifications are set in D6.2. Using data – particularly on user
perceptions - generated on social media and other crowdsourcing platforms is fraught with ethical and
data protection challenges. Furthermore, doubts remain about the research value of those data. These
should be borne in mind as we move to setting the functional specifications in D6.2 and further into
the WP. While the project recognises that the research tool of the project is looking to be context
relevant in the future, the ethical, data protection and value of using the input data remain.
The Application Server
The application server is where the PACT DSS Application Logic resides. The rules and algorithms
implementing DSS functionalities are in this layer and divided into three modules, each one grouping a
particular set of functionalities: (i) the HMI, Dialogue Management & Visualisation module, responsible for
handling the presentation and interaction with the User; (ii) the Decision Trees & Engine Module,
responsible for the creation of Decision Support Trees and graphs according to the specific context,
technologies and case under investigation by the user, based on knowledge and data provided by the (iii)
Knowledge Management Module, a group of services responsible for extraction of knowledge and data
management provided by the Database.
The PACT DSS Application Server is the overall responsible layer for implementing the needed intelligence
for the PRFST toolbox as described in PACT D5.2. Required data is retrieved from the database, fed into the
decision support process and presented accordingly to the user through multiple visualisation modes (such
as charts, spider diagrams and colour coded tables) as rational arguments towards supporting the user’s
decision process. As a decision support tool however, it should be noted that the PACT DSS is not a legally
bound tool responsible of taking or validating legally the user’s decisions. The user is the expert responsible
of taking decisions, and the PACT DSS is a convenient and user-friendly tool to help him reach to the
decision faster and with a better view of all associated parameters, knowledge and arguments.
HMI, Dialogue Management & Visualisation Module
This module is responsible for the HMI, managing the dialogue and interaction space with the user, while
implementing visualisation components from the data and knowledge provided by the Knowledge
Management Module and from the graphs and conceptual trees provided by the Decision Trees & Engine
Module. The user dialogue with the PACT DSS will promote an interface users will find friendly and intuitive.
To achieve this, the user will perform security technologies selection and context parameterisation through
assistive UI components which will minimise strenuous UI activities like excessive clicks and complicated
structures navigation, by applying optimal design patterns including “one-click-away”, auto completion
input forms, drop-down selection lists and personalisation capabilities according to the specific user. User
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
32
Profile management is therefore also part of this module, adapting the interface to the particular user
needs and preferences, as well as to user decision and access level.
Regarding visualisation components, the module will provide convenient visualisation modalities in order to
provide the user with the necessary insight on decision-affecting factors. According to the nature and
structure of knowledge/data the optimal representation changes according to the complexity of the
knowledge structure and the relationships between multiple criteria, with a key visualisation mode being
the presentation of a user friendly representation of decision trees (full blown trees may be complicated as
in Figure 12) managed by the Decision Trees & Engine Module. E.g. in PACT D5.2 data and knowledge
correlations are visualised using the sample bar chart (Figure 9) and spider diagram (Figure 10). For each
technology two colours are used - positive effect is represented using the colour on right and negative
effect is represented using the colour on left Refer to PACT D5.2 for better understanding colour
representation and semantics.
Figure 9 Sample PACT bar chart
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
33
Figure 10 Sample PACT spider diagram
As explained in PACT D5.2: “Apart from decision trees there are other kind of charts that are useful in order
to make decision makers aware of how suitable are the options in accordance with their compliance to
privacy criteria. The objective of charts are to show decision makers in a clear way a comparison of the
possibilities according to the weights assigned to show how much they are privacy respecting”. For more
information on the charts and their use within the PRFST methodology please refer to section 4 of PACT
D5.2.
Knowledge and assessment of values will in some cases be presented using easy to understand colour
coding, as in the case of the Privacy Threat Index (PTI) of the PRFST (Figure 11), with low, medium and high
threat-impact combinations are coloured from green to yellow to red, depending on the level of attention
needed by the user in terms of privacy issues and corrective actions needed.
Threat Likelihood Impact
Low Medium High
High
Medium
Low Figure 11 Privacy Threat Index, Colour Scheme representation
The UI dialogue will go through the specific security technology case study through a guided step-by-step
structure, which will allow fast and convenient transition to any step needed to be revisited or edited. After
completing all steps, the final assessment, all user selections, input and annotations will also be provided in
an exportable formal (e.g. PDF file) as a consolidated impact assessment report.
Decision Trees & Engine Module
The Decision Trees & Engine Module is responsible for receiving relevant data and building the sequential
decision process in the form of a conceptual decision tree, “including decision alternatives, events (states of
nature), probabilities/weights and outcome of alternatives related with the sequential decisions” (PACT
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
34
D5.2). The module therefore builds the tree hierarchy including multiple available attributes such as factors
affecting the public perception and trust.
SUR
VEI
LLA
NC
E TE
CH
NO
LOG
Y SE
LEC
TIO
N
DA5
DA4
DA3
DA2
DA1
E1
E2
E3
E1
E2
E3
E1
E2
E3
E1
E2
E3
E1
E2
E3
O51
O52
O53
O41
O42
O43
O31
O32
O33
O21
O22
O23
O11
O12
O13
Figure 12: Sample decision tree
As shown in Figure 12, the full blown decision tree shows the overall picture when there are several
options to choose. It is the Visualisation’s module task to ensure efficient and user-friendly visualisation of
the knowledge encapsulated in the decision trees, while the decision trees module manages relevant
knowledge structures and hierarchies, offering the necessary intelligence and application logic to guide the
user through the optimal decision path. The module supports the user’s decision and does not by any
means enforce decisions or drive the decision process in an automated way. It is important to clarify that
the design of the PACT DSS is user-driven, and decisions are taken by the user himself, with the system
supporting the process through multiple modalities and presenting quantitative and qualitative information
regarding the impact of each decision path.
Knowledge Management Module
The Knowledge Management Module is responsible for managing and feeding the empirical and theoretical
knowledge to all other modules of the application server, namely for presentation, visualisation and
decision process building purposes. The module establishes the connection with the database server,
extracts needed data and provides it in a convenient format through a set of embedded web services. The
services will provide necessary data formatting as well as any necessary additional mathematical
functionality not covered by the stored data results.
Given the large PACT dataset which includes the whole survey results, the Knowledge Management Module
will focus on the efficient management and building of relevant and focused context around the decision
process and user selected case/technology under investigation, enabling a context-driven and specific
approach. To achieve this, additional context parameters will either be provided through the HMI dialogue,
user profiling and/or location-aware application settings.
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
35
The Database Server
The data layer consists of the PACT DSS database and database server, which includes all data stored in a
convenient structure for the PACT system to use. Data stored in the database will include both theoretical
and empirical PACT findings. The former will contain theoretical input assigned to specific use cases as in
PACT D1.6, as well as risks and privacy threats associated with particular technologies as per the mapping in
PACT D1.3. The data base E-R (Entity Relationship) diagram will interconnect the risk mapping and
technology taxonomy with key decision-influencing knowledge elements such as related regulatory
frameworks and laws, associated ethical values, privacy goals & assets and other attributes.
The E-R diagram will extend and include relationships to the empirical set of data, encapsulating the public
perception and opinion into the overall picture and associating multiple attributes and parameters that
may be affecting it. For this purpose, task 6.1 and partner NCSRD has collaborated with RAND and T4.2 and
produced a data template and format convenient for porting the PACT survey results into the DSS
database. The template consists of a single 2-dimensional table per country per context/scenario, with each
row corresponding to a different technology choice/attribute (e.g. standard CCTV, CCTV with embedded
intelligence, etc) and each column corresponding to a different demographic or other parameter (e.g. age,
income, education etc). Figure 13 shows an extract of the survey data table.
Figure 13 Survey Data Table Sample for Travel context
Attribute Observed Predicted Std. dev nσ Observed Predicted Std. dev nσ
Standard CCTV – working like a television 4.173 4.213 48,9 -0,8 4.919 4.904 52,4 0,3
Advanced CCTV that can detect abandoned bags 4.801 4.789 51,8 0,2 5.638 5.641 55,9 -0,1
Advanced CCTV that can recognise suspicious movements of people 4.603 4.582 50,6 0,4 5.196 5.219 53,9 -0,4
Advanced CCTV that can recognise faces 4.583 4.602 50,2 -0,4 5.335 5.306 53,5 0,5
No CCTV cameras 2.798 2.800 44,7 0,0 2.840 2.831 45,8 0,2
NA 0 0 0,0 0,0 0 0 0,0 0,0
CCTV information not stored for future use-only used in real time monitoring 3.058 3.000 43,3 1,3 3.402 3.474 46,5 -1,6
CCTV information stored for 3 days 3.124 3.102 41,3 0,5 3.555 3.604 44,2 -1,1
CCTV information stored for 7 days 4.367 4.429 48,4 -1,3 5.110 5.041 51,3 1,3
CCTV information stored for 15 days 4.441 4.470 48,6 -0,6 5.279 5.228 52,2 1,0
CCTV information stored for 45 days 3.170 3.185 43,2 -0,3 3.742 3.723 46,4 0,4
No CCTV cameras 2.798 2.800 44,7 0,0 2.840 2.831 45,8 0,2
Only police departments in the [UK] have access to the camera information 6.609 6.579 59,4 0,5 7.656 7.697 63,9 -0,6
All European police departments have access to the camera information 6.292 6.328 58,8 -0,6 7.335 7.302 62,8 0,5
All police departments worldwide have access to the camera information 5.259 5.279 56,2 -0,3 6.097 6.070 60,0 0,4
No CCTV cameras 2.798 2.800 44,7 0,0 2.840 2.831 45,8 0,2
NA 0 0 0,0 0,0 0 0 0,0 0,0
NA 0 0 0,0 0,0 0 0 0,0 0,0
No security personnel 3.532 3.460 47,7 1,5 3.879 3.960 50,9 -1,6
Unarmed security personnel employed by a private company 4.546 4.633 50,5 -1,7 5.339 5.285 53,8 1,0
Armed security personnel employed by a private company 4.173 4.186 49,3 -0,3 4.792 4.760 52,3 0,6
Unarmed police 4.578 4.603 50,1 -0,5 5.267 5.264 53,4 0,1
Armed police 4.129 4.104 48,7 0,5 4.651 4.631 51,5 0,4
NA 0 0 0,0 0,0 0 0 0,0 0,0
People randomly selected for physical search and bag check 5.614 5.710 60,5 -1,6 6.597 6.521 64,4 1,2
People randomly selected to go through metal detector or full body scanner 7.318 7.228 64,8 1,4 8.285 8.327 69,3 -0,6
No physical security checks 8.026 8.047 65,4 -0,3 9.046 9.053 69,1 -0,1
NA 0 0 0,0 0,0 0 0 0,0 0,0
NA 0 0 0,0 0,0 0 0 0,0 0,0
NA 0 0 0,0 0,0 0 0 0,0 0,0
10 seconds 3.477 3.318 44,5 3,6 4.083 3.834 47,5 5,2
30 seconds 1.768 1.765 32,8 0,1 1.993 2.025 34,9 -0,9
1 minute 2.104 2.151 35,6 -1,3 2.287 2.405 37,5 -3,1
2 minutes 1.975 2.090 35,2 -3,3 2.254 2.365 37,4 -3,0
5 minutes 3.608 3.614 48,0 -0,1 4.265 4.219 51,7 0,9
None 8.026 8.047 65,4 -0,3 9.046 9.053 69,1 -0,1
None of these 7.247 7.219 71,7 0,4 8.339 8.367 76,9 -0,4
Total 28.205 28.205 32.267 32.267
Male Female
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
36
As described previously the PACT DSS is context-specific and the data model of the PACT data base builds
upon the context as specified in the PACT survey, namely the Travel, Health and Internet scenarios.
However, the design will follow optimal patterns in order to be modular and extensible, having the ability
to integrate future empirical studies in other context, or enrich the existing scenarios without overhead or
need for extended offline time for the PACT DSS system.
4.2 Functional and Technical Requirements
Based on the user requirements presented in chapter 3 and the design considerations expressed through
the DSS architecture in section 4.1, T6.1 proceeded with defining the Functional and Technical
Requirements of the PACT DSS. Functional requirements describe the functions of the DSS, namely the
inputs, the behaviour and the outputs of the system. On the other hand, technical requirements describe
quality traits of the system, characterising its operation and pertaining its technical aspects. Both constitute
essential elements and results of requirement analysis, an important step in software engineering.
Following analysis, functional and technical requirements are described in a way that the needs of the users
are translated into system functions, behaviours and operational characteristics, which are in turn
understandable by developers in order to proceed with system specifications and implementation. For this
purpose, functional and technical requirements are clearly linked to user requirements in order to ensure
all identified real needs are being met.
The requirements are organised according to the Architecture to overall requirements and to individual
component requirements.
4.1.1 Overall System Requirements
In this section the functional and technical requirements corresponding to general functionalities and
technical traits of the system as a whole are presented.
Functional Requirements
This section is dedicated to the activities and processes the system must perform in general and specifically
for information retrieved (input) and delivered (output) as well as for management purposes. Tables with
corresponding requirements follow respectively.
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Ma
nd
ato
ry
Opti
onal
D61.F-
19
Context-specific decision and
assessment case
The user will select the context
and specific security technology
application to be assessed among
available options of the PACT DSS.
D61.U-3 X
D61.F-
20
Assets The DSS will connect with selected
case related tangible and
intangible assets.
D61.U-8 X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
37
D61.F-
21
Decision-making process support The user will be offered with
visualised rationale arguments at
each key decision choice point to
support the decision-making
process.
D61.U-11 X
D61.F-
22
Multiple factors assessment The user will assess the impact
through evaluation of multiple
factors: associated risks, privacy,
ethics, social acceptance and
public perception.
D61.U-2 X
D61.F-
23
Theoretical and empirical
knowledge
Relevant theoretical and empirical
knowledge will be provided to the
user in an efficient, relevant and
context-driven manner.
D61.U-4 X
D61.F-
24
Measurable index A measurable index for
assessment will be offer to the
User, the Privacy Impact (or
Threat) Index.
D61.U-7 X
D61.F-
25
Laws and ethics The PACT DSS will provide quick
references to related laws,
regulations, standards as well as
ethics issues, highlighting any
known tension or controversies.
D61.U-6,
D61.U-9
X
D61.F-
26
Storing and retrieving multiple
assessment cases
The user shall be able to save his
work and resume it at a later
point. The User will be able to
hold multiple ongoing assessment
cases.
D61.U-2 X
D61.F-
27
Security Terminology PACT DSS will use a standard
security language based on
relevant European
recommendations (e.g. STACCATO
security taxonomy).
D61.U-5 X
D61.F-
28
Input PACT DSS will be able to receive
input from the user for a
particular security
technology/scenario under
investigation.
D61.U-12 X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
38
D61.F-
29
Output: Impact assessment report PACT DSS will produce an impact
assessment report which he can
save it for future reference, print
it or produce an exportable file
and sharable result.
D61.U-1 X
D61.F-
30
Online collaboration PACT DSS will enable multiple user
collaboration on a single decision
problem, offering sharing/inviting,
parallel working and
synchronization functionalities.
D61.U-2,
D61.U-10
X
D61.F-
31
User Profile Management PACT DSS will support user
registration and use via login. This
will effectively support profile
management, personalised user
preferences and personal
assessment cases portfolio.
D61.U-2 X
D61.F-
32
Anonymous login PACT DSS will also operate for
anonymous users, i.e. without
need for login.
D61.U-2 X
D61.F-
33
Social Media –crowdsourcing The PACT DSS will be able to
receive updates regarding social
acceptance and public perception
via social media and
crowdsourcing techniques.
D61.U-4 X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
39
Technical Requirements
This section includes the requirements encapsulating the technical aspects the overall system must fulfil in
general, for targeting usage environment and platform, achieving satisfactory response times, scalability
purposes and meeting security, privacy and ethics requirements. Tables with corresponding requirements
follow respectively.
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Ma
nd
ato
ry
Opti
onal
D61.T-34 Online access The PACT DSS will be a web
application and will be accessed
online.
D61.U-17 X
D61.T-35 Platform independence The PACT DSS will be able to
operate smoothly across all
popular platforms and web
browsers.
D61.U-17 X
D61.T-36 System response time The DSS shall be able to respond
to any user request within a
specified period of time.
D61.U-17 X
D61.T-37 Concurrent users support The DSS shall be able to operate
smoothly under increased load.
D61.U-17 X
D61.T-38 Extensibility The PACT DSS shall be able to
interface and use data from
multiple sources.
D61.U-4 X
D61.T-39
Protected Access Access for registered users will be
protected through use of
password.
D61.U-2 X
D61.T-40 Integrity of information The integrity of information
processed by the system must be
checked.
D61.U-18 X
D61.T-41 Disclaimer A disclaimer will be provided to
the users explaining the use of
profile data as well as the
boundaries of the PACT DSS tool.
D61.U-18,
D61.U-2
X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
40
4.1.2 Individual Component Requirements
User Layer, HMI, Dialogue Management & Visualisation Module
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Ma
nd
ato
ry
Opti
onal
D61.F-
42
Visualisation Modes Visualisation modes used by PACT
DSS include bar charts, spider
diagrams and colour coded tables.
D61.U-11,
D61.U-14,
D61.U-16
X
D61.F-
43
Minimum effort selection Use of autocomplete and selection
forms will be used to minimise
user effort and overhead.
D61.U-13,
D61.U-12,
D61.U-14
X
D61.F-
44
Assistive Step-by-step procedure The HMI dialogue will be
structured in an assistive step by
step procedure, with the ability of
re-visiting any step at any time
without losing any data or
progress.
D61.U-15,
D61.U-13
X
D61.T-
45
UI fast navigation Navigation throughout the PACT
DSS UI shall be reduced to the
minimum number of clicks.
D61.U-15,
D61.U-13
X
Decision Trees & Engine Module
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Ma
nd
ato
ry
Opti
onal
D61.F-
46
Decision Engine The PACT DSS Decision Engine will
be an implementation of
algorithms and structures that will
utilize the existing knowledge to
apply criteria and offer advice on
the scenarios specified.
D61.U-2 X
D61.F-
47
Sequential decision process The user decision process for the
specific case/security technology
will be modelled as a conceptual
decision tree.
D61.U-2,
D61.U-15
X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
41
D61.F-
48
Tree optimal path The system will calculate at all
times the tree optimal path for
minimum impact, posing relevant
arguments and indicators.
However the user is responsible
for driving the decision process
throughout the tree at his own
free will and choice.
D61.U-2 X
D61.F-
49
Decision sequence transition Transition through decision
sequence should be allowed
without losing progress or data.
D61.U-2,
D61.U-15
X
Knowledge Management Module & Database Server
ID Brief Summary Requirement
Requireme
nt Source
Ref.
Ma
nda
tory
Opti
onal
D61.T-
50 Knowledge Database PACT DSS will hold an updatable
database containing models and
statistics pertaining to privacy
and security concerns, including
fine- and coarse-grained
information and theoretical
knowledge based on context.
D61.U-10,
D61.U-18,
D61.U-4,
D61.U-5,
D61.U-6,
D61.U-8
X
D61.T-
51
Knowledge management Knowledge and data will be
managed through a set of web
services, offering efficient
management.
D61.U-10,
D61.U-18,
D61.U-12,
D61.U-4
X
D61.T-
52
Data model extensibility The data model will be easily
extensible in order to support
updates from future empirical
research.
D61.U-10 X
D61.T-
53
Internal data format An internal data model will be
applied to the knowledge
supplied so it can be
meaningfully stored and
become available for use.
D61.U-10 X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
42
5 Requirements Traceability Matrix
The following table, called the Requirements Traceability Matrix, consolidates all requirements defined in
this document, for purposes of easy reference and convenient reading by future WP6 activities.
User Requirements
ID Brief Summary
Requirement Source Functional / Non-
Functional
Domain
D61.U-1 Privacy Impact Assessment
The system should provide a tangible report in the form of a privacy impact assessment for security technologies
Security Stakeholders/ Privacy Advocate
Functional General, Privacy
D61.U-2 Impact assessment assistance
Given that 1/3 impact assessment reports are sent back to be re-worked, the PACT DSS should offer personalised support, assistance and informal validation for impact assessment procedures.
High Level Policy Panel
Functional General, Security, Privacy
D61.U-3 Context-specific
The PACT DSS should focus in the particular context and scope of the scenario/technology under investigation.
Security Stakeholders/ High Level Policy Panel
Functional General, Security
D61.U-4 Consolidated knowledge
The system should consolidate theoretical and empirical data from multiple sources.
Security Stakeholders/ High Level Policy Panel
Non Functional
General, Security
D61.U-5 Security Language
The system should avoid negative connotations connected with specific terms, and should “talk” the same language with the users.
High Level Policy Panel
Functional General, Security, Privacy
D61.U-6 Laws and Ethics
The system should highlight case-specific connected legal and ethical issues, along with any potential tension between those.
High Level Policy Panel
Functional Legal, Ethical
D61.U-7 Measurable Index
The system should use whenever possible measurable index, applied e.g. to PTI.
High Level Policy Panel
Functional General, Security
D61.U-8 Intangible assets
The system should take into account intangible assets
Security Stakeholders
Functional Privacy, Security
D61.U-9 Compliance The system should check if proposed security solution complies with current standards, practices and legislations
Security Stakeholders/ High Level Policy Panel
Functional Legal, Privacy, Security
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
43
D61.U-10 Update Knowledge Base
Ability to update the knowledge base including results and standards taken into account etc.
High Level Policy Panel
Functional General
D61.U-11 Data Visualisations
The system should provide useful data visualizations to the user.
Security Stakeholders
Functional General
D61.U-12 Data Input The HMI must provide a way to input scenario-specific data
Security Stakeholders/ Privacy Advocate
Functional General
D61.U-13 Non-cluttered visualizations
The system’s HMI should not be cluttered with too much data to visualize simultaneously.
Security Stakeholders
Non-Functional
General
D61.U-14 Simple visualizations
Data visualizations should be simple, non-complex with the ability to expand into more complex views if necessary
Security Stakeholders
Non-Functional
General
D61.U-15 Wizard-like interface
The HMI should allow the data input and data processing in clearly defined steps, making it easy for the user to track back their steps if required.
Security Stakeholders
Non-Functional
General
D61.U-16 Direct Manipulation Icons
Use of icons in HMI design, with clear mapping to real-world concepts
Security Stakeholders
Non-Functional
General
D61.U-17 Easy Access Access to the PACT DSS should be easy and shouldn’t require too much ICT skills on behalf of the users
Security Stakeholders
Non-Functional
General
D61.U-18 Reliability PACT DSS should provide reliable information at all times.
Security Stakeholders
Non-Functional
General
Functional and Technical Requirements
Overall System Functional Requirements
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Manda
tory
Opti
onal
D61.F-19 Context-specific decision
and assessment case
The user will select the context and
specific security technology
application to be assessed among
available options of the PACT DSS.
D61.U-3 X
D61.F-20 Assets The DSS will connect with selected
case related tangible and intangible
assets.
D61.U-8 X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
44
D61.F-21 Decision-making process
support
The user will be offered with
visualised rationale arguments at
each key decision choice point to
support the decision-making
process.
D61.U-11 X
D61.F-22 Multiple factors
assessment
The user will assess the impact
through evaluation of multiple
factors: associated risks, privacy,
ethics, social acceptance and public
perception.
D61.U-2 X
D61.F-23 Theoretical and empirical
knowledge
Relevant theoretical and empirical
knowledge will be provided to the
user in an efficient, relevant and
context-driven manner.
D61.U-4 X
D61.F-24 Measurable index A measurable index for assessment
will be offer to the User, the Privacy
Impact (or Threat) Index.
D61.U-7 X
D61.F-25 Laws and ethics The PACT DSS will provide quick
references to related laws,
regulations, standards as well as
ethics issues, highlighting any known
tension or controversies.
D61.U-6,
D61.U-9
X
D61.F-26 Storing and retrieving
multiple assessment cases
The user shall be able to save his
work and resume it at a later point.
The User will be able to hold
multiple ongoing assessment cases.
D61.U-2 X
D61.F-27 Security Terminology PACT DSS will use a standard
security language based on relevant
European recommendations (e.g.
STACCATO security taxonomy).
D61.U-5 X
D61.F-28 Input PACT DSS will be able to receive
input from the user for a particular
security technology/scenario under
investigation.
D61.U-12 X
D61.F-29 Output: Impact assessment
report
PACT DSS will produce an impact
assessment report which he can save
it for future reference, print it or
produce an exportable file and
sharable result.
D61.U-1 X
D61.F-30 Online collaboration PACT DSS will enable multiple user
collaboration on a single decision
problem, offering sharing/inviting,
parallel working and
synchronization functionalities.
D61.U-2,
D61.U-10
X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
45
D61.F-31 User Profile Management PACT DSS will support user
registration and use via login. This
will effectively support profile
management, personalised user
preferences and personal
assessment cases portfolio.
D61.U-2 X
D61.F-32 Anonymous login PACT DSS will also operate for
anonymous users, i.e. without need
for login.
D61.U-2 X
D61.F-33 Social Media –
crowdsourcing
The PACT DSS will be able to receive
updates regarding social acceptance
and public perception via social
media and crowdsourcing
techniques.
D61.U-4 X
Overall System Technical Requirements
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Manda
tory
Opti
onal
D61.T-34 Online access The PACT DSS will be a web
application and will be accessed
online.
D61.U-17 X
D61.T-35 Platform independence The PACT DSS will be able to operate
smoothly across all popular
platforms and web browsers.
D61.U-17 X
D61.T-36 System response time The DSS shall be able to respond to
any user request within a specified
period of time.
D61.U-17 X
D61.T-37 Concurrent users support The DSS shall be able to operate
smoothly under increased load.
D61.U-17 X
D61.T-38 Extensibility The PACT DSS shall be able to
interface and use data from multiple
sources.
D61.U-4 X
D61.T-39
Protected Access Access for registered users will be protected through use of password.
D61.U-2 X
D61.T-40 Integrity of information The integrity of information
processed by the system must be
checked.
D61.U-18 X
D61.T-41 Disclaimer A disclaimer will be provided to the
users explaining the use of profile
data as well as the boundaries of the
PACT DSS tool.
D61.U-18,
D61.U-2
X
User Layer, HMI, Dialogue Management & Visualisation Module Requirements
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
46
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Manda
tory
Optio
nal
D61.F-42 Visualisation Modes Visualisation modes used by PACT
DSS include bar charts, spider
diagrams and colour coded tables.
D61.U-11,
D61.U-14,
D61.U-16
X
D61.F-43 Minimum effort selection Use of autocomplete and selection
forms will be used to minimise user
effort and overhead.
D61.U-13,
D61.U-12,
D61.U-14
X
D61.F-44 Assistive Step-by-step
procedure
The HMI dialogue will be structured
in an assistive step by step
procedure, with the ability of re-
visiting any step at any time without
losing any data or progress.
D61.U-15,
D61.U-13
X
D61.T-45 UI fast navigation Navigation throughout the PACT DSS
UI shall be reduced to the minimum
number of clicks.
D61.U-15,
D61.U-13
X
Decision Trees & Engine Module Requirements
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Manda
tory
Optio
nal
D61.F-46 Decision Engine The PACT DSS Decision Engine will
be an implementation of algorithms
and structures that will utilize the
existing knowledge to apply criteria
and offer advice on the scenarios
specified.
D61.U-2 X
D61.F-47 Sequential decision
process
The user decision process for the
specific case/security technology
will be modelled as a conceptual
decision tree.
D61.U-2,
D61.U-15
X
D61.F-48 Tree optimal path The system will calculate at all times
the tree optimal path for minimum
impact, posing relevant arguments
and indicators. However the user is
responsible for driving the decision
process throughout the tree at his
own free will and choice.
D61.U-2 X
D61.F-49 Decision sequence
transition
Transition through decision
sequence should be allowed without
losing progress or data.
D61.U-2,
D61.U-15
X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
47
Knowledge Management Module & Database Server Requirements
ID Brief Summary Requirement
Requirem
ent
Source
Ref.
Manda
tory
Option
al
D61.T-50 Knowledge Database PACT DSS will hold an updatable
database containing models and
statistics pertaining to privacy and
security concerns, including fine-
and coarse-grained information and
theoretical knowledge based on
context.
D61.U-10,
D61.U-18,
D61.U-4,
D61.U-5,
D61.U-6,
D61.U-8
X
D61.T-51 Knowledge management Knowledge and data will be managed
through a set of web services,
offering efficient management.
D61.U-10,
D61.U-18,
D61.U-12,
D61.U-4
X
D61.T-52 Data model extensibility The data model will be easily
extensible in order to support
updates from future empirical
research.
D61.U-10 X
D61.T-53 Internal data format An internal data model will be
applied to the knowledge supplied so
it can be meaningfully stored and
become available for use.
D61.U-10 X
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
48
6 DSS Validation Criteria
This chapter defines the validation criteria for the target PACT DSS, including user requirement based
validation and acceptance as well as performance and technical success measures for the system and
individual components. Following completion and delivery of the PACT DSS software tool, its successful
operation will be validated against the criteria defined in this chapter. Depending on the criteria type,
validation will be performed by either by technical experts in case e.g. of performance validation or by end
users in case of user perspective criteria. The latter case will include volunteers from the combined PACT
experts groups. T6.4 “Validation and User Evaluation” is responsible for planning and performing relevant
validation tests and acquiring “hands-on” user evaluation feedback through the T6.4 cluster of end-user
participants and volunteers from external experts groups. This is planned to be conducted during the
combined key event consisting of the PACT consortium meeting, SAG and PAP annual meeting and HLPP
final event, 18-19 December 2014 in Brussels, Belgium.
6.1 User Perspective Criteria Being a user-driven system, the PACT DSS User Perspective Validation Criteria are of outmost importance
for the successful validation of the system. By analysing the user feedback reported in chapter 3, the
following user perspective validation criteria are identified for the PACT DSS:
Reduction on average decision process time (including collaboration with other users)
Improving decision specific context visibility & access to necessary knowledge for related security
technologies decision making
Reduction on average time for compiling a privacy impact assessment report
Convenience and acceptance of PACT impact assessment report
Usability & User friendliness
Cost effectiveness (value for money)
6.2 Component Specific Validation Criteria and Success Measures
User Layer, HMI, Dialogue Management & Visualisation Module
HMI validation will feature qualitative and quantitative evaluation criteria. This goal can be achieved by
combining elements from various Usability Inspection paradigms, focusing on Cognitive Walkthroughs. A
cognitive walkthrough is essentially a qualitative task-driven process, where the end user is requested to
perform specific tasks. During the process, the user is encouraged to comment on the system’s ease of use,
responsiveness etc. Common questions during Cognitive Walkthroughs are, for example23:
Does the user understand which subtasks are needed to reach the user's goal?
Is the correct action visible and available?
Does the user get feedback?
Will the user know if their actions are correct or not?
Etc.
23 Blackmon, M. H. Polson, P.G. Muneo, K & Lewis, C. (2002) Cognitive Walkthrough for the Web CHI 2002 vol.4 No.1 pp463–470
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
49
By providing the appropriate set of questions during the PACT DSS Validation, the end-users should be able
to offer insight on the efficiency of the HMI design, especially in terms of the main design criteria that they
identified during the PACT workshop (i.e. simplicity, responsiveness, forgiveness, linguistic clarity, aesthetic
integrity, as mentioned in Section 3.2). Apart from providing a questionnaire to the user, it is also possible
to implement a Think-Aloud protocol 24 during system validation. During “Think-aloud” evaluation,
participants are encouraged to provide real-time verbal reports of their user experience.
During the Cognitive Walkthrough, the users will be asked to answer appropriate questions on the look-
and-feel of the HMI design thus providing a qualitative evaluation of the user interface. In addition to a
Cognitive Walkthrough questionnaire, a number of free usability tools can be used, such as:
Mouse tracking software: By tracking mouse movement and click patterns it is easy to estimate user effort in performing a scenario, as well as discovering possible bottlenecks.
Eye-tracking software: By tracking eye movement, it is possible to identify which UI elements dominate the user’s attention and if they become distracting.
Examples of free tools for usability inspection include Simple Mouse Tracking25 and Usabilla26. In cases
where human observation is required, the users will be debriefed and asked for their consent prior to their
participation in the validation process.
A more quantitative approach may also be implemented, by utilizing Fitt’s Law and the Keystroke-level
model (KLM) to estimate user effort (without requiring human observation). Fitts's law (often cited as Fitts'
law) is a theoretical model of human movement that predicts the time required to move to a target area, as
a function of the distance to the target and the target size. Fitts's law is thus used to model the act of
pointing and/or moving a mouse. KLM provides a time estimate for the completion of a task, by breaking it
down in subtasks (pointing, clicking, typing etc.) and taking into account the mental preparation of the user
as they plan their actions. It can be summarized in the following steps27:
Step 1 — “Obtain a working prototype of computer interface or a step-by-step operational description of a
task.”
Step 2 — “Identify the goals or the desired outcome of work”
Step 3 — “For each of these goals, find subgoals or tasks that achieve the main goals.”
Step 4 — “Identify methods to main goals and all subgoals.”
Step 5 —“Convert description of methods to pseudo-code (the terminology that is described above).”
Step 6 — “State any and all assumptions used in the making of pseudo-code and goals.”
Step 7 — “Determine appropriate mental or keystroke operators for each step.”
Step 8 — “Assign time values to mental or keystroke operators.”
Step 9 — “Add up execution times for operators.”
Step 10 —“Adjust total time of task to be sensitive by age of expected.”
By combining both qualitative and quantitative measures, a comprehensive evaluation of the HMI design
will be achieved.
24
Lewis, C. H. (1982). Using the "Thinking Aloud" Method In Cognitive Interface Design (Technical report). IBM. RC-9265. 25 Simple Mouse Tracking http://smt.speedzinemedia.com (accessed February 2014) 26 Usabilla https://usabilla.com (accessed February 2014) 27 Kieras, D. (1993, 2001), "Using the Keystroke-Level Model to Estimate Execution Times", (On-line handout).
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
50
Decision Trees & Engine Module
Validation of the Decision Trees & Engine Module will be based on the following criteria:
Extensible design and compliance with any valid tree structure
Generated tree consistency
Tree optimal path consistency
Knowledge Management Module & Database Server
Validation of the knowledge and data management services will be based on the following criteria:
Services failures should occur only in communication failure cases
Integrity checks of data
Data consistency and PACT DSS E-R compliance
Availability of services
6.3 Functionality Validation Criteria The PACT DSS functionalities will be validated according to the functional requirements provided in section
4.2 and to the use cases that will be extracted during the software engineering process (task 6.2). Namely
functionality validation will occur by successfully completing a checklist of all required system
functionalities as described by the requirements in this deliverable.
6.4 Performance Validation Criteria
Given its architecture, the PACT DSS in general inherits the performance traits of the online platform it is
deployed upon. The overall system performance will be validated based on the key operations and
performance traits of the system, as these are identified in the preliminary architecture, user and technical
requirements. The performance criteria involve assessment report creation time, concurrent users support
and system response time. Namely, the following criteria will be used for performance validation:
Assessment report creation time
Maximum concurrent users
Average system response time
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
51
7 Conclusions
This deliverable presented the output of Task “T6.1 DSS Architecture Technical Specifications and System
Validation definition”. This included a review and assessment of the relevant state of the art of current DSS
development as carried on by relevant EU research projects - PRISMS, PARIS, SURPRISE, VALUSEC and DESSI
– so to identify which input can be drawn from these projects. In parallel, T6.1 partners collected and
extracted user requirements based on the PRFST-DSS joint workshop results as well as feedback gathered
from all PACT stakeholder groups: the Stakeholder Advisory Group, the Privacy Advocates Panel and the
High Level Policy Panel. Following these tasks, the preliminary DSS architecture was defined and described,
producing also the relevant functional and technical requirements and consolidated Requirements
Traceability Matrix for reference. The architecture will be finalised in the task T6.2 software engineering
activities. Finally, the PACT DSS validation criteria were defined including the user perspective criteria along
with the success measures for each DSS component, functionality and performance validation. Following
completion and delivery of the PACT DSS software tool, its successful operation will be validated against the
criteria defined in this document.
To conclude this document, this section sums up the following important points and DSS features:
The PACT DSS is a decision support software tool implementing the PRFST toolbox as described in
PACT D5.2. Required context data is retrieved from the system database, fed into the decision
support and decision-tree process and presented accordingly to the user through multiple
visualisation modes (such as charts, spider diagrams and colour coded tables) as rational arguments
towards supporting the user’s decision process.
The PACT DSS aims to support decision process for security technologies investments and
assessment of impact in terms of privacy, ethics, social acceptance and public perception, assisting
the decision process of experts through efficient visualisation and user-DSS interaction based on
PACT’s theoretical and empirical findings.
The PACT DSS is context-specific and the data model of the PACT data base builds upon the context
as specified in the PACT survey, namely the Travel, Health and Internet scenarios. However, the
design will follow optimal patterns in order to be modular and extensible, having the ability to
integrate future empirical studies in other context, or enrich the existing scenarios without
overhead or need for extended offline time for the PACT DSS system.
PACT DSS aims at the following user-perspective success criteria:
o Reduction on average decision process time (including collaboration with other users)
o Improving decision specific context visibility & access to necessary knowledge for related
security technologies decision making
o Reduction on average time for compiling a privacy impact assessment report
o Convenience and acceptance of the produced PACT impact assessment report
o Usability & User friendliness
o Cost effectiveness (value for money)
D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635
52
The PACT DSS is not a legal instrument responsible of validating legality of the user’s decisions. The
user is the expert responsible of taking decisions, and the PACT DSS is a convenient and user-
friendly tool to help him reach to the decision with a better view of associated parameters,
knowledge and arguments. It is important to highlight that the design of the PACT DSS is user-
driven, and decisions are taken by the users themselves, with the system supporting the process
through multiple modalities and presenting quantitative and qualitative information regarding the
impact of each decision path.
o During T6.1 by analysing user needs and defining the DSS behaviour and characteristics (as
these are expressed in terms of functional and technical requirements), we have placed a
significant effort to clearly define the DSS scope and to strengthen appropriate premises
within legal and ethical boundaries.
o PACT will also provide a socio-economic impact study and a political impact study (cf.
activities ST7.3.1 and ST7.3.2, PACT DoW, to be included in PACT D7.5) that will further
clarify the risks and limits of a DSS operating beyond constraints and limitations
expressed here.