Miraculous-Life...4.2.6 UI Definition and Guidelines 84 5 Conclusions 87 Appendix A Questionnaire...

7

th Framework Programme

Miraculous-Life

Grant Agreement No. 611421

Public Miraculous-Life i

Project Identification

Project number No. 611421

Duration 1st Dec 2013 – 30

th Nov 2016

Coordinator Andreas Hochgatterer

Coordinator Organisation AIT Austrian Institute of Technology GmbH, Austria

Website www.miraculous-life.eu

Miraculous-Life

Miraculous-Life for Elderly Independent Living

Document Identification

Deliverable ID: D1.2b Specification of use case scenarios and User Interface

Release number/date V19 20.07.2015

Checked and released by Pedro Trindade (CITARD), Ben Loke (Noldus)

Work Status Select one: Not Started, Work in Progress, Finalizing, Finished

Review Status Select one: Not reviewed, In Review, Request for changes, Accepted

Key Information from “Description of Work”

Deliverable Description The role of this deliverable is twofold. Firstly, it aims to define a detailed specification of the use case scenarios. Secondly, it aims to provide a detailed description and specification of the main user interface.

Dissemination Level PU = Public

Deliverable Type R = Report

Original due date (a) Project Month 8 / 31 July 2014, (b) Project Month 12 / 31 November 2014

Authorship& Reviewer Information

Editor Pedro Trindade (CITARD), Christophoros Christophorou (CITARD), Ben Loke (Noldus)

Partners contributing UniGe, UCY, ORBIS, CITARD, MRPS, Noldus, AIT

Reviewed by Panayiotis Andreou (UCY)

Deliverable 1.2b Specification of use case scenarios and User Interface

Public Miraculous-Life ii

Release History

Release Number

Date Author(s) Release description /changes made

V01 19.05.2014 BL/Noldus First version of Del Template, draft TOC.

V02 11.07.2014 DC/MRPS, CW/ORBIS First contribution on the deliverable.

V03 22.07.2014 DC/MRPS, CW/ORBIS Major revision in use cases and interaction requirements.

V04 05.08.2014 MW/Noldus, NA/Noldus Added user interface specifications and finalized for review.

V05 06.08.2014 PA/UCY Reviewed version.

V06 15.08.2014 RW/ORBIS, EC/CITARD, PT/CITARD, DC/MRPS, NA/Noldus.

Contributions received from all partners addressing the comments of the reviewer.

V07 19.08.2014 CC/CITARD, NA/Noldus, BL/Noldus

Final version of D1.2a.

V08 04.11.2014 NA/Noldus, BL/Noldus, RW/ORBIS, DC/MRPS

Updates on the User Interface design based on D6.4a.

V09 22.11.2014 MB/UniGe, DC/MRPS, CW/ORBIS

Updates on all the use cases and integration of new use-cases from UniGe, MRPS, ORBIS, etc.

V10 26.11.2014 ALL PARTNERS Finalization of use cases.

V11 29.11.2014 NA/Noldus, BL/Noldus Updates on the User Interface design.

V12 01.12.2014 NA/Noldus, BL/Noldus, RW/ORBIS, DC/MRPS, CC/CITARD

Finalization of User Interface design.

V13 05.12.2014 BL/Noldus, CC/CITARD Complete version of the deliverable and provision to the assigned reviewer for review.

V13 09.12.2014 PA/UCY Comments provided by the Reviewer.

V14 15.12.2014 ALL PARTNERS Updates based on the comments received by the reviewer.

V15 16.12.2014 BL/Noldus, CC/CITARD Final version of D1.2b released.

V16 17.06.2015 PT/CITARD, NA/Noldus, BL/Noldus, DC/MRPS, CW/ORBIS, JM/ORBIS

Following EU Review comments, chapter 3 was added to provide a meaningful cost-benefit analysis of the use cases.

V17 21.06.2015 PA/UCY, PT/CITARD, NA/Noldus, BL/Noldus, DC/MRPS, CW/ORBIS, JM/ORBIS

Internal Review performed and addressed

V18 18.07.2015 EC/CITARD, CC/CITARD, SK/CITARD, PT/CITARD, DC/MRPS, CW/ORBIS,

Input from different partners to provide a more detailed technical perspective and relevance analysis, following the discussion in the project


Public Miraculous-Life iii

JM/ORBIS, BL/Noldus, CT/UniGe, MB/UniGe, MF/UniGe, ES/AIT, SH/AIT, CS/FH-IGD

consortium meeting in Cyprus, 25th of June

2015.

V19 20.07.2015 EC/CITARD, CC/CITARD, SK/CITARD, PT/CITARD, DC/MRPS, CW/ORBIS, JM/ORBIS, BL/Noldus, CT/UniGe, MB/UniGe, MF/UniGe, ES/AIT, SH/AIT, CS/FH-IGD

Finalization of the deliverable.


Public Miraculous-Life iv

Miraculous-Life Consortium

Miraculous-Life (Contract No. 611421) is a project within the 7th Framework Programme. The consortium members are:

Contact person: Andreas Hochgatterer

Email: [email protected]

Partner 1 AIT AUSTRIAN INSTITUTE OF TECHNOLOGY GMBH (AIT, Project Coordinator, AT)

Contact person: Dimitri Konstantas


Partner 2: UNIVERSITY OF GENEVA (UniGe, CH)

Contact person: Prof. George Samaras


Partner 3: UNIVERSITY OF CYPRUS (UCY, CY)

Contact person: Cindy Wings


Partner 4 ORBIS MEDISCH EN ZORGCONCERN (ORBIS, NL)

Contact person: Carsten Stocklöw


Partner 5 FRAUNHOFER IGD (Fh-IGD, DE)

Contact person: Ben Loke


Partner 6 Noldus Information Technology BV (Noldus, NL)

Contact person: Eleni Christodoulou


Partner 7 CITARD SERVICES LTD (Citard, CY)

Contact person: Sascha Fagel


Partner 8 ZOOBE MESSAGE ENTERTAINMENT GMBH (Zoobe, DE)

Contact person: Donato Cereghetti


Partner 9 MAISON DE RETRAITE DU PETIT-SACONNEX (MRPS, CH)

mailto:[email protected]










Public Miraculous-Life v

Table of Contents

Release History II

Miraculous-Life Consortium IV

Table of Contents V

Table of Figures VII

List of Tables VIII

Abbreviations IX

Executive Summary 1

1 About this Document 2

1.1 Role of the deliverable 2

1.2 Relationship to other Miraculous-Life deliverables 2

1.3 Structure of this document 3

1.4 Updates to this document 3

2 User scenarios and use cases 5

2.1 First user scenario: Luc Petit 5

2.1.1 Use case: Contact list (Co-Net Service) 6

2.1.2 Use case: Message System (Co-Net Service) 9

2.1.3 Use case: Shopping Assistance (Care & Wellness Service) 10

2.1.4 Use case: Medication Reminder (Medication Service – Care & Wellness Service) 12

2.1.5 Use case: Wake-up Calls (Agenda Service – Care & Wellness Service) 15

2.2 Second user scenario: Nicole Framboise 16

2.2.1 Use case: Periodic Advice (Agenda Service – Care & Wellness Service) 16

2.2.2 Use case: Mode of the system: Active vs Passive Mode 17

2.2.3 Use case: Configure the VSP Speech (Dialogue Management) 18

2.2.4 Use case: Fall Detection (Safety Service) 18

2.2.5 Use case: Dangerous Objects Adviser (Safety Service) 20

2.2.6 Use case: Danger Situations Adviser (Safety Service) 20

2.2.7 Use case: Call for Help (Safety Service) 21

2.2.8 Use case: Windows reminder (Household Adviser) 22

2.2.9 Use case: Sleeping reminder (Household Adviser) 23

2.3 Third user scenario: Debora De Jong 23

2.3.1 Use case: Agenda (Agenda Service – Care & Wellness Service) 24

2.3.2 Use case: Events/Group activities (Agenda Service - Care & Wellness Service) 29


Public Miraculous-Life vi

2.3.3 Use case: Appointment Reminder (Agenda Service - Care & Wellness Service) 32

2.3.4 Use case: Object Location Assistance and Reminder (Guidance Service) 34

2.3.5 Use case: Notification Service (Co-Net Service) 36

2.4 Fourth user scenario: July Appetite 37

2.4.1 Use case: Meal preparation 37

2.5 Fifth user scenario: Gunter Robertson 38

2.5.1 Use case: Motivation for Physical Activity (Guidance Service) 38

2.5.2 Use case: Physical Activity Service (Guidance Service) 42

2.5.3 Use case: Social Bonding 43

3 Cost Benefit analysis of the use-cases 54

3.1 Introduction 54

3.2 The Use Cases Development Process 54

3.3 Natural Interaction in the Use-Cases 55

3.4 Cost-Benefit perspective 57

3.5 Final Remarks 75

4 User Interface definition 77

4.1 Human-Computer Interaction analysis 77

4.2 Human-Computer interface specification 78

4.2.1 Introduction 78

4.2.2 Interaction design 79

4.2.3 VSP Main screen 80

4.2.4 VSP Services screen 82

4.2.5 Deactivate VSP screen 84

4.2.6 UI Definition and Guidelines 84

5 Conclusions 87

Appendix A Questionnaire for prioritizing the services of the Miraculous-Life system 88

Appendix B Table with extended details for the feasibility analysis of the use-cases 94

Appendix C Ranking of Miraculous-Life use cases, based on the Relevance Questionnaire - Both end-users organizations (Caregivers = 25, Elderly = 14) 96

Appendix D Ranking of Miraculous-Life use cases, based on the Relevance Questionnaire - MRPS (Caregivers = 11, Elderly = 7) 97

Appendix E Ranking of Miraculous-Life use cases, based on the Relevance Questionnaire - Orbis (Caregivers = 14, Elderly = 7) 98


Public Miraculous-Life vii

Table of Figures

Figure 1: Contact list in the Miraculous Life system. 6

Figure 2: Details of contact in the Miraculous Life system. 7

Figure 3: Main screen of the shopping list service. 10

Figure 4: The agenda of the Miraculous-Life system 24

Figure 5: Evaluation results from elderly and caregivers of Orbis and MRPS for the relevance of the needs addressed by the 23 use cases based on a Likert scale from 1 to 10 (1 = not a priority; 10 = essential priority) 59

Figure 6: Cost-Benefit matrix diagram based on the empirical analysis of the use cases 76

Figure 7: VSP Main screen 80

Figure 8: Show clock 82

Figure 9: VSP Services screen - Example of the Agenda service 83

Figure 10: Agenda screen - Activity 83

Figure 11: Passive / Deactivated VSP screen 84

Figure 12. Mary: the Virtual Support Partner (VSP) of the Miraculous Life system. 88


Public Miraculous-Life viii

List of Tables

Table 1: Active vs Passive Mode 17

Table 2: Technical perspective and Relevance of the use cases, including both qualitative and quantitative their analysis. This information constitutes the Cost-Benefit analysis for each use case. 60


Public Miraculous-Life ix

Abbreviations

Abbrev. Description

AAL Ambient Assisted Living

Co-Net Collaborative Care Network

ICT Information and Communications Technology

KB Knowledge Base

VCT Virtual Care Team

VSP Virtual Support Partner

WP Work Package

DoW Description of Work (Annex I)


Public Miraculous-Life 1

Executive Summary

The main aim of the Miraculous-Life project is to design, develop and evaluate an innovative user-centric technological solution, the Virtual Support Partner (VSP), attending to the elder daily activity and safety needs, while the elder goes about his normal daily life. The VSP will provide implicit support which is based on behaviour and emotional understanding and will interact with the elder exhibiting distinctive emotions, delivered in a human like way simulating in essence the interaction with a real life partner.

To bring user context to life, 5 user scenarios are developed and 23 use cases have been defined giving detailed realistic examples of how users will carry out their daily life activities at home based on the Virtual Support Partner (VSP) model. It considers the interaction of the system with different formal and informal care actors. The results will serve as an aid to understand and clarify user requirements and expectations of the system’s functionality and to provide a basis for the definition of the different system features. In this second version of the document (D1.2b) the scenarios and use cases are refined and examined on their relevance, and feasibility with the technical partners to have a resulting set of use cases that will be implemented in the Miraculous Life system.

Moreover, the analysis of the interaction requirements is presented, that is needed to specify the human-computer interface. As the human-computer interface must be as natural as possible, the VSP should listen to the user, speak back to the user and detect its emotional state. To complete the interaction part of the system, a detailed design of the system’s user interface is developed. In D1.2b, the interaction design has been refined and the results of the first Pre-trial (included in deliverable D6.4a) were taken into account to improve the user interface. Thus, an updated user interface is presented which should improve the interaction between the user and the system and perform the service functions as described in the use cases.



1 About this Document

1.1 Role of the deliverable

The role of this deliverable is twofold. Firstly, it aims to define a detailed specification of the use case scenarios. More specifically, this deliverable describes the scenarios and the use cases that brings user context to life giving detailed realistic examples of how users will carry out their daily life activities at home based on the VSP model and considering interaction with different formal and informal care actors. Secondly, it aims to provide a detailed description and specification of the main user interface. The interaction of the user with the system has been analysed and a detailed description and specification of the main user interface is provided to serve as input for the system development as a whole and the interaction modules of WP2, WP3 and WP4 specifically.

1.2 Relationship to other Miraculous-Life deliverables

The deliverable is related to the following Miraculous-Life deliverables:

Deliverable Relation

D1.1 Specification of user needs analysis and design of VSP model: This document describes the end user needs and requirements analysis, identifies needed services and safety issues and defines the functional specification of the system. Furthermore the psychological definition of the VSP will be provided with the dialogue pattern together with the specification of the operational model of the VSP including the dialogues.

D1.2 will consider the results provided by D1.1 for the definition of the user scenarios, the use cases and the interface definition.

D2.1 Specification of the user emotion recognition component: This document presents the design of the human-computer interaction (i.e., facial emotion recognition, gesture-based emotion recognition, emotion recognition from audio speech).

D1.2 (in association with D1.1) will be provided as input to D2.1 and will be considered during the design of the components responsible for the human-computer interaction.

D3.1 Specification of the User Behaviour Analysis and Environment Analysis component: This document presents the design of the Virtual Support Partner (i.e., design algorithms for environment context analysis and human behaviour understanding, design the overall behaviour and top-level decision making logic of the Virtual Support Partner, design the expressive virtual avatar with human-like facial expression, emotional voice and gestures, etc.)

D1.2 will be provided as input to D3.1 and will be considered during the design of the Virtual Support Partner.

D4.1 Design and specification of ICT-based services and safety services: This document presents the design of a set of Home Daily ICT services considering the categories of: Care & Wellness, Guidance, and Education/Leisure aiding the elder in the execution of determined daily life activities at home as well as the design of safety services that will fulfil safety needs of the elder in carry out his/her daily life activities at home.

D1.2 will be provided as input to D4.1 and will be considered during the design of the Home Daily ICT and Safety Services.

D4.2 Specification of Co-Net: This document presents the design of the Collaborative care Network (Co-Net).



D1.2 will be provided as input to D4.2 and will be considered during the design of the Collaborative care Network (Co-Net) database and services that this component will provide.

D5.1 Specification of overall system architecture and security and privacy infrastructure: This document presents overall design of the Miraculous-Life system architecture.

D1.2 will be provided as input to D5.1 and will be considered during the system architecture design and specification.

D5.2 Development of the KnowledgeBase: This document specifies the knowledge base where the data is stored.

D1.2 will be provided as input to D5.2 and will be considered mainly during the design and specification of this knowledge base data model.

D6.1 Trials specification and design: This document designs the trials and describes the overall evaluation approach that will be used for the trials evaluation.

D1.2 will be provided as input to D6.1 and it will be considered during the design and specification of the trials.

D6.4 Pilot acceptance evaluation results: This document assesses the acceptance of the final system based on experiences and evaluation data gathered by the two pilots.

D1.2 will provide the use cases that are evaluated in D6.4.

D6.5 Overall system evaluation and initial deployment: This document describes the evaluation of the system and identifies initial deployment possibilities.

D1.2 will provide the use cases on which the system is based and, therefore, play an important role as input for D6.5.

1.3 Structure of this document

This document is divided into two parts, addressing the two objectives of this deliverable. Following the current introductory chapter, the rest of this document is structured as follows. Chapter 2 describes in detail the user scenarios and use cases giving detailed realistic examples of how users carry out their daily life activities at home, based on the VSP model and considering interaction with different formal and informal care actors. Chapter 3 provides a clear cost-benefit analysis of the use cases, which reflect both the technical perspective and the relevance of the needs addressed in the use cases. Chapter 4 analyses the human-computer interaction and provides a detailed description and specification of the main user interface. Finally, the main conclusions are provided in Chapter 5.

1.4 Updates to this document

The Miraculous-Life project, had the first project review meeting in month 14 (M14) of the project and the consortium was asked to provide a clear cost-benefit analysis of the use case with the additional double-perspective:

i) Technical Feasibility of the Use Cases;

ii) Relevance of the needs addressed in the use case.

Therefore, in this revised version of D1.2b a new chapter (Chapter Error! Reference ource not found.) is included that analyses the technical feasibility and relevance regarding the needs addressed by each of the use cases selected. The technical partners



will consider this information in order to prioritize the development of the use cases, as well as, to better facilitate resource allocation, thus minimizing any risks that could result in failure to implement the use cases. Additionally, the end-users organizations gain insight about which use-cases provide greater benefit to the end-users



2 User scenarios and use cases

To bring user context to life, five (5) different user scenarios and a total of twenty-three (23) use cases are developed, giving detailed realistic examples of how users carry out their daily life activities at home, based on the VSP model and considering interaction with different formal and informal care actors. The results will serve as an aid for understanding and clarifying user requirements and expectations of the system’s functionality; and to provide a basis for the definition of the different system features.

Each user scenario is followed by several use cases. The use cases are, organized in situations and described in a stepwise manner. Moreover, as the most important objective of this project is to demonstrate the communication aspects between the Avatar and the elderly, in the use cases presented below emphasis is given on the dialogs between the end-user and the VSP, thus assisting the technical partners on the development of the dialogue patterns. Each use case is described aiming to address all possible interactions and situations, identified so far, between the user and the system.

At the current stage of the Miraculous Life project, the technology behind the speech recognition and the avatar interaction are not capable of complete natural interactions. Hence, the proposed use cases are adapted to cope with the limitations introduced by the interaction components. The use cases described in this deliverable will be demonstrated either during the second Pre-Trial (which starts on Month 16 of the project) or during the Trials (which starts on Month 24 of the project). As the most important objective of this project is to demonstrate the communication aspects between the Avatar and the elderly, for the second Pre-trial (Month 16), emphasize will be mainly given on demonstrating those use cases that prove in the best possible way the objectives set for the communication aspects between the Avatar and the Elderly. However, during the Trials, the rest of the use cases focusing also on the different services provided by the Miraculous Life system will be demonstrated.

The planning describing which use cases (or situations of the different use cases) will be demonstrated during the second Pre-Trial and which during Trials, is defined based on feedback received from the WP2, WP3, WP4 and WP5 technical Work Packages. This planning is described in an internal document with title “D1.2b Use Cases - Planning for Pre-Trials and Trials”. However, the final planning, due to its high dependency on the work flow performed under WP2, WP3, WP4 and WP5 Work Packages and the problems that may encountered during the development and the integration of the different system components, will be discussed again, decided and confirmed by all the technical partners on Month 14 of the project.

2.1 First user scenario: Luc Petit

Luc Petit is 85 years old. He lives alone in an apartment including a kitchen open on the dining area, a large bedroom, a hall, a bathroom and a balcony (47 square meters) in the Colladon Residence. He feels lonely since his wife passed away one year ago and he suffers from slightly memory problems. He would like to use the Miraculous Life system (1) to stay in touch with his family and with the caregivers, (2) to obtain shopping assistance, (3) to benefit from the medication reminder service.



2.1.1 Use case: Contact list (Co-Net Service)

Note: All the persons listed in the Co-Net Service will be added before the beginning of the trial. Only administrators will be able to (1) add new persons in this list and (2) assign persons to the Virtual Care Teams of each primary end-user. A number of predefined Virtual Care Team types (e.g., contact list, friends, emergency contacts, etc.) will be recorded in the KB.

ELDERLY ASPECTS

Situation: Luc opens the Contact List Service

State 1: Luc approaches the device and asks: “Open Contact list”. Go to the next state.

State 2: The VSP is on the right, showing the contact list (e.g., see Figure 1). Each line of the list is composed by: (1) the photo of the contact (if available), (2) the first name of the contact, (3) the last name of the contact. The VSP continues the interaction: “Luc, here you are. How can I help you?” Go to state 3, 7, 11 or 15.

Situation: Luc wants to see the contact details of his friend Thom

State 3: Luc answers: “See details of Thom”. Go to the next state.

State 4: The VSP is on the right. The contact list disappears; while the details of the fourth entry (Thom Bellamy) are shown on the screen (e.g., see Figure 2). The VSP continues the interaction: “Luc, here you are.” Go to the next state.

State 5: Luc now sees on the screen all he wanted to know. Luc says: “Thank you”. Go to the next state.

State 6: The VSP says: “Okay” and returns to the main screen of the Contact List Service. Go back to state 3 and check the details of another member in the Contact list or go to state 7, 11 or 15.

Figure 1: Contact list in the Miraculous Life system.



Figure 2: Details of contact in the Miraculous Life system.

Situation: Luc wants to call his friend Thom

State 7: Luc answers: “Call Thom”. Go to the next state.

State 8: The VSP is on the right. The VSP becomes happy that Luc wants to interact socially with the others and continues the interaction: “Ok Luc, I am calling Thom now”. Go to the next state.

Component Action

Emotion The avatar looks happy

State 9: The “calling window” appears on the screen, showing the outgoing call. Thom answers to the call: the conversation starts. Go to the next state.

State 10: Luc (or Thom) ends the conversation by pressing the “End Call” button. The call window is now closed. The VSP says: “Anything else?” and returns to the main screen of the Contact List Service. Go to state 3, 7, 11 or 15.

Situation: Luc wants to send a message to his friend Thom

State 11: Luc answers: “Write a message to Thom”. Go to the next state.

State 12: The VSP is on the right. The VSP continues the interaction: “Ok Luc”. Go to the next state.

State 13: A form appears on the screen, allowing Luc to write a message to Thom via the keyboard of the tablet. Go to the next state.

State 14: Luc sends the message by pressing the “Send Message” button or by saying “Send Message”. In this state, a “Cancel” button is also designed, allowing the user to go back to the main screen of the Contact List service without sending the message. In addition, a voice command “Cancel Message” to the VSP can be used. The VSP becomes happy that Luc is socially interacting with others and continues the interaction: “Anything else?” and returns to the main screen of the Contact List Service. Go to state 3, 7, 11 or 15.

Component Action




Situation: Luc quits the Contact List Service

State 15: Luc answers: “I want to quit the Contact List Service”. The VSP is on the “full screen mode”. STOP.

Situation: Luc receives a call

State 16: By default, the ‘Call Software’ runs in the background of the Miraculous Life system, allowing the elderly to receive calls when the Miraculous Life system is turned on. It’s 14.00 PM and Luc receives a call from his daughter Miriam. The “Calling Window” appears on the screen, showing the incoming call. Go to the next state or to state 19.

State 17: Luc heard the incoming call ring. Luc approaches the device and presses the button “Answer”. In alternative, Luc tells to the VSP: “Answer”. The conversation starts. Go to the next state.

State 18: Luc (or Miriam) ends the conversation by pressing the button “End Call”. The call window is now closed and the VSP is on the “full screen mode”. The VSP concludes the interaction: “I hope you had a nice conversation, Luc. If you ever need anything, don't hesitate to ask me”. STOP.

State 19: Luc does not hear the incoming call ring, or does not want to answer to the call. He continues to read a book (or watch TV). A notification will be triggered by the system, 5 minutes after the missed call. STOP.

Situation: Luc receives a new message

State 20: Luc received a new message from Mike. A notification is triggered by the system with a sound. The VSP: “Luc, you have a new message from Mike. Do you want to see it?” Go to state 21 or 27.

State 21: Luc answers “Show me the message” Go to the next state.

State 22: The VSP says: “Okay, here it is”. The service “Message System” is automatically open and the message written by Mike appears on the screen. Go to state 23, 24, 25 or 27.

State 23: Luc: “Answer back to Mike”. Go to situation: Luc wants to send a message to his friend Thom; Go to state 12.

State 24: Luc: “Call Mike”. Go to situation: Luc wants to call his friend Thom; Go to state 8.

State 25: Luc: “Delete the message”. Go to the next state.

State 26: The VSP: “Ok Luc, the message has been deleted. If you need something, don’t hesitate to ask me”. The VSP is on the “full screen mode”. STOP.

State 27: Luc: “Remind me later”. Go to the next state.

State 28: The VSP: “I will remind you again in 30 minutes”. STOP.



2.1.2 Use case: Message System (Co-Net Service)

Note that a code colour could be used in order to distinguish between (1) the invitations (see use case 2.3.1 Agenda), (2) the notifications (see use case 2.3.5 Notification Service), and (3) the messages sent by others primary end-users (see use case 2.1.1 Contact List). Designers should also consider implementing a filtering tool.

ELDERLY ASPECTS

Situation: The elderly opens the Message System Service

State 1: Luc approaches the device and says: “Open Message System”. Go to the next state.

State 2: The VSP is on the right, showing all the messages in chronological order received by the elderly and the possible user actions. The VSP continues the interaction: “Luc, at the screen you see the messages you received. What do you want to do?” Go to state 3, 9 or 14.

Situation: The elderly wants to see a message

State 3: Luc answers to the VSP: “Show me message 6”. Go to the next state.

State 4: The message written by Mike appears on the screen. The VSP: “Here you are, Luc. Now, what do you want to do?”. Go to state 5, 6 or 7.

State 5: Luc: “Answer back to Mike”. Go to situation: Luc wants to send a message to his friend Thom; state 11 of use case 2.1.1.

State 6: Luc: “Call Mike”. Go to situation: Luc wants to call his friend Thom; state 7 of use case 2.1.1.

State 7: Luc: “Delete the message”. Go to the next state.

State 8: The VSP: “Ok Luc, the message has been deleted. How can I help you now?” and returns to the main screen of the Message System Service. Go to the state 3, 9 or 14.

Situation: The elderly wants to write a new message

State 9: Luc answers to the VSP: “Write a new message”. Go to the next state.

State 10: The VSP is on the right: “Ok Luc. To whom would you wish to send the message?” Go to the next state.

State 11: The contact list appears on the screen. Luc says: “To Donato”. Go to the next state.

State 12: A form appears on the screen, allowing Luc to write a message to Donato via the keyboard of the tablet. Go to the next state.

State 13: Luc sends the message by pressing the button “Send Message” or by saying “Send Message” to the VSP. In this state, a “Cancel” button is also designed, allowing the user to come back to the main screen of the Message System Service without sending the message. The VSP says: “Anything else?” and returns to the main screen of the Message System Service. Go to the state 3, 9 or 14.



Situation: The elderly quits the service

State 14: Luc answers: “I want to quit the Message System”. The Message System is now closed and the VSP is on the “full screen mode”. STOP.

2.1.3 Use case: Shopping Assistance (Care & Wellness Service)

Note that the predefined list of shopping items will be identified with the help of end-users (via a questionnaire).

ELDERLY ASPECTS

Situation: Luc opens the Shopping Assistance Service

State 1: Luc approaches the device and says: “Open Shopping Assistance”. Go to the next state.

State 2: The VSP is on the right, showing the shopping list and the possible user actions (e.g., see Figure 3). The VSP continues the interaction: “Luc, at the screen you see your shopping list. What do you want to do?” Go to state 3, 16, 22, 28 or 35.

Figure 3: Main screen of the shopping list service.

Situation: Luc adds an item in the shopping list

State 3: Luc answers to the VSP: “Add an item”. Go to the next state.

State 4: The VSP continues the interaction: “What item do you want to add in the list?” Go to state 5, 8, 11 or 13.

State 5: Luc answers: “Aubergines”. The system recognizes the word “Aubergine”. Go to the next state.

State 6: The VSP continues the interaction: “How many aubergines do you need?” Go to the next state.

State 7: Luc answers: “Two”. Go to the state 15.



State 8: Luc answers: “Sugar”. The system recognizes the word “Sugar”. Go to the next state.

State 9: The VSP continues the interaction: “How much sugar do you need?” Go to the next state.

State 10: Luc answers: “One kilogram”. Go to the state 15.

State 11: Luc answers: “Milk”. The system recognizes the word “Milk”. The VSP continues the interaction: “How many litres do you need?” Go to the next state.

State 12: Luc answers: “four”. Go to the state 15.

State 13: Luc answers: “Fois gras”. The system does not recognize the word “Fois gras”. The VSP continues the interaction: “I am sorry, but I did not understand you. Could you please write this item manually?”. Go to the next state.

Component Action

Emotion The avatar looks sad

State 14: A text-input form appears on the screen, allowing Luc to add manually the item “Fois gras” on the shopping list (via the keyboard of the tablet). The user writes “Fois gras” and confirms the new entry by saying “Add the item” to the VSP. Go to the next state.

State 15: The VSP continues the interaction: “I added the item to the shopping list. What do you want to do now?” Go to state 3, 16, 22, 28 or 35.

Situation: Luc removes an item from the shopping list

State 16: Luc answers to the VSP: “Remove item two”. Go to the next state.

State 17: The VSP: “Luc, are you sure you want to delete the item ‘Aubergine’?”. Go to state 18 or 20.

State 18: Luc: “Yes”. Go to the next state.

State 19: The VSP: “Ok, the item ‘Aubergine’ has been removed from the list. How may I help you?” Go to state 3, 16, 22, 28 or 35.

State 20: Luc: “No”. Go to the next state.

State 21: The VSP: “The item ‘Aubergine’ hasn’t been removed from the list. How may I help you?” Go to state 3, 16, 22, 28 or 35.

Situation: Luc removes the whole shopping list

State 22: Luc answers to the VSP: “Remove the whole shopping list”. Go to the next state.

State 23: The VSP continues the interaction: “Luc, are you sure that you want to remove all the items contained in the shopping list?” Go to state 24 or 26.

State 24: Luc answers: “Yes”. Go to the next state.

State 25: The shopping list is now empty. The VSP continues the interaction: “I removed all the items from the shopping list, Luc. How can I help you?”. Go to state 3, 16, 22, 28 or 35.



State 26: Luc answers: “No”. Go to the next state.

State 27: The VSP continues the interaction: “Ok, I did not remove your shopping list. How can I help you?”. Go to state 3, 16, 22, 28 or 35.

Situation: Luc sends the shopping list by message

State 28: Luc answers to the VSP: “Send the shopping list”. Go to the next state.

State 29: The VSP knows that Luc always prefers to send the shopping list to Kate Nielsen and asks “Do you want to send it to Kate Nielsen?” Go to the next state or state 31.

Component Action

Memory Person often receiving the shopping list from the Luc

State 30: Luc answers: “Yes”. Go to state 34.

State 31: Luc answers: “No”. Go to the next state.

State 32: The VSP is on the right, showing the contact list. Each line of the list is composed by (1) the photo of the contact, (2) the first name of the contact, (3) the last name of the contact. The VSP continues the interaction: “Who should I send the shopping list to?” Go to the next state.

State 33: Luc answers: “Two, VSP”. Go to the next state.

State 34: The VSP says: “Luc, I just sent the list to John Bowes (or to Kate Nielsen if state 30 occurs). Do you want that I remove the whole shopping list?” Go to the next state or to the state 24 or 26.

Situation: Luc quits the shopping list

State 35: Luc answers: “I want to quit the shopping list”. The Shopping Assistance Service is now closed and the VSP is on the “full screen mode”. STOP.

2.1.4 Use case: Medication Reminder (Medication Service – Care & Wellness Service)

CAREGIVER ASPECTS

Situation: A caregiver adds, modifies or removes a new medicament in the agenda

State 1: François Fournier (MRPS caregiver) accesses to the caregiver interface and selects the user Luc. Go to the next state.

State 2: François adds a new medication in the agenda*. Each entry is defined by fields such as: (1) Medication (what), (2) Timing (when), (3) Amount (how much), (4) Frequency of the reminder (daily, every other day, weekly, etc.). STOP.

* For future development of the Miraculous Life system, note that the site http://ch.oddb.org/en provides an

open drug database for Switzerland.

http://ch.oddb.org/en



ELDERLY ASPECTS

Situation: First medication reminder at 10:00 hrs

State 1: The VSP says: “Luc, you have a medication reminder. What would you like to do with it?” Go to the state 2, 4, 6, 8 or 11.

State 2: Luc answers “Show me the reminder”. Go to the next state.

State 3: The VSP answers: “It’s 10:00 hrs, Luc. You should take the medicine Actos”. At the same time a text bubble on the screen is showing the following text:

Medication: Actos.

Time: 10:00 hrs.

Dose: 2 pills.

Go to state 4, 6, 8 or 11.

State 4: Luc: “Remind me later” or “I will take it”. Go to the next state.

State 5: The VSP: “I will remind you again in 10 minutes to check if you took it”. STOP.

State 6: Luc: “I’ve (already) taken it”. Go to the next state

State 7: The VSP is happy that Luc took the medication and says: “OK, I will then delete this reminder”. STOP.

Component Action


State 8: Luc: “I don’t want to take it”. Go to the next state.

State 9: The VSP: “Luc, are you sure you don’t want to take your medication? A notification will be sent to your caregivers”. Go to state 6, 10 or 11.

Component Action

Emotion The avatar looks worried

State 10: Luc: “Yes, I don’t want to take it”. A notification is sent to the caregivers. STOP.

State 11: Luc did not answer to the VSP. A new reminder will be triggered 10 minutes later. STOP.

Situation: Second medication reminder, later at 10:10 hrs

State 1: The VSP says: “Luc, you have a medication reminder. What would you like to do with it?” Go to state 2, 4, 6, 10 or 13.

State 2: Luc answers “Show me the reminder” Go to the next state.



State 3: The VSP answers: “At 10:00 hrs, you had to take the medicine Actos. Did you take it?” At the same time a text bubble in the screen is shown the following text:

Medication: Actos.

Time: 10:00 hrs.

Dose: 2 pills.

Go to state 4, 6, 8, 10 or 13.

State 4: Luc: “Remind me later” or “I will take it”. Go to the next state.

State 5: The VSP: “I will remind you again in 10 minutes to check if you took it”. STOP.

State 6: Luc: “Yes” or “I’ve (already) taken it”. Go to the next state


Component Action



State 9: The VSP: “It is time to take this medicine. It is not wise to postpone it more”. Go to state 4, 6, 10 or 13.

Component Action

Emotion The avatar expresses directive behaviour


State 11: The VSP: “Luc, are you sure you don’t want to take your medication? A notification will be sent to your caregivers”. Go to state 6, 12 or 13.

Component Action



State 13: Luc did not answer to the VSP. A new reminder will be triggered 10 minutes later. STOP.

Situation: Third medication reminder, later at 10:20 hrs

State 1: The VSP says: “Luc, you have a medication reminder. This is the last reminder. What would you like to do with it?” Go to state 2, 4, 8 or 11.

State 2: Luc answers “Show me the reminder” Go to the next state.



State 3: The VSP answers: “At 10:00 hrs, you had to take the medicine Actos. Did you take it?” At the same time a text bubble on the screen is shown the following text:

Medication: Actos.

Time: 10:00 hrs.

Dose: 2 pills.

Go to state 4, 6, 8 or 11.

State 4: Luc: “Yes” or “I’ve (already) taken it”. Go to the next state.


Component Action



State 7: The VSP: “It is time to take this medicine. It is not wise to postpone it more”. Go to state 4, 8 or 11.

Component Action



State 9: The VSP: “Luc, are you sure you don’t want to take your medication? I will inform your caregivers if you do not”. Go to state 4, 10 or 11.

Component Action



State 11: Luc did not answer to the VSP. The interaction is completed. The reminder will be triggered on the next interaction with the user (i.e., when the user will approach to the system). STOP.

2.1.5 Use case: Wake-up Calls (Agenda Service – Care & Wellness Service)

ELDERLY ASPECTS

Situation: The elderly sets the alarm clock

State 1: The VSP is in the “full screen mode”. Luc approaches the device. The VSP begins the interaction: “Luc, how’s your day?” Go to the next state.

State 2: Luc answers: “I am fine. VSP, I want to set the alarm clock”. Go to the next state.



State 3: The VSP continues the interaction: “This is a good idea. At what time do you want to get up?” Go to the next state.

State 4: Luc answers: “At 7.00 am”. Go to the next state.

State 5: The VSP concludes the interaction: “The alarm has been set and will sound at the time selected. If you ever need anything, don't hesitate to ask me!”. STOP.

Situation: The elderly wakes-up with the avatar

State 1: It’s 7.00 am in the morning. The alarm clock is triggered by the system.

State 2: The elderly wakes up and heads to the device in order to turn off the alarm. The system knows that the elderly just woke up and the avatar starts the first interaction of the day.

2.2 Second user scenario: Nicole Framboise

Nicole Framboise is 68 years old. She lives alone in a big apartment in the Colladon Residence. Nicole leads a semi-independent life. Nicole suffers from depression. She also does not eat and drink sufficiently. Finally she’s anxious about daily living in her apartment. She would like to have a system that can meet her safety needs.

2.2.1 Use case: Periodic Advice (Agenda Service – Care & Wellness Service)

CAREGIVER ASPECTS

Situation: A caregiver adds, modifies or removes a period advice for a particular elderly

State 1: François Fournier (MRPS caregiver) accesses to the caregiver interface and selects the user Nicole. Go to the next state.

State 2: François adds a new periodic advice for Nicole Framboise as an agenda item. A periodic advice is defined by:

1. The name of the advice (in this case: ‘drink reminder’). Note that the name of the advice is shown in the back-end (caregivers interface), but not in the front-end (elderly) interface.

2. The text of the advice that will be read by the VSP (see state 3, elderly aspects)

3. The timing (when)

4. The frequency of the advice (daily, every other day, weekly)

Notes: A list of standard advices should already be available on the back-end (for example: drink reminder, lunch reminder, dinner reminder, etc.). In addition, caregivers should be able to see the video of the VSP associated to the advice. This will ensure that the text of the advice is interpreted correctly by the text-to-speech module.

ELDERLY ASPECTS



Situation: The elderly receives a periodic advice (in this case a ‘drink reminder’)

State 1: It’s 15:00 hrs. A periodic advice it’s triggered by the system. The VSP says: “Nicole, you have a new advice from François Fournier. What would you like to do with it?” Go to state 2, 4, or 6.

State 2: Nicole answers “Show me the advice” Go to the next state.

State 3: The VSP reads the text previously written by François Fournier: “Dear Nicole, the weather has been so hot lately. It’s important to stay adequately hydrated. Don’t forget to drink a glass of water from time to time, even if you don't perceive it necessarily”. At the same time a text bubble in the screen is shown the following text:

From: François Fournier

Advice: “Dear Nicole, the weather has been so hot lately. It’s important to stay adequately hydrated. Don’t forget to drink a glass of water from time to time, even if you don't perceive it necessarily”

Time: 15:00 hrs.

Go to state 4 or 6.

State 4: Nicole: “Delete the advice” Go to the next state.

State 5: The VSP: “This advice is deleted now”. STOP.

State 6: Nicole did not answer to the VSP. The interaction is completed. The reminder will be triggered on the next interaction with the user (i.e., when the user will approach to the system). If no interaction with the user occurs in the waiting configured period, the advice will be automatically deleted. STOP.

2.2.2 Use case: Mode of the system: Active vs Passive Mode

Table 1: Active vs Passive Mode

Active Passive

Appointment Reminder ON OFF

Danger Situations Adviser ON ON

Dangerous Objects Adviser ON ON

Fall Detection ON ON

Medication Reminder ON ON

Motivation for Physical Activity ON OFF

Notification Service ON OFF

Periodic Advice ON OFF

Wake-up Calls ON ON

Note that in the passive mode, speech recognition only responses to the sentence ‘’Switch to active mode’’ or the system can be activated by touch via the button ‘’Activate’’. In the passive mode, the user cannot interact with the system in other ways.

ELDERLY ASPECTS



Situation: Nicole switches the system to the Passive mode

State 1: Nicole: “Switch to the Passive Mode”. Go to the next step.

State 2: The VSP: “Ok Nicole. The appointment reminders, notifications and periodic advices are now disabled.” Reminders and notifications will be postponed until the user switches the system back to the Active mode. STOP.

Situation: Nicole switches the system to the Active mode

State 1: Nicole: “Switch to the Active Mode”. Go to the next step.

State 2: The VSP: “Ok Nicole. The appointment reminders, notifications and periodic advices are now enabled.” STOP.

2.2.3 Use case: Configure the VSP Speech (Dialogue Management)

Situation: Nicole does not understand the VSP because the VSP talks too fast/slow/soft/loud

State 1: Nicole says “I did not understand you” or clicks on the “Configure VSP Speech” button to configure the avatar interaction parameters. Go to the next state.

State 2: The VSP asks: “Do you want me to speak louder, softer, faster or slower?” Go to state 3, 4, 5, and 6.

State 3: Nicole answers: “Slower”. Go to state 7

State 4: Nicole answers: “Faster”. Go to state 8

State 5: Nicole answers: “Louder.” Go to state 9

State 6: Nicole answers: “Softer”. Go to state 10

State 7: The VSP adapts its speed according to Nicole’s answer and asks: “Ok. I will speak slower from now on. Did you understand me?” Go to state 11 or 12.

State 8: The VSP adapts its speed according to Nicole’s answer and asks: “Ok. I will speak faster from now on. Did you understand me?” Go to state 11 or 12.

State 9: The VSP adapts its volume according to Nicole’s answer and asks: “Ok. I will speak louder from now on. Did you understand me?” Go to state 11 or 12.

State 10: The VSP adapts its volume according to Nicole’s answer and asks: “Ok. I will speak softer from now on. Did you understand me?” Go to state 11 or 12.

State 11: Nicole answers: “Yes”. STOP.

State 12: Nicole answers: “No”. Go to state 2.

2.2.4 Use case: Fall Detection (Safety Service)

Remark: This use-case is active whether the system is in “active mode” or “passive mode”. The system will always collect and process data from sensors for safety concerns.

CAREGIVER AND ELDERLY ASPECTS



State 1: In the middle of the night, Nicole wakes up to go to the toilet. She stumbles and falls on the floor. Go to the next state.

State 2: The Miraculous Life system detects the figure of the elderly on the floor. The VSP becomes worried and asks Nicole: “Nicole, it seems that you fell down. Shall I ask help from your caregivers?”. Go to state 3, 5 or 6.

Component Action


State 3: Nicole stands up. She feels good and answers: “No, I am ok now”. Go to the next state.

State 4: The VSP appears relieved and says: “Ok, Nicole. Let me know if you will need support from your caregivers”. STOP.

Component Action

Emotion The avatar looks relieved

State 5: Nicole answers: “Yes” or “Call for help”. Go to state 7.

State 6: Nicole is lying on the ground and she does not answer to the VSP. Go to state 7.

State 7: The VSP continues the interaction: “Do not worry, Nicole. I warned the caregivers. They will be here soon”. The Miraculous Life system automatically makes a call to predefined contacts (in a predefined order of priority). Go to the next state.

State 8: Sylvie Rousseau (MRPS caregiver) answers the call of the Miraculous Life system. An automatic audio message is trigged: “This is a pre-recorded message provided by the Miraculous Life system: Nicole is in her apartment and she needs help immediately. Can you intervene in the next minutes in order to provide support to her?” Go to the next state or to state 12.

State 9: The caregiver answers: “Yes”. Go to the next state.

State 10: A new pre-recorded message is trigged by the system: “You have confirmed that you can intervene to assist Nicole Framboise. Thank you. Nicole is waiting for you”. The call ends automatically. Sylvie goes to Nicole's apartment to assist the elderly. Go to the next state.

State 11: Now that Sylvie confirmed that she will come, the VSP says to Nicole: “Sylvie Rousseau is on her way to help you. Everything will be ok”. STOP

Component Action


State 12: The caregiver answers: “No”. Go to the next state.

State 13: A new pre-recorded message is trigged by the system: “You cannot intervene to assist Nicole Framboise. Don’t worry, I will call someone else”. The call ends automatically. The Miraculous Life system makes a new call to the following predefined contact. Go to state 8.



2.2.5 Use case: Dangerous Objects Adviser (Safety Service)

ELDERLY ASPECTS

State 1: Nicole is in the sitting room. Nicole is preparing to get out the house for a walk. Go to the next state.

State 2: The system recognizes an unknown object at the ground in her walking direction and interprets this object as potential danger for the elderly. The VSP appears worried and starts a new interaction: “Nicole, I see an object at the ground in your walking direction. It could be dangerous! I think you should move this object in a safer place.” Go to the next state or to state 5 or 7.

Component Action


State 3: Nicole sees the umbrella on the ground. She picks it up and places it on the umbrella stand, by answering to the VSP: “Thank you, I moved it”. Go to the next state.

State 4: The Miraculous Life system detects that the potential risk has been raised. The VSP continues the interaction: “You’re welcome Nicole”. STOP.

Component Action


State 5: Nicole did put the umbrella there on purpose. So, she will not forget it and answers to the VSP: “It is ok. I just put it there myself”. Go to the next state.

State 6: The Miraculous Life system accepts Nicole’s argument and says “OK Nicole, just be aware not to stumble over it”. STOP.

Component Action

Emotion The avatar looks neutral

State 7: Nicole does not react to the advice provided by the VSP and she goes to the toilet. Go to the next state.

State 8: Nicole comes back to the sitting room. A new advice will be triggered by the system 30 minutes later. Note: the Miraculous Life system will continuously trigger advices until the elderly moves the object. STOP.

2.2.6 Use case: Danger Situations Adviser (Safety Service)

ELDERLY ASPECTS

State 1: Nicole is cooking lunch by consulting a recipe contained in the Meal Preparation Service. Go to the next state.

State 2: The recipe of the service “Meal Preparation” was open for more than 1 hour and the elderly didn’t interact with it for more than 1 hour. The system evaluates this situation



as being potentially dangerous for the elderly. The VSP starts a new interaction: “Nicole, I could be wrong, but you were preparing a meal. Did you turn off the stove?” Go to state 3, 5, 7 or 9.

Component Action


State 3: Nicole: “Yes I turned off the stove. Thank you, Mary”. Go to the next state.

State 4: The Miraculous Life system detects that the potential risk has been removed and feels relieved. The VSP continues the interaction: “You’re welcome Nicole.” STOP.

Component Action


State 5: Nicole: “I wasn’t cooking”. Go to the next state.

State 6: The VSP: “Okay, I'm sorry I bothered you Nicole.” STOP.

Component Action


State 7: Nicole: “I am still cooking”. Go to the next state.

State 8: The VSP: “Okay, Nicole. Enjoy your meal.”

Component Action


State 9: Nicole does not react to the advice. Go to the next state.

State 10: Ten minutes later, a new reminder is triggered by the system. Note: the Miraculous Life system will continuously trigger advices until the elderly answers to the advice. STOP.

2.2.7 Use case: Call for Help (Safety Service)

ELDELRY ASPECTS

State 1: Nicole does not feel good. She has strong headache and stomach-ache. She asks for help by saying: “Mary, help me”, “Mary, this is an emergency” or “Mary, I am not feeling well”.

State 2: The VSP became worried and asks for a confirmation: “Hey Nicole. Apparently you need help. Shall I call your caregivers?” Go to state 3 or 10

Component Action




State 3: Nicole answers: “Yes”. Go to the next state.

State 4: The VSP continues the interaction: “Do not worry, Nicole. I warned the caregivers. They will be here soon”. The Miraculous Life system automatically makes a call to predefined contacts (in a predefined order of priority). Go to the next state.

State 5: Sylvie Rousseau (MRPS caregiver) answers the call of the Miraculous Life system. An automatic audio message is trigged: “This is a pre-recorded message provided by the Miraculous Life system: Nicole is in her apartment and she needs help immediately. Can you intervene in the next minutes in order to provide support to her?” Go to the next state or to state 8.

State 6: The caregiver answers: “Yes”. Go to the next state.

State 7: A new pre-recorded message is trigged by the system: “You have confirmed that you can intervene to assist Nicole Framboise. Thank you. Nicole is waiting for you”. The call ends automatically. Sylvie goes to Nicole's apartment to assist the elderly. STOP.

Component Action


State 8: The caregiver answers: “No”. Go to the next state.

State 9: A new pre-recorded message is trigged by the system: “You cannot intervene to assist Nicole Framboise. Don’t worry, I will call someone else”. The call ends automatically. The Miraculous Life system makes a new call to the following predefined contact. Go to the state 5.

State 10: Nicole answers: “No, that’s just okay for now”. Go to the next state.

State 11: The VSP appears relieved: “Ok, Nicole. Let me know if you will need support from your caregivers. STOP.

Component Action


Note Optionally: A sensor will be installed in the bathroom of the seniors. When this sensor detects that the user is in the bathroom and not moving for more than 45 minutes, the system will automatically trigger the use case “call for help”.

2.2.8 Use case: Windows reminder (Household Adviser)

State 1: It’s 10 am. A sensor is placed on each window. The system detects that the windows this morning have not been opened by the user. On the next interaction with the system (i.e., when the user will approach the system), the VSP will remind the user to open the windows: “Welcome back, Nicole. Today, you have not yet opened the windows. It’s important to get proper air circulation in the apartment. What about opening the windows?” Go to the next state.



Component Action

Emotion The avatar looks compassionate

State 2: Nicole answers: “Thank you for the reminder”. Go to state 3 or 8.

State 3: Nicole opens the windows. The system detects that the windows is now opened. Go to state 4 or 5.

State 4: 30 minutes later, Nicole closes the windows. The interaction is completed. STOP.

State 5: 1 hour later, the system detects that the windows are still opened. A new reminder is triggered by the system: “Nicole, don’t forget to close the windows!” Go to state 6 or 7.

Component Action


State 6: Nicole answers: “Thank you for the reminder”. The interaction is completed. STOP.

State 7: Nicole does not react on the windows reminder. The reminder will be triggered on the next interaction with the user (i.e., when the user will approach the system).

State 8: Nicole does not open the windows. The interaction is completed. STOP.

2.2.9 Use case: Sleeping reminder (Household Adviser)

State 1: It’s 2 am. The user is sleeping on the couch (living room) with the TV on. A sensor is placed on the bedroom and the user is not detected on the bed. The avatar tries to wake up the elderly and motivates him to go to the bed: “Nicole, wake up!” After 10 seconds, the VSP continues the interaction: “You know Nicole, you should sleep on your bed, not here. What about moving to your bed?”. Go to state 2 or 3.

Component Action


State 2: Nicole answers: “Thank you for the reminder”. STOP.

State 3: Nicole continues to sleep and does not react on the advice. A new reminder will be triggered 45 minutes later.

2.3 Third user scenario: Debora De Jong

Debora De Jong is 88 years old. She lives alone in the elderly care centre of Orbis Hoogstaete in an apartment including a living room with a small kitchenette, a bedroom and bathroom (46 square meters). She suffers from Chronic Obstructive Pulmonary Disease (COPD) and needs oxygen. For long distances, she uses a mobility scooter.



She feels lonely, especially in the evening, since her husband passed away two years ago. She needs physical and emotional support. She needs help with showering and getting (un)dressed considering her oxygen supply and she needs reminders for taking her medication. Emotionally, she needs to be supported by a person in the morning and evening, to start and close the day.

She participates in several activities organized in the care centre and she loves to play cards with her neighbour Susan. She is always looking for social contacts and her daughter visits her twice a week. With all these various activities Debora’s calendar is quite full and she sometimes misses some appointments with friends or organized activities. Therefore she would like the system to remind her of her appointments and show her the daily schedule of her activities.

2.3.1 Use case: Agenda (Agenda Service – Care & Wellness Service)

ELDERLY ASPECTS

State 1: Debora asks: “Open Agenda, please”. Go to the next state.

State 2: The VSP is on the right, showing the agenda (i.e., a list of the appointments and activities of the elderly planned for today). The VSP continues the interaction: “Welcome to the agenda. On the screen you can see you appointments and activities planned for today. How can I help you?” Go to state 3, 9, 32, 41, 43 or 45

Figure 4: The agenda of the Miraculous-Life system

Situation: See the details of an Agenda Item

State 3: Debora asks: “Show me details of appointment 1”. Go to the next state.

State 4: The VSP says: “Okay, here it is” while the list at the left side is now changed into a detailed overview of the activity asked by the user. If the user requested the details of an agenda item, in the detailed overview she/he sees (1) the date of the activity, (2) the time, (3) the location, (4) the price, (5) the accessories needed and (6) the participants (e.g., the creator of the agenda item and the persons who accepted his/her invitation)

Activity: Card Games



Date: 13.11.2014

Time: 11:00-12:00 hrs.

Location: Restaurant

Price: 0

Accessories: Cards

Participants: Cindy, Rachelle, Inge

If the user requested the details of an agenda item created by him or her, in the detailed overview she/he sees (1) the date, (2) the time and (3) the location of the activity.

Activity: Doctor

Date: 13.11.2014

Time: 16:00-17:00 hrs.

Location: Sittard

Finally if the user requested the details of an event created by another primary user of the Miraculous Life system, in the detailed overview he/she sees (1) the date, (2) the time of the activity, (3) the location and (4) the participants (e.g., the creator of the agenda item and the persons who accepted his/her invitation).

Activity: Poker

Date: 13.11.2014

Time: 19:00-20:00 hrs.


Participants: Donato, Debora, Susan

Go to the next state or state 7

State 5: Debora asks: “Read me the details”. Go to the next state.

State 6: The VSP reads the activity details shown on the screen aloud. Go to the next state.

State 7: Debora now sees where she has to be, at what time and knows all she wanted to know. Debora says: “Thank you”. Go to the next state.

State 8: The VSP says: “Okay” and returns to the main screen of the Agenda. Go to state 3, 9, 32, 41, 43 or 45.

Situation: Add Agenda Items and Inviting persons

State 9: Debora asks: “I’d like to add a new event”. Go to the next state.

State 10: The VSP continues the interaction: “Ok, Debora”. The Agenda disappears; while a text-input form appears on the screen, allowing Debora to add manually the new event (via the keyboard of the tablet) by specifying: (1) the nature of the activity (what), (2) the date, (3) the time and (4) the location of the activity (where). After that, the user can invite persons to participate. Note that the system proposes standard activities (like going for a walk, apero in the restaurant, birthday party, etc.). Debora confirms the new appointment by saying “Validate it” to the VSP. Go to the next state.



State 11: The VSP says: “I added the item in your agenda” while a detailed overview of the activity is shown on the left side. Go to the next state or state 16.

Activity: Shopping

Date: 13.11.2014

Time: 16:00-17:00 hrs.

Location: Center

State 12 (Conditional): If Debora often invites “Emilia” for the activity “Shopping”, the VSP will suggest to invite her to this activity when created. It says “You often invite Emilia to activity “Shopping”. Would you like to invite Emilia again?” Go to the next state or state 14.

Component Action

Memory Person often invited to activity “Shopping”

State 13: Debora: “Yes”. Go to state 15.

State 14: Debora: “No”. Go to state 16.

State 15: The VSP becomes happy that Nicole is socially active, sends an invitation to Emilia and says “An invitation for the activity “Shopping” has been sent to Emilia.” Go to the next state.

Component Action


State 16: VSP asks Debora if she would like to invite someone (else). “Would you like to invite someone to this activity?” Go to the next state or state 18.

State 17: Debora: “Yes”. Go to state 19.

State 18: Debora: “No”. The system returns back to the Agenda screen. Go to state 3, 9, 32, 41, 43 or 45

State 19: VSP asks then “Who would you like to invite then?” Go to the next state.

State 20: Debora: “Susan”. Go to the next state.

State 21: VSP becomes happy that Nicole is socially active, sends an invitation to Susan and says “An invitation for the activity “Shopping” has been sent to Susan.” and returns back to the Agenda screen. Go to state 3, 9, 35, 41, 43 or 45

Component Action


Situation: Susan received an invitation

State 22: Immediately after Debora created the social event, a notification is sent to Susan. The VSP: “Susan, Debora invited you for a social activity. Do you want to see it?” Go to state 23, 25 or 34.



State 23: Susan answers “Show me the activity” Go to the next state.

State 24: The VSP says: “Okay, here it is”. A detailed overview of the agenda item created by Debora is shown on the screen. In this detailed overview the user sees the date, the time of the activity, the location and the participants (e.g., the creator of the agenda item and the persons who accepted his/her invitation)

Activity: Shopping

Date: 13.11.2014

Time: 16:00-17:00 hrs.

Location: Center

Participants: Debora

Go to state 25, 27, 29 or 34.

State 25: Susan: “Remind me later”. Go to the next state.

State 26: The VSP: “I will remind you again in 30 minutes”. STOP.

State 27: Susan: “I will participate”. Go to the next state.

State 28: The VSP: “Susan, this social activity was added in your agenda. I will also send a message to Debora to confirm your presence”. The social activity is added in her agenda. A new notification is sent to Debora:

From: Susan

Notification: “Susan confirmed her presence to the activity “Shopping”

STOP.

Component Action


State 29: Susan: “I will not participate”. Go to the next state.

State 30a (Conditional): The VSP realizes that Susan already is socially active and is ok with it that she does not want to participate. STOP

Component Action

Memory Frequency of Susan being social active over one week


State 30b (Conditional): The VSP becomes sad that Susan is not socially active and says: “I am sad to hear that you do not want to participate. You have not been very socially active lately. I hope you change your mind”. Go to the next state.

Component Action

Memory Frequency of Susan being social active over one week




State 31: Susan: “OK, I will participate”. Go to state 28.

State 32: Susan: “I do not want to participate”. Go to the next state.

State 33: The VSP says: “Okay, as you like!” and changes to a neutral expression. STOP.

Component Action


State 34: Susan did not answer to the VSP. A new notification will be triggered 30 minutes later. STOP.

Situation: Remove an Agenda Item

State 35: Debora answers to the VSP: “Remove Appointment two”. Go to the next state.

State 36: The VSP: “Debora, are you sure you want to delete the appointment two?”. Go to state 37 or 39

State 37: Debora: “Yes”. Go to the next state.

State 38: The VSP: “Ok, the appointment two has been removed from the list. How may I help you?” STOP.

State 39: Debora: “No”. Go to the next state.

State 40: The VSP: “The appointment two hasn’t been removed from the list. How may I help you?” STOP.

Use cases: 1. The user ABC subscribes to a group activity (in the Group Activity service). After

doing that, the user ABC deletes this agenda item from his/her agenda. The user ABC still can find this activity on the service “Group Activity” and, potentially, re-subscribe.

2. The user ABC creates an agenda item and invites the users GHI and XYZ. The users GHI and XYZ accept the invitation made by the user ABC. After doing that, the user ABC or XYZ (creator or participant) deletes this agenda item from his/her agenda. The agenda item is removed exclusively on the ABC or XYZ agenda.

Situation: Debora changes the day

State 41: Debora: “Show me activities of previous day”. Go to the next state.

State 42: The VSP is showing the list of the activities for the asked day at the left side of the screen and says: “At the screen you see a list with the activities of [yesterday].” Go to state 3, 9, 35, 43 or 45.

State 43: Debora: “Show me activities of the next day”. Go to the next state.



State 44: The VSP is showing the list of the activities for the asked day at the left side of the screen and says: “At the screen you see a list with the activities of [tomorrow]”. Go to state 3, 9, 35, 41 or 45.

Situation: Debora quits the Agenda service

State 45: Debora answers: “I want to quit the agenda”. The Agenda Service is now closed and the VSP is on the “full screen mode”. STOP.

2.3.2 Use case: Events/Group activities (Agenda Service - Care & Wellness Service)

Note that group activities could be activities created by caregivers or by the animation team of the elderly care institution.

CAREGIVER ASPECTS

The caregiver needs to add the activities organised by the care organisation into the system via his work computer which is connected to a remote server which sends the information to all local servers of the elderly.

The caregiver needs to add, edit and delete activities in the system.

In adding activities we need the following fields:

Activity

Price of activity

Location

Date

Time

Duration

Frequency

If an activity is added and it overlaps with another earlier added activity the caregiver needs to receive a pop up which shows with which activity it overlaps and if the caregiver still wants to add this new activity.

ELDERLY ASPECTS

Situation: Debora wants to know which events/group activities are proposed today

[Daily schedule]:

State 1: Debora asks: “Open Agenda, please”. Go to the next state.

State 2: The VSP is on the right, showing the agenda (i.e., a list of the appointments and activities of the elderly planned for today). The VSP continues the interaction: “Welcome to the agenda. On the screen you can see your appointments and activities planned for today. How can I help you?” Go to the next state.

State 3: Debora asks: “Show me the group activities.” Go to the next state.



State 4: The VSP is showing the list now at the left side of the screen and says: “At the screen you see a list with the activities of today.” Go to the next state.

State 5a (Conditional): The VSP realises that today’s activity list contains an activity from the category “social activities” and knows that Debora did not participate in any “social activity” for more than a week and says “You haven’t participated to any social activity for more than a week. I see that today the social activity “drinking coffee” is organised. Would you like to register to this activity?” Go to state 6 or 7.

Component Action

Memory Get categories of activities where user did not participate for more than a week

State 5b (Conditional): The VSP realises that one of Debora’s favourite activities is in the list and says “I see that your favourite activity “drinking coffee” is today. Would you like to register to this activity?” Go to state 6 or 7.

Component Action

Memory Get top 5 activities where the user participated most

State 5c (Conditional): The VSP knows that Debora often participates in activities of the category “social activities” and suggests to Debora to participate to an activity of the same category by asking her “I know that you like social activities, what do you think of the activity “drinking coffee” of today? Would you like to register to this activity?” Go to state 6 or 7.

Component Action

Memory Get the category of events where the user participated most

State 6: Debora answers: “Yes”. Go to state 8

State 7: Debora answers: “No” and the system returns to the main screen of the activity service. Go to state 3, 9, 11, 13, 15, 21, 23 or 25.

State 8: The VSP becomes happy that Debora participates to activities and answers “You are registered for drinking coffee at 10:00 hrs. Anything else?” and returns to the main screen of the activity service. Go to state 3, 9, 11, 13, 15, 21, 23 or 25.

Component Action


State 9: Debora asks: “Read me the group activites”. Go to the next state.

State 10: The VSP says: ”Activity one is drink coffee and is at 10:00 hrs., activity two is play card games and is at 11:00 hrs., and activity three is do yoga and is at 15:00 hrs. Let me know if you would like to register to an activity or to see more details about it”. Go to state 9, 11, 13, 15, 21, 23 or 25

State 11: Debora: “Register for activity 2”. Go to the next state.



State 12: The VSP becomes happy that Debora participates to activities and answers “You are registered for card games at 11:00 hrs. Anything else?”. Go to state 9, 13, 15, 21, 23 or 25

Component Action


State 13: Debora: “Register for activity 3”. Go to the next state.

State 14: The VSP becomes happy that Debora participates to activities and answers “You are registered for yoga at 15:00 hrs. Anything else?”. Go to state 9, 11, 13, 15, 21, 23 or 25

Component Action


Situation: Debora wants to see the details of one particular activity

State 15: Debora says: “Show me the details of activity 1”. Go to the next state.

State 16: The VSP says: “Okay, here it is” while the list at the left side is now changed into a detailed overview of the activity “drink coffee”. In this detailed overview you see the time of the activity, the location, the price, the accessories needed and the participants (e.g., the end-users who subscribes to this group activity)

Activity: drink coffee

Time: 10:00-11:00 hrs.

Location: Coffee corner department 1

Price: 0

Accessories: No


Go to the next state or state 19

State 17: Debora asks: “Read me the details”. Go to the next state.

State 18: The VSP reads the activity details shown on the screen aloud. Go to the next state.

State 19: Debora now sees where she has to be, at what time and knows all she wanted to know. Debora says: “Thank you”. Go to the next state.

State 20: The VSP says: “Okay” and returns to the main screen of the activity service. Go to state 3, 9, 15, 21, 23 or 25.

State 21: Debora wants to see the details of one particular activity, card games. Debora says: “Show me details of activity card games”. Go to the next state



State 22: The VSP says: “Okay, here it is” while the list at the left side is now changed into a detailed overview of the activity “card games”. In this detailed overview you see the time of the activity, the location, the price, the accessories needed and the participants (e.g., the end-users who subscribes to this group activity).

Activity: card games

Date: 13.11.2014

Time: 11:00-12:00 hrs.


Price: 0

Accessories: Cards


Go to state 19.

Situation: Debora changes the day

State 23: Debora: “Show me previous day”. Go to the next state.

State 24: The VSP is showing the list of the activities for the asked day at the left side of the screen and says: “At the screen you see a list with the activities of [yesterday].” Go to state 3, 9, 15, 21, 23 or 25.

State 25: Debora: “Show me the next day”. Go to the next state.

State 26: The VSP is showing the list of the activities for the asked day at the left side of the screen and says: “At the screen you see a list with the activities of [tomorrow]”. Go to state 3, 9, 15, 21, 23 or 25.

2.3.3 Use case: Appointment Reminder (Agenda Service - Care & Wellness Service)

ELDERLY ASPECTS

Situation: Later today at 10:00 hrs.

State 1: The VSP says: “Debora you have an appointment reminder. What would you like to do with it?” Go to state 2, 4, 6 or 8.

State 2: Debora answers “Show me the reminder”. Go to the next state.

State 3: The VSP answers: “In one hour your activity card games starts in the restaurant. “Do not forget to bring your cards.” “This activity is for free.” At the same time a text bubble in the screen is shown the following text:

Activity: Card games

Time: 11:00-12:00 hrs.


Price: 0



Accessories: Cards

Go to state 4, 6 or 8.

State 4: Debora: “Remind me later”. Go to the next state.

State 5: The VSP: “I will remind you again at 10:30”. STOP.

State 6: Debora: “Delete the reminder”. Go to the next state.

State 7: The VSP: “This reminder is deleted now”. STOP.

State 8: Debora did not answer to the VSP. A new reminder will be triggered 30 minutes later. STOP.

Situation: Later at 10:30 hrs

State 1: The VSP says: “Debora you have an appointment reminder. What would you like to do with it?” Go to state 2, 4, 6 or 8.

State 2: Debora answers “Show me the reminder”. Go to the next state.

State 3: The VSP answers: “In 30 minutes your activity card games starts in the restaurant. “Do not forget to bring your cards.” “This activity is for free.” At the same time a text bubble in the screen is shown the following text:


Time: 11:00-12:00 hrs.


Price: 0

Accessories: Cards



State 5: The VSP: “I will remind you again at 10:50”. STOP.

State 6: Debora: “Delete the reminder” Go to the next state.


State 8: Debora did not answer to the VSP. A new reminder will be triggered 20 minutes later. STOP.

Situation: Later at 10:50 hrs

State 1: The VSP says: “Debora you have an appointment reminder. This is the last reminder. What would you like to do with it?” Go to state 2, 4 or 6.

State 2: Debora answers “Show me the reminder” Go to the next state.

State 3: The VSP answers: “In 10 minutes your activity card games starts in the restaurant. “Do not forget to bring your cards.” “This activity is for free.” At the same time a text bubble in the screen is shown the following text:




Time: 11:00-12:00 hrs.


Price: 0

Accessories: Cards

Go to state 4 or 6.

State 4: Debora: “Delete the reminder” Go to the next state.


State 6: Debora did not answer to the VSP. The interaction is completed. After 15 minutes, the reminder will be automatically deleted. STOP.

2.3.4 Use case: Object Location Assistance and Reminder (Guidance Service)

Note that the full list of detectable objects is not clear yet. The full list of detectable objects will be provided either in D2.1 or D4.1.

CAREGIVER ASPECTS

Situation: A caregiver adds, modifies or removes an object from the list

State 1: Sylvie Rousseau (caregiver) accesses to the caregiver interface and selects the user Debora. Go to the next state.

State 2: Sylvie adds a new entry in the object location reminder list. Each entry is defined by the following fields: (1) Name of the object, (2) Room, (3) Location. STOP.

ELDERLY ASPECTS

Situation: The elderly is looking for an object in the apartment

State 0: Debora is repeatedly going from one place to the other. Go to the next state.

State 0.1: The system's behaviour analysis module recognizes that Debora might be searching for something, so the VSP asks Debora "Are you searching for something?" Go to state 0.2 or 0.3

Component Action

Behaviour Analysis User is searching

State 0.2: Debora answers: "Yes". Go to state 2

State 0.3: Debora answers: "No". Go to state 0.4

State 0.4: The VSP concludes the interaction: “Okay, Debora. I’m sorry to interrupt you. Let me know if you need something”. STOP.

Situation: The elderly opens the object localisation service



State 1: Debora tells the VSP: “Open object localisation”. Go to the next state.

State 2: VSP: “Welcome to the object localisation service. How can I help you?” Go to state 3, 11, 14 or 20.

Situation: Debora can’t find her hat

State 3: Debora: “Find the hat”. Go to state 4 or 6.

State 4: The system detects the hat on her couch. The VSP says: “I think your hat is on your couch. Did you found it?”. Go to state 5 or 9.

State 5: Debora says: “No”. Go to the next state.

State 6: The system scans the room but can’t find the hat. The system now automatically switches over to the object location reminder. The VSP tells: “Apparently, I can’t detect your hat via the camera. But normally, you store your hat in the table of your living room. Did you found the object?”. Go to state 7 or 9.

State 7: Debora says: “No”. Go to the next state.

State 8: The VSP looks sorry: “I’m sorry, but I can’t help you more than this. I hope you will find your hat soon” Go to the state 3, 11, 14 or 20.

Component Action


State 9: Debora says: “Yes”. Go to the next state.

State 10: The VSP: “I am glad to hear that! Anything else?” Go to the state 3, 11, 14 or 20.

Component Action


Situation: Debora adds a new object to the list of stored objects

State 11: Debora: “I’d like to add an object”. Go to the next state.

State 12: VSP: “Ok, Debora”. A form appears on the screen, allowing Debora to add manually the new object by (1) choosing the object from a list previously created by the administrator and (2) specifying the room and the location of the object (from a predefined list created by the administrator). Note that the system proposes standard rooms (like bathroom, living room, etc.) and locations (like table, couch, etc.). Debora confirms the new object by saying “Validate it” to the VSP. Go to the next state.

State 13: The main menu of the object localisation service is now shown on the screen. The VSP continues the interaction: “I added the new object in the database. And now, what do you want to do?”. Go to the state 3, 11, 14 or 20.

Situation: Debora removes an object from the list of stored objects



State 14: Debora answers to the VSP: “Remove glasses”. Go to the next state.

State 15: The VSP: “Debora, are you sure you want to delete the glasses from the list?”. Go to state 16 or 18.

State 16: Debora: “Yes”. Go to the next state.

State 17: The VSP: “Ok, the glasses have been removed from the list. How may I help you?” Go to state 3, 11, 14 or 20.

State 18: Debora: “No”. Go to the next state.

State 19: The VSP: “The glasses haven’t been removed from the list. How may I help you?” Go to state 3, 11, 14 or 20.

Situation: Debora quits the Object Localisation Service

State 20: Debora answers: “I want to quit the object localisation service”. The Object Location Service is now closed and the VSP is on the “full screen mode”. STOP.

2.3.5 Use case: Notification Service (Co-Net Service)

CAREGIVER ASPECTS

Situation: A caregiver sends a notification to Debora

State 1: Lotta de Graaf (Orbis receptionist) accesses to the caregiver interface and selects the user Debora. Go to the next state.

State 2: The receptionist sends a notification to Debora with the text “A package is waiting for you at the reception. Please pick it up before 5:00 PM.” STOP.

ELDERLY ASPECTS

Situation: A package is delivered at the reception of Orbis Hoogstaete for Debora

State 1: Immediately, after the receptionist sends the notification, the VSP says: “Debora, you have a new notification. What would you like to do with it?”. Go to state 2, 4, 6 or 8.

State 2: Debora answers “Show me the notification”. Go to the next state.

State 3: The VSP reads the text previously written by Lotta de Graaf: “A package is waiting for you at the reception. Please pick it up before 5:00 PM.” At the same time a text bubble in the screen is shown the following text:

Notification: “A package is waiting for you at the reception. Please pick it up before 5:00 PM.”



State 5: The VSP: “I will remind you again in 1 hour”. STOP.

State 6: Debora: “Delete the notification” Go to the next state.

State 7: The VSP: “This notification is deleted now”. STOP.



State 8: Debora did not answer to the VSP. The interaction is completed. The notification will be triggered on the next interaction with the user (i.e., when the user will approach to the system). STOP.

2.4 Fourth user scenario: July Appetite

July Appetite is 80 years old. She lives alone in an apartment including a large kitchen open on the dining area, a large bedroom, a hall, a bathroom and a balcony (47 square meters) in Silverstaete Residence. She was a former cook and likes to eat at the restaurant of Hoogsteate but also likes to make simple things herself in her kitchen. She is diabetic and follows a sugar free diet which makes it sometimes more difficult to cook, as she has some slightly starting memory problems. She also has an allergy for nuts. She would like to use the Miraculous Life system (1) to show her how to prepare a simple meal (that is safe and healthy for her) and (2) make a shopping list for a specific recipe.

2.4.1 Use case: Meal preparation

Situation: The elderly opens the Meal Preparation Service

State 1: July tells the VSP: “Open Meal Preparation”. Go to the next state.

State 2: VSP: “Welcome to the Meal Preparation Service. How can I help you?” The list of all the recipes available appears on the screen. Note that this list is in agreement with his dietary requirements and allergies. Go to state 3 or 18.

Situation: The elderly consults a recipe

State 3: July tells the VSP: “Show me omelette with cheese”. Go to the next state.

State 4: The VSP is on the right, showing the ingredients needed for the asked recipe:

Omelette with cheese: ingredients

large free-range eggs

sea salt

freshly ground black pepper

1 small knob butter

1 small handful Cheddar cheese, grated, optional

The VSP: “On the screen, you can see the ingredients needed for the recipe omelette with cheese. How can I help you, July?” Go to state 5, 7, 11, 13, 17, or 18.

State 5: July asks: “Read me the ingredients”. Go to the next state.

State 6: The VSP reads the ingredients shown on the screen aloud. The VSP concludes the list of ingredients by saying: “Anything else?” Go to state 5, 7, 11, 13, 17, or 18.

State 7: July asks: “Add an ingredient in the shopping list” Go to the next state.

State 8: The VSP: “Which ingredient would you like to add?” Go to the next state.

State 9: July: “Sea salt”. Go to the next state.



State 10: The VSP: “I just added the ingredient ‘sea salt’ on your shopping list, July. How can I help you?”. Go to state 5, 7, 11, 13, 17, or 18.

State 11: July asks: “Add all the ingredients in the shopping list” Go to the next state.

State 12: The VSP: “I just added all the ingredients of the recipe ‘omelette with cheese’ to your shopping list, July. Anything else?” Go to state 5, 7, 11, 13, 17, or 18.

State 13: July asks: “Show me the procedure of the recipe”. Go to the next state.

State 14: The VSP is on the right, showing the procedure of the recipe in txt:

“Beat together eggs and water until blended. In a 10-inch omelette pan heat butter until just hot enough to sizzle a drop of water. Pour in egg mixture.”

The VSP: “On the screen, you can see the ingredients needed for the recipe omelette with cheese” Go to the next state.

State 15: July: “Go back to the ingredients”. Go to the next state.

State 16: The VSP: “Here you are, anything else?” Go to state 5, 7, 11, 13, 17, or 18.

State 17: July: “Go back to the recipe list”. Go to state 2.

Situation: The elderly quits the Meal Preparation Service

State 18: July: “I want to close the Meal Preparation Service” Go to the next state.

State 19: The Meal Preparation Service is now closed and the VSP is on the “full screen mode”. STOP.

2.5 Fifth user scenario: Gunter Robertson

Gunter Robertson is 78 years old. He lives alone in an apartment including a kitchen open on the dining area, a large bedroom, a hall, a bathroom and a balcony (47 square meters) in the Colladon Residence. Gunter uses a calendar to write down friends’ and family’s birthdays. Since he does not consult his calendar daily, he often forgets to send greetings to his friends. Gunter spends most of his time at home, watching TV. He also needs motivation in order to perform physical activity.

2.5.1 Use case: Motivation for Physical Activity (Guidance Service)

Situation: The system detects that motivation for physical activity should be triggered. Note that: (1) this functionality is triggered only if the user does not express a negative mood state, (2) this functionality will not be triggered more than 3 times per week.

State 1: Physical inactivity is detected by: (1) user’s behaviour in the apartment, (2) the use of the service “Guidance during physical activity”, (3) the user’s participation in physical activities. The cumulative criteria for detecting physical immobility are the following:

The elderly has not participated in physical activities in the last 3 days; [AND]



The elderly has not used the “Physical Activity Service” (videos) in the last 3 days; [AND]

The system detects that the elderly is physically inactive inside his apartment; meaning that the elderly did not engage in significant physical activity (activity that has walking in it) for more than 4 hours in a time frame where you would expect walking.

Note that these criteria changes as a function of the behaviour of the user. If the three cumulative criteria are met, go to the next state.

Component Action

Behaviour Analysis Physically inactive of the elderly for the last 4 hours

Recognised user mood

Memory Participation to organised physical activities in last 3 days

Usage number of the “Physical Activity Service” (videos) in the last 3 days

Usage of the “Motivation for Physical Activity” in the last week

State 2: The system detects that Gunter is in the room. According to the criteria described in the previous state, motivation for physical activity should be triggered. The VSP initiates a new conversation trying to motivate him to perform physical activities: “Hey Gunter. Performing physical activities is healthy for your body, mind and soul. I would like to suggest you several activities that might interest you. Do you want to have a look?” Go to state 3, 4 or 5.

State 3: Gunter answers: “Yes”. Go to state 7.

State 4: Gunter answers: “No”. Go to state 21.

State 5: Gunter answers: “Remind me later”. Go to the next state.

State 6: The VSP: “Ok, I will remind you again in 1 hour”. One hour later, go back to the state 2.

State 7: The VSP suggests three physical activities and this according to the preferences of the end-user. The VSP can propose three kinds of activities: (1) the physical group activity organized by the home care institution (or by others primary end-users), (2) the physical group/individual activity created by the seniors and (3) the videos contained in the “Physical Activity Service”

In this scenario, the system knows that Gunter used to participate in the activity “Thai Chi” in the past. Nevertheless, Gunter did not subscribe yet for the next course. The VSP knows that Gunter is potentially interested in this activity and try to motivate him to subscribe: “In four days, the activity ‘Thai Chi’ will take place in the ‘Colladon Room’. Why don’t you participate, Gunter?” At the same time a text bubble in the screen is shown the following text:

Activity: Thai Chi



Date: 14.11.2014

Time: 11:00-12:00 hrs.

Location: Colladon Room

Go to state 8 or 10

Component Action

Memory Preferred physical activity

State 8: Gunter answers: “Register me for this activity” Go to the next state.

State 9: The VSP is on the “full screen mode”. The VSP concludes the interaction: “I am happy to hear that, Gunter. You are registered for the activity ‘Thai Chi’, on Monday at 11:00 hrs.” STOP.

Component Action


State 10: Gunter answers: “Propose me another activity”. Go to the next state.

State 11: Gunter – from time to time – creates an instance of the physical activity named “going for a walk” and invites his friend Luc. The VSP knows that Gunter could be potentially interested in this activity: “Gunter, what about the activity ‘going for a walk’? Maybe Luc will be happy to join!” Go to state 12 or 15.

Component Action

Memory Activity often created

Friend often invited

State 12: Gunter answers: “Create activity ‘going for a walk’”. Go to the next state.

State 13: The VSP continues the interaction: “Ok, Gunter! I am happy to hear that”. A text-input form appears on the screen, allowing Gunter to add manually the new event (via the keyboard of the tablet) by specifying: (1) the name of the activity (what), (2) the date, (3) the time, (4) the location of the activity (where), and (5) the persons invited (in this case Luc). Note that the system proposes standard activities (like going for a walk, etc.). Note also that the field regarding the name of the activity (“what” - i.e., “going for a walk”) and the persons invited were already filled in by the system. Gunter confirms the new appointment by saying “Validate it” to the VSP. Go to the next state.

State 14: The VSP continues the interaction: “I added the activity in your agenda. An invitation was also sent to Luc, as you requested”. A notification is sent to Luc and a new entry is created in his activity list. STOP.

Component Action


State 15: Gunter answer: “Propose me another activity”. Go to the next state.



State 16: This is the third and the last suggestion made by the VSP. This suggestion is the least likely to be accepted by the user.

Rarely, Gunter performs physical activity by using the “Physical Activity Service”. The VSP try thus to motivate him to use this service “here and now”: “Gunter, what about the “Physical Activity Service”? I am sure you will find some interesting videos” Go to state 17, 19 or 20.

Component Action

Memory Usage of the Physical Activity Service

State 17: Gunter answer: “Go to the Physical Activity Service”. Go to the next state.

State 18: The VSP: “I am happy to hear that, Gunter!” The service “Guidance during physical activity” is now open and Gunter selects the video. STOP.

Component Action


State 19: Gunter answers: “Come back to your first suggestion”. Go to state 7.

State 20: Gunter answers: “Don’t bother me anymore”. Go to state 21.

State 21: Gunter refused the suggestions made by the VSP. In this situation, the VSP adopts two possible behaviours according to their efficiency in the past situations: (1) a sad behaviour, (2) a directive behaviour. Go to state 21a (sad behaviour) or 21b (directive behaviour).

Component Action

Memory Most successful motivation strategy for physical activities

State 21a: The VSP appears slightly sad and worried: “Oh! It seems that you don’t want to perform any physical activity. That’s makes me sad, Gunter. I am worried for you. Don’t you really want to have a look at my suggestions?” Go to state 22a or 24a.

Component Action


State 22a: Gunter answers: “Go to the suggestions”. Go to the next state.

State 23a: The VSP now looks Happier: “Great!”. Go to state 7.

Component Action


State 24a: Gunter answers: “No, thank you”. Go to the next state.



State 25a: The VSP appears slightly sad and worried: “Ok, Gunter. Feel free to do as you please.” The system understands that this strategy is not efficient. Next time, the VSP will be more likely to motivate the elderly using a directive behaviour. STOP.

Component Action


State 21b: The VSP appears directive and their non-verbal behaviour is steady: “It seems that you don’t want to perform any physical activity. I am worried, Gunter… Come on, follow me! I want to propose you some physical exercise: it will not take more than 10 minutes! I promise you!“. Go to state 22b or 24b.

Component Action


State 22b: Gunter answers: “Go to the Physical Activity Service”. Go to the next state.

State 23b: The VSP now looks Happier: “Great!”. The service “Guidance during physical activity” is now open and Gunter selects the video. STOP.

Component Action


State 24b: Gunter answers: “No, thank you”. Go to the next state.

State 25b: The VSP appears slightly sad and worried: “Ok, Gunter. Feel free to do as you please.” The system understands that this strategy is not efficient. Next time, will be more likely to motivate the elderly using a “sad behaviour”. STOP.

Component Action


2.5.2 Use case: Physical Activity Service (Guidance Service)

Note: the videos will be recorded in both end-user institutions with an expert assistance, and – potentially – with seniors as demonstrators. The videos will have duration of 4-5 minutes each.

CAREGIVER ASPECTS

Situation: The caregiver assigned the video to the users

State 1: Sylvie Rousseau (caregiver) accesses the caregiver interface and selects the user Gunter. Go to the next state.

State 2: Sylvie chooses the videos that are available for Gunter, according to his specific health condition. STOP.



ELDERLY ASPECTS

Situation: The service is initiated by the user or after a suggestion like in the use case above (use case 2.5.1)

State 1: The user initiates the “Physical Activity” Service by saying “Open Physical Activity Service”. Go to the next state.

State 2: The VSP: “Welcome to the Physical Activity Service, Gunter. Please choose the type of exercise that you would like to perform”. On the screen appears a list of categories (for example: lower body stretches, upper body stretches, posture. The categories and the videos will be identified and recorded by the end-users institutions). Go to the next state.

State 3: Gunter selects by voice one of the categories presented on the screen: “Upper body stretches”. Go to the next state.

State 4: The list of all videos contained in this category appears on the screen. Each line of the list is composed by (1) a frame (image) of the video, and (2) the name of the video. The VSP continues the interaction: “Here are the available videos for the category ‘upper body stretches’. Please select the video you would like to watch, Gunter”. Go to state 5 or 6.

State 5: Gunter: “Select video 4”. Go to state 7.

State 6: Gunter: “Go back to the main menu”. Go to state 2.

State 7: The VSP gives an introduction about the selected video: “The purpose of this video is to increase the flexibility and range of motion of your hand and fingers. The length of the video is 5 minutes. What do you want to do?” Go to state 8 or 9.

State 8: Gunter: “Play the video”. Go to state 10.

State 9: Gunter: “Go back to the list of video”, Go to state 4.

State 10: The video is played by the system (layout: full-screen). The user is able to pause and stop the video. Note that the system will not track if the user is actually performing some exercise or not. Once the video is played, go to the next state.

State 11: The VSP concludes the interaction by motivating the user with some analytics: “Well done, Gunter. This is the fifth time you use this service this week! I hope to use this service again soon”. The VSP is on the “full screen mode”. STOP.

Component Action


2.5.3 Use case: Social Bonding

Situation: First interaction of the morning (‘how are you today?’)

State 1: The VSP is in the ‘passive mode’. Gunter just woke up. He approaches the VSP for the first interaction of the morning. The VSP starts the interaction: “Good morning Gunter. It’s Friday 16 November. How are you today?” Go to state 2 or 4.



Component Action


State 2: Gunter: “I am fine, thanks”. Go to the next state.

State 3: The VSP looks happy: “I am happy to hear that, Gunter”. Go to the second interaction of the morning.

Component Action


State 4: Gunter: “I don’t feel well”. Go to the next state.

State 5: The VSP appears worried: “I hope it's not anything serious. Is everything okay with your health? Go to state 6 or 12.

Component Action


State 6: Gunter answers: “No, it isn’t”. Go to the next state.

State 7: The VSP looks worried: “Shall I ask help from your caregivers?” Go to state 8 or 10.

Component Action


State 8: Gunter answers: “Yes”. Go to the next state.

State 9: The VSP performed the “Call for Help” procedure as described in 2.2.7. STOP

State 10: Gunter: “No, that’s just okay for now”. Go to the next state.

State 11: The VSP appears worried and slightly suspicious: “Ok, Gunter. Let me know if you will need support from your caregivers.” Go to the second interaction of the morning. At 11:00, the situation “the VSP checks the health status of the elderly (‘do you feel better now?)”, will be also triggered.

Component Action


State 12: The system makes the hypothesis that the user feels bad “emotionally”. Go to the situation “Reaction on persisting sadness state” and then to the second interaction of the morning.

Situation: Second interaction of the morning (agenda)



State 1: The second interaction of the morning starts immediately after the first one. The VSP: “You have three activities in your agenda for today. Do you want to have a look, Gunter?” Go to state 2 or 3.

State 2: Gunter answers: “Yes”. Go to state 2 of the use case 2.3.1. Once the elderly consulted his agenda, go to the third interaction of the morning.

State 3: Gunter answers: “Maybe, I will do later”. Go to the third interaction of the morning.

Situation: Third interaction of the morning (notifications)

State 1: The third interaction of the morning starts immediately after the second one and shows the user all the notifications (agenda, missed phone calls, custom reminders, etc.) and says: “Gunter, you received 2 notifications during the last night. Do you want to have a look?” Go to state 2 or 3.

State 2: Gunter answers: “Yes”. The VSP shows sequentially all the notifications received by the end-user. Once this iterative process is completed, go to state 4.

State 3: Gunter answers: “No”. Go to state 4.

State 4: The VSP concludes the first interaction of the morning: “Okay Gunter, have a nice day!”.

Component Action


Situation: The VSP checks the health status of the elderly (‘do you feel better now?’)

This situation is triggered if during the first interaction of the morning the user reported having health problems and chose not calling caregivers. In this case this interaction is triggered at 11:00.

Component Action

Memory Occurrence of not feeling well today

State 1: It’s 11:08. Gunter approaches to the system. The system detects the user in front of the tablet, the VSP starts the interaction: “Welcome back, Gunter. How are you feeling now? Do you feel better?” Go to state 2 or 4.

State 2: Gunter: “Yes, I feel better”.

Component Action


State 3: The VSP looks relieved: “Great! I am happy to hear that.” STOP.

State 4: Gunter: “I still don’t feel well”. Go to the next state.



Component Action


State 5: The VSP looks worried: “Shall I ask help to your caregivers?” Go to state 6 or 8.

State 6: Gunter answers: “Yes”. Go to the next state.

State 7: The VSP performed the “Call for Help” procedure as described in 2.2.7. STOP

State 8: Gunter: “No, that’s just okay for now”. Go to the next state.

Component Action


State 9: The VSP appears worried and slightly suspicious: “Ok, Gunter. Let me know if you will need support from your caregivers.” STOP.

Situation: Missed Appointment Reminder

State 1: During the morning, Gunter did not react to an appointment reminder. It’s 12:30, Gunter approaches to the system. The VSP starts the interaction: “Welcome back Gunter. You have a missed appointment reminder. What would you like to do with it?” Go to state 2, 4, 6 or 14.

State 2: Gunter answers “Show me the reminder” Go to the next state.

State 3a (Conditional): The missed appointment was a group activity. The VSP answers: “Today at 11:00 you had the activity Card games. Did you participate?” At the same time a text bubble in the screen is shown the following text:


Time: 11:00-12:00 hrs.


Price: 0

Accessories: Cards


State 3b (Conditional): The missed appointment was an invitation made by another elderly. The VSP answers: “Today at 10:00 you had the activity “going for a walk”. Your friend Cindy invited to join. Did you participate?” At the same time a text bubble in the screen is shown the following text:

Activity: going for a walk

Date: 13.11.2014

Time: 10:00-11:00 hrs.

Location: Orbis park

Participants: Cindy, Gunter




State 4: Gunter: “I participated to the activity”. Go to the next state.

State 5: The VSP deletes the reminder and says: “Great! Nice to hear that you are active”. STOP.

Component Action


State 6: Gunter: “I’ve forgot it”. Go to the next state.

State 7a (Conditional): The missed appointment was a group activity. The VSP looks slightly sad: “Oh, what a pity! Things like that happen, Gunter.” Now the VSP looks more proactive: “What about looking for a new group activity?” Go to state 8 or 13.

Component Action


State 7b (Conditional): The missed appointment was an invitation made by another primary end-user. The VSP looks slightly sad: “Oh, what a pity! Things like that happen, Gunter. Don’t you want to write a short message to Cindy to apologize yourself?” Go to state 9 or 13.

Component Action


State 8: Gunter: “Show me group activities, please”. The Group Activity Service is now activated. STOP.

State 9: Gunter: “Write a message to Cindy”. Go to the next state.

State 10: The VSP is on the right. The VSP continues the interaction: “Ok Gunter”. Go to the next state.

State 11: A form appears on the screen, allowing Gunter to write a message to Cindy via the keyboard of the tablet. Go to the next state.

State 12: Gunter sends the message by pressing the “Send Message” button. In this state, a “Cancel” button is also designed, allowing the user to go back to the main screen without sending the message. The VSP becomes happy that Gunter is socially interacting with others. STOP.

Component Action


State 13: Gunter: “Maybe, I will do later”. STOP.

State 14: Gunter: “Delete the reminder” Go to the next state.




Situation: Missed Medication Reminder

State 1: During the morning, Gunter did not react to a medication reminder. It’s 11:15, Gunter approaches to the system. The VSP starts the interaction: “Welcome back Gunter. You have a missed medication reminder. What would you like to do with it?” Go to state 2, 4, 6 or 13.

Component Action


State 2: Gunter answers “Show me the reminder” Go to the next state.

State 3: The VSP answers: “It’s 11:15 hrs, Gunter. Did you take the Actos medication this morning? At the same time a text bubble in the screen is shown the following text:

Medication: Actos.

Time: 10:00 hrs.

Amount: 2 pills.


State 4: Gunter: “I’ve (already) taken it”. Go to the next state.

State 5: The VSP: “Great! I will delete the reminder now”. STOP.

Component Action


State 6: Gunter: “I don’t remember”. Go to the next state.

State 7: The VSP: “You should be more careful Gunter: medications should be taken on time. Maybe André [MRPS caregiver] can help you deciding whether or not taking your medication.” Go to state 8 or 11.

Component Action


State 8: Gunter: “Call André”. Go to the next step.

State 9: The “calling window” appears on the screen, showing the outgoing call. André answers to the call: the conversation starts. Go to the next state.

State 10: Gunter (or André) ends the conversation by pressing the button “End Call”. The call window is now closed. STOP.

State 11: Gunter: “No thank you, I don’t need to call Andre”. Go to the next state.

State 12: The VSP concludes the interaction: “Okay Gunter. This reminder is deleted now”. STOP.



Component Action



State 14: The VSP: “Gunter, are you sure you don’t want to take your medication? A notification will be sent to your caregivers”. Go to state 4 or 15.

Component Action


State 15: Gunter: “Yes, I don’t want to take it”. A notification is sent to the caregivers. STOP.

Situation: The user comes back after an activity

State 1: Gunter comes back from a group activity (or from an activity organized by a friend). On the next interaction with the system, the VSP will try to understand whether the elderly appreciated or not the activity.

Component Action

Memory End time of last activity

State 2: The VSP: “Welcome back, Gunter. Today at 10:00 you had the activity Thai Chi. Did you appreciate it?” Go to state 3, 4 or 5.

State 3: Gunter: “Yes a lot!” Go to state 6.

State 4: Gunter: “It was not bad” Go to state 7.

State 5: Gunter: “Not really” Go to state 7.

State 6: The VSP concludes the interaction: “I am happy to see you so enthusiastic, Gunter”. STOP.

Component Action


State 7: The VSP concludes the interaction: “Oh, maybe next time you will appreciate it more!” STOP.

Component Action

Emotion The avatar expresses compassion

Note that the system stores the information provided by the user. This information will be used by the VSP in order to suggest activity to perform.



Situation: Reaction on persisting blame affective state (anger, disgust)

State 1: The system detects that the user is on a persisting blame affective state. Go to the next state.

State 2: Gunter approaches to the system (or he’s already interacting with it). The VSP starts a new interaction: “Gunter, you look upset. Can I do something for you?” Go to state 3 or 4.

Component Action


State 3: Gunter: “Yes, please”. Go to state 6.

State 4: Gunter: “No Mary, it’s only your impression”. Go to the next state.

State 5: The VSP: “Okay Gunter, good for you” STOP.

Component Action


State 6: The VSP looks particularly empathic and compassionate: “Gunter, staying in touch with your friends and family or participating in relaxing activities might help you calm down. Why don’t you plan the activity ‘walking in the park’? Maybe your friend Maria will join you. Go to state 7, 8, 9, 10, 11 or 12.

Component Action

Memory Activity often created when angry

Friend often invited when angry


State 7: Gunter: “Call Maria”. A call with Maria is established. STOP.

State 8: Gunter: “Call another person” Go to the Contact List service. The VSP continues the conversation: “Here you are Gunter, who do you want to call?” STOP.

State 9: Gunter: “Invite Maria to visit me”. A new agenda item “Invite a friend” appears on the screen. Note that the activity “invite a friend” will be proposed as a standard activity in the Agenda Service (see Situation: Add Agenda Items and Inviting persons). The system already selected Maria as an invited person. STOP.

State 10: Gunter: “Invite someone else” A new agenda item “Invite a friend” appears on the screen. Note that the activity “invite a friend” will be proposed as a standard activity in the Agenda Service (see Situation: Add Agenda Items and Inviting persons). Contrary to the state 9, the system didn’t select Maria as an invited person. Gunter will choose thus the person to invite. STOP.

State 11: Gunter: “Create activity ‘walking in the park’”. A new agenda item “walking in the park” appears on the screen. Note that the activity “walking in the park” will be proposed



as a standard activity in the Agenda Service (see Situation: Add Agenda Items and Inviting persons). STOP.


The system knows that Gunter often invites Luc to the relaxing activity “walking in the park”. Maria could be a friend or a member of the Gunter’s family.

Situation: Reaction on persisting fearful/stressful affective state

State 1: The system detects that the user is on a persisting fearful/stressful affective state. Go to the next state.

State 2: Gunter approaches to the system (or he’s already interacting with it). The VSP starts a new interaction: “Gunter, you look worried about something. May I do something for you?” Go to state 3 or 4.

Component Action




State 5: The VSP: “Okay Gunter, good to hear that you are ok!” STOP.

State 6: The VSP looks particularly empathic and compassionate: “Gunter, I don’t know why you are stressed. But I am sure that Maria or André can support you and make you feel better.” Go to state 7, 8, 9 or 10.

Component Action

Memory Friend often invited when fearful


State 7: Gunter: “Call Maria”. A call with Maria is established. STOP

State 8: Gunter: “Call André”. A call with André is established. STOP

State 9: Gunter: “Call another person” Go to the Contact List service. The VSP continues the conversation: “Here you are Gunter, who do you want to call?” STOP.

State 10: “Maybe, I will do later”. STOP.

Maria could be a friend or a member of the Gunter’s family; while André is a formal caregiver.

Situation: Reaction on persisting sadness state



State 1: The system detects that the user is on a persisting sadness state. Go to the next state.

State 2: Gunter approaches to the system (or he’s already interacting with it). The VSP starts a new interaction: “Oh Gunter, It seems that you are feeling sad. Can I do something for you?” Go to state 3 or 4.

Component Action




State 5: The VSP: “Okay Gunter, good to hear that you are ok!” STOP.

Component Action


State 6a (Conditional): The VSP realises that Gunter (1) subscribed in all group activities proposed today by caregivers and (2) accepted all the invitations made by others primary end-users for today. The VSP looks particularly empathic and compassionate: “I am sorry that you are feeling sad, Gunter. Staying in touch with your friends and family might help you feel better. Why don’t you call or invite your friend Maria? I am sure that she will cheer you up”. Go to state 9, 10, 11, 12 or 13.

Component Action

Memory Friend often invited when sad


State 6b (Conditional): The VSP realises that Gunter subscribed in all group activities proposed today by caregivers but didn’t accept all the invitations made by others primary end-users for today. The VSP looks particularly empathic and compassionate: “I am sorry that you are feeling sad, Gunter. I am sure that participating in social activities or speaking with a friend will cheer you up. Yesterday Miguel invited you to the activity “going for a walk”. Why don’t you participate? Or maybe do you prefer to call or invite someone? Maybe speaking with Maria will help you feel better. Go to state 8, 9, 10, 11, 12 or 13.

Component Action


State 6c (Conditional): The VSP realises that Gunter didn’t subscribe in all group activities proposed today by caregivers but accepted all the invitations made by others primary end-users for today. The VSP looks particularly empathic and compassionate: “I am sorry that you are feeling sad, Gunter. Maybe staying in touch with your family or participating in group activities will make you feel better. Today at 4:00 pm the group



activity “Thai Chi”, will take place. Why don’t you participate? Or what about calling, or inviting Maria? I am sure that she will cheer you up”. Go to state 7, 9, 10, 11, 12 or 13.

Component Action


State 6d (Conditional): The VSP realises that Gunter (1) didn’t subscribe in all group activities proposed today by caregivers and (2) didn’t accept all the invitations made by others primary end-users for today. The VSP looks particularly empathic and compassionate: “I am sorry that you are feeling sad, Gunter. I am sure that participating in social activities or speaking with a friend will cheer you up. Yesterday Miguel invited you to the activity “going for a walk”. In addition today at 4:00 pm the group activity “Thai Chi”, will take place. Why don’t you participate in one of those activities? Or what about calling Maria? I am sure that she will cheer you up. Go to state 7, 8, 9, 10, 11, 12 or 13.

Component Action


State 7: Gunter: “Show Group Activities”. Go to the Group Activity Service. STOP.

State 8: Gunter: “Show Invitations”. Go to the “Message Service” and filter the Invitations. STOP.

State 9: Gunter: “Call Maria”. A call with Maria is established. STOP.

State 10: Gunter: “Call another person”. Go to the Contact List service. The VSP continues the conversation: “Here you are Gunter, who do you want to call?” STOP

State 11: Gunter: “Invite Maria to visit me”. A new agenda item “Invite a friend” appears on the screen. Note that the activity “invite a friend” will be proposed as a standard activity in the Agenda Service (see Situation: Add Agenda Items and Inviting persons). The system already selected Maria as an invited person. STOP.

State 12: Gunter: “Invite someone else”. A new agenda item “Invite a friend” appears on the screen. Note that the activity “invite a friend” will be proposed as a standard activity in the Agenda Service (see Situation: Add Agenda Items and Inviting persons). Contrary to state 11, the system didn’t select Maria as an invited person. Gunter will choose thus the person to invite. STOP.


Note that Maria could be a friend or a member of the Gunter’s family.

Situation: Reaction on persisting positive affective state (happiness)

If the system detects a positive affective state on the user, the VSP will express the same emotional state, and this in every state of every use case (mimicry). Thus, if the end-user is happy, the VSP should appear happy too. This should reinforce positive emotional state in the end-user.



3 Cost Benefit analysis of the use-cases†

3.1 Introduction

Following the 1st project review meeting (M14) the consortium was asked to provide a clear cost-benefit analysis with the additional double-perspective:

i) Technical perspective – feasibility, major obstacles;

ii) Relevance of the needs addressed in the use case.

This chapter presents this analysis, which identifies the most meaningful use-cases considering their feasibility and relevance regarding the needs addressed by each of them. The technical partners will utilize this analysis in order to prioritize the development of the use cases, as well as, to better facilitate resource allocation, thus minimizing any risks that could result in failure to implement the use cases. Additionally, the end-users organizations gain insight about which use-cases provide greater benefit to the end-users,

Furthermore, this analysis enables the consortium to assess the use cases in order to avoid spending resources in implementing parts of the use-cases where their applicability could be limited due to feasibility constrains. Therefore, it is expected that during the development some of the use cases described in chapter 2 might be slightly simplified in order to overcome these feasibility constrains, but without affecting the objectives and the goals of the use cases.

Along with the cost-benefit analysis, a new section that details the workflow of building each use-case (carefully revised and validated from all related partners) is introduced (section 3.2). The structure of this chapter is as follows: First, the workflow of building and revising the use-cases, including a feasibility analysis, is described. Next, the benefit analysis process is presented and the benefit provided by each use-case is demonstrated. A detailed presentation about how the natural interaction of the use-cases was used as a common strategy to set the ground for building a coherent and technically validated VSP follows. At the end, the final remarks for this chapter are presented.

3.2 The Use Cases Development Process

The first version of this deliverable (D1.2a) was released in month 8 of the project (M8), followed in M12 by a second (final) version (D1.2b). During the mediate period between these two versions (i.e., from M09 – M12), we extracted the end-user requirements (utilizing as input D1.1a released in M8) and in collaboration with WP4 where the services were defined, created a prioritized list of services, based on the major goals of the project from the end-user’s perspective. This information was utilized for the revision of the use cases presented in the previous version of this deliverable (M8 – D1.2a).

With this ranking of services, presented in Appendix B, WP1 defined a revision process, which started with the careful revision of the use-cases in order to become more meaningful and connected to the user needs resolved by the associated services. Both end-user organizations revised the use-cases and presented them to allow a wide discussion within WP1.

† This sole chapter was added in M19 following the 1

st review meeting report.



After a general discussion, we identified a number of use cases needed to showcase the different services that Miraculous-Life provides. Following this step, each use-case was subject to thorough analysis, including a detailed feasibility analysis involving both end-user organizations and all technical partners participating in the development of each use-case. This included exchange of information via email, direct communication using Skype and/or physical meetings. Following the validation and agreement for implementation from the technical partners involved in the use cases and end-user organizations, Noldus and Citard integrated the use-cases in D1.2b.

The process described above concluded in M12 resulting in a final list of 23 use cases (carefully inspected to become coherent and feasible). This process enabled us to reduce implementation risks, as well as, address obstacles such as the limitation of depth-of-filed cameras (Kinect) or limitations in the dialog patterns. Appendix B presents the list of the use-cases, including the technical partners that validated the technical feasibility of each use-case.

Furthermore, an internal document was produced on M14, specifying the planning for the use cases demonstration. More specifically this document, entitled “D1.2b Use Cases - Planning for Pre-Trials and Trials”, gathered feedback from all technical work packages to identify which services would be demonstrated on M16 (2nd pre-trial) and which services on M24 (final trials), thus enabling the proper organization and planning for the implementation and the testing of the use-cases before the project final trials.

3.3 Natural Interaction in the Use-Cases

Natural interaction is the heart of the system and is crucial for the acceptance of the system for the elderly. Within the Miraculous Life project, natural interaction includes multimodal dialogue management, affective user modelling and adaptation, personality, cognitive and affective capabilities, human-like memory structures and processes and perception and awareness. As each individual part of the natural interaction could be a project goal on its own, the idea of Miraculous Life is to adapt and enhance existing tools and techniques provided by partners or available elsewhere and improve the VSP technology. The use cases are setup in such a way to accommodate all these aspects of natural interaction as much as possible. However, the final presentation of the use cases is the result of intensive negotiation about what is technologically possible to be developed by the technical partners and what is acceptable or required by the elderly and their caregivers (represented by the end-user organizations).

An illustrative example is to discuss the process of including speech recognition in the use cases. The speech-controlled interface is identified by the end-users as crucial for the interaction of the elderly with the VSP, as elderly will not accept a system without speech recognition. The technical partners Noldus and UniGe already confirmed in an early stage of the project, that currently available speech recognition software needs a lexicon to recognize what the person says and therefore needs much data, in the desired language to be recognized. The constraint from the end-user’s locations implies we need specific dialects, for which no data is available yet, particularly for the Dutch language. Therefore, the speech recognition is limited to command-based speech recognition such as used in human-computer interaction or gaming. In the discussion with the end-users, we believe this solution can work because of two reasons: (1) for the command-based speech recognition there are dictionaries available for Dutch and French and (2) elderly can be asked to give commands in the official language. The dialogues in each use case are



proposed, revised and tested by the technical partners and end-user organizations on feasibility, acceptance and flow. As a result, long sentences are reduced to small commands, the text expected to be spoken is displayed as much as possible on the interface to help the elderly with the spoken commands and free speech is replaced by the possibility to have text field input.

Recent developments in the field and an early inventory of existing tools from the partners, lead to the decision to use the Microsoft Kinect One sensor as the main sensor of the system as it (1) provides the VSP with all the necessary sensor modalities (sound, video, and human motion), (2) the accompanying software development kit provides state-of-the-art components we need, and (3) it is affordable. Tests at end user locations proved that this sensor is capable of satisfying the project’s needs. To ensure 100% coverage (e.g., for safety purposes), we need multiple Kinect One sensors. However, technologically, it is not possible to connect more than one Kinect One sensor on a single PC and, even more important, the elderly protested to have more than one Kinect One sensor in their homes. Although limiting, the technical partners (Fh-IGD, UniGe and Noldus) think that the Kinect One is the only way to capture human motion within the project and we accepted the focus to a single hotspot.

The restriction of only one Kinect One sensor and the use of the tablet sensors elsewhere have serious effects on the (natural) interaction. For the affective user modelling and adaptation, consider how emotions are measured and how the VSP should respond. For the affective state recognition, Noldus and Zoobe have already developed technology to recognize emotions from face and speech, respectively. However, these techniques are validated only under specific constraints. The Zoobe’s emotion recognition from speech module is trained using a corpus recorded in a lab environment and the Noldus FaceReader is validated for human-computer interaction under perfect illumination conditions with a fixed distance between the user and the camera. Although it is part of the project to adapt these existing tools to the VSP, their performance will be affected, as the natural environment is more challenging. In developing the use cases, it is necessary to make a trade-off between system stability and technology. If we want the VSP to respond to an affective state of the elderly, we need to make sure the VSP is correct. Hence, we need to guarantee that the VSP has multiple measurements over time in order to confirm that the affective state estimation is correct. The decision of using only one Kinect One sensor and the sensors in the tablet decreases the circumstances to have robust measurements, thus requiring measuring for a longer period. Consequently, only situations where the VSP responds to the elderly’s persistent affective states are defined in the use cases (see for example use cases 2.5.1 and 2.5.3).

The development and implementation of some other aspects of the natural interaction, like for example, the multimodal dialogue management, and the human-like memory structures are also considered in the construction of the use cases as UniGe designed and discussed specific use cases with the end-user organizations to guarantee these components are present when appropriate and feasible. In addition, Zoobe, responsible for the emotion expression of the avatar and the text-to-speech, validated the requests from the end-users to incorporate a natural appearance of the avatar in the use cases. It is worth mentioning that the selection of the emotions expressed by the VSP in D1.2b, was based on the theoretical framework presented in the revision of D1.1a.



3.4 Cost-Benefit perspective

In order to provide a cost benefit analysis of the use cases we considered a technical perspective to understand the Cost, and a relevance analysis of the use cases to assess their Benefit. To create the technical perspective of the use cases, we performed the following analysis per each use case:

Technical Feasibility assessment

Risk Analysis

Risk Mitigation

To address the benefit analysis, we performed the analysis of the relevance of the needs addressed by the use cases. Particularly:

The relevance assessed by the elderly of ORBIS and MRPS

The relevance assessed by the care professionals of ORBIS and MRPS;

The project objectives;

Cost Analysis:

To understand the cost of each use case, we asked the technical partners to provide a technical perspective for each use case, by performing an analysis on four (4) aspects:

Score of Technical Feasibility: Using a numerical classification of the feasibility of a use case it became possible to have clear empirical perspective of the cost of the use case. The scale for this assessment is:

o Excellent (5): The use case is feasible and any shortcomings, if any, are minor.

o Very Good (4): The use case is feasible but a small number of shortcomings are present.

o Good (3): The use case is feasible but a number of shortcomings are present.

o Fair (2): Significant shortcomings are present.

o Poor (1): Serious inherent shortcomings are present.

Risk Assessment on achieving the use case objectives: Using a simple scale of None, Low, Medium or High.

o None: No Impact on achieving the use case objectives.

o Low: Little Impact on achieving the use case objectives.

o Medium: Moderate impact on achieving desired results, to the extent that one or more of the use case objectives will fall below goals but above minimum acceptable levels.

o High: Significant impact on achieving desired results, to the extent that one or more of the use case objectives will fall below acceptable levels.

Risk Explanation: This element is connected to the previous ones, detailing the risks existing the given use case.

Risk Mitigation: If risks are identified in the use case, technical partners already share their perspective and mitigation actions will be enabled if needed.



This technical perspective, which reflects the cost analysis of each use case, is presented in the second column of Table 2.

Benefit Analysis (Qualitative and Empirical Perspective):

The benefit analysis was performed considering both a qualitative perspective and an empirical perspective. For the qualitative perspective, we analysed the relevance and usefulness of each use case for both the elderly and the caregivers. Table 2 (third and fourth columns) presents in detail the benefit found in each use case, in terms of the relevance of the needs for each type of user (elderly or caregiver), in line with the project objectives as specified in the DoW.

For the empirical perspective, a user questionnaire (see Appendix A) was built and administrated to MRPS and Orbis caregivers and professionals in order to evaluate the relevance of the 23 use cases specified in the D1.2b. Participants were asked to rate each use case by using a 10-points Likert scale (1 = not a priority; 10 = essential priority). Similarly, the same questionnaire was utilized to obtain this information from the elderly of both end-user organizations.

The MRPS elderly group included seven people and the caregiver group included nine people (1 nursing auxiliary, 3 animators; 2 nurses; 2 members of the “resident service” team; 1 care coordinator).

The Orbis elderly group included seven people and the caregiver group included fourteen people (4 carers; 4 nurses; 3 policy advisors; 2 animators; 1 project manager).

The information shown in Figure 5 summarises the results collected in both end-user organizations (the complete rankings are presented in Appendix C (for both end-users organizations), Appendix D (for MRPS) and Appendix E (for Orbis)).

Overall, all the use cases proposed in the D1.2b are considered as being relevant and meaningful by caregivers and professionals of both end-user organizations. This was the expected result since the use cases specified in the D1.2b are based on the end-user requirements identified and presented in D1.1a.

To provide a single and overall empirical benefit analysis it is necessary to combine the information from the elderly and caregivers. In Figure 5, the tendency curves of both the elderly and caregivers have a strong similarity. Therefore, it becomes clear that both populations have the same relative opinion of the relevance of the use cases. An offset is identifiable which could be explained by the different nature of the types of populations (caregivers and elderly). Furthermore, a less stable standard deviation is identified in the elderly results (Table 2 presents these values). It can be concluded that to achieve a final relevance value any decision on weights associated to elderly and caregivers’ results would just be reflected in a minor change of the offset of the relevance of the use cases and not to their tendency.

Therefore, the decision to use an average relevance value (black tendency curve in Figure 5) is a balanced choice since it does not compromise the relative importance between use cases and still encompasses both elderly and caregivers results.

The overall benefit for each use case is also included Table 2.



Figure 5: Evaluation results from elderly and caregivers of Orbis and MRPS for the relevance of the needs addressed by the 23 use cases based on a Likert scale from 1 to 10 (1 = not a priority; 10 = essential priority)



Table 2: Technical perspective and Relevance of the use cases, including both qualitative and quantitative their analysis. This information constitutes the Cost-Benefit analysis for each use case.

Use Case Technical Perspective Relevance/Usefulness for Elderly Relevance/Usefulness for Caregivers

2.1.1 Contact list

Partners Involved: AIT, CITARD, UniGe

Score of Technical Feasibility: 5

Risk Assessment: Low

Risk Explanation: Technical difficulties may occur when developing the phone line based dispatching technology;

Risk Mitigation: Use of existing cloud based dispatching services (e.g., Nexmo.com).

It allows the elderly to have direct and easy access to their social contacts and formal/informal caregivers.

Increase the elderly social interaction, preventing loneliness and isolation, contributing thus positively to their wellbeing.

Promotes collaboration, between the elderly and formal/informal caregivers, in order to better and more efficiently assist the elderly and support him/her through his/her daily activities.

Improve highly the efficiency and continuity of integrated care provision to the elderly at home, resulting in reduction of demand of care resources and burden of care.

Usefulness/Relevance Assessment for Elderly:

Mean: 6.07 SD: 2.50

Usefulness/Relevance Assessment for Caregivers:

Mean: 8.64 SD: 1.22

Overall Benefit assessment: 7.72

Based on the Usefulness/Relevance assessment by the Elderly and Caregivers in a 1 to 10 Likert scale.




2.1.2 Message System

Partners Involved: AIT, CITARD, UniGe



Risk Explanation: Technical difficulties may occur when developing the phone line based dispatching technology;

Risk Mitigation: Use of existing cloud based dispatching services (e.g., Nexmo.com).

Increase elderly social interaction contributing thus positively to their well.

Facilitates communication between the elderly and his/her formal/informal caregivers and other users of the Miraculous-Life system, thus preventing loneliness and isolation.

Prevents loneliness and therefore improves efficiency and continuity of integrated care provision to the elderly at home. Results in reduction of demand of care resources and burden of care.

Promotes collaboration, between the elderly and formal/informal caregivers, in order to better and more efficiently assist the elderly and support him/her through his/her daily activities


Mean: 5.89 SD: 2.70


Mean: 8.20 SD: 1.76



2.1.3 Shopping Assistance

Partners Involved: AIT, CITARD, UCY, UniGe



Risk Explanation: Minor technical difficulties may occur when developing the phone line based SMS service;

Risk Mitigation: Use of existing cloud based SMS services (e.g., Nexmo.com).

Assist the elderly to remain longer active and independent, by maintaining his/her autonomy and help him/her increase their confidence.

Skills and capabilities are thus maintained for longer.

The elderly maintain their social interactions with their informal caregivers and/or friends who volunteered to do the shopping for them.

Stimulating independence and thereby reducing care burden since the shopping will be done by an informal caregiver.


Mean: 5.61 SD: 2.53


Mean: 7.71 SD: 1.78






2.1.4 Medication Reminder

Partners Involved: CITARD, UCY, UniGe


Risk Assessment: None

Prevents forgetfulness and allows the elderly to feel more independent by managing his medication intake on time. This prolongs the period the elderly stays at home.

Improves safety in elderly Health Care by notifying the caregivers in case the elderly does not take important medication, thus prolonging their quality of life in their home environments.

Stimulating elderly independence and thereby reducing care burden.

Reduction of care stress and workload by the formal and informal caregivers since they only have to attend the elderly in case the system notifies them.


Mean: 6.25 SD: 3.00


Mean: 8.84 SD: 1.34

Overall Benefit assessment:7.91


2.1.5 Wake-up Calls

Partners Involved: AIT, CITARD, UCY, UniGe



Prolongs the autonomy and independence of the elderly, providing the opportunity to set up his own wake-up calls.

Stimulates the elderly to start an active day rhythm on his own pace.

Stimulating elder independence and thereby reducing care burden.

The caregiver does not have to attend and worry about waking up the elderly.


Mean: 7.10 SD: 2.10


Mean: 7.60 SD: 1.83






2.2.1 Periodic Advice




Ensures the elderly is advised to keep healthy and active, thus improves their quality of life. In addition, keeping healthy promotes an active lifestyle for the elder and prolongs their autonomy.

Certain safety advice is communicated to the elderly, thus promoting independent leaving and prolonging their life in their home environments.

Improve highly the efficiency and continuity of integrated care provision to the elderly at home. Resulting in reduction of demand of care resources and burden of care.


Mean: 6.39 SD: 2.35


Mean: 7.68 SD: 2.12



2.2.2 Mode of the system: Active vs Passive Mode

Partners Involved: AIT, UniGe



Stimulate autonomy by letting the elderly choose when they need an active or passive system.

The elder feels understood and confident of having control of the system and perceive the system as a companion and not as a directive tool.

Autonomy is given to the elderly, thus less care burden.


Mean: 7.61 SD: 2.44


Mean: 6.71 SD: 2.66






2.2.3 Configure the VSP Speech

Partners Involved: AIT, UniGe



Personal needs change over time due to capabilities degradation; the elderly can adjust the system according to his/her needs (e.g., speech, audio, voice, pace).

Provides understanding of the difference in capabilities, thus increase the elderly’s satisfaction in using the system and enhances the engagement of the user in interacting with the agent.

Allows the elderly to adjust settings on their own, thus promotes autonomy and independence therefore, reducing care burden.


Mean: 8.54 SD: 1.12


Mean: 8.39 SD: 1.57






2.2.4 Fall Detection

Partners Involved: AIT, Fh-IGD, UniGe


Risk Assessment: Medium

Risk Explanation: 1) False Fall Detection event possible to occur as identification or variation of movement by Kinect camera is not accurate;

2) Technical difficulties may occur when developing the phone line based dispatching technology

Risk Mitigation: 1) When a Fall is detected do not notify the caregivers immediately. Ask the elderly first if he/she is ok otherwise notify;

2) Use of existing cloud based dispatching services (e.g., Nexmo.com)

Improve Safety and Quality in elderly Health Care (e.g., by immediate notification of the caregivers in case of a fall due to a heart attack) thus prolonging their life in their home environments.

Delay inclusion into care centers (since their life at home is prolonged).

Reduction of care stress by the formal and informal caregivers.

Reduction of the demand of care resources and of the burden of care by the formal and informal caregivers.


Mean: 8.00 SD: 2.48


Mean: 9.32 SD: 0.90



2.2.5 Dangerous Objects Adviser

Partners Involved: AIT, Fh-IGD, UniGe



Risk Explanation: Identification of (very small) objects by a Kinect camera is not accurate or not possible.

Risk Mitigation:

A list will be provided with the objects that can be detected and identified by a Kinect camera. For very small objects that cannot be identified by the Kinect camera another technical solution will be adopted

Improves safety (e.g., by immediate notification of dangerous objects) thus preventing a possible accident and prolonging the elders’ life in their home environments (e.g., preventing admission in hospital and nursing home)

The elderly feel protected and cared for therefore, motivated to continue their active life at home.

Reduction of care stress, the demand of care resources and of the burden of care, by the formal and informal caregivers.


Mean: 7.57 SD: 2.68


Mean: 8.20 SD: 1.66






2.2.6 Danger Situations Adviser

Partners Involved: UniGe



Improves safety (e.g., by detecting a possible hazardous situation) thus preventing a possible accident and prolonging the elderly’s life in their home environments (e.g., preventing admission in hospital and nursing home)


Reduces stress; demand of care resources; and the burden of care by the formal and informal caregivers.


Mean: 6.61 SD: 3.20


Mean: 8.60 SD: 1.50



2.2.7 Call for Help

Partners Involved: AIT, CITARD, Fh-IGD, UniGe



Risk Explanation: Minor technical difficulties may occur when developing the phone line based dispatching technology

Risk Mitigation: Use of existing cloud based dispatching services (e.g., Nexmo.com)

Improves safety (e.g., by intervening in case of an emergency) thus preventing a possible health and safety situation and prolonging the elderly’s life in their home environments (e.g., preventing admission in hospital and nursing home)


In addition, the elderly is reassured and informed at every stage of the communication of the system with his/her caregivers, thus increases the satisfaction of the elderly in using the system

Facilitate communication, between the elderly, the formal/informal and other users of the Miraculous-Life system therefore, the elderly feels secure.

Promote communication and collaboration between the elderly and formal caregivers, in order to efficiently assist the elderly in an emergency situation.

Reduction of care stress from formal and informal caregivers.


Mean: 8.11 SD: 1.42


Mean: 9.52 SD: 0.71






2.2.8 Windows Reminder

Partners Involved: AIT, Fh-IGD, UCY, UniGe



Risk Explanation: 1) Minor technical difficulties may occur when integrating certain commercial window sensors.

2) If power is lost, sensor’s connection to the workstation will be lost.

Risk Mitigation: 1) Use of another type/brand of commercial window sensor.

2) If connection with the sensor is lost, the elderly/caregiver/administrator will be notified.

Improves healthy leaving (e.g., by letting in fresh air) thus preventing a possible health hazard, prolonging the elderly’s life in their home environments (e.g., preventing admission in hospital and nursing home).

The elderly feel cared for therefore, motivated to continue their active life at home. In addition, it Prevent forgetfulness.

Stimulate elderly’s independence and thereby reduce care burden and care stress in formal and informal caregivers.


Mean: 5.25 SD: 2.56


Mean: 7.12 SD: 1.88



2.2.9 Sleeping Reminder

Partners Involved: AIT, UCY, UniGe



The elderly feels cared for, thus motivated to continue leaving at home. Motivate the elderly to sleep on the bed, improve thus both, quality of sleep and autonomy that can have a direct effect on everyday activities. In this way promotes an active lifestyle.

Stimulate independence and thereby reduce care burden and care stress in formal and informal caregivers.


Mean: 4.04 SD: 2.27


Mean: 6.68 SD: 2.12






2.3.1 Agenda




Motivate the elderly to remain active and independent by facilitating social.

Assisting the elderly to create and be reminded of social activities, preventing thus loneliness and social isolation.

The elderly can organize their life and free time better, improving thus their autonomy, independence and confidence.

Prevent loneliness and therefore improve efficiency and continuity of integrated care provision to the elderly at home.



Mean: 6.25 SD: 2.28


Mean: 8.16 SD: 1.75



2.3.2 Event/Group activities

Partners Involved: UCY, UniGe



Motivate the elderly to remain longer active and independent by introducing group activities/events, thus encouraging the elderly to get socially involved and prevent inactivity.

Facilitate social interaction, between the elderly, the formal/informal caregivers and other users of the Miraculous-Life system.

Motivate the elderly in participating to the activities organized by the animation department, preventing thus loneliness and prevent early capability degradation by keeping the elderly active.

In addition, the elderly feels independent and autonomous by managing his/her own social interaction.

Promote communication and collaboration between the elderly and caregivers, in order to efficiently support the elderly through his/her daily activities.




Mean: 6.11 SD: 2.29


Mean: 8.56 SD: 1.16






2.3.3 Appointment Reminder




The elderly remains reassured and not stressed that he might forget an appointment. Motivates the elderly to remain active, by attending planned events and activities.

It prevents forgetfulness, allowing the elderly to feel autonomous, active and competent.




Mean: 6.89 SD: 2.66


Mean: 8.60 SD: 1.12



2.3.4 Object Location Assistance and Reminder

Partners Involved: Fh-IGD, UniGe



Risk Explanation: Identification of (very small) objects by Kinect camera is not accurate or not possible.

Risk Mitigation:

The elderly feels empowered by having assistance to locate items in their household. This motivates the elderly to continue with everyday activities and not feeling demotivated. Independence is also promoted since the elderly does not need to ask for help from formal/informal caregivers.

Reduction of the demand of care resources and of the burden of care by the formal and informal caregivers.

Reduction of care stress in formal and informal caregivers.


Mean: 6.39 SD: 3.41


Mean: 7.88 SD: 1.69




Restrict the identification to a set of objects that can be identified with acceptable accuracy by the Kinect camera; Colored markers may be used for some of the objects.

In case the objects that may be useful for the elderly to detect, cannot be recognized by the Kinect camera, another technology may be adopted. E.g., Colored markers may be used for some of the objects.



2.3.5 Notification Service




Facilitate communication, between the elderly and the formal/informal caregivers.

Improves the autonomy of the elderly informing them about certain situation and allowing the elder to decide for the action to be taken.

Promote communication and collaboration between the elderly and caregivers, in order to efficiently assist the elderly and support him/her through his/her daily activities.



Mean: 6.18 SD: 2.52


Mean: 7.84 SD: 1.95



2.4.1 Meal Preparation




Assists the elderly to prepare a meal and provides access to ingredients and recipes procedures. Promotes healthy leaving, helps the elderly to remain active and have autonomy and independence with respect to his dietary selections.

The elderly thus can be energetic and remains active for longer at home.






Mean: 3.82 SD: 2.41


Mean: 6.56 SD: 2.20






2.5.1 Motivation for Physical Activity

Partners Involved: Fh-IGD, Noldus, UCY, UniGe



Risk Explanation: 1) In a real-life environments the emotion recognition and/or behavior analysis, may be inaccurate (in this case the service will not be triggered) ;

2) Sensors are not able to cover the entire space of the residence, therefore there may be significant gaps in the recognized behaviors;

Risk Mitigation: 1) Build a personal emotion profile which will be used as reference for the emotion measurements (which will carried out for a certain time period instead of instantaneous); This will improve the reliability of the recognized emotion.

2) Revise behavior analysis module algorithms based on recognition results using real data in order to improve the recognition rates and models.

3) Introduce low cost sensors (ex. PIR, pressure) to help cover the gaps;

Stimulate and motivate the elderly in performing physical activities therefore, remain longer active and independent.

Physical activity helps the elderly to keep fit and able to continue performing his daily activities and prevents early degradation of skills and capabilities.



Mean: 5.89 SD: 2.96


Mean: 6.88 SD: 1.94






2.5.2 Physical Activity Service

Partners Involved: MRPS, Orbis, UniGe



Risk Explanation:

The physical exercises in the videos are not appropriate for the elderly.

Risk Mitigation:

Film videos specifically designed by experts (e.g., physiotherapists) in the two care-giver organizations.

The elderly is provided with specific activities suitable for his/her capabilities and potentials. Thus, he is not getting demotivated or possibly injured by attempting activities that he/she cannot perform.

Promotes independence by allowing the elderly to select from the predefined activities therefore, improve their confidence.



Mean: 6.46 SD: 2.42


Mean: 6.84 SD: 2.08



2.5.3 Social Bonding

Partners Involved: CITARD, Noldus, UCY, UniGe, Noldus



Risk Explanation:

In real life environments, where we don’t have control over important factors like: ambient (sun or other) light, the location, position and direction of an elder person, may result in inaccurate emotion recognition;

Social interaction prevents isolation and depression.

This is facilitated using emotions and expressions of the avatar that helps the elderly feel cared for, important and makes his/her communication with the system satisfying and natural.

Allows the elderly to get the impression of having a company, thus they do not feel lonely and isolated.

The elderly feel better understood and develop a sense of security of leaving alone in their home for longer.


Reduction of care stress in formal and informal caregivers.


Mean: 5.04 SD: 3.42


Mean: 7.64 SD: 2.15




Risk Mitigation:

Build a personal emotion profile that will be used as reference for the emotion measurements (which will carried out for a certain time period instead of instantaneous); This will improve the reliability of the recognized emotion.





3.5 Final Remarks

This chapter presented a cost-benefit analysis of the use-cases with the perspective of enabling a prioritized effort in their implementation in subsequent work packages. We detailed the revision process of the use cases since their initial version in D1.2a. We benefited from the pre-trials and performed a deep feasibility analysis to create meaningful and valuable use-cases. All technical partners were deeply involved in the process, thus minimizing the risks of failure while considering technological or resource limitations.

In order to perform the Cost-Benefit analysis of each use case we identified the Cost based on a technical perspective provided by all the involved technical partners, and identified the Benefit of each by considering both qualitative and empirical perspectives regarding the relevance of the use cases and the need they address. This information was compiled and presented in Table 2.

In order to make an overall assessment of the Cost-Benefit of all use cases we created the following matrix diagram (Figure 6), in which we present the Cost and Benefit variables according to their previously identified empirical perspective. The Cost-Benefit matrix diagram can be used to target selected use cases in the context of specific actions and changes during the project lifetime.

If it is identified later in the project that there is a need to simplify some use cases due to unforeseen resource constrains it is suggested to consider use cases 2.2.9 and 2.4.1 which have the less benefit for the users. Use cases 2.2.4, 2.2.5, 2.3.4, 2.5.1 and 2.5.3 could also be considered for simplification, even though their benefit is higher than others, as a number of shortcomings are identified to be present. Therefore, it is expected that during the development of the use cases some of the use cases might be (slightly) simplified in order to overcome any feasibility constrains identified, but without affecting the objectives and the goals foreseen to be achieved by the use cases.

This thorough Cost-Benefit analysis enables targeted and prioritized development effort for the use-cases by the relevant work packages.



Figure 6: Cost-Benefit matrix diagram based on the empirical analysis of the use cases



4 User Interface definition

This chapter presents the analysis of the interaction requirements needed to specify the Human-Computer interface that is planned in Task 1.3, providing input to the multi-modal interface to be defined in WP2. The majority of information collected in Task 1.2 describes how emotion is used in human-to-human communication. In this chapter, we take the results of Task 1.2 as input and adapt to how humans will interact with the VSP.

The defined user interface and the interaction design are based on the user scenarios and use cases described in chapter 2 (User scenarios and use cases). The aim of the user interface and interaction design, provided in this chapter, is to make interaction between the elderly person and the VSP possible and also assist in performing all the defined use cases described above. In addition, the pilot results described in D6.4a, specifically the recommendations for the interface and design, were an important input to improve the user interface.

Note: The intention is to define the main user interface and to define guidelines. The few services screens shown here are just for illustrating how the user interface could look like. The guidelines and general ideas shown in this chapter should be sufficient to define the service screens when they are developed.

4.1 Human-Computer Interaction analysis

To make the interaction of the elderly person with the VSP as natural and easy as possible it is the common thought that the main interaction will be based on the speech of the elder person. Commands will be filtered from the spoken words and will be used to control the interaction with the VSP. The VSP will use speech as the main interaction with the elderly person, using text to speech technology, to support and simulate the natural spoken interaction between two persons as much as possible. To enhance and support the interaction the spoken text of the VSP will also be shown on the screen(s).

As mentioned in the outcome of the user needs the VSP should have a first name of how the elderly can call him. The elderly prefers also to be addressed by the VSP by his first name. This makes the interaction more natural and intuitive between the elderly and the VSP.

The avatar has 3 different interaction modes as described in D1.1a:

a) Listening mode

b) Waiting mode

c) Talking mode

The avatar should not only be able to speak (active, talking mode) but also have passive states to appear as attentive (listening mode) and to be present without being engaged in a conversation (waiting mode).

The system should be adaptable for the elderly. This means for example that the elderly will decide at the beginning (but also during the use of the system) which services, described in the above scenarios, are active in his personal VSP and which services are not.



The volume of the speech of the VSP should be modulated by the environmental factors: if the environment is quite (e.g., the elderly is reading), the volume of the VSP should be relatively low; if the environment is noisy (e.g., the elderly is watching on TV), the volume of the VSP should be relatively high.

The expression of the avatar in response to the emotion of the elderly is described in the emotion component in the User scenarios and use cases. There are defined 7 emotional states of the avatar:

1) Happy,

2) Sad,

3) Worried,

4) Relieved,

5) Compassionate (expresses sympathy),

6) Directive behaviour and

7) Neutral.

The VSP should be able to instantiate spontaneous interaction. At random times (when the user is present), the VSP could ask: “How are you?” to stimulate interaction. Also the VSP should show some movement with the body, head and/or eyes (blinks) to avoid a static non-natural appearance.

4.2 Human-Computer interface specification

4.2.1 Introduction

A first design was made and described in D1.2a and tested in the pre-trial. The design of the user interface required a major update to take into account the recommendations and suggestions made on the User Interface and the interaction design in D6.4 Pilot acceptance evaluation results. The main recommendations were:

1) VSP too big in main screen

2) VSP too small in service screen

3) Bigger clock desired

4) Bigger texts required

5) Fresher colours for more positive feelings

6) Better contrasts between texts and backgrounds

7) Add instruction: “Open…”

8) Button for previous step

9) Breadcrumbs

10) No other service can be opened directly when a service is active

Besides this, since the main interface device will be a tablet (the Samsung Galaxy Tab 4, 10.1” has been selected) the design has been updated to take into account the guidelines for touch interaction. Some design decisions are pointed out in this section:



The Samsung Galaxy Tab 4, 10.1” has a display resolution of 1280 x 800 pixels‡.

This means that the tablet display shows 149 Dots Per Inch (DPI)§.

149 DPI is considered an MDPI screen for which 1 DP (Density Independent Pixel)

can be considered as 1 pixel**. Therefore the designs in the following sections are

designed with a resolution of 1280 x 800 pixels. So what you see here is what you

can expect to see in the tablet too.

Guidelines on optimal button sizes are described on the Android Developer

website††. This website describes that 48 DP (Density Independent Pixels)

translates to a physical size of about 9 mm (with some variability). This is

comfortably in the range of recommended target sizes (7-10mm). The 48 DP

includes 4 DP spacing to the sides, so the actual button is 40 DP (equals 40 pixels)

only. The designs provided in this document use 48 pixels (equals 48 DP) plus 4

pixels (equals 4 DP) margin to each side. For the spacing between buttons 8 DP is

used (this equals two times the margin around the buttons).

The Android Developer website‡‡ also recommends some text sizes, however

larger text sizes are used so that elderly can read the button texts more easily. The

largest text in the Android Developer guidelines are: 22 SP (Scale Independent

Pixel) but in the design text-size 32 pixels (equals 32 SP) is used for the button

texts.

The design only allows a landscape alignment. A portrait alignment should be

prevented.

Note: A tablet or screen with different resolution or DPI will not show the user interface

correctly (it will be distorted), unless it was taken into account in the implementation of the

UI for the system.

4.2.2 Interaction design

The interaction between the elderly and the VSP will be guided with a screen that shows the avatar and some buttons which provide the services described in chapter 2 (User scenarios and use cases). The avatar represents the virtual partner and it should show as natural behavior as possible. That means the VPS should visually move or blink with the eyes once in a while, show emotions as described in the uses cases, and verbally it should speak the language of the elderly (for our pilot sites it is Dutch or French) as natural as possible.

Direct response on speech, activity and behavior (as described in the use cases) will be needed to have a natural interaction between the elderly and the VSP as much as possible. We can distinguish two modes of interaction:

1) VSP Main mode

‡ Samsung website (Accessed: 09-12-2014): http://www.samsung.com/us/mobile/galaxy-tab/SM-T530NZWAXAR

§ DPI Love website (Accessed: 09-12-2014): http://dpi.lv/#1280×800

** Skinkers website (Accessed: 09-12-2014): http://labs.rampinteractive.co.uk/android_dp_px_calculator/

†† Android Developer website (Accessed; 09-12-2014): http://developer.android.com/design/style/metrics-grids.html

‡‡ Android Developer website (Accessed: 09-12-2014): http://developer.android.com/design/style/typography.html

http://www.samsung.com/us/mobile/galaxy-tab/SM-T530NZWAXAR

http://dpi.lv/#1280×800

http://labs.rampinteractive.co.uk/android_dp_px_calculator/

http://developer.android.com/design/style/metrics-grids.html

http://developer.android.com/design/style/typography.html



2) VSP Services mode

For the main mode a large main screen will be shown with a large avatar and this will be shown initially and also whenever a use case/a service for the user have ended. Most of the time, this screen will be the one that shown when the system is active. In the service mode the focus is on the service itself, and the service will use a large area of the screen, the avatar is much smaller and only a few important buttons (functions) will be shown.

The following functions are available in both modes and have a fixed position such that the user can easily find them, they are:

Open/Close clock

Activate/Deactivate VSP

Configure speech

Emergency call.

The next two paragraphs describe the design and interaction of both modes in more detail.

4.2.3 VSP Main screen

From the main screen the user can see which services and functions are available. It also serves as memory list and supports the interaction because the user will see it each time. Each of the services can be activated by calling “Open” and the name of the service. For example, “Open Agenda” will open the agenda service, as described in detail in chapter 2 (User scenarios and use cases). Note also the word “Open” appeared at the top of the buttons as additional reference. Naturally, the buttons can also be touched to open the services.

Figure 7: VSP Main screen



Compared to the layout of the main screen presented in D1.2a, the layout described in this second version of the deliverable (D1.2b) has been changed. The size of the avatar component is now reduced to give room to a larger services menu area. The position of Services menu has been moved to the right for two reasons:

a) The avatar is considered the most important part of the system and should be on the left side since that normally will be seen first assuming most users are reading from left to right and top-down (in a so called F-pattern),

b) It’s more convenient to have the service buttons on the right side assuming most users are right-handed.

The colours of the buttons have been changed into blue with white characters (following Miraculous Life colour schemes). The caregivers have the experience that it is easier to explain to the elderly that the blue coloured “things” on the right are buttons which they can press to activate a service. The elderly recognize them more easily as buttons in comparison with the light coloured buttons of the initial design. The contrast of the button and font colour has been tested (with separate contrast measuring software tool) to ensure readability by the elderly.

The Miraculous Life logo is shown transparent in the left top corner of the avatar window. The analogue clock will not be shown continuously anymore but the user can ask the time or press the “Open clock” button to get the analogue clock visualized. This is done to gain more space for the smaller avatar and the buttons (see Figure 8).

Note it will be good if the clock would contain more detail but that is not shown here since it’s not the goal to define the detailed interface here.

Two additional buttons are placed in the lower right corner:

1) To switch between active and passive mode (Activate / Deactivate VSP)

2) To configure VSP speech.



Figure 8: Show clock

4.2.4 VSP Services screen

As can be seen in Figure 9, the layout of the VSP Service screen has been changed together with the main screen to form a coherent setup for the user. The avatar is more prominent because it has a bigger area now. The 4 buttons on the lower right part of the Service menu will be shown always and have a static position. Switching between the screens will not result in a restless interface. On top there are two buttons:

1) A “Return to the main service screen” button to allow quick navigation when this needed.

2) A “Close the service” button to return to the main VSP screen.

The name of the service will be shown on these buttons, i.e., “Return to agenda”, “Close Agenda” to have extra feedback on which service has been active at the moment.

In case a service is activated, the avatar will move from its central position to the right position in the services menu. Depending on the selected service only the face will be shown to make the interaction, especially speech and expression more clear to the user or the complete avatar as shown in Figure 9.

Note that the Miraculous Life logo (see Figure 7) is removed from the avatar screen to make space for the avatar. Since the VSP is mainly a speech based system, the user should receive some feedback on what texts/commands to use. The possible options are presented to the user in a similar button style as the services menu uses. By presenting commands in a consistent manner, users will recognize easily commands that can be used to proceed.



Figure 9: VSP Services screen - Example of the Agenda service

Figure 10: Agenda screen - Activity



Using a consistent button style is particularly handy when the choices are “dynamic”. For example, a screen with an activity where the user can select to participate or not. Where the “Participate” button has been coloured green and the Cancel button neutral (for example see Figure 10).

4.2.5 Deactivate VSP screen

In the main screen, the Deactivate VSP button switches the VSP system into passive mode and from the service screen the Activate VSP button switches the system back into active mode. See Figure 11 for a detailed description.

The screen shows a screen with a lock (or something similar) to indicate that the system is in passive mode now. The other services are disabled and greyed out to indicate that they cannot directly be reached; activation of the VSP will be necessary first.

The main functions will remain accessible when the VSP has been deactivated.

Figure 11: Passive / Deactivated VSP screen

4.2.6 UI Definition and Guidelines

Main screen definitions:

The VSP size = 800 x 800 pixels

The menu width = 480 pixels

The menu buttons height is: 48 pixels

The menu buttons width is: 464 pixels



The menu button icons are 32 x 32 pixels

The menu button texts are 32 pixels

All texts in the interface are Roboto (regular)

The spacing between buttons is: 8 pixels

The service screen definitions:

The service shows breadcrumbs (text size = 32 pixels, transparency 60%)

The sizes of the instruction- and other texts are 24 pixels

The button height in the service is 48 pixels

The button width in the service depends on the content; in the example 200 pixels

In order to get a consistent look and feel the following guidelines has been defined:

All texts

o Font style = Roboto regular

Breadcrumbs

o Font size = 32px

o Font / icon colour = RGB (52,73,94) or Hex (#34495E)

o Font transparency: 60%

Regular texts

o Font size = 32px

o Font colour = RGB (52,73,94) or Hex (#34495E)

Buttons (not selected)

o Background colour = RGB (241,173,198) or Hex (#F1ADC6)

o Font / icon colour = RGB (255,255,255) or Hex (#FFFFFF)

Selected buttons

o Background colour = RGB (17,74,43) or Hex (#114A2B)


Emergency button

o Background colour = RGB (151,0,12) or Hex (#97000C)


Negative response button (e.g. Cancel) , similar to not selected button colour



o Background colour = RGB (241,173,198) or Hex (#F1ADC6)


Positive response button (e.g. Participate)

o Background colour = RGB (30,130,76) or Hex (#1E824C)




5 Conclusions

To bring user context to life, five (5) different user scenarios and a total of twenty-three (23) use cases are developed, giving detailed realistic examples of how users carry out their daily life activities at home. The aim of the use cases described is to illustrate in detail how the users interact with the VSP for a specific service. As the speech recognition and buttons in the screen provide the means for the user to interact with VSP, the use cases described illustrate also how this interaction works.

In this second version of D1.2 (D1.2b) the end-user organizations, in close collaboration with the technical partners involved in each use case, refined the existing use cases heavily and defined additional use cases to match the user scenarios, as much as possible, with reality. Also the different states describing the work flow of use cases are described in more detail providing also, when applicable, the emotional expression of the avatar. Moreover, the different use cases were discussed closely together with all the technical partners, who assessed their feasibility and importance for the project and parts were taken out in order to focus on the most important aspects. Two new use cases, described in Use case: Windows reminder (Household Adviser) and Use case: Sleeping reminder (Household Adviser) were added, requiring additional sensors to be used by the system in order to detect that the elderly is not in bed or if a window is open or not.

A cost-benefit analysis of all the use cases was also presented in this document. We detailed the process for building the use cases and identified the relevance of the needs they addressed. The technical partners were included in the revision process of the use cases thus allowing a strong feasibility analysis and validating all use cases from the technical perspective. A questionnaire was created and filled by the elderly and professionals of both end-user organizations and the results demonstrated that all use cases were considered meaningful and relevant. A detailed qualitative analysis was performed to assess the relevance of each use case. Based on these results a strong development effort can target selected use cases by the relevant technical work packages.

An important aspect of the VSP is its user interface as it determines the way the elderly or caregiver interacts with the system. Although we have quite some advanced sensor technology available to recognize activities, capture speech and the emotional state, the VSP should respond to these measurements. The priority of responses, the type of responses and the appearance of the avatar have been described. The interface of the VSP that is visible on the screen must support the natural interaction. If speech recognition fails, the interface must provide buttons with which the user can still indicate his or her choice.

The results obtained from the pre-trial tests were used to improve the user interface and provide a smooth interaction between elderly and the VSP. The main screen and the service screen layout have been updated to match with the use cases and the recommendations of D6.4a. This led to a drastic update of the user interface with a different layout, different colours and a guideline that defines details of the user interface, which should make it possible to define the service screens when they will be developed.

This deliverable describes in detail the different use cases and defines the user interface and interaction design guidelines that should be considered as an important input to all the technical work packages (WP2, WP3, WP4 and WP5) during the development of the Miraculous Life system.



Appendix A Questionnaire for prioritizing the services of the Miraculous-Life system

QUESTIONNAIRE FOR PRIORITIZING THE SERVICES OF THE MIRACULOUS LIFE

SYSTEM

Aim of the questionnaire

The aim of this questionnaire is to prioritize the service we have planned to integrate in the final version of the Miraculous Life solution. Your input is extremely valuable to us!

Miraculous Life concept

The main aim of the Miraculous-Life project is to design, develop and evaluate an innovative usercentric technological solution, the Virtual Support Partner (VSP), attending to the elder (65+) daily activity and safety needs, while the elder goes about his normal daily life. The VSP will provide implicit daily activities support which is based on behaviour and emotional understanding and appropriate respond exhibiting distinctive emotions, deliver in a human like way simulating in essence the interaction with a real life partner.

Figure 12. Mary: the Virtual Support Partner (VSP) of the Miraculous Life system.



Part1 – General questions

A1. Sex:

Male

Female

A2. What is your profession?

Care coordinator

Nurse

Nursing auxiliary

Animation coordinator

Concierge / service to the residents

Management

Physiotherapist

Elderly care physician

Other, namely ………………



Part2 – Service questions

1. Contact list: the senior can view the list of his/her contacts (friends, relatives, and caregivers), establish a free telephone conversation (thought the ML system) and write a new message to one of his/her contacts.

Please rate the relevance of this service from 1 (not a priority) to 10 (essential priority): _____

2. Message System: the senior can view the messages received in the past and write a new message to one of his/her contacts.


3. Shopping Assistance: the senior can add or remove items from his/her shopping list and send the shopping list by message (to an informal caregiver).


4. Medication Reminder: At a set times, the VSP reminds the senior to take his/her medication. The senior can (1) postpone the reminder, (2) inform the system that he/she took the pills or (3) inform the system that he/she does not want to take the medication.


5. Wake-up Calls: the senior can set the alarm clock. The alarm clock will be triggered in the morning at the time selected by the senior.


6. Periodic Advice: Periodic advices are messages (written by caregivers, or receptionists, or animators) that are periodically read by the VSP. For example, each day at 2 p.m. the drink reminder could be triggered by the system: “Dear Nicole, the weather has been so hot lately. It’s important to stay adequately hydrated. Don’t forget to drink a glass of water from time to time, even if you don't perceive it necessarily”.


7. Mode of the system: The senior can change the mode of the Miraculous Life system (active vs. passive). In the active mode, all the services of the ML are available. In the



passive mode, the system is deactivated meaning that (1) the senior cannot interact with the system and (2) the reminders, period advices aren’t triggered by the system.


8. Configure the VSP Speech: the senior can configure the speech of the VSP (i.e. ask her to speak louder, softer, faster, or slower).


9. Fall Detection: the system is able to detect a fall event in the apartment of the senior and alert the appropriate person (caregivers) in case of need.


10. Dangerous Objects Adviser: the system is able to detect dangerous objects on the floor that could potentially cause a fall. When a dangerous object is detected by the system, the VSP warns the senior for obstacles, reducing the risk of fall.


11. Oven and stove reminder: the system is able to detect if the senior is cooking on his/her apartment. When the user is cocking, the VSP reminds him/her to switch off the stove and the oven.


12. Call for Help: the senior can establish a phone conversation with caregivers when needed (emergency call).


13. Windows reminder: each morning, the system check if the senior have opened the window of his/her apartment. The VSP reminds him/her to open the windows, if the user forget it.




14. Sleeping reminder: the system is able to detect if the senior is not in the bed during the night. If the user is not detected in the bed, the VSP suggests him/her to go to sleep (at a set time).


15. Agenda: the senior can consult his/her agenda, add or remove items. In addition, the senior can create social activities (i.e. inviting friend or relatives to join activities).


16. Events/Group activities: the senior can consult and subscribe to the activities organized by the institution.


17. Appointment Reminder: the VSP reminds the senior of his/her appointments and activities before the events begins. The senior can postpone or delete the reminder.


18. Object Location Assistance and reminder: the system is able to find the objects lost by the senior in the apartment. If the system fails in locating the object lost, the VSP reminds the senior the places where he/she is used to put the object.


19. Notification Service: Notifications are messages (written by caregivers, or receptionists, or animators) that are read by the VSP. For example: “Mr. Smith has recently moved in MRPS... and he is your new neighbour! Why not invite him for a coffee?” The senior can postpone or delete the notification.


20. Meal preparation: the senior can consult recipes that were selected according to his/her dietary requirements and allergies.




21. Motivation for Physical Activity: the system is able to detect if the senior is not performing sufficient physical activity. If needed, the VSP motivates the senior in performing physical activities.


22. Physical Activity Service: the senior can watch short video clips demonstrating physical exercises that he/she can performing at home.


23. Social Bonding: the VSP provides company to the senior by interacting in an empathic way and considering his/her emotional and health status.




Appendix B Table with extended details for the feasibility analysis of the use-cases

Use-case

Description Subtasks or Services

Associated questionnaire section — from

D1.1a B1

Associated mean value

(1 means "very important") taken

from D1.1a

Ranking

(1 means highest priority)

Project Goals

Ranking


Final Ranking


Technical partners involved in the use-case

revision

2.1.1 Contact list Co-Net service

C5 1,8 3 2 11 Citard

Calling service AIT (Fh-IGD)

2.1.2 Message System Co-Net service C5 1,8 3 2 11 Citard

2.1.3 Shopping Assistance Calling service

C15 1,79 2 2 10 UCY

SMS Server AIT

2.1.4 Medication Reminder Medication service

C2 2,21 9 1 4 UCY

Notification service Citard

2.1.5 Wake-up Calls Agenda service

(Identified by WP4) 2 21 UCY

Notification service Citard

2.2.1 Periodic Advice Agenda service (Identified in the DoW) 1 6 UCY, Citard

2.2.2 Mode of the system: Active vs Passive Mode

Passive mode (Identified in the DoW) 2 21 UniGe

2.2.3 Configure the VSP Speech Dialogue Management (Identified in the DoW) 1 6 UniGe

2.2.4 Fall Detection Safety service C21 2 5 2 13 Fh-IGD

2.2.5 Dangerous Objects Adviser Safety service C21 2 5 2 13 Fh-IGD

2.2.6 Dangerous Situations Adviser Safety service C21 2 5 2 13 Fh-IGD

2.2.7 Call for Help Acquisition motion capturing

C21 2 5 2 13 Fh-IGD

High-Level activity detection UniGe



Use-case

Description Subtasks or Services

Associated questionnaire section — from

D1.1a B1

Associated mean value

(1 means "very important") taken

from D1.1a

Ranking


Project Goals

Ranking


Final Ranking


Technical partners involved in the use-case

revision

Safety service Fh-IGD (with AIT)

2.2.8 Windows reminder

Acquisition (PIR sensors)

C13 2,67 18 2 19

AIT

High-level activity recognition UniGe

Household adviser service UCY

2.2.9 Sleeping reminder

Acquisition (PIR sensors)

C13 2,67 18 2 19

AIT

High-Level activity recognition UniGe

Household adviser service UCY

2.3.1 Agenda Agenda service C3 1,73 1 1 1 UCY

2.3.2 Event/Group activities Agenda service C14 2,33 11 1 5 UCY

2.3.3 Appointment reminder Agenda service C3 1,73 1 1 1 UCY, Citard

2.3.4 Object Location Assistance and Reminder Object localization service C12 2,13 8 2 17 UniGe

2.3.5 Notification service Co-Net service (Identified by WP4) 1 6 Citard

2.4.1 Meal preparation Meal preparation service (Identified in the DoW) 2 21 UCY

2.5.1 Motivation for Physical Activity Guidance service

C16 2,07 7 1 3 UniGe

Memory service UniGe

2.5.2 Physical Activity Service Guidance service C9 2,47 13 2 18 UniGe

2.5.3 Social bonding

VSP service

(Identified in the DoW) 1 6

UniGe

Reaction on persisting affective states

UniGe, Noldus



Appendix C Ranking of Miraculous-Life use cases, based on the Relevance Questionnaire - Both end-users organizations (Caregivers = 25, Elderly = 14)

Caregivers Elderly

Use Case Mean STD Use Case Mean STD

B12. Call for Help 9,52 0,71 B8. Configure the VSP Speech 8,54 1,12

B9. Fall Detection 9,32 0,90 B12. Call for Help 8,11 1,42

B4. Medication Reminder 8,84 1,34 B9. Fall Detection 8,00 2,48

B1. Contact list 8,64 1,22 B7. Mode of the system 7,61 2,44

B11. Oven and stove reminder 8,60 1,50 B10. Dangerous Objects Adviser 7,57 2,68

B17.Appointment Reminder 8,60 1,12 B5. Wake-up Calls 7,11 2,10

B16. Events/Group activities 8,56 1,16 B17.Appointment Reminder 6,89 2,66

B8. Configure the VSP Speech 8,39 1,57 B11. Oven and stove reminder 6,61 3,20

B2. Message System 8,20 1,76 B22. Physical Activity Service 6,46 2,42

B10. Dangerous Objects Adviser 8,20 1,66 B6. Periodic Advice 6,39 2,35

B15. Agenda 8,16 1,75 B18. Object Location Assistance and reminder

6,39 3,41

B18. Object Location Assistance and reminder

7,88 1,69 B4. Medication Reminder 6,25 3,00

B19. Notification Service 7,84 1,95 B15. Agenda 6,25 2,28

B3. Shopping Assistance 7,71 1,78 B19. Notification Service 6,18 2,52

B6. Periodic Advice 7,68 2,12 B16. Events/Group activities 6,11 2,29

B23. Social Bonding 7,64 2,15 B1. Contact list 6,07 2,50

B5. Wake-up Calls 7,60 1,83 B2. Message System 5,89 2,70

B13. Windows reminder 7,12 1,88 B21. Motivation for Physical Activity 5,89 2,96

B21. Motivation for Physical Activity 6,88 1,94 B3. Shopping Assistance 5,61 2,53

B22. Physical Activity Service 6,84 2,08 B13. Windows reminder 5,25 2,56

B7. Mode of the system 6,71 2,66 B23. Social Bonding 5,04 3,42

B14. Sleeping reminder 6,68 2,12 B14. Sleeping reminder 4,04 2,27

B20. Meal preparation 6,56 2,20 B20. Meal preparation 3,82 2,41



Appendix D Ranking of Miraculous-Life use cases, based on the Relevance Questionnaire - MRPS (Caregivers = 11, Elderly = 7)

Caregivers Elderly


B9. Fall Detection 9,91 0,30 B8. Configure the VSP Speech 9,36 0,94

B12. Call for Help 9,82 0,60 B10. Dangerous Objects Adviser 9,14 1,57

B17.Appointment Reminder 9,27 0,90 B9. Fall Detection 9,00 1,91

B1. Contact list 9,09 1,22 B17.Appointment Reminder 8,79 0,57

B16. Events/Group activities 9,09 1,22 B4. Medication Reminder 8,21 2,23

B2. Message System 8,91 1,76 B11. Oven and stove reminder 8,07 2,39

B4. Medication Reminder 8,91 1,70 B12. Call for Help 8,07 1,69

B11. Oven and stove reminder 8,91 1,70 B18. Object Location Assistance and reminder

8,07 3,01

B15. Agenda 8,82 1,66 B5. Wake-up Calls 7,93 2,17

B19. Notification Service 8,55 1,75 B7. Mode of the system 7,64 2,21

B6. Periodic Advice 8,45 2,62 B22. Physical Activity Service 7,64 1,89

B8. Configure the VSP Speech 8,34 1,99 B15. Agenda 7,50 1,61

B5. Wake-up Calls 8,27 1,90 B6. Periodic Advice 7,36 2,36

B13. Windows reminder 8,27 1,62 B16. Events/Group activities 7,36 1,25

B10. Dangerous Objects Adviser 8,18 1,89 B21. Motivation for Physical Activity 7,21 1,91


7,91 1,81 B1. Contact list 7,00 2,00

B3. Shopping Assistance 7,50 2,27 B3. Shopping Assistance 6,93 2,28

B23. Social Bonding 7,33 3,20 B19. Notification Service 6,93 1,97

B20. Meal preparation 7,09 2,12 B2. Message System 6,64 2,66

B22. Physical Activity Service 7,00 2,05 B13. Windows reminder 6,64 2,06

B7. Mode of the system 6,30 3,47 B23. Social Bonding 6,42 3,75

B21. Motivation for Physical Activity 6,30 2,21 B14. Sleeping reminder 5,36 2,36

B14. Sleeping reminder 6,27 2,80 B20. Meal preparation 5,21 2,20



Appendix E Ranking of Miraculous-Life use cases, based on the Relevance Questionnaire - Orbis (Caregivers = 14, Elderly = 7)

Caregivers Elderly


B12. Call for Help 9,29 0,73 B12. Call for Help 8,14 1,21

B9. Fall Detection 8,86 0,95 B8. Configure the VSP Speech 7,71 0,49

B4. Medication Reminder 8,79 1,05 B7. Mode of the system 7,57 2,82

B8. Configure the VSP Speech 8,43 1,22 B9. Fall Detection 7,00 2,71

B11. Oven and stove reminder 8,36 1,34 B5. Wake-up Calls 6,29 1,80

B1. Contact list 8,29 1,14 B10. Dangerous Objects Adviser 6,00 2,71

B10. Dangerous Objects Adviser 8,21 1,53 B6. Periodic Advice 5,43 2,07

B16. Events/Group activities 8,14 0,95 B19. Notification Service 5,43 2,94

B17.Appointment Reminder 8,07 1,00 B22. Physical Activity Service 5,29 2,43

B3. Shopping Assistance 7,86 1,41 B1. Contact list 5,14 2,73


7,86 1,66 B2. Message System 5,14 2,73

B23. Social Bonding 7,85 1,07 B11. Oven and stove reminder 5,14 3,39

B2. Message System 7,64 1,60 B15. Agenda 5,00 2,24

B15. Agenda 7,64 1,69 B17.Appointment Reminder 5,00 2,58

B19. Notification Service 7,29 1,98 B16. Events/Group activities 4,86 2,48

B21. Motivation for Physical Activity 7,29 1,68 B18. Object Location Assistance and reminder

4,71 3,09

B5. Wake-up Calls 7,07 1,64 B21. Motivation for Physical Activity 4,57 3,36

B6. Periodic Advice 7,07 1,44 B3. Shopping Assistance 4,29 2,14

B7. Mode of the system 7,00 2,00 B4. Medication Reminder 4,29 2,36

B14. Sleeping reminder 7,00 1,41 B13. Windows reminder 3,86 2,34

B22. Physical Activity Service 6,71 2,16 B23. Social Bonding 3,86 2,85

B13. Windows reminder 6,21 1,58 B14. Sleeping reminder 2,71 1,25

B20. Meal preparation 6,14 2,25 B20. Meal preparation 2,43 1,81

Date post:	17-Apr-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

Miraculous-Life...4.2.6 UI Definition and Guidelines 84 5 Conclusions 87 Appendix A Questionnaire...

Documents