Slide 1 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
© 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
The Story of Our LivesFrom Sensors to Stories in Self-monitoring Systems
Dana Pavel, Vic Callaghan, Anind K. Dey (CMU), Francisco Sepulveda, Michael Gardner
School of Computer Science and Electronic EngineeringUniversity of Essex12 September 2012
Slide 2 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Outline• Motivation and (current) goals• MyRoR system• Summary of experiments
Slide 3 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
MyRoR Background
• Lifestyle management is an important area for individuals and societies
• Most current solutions focus on showing what and less why it happened
• End users are often left out of the loop
Slide 4 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Main goals
• Provide a more comprehensive picture of user lives
• Better understand what information people find interesting
• Create better correlations and visualisations capable to capture such diverse information
• Involve the user in all aspects: from information gathering to processing to usage
Slide 5 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Towards a comprehensive user context
Environmental noise level
Battery level (phone)
Signal strength (phone)
Devices around (BT, WLAN-
based)
Location (GPS, cell ID, country code,
WLAN based)
URLs visited
ECG/Heart rate
3-axis accelerom
eter
Event button (user
pressed)
Email dataCalendar
event
Keystrokes
Application context
Time
Mental context
(interest, focus, etc.)
Availabilitycontext
(people or resource)
Physical context (position, direction,
distance, speed, proximity)
Temporal context
(absolute, relative,
duration)Activity context
Emotional context
Social context(communication,
identity)
Slide 6 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Slide 7 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
System view
Slide 8 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Main challenges• Synchronizing input devices• Different formats for input data• Correlating and interpreting data• Evolving platform in time (over different
situations, over life changes)• Presenting information to end users
– Detailed vs. abstract– Interactive– Evolving
Slide 9 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Desktop interface in MyRoR
Main interface
Abstract, story-based visualisations
Detailedvisualisations
Slide 10 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
MyRoR storiesWhy: • Stories are most natural ways of “representing” information• Stories can bring together different types of recorded
information in a fun, concise and engaging way• Stories require a focus on what is important (“collection of
meaningful events” – Kevin M. Brooks)
How: • MyRoR stories combine various media created based on
interpreting available user information• Take into account user annotationsOne main challenge:• What is ‘meaningful’?
Slide 11 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Story creation processMyRoR DB
Rule Engine/KBs
Slide 12 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Time
Text version
Iconversion
Location-based
background
Example: An event within a story
Slide 13 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Experiments and studies• Online survey – 38 participants
– Main focus: • Explore attitudes towards self-reflection• Test if such system is perceived as useful • Explore issues around story creation and customisation
– http://ieg.essex.ac.uk/myror/survey/intro.php
• Hands-on experiments – 6 participants– Main focus:
• Importance of information• Story creation and sharing• Interactions with the system
Slide 14 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Results from survey (1/2)
Q1: Do you often think back about what happened during the day? Q2: Do you think about what triggered a certain emotion or behaviour?Q3: Do you usually propose any change based on self-reflection?
Slide 15 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Results from survey (2/2)Q5: Do you use (or have used) any self-monitoring systems (e.g., fitness-related)? Q6: Would you find useful having a system as presented in the scenario? Q7: If you were to be using such a system would you like to be able to see a story generated based on your activity data?
Slide 16 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Quick Learnings (1/2)• It’s good to have more contextual information as it helps remember, understand
and reflect on what happened
• The usefulness of information cannot be detached from the context it was recorded in– We cannot create lifestyle management systems that only focus on a certain type of
information
• Need to create systems that can dynamically change the importance of information based on user, situations and evolve over time
• Giving people the means to create annotations yields interesting results!– Difference between implicit vs. explicit meaning– Annotations as means for self-reflection– Need better ways for annotating (e.g., watch, pendant?)
Slide 17 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Quick Learnings (2/2)• The story-based multi-media concept was preferred
over graphs as a quick view into recorded data– Combine narrative with graphic
• Customization is desired but should not take too long– Background picture is important
• Stories should be able to change and evolve– Change focus, add new information
• Fun way of sharing recorded information– Needs means to adapt stories for sharing
Slide 18 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Future…• Further develop the stories
– Dynamic creation based on changing information, interest and audience
• Apply system to various areas– E.g., health area (look into embedding various
psychological/behavioural/persuasive models)
Slide 19 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
© 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Contact info: [email protected] survey: http://ieg.essex.ac.uk/myror/survey/intro.phpWork is performed under PAL project: http://palproject.org.uk
Thank you!
Slide 20 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
© 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Background material
Slide 21 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Examples of information processing
Slide 22 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Remote access to system (phone-blog)
Protected access to
collected information
Updating system with data (photos, blog entries, documents)
Story playing
Slide 23 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Mapping emotional states to colours
Slide 24 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Using colours to represent emotional states
Slide 25 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Visualisations in self-monitoring systems
From Affective Diary work at SICS (Stahl, Hook, etc.)
From Garmin software
Using Google Visualizations
Slide 26 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Daily story
Visualisations for information collected and derived stored in the personal database
Information collected on demand from remote servers
Calendar-based interface
Own notes
Slide 27 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
Based on user-based actions/annotations
Determining “meaningful” events
Based on more interesting context data• E.g., calls/messages, changes in location, heart rate• Allowing end user to select interesting information ! Need to be keep the story short!
Slide 28 © 2009-2012 The Copyright in this document belongs to the PAL Consortium and no part of this document should be used or copied without their prior written permission.
PAL project• This work is part of the PAL project (
http://palproject.org.uk), funded by EPSRC and TSB
• PAL looks at self-monitoring solutions as part of future healthcare scenarios
• PAL also includes aspects of security and privacy in the context of various usages of recorded personal data
