INVESTIGATINGSELF-REPORTING BEHAVIOR
IN LONG-TERM STUDIESAndreas Möller ✽, Matthias Kranz ❖,
Barbara Schmid ✽, Stefan Diewald ✽, Luis Roalter ✽
✽ Technische Universität München, Germany❖ Universität Passau, Germany
Logging
Self-Reporting
DATA COLLECTION
Forgetting
Annoyance
Sluggishness
Self-Perception
RESEARCH QUESTIONS
Accuracy? Change over time?
Influence ofreporting frequency?
Reliability maximization?
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
BACKGROUND ■ Electronic diaries show higher
compliance(Hufford & Shields, 2002)
■ Mobile phone as survey tool (Consolvo et al., 2007)
Consolvo et al., 2007
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
CONTRIBUTIONS■ Empirical quantitative long-term data
on reliability of informationcollected with mobile devices
■ Recommendations for maximizingresult reliability
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
METHODOLOGYEvaluatereporting behavior
Ground truthCan be gained in automated
wayLimited effort
Smartphoneusage
GOAL
REQUIRE-MENTS
SOLUTION
Frequently used apps
Installed by everyone
Mail Facebook
Frequently used apps
Installed by everyone
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
PROCEEDING
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 6 + 4
Pre-Questionnaire Reminder Emails Post-Questionnaire
Post-Post-Questionnaire
Requested Self-Reports& Logging
TASK■ Answer questionnaire after Facebook or Mail
usage■ Report as accurate as possible
1. How long did you use the app?
2. How many times did you use the appwithout answering a questionnaire?
1 direct report
n indirect reports
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
SELF-REPORTING AND EXPERIENCE SAMPLING ASSISTANT
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
SERENA■ App usage logging■ Server upload■ Questionnaire triggers
□ Event-based□ Time-based□ Manually
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
3 CONDITIONS
Voluntary Interval Event
No trigger Daily trigger Trigger afterapp usage
30 Participants3,631 Mail usages3,181 Facebook usages
SESSIONS
Voluntary Interval Event
Amount of reported Facebook usages
37.6%
63.8%54.3%
Indirect reports
Direct reports
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
DURATIONS
Voluntary
Interval
Event
1:29
1:29
1:22
2:52
3:02
3:35
Facebook sessions
RealSelf-Reported
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
OVER TIME■ Self-report ratio decreases■ Actual usage decreases
„Answering the questionnairechanged my Facebook usage habits.“
Voluntary 2.3Interval 2.2Event 3.5
5 = strongly agreeA. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
DISCUSSION
Subjects reported max. 70% of usage
Commitment decreased
Reports may influence actual
behavior
Subjects overestimated session length
Reminder emails pushed
commitment in 2nd phase
Behavior change with increasing
burden
Little control: forgetting
High control: burden
Trigger influence lower than
hypothesized
App usage decreased
Most subjects would report max. 4 weeks
[email protected]://vmi.lmt.ei.tum.de/serena
Please cite this work as follows:
A. Möller, M. Kranz, B. Schmid, L. Roalter, S. DiewaldInvestigating Self-Reporting Behavior In Long-Term StudiesIn: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), pp. 2931-2940, Paris, France, April-May 2013.
If you use BibTex, please use the following entry:@inproceedings{chi2013selfreport, author = {Andreas M\"{o}ller and Matthias Kranz and Barbara Schmid and Luis Roalter and Stefan Diewald}, title = {Investigating Self-Reporting Behavior In Long-Term Studies}, booktitle = {Proceedings of the 2013 ACM annual conference on Human Factors in Computing Systems}, pages = {2931--2940}, series = {CHI '13}, year = {2013}, isbn = {978-1-4503-1899-0}, location = {Paris, France}, numpages = {10}, publisher = {ACM}, address = {New York, NY, USA},}
