Passive sensing of circadian rhythms for individualized models
of cognitive performance
Julie Kientz, Tanzeem Choudhury
Saeed Abdullah, Elizabeth Murnane, Mark Matthews, Matt Kay
Cognitive capabilities vary over time
2
Alertness: basic building block of cognitive performance
3
fatigue and sleepiness =
alcohol intoxication
4
Fatigue is involved in 30% of all road accidents in US
NTSB. Safety report NTSB/SR-99/01
5
36% increase in serious medical errors
6
Williamson, Ann, et al. "The link between fatigue and safety." Accident Analysis & Prevention 43.2 (2011): 498-515.
Negative impact on learning and problem solving
7
8
Alertness
Chronotype
Sleep CircadianMisalignment
Stimulants
9
Circadian Rhythm: biological processes following a roughly 24-hour period
circa: about, diem: a day
10
Almost every neurobehavioral process displays circadian rhythms
11
Chronotype: Individual differences in temporal preference resulting from circadian rhythms (early and late types)
12
Circadian Rhythms and Alertness
• Internal time dictates optimal peak alertness period
• Alertness drops during mid-day dip
• Sleep is a crucial factor
13
Studying alertness beyond controlled lab environment
14
Continuous assessment of alertness based on in-situ data in a real-world setup
15
• How do body clock, time of day, and stimulant intake impact alertness?
• Do phone usage patterns reflect fatigue and sleepiness?
• Can we automatically assess alertness using passively sensed phone data?
16
Methodology
17
Population
• University-aged individuals
• Massive risk of circadian misalignment
• Largest and most habituated technology users
18
Participants & Procedure• 20 participants
• 7 male, 13 female
• 18-29 years old
• Android users
• 40 days
19
• Data
• Daily sleep diary
• 4-times-per-day alertness assessment (EMA)
• Phone use logs
• Interviews
Sleep Data
MSFSC = MSF −0.5(SDF −(5∗SDW +2∗SDF)/7)
1 2 3 4 5 6 7 8 9
35
0
5
10
15
20
25
30
Chronotype
% o
f Sam
ple
Larks Owls
extremeEarlytype
moderateEarlytype
slightEarlytype
Normaltype
slightLatetype
moderateLatetype
extremeLatetype
Chronotype:
20
21
Smartphone Toolkit
22
PVT
From PVT to Alertness
• Median response time from a PVT session
• Establish individual baseline across all session
• Alertness is departure from baseline
23
Results
24
Alertness varies across time
25
Early and late types have different performance pattern
26
Daylight saving time (DST)
• Social clock-shifting
• Known to cause circadian disruptions
• 70 countries observe DST, impacting 1.6 billion people
27
Negative impact of DST
28
Stimulant intake
• Positive stimulants
- Caffeine intake, napping, doing exercise, nicotine intake
• Negative stimulants
- Alcohol consumption, having meals, relaxation
29
Stimulant intake
• 5.1% increase after positive stimulants
• 1.37% drop after negative stimulants
• Statistically significant (t = 2.2, p = 0.03)
30
Self-Assessment of Alertness
• Self-assessment
- Tiredness, energy and concentration level
• Response time differs significantly between high and low self-ratings
- fatigued individuals are usually aware of reduced capability
32
230 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
2500
0
500
1000
1500
2000
App
licat
ion
Usa
ge E
vent
s
Entertainment Time & WeatherCommunicationProductivity BrowsingEmailSocial Media
Rhythms in App Use
33
Productivity vs. Entertainment
20
0
5
10
15
% o
f Usa
ge E
vent
s
EntertainmentProductivity
Mon Tues Wed Thu Fri Sat Sun
• Work days
• Free days
• Mid-week dip
34
Usage Differences by Chronotype
• Early Types
• 25% more productivity apps
• 18-28% fewer entertainment apps
• Late Types
• 22-68% more productivity apps
• 15-50% less entertainment apps
100
-100
-80-60-40-20
020
406080
Earl
y-La
te U
sage
Cha
nge
(%)
Morning(6AM-12PM)
Afternoon(12PM-6PM)
Evening(6PM-12AM)
Night(12AM-6AM)
EntertainmentProductivity
35
Internal Time
230 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
InT
Performance
6
-8
-7
-6
-5
-4
-3
-2
-10
1
2
3
4
5
Usage
Entertainment CMC
Alertness
Productivity
InT = ExT - MSFSC
“To wake myself up, I’ll have to look at things on the phone like Facebook or Tumblr.”
“In morning classes, I have less attention and am very tired so I’ll browse the phone. Using tactics like social media, I focus on the screen to try to keep my eyes open.”
———
“Every time before I go to bed, I play a card game until I feel sleepy.”
“I use my phone when falling asleep. Especially if I’m having trouble falling asleep, I’ll play a game or talk to my boyfriend until I fall asleep.”
36
App Use and Sleep
• Less sleep: less productivity (r=0.43), more entertainment apps (r=-0.19)
• Adequate sleep: 61% more productivity apps
• Inadequate sleep: 33% more entertainment apps
• Nightly use events reflect sleep interruptions
37
Predicting alertness from phone data
• PVT is not suitable for longitudinal deployment
• Passive inferring of alertness can enable a new suite of HCI applications
• Can data from mobile phone predict alertness?
38
Predicting Alertness• Stochastic Gradient Descent (SGD) with Huber loss
function
• Standardize all features to have zero mean and unit variance
• L1-norm as regularization term
- α = 10-8
- learning rate: γt = γ0 · t−1/4 (with γ0 = 0.01)
39
Predicting Alertness
• 10 fold cross-validation
• RMSE of 11.39 across all participants
• Accurate enough for scalable deployment
Internal Time
Avg. time between phone usage sessions
Short Session frequency
Phone usage duration
Relative sleep need
Top-ranking features for predicting alertness
40
Implications & Applications
41
Task Scheduling
• When to do what?
- based on cognitive demand and assessed alertness
• Better team collaboration
- grouping members with similar circadian characteristics
42
Learning and Education
• Circadian disruptions adversely affect memory and learning abilities
• Learning and memorization aligned with individual alertness rhythms in school
43
Accident Prevention
• Assistive systems for drivers
• Continuous monitoring to prevent industrial accidents
44
Future work: • Circadian-aware technology • Informatics tools & intervention studies
Contributions: • In-situ alertness sensing • Manifestations of biological rhythms in mobile use • Automated alertness prediction
45