1
Making Continuous Mobile Sensing More Energy-Efficient
Archan MisraSingapore Management University
Sep 27, 2012
Karl Aberer (EPFL), Rajesh Balan (SMU), Dipanjan Chakraborty (IBM),Lipyeow Lim (U. Hawaii), Sougata Sen (SMU), Vigneshwaran Subbaraju(SMU), Zhixian Yan (EPFL)
The Opportunity
2
Smartphones: A Beehive of sensing activity
*Reproduced from: “Sensing Your World”, Mike Thompson, blogs.synopsys.com
Increasing penetration ofsensors in mobile devices/tablets.
• Accelerometers• Compass• Gyroscope• Barometer
Projected Penetration of Inertial Sensors, reproduced from Yole Developpement Report, 2011
The Applications of Sensing
3
Location-based ServicesIndoor Location
Wellness & Lifestyle
PureRunner BeWell
Social Networking & Interaction
WalkBase
Color Postural Recognition (ETH)
ShopKick
The Problem
4
The Energy Overheads:• Activating/Sampling the Sensor• Processing the Sensor Data Stream•Transmitting the Results to the “Cloud”
Power Consumption Observed on a Test Samsung Galaxy S3
• OK for ‘intermittent’ sensing.• Need research to address“continuous sensing”!
20-30% increase in overhead of motion-
related activity recognition
The Different Steps in “Making Sense”
5
Sensing
Feature Extraction
Classification
Context Deduction
High-Level Query
Mean, Fourier Coefficients, Entropy
Stand, Walk
Queuing, Exercising
Is Archan ‘queuing’ alone or with friends?
Accel(x,y,z)
The Research Threads
6
Make the Sensing Process More Adaptive andEfficient
Make the Querying Logic Smarter on anIndividual Phone
Make the Querying Logic Smarter Across ManyPhones
A3R: Adaptive Sensing & Feature Extraction
(accelerometer)
ACQUA : query optimization
Cloud Query Coordination Service
High-Level Query
Research #1: A3R
7
A3R: Adaptive Accelerometer-based Activity Recognition
8
Key Idea: Adjust accelerometer “parameters” based on the current activity of the individual.
Two parameters:• Accelerometer sampling frequency (SF)• Classification Features (CF)
Goal: reduce energy overhead of activity recognition without sacrificing accuracy
Energy Overhead Variation
• Energy overhead increases with SF.• Non-linear increase when frequency-domain features (CF)
are selected along with time-domain features.
9
Graph of Energy Consumptionon Samsung S2
Classification Accuracy• Different combination of Activites (Aggregated)
– <sampling frequency, feature choice>
10
• Activity-specific– separate study for each activity
A3R Algorithm for Continuous Activity Recognition
11
Activity Unknown<Fmax, T+F>
Stand<16,T>
Sit-relax<5,T>
Slow Walk
<16,T>
Escalator down
<100,T+F>
Ave_Conf> ∆
Ave_Conf< ∆
Ave_Conf> ∆
Ave_Conf> ∆
Ave_Conf> ∆
Ave_Conf< ∆
Ave_Conf< ∆
Ave_Conf< ∆
• The initial (SF, CF) default: highest sampling freq (SF) & richest feature set (CF)
• Classifier confidence: conf_vector = [p1, p2, …, pn]Average the confidence over a window & find the max
If ave-conf < Δconf
– use maximum (SF,CF)else
– use smart (SF,CF)
A3R: Insight into Activity Behavior of Real Users
user1 user2 user3 user4 user5 user60
2
4
6
8
10
12
14
16
x 104
coun
t of a
ctiv
ities
sit sitRelax normalWalk slowWalk stand stairs
Nokia N95 data: 6 users, 2-4 weeks each Non-adaptive vs. A3R
Full100, Full50, Full16: both {time + freq} features A3R: adaptive feature & sampling frequency
12
Evaluation II: In-Situ Study for Android Users
Continuous study on two android phones User 1: Samsung Galaxy II User 2: HTC Nexus I
Battery Remaining Each day (3 cases) A3R: adaptive feature & sampling frequency Non-adaptive: {50}Hz + {time+freq} features No activity recognition
13
Research #2: ACQUA:
14
ACQUA= Acquisition-Cost Aware Continuous Query Adaptation
ACQUA: Historical Scenario
SPO2
ECG
HR
Temp.
Acc.
...
IF Avg(Window(HR)) > 100AND Avg(Window(Acc)) < 2 AND AVG(Window(Tep))>80F
THEN SMS(caregiver)
Body-worn health & wellness sensors
Phone runs a complex event processing (CEP)
engine with rules for alerts
15
Can we save ‘energy’ by avoiding the need to download all data from all sensors, all the time?
ACQUA: Dynamically Changing Order of Retrieving Sensor Data
Predicate Avg(HR,5)>100
Max(SpO2,10)<90
Acquisition 5 * .02 = 0.1 nJ 10 * .008 = 0.08 nJPr(false) 0.95 0.8
if Avg(Heart-Rate, 5)>100 AND Max(Sp02,10)<90 then ALERT (STRESS).
Acq./Pr(f) 0.1/0.95 0.08/0.8
16
Query Sp02 first or HR first?• Different events are less “likely” (at present)• Different sensors need different amount of
energy to transfer data
#1: Evaluate predicates with lowest energy consumption first:
{Sp02, HR}
#2: Evaluate predicates with highest false probability first
{HR, Sp02}
#3: Evaluate predicate with lowest normalized acquisition cost first (#1/#2): {Sp02, HR}
ACQUA Architectural Components
17
Asynchronous Event Engine • Maintains partial query evaluation state
Dynamic QueryEvaluation Optimizer• Determines retrieval sequence for sensor streams
Query Logic Specification Module• Subset of Stream-SQL query syntax
Cost Modeler• External specification of sensor-specific trx. Cost model• Dynamic evaluation of stream selectivity
C(.); P(.)Normalized Query Syntax
Push/Pull, Batch commands
Dynamic Sensor Control (DSC)
ACQUA Components on a
Single Phone
Example: ω(Evaluation Period)=3
• Time 5: P2,P1,P3• Time 8: acquisition cost for A becomes
cheaper, because some tuples are already in buffer P1, P2, P3
Acquisition cost depends on state of the buffer at time t
18
P1
P2 P3
Dynamics of SelectivityThe sequence decision is made at EVERY
evaluation instant!
Performance Results (Simulations)Bluetooth 802.11
Ener
gyBy
tes
19
Research #3: Cloud Coordinated Mobile Sensing
20
Illustrating the Coordination Service
21
Inform when – “At least 3 PhD students of 2012 taking ISM are co-located”, so that – “I can discuss assignment with them”
Let me know if – “when at least one of 3 persons, A, B and C, are back in office and not using their cellphone, so that – “I can discuss assignment with them”
Can we save ‘energy’ by better coordinating the queries across large number of phones?
Sensing Coordination Service
Application 1
<M1,Q1>
Application 2<M2,Q2>
Q1 Q2
Logic Flow of the Coordination Service
Optimize JOINTLY (Q1, Q2):
ACQUA on Multiple Phones
Optimization Energy Savings (compared to ‘all transfer)
Individual Initial 48%
Jointly Additional 27%
23
Q1: Inform when – “A is standing and near the security gate”
Q2: Inform when – “One of A or B is standing”
Sensing Coordination Service
StandingA(Accel)StandingB(Accel)
LocationA(Wi-Fi)
Optimize individually:Q1: {locationA(Wi-Fi), StandingA(Accel)}Q2: {StandingB(Accel), StandingA(Accel) } {StandingA(Accel)
locationA(Wi-Fi)
StandingB(Accel)
Preliminary Numerical Investigation
24
LiveLabs Cloud Service
5 minutes later
Lifestyle Company
If a group of 4 or more people exit from Café after sitting down for 10 minutes, send
SMS with a “Movie Discount”
10 minutes later
4 in a group sitting down at
a Café
4 in a group left after 10 mins
LiveLabs software continuously monitors (location, activity, …)
Show this notification and get 20% on all
Movies
LiveLabs: A Future Testbed for Mobile Sensing(Jointly with Prof. Rajesh Balan)
LiveLabs: a large-scale research testbed for “mobile systems innovation” let’s companies & researchers run LARGE-SCALE trials and experiments with novel context-based mobile applications and services testbeds at 3 key locations with 30,000 users
Conclusions• Energy is the most critical resource constraint in
mobile sensing.• Advances include
– Adaptive adjustment of individual sensor– Query optimization of queries on individual phone– Joint optimization of queries across multiple phones.
• Ability to predict ‘context’ is the key.– Currently, context is inferred on per-phone basis.– Future—context itself inferred ‘collectively’?– Open issue: scaling context to citizen-scale
environments.
25
The End!
Thanks! Questions?
26
• “Energy-Efficient Continuous Activity Recognition on Mobile Phones: An Activity-Adaptive Approach” by Z. Yan, V. Subbaraju, D. Chakraborty, A. Misra and K. Aberer, 16th Annual International Symposium on Wearable Computers (ISWC), June 2012.
• "Optimizing Sensor Data Acquisition for Energy-Efficient Smartphone-based Continuous Event Processing", by Archan MISRA and Lipyeow LIM, IEEE Int. Conference on Mobile DataManagement (MDM), May 2011.
• "The Case for Cloud-Enabled Mobile Sensing Services", by Sougata SEN, Archan MISRA, Rajesh Krishna BALAN, and Lipyeow LIM,, Mobile Cloud Computing Workshop (MCC'12), in conjunction with ACM SIGCOMM, August 2012.