Post on 09-Apr-2018
transcript
8/7/2019 Darwin Phones
1/41
DARWIN PHONES: THE EVOLUTIONOF SENSING AND INFERENCE ON
MOBILE PHONES
PRESENTED BY: BRANDON OCHS
Emiliano Miluzzo, Cory T. Cornelius, Ashwin Ramaswamy, Tanzeem Choudhury, Zhigang Liu,Andrew T. Campbell, "Darwin phones: the evolution of sensing and inference on mobile
phones," In Proc. of 8th ACM Conference on Mobile Systems, Applications, and Services(MobiSys), 2010, pp. 5-20.
8/7/2019 Darwin Phones
2/41
What does Darwin do?
A Smartphone platform for urban sensing
Proof of concept model uses microphone
Communicates with other local devices to improveinference accuracy (collaborative inference)
Framework can be expanded to gather
information using a range of sensor data
8/7/2019 Darwin Phones
3/41
8/7/2019 Darwin Phones
4/41
Common Urban Sensing Challenges
Human burden of training classifiers
Ability to perform reliably in different environments
(indoor vs outdoor)
The ability to scale to a large number of phones
without hurting usability and battery life.
Darwin overcomes all of these through
classifier/model evolution, model pooling, andcollaborative inference
8/7/2019 Darwin Phones
5/41
Types of Learning
Supervised: Given a fully-labeled training set
Semi-Supervised: Given a small training set that isevolved
Unsupervised: No training set is given
8/7/2019 Darwin Phones
6/41
Darwin Steps
Evolution, Pooling, and Collaborative Inference
These represent Darwins novel evolve-pool-collaborate model implemented onmobile phones
8/7/2019 Darwin Phones
7/41
Classifier Evolution
Automated approach to updatingmodels over time
Needs to account for variability insensing conditions and settings
Variability in background noise andphone location require separatemodels
8/7/2019 Darwin Phones
8/41
Model Pooling
Reuses models that have already been
built and evolved on other phones
Exchange classification models
whenever the model is available fromanother phone
Classifiers do not need to be retrained,
which increases scalability Can pool models from backend servers
8/7/2019 Darwin Phones
9/41
Collaborative Inference
Combines results from multiple phones
Run inference algorithms in parallel on
the same classifiers
System is more robust to degradation in
sensing quality
Increases accuracy
8/7/2019 Darwin Phones
10/41
Darwin Design: Computation
Reduces the on-the-phone computation by
offloading some of the work to backend servers
Backend server uses a machine learning algorithm
to compute a Gaussian Mixture Model (2 hours for
15 seconds of audio)
Feature vectors are computed
locally
8/7/2019 Darwin Phones
11/41
Darwin Design: Context
Context (in/out of pocket, in/out of bag) will impact
the sensing and inference capability
Classifier evolution makes sure the classifier of an
event is robust across different environments
8/7/2019 Darwin Phones
12/41
Darwin Design: Co-location
Accounts for a group of co-located phones runningthe same classification algorithm and sensing thesame event but computing different inference results
Phones pool classification models when collocatedor from backend servers
Compares against its own model and the co-locatedmodel
Drastically reduces classification latency Exploits diversity of different phone sensing context
viewpoints
8/7/2019 Darwin Phones
13/41
Speaker Recognition
Attempts to identify a speaker by analyzing the
microphones audio stream
Suppresses silence, low amplitude audio, and chunks
that do not contain human voice
Reduce false positives by pre-processing in 32msblocks
8/7/2019 Darwin Phones
14/41
Speaker Modeling
Feature vector consisting of
Mel Frequency CepstralCoefficients
Each speaker is modeled
with 20 Gaussians
An initial speaker model is built by collecting a short
training sample
8/7/2019 Darwin Phones
15/41
Classifier Evolution: Training Step
Short training phase (30 seconds) used to build a
model which is later evolved
First 15 seconds used as the training set
Last 15 seconds used as baseline for evolution
8/7/2019 Darwin Phones
16/41
Classifier Evolution: Evolution Step
Semi-supervised learning strategy
If the likelihood of the incoming audio stream is
much lower than any of the baselines then a new
model is evolved
8/7/2019 Darwin Phones
17/41
8/7/2019 Darwin Phones
18/41
Collaborative Inference
Local inference phase can be broken into threesteps:
Local inference operated by each individual phone
Propagation of the result of the local inference to theneighboring phones
Final inference based on the neighboring mobilephones local inference results
Each node individually operates inference on thesensed event
Results and confidence broadcasted
8/7/2019 Darwin Phones
19/41
8/7/2019 Darwin Phones
20/41
Privacy and Trust
Raw sensor data is not stored on or leaves the
mobile phone
The content of a conversation or raw audio data is
never disclosed
Users can choose to opt out of Darwin
8/7/2019 Darwin Phones
21/41
Experimental Results
Tested using a mixture of five N97 and iPhones
used by eight people over a period of two weeks
Audio recorded in different locations
Classifier trained indoors
8/7/2019 Darwin Phones
22/41
Experiment 1 Parameters
Three people walk along a sidewalk of a busy road
and engage in conversation
The speaker recognition application without the
Darwin components runs on each of the phones
carried by the people
8/7/2019 Darwin Phones
23/41
8/7/2019 Darwin Phones
24/41
Experiment 2 Parameters
Meeting setting in an office environment where 8
people are involved in conversation
The phones are located at different distances from
people in the meeting, some on the table and some
in peoples pockets
8/7/2019 Darwin Phones
25/41
Experiment 2 Results
8/7/2019 Darwin Phones
26/41
Experiment 2 Results
8/7/2019 Darwin Phones
27/41
Experiment 3 Parameters
Five phones in a noisy restaurant
Three of the five people are engaged in
conversation
Two of the five phones are placed on the table
Phone 4 Is the closest phone to speaker 4 and also
the closest phone to another group of people
having a loud conversation
8/7/2019 Darwin Phones
28/41
Experiment 3 Results
8/7/2019 Darwin Phones
29/41
Experiment 3 Results
8/7/2019 Darwin Phones
30/41
Experiment 3 Results
8/7/2019 Darwin Phones
31/41
Experiment 3 Results
8/7/2019 Darwin Phones
32/41
Experiment 4 Parameters
Five people walk along a sidewalk and three of
them are talking
The greatest improvement is observed by speaker
1, whose phone is clipped to their belt
8/7/2019 Darwin Phones
33/41
Experiment 4 Results
8/7/2019 Darwin Phones
34/41
Experiment 4 Results
8/7/2019 Darwin Phones
35/41
Time and Energy Measurements
Baselines for power use determined
Measurements performed using the Nokia Energy
Profiler tool
No data gathered for the iPhone
Smart duty cycling required later to save battery
life
8/7/2019 Darwin Phones
36/41
Time and Energy Measurements
8/7/2019 Darwin Phones
37/41
8/7/2019 Darwin Phones
38/41
Future Work
Duty cycling for improved battery life
Simplified classification techniques
8/7/2019 Darwin Phones
39/41
Improvements On The Paper
Studies dont show conclusive evidence; there should
be separate control models for each of the
scenarios
8/7/2019 Darwin Phones
40/41
Conclusion
The Darwin system combines classifier evolution,
model pooling, and collaborative inference
Results indicate that the performance boost offered
by Darwin off sets problems with sensing context
The Darwin system provides a scalable framework
that can be used for other urban sensing
applications
8/7/2019 Darwin Phones
41/41
References
[1] Emiliano Miluzzo, Cory T. Cornelius, Ashwin Ramaswamy, Tanzeem
Choudhury, Zhigang Liu, Andrew T. Campbell, "Darwin phones: the evolution
of sensing and inference on mobile phones," In Proc. of 8th ACM
Conference on Mobile Systems, Applications, and Services (MobiSys), 2010,
pp. 5-20.
[2] H. Ezzaidi and J. Rouat. Pitch and MFCC Dependent GMM Models for
Speaker Identification systems. In Electrical and Computer Engineering,
2004. Canadian Conference on, volume 1, 2004
[3] H. Ezzaidi and J. Rouat. Pitch and MFCC Dependent GMM Models for
Speaker Identification systems. In Electrical and Computer Engineering,
2004. Canadian Conference on, volume 1, 2004.