Darwin Phones

8/7/2019 Darwin Phones

1/41

DARWIN PHONES: THE EVOLUTIONOF SENSING AND INFERENCE ON

MOBILE PHONES

PRESENTED BY: BRANDON OCHS

Emiliano Miluzzo, Cory T. Cornelius, Ashwin Ramaswamy, Tanzeem Choudhury, Zhigang Liu,Andrew T. Campbell, "Darwin phones: the evolution of sensing and inference on mobile

phones," In Proc. of 8th ACM Conference on Mobile Systems, Applications, and Services(MobiSys), 2010, pp. 5-20.


2/41

What does Darwin do?

A Smartphone platform for urban sensing

Proof of concept model uses microphone

Communicates with other local devices to improveinference accuracy (collaborative inference)

Framework can be expanded to gather

information using a range of sensor data


3/41


4/41

Common Urban Sensing Challenges

Human burden of training classifiers

Ability to perform reliably in different environments

(indoor vs outdoor)

The ability to scale to a large number of phones

without hurting usability and battery life.

Darwin overcomes all of these through

classifier/model evolution, model pooling, andcollaborative inference


5/41

Types of Learning

Supervised: Given a fully-labeled training set

Semi-Supervised: Given a small training set that isevolved

Unsupervised: No training set is given


6/41

Darwin Steps

Evolution, Pooling, and Collaborative Inference

These represent Darwins novel evolve-pool-collaborate model implemented onmobile phones


7/41

Classifier Evolution

Automated approach to updatingmodels over time

Needs to account for variability insensing conditions and settings

Variability in background noise andphone location require separatemodels


8/41

Model Pooling

Reuses models that have already been

built and evolved on other phones

Exchange classification models

whenever the model is available fromanother phone

Classifiers do not need to be retrained,

which increases scalability Can pool models from backend servers


9/41

Collaborative Inference

Combines results from multiple phones

Run inference algorithms in parallel on

the same classifiers

System is more robust to degradation in

sensing quality

Increases accuracy


10/41

Darwin Design: Computation

Reduces the on-the-phone computation by

offloading some of the work to backend servers

Backend server uses a machine learning algorithm

to compute a Gaussian Mixture Model (2 hours for

15 seconds of audio)

Feature vectors are computed

locally


11/41

Darwin Design: Context

Context (in/out of pocket, in/out of bag) will impact

the sensing and inference capability

Classifier evolution makes sure the classifier of an

event is robust across different environments


12/41

Darwin Design: Co-location

Accounts for a group of co-located phones runningthe same classification algorithm and sensing thesame event but computing different inference results

Phones pool classification models when collocatedor from backend servers

Compares against its own model and the co-locatedmodel

Drastically reduces classification latency Exploits diversity of different phone sensing context

viewpoints


13/41

Speaker Recognition

Attempts to identify a speaker by analyzing the

microphones audio stream

Suppresses silence, low amplitude audio, and chunks

that do not contain human voice

Reduce false positives by pre-processing in 32msblocks


14/41

Speaker Modeling

Feature vector consisting of

Mel Frequency CepstralCoefficients

Each speaker is modeled

with 20 Gaussians

An initial speaker model is built by collecting a short

training sample


15/41

Classifier Evolution: Training Step

Short training phase (30 seconds) used to build a

model which is later evolved

First 15 seconds used as the training set

Last 15 seconds used as baseline for evolution


16/41

Classifier Evolution: Evolution Step

Semi-supervised learning strategy

If the likelihood of the incoming audio stream is

much lower than any of the baselines then a new

model is evolved


17/41


18/41

Collaborative Inference

Local inference phase can be broken into threesteps:

Local inference operated by each individual phone

Propagation of the result of the local inference to theneighboring phones

Final inference based on the neighboring mobilephones local inference results

Each node individually operates inference on thesensed event

Results and confidence broadcasted


19/41


20/41

Privacy and Trust

Raw sensor data is not stored on or leaves the

mobile phone

The content of a conversation or raw audio data is

never disclosed

Users can choose to opt out of Darwin


21/41

Experimental Results

Tested using a mixture of five N97 and iPhones

used by eight people over a period of two weeks

Audio recorded in different locations

Classifier trained indoors


22/41

Experiment 1 Parameters

Three people walk along a sidewalk of a busy road

and engage in conversation

The speaker recognition application without the

Darwin components runs on each of the phones

carried by the people


23/41


24/41


Meeting setting in an office environment where 8

people are involved in conversation

The phones are located at different distances from

people in the meeting, some on the table and some

in peoples pockets


25/41

Experiment 2 Results


26/41



27/41


Five phones in a noisy restaurant

Three of the five people are engaged in

conversation

Two of the five phones are placed on the table

Phone 4 Is the closest phone to speaker 4 and also

the closest phone to another group of people

having a loud conversation


28/41



29/41



30/41



31/41



32/41


Five people walk along a sidewalk and three of

them are talking

The greatest improvement is observed by speaker

1, whose phone is clipped to their belt


33/41



34/41



35/41

Time and Energy Measurements

Baselines for power use determined

Measurements performed using the Nokia Energy

Profiler tool

No data gathered for the iPhone

Smart duty cycling required later to save battery

life


36/41

Time and Energy Measurements


37/41


38/41

Future Work

Duty cycling for improved battery life

Simplified classification techniques


39/41

Improvements On The Paper

Studies dont show conclusive evidence; there should

be separate control models for each of the

scenarios


40/41

Conclusion

The Darwin system combines classifier evolution,

model pooling, and collaborative inference

Results indicate that the performance boost offered

by Darwin off sets problems with sensing context

The Darwin system provides a scalable framework

that can be used for other urban sensing

applications


41/41

References

[1] Emiliano Miluzzo, Cory T. Cornelius, Ashwin Ramaswamy, Tanzeem

Choudhury, Zhigang Liu, Andrew T. Campbell, "Darwin phones: the evolution

of sensing and inference on mobile phones," In Proc. of 8th ACM

Conference on Mobile Systems, Applications, and Services (MobiSys), 2010,

pp. 5-20.

[2] H. Ezzaidi and J. Rouat. Pitch and MFCC Dependent GMM Models for

Speaker Identification systems. In Electrical and Computer Engineering,

2004. Canadian Conference on, volume 1, 2004

[3] H. Ezzaidi and J. Rouat. Pitch and MFCC Dependent GMM Models for

Speaker Identification systems. In Electrical and Computer Engineering,

2004. Canadian Conference on, volume 1, 2004.

Date post:	09-Apr-2018
Category:	Documents
Upload:	jagdishmehtaonline123
View:	215 times
Download:	0 times

Darwin Phones

Documents