Phone As A Sensor Technology: mHealth and Chronic Disease

> slide to unlock

eric c. larson | eclarson.com

phone-as-a-sensor technology:

Assistant Professor Computer Science and Engineering

mhealth and chronic disease

MiPhone 9 Now with iColon.

> slide to unlock

Comp Sci. & Engr.

Comp Sci. & Engr.

databases & data mining

algorithms arch

AI & robotics

OS

networking

languages

symbolic computing

software

mobileComp Sci. & Engr.


algorithms arch

AI & robotics

OS

networking

languages

symbolic computing

software



algorithms arch

AI & robotics

OS

networking

languages

symbolic computing

software

mhealth



algorithms arch

AI & robotics

OS

networking

languages

symbolic computing

software

mhealth

what is mhealth?

the promise of mhealth:


revolutionize medicine


revolutionize medicineeliminate doctor visits


revolutionize medicineeliminate doctor visitsremote / automatic diagnosis


revolutionize medicineeliminate doctor visitsremote / automatic diagnosisequalize developing countries





stress check

glucose buddyfitness trainer

heart rate

zombie run current mhealth

stress check


heart rate

zombie run current mhealth43,000 apps for health on the app store

stress check


heart rate

zombie run current mhealth43,000 apps for health on the app store 96% are for calorie counting & exercise

stress check


heart rate

zombie run current mhealth43,000 apps for health on the app store 96% are for calorie counting & exercise4% are remote monitoring

consider physician’s needs

consider physician’s needsconnecting with patient

consider physician’s needsconnecting with patient tracking baselines

consider physician’s needsconnecting with patient tracking baselinespersonalized trending data

consider physician’s needsconnecting with patient tracking baselinespersonalized trending datamanaging chronic disease

consider physician’s needsconnecting with patient tracking baselinespersonalized trending datamanaging chronic disease

30% of all US healthcare spending is on chronic disease

mhealth and chronic disease management

mhealth and chronic disease management

compliance?cost?

doctor patient?data reliability?

compliance

compliance

baseline quantitysensor

phone as a sensorbaseline quantitysensor


embedded sensors


embedded sensors processing

estimated

accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)


compliance++;cost--;dr_pat *= 10;


compliance++;cost--;dr_pat *= 10;

data reliability?

what can the mobile phone sense with clinical accuracy?

future research

lung function jaundice

future research


spirometer lung function??

lung function

asthma COPD cystic fibrosis

evaluates pulmonary impairments

spirometer

device that measures amount of air inhaled and

exhaled.

using a spirometer

flow

volume

volum

e

time

using a spirometer

flow

volume

volum

e

time

using a spirometer

flow

volume

volum

e

time

volume-time graphvo

lume

time

volume-time graphvo

lume

time

volume-time graphvo

lume

time

FEV1

FVC

FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity

volume-time graphvo

lume

time1 sec.

FEV1

FVC


volume-time graphvo

lume

time1 sec.

FEV1

FVCFEV1% = FEV1/FVC



FEV1% = FEV1/FVC


FEV1% = FEV1/FVC

> 80% healthy60 - 79% mild40 - 59% moderate

< 40% severe

flow-volume graphflo

w

volume

flow-volume graphflo

w

volume

flow

volumeFEV1 FVC

1 sec.

PEF

PEF: Peak Expiratory Flow FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity

flow-volume graph

flow

volume

normal

flow-volume graph

flow

volume

normalobstructive

flow-volume graph

obstructive diseases

!

resistance in air path leads to reduced air flow

obstructive diseases

!

resistance in air path leads to reduced air flow

restrictive diseases

!

lungs are unable to pump enough air and pressure

restrictive diseases

!

lungs are unable to pump enough air and pressure

flow-volume graphFlo

w

Volume

normalobstructive

flow-volume graphFlo

w

Volume

normal

restrictiveobstructive

clinical spirometry

home spirometry

home spirometry

!

faster detection rapid recovery

trending

home spirometry

high cost barrier patient compliance

less coaching limited integration

challenges with

SpiroSmart

availability cost portability more effective coaching interface integrated uploading

Using SpiroSmart

Using SpiroSmart

Using SpiroSmart

]

Using SpiroSmart

]

Using SpiroSmart

]

Using SpiroSmart

Using SpiroSmart

flow rate volume

lung functionairflow sensor

flow rate volume


sound pressure microphone

flow rate volume


sound pressure microphone processing

estimated

audio

audio

flow features

audio

flow features

measures regression

FEV1FVCPEF

0 1 2 3 40

5

10

15

Flow

(L/s

)

Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

0 1 2 3 40

5

10

15

Flow

(L/s

)Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

audio

flow features

measures regression

curve regression

FEV1FVCPEF

0 1 2 3 40

5

10

15

Flow

(L/s

)

Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

0 1 2 3 40

5

10

15

Flow

(L/s

)Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

audio

flow features

measures regression

curve regression

lung functionFEV1FVCPEF

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features estimation

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude


vocal tractsource output

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude


0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

resonance tracking0 1 2 3 4 5 6 7

−1

−0.5

0

0.5

1lpc8raw

time(s)

amplitude

flow estimation features

vocal tractsource output 0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1lpc8raw

time(s)

amplitude

auto-regressive estimate

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measures regression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

ground truth

feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measures regression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e7.1

PEF featuresground truth

feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measures regression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e7.1

0.330.35


feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measures regression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e7.1

0.330.35

3.2

FEV1 features


feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measures regression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e7.1

0.330.35

3.2

0.120.17

FEV1 features


feature 1

feature 2

FEV1 features PEF features

measures regression

FEV1 features

bagged decision tree

PEF features


measures regression

FEV1 features


output

PEF features


output

measures regression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

feature 1

feature N

window

ed machine

learning regression

......

curve regression

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

feature 1

feature N

curve output

window

ed machine

learning regression

......

curve regression


−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

curve regression


−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

CRF

curve regression


−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

CRF

!

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

curve regression

study design

x 3

x 3

study enrollment

study enrollment

participants 5218-75 years old, mostly healthy

study a

study enrollment


study a

participants 1012-17 years old, mixed healthy/abnormal

study b

study enrollment


study a

participants 1012-17 years old, mixed healthy/abnormal

study b

participants 5610-69 years old, mostly abnormal

study c

enrolledby

pulmonologists

resultsmeasures regression

resultsmeasures regression

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

resultscurves regression

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


can SpiroSmart curves be used for diagnosis?

survey

• normal/abnormal subjects curves

survey

• 5 pulmonologists


survey

• 5 pulmonologists


• unaware if from SpiroSmart / spirometer

survey

results

normal minimal obstructive mild obstructive moderate obstructive severe obstructive

restrictive

inadequate

results


restrictive

inadequate

identical 64%

results


restrictive

inadequate

one off 10%

identical 64%

results


restrictive

inadequate

one off 10%

identical 64%

results


restrictive

inadequate false positive 14%

one off 10%

identical 64%

results


restrictive

inadequate

false negative 4%

false positive 14%

one off 10%

identical 64%

results


restrictive

inadequate

error 8%false negative

4%

false positive 14%

one off 10%

identical 64%

results


restrictive

inadequate

error 8%false negative

4%

false positive 14%

one off 10%

identical 64%

!

!

abnormal vs normal 96%

appropriate for trending and screening

global health non-profit


patient and doctors


patient and doctors pharmaceutical drug trials

future research


future research


neonatal jaundice in the US

kernicterus: 21

hazardous jaundice: 1158

extreme jaundice: 2,317

severe jaundice: 35,000

phototherapy: 290,000

visible jaundice: 3.5 million

births/year: 4.1 million

Method Accuracy Disadvantages

TSB Gold standard (r=1.0)

Painful, costly, inconvenient, delayed

TcBAccurate

(r= 0.75 -0.93) Meter = $7000

tips $5 unavailable in most

physician offices

Visual assessment (provider or parent)

not accurate (r= 0.36 - 0.7), underestimates

severity

No standardization lighting, pigmentation

bilirubin level in blood

jaundice levelblood draw



yellowness camera



yellowness camera processing

estimated

bilicam

participants 48 newborns0-4 days old, collected in nursery

3 hospitals in Washington & Philadelphia

study A

participants 48 newborns0-4 days old, collected in nursery

3 hospitals in Washington & Philadelphia

study A

Color Linearization

• Camera Settings Adjustment

• Light Source Estimation

Image Segmentation

• Quality Control for Distance, Lighting, and Shadow

• Sternum, Forehead, Card Segmented

Color Calibration

• Dynamic Least Squares Regression

• Automatic Feature Selection

Neonatal Skin Response to Bilirubin

• Skin Independent Color Transformations Applied

• Multivariate Machine Learning Regression

bilicam

biliru

bin

level

bilicam estimation0

15

155 10

5

10

mg/dlr=0.91

bilicam initial results

20

20

biliru

bin

level

bilicam estimation0

15

155 10

5

10

mg/dlr=0.91


20

20

non whitewhite

biliru

bin

level

bilicam estimation0

15

155 10

5

10

mg/dlr=0.91


20

20

non whitewhite

TcB = 0.85

biliru

bin

level

bilicam estimation0

15

155 10

5

10

mg/dlr=0.91


20

20

non whitewhite

TcB = 0.85BiliCam = 0.84

bilicam future work

• near term: screeningbilicam future work

• near term: screening• medium term: more data

bilicam future work

• near term: screening• medium term: more data• long term: developing world

bilicam future work

• near term: screening• medium term: more data• long term: developing world

“in many resource poor nations,hyperbilirubinemia is the second or

third leading cause of infantmortality and disability”

bilicam future work

future research


future research


oxygen volume, VO2

future research

cardiac output and blood pressure

intra ocular pressure

intra ocular pressure

PressCam

eclarson.com [email protected] @ec_larson> slide to unlock

Thank You!


eclarson.com [email protected] @ec_larson> slide to unlock




collaborators: !Joseph Camp Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Gaetano Boriello Mayank Goel Lilian DeGreef


eclarson.com [email protected] @ec_larson




collaborators: !Joseph Camp Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Gaetano Boriello Mayank Goel Lilian DeGreef

Date post:	23-Dec-2014
Category:	Technology
Upload:	eric-larson
View:	93 times
Download:	1 times