A Practical Approach to Recognizing Physical Activities

Jonathan Lester, Tanzeem Choudhury, and Gaetano Borriello

In Proceedings of the Fourth International Conference on Pervasive Computing (2006)

Benjamin Stokes, Presenter -- 1/24/11For CS 546: Intelligent Embedded Systems

The Challenge

Personal activity recognition (in highly constrained use contexts)

• Healthcare sector: demand growing, currently relies on paid observer or self-reporting;

• Deficiencies: cost, accuracy, scope, coverage and obtrusiveness.

Proposed Solution

To investigate three pressing constraints:

1. Unpredictable sensor location (wrist, waist, shoulder)

2. Minimal training across individuals

3. Cost (and sensor) minimizing

Technical Preview

Related Links

• MSP Research Initiative@ University of Washington & Intel Research Seattle: http://seattle.intel-research.net/MSP/

• Wiki for the Mobile Sensing Platformhttp://ubi.cs.washington.edu/wiki/index.php/Main_Page

• Jonathan Lester research page (with details on the Mobile Sensing Platform) http://www.cs.washington.edu/homes/jlester/research.html – Includes a great white paper on their MSP design justification and

experience

Experimental Design

• 12 volunteers given a sequence of activities over several days (observer annotates true event)

• 8 different activities (sitting, standing, walking, walking up/down stairs, riding elevator, brushing teeth – selected as useful for elder care)

• Recognition trained & tested via activity classification algorithm developed earlier

Technical Design

Available Sensors

Raw Data (12 hrs gathered)

(Data from 2nd Data Set)

Data Preprocessing

18,000 samples of data per second so must summarize…

Result: 4 Hz @ 651 features

Analysis Stage: Classification

Window: 15 second sliding (decreases error; reveals transitions) • with 5 second overlap

For each activity…•True activity type is observed by a human•All events of that type are divided into 4 “folds” (for training/testing, i.e., 3 :1)

Investigating Sensor Location

Several options:1. Any location (of three) = ideal

…or…

2. Shoulder only3. Waist only4. Wrist only

Learning Model

Often, learning models are either:a) discriminative: to learn the class boundaries without

regard for densitiesb) generative: to learn the class densities

…they have a mix. Specifically:1. Top 50 features as most discriminating (< 10%)2. To recognize activities, Hidden Markov Models

(HMMs – i.e., a simple Bayesian network); includes “temporal smoothing”

(Image Source: Wikipedia)

Confusion Matrix (for the “location independent” condition)

Precision/recall

Variation Across Users

• Can anyone benefit?• How much training?

Train on 1-12 users (folds 3:1), test on all 12

• Approaches 80% accuracy if testing on outsiders(approaches 95% accuracy if tested within group)

Fewer sensors possible?

Which are most important?• Accelerometer (motion of user)• Audio (changing environment)• Barometric pressure (env.; in GPS for altitude)

Compare best sensor : top three (all locations)38.96% recall : 81.38%

…so use three!!

Findings SummaryInvestigated…

(1) location sensitivity, Can recognize context within our constraints! (and works across locations)

(2) variations across users, Can be pre-trained by other individuals.

(3) which sensor modalities.Can use fewer & cheaper sensors.

Critiques, Future Research• Curious: Defensive about dual analysis techniques?• Limitation: Excluded unclassified activities

(Overlooks 5 of 12 hours low ambiguity tolerance.)• Conceptual need: meta-classification to connect activities

(e.g., “making the rounds in hospital”)• Suggestion: Cluster population groups for performance.

(Here it was just “healthy individuals.”)• Suggestion: Consider time series data? (e.g., sitting

typically followed by standing, which precedes walking)

Related Links

• MSP Research Initiative@ University of Washington & Intel Research Seattle: http://seattle.intel-research.net/MSP/

• Wiki for the Mobile Sensing Platformhttp://ubi.cs.washington.edu/wiki/index.php/Main_Page

• Jonathan Lester research page (with details on the Mobile Sensing Platform) http://www.cs.washington.edu/homes/jlester/research.html – Includes a great white paper on their MSP design justification and

experience