Sensor Fusion for Energy Expenditure Estimation
Dominik Schuldhaus1, Sabrina Dorn1, Heike Leutheuser1, Alexander Tallner2,
Jochen Klucken3, Bjoern M. Eskofier1
December 4, 2013
1Digital Sports Group, Pattern Recognition Lab, University of Erlangen-Nuremberg, Germany
2Institute of Sport Science and Sport, University Erlangen-Nuremberg, Germany
3Department of Molecular Neurology, University Hospital Erlangen, Germany
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion 2
World Health Organization
Overweight and obesity:
fifth leading risk for global deaths
[http://www.foxnews.com]
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion 3
Physical Activity (PA)
0
50
100
%
Active
Not active
Active State Distribution
Assessment of PA
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Self reports
Energy Expenditure
4
Assessment of PA
[www.mojolondon.co.uk]
Energy Expenditure Estimation
in
Daily Life
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
State-of-the-Art Algorithms
5
Sensor
Literature
• 1994: Bouten et al.
• 2010: Vathsangam et al.
• 2012: Liu et al.
Small and lightweight sensors
• Fusion of sensor data
Estimation of energy expenditure
• Daily life activities
• Running on treadmill
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
[Hall et al. 1997]
Feature Level Fusion
6
Feature Extraction
Regression
Preprocessing Preprocessing Preprocessing
Sensor 1 Sensor 2 Sensor 3
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
[Hall et al. 1997]
Feature Level Fusion
7
Feature Extraction
Regression
Preprocessing Preprocessing Preprocessing Preprocessing
Sensor 1 Sensor 2 Sensor 3 Sensor 4
Major need:
No retraining of system
after adding sensors
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
[Hall et al. 1997]
Feature Level Fusion
8
Feature Extraction
Regression
Preprocessing Preprocessing Preprocessing Preprocessing
Sensor 1 Sensor 2 Sensor 3 Sensor 4
Major need:
No retraining of system
after adding sensors
Decision level fusion
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Study: Data Collection
9
# Participants 10
# Male 7
Age [years] 49 ± 12
Height [cm] 178 ± 10
Weight [kg] 80.7 ± 14.6
EnEx database:
http://www.activitynet.org
http://www.activitynet.org/
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Study: Sensor Setup
10
SHIMMER 3-D accelerometer 1.5 g (hip)
6.0 g (ankle)
3-D gyroscope 500 °/s (hip)
2000 °/s (ankle)
Sampling rate 204.8 Hz
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Study: Exercises
11
Traditional
• 3.2 km/h
• 4.8 km/h
• 6.4 km/h
Oscillating
• 3.2 km/h
• 4.8 km/h
• 6.4 km/h
• Each speed level: 6 min
• Treadmill
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Reference System
12
Spirometry system
Metabolic equivalent
(MET)
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Study: Example Data
13
• Treadmill - traditional
• Three speed levels
• Angular velocity in sagittal plane (ankle)
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Study: Example Data (2)
14
3.2 km/h 4.8 km/h
6.4 km/h
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Proposed System
15
Decision Level Fusion
Preprocessing Preprocessing Preprocessing Preprocessing
HP_ACC HP_GYR AK_ACC AK_GYR
Feature Extraction Feature Extraction Feature Extraction Feature Extraction
Regression Regression Regression Regression
HP: hip - AK: ankle - ACC: accelerometer - GYR: gyroscope
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Preprocessing
16
Non-overlapping sliding windows: 30 sec
1.
2.
3.
Steady state segmentation: last three minutes
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Feature Extraction
17
Features
Extrema
• Minimum
• Maximum
Statistical
• Mean of abs. amplitudes
• Standard deviation
• 10 / 25 / 50 / 75 / 90 th percentile
9 / axis
x
(3 axes accel or 3 axes gyro)
=
27 / sensor type
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Regression
18
Comparison of algorithms
• Support Vector Regression (SVR)
• Classification and Regression Trees (CART)
• Multiple Linear Regression (MLR)
Performance assessment
• Mean absolute error [MET]
• Leave-one-subject-out cross-validation
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Proposed System
19
Decision Level Fusion
Preprocessing Preprocessing Preprocessing Preprocessing
HP_ACC HP_GYR AK_ACC AK_GYR
Feature Extraction Feature Extraction Feature Extraction Feature Extraction
Regression Regression Regression Regression
HP: hip - AK: ankle - ACC: accelerometer - GYR: gyroscope
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Proposed System
20
HP: hip - AK: ankle - ACC: accelerometer - GYR: gyroscope
HP_ACC HP_GYR AK_ACC AK_GYR
MET MET MET MET
Mean
MET
Only adjustment after adding sensors:
Mean function
No retraining of system
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Results
21
Algorithm HP_ACC HP_GYR AK_ACC AK_GYR
SVR 0.64 ± 0.45 0.79 ± 0.71 0.71 ± 0.53 0.61 ± 0.41
CART 0.61 ± 0.36 0.96 ± 0.72 0.71 ± 0.51 0.67 ± 0.50
MLR 0.77 ± 0.58 0.79 ± 0.71 0.66 ± 0.48 0.85 ± 0.46
Mean absolute error [MET]
Decision level fusion
0.50 ± 0.13
Performance improvement by 18 %
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Summary
22
0
50
100
Mon Tue Wed
% Active
Not active
Active State Distribution
• Sensor-based energy expenditure estimation
• Decision level fusion
• Mean absolute error: 0.5 MET
• No retraining of system after adding sensors
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Outlook
23
• Comparison of fusion algorithms
• Adding physiological sensors
• Testing on daily life activities
December 4, 2013 | Dominik Schuldhaus | University of Erlangen-Nuremberg | Decision Level Fusion
Outlook: Benchmark Dataset
24
• Comparison of algorithms difficult:
No common used benchmark dataset
• http://www.activitynet.org
http://www.activitynet.org/
Thank you for your attention!
Bavarian Ministry of
Economic Affairs,
Infrastructure, Transport and
Technology