1

A Preliminary Study of Sensing Appliance Usage for Human Activity Recognition Using Mobile Magnetometer

Mi Zhang and Alexander A. SawchukDepartment of Electrical Engineering

University of Southern Californiamizhang@usc.edu

UbiMI workshop at ACM Ubicomp Conference, September 8, 2012

2

Introduction

• Background

Human activity recognition is one of the most basic problems in Ubiquitous Computing.

Applications: surveillance, security, health care, etc. In this work, we focus on recognizing household activities by detecting appliance

usage.

• Existing Technology

Courtesy to Opportunity Project

Computer Vision RFID

Courtesy to Intel Research

Smart Meter

Courtesy to IBM

3

Magnetic Field Sensing

• Idea

When household appliance is in operation, it generates electromagnetic waves. The electric component has been demonstrated to infer appliance usage at home. How about the magnetic component?

• Magnetometer

The magnetometer measures the strength and the direction of the earth’s magnetic field in 3D space.

It is mainly used for outdoor navigation (referred to as compass).

Magnetometer can also be used for detecting magnets and ferromagnetic materials(referred to as metal detector).

4

Key Observation

• More Interestingly

When household appliances are in operation, the magnetic field around them presents a different pattern compared to the scenarios when these devices are turned off.

Hair Dryer is OFF Hair Dryer is ON

These changes exhibit different patterns for different devices and act as the signatures of the devices.

Our Framework

5

Magnetometer

Sliding Window

at 6sClassificationFeature

Extraction

1

2

1

nnf

ff

F

222 )()()()( tmagtmagtmagtM zyx

Overall Magnetic Field Strength

Standard Deviation, Mean Derivatives, Mean Crossing Rate, Dominant Frequency, Dominant Frequency Magnitude, Energy, Spectral Entropy

6

Evaluation

• Sensing Hardware and Software

Hardware: 3-axis magnetometer in iPhone 4GS, sampling at 15Hz.

Software: techBASIC mobile application. Data analysis is performed using MATLAB.

• Household Activities

Laptop Hair Dryer TV Mobile Phone Microwave

7

Preliminary Results• Scatter Plots

• Recognition Accuracy

Energy Features Statistical Features

Laptop Microwave TV Hair Dryer Mobile Phone

84.3% 100% 81.6% 92.7% 83.4%

8

Limitations and Future Work

• Sampling Rate is LOW

High frequency components in the magnetic field signal that may contain important information are not captured.

• Test on More Appliances

• Combine with Other Environmental Sensors

light sensor, temperature sensor, and motion sensors

9

Any Questions?

10

Thank You