Introduction Methods Experiments and Results Conclusion
Radio Tomographic Imaging and Tracking ofStationary and Moving People via Kernel
Distance
Yang Zhao, Neal Patwari, Jeff M. Phillips, SureshVenkatasubramanian
April 11, 2013
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Outline
1 IntroductionDevice-Free LocalizationLimits of Previous Methods
2 MethodsHistogram DifferenceImaging and Tracking via Kernel Distance
3 Experiments and ResultsExperimentsResults
4 Conclusion
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Outline
1 IntroductionDevice-Free LocalizationLimits of Previous Methods
2 MethodsHistogram DifferenceImaging and Tracking via Kernel Distance
3 Experiments and ResultsExperimentsResults
4 Conclusion
Introduction Methods Experiments and Results Conclusion
Device-Free Localization
Device and Device Free Localization of People
5 718 19 20 2114 15 16 17
1
2
3
4
5 6 7 8 9 10 11 12 13
T
5 718 19 20 2114 15 16 17
1
2
3
4
5 6 7 8 9 10 11 12 13
Radio device localization: RFID tag, Mobile Phone
Device free localization (DFL):
Idea: directly use human motion/presence as signalApplications: emergency response, smart homes, context-awarecomputing, etc.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Device-Free Localization
Received Signal Strength (RSS)-Based DFL
RSS: available in standard wireless devices
Fingerprint-based DFL: training is needed for eachenvironment, not applicable for emergency situations
Model-based DFL: [Patwari and Agrawal, 2008] modelsshadowing effects of people/objects on RSS, and proposes theidea of radio tomographic imaging (RTI)
Patwari and Agrawal. Effects of Correlated Shadowing: Connectivity,
Localization, and RF Tomography. In Proc. of the 7th ACM/IEEE IPSN,
April 2008.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Limits of Previous Methods
Previous DFL Methods
Shadowing-based RTI: first RTI method to image shadowingloss due to human presence, not robust to non-line-of-sight(NLOS) environments;Variance-based RTI (VRTI) [Wilson and Patwari, 2011]:works in NLOS environments, but cannot locate stationarypeople;Sequential Monte Carlo (SMC) [Wilson and Patwari, 2012]:can locate stationary and moving people, but requiresempty-area calibration, and is not a real-time method.
Wilson and Patwari. See-Through Walls: Motion Tracking Using Variance-Based Radio Tomography
Networks. IEEE Trans. Mobile Computing, May 2011, pp.612-621.
Wilson and Patwari. A Fade Level Skew-Laplace Signal Strength Model for Device-Free Localization with
Wireless Networks. IEEE Trans. Mobile Computing, June 2012, pp. 947-958.Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Limits of Previous Methods
Goal: Break These Limits
Features RTI VRTI SMC
Through-wall? No Yes Yes
Stationary people? Yes No Yes
Real-time? Yes Yes No
Empty-area calibration? Yes No Yes
Table: Features of previous DFL methods.
Goal: a real-time method capable of imaging both stationaryand moving people in both LOS and NLOS environmentswithout training or empty-area calibration.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Outline
1 IntroductionDevice-Free LocalizationLimits of Previous Methods
2 MethodsHistogram DifferenceImaging and Tracking via Kernel Distance
3 Experiments and ResultsExperimentsResults
4 Conclusion
Introduction Methods Experiments and Results Conclusion
Histogram Difference
Idea: Use RSS Histogram Instead of RSS Mean or Variance
Notation of RSS histogram at time n:
hn =∑i
wn,i Iy i (1)
where
y i : RSS at time i (discrete value, quantization 1 dBm)
N = ymax − ymin + 1: range of RSS histogram
I: N-length indicator vector
wn,i : weight for Iy i
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Histogram Difference
Two Weighting Schemes
Uniform weight (FIR filter)
wn,i =
{1T 0 ≤ i ≤ T − 1
0 otherwise(2)
Exponentially weighted moving average (EWMA)
wn,i =
{β(1− β)n−i i ≤ n
0 otherwise(3)
where 0 < β < 1 is the forgetting factor
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Histogram Difference
Two Types of RSS Histograms
EWMA: an IIR filter with lower computational and memorycomplexity
hn = (1− β)hn−1 + βIyn (4)
only requires current RSS yn and previous histogram hn−1.
Long-term histogram (LTH) q and short-term histogram (STH) p
FIR scheme: LTH q with a high T value
IIR scheme: LTH q with a low β value (β = 0.05 for LTH;β = 0.9 for STH)
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Histogram Difference
Observations of RSS Histograms
Online LTH (IIRusing online data) issimilar to empty-areaLTH (FIR usingoffline data)
STH with people isdifferent from LTH
STH w/o people issimilar to LTH
Idea: quantify thedifference betweenonline LTH and STH
12
34
−48−47
−46−45
−44−43
−42−41
0
0.2
0.4
0.6
0.8
1
RSS (dBm)
Fre
quency
Online LTH
Empty−room LTH
STH with person
STH w/o person
Figure: RSS histograms (same link).
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Histogram Difference
Histogram Difference Metrics
Many ways to measure the difference D(p,q) between p and q:
Earth mover’s distance (very computationally expensive)
Kullback-Leibler divergence (DKL(p,q) =∑
k pk log pkqk
)
Kernel distance (symmetric distance metric)[Joshi, et al. 2011]
Joshi, et al. Comparing Distributions and Shapes Using the Kernel
Distance. In Proc. of the 27th ACM symposium on computational
geometry, June 2011.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Histogram Difference
Kernel Distance
Definition:
DK (p,q) = pTKp + qTKq− 2pTKq, (5)
where K is an N by N kernel matrix from a 2-D kernel function.
Gaussian kernel:K(yj , yk) = exp
(− |yj−yk |
2
σ2G
), where yj and yk are the jth and
kth elements, and σ2G is the kernel width parameter.
Epanechnikov kernel:
K(yj , yk) = 34
(1− |yj−yk |
2
σ2E
)I|yj−yk |≤σ2
E, where Ia is the
indicator function.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Histogram Difference
Efficient Computation: O (N) Instead of O(N2)
Rewrite (5) as:
D(p,q) = (K12p)TK
12p + (K
12q)TK
12q− 2(K
12p)TK
12q
Let u = K12p, v = K
12q:
D(p,q) = ‖u− v‖l2
Updating un and vn:
un = (1− βp)un−1 + βpK12 Iyn
vn = (1− βq)vn−1 + βqK12 Iyn
where the term K12 Iyn is simply the ynth column of matrix K
12 .
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Histogram Difference
Example of Kernel Distance
0 50 100 150Time (n)
100
90
80
70
60
50
40RS
S (d
Bm)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Kern
el d
ista
nce
Figure: RSS (×) and kernel distance (+) time series for a link which aperson crosses at n = 23 and n = 120.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Imaging and Tracking via Kernel Distance
RTI Idea
Divide the area into P pixelsEstimate pixel values x = [x1, x2, · · · , xP ]T , which representhuman presence, from RSS link measurements and model W[Patwari and Agrawal, 2008]
Figure: One example of RTI image estimate.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Imaging and Tracking via Kernel Distance
Kernel Distance-Based Radio Tomographic Imaging(KRTI)
Notation: Let d = [d0, . . . , dL−1]T denote a vector with Lkernel distances, dl = D(pl ,ql)
Problem: Estimate image x from d and model W (ill-posedinverse problem L < P)
Regularized least squares solution:
x = ΠKd where ΠK = (W TW + αC−1x )−1W T .
where α is the regularization parameter, and Cx is thecovariance of the image used in regularization.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Imaging and Tracking via Kernel Distance
Localization and Tracking
The location of a person is estimated as:
z = rq where q = arg maxp
xp
where xp is the pth element of vector x.
For a moving person: Use a Kalman filter on the localizationestimates to track locations over time.
Localization and tracking of multiple people can be achievedfrom KRTI images.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Outline
1 IntroductionDevice-Free LocalizationLimits of Previous Methods
2 MethodsHistogram DifferenceImaging and Tracking via Kernel Distance
3 Experiments and ResultsExperimentsResults
4 Conclusion
Introduction Methods Experiments and Results Conclusion
Experiments
Experiment Testbed
�
��
�
�
�
�
�
A
B
C
�
��
�
�
�
�
�
A
B
C
DE
DE
ZigBee radio node: 2.4 GHz, IEEE 802.15.4
Spin: token passing protocol; when one transmits,others measure RSS
Packet data: node ID and measured RSS values
Laptop-connected note listens all traffic to recordall pairwise RSS
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Experiments
Experiments
34 radio nodes are deployed around a 9 m by 8 m (Exp. 1 - 4) or a12 m by 5 m (Exp. 5) area to locate a stationary person (Exp. 1)or a moving person (Exp. 2 - 5).
Name Task Description
Exp.1 stationary person calm day through-wall
Exp.2 moving person calm day through-wall
Exp.3 moving person windy day with fans
Exp.4 moving person windy day with fans
Exp.5 moving person at a cluttered bookstore
Table: Experimental datasets (new data Exp. 3 and 4 reported in thispaper is available at http://span.ece.utah.edu/data-and-tools).
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Experiments
Experimental Layout (through-wall)
0 2 4 6 8 10X (m)
�20
2
4
6
8Y (
m)
Living room
Kitchen
Door
Door
Camera
Tree
Nodes on stands
Nodes not on stands
Figure: Experiment layout of Exp. 1 - 4.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Experiments
Experimental Layout (cluttered bookstore)
2 0 2 4 6 8 10 12X (m)
1
0
1
2
3
4
5
6
Y (m
)
A
D C
B
RF sensorsShelvesPath
Figure: Experiment layout and picture of Exp. 5.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Results
Imaging of a Stationary Person (Exp. 1)
0 2 4 6 8m
0
1
2
3
4
5
6
7
8
m
0.000.020.040.060.080.100.120.140.160.180.20
0 2 4 6 8m
0
1
2
3
4
5
6
7
8
m
0.000.020.040.060.080.100.120.140.160.180.20
Figure: Imaging results of a stationary person (true location shown as ×)from KRTI (left) and VRTI (right).
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Results
Localization of a Stationary Person (Exp. 1)
0 2 4 6 8 10X (m)
0
2
4
6
8
Y (m
)
Known locationsEstimatesRF sensors
0 2 4 6 8 10X (m)
0
2
4
6
8
Y (m
)
Known locationsEstimatesRF sensors
Figure: Location estimates of a person standing at twenty locations fromKRTI (left, average error 0.71 m, 0.03 sec per estimate) and SMC (right,average error 0.83 m, 3 to 4 minutes per estimate).
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Results
Localization of a Moving Person (Exp. 2 - 5)
Figure: RMSE comparison with variance-based methods – VRTI andsubspace variance-based radio tomography (Zhao and Patwari. NoiseReduction for Variance-Based Device-Free Localization and Tracking. In Proc.of the 8th IEEE SECON, June 2011.)
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Introduction Methods Experiments and Results Conclusion
Results
Tracking of a Moving Person
2 0 2 4 6 8 10 12X (m)
10123456
Y (m
)
Figure: Kalman filter tracking result of Exp. 5 (true path shown as dashline).
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
Outline
1 IntroductionDevice-Free LocalizationLimits of Previous Methods
2 MethodsHistogram DifferenceImaging and Tracking via Kernel Distance
3 Experiments and ResultsExperimentsResults
4 Conclusion
Introduction Methods Experiments and Results Conclusion
Conclusion
Using kernel distance in radio tomography allows us to locateboth stationary and moving people in an LOS or non-LOSenvironment in real-time;
Instead of using “empty-area” calibration, KRTI uses onlinecalibration, which enables applications in emergencysituations;
Real-world experiments show that KRTI outperforms previousDFL methods in localization accuracy and computationalefficiency.
Yang Zhao, Neal Patwari, Jeff M. Phillips, Suresh Venkatasubramanian University of Utah
Radio Tomographic Imaging and Tracking of Stationary and Moving People via Kernel Distance
