Indoor Localization
Qian Zhang
Indoor localization platform providing high accuracy could enable a host of applications
Targeted Location Based Advertising
Indoor Navigation (e.g. Airport Terminals)
Real Life Analytics (Gym, Office, etc..)
Applications of Indoor Localization
Lots of Technologies!
Ultrasonic time of flight
Stereo camera
Ad hoc signal strength
Physical contact
WiFi Beacons
Infrared proximity
Laser range-finding
Array microphone
Floor pressure Ultrasound
• Technologies to be covered in this Chapter:
• Wireless-based solution
• VLC-based solution
• Multi-source based solution
Agenda
01
02
03
Wireless-based Solutions
Multi-source based solutions
VLC-based solutions
Wireless Technologies for Localization Name Effective Range Pros Cons
GSM 35km Long range Very low accuracy
LTE 30km-100km
Wi-Fi 50m-100m Readily available; Medium range
Low accuracy
Ultra Wideband 70m High accuracy High cost
Bluetooth 10m Readily Available; Medium accuracy
Short range
Ultrasound 6-9m High accuracy High cost, not scalable
RFID & IR 1m Moderate to high accuracy
Short range, Line-Of-Sight (LOS)
NFC <4cm High accuracy Very short range
• Fingerprinting: Radar
• Fingerprinting: PinLoc
• SpotFi: Decimeter Level Localization using WiFi
• Push the Limit of WiFi based Localization for Smartphones
• Accurate RFID Positioning in Multipath Environments
Fingerprinting
• Mapping solution
• Address problems with multipath
• Better than modeling complex RF propagation pattern
Fingerprinting
SSID (Name) BSSID (MAC address) Signal Strength (RSSI)
linksys 00:0F:66:2A:61:00 18
starbucks 00:0F:C8:00:15:13 15
newark wifi 00:06:25:98:7A:0C 23
• Easier than modeling
• Requires a dense site survey
• Usually better for symbolic localization
• Spatial differentiability
• Temporal stability
Fingerprinting
Received Signal Strength (RSS) Profiling Measurements
• Construct a form of map of the signal strength behavior in the coverage area
• The map is obtained: – Offline by a priori measurements
– Online using sniffing devices deployed at known locations
• They have been mainly used for location estimation in WLANs
• Different nodes: – Anchor nodes
– Non-anchor nodes
– A large number of sample points (e.g., sniffing devices)
• At each sample point, a vector of signal strengths is obtained – jth entry corresponding to the jth anchor’s transmitted signal
• The collection of all these vectors provides a map of the whole region
• The collection constitutes the RSS model
• It is unique with respect to the anchor locations and the environment
• The model is stored in a central location
• A non-anchor node can estimate its location using the RSS measurements from anchors
Received Signal Strength (RSS) Profiling Measurements
RADAR: An In-Building RF-Based User Location and Tracking system
Paramvir Bahl and Venkata N. Padmanabhan
•Functional Components • Base Stations (Access Points)
• Mobile Users
•Fundamental Idea in RADAR • Signal Strength is a function of the receiver’s location
• Road Maps
•Techniques to build the Road Maps • Empirical Method
• Radio Propagation Model
•Search Techniques • Nearest Neighbor in Signal Space (NNSS)
• NNSS Avg.
• Viterbi-like Algorithm
Data Collection
• Key Step in the proposed approach
• Records the Radio Signal as a function of the user location
• Off-Line Phase • Construct/validate models for signal propagation
• Real-Time Phase (Infer location of user)
• Every packet received by the base station, the WiLIB extracts • Signal Strength
• Noise floor at the transmitter
• Noise floor at the receiver
• MAC address of the transmitter
Data Processing
• Traces collected from the off-line phase are unified into a table consisting of tuples of the format
[ x, y, d, ss(i), snr(i) ] I € {1,2,3}
• Search Algorithm
• NNSS
• NNSS – Avg.
• Viterbi-like Algorithm
• Layout Information
Algorithm and Experimental Analysis
Empirical Method
• 280 combinations of user location and orientation (70 distinct points, 4 orientations on each point)
• Uses the above empirical data recorded in the off-line phase to construct the search space for the NNSS Algorithm
• Algorithm (Emulates the user location problem) • Picks one location and orientation randomly
• Searches for a corresponding match in the rest of the 69 points and orientations
•Comparison with • Strongest Base Station
• Random Selection
Error Distance Values
• Multiple Nearest Neighbor • Increases the accuracy of the Location Estimation
Figure : Multiple Nearest Neighbors T – True Location
G – Guess N1,N2,N3 - Neighbors
N1
N3 N2
G
T
Empirical Method (Cntd. )
Empirical Method (Cntd. )
• Impact of Number of Number of Samples • Accuracy obtained by all the samples can be obtained if only a few samples
are taken
• Impact of User Orientation •Off-line readings for all orientations is not feasible •Work around is to calculate the error distance for all combinations
No. Of Real-Time Samples Error Distance degradation
1 30%
2 11%
3 4%
• Tracking a Mobile User
• Analogous to the user location problem
• New Signal Strength data set
• Window size of 10 samples
• 4 Signal Strength Samples every second
• Limitation of Empirical Method
• To start off with needs an initial signal strength data set
• Relocation requires re-initialization of the initial data set
Empirical Method (Cntd. )
Radio Propagation Model
• Introduction • Alternative method for extracting signal strength information
• Based on a mathematical model of indoor signal propagation
• Issues • Reflection, scattering and diffraction of radio waves
• Needs some model to compensate for attenuation due to obstructions
• Models
• Rayleigh Fading Model : Infeasible
• Rician Distribution Model : Complex
• Wall Attenuation Factor
Wall Attenuation Factor
Radio Propagation Model (Cntd. )
• Advantages: • Cost Effective
• Easily Relocated
Conclusion
• RF-based user location and tracking algorithm is based on • Empirically measured signal strength model
• Accurate
• Radio Propagation Model
• Easily relocated
• RADAR could locate users with high degree of accuracy
• Median resolution is 2-3 meters, which is fairly good
• Used to build “Location Services” • Printing to the nearest printer
• Navigating through a building
• Fingerprinting: Radar
• Fingerprinting: PinLoc
• SpotFi: Decimeter Level Localization using WiFi
• Push the Limit of WiFi based Localization for Smartphones
• Accurate RFID Positioning in Multipath Environments
While most WiFi based localization schemes operate with signal strength based information at the MAC layer, PinLoc recognizes the possibility of leveraging detailed physical (PHY) layer information
Fingerprinting Wireless Channel
• 802.11 a/g/n implements OFDM – Wideband channel divided into subcarriers
– Intel 5300 card exports frequency response per subcarrier
Frequency subcarriers
1 2 3 4 5 6 7 8 9 10 39 48
phase and magnitude over 30 subcarriers richly capture the scattering in the environment
• Two key hypotheses need to hold:
Temporal
• Channel responses at a given location may vary over time
• However, variations must exhibit a pattern – a signature
1.
Spatial
• Channel responses at different locations need to be different 2.
Is WiFi Channel Amenable to Localization?
channel responses from multiple OFDM subcarriers can be a promising location signature
• Measured channel response at different times – Using Intel cards
cluster2
cluster2
cluster1
cluster1
Observe: Frequency responses often clustered at a location
Variation over Time
But not necessarily one cluster per location
cluster2
cluster2
cluster1
cluster1
2 clusters with different
mean and variance
But not necessarily one cluster per location
• Measured channel response at different times – Using Intel cards
Variation over Time
Unique clusters per location
How Many Clusters per Location?
Do all 19 clusters occur
with same frequency?
Most
frequent
cluster
2nd
most
3rd
4th Others
3 to 4 clusters heavily dominate, need to learn these signatures
Unique clusters per location
Cluster Occurrence Frequency
Spatial
• Channel responses at different locations need to be different 2.
Clusters with different
mean and variance
Is WiFi Channel Amenable to Localization?
Temporal
• Channel responses at a given location may vary over time
• However, variations must exhibit a pattern – a signature
1.
Location Signature
What is the Size of a Location?
● Localization granularity depends on size ● RSSI changes in orders of several meters (hence, unsuitable)
Cross correlation with signature at reference location
Channel response changes every 2-3cm
3 cm apart
2 cm apart
Define “location” as 2cm x 2cm area, call them pixels
What is the Size of a Location?
● Localization granularity depends on size ● RSSI changes in orders of several meters (hence, unsuitable)
Will all pixels have unique signatures? But …
Real (H(f))
Im (
H(f
))
Self
Similarity
Cross
Similarity > Max ( )
Pixel 1
Pixel 2
Pixel 3
For correct pixel localization
Self
Similarity
Cross
Similarity > Max ( ) 0 -
Self – Max (Cross)
AP1
Self – Max (Cross)
AP2
Self – Max (Cross)
AP1 and AP2
67% pixel accuracy even with multiple APs
Opportunity:
- Humans exhibit natural (micro) movements
- Likely to hit several nearby pixels
- Combine pixel fingerprints into super-fingerprint
67% accuracy inadequate …
can we improve accuracy?
Intuition: low probability that a set of pixels
will all match well with an incorrect spot
From Pixels to Spots
Combine pixel fingerprints from a 1m x 1m box.
Spot
Pixel
2cm
PinLoc: Architecture and Modeling
Test Data
Parameters: (wK, UK, VK)
Variational Inference (Infer.NET)
PinLoc measures the CFRs at spots of interest during the training phase and tries to identify as many of the unique clusters as possible during a war-driving period
Per pixel signature
Real (H(f))
Im (
H(f
))
Per spot signature
Real (H(f))
Im (
H(f
))
• Evaluated PinLoc (with existing building WiFi) at:
– Duke museum
– ECE building
– Café (during lunch)
• Roomba calibrates
– 4m each spot
– Testing next day
– Compare with Horus (best RSSI based scheme)
PinLoc Evaluation
• 90% mean accuracy, 6% false positives
• WiFi RSSI is not rich enough, performs poorly - 20% accuracy
Accuracy per spot False positive per spot
Performance
• Fingerprinting: Radar
• Fingerprinting: PinLoc
• SpotFi: Decimeter Level Localization using WiFi
• Push the Limit of WiFi based Localization for Smartphones
• Accurate RFID Positioning in Multipath Environments
SpotFi: Decimeter Level Localization using WiFi
Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, Sachin Katti
Stanford University
Requirement for Ideal Localization System
1. Easily Deployable
• Commercial WiFi chips
1. Easily Deployable
• Commercial WiFi chips
• No hardware or firmware change
4
1. Easily Deployable
• Commercial WiFi chips
• No hardware or firmware change
• No User Intervention
5
2. Universal
• Localize any WiFi device
• No specialized sensors
3. Accurate
1 m
• Error of few tens of centimeters
State-of-the-art
System Deployable Universal Accurate
RADAR, Bahl et al, ’00
HORUS, Youssef et al, ’05
ArrayTrack, Xiong et al, ’13
PinPoint, Joshi et al, ’13
CUPID, Sen et al, ’13
LTEye, Kumar et al, ’14
Phaser, Gjengset et al, ’14
Ubicarse, Kumar et al, ’14
SpotFi, Kotaru et al, ’15
System Overview
Localization - Overview
Localization - Overview
Challenge - Multipath
Solving The Multipath Problem
State-of-the-art
Model signal on antennas alone Model signal on both antennas and
subcarriers
SpotFi
Sub
carr
iers
Antennas
𝒇𝟏
𝒇𝟐
𝒇𝟑
𝒇𝟒
Overall Architecture
• SpotFi collects CSI and RSSI measurements from all the APs that can hear the packet transmitted by the target • SpotFi calculates the ToF and AoA of all the propagation paths from the target to each of the APs • SpotFi then identifies the direct path between the target and the AP that did not undergo any
reflections • SpotFi estimates the location of the target by using the direct path AoA estimates and RSSI
measurements from all the APs
Step 1: Resolve Multipath
𝜽𝟏, 𝝉𝟏
𝜽𝟐, 𝝉𝟐
Equal Distance
Line
Signal Modeling
Phase
Distance travelled by the WiFi signal
Ph
ase
1 / frequency
0
𝐏𝐚𝐭𝐡 𝐃𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐜𝐞 =𝟐𝝅
𝒘𝒂𝒗𝒆 𝒍𝒆𝒏𝒈𝒕𝒉∗ (𝑷𝒉𝒂𝒔𝒆 𝑫𝒊𝒇𝒇𝒆𝒓𝒆𝒏𝒄𝒆)
Equal Phase Line
Signal Modeling – AoA (Angle of Arrival)
Signal Modeling – AoA (Angle of Arrival)
Uniform linear array consisting of M antennas:
• For AoA of θ, the target’s signal travels an additional distance of d*sin(θ) to the second antenna in the array compared to the first antenna
• This results in an additional phase of -2π*d*sin(θ)*f/c at the second antenna
Signal Modeling - AoA
Define Φ1 = e−𝑗2𝜋𝑑sin𝜃1
𝑐𝑓
𝜽𝟏
1
Γ1 is complex attenuation of the path. Φ1 depends on AoA
Phase at the antenna 1: 𝑥1 = Γ1 Phase at the antenna 2: 𝑥2 = Γ1Φ1 Phase at the antenna 3: 𝑥3 = Γ1Φ1
2 2 3
Say There Are Two Paths…
Say There Are Two Paths…
𝑥1 = Γ1 𝑥2 = Γ1Φ1 𝑥3 = Γ1Φ1
2
Say There Are Two Paths…
𝑥1 = Γ1 + Γ2 𝑥2 = Γ1Φ1 + Γ2Φ2 𝑥3 = Γ1Φ1
2 + Γ2Φ22
Problem Statement
CSI - Known
𝑥1 = Γ1 + Γ2 𝑥2 = Γ1Φ1 + Γ2Φ2 𝑥3 = Γ1Φ1
2 + Γ2Φ22
Problem Statement
𝑥1 = Γ1 + Γ2 𝑥2 = Γ1Φ1 + Γ2Φ2 𝑥3 = Γ1Φ1
2 + Γ2Φ22
Parameters - Unknown
Problem Statement
Number of paths (or AoAs) < Number of antennas (or equations)
𝑥1 = Γ1 + Γ2 𝑥2 = Γ1Φ1 + Γ2Φ2 𝑥3 = Γ1Φ1
2 + Γ2Φ22
Typical Indoor Multipath
That’s A Problem
State-of-the-art Commodity WiFi chips
Number of antennas/equations should be at least 5
How To Obtain More Equations?
Model signal on both antennas and subcarriers
Sub
carr
iers
Antennas
𝒇𝟏
𝒇𝟐
𝒇𝟑
𝒇𝟒
𝒇𝟏
𝒇𝟐
Each Subcarrier Gives New Equations
Define 𝜴𝟏 = 𝒆−𝒋𝟐𝝅 𝒇𝟐−𝒇𝟏 𝝉𝟏
Γ1 is complex attenuation of the path. Ω1 depends on incoming signal ToF
Phase at first subcarrier: 𝑥1 = Γ1 Phase at second subcarrier: 𝑥2 = Γ1Ω1
Signal Modeling – ToF (Time of Flight)
Estimate both AoA and ToF
More number of equations in terms of parameter of our interest
Say There Are Two Paths…
At first subcarrier, for 3 antennas
𝑥1 = Γ1
𝑥2 = Γ1Φ1
𝑥3 = Γ1Φ12
At second subcarrier, for 3 antennas
𝑦1 = Γ1Ω1
𝑦2 = Γ1Φ1Ω1
𝑦3 = Γ1Φ12Ω1
Say There Are Two Paths…
At first subcarrier, for 3 antennas
𝑥1 = Γ1 + Γ2
𝑥2 = Γ1Φ1 + Γ2Φ2
𝑥3 = Γ1Φ12 + Γ2Φ2
2
At second subcarrier, for 3 antennas
𝑦1 = Γ1Ω1 + Γ2
𝑦2 = Γ1Φ1Ω1 + Γ2Φ2Ω2
𝑦3 = Γ1Φ12Ω1 + Γ2Φ2
2Ω2
𝑥1 = Γ1 + Γ2
𝑥2 = Γ1Φ1 + Γ2Φ2
𝑥3 = Γ1Φ12 + Γ2Φ2
2
𝑦1 = Γ1Ω1 + Γ2
𝑦2 = Γ1Φ1Ω1 + Γ2Φ2Ω2
𝑦3 = Γ1Φ12Ω1 + Γ2Φ2
2Ω2
Problem Statement
Sub
carr
ier
1
Sub
carr
ier
2
CSI - Known
𝑦1 = Γ1 + Γ2
𝑦2 = Γ1Φ1 + Γ2Φ2
𝑦3 = Γ1Φ12 + Γ2Φ2
2
𝑦1 = Γ1Ω1 + Γ2
𝑦2 = Γ1Φ1Ω1 + Γ2Φ2Ω2
𝑦3 = Γ1Φ12Ω1 + Γ2Φ2
2Ω2
Problem Statement
Sub
carr
ier
1
Sub
carr
ier
2
Parameters - Unknown
𝑥1 = Γ1 + Γ2
𝑥2 = Γ1Φ1 + Γ2Φ2
𝑥3 = Γ1Φ12 + Γ2Φ2
2
𝑦1 = Γ1Ω1 + Γ2
𝑦2 = Γ1Φ1Ω1 + Γ2Φ2Ω2
𝑦3 = Γ1Φ12Ω1 + Γ2Φ2
2Ω2
Problem Statement
Sub
carr
ier
1
Sub
carr
ier
2
Number of equations =
Number of Subcarriers x
Number of Antennas
AoA, ToF Estimates
𝜽𝟏, 𝝉𝟏
𝜽𝟐, 𝝉𝟐
Step 2: Identify Direct Path
𝜽𝟏, 𝝉𝟏
𝜽𝟐, 𝝉𝟐
𝜽𝟏, 𝝉𝟏
AoA, ToF Estimates
𝜽𝟏, 𝝉𝟏
𝜽𝟐, 𝝉𝟐
Use Multiple Packets
𝜽𝟏, 𝝉𝟏
𝜽𝟐, 𝝉𝟐
Use Multiple Packets
Use Multiple Packets
Use Multiple Packets
Direct Path Likelihood
Higher weight
Higher weight
Higher weight
Lower weight
Lower weight
• Smaller ToF
Direct Path Likelihood
Higher weight
Lower weight
Lower weight
Lower weight
• Smaller ToF
• Tighter Cluster
Lower weight
Direct Path Likelihood
Higher weight
Higher weight
Lower weight
Lower weight
• Smaller ToF
• Tighter Cluster
• More Packets
Lower weight
Highest Direct Path Likelihood
Step 3: Localize The Target
𝜽𝟏, 𝝉𝟏
𝜽𝟐, 𝝉𝟐
𝜽𝟏, 𝝉𝟏
Use Multiple APs
Direct Path AoA = 45 degrees Signal Strength = 10 dB
Direct Path AoA = 10 degrees Signal Strength = 30 dB
Find location that best explains the AoA and Signal Strength
at all the APs
Direct Path AoA = -45 degrees Signal Strength = 20 dB
Use Different Weights
Use different weights for different APs
Direct Path AoA = 45 degrees Signal Strength = 10 dB Direct Path Likelihood
Direct Path AoA = 10 degrees Signal Strength = 30 dB Direct Path Likelihood
Direct Path AoA = -45 degrees Signal Strength = 20 dB Direct Path Likelihood
Evaluation
52 m
Testbed
Access point Target
AP Locations Target Locations
0
0.2
0.4
0.6
0.8
1
0.05 0.5 5
Emp
iric
al C
DF
Localization Error (m)
Indoor Office Deployment
52 m
16 m
0.4 m
ArrayTrack Ubicarse SpotFi
0.3 m 0.4 m 0.4 m
AP Locations Target Locations
Stress Test – Obstacles Blocking The Direct Path
AP Locations Target Locations 52 m
0
0.2
0.4
0.6
0.8
1
0.05 0.5 5
Emp
iric
al C
DF
Localization Error (m)
Stress Test – Obstacles Blocking The Direct Path
1.3 m
AP Locations Target Locations 52 m
Effect of WiFi AP Deployment Density
0
0.2
0.4
0.6
0.8
1
0.05 0.5 5
Emp
iric
al C
DF
Localization Error (m)
3 APs
4 APs
5 APs
0.8 m
Conclusion
• Deployable: Indoor Localization with commercial WiFi chips
• Accurate: Accuracy comparable to state-of-the-art localization systems which are not suitable for wide deployments
• Universal: Simple localization targets with only a WiFi chip
• Fingerprinting: Radar
• Fingerprinting: PinLoc
• SpotFi: Decimeter Level Localization using WiFi
• Push the Limit of WiFi based Localization for Smartphones
• Accurate RFID Positioning in Multipath Environments
Push the Limit of WiFi based Localization
for Smartphones
Hongbo Liu, Yu Gan, Jie Yang, Simon Sidhom, Yan Wang, Yingying Chen
Department of Electrical and Computer Engineering Stevens Institute of Technology
Fan Ye IBM T. J. Watson Research Center
The Need for High Accuracy Smartphone Localization
Shopping Mall Airport
Help users navigation inside large and complex indoor environment, e.g., airport,
train station, shopping mall.
Understand customers visit and stay patterns for business
Train Station
Smartphone Indoor Localization - What has been done?
Contributions in academic research
Commercial products
Localization error up to 10 meters
Google Map Shopkick
Locate at the granularity of stores
WiFi indoor localization
High accuracy indoor localization
WiFi enabled smartphone indoor localization
RADAR [INFOCOM’00], Horus [MobiSys’05], Chen et.al[Percom’08]
Cricket [Mobicom’00], WALRUS [Mobisys’05], DOLPHIN [Ubicomp’04], Gayathri et.al [SECON’09]
SurroundSense [MobiCom’09], Escort [MobiCom’10], WILL[INFOCOM’12], Virtual Compass [Pervasive’10]
Is it possible to achieve high accuracy localization using most prevalent WiFi infrastructure?
0
5
10
15
20
25
30
35
40
45
AP 1 AP 2 AP 3 AP 4
6 - 8 meters ~ 2 meters
Root Cause of Large Localization Errors
Permanent environmental settings, such as furniture placement and walls.
Transient factors, such as dynamic obstacles and interference.
Am I here?
I am around here.
32: [ -22dB, -36dB, -29dB, -43dB ]
48: [ -24dB, -35dB, -27dB, -40dB]
Orientation, holding position, time of day, number of samples
Physically distant locations share similar WiFi Received Signal Strength !
Rec
eive
d S
ign
al S
tren
th
(dB
m) WiFi as-is is not a suitable candidate for high accurate
localization due to large errors
Is it possible to address this fundamental limit without the need
of additional hardware or infrastructure?
Inspiration from Abundant Peer Phones in Public Place
Increasing density of smartphones in public spaces
Provide physical constraints from nearby peer phones
How to capture the physical constraints?
Target
Peer 1
Peer 2
Peer 3
Basic Idea
WiFi Position Estimation Acoustic Ranging
Interpolated Received Signal Strength Fingerprint Map
Exploit acoustic signal/ranging to construct peer constraints Target
Peer 1 Peer 2
Peer 3
• Peer assisted localization
• Fast and concurrent acoustic ranging of multiple phones
• Ease of use
System Design Goals and Challenges
Exactly what is the algorithm to search for the best fit position and quantify the signal similarity so that to reduce large errors?
How to design and detect acoustic signals?
Need to complete in short time.
Not annoy or distract users from their regular activities.
Rigid graph construction
Sound signal design
Acoustic signal detection
System Work Flow
Identify nearby peers
Beep emission strategy
Only phones close enough can detect recruiting signal
Peer phones willing to help send their IDs to the server
Employ virtual synchronization scheme based on time-multiplexting
Deploy extra timing buffers to accommodate variations in the reception of the schedule at different phones, e.g., 20 ms
Peer recruiting & ranging
Peer assisted localization
Peer recruiting & ranging
WiFi position estimation
Peer recruiting & ranging
Minimizing the impact on users’ regular activities
Fast ranging
Unobtrusive to human ears
Robust to noise
Change point detection
Correlation method
16 – 20 KHz
ADP2
Lab Train Station Shopping Mall Airport
HTC EVO
Construct the graph G and G’ based on initial WiFi position estimation and the acoustic ranging measurements.
Graph G based on WiFi position estimation
Rigid Graph G’ based on acoustic ranging
Peer recruiting & ranging
Rigid graph construction
Peer assisted localization
WiFi position estimation
Rigid graph construction
Rigid graph construction
System Work Flow
System Work Flow
Peer assisted localization
Peer recruiting & ranging
Rigid graph construction
Peer assisted localization
WiFi position estimation Peer assisted localization
Graph Orientation Estimation Translational Movement
WiFi based graph Acoustic ranging graph
• Prototype Devices
• Trace-driven statistical test Feed the training data as WiFi samples
Perturb distances with errors following the same distribution in real environments
Prototype and Experimental Evaluation
ADP 2 HTC EVO
• Localization performance across different real-world environments (5 peers)
Localization Accuracy
Peer assisted method is robust to noises in different environments
Median error 90% error
Lab Train Station Shopping Mall Airport
• Overall Latency
• Energy Consumption
Overall Latency and Energy Consumption
Negligible impact on the battery life
• e.g., with additional power consumption at about 320mW on HTC EVO - lasts 12.7 hours with average power of 450mW
Pose little more latency than required in the original WiFi localization about 1.5 ~ 2 sec
• Peer Involvement
• Movements of users
• Triggering peer assistance
Discussion
Provides the technology for peer assistance
Up to users to decide when they desire such help
Do not pose more constraints on movements than existing WiFi methods
Affect the accuracy only during sound-emitting period
• Happens concurrently and shorter than WiFi scanning
Use incentive mechanism to encourage and compensate peers that help a target’s localization
• Leverage abundant peer phones in public spaces to reduce large localization errors
• Exploit minimum auxiliary COTS sound hardware readily available on smartphones
• Demonstrate our approach successfully pushes further the limit of WiFi localization accuracy
Conclusion
Aim at the most prevalent WiFi infrastructure
Do not require any special hardware
Utilize much more accurate distance estimate through acoustic ranging to capture unique physical constraints
Lightweight in computation on smartphones
In time not much longer than original WiFi scanning
With negligible impact on smartphone’s battery life time
• Fingerprinting: Radar
• Fingerprinting: PinLoc
• SpotFi: Decimeter Level Localization using WiFi
• Push the Limit of WiFi based Localization for Smartphones
• Accurate RFID Positioning in Multipath Environments
Accurate RFID Positioning in Multipath Environments
Jue Wang & Dina Katabi ACM Sigcomm 2013
RFIDs
Battery-free RF stickers with unique IDs
RFIDs
5-cent stickers to tag any and every object
Reader’s range is ~15m
Imagine you can localize RFIDs to within 10 to 15 cm!
No more customer checkout lines
If we can locate RFID to within 10 to 15cm
No more customer checkout lines
If we can locate RFID to within 10 to 15cm
The Challenge: Multipath Effect
Localization uses RSSI or Angle-of-Arrival (AoA)
But, signal bounces off objects in the environment
Angle of signal is not the direction of the RFID
Multipath propagation limits the Accuracy of RFID localizations
PinIt
Accurate RFID localization (e.g., 10 to 15cm) even in
multipath and non-line-of-sight settings
• Focuses on proximity to reference RFIDS
• Exploits multipath effects to increase accuracy
PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections
PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections
PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections
PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections
Nearby RFIDs have similar profiles with smaller shifts in the peaks
Implementation & Evaluation
• Implemented a PinIt Reader in USRP
• Commercial off-the-shelf RFIDs
• Mounted the antenna on an iRobot that slides back and forth
Positioning Accuracy
• 200 RFIDs deployed on the shelves in the library spaced by 15 cm
PinIt improve the accuracy by 6x in comparison to AoA and 10x in comparison to RSSI
Automatic Checkout
Five items in two adjacent baskets at checkout
Which Items Belong to Which Basket?
Is the Cookie Bag in the Orange or Blue Basket?
i
i+2
i
i i
time time
Why Dynamic Time Warping (DTW)?
Any distance (Euclidean, Manhattan, …) which aligns the i-th point on one time series with the i-th point on the other will produce a poor similarity score.
A non-linear (elastic) alignment produces a more intuitive similarity measure, allowing similar shapes to match even if they are out of phase in the time axis.
C
Q
C
Q
How is DTW Calculated?
KwCQDTWK
k k1min),(
Every possible warping between two time series, is a path though the matrix. We want the best one…
(i,j) = d(qi,cj) + min{ (i-1,j-1), (i-1,j ), (i,j-1) }
This recursive function gives us the minimum cost path
Warping path w
C
Q
One more note
Warping path w
The time series can be of different lengths..
C Q
Is the Noodle in the Orange or Blue Basket?
Brief Summary
• PinIt provides accurate RFID positioning even in multipath
and NLOS settings
• It uses DTW to compare RFID multipath profiles
• It enables new applications including eliminating checkout
lines, object tracking in libraries and pharmacies, smart
homes, …
Agenda
01
02
03
Wireless-based Solutions
Multi-source based solutions
VLC-based solutions
Wearables Can Afford: Light-weight Indoor Positioning with Visible Light
Zeyu Wang, Zhice Yang, Jiansong Zhang, Chenyu Huang,Qian Zhang
Hong Kong University of Science and Technology
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
Accuracy is not enough (~several meters)
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
• Dedicated localization infrastructure
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
• Dedicated localization infrastructure
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
• Dedicated localization infrastructure
Indoor Localization
• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map
• Dead reckoning: Use inertial sensors to calculate moving path
• Dedicated localization infrastructure
Complex and high-cost to handle RF multipath
Visible Light Positioning
Visible Light Positioning
…1
…1
...1…
Visible Light Positioning
…1
…1
...1…
…2
…2
...2…
• Visible Light Positioning (VLP) is an emerging positioning technique that based Visible Light Communication (VLC) – Light bulbs are densely deployed
Location anchors are ubiquitous
– Light beam is very directional
No multipath, localization is simple and accurate
– More…
• Light is free of radio wave
• Positioning through light bulbs is green in energy
Visible Light Positioning
How VLC generally works?
• Modulate Light Intensity
Normal Light
Modulated Light
Time
Problem in VLC: Flickering
10Hz 100Hz >1000Hz
Consequence: Overhead in Client
• Additional Receiving Device – Using customized light sensor that
requires cumbersome calibration[1]
• High Computational Overhead – Using very high resolution camera to
extract the roller shuttering patterns[2]
>1000Hz
[1] L. Li etc. “Epsilon: A visible light based positioning system” in NSDI’14 [2] Y.-S. Kuo etc. “Luxapose: Indoor positioning with mobile phones and visible light” in Mobicom’14
Must be LED
These overhead can hardly be afforded in wearables. Can they be eliminated?
Idea: Flickering-free Modulation
• Instead of changing the intensity, we modulate information by changing the polarization of light Human eyes CANNOT perceive changes in polarization
Therefore low baud rate in transmitters
Therefore low decoding overhead in clients
PIXEL
Review the display mechanism of LCD !
Back Light
Polarizing Film
PIXEL: One Pixel from LCD
Polarizing Film Eyes
0V
PIXEL: One Pixel from LCD
Back Light
Polarizing Film
Polarizing Film Eyes
Voltage
Liquid Crystal
5V
PIXEL: One Pixel from LCD
Liquid Crystal
Back Light
Polarizing Film
Polarizing Film Eyes
Camera
Eyes
VLC Transmitter
PIXEL: VLC Transmitter
Voltage
Liquid Crystal
Back Light
Polarizing Film
Polarizing Film
Eyes
Locatio
n…
Sun
PIXEL: VLP Architecture
Polarizing Film
VLC Transmitter
… Lo
cation
…
… Lo
cation
…
… Lo
cation
…
…
Challenge: User Mobility
SNR
45° 135°
Challenge: User Mobility (Cont.)
Receiving Direction
Voltage “Low”
Voltage “High”
Received Light Intensity
Solution: Dispersion
Solution: Dispersion (Cont.)
Disperse the Polarization of Different Colors into Different Directions
Dispersor
Solution: Dispersion (Cont.)
Receiving Direction
Received Color
SNR
45° 135°
Voltage “Low”
Voltage “High”
Positioning Method
1 3
2
1
2
3
Positioning Method
1 3
2
1
2
3
Challenges: Less Beacons
• Existing methods for camera-based VLC localization require multiple beacon lamps(3 or more) being captured in a single image
• Field Test: 2 or less beacon lamps can be captured by the front camera in normal holding position Portable cameras do not have wide Field of View
The ceiling of buildings is normal limited to several meters.
Example: 3m below, camera of iPhone 6 can only cover 3*3m2 of the ceiling.
Challenges: Less Beacons (Cont.)
Challenges: Less Beacons (Cont.)
1 2
Location Ambiguity
• The position of the receiver has 6 degrees of freedom: 3 in location and 3 in 3D orientation.
• Each received beacon adds 2 AoA constraints to the position and orientation.
• The gravity sensor adds 2 constraints to the 3D orientation.
Two beacons are enough
Solution: Sensor Assisted Localization
1 2
Location Ambiguity
Gravity
Solution: Sensor Assisted Localization
Implementation
• VLC Transmitter
– Polarizing film ($0.001/cm2)
– LCD with only one pixel ($0.03/cm2)
– Glass box with optical rotation liquid
– 14Hz Baud Rate
– Location Beacon
• 5bit Preamble + 8bit Location ID + 4bit CRC
• Client
– Polarizing film ($0.001/cm2)
– Android App with VLC decoding and VLP algorithm
• Smart phone: Galaxy S II (1.2GHz CPU, 8 Megapixel Camera)
• Wearable: Google Glass
Evaluation-VLC
VLC Transmitter
𝜃 30
25
20
15
10
5
0
SNR
(d
B)
0 20 40 60 80 100 120 140 160 180
Receiver's Orientation 𝜃 (degree)
w/o dispersor
with dispersor
30 × 40
60 × 80
Evaluation-VLC
30
25
20
15
10
5
0
SNR
(d
B)
1 2 3 4 5 6 7 8 9 10 11 12 13
Distance (m)
120 × 160
VLC Transmitter
𝑑
Evaluation-VLP
1 2
3
4 5
7 8
6
1.8m
2.4m
1
0.8
0.6
0.4
0.2
0 0 10 20 30 40 50
Positioning Error (cm)
CD
F
Google Glass 300MHz Google Glass 600MHz Google Glass 800MHz
Evaluation-VLP
1
0.8
0.6
0.4
0.2
0 0 50 100 150 200 250
VLP Processing Time Cost (ms)
CD
F Samsung Galaxy SII 1200MHz
Conclusion
• We introduce a light weight VLC method that based on modulating light’s polarization
• We propose to use optical rotation material/dispersor to hand users’ mobility
• We implement and evaluate the VLP system, and results show submeter accuracy can be achieved in both smart phone and wearables.
Agenda
01
02
03
Wireless-based Solutions
Multi-source based solutions
VLC-based solutions
SurroundSense: Mobile Phone Localization via Ambience Fingerprinting
Ionut Constandache, Martin Azizyan and Romit Roy Choudhury
Context
Pervasive wireless connectivity
+
Localization technology
=
Location-based applications
Location-Based Applications (LBAs)
• For Example: – GeoLife shows grocery list when near Walmart
– MicroBlog queries users at a museum
– Location-based ad: Phone gets coupon at Starbucks
• iPhone AppStore: 3000 LBAs, Android: 500 LBAs
Location-Based Applications (LBAs)
• For Example: – GeoLife shows grocery list when near Walmart
– MicroBlog queries users at a museum
– Location-based ad: Phone gets coupon at Starbucks
• iPhone AppStore: 3000 LBAs, Android: 500 LBAs
• Location expresses context of user – Facilitates content delivery
Location is an IP address As if for content delivery
Thinking about Localization
from an application perspective…
Emerging location based apps need
place of user, not physical location
Starbucks, RadioShack, Museum, Library
Latitude, Longitude
We call this Logical Localization …
Can we convert from
Physical to Logical Localization?
Can we convert from
Physical to Logical Localization?
State of the Art in Physical Localization:
1. GPS Accuracy: 10m
2. GSM Accuracy: 100m
3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-100m
Widely-deployable localization technologies have errors in the range of several meters
Can we convert from
Physical to Logical Localization?
State of the Art in Physical Localization:
1. GPS Accuracy: 10m
2. GSM Accuracy: 100m
3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-100m
Several meters of error is inadequate
to logically localize a phone
Physical Location Error
Several meters of error is inadequate
to logically localize a phone
RadioShack Starbucks
Physical Location Error
The dividing-wall problem
Contents
• SurroundSense
• Evaluation
• Limitations and Future Work
• Conclusion
It is possible to localize phones by sensing the ambience
Hypothesis
such as sound, light, color, movement, WiFi …
Sensing over multiple dimensions extracts more information from the ambience
Each dimension may not be unique,
but put together, they may provide a
unique fingerprint
B A C D E
Should Ambiences be Unique Worldwide?
F G
H J
I
L M N
O
P Q
Q R
K
SurroundSense
• Multi-dimensional fingerprint – Based on ambient sound/light/color/movement/WiFi
Starbucks
Wall
RadioShack
Should Ambiences be Unique Worldwide?
B A C D E
F G
H J
I
K L
M N O
P Q
Q R
GSM provides macro location (strip mall) SurroundSense refines to Starbucks
+
Ambience Fingerprinting
Test Fingerprint
Sound
Acc.
Color/Light
WiFi
Logical Location
Matching
Fingerprint Database
=
Candidate Fingerprints
GSM Macro Location
SurroundSense Architecture
Fingerprints
• Sound:
(via phone
microphone)
• Color:
(via phone
camera)
Amplitude Values -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
No
rmal
ized
Co
un
t
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
Acoustic fingerprint
(amplitude distribution)
Color and light fingerprints on HSL space
Ligh
tnes
s
1
0.5
0
Hue
0
0.5
1 0 0.2 0.4 0.6
0.8 1
Saturation
Fingerprints
• Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Moving
Fingerprints • Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Moving
Queuing
Fingerprints
• Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Queuing Seated
Moving
Fingerprints
• Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Pause for product browsing
Moving
Fingerprints
• Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Pause for product browsing
Short walks between product browsing
Moving
Fingerprints
• Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Walk more
Moving
Fingerprints
• Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Walk more Quicker stops
Moving
Fingerprints
• Movement: (via phone accelerometer)
• WiFi: (via phone wireless card)
Cafeteria Clothes Store Grocery Store
Static
ƒ(overheard WiFi APs)
Moving
Discussion
• Time varying ambience – Collect ambience fingerprints over different time windows
• What if phones are in pockets? – Use sound/WiFi/movement
– Opportunistically take pictures
• Fingerprint Database – War-sensing
Contents
• SurroundSense
• Evaluation
• Limitations and Future Work
• Conclusion
Evaluation Methodology
• 51 business locations – 46 in Durham, NC
– 5 in India
• Data collected by 4 people – 12 tests per location
• Mimicked customer behavior
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Acc
ura
cy (
%)
Cluster
Localization accuracy per cluster
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Acc
ura
cy (
%)
Cluster
Localization accuracy per cluster
Multidimensional sensing
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Fault tolerance
Acc
ura
cy (
%)
Cluster
Localization accuracy per cluster
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Acc
ura
cy (
%)
Cluster
Localization accuracy per cluster Sparse WiFi APs
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
No WiFi APs
Acc
ura
cy (
%)
Cluster
Localization accuracy per cluster
Evaluation: Per-Scheme Accuracy
Mode WiFi Snd-Acc-WiFi Snd-Acc-Lt-Clr SS
Accuracy 70% 74% 76% 87%
Evaluation: User Experience
Random Person Accuracy
Average Accuracy (%) 0 10 20 30 40 50 60 70 80 90 100
1
0.9
0.8
0.7
0.6
0.5
C
DF
0.4
0.3
0.2
0.1
0
WiFI
Snd-Acc-WiFi
Snd-Acc-Clr-Lt
SurroundSense
Economics forces nearby businesses to be different
Not profitable to have 3 coffee shops
with same lighting, music, color, layout, etc.
SurroundSense exploits this ambience diversity
Why does it work?
The Intuition:
Contents
• SurroundSense
• Evaluation
• Limitations and Future Work
• Conclusion
Limitations and Future Work
• Energy-Efficiency
• Localization in Real Time
• Non-business locations
Limitations and Future Work
• Energy-Efficiency – Continuous sensing likely to have a large energy draw
• Localization in Real Time
• Non-business locations
Limitations and Future Work
• Energy-Efficiency – Continuous sensing likely to have a large energy draw
• Localization in Real Time – User’s movement requires time to converge
• Non-business locations
Limitations and Future Work
• Energy-Efficiency – Continuous sensing likely to have a large energy draw
• Localization in Real Time – User’s movement requires time to converge
• Non-business locations – Ambiences may be less diverse
Contents
• SurroundSense
• Evaluation
• Limitations and Future Work
• Conclusion
SurroundSense
• Today’s technologies cannot provide logical localization
• Ambience contains information for logical localization
• Mobile Phones can harness the ambience through sensors
• Evaluation results: – 51 business locations,
– 87% accuracy
• SurroundSense can scale to any part of the world
End of This Chapter