Using Sensors and Location Data for

Cutting-edge User Experiences in Mobile

Applications

Overview

Modern mobile devices like tablets are equipped with various sensors (accelerometer, gyroscope,

magnetometer, GPS, etc.). To fully explore the use of the fused motion and location data, this article

provides information on the best practices and importance of integrating/filtering/tuning the sensor

data in end-user applications. The ultimate goal is to empower application developers to create

cutting-edge user experiences in their applications.

This article discusses:

• Overview of sensors, data provided by Windows* Runtime APIs, usage scenarios, and challenges on

using the sensor data in applications

• Illustrative examples on how to integrate, filter, and tune sensor data in user applications for an

immersive, seamless end-user experience

• Optimizing the location user-experience by tuning location parameters and filtering location data

noise

Contents

Overview ....................................................................................................................................................... 1

Contents ........................................................................................................................................................ 1

Sensor Overview ........................................................................................................................................... 2

Motion Sensors in Windows* 8 Runtime.................................................................................................. 3

Location Sensors in Windows* 8 Runtime ................................................................................................ 4

Motion Sensors ............................................................................................................................................. 5

Accelerometer........................................................................................................................................... 5

Gyrometer ............................................................................................................................................... 10

Sensor Fusion .......................................................................................................................................... 11

Location Sensors ......................................................................................................................................... 12

Filtering Location Data ............................................................................................................................ 14

Conclusion ................................................................................................................................................... 15

About the Author ........................................................................................................................................ 15

Sensor Overview

The types and form factors of devices available with sensors is continuing to expand. Various phone

designs, a range of tablet sizes, UltraBook™ devices, convertible tablets, etc. are just a few examples

of the many form factors available with sensors. These unique devices open up interesting

opportunities for developers to create sensor-based applications.

In addition to new form factors, the types and capabilities of the sensors available to consumers and

application developers are ever increasing. Most mobile devices today are equipped with

accelerometers, gyrometers, magnetometers, GPS, Wi-Fi*, touch displays, Bluetooth*, NFC, and more.

Sensors that were once available only internally to the device such as thermometers are now

making their way to device APIs. The additional sensors available to application developers create

new opportunities for developers to create compelling applications.

Motion Sensors in Windows 8 Runtime

New for Windows 8 UI application development is a powerful set of APIs for using sensors. The APIs

include a set of common interfaces for most of the sensor components and provide both event and

polling-based mechanisms for retrieving data.

The following image shows an overview of the different components involved in using the Windows

8 Runtime for motion sensor application development.

Inputs to the Windows 8 Runtime displayed are the raw sources of motion sensor data. The Ambient

Light Sensor is shown for completeness since it uses the same common API for data retrieval.

Outputs of the Runtime expose several components available to you as an application developer.

These include components that use multiple sensors to provide sensor reading data such as the

Compass, Inclinometer, and Orientation Sensor.

Windows.devices.sensors is the namespace you will find the APIs to retrieve sensor event data.

The following snippet is C# code showing an example of how to retrieve the accelerometer present

in the system and setup a method AccelerometerReadingChanged that will be called whenever

the accelerometer data is updated by the system.

Accelerometer acc = Windows.Devices.Sensors.Accelerometer.GetDefault(); if (acc != null) {

acc.ReadingChanged +=new typedEventHandler<Accelerometer, AccelerometerReadingChangedEventArgs>(AccelerometerReadingChanged);

}

The interval at which the event will be called is configurable. The property

minimumReportInterval will tell you what the highest possible frequency is for this sensor. This

value changes from device to device and is also influenced by the driver.

Location Sensors in Windows 8 Runtime

The location sensor APIs have a similar event-based interface and provide a common framework for

retrieving location data that could come from a variety of location data sources.

The diagram below illustrates how Windows 8 Runtime uses a variety of sources of location data,

multiplexes the geolocation data from all available location sensors, and returns the “best” location

data.

The Windows 8 Runtime namespace windows.devices.geolocation is used to access the

geolocation information. Creation of a location object is done with the following code:

Geolocator geo = new Geolocator();

To poll and detect the device’s current location use Geolocator.GetGeopositionAsync().

if (geo != null) { IGeoposition pos = await geo.GetGeopositionAsync();

}

This could be a lengthy operation and is denoted as being asynchronous. The use of the await

keyword makes this easy to handle in your UI code. By using await, the calling thread is not blocked

while this call is made and once GetPositionAsync returns, code execution continues after the async

function. To respond to location updates, hook an event callback geo_PositionChanged as below:

if (geo != null) { geo.PositionChanged += new TypedEventHandler<Geolocator,

PositionChangedEventArgs>(geo_PositionChanged); }

Additionally, tunable options for controlling how often you are notified about events include desired

accuracy, report time interval, and movement threshold.

Motion Sensors

Accelerometer-, compass-, and gyroscope-based apps are some of the most popular and have a

variety of applications including:

Augmented Reality

Games

Orientation

Pedometer

Navigation

Remotely-controlled Devices

Biofeedback

The Windows 8 runtime provides robust APIs and sensor fusion to incorporate in your applications,

but sometimes, applications need to build on top of this for the best user experience. The following

sections go into detail about how some of the common motion sensors work, what can be done to

improve the data of the sensor, and what are some of the limitations you might run into.

Accelerometer

The accelerometer sensor measures how fast the velocity of the device is changing over time. This

could be measuring the device shaking, moving from point A to B, or rotating. The following image

displays some characteristics of the accelerometer as the device is rotated by an angle

counterclockwise.

Using gravity to our advantage, we can quickly determine the angle of rotation by looking at the

angle between the X,Y accelerometer values and the –Y axis. The alternative way to determine the

rotation would be to isolate gravity and measure just the changes in the accelerometer values that

occurred during the rotation. You would then integrate the values to give you velocity and once

again to give you distance travelled.

One major issue with both of these methods is that the data source of the accelerometer is rather

noisy. The following graph looks at the raw sensor readings as the device is rotated 90 degrees.

Starting at the left of the graph, the device is in a landscape position with the -Y axis pointing

straight down. Here you see the effect of gravity with a -1 value. After the rotation is complete, the

–X axis is pointing straight down and gravity is now reporting -1 g’s, while the Y axis no longer has

any gravity component. The other notable characteristic of the graph is the amount of noise or jitter

in the lines. Using this raw data directly in your application may not produce ideal results. One

straightforward algorithm to deal with the jitter is a smoothing function that looks like this:

( )

= .15

Corresponding C# code might look like the following:

CalcRollingValue(double val, double prevVal) { return val * .15 + .85f * prevVal; }

This rolling average function removes high frequency noise by only allowing 15% of the current

value to be used at any new data point. The downside is that a large change in the accelerometer

values that you cared about would not be fully realized for several samples. As a result, the smaller

the alpha value the larger the lag introduced. Here is the same graph of a rotation with the

smoothing filter applied:

Immediately visible is that the noise visible in the original graph is much more subtle in the filtered

graph.

One other way to use the smoothing algorithm is to assist in isolating the gravity component of

acceleration values. To do this, subtract the rolling value from the instantaneous value and you are

left with only the change in value from the average. The following code and graph illustrate what

this looks like with the same rotation of 90 degrees:

float accelX = currentAccel.x - rollingValueX; float accelY = currentAccel.y - rollingValueY;

A further improvement on the rolling filter is to use more advanced algorithms such as a Kalman

filter. The filter has a couple of interesting properties that allow it to provide better results. First,

the algorithm is able to use more than one source of data allowing it to use both inputs as sources

of measurement data and provide more accurate results. Second, the algorithm has a two-phase

approach that allows it to adapt based on the measurement noise and then update and correct the

prediction for the next measurement. The following diagram illustrates the approach:

The following graph shows the accelerometer output of a game that requires a quick response in

order to play well. The time span of all the motion data is only about 800 ms and varies quite a bit in

that short time. You can see in this case, the Kalman filter does a pretty good job of capturing the

large changes in accelerometer data but still creates smooth data values.

Knowing the limitations of the accelerometer data available to the application is important. The

following example of one of the limitations comes from an attempt to build an application that

detects directional tap on the device using the accelerometer. In the hardware accelerometer the

pulse caused by tapping the side of the device might look something like this to the sensor:

A large pulse followed by the signal quickly dampening, using this level of detail an algorithm could

be created to figure out that the device was tapped and what direction it came from. However, the

fidelity of the data at the application level is not this good and the following images illustrate what

happens as the sampling rate is increased and the samples don’t align with the pulse as expected:

With even higher sample rates, writing an algorithm to detect the pulse is going to be difficult:

Gyrometer

The gyrometer measures rotational velocity in radians or degrees per second. To use the sensor to

measure rotation of the device, the time between readings must be stored and multiplied by the

velocity in order to calculate distance travelled.

double angle = -reading.AngularVelocityZ * secDelta;

The angle could then be used directly in your application.

The major drawback of the gyrometer is that it knows nothing about orientation of the device so

over time any error in the gyrometer values shows up as drift in your application.

Sensor Fusion

Sensor fusion offers the best solution by using multiple sources of data to report the orientation

more accurately. Accelerometer and gyrometer data are both used as inputs to the fused

orientation output. Depending on your application, the orientation can be consumed as pitch, yaw,

roll angles via the inclinometer or more advanced representations in the form of quaternions or a

rotation matrix.

In Windows 8 some of the aggregation and processing of sensor data is done in hardware, which

provides for greater accuracy and less power consumption. Using the inclinometer’s yaw reading,

the application can measure the same rotation of the device.

float angle = reading.YawDegrees;

The angle of rotation around the z-axis is given as yaw in degrees.

Location Sensors

Most modern tablets, mobile phones, and PCs have location sensor capabilities. These devices

normally use multiple positioning methods to provide different granularities of location data. The

sources of position data vary in terms of accuracy, startup time, and power signature and include

the following:

GPS

A-GPS

Cell tower triangulation

Wi-Fi triangulation

IP Address

Applications that make use of location sensors range from games to navigation and some examples

are shown below:

Search for Points of Interest (POIs)

Geotagging ‒ adding location information to files such photos

Games ‒ Geocaching

Outdoors and fitness

Pedestrian or vehicle navigation

A few common sources of location data inaccuracy include IP resolution over corporate intranets,

Wi-Fi AP moves, and cell tower ID movement. The following image shows what happens when you

are indoors using only IP resolution and network traffic is being routed through corporate

headquarters:

Although the device was physically located in Arizona, the location data indicates positions in

Arizona and California.

Additional sources of location issues arise from GPS data error caused by situations such as being

indoors, atmospheric delays, and building or tree interference. The following image shows a route

being walked through the front lawn when the actual path never strayed from the sidewalk.

The following sections look at possible methods to increase the accuracy of the data used in

location-aware applications.

Filtering Location Data

geocoordinate.accuracy is used to retrieve the accuracy of the location in meters—the larger

the value, the less accurate the data. A simple filtering mechanism is to throw out values that are

not useful, use the geocoordinate.accuracy value to filter out the location sensor data that exceeds

the accuracy threshold.

if (position.coordinate.accuracy > X) { mark the location data invalid;

}

In situations where the device is moving, we can identify some GPS data errors using the following

algorithm that looks at the distances between neighboring points. The following pseudo-code

demonstrates the details:

if (distance (Pk, Pk+1) > Th1 meters && Distance (Pk, Pk+2) < Th2 meters) {

mark Pk+1 invalid;

}

Th1 = Speed1*TimeDelta1*C Th2=Speed1*TimeDelta2*C

C is a tuning constant based on the characteristics of your application. While the device is moving,

points are collected and the distances between two successive points are compared. If the distance

travelled by the first point is larger than some threshold but the distance travelled by next point is

within the threshold, the first point can be ignored.

An algorithm that is best in cases where your application is used in a stationary position is to look at

the centroid computed by a series of points. With a computed centroid, outliers can easily be culled

by comparing against a radius from the centroid.

Conclusion

We have shown a variety of possible filters to use in creating a sensor-driven mobile application.

These filters may be what you need to add another level of refinement to your application and

create a better user experience. Knowing the limitations and accuracy of the sensors you are using

will help in understanding what performance you can expect in your application.

As more devices are available with sensors tightly integrated, developers will always be ready to

create new features and come up with exciting possibilities for applications built around sensors.

Access to lower level, more accurate sensor data is ever increasing and will allow developers to

create even more exciting applications.

Additional information about the topics covered in this article:

– Ultrabook™ and Tablet Windows 8* Sensors Development Guide -http://software.intel.com/en-

us/articles/ultrabook-and-tablet-windows-8-sensors-development-guide/

– Using Accelerometer in Windows* Style App and a Case Study of Tap Detection -

http://software.intel.com/en-us/articles/using-accelerometer-in-windows-8-metro-style-app-

and-a-case-study-of-tap-detection/

– MSDN - http://msdn.microsoft.com/en-us/library/windows/apps/

– J. Kerr, F. Raab, E. Ramirez, “Best Practices in Data Reduction and Analysis of GPS Data” -

http://216.92.169.205/files/DataReduction_GPS_Workshop.pdf

– Greg Welch and Gary Bishop, “An Introduction to the Kalman Filter” -

http://www.cs.unc.edu/~welch/media/pdf/kalman_intro.pdf

Acknowledgements

Special thanks to Miao Wei and Sushu Zhang for contributing content to this article.

About the Author

Nathan Totura is an application engineer in Intel's Software and Services Group.

Currently working on the Intel® Atom™-enabling team he helps connect software

developers with Intel technology and resources. Primarily these technologies

include tablets and handsets on the Android*, Windows 8 and Tizen* platforms.

Notices

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS

OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS

DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL

ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO

SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A

PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER

INTELLECTUAL PROPERTY RIGHT.

UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR

ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL

INJURY OR DEATH MAY OCCUR.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not

rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel

reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities

arising from future changes to them. The information here is subject to change without notice. Do not finalize a

design with this information.

The products described in this document may contain design defects or errors known as errata which may cause

the product to deviate from published specifications. Current characterized errata are available on request.

Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your

product order.

Copies of documents which have an order number and are referenced in this document, or other Intel literature,

may be obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm

Software and workloads used in performance tests may have been optimized for performance only on Intel

microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer

systems, components, software, operations, and functions. Any change to any of those factors may cause the

results to vary. You should consult other information and performance tests to assist you in fully evaluating your

contemplated purchases, including the performance of that product when combined with other products.

Any software source code reprinted in this document is furnished under a software license and may only be used

or copied in accordance with the terms of that license.

Intel, Ultrabook, and the Intel logo are trademarks of Intel Corporation in the US and/or other countries.

Copyright © 2012 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

Optimization Notice

This sample source code is released under the Intel Sample Source Code License Agreement

Optimization Notice

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for

optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3,

and SSE3 instruction sets and other optimizations. Intel does not guarantee the availability,

functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel.

Microprocessor-dependent optimizations in this product are intended for use with Intel

microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel

microprocessors. Please refer to the applicable product User and Reference Guides for more

information regarding the specific instruction sets covered by this notice.

Notice revision #20110804