Windows 7: The Sensor and Location Platform: Building Context-Aware Applications

Dan PolivyLead Program ManagerMicrosoft Corporation

PC25

…your computer knew where youwere and adjusted itself accordingly? Update local weather, news, events Automatically adjust clock

…your computer could sense its environment and optimize your experience? Adjust display backlight based on ambient light Optimize UI elements for improved readability

What If…?

Platform Introduction and Scenarios Components and Architecture Location-awareness demos Light-aware UI demos Sensor Development Kit Overview

Agenda

Location-Aware Weather Gadget

demo

Location devices are exposed as virtual COM ports Exclusive application access Not secure Proprietary data formats (NMEA, others)

GPS doesn’t work indoors Hard to support multiple technologies at once

Sensors are integrated as vertical solutions Applications need to know sensor

hardware specifics Limited adoption and scope

Limitations Of Sensors Today

Windows Sensor And Location Platform

Provides unified driver model for all types of sensor devices Physical sensors (e.g., GPS devices, Light Sensors) Logical sensor (e.g., Wi-Fi triangulation resolver)

Provides standard APIs for accessing sensors Sensor API: C++/COM

Raw access to any sensor Location API: C++/COM, IDispatch (scriptable)

Abstracted API for location data Puts user in control of information disclosure

Sensor Architecture

UMDF Sensor Driver

Sensor Class Extension

Sensor API

Application

Location and Other

Sensors Control Panel

SensorDevice

Application

UserSystem

Location Architecture

UMDF Sensor Driver

Sensor Class Extension

Sensor API

Gadget or Script Application

Location and Other

Sensors Control Panel

SensorDevice

Location APILocation IDispatch Interface

Application

UMDF Sensor Driver

Sensor Class Extension

Logical Location Sensor(Triangulation)

UserSystem

Single API call to answer “Where am I?” Independent of provider used to

determine location Synchronous and Asynchronous models Script/automation compatible

Automatic transition between providers Most accurate providers have priority

Concurrent access for multiple applications Default Location

Provided by user as fallback when no other sources are available

Location Platform Benefits

Location COM API

interface ILocation : IUnknown{

HRESULT RegisterForReport(ILocationEvents* pEvents,REFIID reportType,ULONG dwRequestedReportInterval);

HRESULT GetReport(REFIID reportType,ILocationReport** ppLocationReport);

// other functions};

interface ILocationEvents : IUnknown{

HRESULT OnLocationChanged(REFIID reportType,ILocationReport* pNewReport);

HRESULT OnStatusChanged(REFIID reportType,LOCATION_REPORT_STATUS newStatus);

};

Geographic Data (ILatLongReport) Latitude, longitude, altitude,

associated error required Most common format Best format for precise location Can reverse geo-code later

Civic Address (ICivicAddressReport) Zip, Country required Most human readable Best for ‘rough’ location estimates,

street directions

Types Of Location Data

Location Reports

interface ILatLongReport : ILocationReport {

HRESULT GetLatitude(DOUBLE* latitude);HRESULT GetLongitude(DOUBLE* longitude);HRESULT GetAltitude(DOUBLE* altitude);

HRESULT GetAltitudeError(DOUBLE* altitudeError);HRESULT GetErrorRadius(DOUBLE* errorRadius);

};

interface ICivicAddressReport : ILocationReport{

HRESULT GetAddressLine1(BSTR* pbstrAddress1);HRESULT GetAddressLine2(BSTR* pbstrAddress2);HRESULT GetCity(BSTR* pbstrCity);HRESULT GetStateProvince(BSTR* pbstrState);HRESULT GetPostalCode(BSTR* pbstrPostalCode);HRESULT GetCountryRegion(BSTR* pbstrCountry);

};

Scripting The Location API

demo

Location data is considered PII User consent is required to share data

All sensors are disabled by default Admin rights required to enable a sensor Sensors can be

configured on a per-user basis

“Enable Dialog” invoked by applications

Privacy And Access Control

Location Summary

C++/COM IDispatch (script, .NET interop)

Lat/Long and Address formats

Opt-in via ‘Enable Dialog’ orControl Panel

Single, convenient API for accessing current location

Enables applications to be

device-agnostic

User consent required toaccess data

Working Outside With Windows Vista

What Is A Sensor?

Enumerated via category and type Category represents what is being sensed Type represents how it is sensed

Properties Read-only or read-write

Data Property keys specify data field, units, and type

for associated data Events

Data-driven State

Ambient Light Sensors

Measure light intensity (i.e., illuminance) in LUX (lumens per square meter)

Windows 7 includes class driver support for ACPI light sensors Working with OEMs to integrate light

sensors into notebook designs Adaptive brightness feature

supported by Windows 7 OS automatically adjusts display backlight

Light-aware applications can use these sensors to optimize UI content for various lighting conditions

Light-aware UI

demo

Light-Aware Applications

UI without light-awareness,40% screen brightness

UI with light-awareness,100% screen brightness

Photos taken in direct sunlight

Enumerating Light Sensors

#include <sensors.h>

HRESULT hr;CComPtr<ISensorManager> pSensorManager;

pSensorManager.CoCreateInstance(CLSID_SensorManager);

CComPtr<ISensorCollection> pALSCollection;CComPtr<ISensor> pALSSensor;

// Get all the ALS sensors on the systempSensorManager->GetSensorsByType(SENSOR_TYPE_AMBIENT_LIGHT, &pALSCollection);

hr = pSensorManager->RequestPermissions(0, // Owner window pALSCollection, // Collection of sensors requiring permissionsTRUE); // Modal flag

if(SUCCEEDED(hr)){ pALSCollection->GetAt(0, &pALSSensor);}

Getting Current Light Level

STDMETHODIMP CALSEventSink::OnDataUpdated(ISensor* pSensor, ISensorDataReport* pNewData)

{PROPVARIANT lightLevel; PropVariantInit(&lightLevel);

// Get the sensor reading from the ISensorDataReport objectpNewData->GetSensorValue(SENSOR_DATA_TYPE_LIGHT_LEVEL_LUX, &lightLevel);

// Extract the float value from the PROPVARIANT objectfloat luxValue = V_FLOAT(lightLevel);

// Normalize the light sensor datadouble lightNormalized = ::pow(luxValue, 0.4) / 100.0;

// Handle UI changes based on the normalized LUX data // which ranges from 0.0 - 1.0 for a lux range of// 0 lux to 100,000 lux, this method represents such a // handler that would be implemented in your application UpdateUI(lightNormalized);

PropVariantClear(&lightLevel);return S_OK;

}

How To Build Light-Aware UI

Scale Change font size/zoom level ‘Weight’ of lines

Contrast Color

Saturation Complementary vs Adjacent

Smooth transitions between changes are extremely important for overall user experience Fades or animations Data smoothing/hysteresis

Sensor API Summary

C++/COM Synchronous and asynchronous

data access

Discovery via category or type

Data and properties mappedto PROPERTYKEY andPROPVARIANT pairs

Rich API for accessing raw sensor data

Direct access to individual sensors

Extensible architecture

Based on Freescale JM Badge Board (HID)

Sensors Ambient light sensor 3D Accelerometer Dual touch strip sensors

Developer tools Sample firmware code Sample driver code Diagnostic and sample

applications Light-aware MSDN Reader Marble game

Sensor Development Kit

Limited quantitiesavailable in the

Windows 7 booth!

Samples C++/COM samples in Windows SDK JScript location sample in Windows SDK WPF/.NET samples in

Sensor Development Kit CD Tools

Light Simulator (SDK) Hardware

Get a Sensor Development Kit at PDC! Will be available online in the future

(timeline TBD)

Getting Started

Utilize the Windows Sensor and Location platform to build location-aware applications, gadgets and services

Implement light-aware UI in your applications to support betterreadability and UX under various environmental conditions

Build environmentally-aware applications using additional sensors with the sensor and location platform

Call To Action

Booth Windows 7

Hands On Lab PCHOL13: Writing a Location-Aware Gadget

Whitepaper Introducing the Windows Sensor

and Location Platform Implementing Light-Aware UI Using the

Windows Sensor and Location Platform

Additional Content At PDC

Sensor and Location WHDC sitewww.microsoft.com/whdc/sensors

Windows SDK Virtual Earth Map gadget (winui\Sidebar\

LocateMe.gadget) Light Simulator and sample

For business inquiries, please contact: Sensor Platform: sensext@microsoft.com Location Platform: locext@microsoft.com

Resources

Evals & Recordings

Please fill

out your

evaluation for

this session at:

This session will be available as a recording at:

www.microsoftpdc.com

Please use the microphones provided

Q&A

© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market

conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.