8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

1/17

Sensor In ut for Mobile Devices

Mark Shkolnikov

President

KEYnetik, Inc

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

2/17

From Single Sensor -

- To Sensor Array

Traditional use of sensors

Ambient light dim down display Accelerometer screen orientation

Proximity lock down touch screen

New paradigm Sensor Array Integration of sensors

Quantity makes new quality

From automation of disparate functions tointelligent device

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

3/17

Array of Sensors

deduce whats u

Sense whats around

and help the user

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

4/17

Smartphone+ Sensory Input= Intelligent Device

Fewer Commands - Fewer Keys

Better User ExperienceFewer DesignConstraints

Sensors pay for themselvesMost expensive real estate is the front surfaceof mobile device+

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

5/17

Compensation of ErrorsSynchronization and cross correlation of data

All Data

Perceptive Filtering

Situation AwarenessWhere? Whats going on?

Task AwarenessWhat needs to done?

Actionable Data

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

6/17

Spatially Aware Inference Logic

Sensors work as an orchestra with motion

sensor being a conductor

High confidence data is analyzed, other

Data from disparate sources turns intocohesive actionable profile

Example: Hi-N-Bye for Nokia 5800

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

7/17

Example: Disambiguation of Image

with Accelerometer

Really falling?

Cannot hold yourcamera straight?

Both?

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

8/17

Importance and Challenges of

Motion Perception

Motion affects all other sensors

(pressure, image, sound, light, temperature, compass)

Motion is very complex(six degrees of freedom = six time variables)

Orientation (roll, pitch and yaw angles) Accelerations in 3D

Tri-axis accelerometer jams six variables

into three creating ambiguity

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

9/17

Perceptive Disambiguation -

- Strengths and Weaknesses

Assessment Vs. Precision

Very efficient for recognition

Cannot substitute accurate

measurements

Can separate high confidence and lowconfidence data

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

10/17

Motion Sensors Accelerometers

Sense both inertia and gravity

Consume little power

Cannot tell

Gravity from inertia

Gyroscopes

Immune to gravity and lateral forces

Sense only rotation over internal reference

Need time to start up and recover from impact

Draw substantial power

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

11/17

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

12/17

Accelerometer Pair

1 + 1 = 3Two sensors + distance in between

build up trajectory with single integration

Can emulate gyroscope

Low power consumption

Instant to start and immune to impact

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

13/17

Accelerometers Vs Gyroscope

SingleAccelerometer

DualAccelerometers

Gyroscope

Orientationtowards gravity

Yes Yes No

Angular rotationrate

No Yes Yes

Radius of rotation No Yes NoReaction toimpact

None NoneTemporaryblackout

Start up time None None Same as blackout

Powerconsumption

Low 2 x Low 10 x Low

Price Low 2 x Low 5 x Low

13

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

14/17

Dual Accelerometer Platform

Support all motion input applicationsImage stabilization, UI, athletic monitor, gaming

Unique ability to filter motion signals

Cost less and consume less power thangyroscopes

Immune to motion interference Works with SAIL

14

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

15/17

Use of Radius of Rotation for UIForgiving execution with precise

response guaranteed by

anthropometry (wrist length Vs.elbow/hand)

15

Freedom to move device aroundto stay comfortable and maintainbest possible viewing angle

Multitask: control the device withmotion and have pedometer

running

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

16/17

Turn Phone into Gaming Controller

Practice golf swings, fencinglunges, tennis backhands and

bowling, put a spin on the ball!

Play with friends or alone

Measure radiuses andaccelerations as you ski orbike, time out your jumps

Use your phone as personalblack box / emergency

beacon

16

8/14/2019 Device Creation_Mark Shkolnikov_KEYnetik

17/17

. .