Overview on Gesture Recognition

Spring Semester, 2010

Gesture

A form of non-verbal communication in which visible bodily actions communicate particular messages

Include movement of the hands, face, or other parts of the body

Gesture Recognition

Vision / Wave

Wearable-sensor

Acceleration / Gyro

…

Vision and Wave

MS– Project natal

Nokia– 3D device control interface by using

ultrasonic transducers

Wearable Sensors

Accelerometer

Nintendo Wii

Mobile phones

Accelerometers on Mobile Phones

Today’s Paper

Natural Throw and Tilt Interaction between Mobile Phones and Distant Displays -Dachselt, R. and Buchholz, R. presented in CHI 2009

Throw and Tilt – Seam-less Interaction across Devices Using Mobile Phone Gestures -Dachselt, R. and Buchholz, R., MEIS 2008

Goal

Combine the mobile phones with large displays– Most mobile phones are able to store and

manage all kinds of personal information and media

– All of these devices are inherently limited in terms of their screen size and amount of information they can display

Throw and Tilt

Use intuitive throwing gestures to transfer personal media data such as music,

pictures, diary entries and map locations to a large screen

Use a simple, intuitive, and generic set of tilt gestures to ensure natural interaction

Tilt Interaction

Gesture set

Linear mapping

Throwing Data

Throw– Use a threshold of the Euclidean length of

the raw force vector delivered by the sensor

Fetch-back

Application Scenarios

The zoomable music browser

Image / media browser

Geo-Referenced Data on Maps

Demo

Summary and Discussion

Give a brief overview on gesture recognition

The simple but powerful gestures can be used to control various applications– Additional application scenarios: Combining

private mobile and public displays

Multi-user collaboration, proximity, and directionality of devices