vSmiley : The imaging of vSmiley : The imaging of

emotions through vibration emotions through vibration

patterns. patterns.

Deepa Mathew

Department of Computer Sciences

University of Tampere, Finland

deepa.mathew@uta.fi

April, 2005

AAFG 2005

Deepa.Mathew p 02_18 18.04.2005

INTRODUCTION

Tactons are becoming a popular widget in various research programs

to build new multi-model or multi-tasking user-interfaces, mainly

because of it’s ability to directly interpret with human skin (cutaneous),

which in turn gives an impression of ‘sense of feel’.

Various tactile patterns can be designed in a way, even to express

emotion with sense of feel when it is presented as Smiley or in any

other forms like icons, graphics and so on.

Using tactons can even be a better option when building an user-

interface for deaf/blind and also deaf-blind users

In the game “vSmiley” we have used the vibrating patterns (Tactons)

for imaging of emotions. And this game is also intended for deaf or

visually impaired children, using vibrations (Tactons) as the common

root.

vSmiley

GOAL

•The goal of the project vSmileys is an empirical study of the

possibility to substitute emoticons by vibro-tactile patterns.

• Tactons could be a usesful means of communicating information

in user interfaces for the visually challenged and people with

hearing disabilities. So, the game could be a starting point to get

familiarised with the sematic patterns.

vSmiley

Deepa.Mathew p 03_18 18.04.2005

METHOD

Similar to the envelope of sound patterns (earcons) mechanical

vibrations can be composed into the vibro-tactile patterns (tactons).

Some earcons were produced and widely used to signify a

semantically-completed messages and feedback cues in GUI. Vibro-

tactile patterns are employed as a way of communication especially

for and between people with sensorial deficit (deaf, blind, deaf-blind).

vSmiley

Deepa.Mathew p 04_18 18.04.2005

Figure1: Sound wave envelope

TECHNIQUES

The tactons were designed for 9 smileys signifying different emotions.

The aim of the pilot testing through the game was estimation of relative

identification of the semantics of the constructed tactons (vSmileys) by

comparing them with the emotion of the smileys presented visually.

Basic vibration patterns (Figure 2) have been used and combined to

get the desired sample (vSmiley) which should contain semantics

similar to the graphical prototype (smiley). Basic patterns can be

combined to create composite vibro-tactile messages.

vSmiley

Deepa.Mathew p 05_18 18.04.2005

Figure2: Vibrating patters used in designing vSmileys, Immersion Studio 4.0

Figure 3: Logitech iFeel optical mouse

Apparatus and Procedure

Software:

‘Immersion Studio 4.0’ was used in designing vSlileys to transform

smileys into composite vibro-tactile patterns.

Hardware:

An Logitech iFeel optical mouse (Figure 4) connected via USB port was

used to display the vibro-tactile patterns.

vSmiley

Deepa.Mathew p 06_18 18.04.2005

http://www.dcs.gla.ac.uk/~stephen/papers/WorldHaptics_Brown.pdf

Deepa.Mathew p 07_18 18.04.2005

vSmiley

Figure 6: C2 Tactor attached to finger.

There are some other devices as well that could be used. For

example: Engineering Acoustics C2 Tactor (Figure 4) and TACTAID

VBW32 transducer (Figure 5)

These devices was used in the research done by Brown et al [1]

Figure 5: TACTAID VBW32 transducer.

Figure 4: Engineering Acoustics C2 Tactor [2]

DESIGNING THE TACTONS

Before vSmileys have been created, the appropriate parameters for

vibro-tactile patterns in which information can be encoded must be

identified [1].

The most obvious parameters to use in vSmiley construction are the

basic parameters of basic vibratory patterns such as frequency,

amplitude, waveform (envelope: attack, sustain, decay) and duration

of each segment/burst. However, in designing the vSmileys have

been used only the frequency and duration, these parameters were

changed to encode 9 smileys.

Semantic information could be encoded by manipulating the duration

of pulses. While duration alone could be used as a parameter,

combining pulses of different durations to form rhythms would offer

more flexibility [1].

vSmiley

Deepa.Mathew p 08_18 18.04.2005

TESTING PROCEDURE

Five participants, (2 females and 3 males) with normal vision and

hearing took part in the pilot testing of the vSmileys game.

None of the participant had any previous experience in this kind of

testing and especially had no idea about tactons.

They are in the age group of 23-28 (2 students and 3 employees)

Five minutes training time was given to each of them to memories

vSmileys. If needed, then extra time was given.

After the training time was over the participants where asked to

identify different vSmileys by feeling the vibratory patterns prsented

with Logitech iFeel optical mouse with built-in shaking motor. The

test consisted of 90 trials (9 vSmileys x 10 times) and the tactons

where selected randomly. Data were recorded on the recognition of

the correct smiley and stored in a log file after completion the game.

vSmiley

Deepa.Mathew p 09_18 18.04.2005

The game is played as explained below

• Click on each smiley as many times as needed to memories the vSmileys (vibro-tactile patterns).

• After player have memorised all the patterns, s/he can activate the testing phase by tapping the space bar.

• Tapping the spacebar produces the test-pattern. Player can repeat this vSmiley as many times as needed.

• The player has to identify the test-pattern and point by click the correct smiley having the same meaning.

vSmiley

Deepa.Mathew p 10_18 18.04.2005

Figure7: vSmiley game

RESULTS AND DISCUSSIONS

vSmiley

Deepa.Mathew p 11_18 18.04.2005

The players had some trouble choosing the right smiley in the

beginning but the results were gradually improved.

And they were able to distinguish the vibrating patterns and identify the

correct smiley after few trials.

But they had some trouble when identifying the somewhat-similar

vSmileys. They got easily confused and made more errors.

vSmiley

Deepa.Mathew p 12_18 18.04.2005

The average number of repeatitions per vSmiley in training phase for each player.

Each of the players had little difficulty in memorising the 6th and 9th vSmileys. Maybe because of its similarity.

0

2

4

6

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Rep

etiti

on p

er im

age,

n

Series1 Series2 Series3 Series4 Series5

vSmiley

Deepa.Mathew p 13_18 18.04.2005

0

2

4

6

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Rep

etiti

ons

per

imag

e, n

Series1 Series2 Series3 Series4 Series5

The average number of repeatitions per vSmiley in testing phase for each player.

Deepa.Mathew p 14_18 18.04.2005

vSmiley

0

2

4

6

8

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Wro

ng t

rial,

n

Series1 Series2 Series3 Series4 Series5

The average number of wrong trials per vSmiley.

vSmiley

Deepa.Mathew p 15_18 18.04.2005

The average number of repetitions per image throughut the test

for all the players.

0

1

2

3

4

5

6

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Rep

etiti

ons

per

imag

e, n

Repet. in training phase Repet. in testing phase

vSmiley

Deepa.Mathew p 16_18 18.04.2005

The average error rate during the test.

Some vSmileys were easily distinguished and some were not.

0

1

2

3

4

5

6

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Err

ors,

n

vSmileyCONCLUSIONS

Vibro-tactile patterns and semantic sequences can play a vital role for

building new multi-model or multi-tasking user-interfaces especially

for users having a sensorial deficit and also introduce a possible

alternative way of communication.

Tactons were already been used in areas such as tele-operation or

displays for blind people to provide sensory substitution [2]. But not to

its true abilities.

More research should be done to know the true usefulness of the

tactile patterns. And based on the pilot experiment it is obvious that it

is easy to identify and differentiate different vibrating patterns. And

with proper research and knowledge, tactons could replace speech or

short speech messages which could be vital for the deaf users.

Deepa.Mathew p 17_18 18.04.2005

vSmiley

Click to go the first slide

References

[1] Lorna M. Brown, Stephen A. Brewster and Helen C. Purchase. A First Investigation into the Effectiveness of Tactons.Glasgow Interactive Systems Group, Department of Computing Science, University of Glasgow, UK

[2] Dr Steven Wall, An investigation of multimodal interaction with tactile displays. Department of Computing Science, University of Glasgow.

Deepa.Mathew p 18_18 18.04.2005

Thank you!

Any Questions?