CursiveA novel interaction

technique for controlling expressive avatar gesture

Francesca Barrientos and John CannyUC Berkeley

UIST 12 November 2001 , Orlando Florida

Cursive 212 November 2001

Avatar worlds

• Verbal communication– Chat– Spoken

• Nonverbal communication– Mechanisms

• Change image• Play animation

– Gestures have continuous qualities

Cursive 312 November 2001

Current techniques• Specialized widgets

Vlnet expression panel1. Select2. Modulate3. Trigger• 3 step process

ComicChat EmotionWheel• Select and modulate• Hunt and click diverts attention

• Avatar agencyBodyChat

• Driven by chat text• Lacks direct control

Cursive 412 November 2001

Expressive and Controllable

• Want gesture to be expressive– Range of gestures– Vary quality of gesture

• Single step - modeless• Enable co-occuring verbal and nonverbal

communication• Solution: pen gesture input

Cursive 512 November 2001

Mapping pen to avatar gesture• Pen gesture carries symbolic and qualitative data• Human/avatar gesture can be symbolic and qualitative• Pen gesture simultaneously selects avatar gesture and

modulates multiple expressive qualities

Data Pen Avatar

Symbolic Letter of alphabet Gesture type

QualitativeSize Magnitude

Duration SpeedPressure Effort

Cursive 612 November 2001

Conceptual modelLibraryGenerated offline

InputUser writes letter in GUI

GenerationGesture selection and modulation

Animation Gesture performed

GUI library

Gesture generator

Avatar animator

w-s-r-

Cursive 712 November 2001

Example: modulating size

• Writing letter ‘s’ triggers animation

• Single gesture• Modulation along

single parameter• Writing size

mapped to gesture’s size

Cursive 812 November 2001

Extraction

Character recognizer

Feature extractor

w-s-r-

Speed .5Size .3

Inputs modulate gesture

Style parameters

Gesture Typew-s-r-

Type library

GUI

Cursive 912 November 2001

Modulation through multilinear interpolation

• Joint trajectory I– Rotation angles over time

• Gesture type β– Semantic category– Set of 2n prototypes - G– n style parameters– Prototypes represent

extremal trajectories• Gesture instance Iβ(u,v )

– Vector of joint trajectories – Multilinear interpolation on

type produces instance

Speed u

Size v

k

l

Gβ

Iβ(0,1)

Iβ(0,0)

Iβ(1,1)

Iβ(1,0)

Iβ(l,k)Iβ(0,k) Iβ(1,k)

Cursive 1012 November 2001

Speed parameter special case

• Interpolation along curves of different durations

• Time dilation step– Determine duration for

interpolated curve– Choose sample rate on

interpolated curve– Compress slower curve -

sample at proportionately slower rate

– Sample faster trajectory at proportionately faster rate

Angle θ

t

Cursive 1112 November 2001

ArchitectureGesture producer

Animator

Animationcommands

VRML

scriptGesturecommands-

w-

s-

r-

Cursive 1212 November 2001

Communication

• User logs into Vworld server

• Other visitors receive notification

• Other visitors request and download avatar copy

• Animator opens socket connection to producer

• Generator sends gesture commands via socket

Web server

VWorld server

Cursive communicates with any copies of user’s avatar user hostvisitor host

loginnotificationrequest avataropen socketsends commandsDownload avatar

Cursive 1312 November 2001

Summary

• Current techniques lack expressive control of avatar gesture

• Apply pen gesture to avatar gesture• Modulate recorded gestures using

multilinear interpolation• Augmented VRML avatars can drop into

multi-user virtual worlds

Cursive 1412 November 2001

Conclusion

• Expressive avatar gesture• Single action invokes and

modulates gesture• Multidimensional

modulation• Enables co-occuring

verbal and non-verbal communication

Pen gestures control Thank Erin.