Download - Cursive A novel interaction technique for controlling expressive avatar gesture Francesca Barrientos…

CursiveA novel interaction

technique for controlling expressive avatar gesture

Francesca Barrientos and John CannyUC Berkeley

UIST 12 November 2001 , Orlando Florida

Cursive 212 November 2001

Avatar worlds

• Verbal communication– Chat– Spoken

• Nonverbal communication– Mechanisms

• Change image• Play animation

– Gestures have continuous qualities


Current techniques• Specialized widgets

Vlnet expression panel1. Select2. Modulate3. Trigger• 3 step process

ComicChat EmotionWheel• Select and modulate• Hunt and click diverts attention

• Avatar agencyBodyChat

• Driven by chat text• Lacks direct control


Expressive and Controllable

• Want gesture to be expressive– Range of gestures– Vary quality of gesture

• Single step - modeless• Enable co-occuring verbal and nonverbal

communication• Solution: pen gesture input


Mapping pen to avatar gesture• Pen gesture carries symbolic and qualitative data• Human/avatar gesture can be symbolic and qualitative• Pen gesture simultaneously selects avatar gesture and

modulates multiple expressive qualities

Data Pen Avatar

Symbolic Letter of alphabet Gesture type

QualitativeSize Magnitude

Duration SpeedPressure Effort


Conceptual modelLibraryGenerated offline

InputUser writes letter in GUI

GenerationGesture selection and modulation

Animation Gesture performed

GUI library

Gesture generator

Avatar animator

w-s-r-


Example: modulating size

• Writing letter ‘s’ triggers animation

• Single gesture• Modulation along

single parameter• Writing size

mapped to gesture’s size


Extraction

Character recognizer

Feature extractor

w-s-r-

Speed .5Size .3

Inputs modulate gesture

Style parameters

Gesture Typew-s-r-

Type library

GUI


Modulation through multilinear interpolation

• Joint trajectory I– Rotation angles over time

• Gesture type β– Semantic category– Set of 2n prototypes - G– n style parameters– Prototypes represent

extremal trajectories• Gesture instance Iβ(u,v )

– Vector of joint trajectories – Multilinear interpolation on

type produces instance

Speed u

Size v

k

l

Gβ

Iβ(0,1)

Iβ(0,0)

Iβ(1,1)

Iβ(1,0)

Iβ(l,k)Iβ(0,k) Iβ(1,k)


Speed parameter special case

• Interpolation along curves of different durations

• Time dilation step– Determine duration for

interpolated curve– Choose sample rate on

interpolated curve– Compress slower curve -

sample at proportionately slower rate

– Sample faster trajectory at proportionately faster rate

Angle θ

t


ArchitectureGesture producer

Animator

Animationcommands

VRML

scriptGesturecommands-

w-

s-

r-


Communication

• User logs into Vworld server

• Other visitors receive notification

• Other visitors request and download avatar copy

• Animator opens socket connection to producer

• Generator sends gesture commands via socket

Web server

VWorld server

Cursive communicates with any copies of user’s avatar user hostvisitor host

loginnotificationrequest avataropen socketsends commandsDownload avatar


Summary

• Current techniques lack expressive control of avatar gesture

• Apply pen gesture to avatar gesture• Modulate recorded gestures using

multilinear interpolation• Augmented VRML avatars can drop into

multi-user virtual worlds


Conclusion

• Expressive avatar gesture• Single action invokes and

modulates gesture• Multidimensional

modulation• Enables co-occuring

verbal and non-verbal communication

Pen gestures control Thank Erin.