Cursive

A novel interaction technique for controlling expressive avatar gesture

Francesca Barrientos and John Canny

UC Berkeley

UIST 12 November 2001 , Orlando Florida

Cursive

pen gestures

> Type of avatar is used in graphics chat worlds

Avatar worlds

Verbal communicationChatSpoken

Nonverbal communication

MechanismsChange imagePlay animationGestures have continuous qualities

Cursive

> How specify continuous qualities

10.bin

Current techniques

Specialized widgets

Vlnet expression panel

Select

Modulate

Trigger

3 step process

ComicChat EmotionWheel

Select and modulate

Hunt and click diverts attention

Avatar agency

BodyChat

Driven by chat text

Lacks direct control

Cursive

Limitations of each method

> Want best of both worlds

Expressive and Controllable

Want gesture to be expressiveRange of gesturesVary quality of gestureSingle step - modelessEnable co-occuring verbal and nonverbal communicationSolution: pen gesture input

Cursive

> How apply pen gesture

Mapping pen to avatar gesture

Pen gesture carries symbolic and qualitative dataHuman/avatar gesture can be symbolic and qualitativePen gesture simultaneously selects avatar gesture and modulates multiple expressive qualities

DataPenAvatarSymbolicLetter of alphabetGesture typeQualitativeSizeMagnitudeDurationSpeedPressureEffort

Cursive

Compact

Natural (writing is previously learned skill)

> PROCESS

Conceptual model

LibraryGenerated offline

InputUser writes letter in GUI

GenerationGesture selection and modulation

Animation Gesture performed

GUI

library

Gesture generator

Avatar animator

w-

s-

r-

Cursive

See

Example: modulating size

Writing letter s triggers animationSingle gestureModulation along single parameterWriting size mapped to gestures size

Cursive

Closer look

Extraction

Character

recognizer

Feature extractor

w-

s-

r-

Inputs modulate gesture

Style

parameters

Gesture Type

Type library

GUI

Speed.5Size.3

w-

s-

r-

Cursive

Explain modulate

Modulation through multilinear interpolation

Joint trajectory IRotation angles over timeGesture type Semantic categorySet of 2n prototypes - Gn style parametersPrototypes represent extremal trajectoriesGesture instance I(u,v )Vector of joint trajectories Multilinear interpolation on type produces instance

Speed u

Size v

k

l

G

I(0,1)

I(0,0)

I(1,1)

I(1,0)

I(l,k)

I(0,k)

I(1,k)

Cursive

Actually this is simplified

Speed parameter special case

Interpolation along curves of different durationsTime dilation stepDetermine duration for interpolated curveChoose sample rate on interpolated curveCompress slower curve - sample at proportionately slower rateSample faster trajectory at proportionately faster rate

Angle

t

Cursive

Briefly describe implementation

Architecture

Gesture producer

Animator

Gesturecommands

Animation

commands

VRML

script

w-

s-

r-

-

Cursive

Show mechnisms

Communication

User logs into Vworld server Other visitors receive notificationOther visitors request and download avatar copyAnimator opens socket connection to producerGenerator sends gesture commands via socket

Cursive communicates with any copies of users avatar

user host

visitor host

Web server

VWorld server

login

notification

request avatar

open socket

sends commands

Download avatar

Cursive

Summarize

Summary

Current techniques lack expressive control of avatar gestureApply pen gesture to avatar gestureModulate recorded gestures using multilinear interpolationAugmented VRML avatars can drop into multi-user virtual worlds

Cursive

Conclusion

Expressive avatar gesture

Single action invokes and modulates gesture

Multidimensional modulation

Enables co-occuring verbal and non-verbal communication

Pen gestures control

Thank Erin.

Cursive