Virtual Reality and Virtual Reality and Digital Characters: Digital Characters: New Modalities for New Modalities for Human Computer Human Computer

Interaction Interaction G2V2 TalkG2V2 Talk

September 5September 5thth, 2003, 2003Benjamin LokBenjamin Lok

Welcome!Welcome!

My name is Benjamin My name is Benjamin Lok.Lok. Pronounced like “lock” as Pronounced like “lock” as

in lock the door.in lock the door. Not Loke as in Tone LNot Loke as in Tone Lōcōc.. Feel free to call me Ben.Feel free to call me Ben.

A bit about myselfA bit about myself BackgroundBackground InterestsInterests

Quick Research BioQuick Research Bio University of TulsaUniversity of Tulsa

Medical InformaticsMedical Informatics University of North CarolinaUniversity of North Carolina

Tracking in Virtual Environments (MS) – 3rdTech HiBallTracking in Virtual Environments (MS) – 3rdTech HiBall Avatars in Virtual EnvironmentsAvatars in Virtual Environments Interaction in Virtual EnvironmentsInteraction in Virtual Environments

University of North Carolina at Charlotte (Postdoc)University of North Carolina at Charlotte (Postdoc) Digital CharactersDigital Characters Virtual RealityVirtual Reality Computer VisionComputer Vision

Research AreasResearch Areas Interactive Computer Graphics (Rendering)Interactive Computer Graphics (Rendering) Virtual RealityVirtual Reality Human Computer InteractionHuman Computer Interaction Digital CharactersDigital Characters Assistive Technologies Assistive Technologies Computational Geometry / Computer Vision / Image Computational Geometry / Computer Vision / Image

ProcessingProcessing

Human Computer Human Computer InteractionInteraction

Q: What is the field of HCI?Q: What is the field of HCI? A: Discipline concerned with the A: Discipline concerned with the studystudy, , designdesign, ,

construction construction and and implementation implementation of human-of human-centric interactive computer systemscentric interactive computer systems

Not only Computer Scientists!Not only Computer Scientists! Comp Science – application and design of interfacesComp Science – application and design of interfaces Sociology and Anthropology – how humans and Sociology and Anthropology – how humans and

technology adapt to each othertechnology adapt to each other Ergonomics – safetyErgonomics – safety Psychology – cognitive processes and behavior of Psychology – cognitive processes and behavior of

usersusers Linguistics – Development of human and machine Linguistics – Development of human and machine

languageslanguages

From: www.pcwebopedia.com

Human Computer Human Computer InteractionInteraction

We are concerned about We are concerned about designing designing the the interface between the user and the system.interface between the user and the system.

Examples of HCI topics:Examples of HCI topics: Graphical User InterfaceGraphical User Interface

WidgetsWidgets Interfaces themselvesInterfaces themselves

Mice, keyboard, joystickMice, keyboard, joystick Evaluation of interfacesEvaluation of interfaces

Fun QuestionFun Question From From www.asktog.comwww.asktog.com A right-handed user is known to be within 10 A right-handed user is known to be within 10

pixels of the exact center of a large, 1600 X 1200 pixels of the exact center of a large, 1600 X 1200 screen. You will place a single-pixel target on the screen. You will place a single-pixel target on the screen that the user must point to exactly. List screen that the user must point to exactly. List the five pixel locations on the screen that the the five pixel locations on the screen that the user can access fastest. For extra credit, list user can access fastest. For extra credit, list them in order from fastest to slowest access.them in order from fastest to slowest access.

A: (Right handed user) A: (Right handed user) The pixel immediately at the current cursor locationThe pixel immediately at the current cursor location The bottom-right corner. The bottom-right corner. The top-left corner. The top-left corner. The top-right corner. The top-right corner. The bottom-left corner.The bottom-left corner.

Virtual RealityVirtual Reality

Computer generated Computer generated synthetic environmentsynthetic environment Virtual WorldsVirtual Worlds ImmersiveImmersive Sensory FeedbackSensory Feedback InteractionInteraction

Digital CharactersDigital Characters A new medium to A new medium to

interact with system interact with system informationinformation

Spectrum from the Spectrum from the paper clip -> AIBOpaper clip -> AIBO

Why would we want a Why would we want a digital character?digital character? DynamicDynamic Better than keyboard Better than keyboard

and mouse for some and mouse for some taskstasks

Natural interactionNatural interaction

Current ResearchCurrent Research

Combine HCI with VRCombine HCI with VR If all things in VR are virtual, how do If all things in VR are virtual, how do

you interact with the environment?you interact with the environment?

Combine HCI with digital charactersCombine HCI with digital characters How do you interact with digital How do you interact with digital

characters?characters? Interaction = input Interaction = input andand output output

Merging real Merging real and virtual spacesand virtual spaces

Interacting with Real Interacting with Real Objects in VRObjects in VR

How How important important are real are real objects?objects?

A: For A: For cognitive cognitive (problem (problem solving), solving), very very important!important!

Real Space

Purely Virtual

Hybrid

Vis. Faithful Hybrid

NASA Case NASA Case StudyStudy

Videos of TaskVideos of Task

Having a hybrid environment provides substantial benefits in prototype design.

Locomotion in VRLocomotion in VR Another aspect of ‘natural Another aspect of ‘natural

interaction’ is locomotion interaction’ is locomotion (define).(define).

Most common way:Most common way: Use a ‘virtual walking’ Use a ‘virtual walking’

metaphor.metaphor. Does this reduce effectiveness?Does this reduce effectiveness?

We can test this because of We can test this because of new wide-area tracking new wide-area tracking technologies.technologies.

Locomotion in VR VideoLocomotion in VR Video

Play videoPlay video

ResultsResults

Mean Cognition Questionnaire Scores

15.73

13.00

13.7514.18

8.00

10.00

12.00

14.00

16.00

18.00

20.00

Condition

Sc

ore

RW VW6 VW3 M

Sketch Map - All Object Positions

25.27

19.42 19.45

17.42

0

5

10

15

20

25

30

Condition

Sco

re

RW VW6 VW3 M

SUS Means

4.11

3.78

2.47

4.05

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

Condition

SU

S M

ea

n

RW VW6 VW3 M

Natural interaction is better!

Avatars in VEAvatars in VE Most virtual environments do not provide an Most virtual environments do not provide an

avatar (user self-representation)avatar (user self-representation) Why? Because tracking the human body is Why? Because tracking the human body is

difficult.difficult. Solution: Use simple computer vision to track Solution: Use simple computer vision to track

colored markers to generate an avatarcolored markers to generate an avatar

STRAPS VideoSTRAPS Video

Play VideoPlay Video

Digital CharactersDigital Characters

http://http://movies.yahoo.com/shop?dmovies.yahoo.com/shop?d==hv&idhv&id=1807432839&cf=trailer=1807432839&cf=trailer

Thinking

Perceiving

Responding

Thinking

Perceiving

Responding

Digital Character

Participant

Interaction

DIgital ANimated Avatar DIgital ANimated Avatar (DIANA)(DIANA)

Upcoming Upcoming ResearchResearch

How do we naturally interact How do we naturally interact with a digital character?with a digital character? DisplayDisplay PerceptionPerception EfficacyEfficacy

Combine Straps with DIANACombine Straps with DIANA Avatar TelephoneAvatar Telephone Acting practiceActing practice

VR InteractionVR Interaction

Getting real objects into virtual Getting real objects into virtual environmentsenvironments

How do you naturally interact with virtual How do you naturally interact with virtual objects?objects?

Questions?Questions?

Thinking

Perceiving

Responding

Thinking

Perceiving

Responding

Digital Character

Participant

Interaction