Charles Hughes, School of Computer ScienceCharles Hughes, School of Computer Science

Media Convergence LaboratoryMedia Convergence Laboratory

School of Film and Digital MediaSchool of Film and Digital Media

June 15, 2005June 15, 2005

Mixed RealityMixed Reality

Trompe l’oTrompe l’oëëil in the 21st il in the 21st CenturyCentury

15-June-2005

Mixed RealityMixed Reality 2

Trompe l’oëil

To deceive the viewer as to its reality.

15-June-2005

Mixed RealityMixed Reality 3

See-Through HMDsSee-Through HMDs

OpticalOptical

VideoVideo

15-June-2005

Mixed RealityMixed Reality 4

REALITYVIRTUAL

Augmented Reality

Augmented Virtuality

Dark RidesDark Rides

RidesRides

Video GamesVideo Games

Extending Mixed RealityExtending Mixed Reality“Magic is behind the “Magic is behind the

eyeballs”eyeballs”

IMAGINED

MR ExperiencesMR Experiences

Examples from our Examples from our Media Convergence LabMedia Convergence Lab

EntertainmentEntertainment

15-June-2005

Mixed RealityMixed Reality 7

Time Portal (AR)Time Portal (AR)

Animatic

MR @ MR @ SIGGRAPH’03SIGGRAPH’03

Visual+audio+SFX

CollaborationCollaboration

15-June-2005

Mixed RealityMixed Reality 9

Placing Real Entities into Virtual Environments

Augmented VirtualityAugmented Virtuality

15-June-2005

Mixed RealityMixed Reality 10

Demo Dome Demo Dome CharacteristicsCharacteristics

Relative to CAVERelative to CAVE LightweightLightweight InexpensiveInexpensive Can have distinct POVsCan have distinct POVs

E.g., consider a city planner and an E.g., consider a city planner and an architectarchitect

Each wants specialized informationEach wants specialized information But both need to have a common But both need to have a common

contextcontext

EducationEducation

15-June-2005

Mixed RealityMixed Reality 12

MR Sea Creature @ OSCMR Sea Creature @ OSC

Artist’s ConceptArtist’s Concept

MR ROVERMR ROVER

15-June-2005

Mixed RealityMixed Reality 13

15-June-2005

Mixed RealityMixed Reality 14

Water’s Journey through Water’s Journey through ……

TrainingTraining

15-June-2005

Mixed RealityMixed Reality 16

MR MINI-MOUTMR MINI-MOUT

15-June-2005

Mixed RealityMixed Reality 17

Merging RealitiesMerging Realities

FFWFFW

Multi-PlayerMulti-Player

CGFCGF

SniperSniper

MOUTMOUT

SFXSFXVR SetsVR Sets

VR ActorsVR Actors

HostilesHostiles

VehicularVehicular

VR CuesVR Cues

HUDHUD

15-June-2005

Mixed RealityMixed Reality 18

ResearchResearch

15-June-2005

Mixed RealityMixed Reality 20

MR Software SuiteMR Software Suite

Platform neutralPlatform neutral Story Engine Story Engine

Agent-basedAgent-based Plug-in architecturePlug-in architecture

Audio EngineAudio Engine Constraint-based speaker usageConstraint-based speaker usage

Graphics EngineGraphics Engine DMX EngineDMX Engine ProtocolsProtocols

Command and control streamsCommand and control streams Behavior, GUI and device interfacesBehavior, GUI and device interfaces

15-June-2005

Mixed RealityMixed Reality 21

Visual RenderingVisual Rendering

Interactive-time and Realistic (GPU)Interactive-time and Realistic (GPU) Global IlluminationGlobal Illumination

Shading and shadowsShading and shadows Interaction of real and virtualInteraction of real and virtual

Ambient lightingAmbient lighting Color mapping so virtual reactColor mapping so virtual react

Story-based renderingStory-based rendering Various HDR projectsVarious HDR projects

15-June-2005

Mixed RealityMixed Reality 22

Fire/ShadowsFire/Shadows

15-June-2005

Mixed RealityMixed Reality 23

Pegasus in CSBPegasus in CSB

15-June-2005

Mixed RealityMixed Reality 24

Charles E. Hughes

School of Computer Science & Media Convergence Lab & School of Film and Digital Media & Text and Technology Program (English)

E-Mail: ceh@cs.ucf.edu

Home Page: http://www.cs.ucf.edu/~ceh/