Outline
• Some basics• Why perception is important
– For Modelling– For Rendering– For Animation
• Future research - multisensory perception and crowds
Why do we care?• Rapid Developments in Graphics:
– Algorithmic: e.g., fluid, cloth, humans, etc…– Hardware: desktop PCs – Low-end mobile devices, PDAs, cell phones
• More than just technical challenges:– Fidelity? Plausibility? Presence? – Perceptibility of errors? Evaluation? Metrics?
Must consider human perception!
Modelling
Example – Model simplification
http://www.ixbt.com/
How to measure fidelity?
Used experimental measures to evaluate simplification algorithms– Naming times– Ratings– Forced-choice preferences
Original (top), QSlim at 80% (middle) and Vclust 80% (bottom)
Watson et al. 2001
• Users can guide the simplification process
Pojar and Schmalstieg 2003Kho and Garland 2003
• Or salient features can be found automatically, or using an eye-tracker
Howlett et al.2004Lee et al.2005
Perceptual simplification
We used an eye-tracker to determine the prominent features of models
Finding Salient Features
• We gathered information on where a participant was fixating while viewing a set of models.
Evaluation• We incorporated
fixation data to produce a perceptual simplification metric
• Then asked people to name them, match them, and choose between them
Rendering
Image Fidelity• Fidelity to what ?
Stimulus Response Semantics• How to assess fidelity ?
– Metrics• How to apply the notion to image production ?
– Perceptually informed rendering• To image reproduction ?
– Tone mapping, contrast reduction, display device design
• How to evaluate perceptual methods ?
Measuring ErrorProblem:Problem:
Model the Human Visual System (HVS)VDP: Visible Differences
VQEG/ModelfestSSIM: Structured Similarity
Solution:Solution:
Visible Difference Predictors
Two Images(e.g., frames of an animation)
Probabilities of difference detection
Daly/Myskowski
Example: Perceptual Rendering• Can do user studies to
find out what rendering components are perceptually important
• Then develop a metric to guide where to concentrate computationally expensive illumination components with respect to image quality.
Ferwerda et al., 2004
High Dynamic Range image reproduction
High Dynamic Range display device• Seetzen et al. 2004 present two designs for a HDR display
device, based on the idea of a modulated backlight– 1) LEDs array and 2) Digital Mirror Device
• A standard LCD device provides for color and further intensity modulation
• Limitations of the eye’s ability to perceive local contrast (ratio of 150 to 1) are taken into account to determine:– The minimum number of LEDs necessary in the array– Adequate resolution and blur for the DMD projector– The optimal number of bits necessary to drive each backlight
High Dynamic Range image reproduction
Spatial kernel f
Influence g in the intensity domain for the central pixel
Input
Output
Weights
Images courtesy of Fredo Durand and
Julie Dorsey
Visual Attention and Tasks
Attention is largely controlled by task:– Scene rendered at low,
high and selective resolution.
– Task allocated…– Difference in quality
largely not noticed by participants
Selective Quality Image
Cater et al. 2003
Three varieties of realism
• Physical realism– Same visual stimulation as scene– Highly computationally expensive
• Photo-realism– Same visual response as scene– Takes observer’s visual system into account
• Functional realism– Same visual information as scene– Information useful for completing a task
Ferwerda 2003
Human facial illustrationsGooch et al. 2004
Images courtesy of Bruce Gooch
• Presents a new technique for automatic NPR generation of faces from photos– Photo illustration caricature
• Evaluates functional value of images– Recognition task– Learning task – slower with photos– Accuracy, speed
Human facial illustrationsGooch et al. 2004
• Radar display– Triangles– Circles
• Which is the threat?– Invariant of a sharp
shape invokes connotations of threat
Rendering and affectDuke et al. 2003
Experiments
• Assessment of danger and safety– Radar, door, house & trees
• Assessment of strength and weakness– Radar, strongest man, weakest man
• Goal-directed interaction– Paths, object selection
Results
• Demonstration of how rendering style can convey meaning and influence judgement
• Illustrate how semantics, affect and other high-level invariants need to be taken into account when analysing rendering methods, not just perceptual adequacy and realism.
Distance and Scale in VEs• What affects perception of distance in VEs?
– ‘Triangulated’ walking task Thompson et al. 2003– Image Quality? – Makes no difference
• Is it the HMD? Creem-Regehr et al. 2002– No… artificially generating these restrictions in the
real world did not produce the same problems– No need to see own body (see also Lok et al. 2003)
• However, some cues are important Hu et al. 2002– Shadows and Interreflection affected performance in
a placement task
Presence
• Presence = the sense of “being there”.– In the past, questionnaires and interviews
were predominant– Physiological measurements proposed as a
viable alternative– Shock of entering room with precipice induces
physiological response Meehan et al. 2002– Perhaps can also be used to measure and
predict Breaks in Presence (BIP)
Slater et al. 2004
Animation
Measuring Error
Reitsma and Pollard’03
Accuracy vs. Plausibility
Chenney and Forsyth’00
Collision HandlingCollision Handling
Collisions and PerceptionCollisions and Perception
Evaluating the Visual Fidelity of Physically Based Animations.
Collisions and Attention
Multisensory Perception: 1The multisensory brain:• Areas of brain not
unisensory but active to other sensory information
• Growing body of evidence:– Visual areas active during
tactile perception– Visual areas active during
tactile object recognition– Auditory areas of brain are
active during lip reading (no sound)
Multisensory Perception: 2
• You simply cannot predict perception by studying the senses in isolation
The senses influence each other:
Perceptual metrics for crowds• New metrics to evaluate human simulations, taking
account of:– Multisensory information: vision, motion and sound– Crowd and scene scale– Task
• …based on psychophysical and neuroimagingresults
• …which will be used to:– Devise new multisensory strategies for optimal LOD
control– Implement new error and comparison measurements for
evaluation
State of the Art
Dead Rising (Xbox®360), Capcom, 2006
State of the Art
Madden NFL (Xbox®360), EA, 2006
State of the Art
Project Gotham Racing (Xbox®360), Microsoft, 2005
State of the Art
Assassin’s Creed (Xbox®360), Ubisoft
Tsingos et al. - SIGGRAPH 2004
Sound rendering for crowds
Wand and Straβer PBG 2005