Slide 1

SID Display Week 2016Light Field, Focus-tunable, and Monovision Near-eye Displays

Gordon Wetzsteinwww.computationalimaging.orgMay 25, 2016

1

http://www.slideshare.net/StanfordComputationalImaging/light-field-focustunable-and-monovision-neareye-displays-for-virtual-and-augmented-realityDont take photos, just download the slides!

2

E&MWave Optics

Geometric Optics

Modern Signal Processingand OptimizationWhat is Computational Imaging?

3

++opticscomputationsensingComputational ImagingWhat is Computational Imaging?

[Devebec,Nayar,]HDR ImagingSuper-resolutionEDOFLight FieldsCompressive Imaging[Baker,Ben-Ezra,]

[Dowski,Nayar,][Levoy,][Baraniuk,]

4

What is Computational Imaging?

[Devebec,Nayar,]HDR ImagingSuper-resolutionEDOFLight FieldsCompressive Imaging[Baker,Ben-Ezra,]

[Dowski,Nayar,][Levoy,][Baraniuk,]

piximnexusphones

light.co

Ricoh

LytroGoogle??

5

computationoptics &electronicshuman visual system

++interactionComputational DisplaysWhat is Computational Imaging?[Seetzen,]HDR DisplaySuper-resolutionEDOFLight Fields

Near-eyeDisplays[Wetzstein, ]

6

Stanford Computational Imaging Group

=?

(Some) Technology ChallengesVergence-accommodation conflict (VAC)

Vestibular-visual conflict (motion sickness)

(Some) Technology ChallengesVergence-accommodation conflict (VAC)

Vestibular-visual conflict (motion sickness)

ARocclusionsaesthetics / form factorbattery lifeheatwireless operation

low-power computer visionregistration of physical / virtual world and eyes consistent lightingscanning real worldVAC more importantdisplay contrast & brightnessfast, embedded GPUs

Top ViewReal World:

Vergence & Accommodation Match!

17

VR/AR Display Optics = Magnifier

Top ViewNear-eye Displays Today (all stereo displays):

Vergence-Accommodation Mismatch!virtual image

19

Light Field Cameras

Light Field StereoscopeACM SIGGRAPH 2015

BacklightThin Spacer & 2nd panel (6mm)Magnifying LensesLCD PanelLight Field StereoscopeACM SIGGRAPH 2015

Target Light Field

Input: 4D light field for each eyeModel Courtesy of Bushmills Irish Whiskey

22

Multiplicative Two-layer Modulation

Model Courtesy of Bushmills Irish WhiskeyInput: 4D light field for each eye

23

Multiplicative Two-layer Modulation

Parallax over the Pupil

Model Courtesy of Bushmills Irish WhiskeyInput: 4D light field for each eye

24

Multiplicative Two-layer Modulation

Reconstruction:[Wetzstein et al 2012]Model Courtesy of Bushmills Irish WhiskeyInput: 4D light field for each eye

25

Traditional HMDs- No Focus CuesThe Light Field HMDStereoscopeLight Field StereoscopeACM SIGGRAPH 2015

26

Traditional HMDs- No Focus CuesThe Light Field HMDStereoscopeLight Field StereoscopeACM SIGGRAPH 2015

27

Model Courtesy of Paul H. ManningTraditional HMDs- No Focus CuesThe Light Field HMDStereoscopeLight Field StereoscopeACM SIGGRAPH 2015

28

Traditional HMDs- No Focus CuesThe Light Field HMDStereoscopeModel Courtesy of Paul H. ManningLight Field StereoscopeACM SIGGRAPH 2015

29

MonovisionACM SIGCHI 2016

30

MonovisionACM SIGCHI 2016

Evaluation through user studyDisplay ModesConventional

32

Evaluation through user studyDisplay ModesConventionalConventional + DoF

33

Evaluation through user studyDisplay ModesConventionalConventional + DoFAdaptive Focus

Gaussian Thin Lens Equation

o lens to screen distancedv lens to virtual image distf focal length of lens

34

Evaluation through user studyDisplay ModesConventionalConventional + DoFAdaptive FocusAdaptive Focus + DoF

35

Evaluation through user studyDisplay ModesConventionalConventional + DoFAdaptive FocusAdaptive Focus + DoFMonovision

36

User Preference study

User performance studies faster & more accurate

MonovisionACM SIGCHI 2016

37

User Preference study

User performance studies faster & more accurate

MonovisionACM SIGCHI 2016

38

Monovision

39

User Study SummaryMonovisionBetween conventional and adaptive focus in preferenceImproved time to fuse in visual clarity taskAdaptive focusMost preferred Improved time to fuse in visual clarity taskWith DoF improved accuracy in depth judgment taskSoftware only approach doesnt have measurable effect on VAC

40

Conclusion Use gaze contingent focus!MonovisionEasy to implement, effectiveGaze-contingent focusShows highest potential in our studyEye trackingFocus-tunable / actuated display

41

Conclusion Use gaze contingent focus!MonovisionEasy to implement, effectiveGaze-contingent focusShows highest potential in our studyEye trackingFocus-tunable / actuated display

42

EE 267 - http://stanford.edu/class/ee267/

Acknowledgements

Robert KonradFu-Chung HuangEmily Cooper

Gordon WetzsteinComputational Imaging GroupStanford Universitystanford.edu/~gordonwz

www.computationalimaging.org

45

User Preference Study

Asked users to visually explore 3 VR environmentsIn each one, uniquely rank the 5 display modesOverall quality of their experience46

User Preference - ResultsGoodBad

Bar heights represent the mean rankingError barsPoints represent individual sample points1 was the best (most preferred rank)Preferred the conventional mode the least47

User Preference - Results

Slightly better, but not statistically significant effectMean of 3.548

User Preference - Results

Mean of 2.2

49

User Preference - Results

Mean of 2.450

User Preference - Results

Mean of 3.2Falls somewhere in between adaptive, and conventionalInteresting because it isnt necessarily a natural effect with each eye seeing a differently focused imageAdaptive Focus (with and without DOF), showed statistical effect over the conventional modeFriedman test (and followup Tukey-Kramer method)51

Visual Clarity Study

Users identified the orientation of a small letter E placed on target within sceneMeasured reaction time (time-to-fuse) and accuracyTime-to-fuse correlated with amount of VACLarge VAC increases time to fuseExpected conventional to perform poorly compared to adaptive focusUser asked to fixate on initial target, at either 0.4, 1, or 2DSecond target appeared at relative distance: -0.5, -0.2, 0, 0.2, 0.5 DRemove relative object size cue22.59 arcminutes

52

Visual Clarity Reaction Time

fasterslower

Just showing closest case for nowInitial target at 2D with the secondary targets placed at the distances belowResults compared to conventional mode, representing by the dashed linePoints above the line were faster, and points below were slower53

Visual Clarity Reaction Time

fasterslower

Just adding in DoF effect didnt show a positive effect54

Visual Clarity Reaction Time

fasterslower

As expected adaptive focus reduces VAC and results in better reaction times55

Visual Clarity Reaction Time

fasterslower

Shows statistical significance compared to conventional mode in closer areas56

Visual Clarity Reaction Time

fasterslower

Shows statistical effect over conventional at closer target distancesThe relative increase at close target distances is largely due to the fact that reaction times in normal mode slowed down at these distances, while the the reaction times in monovision and adaptive focus modes remained largely stableAs expected, because the conflict for the conventional mode with the screen placed at 1.3m results in having the strongest conflict at close distances57

Visual Clarity Reaction Time

Not much effect in the farther viewing distances, as expected because the VAC is weaker there for the conventional mode58

Visual Clarity Accuracy

Average accuracy across all trial types 91%Not only better reaction times, but also better accuracy in close by targets59

Depth Judgement Study

Similar setup to visual clarity study with some initial target and relative distancesUsers asked to make relative depth judgement between two these two targetsExpected some improvement of adaptive focus over conventional because we support an additional depth cue (accommodation)Remove relative object size cue60

Depth Judgement Reaction Time

Did not observe a significant effectReaction times tended to slow down for closer targetsOne maybe think that monovision may be slightly slower because of reduction in stereopsis, but we didnt observe that effect61

Depth Judgement Accuracy

Adaptive Focus + DOF has statistical effect over conventional mode62