Date post: | 21-Jan-2018 |
Category: |
Technology |
Upload: | intel-software |
View: | 331 times |
Download: | 2 times |
Open XR – And the Future of AR/VR/MR Gaming
Brent Insko, Virtual Reality Center of Excellence
October, 2017
Where is VR today?
2
3
Gartner Hype Cycle 2017
© 2017 Gartner, Inc.
What is holding VR back?
4
What is holding VR back?
• Content• Comfort• Cost
5
6
7
Platforms
8
Platforms VR Platforms
9
3 PCVR Verticals
Oculus Store Microsoft Store
Mixed Reality App
Mixed RealityRuntime
Oculus App
OculusRuntime
Mixed RealityHMDs
OculusRift
Steam Store
SteamVRApp
SteamVRRuntime
HTC Vive(+ HDK & others)
10
11
12
13
What is a VR Runtime?
14
15
High Level VR Software Stack
VR RuntimeServer Compositor
DirectX (+ other system APIs)
Windows
VR Application Browser
VRHardware
OpenXRAPIs
WebVRGame Engine
Application interface
Driver(HW) interface
• Takes Windows surfaces from multiple applications
• Composites onto a single desktop in 2.5D
• Handles transparency/ translucency
Windows Compositor… Desktop Window Manager (DWM)
VR Compositor
T1 T2 T3
• Composition• Time Warp• Barrel Distortion• Chromatic Aberration
Correction
VR Compositor
18
• Head and controller positions
• Controller state• Input events
• Game images• Haptics events
Information Flow
19
Why does Intel care?
20
VR Vision – The PC PlatformThe PC is doing more than ever to stimulate the user and drive their senses
Physically driven audioAudio effects that are true to life, reflecting off
objects and surface materials, accounting for distance and orientation.
Intense graphicsHigher resolution, faster frame rates, and depth capable displays, fully immerse players in the experience.
Hands-free inputDepth cameras incorporated into HMDs and
connected to the PC track arm and hand movement allow controller-less input
Multiplayer supportSimulation accounts for multiple players in the
same room – including both players VR experience being driven off of one high-performance PC.
Wireless freedomHigh-speed, low-latency wireless driving of all peripherals.
Haptic feedbackVest and controllers mimic in-game interactions to further stimulate the player and make interactions between teammates more physical.
Vest reacts with jolt from enemyweapon fire.
Vest and controllers relay low health condition.
Physically accept a healing item handed to you by a teammate. Object heft is conveyed to you.
1
2
3
We believe VR will evolve to allow multi-modal input and feedback. Here we show a variety of the technologies that the PC will be providing feedback to the user.
CPU WorkloadsParticle effects and physics, Artificial
intelligence, Destruction and Object Persistence, Initialization & level setup, Spatial binaural 3D
audio, frame rate, mega-tasking workloads
VR Vision – The Sensor PlatformFully immersing users in VR will include full awareness of their state and movement
Facial expression trackingMask sensors map your reactions to your in
game character, enhancing non-verbal communication and heightening involvement.
Object scanningDepth cameras in HMD allow scaling real
world objects to bring in game. Swing a favorite toy sword or bring a child’s art class
project into social spaces.
Eye trackingPlayers eyes are reflected in their in game character, expressing emotion and intent. Gaze tracking can unlock UI triggers, and drive rendering performance focus.
Biometric sensorsHeart rate, pupil dilation and galvanic skin response can be used to convey emotional responses such as fear or excitement.
Responses can be analyzed to tailor game difficulty, give cues to other players, or to limit fatigue.
At the same time, the PC will be capturing a wide array of conscious and unconscious input from the player, and feeding this back to the simulation for processing
Intuitive ControllersControllers evolve to intuitively
communicate greater degrees of intent, with analog triggers, contact sensing, and greater
degrees of affordance
Physics-driven presence Motion tracking of HMDs and controllers, combined with 3D cameras, and CPU-driven physics are used to track and confer full body articulation .
23
3rd Person Camera AI (Yoann Pignole Gamasutra 2015)
GFX
Source(PC) Sink(HMD)
Panel(Output)
Sensors(Input)
RenderTimeWarpBarrelDistortionChromaCorrection
Display(Scalar)
PoseInputImagingData
Tx Rx
USBDisplay
Displaylink Wireless VR
Intel’s partnership with ISVs on VR content
Intel Projects
2020 Virtual Reality Vision
Adventure together with friends, compete with rivals, or bring the world along for the show
Open Innovation & Best of class performance allow VR to thrive and lead on PCUsage: Truly Immersive multiplayer gaming allows players to travel to other worlds, whether next to each other or across the globe
Focus areas: Gaming, eSports, Social
Key capability areas (hypotheses)
• Mind-tripping quality: Photo-real, deep behavioral simulation, responsive enough to exceed sensory reqs. “Believed I was there”
• Wireless HMDs and peripherals and/or wearable form factors for unhindered and unlimited movement
• Multi sensory stimulation: physiological-transparent quality display & graphics, physically modeled 3D Audio, (haptic stimulation), (vestibular dampening),
• Multi-model input, sensing, and responsive simulation: eye-tracking, expression tracking, tactile feedback, biometric feedback (galvanic response, heart rate, etc), voice & outside audio cuing, environment capture
• Social & competitive: Your expressions & movements driving your presence in virtual interactions with others
New York Los Angeles London
WW Spectators
The Future of AR/VR/MR Gaming & OpenXR• Engaging more senses, better experiences
à more immersion
à more presence
• More innovation
25
The Future of AR/VR/MR Gaming & OpenXR• Engaging more senses, better experiences
à more immersion
à more presence
• More innovation
• Dependent on:
à more content more places
à more hardware more places
à easier cross-platform development & testing
26
Placeholder Footer Copy / BU Logo or Name Goes Here
28
à more content more placesà more hardware more placesà easier cross-platform development & testing