!
• Founder & President!
• 12 years in VR!
!
• - French railways!
!
•
Sébastien Kuntz
• Simplify VR!
• Creation!
• Deployment!
• So YOU can spend more time creating applications
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Rendering speed!
• MiddleVR for Unity!• Conclusion
Plan
Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
• Platform for creating interactive 3d applications!
• Widely used!
• 2 million registered developers!
• Certified by the US Army and Air Force!
• Clients : Bigpoint, Cartoon Network, Coca-Cola, Disney, Electronic Arts, LEGO, Microsoft, NASA, Nickelodeon, Ubisoft, US Army, Warner Bros.
• High-quality rendering!
• Deferred renderer!
• Occlusion culling!
• Realtime shadows!
• Ambient occlusion!
• Light mapping
Unity 3D Features• Programming!
• C#, Javascript!
• Profiler!
• Physics!
• NVidia PhysX!
• Cloth!
• Soft/rigid bodies!
• Ragdolls!
• Cars
3D formats !
FBX, Collada, 3DS, OBJ…!
Audio engine!
Networking!
Terrain
Editor
• Simple to use!
• Create your own tools inside Unity
Asset store
• Buy / sell assets!
• 3D models!
• Textures!
• Shaders!
• Scripts!
• Editor plugins
Desktop
Plan• Unity!
Challenges!• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Plan• Unity!• Challenges!
Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Virtual reality
Presence in a virtual world
(c) eMagin(c) Barco
Experience of presenceis in your brain !
Cognitive presencePerceptive presence
• Build hardware
To create and maintain presence, you need to ...
• Build software
• Build user experience
Plan• Unity!• Challenges!
• Presence!
Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Device management
Physical device!!
Driver Application
Device management
Tracker 1! Driver Tracker 1
Application
Tracker 2!!
Driver Tracker 2
Plan• Unity!• Challenges!
• Presence!• Device management!
Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Correct perspective
(c) Johnny Lee
Correct perspective
Stereoscopy
Warping & Blending
Graphics cards limitations
• Number of outputs!
• Processing power
multiple graphics cards
• Pro!
• More outputs!
• More processing power!
• Con!
• 3D engines are not optimized for multi-pipe
Cluster
• Pro!
• More outputs!
• More processing power!
• Con!
• 3D engines don’t support clustering
• No seams across displays require 3 layers of sync!
• 3D objects synchro (Scenelock)!
• New images display (Swaplock)!
• Left/Right eyes (Genlock)
Cluster
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!
Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Interactions
• Many different 3D user interactions!
• Navigation!
• Selection!
• Manipulation!
• Draw!
• System control!
• etc.
Interactions are dependent on hardware
Menus
Haptics (force feedback)
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!
Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Deployment
• Modify application to!
• Change drivers!
• Change displays!
• Change interactions
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!
Rendering speed!• MiddleVR for Unity!• Conclusion
Rendering speed
Plan• Unity!
Challenges!• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Rendering Speed!
• MiddleVR for Unity!• Conclusion
Application
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
MiddleVR for Unity!• Conclusion
• Build hardware
To create and maintain presence, you, the author, need to ...
• Build software
• Build user experience
“VR is mainly a software problem”!Jaron Lanier
What is MiddleVR ?
• VR plugin !
• Functionalities ○Manage input devices
•3D trackers, keyboard, mouse, joystick !
○Manage display •Viewports, cameras, stereoscopy, cluster
Desktop
http://www.youtube.com/watch?v=EEKH-1EilXs
MiddleVR tutorial
Device management
Device management
Supported Devices• Via VRPN :
• A.R.T • Vicon • Optitrack • Wiimote !
• Polhemus Fastrak Tracker, 3Space Tracker, Patriot Trackers,
• Ascension Flock of Bird • Intersense IS600, IS900 • 3dTech HiBall-3000 Wide Area Tracker, • Advanced Realtime Tracking Gmbh DTrack
Client, • World Viz Precision Position Tracker PPT
1.2, • Natural Point Optitrack Rigid Body Toolkit, • Phasespace optical tracking OWL, • ...
• Native drivers • Oculus Rift • Leap Motion • Kinect (Microsoft SDK) • PNI SpacePoint
Fusion • GameTrak • Razer Hydra • Motion Analysis (beta)
First integration of Haption’s haptic devices
http://www.youtube.com/watch?v=lnoE_VclnOI
Stereoscopy
• Active stereoscopy (OpenGL Quad-Buffer)!
• Which Unity can’t natively handle!
• Passive stereoscopy
Cluster & Multi-pipe
• Scenelock!
• Swaplock!
• Genlock!
• Multi-pipe (soon)
Supported displays
• Virtually any HMD • VR-Wall, Powerwall • Workbench, Holobench • HoloStage • CAVEs • 3D TVs • zSpace • Igloo Vision • …
Deployment
• Abstractions!
• Devices!
• 3D Nodes (User)!
• Interactions (soon)
Device management
Driver Tracker 1
Application
Driver Tracker 2
Device abstraction
Virtual Device
Application
Driver Tracker 1
Driver Tracker 2
Device management
Driver
Virtual Device Application
Simulation
Replay
Virtual Device 1
Hand (3D Node) Application
Virtual Device 2
Inverse Kinematics
User abstraction
Interactions• MiddleVR includes high-level interactions!
• Navigation!
• Selection!
• Manipulation!
• … More to come!
• Future: Interactions abstractions!
• Dynamically change interactions based on hardware
Interactions abstraction?Grab-air
Navigation
Application
Point
Steering
Selection
Manipulation
Raycasting
Go-go
Hand
Homer
Hand
WIM
Menus
• Ability to create menus and GUIs in HTML5 (soon)
Speed• Unity!
• Many options for optimizations!
• Occlusion culling!
• Static batching!
• Lightmap baking!
• etc.!
• 32-bit editor only!
• But player is 32-bit & 64-bit !!
• Subdivide parts in editor!
• Compositing!
• Cloud
Compositing Stereoscopy
Compositing Image decomposition
VR in the Cloud
• Temporary increase in power!
• Combine with compositing
Perceptive illusions
• Playing with limitations of human perception!
• Sensitivity of senses!
• Selectivity of attention!
• Useful to lower cost/complexity of VR systems
Perceptive illusions
(c) Franck Steinicke - Wurzburg University
Redirected walking
Change blindness :
(c) Evan Suma - USC/ICT
Perceptive illusionsChange blindness
