www.swing3d.io
Building Blocks of WebVRIntroduction to VR and WebVRwith Mirek Ciastek, SwingDev’s Front-end Developer
What’s virtual reality
It’s a computer-generated scenario that simulates a realistic experience. The immersive environment can be similar to the real world in order to create a lifelike experience grounded in reality or sci-fi.
Back to the past
90’s and recently
Today
Designing for VR
Key concpets of VR
Head-mounted display (HMD) Position / Orientation / Velocity / Acceleration
Field of ViewStereoscopic vision
UX of VR
• Comfortable using
• In-world interface
• Immersive sounds
• Uninterrupted movement
• Interaction with world
Problems of VR
Hardware
Health issues
Best experience = High price
+
VR in browser
From VR to WebVR
WebVR is an open specification that makes it possible to experience VR in your browser. The goal is to make it easier for everyone to get into VR experiences, no matter what device you have.
All this thanks to browser environment!
APIs useful for WebVR
• WebVR API - manages displays
• WebGL - provides a way to run 3D in the browser
• Gamepad API - helps with using controllers
• Web Audio API - if you want to add some sound
• Web Video API - for 360 videos
• Web Workers - might give good performance boost
Libraries
Let’s make a VR in the browser
Make a VR scene
setScene () { this.camera = new THREE.PerspectiveCamera( 60, ASPECT_RATIO, 0.01, 10000 )
this.scene = new THREE.Scene() this.scene.add(this.camera) this.scene.background = new THREE.Color().setHSL(0.6, 0, 1) this.scene.fog = new THREE.Fog( this.scene.background, 1, 5000 )}
setRenderer () { const canvasEl = document.getElementById('scene')
this.renderer = new THREE.WebGLRenderer({ canvas: canvasEl }) this.renderer.setPixelRatio(window.devicePixelRatio) this.renderer.setSize(window.innerWidth, window.innerHeight) this.renderer.gammaInput = true this.renderer.gammaOutput = true this.renderer.setClearColor(0x000000)
this.renderer.shadowMap.enabled = true}
setComponents () { const sky = Sky() const ground = Ground() this.wall = new Wall(this.renderer) this.turret = new Turret(this.renderer) this.cursor = new Cursor()
this.turret.init() .then(this.handleModelLoad)
this.wall.init() .then(this.handleModelLoad)
this.scene.add(this.cursor.mesh) this.scene.add(sky) this.scene.add(ground)}
setLights () { const { hemiLight, dirLight } = makeLights() this.scene.add(hemiLight) this.scene.add(dirLight)}
Allow to move camera
MousePanControlshandleMouseMove = (e) => { if (!this.tracking) { return }
const width = this.target.innerWidth || this.target.clientWidth const height = this.target.innerHeight || this.target.clientHeight
const deltaX = (typeof this.lastX === 'number') ? e.screenX - this.lastX : 0 const deltaY = (typeof this.lastY === 'number') ? e.screenY - this.lastY : 0 this.lastX = e.screenX this.lastY = e.screenY
this.yaw += THREE.Math.degToRad( deltaX / width * this.camera.fov * this.camera.aspect )
this.pitch += THREE.Math.degToRad(deltaY / height * this.camera.fov) this.pitch = Math.max(-HALF_PI, Math.min(HALF_PI, this.pitch))};
DeviceOrientationControlsupdate () { if (!this.enabled) { return } const alpha = this.deviceOrientation.alpha || 0 const beta = this.deviceOrientation.beta || 0 const gamma = this.deviceOrientation.gamma || 0 const orientation = this.screenOrientation
// Update the camera rotation quaternion const quaternion = this.camera.quaternion euler.set(beta, alpha, -gamma, 'YXZ') quaternion.setFromEuler(euler)
if (this._initialAlpha !== null) { rotation.setFromAxisAngle(Y_UNIT, -this._initialAlpha) quaternion.premultiply(rotation) }
quaternion.multiply(SCREEN_ROTATION) rotation.setFromAxisAngle(Z_UNIT, -orientation) quaternion.multiply(rotation)}
VRControlsclass VRControls { constructor (camera, vrDisplay) { this.camera = camera this._vrDisplay = vrDisplay }
update (options) { const pose = options.frameData ? options.frameData.pose : null if (pose) { if (pose.position) { this.camera.position.fromArray(pose.position) } if (pose.orientation) { this.camera.quaternion.fromArray(pose.orientation) } } }}
Allow to interact with world
GamepadControlslistenGamepadEvents () { const gamepads = getGamepads() // navigator.getGamepads
for (let i = 0; i < gamepads.length; i += 1) { if (gamepads[i]) { const buttons = gamepads[i].buttons
for (let j = 0; j < buttons.length; j += 1) { if (isPressed(buttons[j])) { this.batchedEvents.push({ type: 'GamepadEvent', eventType: 'keydown', button: buttons[j], gamepad: i }) } } } }}
Check for box hit
hit (origin, direction) { raycaster.set(origin, direction)
const intersects = raycaster.intersectObjects(this.root.children)
for (let i = 0; i < intersects.length; i += 1) { const { object } = intersects[i] this.root.remove(object) }
if (this.root.children.length === 0) { this.build() }}
Add some extra features
Crosshairconst texture = new THREE.TextureLoader().load('images/crosshair.png')const geometry = new THREE.PlaneGeometry( DEFAULT_CURSOR_WIDTH, DEFAULT_CURSOR_WIDTH)
const material = new THREE.MeshBasicMaterial({ transparent: true, side: THREE.DoubleSide, depthTest: false, depthWrite: false, map: texture})
this.mesh = new THREE.Mesh(geometry, material)this.mesh.raycast = () => nullthis.mesh.renderOrder = 1
Explosionsconst particleSettings = { texture: { value: textureLoader.load(smokeImage) }, depthTest: true, depthWrite: false, blending: THREE.NormalBlending, maxParticleCount: 1000}
const emitters = [ { particleCount: 600, type: SPE.distributions.SPHERE, position: { radius: 0.1 }, maxAge: { value: 0.5 }, activeMultiplier: 20, velocity: { value: new THREE.Vector3(1.2) }, size: { value: 1.5 }, opacity: { value: [0.5, 0] } }]
// init particlesinitParticles () { this.particleGroup = new SPE.Group( particleSettings )
this.particleGroup.addPool(1, emitters, false) this.headGroup.add(this.particleGroup.mesh)}
// play sound and show smoketriggerSmoke () { player.play('shot') this.particleGroup.triggerPoolEmitter( 1, smokePosition )}
I used Shader Particle Engine
Soundsclass SoundPlayer { constructor () { this.context = new (window.AudioContext || window.webkitAudioContext)() this.bufferLoader = new BufferLoader( this.context, sounds, this.handleLoad )
this.loaded = false this.bufferLoader.load() }
handleLoad = () => { this.loaded = true }
play (soundName) { if (this.loaded) { const source = this.context.createBufferSource() source.buffer = this.bufferLoader.bufferList[soundName]
source.connect(this.context.destination) source.start(0) } }}
…and allow to enter VR mode
VRDisplay API provides all needed methods and properties to serve your scene in VR compatible format
• eyes position - you need to slice your scene in half
• isPresenting flag - check if you can serve VR frames
• frameData - includes device position, etc.
• submitFrame - remember to call it when your frame is prepared
Check three.js VREffect interface for more details
How you can extend it
• Use any WebVR framework (A-frame or ReactVR) - the farther you go the harder it gets to maintain all the things
• Add any UI to allow user to track their progress
• Add some physics - make the wall collapse
• Add support for 3DoF and 6DoF controller - yes, you can do it in your browser!
Further reading 1/2• The User Experience of Virtual Reality
• Virtual Reality Basics
• VR for UX Designers: What I Learned During My First Project
• History Of Virtual Reality
• The very real health dangers of virtual reality
• The Story of VR - A Look at the History Behind Virtual Reality
• Design Practices in Virtual Reality
• The Future of Virtual Reality
• Designing Screen Interfaces for VR (video)
Further reading 2/2• WebVR concepts
• Introduction to VR Web
• Building Virtual Reality on the Web with WebVR (video)
• Using the WebVR API
• Using the Gamepad API
• Using VR controllers with WebVR
• Getting started with three.js
• Web Audio API
Thank you