Date post: | 26-Jan-2015 |
Category: |
Technology |
Upload: | yahoo-developer-network |
View: | 106 times |
Download: | 1 times |
Mic, Camera, Action!
Shwetank Dixit
Opera Software
about meWeb Evangelist, Web
Standards Team
Member, W3C Mobile Web
for Social Development
Group
Twitter: @shwetank
Email: [email protected]
The web is awesome!
but Native apps sometimes have an edge
But that edge is getting smaller
We’re adding more ways to get input from the
user
We’re making existing ways easier
Like web forms
We want to
StopWriting boring JS
We want to
StartWriting exciting JS
Mic
MicDEMO
MicWeb Audio API + getUserMedia
getUserMediaif (navigator.getUserMedia){ navigator.getUserMedia({audio: true}, a_success, a_error);}
getUserMediaMic returns raw audio samplesCannot be played natively by the audio playerNeed Web Audio API to process it and then play it
getUserMediaCreate audio context and media stream source
getUserMedia//Create audio context and media stream source
var audioContext = new AudioContext(); // Prefix it for various browsers
// Create an AudioNode from the stream.var mediaStreamSource = audioContext.createMediaStreamSource( stream );
getUserMedia//Create audio context and media stream source
var audioContext = new AudioContext(); // Prefix it for various browsers
// Create an AudioNode from the stream.var mediaStreamSource = audioContext.createMediaStreamSource( stream );
// Connect audio loop to hear your own voicemediaStreamSource.connect(context.destination);
Recorder.jsGreat library to help you record mic
input using getusermedia
Recorder.jsbuffers the raw audio samples.
converts it to format which can be
played by an <audio> control
Camera
Use camera with getusermediavar options = {audio: false, video: true};
Use camera with getusermediavar options = {audio: false, video: true};
navigator.getUserMedia(options, v_success, v_error);
Use camera with getusermedia// In the HTMLvar video = document.querySelector('video');.........// In the JS Codefunction v_success(stream){ video.src = window.URL.createObjectURL(stream) || stream;}
Use camera + Video + Canvas for new tricksvar button = document.querySelector('#button'); button.addEventListener('click',snapshot, false);......function snapshot(){ var c = document.querySelector('canvas'); var ctx = c.getContext('2d'); var cw = c.clientWidth; var ch = c.clientHeight; ctx.drawImage(video_element, 0, 0, cw, ch); }
DemoNotice you can click on a polaroid
and save that image. You can create data URIs of canvases
Manipulating the image from the webcam
Once you get the snapshot of the webcam
into canvasyou can go crazy with it
DEMOWarholiser
Action!
Tracking user actionsMotion detection
gesture recognition
headtrackr.jshttps://github.com/auduno/headtrackr/
headtrackr.jshttp://auduno.tumblr.com/post/25125149521/head-tracking-with-
webrtc
Face Detection
X, Y position of face
Width, Height of face relative to canvas
headtrackr.js
Demo
Voice driven apps
Voice modulation apps
Native photo capture in web apps
Live image manipulation and effects
Video conferencing (using WebRTC)Accessibility enhancements to apps
Possible use cases