+ All Categories
Home > Technology > Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Date post: 26-Jan-2015
Category:
Upload: yahoo-developer-network
View: 106 times
Download: 1 times
Share this document with a friend
Description:
Talk by Shwetank. Using getUserMedia you can now access the camera and microphone using just plain old JavaScript! In this talk, we'll take a look at the various use cases for it and what all possibilities it opens up for web applications in the future.
Popular Tags:
36
Mic, Camera, Action! Shwetank Dixit Opera Software
Transcript
Page 1: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Mic, Camera, Action!

Shwetank Dixit

Opera Software

Page 2: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

about meWeb Evangelist, Web

Standards Team

Member, W3C Mobile Web

for Social Development

Group

Twitter: @shwetank

Email: [email protected]

Page 3: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

The web is awesome!

Page 4: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

but Native apps sometimes have an edge

But that edge is getting smaller

Page 5: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

We’re adding more ways to get input from the

user

Page 6: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

We’re making existing ways easier

Like web forms

Page 7: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

We want to

StopWriting boring JS

Page 8: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

We want to

StartWriting exciting JS

Page 9: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Mic

Page 10: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

MicDEMO

Page 11: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

MicWeb Audio API + getUserMedia

Page 12: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

getUserMediaif (navigator.getUserMedia){ navigator.getUserMedia({audio: true}, a_success, a_error);}

Page 13: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

getUserMediaMic returns raw audio samplesCannot be played natively by the audio playerNeed Web Audio API to process it and then play it

Page 14: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

getUserMediaCreate audio context and media stream source

Page 15: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

getUserMedia//Create audio context and media stream source

var audioContext = new AudioContext(); // Prefix it for various browsers

// Create an AudioNode from the stream.var mediaStreamSource = audioContext.createMediaStreamSource( stream );

Page 16: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

getUserMedia//Create audio context and media stream source

var audioContext = new AudioContext(); // Prefix it for various browsers

// Create an AudioNode from the stream.var mediaStreamSource = audioContext.createMediaStreamSource( stream );

// Connect audio loop to hear your own voicemediaStreamSource.connect(context.destination);

Page 17: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Recorder.jsGreat library to help you record mic

input using getusermedia

Page 18: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Recorder.jsbuffers the raw audio samples.

converts it to format which can be

played by an <audio> control

Page 19: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Camera

Page 20: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Use camera with getusermediavar options = {audio: false, video: true};

Page 21: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Use camera with getusermediavar options = {audio: false, video: true};

navigator.getUserMedia(options, v_success, v_error);

Page 22: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Use camera with getusermedia// In the HTMLvar video = document.querySelector('video');.........// In the JS Codefunction v_success(stream){ video.src = window.URL.createObjectURL(stream) || stream;}

Page 23: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Use camera + Video + Canvas for new tricksvar button = document.querySelector('#button'); button.addEventListener('click',snapshot, false);......function snapshot(){ var c = document.querySelector('canvas'); var ctx = c.getContext('2d'); var cw = c.clientWidth; var ch = c.clientHeight; ctx.drawImage(video_element, 0, 0, cw, ch); }

Page 24: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

DemoNotice you can click on a polaroid

and save that image. You can create data URIs of canvases

Page 25: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action
Page 26: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Manipulating the image from the webcam

Page 27: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Once you get the snapshot of the webcam

into canvasyou can go crazy with it

Page 28: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

DEMOWarholiser

Page 29: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Action!

Page 30: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Tracking user actionsMotion detection

gesture recognition

Page 31: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

headtrackr.jshttps://github.com/auduno/headtrackr/

Page 33: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Face Detection

X, Y position of face

Width, Height of face relative to canvas

headtrackr.js

Page 34: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Demo

Page 35: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Voice driven apps

Voice modulation apps

Native photo capture in web apps

Live image manipulation and effects

Video conferencing (using WebRTC)Accessibility enhancements to apps

Possible use cases

Page 36: Yahoo Hack India: Hyderabad 2013 | Mic, Camera, Action

Cheers!More questions? Ask me now

or contact me at:[email protected]

or on twitter - @shwetank


Recommended