IS-4081, Rabbit: Reinventing Video Chat, by Philippe Clavel

REINVENTING VIDEO CHAT!

2 | RABBIT -‐ REINVENTING VIDEO CHAT | NOVEMBER 19, 2013 | CONFIDENTIAL

!  Rabbit is a new way to share live experiences with anyone, anywhere

!  The core product is a plaMorm that enables unlimited video chat and content sharing

!  Backed by Google Ventures, CrunchFund and Michael Birch

!  Launched Private Beta in February on Mac, extending to Windows and Web in Q1 2014


DEMO

!  Rabbit is currently in Beta and available on Mac OS X

!  We are working closely with AMD on a Windows version

!  We expect a huge performance gain…

Let’s see a demo…!


WHAT IS RABBIT ?

Video chat with an unlimited number of people!


WHAT IS RABBIT ?

Unique video chat experience!


WHAT IS RABBIT ?

Share anything with everyone in a room!


WHAT IS RABBIT ?

Watch together!


CHALLENGES?

!  In short Rabbit allows: ‒ Unlimited number of people in video chat while watching content together ‒  Share Anything you have access to on your computer ‒  Including Audio… ‒ All that with a unique UI and UX…

!  So some tech challenges: ‒  Stream Audio/Video of 30+ streams of LIVE content ‒ Render 30+ streams with Audio Video Synchronizadon ‒ Guarantee nice and dynamic UI at 60fps+ ‒  Screencast any App running on your computer ‒ Audio Capture, Render and Processing…. ‒ Allow unlimited number of people to join a room


A LOT OF STREAMS…


A LOT OF STREAMS…


STREAM A LOT OF STREAMS – HOW?

!  Star5ng Point: ‒ Use Network Friendly Codecs: vp8, Speex ‒ Use Standards: RTSP, RTP, RTCP ‒ Use open source library for network (Live555) ‒  Target High Bandwidth Environment

!  Issue: ‒  Codec makers:

‒  Vp8 didn’t output video quality good enough for real-‐dme encoding when we started ‒  Opus beker than Speex for audio

‒  Standards are great, but…. ‒  RTP/RTCP has port rules that are really painful (2 ports per stream) ‒  RTSP: command structure is heavily geared toward playing movies

‒  Live555 is great for networking and streaming but: ‒  Single Threaded only ‒  Does not scale past 100 streams on server ‒  Some slow network code (read byte by byte, etc..) ‒  No real documentadon…


STREAM A LOT OF STREAMS – HOW!

‒ Custom RTSP/RTP/RTCP stack: ‒ RTP uses one port for all streams ‒ RTSP stack is just publish, play and stop… ‒ RTCP implemented pardally for our needs (live streaming) ‒ Custom UDP Framing/Deframing ‒ Error Cancelladon on an as-‐needed basis

‒ Streaming Server (Hop) developed from scratch ‒ Support high number of streams, 5k+ clients simultaneously ‒ Based on RTP ‒ Deals with hole punching and NATs ‒ Only one port used for streaming


STREAM A LOT OF STREAMS – KEY BENEFITS WITH AMD!

!  While our network layer does not change, our encoding and decoding pipeline gets great benefits from AMD hardware.

!  Encoding: VCE ‒ Designed for low power and low latency encoding (Ideal for real-‐dme encoding) ‒ We are working closely with AMD to use it for all our encoding needs (5 different streams in real dme) ‒ We are able to offload all our encoding from CPU (easy interop with OpenCL)…

!  Decoding: UVD ‒ Using ffmpeg for automadc hardware decoding using UVD via DXVA (simple API call) ‒ When we do computadon (OpenCL/CV), we benefit from HSA SVM transparently (less memcopy)… ‒ UVD is used to decode most of our incoming streams.

VCE and UVD are drastically lowering Rabbit’s CPU usage!


RENDER A LOT OF STREAMS – HOW?

!  Where we started: ‒  Let’s decode using ffmpeg ‒  Just push the data to the Cocoa NSView ‒  Just push Audio to the speakers

!  Issue: ‒ Decoding is slow and falls behind… ‒  Simple UIview consumes too much CPU ‒ Audio is out of synch and can be choppy…


RENDER A LOT OF STREAMS – HOW !

!  Full control of UI render pipeline: Go OpenGL… ‒ A lot of video data pushed (15fps or 30fps) ‒ Minimize memcpy ‒ Use shaders for render opdmizadon ‒ Be smart and lean on the endre pipeline…

!  Full control of Audio Pipeline ‒ Minimize memcpy ‒ Use Low level Audio render (Core Audio)

!  A/V Synch ‒ Buffer both streams on the rendering end ‒  Synchronized pushes to render ‒ Cover for packet loss at network layer


RENDER A LOT OF STREAMS – NOW WITH AMD!

Capture Network Out

Network In Render

To GPU To CPU

To GPU

Color Scale Crop

Encode Face Detect

OpenCV filters

Decode Color Scale Crop

Face Detect

OpenCV filters


RENDER A LOT OF STREAMS – NOW WITH AMD!

!  HSA: ‒ HSA seamlessly shares memory directly with the graphic card… fewer copies are very important for us ;) ‒ By using opdmized Face Detecdon, we plan to drasdcally enhance the user experience (people always in focus in all bubble, etc…)

!  OpenCL/OpenCV : ‒  Scaling / pan and crop offloaded ‒ Adaptadve bilateral filter (noise reducdon) ‒ CLAHE filter (normalize light) ‒  Scale and Crop ‒ Audio focus calculadon


RENDER 60 FPS -‐ HOW!

!  Issue: ‒ Graphics look choppy in most Video Chat Apps ‒ Priority is on Video not Audio ‒ Video is CPU and GPU intensive ‒ There must be a beker way ;)

!  Rabbit = Full game engine: ‒ Game loop ‒  Low level rendering ‒ Nice threading to handle UI separated from processing ‒ Minimize memcpy and memory allocadon ‒ And lot of fun opdmizadon on AMD too;)

Focus on UI/UX gives a very different feel!


SCREENCAST ANY APP ?


SCREENCAST ANY APP – HOW ?

!  Star5ng point: App Capture… Easy! ‒  Let’s capture the images of the App at high fps ‒  Let’s capture the audio of the App ‒  Let’s broadcast…

!  Issue: ‒ Capturing the image of a given window is not as simple as capturing the full desktop ‒ App only audio is very complex to capture ‒ Nice quality broadcast means big NAL and potendal packet loss or delay


SCREENCAST ANY APP – HOW ?

!  Video ‒ Real-‐ dme capture of an App’s window ‒ Real-‐dme encoding using x264 on Mac/VCE on AMD ‒  Slicing used ‒ Added Forward Error Correcdon (FEC) layer to correct loss on large NALs

!  Audio ‒ No way to capture App only audio on Mac ‒ Custom audio capture of system audio on Mac ‒ Real dme encoding using Opus ‒ Added custom Error Cancelladon (EC) to support audio loss ‒ Custom render of full Audio (Rabbit Ears)


Q & A

Quesdons?


DISCLAIMER & ATTRIBUTION

The informadon presented in this document is for informadonal purposes only and may contain technical inaccuracies, omissions and typographical errors.

The informadon contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, sovware changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligadon to update or otherwise correct or revise this informadon. However, AMD reserves the right to revise this informadon and to make changes from dme to dme to the content hereof without obligadon of AMD to nodfy any person of such revisions or changes.

AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.

AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

ATTRIBUTION

© 2013 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinadons thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdicdons. SPEC is a registered trademark of the Standard Performance Evaluadon Corporadon (SPEC). Other names are for informadonal purposes only and may be trademarks of their respecdve owners.

Date post:	13-Jan-2015
Category:	Technology
Upload:	amd-developer-central
View:	929 times
Download:	1 times

IS-4081, Rabbit: Reinventing Video Chat, by Philippe Clavel

Technology