Fraunhofer IIS 1 Spatial Conferencing

Redmond, 16 Oct 2014

Spatial Conferencing

HP Baumeister

Fraunhofer IIS 2 Spatial Conferencing

Telepresence Rooms: High End Conferencing

Fraunhofer IIS 3 Spatial Conferencing

General Reaction: As if speaking to someone in the same room

Fraunhofer IIS 4 Spatial Conferencing

General Reaction: As if speaking to someone in the same room

Why?

In part, because the audio is transparent, and comes from the appropriate direction!

Spatial Audio

Fraunhofer IIS 5 Spatial Conferencing

Spatial Audio

audio: 3 channels of Fraunhofer AAC-LD @ 48kHz video: 3 x H.264 @ 1080p

Fraunhofer IIS 6 Spatial Conferencing

Telepresence for Everyone: Spatial Conferencing

Bring the telepresence experience to smartphones

Using regular headsets/earphones

Fraunhofer IIS 7 Spatial Conferencing

Telepresence for Everyone: Spatial Conferencing

Billions of smartphone and tablets

Fraunhofer IIS 8 Spatial Conferencing

Spatial Conferencing: Key Elements

Highest Quality Audio: Full-HD Voice

Spatialization

Fraunhofer IIS 9 Spatial Conferencing

Spatial Conferencing: Full-HD Voice

14 kHz+ audio bandwidth

Fraunhofer IIS 10 Spatial Conferencing

Full-HD Voice Audio Quality

source: http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_52/Docs/S4-090080.zip (Feasibility study on EVS audio bandwidth, Ericsson 3GPP/SA4 01/2009)

0 1 2 3 4 5

Mean Opinion Score

Fair Good Excellent Bad Poor

NB

WB

SWB

Fraunhofer IIS 11 Spatial Conferencing

Spatial Conferencing Full-HD Voice

14 kHz+ audio bandwidth

Very low delay

Multi-channel capability

Fraunhofer IIS 12 Spatial Conferencing

Spatial Conferencing Full-HD Voice

ELD = Enhanced Low Delay

AAC-ELD is a low latency mode of the widely adopted music codec AAC.

CD-like audio quality for telephone calls

full audio bandwidth up to 20 kHz

low coding delay down to 15 ms, crucial for natural conversations

optimized for a bit-rate range of 24 kbit/s to 64 kbit/s per channel

AAC-ELDv2 enhances bit-rate range down to 24 kbit/s stereo

Most widely adopted codec offering Full-HD Voice today

FaceTime

Native in iOS

Native in Android

Fraunhofer IIS 13 Spatial Conferencing

Spatial Conferencing for Everyone Spatialization

Geometric Localization of Participants

With or without Video

Fraunhofer IIS 14 Spatial Conferencing

Spatial Conferencing Head Related Transfer Function

Fraunhofer IIS 15 Spatial Conferencing

Spatial Conferencing Head Related Transfer Function

Credit: University of Maryland

Fraunhofer IIS 16 Spatial Conferencing

Spatial Conferencing Out of Head Experience

Stereo = Inside Head

Spatial Rendering = outside Head

Fraunhofer IIS 17 Spatial Conferencing

Audio Conferencing

MCU

Fraunhofer IIS 18 Spatial Conferencing

Audio Conferencing single channel everyone is in the same location

MCU

Fraunhofer IIS 19 Spatial Conferencing

Spatial Conferencing Spatialization participants can be localized as if in the room

MCU

positioning &

out of head

Full-HD Voice (spatialized)

Fraunhofer IIS 20 Spatial Conferencing

Spatial Conferencing MCU

Fraunhofer IIS 21 Spatial Conferencing

3-Point conference via IP

Full Band Audio, 44.1 kHz

Immersive, spatial experience

Spatial Conferencing Demo Setup

3x Nexus 5 - Android 4.4.3.2.1 - VoIP App using native ELD

standard in-ear headset

Mac-Mini running MCU with binaural processing

64 kbps mono

128 kbps stereo

Fraunhofer IIS 22 Spatial Conferencing

Developed and evaluated in collaboration with

Spatial Conferencing Trials

Full-HD VoicePSTN

I found it really amazing, because one really

thought you are sitting in the same room

Fraunhofer IIS 23 Spatial Conferencing

Member Companies should consider spatial conferencing in future systems

Define uses cases

Interface Specification / Standardisation

Interoperability

Spatial Conferencing IMTC Opportunities

Fraunhofer IIS 24 Spatial Conferencing

See me: HP Baumeister

Call me: 408 573 9903

Mail Me hp.baumeister@dmt.fraunhofer.org

Thank you! Questions? Live Demo?

Fraunhofer IIS 25 Spatial Conferencing

Spatial Conferencing Spatialization

Full-HD Voice (mono)

MCU

positioning &

out of head

Full-HD Voice (spatialized)