Date post: | 30-Mar-2015 |
Category: |
Documents |
Upload: | kenia-boatwright |
View: | 223 times |
Download: | 1 times |
Immersive Media Vision Create seamless immersive environment for distributed,
interactive real and virtual events Reproduce audio, video and other senses with fidelity
approaching the limits of human perception
Relevant to any type of human interaction: entertainment, education, communications
IMSC Entertainment Vision:RMI
A fusion of internet browsing with a theater-like immersive experience
HD Video at up to 45 Mbits/sec 10.2 ch Immersive Audio (16 Mbits/sec)
Streaming over the Internet on-demand to a mouse click (20 min prog)
RMI demonstrates IMSC innovations in several critical technical areas:
Immersed in a college football game
Doctors assisting in a remote procedure
Business people negotiating like they
are in the same room
Students visiting an aquarium a thousand
miles away
Streaming media
servers
Immersive audio
capture and
rendering
Network protocols for error
correction Synchronization
Applications
Premier Demonstration May 9, 2002
Cross-country streaming from ISI-East in Virginia
NY Times coverage in “Circuits” section
NBC-TV and KTLA http://imsc.usc.edu/rmi/ http://www.east.isi.edu/NGI-
S/
Internet2 Fall 2002 Member Meeting
Audio: 10.2 channel, immersive sound systemVideo: HDTV 1920x1080i
New World Symphony, Miami Beach
550 seat USC Bing Theater
University of Maryland Server
USC
USC Client
New World Symphony
Server
Georgia Tech Server
RMI Experimental Setup Synchronized immersive audio and HDTV streamed playback
from Yima server over Internet2 Control of end-to-end process: capturing, network interface,
transmission, rendering CENIC 2003 “Gigabit or Bust” Award
Remote Master Class
LA Philharmonic cellist located at our USC lab Student at New World Symphony in Miami Beach Three hour session with MPEG2 video and 10.2 immersive audio Teacher reports that student "was really there" with immersive audio Many psychophysical, perceptual and artistic tests to be done
Recent Developments...
Two-way audio communication to New World Symphony in Miami Beach, Florida 8-channel audio – low latency
Live video capture/transmission (DVCam) Near term prospects
RMI theater at Inha University (Korea) 2-way live HD-video/immersive audio
streaming for Internet collaboration
Distributed Immersive Performance Create seamless immersive interactive environment Distributed musicians, conductor (active) and audience
(passive) Q: Why a musical performance?
A: A distributed orchestra is one of the most challenging immersive, interactive environments
Scenario: Multiple soloists scheduled in different cities Under served areas can be reached
IMSC Communication VisionInterAct!
UWB console captures 3D model and problem details
PDA provides spatial array of participants and sonic
translation of details
Immersive workstation coordinates video, avatars, graphics, audio, and haptics to facilitate problem resolution
Research Highlights
immersive audio multichannel and HRTF approaches - holistic DSP approach
streaming servers distributed and scalable architecture
computer vision computational framework for grouping based on tensor voting, tracking for augmented realities and SFX
graphics & animation 3D DSP mesh processing, compression, mesh operations, hair modeling and animation
multimodal emotive interfaces Speech and dialog, vision sensing of body and hands, facial expressions analysis and expressive avatars
virtual reality -- chair of VR2003 in LA applications to psychology (ADD diagnosis), haptics applications, and user studies
perception & cognitive modeling theoretical foundations for understanding benefits of media immersion
UWB wireless leadership in technology, FCC regulation, and commercialization
IMSC has produced ground breaking fundamental research in:
Sensory Interfaces (SI)
Robust Vision Systems Segmentation, motion, 3D body and hand tracking, tensor
voting, activity analysis, recognition and tracking Speech Recognition and Synthesis
Emotive dialog, translation, emotion analysis Immersive Audio
Sonic visualization, spatialization, autocalibration Facial Analysis -- Gestures and Animation
Expression signatures for emotion and recognition Video fusion with 3D models for Surveillance
Structure extraction from LiDAR and texture projection Digital Geometry (3D) Processing
Compression, surface and topology smoothing, mesh simplification