1 Multimedia Foundations

8/10/2019 1 Multimedia Foundations

1/27

Presented by [email protected] 1.0.0
mailto:[email protected]:[email protected]


2/27

What is Multimedia?

Multimedia has no strict definition.In our context, multimedia indicates thecomputer technology (multimedia computing) formore efficient communication by using different

media types:Text

Audio and speech

Images

GraphicsVideo

2


3/27

Multimedia System

Multimedia involves more thansimple addition of new datatypes.

It integrates a wide range ofsymbol modes simultaneously intoa coherent framework.

The framework is usually denotedas a multimedia system.

3


4/27

Challenges of Multimedia Computing

Developing a successful multimedia system is non-trivial.Continuous media types such as video need a lot ofspace to store and very high bandwidth to transmit.

They also have tight timing constraints.

Automatically analyzing, indexing and organizinginformation in audio, image and video is much harderthan from text.

Multimedia involves many different research areasand needs more complex and more efficientalgorithms and hardware platforms.

4


5/27

Multimedia is Multidisciplinary5

Image, audio,

speech processing

Computer vision,

pattern recognition

Computer

networks,

operating system

Human computer

interaction Computer

graphics

Multimedia

computing


6/27

Example Multimedia Systems6


7/27



8/27


Navigation Learning Environmentin MITs Project Athena


9/27


Photosynth of Microsoft Live Labs.
http://labs.live.com/photosynth/whatis/http://labs.live.com/photosynth/whatis/


10/27

Multimedia Computing

Multimedia systems involve some basic enablingtechniques:

Multimedia data representation and compression.

Multimedia data processing and analysis.

Transmitting multimedia data through communicationnetworks.

Multimedia database, indexing and retrieval.

10


11/27

Data Compression11

Raw image takes about 6M bytes

(without header information)24k bytes with jpeg, Q=50


12/27

JPEG Illustration12

r g b

0

10

20

30

40

5060

0

10

20

30

40

5060

0

1

2

xy

|dc

t(b

-0

.5

)|

Transformto frequency

domain

Quantization

Lossless compression

0010001 .

decoderr gb

uv


13/27

Media Processing and Analysis

In applications such as digital library,automatic data analysis has to be done toextract semantic meanings from audios,images and videos.

Based on media processing methods such as

object tracking (face, eyes),object recognition,

gesture recognition, etc.,

we can build more effective humancomputer interfaces.

13


14/27

Tracking Object by Image Matching14
http://localhost/var/www/apps/conversion/tmp/scratch_4/bear.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/hand.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/book.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/car_back_big.avi


15/27

Multiple Object Tracking15

Toys Squash

LabDouble Squash
http://localhost/var/www/apps/conversion/tmp/scratch_4/double_again_small.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/indoor_3_lp.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/single_squash.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/bear_video.AVI


16/27

Finding Objects in Clutter16


17/27

Object Recognition17

David G. Lowe, "Object recognition from local scale-invariant features," ICCV99.


18/27

Posture Detection in Images18

TemplateImage


19/27

Dealing with Multiple Objects 19


20/27

Action Detection20

Sign language gesture detection

Detecting actions in baseball sequence


21/27

Finding Action Clusters in Image Database 21


22/27

Finding Action Clusters (cont)22


23/27

Action Recognition23

video1 video2

Ramanan, D., Forsyth, D. A., Zisserman, A. "Strike a Pose: Tracking People by Finding Stylized Poses."

Computer Vision and Pattern Recognition (CVPR), San Diego, CA, June 2005.

video3
http://localhost/var/www/apps/conversion/tmp/scratch_4/kwanPostSmoothGreen_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/kwanPostSmoothGreen_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/lolaRunSmooth_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/lolaRunSmooth_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/lolaRunSmooth_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/lolaRunSmooth_divx.avi


24/27

Media Delivery

Transmitting multimediadata across the networkis another topic inmultimedia computing.

There are differentissues when we transmit

video and audio throughpacket network.

Quality of service.

Synchronization.

Error and congestioncontrol.

Session setup andbook keeping.

24


25/27

Multimedia Database and Indexing

Multimedia database has to deal with largemedia files.

Multimedia data needs new datastructures, indexing and searchingmethods.

Content based multimedia retrieval is stillan ongoing research topic.

25


26/27

Useful Resources

JournalsIEEE MultimediaIEEE Transaction on MultimediaIEEE Transaction on Image ProcessingIEEE Transaction on Pattern Recognition and

Machine IntelligenceConferences

ACM MultimediaInternational Conference on Multimedia and Expo(ICME)IEEE Computer Vision and Pattern Recognition(CVPR)International Conference on Computer Vision(ICCV)

26


27/27

Thank You27

Q & A

Date post:	02-Jun-2018
Category:	Documents
Upload:	sandi-ahmad-sadi
View:	216 times
Download:	0 times

1 Multimedia Foundations

Documents