Date post: | 02-Jun-2018 |
Category: |
Documents |
Upload: | sandi-ahmad-sadi |
View: | 216 times |
Download: | 0 times |
of 27
8/10/2019 1 Multimedia Foundations
1/27
Presented by [email protected] 1.0.0
mailto:[email protected]:[email protected]8/10/2019 1 Multimedia Foundations
2/27
What is Multimedia?
Multimedia has no strict definition.In our context, multimedia indicates thecomputer technology (multimedia computing) formore efficient communication by using different
media types:Text
Audio and speech
Images
GraphicsVideo
2
8/10/2019 1 Multimedia Foundations
3/27
Multimedia System
Multimedia involves more thansimple addition of new datatypes.
It integrates a wide range ofsymbol modes simultaneously intoa coherent framework.
The framework is usually denotedas a multimedia system.
3
8/10/2019 1 Multimedia Foundations
4/27
Challenges of Multimedia Computing
Developing a successful multimedia system is non-trivial.Continuous media types such as video need a lot ofspace to store and very high bandwidth to transmit.
They also have tight timing constraints.
Automatically analyzing, indexing and organizinginformation in audio, image and video is much harderthan from text.
Multimedia involves many different research areasand needs more complex and more efficientalgorithms and hardware platforms.
4
8/10/2019 1 Multimedia Foundations
5/27
Multimedia is Multidisciplinary5
Image, audio,
speech processing
Computer vision,
pattern recognition
Computer
networks,
operating system
Human computer
interaction Computer
graphics
Multimedia
computing
8/10/2019 1 Multimedia Foundations
6/27
Example Multimedia Systems6
8/10/2019 1 Multimedia Foundations
7/27
Example Multimedia Systems7
8/10/2019 1 Multimedia Foundations
8/27
Example Multimedia Systems8
Navigation Learning Environmentin MITs Project Athena
8/10/2019 1 Multimedia Foundations
9/27
Example Multimedia Systems9
Photosynth of Microsoft Live Labs.
http://labs.live.com/photosynth/whatis/http://labs.live.com/photosynth/whatis/8/10/2019 1 Multimedia Foundations
10/27
Multimedia Computing
Multimedia systems involve some basic enablingtechniques:
Multimedia data representation and compression.
Multimedia data processing and analysis.
Transmitting multimedia data through communicationnetworks.
Multimedia database, indexing and retrieval.
10
8/10/2019 1 Multimedia Foundations
11/27
Data Compression11
Raw image takes about 6M bytes
(without header information)24k bytes with jpeg, Q=50
8/10/2019 1 Multimedia Foundations
12/27
JPEG Illustration12
r g b
0
10
20
30
40
5060
0
10
20
30
40
5060
0
1
2
xy
|dc
t(b
-0
.5
)|
Transformto frequency
domain
Quantization
Lossless compression
0010001 .
decoderr gb
uv
8/10/2019 1 Multimedia Foundations
13/27
Media Processing and Analysis
In applications such as digital library,automatic data analysis has to be done toextract semantic meanings from audios,images and videos.
Based on media processing methods such as
object tracking (face, eyes),object recognition,
gesture recognition, etc.,
we can build more effective humancomputer interfaces.
13
8/10/2019 1 Multimedia Foundations
14/27
Tracking Object by Image Matching14
http://localhost/var/www/apps/conversion/tmp/scratch_4/bear.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/hand.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/book.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/car_back_big.avi8/10/2019 1 Multimedia Foundations
15/27
Multiple Object Tracking15
Toys Squash
LabDouble Squash
http://localhost/var/www/apps/conversion/tmp/scratch_4/double_again_small.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/indoor_3_lp.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/single_squash.AVIhttp://localhost/var/www/apps/conversion/tmp/scratch_4/bear_video.AVI8/10/2019 1 Multimedia Foundations
16/27
Finding Objects in Clutter16
8/10/2019 1 Multimedia Foundations
17/27
Object Recognition17
David G. Lowe, "Object recognition from local scale-invariant features," ICCV99.
8/10/2019 1 Multimedia Foundations
18/27
Posture Detection in Images18
TemplateImage
8/10/2019 1 Multimedia Foundations
19/27
Dealing with Multiple Objects 19
8/10/2019 1 Multimedia Foundations
20/27
Action Detection20
Sign language gesture detection
Detecting actions in baseball sequence
8/10/2019 1 Multimedia Foundations
21/27
Finding Action Clusters in Image Database 21
8/10/2019 1 Multimedia Foundations
22/27
Finding Action Clusters (cont)22
8/10/2019 1 Multimedia Foundations
23/27
Action Recognition23
video1 video2
Ramanan, D., Forsyth, D. A., Zisserman, A. "Strike a Pose: Tracking People by Finding Stylized Poses."
Computer Vision and Pattern Recognition (CVPR), San Diego, CA, June 2005.
video3
http://localhost/var/www/apps/conversion/tmp/scratch_4/kwanPostSmoothGreen_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/kwanPostSmoothGreen_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/lolaRunSmooth_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/lolaRunSmooth_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/lolaRunSmooth_divx.avihttp://localhost/var/www/apps/conversion/tmp/scratch_4/lolaRunSmooth_divx.avi8/10/2019 1 Multimedia Foundations
24/27
Media Delivery
Transmitting multimediadata across the networkis another topic inmultimedia computing.
There are differentissues when we transmit
video and audio throughpacket network.
Quality of service.
Synchronization.
Error and congestioncontrol.
Session setup andbook keeping.
24
8/10/2019 1 Multimedia Foundations
25/27
Multimedia Database and Indexing
Multimedia database has to deal with largemedia files.
Multimedia data needs new datastructures, indexing and searchingmethods.
Content based multimedia retrieval is stillan ongoing research topic.
25
8/10/2019 1 Multimedia Foundations
26/27
Useful Resources
JournalsIEEE MultimediaIEEE Transaction on MultimediaIEEE Transaction on Image ProcessingIEEE Transaction on Pattern Recognition and
Machine IntelligenceConferences
ACM MultimediaInternational Conference on Multimedia and Expo(ICME)IEEE Computer Vision and Pattern Recognition(CVPR)International Conference on Computer Vision(ICCV)
26
8/10/2019 1 Multimedia Foundations
27/27
Thank You27
Q & A