Multimedia SystemsPart 1

Mahdi Vasighiwww.iasbs.ac.ir/~vasighi

Department of Computer Science and Information Technology,Institute for Advanced Studies in Basic Sciences, Zanjan, Iran

The course includes fundamental concepts ofmultimedia, Speech, Image and Video processingincluding international standards.

Multimedia Concept and Topics Audio Fundamentals Audio Coding and Standard Image/Video Fundamentals Image/Video Coding MPEG Coding Standards

Teaching Plan

Fundamentals of MultimediaSecond Edition

Ze-Nian LiMark S. DrewJiangchuan Liu

Springer, 2014ISSN 1868-0941

Textbook

Midterm Exam 20% Final Exam 50% Homework 20% Attendance 10%

Grading Policy

What is Multimedia?

Multi: more than oneMedium (singular): middle, intermediary, meanMedia (plural): means for conveying information

mass mediatransmission mediastorage mediainteraction media

the term “multimedia” may have quite different, even opposing, viewpoints.

Multimedia is the field concerned with the computer controlledintegration of text, graphics, images and videos, animation, audio,and any other media.

Every type of media information can be represented, stored,transmitted and processed digitally.

What is Multimedia?

A Multimedia Application is an application which uses acollection of multiple media sources e.g. text, graphics, images,sound/audio, animation and/or video.

multimedia content is recorded and played, displayed,or accessed by digital information content processingdevices

This leads to a wide variety of research topics:

• Multimedia processing and coding• Multimedia system support and networking• Multimedia tools and applications

What is Multimedia?

What is Multimedia?

Multimedia research is also highly interdisciplinary

Multimedia Computing

ComputerNetwork

& OS

ComputerVision &Pattern

Recognition

ComputerGraphics

Human ComputerInteraction

Image, Audio, speech

Processing

Multimedia History

1826 : sliding wooden box camera, first natural image (Joseph Nicéphore)1839 : Commercial camera with wet silver-surfaced copper plates1870s: Commercial camera with dry plates1877: phonograph, record and reproduce sound (Thomas Alva Edison)1887: motion picture camera (Thomas Alva Edison) 1895: conducted the first wireless radio transmission (Guglielmo Marconi )1884: first electromechanical television system (Paul Gottlieb Nipkow)1907: cathode ray tube (CRT)

1967: Architecture Machine Group at MIT1969: an early hypertext editor called FRESS1982: The Compact Disc (CD) by Philips and Sony1991: MPEG-1 1991: personal digital assistant (PDA)1992: JPEG1995: JAVA language1996: digital versatile disc (DVD)1998: Handheld Mp3 audio players

An

alogu

eD

igital

HyperText

Memex (1945): personalized memory included associative linksXanadu project (1960): Hypertext

Hypertext is a text which contains links to other texts andTraversal through pages of hypertext is therefore usually non-linear.

HyperTextNormalText

HyperMedia

HyperMedia is not constrained to be text-based and can includeother media, e.g., graphics, images, and especially continuousmedia (sound and video)

Example:The World Wide Web (WWW) Multimedia Authoring (PDF)Computer GamesVirtual realityDigital video editing Multimedia Database systems

1990: Hypertext Markup Language (HTML) and the Hypertext Transfer Protocol (HTTP) for this purpose

Multimedia Systems

Where multimedia used to be … digitally.

A Multimedia System is a system capable of processingmultimedia data and applications and characterized by theprocessing, storage, generation, manipulation and transmission ofMultimedia information.

It has four basic characteristics:• Computer controlled.• Integrated• Data are represented digitally.• The Interactive interface

Represented Stored TransmittedProcessed

Multimedia Systems

Challenges

Digital representation• Analog to digital conversion, sampling

Large data requirements• storage, compression

Distributed networks• bandwidth, QoS

Maintaining temporal relationship between data• Sequencing (playing frames in correct order/time frame in video)• Synchronization (inter-media scheduling, e.g. Lip synchronization)

Multimedia Systems

Major uses:• Education and training• Leisure and entertainment• Information provision• Virtual reality and simulations

Benefits of Multimedia

Your Senses Are Your Raw Information Learning Portals

83.0% – Sight (of which 20 % is remembered)11.0% – Hearing (of which 30 % is remembered)03.5% – Smell01.5% – Touch01.0% – Taste

where 50 % of what is both seen and heard is rememberedfurther 80 % of what is seen, heard and done, is rememberedThat is, multiple, media, and interactive should be a good thing

A Classification of Multimedia

Multimedia Data:• Text - ASCII/Unicode, HTML, Postscript, PDF• Audio – Sound, music, speech, structured audio (e.g. MIDI)• Still Image - Facsimile, photo, scanned image• Video (Moving Images) – Movie, a sequence of pictures• Graphics – Computer produced image• Animation – A sequence of graphics images

• Discrete Media (DM, Static): text, image, graphics• Continuous Media (CM, Dynamic): audio, video, animation

• Captured vs Synthesized media• Standalone vs Networked media

• Journals• IEEE Multimedia

• IEEE Transaction on Multimedia

• IEEE Transaction on Image Processing

• IEEE Transaction on Pattern Recognition and Machine Intelligence

• Conferences• ACM Multimedia

• International Conference on Multimedia and Expo (ICME)

• IEEE Computer Vision and Pattern Recognition (CVPR)

• International Conference on Computer Vision (ICCV)

User Resource