REQUIREMENTS FORMULTIMEDIA PROCESSING
MULTIMEDIA SYSTEMS
IREK DEFEE
• WHAT ARE THE RQUIREMENTS FOR
HANDLING MULTIMEDIA DATA ?
1. PERCEPTUAL REQUIREMENTS
For the user we would like to provide
maximal perceptual stimulation
THIS REQUIRES:
- OPTIMAL STIMULATION OF EACH
SENSE
- OPTIMAL COORDINATION OF ALL
SENSES - INVOLVEMENT OF POSSIBLE MANY SENSES (ACTIVE) THIS MEANS INVOLVEMENT OF: - ACOUSTICAL (HEARING,VOICE) - VISUAL - HAPTIC (TOUCH, MOTORIC, VESTIBULAR (orientation in space) ) ON FULL SCALE (e.g. SPATIAL PROPERTIES)
THE PROBLEM IS HOW THIS CAN BE
DONE TECHNICALLY:
- ACOUSTICAL STIMULATION
THIS IS NOW RELATIVELY EASY
(SPATIAL (MULTICHANNEL SOUND)
- VISUAL STIMULATION
SPATIAL (3-D) IMAGES ARE
PROBLEM, HIGH RESOLUTION TOO
HAPTIC STIMULATION
THERE ARE MANY INPUT DEVICES
(JOYSTICKS), BUT FULL STIMULATION INCLUDING MOTOR
SENSE, SKIN SENSORS AND
VESTIBULAR SYSTEM IS DIFFICULT
IN FACT WE CAN TALK ABOUT
TWO TYPES OF SYSTEMS FOR
STIMULATION:
- CLOSELY COUPLED TO SENSORY SYSTEM (BY GLOVES, UNIFORMS, EYE DISPLAYS, HEADPHONES, MOVING SIMULATORS) THEY ALLOW ”FULL IMMERSION”, OR CREATION OF VIRTUAL REALITY MIGHT BE TIRING TO USE - LOOSELY COUPLED TO SENSORY
SYSTEM (NORMAL SCREENS, SPEAKERS, JOYSTICKS ETC.)
MIGHT BE MORE COMFORTABLE
IN THIS COURSE WE ARE TALKING
ABOUT ’LOOSELY’ COUPLED
MULTIMEDIA SYSTEMS WITH
STANDARD SENSORY INTERFACES
VIRTUAL REALITY IS A TOPIC OF
OTHER COURSES
WHAT ARE THE TECHNICAL
REQUIREMENTS FOR MULTIMEDIA
SYSTEMS ?
1. HANDLING OF LARGE DATA SETS
E.G. AUDIO
5 CHANNELS X 16-BIT X 44 000 SAMPLES/SEC = 3 520 000 BITS/SEC
~450 KB/sec = 27 MB/min=1,6GB/h
SECOND EXAMPLE, VIDEO
RESOLUTION OF STANDARD TV
PICTURE IS 720X576 POINTS, EACH
POINT IS 24-BIT (8-BIT R, G, B)
THERE ARE 25 PICTURES PER SECOND
SO THE DATA RATE IS
720X576X25X24~ 250 Mb/sec = 1.8GB/min
= 108 GB/h
FOR HIGHER RESOLUTIONS:
A FORMAT FOR HIGH-DEFINITION TV
IS PROPOSED
PICTURE SIZE IS 1920X1080 POINTS
IT IS ABOUT 5X TIMES LARGER THAN
STANDARD TV. DATA RATES ARE
THEN, 1.3 Gb/sec, 9 GB/min, 540 GB/h
DATA RATES FOR AUDIO AND VIDEO ARE NOT PRACTICAL,
COMPRESSION IS REQUIRED
- COORDINATION AND SYNCHRONIZATION
- HUMAN SENSORY SYSTEM IS VERY SENSITIVE TO COORDINATION OF EVENTS IN TIME AND SPACE
MULTIMEDIA DATA, ESPECIALLYAUDIO AND VIDEO DEPEND CRITICALLY ON TIME AND SPACE
• EXAMPLES:
- LIP SYNCHRONIZATION IN TV AND
CINEMA, VOICE AND LIP MOVEMENT
MUST BE SYNCHRONIZED, OTHERWISE IT IS VERY ANNOYING
SYNCHRONIZATION ERROR FOR BIG
PICTURES MUST BE IN THE LIMIT OF +20 ; –40 MILISECONDS
- SPATIAL SOUND
HEARING SYSTEM ESTABLISHES
LOCATION OF SOUND SOURCES
BASED ON DIFFERENCE OF TIME OF
ARRIVAL AT EARS
PRECISION REQUIRED IS ON THE ORDER OF SEVERAL 10’s OF MICROSECONDS.
• THIRD REQUIREMENT: FAST RESPONSE
WHENEVER WE RESPOND (E.G. BY
CLICKING), WE EXPECT IMMEDIATE
RESPONSE.
IMMEDIATE MEANS ”NO NOTICEABLE DELAY” FOR PEOPLE.
IT MEANS LESS THAN 300 ms=0.3 sec
• THUS, MULTIMEDIA DATA REQUIRE
HIGH-DATA RATE
& TIME/SPACE CRITICAL PROCESSING
WITH
FAST RESPONSE TO EXTERNAL
EVENTS
THESE ARE ATTRIBUTES OF
REAL-TIME PROCESSING
• DEMANDS FOR COMPUTATIONAL POWER ARE HIGH
• ONLY RECENTLY THIS BECAME POSSIBLE
Multimedia systems requirements
CONTENT DELIVERY
DISTRIBUTION
CONSUMER
Recall the multimedia system is for service offering:
What users expect from a service they use?- They expect satisfying Quality of Service – QoS QoS includes technical aspects which are most
important for engineers (e.g. good communication link)
QoS in broad sense includes all other aspects which make customer satisfied (e.g. support, price)
What is QoS ?
• QoS is degree to which customer expectations are satisifed
• QoS includes technical parameters for
quality: e.g. reliability of communication
• QoS may also include other parameters
for satisfaction (attraction, support, price, etc.)
• What service owners expect? They expect proper environment that they can run their service Especially that their buisness can run without problems Unfortunately for network buisnesses there are big problems with this What is absolutely necessary is SERVICE PROTECTION
Good buiness over the network needs as the most critical requirement
STRONG SERVICE PROTECTION
CONTENT DELIVERY
DISTRIBUTION
CONSUMER
Every part of the system has to be strongly protected againstthose who want to destroy the service integrity
Overall architecture of multimedia system
There are two ”layers”: Hardware layer: servers, networks and terminalsSoftware layer: content, service, provisioning, networking, middleware, applications
Multimedia system
• Composed of hardware/software components• Goal: Creation and delivery of media content from
production to consumer at proper QoS in an economic way
This means business is sustainable and consumers are satisfied
Examples of multimedia systems
Content delivery system examples:
• Television/(radio?) (old, basic system)
• Web/Internet type
• Mobile (radio networks)
Content distribution type examples:
• Music
• Movies (audio/visual content
• Games
• Virtual worlds (Check Second life on the Web)
• eBooks? eMagazines? eNewspapers?