MPEG 4: The ultimate low bit rate formatwww.chiariglione.org/mpeg/

Overview: ISO/IEC 14496 Coding of audio-visual objects Low bit rate multimedia system - typically less than MPEG 2 videoObject based - each element is coded separatelyOpen ended system that can continue to develop in the futureScalable & InteractiveVersion 1: October 1998

VersionsVersion 1

ProfilesQualityComplexityDVDVideo CDMobilesHDTVDigital cinema

Current uses:

3G mobile phones,Portable devices, PDAs, video iPodInteractive television / IPTVNew interactive multimedia formatsWeb pagesInteractive music formatSecurity systems

Basics:Object based system: using Natural and/or Synthetic objectsMakes use of local processing power to recreate sounds and imagesThis makes it one of the most efficient compression systems

Basics: Object TypesPhotos - JPE, GIF, PNG,Video - MPEG-2, Divx, AVI, H.264, QuickTimeSpeech - CELP, HVXC, Text to SpeechMusic - AAC, MP3, surroundSynthetic musicGraphics - Java codeTextAnimated objects, e.g., talking heads

Basics:The selected objects are put together into a 2D or 3D scene In 3D the viewer can change the shape of the image and view it from other positions in the 3D space. Similar to VRMLEach object is compressed using the best and most optimum method for that type of data

Basics: Virtual StudioVirtual production techniques are increasingly used in TV production Well known chroma-key method using a blue/green screen backgroundActors are overlaid on to a virtual studio background imageThe composition of the screen image and the sound can now take place in the decoder at home

BIFS:Binary Format for Scene descriptorNew description language like HTML but written in binary not in English Has scalable levels for audio and video which can be set by access rights or by interrogation of the receiver to set the best Quality of Service (QoS)

Synchronised StreamingEach element can be time stamped to synchronise with other objects in the frame Flexi Time: The viewer can vary the time for playback There are 3 types, set by the producer, minimum, maximum and optimalAudio can be set to change pitch or stay fixed

Compression: SpeechHVXC - Harmonic Vector Excitation CodingCELP - Code Excited Linear Prediction2 - 24 K bit/secSynthesised speech: Text to speech synthesis, 200-1200 bit/secVery low delay, 20 ms, for video phone use MP3 takes too long to encode/decode

Compression: Natural AudioMPEG - AAC (Advanced Audio Coding)MP3, AAC, 5.1 surround6 - 380 K bit/sec

Parametric audio codingSynthesised audio: Spectral noise re-synthesised This process separates unique audio sounds from predictable noise shapes which can then be re-synthesised locally Signal is represented by three objects:Transients: localized in time,Sinusoids: localized in frequency,Noise: no strict localization

Parametric audio codingTransients: Castanets

Parametric audio codingSinusoids: Harpsichord

Parametric audio codingNoise: Heavy metal

Compression: Structured Audio SAOL - Structured Audio Orchestra Language (pronounced sail)Down loadable sound fontsWavetable synth + GM2 type spec.Any kind of virtual instrumentsVirtual effects algorithms and mixersMIDI data rates e.g. 300 bit/sec

Interactive Audio Download and remix tracksFlash interface and compressed audio loopswww.yourspins.com

Compression: VideoOverall MPEG-4 supports a wide range of standards from very small, poor quality, pictures up to HDTV MPEG 1 & 2MPEG 4 - part 2MPEG 4 - part 10, 'Advanced Video Coding AVC H.264 - high quality video codecDeveloped jointly by MPEG and the ITU

Compression: Video - H.264Half to one quarter of the normal bit rate of MPEG-2Scalable from 3G to HDMore advanced B frame operation where the frame can link to any frame in the video sequenceSmaller 4X4 grids of pixels4 Profiles and 16 LevelsBit rates from 64 Kbps - 240 Mbps

Compression: VideoMixture of pixel based and vector graphicsVideo is no longer a rectangular shape, it can be any shapeSynthetic images with bit rates from 5 kbit/sec - 10 Mbit/secSupports the mapping of video textures onto moving objects and meshes

2D mesh model of a fishBy deforming the mesh the fish can be animated

Animated Objects:The Animation Framework eXtension, AFXFace animation: The face models are not part of MPEG-4 only the movement codes for the expressions, eye movement, etc.

Body animation works in the same way and can be used in games

Future Options:MPEG-4 is still being developed and all new parts will work with the old formatsStudio quality versions for HDTVDigital cinema 45 - 240 Mbit/sec H.264Home video cameras with MPEG-4 output straight to the web from the hard drive

Future Options:Integrated Service Digital Broadcast (ISDB)Newspaper + TV + dataIntegration with MPEG 7 databasesGames with 3D texture mapping

Future Options:TeleVision Modelling Language (TVML)Computer generated TV programs + presenters - Max Headroom??

Future Options:Information boothsTalking objects - fridge, cars, toaster?Security cameras over the webInteractive manuals and training materialsNew downloadable interactive music format, SAOL

MPEG 7

MPEG 7Multimedia Content Descriptor StandardDatabase system to automatically define, organise and search for text, pictures, sound FX, graphics, video clips, songs, music, etc.On-line Music libraryAutomatic identification of musicUses XML to store metadata

MPEG 7Proposed uses:Live broadcast monitoring, radio outputDigital libraries e.g., image catalogue, musical dictionary, bio-medical imaging, sound FX, film, video and radio archivesCultural services history museums, art galleries, etc.

MPEG 7Home entertainment e.g., systems for the management of personal multimedia collections, e.g. music, home video, searching a game, karaokeE-Commerce e.g., personalised advertising, on-line catalogues, directories of e-shopsEducation e.g., repositories of multimedia courses, multimedia search for support material

MPEG 7Investigation services e.g., human characteristics recognition, forensicsJournalism e.g. searching speeches of a certain politician using their name, voice or faceMultimedia directory services e.g. Yellow Pages, tourist information, geographical information systems

MPEG 7Multimedia editing e.g., personalised electronic news service, media authoringSocial e.g. on-line dating servicesSurveillance e.g., traffic control http://www.eptascape.com/products/demoflv.htm

MPEG 21

MPEG 21An infrastructure for the delivery and consumption of multimedia contentUsers are seen as either creators, consumers, rights holders, content providers, or distributors

MPEG 21Every media element is defined as a Digital ItemMetadata defines what media we can use, what we can do with it and who owns itDesigned to work with MPEG 4 files and MPEG 7 database