IEE 5037 Multimedia CommunicationsLecture 15: MPEG System and Digital Right Management: from MPEG-2 to
MPEG-7 and MPEG-21
Dept. Electronics Engineering,N
ational Chiao T
ung University Adapted from Prof. Hang’s slides and many other’s slides
1
Institute of Electronics,National C
hiao Tung U
niversity
Overview: Beyond the Compression
• System level problems– How to packet and composite these A/V contents?
• Buffer management• A/V synchronization• Program information• A/V composition• Defined in MPEG-2 system and MPEG-4 system
– How to management these A/V contents• Content management: search by metadata descriptions
– MPEG-7• Digit right management: framework for delivery and
consumption– MPEG-21
2
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-2 System Standard
• MPEG Standards: System, Video, and Audio• MPEG-2 system applications -- Broadcast digital
TV, Cable digital TV, Interactive TV, etc.• MPEG-2 System specifications:
– Provide timing information for buffer management, audio/video synchronization, ...
– Multiplex audio, video, data and control bit streams
– Support access control (security), program specific information, digital storage control,
3
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-2 System Model
Pack /
Mux
Videoencoding Buffer
Audioencoding Buffer
VideodecodingBuffer
ChannelUnpack /
DemuxAudio
decodingBuffer
Video
in
Audio
in
Buffer Buffer
Video
out
Audio
out
Data
outData
in
Elementary Stream (ES): One type of data, e.g., coded video, coded audio, etc.Packetized Elementary Stream (PES): With stream_id and PES header --- different headers for different contents (e.g., A/V: PES_priority, DSM_trick_play, PTS …)
4
Institute of Electronics,National C
hiao Tung U
niversity
5
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-4 Systems Goals
• AVO (Media Object) -- object-based representation of audiovisual content.
• Describe the composition of the AVOs to create compound objects that form a Scene.
• Multiplex and synchronize AVO streams for transportation over network with QoS.
• Interact with the Scene generated at the receiver end.
6
Institute of Electronics,National C
hiao Tung U
niversity
Architecture of MPEG-4
MPEG-4 Systems
DMIF
Visual AudioOther
Visual / AudioStandards
7
Institute of Electronics,National C
hiao Tung U
niversity
Architecture of MPEG-4
DAI
ESI
Delivery aware, media unaware :14496-6 DMIF14496-1 FlexMux
Media aware, delivery unaware :14496-2/3 Audio and Visual14496-1 Scene Description and Object Descriptor Protocol
Media and Delivery unaware :14496-1 Systems, adaptation of the MPEG-2 traditional activities
DeliveryLayer
FlexMuxTool
Sync.Layer
CompressionLayer
Composition
8
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-4 DMIF
• DMIF = “Delivery Multimedia Integration Framework”
D M I FThe multimedia content delivery integration framework
Cable,Satellite,
etc.
Broadcast Technology
Internet, ATM,
etc.
InteractiveNetwork Technology
CD,DVD,etc.
Storage Technology
9
Institute of Electronics,National C
hiao Tung U
niversity
Terminologies
• Access Unit– -- the smallest data entities, e.g., video frame
• Elementary Stream– -- composed by access units
• SL-Packetized Stream (SPS)– -- Synchronization Layer Packetized Stream
• FlexMux Stream– -- composed by one or more SPS
10
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-4 DMIF (cont.)
• DAI = DMIF Application Interface• DNI = DMIF Network Interface
SL SL SL
TransMux Layer
FlexMux
TransMux Streams
FlexMux Channel
TransMux Channel FlexMux Streams DMIF Network Interface
DMIF Application Interface
Elementary Stream Interface
SL-Packetized Streams
Elementary Streams
FlexMux
Sync Layer
DMIF Layer
SL SLSL
FlexMux
SL
(RTP)UDP
IP
(PES)MPEG2
TS
AAL2ATM
H223PSTN
....
....
....DABMux
File Broad-cast
Inter-active (not specified in MPEG-4)
Del
iver
y La
yer
11
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-4 Terminal
End user PresentationEnd user Presentation
Composition & Rendering
Composition & Rendering
Hierarchical, Interactive, Audiovisual Scene
Network
Layer
Network
Layer
DecompressionDecompression
Primitive AV Objects
...
DemultiplexDemultiplex
...
Elementary Stream
Scene Description
Scene Description
Upstream DataUser Events, Class Request, ...
12
Institute of Electronics,National C
hiao Tung U
niversity
Scene Description -- BIFS
• Coded scene description in a compact binary form Called “Binary Format for Scene -- BIFS”
• BIFS-Command – -- To modify a set of scene properties at a given time
instance.• BIFS-Animation
– -- To continuously update certain fields in the scene graph.
13
Institute of Electronics,National C
hiao Tung U
niversity
Application Example
-- Ping-pong Game• Green Bar and Blue Bar controlled
by Player1 & Player2 with keyboard• We can change the speed of the
ball (square)• The ball contains an H.263 video
sequence• As ball moves, we sendEventIn to
update the position in BIFS
14
Institute of Electronics,National C
hiao Tung U
niversity
15
Institute of Electronics,National C
hiao Tung U
niversity
Motivation of MPEG-7 and MPEG-21
16
Institute of Electronics,National C
hiao Tung U
niversity
MPEG7
• MPEG7 為因應日漸繁多的多媒體資訊,期望能夠快速且有效的檢索出用戶所需的不同類型的多媒體資料。
• 於1998年10月提出,於2001年公佈。• MPEG7 可針對各種不同類型的多媒體資訊進行標準化的描述,並將該描述與所描述的內容相聯繫,以實現快速有效的檢索。
17
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-7 (ISO/IEC 15938)
• Content Management– Increasing availability of multimedia content– Finding, selecting, filtering desired material difficult– Desire for information about the content
• Objective– Standardized content-based descriptions for multiple types of
audio-visual information– Enable rapid and efficient content identification– Suitability for large range of applications
• MPEG-1, -2 and -4 represent the content “the bits”
• MPEG-7 represents information about the content“the bits about the bits”
18
Institute of Electronics,National C
hiao Tung U
niversity
Multimedia Semantic Framework
19
Institute of Electronics,National C
hiao Tung U
niversity
Multimedia Semantic Framework
20
Institute of Electronics,National C
hiao Tung U
niversity
Towards Knowledge Management andTransaction Enrichment for Digital Media
21
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-7 OverviewXML Metadata for Multimedia Content Description
22
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-7 Standard Scope
23
Institute of Electronics,National C
hiao Tung U
niversity
Annotation Example
24
Institute of Electronics,National C
hiao Tung U
niversity
Example: Semantic Concept Detection
25
Institute of Electronics,National C
hiao Tung U
niversity
Example: Personalization and Summarization
26
Institute of Electronics,National C
hiao Tung U
niversity
27
Institute of Electronics,National C
hiao Tung U
niversity
MPEG21
• 多媒體版權的管理• 定義各種可供傳播與交易的數位元件基本單位(Digital Item),並且讓各種使用者(包括製作、傳播與消費者)擁有同樣取得數位元件的權利。MPEG組織希望MPEG 21能夠有效釐清數位多媒體專利權的歸屬議題,未來的使用者也將有機會在任何時間、地點、設備上(Any time,any place,any device)取得數位多媒體資訊。
28
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-21 (ISO/IEC 21000)
• What?– Multimedia Framework for multimedia delivery and consumption– Content creator and content consumer as focal points
• Why?– Many elements (standards) exist for delivery and consumption of
multimedia contents– Absence of 'big picture” to describe how elements relate to each
other– Increase interoperability to allow existing components to be used
together by filling gaps• Why now?
– HW building blocks and infrastructure in place– Compression, transmission, description standards are ready
29
Institute of Electronics,National C
hiao Tung U
niversity
30
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-21 Standards
31
Institute of Electronics,National C
hiao Tung U
niversity
32
Institute of Electronics,National C
hiao Tung U
niversity
AuthorAuthorAgentAgent
PublisherPublisherAggregatorAggregator
DistributorDistributorWholesalerWholesaler eTailer ConsumerConsumer
Authoring Tools Publishing S/WAsset ManagementContent PackagingContent Hosting
WWW. Store FrontRights Clearing User Interface
Digital Rights
Meta Data
Digital Content Products & ServicesDigital Content Products & Services
Digital IdentificationDigital Identification
Digital Supply Chain
33
Institute of Electronics,National C
hiao Tung U
niversity
34
Institute of Electronics,National C
hiao Tung U
niversity
Multiple Flows and Transactions
• Between any two points there are flows:• Information about content• Authentication between peer entities• Content• Content usage rights• Technology usage rights• Delivery usage rights• Money
35
Institute of Electronics,National C
hiao Tung U
niversity
36
Institute of Electronics,National C
hiao Tung U
niversity
Current Practice
• Today’s media contain implicit or explicit rules– A book can be read and resold– A CD may be played, but may not be copied– A video cassette is for private consumption, can be
rented but may not be copied– A public broadcast can be watched because license fee
has been paid– A commercial broadcast can be watched because one
undertakes to watch commercials– A pay TV broadcast can be watched because one has
paid a monthly subscription
37
Institute of Electronics,National C
hiao Tung U
niversity
Future Practice?
• The Multimedia Framework gives unlimited flexibility:
• One can buy an MPEG-21 Digital Item and:– Copy it to a portable device once only– Rent it for 24 hours– Rent it for 10 playbacks– Distribute it to 10 friends and get a copy for free– Access it for free at a low quality, and for a fee at high
quality– ??????
38
Institute of Electronics,National C
hiao Tung U
niversity
MPEG-21 Objectives
• Vision– To define a multimedia framework to enable transparent use of
multimedia resources across a wide range of networks and devices used by different communities
• Purpose– Enable electronic creation, delivery, trade of digital multimedia
content• Goals
– Provide access to information and services from almost anywhere at anytime with ubiquitous terminals and networks
– Identify, describe, manage, and protect multimedia content to support delivery chain of content creation, production, delivery, and consumption
39
Institute of Electronics,National C
hiao Tung U
niversity
Fundamental Concept: The Digital Item
• A structured digital object with a standard representation, identification and meta-data
• The fundamental unit of distribution and transaction in the MPEG-21 framework
• Digital Item = resource + metadata + structure• Resource: individual asset, e.g., MPEG-2 video• Metadata: descriptive information, e.g., MPEG-7• Structure: relationships among parts of the item
40
Institute of Electronics,National C
hiao Tung U
niversity
Resources
MPEG-1
MPEG-2
MPEG-4
MPEG-7
Metadata New Metadata& Resource
Forms
Structure
MPEG-21
Digital Item
41
Institute of Electronics,National C
hiao Tung U
niversity
Benefits of Digital Item
• Tangibility– content is more than files on a disk
• Deliverability– more automated, less end-user involvement
• Configurability– express options/augmentations for specific users,
groups, locales, prices, etc.
42
Institute of Electronics,National C
hiao Tung U
niversity
ISO/IEC 21000-1Technical Report (TR)
• Published Sept 2001• Outlines objectives and vision of 7 elements for
the MPEG-21 – Digital Item Declaration– Digital Item Representation– Digital Item Identification and Description– Digital Item Management and Usage– Intellectual Property Management and Protection– Terminals and Networks– Event Reporting
43
Institute of Electronics,National C
hiao Tung U
niversity
Digital Item Declaration ISO/IEC 21000-2
• Reached FDIS in May 2002• Purpose & Conceptual Model
– Declare a Digital Item’s makeup and structure– Composite whole composed of various parts
• Syntactic Representation – XML-based Digital Item Declaration Language (DIDL)
• Features– Hierarchical, generic structure– Flexible meta-data expression– Reusable and configurable elements
FDIS Final Draft Information Standard
44
Institute of Electronics,National C
hiao Tung U
niversity
Digital Item Identification ISO/IEC 21000-3
• Provides means to uniquely identify and describe Digital Items
• Defines the relationship between Digital Items and existing ID systems – This part does not specify any new ID systems
• Identification– syntax: URI of the form
urn:mpeg:mpeg21:diid:sss:nnn• sss denotes the identification system • nnn denotes a unique identifier within that identification system
FCD – Final Committee Document
45
Institute of Electronics,National C
hiao Tung U
niversity
IP Management & ProtectionISO/IEC 21000-4
• Multimedia Digital Rights Management (DRM) – Enables users to express their rights, interest, and
agreements related to Digital Items (DI’s)– Enables users to derive appropriate levels of assurance
that those rights, interests and agreements will be persistently and reliably managed and protectedacross a wide range of networks and devices
46
Institute of Electronics,National C
hiao Tung U
niversity
Intellectual Property Management & Protection (IPMP)
• Management and protection – MPEG-4 audio-visual objects– MPEG-7 descriptors and description schemes – Other DI types including personal data and rights to its
use• Three parts to the standard underway:
– IPMP Framework– Rights Expression Language (REL)– Rights Data Dictionary (RDD)
47
Institute of Electronics,National C
hiao Tung U
niversity
Rights Expression Language ISO/IEC 21000-5
• Targets for Standardization– Machine-readable language
• Declare rights and permissions using the terms as defined in the Rights Data Dictionary
– RDD-REL are intended to provide flexible, interoperablemechanisms to support transparent and augmented use of digital resources
– REL supports guaranteed end-to-end interoperability, consistencyand reliability between different systems and services
CD Committee Draft
48
Institute of Electronics,National C
hiao Tung U
niversity
Rights Data Dictionary ISO/IEC 21000-6
• Targets for Standardization– Model– Core set of terms– The relation between the RDD and the core set of
descriptive terms in the REL
49
Institute of Electronics,National C
hiao Tung U
niversity
Digital Item Adaptation ISO/IEC 21000-7• Targets for standardization
– Usage Environment Description Tools– Resource Adaptability Tools– Systems Related Tools
50
Institute of Electronics,National C
hiao Tung U
niversity
51
Institute of Electronics,National C
hiao Tung U
niversity
52
Institute of Electronics,National C
hiao Tung U
niversity
53
Institute of Electronics,National C
hiao Tung U
niversity
54
Institute of Electronics,National C
hiao Tung U
niversity
55
Institute of Electronics,National C
hiao Tung U
niversity