+ All Categories
Home > Documents > MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS NETWORKING

Date post: 13-Jan-2016
Category:
Upload: brice
View: 53 times
Download: 3 times
Share this document with a friend
Description:
MULTIMEDIA SYSTEMS NETWORKING. SOFTWARE. SERVER. NETWORK. CLIENT. MULTIMEDIA SYSTEM. WE CONSIDER MULTIMEDIA SYSTEMS BUILT-UP ON LARGE SCALE: REACHING PRACTICALLY EVERYBODY, LIKE TV, RADIO AND TELEPHONE TODAY, THAT IS THOUSANDS AND MILIONS OF USERS MULTIMEDIA MEANS BROADBAND - PowerPoint PPT Presentation
Popular Tags:
150
MULTIMEDIA SYSTEMS IREK DEFEE MULTIMEDIA SYSTEMS NETWORKING
Transcript
Page 1: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

MULTIMEDIA SYSTEMSNETWORKING

Page 2: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SERVER CLIENTNETWORK

SOFTWARE

MULTIMEDIA SYSTEM

Page 3: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• WE CONSIDER MULTIMEDIA SYSTEMS BUILT-UP ON LARGE SCALE: REACHING PRACTICALLY

EVERYBODY, LIKE TV, RADIO AND

TELEPHONE TODAY, THAT IS THOUSANDS AND MILIONS OF USERS

• MULTIMEDIA MEANS BROADBAND

STREAMING AND INTERACTIVITY

Page 4: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

MAIN PROBLEMS IN NETWORKING • NETWORKS FOR MULTIMEDIA REQUIRE:- SUFFICIENT/CONTROLLED BANDWIDTH - RESERVATION OF NETWORK

RESOURCES (WE CAN NOT LOSE DATA)- STREAMING - FAST INTERACTIVITY DELIVERY CAN BE WIRED OR WIRELESS:

WIRED - USES PHYSICAL LINKS

WIRELESS – USES RADIO

Page 5: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• A MODEL OF INTERACTIVE SYSTEM IS INTERNET

• A MODEL OF BROADBAND STREAMING ARE TV, RADIO

• MULTIMEDIA SYSTEMS WOULD

BE SOMETHING LIKE INTEGRATION

OF THEM BOTH

• SOMETHING LIKE BECAUSE WE CAN NOT PREDICT IN DETAIL WHAT IT MIGHT BE

Page 6: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• COMMUNICATION SERVICE TYPES- BROADCAST: SINGLE SOURCE, MANY USERS – LIKE TV, 1-WAY DISTRIBUTION- MULTICAST: LIKE BROADCAST BUT USERS SIGN TO A SESSION, ONE

WAY CONNECTION- RETRIEVAL, UNICAST –

ASYMMETRIC CONNECTION, USERS RECEIVE MUCH MORE THAN SEND, E.G. WWW

Page 7: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• COMMUNICATIVE SERVICE –

SYMMETRIC 2-WAY CONNECTION

LIKE TELEPHONE

IN MULTIMEDIA SYSTEMS WE ARE MOSTLY INTERESTED IN THE RETRIEVAL AND COMMUNICATIVE SERVICES WITH HIGH BIT RATE

Page 8: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• STREAMING

STREAMING MEANS DATA ARE SEND

SYNCHRONOUSLY IN TIME AND WILL

REACH USER PRESERVING THEIR TIME

ORGANIZATION. NO DATA LOSS OR

DELAYS ARE ALLOWED.

EXAMPLE: TELEPHONE NETWORK

IN MULTIMEDIA STREAMING IS

ABSOLUTELY NECESSARY

Page 9: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• THIS IS VERY MUCH RELATED TO THE PROBLEM OF NETWORK RESOURCE MANAGEMENT AND

CONTROL• THERE ARE TWO TYPES OF NETWORKS- CONNECTION-ORIENTED- CONNECTIONLESS- IN CONNECTION-ORIENTED

NETWORK RESOURCES ARE ALLOCATED BEFORE

Page 10: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

THE DATA TRANSFER IS STARTED.

IN CONNECTIONLESS NETWORK

RESERVATION IS NOT DONE: DATA

MAY BE TRANSFERRED WITHOUT ANY

GUARANTEE OF RESOURCE

ALLOCATION, OR WITH SOME

GUARANTEE ONLY.

Page 11: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• HOW STREAMING CAN BE ENSURED IN NETWORKING?

THERE ARE THREE BASIC TYPES OF NETWORKING

- CIRCUIT SWITCHING - CONNECTION IS ’SWITCHED’ BEFORE SENDING DATA (”PHONE”)

- PACKET SWITCHING – DATA ARE

ORGANIZED IN PACKETS. EACH PACKET CARRIES ADDRESSES

Page 12: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

- CELL SWITCHING – BETWEEN CIRCUIT AND PACKET SWITCHING

”PACKETS ARE SWITCHED”

(TECHNOLOGY CALLED ATM –

ASYNCHRONOUS TRANSFER MODE)

ALL THESE SYSTEMS HAVE VARIOUS

POSIIVE AND NEGATIVE ASPECTS FOR MULTIMEDIA

Page 13: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

IN CIRCUIT SWITCHING NETWORK RESOURCES ARE FULLY RESERVED FOR STREAM TRANSMISSION

IN PACKET SWITCHING PACKETS ARE

FLOWING IN LITTLE CONTROLLED WAY

MAKING STREAMING PROBLEMATIC

IN CIRCUIT SWITCHING INTERACTIVITY IS SLOW (TIME FOR CONNECTION NEEDED)

IN PACKET SWITCHING INTERACTIVITY IS

FAST

Page 14: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• EXAMPLES:- TELEPHONE SYSTEM – SWITCHING

(connection establishment) IS DONE IN EXCHANGES, SWITCHBOARDS OR SWITCHING CENTERS

- INTERNET - PACKETS ARE DIRECTED IN ROUTERS WHICH READ THEIR

ADDRESSES AND DIRECT THEMThe difference is that only circuit switchingfully guarantees streaming but it is expensive

Page 15: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• ADVANTAGES OF CIRCUIT SWITCHING:

- GUARANTEED DATA DELIVERY

- STREAMING AND TIMING OF DATA

NO PROBLEM

- DISADAVANTAGES:

- CIRCUIT ESTABLISHMENT (”call”)

TAKES TIME

- EXPENSIVE INFRASTRUCTURE

Page 16: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• PACKET SWITCHING

EXAMPLE: INTERNET

DATA ARE ORGANIZED IN PACKETS

PACKETS CARRY HEADERS WITH

SOURCE AND DESTINATION ADDRESS

PACKETS ARE ROUTED IN THE

NETWORK BASING ON THE ADDRESSESS

Page 17: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• ADVANTAGES OF PACKET SWITCHING

- SIMPLICITY, EVEN A PC CAN BE ROUTER

- PACKETS CAN BE ROUTED IMMEDIATELY

- PACKETS CAN BE SEND USING DIFFERENT ROUTES, OR STORED

EASY ADAPTATION TO THE AMOUNT OF TRAFFIC

- IDEAL FOR TIME NON-CRITICAL DATA

Page 18: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• DISADVANTAGES:

- NO GUARANTEE FOR

PACKET DELIVERY

- NO GUARANTEE FOR RESOURCES

NO GUARANTEE FOR PACKET DELIVER ON-TIME, NO STREAMING

Page 19: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• EXAMPLE – WWW. WE GET INSTANTRESPONSE FROM THE PS INTERNETIF THE INTERNET WOULD BE CS, WEWOULD NEED TO WAIT FORCONNECTION ESTABLISHMENT.BUT THERE IS NO RESOURCERESERVATION FOR STREAMS IN PSA SOLUTION FOR THIS PROBLEMCOULD BE INTELLIGENT PACKETSWITCHING AND NETWORKING

Page 20: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• IN INTELLIGENT PS PACKETS HAVE

PRIORITIES ASSIGNED, MULTIMEDIA

PACKETS GET HIGH PRIORITY SO

THEY WILL NOT BE DISTURBED BY

OTHER PACKETS

IF THE NETWORK WOULD HAVE ENOUGH CAPACITY, MULTIMEDIA PACKETS WILL FLOW IN STREAMS

(if there is not enough capacity, no help)

Page 21: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• THUS PACKET SWITCHING CAN BE IN PRINCIPLE BETTER BECAUSE OF STREAMING AND FAST INTERACTIVITY

• CURRENT NETWORKING IS THUS EVOLVING IN THIS DIRECTION

(INTERNET IS BECOMING VERY

POWERFUL)

Page 22: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

CELL SWITCHING OVERVIEW • THIS TECHNOLOGY IS REALIZED IN

ONE PARTICULAR FORM CALLED • ATM – ASYNCHRONOUS TRANSFER• MODEIT IS A COMBINATION OF CIRCUITSWITCHING AND PACKET SWITCHINGPACKETS HAVE CONSTANT, VERYSHORT LENGTH – 48DATA+5HEADER=53BYTES/PACKET

Page 23: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• ATM ENABLES COMBINATION OF

CIRCUIT AND PACKET SWITCHING

• BASIC CONCEPTS ARE VIRTUAL

CHANNEL AND VIRTUAL PATH

H PAYL. H PAYL. H PAYL. H

FLOW OF ATM CELLS H – HEADER 5 BYTESPAYLOAD 48 BYTES

Page 24: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• HEADERS ESTABLISH WHERE THE

PACKETS ARE TRANSMITTED

GFC VPI VCI PT CL HEC

STRUCTURE OF ATM CELL HEADER:

GFC - GENERIC FLOW CONTROLVPI - VIRTUAL PATH IDENTIFIERVCI - VIRTUAL CIRCUIT IDENTIFIERPT - PAYLOAD TYPECL - CALL LOSS PRIORITYHEC – HEADER ERROR CHECK

4 8 16 3 1 8 bits

Page 25: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• FOR MULTIMEDIA DATA

ONE VP COULD BE ALLOCATED

AND SEVERAL VC FOR E.G.

VC=1 VIDEO

VP=5 VC=2 AUDIO

VC=3 TEXT

Page 26: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• IN THE ATM NETWORK THERE ARE

SWITCHES WHICH DIRECT PACKETS

ACCORDING TO THEIR VP AND VC

SWITCH 1 SWITCH 2VP=1

VC=5VP=6VC=13

VP=4VC=2

VP AND VC BETWEEN THE SWITCHES (THERE CAN BE MANY OF THEM HAS TO BE ASSIGNED TO ENABLE END-TO-END TRANSMISSION

Page 27: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• HOW THE ASSIGNMENT IS DONE?

IT CAN BE DONE BY HAND

OR IT CAN BE DONE AUTOMATICALLY USING SPECIAL

SYSTEM FOR MAPPING END POINT

NUMBERS TO VP AND VC OF

INTERMEDIATE SWITCHES

Page 28: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• IN ATM THIS IS SOLVED IN A WAY

WHICH IS SIMILAR TO TELEPHONE

NETWORKS

IT IS POSSIBLE TO ESTABLISH

CONNECTION BETWEEN ANY

END-POINTS BECAUSE THEY HAVE

UNIQUE NUMBERS

Page 29: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• THE ATM CALLS CAN SPECIFY

BANDWIDTH AND OTHER PARAMTERS

THERE ARE SEVERAL CLASSES OF

SERVICES

AAL – ATM ADAPTATION LAYER

AAL1- CONSTANT BITRATE WITH

TIMING

Page 30: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• AAL2 – VARIABLE BIT RATE WITH TIMING

• AAL3/4- VARIABLE BIT RATE WITH

NO TIMING

• AAL5 VARIABLE BIT RATE WITH NO TIMING, CONNECTIONLESS

• FOR THESE AAL’S IT IS SPECIFIED HOW DIFFERENT PROTOCOLS ARE

MAPPED ON THEM

Page 31: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• FOR EXAMPLE MPEG-2 TRANSPORT

STREAM IS MAPPED INTO

2 X188 MPEG-2 PACKETS -> 8 ATM

CELLS

AAL2 IS IDEAL FOR MULTIMEDIA

BUT IT IS DIFFICULT TO IMPLEMENT

(TIMING)

LARGE ATM NETWORKS ONLY RARELY ARE IMPLEMENTED

Page 32: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• BUT IN NEW THIRD GENERATION WIRELESS NETWORKS

- WIRELESS LAN

- WIRELESS CELLULAR

ATM AND AAL2 (WIRELESS) IS SPECIFIED AND IMPLEMENTED

THUS, THEY MAY OPERATE IN THE

ATM MODE

Page 33: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

PACKET SWITCHING – IP INTERNET

IN STANDARD PACKET SWITCHING

THERE IS NO NETWORK RESOURCE

RESERVATION BEFORE THE

PACKETS ARE SEND.

THE BASIC METHOD OF

TRANSFERRING PACKETS ON THE

INTERNET IS THE

UDP – USER DATAGRAM PROTOCOL

Page 34: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

IPv4 Header Structure

Ip version Header_length Tos Total length

Identification Flags and fragmentation

TTL Protocol Header check sum

32 bit Source Ip address

32 bit Destination address

Options(if any)

Data

Page 35: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• UDP/IP PACKETS ARE ROUTED ACCORDING TO THEIR ADRESSES,

ONCE SENT, THEY ARE UNCONTROLLED

THIS CAN BE VERY UNRELIABLE

BUT ON RELIABLE AND HIGH

BANDWIDTH LINKS IT CAN BE

SUFFICIENT FOR MULTIMEDIA

Page 36: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• THE TCP/IP PROTOCOL IS USED TOPROVIDE INFORMATION THAT PACKETS REACHED THEIR

DESTINATION, THERE IS CONFIRMATION SEND ABOUT THEIR

RECEPTION• IF THERE IS NO CONFIRMATION,

PACKETS ARE RESEND • TCP/IP IS GOOD FOR NON-TIME

CRITICAL DATA (EMAIL)

Page 37: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

IP NETWORKING FOR

MULTIMEDIA • FOR MULTIMEDIA DATA PACKET

SWITCHING INTERNET CAN BE VERY BAD

• ON THE OTHER HAND INTERNET IS CHEAP, AVAILABLE AND UBIQUITOUS

Page 38: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

FOR SOLVING PROBLEMS WITH MULTIMEDIA DATA OVER IP PACKET

NETWORK, THERE ARE THREE APPROACHES:

- INTRODUCING RESOURCE RESERVATION, PROTOCOL CALLED

RSVP – RESOURCE RESERVATION PROTOCOL AIMS TO DO IT

- INTRODUCE PACKET PRIORITY LABELLING (DONE IN ROUTERS)

Page 39: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• BOTH WOULD REQUIRE IMPLEMENTATION IN ROUTERS

AND ARE COMPLICATED

• THIRD APPROACH - SIMPLER

- TRYING TO FOLLOW HOW PACKETS

ARE FLOWING THROUGH THE NETWORK AND SEND INFORMATION

TO THE SENDING SIDE

Page 40: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• REAL-TIME PROTOCOL RTP

IN THIS PROTOCOL EACH PACKET

GETS TIME STAMP AND NUMBER

WHEN IT IS SEND. ON THE RECEIVING

SIDE IT IS POSSIBLE TO DETECT IF

PACKETS ARE LOST AND WHAT ARE

THE PACKETS DELAYS. THECOMPLETE

PACKET LOOKS LIKE THIS

Page 41: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• THERE IS A PROTOCOL CALLED

RTCP – REAL TIME CONTROL PROTOCOL WHICH COLLECTS STATISTICAL INFORMATION ABOUT

PACKET DELAYS AND LOSS AND SENDS IT TO THE SENDING SIDE

WHICH MAY REACT IN SOME WAY

Page 42: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

• THE RTP/RTCP COMBINATION IS

SIMPLE AND REALISTIC FOR THE

CURRENT INTERNET:

- DOES NOT CHANGE ANYTHING

IN THE SYSTEM

- PROVIDES CONCISE REPORTING

- SENDING SIDE MAY ESTIMATE

QUALITY OF CONNECTION AND REACT E.G. BY REDUCING BITRATE

Page 43: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP INTRODUCTIONprovides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio or video. does not address resource reservation and does not guarantee quality-of-service for real-time services.

independent of the underlying transport and network layers.

RTP packet consists fixed RTP header, a possibly empty list of contributing sources and the payload data

Page 44: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP header

T

TIMESTAMP

SYNCHRONIZATION SOURCE IDENTIFIER (SSRS)

V P MCC PTX SEQUENCE NUMBER

CONTRIBUTING SOURCE IDENTIFIERS (CSRC)...

Page 45: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP header

• Version (2 bits) identifies the version of RTP

• Padding (1 bit) packet contains one or more additional

padding octets • Extension (1 bit) the fixed header must be followed by one

header extension

• CSRC Count (4 bits) contains the number of CSRC identifiers that follow the fixed header.

V P CCX

Page 46: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP header

• Marker (1 bit) the packet contains marker bits

• Payload Type (7 bits) identifies the format of the RTP payload

• Sequence number (16 bits) increments by one for each RTP data packet sent, and may be used by the receiver to detect packet loss and to restore packet sequence.

M PT SEQUENCE NUMBER

Page 47: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP header

• Timestamp (32 bits) reflects the sampling instant of the first octet in the RTP data packet

• SSRC (32 bits) field identifies the synchronization source.

• CSRC list (0 to 15 items, 32 bits each) identifies contributing sources for the payload. The number of identifiers is given by the CC field.

TIMESTAMP

SYNCHRONIZATION SOURCE IDENTIFIER (SSRS)

CONTRIBUTING SOURCE IDENTIFIERS (CSRC)...

Page 48: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP Applications

• Designed for multiparticipant multimedia conferences– Storage of continuos data– Interactive distributed simulation– Control and measurement applications

Page 49: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Implementation of RTP

• Typically integrated into application processing• No direct coupling at the RTP level between

different media– For example, audio and video are in different sessions

– Enables the choice of receiving only audio signal

• With UDP, RTP uses an even port number and the corresponding RTCP stream uses the next higher odd port number

Page 50: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP/UDP/IP

Page 51: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Application dependent protocol

• Complete implementation of RTP for a particular application requires a profile and a payload format (e.g., media encodings) specification

Page 52: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Profile

• Defines – Mapping of a set of payload formats to payload type

values– Extensions into the RTP and RTCP headers

• RTP Profile for – Audio and Video Conferences with Minimal Control

(IETF 1890)– Source Authentication and Non-Repudiation of Audio

and Video Conferences (Internet Draft)– Interactive media (proposed in journal paper)

Page 53: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Payload Types• RFCs

– H.261 and H.263 video streams; Redundant Audio Data; MPEG1/MPEG2 Video; Bundled MPEG; JPEG-compressed Video; Generic Forward Error Correction

– MPEG-4 IN PREPARATION

• Internet Drafts:– MPEG-4 Streams; Interleaved Media; DV Format Video;

MPEG-2 AAC Streams; DTMF Digits, Telephony Signals; Text Conversation; Real-Time Pointers; DV Audio; MP3 Audio, Shared Multicast Virtual Worlds (SMVW)

• Also proprietary streams can be used

Page 54: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTCP

• RTP control protocol (RTCP)

• Periodic transmission of control packets to all participants in session

Page 55: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Features of RTCP

• Feedback on the quality of data distribution– Evaluate whether problems are local or global

– Transmission rate can be changed according to network situation

• Persistent transport-level identifier for an RTP source called the canonical name or CNAME– Keep track of participants

– Associate multiple data streams from a given participant in a set of related RTP sessions

Page 56: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Features of RTCP (2)

• Observe the number of participants• Scaling of control information transmission rate in

order for RTP to suit also to a large number of participants– Maximum of 5 % of session bandwidth allocated to

RTCP

• Transport minimal session control information– For example display participant identification in the

user interface

Page 57: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTCP Packet Format

• SR: Sender report, for transmission and reception statistics from participants that are active senders

• RR: Receiver report, for reception statistics from participants that are not active senders

• SDES: Source description items– describes participant using ASCII text

• BYE: Indicates end of participation • APP: Application specific functions

Page 58: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Compound RTCP Packet

• Typically multiple RTCP packets are sent together as a compound RTCP packet

• In the following format:– Encryption prefix if packet is encrypted– SR or RR– SDES– Other RTCP packet types

Page 59: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Sender report (SR)

• First section of SR– Version, padding, reception report count,

packet type, and SSRC– Similar to the beginning of RTP header

Page 60: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Second section of SR

• NTP timestamp: 64 bits – Real time when report was sent – Used in combination with timestamps returned in

reception reports to measure round-trip propagation to those receivers

• RTP timestamp: 32 bits – Same time as the NTP timestamp but with the same

random offset as the RTP timestamps in data packets– Used by media- independent receivers to estimate the

nominal RTP clock frequency

Page 61: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Second section of SR (2)

• Sender's packet count: 32 bits – Total number of RTP data packets transmitted

by the sender since starting transmission

• Sender's octet count: 32 bits – Total number of payload octets transmitted in

RTP data packets by the sender since starting transmission

– Used to estimate the average payload data rate

Page 62: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Third section of SR

• SSRC_n (source identifier): 32 bits – SSRC identifier of the source to which the information

in this reception report block pertains

• Fraction lost: 8 bits – Fraction of RTP data packets from source SSRC_n lost

since the previous SR or RR packet was sent

• Cumulative number of packets lost: 24 bits – Total number of RTP data packets from source

SSRC_n that have been lost since the beginning of reception

Page 63: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Third section of SR (2)

• Extended highest sequence number received: 32 bits – Low 16 bits contain the highest sequence number

received in an RTP data packet from source SSRC_n– Most significant 16 bits extend sequence number with

the corresponding count of sequence number cycles

• Interarrival jitter: 32 bits – Estimate of the statistical variance of the RTP data

packet interarrival time– J=J+(|D(i-1,i)|-J)/16

Page 64: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Third section of SR (3)

• Last SR timestamp (LSR): 32 bits – NTP timestamp received as part of the most

recent RTCP sender report (SR) packet from source SSRC_n

• Delay since last SR (DLSR): 32 bits – Delay between receiving the last SR packet

from source SSRC_n and sending this reception report block

Page 65: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Sender Report

• Profile may define extensions to SR

• Format of the receiver report (RR) packet is the same as that of the SR packet except sender information is omitted

Page 66: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Why RRs and SRs?• Sender may modify its transmission rate based on the

feedback• Receivers can determine whether problems are local,

regional or global• Network managers may use profile-independent

monitors that receive only RTCP packets to evaluate the performance of their networks for multicast distribution

• Cumulative counts are used to make measurements over both short and long time periods, and to provide resilience against the loss of a report

Page 67: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Why RRs and SRs? (2)

• Since timestamp is independent of the clock rate for the data encoding, it is possible to implement encoding- and profile-independent quality monitors

• A third-party monitor can calculate the average payload data rate and the average packet rate over an interval without receiving the data

• Jitter measure may indicate congestion before it leads to packet loss– necessary to analyze a number of reports from one

receiver over time or from multiple receivers

Page 68: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SDES: Source description RTCP packet

• CNAME: Canonical end-point identifier SDES item– CNAME should be fixed for a given participant in order

to provide a binding across multiple media tools

• NAME: User name, EMAIL,PHONE,LOC: Geographic user location, TOOL: Application or tool name

• NOTE: Notice/status SDES item– Intended for transient messages describing the current

state, e.g., "on the phone, can't talk"

• PRIV: Private extensions SDES item

Page 69: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP Translators and Mixers

• Firewalls and low bandwidth connections• Connects transport-level "clouds"

– Each cloud is defined by a common network and transport protocol, multicast address or pair of unicast addresses, and transport level destination port

• May add or removes encryption• May change encoding of data or underlying

protocols

Page 70: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Translators• Translator passes through the data streams from

different sources separately• Forwards RTP packets with their SSRC identifier

intact• Receivers cannot detect the presence of a translator• Replicator from multicast to unicast• Application-level filter in firewalls• May, for example, filter non-CNAME SDES

information if bandwidth is limited

Page 71: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Mixer• Intermediate system that receives streams of RTP

packets from sources, combines them and then forwards the combined stream

• Makes timing adjustments among the streams and generates its own timing for the combined stream

• Packets marked with the mixer's own SSRC identifier

• Generates its own reception reports for sources in each cloud and sends them out only to the same cloud

Page 72: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Pros and cons of mixers

• Output bandwidth is limited to that of one source even when multiple sources are active on the input side

• Receivers on the output side don't have any control over which sources are passed through or muted

• Receivers can't do inter-media synchronization of the original streams

Page 73: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Loop detection

• Translators and mixers may create transmission loops

• A loop of data packets to a multicast destination can cause severe network flooding. – All mixers and translators are required to implement a

loop detection algorithm

• SSRCs are kept globally unique for each RTP session, they can be used to detect loops that may be introduced by mixers or translator

Page 74: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP (The Real Time Streaming Protocol)

Page 75: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP introduction

• application level protocol for control over the delivery of data with real-time properties.

• provides an extensible framework to enable controlled, on demand delivery of real-time data, such as audio and video.

• protocol provides means for choosing delivery channels

such as UPD, multicast UDP and TCP

Page 76: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP introduction

• provides means for choosing delivery mechanisms based on RTP

• does not depend on the mechanism used to carry continuous media.

• similar in syntax and operation to HTTP

• control may occur on TCP connection while the data flow via UDP.

Page 77: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP request message

• ANNOUNCE • DESCRIBE• GET_PARAMETER• OPTIONS• PAUSE• PLAY• RECORD• REDIRECT• SETUP• SET_PARAMETER• TEARDOWN

Page 78: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP request message (describe)

• DESCRIBE method retrieves the description or a presentation or media object identified request URL from the server.

– DESCRIBE rtsp://server.example.com/foo RTSP/1.0– Cseg: 312– Accept: application/sdp, application/rtsl,

application/mheg

• response– RTSP/1.0 200 OK– Cseg: 312– Date: 23 Jan 1999 15:35:06 GMT– Content-Type: application/sdp– Content-Length: 376

– SDP part

Page 79: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP request message (announce)

• two purposes:

• 1. When sent from client to server, ANNOUNCE posts the description of a presentation or media object by the request URL to a server.

• 2. When sent from server to client, ANNOUNCE updates the session description in real-time.

• The whole presentation description should be sent, rather than just the additional components.

Page 80: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP request message (options)

• query the server about options it may or may not support

• OPTIONS * RTSP/1.0• Cseq: 1• Require: implicit-play• Proxy-Require: gzipped-message

• Response could be• RTSP/1.0 200 OK• cseg: 1• Public: DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE

• or• RTSP/1.0 551 Option not supported• cseq:1• unsupported: implicit-play

Page 81: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP request message (setup)

• specifies the transport mechanism to be used for the streamed media

• change transport parameter

• SETUP rtsp://example.com/foo RTSP/1.0• Cseg: 302• Session: 47112344• Transport: RTP/AVP;unicast;client_port=…

• If the change is not allowed response is• RTSP/1.0 455 Method Not Valid In This State

•Cseq: 302

Page 82: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP request message (play)

• tells the server to start sending data • PLAY request may be queued• PLAY rtsp://example.com/foo RTSP/1.0• Cseg: 832• Session: 47112344• Range: npt=10-15

• PLAY rtsp://example.com/foo RTSP/1.0• Cseg: 833• Session: 47112344• Range: npt=20-25

Page 83: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP request message

• The PAUSE request causes the stream delivery to be

interrupted temporarily.

• The TEARDOWN request stops the stream delivery freeing the resource associated with it.

• The REDIRECT request informs the client that it must connect to another server location.

Page 84: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP request message

• The GET_PARAMETER request retrieves the value of a parameter of presentation or stream.

• The SET_PARAMETER requests to set the value of a

parameter for a presentation or stream.

• The RECORD request initiates recording a range of media

data according to the presentation description.

Page 85: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTSP example

DESCRIBE

200 OK

SETUP (video)

200 OK

SETUP (audio)

200 OK

PLAY

200 OK

PAUSE

460 (error message)

PAUSE

200 OK

Page 86: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

HIGHER LEVEL PROTOCOLS

DEVELOPED BY MMUSIC GROUP OF IETF INCLUDE SDP AND SIP PROTOCOLS IN ADDITION TO RTP/RTCP/RTSP

Page 87: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Introduction

.

• MMUSIC (Multiparty Multimedia Session Control)

• Internet standard track protocols to support Internet teleconferencing sessions.

Page 88: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP (Session Initiation Protocol)

• application-layer control protocol for creating, modifying and terminating sessions with one or more participants

• Internet multimedia conference, Internet telephone calls and multimedia distribution.

Page 89: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SDP (Session Description Protocol)

• describing multimedia sessions

• session announcement, session invitation, and other forms of multimedia session initiation.

Page 90: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP introduction

• supports name mapping and redirection services

• allows implementation of ISDN and Intelligent Network telephony subscriber service.

• Internet telephony gateways that connect Public Switched Telephone Network (PSTN) parties can also use SIP to set up calls.

Page 91: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP introduction cont.

• an application-layer control protocol for creating, modifying and terminating sessions with one or more participants.

• invite both persons and ”robots”, such as a media storage service.

• can be used to initiate sessions as well as invite members to sessions that have been advertised and established by other means.

Page 92: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP introduction

• does not offer conference control services such as floor control or voting

• doesn’t prescribe how a conference is to be managed• can be used to introduce conference control protocol

• does not reserve resources• can convey to the invited system the information

necessary to do this.

Page 93: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP Addressing

• The SIP URL similar to a mailto or telnet URL i.e. user@host

• User part is a user name or a telephone number • the host part is either a domain name or a numeric

network address

• Sip:[email protected]

Page 94: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP entities

• SIP client is an application program that is able to send SIP methods (invite).

• SIP server is an application that accepts SIP methods and replies with status code.

• SIP proxy is an intermediary program that acts as both a server and client for the purpose of making requests on behalf of the other client

Page 95: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP entities• SIP registrar is a server that accepts REGISTER

requests.

• SIP redirect server is a server that accepts a SIP request, maps a address into zero or more new addresses and returns these addresses to the client.

• Does not initiate or accepts call by itself.

• Location server is used by SIP request or proxy server to obtain information about a callee’s possible locations

Page 96: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP methods

• INVITE (call user)• ACK (confirm

connection)• CANCEL (cancel

operation)• BYE (disconnect)• REGISTER (sign up

with server)• OPTIONS (capability

query)

Page 97: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP invite

• The INVITE method indicates that the user or service is being invited to participate in a session.

• message body contains a description of the session

• type of media it is able to receive • media it is willing to send • parameters such as network destination.

Page 98: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP invite• INVITE sip:[email protected] SIP/2.0• Via: SIP/2.0/UDP kton.bell-tel.com• From: A. Bell <sip:[email protected]>• To: T. Watson <sip:[email protected]>• Call-ID: [email protected]• Cseq: 1 INVITE• Subject: Mr. Watson, come here.• Content-type: application/sdp• Content-length: …

• (SDP)• v=0• o=bell 53655765 2353687637 IN IP4 128.3.4.5• s=Mr. Watson, come here.• c=IN IP4 kton.bell-tel.com• m=audio 3456 RTP/AVP 0 3 4 5

Page 99: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP ack• confirms that the client has received a final response

to an INVITE request

• ACK is used only with INVITE request.

• ACK request may contain final session descriptions to be used by callee.

• If the ACK message body is empty, the callee uses the session description in the INVITE request.

Page 100: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP ack

• ACK sip:[email protected] SIP/2.0• Via: SIP/2.0/UDP kton.bell-tel.com• From: A. Bell <sip:[email protected]>• To: T. Watson <sip:[email protected]

tel.com>• Call-ID: [email protected]• Cseq: 1 ACK

Page 101: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP cancel, bye and options

• cancels a pending request with the same header field values, but doesn’t affect a completed request.

• release the call• may be issued by either caller or callee.

• find out the capabilities of the potential callee• without ”ringing” the designated address

Page 102: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP register

• register the address listed in the To header field with the SIP server.

• REGISTER sip:bell-tel.com SIP/2.0• Via: SIP/2.0/UDP saturn.bell-tel.com• From: sip:[email protected]• To: sip:[email protected]>• Call-ID: [email protected]• Cseq: 1 Register• Contact: <sip:[email protected]

tel.com:3890;transport=udp>• Expires: 7200

Page 103: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP status messages (responses)• 1xx: Informational - request received,

continuing to process the request• 2xx: Success - the action was successfully

received, understood and accepted

• 3xx: Redirection - further action needs to be taken in order to complete the request.

• 4xx: Client error - request contains bad syntax or cannot be fulfilled at this server.

• 5xx: Server error - the server failed to fulfill an apparently valid request.

• 6xx: Global failure - the request cannot be fulfilled at any server.

Page 104: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP status messages (responses)

• 100 Trying

• 180 Ringing

• 181 Call is being forwarded

• 182 Queued

• 200 OK

• 300 Multiple choices

• 301 Moved permanently

• 302 Moved Temporarily

• 303 See other

• 304 Use proxy

• 380 Alternative Service

• 400 Bad request

• 401 Unauthorized

• 402 Payment required

• 403 Forbidden

• 404 Not Found ...

• 480 Temporarily not available

• ….

• 500 Internal server error

• 501 Not implemented

• 502 Bad gateway

• 503 Service unavailable

• 504 Gateway time-out

• 505 SIP version not supported

• 600 Busy everywhere

• 603 Decline

• 604 Does not exist anywhere

• 605 Not acceptable

Page 105: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP status messages (responses)

• SIP/2.0 180 Ringing• Via: SIP/2.0/UDP kton.bell-tel.com• From: A. Bell <sip:a.g.bell@bell-

tel.com>• To: <sip:[email protected]

tel.com>• Call-ID: [email protected]

tel.com• Cseq: 1 INVITE• Content-length:0

• SIP/2.0 200 OK• Via: SIP/2.0/UDP kton.bell-

tel.com• From: A. Bell

<sip:[email protected]>• To: <sip:[email protected]

tel.com>• Call-ID: [email protected]

tel.com• Cseq: 1 INVITE• Contact:

sip:[email protected]• Content-type: application/sdp• Content-length: …

• v=0• o=bell 53655765 2353687637 IN

IP4 128.3.4.5• s=Mr. Watson, come here.• c=IN IP4 kton.bell-tel.com• m=audio 3456 RTP/AVP 0 3 4 5

Page 106: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP operation in proxy modepulver.com

Proxy server

nortel.com

[email protected]

1. INVITE sip:[email protected] SIP/2.0 From: sip:[email protected]

Location Server

jeff

.pu

lve

r

pu

lve

r@v

on

1

2. INVITE sip:pulver@von1 SIP/2.0 From: sip:[email protected]

3. SIP/2.0 200 ok From: sip:pulver@von1

pulver@von1

4. SIP/2.0 100 OK From: sip:[email protected]

5. ACK sip:[email protected] SIP/2.0 From: sip:[email protected]

6. ACK sip:pulver@von1 SIP/2.0 From: sip:[email protected]

Page 107: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP operation in redirect mode

1. INVITE sip:[email protected] From: sip:[email protected]

2. SIP/2.0 320 Moved temporarily Contact: sip:[email protected]

nortel.com

[email protected]

pulver.com

Redirect Server

Location Server

Je

ff.p

ulv

er

pu

lve

r@v

on

1

4. INVITE sip:[email protected] From: [email protected]

3. ACK sip:[email protected] From: sip:[email protected]. SIP/2.0 200 OK

To: [email protected]

6. ACK sip:[email protected] From: sip:[email protected]

Pulver@von1

Page 108: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SIP and H.323

• SIP• 60+ pages• Firewall - friendly

• SIP address = email address

• Personal mobility, IN services

• H.323• 200+ pages• complex, multiple

protocols• yet another address

• Not (yet)

Page 109: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SDP introduction• describing multimedia sessions for the purposes of

session announcement, session invitation, and other forms of multimedia session initiation

• advertise multimedia conference addresses and conference tool-specific information necessary to participation.

• does not incorporate a transport protocol, and is intended to use with other protocols like SIP and RTSP.

Page 110: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SDP includes:

• Session name and purpose• Time(s) the session is active (start, stop, repeat

time and so on• The media comprising the session (type, transport

protocol, format and so on)• Information to receive those media (addresses,

ports, formats and so on)• Bandwidth information• Contact information

Page 111: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

SDP example– v=0– o=mhadley 2890844526 2890842807 IN IP4 126.16.64.4– s=SDP seminar– i=A seminar on the session description protocol– u=http://www.cs.ucl.ac.uk/staff/M.Hadley/dsp.03.ps– [email protected] (Mark Hadley)– p=+44 171 380 7777– c=IN IP4 224.2.17.12/127– b=CT:128– t=2873397496 2873404696– a=recvonly– m=audio 49170 RTP/AVP 0– m=video 51372 RTP/AVP 31– m=application 32461 udp wb– a=orient:portrait

• V,O,S,T & M tags are required

Page 112: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

APPLICATION TO MULTIMEDIA – MPEG-4

EXAMPLE MPEG-4 SYSTEM FITS AND REQUIRES SOMETHING LIKE THE RTP/RTCP/RTP AND SDP/SIP IN MPEG-4 GENERIC NETWORKING SCHEME IS PROVIDED, ABOVE PROTOCOLS CAN BE ADAPTED TO IT THIS IS ONGOING WORK

Page 113: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Streaming Media Formats

Page 114: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

...

Scene Description Stream

Object Descriptor Stream

Visual Stream

Visual Stream

Visual Stream

Audio Stream

Interactive Scene Description

MPEG-4 Systems Principle

STREAMSWHICH CAN BE TRANSPORTEDOVER NETWORK

Page 115: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Example of MPEG-4 scene

Page 116: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Object-based compression and delivery

Page 117: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

...

Decoding

MPEG-4 InteractiveScene

Composition andRendering

PrimitiveAV Objects

Scene DescriptionInformation

...

ElementaryStreams

FlexMuxNetwork

TransMux

...

Object Descriptor

Display andLocal UserInteraction

MPEG-4: An integrated Multimedia System

Page 118: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Interactive AudiovisualScene

Elementary Streams

Composition and Rendering

Display andUser

Interaction

Transmission/Storage Medium

(RTP)UDP

IPMP4

DABMux

DeliveryLayer

DeliveryLayer

FlexMux

DMIF Application Interface

SL SLSL SL ... SyncLayerSyncLayer

Elementary Stream Interface

SceneDescriptionInformation

ObjectDescriptor

... CompressionLayer

CompressionLayer

SL

SL-Packetized Streams

(PES)MPEG-2

TS

AAL2ATM

UpchannelInformation

SLSL

...

Multiplexed Streams

FlexMuxFlexMux

AV ObjectsData

MPEG-4NETWORKING

Page 119: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Example of MPEG-4 over IP

Page 120: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Carriage of MPEG-4 contents over IP Networks

• MPEG-4 is a standard designated for the representation and delivery of multimedia information over a variety of transport protocols;

• MPEG-4 includes:

– Interactive scene management;

– Visual & Audio representations;

– Functional systems (multiplexing, synchronization);

– Object descriptor framework;

• Transfer method for MPEG-4 over IP:– In context with specific IP packets;

– Multiplexed in MPEG-4 Flexmux (carried in IP packets);

– Multiplexed in MPEG-2 TS (carried in IP packets);

Page 121: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

HTTP Transport Protocol

• It is a very simple way to stream media files;

• The wire format is the same as the file-format storage;

• Arbitrary MPEG-4 Intermedia files cannot be streamed directly using HTTP, because of the “loose” in ordering the objects, and possible cross-references within the media file;

• Live streaming for Intermedia files can also be supported using proprietary methods;

Page 122: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Media Control Protocols

• In order to enable full streaming systems, a media control protocol needs to be defined to support the following features:

1. Seeking:

– Forward;

– Rewind;

– Skip.

2. Bandwidth Scalability

3. Live Streaming

Page 123: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

MPEG-4 Systems• In MPEG-4 Systems, the transport of streams is divided in 4 layers:

– Compression Layer: includes elementary (raw) media streams (audio, video, etc.).

– Synchronization Layer (SL): Adds a header to each unit of an elementary stream, which includes time stamps, reference to a clock elementary stream, and identification of key frames (RandomAccessPoint). This is similar to the task of RTP in IP networks. However, SL does not contain payload type (like RTP), and does not contain the Elementary Stream. In addition, an SL packet does not contain an indication of its length, so it must be framed by a lower-level protocol such as FlexMux or RTP.

– FlexMux Layer: Groups elementary streams according to common attributes, such a QoS requirements. This is very simple multiplexing protocol, but also very low overhead.

– TransxMux Layer: This is the actual transport protocol, such as RTP/UDP, MPEG-2, etc. MPEG-4 does not define its own transport protocol, but assumes the application relies on an existing transport protocol. The FlexMux Layer is optional, but the Synchronization Layer is always present.

Page 124: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

TCP/IP & SL-packet over UDP/IP

• TCP/IP:– Not suitable for real-time transfers (high delays and causes jitter, it was

created for reliable transmission over timely delivery);

– Does not support multicast;

– Congestion control mechanism not suitable for AV media;

• SL-packet over UDP/IP:– SL provides: Timing & Sequence numbering;

– UDP provides: Multiplexing, Length field, Checksum service;

– SL+UDP may be used like a transport protocol for AV media;

Page 125: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Problems of SL/UDP/IP

• No other media stream can be synchronized with MPEG-4 data carried directly over UDP;

• The dynamic scene and session control concepts cannot be extended to non MPEG-4 data;

• No defined technique for synchronizing MPEG-4 streams from different servers;

• Streams from different servers may collide (their sources may become unresolvable at destination) in a multicast session;

• Mechanisms need to be defined to protect sensitive MPEG-4 data (RTP supports FEC);

• A feedback channel must be defined for quality control;

Page 126: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP & RTCP

• RTP = Real-time Transport Protocol;

• RTCP = Real-time Transport Control Protocol;

• A session consists of an RTP/RTCP pair of channels

• Usually works over UDP/IP (can work with other protocols also);

• RTP supports:– Multicasting;– Payload type identification;– Time stamping;– Sequence numbering;– Delivery monitoring;– From UDP (Multiplexing, Length field, Checksum service);

Page 127: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP

• RTP Problems:– It does not support the timely delivery of data or any other QoS guarantees;– It does not guarantee delivery, so packets may be delivered out of order or get lost

(no mechanism to recover from packet loss);

• Time Stamp (TS):– Place incoming packet in correct timing order– Initial values are picked randomly and independently for each RTP stream;– Increase in time indicated by each packet;

• Sequence Number (SN):– Detect packet loss;– Increase by one for each packet;

• For video frame that is split into multiple RTP packets: they share the same TS but use different SN;

Page 128: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP Characteristics

• RTP can map only one source stream onto RTP session:– Multiplexing causes problems;– For scale coding, each layer must have its own RTP session and multicast group;

• Within each RTP session, source can change its data format over time;

• It allows FEC (Forward Error Correction);

• Synchronize across different media streams;

• Provide feedback on the quality of data using packet counts;

• Identify and keep track of participants;

Page 129: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

MPEG-4 over RTP/UDP/IP

• Easiest is to wrap the MPEG-4 SL packet in an RTP packet:– High overhead: two full headers;– RTCP may not provide enough control for the MPEG-4 stream;

• Several types of MPEG-4 payloads are being defined by the IETF for different ESs;

Page 130: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

RTP ES & SL payload restrictions• RTP packetization is not media aware:

– No unique scheme can be defined, need a payload definition per payload type (not practical);

– This may require the definition of new session and scene description mechanisms to deal with all the flows;

• Common restrictions:– RTP timestamp corresponds to composition time stamp (CTS) of MPEG-4 stream;– Packets should be sent in decoding order;– Streams can be synchronized using RTCP;

• Map SL stream onto RTP session:– SL header is optional;

• Reduced SL headers does not include:– Sequence number (mapped to RTP header);– Composition Time Stamp (mapped to RTP header);

Page 131: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Streams & Media Control in MPEG-4

• Multiple Strems inMPEG-4:– Port consuming:

• Each AV contains one or more streams;• Each stream needs one RTP session;

– Potential solution:• Selective bundling of ESs-FlexMux -> define a multiplexed MPEG-4 payload (may have

to be defined for multiple payload types);• Generic RTP multiplexing for use with MPEG-4, under development by IETF.

• MPEG-4 Media Control:– Remote interactivity: add or delete a stream, etc.– Media control channel allows renegotiating in time the available network and

processing resources;– Must have an efficient signaling protocol that can handle such messages;

Page 132: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Media Control Framework

• To allow a client and one or more servers to exchange types of control messages and also allow for peer to peer exchange between two or more clients, the framework requires several components:

– A description of the stored or live presentation;

– A set of protocols that can provide proper services for the back channel message delivery;

– A set of protocols that can allocate resources for the involved hosts and networks;

Page 133: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Components of media control (1)

• Presentation Description:

– The client needs to refer to the description of a presentation that expresses the temporal and static properties of the presentation itself;

– Must include information about the media, location of the media, etc.

– Should provide multiple description instances of the same presentation so that the client can specify a given combination that fits its needs/capabilities – the client is the orchestrator of the presentation and the server is participating;

Page 134: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Components of media control (2)

• Client and Server State Representations:

– Out of band signaling is used: the data streams and the control information are carried over separate channels using different protocols, each best suited to their needs and modes of operation;

– As the media streams may be modified by the end user, the server needs to a state if the streams status for each client it is serving;

– The client has to keep track of all the participating streams;

Page 135: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Components of media control (3)

• Basic Media Control Messages:

– A multimedia system should have access to control messages ranging from remote VCR functions such as stop, play, fast forward and fast reverse, to messages generated in response to user actions to modify the presentation of a given object stream such as add or remove or alter, etc.

– The basic control functionality relates to presentation and stream set-up: play, stop, pause, teardown and recording;

Page 136: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Real Time Streaming Protocol (RTSP)

• It is an application level protocol that provides an extensible framework to enable controlled delivery of real-time data, such as audio & video;

• It can be transported over UDP, TCP and is designated to work with RTP and HTTP;

• Provide support for bidirectional communications (frame level timing for remote video editing);

• RTSP does:– Control the transmission of media stream;– Use out-of-band signaling;

• RTSP does not:– Define compression schemes;– Define how AV is encapsulated;– Define how to buffer;

Page 137: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

MPEG-4 and RTSP

• From DMIF’s perspective, RTSP is an application alongside MPEG-4 systems;

• The RTSP client and server interact with the MPEG-4 systems;

• The RTSP client and server control the streams through the DAI by an RTSP-DMIF interface;

• The interface is kept very simple by limiting it to field mapping between the RTSP fields and the DAI primitive parameters;

• The RTSP client server interactions are used to control the MPEG-4 elementary streams;

Page 138: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Session Initiation Protocol (SIP)

• IETF Family of Session Protocols:– Session Description Protocol (SDP);– Session Announcement Protocol (SAP);– Session Initiation Protocol (SIP);

• SIP:– Two basic components: user agent & network server;– Independent of lower layer protocols;– Extensible to be application specific;– Gaining widespread use in IP telephony;

• MPEG-4 and SIP:– Unique ability to control different media types within a single session Multiple

stream transmission in one network session;– User Agent model fits in well with an MPEG-4 Client/Server model (point-to-point

communication);

Page 139: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Toolboxes for transmitting MPEG-4 over internet

• RTP transport of audio/video/… data, quality-of-service feedback;

• RTSP very simple media control of streams;

• SIP inviting people, media servers to sessions – telephony and streaming audio/video;

• HTTP, SDP retrieve media descriptions;

Page 140: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Issues

• Encapsulation of MPEG-4 Sync layer packetized stream:– IEC/ISO 14496-8, framework still in revision;– Lots of issues still remain: time axis, buffer management, etc…

• Interactivity between application and End User:– Description of MPEG-4 content;– Initialization of an MPEG-4 session;

• SIP and IPC (Inter Process communication):– How to describe the dynamic process of channel (stream) setup and release?– What control information is necessary and how to transport it?

• Transport and IP QoS:– Must define a mapping mechanism among the different QoS mechanisms:

transport QoS (not available yet), network QoS, etc.

• And more…e.g. DMIF

Page 141: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

DMIF

• Delivery Multimedia Integration Framework

• DMIF is networking part of the MPEG-4 standard

• DMIF specifies the Delivery Layer of MPEG-4 (mostly in generic form)

Page 142: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

MPEG-4 Standard Structure

DAI

ESI

Delivery Aware, media unaware :DMIFFlexMux

Media Aware, delivery unaware :Audio and Visual CompressionScene Description and Object Descriptor Protocol

Media and Delivery unaware :Systems

DeliveryLayer

FlexMuxTool

Sync.Layer

CompressionLayer

Composition

Related to networking

Page 143: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Why DMIF?

• many different delivery technologies, each with its own peculiarities

• different APIs for different environments (Local Files, Broadcast sources, Interactive servers through a variety of transports)

• no consolidated solution for real time multimedia streaming at certain QoS

• the introduction of QoS support in networks also impacts on charging policies

Page 144: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

DMIF goals

• Separate the delivery aspects from the multimedia application issues

• Guarantee permanence of multimedia application in presence of new delivery technologies

• Allow the concurrent usage of different delivery technologies (e.g. broadcast + local retrieval)

• Allow resource logging to support flexible charging policies

Page 145: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

DMIF

• DMIF addresses the following scenarios:

– multicast/broadcast

– local storage

– remote interactive

Broadcastsource

LocalStorage

Network

Page 146: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

The DMIF reference architecture

• supports the streaming of multimedia content• avoids embedding delivery dependency in a

multimedia application• allows the concurrent usage of multiple delivery

technologies• enables a plug-in approach to add new modules

– to follow technical evolution in content streaming– to allow ad-hoc delivery implementations (e.g. for

charging, admission control, Intelligent Networks …)

Page 147: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

The DMIF reference architecture

Sigmap

TargetDMIF

TargetApp

DNI DAI

Origin.

App

DAI

DM

IF F

ilte

r

Origin. DMIFfor Remote

srv

DNI

Sigmap Network

Origin. DMIFfor Broadcast

TargetDMIF

Target Application

Broadcastsource

Origin. DMIFfor Local Files

TargetDMIF

Target Application

LocalStorage

Page 148: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

MPEG-4 synchronisation, mux & transport

(MPEG-4 own Sync Layer + Flexmux approach)

DAI

ESI

Synchronisation Layer

FlexMux

SL SL SL SL SL SL

FlexMux

SL

HTML page JPG

GIF

VideoAudio

JavaclassOD

BIUBIA

DMIF

DNI

Delivery Layer Control Plane

Applic. Signal.

IP TCP

HTTP UDP

RTP

FlexMux Channels

ES Channels

TransMux Channels

RTCP

QoS Manager

Page 149: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

DNI

MPEG-4 synch., mux & transport (IETF

PROPOSAL, RTP CHANNEL MULTIPLEX)

IP TCP

HTTP UDP

RTP/RTP Mux

Generic MP4 ES (& SL-PDU) to

RTP mapping

HTML page JPG

GIF

Video

AudioJavaclass

ODBIU

BIA

Delivery Layer

DMIF Control Plane

Synchronisation and Protection

TransMux Channels

ES Channels

Applic. Signal.

DAI

ESI

RTCP

Page 150: MULTIMEDIA SYSTEMS NETWORKING

MULTIMEDIA SYSTEMS IREK DEFEE

Conclusions

• MPEG-4 is here in a limited fashion;

• We will see a marked growth in MPEG-4 products as chip sets become more readily available;

• Simple basic MPEG-4 service is not a problem;

• There is a lot of ongoing research on how to deliver MPEG-4 content over IP-based networks;


Recommended