RTP/RTCP/RTSP
Real-Time Protocols
Presented by-1. A. H. M. Kamal (0411054002)2. KAZY NOOR –E- ALAM SIDDIQUEE (0411052009)3. MASUD4.Mahmudul Hasan Razib (0411052060)
Overview
• Streaming performance requirements• Protocol stack for multimedia services• Real-time transport protocol (RTP)• RTP control protocol (RTCP)• Real-time streaming protocol (RTSP)
Real-time multimedia streaming• Real-time multimedia applications
– Video teleconferencing– Internet Telephony (VoIP)– Internet audio, video streaming
(A-PDUs)
Streaming performance requirements– Sequencing
– to report PDU loss – to report PDU reordering – to perform out-of-order decoding
– Time stamping and Buffering – for play out– for jitter and delay calculation
– Payload type identification– for media interpretation
– Error concealment –covers up errors from lost PDU by using redundancy in most-adjacent-frame
– Quality of Service (QoS) feedback – from receiver to sender for operation adjustment
– Rate control –sender reduces sending rate adaptively to network congestion
Ideal Timing – no jitter
00.00.00
00.00.10
00.00.20
00.00.30
00.00.11
00.00.21
00.00.31
Send time
Play time
30 s
eco
nds
First RTP-PDU
Second RTP-PDU
Third RTP-PDU
application
Reality – jitter
00.00.00
00.00.10
00.00.20
00.00.30
00.00.11
Send time
Play time
00.00.21
00.00.25
00.00.35
00.00.37
00.00.47
delay
First RTP-PDU
Second RTP-PDU
Third RTP-PDU 00.00.40
Fourth RTP-PDU 00.00.41
00.00.51
Jitter (contd.)
00.00.00
00.00.10
00.00.20
00.00.30
00.00.11
Send time
Play time
00.00.21
00.00.25
00.00.3500.00.37
00.00.47
First RTP-PDU(0)
Second RTP-PDU(10)
Third RTP-PDU(20) 00.00.40
Fourth RTP-PDU (30) 00.00.41
00.00.51
00.00.18
00.00.28
00.00.38
00.00.48
00.00.58
Jitter (contd.)
Playback bufferAt time 00:00:18
At time 00:00:28
At time 00:00:38
How does Sequence number and Timestamp help ?
Audio silence example:
Solution:
– After receiving no PDUs for a while, next PDU received at the receiver will reflect a big jump in timestamp, but have the correct next seq. no. Thus, receiver knows what happened.
– Why might this cause problems? sen
der
rece
iver
silence
Seq no.1, Tmpst 100Seq no.2, Tmpst 200Seq no.3, Tmpst 300
Seq no.4, Tmpst 600Seq no.5, Tmpst 700
• Consider audio data– What should the sender do during silence?
• Not send anything
• Receiver cannot distinguish between loss and silence
Streaming performance requirements– Sequencing
– to report PDU loss – to report PDU reordering – to perform out-of-order decoding
– Time stamping and Buffering – for play out– for jitter and delay calculation
– Payload type identification– for media interpretation
– Error concealment –covers up errors from lost PDU by using redundancy in most-adjacent-frame
– Quality of Service (QoS) feedback – from receiver to sender for operation adjustment
– Rate control –sender reduces sending rate adaptively to network congestion
TCP is not used because:• TCP does retransmissions unbounded delays• No provision for time stamping• TCP does not support multicast• TCP congestion control (slow-start) unsuitable for real-time transport
RTP + UDP usually used for multimedia services
Support from transport layers
Not like streaming stored audio and videosupport longer delay?
• Video conference• Internet Telephony
Real time interactive audio video
TCP(till now)
RTSP
Protocol stack for multimedia services
RTP RTCP
RTP: Introduction• Provides end-to-end transport functions for real-time
applications– Supports different payload types
• All RTP and RTCP PDUs are sent to same multicast group (by all participants)
• All RTP PDUs sent to an even-numbered UDP port, 2p
• All RTCP PDUs sent to UDP port 2p+1
• Does NOT provide timely delivery or other QoS guarantees– Relies on other protocols like RTCP and lower layers
• Does NOT assume the underlying network is reliable and delivers PDUs in sequence– Uses sequence number
RTP RTCP
Application
UDP
IP
Data Link
Physical
Transport
layer
RTP Session
For an RTP session
Single multicast address
RTP Session
RTP session is sending and receiving of RTP data by a group of participants
For each participant, a session is a pair of transport addresses used to communicate with the group
If multiple media types are communicated by the group, the transmission of each medium constitutes a session.
RTP Synchronization Source
synchronization source - each source of RTP PDUs
Identified by a unique,randomly chosen 32-bit ID (the SSRC)
A host generating multiple streams within a single RTP must use a different SSRC per stream
RTP Basics of Data Transmission
RTP PDUs
RTP Packet
RTP Header Audio/Video Chunk
RTP Packet
RTP Header Audio/Video Chunk
RTP Header
RTP PDU HeaderIncremented by one for each RTP PDU:
• PDU loss detection•Restore PDU sequence
Sampling instant of first data octet• multiple PDUs can have same timestamp• not necessarily monotonic• used to synchronize different
media streams
Payload type
Identifies synchronization source
(used by mixers)Identifies contributing sources
RTP Payload-Type
Payload-Type Number Audio Format Sampling Rate (kHz) Rate (kbps)
0 PCM u-low 8 64
1 1016 8 4.8
3 GSM 8 13
7 LPC 8 2.4
9 G.722 16 48-64
14 MPEG Audio 90 -
15 G.728 8 16
Audio Payload Type
Video Payload Type
Payload Type Number Video Format
26 Motion JPEG
31 H.261
32 MPEG 1 video
33 MPEG 2 video
MixerRTP mixer - an intermediate system that receives & combines RTP PDUs of one or more RTP sessions into a new RTP PDU
• Stream may be transcoded, special effects may be performed.
• A mixer will typically have to define synchronization relationships between streams.Thus…
Sources that are mixed together become contributing sources (CSRC)
Mixer itself appears as a new source having a new SSRC
Translator
• An intermediate system that…
Connects two or more networks
Multicasting through a firewall
Modifies stream encoding, changing the stream’s timing
Transparent to participants
SSRC’s remain intact
end system 1
end system 2
transl.1from ES1: SSRC=6
from ES2: SSRC=23transl.2
from ES2: SSRC=23from ES1: SSRC=6
authorized tunnel
firewallfrom ES2: SSRC=23from ES1: SSRC=6
RTP: Developing Software App. with RTP
RTP RTCP
UDP
IP
Data Link
Physical
Transport
layer
Application
RTP
Socket
UDP
IP
Data Link
Physical
• Do by hand
• Add RTP header to frame
• Pass to UDP socket
Application
• Use JAVA Class/ C RTP Library
• Send a chunk of media, Payload type no., SSRC, timestamp to interface
Approach One Alternative Approach
RTP Security
Confidentiality
Authentication and Integrity
Allows encription to RTP Packets
Believes to lower levels to ensure authentication.
Integrity by sanity checking descriptor header: protocol version no, pkt length, payload type.
Key Management Conference type App:
Combination of SIP, SAP, SDP
RTP Control Protocol (RTCP) RTCP specifies report PDUs exchanged between sources and destinations of multimedia information
receiver reception report
sender report
source description report
Reports contain statistics such as the number of RTP-PDUs sent, number of RTP-PDUs lost, inter-arrival jitter
Feedback Information: modify sender transmission rates and for diagnostics purposes-whether problems are local, regional, global
• Timestamps in RTP PDUs are tied to the individual video and audio sampling clocks timestamps are not tied to the wall-clock time, or each other!
Synchronization of streams using RTCP
• Each RTCP sender-report PDU contains (for most recently generated PDU in associated RTP stream):
The timestamp of RTP PDU The wall-clock time for when PDU was created
• Receivers can use this association to synchronize the playout of audio and video
Internetwork
RTP audio
RTCP audio
RTP video
RTP video
RTCP bandwidth scaling
Solution• RTCP attempts to limit its
traffic to 5% of the session bandwidth to ensure it can scale!
• RTCP gives 75% of this rate to the receivers; and the remaining 25% to the sender.
Example • Suppose one sender, sending
video at a rate of 2 Mbps. Then RTCP attempts to limit its traffic to 100 Kbps.
• The 75 kbps is equally shared among receivers: – With R receivers, each
receiver gets to send RTCP traffic at 75/R kbps.
• Sender gets to send RTCP traffic at 25 kbps.
Problem• What happens when there is one sender and many receivers? RTCP reports scale linearly with the number of participants and would match or exceed the amount of RTP data! More overhead than useful data!
RTCP bandwidth scaling
Sender Can Send T=# of Senders
.25x.05xSession BWAvg. RTP packet size
Receiver Can Send T=# of Senders
.75x.05xSession BWAvg. RTP packet size
Real-Time Streaming Protocol (RTSP)
• Application layer protocol (default port 554)• Usually runs on RTP for stream & TCP for control• Provides the control channel to transmit• Uses out-of-band signaling• Usable for Live broadcasts / multicast
Also known as “Network remote control” for multi-media servers.
web browser
media player
Web Server
Web Server/Media server
RTSP Overview
RTSPpres. desc,streaming commands
RTP/RTCPaudio/video content
Presentation
descriptor
HTTPpresentation descriptor
•Reference to several continuous media file
•Directive for synchronization
RTSP Session
media server
RTSPserver
datasource
media player
AVsubsyste
m
RTSPclient
RTSP OK
RTSP PLAY
RTSP OK
RTP AUDIO
RTP VIDEO
RTSP TEARDOWNRTSP OK
get UDP portchooseUDP port
RTSP SETUP
Default port 554
RTCP
TCP
UDP
Example:Media on demand (Unicast)
Media server A
audio.example.com
Media server V
video.example.com
Web server W
-holds the media descriptors
Client C
RTSP Message sequence
C
W
V
A
C->V : SETUP rtsp://video.example.com/twister/video.en RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=3058-3059
A-> C : RTSP/1.0 200 OK
Cseq:1
Session: 23456789
Transport : RTP/AVP/UDP;unicast;client_port=3058-3059
server_port=5002-5003
C -> W : GET/Twister.sdp HTTP/1.1
Host: www.example.com
Accept: application/sdp
W-> C : HTTP/1.0 200 OK
Content-Type: application/sdp
C-> A : SETUP rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=3056-3057
A-> C : RTSP/1.0 200 OK
Cseq:1
Session: 12345678
Transport : RTP/AVP/UDP;unicast;client_port=3056-3057
server_port=5000-5001
RTSP Message sequence (contd.)
C
W
V
A
C->V: PLAY rtsp://video.example.com/twister/video RTSP/1.0
Cseq: 2
Session: 23456789
V->C: RTSP/1.0 200 OK
Cseq: 2
Session: 23456789
RTP-Info: url=rtsp://video.example.com/twister/video;
seq=12312232;
C->A: PLAY rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq: 2
Session: 12345678
A->C: RTSP/1.0 200 OK
Cseq: 2
Session: 12345678
RTP-Info: url=rtsp://audio.example.com/twister/audio.en;
seq=876655;
RTSP Message sequence (contd.)
C
W
V
A
C->A: TEARDOWN rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq: 3
Session: 12345678
A->C: RTSP/1.0 200 OK
Cseq: 3
C->V: TEARDOWN rtsp://video.example.com/twister/video RTSP/1.0
Cseq: 3
Session: 23456789
V->C: RTSP/1.0 200 OK
Cseq: 3
RTCP compound PDU
SRsenderreport
receiverreport
receiverreportSS
RC
SS
RC
SS
RC
source 2 source 3
RTCP PDU 1
SDES CNAME PHONE
SS
RC
RTCP PDU 2
compound PDU(single UDP datagram)
RTCP processing in Translators
• SR sender information : Does not generate their own sender information(most of the times), but forwards the SR PDUs received from one side to other
• RR reception report blocks : Does not generate their own RR reports (most of the times), but forwards RR reports received from one side to another. SSRC are left intact
• SDES : Forwards without changing the SDES info. but may filter non CNAME SDES, if bandwidth is limited
• BYE : Forwards BYE PDU unchanged. A translator about to cease forwarding, send a BYE PDU to each connected nodes
RTCP processing in Mixers• SR sender information : Generates its own SR info. Because the characteristics of source stream is lost in the mix. The SR info is sent in same direction as the mixed stream
• RR reception report blocks : Generates its own reports for sources in each cloud and sends them only to same cloud
• SDES : Forwards without changing the SDES info. but may filter non CNAME SDES, if bandwidth is limited
• BYE : Forwards BYE PDU unchanged. A mixer about to cease forwarding, send a BYE PDU to each connected nodes
Source description PDUsMay contain:
– a CNAME item (canonical identifier/name) – a NAME item (real user name) – an EMAIL item – a PHONE item – a LOC item (geographic location) – a TOOL item (application name) – a NOTE item (transient msg, e.g. for status) – a PRIV item (private extension)
Value
1
2
3
4
5
6
7
8
CNAME=1 length user and domain name
Here is the end of my presentation on RTP
=Thanks to all=