82
1 Dude, Where is My Packet?
1
Dude, Where is My Packet?
NUS.SOC.CS5248OOI WEI TSANG
2
Overview
Characteristics of the InternetGeneral techniquesError recovery for audioEffect of loss on MPEGError recovery for MPEG
3
Loss Characteristics of The Internet
NUS.SOC.CS5248OOI WEI TSANG
4
Characteristics of Internet60-70% of paths do not show
any loss
Those with loss have an average of 4.5 – 6% packet loss
[Paxson97] End-to-end Internet packet dynamics
NUS.SOC.CS5248OOI WEI TSANG
5
Packet Loss Pattern
1000
30
NUS.SOC.CS5248OOI WEI TSANG
6
Characteristic of InternetBursts of loss are typically short
(2-3 consecutively loss packets)
Long burst do occur
Burst may occur periodically
NUS.SOC.CS5248OOI WEI TSANG
7
Wireless Link
Loss rate measured in my office 10 ~ 20%
up to 50% reported!
8
General Error Recovery Techniques
Perkins, Hodson and HardmanIEEE Network Magazine 1998
NUS.SOC.CS5248OOI WEI TSANG
9
Retransmission
1 2 3 4
1 2 4
3
3
NUS.SOC.CS5248OOI WEI TSANG
10
Redundant Data
1 2 3 4
1 2 4
2
2 3
3 4 5
53
NUS.SOC.CS5248OOI WEI TSANG
11
2 4
Error Concealment
1 2 3 4
1
12
Retransmission
for audio
NUS.SOC.CS5248OOI WEI TSANG
13
Retransmission is Bad
Need scalable feedbackNACK consumes bandwidthMay need to retransmit
frequentlyRetransmission maybe useless
NUS.SOC.CS5248OOI WEI TSANG
14
Retransmission is GoodOnly retransmit when neededExact recovery
NUS.SOC.CS5248OOI WEI TSANG
15
Scalable RetransmissionOn packet loss
T = random(0, RTT)wait for Tmulticast NACK
On receiving NACK from otherssuppress own NACK
NUS.SOC.CS5248OOI WEI TSANG
16
Retransmit when
group size is small loss rate is low large latency acceptable
17
Redundant Data
for audio
NUS.SOC.CS5248OOI WEI TSANG
18
Parity FEC
1011001
1000010
0001001
XOR 0010010
NUS.SOC.CS5248OOI WEI TSANG
19
Parity FEC
1011001
1000010
0001001
0010010
NUS.SOC.CS5248OOI WEI TSANG
20
Parity FEC Ordering
NUS.SOC.CS5248OOI WEI TSANG
21
Parity FEC Ordering
NUS.SOC.CS5248OOI WEI TSANG
22
Parity FEC Ordering
NUS.SOC.CS5248OOI WEI TSANG
23
Parity FEC Ordering
NUS.SOC.CS5248OOI WEI TSANG
24
Parity FEC Ordering
NUS.SOC.CS5248OOI WEI TSANG
25
Reed-Solomon Code
RS(n,k)
RS
NUS.SOC.CS5248OOI WEI TSANG
26
Reed-Solomon Code
NUS.SOC.CS5248OOI WEI TSANG
27
Media Specific FEC
1 2 3 4
1 2 4
2
2 3
3 4 5
53
NUS.SOC.CS5248OOI WEI TSANG
28
Interleaving
NUS.SOC.CS5248OOI WEI TSANG
29
Pro and Cons
Media Independent FECOverhead for redundant dataExact recoveryCould be computationally costly
Media Specific FECApproximate recovery only
30
Error Concealment
NUS.SOC.CS5248OOI WEI TSANG
31
Effect of Loss on AudioSpeech
Human ears can interpolateLoss up to length of phoneme can
still be tolerable
NUS.SOC.CS5248OOI WEI TSANG
32
Insertion-based RepairSpliceSilence SubstitutionNoise SubstitutionRepetition
NUS.SOC.CS5248OOI WEI TSANG
33
Other Repair Methods
InterpolationRegeneration
NUS.SOC.CS5248OOI WEI TSANG
34
Pro and Cons
Could be computationally costlyApproximate recovery onlyDoes not work well for long burst
of packet loss
35
Colin’s Recommendations
NUS.SOC.CS5248OOI WEI TSANG
36
Non-Interactive Apps
InterleavingFECRetransmission for unicast only
NUS.SOC.CS5248OOI WEI TSANG
37
Interactive ApplicationsMedia Specific FEC
NUS.SOC.CS5248OOI WEI TSANG
38
Error Concealment
Repeat
39
Effects of Packet Loss on MPEGJill Boyce and Robert GaglianelloACM Multimedia 1998
NUS.SOC.CS5248OOI WEI TSANG
40
Measurement-based StudyNeed to understand the problem
before proposing solution
NUS.SOC.CS5248OOI WEI TSANG
41
Data Gathering MethodFrom: NYC13 , Austin21, London18
To: Holmdel, NJ
NUS.SOC.CS5248OOI WEI TSANG
42
Data Gathering MethodVideo:
Two 5-mins MPEG30 fps384 kbps and 1 MbpsIBBPBBPBBPBBPBBQSIF 176x112 and SIF 352x240One row per slice
NUS.SOC.CS5248OOI WEI TSANG
43
Average Packet Loss
0
2
4
6
8
10
12
14
Packet Loss Rate
(% )
New York Texas London
1Mbps384kbps
NUS.SOC.CS5248OOI WEI TSANG
44
Packet Loss vs Time
NUS.SOC.CS5248OOI WEI TSANG
45
How to Fill Packets
B 1 frame 1 packetP 1 slice 1 packetI 1 slice 1 packet
NUS.SOC.CS5248OOI WEI TSANG
46
Packet Size Distributions
NUS.SOC.CS5248OOI WEI TSANG
47
Packet Size vs Loss Rate
NUS.SOC.CS5248OOI WEI TSANG
48
Lesson
1 slice per packet even if larger than MTU
If smaller packets means higher loss rate: fill packet until MTU
Else one slice per MTU
Alt: Change size of slice
NUS.SOC.CS5248OOI WEI TSANG
49
Frame Effected by Errors
NUS.SOC.CS5248OOI WEI TSANG
50
Correlation of Packet Loss
51
Overview of Error Recovery for Video
NUS.SOC.CS5248OOI WEI TSANG
52
Methods
RetransmissionError ConcealmentFECLimiting Error Propagation
Reference Frame Selection
NUS.SOC.CS5248OOI WEI TSANG
53
Reference Frame Selection
I P B B P
X
I
NUS.SOC.CS5248OOI WEI TSANG
54
Methods
RetransmissionError ConcealmentFECLimiting Error Propagation
Reference Frame SelectionChanging Temporal Pattern
55
Changing Temporal Patterns
Injong RheeSIGCOMM ‘98
NUS.SOC.CS5248OOI WEI TSANG
56
Basic Idea
“Better Late Than Never!”, orLate packet is still useful
NUS.SOC.CS5248OOI WEI TSANG
57
MPEG Frame Pattern
I B B P B B P
I P B B P B B
NUS.SOC.CS5248OOI WEI TSANG
58
H.261 Error Propagation
I P P P P P PX X X X X X
I P P P P P PX X X X
retransmission
loss
NUS.SOC.CS5248OOI WEI TSANG
59
H.261 Frame Pattern
I P P P P P PX
loss
NUS.SOC.CS5248OOI WEI TSANG
60
I P P P P P P
H.261 Frame Pattern
X
loss
X X
retransmission
NUS.SOC.CS5248OOI WEI TSANG
61
PTDD
Periodic Temporal Dependency Distance
Large PTDDLater DeadlinesLonger Error PropagationsLess Temporal Dependencies
NUS.SOC.CS5248OOI WEI TSANG
62
QAL
I P P P
I P P P Base Layer
EnhancementLayer
NUS.SOC.CS5248OOI WEI TSANG
63
QAL
I P P P
I P P P Base Layer + FEC
EnhancementLayer
X
NUS.SOC.CS5248OOI WEI TSANG
64
QAL + PTDD
I P P P
I P P P Base Layer
EnhancementLayer
65
SR-RTP
Feamster and BalakrishnanPV’02
NUS.SOC.CS5248OOI WEI TSANG
66
Overview
An analytical model for packet loss
SR-RTPPost Processing
NUS.SOC.CS5248OOI WEI TSANG
67
Analytic Model
p : prob. packet loss f : observed frame rate : frame drop rateSi : mean num. of packets in
frames of type iP(i) : prob. frame type is iP(F) : prob. frame is useless
NUS.SOC.CS5248OOI WEI TSANG
68
p : prob. packet loss f : observed frame rate : frame drop rateSi : mean num. of packets in frames of type iP(i) : prob. frame type is iP(F) : prob. frame is useless
NUS.SOC.CS5248OOI WEI TSANG
69
Is it accurate?
NUS.SOC.CS5248OOI WEI TSANG
70
SR-RTP
Extensions to RTP for selective reliability
NUS.SOC.CS5248OOI WEI TSANG
71
SR-RTP
0 Length
ADU Sequence Number
ADU Length
ADU Offset
Priority Layer Number
NUS.SOC.CS5248OOI WEI TSANG
72
Feamster & Balakrishnan’s ADU1 ADU = 1 Frame
NUS.SOC.CS5248OOI WEI TSANG
73
Loss Detection
LEN: 50ADU: 0ADU Size: 100ADU Off: 0
LEN: 20ADU: 0ADU Size: 100ADU Off: 80
LEN: 50ADU: 1ADU Size: 150ADU Off: 40
LEN: 50ADU: 3ADU Size: 50ADU Off: 0
NUS.SOC.CS5248OOI WEI TSANG
74
Retransmission RequestBased on priorityExample:
I-Frame: highest priorityP-Frame: higher priority if closer
to prev IB-Frame: no retransmition
NUS.SOC.CS5248OOI WEI TSANG
75
Error Concealment for VideoRepeat pixels from previous
frameInterpolate pixels from
neighbouring regionInterpolate motion vectors from
previous frame
NUS.SOC.CS5248OOI WEI TSANG
76
Inferred MV for P Frame
I P
NUS.SOC.CS5248OOI WEI TSANG
77
Inferred MV for I Frame
P P I
NUS.SOC.CS5248OOI WEI TSANG
78
Evaluation
Peak Signal to Noise Ratio (PSNR)
2552 over average square of differences in pixel values
Calculated in dB (10 log 10)
NUS.SOC.CS5248OOI WEI TSANG
79
Performance of SR-RTP
80
Today’s Summary
NUS.SOC.CS5248OOI WEI TSANG
81
How to recover packet lossRetransmissionFECError Concealment
NUS.SOC.CS5248OOI WEI TSANG
82
How to limit the damage of error InterleavingKey frame selectionChange reference frame
