Post on 02-May-2018
transcript
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 1
Interframe coding of video signals
Adaptive intra-interframe prediction Conditional replenishment Rate-distortion optimized mode selection Motion-compensated prediction Hybrid coding: combining interframe
prediction and intraframe transform coding
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 2
Interframe coding of video signals
. . . exploits similarity of successive pictures
Previous frame Current frame
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 3
“It has been customary in the past to transmit successive complete images of the transmitted picture.” [...] “In accordance with this invention, this difficulty is avoided by transmitting only the difference between successive images of the object.”
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 4
Interframe DPCM
Entropy coder Entropy decoder Quantizer
Frame store
e s + - s ^
s'
e' e'
s ^
s'
Coder Decoder
Frame store
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 5
Adaptive intra-interframe Prediction
Predictor is switched between two states: A: Intraframe prediction for moving or changed areas.
B: Interframe prediction (previous frame prediction) for still areas of the picture.
Sintra = a1S1 + a2S2 + a3S3 + a4S4 Sinter = S20
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 6
Intra-interframe DPCM: feedback adaptation
Entropy coder
Interframe predictor
Intraframe predictor
Entropy decoder Quantizer
Interframe predictor
Intraframe predictor Predictor
adaptation
e s + - s ^ s ^
inter
s ^ intra
s'
e' e'
s ^ s ^
inter
s ^ intra
s'
Predictor adaptation
Coder Decoder
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 7
Intra-interframe DPCM: feedforward adaptation
Interframe predictor
Intraframe predictor
Entropy decoder Entropy coder Quantizer
Interframe predictor
Intraframe predictor
Predictor adaptation
e s + - s ^ s ^
inter
s ^ intra
s'
e' e'
s ^ s ^
inter
s ^ intra
s'
intra-/interframe switching information
Coder Decoder
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 8
Conditional replenishment
Still areas: repeat from frame store Moving areas: encode and transmit address and waveform
CODER DECODER
SIGNAL INPUT
SEGMENTER (MOVEMENT DETECTOR)
CODING, ADDRESSING, BUFFERING
BUFFERING, DECODING, ADDRESSING
FRAME DELAY
TRANSMISSION CHANNEL
SIGNAL OUTPUT
Change detector
Frame store Frame store
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 9
Change detection Example of a pixel-wise change detector
Example of a block-wise change detector
ABS Average of 3x3 window
Eliminate isolated points or pairs of points
+
-
Current frame
Previous frame
Decision changed/ unchanged
Threshold
ABS Accumulate over NxN blocks
+
-
Current frame
Previous frame
Decision changed/ unchanged
Threshold
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 10
Example: pixel-wise change detection
Two successive video frames Change detection
mask
[Xinqiao Liu, EE368B class project, 2000]
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 11
Crawford noise reduction filter
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 12
“Dirty Window” effect
Conditional replenishment scheme with change detection threshold set too high leads to the subjective impression of looking through a dirty window.
Moving area picked up by change detector
Moving areas missed by change detector
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 13
Rate-distortion optimized mode selection
How to choose the decision threshold, if distortion D shall be minimized for a given rate R?
Assumptions Blockwise mode selection, block index i Additive overall distortion and rate
Lagrangian cost function
Strategy: minimize Ji for each block i separately, using a common Lagrange multiplier λ
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 14
Rate-distortion optimized mode selection (cont.)
D
R
D1(R1)
D2(R2)
• Consider 2 blocks with D(R) = D1(R1) +D2(R2) • Intra-coding only
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 15
Rate-distortion optimized mode selection (cont.)
D
R
D1(R1)
D2(R2)
Interframe replenishment
Intraframe coding
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 16
Motion-compensated prediction
x
y
stationary background
moving object
shifted object
time t
previous frame
current frame
Prediction for the luminance signal S(x,y,t) within the moving object:
S(x, y,t) = S(x − dx , y − dy ,t − Δt)
„Displacement vector“
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 17
Combining transform coding and prediction
- -
Transform domain prediction Space domain prediction
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 18
Intra-frame Decoder
Motion- Compensated
Predictor
Control Data
DCT Coefficients
Motion Data
0
Intra/Inter
Coder Control
Decoder
Motion Estimator
Intra-frame DCT Coder -
Motion-compensated hybrid coder
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 19
Intra-frame Decoder
Motion- Compensated
Predictor
Control Data
DCT Coefficients
Motion Data
0
Intra/Inter
Decoder
Motion-compensated hybrid decoder
Bernd Girod: EE398A Image and Video Compression Interframe Coding no. 20
Reading
B. Girod, H. Almer, L. Bengtsson, B. Christensson, P. Weiss, “A Subjective Evaluation of Noise-Shaping Quantization for Adaptive Intra-/Interframe DPCM Coding of Color Television Signals,” IEEE Trans. Communications, vol. 36, no. 3, pp. 332-346, March 1988.
B. Haskell, P. Gordon, R. Schmidt, J. Scattaglia, “Interframe Coding of 525-Line, Monochrome Television at 1.5 Mbits/s,” IEEE Trans. Communications, vol. 25, no. 11, pp. 1339-1348, Nov. 1977.
S. Ericsson, “Fixed and Adaptive Predictors for Hybrid Predictive/Transform Coding,” IEEE Trans. Communications, vol. 33, no. 12, pp. 1291-1302, Dec. 1985.