Post on 19-Dec-2015
transcript
Shengkai Lin@CMLAB, NTUCSIE 2004
1
Ch. 6- H.264/AVCCh. 6- H.264/AVCPart I (pp.160~199)Part I (pp.160~199)Ch. 6- H.264/AVCCh. 6- H.264/AVC
Part I (pp.160~199)Part I (pp.160~199)
Sheng-kai LinSheng-kai Lin
Shengkai Lin@CMLAB, NTUCSIE 2004
2
Outline• Introduction• Baseline profile
– Reference Picture Management– Slice– Inter-prediction– Intra-prediction– Deblocking filter– Transform and Quantisation
Shengkai Lin@CMLAB, NTUCSIE 2004
3
H.264/AVC Introduction (I)
• 4:2:0• Define VCL and NAL
– VCL: Video Coding Layer– NAL: Network abstraction
• Variable block size• Multiple reference frame• ¼ sample motion compensation
Shengkai Lin@CMLAB, NTUCSIE 2004
5
Terms (I)• Picture (frame)
– one or more slices
• Slice– group of macroblock (1~ )– minimal inter-dependency between
coded slices can limit the error propagation
• Macroblock
Shengkai Lin@CMLAB, NTUCSIE 2004
6
Terms (II)• I slice
– I macroblock only
• P slice– P, I macroblock
• B slice– B,I macroblock
• SI slice• SP slice
Shengkai Lin@CMLAB, NTUCSIE 2004
7
Terms (III)• I macroblock
– intra prediction from decoded samples in the current slice• for entire macroblock• for each 4*4 block
• P macroblock– inter prediction using past references
• 16*16, 16*8, 8*16, 8*8, 4*8, 8*4, 4*4
Shengkai Lin@CMLAB, NTUCSIE 2004
8
Terms (IV)• B macroblock
– inter prediction using past/ future references
• list 0– past reference frame buffer
• list 1– future reference frame buffer
• RBSP (Raw Byte Sequence Payload)– NAL payload
Shengkai Lin@CMLAB, NTUCSIE 2004
12
Outline• Introduction• Baseline profile
– Reference Picture Management– Slice– Inter-prediction– Intra-prediction– Deblocking filter– Transform and Quantisation
Shengkai Lin@CMLAB, NTUCSIE 2004
13
Reference Picture Management (I)
• Reference pictures are stored in decoded picture buffer (DPB)
• short/long term reference picture• A decoded frame may be marked as
– unused for reference– short term picture– long term picture
Shengkai Lin@CMLAB, NTUCSIE 2004
14
Reference Picture Management (II)
• “Sliding Window” memory management– Keep #(long_term_pic+ short_term_pic)– Remove short term picture if lack of space
• Adaptive memory control– issued by encoder– change the type of the ref frame
• IDR (Instantaneous Decoder Refresh)– clear ref buffer– I frame
Shengkai Lin@CMLAB, NTUCSIE 2004
15
Outline• Introduction• Baseline profile
– Reference Picture Management– Slice– Inter-prediction– Intra-prediction– Deblocking filter– Transform and Quantisation
Shengkai Lin@CMLAB, NTUCSIE 2004
16
Redundant coded picture
• Send the duplicated part or all of a coded picture
• Error resilience
Shengkai Lin@CMLAB, NTUCSIE 2004
17
Arbitrary Slice Order (ASO)
• The decoding order of the slices could be arbitrary
• Application example– reduce end-end transmission delay in
RT app
Shengkai Lin@CMLAB, NTUCSIE 2004
18
Slice Group (I)• former called “FMO”
– Flexible Macroblock Ordering• a subset of the macroblocks and may
contain one or more slices• Application example
– Error resilience
Shengkai Lin@CMLAB, NTUCSIE 2004
20
Outline• Introduction• Baseline profile
– Reference Picture Management– Slice– Inter-prediction– Intra-prediction– Deblocking filter– Transform and Quantisation
Shengkai Lin@CMLAB, NTUCSIE 2004
21
Inter Prediction• Variable block size• ¼ pixel motion compensation
Shengkai Lin@CMLAB, NTUCSIE 2004
22
Interpolation Samples (I)
1. Generate ½ samples by six tap FIR• b, h, m, s • FIR para :(1/32, -5/32, 5/8, 5/8, -5/32, 1/32)
2. Generate center ½ sample by FIR• j
3. Generate ¼ sample by linear interpolation
Shengkai Lin@CMLAB, NTUCSIE 2004
24
Motion vector prediction (I)
• Encoding MV can cost a lot of bits– especially small partition sizes
• Neighboring MV is highly correlated– predictable– encode the diff MVD and transmit it
Shengkai Lin@CMLAB, NTUCSIE 2004
25
Motion vector prediction (II)
• Partitions excluding 16 × 8 and 8 × 16, MVp = (MVA+ MVB + MVC) /3
• For 16 × 8 partitions, MVp of the upper 16 × 8 = MVB ;MVp for the lower 16 × 8 = MVA
Shengkai Lin@CMLAB, NTUCSIE 2004
26
Motion vector prediction (III)
• For 8 × 16 partitions, MVp for the left 8 × 16 = MVA ;MVp for the right 8 × 16 = MVC
• For skipped macroblocks, do as 16 × 16 Inter mode
Shengkai Lin@CMLAB, NTUCSIE 2004
28
Outline• Introduction• Baseline profile
– Reference Picture Management– Slice– Inter-prediction– Intra-prediction– Deblocking filter– Transform and Quantisation
Shengkai Lin@CMLAB, NTUCSIE 2004
29
Intra Prediction• For Luma samples
– 4*4 block• 9 prediction modes
– 16*16 block• 4 modes
– I_PCM• transmit the encoded samples w/o pred. & tra
ns
Shengkai Lin@CMLAB, NTUCSIE 2004
32
8*8 Chroma Prediction Modes
• Similar to 16*16 luma intra-prediction except the different numbering.– DC(mode 0) – horizontal (mode 1)– vertical(mode 2) – plane (mode 3).
Shengkai Lin@CMLAB, NTUCSIE 2004
33
Signaling Intra Prediction Modes (I)
• The choices of the mode should be signaled to the decoder.– might cost lots of bits.
• Intra modes for neighboring 4 × 4 blocks are often correlated– predict the parameters !
• The prediction mode for luma coded in Intra-16×16 mode or chroma coded in Intra mode is signalled in the macroblock header and the following method is not used.
Shengkai Lin@CMLAB, NTUCSIE 2004
34
Signaling Intra Prediction Modes (II)
• If A and B are available, C = min (A,B)• else if (neither A nor B are available) C = 2
(DC)• else C = available (A,B)
B
CA
Shengkai Lin@CMLAB, NTUCSIE 2004
35
Signaling Intra Prediction Modes (III)
• Flag prev_intra4×4_pred_mode– 1: most probable prediction mode is
used.– 0: Flag rem_intra4×4_pred_mode
indicates the change of the mode
Shengkai Lin@CMLAB, NTUCSIE 2004
36
Signaling Intra Prediction Modes (IV)
• If rem_intra4×4_pred_mode < most_probable_mode, the prediction mode = rem intra4×4 pred mode
• else the prediction mode = (rem_intra4×4_pred_mode+1)
Shengkai Lin@CMLAB, NTUCSIE 2004
37
Outline• Introduction• Baseline profile
– Reference Picture Management– Slice– Inter-prediction– Intra-prediction– Deblocking filter– Transform and Quantisation
Shengkai Lin@CMLAB, NTUCSIE 2004
38
Deblocking filter• Filter 4 vertical/horizontal boundaries of luma• Filter 2 vertical/horizontal boundaries of chroma• Affect up to 3 samples on the either side.• The filter is stronger at places where there is likely
to be significant blocking distortion– e.g.: such as the boundary of an intra coded macroblock
or a boundary between blocks that contain coded coefficients.
Shengkai Lin@CMLAB, NTUCSIE 2004
39
Outline• Introduction• Baseline profile
– Reference Picture Management– Slice– Inter-prediction– Intra-prediction– Deblocking filter– Transform and Quantisation
Shengkai Lin@CMLAB, NTUCSIE 2004
40
Transform and Quantisation
• 3 transforms– DCT-base transform for all 4*4 residual b
lock– Hadamard transform for 4*4 luma DC co
efficient (in 16*16 intra)– Hadamard transform for 2*2 chroma DC
coefficient
Shengkai Lin@CMLAB, NTUCSIE 2004
41
DCT-base transform (I)• Integer transform• Might be zero mismatch• The code part of the algo need add and shift
only• Scale multiplication could be integrated into
quantiser.• Inverse operation could be down by 16-bit ar
ithmetic.
Shengkai Lin@CMLAB, NTUCSIE 2004
43
Quantisation• Combine with the scale of the DCT• Skip the rest details
Shengkai Lin@CMLAB, NTUCSIE 2004
44
4*4 DC luma Transform and Quantisatoion
• |ZD(i, j)| = (|YD(i, j)| MF(0,0) + 2f ) >> (qbits +1)• sign (ZD(i, j)) = sign (YD(i, j))
Shengkai Lin@CMLAB, NTUCSIE 2004
45
2*2 DC chroma Transform and Quantisatoion
• |ZD(i, j)| = (|YD(i, j)| MF(0,0) + 2f ) >> (qbits +1)• sign (ZD(i, j)) = sign (YD(i, j))