April13,2009Slide1
HowCompressionWorks
HowdyPierceCardinalPeak,LLC
April13,2009Slide2
VideoCompression
• Uncompressed,full‐colorvideoisBIG• FullD1video(720x480@30fps)
720x480x24bppx29.97fps=248Mbpsofdata!
April13,2009Slide3
VideoCompression
• Uncompressed,full‐colorvideoisBIG• FullD1video(720x480@30fps)
720x480x24bppx29.97fps=248Mbpsofdata!
• Subsampledcolor(4:2:0) 720x480x12bppx29.97fps=124Mbps
• Tobeviablecommercially,weneedtogetthisintothe1.0to10.0Mbpsrange
April13,2009Slide4
2typesofcompression
• Lossless–getbackwhatyouputin• Lossy–getbacksomethingclosetowhatyouputin
April13,2009Slide5
HowManyBits?
• Howmanybitsshouldittaketorepresenta32‐symbolalphabet?
SPC F Q' G R, H S‐ I T. J U; K VA L WB M XC N YD O ZE P
April13,2009Slide6
HowManyBits?
• Weknowsomethingaboutthedistribu]on
Distribu]onbasedonWutheringHeightsbyEmilyBrönte
Sym P Sym P Sym PSPC 17.28% F 1.69% Q 0.07%' 1.08% G 1.67% R 4.66%, 1.54% H 5.09% S 4.78%‐ 0.32% I 5.64% T 6.82%. 0.82% J 0.10% U 2.34%; 0.29% K 0.62% V 0.72%A 6.09% L 3.24% W 1.66%B 1.09% M 2.08% X 0.13%C 1.90% N 5.64% Y 1.70%D 3.76% O 5.83% Z 0.03%E 10.05% P 1.26%
April13,2009Slide7
HuffmanCode
S RE
V KPW ,
I N H
Q ZJ
F GLT A O
X ‐ ;
.YD
B ‘UM C
SPC
April13,2009Slide8
HuffmanCode
S RE
V KPW ,
I N H
Q ZJ
F GLT A O
X ‐ ;
.YD
B ‘UM C
SPC
0 1
0 1 01
0
0
1
1
0
0
0
0
0
0
0 11
1
1
11
Sowewouldcodeeachsymbolasfollows:
001 E111 SPC0000 S0001 R0100000 V
…etc
April13,2009Slide9
HuffmanCode
S RE
V KPW ,
I N H
Q ZJ
F GLT A O
X ‐ ;
.YD
B ‘UM C
SPC
0 1
0 1 01
0
0
1
1
0
0
0
0
0
0
0 11
1
1
11
3bits4bits5bits6bits7bits
9bits10bits11bits
April13,2009Slide10
EntropyCoding(akaVariableLengthCoding)
• Weendupusingfewerbitsformorecommonleners,andmorebitsforlesscommonleners
Weightedaveragenumberofbitspersymbol:4.29
Sym P bits Sym P bits Sym P bitsSPC 17.28% 3 F 1.69% 6 Q 0.07% 11' 1.08% 7 G 1.67% 6 R 4.66% 4, 1.54% 6 H 5.09% 4 S 4.78% 4‐ 0.32% 9 I 5.64% 4 T 6.82% 4. 0.82% 7 J 0.10% 10 U 2.34% 6; 0.29% 9 K 0.62% 7 V 0.72% 7A 6.09% 4 L 3.24% 5 W 1.66% 6B 1.09% 7 M 2.08% 6 X 0.13% 9C 1.90% 6 N 5.64% 4 Y 1.70% 6D 3.76% 5 O 5.83% 4 Z 0.03% 11E 10.05% 3 P 1.26% 6
April13,2009Slide11
ClaudeShannon(1916‐2001)
• “Fatherofinforma]ontheory”
• Laidthetheore]calgroundworkfordatacompressionandmodula]on
April13,2009Slide12
EntropyCoding:Theory
• AccordingtoShannon’sSourceCodingTheory,theop]mallengthforacodeis
−logb(P)
• Where
Pistheprobabilityforthesymbolbeingcoded
bisthenumberofsymbolsusedtomaketheoutputcodes(i.e.,always2forbinary)
April13,2009Slide13
EntropyCoding
• HowclosedoesourHuffmancodecome?
Weightedaveragenumberofbitspersymbol:4.29Weightedaveragebitspersymbol,theore]callimit:4.26
Sym P bits log2(P) Sym P bits log2(P) Sym P bits log2(P)SPC 17.28% 3 2.53 F 1.69% 6 5.88 Q 0.07% 11 10.40' 1.08% 7 6.53 G 1.67% 6 5.90 R 4.66% 4 4.42, 1.54% 6 6.02 H 5.09% 4 4.30 S 4.78% 4 4.39‐ 0.32% 9 8.30 I 5.64% 4 4.15 T 6.82% 4 3.87. 0.82% 7 6.93 J 0.10% 10 10.00 U 2.34% 6 5.41; 0.29% 9 8.43 K 0.62% 7 7.34 V 0.72% 7 7.12A 6.09% 4 4.04 L 3.24% 5 4.95 W 1.66% 6 5.91B 1.09% 7 6.51 M 2.08% 6 5.59 X 0.13% 9 9.59C 1.90% 6 5.72 N 5.64% 4 4.15 Y 1.70% 6 5.88D 3.76% 5 4.73 O 5.83% 4 4.10 Z 0.03% 11 11.67E 10.05% 3 3.31 P 1.26% 6 6.31
April13,2009Slide14
EntropyCoding:Theory
• Entropycodingislossless• ItturnsoutthatHuffmancodesareop]mal– IFrestrictedtocodingonesymbolata]me
• However,itispossibletodobenerbycompressingmorethanonesymbolata]me– Arithme]ccoding
April13,2009Slide15
HowtoCreateaHuffmanCode
• Orderthesymbolsbyprobability
• Lowest2symbolsbecomeasub‐tree
• Iterateun]lallsymbolsarehandled
April13,2009Slide16
HuffmanCodeExercise
• MakeyourownHuffmancodewiththissetofsymbols:
Sym PN. 16.05%NE. 4.17%E. 32.26%SE. 8.19%S. 12.20%SW. 4.65%W. 8.03%NW. 14.45%
April13,2009Slide17
HuffmanCodeExercise
NW NE
SW NESSE W
Weightedaveragenumberofbitspersymbol:2.77Weightedaveragebitspersymbol,theore]callimit:2.71
Sym P bits log2(P)N. 16.05% 3 2.64NE. 4.17% 4 4.58E. 32.26% 2 1.63SE. 8.19% 3 3.61S. 12.20% 3 3.04SW. 4.65% 4 4.43W. 8.03% 3 3.64NW. 14.45% 3 2.79
April13,2009Slide18
VideoandImageCompression
• Mostcommonformsofvideoandimagecompressionhavethesamecentralconcept
– JPEG– MPEG‐1– MPEG‐2
– MPEG‐4– H.264
• Obviously,therearedifferencestoo
April13,2009Slide19
VideoandImageCompression8x8block
April13,2009Slide20
VideoandImageCompression8x8block
101 100 94 102 97 91 88 83101 99 98 103 93 93 107 11098 97 97 97 103 101 94 10097 98 99 100 103 105 101 9699 100 104 104 100 107 109 8999 101 106 105 116 113 87 5894 69 70 66 79 70 40 2659 30 27 33 32 37 45 41
SourceMatrix
April13,2009Slide21
HowManyBits?
• Howmanybitsshouldittaketocodethisblock?
SourceMatrix
April13,2009Slide22
3KeyStepsofImageCompression
1. DiscreteCosineTransform2. Quan]za]on3. EntropyCoding
DCT Quan]za]on EntropyCoding8x8block
codeddata
April13,2009Slide23
DiscreteCosineTransform
• Transformsonematrixintoanothermatrix
• Reversible&lossless
• Thetransformedmatrixhassomeinteres]ngproper]es
– Mostinforma]onrepresentedinthelow‐ordercoefficients
April13,2009Slide24
DiscreteCosineTransform
DCT
SourceMatrix
DCTCoefficients
April13,2009Slide25
DiscreteCosineTransform
DCTthe“DCcoefficient”(avgofini]almatrix)
Increasinglyhigherfrequenciesin
ver]caldomain
Increasinglyhigherfrequenciesinhorizontaldomain
DCTCoefficients
SourceMatrix
April13,2009Slide26
3KeyStepsofImageCompression
1. DiscreteCosineTransform2. Quan]za]on3. EntropyCoding
DCT Quan]za]on EntropyCoding8x8block
codeddata
April13,2009Slide27
Quanfzafon
OriginalValues Quan]zedValues
• Quan]za]onisthelossystepinimagecompression
April13,2009Slide28
Quanfzafon
• Forimagecompression,therearetwostepstoquan]za]on:– Applyaweigh]ngmatrixtoDCTcoefficients
– ApplyauniformscalefactorMPEG‐2DefaultWeigh]ngMatrix
April13,2009Slide29
AherQuanfzing
DCTCoefficients
Quan]zedwithMPEG‐2weigh]ngmatrixandscale=8
Quan]za]on
April13,2009Slide30
AherQuanfzing
DCTCoefficients
Quan]zedwithMPEG‐2weigh]ngmatrixandscale=8
Quan]za]on
Thisisthedataweneedtotransmitto
thedecoder
April13,2009Slide31
ReversingtheProcess
DCTQuan]za]on
(q=8)Inverse
Quan]za]onInverseDCT
OriginalSourceMatrix RecoveredOutput(q=8)
April13,2009Slide32
HowLossyIsIt?OriginalSourceMatrix RecoveredOutput(q=8)
April13,2009Slide33
HowLossyIsIt?OriginalSourceMatrix RecoveredOutput(q=8)
ErrorMatrix
SumofSquaresoferrormatrix:4335
April13,2009Slide34
ComparingQuanfzafonFactors
Quan]zedwithMPEG‐2weigh]ngmatrixandscale=8
• Sum‐of‐squareserror:4335• Numberofnon‐zerocoefficients:11
Quan]zedwithMPEG‐2weigh]ngmatrixandscale=1
• Sum‐of‐squareserror:231• Numberofnon‐zerocoefficients:39
SameDCTCoefficients
April13,2009Slide35
3KeyStepsofImageCompression
1. DiscreteCosineTransform2. Quan]za]on3. EntropyCoding
DCT Quan]za]on EntropyCoding8x8block
codeddata
April13,2009Slide36
Run‐LengthCoding
• Applyazig‐zagscanpanern: 86,1,7,5,‐1,0,1,0,0,2,‐1,1,0,‐1,0,0,0,0,‐1,0,0,…
• Notethattherearealotofrunsofzeros!• Breakinto“words”ofXzeros,followedbyacoefficient
Quan]zeddatawithMPEG‐2weigh]ngmatrixandscale=8
April13,2009Slide37
Run‐LengthCoding
• TheDCcoefficient(86)issentdirectly* =8bits
• Remainingcoefficientsaresentas10“words”,plusaspecialend‐of‐blockword
86,1,7,5,‐1,0,1,0,0,2,‐1,1,0,‐1,0,0,0,0,‐1,0,0,…
(*)Inprac]ce,it’smuchmorecomplexthanthis,butforsakeofsimplicity…
April13,2009Slide38
MPEG‐2HuffmanTable
April13,2009Slide39
Run‐LengthCoding
• ApplyingtheMPEG‐2Huffmancodetothissetofwords,weconsume52bits
• So,wecodetheen]re8x8blockin60bits!
April13,2009Slide40
ContextAdapfveVLC(CAVLC)
• OneproblemwiththeMPEG‐2approach:– Huffmantablesarefixed• Reflectthedistribu]onofwordsinsomesmallsampleofsourcematerialwhenthestandardwaswrinen
– Ifthisdoesn’treflectthedistribu]onofwordsinthevideobeingencoded,wegetinefficiency
• H.264improvesonthisbyhaving5differentVLCtables– Selectedcontextuallybasedon#ofnon‐zerocoefficientsinneighboringblocks
April13,2009Slide41
ThankYou!