1

Video Compression

CSC 9YH

Evolution of video mediums

• Film– Invented in late 18th

century, still widely used today

• VHS– Released in 1976,

rapidly disappearing

2

Evolution of video mediums

• DVD– Released in 1996,

dominant for over a decade

• Hard Disk– Around for many

years, only recently widely used for storing video (helped by explosion of Internet)

Transition from analog mediums to digital mediums

• The “N word”– Analog signals are

prone to corruption by noise

• Economics– Optical media is

cheaper to produce than magnetic media

• Creates need to convert analog video to digital format

3

Video digitization

• Newer digital video cameras have on-board hardware to capture directly to digital format

• Older film can be scanned with special machines to produce digital stream

Real time video encoding

• Motion estimation will be worse, so need higher bitrate to compensate

• Very hard to do in software, need dedicated hardware or hardware assistance

• Tivo, iPlayer ‘live’ do this

4

Video Encoding/Compression• Once video is in digital format, it makes sense to

compress it

• Similarly to image compression, we want to store video data as efficiently as possible

• Again, we want to both maximize quality and minimize storage space and processing resources

• This time, we can exploit correlation in both space and time domains

TMI! (Too Much Information)

• Unlike image encoding, video encoding is rarely done in lossless form

• No storage medium has enough capacity to store a practical sized lossless video file– Lossless DVD video - 221 Mbps– Compressed DVD video - 4 Mbps– 50:1 compression ratio!

5

Definitions

• Bitrate– Information stored/transmitted per unit time– Usually measured in Mbps (Megabits per second)– Ranges from < 1 Mbps to > 40 Mbps

• Resolution– Number of pixels per frame– Ranges from 160x120 to 1920x1080

• FPS (frames per second)– Usually 24, 25, 30, or 60– Don’t need more because of limitations of the human

eye

Scan types

• Interlaced scan– Odd and even lines

displayed on alternate frames

– Initially used to save bandwidth on TV transmission

– When displaying interlaced video on a progressive scan display, can see “comb effect”

6

Scan types• Progressive scan

– Display all lines on each frame

– New “fixed-resolution”displays (such as LCD, Plasma) all use progressive scan

– Deinterlacing is not a trivial task

MPEG (Moving Pictures Expert Group)

• Committee of experts that develops video encoding standards

• Until recently, was the only game in town (still the most popular, by far)

• Suitable for wide range of videos– Low resolution to high resolution– Slow movement to fast action

• Can be implemented either in software or hardware

7

Bitrate allocation

• CBR – Constant BitRate– Streaming media uses this (but is changing!)– Easier to implement

• VBR – Variable BitRate– DVD’s use this– Usually requires 2-pass coding– Allocate more bits for complex scenes– This is worth it, because you assume that you encode

once, decode many times

Lossless Compression

• Run Length Encoding (RLE):– aaaaaaabbbb

• 7a4b

– abababababa• 1a1b1a1b1a1b1a1b1a1b1a

– Lossless compression relies on input being non-random to achieve compression.

8

Lossy Compression

• Removes information• Does so “intelligently”• For media files, remove what is least

noticeable by the senses.

JPEG Compression• Lossy (JPEG-2000 can be lossless)• JFIF file format and JPEG compression• 8x8 blocks matched to

combinations of 64 basic patterns

• ‘Quality’ determineshow hard it tries to match

9

JPEG Compression

Video Compression

• Series of compressed images (JPEG)• But can make smaller by compressing

what’s common between frames• Only save the information that’s changed

between frames• Match macroblocks to previous and

possibly next frames

10

MPEG:AVideoCompressionStandardforMultimedia

Applications

Largely based on Communications of the ACMVolume 34, Number 4

Pages 46-58

Outline

• Introduction• MPEG Goals• MPEG Details• Performance and Such• Summary

11

The Need for Video Compression• High-Definition Television (HDTV)

– 1920x1080 – 30 frames per second (full motion)– 8 bits for each three primary colorsàTotal 1.5 Gb/sec!

• Each cable channel is 6 MHz– Max data rate of 19.2 Mb/sec– Reduced to 18 Mb/sec w/audio + control …àCompression rate must be 83:1!

Compatibility Goals

• CD-ROM and DAT key storage devices– 1-2 Mbits/sec for 1x CD-ROM

• Two types of application videos:– Asymmetric (encoded once, decoded many)

• Video games, Video on Demand– Symmetric (encoded once, decoded once)

• Video phone, video mail …• (How do you think the two types might influence

design?)• Video at about 1.5 Mbits/sec• Audio at about 64-192 kbits/channel

12

Requirements• Random Access, Reverse, Fast Forward, Search

– At any point in the stream– Can reduce quality somewhat during task, if

needed• Audio/Video Synchronization

– Even when under two different clocks• Robustness to errors

– Not catastrophic if bits lost• Coding/Decoding delay under 150ms

– For interactive applications• Editability

– Modify/Replace frames

MPEG Compression

• Compression through– Spatial– Temporal

13

Spatial Redundancy• Take advantage of similarity among

most neighboring pixels

Spatial Redundancy Reduction• RGB to YUV

– less information required, same visually• Macro Blocks

– Take groups of pixels• DCT

– Represent pixels in blocks with fewer numbers

• Quantization– Reduce data required for co-efficients

• Entropy coding– Compress

14

Spatial Redundancy Reduction

Zig-Zag Scan,Run-length

coding

Quantization• major reduction• controls ‘quality’

“Intra-FrameEncoded”

Groupwork

• When may spatial redundancy reduction be ineffective? What kinds of images/movies?

15

Groupwork

• When may spatial redundancy reduction be ineffective?– High-resolution images and displays

• May appear ‘coarse’– A varied image or ‘busy’ scene

• Many colors, few adjacent

Loss of ResolutionOriginal (63 kb)

Low (7kb)

Very Low (4 kb)

16

Temporal Redundancy• Take advantage of similarity between

successive frames

950 951 952

(Simpsons News Clip Here)

“Talking Head”

Temporal Activity

17

Temporal Redundancy Reduction

Temporal Redundancy Reduction

18

Temporal Redundancy Reduction

• I frames are independently encoded• P frames are based on previous I, P frames• B frames are based on previous and following I and P

frames– In case something is uncovered

Group of Pictures (GOP)

• Starts with an I-frame• Ends with frame right before next I-frame• “Open” ends in B-frame, “Closed” in P-

frame– (What is the difference?)

• MPEG Encoding parameter, but ‘typical’:– I B B P B B P B B I– I B B P B B P B B P B B I

• Why not have all P and B frames?

19

Groupwork

• When may temporal redundancy reduction be ineffective?

Groupwork

• When may temporal redundancy reduction be ineffective?– Many scene changes– High motion

20

Non-Temporal Redundancy

• Many scene changes

(“Mixbag clip here)

MPEG Layers

• Sequence Layer• Group of Pictures Layer

21

Typical MPEG Parameters

Typical Compression PerformanceType Size Compression---------------------

I 18 KB 7:1

P 6 KB 20:1

B 2.5 KB 50:1

Avg 4.8 KB 27:1

---------------------

• Note, results in Variable Bit Rate, even if frame rate is constant

22

MPEG Today

• MPEG video compression widely used– digital television set-top boxes– HDTV decoders– DVD players– video conferencing– Internet video– ...

MPEG Today• MPEG-2

– Super-set of MPEG-1– Rates up to 10 Mbps (720x486)– Can do HDTV (no MPEG-3)

• MPEG-4– Around Objects, not Frames– Lower bandwidth

• MPEG-7– Not (yet) a standard– Allows content-description (ease of searching)

• MP3– For audio– MPEG Layer-3