CMSC 100 Storing Data: Huffman Codes and Image Representation Professor Marie desJardins Tuesday,...

CMSC 100CMSC 100

Storing Data: Huffman Codes and Image Storing Data: Huffman Codes and Image

RepresentationRepresentation

Professor Marie desJardins

Tuesday, September 18, 2012

Tue 9/18/121CMSC 100 -- Data Compression

Data Compression: Data Compression: MotivationMotivation

Memory is a finite resource: the more data we have, the more space it takes to store Same with bandwidth: the more data we need to send, the

more time it takes

Data compression can reduce space and bandwidth Lossless compression: Store the exact same data in less

space Lossy compression: Store an approximation of the data in

less space


2

Time and Space Time and Space TradeoffsTradeoffs

Data compression trades (computational) time for space and bandwidth: It takes time to convert the original data D to the compressed

format DC

It takes time to convert compressed data DC back to a viewable format D’

Compression ratio:

Space savings:


3 €

CR =Length(DC )

Length(D)

€

SS =1−CR

Lossless vs. Lossy Lossless vs. Lossy CompressionCompression

Lossless: Save space without losing any information Take advantage of repetition and self-similarity (e.g., solid-

color regions in an image)

Lossy: Save space but lose some information Lose resolution or detail (e.g., “pixillate” an image or remove

very high/low frequencies in a sound file)


4

Encoding StrategiesEncoding Strategies Run-length encoding: replace n instances of object x with the

pair of numbers (n,x)

Frequency-dependent encoding: use shorter representations (fewer bits) for objects that appear more frequently in a document

Relative or differential encoding: when x is followed by y, represent y by the difference y-x (which is often small in images etc. and can therefore be represented by a short code)

Dictionary encoding: Create an index of all of the objects (e.g., words) in a document, then replace each object with its index location (can save space if there is a lot of repetition)


5

Image and Sound Image and Sound FormatsFormats

Images Row-by-row bitmaps in different color spaces:

RGB (one byte per color = 24 bits = 17M different colors), a.k.a “True Color” (used in JPEG formats) (How much storage for one True Color 2Kx3K digital camera image?)

Color palette: Use only one byte to index 256 of the 17M 24-bit colors (used in GIF formats) (How much storage for one 24-bit color 200x300 image on a website?)

Variable resolution provides different image sizes and levels of fidelity to an original (continuous or very high-resolution digital) image

Sound Convert continuous sound to digital by sampling (variable-rate) Each sample can be represented with varying levels of resolution (“bit

depth”) (MP3: 44K samples/second, 16 bits/sample – how much storage for one minute of sound?)


6

Compression Ratio: Compression Ratio: ExampleExample

Suppose I have a 2M .PNG (bitmap) image and I store it in a compressed .JPG file that is 187K. What is the compression ratio? What is the space savings?


7

Huffman CodingHuffman Coding Lossless frequency-based encoding

Huffman coding is (space-)optimal in the sense that if we need the exact distribution (frequency) of every object, we will be able to represent the document in the shortest possible number of bits

Downside: It takes a while to compute

Goal #1: Length of each object should be related to its frequency Specifically: length is proportion to the negative log of the frequency

Goal #2: Code should be unambiguous Since objects will be encoded at different lengths, as we read the

bits, we need to know when we’ve reached the end of one object and should begin processing the next one

This type of code is called a prefix code


8

Using a Prefix CodeUsing a Prefix Code


9

A E

LH O

SC

How would you represent“HELLO” using this code?

0 1

Note: By convention, the left branch is 0;the right branch is 1

0 1 0 1

0 1

0 1

0 1

Interpreting a Prefix Interpreting a Prefix CodeCode


10

A E

LH O

SC

What does “1110000110110111110”mean in this code?

0 1

0 1 0 1

0 1

0 1

0 1



11

A E

LH O

SC

What does “1110000110110111110”mean in this code?

0 1

0 1 0 1

0 1

0 1

0 1

C



12

A E

LH O

SC

What does “1110 | 000110110111110”mean in this code?

0 1

0 1 0 1

0 1

0 1

0 1

C



13

A E

LH O

SC

What does “1110 | 000110110111110”mean in this code?

0 1

0 1 0 1

0 1

0 1

0 1

C H



14

A E

LH O

SC

What does “1110 | 000 | 110 | 110 | 1111 | 10”mean in this code?

0 1

0 1 0 1

0 1

0 1

0 1

C H O O S E

A OSPC

TL W

C! PM US

Y

RE

Decode the Message:

0111110010100101011011100011110111110110 010 00111111110 010

0110001110 010 0110001110 010 0110001110 010

0001100000100100000000110 010 011111001000000 01110

0 1

Tue 9/18/12

15CMSC 100 -- Data Compression

0 1

0 1

0 1

0 1 0 1

0 1

0 1

0 10 10 1

0 1 0 1 0 1

Encoding AlgorithmEncoding Algorithm Frequency distribution:

Set of k objects, o1...ok

Number of times of each object appears in the document, n1...nk

Construct a Huffman code as follows:n Pick the two least frequent objects, oi and oj

n Replace them with a single combined object, oij, with frequency ni+nj

n If there are at least two objects left, go to step 1

Visually:1. Each of the original objects is a leaf (bottom node) in the prefix tree

2. Each combined objects represents a 0/1 split where the “children” are the two objects that were combined

3. In the last step, we combine two subtrees into a single final prefix tree


16

Encoding ExampleEncoding Example SHE SELLS SEASHELLS BY THE SEASHORE


17

Encoding ExampleEncoding Example SHE SELLS SEASHELLS BY THE SEASHORE

Frequency distribution: A – 2 B – 1 E – 7 H – 4 L – 4 O – 1 R – 1 S – 8 T – 1 Y – 1 <SPC> – 5


18

Encoding ExampleEncoding Example


19

2

O1B1



20

2

O1B1

2

T1R1

3

Y1A2



21

2

O1B1

2

T1R1

3

Y1A2

4

7



22

2

O1B1

2

T1R1

3

Y1A2

4

7 8

L4H4

CMSC 100 -- Data Compression


Tue 9/18/12

23

2

O1B1

2

T1R1

3

Y1A2

4

7 8

L4H4

12

E7_5



Tue 9/18/12

24

2

O1B1

2

T1R1

3

Y1A2

4

7 8

L4H4

12

E7_5

15



Tue 9/18/12

25

2

O1B1

2

T1R1

3

Y1A2

4

8

L4H4

12

E7_5

15

20

S87

35

Green Eggs and HamGreen Eggs and Ham


26

Green Eggs and HamGreen Eggs and Ham

I am Sam

I am Sam

Sam I am

That Sam-I-am!

That Sam-I-am!

I do not like

that Sam-I-am!

Do you like

green eggs and ham?

I do not like them,

Sam-I-am.

I do not like

green eggs and ham. Tue 9/18/12

27


Symbols (not letters!) are words.Ignore spaces and punctuation.

Date post:	03-Jan-2016
Category:	Documents
Upload:	chad-potter
View:	216 times
Download:	0 times

CMSC 100 Storing Data: Huffman Codes and Image Representation Professor Marie desJardins Tuesday,...

Documents