CS 414 - Spring 2012 CS 414 – Multimedia Systems Design Lecture 4 – Visual Perception and...

CS 414 - Spring 2012

CS 414 – Multimedia Systems Design Lecture 4 – Visual Perception and Digital Image Representation

Klara Nahrstedt

Spring 2012


Administrative

Groups are formed and names have been sent to TSG and Barb Leisner

We will inform you about group directories as soon as we have information from TSG

Administrative Leasing Process from Barb Leisner

Lease one Logitech camera - two cameras within one group to start MP1, and then for MP2/MP3.

Leasing process starts on January 25 Pick up the camera from Barb Leisner office, 2312 SC Bring your student ID to sign for the camera Each cs414 student is responsible for his/her own camera

if you loose it (or badly damage) and you don’t have police report, you pay for it (charged to your student account at the end of the semester)

Hours to pick up camera: Monday –Friday 9am-5pm No camera pickup on Saturday and Sunday


Today Introduced Concepts

Two Important Metrics for Digital AudioSignal-to-Noise Ratio (dB) Digital Audio Data Rates (bits per second)

Human Visual System Digital Images

SamplingQuantizationSpatial Resolution


Signal-to-Noise Ratio(metric to quantify quality of digital audio)


Signal To Noise (SNR) Ratio

Measures strength of signal to noise

SNR (in DB)=

Given sound form with amplitude in [-A, A]

Signal energy =

)(log10 10energyNoise

energySignal

A

0

-A

2

2A


Modeling of Noise - Quantization Error Difference between actual and sampled value

amplitude between [-A, A] quantization levels = N

e.g., if A = 1,N = 8, = 1/4

N

A2


Compute Signal to Noise Ratio

Signal energy = ; Noise energy = ;

Noise energy =

Signal-to-Noise =

SNR depends on number of bits (number of quantization levels) assigned to signal

Every bit increases SNR by ~ 6 decibels

12

2N

A2

2

2

3 N

A

2

3log10

2N

2

2A


Audio Data Rate Data rate = best sample rate * quantization (bits

per sample) * channel

Derived from Nyquist Data Rate = 2*H log2N, where H is basic sampling

rate, N is number of quantization levels

Compare rates for CD vs. mono audio 8000 samples/second * 8 bits/sample * 1 channel

= 8 kBytes / second 44,100 samples/second * 16 bits/sample *

2 channel = 176 kBytes / second ~= 10MB / minute


Integrating Aspects of Multimedia


Image/VideoCapture

Image/Video InformationRepresentation

MediaServerStorage

Transmission

CompressionProcessing

Audio/VideoPresentationPlaybackAudio/Video

Perception/ Playback

Audio InformationRepresentation

Transmission

AudioCapture

A/V Playback

Human Visual System

Eyes, optic nerve, parts of the brain

Transforms electromagnetic energy

Human Visual System

Image Formation cornea, sclera, pupil,

iris, lens, retina, fovea Transduction

retina, rods, and cones Retina has photosensitive

receptors at back of eye Processing

optic nerve, brain

Rods vs Cones (Responsible for us seeing brightness and color)

Contain photo-pigment Respond to low energy Enhance sensitivity Concentrated in retina,

but outside of fovea One type, sensitive to

grayscale changes

Contain photo-pigment Respond to high energy Enhance perception Concentrated in fovea,

exist sparsely in retina Three types, sensitive to

different wavelengths

Cones Rods


Tri-stimulus Theory 3 types of cones (6/7 Mil. of them)

Red = L cones, Green = M cones, Blue = S cones Ratio differentiates for each person E.g., Red (64%), Green (32%), rest S cones E.g., L(50.6%), M(44.2%), rest S cones

Each type most responsive to a narrow band electro-magnetic waves red and green absorb most energy, blue the least

Light stimulates each set of cones differently, and the ratios produce sensation of color


Color and Visual System Color refers to how we

perceive a narrow band of electromagnetic energy source, object, observer

Visual system transforms light energy into sensory experience of sight

Color Perception (Color Theory) Hue

Refers to pure colors dominant wavelength of the

light Saturation

Perceived intensity of a specific color

how far color is from a gray of equal intensity

Brightness (lightness) perceived intensity


Hue Scale

Saturation

Original

lightness

Source: Wikipedia

Color Perception (Hue)


Relation between “hue” of colors (spectrum of colors) with maximal saturation in HSV and HSL with Their corresponding RGB coordinates

HSV = Hue, Saturation, Value (Lightness)HSL = Hue, Saturation, Lightness

Digitalization of Images – Capturing and Processing


Capturing Real-World Images

Picture – two dimensional image captured from a real-world scene that represents a momentary event from the 3D spatial world


W3

W1W2

r

s

F r= function of (W1/W3); s=function of (W2/W3)

Image Concepts - Sampling

An image is a function of intensity values over a 2D plane I(r,s)

Sample function at discrete intervals to represent an image in digital formmatrix of intensity values for each color plane intensity typically represented with 8 bits

Sample points are called pixels


Digital Image Sampling

Sample = pixel

Image Size (in pixels)

Image Size = Height x Width (in pixels) 320x240 pixels 640x480 pixels 1920x1080pixels


Digital Images - Quantization

Quantization = number of bits per pixel Example: if we would sample and quantize

standard TV picture (525 lines) by using VGA (Video Graphics Array), video controller creates matrix 640x480pixels,

and each pixel is represented by 8 bit integer (256

discrete gray levels)


Image Representations Black and white image

single color plane with 2 bits Grey scale image

single color plane with 8 bits Color image

three color planes each with 8 bits

RGB, CMY, YIQ, etc. Indexed color image

single plane that indexes a color table

Compressed images TIFF, JPEG, BMP, etc. 2gray levels4 gray levels

Digital Image Representation (3 Bit Quantization)


Color QuantizationExample of 24 bit RGB Image


24-bit Color Monitor

Image Representation Example

128 135 166 138 190 132

129 255 105 189 167 190

229 213 134 111 138 187

135 190

255 167

213 138

128 138

129 189

229 111

166 132

105 190

134 187

24 bit RGB Representation (uncompressed)

Color Planes

Graphical Representation


Image Properties (Color)


Color Histogram


Spatial and Frequency Domains Spatial domain

refers to planar region of intensity values at time t

Frequency domain think of each color plane

as a sinusoidal function of changing intensity values

refers to organizing pixels according to their changing intensity (frequency)


Spatial Resolution and Brightness Spatial Resolution

(depends on: ) Image size Viewing distance

Brightness Perception of brightness

is higher than perception of color

Different perception of primary colors

Relative brightness: green:red:blue=

59%:30%:11%

CS 414 - Spring 2011Source: wikipedia

Image Size (in Bits)

Image Size = Height x Width X Bits/pixel

Example: Consider image 320x240 pixels with 8 bits per

pixel Image takes storage 7680 x 8 bits = 61440

bits or 7680 bytes


Summary Important Image Processing Functions (see

Computer Vision/Image Processing classes) Filtering Edge detection Image segmentation Image recognition

Formatting Conditioning Marking Grouping Extraction Matching

Image synthesis


Date post:	19-Jan-2016
Category:	Documents
Upload:	ginger-shepherd
View:	219 times
Download:	5 times

CS 414 - Spring 2012 CS 414 – Multimedia Systems Design Lecture 4 – Visual Perception and...

Documents