1 Light and Color. 2/118 Topics The Human Visual System Displaying Intensity and Luminance Display...

transcript

Light and Color

Topics

The Human Visual System Displaying Intensity and Luminance Display Using Fixed Intensities Understanding Color Display of Color Color Models

Structure of the Human Eye

Image taken from http://hyperphysics.phy-astr.gsu.edu/hbase/vision/eye.html

Main Parts of the Eye

Cornea

Cornea Provides most refraction

Cornea Iris

Opens and Closes to let in more/less light

Cornea Iris

Opens and Closes to let in more/less lightHole is the pupil

Cornea Iris Lens

10/118

Cornea Iris Lens

Flexible - muscles adjust shape

11/118

Cornea Iris Lens

Flexible - muscles adjust shapeAllows fine-detail focus

12/118

Cornea Iris Lens Retina

13/118

Layer of receptor cells at back of eye

14/118

Layer of receptor cells at back of eye Center of focus is the fovea

15/118

Layer of receptor cells at back of eye Center of focus is the fovea Optic nerve collects information from retina, and

brings to the back of the brainCauses a blind spot!Cephalopods (octopus) do not have blind spots

16/118

Focusing Light

For a point in focal plane, all light emitting from that point goes to same point on retina

17/118

Focusing Light

For a point not in focal plane, light gets spread across retina

18/118

Display Screens and Focus

For the usual (current) display systems, we maintain one focus planeEven for stereo displaysEven if the image tries to simulate

differing focusThis is different than nature…

19/118

Distinguish brightness only Best response to blue-green light Prevalent except at fovea (more peripheral) About 100x more sensitive than cones About 100 million in retina

20/118

Color response Centered around fovea

About 147,000 cones/mm2 at fovea2mm away from fovea: 9,500 cones/mm2

6.3 - 6.8 million in retina

21/118

How We See

Individual receptors give response Vision is limited by density of cells in part of

brainHigh detail, good color at center of visionPeripheral vision can see dimmer light, but

mainly black & white, and low resolution

22/118

Vision in the Brain

Signals are carried by optic nerve to brain Brain processes to reconstruct image

Best understood part of brain function, but still many ill-understood parts

Many aspects of vision are “hard-wired”Can lead to optical effects with significant

graphics impact• Mach banding.• Filling in of “blind spot”

23/118

Topics

24/118

Intensity and Luminance

Intensity/Luminance: How much light energy there isThe amount of energy carried by photons

25/118

Intensity and Luminance

Intensity/Luminance: How much light energy there isThe amount of energy carried by photons

Brightness: the perceived intensityEye does not respond to equal intensity

changes equallyEye notices the ratio of intensities

26/118

Brightness Levels

Intensity changes of 1->2, 2->4, 4->8 appear the sameExample: 3-way lightbulb

50->100 seems like bigger change than 100 -> 150

Display devices might limit the number of discrete intensity levels available

27/118

Brightness in Display Devices

Assume:The maximum intensity of a display is 1.0The minimum is I0

It can display n+1 intensity levels Then, to get equal brightness increments, it

should display levels: I0, rI0, r2I0, …, rnI0=1.0

28/118

Brightness Levels

Human eye generally can notice r>1.01 So, to have a “smooth” display, we need to make

sure that r<1.01i.e. we need enough “levels” of display

For a given display, we need

1.01nI0=1.0

1.01n = 1.0/I0

n log 1.01 = - log I0

n = -log I0 / log 1.01

n = -log1.01I0

29/118

Gamma Correction

Designed to compensate for how humans perceive intensity of light Iout = k Iin

Iin, Iout = intensityk, are device-specific terms

Typically, = 2.0 to 2.5 (most displays use 2.2) Combining colors/intensity is not linear! Must

convert to linear space, combine, convert back

30/118

Original 1080p

31/118

YouTube 360p

32/118

Gamma Corrected 360p

33/118

Original 1080p

34/118

Gamma and non-CRT Displays

Gamma is meant to model CRTs only Other displays (e.g. LCD) may have very different

response curves between intensity and value sent They do not match the gamma curve Often manufacturers adjust responses to try to

mimic CRT behavior Rarely is any device’s response curve exactly

what is desired/modeled This can get much more complicated

Major effort just to compensate for gamma

35/118

Dynamic Range

1/I0 is called dynamic range The ratio of maximum to minimum intensity

(remember, we set 1.0 = max) Varies by display device This is the maximum a device can possibly

display vs. the minimum it can display.

36/118

Dynamic Range

1/I0 is called dynamic range The ratio of maximum to minimum intensity

(remember, we set 1.0 = max) Varies by display device This is the maximum a device can possibly

display vs. the minimum it can display. Contrast: the maximum vs. minimum it can display

at the same time. i.e. for one image on screen, maximum vs.

minimum.

37/118

Topics

38/118

Limited Display Levels

We can’t always get a continuous range of display levels Printing: either ink is there or not there

We can adjust amount of ink (i.e. how much space it takes), but not its intensity

Other display might limit the number of levels – e.g. 256 levels of intensity.

We need ways to mimic continuous colors with discrete levels

39/118

Example

40/118

Example

41/118

Halftoning/Dithering

Idea: Eyes integrate over an areaAll light hitting one receptor cell is

combined.Eye only cares about the integrated

information from an entire area So, we can get varying intensity by filling in

fractions of areas

42/118

Halftoning

Subdivide image into blocks of pixels. You will lose resolution! e.g. a 100x100 region divided into 4x4 blocks of

pixels can only display a 25x25 image. The number of intensity levels is related to the

number of pixels in a block 2x2 = 5 intensity levels 3x3 = 10 intensity levels nxn = n2+1 intensity levels

43/118

Example: 2x2 block

Level 0: Intensities 0.0 – 0.2

44/118

Example Image

0.00 0.20 1.00 0.02 0.50 0.20

0.15 0.05 0.01 0.05 0.08 0.90

0.20 0.25 0.25 0.45 0.70 0.60

0.22 0.25 0.60 0.90 0.50 0.80

0.30 0.50 0.80 0.90 1.00 0.90

0.40 0.20 0.61 0.61 0.75 0.95

45/118

Example Image

0.00 0.20 1.00 0.02 0.50 0.20

0.15 0.05 0.01 0.05 0.08 0.90

0.20 0.25 0.25 0.45 0.70 0.60

0.22 0.25 0.60 0.90 0.50 0.80

0.30 0.50 0.80 0.90 1.00 0.90

0.40 0.20 0.61 0.61 0.75 0.95

0.10 0.27 0.42

0.23 0.55 0.65

0.35 0.73 0.90

46/118

Example Image

Average Intensities over Blocks Halftone Image

47/118

2x2 Halftone Example

48/118

49/118

50/118

Halftoning Patterns

The pattern you use to fill makes a difference.Want a “random” pattern, so that artificial

artifacts don’t appearBrain is very good at recognizing some

things, like lines

51/118

Example: 3x3 block

Bad pattern:

Better pattern:

.36 .42

.45 .51

.36 .42

.45 .51

52/118

3x3 Bad Halftone Example

53/118

3x3 Good Halftone Example

54/118

Dithering

Halftoning loses resolution – this is bad. Dithering:

Keep same resolution Change halftoning pattern into a

probability/threshold function (dither pattern) Overlay dither pattern on entire image Fill in a pixel iff it is of higher intensity than the

dither value

55/118

Dither Pattern Example

Intensities 0.0 – 0.2

Halftoning Patterns

56/118

0.2 0.6

0.8 0.4

Cells are filled in only when intensity is larger than the given value

57/118

0.34 0.45 0.56 0.67

0.42 0.65 0.78 0.88

0.21 0.33 0.57 0.77

0.01 0.22 0.44 0.55

0.2 0.6

0.8 0.4

Overlay dither pattern

58/118

0.34 0.45 0.56 0.67

0.42 0.65 0.78 0.88

0.21 0.33 0.57 0.77

0.01 0.22 0.44 0.55

0.2 0.6

0.8 0.4

Fill those cells largerthan dither value

59/118

0.34 0.45 0.56 0.67

0.42 0.65 0.78 0.88

0.21 0.33 0.57 0.77

0.01 0.22 0.44 0.55

0.2 0.6

0.8 0.4

60/118

0.34 0.45 0.56 0.67

0.42 0.65 0.78 0.88

0.21 0.33 0.57 0.77

0.01 0.22 0.44 0.55

0.8 0.4

61/118

0.34 0.45 0.56 0.67

0.42 0.65 0.78 0.88

0.21 0.33 0.57 0.77

0.01 0.22 0.44 0.55

62/118

0.34 0.45 0.56 0.67

0.42 0.65 0.78 0.88

0.21 0.33 0.57 0.77

0.01 0.22 0.44 0.55

63/118

0.34 0.45 0.56 0.67

0.42 0.65 0.78 0.88

0.21 0.33 0.57 0.77

0.01 0.22 0.44 0.55

Repeat for rest of image

64/118

65/118

3x3 Dither Example

66/118

Error Diffusion

Another way to set levels without changing resolution

Assume pixels being visited in some orderLeft to right, top to bottom

Roundoff error (from filling/not filling) gets diffused to adjacent pixels not visited yet

Can be combined with halftoning/dithering

67/118

Error Diffusion

Pattern:

Example: Assume we can display levels in increments of 50.

Error 7/16

3/16 5/16 1/16

66 77 107

14 27 60 70

Given:

68/118

Error Diffusion

Pattern:

Error 7/16

3/16 5/16 1/16

66 77 107

14 27 60 70

Round to 50

69/118

Error Diffusion

Pattern:

Error 7/16

3/16 5/16 1/16

50 +16

77 107

14 27 60 70

Round to 50Error of 16

70/118

Error Diffusion

Pattern:

Error 7/16

3/16 5/16 1/16

50 77+7 107

14+3 27+5 60+1 70

Round to 50Error of 16

71/118

Error Diffusion

Pattern:

Error 7/16

3/16 5/16 1/16

50 84 107

17 32 61 70

72/118

Error Diffusion

Pattern:

Error 7/16

3/16 5/16 1/16

50 84 107

17 32 61 70

Round to 100

73/118

Error Diffusion

Pattern:

Error 7/16

3/16 5/16 1/16

50100-16

17 32 61 70

Round to 100Error of -16

74/118

Error Diffusion

Pattern:

Error 7/16

3/16 5/16 1/16

50 100 107-7

17 32-3 61-5 70-1

Round to 100Error of -16

75/118

Error Diffusion

Pattern:

Error 7/16

3/16 5/16 1/16

50 100 100

17 29 56 69

76/118

3x3 Dither Example

77/118

Error Diffusion Example

78/118

Topics

79/118

Seeing Color

What is color?

Seeing Color

http://www.johnsadowski.com/big_spanish_castle.html 80/118

81/118

Light is carried by photons, traveling with different wavelengths/frequencies

c=Speed of light = c (assume constant)Wavelength = Frequency = So, Wavelength & Frequency are

interchangable, and inverse of each other

82/118

Visible Light

Image taken from http://www.andor.com/image_lib/lores/introduction/introduction%20(light)/intlight%201%20small.jpg

83/118

Visible Light

The response of the cells in our eye let us see from ~400nm (violet) to ~700nm (red).

If all light arriving is one wavelength, we see it as one color.

84/118

Visible Light

Images taken from http://www.newport.com/Information-on-Oriel-Spectral-Irradiance-Data/383232/1033/catalog.aspx

The response of the cells in our eye let us see from ~400nm (violet) to ~700nm (red).

If all light arriving is one wavelength, we see it as one color.

But, Light usually comes in as a spectrum – different intensities at all the various wavelengths.

85/118

Tristimulus Theory

There are 3 kinds of cones in the eye that respond differently to different wavelengths of light.

86/118

Tristimulus Theory

There are 3 kinds of cones in the eye that respond differently to different wavelengths of light.

Anything that stimulates these cones in the same way appears as the same color. This is fundamental to our ability to reproduce

different colors! A monitor can stimulate the cones to make it

appear that a particular color is being generated.

87/118

Response of Cones

3 Cones, Rods respond to light differently

Image taken from http://www.unm.edu/~toolson/human_cone_response.htm

88/118

Receptor Response

Though cones often called “red, green, blue”, the peak responses are actually in the yellow, yellow-green, and violet.

Altogether, overall response is Gaussian-like, centered on yellow/greenSun and plants…

Image taken from http://www.unm.edu/~toolson/human_cone_response.htm

Receptor Response

89/118

90/118

Tristimulus

Since 3 different cones, the space of colors is 3-dimensional.

We need a way to describe color within this 3 dimensional space.

We want something that will let us describe any visible color…

91/118

Topics

92/118

The CIE XYZ system

CIE – Comission Internationale de l’Eclairage International Commission on Illumination Sets international standards related to light

Defined the XYZ color system as an international standard in 1931

X, Y, and Z are three Primary colors. All visible colors can be defined as a combination of these three colors. Defines the 3 dimensional color space

93/118

Color Matching Functions

Given an input spectrum, , we want to find the X, Y, Z coordinates for that color.

Color matching functions, , , and

tell how to weight

the spectrum when

integrating:

)(x )(y )(z

Image taken from http://upload.wikimedia.org/wikipedia/commons/8/87/CIE1931_XYZCMF.png

94/118

XYZ space

The visible colors form a “cone” in XYZ space.For visible colors, X,

Y, Z are all positive.But, X, Y, and Z

themselves are not visible colors!

Image taken from http://fourier.eng.hmc.edu/e180/handouts/color1/node27.html

95/118

Luminance and Chromaticity

The intensity (luminance) is just X+Y+Z.Scaling X, Y, Z just increases intensity.We can separate this from the remaining

part, chromaticity. Color = Luminance + Chromaticity

Chromaticity is 2D, Luminance is 1D To help us understand chromaticity, we’ll fix

intensity to the X+Y+Z=1 plane.

96/118

Chromaticity Diagram

Project the X+Y+Z=1 slice along the Z-axis

Chromaticity is given by the x, y coordinates

97/118

White Point

White: at the center of the diagram.

98/118

Spectral Colors

Visible Spectrum along outside curve

99/118

Saturation

As you move on line from white to edge, you increase the saturation of that color.

Royal blue, red: high saturation

Carolina blue, pink: low saturation

100/118

Hue is the “direction” from white.

Combined with saturation, it gives another way to describe color

Also called dominant wavelength

101/118

Non-Spectral Colors

Non-spectral colors: do not correspond to any wavelength of light. i.e. not seen in

rainbowe.g. maroon,

purple, magenta

102/118

If we have two colors, A and B, by varying the relative intensity, we can generate any color on the line between A and B.

Combining Two Colors

103/118

Complementary Colors

Complementary colors are those that will sum to white.

That is, white is halfway between them.

104/118

Combining Three Colors

If we have three colors, A, B, and C, by varying the relative intensity, we can generate any color in the triangle between them.

105/118

Display devices generally have 3 colors (a few have more).e.g. RGB in monitor

The display can therefore display any color created from a combination of those 3.

This range of displayable colors is called the gamut of the device.

106/118

Differing Gamuts

Different devices have different gamutse.g. differing

phosphors So, RGB on one

monitor is not the same as RGB on another

107/118

Device Gamuts

For monitors, typically have colors in the Red, Green, Blue areasHelps cover lots of visible spectrum

But, not all (in fact, nowhere close to all) of the visible spectrum is ever represented

Since all 3 colors are visible, can’t possibly encompass full visible spectrum!

108/118

Gamuts

Red: typical monitor gamut

Blue: maximum gamut with 3 phosphors

What does cyan look like?

109/118

110/118

Topics

111/118

Color Models

CIE’s XYZ system is a standard, but not very intuitive.

As we saw with saturation and hue, there’s more than one way to specify a color.

A variety of color models have been developed to help with some specifications.

112/118

Red, Green, Blue Common specifications for most monitors

Tells how much intensity to use for pixels Note: Not standard – RGB means different

things for different monitors Generally used in an additive system

Each adds additional light (e.g. phosphor)Combine all three colors to get white

113/118

Cyan, Magenta, Yellow Commonly used in printing Generally used in a subtractive system:

Each removes color from reflected lightCombine all three colors to get black

Conceptually, [C M Y] = [1 1 1] – [R G B]Complimentary colors to RGB

114/118

Cyan, Magenta, Yellow, Black Comes from printing process – since CMY

combine to form black, can replace equal amounts of CMY with Black, saving ink.

K = min(C, M, Y) C = C-K M = M-K Y = Y-K

115/118

YIQ / YUV

NTSC, PAL standards for broadcast TV Backward compatible to Black and White TV Y is luminance – only part picked up by

Black and White Televisions Y is given most bandwidth in

signal I, Q channels (or U,V) contain

chromaticity information

116/118

Hue, Saturation, Value Used as a user-friendly

way to specify color. Hue – angle around

cone Saturation – how far

from center Value (luminance) –

how high up cone (black at bottom, white at top center).

Image taken from http://www.mandelbrot-dazibao.com/HSV/HSV.htm

117/118

Lab and Luv

Perceptually-based color spaces (CIE standards)

Idea: want the distance between colors in the color space to correspond to intuitive notion of how “similar” colors are

Not perfect, but much better than XYZ or RGB color spaces.

118/118

Representing Color

Generally, store 3 color channels in equal bits Not necessary, though, e.g. YIQ Can sometimes get better mapping of color space

for an application by adjusting bitse.g. 10 bits R, 8 bits G, 6 bits B

Color indexing: give each color a numerical identifier, then use that as reference Good for specifying with a limited palette

Note: you can use dithering/halftoning with color channels, similar to how you would for achromatic light.

119/118

120/118

Dynamic Range and Contrast of Various Display Mediums

Display Type Dynamic Range Contrast

Newsprint 10

Printed Photo ~100

Slide Photo ~1000

CRT >10,000 75 - 219

LCD ~500 - 700 ~500 - 700

Plasma ~100 - 600 ~100 - 500

DLP 1381 274 - 332

121/118

Cornea Provides most refractionReshaped in refractive surgery

Corrects nearsightedness caused by elongation of the eyeball

122/118

Cornea Iris Lens

Flexible - muscles adjust shapeAllows fine-detail focusToughens with age, leading to reading

glassesCataracts form here

123/118

Focusing Light

For a point not in focal plane, light gets spread across retinaAlso happens in

nearsightednessOur brain is good

at eliminating

these signals

124/118

Hue, Lightness, Saturation

A variation on HSV, but with a double cone

Lightness: black at base (0), white at top (1)

Image taken from http://en.wikipedia.org/wiki/Image:Color_cones.png

1 Light and Color. 2/118 Topics The Human Visual System Displaying Intensity and Luminance Display...

Documents