Date post: | 21-Dec-2015 |
Category: |
Documents |
Upload: | myron-thornton |
View: | 222 times |
Download: | 0 times |
2-1
2.1. Image representations, types
and concepts
2.2. Image digitization
2.3. Digital image properties
2.4. Color images
2.5. Cameras
Ch. 2 – The image, its representation,
type, and properties
2-2
○ Function
2.1. Image representations, types and concepts
• Continuous – A, B: continuous• Discrete – A: discrete, B: continuous• Digital – A, B: discrete
Digital image f(x,y): a 2-D digital function
○ Scene g(x,y,z): a 3-D continuous function
Pixel: picture elementGray level: pixel value
Origin
: , where : domain, : rangef A B A B
○
2-3
Pin-hole camera
○ Image Formation
2-4
○ Perspective Projection
2-5
Drawbacks of pinhole camera: (i) Big pinhole - Averaging rays blurs image
Small pinhole - Diffraction effect blurs image
2 mm 1 mm 0.35 mm 0.15 mm 0.07 mm
(ii) Images are relatively dark
Solution : * Pinholes Lenses Lenses: gather light sharp focus
2-6
Intensity (Gray Scale) Image
○ Types of ImagesBinary Image
2-7
Color Image
Indexed (or Palette) Color Image
2-8
Multispectral Image
2-9
X-Ray Image Gamma-Ray Image
2-10
Ultraviolet Image
Radio Image
2-11
Ultrasound Image
2-12
X-Ray Transmission Computerized Tomography Image
2-13
Range Image
2-14
Moire Image
2-15
○ Biological Visual Systems
humancattle, horsebird, chicken, duckbatsnakeinsect, fly, bee
2-16
Camera
Image array
2.2. Image Digitization
Types of resolutions: Spatial (space), Radiometric (intensity), Spectral (color), Temporal (time).
Sensor array
2-17
2.2.1. Spatial Resolution
2-18
False contours
2.2.2. Radiometric Resolution
2-19
Non-uniform resolution -- to reduce storage while preserve as much information as possible
* Radiometric resolution (quantization): Busy area – fine, dense Inactive region – coarse, sparse
* Spatial resolution (sampling): Sharp region – fine, dense Smooth area – coarse, sparse
Non-uniform
* Spectral resolution (quantization):* Time resolution (sampling):
2-20
Uniform sampling
Non-uniform sampling
Non-Uniform Sampling
128 by 128
64 by 64
2-21
Adaptive mesh
Adapted mesh
Gradient image
Adaptive sampling
Input image
Spring modelSampled image
2-22
2.3. Digital Image Properties ○ Metric Properties
( , ) 0,D p q ( , ) ( , ),D Dp q q p( , ) ( , ) ( , )D D D p z p q q z
2 2( , ) ( ) ( )ED x s y t p q
4 ( , ) | | | |D x s y t p q
8 ( , ) max{| |,| |}D x s y t p q
Euclidean City-block
Chess-board
Quasi-Euclidean( 2 1)
( , )( 2 1) otherwise
x s y t x s y tD
x s y t
p q
( , ), ( , )x y s t p q (i) Distances
2-23
(ii) Adjacency
4-neighbors 8-neighbors
(iii) Distance Transform (Chamfering)
2-24
2-25
2-26
(i) Crack edges: each pixel has 4 crack edges
(ii) Inner, outer borders
(iii) Convex, concave
(vi) Convex hull: the smallest convex region containing the input region
○ Geometrical Properties
2-27
○ Topological properties(i) Rubber Sheet Transform (RST): preserves the contiguity of object parts and the number of holes in regions (no cut and joint). e.g., Imagine a rubber balloon with an object painted on it
(ii) Euler-Poincare Characteristic: Euler number (#regions - #holes) is invariant under RST
(iii) Topological components:
2-28
2.3.2. HistogramsHistogram
Probability distribution
2-29
Information theory -- amount of informationPhysical system -- measure of disorder Confidence theory -- amount of uncertaintyImage -- degree of redundancy
1
20
( ) ( ) log ( )L
i ii
H X k p x p x
2.3.3. Entropy
2-30
2.3.4. Visual Perception
(i) Perceptual Grouping (Organization)
2-31
2-32
(iii) Overshoot and Undershoot
(iv) Subjective Objects
(ii) Simultaneous Contrast
Mack band pattern
The human brain tricks us whenever
it can!
(v) Optical Illusion and Visual Phenomena
2-33
Are the lines parallel or not?
2-34
Coil or circle?
If something‘s rotating – go home, you need a break!
2-35
It doesn‘t move!
2-36
It doesn‘t move too!
2-37
It doesn‘t move too!
2-38
Concentrate on the cross in the middle, after a while you will notice that this moving purple dot will turn green!
Look at the cross a bit longer and you‘ll notice that all dots except the green one will disappear.
2-39
2-40
2.3.5. Image Quality
Image degradation results from (i) noise, (ii) error, (iii) distortion, (iv) blurring
where g(x,y): input image, f(x,y): degraded image H: degradation process, n(x,y): noise
Degradation model:
( , ) [ ( , )] ( , )f x y H g x y n x y
2-41
Linearity:
Additivity:
Homogeneity:
Position (or space) invariance:
1 1 2 2 1 1 2 2[ ( , ) ( , )] [ ( , )] [ ( , )]H k g x y k g x y k H g x y k H g x y
1 2 1 2[ ( , ) ( , )] [ ( , )] [ ( , )]H g x y g x y H g x y H g x y
1 2, : constantsk k
[ ( , )] [ ( , )]H kg x y kH g x y
( , ) [ ( , )]f x y H g x y
Degradation processes H
2-42
Image:
If H is linear,
If H is homogeneous,
( , ) ( , ) ( , )g x y g x y d d
( , ) [ ( , )] [ ( , ) ( , ) ]f x y H g x y H g x y d d
( , ) [ ( , ) ( , )]f x y H g x y d d
( , ) ( , ) [ ( , )]f x y g H x y d d
Let
[ ( , )]H x y
( , , , ) [ ( , )]h x y H x y
: impulse response of H
: point spread function (PSF) ( , ) ( , ) ( , , , )f x y g h x y d d Then,
Degraded image:
2-43
If H is position invariant,
( , ) ( , ) ( , )f x y g h x y d d Then,
( , ) ( , ) ( , )f x y g x y h x y
Simplified degradation model:
( , ) ( , ) ( , ) ( , )f x y g x y h x y n x y
( , , , ) [ ( , )] ( , )h x y H x y h x y
In frequency domain,
( , ) ( , ) ( , ) ( , )F u v G u v H u v N u v
2.3.6. Noise
-- Originating from image acquisition, digitization,
or transmission
○ White noise: the noise whose Fourier spectrum is constant
( , ) ( , ) ( , )f x y g x y n x y
( , ) ( , ) ( , )f x y g x y n x y
e.g., Television raster, film material
e.g., Physical processes
FT
( , )n x y
( , )n x y ( , )N u v2-44
Additive:
Multiplicative:
2-45
○ Random noise:
Objectives: i) examine the performance of a developed algorithm ii) help to remove noise from an image
(i) Salt-and-pepper (impulse) noise
for
( ) for
0 otherwise
a
b
P x a
p x P x b
2-46
(ii) Uniform noise
1if
( ) ,0 otherwise
a x bp x b a
22 ( )
[ ] , [ ]2 12
a b b aE x Var x
2 2 2
1 1[ ] ( )
1 1 1 1 ( )
2 2 2
b b
a a
E x xp x dx x dx xdxb a b a
b a bx b a
ab a b a
2-47
2 2
2 2
2 2
2 2 2 2
2 2 2
[ ] [( [ ]) ]
[ 2 [ ] ( [ ]) ]
[ ] 2 [ ] [ ] ( [ ])
[ ] ( [ ]) ( ) ( )2
1 ( ) ( ) /12
2
b
a
Var x E x E x
E x xE x E x
E x E x E x E x
a bE x E x x p x dx
a bx dx a b
b a
2-48
(iii) Rayleigh noise2( ) /2
( ) for ( )
0 for
x a bx a e x ap x b
x a
2
/ 4
(4 )
4
a b
b
(iv) Erlang (gamma) noise1
for 0( ) ( 1)!
0 for 0
b baxa x
e xp x b
x
2 2
/
/
b a
b a
2-49
(v) Exponential noise
for 0( )
0 for 0
axae xp x
x
2 2
1/
1/
a
a
(vi) Gaussian noise2
2
( )
21( ) ,
2
x
p x e
Method 1:
2-50
2-51
Method 2:
cdf
inverse
-20 -10 0 10 200
0.02
0.04
0.06
0.08normal distribution pdf, =0, =5
-20 -10 0 10 200
0.2
0.4
0.6
0.8
1normal distribution CDF, =0, =5
2
2
2exp
2
1),;(
x
xf
1( ) 1
2 2
xF x erf
1 1( ) 2 2 1 , ( )F p erf p p F x
若 p 為均勻分佈,則 F-1(p) 為常態分佈
Method3:
Definition:
By Taylor expansion:
Error Function Erf(x)
2-54
◎ Estimation of noise parameters
Gaussian Rayleigh Uniform
Image segmentation (i) manually or (ii) automatically
2-55
Steps: 1. Segment the image 2. Choose a uniform image region 3. Compute its histogram 4. Compute the mean and variance of the histogram
5. Determine the probability distribution from the shape of the histogram 6. Estimate the parameters of the probability distribution of the histogram
( )ih x
2 2( ), ( ) ( )i i i ix h x x h x
( )ih x
2,
2-56
Example: Rayleigh noise:
24 (4 ),b
/ 4, / 4a b a b
From 2 (4 ) / 4,b 24 /(4 )b
From
24
4(4 ) 4a
2( ) /2( ) for
( )0 for
x a bx a e x ap x b
x a
2-57
1. Signal-to-Noise Ratio (SNR)2 2
( , ) ( , )
( , ), ( , ),x y x y
E n x y F f x y /SNR F E
1010logdBSNR SNR (logarithmic scale)
2. Peak Signal-to-Noise Ratio (PSNR)
Measures of Image Quality
where I: input image, K: output image
2.4. Color Images2.4.1. Physics of Color
-- The world is colorless. Human color perception is carried out by the nervous system, which interprets differently to distinct electromagnetic wavelengths.
-- Electromagnetic spectrum
2-58
2-59
-- Observed color of objects Surface reflection rebounds incoming energy, whose spectrum remains the same as that of the light source Body reflection: The incoming energy diffuses into surface and reflects from its internal pigment
-- Visible spectral colors
2-60
2.4.2. Human Color Perception
Human Eye FOV: Field Of View (FOV)
width × height = 160 deg × 135 deg
2-61
Eyeball
Camera
2-62
Retina is composed of photoreceptors
retina
photoreceptors
neural fiber
light
2 types of photo-receptors: rods and cones
2-63
Rods : sensitive to intensity
Cones: sensitive to color
Types of cones: L(red), M(green), S(blue)
2-64
○ RGB Color Space
1 2 3( ) ( ) ( ) ( )C w R w G w B
1 2 3( ), ( ), ( )w w w : color matching functions may be negative
2.4.3. Color spaces – in which colors are made up
R, G, B: primary colors, real
of different amounts of primary colors
2-65
○ CIE XYZ Color Space
CIE (Commission Internationale d’Eclairage):
an organization responsible for color standard
X,Y,Z: not real primaries, Y: luminance
。 The volume of visible
colors in CIE XYZ space
is a cone
Their color matching functions are positive everywhere
2-66
。 The relationship between RGB and XYZ
0.431 0.342 0.178
0.222 0.707 0.071
0.02 0.130 0.939
X R
Y G
Z B
3.063 1.393 0.476
0.969 1.876 0.042
0.068 0.229 1.069
R X
G Y
B Z
2-67
○ CIE xy Color Space -- The section intersects
the XYZ space with the plane 1X Y Z
, , X Y Z
x y zX Y Z X Y Z X Y Z
Since x + y + z = 1, a color can be specifiedby x and y alone.
Chromaticity Diagram
2-68
。 Chromaticity Diagram(i) Spectral locus: the curved boundary along which colors of single wavelengths are viewed
(ii) Hue changes as one moves around the spectral locus
(iv) Colors that lie farther away from the neutral point are more saturated
(iii) Neutral point: the color whose weights are equal for all three primaries
2-69
。 RGB Gamut – The colors correspond to
positive matching values
2-70
Gamut which can be displayed by display devices
CRT monitor
Color film
Color printer
2-71
○ HSV (Hue, Saturation, Value) Color Space
Hue: varies from red greenSaturation: varies from red pinkValue: varies from black white
2-72
○ (i) RGB HSV
If R = V, then
If G = V, then
If B = V, then
If H ends up being negative, add 1
If (R,G,B) = (0,0,0), then (H,S,V) = (0,0,0)
max{ , , },
min{ , , }, /
V R G B
V R G B S V
1
6
G BH
1
26
B RH
1
46
R GH
2-73
。 Example: (R, G, B) = (0.2, 0.4, 0.6)
max{ , , } max{0.2,0.4,0.6} 0.6
min{ , , } 0.6 min{0.2,0.4,0.6} 0.4
/ 0.4 / 0.6 0.6667
V R G B
V R G B
S V
Since 0.6,
1 1 0.2 0.44 4 0.5833
6 6 0.4
B V
R GH
2-74
(ii) HSV RGB
6
6
(1 )
(1 )
[1 (1 )]
H H
F H H
P V S
Q V SF
T V S F
0
1
2
3
4
5
H R G B
V T P
Q V P
P V T
P Q V
T P V
V P Q
2-75
。 Example: (H, S, V) = (0.5833, 0.6667, 0.6)
6 6(0.5833) 3
6 6(0.5833) 3 0.5
(1 ) 0.6(1 0.6667) 0.2
(1 ) 0.6(1 0.6667 0.5) 0.4
[1 (1 )] 0.6[1 0.6667
(1 0.5)] 0.4
H H
F H H
P V S
Q V SF
T V S F
Since 3,
( , , ) ( , , ) (0.2, 0.4, 0.6)
H
R G B P Q V
2-76
○ YIQ Color Space – Used for TV and video
0.299 0.587 0.114
0.596 0.274 0.322
0.211 0.523 0.312
Y R
I G
Q B
1.0 0.956 0.621
1.0 0.272 0.647
1.0 1.106 1.703
R Y
G I
B Q
Y : luminance information I, Q : color information
2-77
○ Uniform Color Space
。 Macadam ellipse -- the noticeable difference of a color forms the boundary of the color in a color space and can be fitted with an ellipse
。 Noticeable difference – the difference when modifying a color until one can tell it has changed
2-78
The color difference in CIE xy space is poor:
The distance in the space is a guide to color difference
(a) the ellipses at the top are larger than those at the bottom
(b) the ellipses rotate as they move up
2-79
○ CIE u’v’ Color Space – a more uniform
space than the CIE xy color space 4 9
( , ) ( , )15 3 15 3
X Yu v
X Y Z X Y Z
2-80
○ CIE L*a*b* Color Space
– another substantial uniform space
* 1/3
* 1/3 1/3
* 1/3 1/3
116( ) 16
500[( ) ( ) ]
200[( ) ( ) ]
n
n n
n n
YL
Y
X Ya
X Y
Y Zb
Y Z
where , ,n n nX Y Z : the XYZ coordinates of a reference white patch
2-81
○ Summary
○ Secondary colors (primaries of pigments):
Magenta (purple) = R + B = W - G
Cyan = G + B = W - R
Yellow = R + G = W - B
2.4.5. Color and Lightness ConstanciesSurfaces have different colors and brightness when viewed under different lights
2-82
2-83
Color constancy – Intensity-independent description of color
Lightness constancy – Color-independent description of brightness
2-84
2.5. Cameras2.5.1. Photosensitive SensorsTypes: CCD (charge-coupled devices) CMOS (complementary metal oxide semiconductor)
CCD uses a rectangular
grid of electron collection
sites laid over a silicon
wafer.
○ Color CCD: uses the same chip as black and white CCD except that successive rows or columns are made sensitive to R, G or B light using a filter coating that blocks the complementary light.
Bayer pattern: a filter pattern including a mosaics of 2 by 2 blocks form by two G, one R, and one B receptors
2-85
2-86
• Perspective Projection
Summary
• Biological images in addition to engineering images
• Types of resolutions: Spatial (space), Radiometric (intensity), Spectral (color), Temporal (time).
2-87
Euclidean City-block
Chess-board
2 2( , ) ( ) ( )ED x s y t p q
4 ( , ) | | | |D x s y t p q
8 ( , ) max{| |,| |}D x s y t p q
( , ), ( , )x y s t p q• Metric functions
• Non-uniform resolution -- to reduce storage while preserve as much information as possible
2-88
• Image degradation model:
( , ) [ ( , )] ( , )f x y H g x y n x y
• Chamfer distance
Simplified model:
( , ) ( , ) ( , ) ( , )f x y g x y h x y n x y
In frequency domain,
( , ) ( , ) ( , ) ( , )F u v G u v H u v N u v
• Bayer pattern:
2-89
• Color spaces:
• Color and brightness constancies:
• Noise generation and estimation