Midterm Review
Image Processing
CSE 166
Lecture 10
Announcements
• Midterm exam is on May 4
CSE 166, Spring 2020 2
Topics covered
• Image acquisition, geometric transformations, and image interpolation
• Intensity transformations
• Spatial filtering
• Fourier transform and filtering in the frequency domain
• Image restoration
• Color image processing
CSE 166, Spring 2020 3
Image acquisition
CSE 166, Spring 2020 4
Geometric transformations
CSE 166, Spring 2020 5
Geometric transformations
CSE 166, Spring 2020 6
Euclideantransformation
Similaritytransformation
Affinetransformation
Composition and inversion of transformations
CSE 166, Spring 2020 7
Composition of transformations Inverse of transformation
Firsttransformation
Secondtransformation
Thirdtransformation
Intensity transformations
CSE 166, Spring 2020 8
Contrast stretchingfunction
Thresholdingfunction
Intensity transformations
CSE 166, Spring 2020 9
Some basic transformation functions
Gamma transformation
CSE 166, Spring 2020 10
Gamma transformation
CSE 166, Spring 2020
γ < 1
Darkimage
11
Gamma transformation
CSE 166, Spring 2020
γ > 1
Lightimage
12
Piecewise-linear transformations
• Contrast stretching
• Intensity-level slicing
• Bit-plane slicing
CSE 166, Spring 2020 13
Contrast stretching
CSE 166, Spring 2020 14
Intensity-level slicing
CSE 166, Spring 2020 15
Bit-plane slicing
CSE 166, Spring 2020 16
Histogram
CSE 166, Spring 2020
Similar to probability density function (pdf)17
Histogram equalization
CSE 166, Spring 2020 18
Histogram equalization
CSE 166, Spring 2020 19
Histogram equalization
CSE 166, Spring 2020 20
Spatial filtering (2D)
CSE 166, Spring 2020 21
2D correlation
2D convolution
Correlation and convolution (2D)
CSE 166, Spring 2020 22
Smoothing kernels
CSE 166, Spring 2020 23
Average(box kernel)
Weighted average(Gaussian kernel)
Smoothing with box kernel
CSE 166, Spring 2020 24
3x3
11x11 21x21
Inputimage
Smoothing with Gaussian kernel
CSE 166, Spring 2020 25
Standard deviation σ Percent of total volume under surface
1 39.35
2 86.47
3 98.89
Volume under surface greater than 3σ is negligible
Smoothing with Gaussian kernel
CSE 166, Spring 2020 26
σ = 3.521x21
Input image σ = 743x43
Derivatives
CSE 166, Spring 2020 27
Sharpening filters
CSE 166, Spring 2020 28
Overview: Image processing in the frequency domain
CSE 166, Spring 2020 29
Image in spatial domain
f(x,y)
Image in spatial domain
g(x,y)
Fouriertransform
Image in frequency domain
F(u,v)
Inverse Fourier
transform
Image in frequency domain
G(u,v)
Frequency domain processing
Jean-Baptiste Joseph Fourier1768-1830
Review
• Complex numbers
• Euler’s formula
• Complex functions
CSE 166, Spring 2020 30
1D Fourier series
CSE 166, Spring 2020
Sinesand
cosines
31
Periodicfunction
Weighted by magnitude
Shifted byphase
Period T
Sampling
CSE 166, Spring 2020 32
Sampling
CSE 166, Spring 2020
Over-sampled
Critically-sampled
Under-sampled
1/ΔT
Fourier transform of function
Fourier transforms of sampled function
33
Fourier transform of sampled functionand extracting one period
CSE 166, Spring 2020 34
1D
2D
Over-sampled Under-sampled
Recovered Imperfect recoverydue to
interference
Aliasing
CSE 166, Spring 2020 35
Continuous Sampled
IdenticalDifferent
Sampled at same rate
Over-sampled
Under-sampledAlias: a
false identity
Aliasing
CSE 166, Spring 2020
1D
2D
Aliasing
Original
36
Fourier transform of sampled functionand extracting one period
CSE 166, Spring 2020 37
1D
2D
Over-sampled Under-sampled
Recovered Imperfect recoverydue to
interference
2D discrete Fourier transform (DFT)
• (Forward) Fourier transform
• Inverse Fourier transform
CSE 166, Spring 2020 38
Centering the DFT
CSE 166, Spring 2020
1D
2D
In MATLAB, use fftshift and ifftshift
39
Centering the DFT
CSE 166, Spring 2020
Original
DFT(look at corners)
Shifted DFTLog of
shifted DFT
40
Contributions of magnitude and phaseto image formation
CSE 166, Spring 2020
Phase
IDFT: Phase only
(zero magnitude)
IDFT: Magnitude
only (zero phase)
IDFT: Boy magnitude
and rectangle phase
IDFT: Rectangle magnitude
and boy phase 41
2D convolution theorem
• 2D discrete (circular) convolution
• 2D convolution theorem
CSE 166, Spring 2020 42
Filtering using convolution theorem
CSE 166, Spring 2020
Filtering in spatial domain
using convolution
expectedresult
Filtering in frequencydomain
using productwithout
zero-padding
wraparounderror
43
Filtering using convolution theorem
CSE 166, Spring 2020
Filtering in frequencydomain
using product
withzero-padding
no wraparounderror
Gaussian lowpass filter in
frequency domain
44
Fourier transform
Product
Inverse Fourier transform
Zero padding
Filtering using convolution theorem
CSE 166, Spring 2020
Filtering in spatial
domain using
convolution
Filtering in frequencydomain
usingproduct
Identical results
DFT
45
Filtering in the frequency domain
• Ideal lowpass filter (LPF)
– Frequency domain
CSE 166, Spring 2020 46
Filtering in the frequency domain
• Ideal lowpass filter (LPF)
– Spatial domain
CSE 166, Spring 2020 47
H(u,v) h(x,y)
Filtering in the frequency domain
• Gaussian lowpass filter (LPF)
CSE 166, Spring 2020 48
Filtering in the frequency domain
• Butterworth lowpass filter (LPF)
CSE 166, Spring 2020 49
Gaussian LPF
Filtering in the frequency domain
CSE 166, Spring 2020
Ideal LPF Butterworth LPF
50
Highpass filter (HPF)Frequency domain
CSE 166, Spring 2020
Ideal HPF
Gaussian HPF
Butterworth HPF
51
Butterworth HPF
Highpass filter (HPF)Spatial domain
CSE 166, Spring 2020
Ideal HPF Gaussian HPF
52
Filtering in the frequency domain
CSE 166, Spring 2020
Ideal HPF Gaussian HPF Butterworth HPF
53
H (u)
u u
x x
H (u)
h (x)
1
16–– 3
h (x)
2 12 12 1
2 1 8 2 1
2 12 12 1
0 2 1 0
2 1 4 2 1
0 2 1 0
1 2 1
2
1
9–– 3
4 2
1 2 1
1 1 1
1 1 1
1 1 1
Filtering in the frequency domain
CSE 166, Spring 2020
1D
Lowpass filter Sharpening filter54
Frequencydomain
Spatialdomain
Filtering in the frequency domain
CSE 166, Spring 2020
2D
55
Lowpass filter Highpass filter Offset highpass filter
Bandreject filters
CSE 166, Spring 2020 56
ButterworthIdeal Gaussian
Model of image degradation
• Spatial domain
• Frequency domain
CSE 166, Spring 2020 57
Noiseimage
Originalimage
Degradedimage
Degradationfunction
Noiseimage
Originalimage
Degradedimage
Degradationfunction
Model of image degradation, then restoration
CSE 166, Spring 2020 58
Estimate of original imageOriginal image
Histograms of sample patches
CSE 166, Spring 2020
Sample “flat” patches from images with noise
Identify closest probability density function (pdf) match:
59
Gaussian Rayleigh Uniform
Mean filters
CSE 166, Spring 2020
Additive Gaussian
noise
Geometric mean
filtered
Arithmetic mean
filtered
60
X-ray image
Order-statistic filters
CSE 166, Spring 2020
Additivesalt and peppernoise
1x median filtered
3x median filtered
2x median filtered
61
Comparing filters
CSE 166, Spring 2020
Alpha-trimmed mean filtered
Median filtered
Arithmetricmean
filtered
Geometric mean filtered
62
Additive uniform +
salt and peppernoise
Adaptive filters
CSE 166, Spring 2020
AdditiveGaussian
noise
Arithmetricmean filtered
Geometric mean
filtered
Adaptive noise reduction
filtered
63
Periodic noise
CSE 166, Spring 2020
Conjugate impulses
64
Additive sinusoidal noise
DFT magnitude
Notch reject filters
CSE 166, Spring 2020 65
Notch reject filter
CSE 166, Spring 2020 66
Filter in frequency
domain
Estimate of original
image
Degraded image
DFT magnitudeConjugate
impulses
Conjugate impulses
Estimating the degradation function
• Methods
– Observation
– Experimentation
– Mathematical modeling
CSE 166, Spring 2020 67
Estimation of degradation function by experimentation
CSE 166, Spring 2020 68
Imaged (degraded) impulseImpulse of light
Estimation of degradation function by mathematical modeling
CSE 166, Spring 2020
Atmospheric turbulence
model
69
Image restoration
CSE 166, Spring 2020
Inversefiltering
Wienerfiltering
Degraded image
70
Motion blur and
additive noise
Constrained least squares filtering
Less noise
Much less noise
RGB color model
CSE 166, Spring 2020
RGB coordinates
71
RGB color cube
HSI color model:Relationship to RGB color model
CSE 166, Spring 2020
All colors with cyan
hue
RGB color cube rotated such that line joining black and white
(intensity axis) is vertical
72
HSI color model
CSE 166, Spring 2020
RGB color cube rotated such that
observer is on intensity axis, beyond white looking towards
black
Shape does not matter, only angle from red73
RGB color cube
HSI intensity axis
CSE 166, Spring 2020
HSI
RGB
CMYK
74
Color models
CMY
Pseudocolor image processing
• Intensity slicing
• Intensity to color transformations
CSE 166, Spring 2020 75
Intensity slicing
CSE 166, Spring 2020 76
Grayscale to 2 colors
Intensity slicing
CSE 166, Spring 2020
Grayscale to 2 colors
77
Intensity slicing
CSE 166, Spring 2020
Grayscale to 256 colors
Colorbar
78
Intensity to color transformations
CSE 166, Spring 2020
Grayscale input image
RGBoutput image
79
Intensity to color transformations
CSE 166, Spring 2020
X-ray grayscale input image
Misses explosive
RGB output images
Without explosive
With explosive
80
Intensity to color transformations
CSE 166, Spring 2020
Multiple grayscale
input images
SingleRGB
output image
81
Intensity to color transformations
Near infrared (NIR)
Red (R)
NIR,G,B as RGB R,NIR,B as RGB82
Green (G) Blue (B)
Multiple satellite
grayscale input
images
Output RGB imagesCSE 166, Spring 2020
Full-color image processing
• Two approaches
– Process each channel independently
– Process color vector space
CSE 166, Spring 2020 83
Full-color image processing
CSE 166, Spring 2020
Spatial filtering: process each channel independently
84
Full-color image processing
CSE 166, Spring 2020
All RGB channels
HSI intensity channel only
Spatial filtering: image sharpening
85
Difference
Full-color image processing
CSE 166, Spring 2020
Histogram equalization: do not process each channel independently
1. RGB to HSI2. Histogram
equalize HSI intensity channel
3. HSI to RGB86
4. HSI saturation channel adjustment
Next Lecture
• Basis vectors and matrix-based transforms
– After midterm exam
• Reading
– Sections 6.1 and 6.2: Wavelet and Other Image Transforms
CSE 166, Spring 2020 87