1
Image Enhancement in the Spatial Domain
Image Processing with Biomedical Applications
ELEG-475/675Prof. Barner
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 2
Image Enhancement in the Spatial Domain
Algorithms for improving the visual appearance of images
Gamma correctionContrast improvementsHistogram equalizationNoise reductionImage sharpening
Optimality is (often) in the eye of the observerAd hoc
Reading assignments: papers on WebCT
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 3
Spatial Domain Processing
Utilize neighborhood operations
g(x,y)=T [f (x,y)]Simple case: point operations
s=T(r)Contrast stretchingThresholding
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 4
Basic Gray Level Transformations
Image negativesEasier visualization of detail embedded in dark regions
2
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 5
Gray Level Transformation Curves
More general transformations
logInverse logn’th powern’th root
Used to map narrow dark (log/root) or bright (inverse log/power) range to a greater dynamic range
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 6
Fourier Spectrum Example
The spectrum has a large dynamic rangeExample below: 0 to 106
Compress range to view visually (dB’s)
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 7
Power-Law (Gamma) Transformations
Power-law transformation:
s=crγ
Many devices require gamma correction
CRTs have power function intensity-to-voltage responsesMonitor specificApplied in color planesEmbedded in some image representations
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 8
Gamma Correction Example – Monitor
3
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 9
Gamma Correction Example – MR
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 10
Gamma Correction Example – Aerial
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 11
Piecewise-Linear Transformations
Contrast stretchingPoor illuminationSensor dynamic rangeLens aperture settings
Feature extractionMedical imaging
BoneSoft tissue
Gray-level slicingOnly show range of interest
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 12
Histogram Processing
Light, dark, and low contrast images have concentrated histogramsImages with uniform histograms
Contain the full range of gray valuesHave high contrastBetter general visual appearance
4
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 13
Histogram Equalization
We focus on the (normalized) scalar mappings=T(r) 0≤r ≤1
where the following are satisfied:T(r) is single-valued and monotonically increasing in [0,1]0≤T(r) ≤1 for 0≤r ≤1
The single-valued condition allows the inverse transformation to be defined
r=T-1(s) 0≤s ≤1The monotonically increasing condition prevents inversion
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 14
Probability Density Function
Let the PDF of r be pr(r)The CDF is:
Note CDFs are monotonically increasing and have range [0,1]
Defined the RV s=T(r)The PDF of a RV function is
0
( ) ( )r
r
rP r p w dw= ∫
( ) ( )s rdrp s p rds
=
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 15
CDF Distribution
Set T(r)=Pr(r) Thus the PDF of s is
The CDF is uniformly distributed
Independent of the PDF
( ) ( )
1( )( )
1
s r
rr
drp s p rds
p rp r
=
=
=0
( )
( )
( )
r
r
r
ds dT rdr dr
d p w dwdrp r
=
⎡ ⎤= ⎢ ⎥⎣ ⎦=
∫
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 16
HistogramEqualization
CDF mapping of Gray values
Yields uniformed histogramSimple, parameter-free
Discrete caseResults not strictly uniformImplementation issues
5
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 17
Histogram EQMappings
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 18
Histogram Matching
To control performance, map to a specified distribution
Desired PDF: pz(z), CDF: Pz(z)Measured image PDF: pr(r), CDF: Pr(r)
Set u= G(z)=Pz(z) and v=T(r)=Pr(r)Both uniformly distributed
Set mapping asz=G-1[T(r)]
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 19
Histogram Matching Mapping
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 20
Histogram Matching Example
Observation image has poorly distributed histogramConcentration causes problems in standard histogram equalization
6
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 21
Histogram Equalization Result
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 22
Histogram Matching Result
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 23
Local Enhancement
Statistics are not uniform across an imageLocal statistic calculations yield spatially adaptive enhancement
Risk: nonuniform or blocky appearance
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 24
Image Statistics
Gray-levels: r0, r2,…, rL-1Mean:
nth moment
Assume statistics are stationaryLocal evaluations can be utilizedCalculate over neighborhood
1
0
( ) [( ) ] ( ) ( )L
n nn i i
i
r E r m r m p rµ−
=
= − = −∑
1
0[ ] ( )
L
i ii
m E r r p r−
=
= =∑
7
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 25
Heuristic Statistic-Based Enhancement
Defined heuristics to enhance:
Dark background objectEdges and details
Note: image depended ad hoc procedure
0
1 2
( , ) if AND k
( , ) otherwise( , )S Gxy
G S Gxy
E f x y m k MD k D
f x yg x y σ⋅ ≤
≤ ≤⎧⎪= ⎨⎪⎩
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 26
Heuristic EnhancementLocal Statistics and Mask
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 27
Observation and Enhancement Results
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 28
Image Subtraction
Simple pixel-wise operationRemoves reference to show details
Can be applied on bit-planes or independently captured reference
Histogram equalizationimproves detailvisualization
8
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 29
Bit-Plane Application Example
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 30
Mask Mode Radiography Example
Goal:Observe arterial bloodstream pathways
Procedure:Reference (mask): x-ray imageObservation: x-ray image with contrast
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 31
Noise Reduction
Simple observation model: g=f+ηReduce noise by averaging across (fixed) images
Note that
Images must be registered to avoid blurring
1
1 K
ii
g gK =
= ∑
{ }E g f=2 21g K ησ σ=
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 32
AstronomyExample
Applicable for sensor noise
Additive Gaussian model
Increasing the number of images averaged reduces the noise variance
Shown: K=8, 16, 64, and 128
9
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 33
Difference Imagesand Histograms
Increasing K reduces the difference images
MeanVariance
Similar to that integrating characteristics of CCD sensors
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 34
Spatial Filtering
Spatial filtering is based on a moving window or maskIn the linear filtering case:
More compactly:
Must take into account boarder affects
( , ) ( , ) ( , )a b
s a t b
g x y w s t f x s y t=− =−
= + +∑ ∑
1
mn
i ii
R w z=
= ∑
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 35
Simple Smoothing Masks
Simplest a linear filter: spatial averageReduces noise, but introduces blurring
Distance weight samplesCentrally located samples are more importantReduces blurring (somewhat)Example above: simple integer arithmetic
Alternative approach: utilize spectral characteristics
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 36
SmoothingExample
10
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 37
Noise Reduction and Object Detection Example
Hubbell Space Telescope imageAveraging reduces noiseThresholding localizes the most significant objects
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 38
Order-Statistic Filters
Linear (weighted sum) filtersBlur edges and detailsAre susceptible to outliers
Order-statistic filtersPreserve edges and detailsAre less susceptible to outliers
Spatially ordered samples: z1,z2,…,zNRank ordered samples: z(1),z(2),…,z(N)
z(1)≤z(2) ≤ …≤ z(N)Med[z1,z2,…,zN]= z((N+1)/2)
Selection-type filter
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 39
Salt-and Pepper Noise ExampleBad sensors and bit errors yield salt-and-pepper noise
Heavy tailed noise distributionOther examples: Laplacian, Cauchy, α-Stable
Probability of corruption: pOutliers (generally) located in the extremes of the ordered set
Do not affect median
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 40
Maximum Likelihood Estimation and the Generalized Gaussian Distribution
| |( ) exp ,2 (1/ )
kk xf xk
µσ σ
−⎛ ⎞= −⎜ ⎟Γ ⎝ ⎠
1
0
( ) exp( )xx t t dt∞
−Γ = −∫
2
2
1 ( )( ) exp .22Gxf x µσσ π−⎛ ⎞= −⎜ ⎟
⎝ ⎠
where , σ, and k denote the Gamma function, scale, and
tail parameter respectively. The distribution tail decays slower as k decreases.
Special Cases:k = 2 : The standard Gaussian distribution,
k = 1 : The Laplacian distribution,1 | |( ) exp .
2Lxf x µ
λ λ−⎛ ⎞= −⎜ ⎟
⎝ ⎠
11
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 41
a) b)
Figure: Probability Density Functions with unit variances: a) Gaussian density function and b) Laplacian density function.
Probability Density Function Plots
The Laplacian distribution has a lower tail decay rateGreater probability of observing outliers
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 42
Maximum Likelihood (ML) Estimate
Consider a set of observations forming a vector, , we assume that the RVs come from a known density, , that has unknown (but fixed) parameter µ . The ML estimate is given by,
where is the Likelihood function. The ML estimate of µ is obtained as the solution to,
or,
1 2[ , ,..., ]TNx x x=xix
^( ) arg max ( ),ML f
µµ µ=x x |
(.)f
1 2( ) ( ). ( ). . ( )Nf f x f x f xµ µ µ µ=x| | | |
( ) 0,ML
fµ µ
µµ =
∂=
∂x |
( )ln ( ) 0.ML
fµ µ
µµ =
∂=
∂x |
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 43
2
2( )
2
1
1( ) .2
i
i
xN
i i
f eµ
σµσ π
−−
=
=∏x |
( ) ( )2
21 1
( )ln ( ) ln 2 ln( ) ,2
N Ni
ii i i
xf N µµ π σσ= =
−= − − −∑ ∑x |
( )2
22 2 2
1 1 1 1
1ln 2 ln( ) .2 2
N N N Ni i
ii i i ii i i
x xN π σ µ µσ σ σ= = = =
= − − − − +∑ ∑ ∑ ∑
( )( )2 2
1 1
ln ( ) 1 1 0.N N
ii ii i
fx
µµ
µ σ σ= =
∂= − + =
∂ ∑ ∑x | 2^
1
21
1
,1
N
ii i
ML N
i i
xσµ
σ
=
=
=∑
∑
• Under the i.i.d Gaussian distributed statistics assumption:
which is a normalized version of a FIR filter output,
1
,N
i ii
y w x=
= ∑21/i iw σ=with .
ML Estimate: Gaussian Case
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 44
Under the i.i.d Laplacian distributed statistics assumption:
1
1( ) .2
i
i
xN
i i
f eµ
λµλ
−−
=
=∏x |
( )1 1
ln ( ) ln(2 ) .N N
ii
i i i
xf
µµ λ
λ= =
−= − −∑ ∑x |
( )( )1
ln ( )0.
Ni
i i
f xµ µµ µ λ=
∂ −⎛ ⎞∂= =⎜ ⎟∂ ∂ ⎝ ⎠
∑x | ^
11 | ,N
ML i ii
MED xµλ =
⎛ ⎞= ◊⎜ ⎟
⎝ ⎠
where ◊ denotes the replication operator defined as
with Hence, the weighted median filter output is,
, ,..., ,i
i i i i i
w many
w x x x x◊ =
( )1| .Ni i iy MED w x == ◊
1/ .i iw λ=
ML Estimate: Laplacian Case
12
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 45
Sharpening Filters
Objective: Highlight and enhance fine detailDetails may have been blurred in acquisition process
Method: utilize first- and second-order derivativeDerivatives identify signal changes (details/features)
Derivative is not unique – impose requirements:Must be zero in flat regionsMust be nonzero at the onset of ramps and stepsMust be nonzero along ramps
Second-derivative requirements:Must be zero in flat regionsMust be nonzero at the onset and end of ramps and stepsMust be zero along ramps of constant slope
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 46
Derivatives
Utilize difference equations:First derivative:
Second derivative:
Extend to 2D
( 1) ( )f f x f xx∂
= + −∂
2
2 ( 1) ( 1) 2 ( )f f x f x f xx
∂= + + − −
∂
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 47
Scan Line Derivative example
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 48
Derivative Observations
First-order derivatives generate thick edgesSecond-order derivatives
Have stronger response to detailsProduce a double response at step changesOrder of response strength
Point, line, step
Second-order derivative is therefore preferred for enhancement
Use isotropic (rotation invariant) formulation
13
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 49
Laplacian
Laplacian is the simplest isotropic derivative:
Discrete realization:
Isotropic to 90° rotationsAdd diagonal derivatives to make it 45° isotropic
2 22
2 2
f ffx y
∂ ∂∇ = +
∂ ∂
[ ]2 ( 1, ) ( 1, ) ( , 1) ( , 1) 4 ( , )f f x y f x y f x y f x y f x y∇ = + + − + + + − −
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 50
Mask Implementations & Enhancement
Similar definition produces a sign changeEnhancement adds (subtracts) derivative and observed image
2
2
( , ) ( , ) if the center coefficient of the L
( , ) ( , ) if the center coefficient of the Laplacian mask is positive
( , )f x y f x y
f x y f x yg x y
−∇
+∇= aplacian mask is negative⎧⎪⎨⎪⎩
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 51
Example
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 52
Composite Mask & Example
14
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 53
Unsharp Masking
Unsharp masking process:
Originally a darkroom process combining a blurred negative and positive film
Generalization: high-boost filtering
Using the Laplacian to obtain the sharp image fs(x,y)
( , ) ( , ) ( , )sf x y f x y f x y= −
( , ) ( , ) ( , )hbf x y Af x y f x y= −( , ) ( 1) ( , ) ( , )hb sf x y A f x y f x y= − +
2
2
( , ) ( , ) if the center coefficient of the
( , ) ( , ) if the center coefficient of the Laplacian mask is positive
Af x y f x y
hb Af x y f x yf
−∇
+∇= Laplacian mask is negative⎧⎪⎨⎪⎩
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 54
Mask Implementation
Standard Laplacian sharpening: A=1Increasing A reduces the level of sharpening
Scales (brightens) original image
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 55
High-Boost Example
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 56
First Derivatives
2D first derivative:
Magnitude:
For computational simplicity:
x
y
GG
fxffy
∂⎡ ⎤⎢ ⎥∂⎡ ⎤∇ = = ⎢ ⎥⎣ ⎦ ∂⎢ ⎥⎢ ⎥∂⎣ ⎦
1/ 222f ffx y
⎡ ⎤⎛ ⎞∂ ∂⎛ ⎞∇ = +⎢ ⎥⎜ ⎟ ⎜ ⎟∂ ∂⎝ ⎠ ⎝ ⎠⎢ ⎥⎣ ⎦
x yf G G∇ ≈ +
15
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 57
Sobel Operators
3x3 Sobel operator:
Utilizes x and ydirectional masksDistance weighting minimizes smoothingExtensions:
Increased window sizeDiagonal directions
7 8 9 1 2 3
3 6 9 1 4 7
( 2 ) ( 2 )
( 2 ) ( 2 )
f z z z z z z
z z z z z z
∇ ≈ + + − + +
+ + + − + +
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 58
Sobel Example
Common application: edge detectionThreshold Sobel outputBinary edge mask
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 59
Composite OperationsExample I
SharpenAdd original & Laplacian
Identify edgesSobel operator
Image ProcessingEnhancement in the Spatial Domain
Prof. Barner, ECE Department, University of Delaware 60
Composite OperationsExample II
Thicken identified edgesSmooth Sobel output
Form maskProduct of smoothed Sobel and sharpened image
Sharpened imageAddition of original and mask
Only sharpen edges
Final displayApply power law transformation