Image Quality Measures
Omar Javed, Sohaib KhanDr. Mubarak Shah
Factors Affecting Registration Performance
• Mission image quality and content• Reference image quality and content• Mission-Reference differences• Viewing geometry• Quality of DEM• Method of registration
Test Images
• 5 image sequences were used as test images – 08 Oct 99 Image Sequence– 13 Oct 99 Image Sequence– 15 Oct 99 Image Sequence– 16 Oct 99 Image Sequence– 19 Oct 99 Image Sequence
In This Presentation...
• Factors affecting registration performance. • Image quality and content measures
– SNR estimation – Texture measures– Gabor filters
Properties of Mission Imagery Affecting Registration Performance
• Scene Content– Homogenous texture i.e. no distinctive features– Example Images
Properties of Mission Imagery Affecting Registration Performance
• Aperture Problem– Presence of roads or homogenous elongated
features causes error in registration along the direction of elongation
Properties of Mission Imagery Affecting Registration Performance
• Extreme blur
Properties of Mission Imagery Affecting Registration Performance
• Spurious weather phenomenon e.g. clouds, haze ..
Image Quality and Content Measures
• SNR estimation• Texture Measures• Gabor Filters
Blind SNR Estimation• A method to estimate the quality of image is
based on quantityQ=2
fi dr
• The intensity image fi can be modeled by a mixture of Rayleigh pdfs
)2
(
12
22
2
)( i
rm
i ii erwrfi
Algorithm For SNR Estimation• Compute the horizontal and vertical
derivatives of the image• Calculate the gradient magnitude ‘ΔΙ’ from
the derivatives.• Obtain a Histogram of gradient intensity
values from ΔΙ.• Count the number of pixels > 2μ , where μ
is mean of ΔΙ .• Normalize by total number of pixels.
ResultsResults
• 08 Oct 99 SequenceTotal Images = 70
Images with error=12
Unregistered Images=17
Images identified by metric as unregisterable=20
# of false +ves=11
# of false -ves= 20
Misclassification Error= 44.28%
ResultsResults
• 13 Oct 99 SequenceTotal Images = 84
Images with error=18
Unregistered Images=11
Images identified by metric as unregisterable=21
# of false +ves=9
# of false -ves= 17
Misclassification Error= 30.95%
ResultsResults
• 15 Oct 99 SequenceTotal Images = 115
Images with error=6
Unregistered Images=0
Images identified by metric as unregisterable=0
# of false +ves=0
# of false -ves= 6
Misclassification Error= 5.21%
ResultsResults
• 16 Oct 99 SequenceTotal Images = 169
Images with error=19
Unregistered Images=39
Images identified by metric as unregisterable=25
# of false +ves=10
# of false -ves= 44
Misclassification Error= 31.95%
ResultsResults
• 19 Oct 99 SequenceTotal Images = 172
Images with error=15
Unregistered Images=22
Images identified by metric as unregisterable=19
# of false +ves=17
# of false -ves= 35
Misclassification Error= 30.23%
Discussion of Results• Images labeled as low quality
– Red squares indicates large registration error or exclusion from registration
Discussion of Results• Images labeled as high quality
– Red squares indicates large registration error or exclusion from registration
Suitability as an Image Metric• Advantages
– Extreme blur is detected and corresponds well with registration error.
– Low computation time
• Disadvantages– Cloud detection is not robust.– Feature less images are a major cause of
registration error. SNR is not able to detect these images robustly.
Texture
• Gray Level Co-occurrence Matrices (GLCMs)– 2D histogram which encodes spatial relations
• parameters: direction, distance,quantization-levelwindow-size
– Measures are computed on the GLCM• entropy, contrast, homogeneity, energy
Computing GLCM• A GLCM P[i,j] is defined by
– specifying displacement vector d=(dx,dy)– Counting all pairs of pixels separated by d having
gray levels I and j.Input image
Window size
i
j
Distance and DirectionRelationship d
1 ……… i ………. 255
1 ….. j …
……
. 255
+1
P(i, j)
Quantization level
GLCM Measures
• Entropy – Randomness of gray level distribution
• Energy:– uniformity of gray level in a region
i j
jiPjiPEntropy ],[log],[
],[ 2 jiPEnergyi j
GLCM Measures
• Contrast– Measure of difference between gray levels
• Homogeneity– Measure of similarity of texture
],[)( 2 jiPjiContrasti j
i j jijiPyHomogeneit
1],[
Contrast
100 200 300
100
200
300
400100 200 300
100
200
300
400
100 200 300
100
200
300
400100 200 300
100
200
300
400
Entropy
Homogeneity Energy
Contrast
GLCM measures
ResultsResults
• 08 Oct 99 SequenceTotal Images = 70
Images with error=12
Unregistered Images=17
Images identified by metric as unregisterable=19
# of false +ves=5
# of false -ves= 15
Misclassification Error= 28.57%
ResultsResults
• 13 Oct 99 SequenceTotal Images = 84
Images with error=18
Unregistered Images=11
Images identified by metric as unregisterable=12
# of false +ves=7
# of false -ves= 26
Misclassification Error= 39.28%
ResultsResults
• 15 Oct 99 SequenceTotal Images = 115
Images with error=6
Unregistered Images=0
Images identified by metric as unregisterable=15
# of false +ves=15
# of false -ves= 6
Misclassification Error= 18.26%
ResultsResults
• 16 Oct 99 SequenceTotal Images = 169
Images with error=19
Unregistered Images=39
Images identified by metric as unregisterable=32
# of false +ves=13
# of false -ves= 41
Misclassification Error= 31.95%
ResultsResults
• 19 Oct 99 SequenceTotal Images = 172
Images with error=15
Unregistered Images=22
Images identified by metric as unregisterable=47
# of false +ves=18
# of false -ves= 8
Misclassification Error= 15.11%
Discussion of Results• Images labeled as low quality
– Red squares indicates large registration error or exclusion from registration
Discussion of Results• Images labeled as high quality
– Red squares indicates large registration error or exclusion from registration
Suitability as an Image Metric• Advantages
– Homogeneous texture is detected though detection is not robust.
• Disadvantages– It is difficult to fine tune the several parameters
of GLCM’s so that consistent results are obtained for a variety of images.
– Clouds are not detected.– Blur is not detected.
Gabor Filter
• The Gabor function
– is a complex sinusoid centered at frequency (U,V) modulated by a Guassian envelop .
• Gabor function can discriminate between textures
)(22)(
22
22
21),,,,( VyUxj
yx
eeVUyxh
Gabor Filter
• Experiments were done with the following values– Variance of Guassian = 30– Four Gabor kernels
• 1 Horizontal• 1 Vertical• 2 Diagonal
Gabor Kernels
Calculation of Quality metric
• Normalize image intensity values (0 to 255).– Calculate mean of intensity values.– Subtract mean from all intensity.– Add 128 (middle value).
• Determine Gabor response of the image.– Generate four Gabor kernels.– Convolve each kernel with the image.– Multiply the four results.
Calculation of Quality metric
• Perform connected component analysis and clean up small areas of response.
• Count the number of pixels Np in the response area. Normalize by total number of pixels.
• If Np <Tlow label image as low quality.• If Np >Thigh label image as high quality.
Calculation of Quality metric
• If both the previous conditions are not met then calculate spatial covariance of Gabor response.
• If spatial covariance is < Ts label image as low quality otherwise label image as high quality.
Results
• Images of Gabor response
Results
• Result after convolution from vertical kernel
Results
• Result after convolution from horizontal kernel
Results
• Result after convolution from diagonal kernel
Results
• Result after convolution from diagonal kernel
Results
• Results after multiplication and thresholding
Results
• Images of Gabor response
Results
• Images of Gabor response
Results
• Images of Gabor response
Results
• Images of Gabor response
Results
• Images of Gabor response
ResultsResults
• 08 Oct 99 SequenceTotal Images = 70
Images with error=12
Unregistered Images=17
Images identified by metric as unregisterable=26
# of false +ves=2
# of false -ves= 5
Misclassification Error= 10.00%
ResultsResults
• 13 Oct 99 SequenceTotal Images = 84
Images with error=18
Unregistered Images=11
Images identified by metric as unregisterable=12
# of false +ves=4
# of false -ves= 21
Misclassification Error= 29.76%
ResultsResults
• False +ves
• Difficulty– Correct Detection
ResultsResults
• False -ves
ResultsResults
• 15 Oct 99 SequenceTotal Images = 115
Images with error=6
Unregistered Images=0
Images identified by metric as unregisterable=0
# of false +ves=0
# of false -ves= 6
Misclassification Error= 5.21%
ResultsResults
• 16 Oct 99 SequenceTotal Images = 169
Images with error=19
Unregistered Images=39
Images identified by metric as unregisterable=46
# of false +ves=7
# of false -ves= 22
Misclassification Error= 17.15%
ResultsResults
• 19 Oct 99 SequenceTotal Images = 172
Images with error=15
Unregistered Images=22
Images identified by metric as unregisterable=49
# of false +ves=22
# of false -ves= 10
Misclassification Error= 18.6%
Results
• A Sample of Images labeled as low quality– Featureless images
Results
• A Sample of Images labeled as low quality– Cloudy Images
– Blur
Results
• A Sample of Images labeled as high quality
Suitability as an Image Metric• Advantages
– Accurate estimation of amount of texture in an image.
– It can identify hazy, cloudy or featureless images.
– Prediction of success/failure of registration possible.
• Disadvantages– High computation time.