Purdue UniversityPage 1
Color Color Image Fidelity Assessor Image Fidelity Assessor *
Wencheng Wu (Xerox Corporation) Zygmunt Pizlo (Purdue University)Jan P. Allebach (Purdue University)
* Research supported by HP Company while Wencheng Wu was at Purdue
Purdue UniversityPage 2
OutlineOutline
• Introduction
• Spatial color descriptor: chromatic difference
• Structure of Color Image Fidelity Assessor (CIFA)
• Psychophysical experiment and its results
• Test examples
• Conclusion
Purdue UniversityPage 3
IntroductionIntroduction(Motivation)(Motivation)
• Image fidelity assessment is important in the development of
imaging systems and image processing algorithms
Create visually lossless reproduction
Allocate efforts on most visible area
• Subjective evaluation is expensive and slow.
Purdue UniversityPage 4
IntroductionIntroduction(Prior work)(Prior work)
• Simple but not working
Root-Mean-Square Error
• Consider structure of HVS and perceptual process
Achromatic: Daly’s VDP, Lubin’s VDM, Taylor’s Achromatic IFA (IFA)
Color: Jin’s CVDM (Daly’s VDP + Wandell’s Spatial CIE Lab)
Purdue UniversityPage 5
IntroductionIntroduction(CVDM vs. CIFA)(CVDM vs. CIFA)
• Both operate along opponent-color coordinates
• Both incorporate results from electrophysiological and
psychophysical exp.
• They differ in a similar way as VDP vs. IFA
CIFA has closer link between the structure of the model and the psychophysical data used by the model
• CIFA normalize the chromatic responses
This discounts luminance effect in chromatic channels
This reduces the dimension of psychometric LUT
Purdue UniversityPage 6
IntroductionIntroduction(Overview of CIFA)(Overview of CIFA)
• Color extension of Taylor’s achromatic IFA
• The model predicts perceived image fidelity
Assesses visible differences in the opponent channels
Explains the nature of visible difference (luminance change vs. color shift)
Color ImageFidelityAssessor(CIFA)
Ideal
Rendered
Viewing parameters
Image mapsof predicted
visibledifferences
Purdue UniversityPage 7
Chromatic differenceChromatic difference(Definition)(Definition)
• Objective: evaluate the spatial interaction between colors
• First transform CIE XYZ to opponent color space (O2,O3) *
* X. Zhang and B.A. Wandell, “A SPATIAL EXTENSION OF CIELAB FOR DIGITAL COLOR IMAGE REPRODUCTION”, SID-97
tieschromaticiopponent )/,/(),( 3232 YOYOoo
.3,22
minmax
ioo
ci
• Then normalize to obtain opponent chromaticities (o2,o3)
• Define chromatic difference (analogous to luminance contrast c1)
Z
Y
X
O
O
Y
501.059.0086.0
077.029.0449.0
010
3
2
Luminance Red-Green
Blue-Yellow
Purdue UniversityPage 8
Opponent color representationOpponent color representation
(13.3,o2,0.17) (13.3,0.24,o3)(Y,0.24,0.17)
(Y,o2,o3)
Purdue UniversityPage 9
)16/cos(2.0 J)16/cos(1.0 J
Chromatic differenceChromatic difference(illustration)(illustration)
• Chromatic difference is a measure of chromaticity variation
• Chromatic difference is a spatial feature derived from opponent
chromaticity that has little dependence upon luminance
174438.0,235924.0,885.6),,( :)( pixelat gray Floyd"" 32 ooYI.J
)16/cos(05.0 J
0.1 0.10.20.05
• Chromatic difference is the amplitude of the sinusoidal grating
)16/cos(1.0 J
Purdue UniversityPage 10
CIFACIFA
Ideal Y Image
Rendered Y Image
Ideal O2 Image
Rendered O2 Image
Ideal O3 Image
Rendered O3 Image
Blue-yellowIFA
Red-greenIFA
Achromatic*IFA
Chromatic IFAs
* Previous work of Taylor et al
(Y,O2,O3): Opponent representation of an image
Multi-resolution Y images
Image map of predictedvisible luminance
differences
Image map of predictedvisible blue-yellow
differences
Image map of predictedvisible red-green
differences
Purdue UniversityPage 11
PsychometricLUT (f,o2,c2)
Chromatic diff.discriminationRed-green IFARed-green IFA
PsychometricSelector
ChannelResponsePredictor
LimitedMemory
Prob. Sum.
LowpassPyramid
LowpassPyramid
Chromatic Diff.Decomposition
Chromatic Diff.Decomposition
c
+
–
Adaptation level
Contrast Decomposition
Contrast Decomposition
Achromatic IFAAchromatic IFAPsychometric
LUT (f,Y,c1)
Lum. contrastdiscrimination
Contrast: luminance contrast & chromatic difference
Purdue UniversityPage 12
IFA componentsIFA components
• Psychometric LUT Results from psychophysical experiment
Stored in the form of Lookup-Table: (f,Y,c1), (f,o2,c1), (f,o3,c1)
Time consuming, but it is done off-line
• Image processing: Lowpass pyramid: create 5 multi-resolution images
» Lowpass filtering + 2 in horizontal and vertical direction
» Normalized by Y images if it is a chromatic IFA
Signal decomposition: create 8 orientation-specific contrast or chromatic-difference images at each resolution
Lowpass pyramid + Signal decomposition: 40 (5 levels 8 orientations) visual channels for each image pixel
Purdue UniversityPage 13
IFA componentsIFA components(cont’d)(cont’d)
• Image processing (continued):
Psychometric selector: for each pixel at each visual channel, find discrimination threshold by choosing appropriate data from LUT
Channel response predictor: for each pixel at each visual channel, convert chromatic difference to discrimination probability
Limited memory probability summation: for each pixel, combine discrimination probability across all 40 visual channel
3,2,1645.12
erf5.05.0
i
cp
i
5
1
)1(1k
kpP
Purdue UniversityPage 14
Estimating parameters of LUTEstimating parameters of LUT(Stimulus: Isoluminant Gabor patch)(Stimulus: Isoluminant Gabor patch)
• Red-green (O2 or o2) stimulus
Keep Y, O3 (o3) constant
Let O2=Yo2+Yc2cos(.)e(.) or equivalently o2’ =o2+c2cos(.)e(.)
• (Y,o2,o3) specifies the background color, c2 is the chromatic difference
3.0,2.0,885.6),,( :RG1 Floyd"" 32 ooY
)()cos(2.0 e)cos(2.0
Gabor patch f, o2, c2
Purdue UniversityPage 15
away m 1.5 from viewed
1.0 ,1.0 ee,cycle/degr 4 ,)3.0,2.0,885.6(),,( 232 ccfooY
Estimating parameters of LUTEstimating parameters of LUT(Psychophysical method)(Psychophysical method)
• Red-green stimulus: (Y,o2,o3) specifies the background color, c2 is
the ref. chromatic difference
• Which stimulus has less chromatic difference?
3.0,2.0,885.6 3.0,2.0,885.6
)(2 )cos( ec )(
2 )cos()( ecc
-0.02 -0.01 0 0.01 0.020
0.2
0.4
0.6
0.8
1
c
Purdue UniversityPage 16
-0.02 -0.01 0 0.01 0.020
0.2
0.4
0.6
0.8
1Subject WW’s responses
prob
abil
ity
c
)0.0065,0.0011(N
Estimating parameters of LUT Estimating parameters of LUT (Data analysis)(Data analysis)
• Fit subject’s responses to a Normal distribution using probit analysis
• Record the standard deviation as the discrimination threshold
• LUT: rg(f,o2,c2)
Purdue UniversityPage 17
Estimating parameters of LUTEstimating parameters of LUT(List of experimental conditions)(List of experimental conditions)
indicate spatial frequency of 1, 2, 4, 8, 16 cpd
Purdue UniversityPage 18
Representative resultsRepresentative results
• Results for f = 16, 8, 4, 2, 1 cycle/deg are drawn in red, green, blue, yellow, and black.
• Threshold is not affected strongly by the reference chromatic difference
• Chromatic channels function like low-pass filters
-0.2 0 0.2 0.4 0.60
0.05
0.1
0.15
0.2
-0.1 0 0.1 0.2 0.30
0.02
0.04
0.06
Reference c3Reference c2
Thr
esho
ld
Thr
esho
ld
Red-green discrimination atRG1:(Y,o2,o3)=(5,0.2,-0.3)
Blue-yellow discrimination atBY1:(Y,o2,o3)=(5,0.3,0.2)
Purdue UniversityPage 19
CIFA output for example distortionsCIFA output for example distortions(Hue change)(Hue change)
Luminance R-G B-Y
Purdue UniversityPage 20
CIFA output for example distortionsCIFA output for example distortions(Blurring)(Blurring)
Luminance R-G B-Y
Purdue UniversityPage 21
CIFA output for example distortionsCIFA output for example distortions(Limited gamut)(Limited gamut)
Luminance R-G B-Y
Purdue UniversityPage 22
ConclusionConclusion
• CIFA provides good assessment of the perceived visible
differences over a range of image contents and distortion types
• Chromatic difference describes the color percept of HVS
efficiently
• Suggestions on future directions
Add DC component in the LUT in chromatic IFAs
Subjective validation
Improve spatial localization
Take dependency between visual channels into account (in prob. Sum. stage)
Purdue UniversityPage 23
CIFA output for example distortionsCIFA output for example distortions (Limited color quantization)(Limited color quantization)
Luminance R-G B-Y
Purdue UniversityPage 24
CIFA output for example distortionsCIFA output for example distortions (Limited gamut)(Limited gamut)
Luminance R-G B-Y
Purdue UniversityPage 25
CIFA output for example distortionsCIFA output for example distortions(Increased saturation)(Increased saturation)
Luminance R-G B-Y