Post on 30-Dec-2015
transcript
Sharpening from Shadow:
November 2012The Role of Bright Pixels in Illumination Estimation
Hamid Reza Vaezi JozeMark S. DrewGraham D. FinlaysonPetra Aurora Troncoso ReySchool of Computer ScienceSimon Fraser University School of Computer SciencesThe University of East Anglia
OutlineMotivationRelated researchExtending the white-patch hypothesisThe effect if bright pixels in well-known methodsThe bright-pixels frameworkFurther experimentConclusion
2
MotivationRelated researchExtending the white-patch hypothesisThe effect if bright pixels in well-known methodsThe bright-pixels frameworkFurther experimentConclusion
2MotivationWhite-Patch methodOne of the first colour constancy methodsEstimates the illuminant colour by the max response of three channelsFew researchers or commercial cameras use it nowRecent research reconsider white patch Local mean calculation as a preprocessing can significantly improve[Choudhury & Medioni (CRICV09)] [Funt & Li (CIC2010)] Analytically, the geometric mean of bright (specular) pixels is the optimal estimate for the illuminant, based on dichromatic model[Drew et al. (CPCV12)]
3
White-Patch methodOne of the first colour constancy methodsEstimates the illuminant colour by the max response of three channelsFew researchers or commercial cameras use it nowRecent research reconsider white patch Local mean calculation as a preprocessing can significantly improve[Choudhury & Medioni (CRICV09)] [Funt & Li (CIC2010)] Analytically, the geometric mean of bright (specular) pixels is the optimal estimate for the illuminant, based on dichromatic model[Drew et al. (CPCV12)]3Bright Pixels4
Light SourceHighlightsWhite surface
Just a bright surface4Previous Research5
White PatchLocal mean calculation as a preprocessing step for White PatchUsing Specular ReflectionSpecular reflection colour is same as the illumination within a Neutral Interface ReflectionIt usually includes the bright areas of imageIllumination estimation methodIntersection of dichromatic planes [Tominaga and Wandell (JOSA89)]Intersection of the lines generates by chromaticity values of pixels of each surface in the CIE chromaticity diagram by [Lee (JOSA86)] Extending Lees algorithms by constraint on the colours of illumination White PatchLocal mean calculation as a preprocessing step for White PatchUsing Specular ReflectionSpecular reflection colour is same as the illumination within a Neutral Interface ReflectionIt usually includes the bright areas of imageIllumination estimation methodIntersection of dichromatic planes [Tominaga and Wandell (JOSA89)]Intersection of the lines generates by chromaticity values of pixels of each surface in the CIE chromaticity diagram by [Lee (JOSA86)] Extending Lees algorithms by constraint on the colours of illumination
5Grey-based illumination estimation Grey-worldThe average reflectance in the scene is achromaticShade-of-greyMinkowski p-normGrey-edgeThe average of the reflectance differences in a scene is achromatic
6
Grey-based illumination estimation Grey-worldThe average reflectance in the scene is achromaticShade-of-greyMinkowski p-normGrey-edgeThe average of the reflectance differences in a scene is achromatic
6Extending the White Patch Hypothesis7
Let us extend white-patch hypothesis that there is always include any of: white patch, specularities, or light source in an imageGamut of bright pixels, in contradistinction to maximum channel response of the White-Patch method, which include the brightest pixels in the image Removing clipped pixels (exceed 90% of the dynamic range)Define bright pixels as the top T % of luminance given by R+G+B.What is the probability of having an image without strong highlights, source of light, or white surface in the real world?
Extending the White Patch HypothesisLet us extend white-patch hypothesis that there is always include any of: white patch, specularities, or light source in an imageGamut of bright pixels, in contradistinction to maximum channel response of the White-Patch method, which include the brightest pixels in the image Removing clipped pixels (exceed 90% of the dynamic range)Define bright pixels as the top T % of luminance given by R+G+B. What is the probability of having an image without strong highlights, source of light, or white surface in the real world?
7Simple ExperimentExperiment whether or not the actual illuminant colour falls inside the 2D gamut of top 5% brightness pixelsSFU Laboratory Dataset : 88.16%ColorChecker : 74.47%GreyBall : 66.02%
8
White surfaceSpecularityFAILThe Effect of Bright Pixels on Grey-base methods9
ColorChecker Dataset Experiment the effect of bright pixels Run grey-based method for the top 20% brightness pixels in each image, and compare to using all image pixels (colour)
Using one fifth of the pixels performance is better or equal
9The Effect of Bright Pixels on Gamut Mapping methodWhite-patch gamut and canonical white-patch gamut introduced [Vaezi Joze & Drew (ICIP12)]White-patch gamut is the gamut of top 5% bright pixels in an image
10
Adding new constraints based on the white-patch gamut to standard Gamut Mapping constraints outperforms the Gamut Mapping method and its extensions.
Canonical gamut vs. WP canonical gamut10The Bright-Pixels FrameworkIf these bright pixels represent highlights, a white surface, or a light source, they approximate the colour of the illuminantTry Mean, Median, Geomean, p-norm (p=2,p=4) for top T% brightness
11
11The Bright-Pixels FrameworkA local mean calculation can help: Resizing to 64 64 pixels by bicubic interpolationMedian filtering Gaussian blurring filter
It does not help so much on these images
12
ColorChecker Dataset 12DatasetSFU Laboratory [Barnard & Funt (CRA02)]321 images under 11 different measured illuminantsReprocessed version of ColorChecker [Gehler et al. (CVPR08)] 568 images, both indoor and outdoorGreyBall [Cieurea & Funt (CIC03)]11346 images extracted from video recorded under a wide variety of imaging conditionsHDR dataset [Funt et al. (2010)]105 HDR images
13
The Bright-Pixels Method14
Remove clipped pixels Do local mean {no, Median, Gaussian, Bicubic }Select top T% brightness pixels Threshold = {.5%,1%,2%,5%,10%}Estimate illuminant by shade of grey eq. p = {1,2,4,8} if the estimated illuminant is not in the possible illuminant gamut use grey-edge
14Further ExperimentComparison with well-known colour constancy methods
15
15Optimal parameters16
pTblurringSFU Laboratory Dataset2.5 % no Color Checker Dataset22% GaussianGreyBall Dataset21% noHDR Dataset81%GaussianGaussian for high resolution images and no blurring for lower resolution imagesEven .5% threshold is enough for in-laboratory images, for real images threshold should be 1-2%
16Conclusion17Based on current datasets in the field we saw that the simple idea of using the p-norm of bright pixels, after a local mean preprocessing step, can perform surprisingly competitively to complex methods.
Either the probability of having an image without strong highlights, source of light, or white surface in the real world is not overwhelmingly great or the current color constancy datasets are conceivably not good indicators of performance with regard to possible real world images.
Questions?Thank you.18