Date post: | 15-Jan-2016 |
Category: |
Documents |
Upload: | peyton-cashmore |
View: | 217 times |
Download: | 0 times |
Denoising photographs using dark frames optimized by quadratic programming
Manuel Gomez-Rodriguez* Jens Kober†
Bernhard Schölkopf†
†Max Planck Institute for Biological Cybernetics
Tübingen
*Electrical Engineering Department
Stanford University
Long exposure photographs• Long exposure photographs
(e.g., astronomical photographs) contain substantial amounts of noise.
• Dark current noise is the dominant source of noise in long exposure photographs.
• We have access to samples of the joint distribution of the noise of our camera using bias frames and dark frames
Noise profile
• A bias frame – raw image taken with closed shutter and exposure time ~ 0 seconds. The bias value is cased by the readout noise.
• A dark frame – raw image taken with closed shutter and nonzero exposure time. It contains a bias frame plus a component that increases with exposure time, in a way that depends on several other factors (i.e. temperature, ISO setting, …)
• A light frame – raw image to denoise.
The problem
• Given the observed image I + D and a few points sampled from the multidimensional noise distribution, we want to estimate I.
X1
X2
XN
D
I
Noise distribution
The problem
• We want to include the joint statistics of the sensor noise in our denoising method
- How should we combine the dark frames?- Does it generalize to different conditions?- Is the problem computationally tractable?
Denoised imageGiven a noisylong exposure setting
A library of dark frames
Naïve approach
• Single dark frame: record a dark frame of matching exposure time after each long exposure. This dark frame is subtracted from the light frame- It is implemented on commercial cameras- It doubles the amount of time- The temperature tends to change- One-point sample from the joint distribution
of the noise
Average of dark frames approach
• Average of dark frames: a set of dark frames under conditions matching the ones of the light frame. The mean of the set is substracted from the light frame.- Used, for example, in astrophotography- Works well for professional cooled CCDs
with precise temperature control- Better estimate of the expected noise
(multi-point sample)
Our approach
• The distribution of noise for a given camera depends on various conditions, including temperature, ISO settings and exposure time.
• If we knew the conditions for the image to be denoised, we should ideally use a library that matches the conditions of the image. But,- The exact temperature is usually unknown- We cannot store dark frames for every
possible condition
Our approach
• Our method generates a synthetic dark frame from the convex hull of the dark frames D(1)…D(N), taken under different conditions,
such that subtracting it from a noisy image optimizes a quality measure or prior for the class of images to denoise
X1
X2
XN
D
I
Noise distribution
Image prior
Optimization problem
• If the quality measure is the smoothness of the image (i.e. discrete derivative), the convex optimization problem can be formulated as,
where
is the variable, is a real convex cost function, is a set of evaluation points and is the 8-neighbor set of the location in the raw image
•
Quadratic programming problem
• If a quadratic penalty function, , is chosen, the optimization problem is equivalent to the following quadratic program (QP)
where
Solution of the QP
• A solution that generalizes well to the full image should be sparse because only the dark frames that were taking under similar conditions as the noisy image should be used for denoising; this is enforced by the constraints and , implying
= 1 • Our method also allows to estimate in an
indirect way the exposure time, temperature and ISO of a photograph
Evaluation points
• As evaluation points, we use points that have high variance between dark frames and,- The selection of evaluation points is done only
once for a specific camera and a relatively low number of evaluation points (~1000) is enough
- The complexity does not depend on the size of the images but the # of dark frames
- As the solution is usually sparse, we only need to load a few full dark frames to denoise
Evaluation
• The same evaluation metric in the training set S and the test set T to numerically evaluate the performance; however S and T are disjoint → True generalization performance
• Dark frames taken with a Canon EOS 1Ds with,- ISO of 800, 1000, 1250 - Exposure times 1, 2, 4, 8,… 128 seconds, and
21 seconds - Various temperature conditions
have been used for the analysis
Evaluation
• Three problem instances in increasing order of difficulty are proposed to validate our method,
Temperature Exposure time
1st problem Constant and matching the noisy image
Variable, including the same exposure time as the noisy image
2nd problem Variable Variable, including the same exposure time as the noisy image
3rd problem Variable Variable, not including the same exposure time as the noisy image
Evaluation: 1st case
• Light frame with ISO 800, 16 seconds of exposure time
• 18 dark frames: constant temperature, variable exposure time
Correct exposure time!
Not used!
Evaluation: 2nd case
• Light frame: ISO 1000, 16 seconds of exposure time
• 175 dark frames: variable temperature, variable exposure time
Correct exposure time!
Not used!
Evaluation: 3rd case
• Light frame: ISO 1000, 21 seconds of exposure time
• 175 dark frames: variable temperature, variable exposure time (not inc. 21 sec)
200 evaluation points!
Noisy image Our method
Horsehead nebula Barnard 33 in nebula IC 434, flame nebula NGC 2024, Canon EOS 5D with 300mm f/2.8 lens
Our methodBilateral filter
Horsehead nebula Barnard 33 in nebula IC 434, flame nebula NGC 2024, Canon EOS 5D with 300mm f/2.8 lens
Wavelet denoising Our method
Horsehead nebula Barnard 33 in nebula IC 434, flame nebula NGC 2024, Canon EOS 5D with 300mm f/2.8 lens
Wavelet denoising Our method + wavelet denoising
Horsehead nebula Barnard 33 in nebula IC 434, flame nebula NGC 2024, Canon EOS 5D with 300mm f/2.8 lens
Part of Orion constellation. Combination of ca. 10 R, G, and B images, denoised using the proposed method. Canon 200mm lens, SBIG CCD camera using Kodak KAF 11002 CCD chip
Magnified detailed (Running Man Nebula)
Conclusions
• A relatively simple method with low complexity can help denoise long exposure images in raw format
• Our method can beneficially be combined with image-based noise reduction methods
• If available, our method could use evaluation points from the "optical black” (an area around the main image portion of the sensor which does not get light).
• We believe that the proposed method can become a practical tool for digital photography
Thank You!