MotivationRecovering HDR Radiance Maps
Tonemapping
High Dynamic Range Imaging
Mathias Eitz, Claudia Stripf
TU Berlin
January, 16th 2007
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
Outline
1 Motivation
2 Recovering HDR Radiance MapsIdeaComputing Camera Response CurveConstructing HDR Radiance Map
3 TonemappingIdeaGlobal OperatorsReinhard Local Operator
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
Very High World Dynamic Range
10 −6 10 6
illuminance human
high dynamic range
10 −6 10 6
illuminance picture
low dynamic range
Figure: High dynamic vs. low dynamic range
Human can discern very high range of brightness values
Photography limited to much lower dynamic range
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
Tasks
Compute a HDR picture from multiple exposures
But: Dynamic range of display usually very limited, cannotdisplay HDR pic
Two tasks
1 Compute HDR image
2 Tonemapping to compress contrast into displayable range
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
What We Have Done
To solve those two tasks, we have implemented two papers:
P. Debevec - Recovering HDR Radiance Maps fromPhotographs - SIGGRAPH’97Reinhard et al - Photographic Tone Reproduction for DigitalImages - SIGGRAPH’02
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Outline
1 Motivation
2 Recovering HDR Radiance MapsIdeaComputing Camera Response CurveConstructing HDR Radiance Map
3 TonemappingIdeaGlobal OperatorsReinhard Local Operator
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Idea
Take multiple exposures of the same scene
Vary exposure time from small to long to capture completedynamic range
Compute HDR image from those exposures
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Multiple Exposure Photography
10 −6 10 6
illuminance scene
dynamic range of photographed scene
10 −6 10 6
illuminance picture
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Multiple Exposure Photography
10 −6 10 6
illuminance scene
dynamic range of photographed scene
10 −6 10 6
illuminance picture
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Multiple Exposure Photography
10 −6 10 6
illuminance scene
dynamic range of photographed scene
10 −6 10 6
illuminance picture
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Multiple Exposure Photography
10 −6 10 6
illuminance scene
dynamic range of photographed scene
10 −6 10 6
illuminance picture
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Outline
1 Motivation
2 Recovering HDR Radiance MapsIdeaComputing Camera Response CurveConstructing HDR Radiance Map
3 TonemappingIdeaGlobal OperatorsReinhard Local Operator
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Computing Camera Response Curve
Camera response curve f tells us, how scene radiance E ismapped to pixel brighness Z
Using the inverse of f allows us to reproduce actual sceneradiance E
f is different for each camera
compute f from a series of exposures
Camera response curve
Non-linear mapping f from scene radiance Ei and exposuretime ∆tj to pixel brightness Zi
Zij = f (Ei∆tj)
We want to determine f
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Computing Camera Response Curve
Camera response curve f tells us, how scene radiance E ismapped to pixel brighness Z
Using the inverse of f allows us to reproduce actual sceneradiance E
f is different for each camera
compute f from a series of exposures
Camera response curve
Non-linear mapping f from scene radiance Ei and exposuretime ∆tj to pixel brightness Zi
Zij = f (Ei∆tj)
We want to determine f
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Camera Response Curve - Mathematical
Define equation system
Zij = f (Ei∆tj)
f −1 (Zij) = Ei∆tj
ln f −1 (Zij) = ln Ei + ln∆tj
Set g = ln f −1
Solve for g
Solve overdetermined system of equationsg (Zij) = ln Ei + ln∆tj
Solved in a least-squared error sense
Result: Camera response function f
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Camera Response Curve - Mathematical
Define equation system
Zij = f (Ei∆tj)
f −1 (Zij) = Ei∆tj
ln f −1 (Zij) = ln Ei + ln∆tj
Set g = ln f −1
Solve for g
Solve overdetermined system of equationsg (Zij) = ln Ei + ln∆tj
Solved in a least-squared error sense
Result: Camera response function f
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Computation of Response Curve of Canon EOS 350D
Computed from a series of 9 exposures
Exposure time from 14000 to 15
Each exposure two stops apart (i.e. 14000 , 1
1000 , 1250 ,...,15)
λ = 50
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
Response Curve of Canon EOS 350D
−6 −4 −2 0 2 40
50
100
150
200
250
300
log Exposure X
Pix
el V
alue
Z
−8 −6 −4 −2 0 2 40
50
100
150
200
250
300
log Exposure X
Pix
el V
alue
Z
−6 −4 −2 0 20
50
100
150
200
250
300
log Exposure X
Pix
el V
alue
Z
−8 −6 −4 −2 0 2 40
50
100
150
200
250
300
log Exposure X
Pix
el V
alue
Z
Figure: Recovered response curve f
Response Curve of Canon EOS 350D
−7 −6 −5 −4 −3 −2 −1 0 1 2 30
50
100
150
200
250
300
log Exposure X
Pix
el V
alue
Z
Figure: Recovered response curve f
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Outline
1 Motivation
2 Recovering HDR Radiance MapsIdeaComputing Camera Response CurveConstructing HDR Radiance Map
3 TonemappingIdeaGlobal OperatorsReinhard Local Operator
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Idea
What we know
Camera response curve
Multiple exposures of same scene at know exposre times
Each pixel in the scene is correctly exposed at least once
Use this information to assemble HDR image
Each pixel is a weighted combination from the correspondingpixels of all exposures
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Assemble HDR Radiance Map - Mathematical
Radiance
lnEi = g (Zij) − ∆tj
Weighted average
Use weighted average of all exposures to create HDR map
lnEi =sumP
j=1(w(Zij)(g(Zij)−∆tj))sumP
j=1(w(Zij))
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
Assemble HDR Radiance Map - Mathematical
Radiance
lnEi = g (Zij) − ∆tj
Weighted average
Use weighted average of all exposures to create HDR map
lnEi =sumP
j=1(w(Zij)(g(Zij)−∆tj))sumP
j=1(w(Zij))
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaComputing Camera Response CurveConstructing HDR Radiance Map
How to Display a HDR Image
We have now generated a HDR image
Dynamic range of resulting HDR image: ≈ 1 : 100000
Dynamic range of computer screen: much lower
Problem We cannot directly display the resulting HDR image
Solution Tone mapping
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaGlobal OperatorsReinhard Local Operator
Outline
1 Motivation
2 Recovering HDR Radiance MapsIdeaComputing Camera Response CurveConstructing HDR Radiance Map
3 TonemappingIdeaGlobal OperatorsReinhard Local Operator
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaGlobal OperatorsReinhard Local Operator
Idea
Compress high dynamic range to displayable low dynamicrange
Need to preserve details and colors
10 −6 10 6
illuminance scene
dynamic range of photographed scene
10 −6 10 6
illuminance picture
Figure: Tone mapping
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaGlobal OperatorsReinhard Local Operator
Luminance Map
Tone mapping is usually done on the luminance map
Then colors are reapplied
Luminance map
L (x , y) = 0.2 ·Red (x , y) + 0.7 ·Green (x , y) + 0.1 ·Blue (x , y)
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaGlobal OperatorsReinhard Local Operator
Outline
1 Motivation
2 Recovering HDR Radiance MapsIdeaComputing Camera Response CurveConstructing HDR Radiance Map
3 TonemappingIdeaGlobal OperatorsReinhard Local Operator
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaGlobal OperatorsReinhard Local Operator
Global Operators
Definition
Tone mapping operator Maps a HDR image into displayable range[0, 1]
Linear Linear mapping of HDR image into range [0, 1]
Logarithmic Logarithmic mapping of HDR image into [0, 1]
Reinhard Global Reinhard operator L (x , y) = L(x ,y)1+L(x ,y)
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaGlobal OperatorsReinhard Local Operator
Outline
1 Motivation
2 Recovering HDR Radiance MapsIdeaComputing Camera Response CurveConstructing HDR Radiance Map
3 TonemappingIdeaGlobal OperatorsReinhard Local Operator
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaGlobal OperatorsReinhard Local Operator
Local Operators
Adapt to the luminance in the current local neighbourhood
Reinhard local operator
L (x , y) = L(x ,y)1+V (x ,y)
V is the average luminance of a local neighbourhood of acertain size
Find biggest local neighbourhood with fairly even contrast
Done with a Difference of Gaussians approach at multiplescales s
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging
MotivationRecovering HDR Radiance Maps
Tonemapping
IdeaGlobal OperatorsReinhard Local Operator
Reinhard Local Operator
Search the biggest pixel neighbourhood s where|V (x , y , s)| < eps
V is a function of the Difference of Gaussians, telling us wherestrong contrast changes occur
Mathias Eitz, Claudia Stripf High Dynamic Range Imaging