Date post: | 05-Jan-2016 |
Category: |
Documents |
Upload: | sharleen-welch |
View: | 214 times |
Download: | 0 times |
Inverse Global Illumination:Inverse Global Illumination:Recovering Reflectance Models of Real Recovering Reflectance Models of Real
Scenes from PhotographsScenes from Photographs
Inverse Global Illumination:Inverse Global Illumination:Recovering Reflectance Models of Real Recovering Reflectance Models of Real
Scenes from PhotographsScenes from Photographs
Computer Science Division
University of California at Berkeley
Computer Science Division
University of California at Berkeley
Yizhou Yu, Paul Debevec, Jitendra Malik & Tim Hawkins
Image-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and Rendering
• 1st Generation---- vary viewpoint but not lighting– Recover geometry ( explicit or implicit )
– Acquire photographs
– Facade, Plenoptic Modeling, View Morphing, Lumigraph, Layered Depth Images, (Light Field Rendering) etc.
• 1st Generation---- vary viewpoint but not lighting– Recover geometry ( explicit or implicit )
– Acquire photographs
– Facade, Plenoptic Modeling, View Morphing, Lumigraph, Layered Depth Images, (Light Field Rendering) etc.
Image-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and Rendering
• Photographs are not Reflectance Maps !
• 2nd Generation---- vary viewpoint and lighting for non-diffuse scenes– Recover geometry
– Recover reflectance properties
– Render using light transport simulation
• Photographs are not Reflectance Maps !
• 2nd Generation---- vary viewpoint and lighting for non-diffuse scenes– Recover geometry
– Recover reflectance properties
– Render using light transport simulation
Illumination Radiance
Reflectance
Previous WorkPrevious WorkPrevious WorkPrevious Work
• BRDF Measurement in the Laboratory– [ Ward 92 ], [Dana, Ginneken, Nayar & Koenderink 97]
• Isolated Objects under Direct Illumination– [ Sato, Wheeler & Ikeuchi 97 ]
• Isolated Objects under General Illumination– [ Yu & Malik 98], [ Debevec 98]
• BRDF Measurement in the Laboratory– [ Ward 92 ], [Dana, Ginneken, Nayar & Koenderink 97]
• Isolated Objects under Direct Illumination– [ Sato, Wheeler & Ikeuchi 97 ]
• Isolated Objects under General Illumination– [ Yu & Malik 98], [ Debevec 98]
The ProblemThe ProblemThe ProblemThe Problem
• General case of multiple objects under mutual illumination has not been studied.
Global IlluminationGlobal IlluminationGlobal IlluminationGlobal Illumination
Reflectance Properties
Radiance Images
Geometry Illumination
Inverse Global IlluminationInverse Global IlluminationInverse Global IlluminationInverse Global Illumination
Reflectance Properties
Radiance Images
Geometry Illumination
Input Radiance ImagesInput Radiance ImagesInput Radiance ImagesInput Radiance Images
[ Debevec & Malik 97]http://www.cs.berkeley.edu/~debevec/HDR
In Detail ... In Detail ... In Detail ... In Detail ...
Geometry and Camera PositionsGeometry and Camera PositionsGeometry and Camera PositionsGeometry and Camera Positions
Light SourcesLight SourcesLight SourcesLight Sources
Synthesized ImagesSynthesized ImagesSynthesized ImagesSynthesized Images
Original Lighting Novel Lighting
OutlineOutlineOutlineOutline
• Diffuse surfaces under mutual illumination
• Non-diffuse surfaces under direct illumination
• Non-diffuse surfaces under mutual illumination
• Diffuse surfaces under mutual illumination
• Non-diffuse surfaces under direct illumination
• Non-diffuse surfaces under mutual illumination
Lambertian Surfaces under Lambertian Surfaces under Mutual IlluminationMutual IlluminationLambertian Surfaces under Lambertian Surfaces under Mutual IlluminationMutual Illumination
j
ijjiii FBEB j
ijjiii FBEB
• Bi, Bj, Ei measured
• Form-factor Fij known
• Solve for diffuse albedo
• Bi, Bj, Ei measured
• Form-factor Fij known
• Solve for diffuse albedo i
iB
jBijF
Source
Target
Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]
Isotropic Kernel
Anisotropic Kernel
NHi
r
),(
Ksd
2
22
4
]/tan[exp
coscos
1),(
ri
K
yx
yx
ri
K
4
)]/sin/cos(tan[exp
coscos
1),(
22222
( 3 parameters)
( 5 parameters)
Non-diffuse Surfaces underNon-diffuse Surfaces underDirect IlluminationDirect IlluminationNon-diffuse Surfaces underNon-diffuse Surfaces underDirect IlluminationDirect Illumination
2
,,)),(( min arg iisi
i
di IKIL
sd
2
,,)),(( min arg iisi
i
di IKIL
sd
NH
iisid
i IKIL )),((
iisid
i IKIL )),((
P1P2
P1
P2
Non-diffuse Surfaces under Non-diffuse Surfaces under Mutual IlluminationMutual IlluminationNon-diffuse Surfaces under Non-diffuse Surfaces under Mutual IlluminationMutual Illumination
• LPiAj is not known. ( unlike diffuse case, where LPiAj = LCkAj )
• LPiAj is not known. ( unlike diffuse case, where LPiAj = LCkAj )
j j
APCAPsAPAPdPC jivjijijiivKLFLL
j jAPCAPsAPAPdPC jivjijijiiv
KLFLL Cv
Ck
Aj
Pi
LPiAj
LCkAj
LCvPi
Source
Target
Solution: iteratively estimate Solution: iteratively estimate specular component.specular component.Solution: iteratively estimate Solution: iteratively estimate specular component.specular component.
jikjkji APCACAP SLL jikjkji APCACAP SLL
• Initialize
• Repeat– Estimate BRDF parameters for each surface
– Update and
• Initialize
• Repeat– Estimate BRDF parameters for each surface
– Update and
0jik APCS 0jik APCS
jik APCS jik APCS jiAPL
Estimation of Specular Difference SEstimation of Specular Difference SEstimation of Specular Difference SEstimation of Specular Difference S
• Estimate specular component of by Monte Carlo ray-tracing using current guess of reflectance parameters.
• Similarly for
• Difference gives S
• Estimate specular component of by Monte Carlo ray-tracing using current guess of reflectance parameters.
• Similarly for
• Difference gives S Cv
Ck
Aj
Pi
LPiAj
LCkAj
LCvPi
LPiAj
LCkAj
Recovering Diffuse Albedo MapsRecovering Diffuse Albedo MapsRecovering Diffuse Albedo MapsRecovering Diffuse Albedo Maps
• Specular properties assumed uniform across each surface, but diffuse albedo allowed to vary.
•
•
• Specular properties assumed uniform across each surface, but diffuse albedo allowed to vary.
•
•
)()()( xLxLxL sd
)(/)()( xIxLx dd
ResultsResultsResultsResults
• A simulated cubical room• A simulated cubical room
Results for the Simulated CaseResults for the Simulated CaseResults for the Simulated CaseResults for the Simulated Case
Diffuse Albedo
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 4 5 60
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 4 5 6
Specular Roughness
ResultsResultsResultsResults
• A real conference room• A real conference room
Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting
Real
Synthetic
Diffuse Albedo Maps of Identical Diffuse Albedo Maps of Identical Posters in Different PositionsPosters in Different PositionsDiffuse Albedo Maps of Identical Diffuse Albedo Maps of Identical Posters in Different PositionsPosters in Different Positions
Poster A Poster B Poster C
Inverting Color BleedInverting Color BleedInverting Color BleedInverting Color Bleed
Input Photograph Output Albedo Map
Real vs. Synthetic for Novel LightingReal vs. Synthetic for Novel LightingReal vs. Synthetic for Novel LightingReal vs. Synthetic for Novel Lighting
Real
Synthetic
VideoVideoVideoVideo
AcknowledgmentsAcknowledgmentsAcknowledgmentsAcknowledgments
• Thanks to David Culler and the Berkeley NOW project, Tal Garfinkel, Gregory Ward Larson, Carlo Sequin.
• Supported by ONR BMDO, the California MICRO program, Philips Corporation, Interval Research Corporation and Microsoft Graduate Fellowship.
• Thanks to David Culler and the Berkeley NOW project, Tal Garfinkel, Gregory Ward Larson, Carlo Sequin.
• Supported by ONR BMDO, the California MICRO program, Philips Corporation, Interval Research Corporation and Microsoft Graduate Fellowship.
ConclusionsConclusionsConclusionsConclusions
• A digital camera can undertake all the data acquisition tasks involved.
• Both specular and high resolution diffuse reflectance properties can be recovered from photographs.
• Reflectance recovery can re-render non-diffuse real scenes under novel illumination as well as from novel viewpoints.
• A digital camera can undertake all the data acquisition tasks involved.
• Both specular and high resolution diffuse reflectance properties can be recovered from photographs.
• Reflectance recovery can re-render non-diffuse real scenes under novel illumination as well as from novel viewpoints.