View-Dependent Precomputed Light Transport Using Nonlinear Gaussian Function Approximations

Paul Green1 Jan Kautz1 Wojciech Matusik2 Frédo Durand1

MIT CSAIL1 MERL2

ACM Symposium in Interactive 3D Graphics 2006

View-Dependent Precomputed Light Transport Using Nonlinear Gaussian Function Approximations

View-Dependent Precomputed Light Transport Using Nonlinear Gaussian Function Approximations

View-Dependent Precomputed Light Transport Using Nonlinear Gaussian Function Approximations

Geometry & Viewpoint All-Frequency Lighting

Rendered Frame

Reflectance

Interactive 6D Relighting

Goal: 6D Relighting

High-quality view-dependent effects Sharp highlights

Spatially varying BRDFs

Allow rendering w/ large environment maps (e.g. 6x256x256)

Without paying a prohibitive data storage price

Applications

Games Architectural Visualization

Outline

Background / Previous Work PRT Nonlinear Approximation

Our Representation Rendering Fitting Results

Conclusion

Courtesy of Sloan et al. 2003

),( ooL pExit Radiance

Distant Radiance

Incident Radiance

Shadowing

Inter-reflection

Transport function maps distant light to incident light

Can Include BRDF if outgoing direction ωo is fixed

Precomputed Light Transport

),( oo pL

iL

Courtesy of Sloan et al. 2003

Radiance Lo at point p along direction is weighted sum of distant radiance Li

i,

, )()(),(

LT

p

p

p

o

o ioo LTL

Outgoing Radiance

Transport Vector

Distant Radiance (Environment Map)

o

Light Transport

Example

Transport Function Environment Map

BRDF Weighted Incident Radiance

Exit Radiance (outgoing color)

It’s a Dot Product Between Lighting and Transport Vectors!!

Light Transport

Data Size Problem Many GB’s of data Rendering is slow

6x64x64 cubemap ~24,000 mults/vert

Can reduce size in different basis: Spherical Harmonics Wavelets Zonal Harmonics

PRT with Spherical Harmonics

Precomputed Radiance Transfer [Sloan et al 02,03]Low Order Spherical HarmonicsSoft Shadows and Low Frequency LightingNot Suitable For Highly

Glossy Materials Not Practical For High

Frequency Lighting

Image from slides by Ng et al.2003

Nonlinear Wavelet PRT

Nonlinear Wavelet Lighting Approximation [Ng et al 03]

Haar Wavelets

Nonlinear Approximation

All Frequency Lighting

Fixed View For ArbitraryBRDFs

Image from slides by Ng et al.2003

Separable PRT

Factor BRDF into product of view-only and light-only functions [Liu et al 04, Wang et al 04]

Nonlinear Wavelet Approximation [Ng et al 03]

Need factorization per BRDFVery specular materials still

require many coefficients

Liu et al 04

Our Goal: 6D Relighting

High quality view-dependent effects

Representation of transport that enables Spatially varying BRDFs Arbitrary highlight scale compact storage High-res environment maps (e.g. 6x256x256)

Sparse view samplingRequires high-quality interpolation Over view directions Over mesh triangles

Outline

Background / Previous Work PRT Nonlinear Approximation

Our Representation Rendering Fitting Results

Conclusion

Factoring Transport

Represent with SH or Wavelets

View-independent (diffuse)

View-dependent

Nonlinear Gaussian Function Approximation?

Per view

Our Nonlinear Representation

),;()(N

, kkik

ki GwTo

p

k k- mean - std. deviation

Sum of N isotropic Gaussians

Previous work: Nonlinear Wavelet

Nonlinear: Approximating basis set depends on input

Truncate small coefficientsEffect of coefficients is still linear

Linear: First 8 Coefficients

SSE = 140.2

Nonlinear: 8 Largest Coefficients

SSE = 25.1

Our solution: Even More Nonlinear

We don’t start from linear basis No Truncation of coefficients Nonlinear parameter estimation Parameters have nonlinear effects

SSE = 5.61

Nonlinear sum of 2 GaussiansNonlinear: 8 Largest Coefficients

SSE = 25.1

Arbitrary freq bandwidth

Accurate approx w/small storage

Good interpolation

Good visual quality

Advantages of sum of Gaussians

original N = 1 N = 2 Haar 70 terms

Examples

p

1o 2o

Rendering

Approximated Transport Lighting

?

Gaussian Pyramid

Larger σ

Pre-convolve environment with Gaussians of varying sizes

Only done Once

Can start with large cubemap e.g. 6x256x256

Rendering

Tri-linear lookup in Gaussian Pyramid

Exit Radiance (outgoing color)

Approximated Transport Lighting

w,,

w

Rendering Novel ViewsPrecomputed Transport Functions for sparse

set of outgoing directions

Naïve solution: (Gouraud Shading) Interpolate Outgoing Radiance

Cross-fading artifacts)()()1())(~(

21 oooooo tLLttL

p

?

1o 2o

)(~ to

0t 1t

t

Better: Interpolate Parameters

Interpolate Gaussian parameters

Mean, Std. Dev, Weights

Analogous to Phong vs Gouraud shading p

?

1o 2o

)(~ to

0t 1t

t

t=0 t=1t=0.5

Per-Pixel Interpolation

Interpolation Drawbacks

Visibility may not interpolate correctly But is usually plausible

Correspondences Makes fitting more difficult

View-independent View-dependent

Data Fitting Precompute Raw Transport Data Solve large scale nonlinear

optimization problem For each vertex

For each view Fit Gaussians to transport data

p

1o 2o

Nonlinear Optimization

Objective has terms for Fitting the Data

Regularization Angular smoothness

and correspondences

Spatial smoothness

and correspondences

originalN = 2

N = 1

min -2

p

1n

2n

k

Error Plots

Results

6x256x256 Environment Maps Single Gaussian 2.8 GHz P4 1GB RAM Nvidia 6800 Ultra Software screen capture

Contributions

New parametric representation of PLT Compact Storage High Quality Interpolation of parameters Sparse View Sampling Efficient Rendering Spatially varying BRDFs

original N = 2 Haar

Questions?