Frequency DomainNormal Map Filtering

Charles HanBo Sun

Ravi RamamoorthiEitan Grinspun

Columbia University

Normal Mapping

• (Blinn 78)

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Normal Mapping

QuickTime™ and aH.264 decompressor

are needed to see this picture.

• (Blinn 78)• Specify surface

normals

Normal Mapping

A Problem…

• Multiple normals per pixel

• Undersampling• Filtering needed

?

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Supersampling

• Correct results• Too slow

QuickTime™ and aH.264 decompressor

are needed to see this picture.

MIP mapping

• Pre-filter• Normals do not

interpolate linearly

• Blurring of details

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Comparison

supersampled MIP mapped

Representation

a single vector is not enough

how do we represent multiple surface normals?

Previous Work

• Gaussian Distributions– (Olano and North 97)– (Schilling 97)– (Toksvig 05)

• Mixture Models– (Fournier 92)– (Tan, et.al. 05)

3D Gaussian 2D covariance matrix

1D Gaussian

mixture of Phong lobes

mixture of 2D Gaussians

no general solution

Our Contributions

• Theoretical Framework – Normal Distribution Function (NDF) – Linear averaging for filtering– Convolution for rendering– Unifies previous works

• New normal map representations– Spherical harmonics– von Mises-Fisher Distribution

• Simple, efficient rendering algorithms

Normal Distribution Function (NDF)

• Describes normals within region • Defined on the unit sphere• Integrates to one• Extended Gaussian Image (Horn 84)

Normal Distribution Function

QuickTime™ and aH.264 decompressor

are needed to see this picture.

NDF

normal map

Normal Distribution Function

QuickTime™ and aH.264 decompressor

are needed to see this picture.

NDF

normal map

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Normal Distribution Function

NDF

normal map

Normal Distribution Function

NDF

normal map

NDF Filtering

normal map

NDF Filtering

normal map

NDF Filtering

• NDF averaging is linear• Store NDFs in MIP map

Rendering

rendered image

€

B(ωout ) = L(ωin )∫ ρ (ω ⋅n) dωin

normal,

€

n

pixel value lights BRDF

Radially symmetric BRDFs• Lambertian:• Blinn-Phong:• Torrance-Sparrow:• Factored:€

(ωin ⋅n)

€

(ωh ⋅n)S

€

f (θh )g(θd )

€

exp θh2

[ ]

Supersampling

supersampled imagesamples

€

B(ωout ) =1

NL(ωin )∫ ρ (ω ⋅ni)

i

∑ dω

€

B(ωout ) = L(ωin )∫ 1

Nρ (ω ⋅ni)

i

∑ ⎡

⎣ ⎢

⎤

⎦ ⎥dω

Effective BRDF

€

1N L(ωin )∫ ρ(ω ⋅n1) dω

+1N L(ωin )∫ ρ(ω ⋅n2) dω

+M+

1N L(ωin )∫ ρ(ω ⋅nN) dω

samples

Effective BRDF

€

ρeff (ω) =

€

1

Nρ(ω • ni)

i

∑

NDF,

€

γ(n)

€

ρ(ω • n)

€

γ(n)∫

€

dn

€

ρeff (ω) =

Spherical Convolution

• Form studied in lighting– (Basri and Jacobs 01)– (Ramamoorthi and Hanrahan 01)

• Effective BRDF = convolution of NDF & BRDF

€

ρ(ω • n)

€

γ(n)∫

€

dn

€

ρeff (ω) =

NDF

Spherical Convolution

€

=

€

⊗

Effective BRDF BRDF

€

ρ(ω • n)

€

γ(n)∫

€

dn

€

ρeff (ω) =

Previous Work

• Gaussian Distributions– Olano and North (97)– Schilling (97)– Toksvig (05)

• Mixture Models– Fournier (92)– Tan, et.al. (05)

• Our Work

3D Gaussian2D covariance matrix

1D Gaussian

mixture of Phong lobesmixture of 2D

Gaussians

NDF representations

spherical harmonicsvon Mises-Fisher mixtures

Spherical Harmonics

• Analogous to Fourier basis• Convolution formula:

€

ρ(ω • n)

€

γ(n)∫

€

dn

€

ρeff (ω) =

€

ρlmeff = ρ lγ lm

BRDF Coefficients

€

ρlmeff = ρ lγ lm

• Arbitrary BRDFs• Cheaply represented

– Analytic: compute in shader – Measured: store on GPU

• Easily changed at runtime

NDF Coefficients

• Store in MIP mapped textures• Finest-level NDFs are delta functions,

so:

• Use standard linear filtering€

γlm = δ(n)Ylm (ω)dω =∫ Ylm (n)€

ρlmeff = ρ lγ lm

Effective BRDF Coefficients

• Product of NDF, BRDF coefficients • Proceed as usual

€

ρlmeff = ρ lγ lm

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Limitations

• Storage cost of NDF– One texture per coefficient– O( ) cost

• Limited to low frequencies

€

l2

von Mises-Fisher Distribution (vMF)

• Spherical analogue to Gaussian• Desirable properties

– Spherical domain– Distribution function– Radially symmetric

more concentrated less concentrated

Mixtures of vMFs

NDF

number of vMFs

1 2 3 4 5 6

Expectation Maximization (EM)

• From machine learning• Used in (Tan et.al. 05)• Fit model parameters to data

data

NDF

model

vMF Mixture

EM

Rendering

• Convolution– Spherical harmonic coefficients– Analytic convolution formula

• Extensions to EM– Aligned lobes (Tan et.al. 05)– Colored lobes

NDF rendered image

€

γl

QuickTime™ and aH.264 decompressor

are needed to see this picture.

QuickTime™ and aH.264 decompressor

are needed to see this picture.

QuickTime™ and aH.264 decompressor

are needed to see this picture.

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Conclusion

• Summary– Theoretical Framework– New NDF representations– Practical rendering algorithms

• Future directions– Offline rendering, PRT – Further applications for vMFs– Shadows, parallax, inter-reflections, etc.

Thanks!

Tony Jebara, Aner Ben-Artzi, Peter Belhumeur, Pat Hanrahan, Shree Nayar, Evgueni Parilov, Makiko Yasui, Denis Zorin, and nVidia.

http://www.cs.columbia.edu/cg/normalmap