Real-time Shading with Real-time Shading with Filtered Importance SamplingFiltered Importance Sampling

Jaroslav Jaroslav KřivánekKřivánekCzech Technical University in PragueCzech Technical University in PragueMark ColbertMark ColbertUniversity of Central FloridaUniversity of Central Florida

• material design interfaces

• rendering algorithm to back up the interface

• immediate feedback

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 22

MotivationMotivation

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 33

GoalGoal

• image-based lighting (environment maps)– improves material perception [Fleming et al. 2003]

point light natural light

images [Fleming et al. 2003]

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 44

GoalGoal

• arbitrary materials– low to high gloss, different BRDF models

images by Pat Hanrahan

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 55

GoalGoal

• dynamic materials, geometry, lighting– no pre-computation

• production pipeline friendly– minimal code base / single GPU shader

• real-time shadows (env. map)– not necessarily

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 66

Desired ResultsDesired Results

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 77

Related WorkRelated Work

• pre-filtered environment maps [ Kautz et al. 2000 ]

• frequency-space rendering [ Ramamoorthi et al. 2002 ], [ Ng et al. 2004 ]

• Efficient Reflectance and Visibility Approximations for Environment Map Rendering [ Green et al. 2007 ]

• Efficient Rendering of Spatial Bi-directional Reflectance Distribution Functions[ McAllister et al. 2002 ]

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 88

OverviewOverview

• Motivation• Goal• Related Work• Shading Algorithm• Theory• Real-time Shadows• Results• Conclusion

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 99

BRDF Importance SamplingBRDF Importance Sampling

• standard in MC ray tracing

• not used on the GPU

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1010

Deterministic SamplingDeterministic Sampling

• aliasing

40 samples per pixel

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1111

Our ApproachOur Approach

• filtered importance sampling– less filtering where samples are denser– more filtering where they are sparser

),(

1 sizefilter

oipN

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1212

FilteringFiltering

• MIP-maps• level proportional to

log of filter size• independent

of the BRDF

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1313

Filtered Importance SamplingFiltered Importance Sampling

40 samples per pixel

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1414

OverviewOverview

• Motivation• Goal• Related Work• Shading Algorithm• Theory• Real-time Shadows• Results• Conclusion

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1515

Underlying TheoryUnderlying Theory

• why theory? – identify approximations – suggest improvements

• … sampling & filtering– signal processing

sample

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1616

Sampling and ReconstructionSampling and Reconstruction

aliasedoriginal

reconstruct

alias = integration error

DC-term = integral

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1717

Application to Importance SamplingApplication to Importance Sampling

• problem: non-uniform samples

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1818

Conceptual ProcedureConceptual Procedurewarp

(inverse BRDF IS)

pre-filter (=convolve)

warp back(BRDF IS)

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 1919

PracticePractice

• isotropic filter approximation

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2020

ApproximationsApproximations

• isotropic filter shape• constant BRDF / PDF ratio across filter support• tri-linear filtering (MIP-map)

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2121

Anisotropic Filtering ExperimentsAnisotropic Filtering Experiments

• anisotropic filter approximation

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2222

• tex2Dgrad for anisotropic texture look-up• worse image quality – still don’t know why

Anisotropic Filtering ExperimentsAnisotropic Filtering Experiments

16x anisotropic filter

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2323

OverviewOverview

• Motivation• Goal• Related Work• Shading Algorithm• Theory• Real-time Shadows• Results• Conclusion

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2424

• environment importance sampling (bright light sources = strongest shadows)

Real-time ShadowsReal-time Shadows

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2525

• shadow map for each sample

Real-time ShadowsReal-time Shadows

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2626

• convert to spherical harmonics at each texel

visibility function

Real-time ShadowsReal-time Shadows

Real-time ShadowsReal-time Shadows

• spatial filtering

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2727

no filtering after filtering

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2828

• use for rendering

– diffuse• SH coefficient dot product

– glossy• attenuate each sample by the visibility

Real-time ShadowsReal-time Shadows

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 2929

OverviewOverview

• Motivation• Goal• Related Work• Shading Algorithm• Theory• Real-time Shadows• Results• Conclusion

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3030

FIS Results – RMSFIS Results – RMS

Environment Sampling Filtered SamplingReference

n =

35

Sam

ples

n =

17

100

S

ampl

es

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3131

FIS Results – Complex GeometryFIS Results – Complex Geometry

5 Samples 50 Samples

200 Samples Reference

50 S

ampl

es

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3232

FIS Results – BRDF AnisotropyFIS Results – BRDF Anisotropy

• limited anisotropy

x = 0.01x = 0.08

x = 0.01x = 0.29

x = 0.01x = 0.01

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3333

Shadows ResultsShadows ResultsRe

fere

nce

(30,

000

Sam

ples

)O

ur Method

(16 Samples)

Visu

alEr

ror

SH v. Ref 8 samples 16 samples 64 samples

• NVIDIA GeForce 8800 GTX, Intel Core2 Duo, 512x512

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3434

Shadows PerformanceShadows Performance

• NVIDIA GeForce 8800 GTX, Intel Core2 Duo, 512x512

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3535

Shadows PerformanceShadows Performance

polygon countpolygon count

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3636

Video

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3737

ConclusionConclusion

• glossy surface shading– practical, relatively accurate, no pre-computation– signal processing theory

• shadows– fast but very approximate– no pre-computation

• implementation details: GPU Gems 3• Code: graphics.cs.ucf.edu/gpusampling/

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 3838

AcknowledgementsAcknowledgements

• Dan Sýkora• Petr Olšák• Czech Ministry of Education

– “Center for Computer Graphics”• Aktion grant• US National Science Foundation

Additional SlidesAdditional Slides

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 4040

Numerical Integration as Numerical Integration as Signal ReconstructionSignal Reconstruction

• integral = DC term• integration by sampling

1.sample the function2.reconstruct the DC term

• insufficient sampling -> aliasing• alias may affect the DC term -> error• anti-aliasing – pre-filtering

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 4141

Anti-aliasing by Pre-filteringAnti-aliasing by Pre-filtering

reconstruct

sample

band-limit

KřKřiivánek, vánek, ColbertColbertReal-time shading with Filtered Importance SamplingReal-time shading with Filtered Importance Sampling 4242

Stochastic SamplingStochastic Sampling

• noise• slower on the

GPU

40 samples per pixel