Hierarchical Caustic Maps

Chris WymanUniversity of Iowa

Symposium on Interactive 3D Graphics and Games 2008

What is a Caustic?

Light focused via reflection or refraction

• Depend on accurate specular interactions– Themselves difficult in the context of interactive rasterization

• Integrate over specular surfaces– Gather incoming light from each point– Not the sort of data available in rasterization (i.e., global)– Essentially an “area” light – the area of the specular surface

Why are interactive caustics difficult?

Previous Work

• Wavefront techniques (non-interactive)– First application for caustics [Mitchell 92]– March through volumetric scene representation [Ihrke 07]

• Beam techniques (object-space interactive approaches)– Backwards beam tracing [Watt 90] – Hardware accelerated approach [Iwasakai 02]– Warped volumes that accurately represent regions [Ernst 05]

• Particle techniques (image-space interactive approaches)– Illumination mapping [Arvo 86]– Photon mapping [Jensen 01]– Caustic mapping

• [Szirmay-Kalos 05] [Wyman 06] [Kruger 06] [Shah 07]

Object v.s. Image Space

• “Caustic volume” object-space methods:– Consume significant fill rate (as shadow volumes)– Requires extruding geometry (i.e., touch every frame)– Difficult to deal with multiple specular interfaces– Straightforward addition of volumetric caustics

• “Caustic map” image-space methods:– Sampling issues (how many photons is enough?)– Touch each photon (O(n) in number of photons)– Too few photons leads to blurry caustics– Volumetric caustics cost (quite a bit) extra

What is a Caustic Map?

• Idea: Create a map akin to a shadow map– Shadow map essentially stores binary visibility

• (i.e., Is light visible or not?)– Caustic map should store integral visibility

• (i.e., How many times is the light visible?)

1. Render from light, store photon positions• Simple render pass… Reasonably cheap

2. Draw photons as point splats• Cost depends linearly on number of photons

3. Project caustic map• Another simple render pass… Reasonably cheap

What is a Caustic Map?

Notice: Wasted photons innon-specular regions… Badif costs linear in # photons!

What Sort of Cost?

512 x 512 1024 x 1024 2048 x 2048

≈50 fps≈30 fps≈10 fps≈4 fps≈0.6 fps

Investigating Costs…

1. Hidden copy cost from framebuffer → VBOGoes away on newer GPUs, see paper

2. “Wasted” photons → no caustic effectCost nearly as much as caustic photonsCould adapt light view each frame to minimizeMay be unavoidable

• (e.g., due to object concavity, specular algorithm req’s)

3. Oversampling and undersamplingNot uniform effect → improve one, worsen other

• (i.e., wasted photons versus noise)

Hierarchical Processing• Photon buffer stores (x,y,z) of photon hits

• Most photons in buffer do not hit refractor

• These photons still need processing, but they do not contribute to the caustic!

• Can we process these faster than O(n)?– Yes!

Valid caustic photons

512x512 16x16

Create MIP map

(Look at higherlevel image)

Counts valid photons in subtree

Hierarchical Processing• Stream input: low resolution “photons”

– Discard all white (invalid) photons– Stream out valid photon’s children for

more processing • Progressively increase resolution

– Input valid photons from lower res pass– Repeat discard process

• At highest resolution– Only processes valid photons & neighbors

• Use new geometry shaders for efficiency

• Gives ≈ 2 – 4x speedup!

Dealing with Oversampling

• Caustic map is a discrete image• Texels converge after different # of photons

– Focal regions of caustic converge fastest– Some with only 642,1282, or 2562 photons!– Using 40962 photons is a significant waste in these areas

Other regions need morethan 40962 photons!

Currently, to get quality you must accept waste(or vice versa)

Hierarchy to the Rescue!

• Hierarchical process avoids irrelevant photons• Use it to eliminate redundant photons, too

• How?– After checking “photon” validity, check child convergence– Output single “photon cluster” instead of four children– “Photon clusters” are only processed @ highest resolution– Replaces 4, 16, 64, or more photons with 1 cluster

Key Idea• Where photons converge

– Each photon processed – All affect the same texel– All increase caustic

intensity roughly the same

• Photon clusters– Only need processing 1x– Only touch the texel 1x– Significantly reduce

oversampling costs– More evenly distributes

photon costs over caustic map

Caustic Map

Converging Photons

(Roughly 50 –100% performance improvement)

Dealing with Undersampling

• Reflection and refraction can cause photons to diverge an arbitrary amount – An arbitrary number of photons may be needed– Still not possible interactively, even with hierarchical speedups– Thus noise still happens

One approach:• Photons affect > 1 texel• Already done in all caustic

mapping techniques• Trade-off:

– Blur v.s. noise

Varying Photon “Splat” Sizes

• Uniformly increasing photon splat size– Blurs the focal regions! (Ruins the point of caustics…)– Larger splats cost more to rasterize (≈ radius squared)

• Goal: Increase splat size only in noisy areas– I.e., the same regions where photons diverge– Observation: Divergence stored in photon buffer

• (How far apart did two initially adjacent photons hit?)

– Relatively simple to vary splat size dynamically

Ratio of areas (refracted to unrefracted)gives relative splat size

Varying Photon “Splat” Costs

• But, photons can diverge arbitrary amount– Splats may be arbitrary size! (Arbitrarily expensive… BAD)

• Divergent splats are expensive, dim, and blurry– Fragment rasterization costs increase with square of radius– Photon “beams” assumed to have constant intensity

• (Larger, diverging beams → less energy per pixel)

Relatively bright, focused caustic

Much dimmer, as energy spreadover larger region – for more divergent photons, intensity fallsoff quickly (as r2)

Render to MIP-map• Goal: Reduce costs of expensive, unimportant photons

• Idea:– Use multiple caustic maps of different resolutions– Per-splat, choose correct map based on splat size– Scale splat size based on chosen map size

• All rasterized splats roughly same size (within factor of 2)

• Benefits:– High frequency focal regions stored in detailed high-res maps– Low frequency, unimportant regions subsampled in low-res maps– All splats cost roughly the same– Allows caustic splats to exceed hardware size limits!

• We use a MIP-map hierarchy for simplicity

ExampleHierarchy:

Lv 0

Lv 1

Lv 2

Lv 3

Before using map:Resize,

Normalize intensity,Sum all levels

Example

With MIP hierarchy Without MIP hierarchy

Evaluation

5 – 20x 50 – 100% - 50 – 100%

• Hierarchical gather: – Big win! Esp. with many photons

• Combining redundant photons– Good results with 1M+ photons… Not worth it with less.

• Varying splat sizes– Dramatic quality improvement. Only worth it with render-to-mipmap.

Results

Using 250 k photons per light: ~24-32 fps Using 1 M photons per light: ~23-30 fps

2 lights per scene, 400k-600k trianglesFinal rendering at: 20482

Using 4 M photons per light: ~18-25 fps Using 16 M photons per light: ~10-16 fps

Results

Using 250 k photons per light: ~24-32 fps Using 1 M photons per light: ~23-30 fps

Using 4 M photons per light: ~18-25 fps Using 16 M photons per light: ~10-16 fps

2 lights per scene, 400k-600k trianglesFinal rendering at: 20482

Video

beethovenVideo_HQ.DivX.avi

Link: Beethoven Video

Link: Dragon Video

Summary

• 3 hierarchical techniques– Reduce irrelevant photons

• (Processing photon buffer low-res to high-res)

– Reduce redundant photons• (Combining converging photons into cheaper clusters)

– Reduce noise via multi-frequency render-to-mipmap• (Each splat selects correct caustic map based on radius)

• Dramatic performance improvement– More than order of magnitude reduction

to gather costs

Limitations

• Many remain from previous caustic map work– Only as accurate as reflection / refraction model– Still a tradeoff in quality v.s. speed

• Bound by photon count, even with our improvements

– Problem: caustics on surfaces not in shadow map

• New limitations:– Requires newer hardware– Some assumptions in render-to-mipmap (see paper)

• Improves splat numerical robustness• Overblurrs slightly

Questions?

Hierarchical Caustic MapsWhat is a Caustic?Why are interactive caustics difficult?Previous WorkObject v.s. Image SpaceWhat Sort of Cost?Investigating Costs…Hierarchical ProcessingHierarchical ProcessingDealing with OversamplingHierarchy to the Rescue!Key IdeaDealing with UndersamplingVarying Photon “Splat” SizesVarying Photon “Splat” CostsRender to MIP-mapExampleExampleEvaluationResultsResultsVideoSummaryLimitationsQuestions?