Yuhao Zhu http://yuhaozhu.com

Lecture 17: Ray Tracing and Acceleration Structures

CSC 292/572, Fall 2020 Mobile Visual Computing

Logistics

‣ PA1 is due 10/30, 11:30 AM. ‣ Will grade and post WA 3 and your proposal grades by this week.

�2

�3

Graphics

Modeling Rendering

Lighting, Camera, and Material

http://www.cgarena.com/freestuff/tutorials/max/thomas_highway/sergeant.htmlhttps://docs.blender.org/manual/en/dev/render/introduction.html

Visibility Shading

Rasterization Ray tracing

Visibility Problem

�4

‣ Two fundamental classes of visibility algorithms ▹ Object-centric (Rasterization) ▹ Image-centric (Ray tracing)

Rasterization asks: given a point P [x, y, z], what’s the corresponding pixel

coordinates [u, v] on the camera sensor?

Ray tracing asks: given a pixel [u, v] on the sensor, what’s the associated

point in the scene?

Ray Tracing

‣ Pioneered by Arthur Appel (ray casting) ▹ "Some techniques for shading machine renderings

of solids." Proceedings of AFIPS 1968. ‣ Popularized by Tuner Whitted (through

recursive ray tracing) ▹ “An improved illumination model for shaded

display.” SIGGRAPH 1979, CACM 1980. ▹ See Whitted’s own narrative.

�5https://blogs.nvidia.com/blog/2018/08/01/ray-tracing-global-illumination-turner-whitted/

Ray Tracing

‣ Whitted invented recursive ray tracing as a way to achieve (some limited form of) global illumination. It considers (more on shading next time): ▹ Direct lighting. ▹ Indirect lighting assuming perfect reflection and refraction (through recursive ray tracing). ▹ Diffusion/glossy through simple Phone shading model. ‣ Ray-tracing could be used for local illumination ▹ By simply not tracing indirect lighting. See this implementation of Phong’s shading model. ‣ Realizing motion blur, depth of field, penumbras, translucency and fuzzy

reflections in ray-tracing: Distributed Ray Tracing, SIGGRAPH 1984 ‣ Unified theory: The Rendering Equation, SIGGRAPH 1986

�6

Basic Idea

�7

foreach pixel in image ray = buildRay(camera, pixel) foreach triangle in mesh if (P = intersect(ray, triangle)) pixel.color = shade(P) else pixel.color = backgroundColor

Basic Idea

�8

foreach pixel in image ray = buildRay(camera, pixel) foreach triangle in mesh if (P = intersect(ray, triangle)) pixel.color = shade(P) else pixel.color = backgroundColor

Basic Idea

�8

foreach pixel in image ray = buildRay(camera, pixel) foreach triangle in mesh if (P = intersect(ray, triangle)) pixel.color = shade(P) else pixel.color = backgroundColor

Generating a Ray in a Pinhole Camera

‣ Very intuitive: connect a pixel and the pinhole to form a ray. ‣ Remember in actual implementations the canvas is before the pinhole.

�9

f

Canvas Canvas

Details

‣ Step 1: Generate a ray in the camera space. ▹ Note that the pixel coordinates are defined

in the raster space. So need to convert the pixel from raster to camera space.

‣ Step 2: Convert the ray from the camera space to the world space.

�10

World frame

Local frame 1

Camera frame

Local frame 3 (light)

Local frame 2

Ray = O + tDO

||t||

class Ray { public: Ray(), O(0), D(0,0,-1) {} … Vec3f O; Vec3f D; };

D: the direction vector

Generating a Ray Under an Ideal Thin Lens

‣ Challenge: one pixel takes contributions from different parts of the lens, and thus different parts of the scene. ▹ Can’t use a pinhole camera model. ▹ For each pixel, need to sample the lens multiple times to trace multiple rays.

�11

f

f

Generating a Ray Under an Ideal Thin Lens

‣ Goal: given an arbitrary Rout how do we find Rin? ▹ There is a unique ray, Rout, between a pixel P and a sample L on the lens ▹ There is a unique ray, Rin, going into the lens that generates Rout

▹ The closest point on Rin before the lens will hit P

�12

f

f

PRout

Rin

L

Generating a Ray Under an Ideal Thin Lens

‣ Use geometrical optics principles to determine Rin for a given Rout

▹ Rays go through the lens center don’t change their directions

�13

f

f

P

L

Rout

Rin

Generating a Ray Under an Ideal Thin Lens

‣ Use geometrical optics principles to determine Rin for a given Rout

▹ Rays go through the lens center don’t change their directions ▹ Rays parallel to the optical axis pass through the lens focus

�14

f

f

PL

Rout

Rin

Generating a Ray Under an Ideal Thin Lens

‣ Use geometrical optics principles to determine Rin for a given Rout

▹ Rays go through the lens center don’t change their directions ▹ Rays parallel to the optical axis pass through the lens focus ▹ Use the above two the find the intersection point S in the scene, then given any L, Rin is the

ray between S and L

�15

f

f

P

LRout

Rin

S

Can Ray Tracing Capture DOF?

‣ The ray tracing technique described here doesn’t rely on whether the closest hit is actually in-focus on the pixel.

‣ So this technique can inherent trace scene points that are out-of-focus, and thus naturally simulate depth of field.

�16

f

f

Actual sensor plane

Can Ray Tracing Capture DOF?

‣ The ray tracing technique described here doesn’t rely on whether the closest hit is actually in-focus on the pixel.

‣ So this technique can inherent trace scene points that are out-of-focus, and thus naturally simulate depth of field.

�16

f

f

Actual sensor plane

Can Ray Tracing Capture DOF?

‣ The ray tracing technique described here doesn’t rely on whether the closest hit is actually in-focus on the pixel.

‣ So this technique can inherent trace scene points that are out-of-focus, and thus naturally simulate depth of field.

�16

f

f

Actual sensor plane

Lens Sampling

‣ To accurate simulate DOF, we need to sample many points on the lens for a single pixel. Otherwise the image looks noise. ▹ Why? Because sampling only approximates a continuous integration. ▹ The same problem will show up in shading (global illumination).

�17http://www.pbr-book.org/3ed-2018/Camera_Models/Projective_Camera_Models.html

2048 samples/pixel 4 samples/pixel

‣ We will assume that all primitives are converted to triangle meshes. ‣ If not, we need one ray-primitive intersection implementation for each

primitive type in the rendering engine. ▹ Implicit surface ▹ Parametric surface ▹…

Ray-Object Intersection

�18

✓ Which triangle is the first hit? ✓ What’s the coordinates of the hit point?

What Defines a Triangle?

�19

V1

V2

V3

‣ A plane with three vertices. ▹ The vertices are guaranteed to be co-planar. ‣ The plane that the triangles are in can

be calculated from the vertices. ▹ By solving a system of linear equations. ‣ The plane can be expressed as an

implicit surface.

A(x - x0) + B(y - y0) + C(z - z0) = 0

P0: [x0, y0, z0]

Normal (n): [A, B, C]

For any point P (x, y, z) on this plane, the vector formed by P - P0 must be

perpendicular to the normal vector n, i.e., the dot product of (P - P0) and n must be 0

Ax + By + Cz +D = 0 D = - (Ax0 + By0 + Cz0)

Ray-Triangle Intersection

�20

O (Ox, Oy, Oz)

||t|| P = O + tD

Ax + By + Cz + D = 0Px = Ox + Dx × t

Py = Oy + Dy × tPz = Oz + Dz × tA × Px + B × Py + C × Pz + D = 0

t = −A × Ox + B × Oy + C × Oz + D

A × Dx + B × Dy + C × Dz

V1

V2

V3

Ray-Triangle Intersection

�20

O (Ox, Oy, Oz)

||t|| P = O + tD

Ax + By + Cz + D = 0Px = Ox + Dx × t

Py = Oy + Dy × tPz = Oz + Dz × tA × Px + B × Py + C × Pz + D = 0

t = −A × Ox + B × Oy + C × Oz + D

A × Dx + B × Dy + C × Dz

From t, we can calculate P, the intersection point. Is it correct/enough?

V1

V2

V3

Parallel Ray and Triangle

�21

t = −A × Ox + B × Oy + C × Oz + D

A × Dx + B × Dy + C × Dz

O (Ox, Oy, Oz)

||t|| P = O + tD

Ax + By + Cz + D = 0

V1

V2

V3

‣ The denominator is 0 if the normal [A, B, C] is perpendicular to the direction of the ray [Dx, Dy, Dz]. That is, when the ray is parallel to the plane.

‣ Need a special test whether the ray is parallel with the plane.

Triangle Behind or After the Ray?

�22

t = −A × Ox + B × Oy + C × Oz + D

A × Dx + B × Dy + C × Dz

O (Ox, Oy, Oz)

||t|| P = O + tD

Ax + By + Cz + D = 0

V1

V2

V3

‣ A ray doesn’t intersect with a plane if the triangle plane is behind the origin of the ray, i.e., t is negative.

t is positivet is negative

The Intersection Point Could be Outside the Triangle

�23

O (Ox, Oy, Oz) Ax + By + Cz + D = 0

V1

V2

V3

‣ Even if a real intersection point is found, it doesn’t mean the ray intersects with the triangle, because the intersection point could be outside the triangle.

‣ Use barycentric coordinates to test whether a point is outside of a triangle.

Finding the Nearest Intersection

�24https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-generating-camera-rays

intersect = false

foreach pixel in image build a camera ray tNearest = inf foreach triangle in mesh if (P = intersect(ray, triangle)) if (P.distance < tNearest) intersect = true tNearest = P.distance

if (intersect) pixel.color = shade(P) else pixel.color = backgroundColor

Acceleration Structures

�25

‣ Most of the time in ray tracing is spent on ray-triangle intersection, for which brute-force search is prohibitively slow.

‣ Prune the search space using space partitioning trees, which in the context of ray tracing are known as acceleration structures.

https://www.scratchapixel.com/lessons/advanced-rendering/introduction-acceleration-structure/bounding-volume

intersect(space, ray) { if ray doesn’t intersect space boundary: return else: foreach subspace in space if (subspace != empty) intersect(subspace, ray) }

Acceleration Structures

�25

‣ Most of the time in ray tracing is spent on ray-triangle intersection, for which brute-force search is prohibitively slow.

‣ Prune the search space using space partitioning trees, which in the context of ray tracing are known as acceleration structures.

https://www.scratchapixel.com/lessons/advanced-rendering/introduction-acceleration-structure/bounding-volume

intersect(space, ray) { if ray doesn’t intersect space boundary: return else: foreach subspace in space if (subspace != empty) intersect(subspace, ray) }

Acceleration Structures

�25

‣ Most of the time in ray tracing is spent on ray-triangle intersection, for which brute-force search is prohibitively slow.

‣ Prune the search space using space partitioning trees, which in the context of ray tracing are known as acceleration structures.

https://www.scratchapixel.com/lessons/advanced-rendering/introduction-acceleration-structure/bounding-volume

intersect(space, ray) { if ray doesn’t intersect space boundary: return else: foreach subspace in space if (subspace != empty) intersect(subspace, ray) }

Spatial Partitioning Trees

�26

Octree

https://en.wikipedia.org/wiki/Octree

KD Tree (2D example here)

http://groups.csail.mit.edu/graphics/classes/6.838/S98/meetings/m13/kd.html

Bounding Volume Hierarchy

�27https://en.wikipedia.org/wiki/Bounding_volume_hierarchyhttps://www.scratchapixel.com/lessons/advanced-rendering/introduction-acceleration-structure/bounding-volume

Bounding Volume Hierarchy

�27https://en.wikipedia.org/wiki/Bounding_volume_hierarchyhttps://www.scratchapixel.com/lessons/advanced-rendering/introduction-acceleration-structure/bounding-volume

Bounding Volume Hierarchy

�28http://www.pbr-book.org/3ed-2018/Primitives_and_Intersection_Acceleration/Bounding_Volume_Hierarchies.html

Calculate the bounding box for each triangle

Iteratively split the triangles to build a tree. Goal: bounding boxes of partitions shouldn’t overlap much.

Bounding Volume Hierarchy

�28http://www.pbr-book.org/3ed-2018/Primitives_and_Intersection_Acceleration/Bounding_Volume_Hierarchies.html

Calculate the bounding box for each triangle

Iteratively split the triangles to build a tree. Goal: bounding boxes of partitions shouldn’t overlap much.

Bounding Volume Hierarchy

�28http://www.pbr-book.org/3ed-2018/Primitives_and_Intersection_Acceleration/Bounding_Volume_Hierarchies.html

Calculate the bounding box for each triangle

Iteratively split the triangles to build a tree. Goal: bounding boxes of partitions shouldn’t overlap much.

Acceleration Structures Considerations

�29

‣ Time to build the tree vs. time to search. ‣ Can we built the tree offline? ‣ Tight bounding volumes provide more

precise intersect test, but are costly to build and to search.

‣ Tree structures take memory. ‣ Non-uniform geometry: do we need to have

adaptive partitioning strategies?

https://www.scratchapixel.com/lessons/advanced-rendering/introduction-acceleration-structure/bounding-volume-hierarchy-BVH-part1

‣ Realistic shading is made easier by recursive ray tracing. ‣ How to trace “secondary rays” depends on the surface property hit by the

primary ray, i.e., the ray from the camera.

Ray Tracing Makes Shading Easier

�30

Perfect reflective surface Reflection + refraction Diffusion

‣ For a diffuse surface, shooting a ray from a point to the light source allows us to determine the color contributed by the direct light.

Ray Tracing Makes Shading Easier

�31https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-overview/light-transport-ray-tracing-whitted

foreach pixel in image ray = buildRay(camera, pixel) if (P = nearestIntersect(ray, mesh)) shadowRay = buildRay(P, Light) if (nearestIntersect(shadowRay, mesh)) pixel.color = shade(P, Light) else pixel.color = darkColor else pixel.color = backgroundColor

P

LightUses a very simple shading model; will fix later.

‣ Simply trace more rays, recursively, to add reflection, refraction, diffusion, etc.

Recursive Ray Tracing

�32https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-overview/light-transport-ray-tracing-whitted

Assuming surfaces only reflect and/or refract, i.e., no

diffusion and other more complicated behaviors.

‣ Simply trace more rays, recursively, to add reflection, refraction, diffusion, etc.

Recursive Ray Tracing

�33https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-overview/light-transport-ray-tracing-whitted

castRay(ray, mesh) { if (P = nearestIntersect(ray, mesh)) reflectRay = buildReflectRay(P) refractRay = buildRefractRay(P) reflectColor = castRay(reflectRay, mesh)) refractColor = castRay(refractRay, mesh))

P.color = shade(reflectColor, refractColor) return P.color; }

P

Uses a very simple shading model; will fix later.

Trace Direction and When to Stop

�34https://www.scratchapixel.com/lessons/3d-basic-rendering/introduction-to-ray-tracing/raytracing-algorithm-in-a-nutshell

Tracing “forward” from the light source wastes lots of compute. Tracing “backward” from eye/camera captures rays that actually enter eyes.

Either define a depth, beyond which we stop tracing rays, or throw a die before start tracing a ray.

https://www.ebay.com/itm/RUSSIAN-ROULETTE-WHEEL-EDIBLE-ROUND-BIRTHDAY-CAKE-TOPPER-DECORATION-PERSONALISED-/333193900562

Ray Tracing Recap

�35

‣ Ray tracing is prohibitive slow vs. rasterization. ▹ Modern GPUs are well optimized for rasterization. New GPUs support limited form or ray

tracing. Hardware support for ray tracing is an active research area. ‣ Most of the ray tracing time is spent on ray-triangle intersect test. ‣ Ray tracing makes realistic shading easier. ▹ Shading naturally traces lights transport in space, which nicely fits the ray tracing framework. ▹ But technically shading is independent of visibility.

Reference Materials

�36

‣ Acceleration structures: ▹ Primitives and Intersection Acceleration, from PBR. ▹ Introduction to Acceleration Structures, from Scratchapixel. ‣ Pioneering ray tracing papers: ▹ Some techniques for shading machine renderings of solids. Proceedings of AFIPS 1968. ▹ An improved illumination model for shaded display. SIGGRAPH 1979, CACM 1980. ▹ Distributed Ray Tracing, SIGGRAPH 1984. ▹ The Rendering Equation, SIGGRAPH 1986.