IEEE TRANSACTIONS ON VISUALIZATION AND ......ray database obsolete and re-computing the ray-database...

IEEE

Proo

f

Second-Order Feed-Forward Renderingfor Specular and Glossy Reflections

Lili Wang, Naiwen Xie, Wei Ke, and Voicu Popescu

Abstract—The feed-forward pipeline based on projection followed by rasterization handles the rays that leave the eye efficiently: these

first-order rays are modeled with a simple camera that projects geometry to screen. Second-order rays however, as, for example, those

resulting from specular reflections, are challenging for the feed-forward approach. We propose an extension of the feed-forward

pipeline to handle second-order rays resulting from specular and glossy reflections. The coherence of second-order rays is leveraged

through clustering, the geometry reflected by a cluster is approximated with a depth image, and the color samples captured by the

second-order rays of a cluster are computed by intersection with the depth image. We achieve quality specular and glossy reflections at

interactive rates in fully dynamic scenes.

Index Terms—Specular reflections, glossy reflections, fully dynamic scenes, feed-forward rendering, interactive rendering

Ç

1 INTRODUCTION

MOST interactive computer graphics applications ren-der 3D scenes in feed-forward fashion, by projection

followed by rasterization. At a fundamental level, theapproach is efficient since projection is a straight forwardway to avoid considering ray/geometric-primitive pairsthat do not produce an intersection. By comparison, theray tracing pipeline requires acceleration schemes to avoidconsidering geometric primitives that do not intersect agiven ray. However many scenes of interest contain specu-lar reflective surfaces which extend and perturb first-orderrays. The resulting higher order rays cannot be modeledwith a simple camera that provides fast projection, andconsequently the classic feed-forward pipeline cannot ren-der reflections. The palliative approach for renderingreflections in interactive graphics applications is to approx-imate the reflected scene with a panoramic image, e.g., acube map, and to look up the reflected rays into the cubemap. This is a drastic approximation that produces largeerrors for reflected objects that are close to the reflectorsurface (Fig. 1left).

In this paper we propose extending the feed-forwardpipeline to handle second-order rays. Our method is basedon the fact that in the case of specular reflections second-order rays are locally coherent. We take advantage of thiscoherence by grouping second-order rays of nearby pixelsinto clusters. Although the rays in a cluster are coherent,

they usually do not pass through a common point and thusthey cannot be modeled with a conventional pinhole cam-era. One approach is to reduce the size of clusters until thepinhole approximation produces acceptable errors, but thisis inefficient for complex reflectors that require small clus-ters and thus a large number of cameras.

Another approach is to model clusters of second-orderrays using more powerful non-pinhole camera models, butsuch cameras introduce costly projection and non-linear ras-terization, and even a small ray approximation error produ-ces reflection discontinuity between clusters. Instead ofapproximating the rays of the cluster, we approximate the geome-try reflected by the cluster. The reflected geometry is approxi-mated by rendering a depth image for each cluster. Thecolor samples captured by the second order rays of a clusterare computed by intersecting the rays with the cluster’sdepth image.

Our method produces quality reflections at interactiverates (see Fig. 1 and accompanying video, which can be foundon the Computer Society Digital Library at http://doi.ieee-computersociety.org/10.1109/TVCG.2014.2314666). No pre-computation is required, thus our method supports fullydynamic scenes. Second-order rays are clustered on the fly,directly in the output image, which brings support for gen-eral scenes, with large, complex, and numerous reflectors.The reflected scene geometry is approximated efficiently, ondemand: our method only approximates the geometryneeded for the reflections in the current frame, the approxi-mation is done at the appropriate level of detail, and theapproximation is done automatically, without a prerequisitepartitioning of reflected geometry into objects. Our methodalso allows approximating glossy reflections (Fig. 2) by inter-secting the cluster depth image with multiple reflected raysper cluster pixel.

Our method essentially approximates second-order rayswith one additional feed-forward rendering pass for eachcluster. Although, in theory, our method could supporthigher-order rays, handling such rays is less efficient sincethey are less coherent, which translates into a larger numberof clusters. Whereas incorrect first-order reflections are

� L. Wang and N. Xie are with the State Key Laboratory of Virtual RealityTechnology and Systems, School of Computer Science and Engineering,Beihang University, Beijing, China.E-mail: [email protected], [email protected].

� W. Kei is with the Computer Science Department, Macao PolytechnicInstitute, Macau, China. E-mail: [email protected].

� V. Popescu is with the Computer Science Department, Purdue University,305 North University Street, West Lafayette, IN 47907-2107.E-mail: [email protected].

Manuscript received 15 Apr. 2013; revised 19 Feb. 2014; accepted 16 Mar.2014. date of publication xx xx xxxx; date of current version xx xx xxxx.Recommended for acceptance by G. Drettakis.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference the Digital Object Identifier below.Digital Object Identifier no. 10.1109/TVCG.2014.2314666

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 20, NO. X, XXXXX 2014 1

1077-2626� 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

IEEE

Proo

feasily noticeable and disturbing, higher-order reflectioninaccuracies are usually difficult to detect and as such theydo not warrant the additional cost—we render higher orderreflections using environment mapping.

The next section discusses prior work. Section 3 discussesour method in detail. Section 4 presents and discussesresults. Section 5 concludes the paper and sketches direc-tions for future work.

2 RELATED WORK

The problem of rendering specular reflections at interactiverates has been approached from many directions.

2.1 Image Based Rendering and Caching

One group of methods employs pre-computed or pre-acquired color samples. Such classic image-based renderingmethods include the light field [20], the lumigraph [19], andview dependent texture mapping [29]. The lumigraph wasmodified from storing color samples to storing a ray toray mapping, which allows changing the reflective andreflected object independently [21]. An outside-looking-in

Fig. 1. Specular reflections rendered with environment mapping (left), ray tracing (middle), and our method (right). Environment mapping producesincorrect reflections that fail to convey the reflected object’s proximity to the reflector surface (e.g., table leg reflected in floor, floor reflected in vase).Our method renders reflections comparable to those rendered with ray tracing. The frame rate for our method is 15 fps, compared to 1.1 fps for raytracing (i.e., Optix with BVH acceleration).

Fig. 2. Glossy reflections rendered with our method, for two levels ofglossiness, at 5 fps.

2 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 20, NO. X, XXXXX 2014

IEEE

Proo

f

parameterization of the light field has been proposed byrevolving a construction camera around the reflective object[22]. The environment light field map [18] goes in the oppo-site direction of achieving an inside-looking-out parameteri-zation of the light field.

The idea of rendering acceleration by reusing pre-com-puted color has recently evolved into irradiance [37], [38],[39] and radiance [40], [41] caching. Irradiance cachingmethods reuse shading results from nearby pixels and areefficient, but they only apply to diffuse surfaces. Radiancecaching [40] overcomes the diffuse surface limitation bystoring full incoming radiance, which is interpolatedbetween pixels taking into account the pixels’ BRDFs.Pre-convolved radiance caching [41] accelerates radiancecaching by storing pre-computed shading expressionsper surface as opposed to per pixel, at the cost of losinghigh-frequency detail.

These caching approaches are ill-suited for renderingspecular reflections as the appearance of specular surfaceschanges substantially even with a small change in view-point, which leads to impractically large ray databases.Moreover, dynamic scenes are challenging for image-basedrendering methods since any change in the scene makes theray database obsolete and re-computing the ray-database istoo costly to be done per frame.

2.2 Ray Tracing

The ray tracing pipeline [1], [2] naturally supports ren-dering reflections. The major concern is performance.The brute force approach of intersecting every scene tri-angle with every reflected ray is prohibitively expensiveand the goal is to avoid performing intersection teststhat do not yield an intersection. A multitude of accelera-tion schemes have been developed, including level ofdetail [5], KD-trees [3], bounding volume hierarchies(BVH) [4], and beam tracing [43], running on the CPU[11], [12] and on the GPU [13], [14], [15], [43]. Renderingspecular and glossy reflections with ray tracing posesseveral challenges.

One challenge is the large number of per-pixel raysneeded to achieve adequate reflection antialiasing. Whenthe solid angle subtended by the reflected rays at a pixel islarge, a large number of rays are needed for adequate sam-pling of reflected geometry. Another challenge is posed byglossy surfaces. Specular, mirror-like surfaces have coher-ent normals and therefore generate coherent reflectedrays, which can be adequately sampled with a small num-ber of per-pixel rays. Glossy surfaces however generateincoherent reflected rays and they require a large numberof rays per pixel. A third important challenge is that, inthe case of dynamic scenes, the data structure used toaccelerate ray tracing has to be re-computed on the fly. Forexample, the Optix ray tracer used to render the compari-son images in Fig. 1 spends 910 ms for the living roomand 260 ms for the bathroom per frame to re-construct itsBVH tree.

As GPUs remain primarily feed-forward renderingmachines, researchers have attempted to extend the feed-forward pipeline to rendering reflections. There are twofundamental options: processing the reflected triangles

with the feed-forward pipeline, or processing the reflec-tive triangles.

2.3 Feed-Forward Processing of ReflectedTriangles

Consider a triangle that is first reflected before being pro-jected onto the output image. Processing such a reflected tri-angle with the feed-forward pipeline requires overcomingtwo challenges. First, one has to be able to project onto theimage plane a vertex that is first reflected. Second, one hasto perform a non-linear rasterization of the reflected triangle(i.e., curved reflected triangle edges, non-linear variation ofrasterization parameters within the triangle). The secondchallenge can be overcome by subdividing the reflected tri-angle until conventional, linear rasterization provides anacceptable approximation. However, the first challenge isdifficult to overcome. If the reflector were a sphere, projec-ting a reflected vertex requires solving a quartic. For generalreflectors modeled with a triangle mesh no closed-form pro-jection exists.

The problem of projecting reflected vertices has beenaddressed in several ways. One method considers thereflected space subdivision induced by the reflector’s trian-gles; a reflected vertex is projected by looking up the subdi-vision cell that contains it; the lookup is accelerated usingan approximate representation called an explosion map[27]. Another method [8] leverages the coherence ofreflected rays and approximates a group of reflected rayswith a conventional planar pinhole camera. The planar pin-hole cameras are stored at the leafs of a BSP tree that definesa sample-based camera. The sample-based camera projectsreflected vertices with bounded error. A third methodsearches for the projection of a reflected vertex through alocal search executed on the GPU [28]. All these methodsscale poorly with reflector complexity and with the numberof reflectors. Complex and numerous reflectors increase thecomplexity of the explosion map, of the sample-based cam-era, or of the search for the reflected vertex projection, andincrease the number of projections for a given triangle dueto multiple projections.

2.4 Feed-Forward Processing of ReflectiveTriangles

The other option for rendering reflections by projection fol-lowed by rasterization is to process the triangles that formthe reflective surface. The vertices of the reflective triangleare projected as usual, vertex normals are interpolated overthe projected triangle, and per-pixel reflected rays are com-puted straightforwardly. However, finding the color of thesamples captured by the reflected rays is challenging. Toavoid the complexity of ray tracing the scene in search ofthe reflected ray color, several methods resort to approxi-mating the reflected scene geometry.

Environment mappingmakes the drastic assumption thatall reflected geometry is infinitely far away from the reflector[6], [7]. With this assumption, the reflected scene can bemodeled with an environment map (typically parameter-ized as a cube map), and the reflected ray is simply lookedup using solely its direction, ignoring the actual 3D pointfrom where it emanates. The reflection is antialiased

WANG ET AL.: SECOND-ORDER FEED-FORWARD RENDERING FOR SPECULAR AND GLOSSY REFLECTIONS 3

IEEE

Proo

f

through mipmapping in the environment map. The methodscales well with reflector complexity and multiple reflec-tions are handled at no extra cost—the fact that multiplereflected rays intersect the same region of the environmentmap has no consequence on performance. Due to its lowcost and robustness, environment mapping is the method ofchoice for rendering reflections when performance is at apremium. However, when the reflected object is close to thereflective surface, environment mapped reflections arewrong, failing to convey the object’s proximity to the reflec-tive surface (Fig. 1).

Environment maps have been extended to approximatesingle- and multiple-lobe BRDF glossy reflections [23], [24].In an effort to improve reflected geometry approximationquality, environment maps have been enhanced with per-pixel depth [25]. However, the resulting environment maponly captures surfaces visible from its center which leads toserious errors in the reflection due to missing samples asmany reflected rays intersect surfaces that are not part ofthe environment map.

In order to reduce the missing sample errors one optionis to subdivide the reflected scene into objects and toapproximate each object individually for each reflector,using a billboard or a conventional depth image at first [26],and later using a non-pinhole depth image [9]. Good reflec-tions are obtained, but subdividing the scene into reflectedobjects is not always possible and, when it is possible, theapproach does not scale with scene complexity.

A second option is to approximate scene geometry withmore powerful depth images. Layered depth images (LDIs)allow for a variable number of samples along a conventionalcamera ray [33] and they have been used to accelerate indi-rect illumination computation [34], [35]. Although LDIsavoid the redundancy of overlapping depth images, LDIconstruction is laborious (requiring depth peeling or merg-ing overlapping depth images), which precludes their usein the context of dynamic scenes where LDIs would have tobe constructed for every frame.

A third option is to use a flexible non-pinhole camera,such as the graph camera [36], to capture the entire scene ina single-layered image [9]. The graph camera offers closed-form projection and the graph camera depth image can beconstructed (i.e., rendered) for every frame. However,graph camera constructors are limited to simple 2D mazeswith right angle turns, and therefore graph cameras cannotapproximate well the reflected rays resulting from specularreflections in complex scenes like the ones considered inour paper (Fig. 1).

Approximating reflected geometry has also been pur-sued by researchers aiming to accelerate ray tracing.Approximations include geometry fields that can be lookedup to estimate the reflected ray color [16] and mipmappedgeometry images [17]. These approximations cannot becomputed on the fly which precludes fully dynamic sceneswith deforming objects.

Our method falls in this category of methods that feed-forward process reflective triangles and that approximatereflected geometry to simplify reflected ray/geometry inter-section. Our method computes a quality approximation ofthe reflected scene for each frame which results in qualityreflections for fully dynamic scenes, and it does not require

partitioning the reflected scene into objects, which bringsscalability with scene complexity.

3 ALGORITHM

Consider a scene that contains diffuse surfaces, i.e., surfaceswith perfectly diffuse reflectance, specular surfaces, i.e., sur-faces whose reflectance model is well approximated by acombination of a perfectly specular component and of a dif-fuse component, and glossy surfaces, i.e., surfaces whosereflectance model is well approximated by a combination ofa single-lobe symmetrical BRDF and of a diffuse compo-nent. The scene is modeled with triangles.

3.1 Algorithm Overview

Given a desired view V, the scene S is rendered from V withthe following algorithm:

1. Render S from V. For every pixel p record:

a. Diffuse component p.rgbdb. Specular and glossiness levels p.s and p.gc. Normal and depth p.n and p.zd. Reflective object ID, p.rID

2. Cluster non-diffuse pixels.3. For every cluster C, finalize reflections as follows:

a. Construct cluster camera K.b. Render S with K to obtain cluster depth

image D.c. For every pixel p in C:

i. For every reflected ray ri Intersect ri withD,i.e., di ¼ ri \D.

ii. Set non-diffuse component p:rgbn ¼ Gðdi;p:gÞ.

iii. Pixel color p.rgb ¼ LERP ðp:rgbd; p:rgbn; p:sÞ.The algorithm has three main steps. The first step takes arendering pass over the scene to compute the diffusecomponent for every pixel, to set the pixel specular andglossiness levels, to compute the pixel normal and depthby conventional interpolation of vertex values, and to setthe ID of the reflective object to which the pixel belongs.The specular level of a pixel p.s ranges from 0 for per-fectly diffuse to 1 for perfectly specular. The glossinesslevel of a pixel p.g ranges from 0 for perfectly specular,mirror-like reflections, to 1 for a glossy surface with thewidest BRDF lobe. The second step groups neighboringnon-diffuse pixels with similar reflected rays into clustersas described in Section 3.2 The third step computes thereflections and finalizes the frame one cluster at the time.For each cluster, a planar pinhole camera is constructedto encompass all the reflected rays of the cluster asdescribed in Section 3.3 (step 3a above). Then the geome-try reflected by the cluster is approximated by renderingthe scene with the cluster camera (step 3b).

The resulting depth image is used to finalize the compu-tation of the color for the pixels in the cluster (step 3c). Foreach pixel in the cluster, the depth image is intersected withreflected rays, as described in Section 3.4 (step 3ci above).For perfectly specular, mirror-like surfaces (i.e., a p.g of 0),there is a single reflected ray per pixel, defined by the sur-face point, the pixel normal, and the eye position. The larger


IEEE

Proo

f

the glossiness factor p.g, the larger the number of reflectedrays. The non-diffuse component of the current pixel iscomputed by blending the color samples di found at thereflected ray/depth image intersections based on the gloss-iness level p.g, using function Gðdi; p:gÞ, which correspondsto an application chosen BRDF. The final pixel color is com-puted by linearly interpolating the diffuse and non-diffusepixel color components with weights defined by the pixelsspecular level p.s (step 3.c.iii).

3.2 Non-Diffuse Pixel Clustering

The first pass over the scene (step 1 in Section 3.1) computesthe pixel normal which translates to a reflected ray. We takeadvantage of the coherence of per-pixel reflected rays bygrouping nearby non-diffuse pixels into clusters. We havedesigned an algorithm for clustering non-diffuse pixelsbased on the following considerations:

a. There should be as few clusters as possible, sinceeach cluster requires rendering the scene to constructits depth image.

b. The cluster should be small enough such that a con-ventional planar pinhole camera constructed for thecluster captures the samples reflected by the cluster.

c. Clustering should be fast as it runs for each frame.We assign non-diffuse pixels directly to their cluster by bin-ning pixel normals into a 3D array of bins whose resolutionis adapted for each frame to achieve a good tradeoffbetween number of clusters and reflected ray modelingfidelity. Our algorithm proceeds as follows.

Offline:

1. Partition non-diffuse triangles into reflective objectsOnline, once per frame:

2. For every reflective object R:

a. Set normalized screen area aR3. Compute the number of visible reflective objects, nv.4. For each non-diffuse pixel p, assign p to cluster

(i, j, k).

a. i ¼MAXTHETABINS � p:n:u=360o � aR=nv

b. j ¼MAXPHIBINS � p:n:’=180o � aR=nv

c. k ¼ p:rID

Binning normals based solely on their orientation canlead to grouping distant pixels in the same cluster. Forexample in Fig. 1(top row), the vase, the teapot, and the

lamp have patches with identical normals. Grouping allthree patches in the same cluster is inefficient since it wouldresult in unnecessarily large depth images. This problemcould be avoided by building contiguous clusters in bot-tom-up quadtree fashion in the output frame, but such anapproach is slow.

Step 1. We group non-diffuse triangles offline into objects(step 1), and we prevent a cluster from spanning multiplereflective objects. This is done using the reflective object IDas a third dimension of the array in which normals arebinned, in addition to the normal’s spherical coordinates uand ’. The partition of non-diffuse triangles follows the nat-ural subdivision of the scene into objects. Fig. 3 illustratesthe 25 and 15 reflective objects for the living room and thebathroom scenes (Fig. 1), respectively. We render reflec-tions only for the non-diffuse pixels visible in the outputframe—the offline partitioning of the scene into reflectiveobjects is only used for fast clustering. Whereas the numberof reflective objects is fixed, the resolution along the u and ’

dimensions of the array of bins is set online, for each reflec-tive object and for each frame. The u and ’ dimensions ofthe bins depend on two quantities.

Step 2. One quantity, aR, measures the footprint of thereflective object in the output frame, as the percentage ofoutput frame pixels where the reflective object is visible.

Step 3. The second quantity is the number of reflectiveobjects nv that are visible in the output frame, computed asthe number of reflective objects whose aR is not 0 (step 3).

Step 4. Each pixel is assigned to a cluster based on itsnormal and on the index of the reflective object to whichit belongs. The maximum possible u and ’ resolutionMAXTHETABINS x MAXPHIBINS is modulated using aRand nv. MAXTHETABINS and MAXPHIBINS are con-stants that we set to 13 and 8, respectively, for all exam-ples shown in the paper. The larger the relative footprintof the object, the finer the bins, and the larger the numberof reflective objects, the coarser the bins. Although themaximum number of bins for the two scenes is 25� 13�8 ¼ 2;600 and 15� 13� 8 ¼ 1;560, respectively, the num-ber of clusters is given by the number of bins that are notempty. For example, in the case of a single large spherethat covers the entire screen, the maximum number ofclusters is 1� 13� 8=2 ¼ 52, which accounts for the factthat only half of the sphere is visible.

Fig. 4 illustrates the clusters used to render the reflec-tions in the four images from Figs. 1 and 2. The floor definesa single cluster. The table top and the floor define two dif-ferent clusters, although they are oriented the same way.The vase defines more clusters when seen in more detail(top right versus top left). The spherical coordinate systemused over-samples clusters near the pole (see top rightimage in Fig. 4), a small disadvantage outweighed by theadvantage of its simplicity. Clustering based on normals asopposed to based on reflected rays avoids over-clusteringat the silhouette of reflective objects and has good frame toframe stability.

3.3 Cluster Camera Construction

Once clustering is complete, reflections are computed forone cluster at the time. The first step is to construct a

Fig. 3. Visualization of reflective objects. Diffuse objects are shown ingrey.


IEEE

Proo

fconventional planar pinhole camera for the current clusterthat allows approximating the geometry reflected by thecluster. We use a conventional camera for three fundamen-tal reasons. First, conventional camera allows rendering thescene geometry efficiently to obtain a depth image thatapproximates the reflected geometry. Second, reflected raysproject to straight lines onto depth images constructed withconventional cameras, which makes ray/depth image inter-section efficient. Third, a conventional camera models thereflected rays of a planar cluster perfectly. Planar specularsurfaces abound in man-made scene and handling themwell is an important reflection rendering algorithm designconsideration.

The goal is for the camera to capture all the samplesreflected by the cluster. One consideration is for the camerato have enough field of view, such that the camera frustumcontains all the reflected rays of the cluster. A second con-sideration is for the rays of the camera to approximate thereflected rays of the cluster as closely as possible in order toavoid disocclusion errors in the reflection. A disocclusionerror occurs when a reflected ray intersects a surface at asample that is missing from the depth image rendered withthe cluster camera. The camera cluster is constructed as fol-lows (Fig. 5);

1. Set image plane through 3D point Pc and normal nc

a. Pc ¼ SPi=Nb. nc ¼ Sni=N

2. Set far plane at distanceD3. Set center of projection ec

a. Compute e0 ¼ Reflectðe; Pc; ncÞ.b. Displace e0 along e0Pc : ec ¼ e0 þ ðPc � e0Þf:

4. Set the image frame aabb.5. Set the image resolution w x h.

Step 1. The image plane of the cluster camera is definedby the cluster centroid and the cluster normal. The clustercentroid is the average of the 3D reflector surface pointsover all cluster pixels (N is the number of pixels in the clus-ter). The cluster normal is the average over all cluster pixelnormals. The image plane also serves as near plane. Fig. 5illustrates cluster camera construction in 2D, for clarity. Acurved reflector is partitioned into a cluster between pointsP1 and PN , where the surface normals are n1 and nN andthe reflected rays are r1 and rN . The output frame center ofprojection is e.

Step 2. For specular reflections, the far plane of the clustercamera is set to be parallel to the image plane at a distanceD equal to the scene diameter for specular reflections(Fig. 5). For glossy reflections the far plane is set closer tothe image plane, as described in Section 3.5

Step 3. The center of projection ec of the cluster cam-era is defined such that the cluster camera rays approxi-mate the reflected rays of the cluster as well as possible.We construct ec such that the axis aligned bounding boxaabbn of the projections of the near reflected ray end-points be of similar size to the axis aligned boundingbox aabbf of the projections of the far reflected ray end-points. In Fig. 5, the near endpoints of reflected rays r1and rN are P1 and PN ; the far endpoint for ray rN is QN ,and for r1 it is Q1 (actual location of Q1 is not shown tokeep the figure compact).

We set ec in two steps. First, the output frame center ofprojection e is reflected over the cluster camera image planeto e0 (Fig. 6). Then, ec is computed by displacing e0 towardsor away from the centroid of the cluster Pc. The displace-ment is controlled by a scalar value f , which is set such thatthe diagonal of the 2D AABB aabbf be approximately equalto the diagonal of the 2D AABB aabbn. We set f as shown inEquation (1), where dn is the length of the diagonal of aabbnwhen ec is at e0; df is the length of the diagonal of aabbfwhen ec is at e0, and dF is the length of the axis alignedbounding box of the far endpoints of the reflected rays onthe far plane. As ec moves on e0Pc, the projections P 0

i ofpoints Pi do not change much, as points Pi are close to theimage plane. Consequently, the length of the diagonal ofaabbn is approximately constant, and Equation (1) approxi-mates it to the length dn it has when ec ¼ e0. On the other

Fig. 5. Construction of the image plane (also the near plane) and of thefar plane for the cluster camera.

Fig. 4. Visualization of pixel clusters used to render the reflections inFigs. 1 and 2. Diffuse pixels are shown in grey.


IEEE

Proo

f

hand, the length of the diagonal of aabbf is sensitive to theposition of ec on e0Pc. Equation (1) sets ec such that thelength of the diagonal of aabbf is equal to dn:

f ¼ dnðdF � dfÞdfðdF � dnÞ: (1)

Fig. 6 illustrates the case of the convex reflector fromFig. 5. Points Q0

1; P01; P

0c; P

0N;Q

0N correspond to the cluster

camera image plane projection from e0 of the correspond-ing points from Fig. 5. The length of segment Q0

1Q0N corre-

sponds to df , that of P01P

0N corresponds to dn, and that of

Q1QN corresponds to dF . At e0 aabbn is smaller than aabbf ,i.e., df > dn. Since dF > df and dF > dn; 0 < f < 1. Thedisplacement moves ec from e0 towards PC . This decreasesaabbf with respect to aabbn. The cluster camera with centerof projection ec approximates the reflected rays better thana cluster camera with center of projection at e0. The direc-tion of ecP

01 is closer to the direction of r1 than is the direc-

tion of e0Q01.

For a cluster corresponding to a planar reflector, allreflected rays intersect at e0, and consequently the clustercamera should use e0 as a center of projection. Such a clustercamera models the reflected rays perfectly and there are nodisocclusion errors. For a planar reflector the near and farendpoints project from e0 to the same image plane point.Consequently aabbn and aabbf are identical, dn ¼ df , and,according to Equation (1), f becomes 1, which implies thatec is set to e0 as desired.

Step 4. The image frame is set at by projecting thenear and far reflected ray endpoints with the finalizedcenter of projection ec. The image frame is the axisaligned bounding box of these projections. This way thecluster camera has a field of view that is guaranteed toencompass all the reflected rays of the cluster (orangeshaded area in Fig. 6).

Step 5. The image resolution is set to match the resolutionof the cluster. The 3D points of a few pairs of cluster pixelsthat are either in consecutive rows or in consecutive col-umns in the output image are projected with the clustercamera. The average distance between the pairs of projec-tions are used to define the pixel size p. The cluster cameraresolution is defined as w� h, where w ¼ aabb:w=p, andh ¼ aabb:h=p.

3.4 Reflected Ray/Depth Image Intersection

Once the camera cluster is complete, the scene is renderedwith it to obtain the cluster depth image. A depth image is apowerful approximation of geometry: the approximationcan be constructed quickly through conventional renderingto obtain a frame buffer with color and depth per pixel, thedepth image captures geometry with controllable level ofdetail, and one can intersect a depth image with a single rayefficiently. The efficient intersection between a ray and adepth image is well known—it has been used in inverseimage-based rendering by 3D warping [31], in renderingsurface geometric detail [32], and in rendering reflections[26]. We briefly sketch the algorithm here for completeness.

Given a ray r and a cluster depth image DI, the closest .intersection between r and DI, if any, is found by projec-ting r onto DI. Let r0 be the projection of r with the clustercamera that rendered DI; r0 is traversed from the near end-point to the far endpoint with one-pixel steps. Let a and bbe the previous and current steps on r0. If the 2D segments½ð0; zraÞ; ð1; zrbÞ� and ½ð0; DI½a�Þ; ð1; DI½b�Þ� intersect, andintersection is found, and the search stops. zra is the depthalong the ray at a, and DI½a] is the depth in the depth imageat a. A measure of depth that is linear in screen space isused, i.e., proportional to 1/z. If the end of r0 is reachedthere is no intersection.

3.5 Glossy Reflections

A point on a glossy surface does not reflect along a singledirection, but rather along a solid angle centered at the spec-ularly reflected ray (i.e., the ray obtained by reflecting theoutput image ray over the point’s normal). We supportglossy reflections with the following three modifications tothe algorithms described above for specular reflections:

1. The field of view of the cluster camera has to be con-structed to take into account the non-zero solid anglesubtended by a glossy reflected cone. We do this byextending the field of view computed at Step 4 of thecluster camera construction algorithm (Section 3.3)with the angle of the reflected cone. This way theresulting cluster depth image captures the additionalgeometry reflected by the glossy cluster.

2. Glossy surfaces only have well defined reflectionsclose to the reflector surface. We take advantage ofthis fact by setting the far plane of a cluster cameraconstructed to render a glossy reflection based onthe glossiness level. For surfaces that are morematte, the far plane can be closer to the near plane,compared to surfaces that are more specular. Bring-ing in the far plane as much as possible reduces theamount of geometry that has to be reflected. Whenno object is sufficiently close to a glossy surface, theresulting cluster depth image is empty, and no sub-sequent ray/depth image intersections are needed.To provide for a gradual fade away of a glossyreflection as a reflected surface moves progres-sively farther from the glossy surface, the glossyreflection is blended with the diffuse color of thesurface with a weight that decreases to 0 as the dis-tance to the reflected surface becomes the distanceto the far plane.

Fig. 6. Construction of the eye e0 of the cluster camera.


IEEE

Proo

f3. Glossy reflections require multiple reflected rays per

reflector surface point. We intersect several reflectedrays with the cluster depth image, for each glossysurface pixel. The reflected rays sample uniformly acone with apex at the surface point and with an axisdefined by the reflected ray generated by the pixelnormal. The solid angle covered by the conedepends on the glossiness level. For surfaces close tospecular, the angle is small, and therefore the num-ber of rays is small. For a more matte surface theangle is larger requiring additional rays. For theexamples shown in this paper the number of raysper glossy pixel ranges from 8 to 49.

4 RESULTS AND DISCUSSION

We tested our method on two indoor scenes with numerousspecular and glossy reflections. The living room scene(Fig. 1, top andmiddle rows) has 286K triangles, out of which156K are diffuse and 132K are non-diffuse. The bathroomscene (Fig. 1 bottom row) has 90K triangles, out of which 44Kare diffuse and 46K are non-diffuse.

4.1 Quality

4.1.1 Specular (Mirror Like) Reflections

As shown in Fig. 1 and in the accompanying video, avail-able in the online supplemental material, our method

produces quality specular reflections. Unlike in the case ofenvironment mapping, objects close to the reflector arereflected correctly, conveying the proximity between thereflected and reflecting object. There is no reflection discon-tinuity between clusters because the clusters have a slightoverlap, which prevents any gaps, and because the reflectedrays are continuous over the smoothly changing reflectorsurface, which prevents any misalignment of the reflectionfrom cluster to cluster (see Fig. 7 bottom).

The reflections rendered with our method are comparableto reflections rendered by ray tracing. Throughout this paperand the video, available in the online supplemental material,reflections rendered with our method are rendered at a reso-lution of 512� 512 with uniform 2� 2 super-sampling (i.e.,1;024� 1; 024 before output frame reconstruction); the reflec-tions rendered by ray tracing use an equivalent 512� 512output image resolution with four rays per pixel. We useNVIDIA’s Optix ray tracer [30]. Fig. 8 shows pixel valuedifferences between our method and ray tracing. The aver-age absolute pixel channel differences (i.e., L1 norm) aresmall, i.e., 6, 3, and 9, for each of the three rows, respectively(we use eight bit RGB channels with values from 0 to 255).

One reason for the difference is the large anglebetween reflected rays at the reflector edges, which leads

Fig. 7. Disocclusion errors (red) decreasing as more clusters are used,for the image in the middle row of Fig. 1. Diffuse pixels are shown inblack and only the reflective component is shown for non-diffuse pixels.The bottom row shows that there are no reflection discontinuitiesbetween adjacent clusters.

Fig. 8. (Left) Difference images between our method and ray tracing forthe three rows in Fig. 1. (Right) same difference images with intensitiesscaled up by a factor of 50.


IEEE

Proo

f

to minification errors. Another reason for the differenceis the slightly different sampling of the diffuse objects,which produces differences most visible at color edges.Our method uses bilinear interpolation of the intermedi-ate sampling provided by the depth image, whereas raytracing samples the diffuse geometry directly with addi-tional rays.

Differences are also caused by surfaces that should bevisible in the reflection at a cluster but that are not cap-tured by the cluster depth image. The reflected rays donot pass exactly through the cluster camera’s center ofprojection so it can happen that a few reflected rays reachsurfaces that are not visible to the cluster’s camera. Wemeasured the number of pixels per frame where such dis-occlusion errors occur over a sequence of 1,000 frames.The maximum/average percentage of disocclusion errorpixels is 1.3 percent/0.62 percent for the living roomscene, and 3.5 percent/2.75 percent for the bathroomscene. The disocclusion error is controlled by reducingthe size of the clusters. The fewer the reflected rays thatare approximated with a single planar pinhole camera,the higher the approximation fidelity (Fig. 7).

Finally, when curved reflectors have a small screen foot-print, the cluster has only one or a few rays, which can

lead to not finding an intersection between the reflected rayand the depth image of the cluster. When an intersection isnot found, the reflected ray is looked up in an environmentmap (Fig. 9).

Our method approximates the reflected geometry foreach output frame using depth images, which allows adapt-ing the level of detail of the reflected geometry as neededfor the current frame. The resolution of the depth image ren-dered for a cluster is commensurate with the resolution ofthe cluster of reflected pixels. Fig. 10 shows that a lower res-olution for the depth images would lead to blurriness (dis-tant part of floor reflected in vase) and jagged edges (tableleg reflected in floor).

4.1.2 Glossy Reflections

Our approach approximates glossy reflections by intersect-ing multiple reflected rays with a cluster depth image foreach cluster pixel. Fig. 11 shows that our method achievesquality glossy reflections, comparable to those obtained byray tracing. For Fig. 11 our method uses eight reflected raysper glossy pixel. Fig. 12 shows the less noisy but moreexpensive glossy reflections obtained with our methodwhen 49 rays per pixel are used.

Fig. 9. Visualization in red of pixels whose reflected rays are looked up inan environment map, for Fig. 1(top). Diffuse pixels are shown in blackand only the reflective component is shown for non-diffuse pixels.

Fig. 10. Reflections rendered with our algorithm (left) and with depthimages with 3� 3 lower resolution than the resolution computed by ouralgorithm (right). Diffuse pixels are shown in black and only the reflectivecomponent is shown for non-diffuse pixels.

Fig. 11. Glossy reflections rendered for two glossiness levels (left versusright), with our method and without a diffuse component (top), with ourmethod and with a diffuse component (middle), and with ray tracing(bottom).


IEEE

Proo

f4.2 Performance

All performance numbers reported in this paper wererecorded on a PC workstation with a 3.4 GHz Intel(R) Corei7-2600 CPU, with 4 GB of memory, and with an NVIDIAGeForce GTX 570 graphics card.

4.2.1 Specular (Mirror Like) Reflections

The performance of our method and the comparison to raytracing is given in Table 1. Ray tracing used the boundingvolume hierarchy acceleration and the BVH data structureis reconstructed every frame as needed for the dynamicscenes. Performance was measured on two paths of 900frames each, through each of the two scenes. The scenescontain many reflective surfaces and approximately halfthe pixels in a frame are non-diffuse. Our method sustains10 fps for all four paths, and it is about five times fasterthan ray tracing for the simpler bathroom scene, and about10 times faster than ray tracing for the more complex livingroom scene. To explain the better scalability of our methodwith scene complexity we breakdown the performanceanalysis as follows.

Table 2 gives the maximum and average times in milli-seconds for the main steps of our algorithm (see Section 3.1).As expected, the first pass, which entails rendering thescene geometry with simple shading, and the second pass,which entails binning pixels with a simple pass over theimage, take negligible time. The construction of the clustercameras is expensive, as it requires multiple passes over thepixels in the cluster and concurrent writes for the computa-tion of the cluster point, normal, and AABB. Rendering thecluster depth images is also laborious as it implies a passover the scene geometry for each cluster. Finally, computingthe intersection between the reflected rays and the clusterdepth images takes about half the time compared to each ofthe previous two steps.

The rendering parameters that could affect performanceare the number of clusters (Table 3), the output frame reso-lution (Table 4), the cluster depth image resolution (Table 5),and the number of diffuse triangles (Table 6). Tables 3, 4, 5,and 6 report average performance for the living room scene.

Our method renders the non-diffuse triangles and parti-tions the resulting non-diffuse pixels into clusters. Conse-quently, performance does not depend on the number ofnon-diffuse triangles, but only on the number of clusters(Table 3). All steps depend on the output image resolution:for a bigger output image resolution, the first pass renders abigger image, there are more pixels to cluster, camera con-struction handles clusters with more pixels, and the resolu-tion of the depth image for each cluster is higher whichtranslates to longer depth image rendering and reflectedray/depth image intersection times. This translates toslower frame rates for higher resolution (Table 4). However,the cost is not proportional to the number of output imagepixels—higher output image resolutions do not increase thecomplexity of the non-diffuse geometry, thus the number ofclusters remains the same, each cluster grouping a largernumber of pixels.

Fig. 12. Glossy reflections from Fig. 11 now rendered with 49 rays pernon-diffuse pixel.

TABLE 1Performance of Our Method Compared to Optix

TABLE 2Performance in Milliseconds for Various Algorithm

Steps as Defined in Section 3.1

TABLE 3Average Frame Rate for Various Numbers of Clusters

TABLE 4Average Frame Rate for Various Output Resolutions

TABLE 5Average Frame Rate for Various Depth Image Resol’s

TABLE 6Average Frame Rate for Various Numbers of Diffuse Tris


IEEE

Proo

f

The frame rate depends very little on the resolution of thedepth image—forcing the resolution to be half or twice asmuch as the resolution computed by the algorithm does notchange the frame rate substantially (Table 5). For the stepsin Table 2, only the last two depend on the depth image res-olution. Step 3b depends on the number of passes, i.e., thenumber of depth images, and it depends little on the resolu-tion of individual depth images. Step 3ci requires substan-tially less time than Step 3b, thus the variable number ofsteps along the projection of the reflected ray onto the depthimage (Section 3.4) has little influence on the overall framerate. The number of diffuse triangles only affects Step 3b,and reducing the number of diffuse triangles benefits over-all performance (Table 6) until the cost of Step 3b is toosmall compared to that of Step 3a.

We can now explain why our method scales better withscene complexity than ray tracing. For our method, cluster-ing, cluster camera construction, and reflected ray depthimage intersection mainly depend on output image resolu-tion. Because of this, simple scenes (e.g., the Cornell box)rendered at high resolution will be handled faster by raytracing. Whereas depth image construction does depend onscene complexity, the two scenes are still simple enough fordepth image construction to take roughly the same amountof time (Table 6). This is not the case for BVH constructionneeded by ray tracing, which takes substantially longer forthe living room scene (910 ms) compared to the bathroomscene (260 ms). Our method depends on the number of clus-ters (Table 3), which is similar for all four paths used forTable 1, as shown in Table 7. In conclusion, our methodscales better with diffuse scene complexity than ray tracingbecause the conventional GPU rendering of the clusterdepth images scales better than the construction of the accel-eration structure needed for ray tracing.

Regarding the dependence on reflector complexity, ourmethod handles optimally planar reflectors, which are fre-quently encountered in man-made scenes, and ray tracingdoes not. Complex reflectors, with high curvature and highfragmentation are challenging for both our method and raytracing: they imply a large number of clusters for ourmethod, and ray tracing has to filter the reflected geometryby shooting tens or hundreds of rays per pixel. Our methodfilters geometry at a much lower cost during cluster depthimage rendering. Both our method and ray tracing areintended for large reflectors that produce coherent reflec-tions; environment mapping should remain the approach ofchoice for very high complexity reflectors, where it is diffi-cult to judge reflection accuracy and hence the additionalcost is not justified.

4.2.2 Glossy Reflections

The difference between rendering perfectly specular, mirror-like reflections and rendering glossy reflections consists of

intersecting the cluster depth image with multiple reflectedrays for each glossy pixel. However, the performance impli-cation is that, in addition to having to intersect multiple rays,it is also the case that each intersection is more expensive.Whereas the pinhole camera constructed for a clusterapproximates the one-per-pixel specularly reflected rayswell, glossy reflected rays are markedly divergent from thecluster camera rays. Because of this, the length of the seg-ment where a glossy ray projects onto the depth image is lon-ger, leading to more steps for finding the intersection. Theglossier the surface (i.e., the more matte and the less mirror-like), the more divergent the rays, and the higher the ray /depth image intersection cost. Glossy rendering perfor-mance is given in Table 8. The frame rate is lower comparedto that for mirror-like reflections, but the advantage over raytracing is maintained. Higher performance more approxi-mate glossy reflections can be obtained by intersecting a sin-gle ray with the cluster depth image and averaging samplesin a neighborhood centered at the intersection.

In terms of memory requirement, the algorithm scaleswell since the cluster depth images do not have to be storedin memory simultaneously—the memory is reused as soonas the reflection for a cluster is completed. The total amountof GPU memory required for rendering the 512� 512 reflec-tions shown here is 87 MB, most of which (i.e., 64 MB) isused for the framebuffer with position, normal, cluster ID,and color channels.

4.3 Implementation Notes

The first rendering pass that finalizes the diffuse pixels andinitializes the non-diffuse pixels (step 1 in Section 3.1) isdone on the GPUwith a straightforward shader. Non-diffusepixel clustering (step 2) is also donewith a GPU shader, sincea pixel is assigned to a cluster using only the information atthe pixel, and no information from neighboring pixels.

Cluster camera construction (step 3a) requires concurrentwrites (e.g., for the computation of the cluster centroid, nor-mal, and AABB of sample projections) and we perform thestep on the GPU in CUDA using shared memory andatomic operations. The number of costly atomic operationsis reduced whenever possible by using regular operationsto determine a good initial guess. For example, whensearching for a maximum element in an array, running thealgorithm without atomic operations will return one of thelarger elements of the array. This element is then used to ini-tialize the maximum for the rigorous version of the algo-rithm that employs atomic operations. Since this initialvalue is only smaller than a few of the elements of the array,the maximum will only be updated a few times, saving

TABLE 7Number of Clusters for the Four Paths in Table 1

TABLE 8Performance for Glossy Reflections in Fig. 11


IEEE

Proo

f

most atomic operations that occur if the maximum is initial-ized with the customary first element.

Depth image rendering (step 3b), and finalizing the non-diffuse pixel colors which includes reflected ray/depthimage intersection (step 3c) are performed on the GPU. Forstep 3b view frustum culling at object level is used, and forstep 3c intersections for planar reflectors are immediatesince the projection of the reflected ray is a single point, i.e.,the intersection point (see cluster camera construction inSection 3.3). Handling planar reflectors correctly (no disoc-clusion errors) and efficiently (planar reflectors are detectedautomatically in the output frame and only the reflection forthe visible part of each planar reflector is computed) is animportant strength of our method since planar reflectors arefrequently encountered in man-made scenes (Table 9).

4.4 Limitations

As discussed above, our method resorts to several approxi-mations. First, the reflected rays are approximated by fittinga conventional planar pinhole camera to each cluster. Thesmaller the cluster, the better the pinhole’s ray approximatethe actual reflected rays. Since the image captured bythe pinhole is not used directly to form the reflection, theapproximation only has a second order effect on the correct-ness of the reflection. In other words, the reflection is notdistorted, and the samples captured by the pinhole arereflected correctly by computing the intersection with thereflected ray. However, the pinhole does not capture allsamples captured by the actual reflected rays, which leadsto disocclusion errors as discussed.

A second approximation is that the reflected geometry isreplaced with a depth image, which introduces an interme-diate resampling of both geometry and color. The effects ofthis approximation are mitigated by increasing the resolu-tion of the depth images. The third approximation consistsof looking up into an environment map the reflected raysgenerated by non-diffuse pixels of small clusters (e.g., sil-houette pixels in Fig. 9).

Our method is best suited for mirror-like reflections withone ray per pixel, because glossy rays cannot be approxi-mated well by the cluster cameras, and intersecting a glossyray with the cluster depth image is more expensive.

Our method requires one rendering pass for each clusterto render the cluster’s depth image. These passes cannot beavoided as one has to follow the reflected rays to capture thereflected samples. A cluster camera is constructed with thesmallest field of view that encompasses the reflected rays ofthe cluster. The depth images are non-redundant, except forinstances when the same part of the scene is reflected morethan once, and except for a small overlap at the borders thatensures reflection continuity between neighboring clusters.

Performance scalability with diffuse scene complexityhas to be sought along the lines of reducing the cost of these

passes. Partitioning scene geometry with a hierarchicalsubdivision scheme is of course an option, but that is notsuitable for dynamic scenes. Another possibility is toimprove the clustering scheme, which in its present formemphasizes efficiency at the cost of unnecessarily numerousclusters. K-means clustering based on k-d trees [42] wouldresult in fewer clusters and we will investigate whether thatbrings a performance gain sufficient to offset the cost of theslower clustering. We will also investigate grouping clusterswith cameras whose frusta are disjoint and rendering onecompound depth image for each group in a single pass.

Our method achieves second-order feed-forward render-ing, with first order rays being the rays leaving the outputimage eye. This means that our method supports only first-order reflections. When a reflected ray intersects a reflectivesurface, the ray color is simply set to the diffuse componentof the surface. Higher-order reflections also occur in the caseof concave reflectors, which can reflect a ray multipletimes until the ray escapes the reflector to sample the envi-ronment. We handle concave reflector clusters in one of twoways. One way is to handle concave clusters like the convexclusters—reflected rays are intersected with the clusterdepth image, ignoring the second intersection with thereflector surface sampled by the cluster. Another way is todetect that a cluster is concave, by testing whether the centerof a cluster is behind the image plane of the camera cluster,and then to intersect reflected rays with the cluster itself todetect a possible second intersection. When such a secondintersection occurs, the ray color is set to the diffuse compo-nent of the cluster sample. None of the two methods providethe accurate second order reflection, but the second methodprovides amore stable reflection, at the small additional cost.

Finally, our method brings the most benefit close to thespecular end of the specular-glossy-diffuse continuum. Anarrower reflection cone per reflective surface point resultsin more coherent reflected rays that are well approximatedby a cluster pinhole camera and requires fewer ray/depthimage intersections per pixel. When moving towards thediffuse end of the surface reflectance continuum, the raycoherence decreases and the cluster cameras become panor-amas with fields of view that encompass the sum of theupper hemispheres of the cluster pixels. Clustering basedon normals does not pay off anymore as the reflected rays ata point become indifferent to the point’s normal. We haveshown that our method achieves good results for specularreflections and that it can also handle high glossiness. Sup-porting low glossiness or diffuse reflections require a differ-ent strategy for approximating the reflected scene geometry.

5 CONCLUSIONS AND FUTURE WORK

We have presented a method for rendering specular andglossy reflections that achieves quality reflections at interac-tive rates. No pre-computation is required which supportsfully dynamic scenes at substantially higher frame ratesthan a ray tracer that has to reconstruct its acceleration datastructure for every frame. Our method readily works withnormal-mapped reflectors (Fig. 13).

Compared to methods that approximate the projectionof reflected vertices such as the explosion map [27], ourmethod has the advantage of better scalability with

TABLE 9Percentage of Non-Diffuse Pixels that Belong

to Planar Reflectors for the Four Paths in Table 1


IEEE

Proo

f

reflector complexity and of handling multiple reflections atno extra cost. Our method can handle complex reflectorsbecause it does not attempt to provide a constant time solu-tion to the problem of projecting reflected vertices. Instead,the reflected scene vertices are first mapped to a depthimage to which reflected rays are then mapped. Thereflected ray to depth image mapping implies a search,which is more expensive than the constant time explosionmap projection, but which makes complex reflectors tracta-ble. The search is confined to the 1D projection of the ray tothe depth image, which bounds the cost to the resolution ofthe depth image.

Compared to image based rendering and caching meth-ods, our method does well for specular reflections whichrequire a high image or cache resolution that makes themexpensive to construct and search. Our method compressesthe specular reflection data well leveraging the reflectedscene geometry. Compared to other methods based onapproximating the reflected scene, our method has theadvantage of approximating only the part of the sceneneeded for the reflections in the current frame. The approx-imation fidelity is tailored to the needs of the currentframe. Compared to environment mapping our method ismore accurate and compared to methods that rely on viewindependent approximations of reflected geometry ourmethod is more efficient, enabling applications involvingdynamic scenes.

Our method achieves interactive rates on complexreflections. Applications where frame rate is the maindesign consideration could reserve the use of our methodto a subset of the reflective surfaces in a scene. We foreseethat the advantage of our method over ray tracing willincrease as graphics hardware progresses, since we mapwell to the GPU’s strength of rendering by projection fol-lowed by rasterization.

In addition to the future work directions sketched inSection 4.4, the number of clusters could be reduced byreplacing the conventional planar pinhole camera used toapproximate the reflected rays of a cluster with more pow-erful, non-pinhole camera models that can conform to largersets of more diverse rays.

Our method demonstrates that today’s hardware imple-mentation of the feed-forward graphics pipeline is suffi-ciently versatile and prolific to compute not only the colorsamples captured by first order rays leaving the eye, butalso the samples captured by second order rays. In the con-text of our paper, the second order rays were created byreflective surfaces, but, in future work, our method could beextended to other types of rays.

ACKNOWLEDGMENTS

The authors would like to thank Zheng Yang for help withthe implementation and Shuo Wang for help with scenemodeling and animation. This work was supported in partby the National Natural Science Foundation of Chinathrough Projects 61272349, 61190121 and 61190125, by theMacao Science and Technology Development Fund throughProject 043/2009/A2, by the National High TechnologyResearch and Development Program of China through 863Program No. 2013AA01A604, and through Beijing ScienceTechnology Star Plan No. 2009B09.

REFERENCES

[1] T. Whitted, “An improved illumination model for shaded dis-play,” Commun. ACM, vol. 23, no. 6, pp. 343–349, 1980.

[2] A. S. Glassner, An Introduction to Ray Tracing. San Mateo, CA,USA: Morgan Kaufmann, 1989.

[3] T. Foley and J. Sugerman, “KD-tree acceleration structures for aGPU raytracer,” in Proc. ACM SIGGRAPH/EUROGRAPHICS Conf.Graph. Hardware, 2005, pp. 15–22.

[4] N. A. Carr, J. Hoberock, K. Crane, and J. C. Hart, “Fast GPU raytracing of dynamic meshes using geometry images,” in Proc.Graph. Interface, 2006, pp. 203–209.

[5] S. E. Yoon, C. Lauterbach, and D. Manocha, “R-LODs: Fast LOD-Based Ray Tracing of Massive Models,” Vis. Comput., vol. 22,no. 9–11, pp. 772-7–772-84, 2006.

[6] J. Kautz and M. D. McCool, “Approximation of glossy reflectionwith prefiltered environment maps,” in Proc. Graph. Interface,2000, pp. 119–126.

[7] P. Green, J. Kautz, and F. Durand, “Efficient reflectance and visi-bility approximations for environment map rendering,” Comput.Graph. Forum, vol. 26, no. 3, pp. 495–502, 2007.

[8] V. Popescu, E. Sacks, and C. Mei, “Sample-based cameras forfeed-forward reflection rendering,” IEEE Trans. Vis. Comput.Graph., vol. 12, no. 6, pp. 1590–1600, Nov./Dec. 2006.

[9] P. Rosen, V. Popescu, K. Hayward, and C. Wyman, “Nonpinholeapproximations for interactive rendering,” IEEE Comput. Graph.Appl., vol. 31, no. 6, pp. 68–83, Nov./Dec. 2011.

[10] V. Popescu, P. Rosen, L. Arns, X. Tricoche, C. Wyman, and C. M.Hoffmann, “The general pinhole camera: Effective and efficientnon-uniform sampling for visualization,” IEEE Trans. Vis. Comput.Graph., vol. 16, no. 5, pp. 777–790, Sep. 2010.

[11] I. Wald, P. Slusallek, C. Benthin, and M. Wagner, “Interactive ren-dering with coherent ray tracing,” Comput. Graph. Forum, vol. 20,no. 3, pp. 153–165, 2001.

[12] A. Reshetov, A. Soupikov, and J. Hurley, “Multi-level ray trac-ing algorithm,” ACM Trans. Graph., vol. 24, no. 3, pp. 1176–1185, 2005.

[13] T. J. Prucell, I. Buck, W. R. Mark, and P. Hanrahan, “Ray tracingon programmable graphics hardware,” ACM Trans. Graph.,vol. 21, no. 3, pp. 703–712, 2002.

[14] T. J. Purcell, “Ray tracing on a stream processor,” Ph.D. disserta-tion, Dept. Comput. Sci., Stanford Univ., Stanford, CA, USA, 2004.

[15] T. J. Purcell, I. Buck, W. R. Mark, and P. Hanrahan, “Ray tracingon programmable graphics hardware”, in Proc. ACM SIGGRAPHConf., 2002, pp. 703-712.

[16] S. Li, Z. Fan, X. Yin, K. Muller, A. E. Kaufman, and X. Gu, “Real-Time Reflection using Ray Tracing with Geometry Field,” in Proc.Eurographics, 2006, pp. 29–32.

Fig. 13. Reflections on a normal-mapped reflector rendered with ourmethod at 31 fps.


IEEE

Proo

f

[17] X. Yu, R. Wang, and J. Yu, “Interactive glossy reflections usingGPU-based ray tracing with adaptive LOD,” Comput. Graph.Forum, vol. 27, no. 7, pp. 1987–1996, 2008.

[18] J. Yu, J. Yang, and L. McMillan, “Real-Time Reflection Mappingwith Parallax,” in Proc. Symp. Interactive 3D Graph. Games, 2005,pp. 133–138.

[19] S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “TheLumigraph,” in Proc. 23rd Annu. Conf. Comput. Graph. InteractiveTech., 1996, pp. 43–54.

[20] M. Levoy and P. Hanrahan, “Light field rendering,” in Proc. 23rdAnnu. Conf. Comput. Graph. Interactive Tech., 1996, pp. 187–196.

[21] W. Heidrich, H. Lensch, M. F. Cohen, and H. Seidel, “Light fieldtechniques for reflections and refractions,” in Proc. EurographicsSymp. Rendering/Eurographics Workshop Rendering Tech., 1999,pp. 187–196.

[22] Y. Taguchi, A. K. Agrawal, S. Ramalingam, and A. Veeraragha-van, “Axial light field for curved mirrors: Reflect your perspec-tive, widen your view,” in Proc. Comput. Vision Pattern Recog.,2010, pp. 499–506.

[23] J. Kautz and M. D. Mccool, “Approximation of glossy reflectionwith prefiltered environment maps,” in Proc. Graph. Interface,2000, pp. 119–126.

[24] P. Green, J. Kautz, and F. Durand, “Efficient reflectance and visi-bility approximations for environment map rendering,” Comput.Graph. Forum, vol. 26, no. 3, pp. 495–502, 2007.

[25] L. Szirmay-kalos, B. Asz�odi, I. Laz�anyi, and M. Premecz,“Approximate ray-tracing on the GPU with distance impostors,”Comput. Graph. Forum, vol. 24, no. 3, pp. 695–704, 2005.

[26] V. Popescu, C. Mei, J. Dauble, and E. Sacks, “Reflected-sceneimpostors for realistic reflections at interactive rates,” Comput.Graph. Forum, vol. 25, no. 3, pp. 313–322, 2006.

[27] E. Ofek and A. Rappoport, “Interactive reflections on curvedobjects,” in Proc. 25th Annu. Conf. Comput. Graph. Interactive Tech.,1998, pp. 333–342.

[28] P. Estalella, I. Mart�ın, G. Drettakis, and D. Tost, “A GPU-drivenalgorithm for accurate interactive reflections on curved objects,”in Proc. 17th Eurographics Conf. Rendering Tech., 2006, pp. 313–318.

[29] P. E. Debevec, Y. Yu, and G. Borshukov, “Efficient view-depen-dent image-based rendering with projective texture-mapping,” inProc. Eurographics Symp. Rendering/Eurographics Workshop Render-ing Tech., 1998, pp. 105–116.

[30] NVIDIA OptiX ray tracing engine, [Online]. Available: https://developer.nvidia.com/optix,

[31] L. McMillan, “An image-based approach to three-dimensionalcomputer graphics,” Ph.D. dissertation, Dept. Comput. Sci., UNC-CH, , Apr. 1997.

[32] F. Policarpo and M. M Oliveira, “Relief mapping of non-height-field surface details,” in Proc. ACM SIGGRAPH Symp. Interactive3D Graph, GamesRedwood City, CA, USA, Mar. 2006, pp. 55–62.

[33] J. Shade, S. Gortler, L.-w. He, and R. Szeliski, “Layered depthimages,” in Proc. ACM SIGGRAPH 25th Annu. Conf. Comput.Graph. Interactive Tech., 1998, pp. 231–242.

[34] K. B€urger, S. Hertel, J. Kr€uger, and R. Westermann, “GPU render-ing of secondary effects,” in Proc. Vision, Model. Vis. Conf.,Saarbr€ucken, Germany, Nov. 2007, pp. 51–60.

[35] M. Nießner, H. Sch€afer, and M. Stamminger, “Fast indirect illumi-nation using layered depth images,” Visual Comput., vol. 26, nos.6–8, pp. 679–686, 2010.

[36] V. Popescu, P. Rosen, and N. Adamo-Villani, “The graph camera,”ACM Trans. Graph., vol. 28, no. 5, 2009.

[37] G. J. Ward, F. M. Rubinstein, and R. D. Clear, “A ray tracing solu-tion for diffuse interreflection,” in Proc. ACM SIGGRAPH 15thAnnu. Conf. Comput. Graph. Interactive Tech., 1988, pp. 85–92.

[38] G. Greger, P. Shirley, P. M Hubbard, and D. P. Greenberg, “Theirradiance volume,” IEEE Comput. Graph. Appl., vol. 27, no. 2,pp. 32–43, Mar. 1998.

[39] G. J. Ward, “Irradiance gradients,” in Proc. ACM SIGGRAPH Clas-ses, 2008.

[40] J. Kriva�nek and P. Gautron, “Radiance caching for efficient globalillumination,” IEEE Trans. Vis. Comput. Graph., vol. 11, no. 5,pp. 550–561, Sep./Oct. 2005.

[41] D. Scherzer, C. H. Nguyen, T. Ritschel, and H.-P. Seidel, “Pre-con-volved radiance caching,” Comput. Graph. Forum, vol. 31, no. 4,pp. 1391–1397, 2012.

[42] R. Wang, R. Wang, K. Zhou, M. Pan, and H. Bao, “An efficientGPU-based approach for interactive global illumination,” ACMTrans. Graph., vol. 28, no. 9, 2009.

[43] B. Liu, L.-Y. Wei, X. Yang, Y.-Q. Xu, and B. Guo, “Nonlinear beamtracing on a GPU,” Microsoft Res., Redmond, WA, USA, Tech.Rep. MSR-TR-2007-34, Microsoft, 2009.

Lili Wang received the PhD degree from theBeihang University, Beijing, China. She is anassociate professor with the School of ComputerScience and Engineering of Beihang University,and a researcher with the State Key Laboratoryof Virtual Reality Technology and Systems. Herinterests include real-time rendering, realisticrendering, global illumination, soft shadow, andtexture synthesis.

Naiwen Xie received the BE degree in computerscience from Huazhong University of Scienceand Technology in 2011. He is currently workingtoward the PhD degree in the State Key Labora-tory of Virtual Reality Technology and Systemsat Beihang University. His research interestsinclude global illumination, image-based render-ing, and image processing.

Wei Ke received the PhD degree from School ofComputer Science and Engineering, BeihangUniversity. He is a researcher of Macao Poly-technic Institute. His research interests includeprogramming languages, functional program-ming, formal methods, and tool support forobject-oriented and component-based engineer-ing and systems. His recent research focuses onthe design and implementation of open platformsfor virtual reality applications, including program-ming tools, environments, and frameworks.

Voicu Popescu received the BS degree in com-puter science from the Technical University ofCluj-Napoca, Romania in 1995, and the PhDdegree in computer science from the Universityof North Carolina at Chapel Hill, in 2001. He is anassociate professor with the Computer ScienceDepartment of Purdue University. His researchinterests lie in the areas of computer graphics,computer vision, and visualization. His currentprojects include camera model design, visibility,augmented reality for surgery telementoring, and

the use of computer graphics to advance education.

" For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.


Date post:	17-Mar-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

IEEE TRANSACTIONS ON VISUALIZATION AND ......ray database obsolete and re-computing the ray-database...

Documents