A Hierarchical Automatic Stopping Condition forMonte Carlo Global Illumination

Holger Dammertz Johannes Hanika Alexander Keller Hendrik P.A. LenschUlm University Ulm University mental images GmbH Ulm University

James-Franck-Ring James-Franck-Ring Fasanenstrasse 81 James-Franck-RingGermany, 89081 Ulm Germany, 89081 Ulm Germany, 10623 Berlin Germany, 89081 Ulm

Figure 1: An example sequence of blocks. Initially large blocks (shown in saturated colors) are recursively re-fined and terminated individually as the image converges. Note that complex areas in the image, where indirectillumination dominates the appearance (e.g. the ceiling), are detected and refined by the algorithm.

ABSTRACTWe introduce a hierarchical image-space method to robustly terminate computations in Monte Carlo image synthe-sis, independent of image resolution. The technique consists of a robust convergence measure on blocks which areeither recursively subdivided or terminated independently, using a criterion which separates signal and noise basedon integral estimates from two separate sample sets. The technique can be easily implemented, as the evaluationof the error measure only requires a second framebuffer and a list of non-terminated blocks. Based on the stoppingcriterion, one can furthermore sample both the image plane as well as the light sources adaptively. Adaptive sam-pling reduces the number of samples required to gain the same root mean square error to one quarter in some ofour test cases.

KeywordsMonte Carlo, Global Illumination

1 INTRODUCTION AND PREVIOUSWORK

Monte Carlo methods are among the most general tech-niques to solve the global illumination problem. Unbi-ased Monte Carlo methods are guaranteed to convergeto the correct solution but the number of samples re-quired for generating an image where the remainingnoise is hardly visible is not known in advance.

We therefore propose a stopping algorithm which ro-bustly separates the remaining noise in the rendered im-age from the variation due to the scene content, which iseasy to implement, requires very little additional mem-

Permission to make digital or hard copies of all or part ofthis work for personal or classroom use is granted withoutfee provided that copies are not made or distributed for profitor commercial advantage and that copies bear this notice andthe full citation on the first page. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requiresprior specific permission and/or a fee.

ory and we also show how to use it to adaptively samplefrom the light sources. Following the idea of Dippé andWold [DW85], we estimate the rate of change as thedifference between images generated with two differ-ent sample sets.

A lot of work has been done to quantify theperceived differences by the human visual sys-tem [Mys98, BM98, RPG99, YPG01, SCCD04, FP04,SGA∗07, Dal93, MDMS05], error measures based onconfidence intervals, contrast, variance, saliency andentropy have been investigated. For simplicity andfast evaluation, we base our stopping criterion on theremaining color noise in relation to the logarithmicluminance of the sample, but any other error measurecould be used as well.

Rigau et al. [RFS03] presented a similar methodbased on entropy, but they only consider the image as awhole and sub-pixel refinements, not blocks of pixels.

Hachisuka et al. [HJW∗08] also subdivide a hi-erarchy of samples very similar to the MISERalgorithm [PTVF92], but our error measure is based

on two integral estimates, which is able to separatesignal from noise. Also, additional reconstruction isnot necessary because sampling is dense enough (atleast one sample per pixel), and the samples do nothave to be stored explicitly in our case.

In [ODR09] the authors describe an adaptive waveletrendering that is similar to our approach but their scal-ing functions are more restrictive than our blocks andadditionally we also adaptively sample the light sourceand are not restricted to 2d.

For a reliable error estimate, it is not sufficient to lookat the variation inside a single pixel, due to the limitednumber of samples [Mit87]. To overcome this, we eval-uate the error measure on a hierarchy of blocks, whichare recursively refined only as the remaining error dropsbelow a certain resolution-independent threshold (seeFigure 1 for an illustration of the refinement process).This makes sure the algorithm adapts to the true sig-nal, not a noisy estimate. Assuming that the samplesavailable so far have not completely missed any majorlight contribution (i.e. one “firefly”-path is enough), ourscheme operates conservatively.

Painter and Sloan [PS89] also used a hierarchy, animage-space kd-tree, but explicitly stored all samplesand all levels of the hierarchy. We, on the other hand,simply store the integral estimates of the two samplingsets per pixel in two framebuffers. Furthermore, weonly keep track of the leaf blocks, which significantlysimplifies bookkeeping.

Based on our convergence estimate, adaptive sam-pling can be performed by simply supersampling onlythose blocks which are not yet terminated.

Note that this is different to most existing adaptivesampling approaches, since it is not based on placingsamples in regions considered interesting at an earlystage but splits and terminates rather conservatively.This is why the common problem of missed featuressuch as small objects is not a problem.

Furthermore, as the technique is framebuffer-based,it is completely independent of the underlying render-ing algorithm.

2 A HIERARCHICAL CRITERIONTo make sure the algorithm does not stop computationsbefore all important features have been detected, a lotof samples should be drawn before making a decision(see Figure 2 for an illustration, and Figure 3 for a com-parison to a pure per-pixel approach). Since it is notdesirable to shoot a lot of possibly unnecessary rays perpixel, we initially base the evaluation of the error on theimage as a whole.

As the variance in the image will be distributed un-evenly, depending on the problem and the type of pathspace sampler used, the stopping decision should bedone locally. This is reflected by splitting the blockswhen the algorithm has enough confidence not to have

Figure 2: A caustic rendered with way too low samplingdensity. It is not sufficient to make decisions based ononly a few samples, i.e. on small blocks or even pixel-wise. The magnified block for example appears to beconverged already, but it still misses important features(evident through adjacent bright pixels) and thus needsmore samples.

Figure 3: This shows the problem of using a per pixeltermination criterion. Due to the nature of MonteCarlo sampling some pixels terminate too early whichis clearly visible in the left image. The right imageshows our proposed termination criterion with equalsample count.

missed any important features, i.e. the per-block errormeasure drops below a certain threshold εs.

A block only consists of an axis-aligned boundingbox covering an area of the image. The blocks are man-aged in a linear list. That is, if a block is split, it issimply removed and the two resulting child blocks, dis-jointly covering the same area of the image, are addedto the list again. This way, no explicit hierarchy has tobe maintained.

The algorithm continues by drawing new samples(one for each pixel in each remaining block in ourimplementation). If stratification in image space canbe guaranteed, e.g. by using a simple backward pathtracer, only non-terminated blocks have to be sampledfurther. If certain sampling methods cannot be re-stricted in this way, such as for example light tracingmethods, their contribution can still be used, as the stop-ping condition is not directly coupled to adaptive sam-pling. In fact, it is also possible to sample adaptivelyfrom the lights (see Section 3.2).

draw samples

evaluateerror metric

for all blocks:

split blockterminateblock

error very smallerror smallerror large

Figure 4: Illustration of a rendering algorithm using thestopping algorithm.

The image is converged when the error of all blocksis smaller than a threshold εt < εs. For an overview ofthe algorithm see Figure 4. An example sequence ofblocks is depicted in Figure 1. Note how the algorithmautomatically adapts the block size to the image and isthus independent on the resolution.

2.1 Error Metric

We use an error metric based on the pixels in the finalimage, but to get a robust criterion we always evaluatea block of pixels. We compare two independently com-puted images with the same number of samples similarto [DW85, DS04].

In the implementation, the evaluation of the errormetric can be simplified by introducing just a singleadditional accumulation buffer and keeping the normalaccumulation buffer used to compute the final image.In the second buffer we accumulate only samples everysecond rendering pass. This is based on the simple ob-servation that computing a single image I with an evensample count can be split into two images A and B withI = A/2 + B/2⇒ B/2−A/2 = I−A. Thus, using anRGB buffer we estimate the per pixel error as

ep =|Ir

p−Arp|+ |I

gp−Ag

p|+ |Ibp−Ab

p|√Irp + Ig

p + Ibp

.

The square root in the denominator is motivated bythe logarithmic response of the human visual system toluminance. The term here behaves similarly, is easier toevaluate and was found to yield slightly better results.

The error per block is computed by summing over allpixels:

eb =rN ∑

pep

where N is the number of pixels contained in this blockand r is a scaling factor computed as

√Ab/Ai. Ai is

the area of the image, Ab the area of the block underconsideration.

2.2 Block Splitting and Block Termina-tion

The user specifies a single error value v that is usedin all further decisions. From this user-specified valuewe compute two error thresholds εs (splitting) and εt(termination). For simplicity we use εt = v but it is ofcourse possible to rescale v to a more intuitive parame-ter range. In all our tests we used v = 0.0002. Given εt ,we compute εs = 256 · εt . This is an empirical choicethat worked well in all our tested scenes.

The splitting is performed axis-aligned by choosingthe axis where the block has the largest extent. Thesplit position is chosen such that the error measure is asequal as possible on both sides.

3 RESULTSIn this section we analyse our proposed stopping crite-rion for different scenes and additionally examine howwell the proposed error metric can be used as an adap-tive sampling criterion. In the first subsection we use apath tracer with next event estimation. In Section 3.2we propose a simple method to extend this adaptivesampling to a light tracing algorithm.

3.1 Image SpaceFigure 5 on the left shows a simple test scene of a dif-fuse sphere illuminated by an area light source castinga large smooth shadow. The right image shows the dis-tribution of the number of samples in the final image asa heat map. White is the maximum number of samplesand black the minimum. As each block is sampled uni-formly the block structure is still apparent in this image.The graph in Figure 6 shows the number of samples re-quired by the normal rendering algorithm and by ourtechnique to achieve an RMS error below 0.01. In thissimple scene the sample count was reduced from 700Mto 400M (57%). This indicates that our error metric ismeaningful also with respect to the RMS error. Theevaluation of the error metric costs about 10% of therendering time in our implementation.

As another example we show a living room scene inFigure 7. Again on the left the final image can be seenand on the right the sample count distribution. This isa more realistic scene as it is closed and strong indirectillumination is present. The complexity is distributedalmost equally over the whole image. This explains

Figure 5: A simple sphere scene rendered with theproposed stopping condition (left) and the respectiveheat map (right) representing the number of samplesrequired for a converged block. Most effort has to bespent in the penumbra regions.

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

0 1e+08 2e+08 3e+08 4e+08 5e+08 6e+08 7e+08

RM

S E

rror

Samples

normaladaptive

Figure 6: This graph shows the RMS error of the imageshown in Figure 5 with increasing number of samples,for normal and adaptive sampling.

why in this scene sampling adaptively does not presenta huge benefit. This is also visible in the RMS errorgraph shown in Figure 8. Still our termination criterionworks robustly and also isolates the two small areas inthe image where the illumination is more complex dueto glossy objects and a small caustic cast by the monkeyhead on the table (see Figure 9).

3.2 Light SpaceIn this section we show how to extend our criterion toalso be able to adaptively sample in a light tracing set-ting [Vea97]. A light tracer starts paths only from thelight sources and connects hit points to the camera. Thisis very well suited to compute caustics but it is not pos-sible to directly refine samples on the image plane as itis unknown where exactly a light sample will connect.For our termination criterion this is not a problem as itworks equally well independent of the used renderingalgorithm. Adaptive sampling of the image plane is notstraightforward for light tracing, so all samples need tobe distributed equally.

To facilitate adaptive sampling also from the lightsources, we back-project the image error metric to animportance map around each light source. Such animportance map consists in a quantized hemispherearound each emitting triangle, thus representing onlyoutgoing directions, ignoring the starting point on thetriangle and higher-dimensional bounces. This workswell under the assumption that triangles are small with

respect to the illuminated area. This can always beachieved by subdividing the emitting triangles whichdoes not change the final image. We chose this ap-proach over explicitly storing the starting point as wellto reduce the dimensionality and thus the size of themap. This also simplifies the integration into an exist-ing rendering system. The algorithm proceeds by calcu-lating an initial image calculating a few samples. Afterthat each new light sample accumulates the remainingimage error back into its importance map bin at the lightsource side. This way, the image error metric is back-projected with a delay of one iteration. The importancemap can then be used in the next iteration to importancesample light directions (similar to [CAM08]).

As test scene we chose a distorted glass object cast-ing a large caustic (see Figure 10). The top row of Fig-ure 11 shows three importance maps at three differentiterations. The bottom row shows the associated imagespace error blocks. While the top row is heat-map col-ored by the importance, the colors in the bottom roware solely to distinguish the different blocks. It can beclearly seen how large areas of the hemisphere do notcontribute at all to the final image and also how the ter-mination of regions in image space is reflected in thehemispherical importance map. Figure 12 shows theRMS error graph using no adaptivity at all and usingthe described back-projected importance map. In thiscase the number of samples required to get an RMSerror below 0.01 is reduced from 570M light paths to160M light paths (28%).

4 CONCLUSIONWe introduced a stopping condition for Monte Carloimage synthesis which automatically adjusts itself toimage resolution, robustly adapts to local variance, andis very simple to implement on top of any rendering sys-tem. Due to this simplicity it can be easily integrated inGPU based ray tracing systems. The criterion can alsobe used to facilitate adaptive sampling. Additionally,we have shown how back-projection of the image er-ror metric onto hemispheres around the light sourcescan be used to profit from adaptive sampling also whenstarting paths at the light sources.

In the future, it should be possible to extend the sys-tem to complete bi-directional light transport. Thatis, combine adaptive image space sampling and start-ing the light paths in important directions based on theback-projected image space error at the time and alsoconsider all deterministic connections. It would also beinteresting to extend the method to higher dimension-ality, similar to Hachisuka et al. [HJW∗08], i.e. to re-move the dependency on the framebuffer. Furthermore,a mathematical analysis of the variance in path spaceshould be done to gain some knowledge about the worstcase integration error at a given sampling density. Thisway, the initial sampling rate required to assure no fea-

Figure 7: Final image (left) and sample density heat map (right) for the living room scene. The algorithm robustlyfinds the two spots with significantly higher variance than the rest of the image.

0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1

0.11 0.12

0 1e+08 2e+08 3e+08 4e+08 5e+08 6e+08 7e+08

RM

S E

rror

Samples

normaladaptive

Figure 8: This graph shows the RMS errors of normaland adaptive sampling of the image shown in Figure 7with increasing number of samples.

Figure 9: Four images taken from the sequence of ac-tive blocks when rendering the living room scene seenin Figure 7. The difficult spots are isolated quickly.

tures are missed when first evaluating the error measurecould be determined.

The adaptivity of the method could also be improvedupon, if one does not only wish to use it as a termi-nation condition, but also increase efficiency. In anal-ogy to building fast spatial acceleration hierarchies forray tracing, the image space block splits could be op-timized to cut off empty space, i.e. always separateblocks which are likely to terminate in the next itera-tion.

Figure 10: The light tracing test scene. The glass objectis black because a pure light tracer cannot deterministi-cally connect singular materials to the camera.

5 ACKNOWLEDGMENTSThis work has been partially funded by mental imagesGmbH and the DFG Emmy Noether fellowship (Le1341/1-1).

Figure 11: Hemispherical importance map at the lightsource (top row) and respective image error blocks forthe caustic scene (Figure 10).

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0 1e+08 2e+08 3e+08 4e+08 5e+08 6e+08

RM

S E

rror

Samples

normalbackprojected adaptive

Figure 12: RMS error graph for the light tracing exper-iment comparing no adaptivity and our proposed back-projection. The curves terminate when the RMS errordrops below 0.01.

REFERENCES[BM98] BOLIN M., MEYER G.: A perceptually based

adaptive sampling algorithm. ACM Transactionson Graphics (Proc. SIGGRAPH 1998) (1998),299–309.

[CAM08] CLARBERG P., AKENINE-MÖLLER T.: Practi-cal product importance sampling for direct illu-mination. In Computer Graphics Forum (Proc.of Eurographics 2008) (2008), pp. 681–690.

[Dal93] DALY S.: The visible differences predictor: analgorithm for the assessment of image fidelity.179–206.

[DS04] DMITRIEV K., SEIDEL H.-P.: Progressivepath tracing with lightweight local error esti-mation. In Vision, modeling, and visualization2004 (VMV-04) (Stanford, USA, 2004), GirodB., Magnor M., Seidel H.-P., (Eds.), Akademis-che Verlagsgesellschaft Aka, pp. 249–254.

[DW85] DIPPÉ M., WOLD E.: Antialiasing throughstochastic sampling. Computer Graphics (Proc.SIGGRAPH ’85) (1985), 69–78.

[FP04] FARRUGIA J., PÉROCHE B.: A progressive ren-dering algorithm using an adaptive perceptuallybased image metric. Computer Graphics Forum23 (2004), 605–614.

[HJW∗08] HACHISUKA T., JAROSZ W., WEISTROFFER

P., DALE K., HUMPHREYS G., ZWICKER M.,JENSEN H. W.: Multidimensional adaptive

sampling and reconstruction for ray tracing.ACM Transactions on Graphics (Proceedings ofSIGGRAPH ’08) (2008).

[MDMS05] MANTIUK R., DALY S., MYSZKOWSKI K.,SEIDEL H.-P.: Predicting visible differencesin high dynamic range images - model and itscalibration. In Human Vision and ElectronicImaging X, IS&T/SPIEs 17th Annual Sympo-sium on Electronic Imaging (2005), RogowitzB. E., Pappas T. N., Daly S. J., (Eds.), vol. 5666,pp. 204–214.

[Mit87] MITCHELL D.: Generating antialiased imagesat low sampling densities. Computer Graphics(Proc. SIGGRAPH ’87) (1987), 65–72.

[Mys98] MYSZKOWSKI K.: The visible differences pre-dictor: applications to global illumination prob-lems. In Rendering Techniques ’98 (Proc. ofthe Sixth Eurographics Workshop on Rendering)(1998), pp. 233–236.

[ODR09] OVERBECK R. S., DONNER C., RAMAMOOR-THI R.: Adaptive Wavelet Rendering. ACMTransactions on Graphics (SIGGRAPH Asia 09)28, 5 (2009).

[PS89] PAINTER J., SLOAN K.: Antialiased ray tracingby adaptive progressive refinement. ComputerGraphics (Proc. SIGGRAPH ’89) (1989), 281–288.

[PTVF92] PRESS W. H., TEUKOLSKY S. A., VETTER-LING W. T., FLANNERY B. P.: Numericalrecipes in C (2nd ed.): the art of scientific com-puting. Cambridge University Press, New York,NY, USA, 1992.

[RFS03] RIGAU J., FEIXAS M., SBERT M.: Refine-ment criteria for global illumination using con-vex functions. In Compositional Data AnalysisWorkshop (2003).

[RPG99] RAMASUBRAMANIAN M., PATTANAIK S.,GREENBERG D.: A perceptually based physicalerror metric for realistic image synthesis. ACMTransactions on Graphics (Proc. SIGGRAPH’99) (1999), 73–82.

[SCCD04] SUNDSTEDT V., CHALMERS A., CATER K.,DEBATTISTA K.: Top-down visual attentionfor efficient rendering of task related scenes. InIn Vision, Modeling and Visualization (2004),pp. 209–216.

[SGA∗07] SUNDSTEDT V., GUTIERREZ D., ANSON O.,BANTERLE F., CHALMERS A.: Perceptual ren-dering of participating media. ACM Trans. Appl.Percept. 4, 3 (2007), 15.

[Vea97] VEACH E.: Robust Monte Carlo Methods forLight Transport Simulation. PhD thesis, Stan-ford University, 1997.

[YPG01] YEE H., PATTANAIK S., GREENBERG D.: Spa-tiotemporal sensitivity and visual attention forefficient rendering of dynamic environments.In In ACM Transactions on Graphics (2001),pp. 39–65.