Interleaved Sampling Alexander Keller , Wolfgang Heidrich University of Kaiserslautern, [email protected]The University of British Columbia, [email protected]Abstract. The known sampling methods can roughly be grouped into regular and irregular sampling. While regular sampling can be realized efficiently in graphics hardware, it is prone to inter-pixel aliasing. On the other hand these artifacts can easily be masked by noise using irregular sampling which, however, is more expensive to evaluate as it lacks the high coherence of a regular approach. We bridge this gap by introducing a generalized sampling scheme that smoothly blends between regular and irregular sampling. By interleaving the samples of regular grids in an irregular way, we preserve the high coherence and efficiently reduce inter-pixel aliasing thus significantly improving the rendering quality as compared to previous approaches. 1 Introduction Sampling of functions is one of the most fundamental tasks in computer graphics. It has to be performed for applications as diverse as anti-aliased scan-conversion of scenes, illumination from area light sources, cameras with depth-of-field effects, motion blur, and volume rendering. The existing sampling schemes for these applications can be grouped into regular and irregular methods. On the one hand, sampling on regular grids is efficient and simple, and also well suited for hardware rasterization. Unfortunately, the signals that need to be sampled in computer graphics typically are not band limited and often can contain arbitrarily large frequencies. Therefore some amount of aliasing is unavoidable, and sampling by regular grids emphasizes these artifacts. On the other hand, Monte Carlo methods use randomized sample positions instead thus masking the aliasing artifacts by noise that is more visually pleasing. However, per pixel irregular sampling does not exhibit the kind of coherence known from regular sampling. Consequently, these methods are more expensive and not well suited for hardware implementations. The new sampling scheme of interleaved sampling allows to smoothly blend be- tween regular and irregular sampling thus yielding the best and most efficient compro- mise. To this end we interleave samples taken from a number of independent regular grids, and merge them into a single sampling pattern. The relative positions of the reg- ular grids are determined by irregular offsets. Each regular grid can be interpreted as a low resolution image of the scene that can be rendered using the efficient rasteriza- tion algorithms. The samples of all regular grids are interleaved such that in the final high-quality, high-resolution image adjacent pixels are not correlated. Only minimal changes to existing hardware algorithms such as the Accumulation Buffer or multisam- pling allow to exploit the benefits of the new sampling scheme.
Abstract. The known samplingmethodscanroughly be groupedinto regularand irregular sampling. While regular samplingcan be realizedefficiently ingraphicshardware, it is proneto inter-pixel aliasing. On the other handtheseartifactscaneasilybemaskedby noiseusingirregularsamplingwhich,however,is moreexpensiveto evaluateasit lacksthehighcoherenceof aregularapproach.We bridgethis gapby introducinga generalizedsamplingschemethatsmoothlyblendsbetweenregular and irregular sampling. By interleaving the samplesofregulargrids in anirregularway, we preserve thehigh coherenceandefficientlyreduceinter-pixel aliasingthussignificantly improving the renderingquality ascomparedto previousapproaches.
1 Intr oduction
Samplingof functionsis oneof themostfundamentaltasksin computergraphics. It hasto be performedfor applications asdiverseasanti-aliasedscan-conversionof scenes,illumination from arealight sources,cameraswith depth-of-field effects,motionblur,andvolumerendering. The existing samplingschemesfor theseapplications canbegroupedinto regularandirregularmethods.
On the onehand,samplingon regulargrids is efficient andsimple,andalsowellsuitedfor hardwarerasterization. Unfortunately, the signalsthat needto be sampledin computer graphics typically arenot bandlimited andoften cancontainarbitrarilylarge frequencies.Thereforesomeamount of aliasingis unavoidable,andsamplingbyregular gridsemphasizestheseartifacts.
On theotherhand, MonteCarlomethodsuserandomizedsamplepositionsinsteadthusmaskingthe aliasingartifactsby noisethat is morevisually pleasing. However,perpixel irregularsamplingdoesnotexhibit thekind of coherenceknown from regularsampling. Consequently, thesemethods aremoreexpensive andnot well suitedforhardwareimplementations.
The new sampling schemeof interleavedsamplingallows to smoothlyblendbe-tweenregular andirregular samplingthusyielding thebestandmostefficient compro-mise. To this endwe interleave samplestaken from a numberof independentregulargrids,andmerge theminto a singlesamplingpattern.Therelativepositions of thereg-ular grids aredeterminedby irregular offsets. Eachregulargrid canbe interpreted asa low resolutionimageof the scenethat canbe rendered usingthe efficient rasteriza-tion algorithms. Thesamplesof all regular gridsareinterleavedsuchthat in thefinalhigh-quality, high-resolutionimageadjacentpixels arenot correlated. Only minimalchangesto existinghardwarealgorithmssuchastheAccumulationBuffer or multisam-pling allow to exploit thebenefitsof thenew samplingscheme.
Fig. 1. The principleof interleaved samplingillustratedby example: In (a) the classicalAccu-mulationBuffer is illustrated.Thesamesamplingpatternis repeatedfor all pixels. TheFouriertransformof this repeatedpatternclearly shows a high potentialfor inter-pixel aliasing. In (b)a�����
The basicideaof interleaved samplingis intriguingly simple and instantlybecomesobviousfrom Figure1: in theAccumulation Buffer method [HA90] theirregular sam-pling patternis affixed to onepixel and in consequenceperiodically repeatedfor allpixelssuchbearinga high potentialfor inter-pixel aliasing(seeFigure 1a). For inter-leavedsamplinganirregularoffsettingpatternis chosenthatcovers multiplepixels. Byperiodically repeatingthispatternregulargridsstill persistasemphasizedin Figure1b,howeveradjacentpixelsaresampledby different patterns.Notethatalthough thenum-berof regular gridsinvolvedis increasingtheinter-pixel aliasingpotentialis effectivelydecreasedandthetotalnumberof samplesremains thesame.
2.1 Blending betweenRegular and Irr egular Sampling
In a first stepanarbitrary irregular samplingpatternis selected.By periodically tilingtheimageplane,eachsampleof theirregular basispatterngeneratesa regular grid. Ina secondstepthepixel resolutionis fixed. Visually spokenthis means to superimposethepixel graphpaperontotheperiodic samplingpattern.Eachregulargrid thenis ren-
deredusingafastrasterizationalgorithm, andits samplesareaccumulatedin thepixelsdeterminedby thesuperpositionof thegraphpaper. Finally eachpixel is averaged by aboxfilter over thenumberof samplesthatfell inside.
By interleaving theregulargridsby anirregular basispattern, ahighlevel of coher-enceis preserved andcanstill be exploited by rastergraphics hardware,yet the highrisk of localaliasing,i.e. inter-pixel aliasing,is reduceddrasticallysinceadjacent pixelsusedifferent samplingpatterns.
Note that this general interleaving schemedoesnot require an integer ratio of thenumberof pixelscoveredby the irregular basispatternasin theexample of Figure1.Depending ontheirregular basispatternandthegraphpaperresolution it mayalsohap-pen,thatthepixelshavedifferent oversampling rates.Theratioof thesupport of apixelandthesupport of theirregular basispatterndetermines theamount of correlation.Wecanblendbetweentwo limit cases(seealsoFigure4): In thefirst thebasispatterncon-sistsof only onesample,which resultsin correlatedsampling on oneregular grid. Forthesecondcasethesizeof thebasispatternis chosenaslarge astheimagegraphpaperwhich correspondsto thecompletely uncorrelatedcasewhereno regular grid structureis present.
2.2 Theoretical Considerations
Themethod of dependenttests[Sob62, FC62] wasintroducedasageneralization of theMonteCarlomethodof integration. Thebasicideawasto usethesamesetof samplesfor theestimationof multiple integralsinsteadof usinga new realizationof thesetofsamplesfor eachintegral.
TheAccumulation Buffer is thebestexample for the realizationof the method ofdependenttests:thesamesamplingpatternis usedfor all pixel integrals.However dueto the repetition of the samplingpatternfor eachpixel, adjacent pixels canbe highlycorrelatedcausinginter-pixel aliasing(seetheperiodogramin Figure1a).Thiseffect isnot observed in theuncorrelatedapproachastakenby e.g.a softwareray tracerthat isapplyingadifferent samplingpatterntoeachpixel. Thespecificsamplingpatternfor themethod of dependenttestscanbechosenfrom a wide varietyof differentmethods (fora survey see[Gla95]). A summaryof theapproachesthathave beenusedfor graphicshardwarecanbefound in a reportby GossandWu [GW99]. We usesampleswith bluenoisecharacteristicsthat perfectly tile periodically andaregeneratedby a relaxationschemesimilar to theoneintroducedin [MF92].
By interleaving themethod of dependenttestsadjacentpixelsaresampledin anun-correlatedway. Theperiodicity by thereplication of theirregular basispatternis muchreducedascomparedto theperpixel approach.Thusthe inter-pixel aliasingpotentialis attenuated, andlower samplingratesascomparedto the Accumulation Buffer canbeused.This partially wasrecognized in [DW85], however without theawarenessofthe methodof dependenttestsandconsequently without exploiting the coherencein-ducedby thegrid structure.Interleavedsamplingis a generalization of themethodofdependenttestconsidering spectralissues.
3 Applications
Interleavedsamplingasintroducedabove canbeappliedto a number of differentsam-pling tasks,in particularthosefor which the Accumulation Buffer [HA90] hasbeenusedbefore. This includes soft shadows basedon themethod by BrotmanandBadler[BB84], limiteddepth-of-field [HA90, KHM95, HSS97], andInstantRadiosity[Kel97].
Fig. 2. On the left we seethe standardAccumulationBuffer approachfor anti-aliasingat 4samplesperpixel asdepictedin Figure1a.Usingtheinterleavedapproach(seeFigure1b)on theright, thealiasingartifactsarespatiallyseparatedyielding a morevisually pleasingappearance.
In additionto theseapplications,wewill alsoconsiderinterleaved samplingfor texturebasedvolume renderingalongthelinesof Cabraletal. [CCF94].
3.1 Anti-Aliasing by Supersampling
The first application we will look at is supersampleanti-aliasing along the lines ofmultisampling [Ake93] andtheAccumulation Buffer [HA90]. Existingmultisamplingimplementationscanbe interpretedasa specialcasefor interleaved sampling. If weconsider only the � sampleof every pixel in animage,weseethatall thesesamplesarelocatedonaregulargrid with aspacingthatis identicalto thespacingof thepixels (seetheemphasizedsamplein Figure1a).Therefore,efficient incremental scan-conversionalgorithmscanbeappliedon this grid for rasterizinggeometric primitives.
With interleavedsamplingwestretchoutthegridspacingof thesamplessothattheyarenolongeridenticalto thepixel spacing.Therefore,everysampleonaparticular gridcorrespondsto a different sub-pixel positionin oneof the pixels (seethe emphasizedsamplein Figure1b). Stretchingout thegrid spacingalsostretchesout aliasingstruc-turesthusverymuchimproving thevisualqualityascanbeseenin Figure2.
� Modification of Multisample Hardware. The mostgenerally applicable waywould be to adddirect hardwaresupport for interleaved samplingin a similarfashionmultisampling is supported today. The hardware for scanconversiondoesnot needto be modified for this. The only differenceto multisamplingisthat not all the pixels will have the samesub-pixel masks,but the maskis oneof a predeterminedandimplementation-specificnumber of masks. With thesemodifications the geometry only needs to be processedonce,while individualscanconverterstakecareof rasterizingtheregular grids.Therelativepositionsof theregular grids,whichalsodeterminethemasksfor theindividual pixels,wouldbeanimplementation-specific arrangementthatcouldbewired into thehardware.Thesepositions wouldbegeneratedin thedesignphaseof the hardware. Sinceinterleaved samplingis composedof regular grids, thehardwareimplementation will be lessexpensive thanthe 3DlabsWildcat mul-tisampleextension[3Dl], which randomizesthe sampleselectionon a higherresolutiongrid.� Implementation on Curr ent Hardware. Anotherwayto implement interleaved
samplingis to individually renderlower resolutionimagescorresponding to theregulargrids,andto theninterleave thesamplesobtainedthis way by hand. Tothis end, the individual regular grids (low resolutionimages)are rendereddi-rectly into a texture map(anoperation that is supportedon many contemporaryplatforms). With theseregular gridsastextures,we canthenrender full-screenpolygonswhile asecondtextureor ascreen-spacestipplepatternis usedto maskout thepixelsthatshould notbeaffected by theregulargrid.If the whole application is boundedby fill rate, thenrendering multiple copiesof thegeometrywill not degeneratetherenderingperformance. In this case,themethodintroducesonly aconstantoverheadfor interleaving thesamplesafterthelow-resolution regular grids have beenrendered. The costof this processdoesnotdependon theshading complexity of everysamplein theimage.Thus, if theapplicationis limited by therasterizationperformance,this algorithm hasonly aconstantoverheadover supersampling with the samesamplingpatternfor eachpixel. It is therefore a feasiblealgorithmthat is interestingfor applications thathave high shadingcostsfor eachindividual sample,suchascomplex proceduralshaders[POAU00], or volumerendering(alsoseeSection3.3).
3.2 Motion Blur
Thebasicconcept of interleavedsamplingcaneasilybeextended to arbitrary dimen-sions. For example, by simply assigningeachof the interleaved imagesonemomentin time,theabovealgorithmcansupport motionblur: thentheirregular basissamplingpatternis 3-dimensional, wherethefirst two dimensionsareusedfor interleaving andthethird dimensionspecifiesthemoment in time. Interleavedsamplingsoenablesef-ficientandcorrectmotionblur simulationfor e.g.theREYESarchitecture [CCC87]orthe photon map[JC98] usinga very small setof different time samples(i.e. just thesamplesin the basispattern), whereeachimageto be interleaved is rendered by theoriginal algorithm, however for thespecifiedmoment in time.
In Figure4 theresultsof amotionblur simulationat16samplesperpixel areshown.Usinguncorrelatedrandomsamplesno aliasingartifactsarevisible, whereasthestan-dardcorrelatedAccumulationBuffer techniqueexhibitsvisibleartifacts.Usinga ��� interleaved irregularbasispatternof 64sampleswith bluenoisecharacteristics(yielding16 samplesperpixel asin theothercases),we blendbetweenuncorrelatedandcorre-latedsamplingandareableto exploit rasterizationon regular grids. Despitethe lowsamplingratethealiasingartifactsareclearlyreduced at thesameamount of samplingwork.
3.3 VolumeRendering
The next application we consideris texture-basedvolumerendering asdescribed byCabral[CCF94]. This algorithm usesa simpleemission-absorptionmodel for thevol-ume,so that the rendering is reduced to a 1D integration along the viewing ray forevery pixel in the final image. The texture-basedapproachfor computing this 1D in-tegral worksasfollows: thevolumeis slicedwith equidistantplanes.Thesearesortedback-to-front andblendedtogether in theframebuffer. As aresultof this algorithm, weobtaina samplingpatternthatcorrespondsto a regular 3D grid in clip coordinates(i.e.aftertheprojective transform).
Typically, on theorder of � � � slicesareusedfor this kind of application, andeachslice yields a polygon that covers a large portion of the screenandis textured with a
Fig. 3. For volumerendering,we combinea collectionof small resolutionimageswith slightlyoffsetsampleplanesinto a largeresolutionimageby interleaving thesamplesfrom the smallerimages.Exceptfor a final combiningoperation,thefill raterequiredis thesameasfor renderinga largeimagein thefirst place,but therenderingquality is muchbetter.
3D texture with tri-linear interpolation. Thus,the fill-rate required by this method isquitehigh, while thecostof geometric transformationsis negligible. But evenwith � � �slices,thereareoftenstill someseriousaliasingartifactsleft. After all, modernmedicalimaging devicescangeneratedatasetsin the order of � � �� , andtherefore � � � sliceswould resultin anundersamplingof a factorof � . On theotherhandit is not possibleto increasethe numberof slicesarbitrarily, bothbecauseof the performance implica-tions andbecauseof the limited depth of the framebuffer that resultsin quantizationartifactswhenusingalphablending. This could be avoidedby hierarchical summa-tion,whichhowever requiresadditional framebuffersfor theintermediateresultsandinconsequencecannot berealizedefficiently.
Interleaved samplingcanbeusedto improveonthissituation.Sincewedonothavethe fill rate to do supersampling at interactive rates,we areonly going to interleavethe samplesin onedimension; the � -coordinatein cameraspace.Interleaving in thatdimension correspondsto varying the offsetsfor the samplesin � direction. Theseoffsetswill bedifferentfor adjacent pixels,but sinceweareusinginterleavedsampling,they will repeatafter a number of pixels, thus providing us with the coherenceweneedfor exploiting graphics hardware. The methodcanbe considered asa hardwareimplementationof theequidistantray marcher from [PKK00] with interleavedjitteredoffsets.
Using existing hardware, the method then proceeds similarly to the method de-scribedin Section3.1. First, we usethe traditional texture-basedvolumerenderingalgorithm [CCF94]to render lowerresolutionregular grids(i.e. images), eachwith thesamplingplanesslightly offset comparedto the otherregular grids. The costof thisoperation is identicalto the original texture-basedvolume rendering algorithm, sincevolume rendering is completely determinedby therasterizationcost.
As describedin Section3.1,we thencombine thelow-resolution regular gridsintoa high-resolutionimageusingstipplepatterns or 2D textures to maskout the pixelsthat are not affected by the individual regular grid. Figure3 illustratesthis methodfor the configuration with 4 regular grids. The cost of this secondstepis constant,that is, it doesnot depend on the number of samplingplanes. During our testswith
anSGI OctaneVPro, we found this constantcostto beabout thesameasrendering8additional planeswith theoriginal method. Thequality improvements we canachievewith interleaved samplingaremuchhigherthanthe improvements we get by addingthis numberof additional samples.
Figure5 shows a comparison of the algorithm with andwithout interleavedsam-pling for � , � � , and � � sampling planes. Sincewe arenot supersamplingin thisapplication, interleaved samplingreplacesthealiasingeffectswith noisethatresemblesthepatternsknown from ordered dithering. Note,however, thatwe arejittering samplelocations insteadof ditheringcolorvalues.
On an OctaneVPro, the frame rate for the 20 planeversionis ��� � framespersecondboth with andwithout interleaving. For the � � planeversionwe get around18 framesper secondin the interleaved case,and � framesper secondin the non-interleaved case.Note that even for the � � planeversionthereis still somealiasingleft, andthe � � planeversionwith interleaved samplinglooksbetterthanthe � � planeversion without. Also notethatrenderingswith � � sampleplanesalready exhibit quiteseriousquantizationartifactsdueto thelimited depthof theframebuffer. Thus, furtherincreasingthenumberof samplingplaneswill not improve thequality.
The basictechniquesof interleaved samplingpresentedin this papernaturallygener-alizeto all Accumulation Buffer [HA90] andmulti-passframe-buffer [POAU00] tech-niqueswhile preserving the goodspectralpropertiesandthe fastconvergence. So itis straightforward to apply interleaved samplingto weightedsampling[HA90], to thesimulationof extendedlight sources usingdeepshadow maps [LV00], andto globalillumination simulations[Kel97, HDKS00]. Hereinterleavedsamplingcanreplacethe�
-shadows-artifactby noise.Finally it is veryappealing to applyinterleavedsamplingto CCD-chipdesigns.
4 Conclusion
We have presenteda new samplingschemecalled interleavedsamplingthat exhibitsexcellent spectralbehavior andyieldsexcellentqualityespeciallyat low samplingrates,while at thesametime it is simpleandefficient to implement.
Thefundamentalideaof thenew method is to interleave samplesfrom severalreg-ulargridsto form a locally irregular samplingpattern.By stretchingout theperiodicityof theper-pixel samplingpattern, aliasingstructuresarespreadout, too,andin conse-quencemuchlessperceivable.Thehighcoherenceof theinvolvedregulargridsallowsfor veryefficient implementationsin software,andin new aswell ascurrent hardware.Thefeasibilityof interleaved samplinghasbeendemonstratedby anumberof examplesranging from supersampling overmotionblur to volumerendering.
Acknowledgements
For theimagein Figure6 we usedanMR datasetkindly providedby ChristophRezk-Salama,University of Erlangen-Nurnberg. Part of this work wasdone in August 2000,while both authors stayedat the Max-Planck-Institut fur Informatik in Saarbrucken,
Germany. We would like to thankHans-PeterSeidelfor granting theresearchstayofthefirst author, whowasalsosupportedby mentalimages,Germany.
Ake93. K. Akeley, RealityEngineGraphics, ComputerGraphics(SIGGRAPH ’93 ConferenceProceedings),1993,pp.109–116.
BB84. L. BrotmanandN. Badler, Generating SoftShadowswith a DepthBuffer Algorithm,IEEEComputerGraphicsandApplications4 (1984),no.10,71–81.
CCC87. R. Cook, L. Carpenter, and E. Catmull, The ReyesImage RenderingArchitecture,ComputerGraphics(SIGGRAPH’87 ConferenceProceedings),1987, pp.95–102.
CCF94. B. Cabral,N. Cam,andJ. Foran,AcceleratedVolumeRenderingand TomographicReconstruction Using Texture Mapping Hardware, 1994Symposiumon VolumeVi-sualization,October1994,pp.91–98.
DW85. M. DippeandE.Wold,AntialiasingthroughStochasticSampling, ComputerGraphics(SIGGRAPH’85 ConferenceProceedings),1985, pp.69–78.
FC62. A. Frolov andN. Chentsov, On the calculationof certain integrals dependenton aparameter by theMonteCarlo method, Zh. Vychisl. Mat. Fiz. 2 (1962),no. 4, 714–717,(in Russian).
Gla95. A. Glassner, Principlesof Digital Image Synthesis, MorganKaufmann, 1995.GW99. M. GossandK. Wu, Studyof SupersamplingMethods for ComputerGraphicsHard-
ware Antialiasing, Tech.report,HPLabs,1999.HA90. P. HaeberliandK. Akeley, TheAccumulationBuffer: Hardware Support for High-
HDKS00. W. Heidrich, K. Daubert,J. Kautz, andH.-P. Seidel,Illuminating Micro GeometryBasedon PrecomputedVisibility, ComputerGraphics(SIGGRAPH ’00 ConferenceProceedings),2000,pp.455–464.
HSS97. W. Heidrich, Ph. Slusallek,and H.-P. Seidel,An Image-Based Model for RealisticLensSystemsin InteractiveComputerGraphics, GraphicsInterface’97, May 1997,pp.68–75.
Kel97. A. Keller, InstantRadiosity, ComputerGraphics(SIGGRAPH’97 ConferencePro-ceedings),1997, pp.49–56.
KHM95. C. Kolb, P. Hanrahan,and D. Mitchell, A RealisticCamera Model for ComputerGraphics, ComputerGraphics(SIGGRAPH ’95 ConferenceProceedings), 1995,pp.317–324.
LV00. T. Lokovich andE.Veach,DeepShadowMaps, ComputerGraphics(SIGGRAPH ’00ConferenceProceedings),2000,pp.385–392.
MF92. M. McCool andE. Fiume,Hierarchical PoissonDisk SamplingDistributions, Pro-ceedings of GraphicsInterface’92, May 1992,pp.94–105.
PKK00. M. Pauly, T. Kollig, andA. Keller, MetropolisLight Transportfor Participating Me-dia, RenderingTechniques 2000(Proc.11thEurographics Workshopon Rendering),Springer, 2000, pp.11–22.
POAU00. M. Peercy, M. Olano,J. Airey, andJ. Ungar, InteractiveMulti-PassProgrammableShading, Computer Graphics (SIGGRAPH ’00 ConferenceProceedings),2000,pp.425–432.
Sob62. I. Sobol,Theuseof ��-distributionfor error estimationin thecalculationof integrals
by theMonteCarlo method, U.S.S.R. Comput.Math. andMath. Phys.2 (1962), 717– 723.
Fig. 4. Blendingbetweenregularandirregularsamplingfor the exampleof motion blur. In allimagesweuse16samplesperpixel. Fromleft to right: methodof dependenttests(AccumulationBuffer), interleavedsampling,andfor illustrationindependentrandom samples.
Fig. 5. A comparisonof interleavedsamplingfor volumerendering(bottom)with thetraditionaltexture-basedapproachdescribedby Cabralet al. [CCF94](top). Fromleft to right:
� �, � � , and� � �
planes.Notethatin the� � �
planeexamplequantizationartifactsdueto thelimited dynamicrangeof theframebuffer startto appear. To avoid theseartifacts,smallernumbersof planeshaveto be used,in which caseinterleaved samplingproduces muchbetterresultsthan the originalalgorithm.
Fig. 6. Illustrationof our volumerenderingalgorithmwith realdatasets.Left of pair: traditionaltexture basedmethodexhibiting aliasingartifactsthat show up asringing structures.Right ofpair: interleavedsamplingwith drasticallyreduceddepthaliasing.
