09-towardBlurryRasterizerWithNotes-BPS2011

7/29/2019 09-towardBlurryRasterizerWithNotes-BPS2011

1/52


2/52


3/52


4/52

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Depth of Field

Direct viewers attention

Match footage with CGI

Motion Blur

Enhance effect of motion

Reduce temporal aliasing

Motivation


5/52


Temporal Aliasing

Avoid jerky animations at low frame rates.Temporal aliasing is as important as spatial aliasingIncreasing the frame rate decreases temporal aliasing,but it is still there. Crucial for feature film (that only now starttalking about 48 fps).


6/52


Motion Blur Rendering

x

yt

pixelt

x

y

Pixel color is an integral over(x,y,t)

Instead of searching for an analytical solution (very hard), we estimate the integral by Monte Carlo techniques.


7/52


Depth of Field Rendering

pixel

image plane(x,y)

focus planelens(u,v)

Pixel color is an integral over(x,y,u,v)

object out of focusobject in focus


8/52


Depth of Field and Motion Blur

pixel

image plane(x,y)

focus planelens(u,v)

object out of focusobject in focus

Pixel color is an integral over(x,y,u,v,t)

t=0

t=1


9/52


SamplingApproximating the visibility integral



10/52


Approximate Integral

Accumulation buffering

Render scene at many fixedtimes (and lens positions)[Korein83, Haeberli90]

Stochastic sampling [Cook86]Sample pixel at non-uniformly

spaced locations in (x,y,t,u,v)

Aliasing is replaced by noise

uniform

stochastic

Visibility integral in (u,v,t) is very hard to solve analytically.Monte Carlo technique: Sample the integral at many non-uniformlyspaced locations. Stochastic sampling to avoid correlation between dimensions and extends to higher dimensions.Average N samples per pixel with diferent (x,y,t) -> spatial & temporal anti-aliasing!Use stratified sampling to reduce noise. Higher correlation between dimensions=lower noise, but more bias


11/52


12/52


Sampling for Motion Blur (3D)

A set of 64 fixed timesis sufficient to avoid

banding artifacts

Many samples per pixel

needed to reduce noise

4 spp 8 spp 16 spp 32 spp 256 spp

High quality sampling patterns available. On-the-fly generation possible.Use patterns that maximizes minimum distance in 3D,and have good 2D projections (on screen space x,y)Reconstruction filters [Lehtinen2011] can help tremendously here


13/52


Sampling for DOF (4D)

Need many unique lens positions

InterleaveUV (64 lens pos) InterleaveUV (256 lens pos) 64 random spp

Furthermore, out-of-focus regions and high frequencies require morevisibility samples per pixel to reduce noise levels.


14/52

InterleaveUVT Sobol sampling InterleaveUVT Sobol sampling

4 spp 4 spp 16 spp 16 sppBeyond Programmable Shading Course, ACM SIGGRAPH 2011

Sampling for MB+DOF (5D)

Need large number of unique (ui,vi,ti) tuples

Harder to find good sample distributionsImagescourtesy

ofLaineetal.[2011]

takenfromAssas

sinsCreedbyUbisoft

The InterleaveUVT images have 64 unique (u,v,t) positionsHarder to find good sample distributions with stratified samples in 5D space.


15/52


VisibilityThe rasterizer stage



16/52


Recap: Standard Rasterization


Visibility determination in a graphics pipeline - convert geometry to pixels


17/52


The Graphics Pipeline

Vertex Shader

Input Assembler

Hull Shader

Tessellator

Domain Shader

Rasterizer

Pixel Shader

Output Merger

MemorySy

stem(buffers,textures)


18/52


19/52


Test samples = evaluate edge

equations


20/52

Test tile of pixels for overlap

Reject samples in tiles completely outside triangle

Enables coarse occlusion culling, improves coherence


Tiled Traversal

Important optimization for larger triangles.Basis for a hierarchical rasterizer. Improves texture and memory coherence


21/52


22/52


23/52


24/52

S S B d


25/52


Screen-Space Bounds

Bound full motion and all lens positions

Handle moving triangles intersectingz=0 with care

t=0

t=1

z= 0

t=0t=1

image plane

DOF BBox

S ti T l Vi ibilit


26/52


Spatio-Temporal Visibility

Moving triangle may cover largeregion in (x,y,t)

More samples per pixel neededfor low noise levels

We need to quickly cull samplesin both space and time

!

"

!

#

The key is that many more samples need to be tested, and that the complexity is tied to the amount of blur applied (motion and/or DOF).Dimension on visibility query has increased from two to three (or five) dimensions

S S C H ll


27/52

!

"

!

#


Screen Space Convex Hull

Reduce spatial bounds [McGuire2010]

A tile is only visited once, enablingmultisampling & coarse Z culling

No temporal culling

Revert to coarser bounds if moving triangle intersects z=0.The screen space convex hull algorithm only works onprojected vertices with z>0.

Interval


28/52

!

"

!

#


Interval

UseNtime intervals[Cook90, Fatahalian2009]

Bound triangles movement in

each interval and test all samples

in (x,y,t) box

Multiple tri setup and bounding. One tile visited multiple times


29/52


30/52

5D Rasterization


31/52


5D Rasterization

Interval [Cook90, Fatahalian2009]

Many strata in (u,v,t). Find (x,y) bounds for each stratum

Interleaved sampling [Keller2001, Fatahalian2009]

Generate fixed set of(u,v,t) tuples and

rasterize coarsely in (x,y)

Tile-based traversal with (u,v,t) bounds [Laine2011]

Tile-test returns bounds foru,v and tindividually

Interleave needs a large set of tuples. Laine: Tile-bounds for t computed using entire lens bounds.Tile-bounds for (u,v) computed using full motion trail. Lower eciency in 5D than 3D and 4D tests.

Tile Overlap Tests


32/52


Tile-Overlap Tests

Find time interval when moving triangle overlaps

with the screen space tile [Laine2011, Munkberg2011b]

[0,1/4] [0,1/2] [1/4,3/4] [1/2,1] [3/4, 1]

t=0 t=1

Examples: Dual-space bounds [Laine], Tile frustum vs moving AABB & Tile vs moving triangleedges [Munkberg].

Tile-Overlap Test for DOF


33/52


Tile-Overlap Test for DOF

pixel

image planefocus planelens

objects out of focus

focus planelens

objects out of focus

objects in focus

Idea:Only test lensregion that sees

the object througha certain screenspace tile

Lens Bounds Per Tile


34/52


Lens Bounds Per Tile

Separating plane tests determine active lens

region [Akenine-Mller2011, Laine2011]lens separating

linestriangle

focus plane

tile

lens separating

lines

triangle

focus plane

tile


35/52


Shading


Shading Static Primitives


36/52


Shading Static Primitives

Shading after visibility

Shade per 2x2 pixels in screen space

Shading before visibility [Cook87]

Shade in object space per vertex

Decoupled shading and visibility

GPU Derivatives by finite diferences. Supports Multi Sampling Anti-Aliasing (MSAA)Shade once per pixel but take many visibility samples. Very ecient for large primitivesReyes: Shading before visibility. Shade in object space per vertex.Shade a grid of vertices.Ecient SIMD shading.Derivatives by finite diferences. Requires small primitives.Small to large triangles: Hierarchical rasterization needed, but need explicit cache for decoupled sampling and shadingVery small triangles. Hierarchical rasterization only ecient for large blursShading at vertex frequency sucient. Supports decoupled sampling and shading by design

Shading Moving Primitives


37/52


Shading Moving Primitives

Supersampling

Shade each visibility sample

Prohibitively expensive

Extend MSAA [McGuire2010]

Motion blur exampleShade once per covered pixel at a certain time

Texture filters take motion into account [Loviscach2005]

Approaches: Supersampling: Shade every visibility sample. No restrictions on traversal orderCaptures temporally varying shadingMSAA for motion blur and DOF: Extend standard MSAA to motion blur & depth of fieldMay shade arbitrary far outside triangle. Additional shading samples for temporal derivativeRequires a tile-based traversal orderDecoupled Shading: Mapping function between visibility samples and shading samples.Map multiple visibility samples to one shading pointPer-vertex shading (requires pixel-sized primitives)

MSAA Failure Case


38/52


MSAA Failure Case

Reverts to supersampling for large motion

Each sample hits a new triangle

t=0

t=1/4

t=1/2

t=3/4

Movement

A diferent sample time is assigned to each of the four multi-samples

Decoupled Sampling [Ragan-Kelley2011 Burns2010]


39/52


Decoupled Sampling[Ragan-Kelley2011, Burns2010]

Many-to-one mapping function

Between visibility and shading samples

Low shading rate

Even for large motion and defocus

Needs cache for shaded valuesRemapping logic per sample

Assume constant shading in time. In general: design (many to one) mapping functionbetween visibility samples and shading samples.Remap sample to a fixed time and center of lensUse remapped location as cache keyIf cache hit, fetch color, else shade and store color

Decoupled Sampling Example


40/52


Decoupled Sampling Example

T=0

T=1

T=0t=0

t=1

screen space barycentric space shading space

remap to triangle at t=0

snap remappedvisibility sampleto closest shading

samplevisibility samples

Each visibility sample has a position in (x,y,t). Some of them hit the triangle. Compute the barycentric coordinate of the hit point.Remap to the triangle at t=0 and snap to the closest shading point in some shading space. Here, the shading points are in the centerof the pixel.

Reconstruction


41/52


Reconstruction

Reduce noise in stochastic rendering

May drastically reduce required #samples per pixel

Store additional information with samples [Lehtinen2011]

Reference (256 spp)ReconstructedInput image (1 spp)

Imagescourtesyof

Lehtinenetal.[2011]

Important emerging research field for stochastic rendering.Can be seen as an alternative to shading caches. One shading value is reused for many (reconstructed) visibility samples.Lethinen et al store motion vector and sample depth. Enables on-the-fly reprojection of stochastic samples during reconstruction.Egan & Soler: Design sheared filter by looking at frequency domain. Adaptive sampling based on predicted bandwidth.


42/52

View frustum

culledOcclusion

culled

Backface

culled


Culling


View Frustum Culling


43/52


g

Solve for temporal interval when triangle is

inside the view frustum [Laine2011, Munkberg2011b]

z= 0

t=0

t=1

near plane

t=0.6

t=1

Test entire moving triangleGives smaller screen space bboxes (and better dual-space bounds)

Motion Blur Backface Culling


44/52

Backface test is cubic

polynomial in t[Munkberg2011a]

Not conservative to test

facing at t=0 and t=1


g

p0

p1

p2

p0

p1

p2

p0

p1

p2

t=0 t=0.5 t=1

The backface test for a triangle with linear vertex motion is cubic polynomialin t. Cubic polynomial -> up to three zeros -> up to three changes in the signof the signed area

DOF Backface Culling


45/52


g

Lens

BF cull at center of lens

Correct BF cull

DOF Backface Culling


46/52


Lens

g

BF cull at center of lens

Correct BF cull

Occlusion Culling for Motion Blur


47/52


g

Standard z-culling less efficient

Spatio-temporal occlusion culling[Akenine-Mller2007, Boulos2010]

x

t

solid occludervalid for t=[0,1]

solid occludervalid for t=ti

Illustrat

ionadapted

fromBoulosetal.[2010]

Left: Solid occluder is minimal. Right: In the other extreme, a separate occlusion hierarchy per sample time.Middle ground probably most practical approach. In GPU-like pipeline, want coarse z-cull at tile level. A full spatio-temporal z-hierarchy improves culling,but is likely too expensive in terms of memory and bandwidth usageConclusion: Tile based traversal is preferable. Ecient 5D occlusion culling is an open research field

Conclusions


48/52


Stochastic rasterization does not come for free

Many more sample tests and shading evaluations

Higher color, depth & texture bandwidth

Less efficient culling and compression

Efficient shading is a requirementVery easy for end-user if implemented in HW

Q&A


49/52


Thank You:

The Advanced Rendering Technology team at Intel

The Graphics Group at Lund University

Mike Doggett, Jonathan Ragan-Kelley, Samuli Laine,

and Timo Aila for fruitful discussions

References


50/52


[Akenine-Mller2007]Tomas Akenine-Mller, Jacob Munkberg, and Jon Hasselgren,

"Stochastic Rasterization using Time-Continuous Triangles", Graphics Hardware, August 2007.

[Akenine-Mller2011]Tomas Akenine-Mller, Robert Toth, Jacob Munkberg, and Jon Hasselgren,

"Efficient Depth of Field Rasterization using a Tile Test based on Half-Space Culling", technical report, June 2011

[Boulos2010] Solomon Boulos, Edward Luong, Kayvon Fatahalian, Henry Moreton and Pat Hanrahan

Space-Time Hierarchical Occlusion Culling for Micropolygon Rendering with Motion Blur High Performance Graphics 2010

[Burns2010] Burns, C.A., Fatahalian, K., Mark, W.R. A Lazy Object-Space Shading Architecture With Decoupled Sampling. High Performance Graphics, 19-28. 2010

[Cook84] Robert L. Cook, Thomas Porter, Loren Carpenter, Distributed Ray Tracing SIGGRAPH 1984

[Cook86] Robert L. Cook. Stochastic Sampling in Computer Graphics. ACM Transactions on Graphics, Volume 6, Number 1, January 1996.

[Cook87] Robert L. Cook, Loren Carpenter, Edwin Catmull. The Reyes Rendering Architecture. 1987

[Cook90] Robert L. Cook, Thomas Porter, Loren Carpenter. Pseudo-random point sampling techniques in computer graphics. United States Patent 4,897,806. 1990.

[Egan2009] Kevin Egan , Yu-Ting Tseng , Nicolas Holzschuch , Frdo Durand , Ravi Ramamoorthi,Frequency analysis and sheared reconstruction for rendering motion blur, ACM Transactions on Graphics (TOG), v.28 n.3, August 2009

[Fatahalian2009] Kayvon Fatahalian, Edward Luong, Solomon Boulos, Kurt Akeley, William R. Mark and Pat Hanrahan.

Data-Parallel Rasterization of Micropolygons with Defocus and Motion Blur High Performance Graphics 2009

[Fatahalian2011] Kayvon Fatahalian Evolving the Real-Time Graphics Pipeline for Micropolygon Rendering. Ph.D. Dissertation, Stanford University, 2011

[Grunschlo2011] Leonhard Grunschlo, Martin Stich, Sehera Nawaz, Alexander Keller. MSBVH: An Efficient Acceleration Data Structure for Ray Traced Motion Blur

High Performance Graphics 2011

References cont.


51/52


[Gribel2010] Carl Johan Gribel, Michael Doggett, and Tomas Akenine-Mller, "Analytical Motion Blur Rasterization using Compression",

High Performance Graphics, pp. 163-172, June 2010

[Gribel2011] Carl Johan Gribel, Rasmus Barringer, and Tomas Akenine-Mller, "High-Quality Spatio-Temporal Rendering using Semi-Analytical Visibility", SIGGRAPH 2011.

[Hachisuka2008]Toshiya Hachisuka, Wojciech Jarosz, Richard Peter Weistroffer, Kevin Dale, Greg Humphreys, Matthias Zwicker, Henrik Wann JensenMultidimensional adaptive sampling and reconstruction for ray tracing SIGGRAPH 2008

[Hou2010] Qiming Hou, Hao Qin, Wenyao Li, Baining Guo, Kun Zhou. Micropolygon ray tracing with defocus and motion blur. SIGGRAPH 2010

[Haeberli90] Paul Haeberli , Kurt Akeley, The accumulation buffer: hardware support for high-quality rendering,

Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.309-318, 1990

[Korein83] Jonathan Korein , Norman Badler,Temporal anti-aliasing in computer generated animation, Computer Graphics, vol 17, no .3, p.377-388, 1983

[Keller2001]Alexander Keller , Wolfgang Heidrich, Interleaved Sampling, Eurographics Workshop on Rendering Techniques, p.269-276, 2001

[Laine2011] Samuli Laine, Timo Aila, Tero Karras and Jaakko Lehtinen. Clipless Dual-Space Bounds for Faster Stochastic Rasterization. SIGGRAPH 2011

[Lee2009] Sungkill Lee, Elmar Eisemann and Hans-Peter Seidel. Depth-of-field rendering with multiview synthesis. SIGGRAPH Asia 2009[Lee2010] Sungkill Lee, Elmar Eisemann and Hans-Peter Seidel. Real-time lens blur effects and focus control. SIGGRAPH 2010

[Lehtinen2011] Jaakko Lehtinen, Timo Aila, Jiawen Chen, Samuli Laine and Frdo Durand.

Temporal Light Field Reconstruction for Rendering Distribution Effects. SIGGRAPH 2011

[Loviscach2005] Jrn Loviscach, Motion Blur for Textures by Means of Anisotropic Filtering, Eurographics Symposium on Rendering, 2005. pp. 105-110

[Max85] Max N. L., Lerner D. M.: A two-and-a-half-d motion-blur algorithm. SIGGRAPH '85

[McGuire2010] McGuire, Enderton, Shirley, and Luebke, Real-Time Stochastic Rasterization on Conventional GPU Architectures, High Performance Graphics, June, 2010.

References cont.


52/52


[Munkberg2011a] Jacob Munkberg and Tomas Akenine-Mller, "Backface Culing for Motion Blur and Depth of Field", journal of gpu, graphics and game tools, 2011.

[Munkberg2011b] Jacob Munkberg, Petrik Clarberg, Jon Hasselgren, Robert Toth, Masamichi Sugihara, and Tomas Akenine-Mller,

Hierarchical Stochastic Motion Blur Rasterization, High Performance Graphics 2011.

[Navarro2011] Fernando Navarro, Francisco J. Sern and Diego Guitierrez.Motion Blur Rendering: State of the Art. Computer Graphics Forum Volume 30 (2011), number 1 , pp. 3-26

[Overbeck2009] Ryan S. Overbeck , Craig Donner , Ravi Ramamoorthi, Adaptive Wavelet Rendering, ACM Transactions on Graphics (TOG), v.28 n.5, December 2009

[Potmesil81] Potmesil M. Chakravarty I: A lens and aperture camera model for synthetic image generation. SIGGRAPH 81 pp. 389-399

[Ragan-Kelley2011] Jonathan Ragan-Kelley, Jaakko Lehtinen, Jiawen Chen, Michael Doggett, Frdo Durand. Decoupled Sampling for Graphics Pipelines. SIGGRAPH 2011

[Shinya93] Shinya M.: Spatial anti-aliasing for animation sequences with spatio-temporal filtering. SIGGRAPH 93 pp. 289-296

[Shirley2011] Shirley, Aila, Cohen, Eric Enderton, Laine, Luebke, McGuire, A Local Image Reconstruction Algorithm for Stochastic Rendering, ACM Symposium on Interactive

3D Graphics and Games (I3D 2011 proceedings), February 2011

[Soler2009] Cyril Soler, Kartic Subr, Frdo Durand, Nicolas Holzschuch, Franois Sillion.Fourier depth of field. ACM Transactions on Graphics (TOG) , Volume 28 Issue 2, 2009

[Sousa2008] Sousa T.: Crysis next-gen effects. In Proceedings of the Game Developers Conference 2008.

[Sung2002] K. Sung , A. Pearce , C. Wang, Spatial-Temporal Antialiasing, IEEE Transactions on Visualization and Computer Graphics, v.8 n.2, p.144-153, April 2002

[Wald2007] Wald I., Mark. W. R., Gnther J., Boulos S., Ize T., Hunt W., Parker S.G.,Shirley P.:

State of the art in ray tracing animated scenes. Eurographics 2007 State of the Art Reports

[Walter2006] Bruce Walter, Adam Arbree, Kavita Bala, Donald P. Greenberg. Multidimensional Lightcuts. SIGGRAPH 2006

Date post:	03-Apr-2018
Category:	Documents
Upload:	yurymik
View:	213 times
Download:	0 times

09-towardBlurryRasterizerWithNotes-BPS2011

Documents