Under the Hood: 3D Pipeline

Motherboard & Chipset

PCI Express x16

Discrete 3D Graphics Architecture

CPUCPU

NorthBridge

NorthBridge3D GFX3D GFX

LocalMemory

LocalMemory

SystemMemory

SystemMemory

SouthBridge

SouthBridge

DisplayDevices

GraphicsSub-System

AGP / PCI-E

Frame BufferZ-BufferTextureVertex…

Discrete Graphics Cards

• NVIDIA

• ATI

Integrated 3D Graphics (Unified Memory

Architecture)

CPUCPU

NB +3D GFX

NB +3D GFX3D GFX3D GFX

LocalMemory

LocalMemory

SystemMemory

SystemMemory

SouthBridge

SouthBridge

DisplayDevices

GraphicsSub-System

AGP / PCI-E

DisplayDevices

Optional

Frame BufferZ-BufferTextureVertex…

LocalMemory

LocalMemory

Example: Intel Core i7 (1st Generation)

Example: Intel Core i7 (2nd Generation)

Keep in mind that…

3D graphics processor works in parallel to the CPU

Many Views of Graphics Pipeline

• Simple “Front-End/Back-End” view.

• Textbook version in [Foley/van Dam].

• David Kirk’s (nVidia CTO) version presented in EG Hardware Workshop 1998: (slide 05) http://www.merl.com/hwws98/presentations/kirk/index.htm

Simplified View

• The Data Flow:3D Polygons (+Colors, Lights, Normals,

Texture Coordinates…etc.)2D Polygons2D Pixels (I.e., Output Images)

Transform(& Lighting)

Rasterization

Some different versions…

14

OpenGL ES 1.X (Fixed Function Pipeline)

15

RasterizationRasterization

RenderingRendering

RasterizationRasterization

RenderingRendering

Texture MappingTexture MappingTexture MappingTexture Mapping

TrianglesTriangles

Transform &Lighting

Transform &Lighting

Triangle SetupTriangle Setup

RasterizationRasterization

RenderingRendering

Conventional 3D Graphics Pipeline

Texture MappingTexture MappingRasterizationRasterization

RenderingRendering

TrianglesTriangles

Transform &Lighting

Transform &Lighting

Evolution of 3D Graphics Hardware

Pixel ShaderPixel Shader

Vertex ShaderVertex Shader

Surface EngineSurface Engine

Texture MappingTexture Mapping

And some really scary ones…

17

A Detailed 3D Graphics Pipeline

A Quick ReviewBy default, graphic pipeline will do the

following:1) Take as input various per-vertex

quantities (color, light source, eye point, texture coordinates, etc.)

2) Calculate a final color for each vertex using a basic lighting model (OpenGL uses Phong lighting)

3) For each pixel, linearly interpolate the three surrounding vertex colors to shade the pixel (OpenGL uses Gouraud shading)

4) Write the pixel color value to the frame buffer

Why Need Hardware• All parts of graphics pipeline can be

done in software.– But very slowly.– Example: mesaGL

• For some applications, speed is beauty– Games– Walkthrough– Visualization

Evolutions of Graphics Hardware

1. Gouraud-shaded polygons.

2. Then came antialiasing.

3. Then came texture mapping.

4. Now comes programmable shading.

Fixed vs. Programmable

• Starting in 1999 some graphics cards used the standard lighting model and Gouraud shading to draw polygon fragments entirely in hardware

• Implementing the pipeline in hardware made processing polygons much faster, but the developer could not modify the pipeline (hence “fixed function pipeline”)

• New programmable hardware allows programmers to write vertex and pixel programs to change the pipeline

OpenGLFixed Function Vertex

TransformTransform[MVP],[MV],[MV][MVP],[MV],[MV]-T-T

LightinLightingg

[0,1][0,1]

[0,1][0,1]

TexgenTexgen TextureTextureMatrixMatrixnn

ColorColorSecondaryColSecondaryColoror

TexCoordTexCoordnn

EdgeFlagEdgeFlag

NormalNormal

Vertex Vertex (object)(object)

TexCoordTexCoordnn

EdgeFlagEdgeFlag

Vertex (eye)Vertex (eye)

Vertex (clip)Vertex (clip)

Front&BackFront&BackColorColor

Front&BackFront&BackSecondaryColSecondaryColoror

24

FrontFacing

ColorColor

Coord

Depth

OpenGLFixed Function Fragment

TexTexnn

TETEnn

SuSumm

FogFog[0,1][0,1]

Coord

FrontFacing

ColorColor

SecondaryColSecondaryColoror

TexCoord[TexCoord[nn]]

zz (|zzee|,f )

Depth

Programmable Shaders• A concept made popular by Pixar’s

RenderMan.• First appeared in hardware: UNC

PixelFlow – See SIGGRAPH papers by Molnar 1995

and Olano 1997.

• Made affordable by nVidia GeForce3 and XBox.

GL2 Vertex Processor

TemporariesTemporaries

VertexVertexShaderShader

UniformUniform

ColorColorSecondaryColSecondaryColoror

TexCoordTexCoordnn

EdgeFlagEdgeFlag

NormalNormal

Vertex Vertex (object)(object)

TexCoordTexCoordnn

EdgeFlagEdgeFlag

Vertex (eye)Vertex (eye)

Vertex (clip)Vertex (clip)

Front&BackFront&BackColorColor

Front&BackFront&BackSecondaryColSecondaryColoror

27

GL2 Fragment Processor

TexCoord[TexCoord[nn]]

FrontFacing

zz (|zzee|,f )

Coord

FrontFacing

ColorColor

Coord

ColorColor

SecondaryColSecondaryColoror

Depth Depth

TemporariesTemporaries

FragmenFragmentt

ShaderShader

UniformUniform TextureTexture