05-schedulingGraphicsPipeline-BPS2011-ragankelley

7/29/2019 05-schedulingGraphicsPipeline-BPS2011-ragankelley

1/95

Beyond Programmable Shading 2011

Scheduling the Graphics Pipe

JonathanRagan-Kelley,MITCSAIL9 August 2011

Beyond Programmable Shadin


2/95


This talk

HowtothinkaboutschedulingGPU-stylepipeliFourconstraintswhichdriveschedulingdecisioExamplesoftheseconceptsinrealGPUdesignGoals

Know why GPUs, APIs impose the constraints they do.

Develop intuition for what they can do well.

Understand key patterns forbuilding your own pipelines.


3/95


This talk

HowtothinkaboutschedulingGPU-stylepipeliFourconstraintswhichdriveschedulingdecisioExamplesoftheseconceptsinrealGPUdesignGoals

Know why GPUs, APIs impose the constraints they do.

Develop intuition for what they can do well.

Understand key patterns forbuilding your own pipelines.


4/95


Scheduling[n.]:

First, some definitions


5/95Beyond Programmable Shading 2011

Scheduling[n.]:Assigning computations and dat

to resources in space and time.




Scheduling[n.]:Assigning computations and dat

to resources in space and time.


Task[n.]:A single, discrete unit of work.



The workload: Direct3D




dataflow


11/95



dataflow


12/95


The machine: a modern GPU


13/95


Scheduling a draw call as a series o

time


14/95



t

ime


15/95



t

ime


16/95



t

ime


17/95



t

ime


18/95



t

ime


19/95



t

ime


20/95


An efficient schedule keeps hardwa

t

ime


21/95


Choosing which tasks to run when (an

ResourceconstraintsTasks can only execute when there are sufficient resources for the

computationandtheir data.

Coherence

Control coherence is essential to shader core efficiency.

Data coherence is essential to memory and communication efficie

LoadbalanceIrregularity in execution time create bubbles in the pipeline sched

Ordering

Graphics APIs define strict ordering semantics, which restrict possible


22/95


Resource constraints limit scheduling

time


23/95



time


24/95



time

???


25/95



time

??? ???


26/95



time

??? ???


27/95



time

Key concept:Preallocation of resources helps

guarantee forward progress.

??? ???

C h i b l i


28/95


Coherence is a balancing act

Intrinsic tension between:

Horizontal (control, fetch) coherenc

Vertical (producer-consumer) localit

Localityand LoadBalance.

G hi kl d i l


29/95


Graphics workloads are irregula

G hi kl d i l


30/95



G hi kl d i l


31/95



G hi kl d i l


32/95



G aphics o kloads a e i eg la


33/95



But: Shaders are optimized forregular,self-simila

Imbalanced work createsbubbles in the task



34/95



But: Shaders are optimized forregular,self-simila

Imbalanced work createsbubbles in the task

Solution:

Dynamically generating and aggregatingtaskisolates irregularity and recaptures coherence

Redistributing tasks restores loadbalance.

Redistribution after irregular amplifi


35/95



time



36/95



time



37/95



time

Key concept:Managing irregularity by dynamicallygeneraaggregating, and redistributingtasks

Ordering


38/95


Ordering

Rule:

All framebuffer updates must appe

as though all triangles were drawn

strict sequential order

Ordering


39/95


Ordering

Rule:

All framebuffer updates must appe

as though all triangles were drawn

strict sequential order

Key concept:Carefully structuring taskredistributito maintainAPIordering.


40/95


Building a real pipeline

Static tile scheduling


41/95



Multiple cores:1 front-end

n back-end

The simplest thing that could

possibly work.



42/95





43/95





44/95





45/95





46/95





47/95



Localitycaptured within tiles

Resourceconstraints

static = simple

Orderingsingle front-end,sequential processing

within each tile



48/95



Theproblem:loadimbalance

only one task creation

point.

no dynamic task

redistribution.



49/95


g



point.

no dynamic task

redistribution.



50/95


g



point.

no dynamic task

redistribution.

Sort-last fragment shading


51/95


g g

Exem

NVIDIA



52/95


g g

Exem

NVIDIA

Redistrib

fragment

But how

maintain



53/95


g g

Exem

NVIDIA



54/95


Exem

NVIDIA

Comp

shadasync



55/95


Exem

NVIDIA

B

f

F

Unified shaders


56/95


Exem

NVIDIA

Solve load balance by time-multiplexing diffe

stages onto shared processors according to

Unified Shaders: time-multiplexing


57/95


time

Exem

NVIDIA

Unified Shaders: time-multiplexing


58/95


time

Exem

NVIDIA

Prioritizing the logical pipeline


59/95




60/95


5

4

3

2

1

0

priority



61/95


5

4

3

2

1

0

priority



62/95


5

4

3

2

1

0

fixed-size

queue storage

priority

Scheduling the pipeline


63/95


time



64/95


time



65/95


time



66/95


time

Hi

sta

Lo

bu



67/95


time



68/95


Queue sizes and backpressure provide a

natural knob for balancing horizontalbatcoherenceand producer-consumerloc

A real computational graphics pipeline


69/95


Entry Points

Ray Shading

Traversal

Host

Ray Generation Program

Intersection Program

Any Hit Program

Closest Hit Program

Selector Visit Program

Trace

Miss Program

Exception Program

Buffers

Texture SamplersVariables



70/95


Entry Points

Ray Shading

Traversal

Host



Any Hit Program

Closest Hit Program


Trace

Miss Program

Exception Program

Buffers


Pipelineabstractioforraytracing



71/95


Entry Points

Ray Shading

Traversal

Host



Any Hit Program

Closest Hit Program


Trace

Miss Program

Exception Program

Buffers


PipelineabstractioforraytracingApplicationfunctishader-styleprogIntersecting primitives

Shading surfaces, firin



72/95


Entry Points

Ray Shading

Traversal

Host



Any Hit Program

Closest Hit Program


Trace

Miss Program

Exception Program

Buffers


PipelineabstractioforraytracingApplicationfunctishader-styleprogIntersecting primitives

Shading surfaces, firin

PipelinestructureTraversal, acceleration

Order of execution

Resource managemen

Issues in scheduling a ray trace


73/95


Breadthfirstordepthfirsttraversal?Wide execution aggregates more potentially coherent work

Depth-first execution reduces footprint needed.



74/95




OptiX:aswideasthethemachine,butnowide



75/95




OptiX:aswideasthethemachine,butnowide

ExtractingSIMDcoherenceShader core requires SIMD batches for efficiency.

Rays may diverge.

Ray tracing on a SIMD machine


76/95


Scalarray

tracing

traversetraverse

Ray tracing on a SIMD machine


77/95


Scalarray

tracing

traversetraverse

Packettracing

traversetraverse

Breaking packets: SIMT ray trac[Aila & Laine 2


78/95


[Aila & Laine 2

AllowdatadivergenceDifferent rays traverse, intersect different parts of the s

Maintaincontrol(SIMD)coherenceAll rays in bundle either traverse orintersecttogether

Breaking packets: SIMT ray trac[Aila & Laine 2


79/95


while(state != done) {if (state == traverse) traverse();

if (state == intersect) intersect();

}

[

AllowdatadivergenceDifferent rays traverse, intersect different parts of the s

Maintaincontrol(SIMD)coherenceAll rays in bundle either traverse orintersecttogether

A pipeline program as a state mac


80/95


while(myState != DONE) {

nextState = scheduler();

if (myState == nextState)switch(myState) {

case 0: myState = traverse(); break;

case 1: myState = intersector1(); break


case 3: myState = shader1(); break;


}

}



81/95










}

}



82/95










}

}



83/95










}

}



84/95










}

}



85/95










}

}


86/95


Summary

Key concepts


87/95


Think of schedulingthepipeline as mappingtasksonPreallocateresourcesbeforelaunchingatask.Preallocation helps ensure forward progress and prevent deadloc

Graphicsisirregular.Dynamically generating, aggregating and redistributingtasks a

amplification points regains coherence and loadbalance.Ordermatters.Carefully structure taskredistribution to maintain ordering.

Why dont we have dynamic resource all


88/95


Static preallocation of resources guarantee

forward progress.

Tasks which outgrow available resources ccausing deadlock.

Whydon twehavedynamicresourcealle.g. recursion, malloc() in shaders

GeometryShadersareslowbecausethedynamicamplificationinshaders.


89/95


Pick your poison:

AlwaysstreamthroughDRAM.exemplar: ATI R600

Smooth falloff for large amplification, but very slo

amplification (DRAM latency).

Scaledownparallelismtofit.exemplar: NVIDIA G80

Fast for small amplification, poor shader through

parallelism) for large amplification.

Key concepts

Thi k f h d li th i li i t k


90/95





Why isnt rasterization programma

Yes, partly because it is computationally intensive,b


91/95


es, pa y because s co pu a o a y e s e, b

It ishighly irregular.

It must generate and aggregateregular o

It must integrate with an order-preserving

redistribution mechanism.

Key concepts

Thi k f h d li th i li i t k


92/95





Questions for the future


93/95


Can we relax the strictordering requiremen

Can you build a genericscheduler forapplication-definedpipelines?What application-specificinformation wougenericscheduler need to work well?

Starting points to learn more

Thenextstep:parallelprimitiveprocessing


94/95


p p p p g

Eldridge et al. Pomegranate: A Fully Scalable Graphics Arc

SIGGRAPH 2000.

Tim Purcell. Fast Tessellated Rendering on Fermi GF100. H

HPG 2010.

Schedulingcyclicgraphs,insoftware,oncurrentParker et al. OptiX:A General Purpose Ray Tracing EngineSIGGRAPH 2010.

DetailsoftheARMMalidesignTom Olson. Mali-400 MP: A Scalable GPU for Mobile Devi

Hot3D, HPG 2010.


95/95


Thank you

Specialthanks:Tim Purcell, Steve Molnar, Henry Moreton, Steve Parker, Austin Robi

Jeremy Sugerman - Stanford

Mike Houston -AMD

Mike Doggett - Lund University

Tom Olson -ARM

Date post:	03-Apr-2018
Category:	Documents
Upload:	yurymik
View:	213 times
Download:	0 times

05-schedulingGraphicsPipeline-BPS2011-ragankelley

Documents