GLSL Applications: 1 of 2 Joseph Kider Source: Patrick Cozzi – Spring 2011 University of...

GLSL Applications: 1 of 2

Joseph KiderSource: Patrick Cozzi – Spring 2011

University of PennsylvaniaCIS 565 - Fall 2011

Agenda

GLSL ApplicationsPer-Fragment Lighting Image Processing

Finish last week’s slidesOpenGL DrawingOpenGL Multithreading

Per-Fragment Lighting

Slide from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

Per-Fragment Lighting

Images from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

Per-Vertex Lighting Per-Fragment Lighting

Per-Fragment Lighting: Diffuse





uniform vec3 u_Color;

in vec3 fs_Incident;

in vec3 fs_Normal;

in vec3 fs_Texcoord;

out vec3 out_Color;

void main(void)

{

vec3 incident = normalize(fs_Incident);

vec3 normal = normalize(fs_Normal);

float diffuse = max(0.0f, dot(-incident, normal));

out_Color = vec4(diffuse * u_Color, 1.0f);

}




in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{





}

• in vectors are not normalized. Why?• Good practice: don’t write to in variables




in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{





}

• Know the graph:

• Why max?

Graph from http://www.wsd1.org/waec/math/pre-calculus%20advanced/Trigonometry/Graphing/graphingintro.htm

Per-Fragment Lighting: Specular




Per-Fragment Lighting: Specularuniform vec3 u_Color; uniform vec3 u_SpecColor;

uniform float u_SpecHardness;


in vec3 fs_Viewer;

in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{



vec3 H = normalize(-incident + fs_Viewer);

float specular = pow(max(0.0f, dot(H, normal)), u_SpecHardness);


out_Color = vec4(diffuse*u_Color + specular*u_SpecColor,1.0f);

}




in vec3 fs_Viewer;

in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{







}

“half” vector




in vec3 fs_Viewer;

in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{







}

Blinn-Phong shading



Image Processing

Our first look at GPGPU General-Purpose computation on Graphics Processing Units

Input: Image Output: Processed image A kernel runs on each pixel

Image Processing Examples

Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf

Image Negative



Edge Detection



Toon Rendering

Image Processing Questions

Is the GPU a good fit for image processing?

Is image processing data-parallel? What about bus traffic? What type of shader should implement an

image processing kernel?

Image Processing: GPU Setup

Input:TextureViewport-aligned quad, a.k.a full-screen quad

Output: framebuffer…for now

Kernel: fragment shader


1. Render viewport-aligned quad

Fragment shader

2. A fragment is invokedfor each screen pixel

3. Each fragment shader can access any part of the image stored as a texel:gather

4. Each fragment shader executes the kernel and writes the color to the framebuffer




Fragment shader







Fragment shader







Fragment shader







Fragment shader






How do we model the viewport-aligned quad?

Two triangles? One big triangle?

Screen


Which has more vertex shader overhead?Does it matter?

Which is simpler to implement? Which has less fragment shader

overhead?


Triangle edges are redundantly shaded

Fragments are processed in 2x2 blocksWhy?


Triangle edges are redundantly shaded

Image and Chart from http://www.humus.name/index.php?page=News&ID=228

Number of vertices

FPS

Fan Strip Max area


The viewport has the same width and height as the image to be processedThe texture also has the same dimensionsHow does the fragment shader access texels?


Store texture coordinates per vertex

in vec3 Position;

in vec2 Texcoords;

out vec2 fs_Texcoords;

void main(void)

{

fs_Texcoords = Texcoords;

gl_Position = vec4(Position, 1.0);

}

Vertex Shader

uniform sampler2D u_Image;

in vec2 fs_Texcoords;

out vec4 out_Color;

void main(void)

{

out_Color = texture(u_Image, fs_Texcoords);

}

Fragment Shader


Store texture coordinates per vertexWhat memory costs does this incur?

Does it matter?

What bandwidth costs does this incur?What non-obvious optimization does it allow?


Compute texture coordinate in fragment shader

in vec3 Position;

void main(void)

{


}

Vertex Shader


uniform vec2 u_inverseViewportDimensions;

out vec4 out_Color;

void main(void)

{

vec2 txCoord = u_inverseViewportDimensions * gl_FragCoord.xy;

out_Color = texture(u_Image, txCoord);

}

Fragment Shader


Compute texture coordinate in fragment shader

in vec3 Position;

void main(void)

{


}

Vertex Shader



out vec4 out_Color;

void main(void)

{


out_Color = texture(u_Image, txCoord);

}

Fragment Shader

What is u_inverseViewportDimensions?


How do you access adjacent texels?uniform sampler2D u_Image;


out vec4 out_Color;

void main(void)

{

vec4 c0 = texture(u_Image, fs_Texcoords);

vec4 c1 = textureOffset(u_Image, fs_Texcoords, ivec2(-1, 0));

vec4 c2 = textureOffset(u_Image, fs_Texcoords, ivec2( 1, 0));

vec4 c3 = textureOffset(u_Image, fs_Texcoords, ivec2( 0, -1));


out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}


How do you access adjacent texels?uniform sampler2D u_Image;


out vec4 out_Color;

void main(void)

{

vec4 c0 = texture(u_Image, fs_Texcoords);

vec4 c1 = textureOffset(u_Image, fs_Texcoords, ivec2(-1, 0));


vec4 c3 = textureOffset(u_Image, fs_Texcoords, ivec2( 0, -1));


out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}

(-1, 0) (1, 0)

(0, -1)

(0, 1)


textureOffset requires constantsE.g. ivec2(x, y) is not allowed

How else do you access adjacent texels?

Image Processing: GPU Setup How else do you access adjacent texels?



out vec4 out_Color;

void main(void)

{


vec2 delta = 1.0 / textureSize(u_Image);

vec4 c0 = texture(u_Image, txCoord);

vec4 c1 = texture(u_Image, txCoord + (delta * vec2(-1.0, 0.0)));

vec4 c2 = texture(u_Image, txCoord + (delta * vec2( 1.0, 0.0)));

vec4 c3 = texture(u_Image, txCoord + (delta * vec2( 0.0, -1.0)));


out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}




out vec4 out_Color;

void main(void)

{








out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}




out vec4 out_Color;

void main(void)

{








out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}

(-1, 0) (1, 0)

(0, -1)

(0, 1)

Image Processing: Kernel Examples

Image Negativeuniform sampler2D u_Image;


out vec4 out_Color;

void main(void)

{

out_Color = vec4(1.0) - texture(u_Image, fs_Texcoords);

}



Image from http://www.librow.com/articles/article-9

2D Gaussian Gaussian Blur

Filter for 3x3 Gaussian Blur:

[1 2 1]

1/16 * [2 4 2]

[1 2 1]

• The elements add to one• Other filters are also used for

• Edge detection• Sharpen• Emboss• …



Gaussian BlurHow would you implement the fragment

shader?How is the memory coherence?

3x3, 5x5, etc.


Image from http://www.ozone3d.net/tutorials/image_filtering_p2.php


What does this filter do?

[1 1 1]

1/9 * [1 1 1]

[1 1 1]

Image Processing: Read backs

How do we get the contents of the framebuffer into system memory? Print Screen?

It doesn’t matter if we are using:

Efficient read backs are important


glReadPixelsglUseProgram(/* ... */);

glDraw*(/* ... */);

unsigned char rgb = new unsigned char[width * height * 3];

glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, rgb);

// ...

delete [] rgb;



glDraw*(/* ... */);



// ...

delete [] rgb;

Use GPU for image processing



glDraw*(/* ... */);



// ...

delete [] rgb;

Allocate buffer for processed image



glDraw*(/* ... */);



// ...

delete [] rgb; Ask for framebuffer’s color buffer



glDraw*(/* ... */);



// ...

delete [] rgb; You guys are sharp


What is the major problem with glReadPixels?

Image Processing: Use Cases

Photoshop-type applications Post-processing in games On the fly video manipulation Augmented reality

These last three don’t even need read backs

Date post:	19-Dec-2015
Category:	Documents
View:	218 times
Download:	0 times

GLSL Applications: 1 of 2 Joseph Kider Source: Patrick Cozzi – Spring 2011 University of...

Documents