CS 354Interaction &Project 2

Mark KilgardUniversity of TexasMarch 29, 2012

CS 354 2

Today’s material

In-class quiz On compression lecture

Lecture topic Project 2 Interaction

CS 354 3

My Office Hours

Tuesday, before class Painter (PAI) 5.35 8:45 a.m. to 9:15

Thursday, after class ACE 6.302 11:00 a.m. to 12

Randy’s office hours Monday & Wednesday 11 a.m. to 12:00 Painter (PAI) 5.33

CS 354 4

Last time, this time

Last lecture, we discussed Project 2 on Programmable Shaders Compression for Computer Graphics

This lecture Project 2 discussion

Get started now

Interaction

CS 354 5

Daily Quiz

1. True or False: A hardware texture compression scheme must be capable of random access decompression.

2. True or False: Floating point values are required to have at least 23 bits devoted to the mantissa and 8 bits for the exponent.

3. True or False: Run-length encoding works well for digital photographs.

4. If the surface normals in a mesh are each stored as an index into a set of 256 quantized normals rather than as three 32-bit floating point numbers, what compression ratio is achieved over the floating-point representation?

On a sheet of paper• Write your EID, name, and date• Write #1, #2, #3, #4 followed by its answer

CS 354 6

Project 2

Project is NOW available PDF + code to use

Requirements Mostly writing GLSL shaders Small amount of C/C++ code

Convert height map to normal map Implement NormalMap::computeNormal method

Intermediate milestone Monday, April 2nd

Submit snapshots and GLSL code for first two shaders Complete project

Due Friday, April 6th All 9 shaders Embellish for additional credit

CS 354 7

By April 2

Have these first two shaders tasks implemented

Task 0: roll torus

Task 1: apply decal

CS 354 8

Procedurally Generating aTorus from a 2D Grid

2D grid over (s,t)[0,1]

Tessellated torus

CS 354 9

GLSL Standard Library Routines You’ll Need for Project 2

texture2D—accesses a 2D texture through a sampler2D and a 2-component texture coordinate set (s,t)

textureCube—access a cube map with a samplerCube and a 3-component texture coordinate set (s,t,r)

normalize—normalizes a vector cross—computes a cross product of 2 vectors dot—computes a dot (inner) product of 2 vectors max—compute the maximum of two values reflect—compute a reflection vector given an incident vector and a normal

vector vec3—constructor for 3-component vector from scalars mat3—constructor for 3x3 matrix from column vectors *—matrix-by-vector or vector-by-matrix multiplication sin—sine trigonometric function cos—cosine trigonometric function pow—raise a number to a power, exponentiation (hint: specular)

CS 354 10

Interaction withComputer Graphics

Graphics isn’t just about pretty pictures It’s also about interacting with pictures

SketchPad1963Ivan Sutherland

CS 354 11

Event Models

Polling “while loop” reads events & processes them Good for events arriving at a regular rate Appropriate when events aren’t “digital”

May require sampling analog signals

Event-driven Structured to avoid “busy waiting” Callback-driven programming

GLUT’s event loop is callback-driven

CS 354 12

Physical Events vs.Logical Events

Physical events Real-world input Examples

Key presses Physical link layer for

networking Charged-coupled

device (CCD) captures image

May require signal processing to make into a digital event

Prone to noise

Logical events Examples

Key press delivered to a particular window the mouse cusor is in

Byte stream from a network socket

Mouse clicked on an object in a 3D scene

Higher-level

CS 354 13

Routing Events

What entity is interested in an event? How is the event routed to the interested

entity? Example: window system events

Mouse events generated by mouse Button clicks and movement must go to the

correct window region Window regions arranged hierarchically

Two or more events may be connected The press & release of a mouse must be paired

CS 354 14

Model-View-Controller

Software architecture approach Separates concerns

Model

Data and logic

View

Interface / display

Controller

User input

updates modifies

user responds to view with input

modifies

CS 354 15

Interactivity

Interaction Latency Time from visual display to user action to

displayed response to user interaction Low latency provides illusion of continuous

interaction

Graphics device display frames Roughly 60 frames/second provides

continuous interaction Roughly 16.6 milliseconds Display devices tuned to this

CS 354 16

Frame Buffering

Problem with just one framebuffer Updating the buffer you are displaying results in flicker and

tearing One solution

Timing framebuffer updates to not be updating pixels about to be scanned out

Known as “racing the beam”

Solution: Two buffers One to display NOW One to render NOW so to display NEXT Ping-pong between the two buffers Alternatively, copy “draw” buffer to “display” buffer every frame Either approach needs synchornization

CS 354 17

Synchronized Buffer Swaps

Stall generate next frame until displaying the last frame

16.6 milliseconds

CS 354 18

Unsynchronized Buffers Swaps

More frame rendered and lower latency to display But tearing artifacts on displays And some frames rendered but never displayed

Wasted work16.6 milliseconds

tearing artifacts

CS 354 19

Triple Buffering Expensive Compromise

Needs more memory, now 3 buffers Renders frames that may never be displayed But avoids tearing while minimizing latency

16.6 milliseconds

CS 354 20

Complex Interactions

Dragging objects Cut-and-paste Pop-up and pull-down menus Selecting an object within a 3D scene

CS 354 21

State Machines to Managed Complex Interactions

Think of on-screen “button” You can click with the mouse How does it really logically work?

CS 354 22

Manipulators and Choosers

Open Inventor

CS 354 23

Intuitive Interactions

Make computer behave in a way that’s natural to humans NOT the other way around

Good rules of thumb Hint what the result of an interaction will be

Locate highlight buttons Change cursor shape based on its position

Provide immediate and continuous display of state Show mercy, people make mistakes, be able to undo Make manipulation as direct as possible

Examples Dragging and dropping objects Pointing to indicate interest

CS 354 24

Back Projection

User clicks on screen… Question: What object got “clicked”?

Known as “picking” in 3D applications Basic operation in 3D applications

Process Window system maps click to (x,y) in a particular window

Delivers the event to application selecting events on the window Applications gets (x,y) in window space

Walk through scene graph Invert viewport, projection, and modelview transforms

Back project point to object space ray Intersect the ray with each object in scene graph

Does the ray intersect the object? What intersected object is the closest?

Return the nearest object intersecting the ray as a “hit”

CS 354 25

Conceptual Vertex Transformation

glVertex*API

commands

Modelviewmatrix

User-definedclip planes

View-frustumclip planes

to primitiverasterization

object-space coordinates

(xo,yo,zo,wo) eye-space coordinates

(xe,ye,ze,we)

clipped eye-space coordinates

clipped clip-space coordinates Perspective

divisionProjection

matrix

Viewport + Depth Rangetransformation

(xc,yc,zc,wc)

window-spacecoordinates

(xw,yw,zw,1/wc)

normalized device coordinates (NDC)

(xn,yn,zn,1/wc)

clip-spacecoordinates

(xc,yc,zc,wc)

(xe,ye,ze,we)

(xe,ye,ze,we)

CS 354 26

Reversed Vertex Transformation

glVertex*API

commands

Modelviewmatrix

User-definedclip planes

View-frustumclip planes

to primitiverasterization

object-space coordinates

(xo,yo,zo,wo) eye-space coordinates

(xe,ye,ze,we)

clipped eye-space coordinates

clipped clip-space coordinates Perspective

divisionProjection

matrix

Viewport + Depth Rangetransformation

(xc,yc,zc,wc)

window-spacecoordinates

(xw,yw,zw,1/wc)

normalized device coordinates (NDC)

(xn,yn,zn,1/wc)

clip-spacecoordinates

(xc,yc,zc,wc)

(xe,ye,ze,we)

(xe,ye,ze,we)

(x,y) mouse event

CS 354 27

New CurrentInteraction Paradigms

Multi-touch screens Phones, tablets… work surfaces?

Perceptive Pixel

CS 354 28

Where next?

Visual, tactile, gestures, speech, highly responsive, emotionally aware Minority Report interface…

CS 354 29

Next Class

Next lecture Procedural methods Making shape, appearance, and animation from

programs—instead of data Project 2

Start learning GLSL and doing project Due Friday, April 6th

Incremental deadline on April 2nd

First two tasks