IMGD 4000: Hot Topics in Graphics
Prof Emmanuel Agu
Computer Science Dept.Worcester Polytechnic Institute (WPI)
Graphics Processing Unit (GPU)
Entire graphics library (OpenGL, DirectX) in hardware => FAST!! Speed: GPUs renders graphics faster CPUs Programmable: in last 8 years (Shaders)
Located either on PC motherboard (Intel) or Separate graphics card (Nvidia or ATI)
On PC motherboard On separate PCI express card
GPU Evolution
High throughput computation GeForce GTX 280: 933 GFLOP/s
High bandwidth memory GeForce GTX 280: 140 GB/s
High availability to all 180+ million shader‐capable GPUs in the wild
Numbers doubling every 6 months (Moore’s law cubed)
1995 2000 2005 2010
RIVA 1283M xtors
GeForce® 25623M xtors
GeForce FX125M xtors
GeForce 8800681M xtors
GeForce 3 60M xtors
“Fermi”3B xtors
Slide from Stanford course 193G 2010
DesignTradeoff makes GPU Fast Graphics has many calculations, few conditional (if/else, loops) Arithmetic Logic Unit (ALU) is part of GPU for calculations Tradeoff: Assign
More transistors for ALU: faster rendering calculations, math Fewer transistors for control, branches, loops (graphics rarely uses them) ALU chip area: CPU (7%) vs GPU ( > 50%)
Result: GPU runs graphics much faster than CPU
GFLOP/sec growth
GPU!!
CPU!!
GPU Memory Bandwidth
Graph from: CUDA Programming Guide 4.2
GPU!!
CPU!!
146X
Medical Imaging
U of Utah
36X
Molecular Dynamics
U of Illinois, Urbana
18X
Video Transcoding
Elemental Tech
50X
Matlab Computing
AccelerEyes
100X
AstrophysicsRIKEN
149X
Financial simulation
Oxford
47X
Linear AlgebraUniversidad
Jaime
20X
3D UltrasoundTechniscan
130X
Quantum Chemistry
U of Illinois, Urbana
30X
Gene Sequencing
U of Maryland
Aside: GPGPU: GPUs to speedup Non‐Graphics Applications
Trend 1: Mobile GPUs GPUs on Phones, tablets now powerful Mobile GPUs support OpenGL ES OpenGL ES: Subset of OpenGL, shader‐based
Trend 1: Mobile GPUs
Many game titles now ship only smartphones only, no consoles
Unity: Cross Platform Shader Support
Artist specifies high‐level shader on the Material ex: "Bumped Specular", "Tree Leaves", "Unlit“
Run‐time picks specific platform shaderdepending on supported feature set
Testing Mobile Apps: Challenging
Animoca, android mobile developer has to test each app on over 400 different smartphones and tablets Screens Aspect ratios Form factors Input controls OS versions CPU/GPU OpenGL/DirectX
versions……. etc
Recap: What is Graphics?
Trend 2: Capture light, materials, geometry, for rendering data from real world
1. Light
2. Geometry3. Materials
4. Light/Material Interaction(Lighting model)
Next: capturing light
Photograph silver ball in environment High dynamic range
Capturing Light: Light Probes
Using Light Probes Example
Trend 2: Capture rendering data
1. Light
2. Geometry3. Materials
4. Light/Material Interaction(Lighting model)
Next: capturing geometry
Trend 2: Capture Geometry Digitize real object geometry, billions of faces High precision databases of models: shared by many Better realism than packages such as maya Important for sports games?
Mesh
Trend 2: Capture rendering data
1. Light
2. Geometry3. Materials
4. Light/Material Interaction(Lighting model)
Next: capturing materials
Measuring BRDFs: Gonioreflectometer
Murray‐Coleman and Smith Gonioreflectometer. ( Copied and Modified from [Ward92] ).
Measured BRDF Samples BRDF: How materials reflect light Examples: cloth, wood, velvet, etc Mitsubishi Electric Research Lab (MERL)http://www.merl.com/brdf/ Wojciech Matusik MIT PhD Thesis 100 Samples
Capture Time‐Varying BRDFs
Time varying?: surface reflectance changes over time Examples: weathering, ripening fruits, rust, etc
Exactly What Can We Capture?1. Appearance (volume, scattering, transparency, translucency, etc)
2. Geometry
3. Reflectance & Illumination4. Motion
Trend 3: Real‐Time LoD Geometry Management
Previously: Pre‐generate mesh versions offline Geometry shader unit added to GPU in DirectX 10 (2007)
generate new vertices, primitives from mesh
Mesh simplification/tesselation on GPU = Real time LoD Tesselation: Demo
tesselation
Simplification
Far = Less detailed mesh
Near = More detailed mesh
Textures in Games
Most games loaded with textures Computationally cheap way to look pretty
Trend 5: New Directions in Texture Synthesis
Texture = picture or algorithmically rendered Internet (e.g. Flickr) has billions of pictures Example‐Based Texture Synthesis: New images from samples Input: small (photographic) textures Output: produce arbitrarily large texture Approach: discover pattern, extrapolate algorithm, math
input
output
SynthesisAlgorithm
Pixel‐based Algorithm
Random Oriented
Regular Semi-regular
Texture Synthesis on Surfaces
Sample
Multi‐Level Texture Synthesis
Coarse1,000 vertices
Medium4,000 vertices
Fine16,000 vertices
Sample:
Example‐Based Model SynthesisPaul Merrell, I3D 2007
Input: Polygon modelsApproach: Partition models into pieces Place pieces in new arrangement subject to constraints
Partitions RandomPlacement
ConsistentPlacement
Example‐Based Model Synthesis
Example‐Based Model Synthesis
Input
Result
Texture Replacement
Lots of new work around algorithmically altering images (found on web?)
Goal: Alter, remove or replace part of a image algorithmically
Artifact Removal
Detail Hallucination
Alcatraz
Scene Completion using Millions of Photographs
Seam Carving
Local Illumination Consider 1 bounce only OpenGL does this Looks fake E.g. Torque 3D game engine
Global Illumination Global illumination: model interaction of light from all surfaces in scene (track multiple bounces)
translucent surface
shadow
multiple reflection
Trend 5: New Real time Global Illumination Algos
Dynamic scenes: Need real‐time GI, on‐the‐fly Real Time Global Illumination: state‐of‐the art
Calculate GI equations on GPU at run‐time
Improves: Shadows Ambient Occlusion Reflections Transmittance Refractions Caustics
What does RT‐GI look like?
Real‐time Lightingin Games
Better Sky Model
[BrunetonNeyret2008] Single and multiple scattering Light shafts Participating media!!
Different Atmospheres
Time Of Day
Better Shadows
Courtesy Hellgate:London, flagship studios incVariance shadow mapping
Courtesy Nvidia SDK 10
Better Caustics and Refraction
Courtesy Chris Wyman, Univ Iowa
Sub‐Surface Scattering
Marble
Human Skin
Crysis skin demo
Sub‐Surface Scattering
Light Propagation Volumes (LPV) CryEngine: new physically‐based game engine by CryTek LPV: new real time Global Illumination algorithm in CryEngine Runs faster than ray tracing on GPU
Crytek Crisis Engine Screenshots
Demo
Light Propagation Volumes Demo
LPV IdeaMain idea: represent light propagation as Virtual Point Lights (VPL) Re‐project VPL into adjacent cells
References Akenine Moller et al, Real‐Time Rendering, 3rd edition Advances in Real‐Time Rendering in 3D graphics and
games, SIGGRAPH course notes 2009 Anton Kaplanyan and Carsten Dachbacher, Cascaded light
propagation volumes for real‐time indirect illumination, in Proc. Si3D 2010
Hao Chen and Natalya Tatarchuk, Lighting Research at Bungie, Advances in Real‐Time Rendering in 3D Graphics and Games SIGGRAPH 2009 Course notes
Advances in Real‐Time Rendering in 3D Graphics and Games, SIGGRAPH 2010 Course Notes
CS 543/563 slides