Date post: | 20-Jan-2016 |
Category: |
Documents |
Upload: | blake-ball |
View: | 216 times |
Download: | 2 times |
Pixmotor:A Pixel Motion Integrator
Pixmotor:A Pixel Motion Integrator
Ivan NeulanderRhythm & Hues
What is Pixmotor?What is Pixmotor?
A 2D motion blur solution at Rhythm & Hues
• Makes motion-blurred images based on static images generated by 3D renderer
Night At The Museum
Garfield: A Tail of Two Kitties
What is Pixmotor?What is Pixmotor?
A 2D motion blur solution at Rhythm & Hues
• Makes motion-blurred images based on static images generated by 3D renderer
• Not a physically correct but plausible
– Incorporates heuristics to minimize artifacts
Night At The Museum
Garfield: A Tail of Two Kitties
What is Pixmotor?What is Pixmotor?
A 2D motion blur solution at Rhythm & Hues
• Makes motion-blurred images based on static images generated by 3D renderer
• Not a physically correct but plausible
– Incorporates heuristics to minimize artifacts
• Fits well into our production pipeline
– Render objects separately, combine in comp
– Zero-centered shutter minimizesinterpolation distance from static frame
Night At The Museum
Garfield: A Tail of Two Kitties
What is Pixmotor?What is Pixmotor?
A 2D motion blur solution at Rhythm & Hues
• Makes motion-blurred images based on static images generated by 3D renderer
• Not a physically correct but plausible
– Incorporates heuristics to minimize artifacts
• Fits well into our production pipeline
– Render objects separately, combine in comp
– Zero-centered shutter minimizesinterpolation distance from static frame
Night At The Museum
Garfield: A Tail of Two Kitties
MotivationMotivation
• Processing Speed
– Invariant of geometric complexity: Typical 2k element completes in under 1 minute
– Many speed/quality controls for interactive testing
MotivationMotivation
• Processing Speed
– Invariant of geometric complexity: Typical 2k element completes in under 1 minute
– Many speed/quality controls for interactive testing
• Art Direction
– Tweak shutter per-frame without rerendering
– Vary motion blur within a frame
shuttershutter/2 shutter x 2 shutter x 3
MotivationMotivation
• In-Compositor Shading
– For static frames:
• Renderer outputs images ofnormals, positions, etc.;compositor does shading
Diffuse Reflection Shader
Normals Output
MotivationMotivation
• In-Compositor Shading
– For static frames:
• Renderer outputs images ofnormals, positions, etc.;compositor does shading
– For motion-blurred frames:
• Motion blurring data imagesin renderer is incorrect
Ground truth
Diffuse Reflection Shader
Normals Output
MotivationMotivation
• In-Compositor Shading
– For static frames:
• Renderer outputs images ofnormals, positions, etc.;compositor does shading
– For motion-blurred frames:
• Motion blurring data imagesin renderer is incorrect
• Using pixmotor allows doingshading on static images Ground truth
Diffuse Reflection Shader
Normals Output
Pixmotor
Pixmotor Data FlowPixmotor Data Flow
Pixmotor
Color Input
Motion Input
Depth Input
Settings
Computing Motion VectorsComputing Motion Vectors
• When shading a triangle at barycentric point (,,):Compute NDC motion vector as
PNDC(,,){ShutterClose} - PNDC(,,){ShutterOpen}
Shutter Open Shading Moment Shutter Close
Implementation:OverviewImplementation:Overview
• Discretize motion into a set of slices
– Each slice = particular moment within shutter span
– Total number of slices depends on maximum motion vector
• User sets maximum pixel separation between slices
– Compute each slice and add it to accumulation buffer
Implementation:OverviewImplementation:Overview
• Discretize motion into a set of slices
– Each slice = particular moment within shutter span
– Total number of slices depends on maximum motion vector
• User sets maximum pixel separation between slices
– Compute each slice and add it to accumulation buffer
1 slice 3 slices 7 slices 52 slices
Implementation:Main LoopImplementation:Main Loop
• For each slice:
– Copy each pixel from source image to a work buffer, shifting its coordinates based on motion vector
Implementation:Main LoopImplementation:Main Loop
• For each slice:
– Copy each pixel from source image to a work buffer, shifting its coordinates based on motion vector
• Resolve occlusion for overlappingpixels using depth info
Implementation:Main LoopImplementation:Main Loop
• For each slice:
– Copy each pixel from source image to a work buffer, shifting its coordinates based on motion vector
• Resolve occlusion for overlappingpixels using depth info
• Holes form due to divergent motion Use hole-filling heuristics
Implementation:Main LoopImplementation:Main Loop
• For each slice:
– Copy each pixel from source image to a work buffer, shifting its coordinates based on motion vector
• Resolve occlusion for overlappingpixels using depth info
• Holes form due to divergent motion Use hole-filling heuristics
Static Image Single-res WBno hole fill
Double-res WB,no hole fill
Double-res WB,with hole fill
Pixmotor
Single-slice interpolation
Implementation:Hole-Filling HeuristicsImplementation:Hole-Filling Heuristics
• Before Main Loop:
– Blur motion image (optionally)
• Filter kernel must ignore empty pixels
Implementation:Hole-Filling HeuristicsImplementation:Hole-Filling Heuristics
• Before Main Loop:
– Blur motion image (optionally)
• Filter kernel must ignore empty pixels
– Compute motion image gradient
• Force high gradient at image edges
Implementation:Hole-Filling HeuristicsImplementation:Hole-Filling Heuristics
• Before Main Loop:
– Blur motion image (optionally)
• Filter kernel must ignore empty pixels
– Compute motion image gradient
• Force high gradient at image edges
Blur = 0 Blur = 3 pixels Blur = 10 pixels Blur = 60 pixels
Motion Gradient
Result
Implementation:Hole-Filling HeuristicsImplementation:Hole-Filling Heuristics
• During Main Loop:
– Identify holes as empty pixels near high-gradient regions
– Fill holes with nearby solid neighbors along motion path
Implementation:Hole-Filling HeuristicsImplementation:Hole-Filling Heuristics
• During Main Loop:
– Identify holes as empty pixels near high-gradient regions
– Fill holes with nearby solid neighbors along motion path
– Solid pixels at image edges are always presumed holes
Implementation:Hole-Filling HeuristicsImplementation:Hole-Filling Heuristics
• During Main Loop:
– Identify holes as empty pixels near high-gradient regions
– Fill holes with nearby solid neighbors along motion path
– Solid pixels at image edges are always presumed holes
StaticImage
MotionGradient
Hole Fill: off
Hole Fill: on
[demo]
Implementation:Hole-Filling HeuristicsImplementation:Hole-Filling Heuristics
• After Main Loop:
– Option to fix matte for remaining holes:
Unpremultiply partial pixels with solid neighbors(distance along motion vector times user constant)
• Helps create a solid matte where needed for compositingBut: can harden some soft edges
Implementation:Hole-Filling HeuristicsImplementation:Hole-Filling Heuristics
• After Main Loop:
– Option to fix matte for remaining holes:
Unpremultiply partial pixels with solid neighbors(distance along motion vector times user constant)
• Helps create a solid matte where needed for compositingBut: can harden some soft edges
FixMatte: 0.0
FixMatte: 1.0
ResultsResults
• No overlap or transparency: excellent
3D Motion Blur
Pixmotor
ResultsResults
• No overlap or transparency: excellent
• With overlap or transparency: not so good
3D Motion Blur
Pixmotor
Results: Linear 3D MB (easy case)Results: Linear 3D MB (easy case)
Results:Pixmotor (easy case)Results:Pixmotor (easy case)
Results:Linear 3D MB (hard case)Results:Linear 3D MB (hard case)
Results:Pixmotor (hard case)Results:Pixmotor (hard case)
Motion-Blurred Shadows:Our ApproachMotion-Blurred Shadows:Our Approach
• Need to store partial occlusion in depth map use matte channel
Matte Depth
Motion-Blurred Shadows:Our ApproachMotion-Blurred Shadows:Our Approach
• Need to store partial occlusion in depth map use matte channel
• Process depth, matte channels using Pixmotor
Pixmotor
Matte Depth
Motion
Motion-Blurred Shadows:Our ApproachMotion-Blurred Shadows:Our Approach
• Need to store partial occlusion in depth map use matte channel
• Process depth, matte channels using Pixmotor
• Use unpremultiplied values for depth
• Depth map camera settings are unmodified
Pixmotor
Matte Depth
Motion
Matte Depth
Motion-Blurred Shadows:ResultsMotion-Blurred Shadows:Results
Motion-Blurred Shadows:ResultsMotion-Blurred Shadows:Results
Frustrations & Future WorkFrustrations & Future Work
• Transparency, occlusion artifacts
use more fragments per pixelBut: speed will suffer
• Nonlinear motion (e.g. propeller)
Use acceleration + velocity for motion
• Memory-hungry on deep inputs
Process channels in batchesBut: counteracts multithreading
• Apply to reflections, refractions
– Compute motion vectors for these
Demo VideoDemo Video
• Real-time capture of interactive Icy session on aquad-CPU Opteron 2200
demo
Thank YouThank You
• Jonathan Robbins
• Martin Ryan
www.rhythm.com/~ivan/pdfs/sketch2007.pdfwww.rhythm.com/~ivan/pub/sketch2007.ppt