Date post: | 17-May-2015 |
Category: |
Technology |
Upload: | nirmala-last |
View: | 1,500 times |
Download: | 3 times |
Using Photographs to Enhance Videos of a Static
ScenePravin Bhat1, C. Lawrence Zitnick2, Noah Snavely1, Aseem
Agarwala3, Maneesh Agrawala4, Michael Cohen1,2, Brian Curless1, Sing Bing Kang2
EGSR 2007
University of Washington1, Microsoft Research Redmond2 University of California3, Adobe Systems4
An overview of Spacetime Fusion
Motivation
• Low quality video
Input Video
Motivation
• Low quality video• Reconstructed video
Input Video
Reconstructed Video
Motivation
• Low quality video• Reconstructed video
– Reconstructed from photos– Good spatial reconstruction– Bad temporal reconstruction
Input Video
Reconstructed Video
Motivation
• Spacetime Fusion result
Input Video
Spacetime Fusion Result
Motivation
• Spacetime Fusion result– Spatial properties of reconstruction– Temporal properties of input video
Input Video
Spacetime Fusion Result
• Define a 3D gradient field
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
Video frame: t Video frame: t - 1
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
Video frame: t Video frame: t - 1
Gt
Gt(x, y, t) = V(x, y, t) - V(x, y, t - 1)
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
Video frame: t Video frame: t - 1
Gt
Gt(x, y, t) = V(x, y, t) - V(x - u, y - v, t - 1)
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
• Increases compatibility betweentemporal gradients and spatial gradients
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
• Increases compatibility betweentemporal gradients and spatial gradients
• Integrate the 3D gradient field
Spacetime Fusion
Spacetime Fusion
• Integrating the gradient field
Solve linear system:Av = b
• Integrating the gradient field
Solve linear system:Av = b
Constraints:vx, y, t – vx-1, y, t = Gx(x, y, t)vx, y, t – vx, y-1, t = Gy(x, y, t)vx, y, t – vx-u, y-v, t = Gt(x, y, t)
Spacetime Fusion
Applications
Enhanced Exposure
Input Video
Edit Propagation
Edit Propagation
User Edits
Edit Propagation
User Edits
Edit Propagation
User Edits
Edit Propagation
User Edits
Edit Propagation
User Edits
User Edits
Edit Propagation
Edited Video
Edit Propagation
Super-Resolution
Conclusion
• Spacetime fusion
Conclusion
• Spacetime fusion – Combines spatial and temporal gradients
from two different sources
Conclusion
• Spacetime fusion – Combines spatial and temporal gradients
from two different sources– Requires motion vectors for temporal
source• stereo (static scenes)• flow (dynamic scenes)
Conclusion
• Spacetime fusion – Combines spatial and temporal gradients
from two different sources– Requires motion vectors for temporal
source• stereo (static scenes)• flow (dynamic scenes)
– Major applications• Enforcing temporal coherence• Transferring lighting information