Post on 02-Jan-2016
description
transcript
Maintaining Temporal Coherence in Video Retargeting Using Mosaic-Guided Scaling
Author： Tzu-Chieh Yen, Chia-Ming Tsai, Chia-Wen Lin, Senior Member, IEEE
Source： IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 8, Source： AUGUST 2011
Speaker： Jhen-Yu Yang
Advisor： Guo-Shiang Lin
2
Outline
Introduction Methods Results Conclusion
2012/01/10
3
Introduction
基於內容為主的影像畫面濃縮，目的是希望畫面經過濃縮之後能盡量維持畫面中人眼感興趣的區域，並且縮小或裁切掉人眼較不感興趣的區域，使得濃縮後的影像能呈現原始影像所要分享的訊息。
對於濃縮後的視訊如何確保畫面之間在空間軸上以及時軸上的一致性將會是影響視訊品質的關鍵。
目前現存的方法，對於同時包含有背景鏡頭移動與物體移動的影片很難在畫面濃縮後保持在時軸上的一致性。
2012/01/10
4
Introduction
利用全景接圖導引的方式去決定每一個相對應區域的縮放比例，以確保畫面之間在時軸上的一致性。
提出的方法與步驟 先將影片中屬於同一個場景的每一張畫面接合產生一張全
景圖。
並根據全景圖產生一張全域縮放圖。
接著每一張畫面參考全域縮放圖上的資訊與提出的空間軸上限制作最佳化處理。
最後得到每一張畫面個別的局部縮放圖和濃縮後的畫面。
2012/01/10
5
Outline
Introduction Methods Results Conclusion
2012/01/10
6
Method
Overall view
2012/01/10
7
Method
Mosaic-guided scaling methodFrame-level energy maps
○ Represent the visual importance of a pixel in each video frame
Initial local scaling maps○ Determine how to scale a pixel in frame
resizing
Shot-level panoramic mosaic
2012/01/10
8
Method Frame-level energy map
2012/01/10
9
Method
Energy fusion functionCombine the gradient energy and the
PQSM-based saliency map
2012/01/10
10
Method
Initial local scaling mapsUses an energy-based frame resizing
approach
2012/01/10
11
Method
Shot-level panoramic mosaic A panoramic mosaic is generated by
using three stepsFeature points detectionCamera motion estimationFrame registration
2012/01/10
12
Method
Mosaic-Guided video retargetingGlobal scaling mapGlobal map constraintSpatial coherence constraintsIterative optimization procedureFrame resizing based on local maps
2012/01/10
13
Method
Global scaling mapTo synchronize the scaling of a corresponding pixel
in different frames of the shot
2012/01/10
14
Method
The maximum scaling factor
2012/01/10
15
Method
Global map constraintThe first-round local scaling maps are
extracted by
The first-round local scaling maps are set as
2012/01/10
16
Method To ensure the temporal coherence
Use the first-round maps to constrain the iterated local scaling maps ○ by introducing the following distortion cost
2012/01/10
17
Method
Spatial coherence constraintsRegions-of-Interest (ROI) Deformation
Constraint
Spatial Smoothness Constraint
2012/01/10
18
Method
Regions-of-Interest (ROI) Deformation Constraint The scaling factors of the pixels of visually
important foregrounds/backgrounds should be made consistent
2012/01/10
19
Method
To maintain the consistency of foreground object sizethe following spatial scaling inconsistency should
be minimized
2012/01/10
20
Method
Spatial Smoothness ConstraintLimit the sum of the differences between the
scaling factors of every two vertically adjacent pixels/patches on a line not to exceed a threshold as follows
2012/01/10
21
Method
Iterative optimization procedure
2012/01/10
22
Method
Frame resizing based on local scaling maps Pixel fusion based image downscaling
2012/01/10
23
Method
Pixel value of the resized frame is computed by
2012/01/10
24
Outline
Introduction Methods Results Conclusion
2012/01/10
25
Results Compared with other methods
2012/01/10
26
Outline
Introduction Methods Results Discussion
2012/01/10
27
Conclusion
實驗的結果顯示我們提出的視訊濃縮方法可以有效維持濃縮後的視訊畫面在時軸上的一致性，甚至對於包含有背景鏡頭移動與物體移動的影片也能有不錯的效果。
2012/01/10
28
The End
Thanks
2012/01/10