+ All Categories
Home > Documents > Author : Tzu- Chieh Yen, Chia -Ming Tsai, Chia-Wen Lin, Senior Member, IEEE

Author : Tzu- Chieh Yen, Chia -Ming Tsai, Chia-Wen Lin, Senior Member, IEEE

Date post: 02-Jan-2016
Category:
Upload: honorato-pearson
View: 39 times
Download: 1 times
Share this document with a friend
Description:
Author : Tzu- Chieh Yen, Chia -Ming Tsai, Chia-Wen Lin, Senior Member, IEEE Source : IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 8, Source : AUGUST 2011 Speaker : Jhen -Yu Yang Advisor : Guo-Shiang Lin. - PowerPoint PPT Presentation
Popular Tags:
28
Maintaining Temporal Coherence in Video Retargeting Using Mosaic-Guided Scaling Author Tzu-Chieh Yen, Chia-Ming Tsai, Chia-Wen Lin, Senior Member, IEEE Source IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 8, Source AUGUST 2011 Speaker Jhen-Yu Yang Advisor Guo-Shiang Lin
Transcript
Page 1: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

Maintaining Temporal Coherence in Video Retargeting Using Mosaic-Guided Scaling

Author: Tzu-Chieh Yen, Chia-Ming Tsai, Chia-Wen Lin, Senior Member, IEEE

Source: IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 8, Source: AUGUST 2011

Speaker: Jhen-Yu Yang

Advisor: Guo-Shiang Lin

Page 2: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

2

Outline

Introduction Methods Results Conclusion

2012/01/10

Page 3: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

3

Introduction

基於內容為主的影像畫面濃縮,目的是希望畫面經過濃縮之後能盡量維持畫面中人眼感興趣的區域,並且縮小或裁切掉人眼較不感興趣的區域,使得濃縮後的影像能呈現原始影像所要分享的訊息。

對於濃縮後的視訊如何確保畫面之間在空間軸上以及時軸上的一致性將會是影響視訊品質的關鍵。

目前現存的方法,對於同時包含有背景鏡頭移動與物體移動的影片很難在畫面濃縮後保持在時軸上的一致性。

2012/01/10

Page 4: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

4

Introduction

利用全景接圖導引的方式去決定每一個相對應區域的縮放比例,以確保畫面之間在時軸上的一致性。

提出的方法與步驟 先將影片中屬於同一個場景的每一張畫面接合產生一張全

景圖。

並根據全景圖產生一張全域縮放圖。

接著每一張畫面參考全域縮放圖上的資訊與提出的空間軸上限制作最佳化處理。

最後得到每一張畫面個別的局部縮放圖和濃縮後的畫面。

2012/01/10

Page 5: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

5

Outline

Introduction Methods Results Conclusion

2012/01/10

Page 6: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

6

Method

Overall view

2012/01/10

Page 7: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

7

Method

Mosaic-guided scaling methodFrame-level energy maps

○ Represent the visual importance of a pixel in each video frame

Initial local scaling maps○ Determine how to scale a pixel in frame

resizing

Shot-level panoramic mosaic

2012/01/10

Page 8: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

8

Method Frame-level energy map

2012/01/10

Page 9: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

9

Method

Energy fusion functionCombine the gradient energy and the

PQSM-based saliency map

2012/01/10

Page 10: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

10

Method

Initial local scaling mapsUses an energy-based frame resizing

approach

2012/01/10

Page 11: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

11

Method

Shot-level panoramic mosaic A panoramic mosaic is generated by

using three stepsFeature points detectionCamera motion estimationFrame registration

2012/01/10

Page 12: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

12

Method

Mosaic-Guided video retargetingGlobal scaling mapGlobal map constraintSpatial coherence constraintsIterative optimization procedureFrame resizing based on local maps

2012/01/10

Page 13: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

13

Method

Global scaling mapTo synchronize the scaling of a corresponding pixel

in different frames of the shot

2012/01/10

Page 14: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

14

Method

The maximum scaling factor

2012/01/10

Page 15: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

15

Method

Global map constraintThe first-round local scaling maps are

extracted by   

The first-round local scaling maps are set as

2012/01/10

Page 16: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

16

Method To ensure the temporal coherence

Use the first-round maps to constrain the iterated local scaling maps ○ by introducing the following distortion cost

2012/01/10

Page 17: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

17

Method

Spatial coherence constraintsRegions-of-Interest (ROI) Deformation

Constraint

Spatial Smoothness Constraint

2012/01/10

Page 18: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

18

Method

Regions-of-Interest (ROI) Deformation Constraint The scaling factors of the pixels of visually

important foregrounds/backgrounds should be made consistent

2012/01/10

Page 19: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

19

Method

To maintain the consistency of foreground object sizethe following spatial scaling inconsistency should

be minimized

2012/01/10

Page 20: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

20

Method

Spatial Smoothness ConstraintLimit the sum of the differences between the

scaling factors of every two vertically adjacent pixels/patches on a line not to exceed a threshold as follows

2012/01/10

Page 21: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

21

Method

Iterative optimization procedure

2012/01/10

Page 22: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

22

Method

Frame resizing based on local scaling maps Pixel fusion based image downscaling

2012/01/10

Page 23: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

23

Method

Pixel value of the resized frame is computed by

2012/01/10

Page 24: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

24

Outline

Introduction Methods Results Conclusion

2012/01/10

Page 25: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

25

Results Compared with other methods

2012/01/10

Page 26: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

26

Outline

Introduction Methods Results Discussion

2012/01/10

Page 27: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

27

Conclusion

實驗的結果顯示我們提出的視訊濃縮方法可以有效維持濃縮後的視訊畫面在時軸上的一致性,甚至對於包含有背景鏡頭移動與物體移動的影片也能有不錯的效果。

2012/01/10

Page 28: Author : Tzu- Chieh  Yen,  Chia -Ming Tsai,  Chia-Wen  Lin, Senior Member, IEEE

28

The End

Thanks

2012/01/10


Recommended