Motion-aware temporal coherence for video resizing

Yu Shuen Wang, Hongbo Fu, Olga Sorkine, Tong Yee Lee, Hans Peter Seidel

研究成果: Conference contribution

101 引文 (Scopus)

摘要

Temporal coherence is crucial in content-aware video retargeting. To date, this problem has been addressed by constraining temporally adjacent pixels to be transformed coherently. However, due to the motion-oblivious nature of this simple constraint, the retargeted videos often exhibit flickering or waving artifacts, especially when significant camera or object motions are involved. Since the feature correspondence across frames varies spatially with both camera and object motion, motion-aware treatment of features is required for video resizing. This motivated us to align consecutive frames by estimating interframe camera motion and to constrain relative positions in the aligned frames. To preserve object motion, we detect distinct moving areas of objects across multiple frames and constrain each of them to be resized consistently. We build a complete video resizing framework by incorporating our motion-aware constraints with an adaptation of the scale-and-stretch optimization recently proposed by Wang and colleagues. Our streaming implementation of the framework allows efficient resizing of long video sequences with low memory cost. Experiments demonstrate that our method produces spatiotemporally coherent retargeting results even for challenging examples with complex camera and object motion, which are difficult to handle with previous techniques.

原文English
主出版物標題Proceedings of ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09
頁面127:1-127:10
28
版本5
DOIs
出版狀態Published - 2009 十二月 1
事件ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09 - Yokohama, Japan
持續時間: 2009 十二月 162009 十二月 19

Other

OtherACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09
國家Japan
城市Yokohama
期間09-12-1609-12-19

指紋

Cameras
Flickering
Pixels
Data storage equipment
Costs
Experiments

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design

引用此文

Wang, Y. S., Fu, H., Sorkine, O., Lee, T. Y., & Seidel, H. P. (2009). Motion-aware temporal coherence for video resizing. 於 Proceedings of ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09 (5 編輯, 卷 28, 頁 127:1-127:10) https://doi.org/10.1145/1661412.1618473
Wang, Yu Shuen ; Fu, Hongbo ; Sorkine, Olga ; Lee, Tong Yee ; Seidel, Hans Peter. / Motion-aware temporal coherence for video resizing. Proceedings of ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09. 卷 28 5. 編輯 2009. 頁 127:1-127:10
@inproceedings{544dfdb138f542049c8a0a832773ce66,
title = "Motion-aware temporal coherence for video resizing",
abstract = "Temporal coherence is crucial in content-aware video retargeting. To date, this problem has been addressed by constraining temporally adjacent pixels to be transformed coherently. However, due to the motion-oblivious nature of this simple constraint, the retargeted videos often exhibit flickering or waving artifacts, especially when significant camera or object motions are involved. Since the feature correspondence across frames varies spatially with both camera and object motion, motion-aware treatment of features is required for video resizing. This motivated us to align consecutive frames by estimating interframe camera motion and to constrain relative positions in the aligned frames. To preserve object motion, we detect distinct moving areas of objects across multiple frames and constrain each of them to be resized consistently. We build a complete video resizing framework by incorporating our motion-aware constraints with an adaptation of the scale-and-stretch optimization recently proposed by Wang and colleagues. Our streaming implementation of the framework allows efficient resizing of long video sequences with low memory cost. Experiments demonstrate that our method produces spatiotemporally coherent retargeting results even for challenging examples with complex camera and object motion, which are difficult to handle with previous techniques.",
author = "Wang, {Yu Shuen} and Hongbo Fu and Olga Sorkine and Lee, {Tong Yee} and Seidel, {Hans Peter}",
year = "2009",
month = "12",
day = "1",
doi = "10.1145/1661412.1618473",
language = "English",
isbn = "9781605588582",
volume = "28",
pages = "127:1--127:10",
booktitle = "Proceedings of ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09",
edition = "5",

}

Wang, YS, Fu, H, Sorkine, O, Lee, TY & Seidel, HP 2009, Motion-aware temporal coherence for video resizing. 於 Proceedings of ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09. 5 edn, 卷 28, 頁 127:1-127:10, ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09, Yokohama, Japan, 09-12-16. https://doi.org/10.1145/1661412.1618473

Motion-aware temporal coherence for video resizing. / Wang, Yu Shuen; Fu, Hongbo; Sorkine, Olga; Lee, Tong Yee; Seidel, Hans Peter.

Proceedings of ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09. 卷 28 5. 編輯 2009. p. 127:1-127:10.

研究成果: Conference contribution

TY - GEN

T1 - Motion-aware temporal coherence for video resizing

AU - Wang, Yu Shuen

AU - Fu, Hongbo

AU - Sorkine, Olga

AU - Lee, Tong Yee

AU - Seidel, Hans Peter

PY - 2009/12/1

Y1 - 2009/12/1

N2 - Temporal coherence is crucial in content-aware video retargeting. To date, this problem has been addressed by constraining temporally adjacent pixels to be transformed coherently. However, due to the motion-oblivious nature of this simple constraint, the retargeted videos often exhibit flickering or waving artifacts, especially when significant camera or object motions are involved. Since the feature correspondence across frames varies spatially with both camera and object motion, motion-aware treatment of features is required for video resizing. This motivated us to align consecutive frames by estimating interframe camera motion and to constrain relative positions in the aligned frames. To preserve object motion, we detect distinct moving areas of objects across multiple frames and constrain each of them to be resized consistently. We build a complete video resizing framework by incorporating our motion-aware constraints with an adaptation of the scale-and-stretch optimization recently proposed by Wang and colleagues. Our streaming implementation of the framework allows efficient resizing of long video sequences with low memory cost. Experiments demonstrate that our method produces spatiotemporally coherent retargeting results even for challenging examples with complex camera and object motion, which are difficult to handle with previous techniques.

AB - Temporal coherence is crucial in content-aware video retargeting. To date, this problem has been addressed by constraining temporally adjacent pixels to be transformed coherently. However, due to the motion-oblivious nature of this simple constraint, the retargeted videos often exhibit flickering or waving artifacts, especially when significant camera or object motions are involved. Since the feature correspondence across frames varies spatially with both camera and object motion, motion-aware treatment of features is required for video resizing. This motivated us to align consecutive frames by estimating interframe camera motion and to constrain relative positions in the aligned frames. To preserve object motion, we detect distinct moving areas of objects across multiple frames and constrain each of them to be resized consistently. We build a complete video resizing framework by incorporating our motion-aware constraints with an adaptation of the scale-and-stretch optimization recently proposed by Wang and colleagues. Our streaming implementation of the framework allows efficient resizing of long video sequences with low memory cost. Experiments demonstrate that our method produces spatiotemporally coherent retargeting results even for challenging examples with complex camera and object motion, which are difficult to handle with previous techniques.

UR - http://www.scopus.com/inward/record.url?scp=77749264947&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77749264947&partnerID=8YFLogxK

U2 - 10.1145/1661412.1618473

DO - 10.1145/1661412.1618473

M3 - Conference contribution

AN - SCOPUS:77749264947

SN - 9781605588582

VL - 28

SP - 127:1-127:10

BT - Proceedings of ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09

ER -

Wang YS, Fu H, Sorkine O, Lee TY, Seidel HP. Motion-aware temporal coherence for video resizing. 於 Proceedings of ACM SIGGRAPH Asia 2009, SIGGRAPH Asia '09. 5 編輯 卷 28. 2009. p. 127:1-127:10 https://doi.org/10.1145/1661412.1618473