Photo squarization by deep multi-operator retargeting

Yu Song, Xiaopeng Zhang, Fan Tang, Oliver Deussen, Weiming Dong, Tong-Yee Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Squared forms of photos are widely used in social media as album covers or thumbnails of image streams. In this study, we realize photo squarization by modeling Retargeting Visual Perception Issues, which reflect human perception preference toward image ratarget-ing. General image retargeting techniques deal with three common issues, namely, salient content, object shape, and scene composition, to preserve the important information of original image. We propose a new way based on multi-operator techniques to investigate human behavior in balancing the three issues. We establish a new dataset and observe human behavior by inviting investigators to retarget images to square manually. We propose a data-driven approach composed of perception and distillation modules by using deep learning techniques to predict human perception preference. The perception part learns the relations among the three issues, and the distillation part transfers the learned relations to a simple but effective network. Our study contributes to deep learning literature by optimizing a network index and lightening its running burden. Experimental results show that photo squarization results generated by the proposed model are consistent with human visual perception results.

Original languageEnglish
Title of host publicationMM 2018 - Proceedings of the 2018 ACM Multimedia Conference
PublisherAssociation for Computing Machinery, Inc
Pages1047-1055
Number of pages9
ISBN (Electronic)9781450356657
DOIs
Publication statusPublished - 2018 Oct 15
Event26th ACM Multimedia conference, MM 2018 - Seoul, Korea, Republic of
Duration: 2018 Oct 222018 Oct 26

Publication series

NameMM 2018 - Proceedings of the 2018 ACM Multimedia Conference

Other

Other26th ACM Multimedia conference, MM 2018
CountryKorea, Republic of
CitySeoul
Period18-10-2218-10-26

Fingerprint

Distillation
Chemical analysis
Deep learning

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction

Cite this

Song, Y., Zhang, X., Tang, F., Deussen, O., Dong, W., & Lee, T-Y. (2018). Photo squarization by deep multi-operator retargeting. In MM 2018 - Proceedings of the 2018 ACM Multimedia Conference (pp. 1047-1055). (MM 2018 - Proceedings of the 2018 ACM Multimedia Conference). Association for Computing Machinery, Inc. https://doi.org/10.1145/3240508.3240623
Song, Yu ; Zhang, Xiaopeng ; Tang, Fan ; Deussen, Oliver ; Dong, Weiming ; Lee, Tong-Yee. / Photo squarization by deep multi-operator retargeting. MM 2018 - Proceedings of the 2018 ACM Multimedia Conference. Association for Computing Machinery, Inc, 2018. pp. 1047-1055 (MM 2018 - Proceedings of the 2018 ACM Multimedia Conference).
@inproceedings{af36a2100b12465a8e128585177235f6,
title = "Photo squarization by deep multi-operator retargeting",
abstract = "Squared forms of photos are widely used in social media as album covers or thumbnails of image streams. In this study, we realize photo squarization by modeling Retargeting Visual Perception Issues, which reflect human perception preference toward image ratarget-ing. General image retargeting techniques deal with three common issues, namely, salient content, object shape, and scene composition, to preserve the important information of original image. We propose a new way based on multi-operator techniques to investigate human behavior in balancing the three issues. We establish a new dataset and observe human behavior by inviting investigators to retarget images to square manually. We propose a data-driven approach composed of perception and distillation modules by using deep learning techniques to predict human perception preference. The perception part learns the relations among the three issues, and the distillation part transfers the learned relations to a simple but effective network. Our study contributes to deep learning literature by optimizing a network index and lightening its running burden. Experimental results show that photo squarization results generated by the proposed model are consistent with human visual perception results.",
author = "Yu Song and Xiaopeng Zhang and Fan Tang and Oliver Deussen and Weiming Dong and Tong-Yee Lee",
year = "2018",
month = "10",
day = "15",
doi = "10.1145/3240508.3240623",
language = "English",
series = "MM 2018 - Proceedings of the 2018 ACM Multimedia Conference",
publisher = "Association for Computing Machinery, Inc",
pages = "1047--1055",
booktitle = "MM 2018 - Proceedings of the 2018 ACM Multimedia Conference",

}

Song, Y, Zhang, X, Tang, F, Deussen, O, Dong, W & Lee, T-Y 2018, Photo squarization by deep multi-operator retargeting. in MM 2018 - Proceedings of the 2018 ACM Multimedia Conference. MM 2018 - Proceedings of the 2018 ACM Multimedia Conference, Association for Computing Machinery, Inc, pp. 1047-1055, 26th ACM Multimedia conference, MM 2018, Seoul, Korea, Republic of, 18-10-22. https://doi.org/10.1145/3240508.3240623

Photo squarization by deep multi-operator retargeting. / Song, Yu; Zhang, Xiaopeng; Tang, Fan; Deussen, Oliver; Dong, Weiming; Lee, Tong-Yee.

MM 2018 - Proceedings of the 2018 ACM Multimedia Conference. Association for Computing Machinery, Inc, 2018. p. 1047-1055 (MM 2018 - Proceedings of the 2018 ACM Multimedia Conference).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Photo squarization by deep multi-operator retargeting

AU - Song, Yu

AU - Zhang, Xiaopeng

AU - Tang, Fan

AU - Deussen, Oliver

AU - Dong, Weiming

AU - Lee, Tong-Yee

PY - 2018/10/15

Y1 - 2018/10/15

N2 - Squared forms of photos are widely used in social media as album covers or thumbnails of image streams. In this study, we realize photo squarization by modeling Retargeting Visual Perception Issues, which reflect human perception preference toward image ratarget-ing. General image retargeting techniques deal with three common issues, namely, salient content, object shape, and scene composition, to preserve the important information of original image. We propose a new way based on multi-operator techniques to investigate human behavior in balancing the three issues. We establish a new dataset and observe human behavior by inviting investigators to retarget images to square manually. We propose a data-driven approach composed of perception and distillation modules by using deep learning techniques to predict human perception preference. The perception part learns the relations among the three issues, and the distillation part transfers the learned relations to a simple but effective network. Our study contributes to deep learning literature by optimizing a network index and lightening its running burden. Experimental results show that photo squarization results generated by the proposed model are consistent with human visual perception results.

AB - Squared forms of photos are widely used in social media as album covers or thumbnails of image streams. In this study, we realize photo squarization by modeling Retargeting Visual Perception Issues, which reflect human perception preference toward image ratarget-ing. General image retargeting techniques deal with three common issues, namely, salient content, object shape, and scene composition, to preserve the important information of original image. We propose a new way based on multi-operator techniques to investigate human behavior in balancing the three issues. We establish a new dataset and observe human behavior by inviting investigators to retarget images to square manually. We propose a data-driven approach composed of perception and distillation modules by using deep learning techniques to predict human perception preference. The perception part learns the relations among the three issues, and the distillation part transfers the learned relations to a simple but effective network. Our study contributes to deep learning literature by optimizing a network index and lightening its running burden. Experimental results show that photo squarization results generated by the proposed model are consistent with human visual perception results.

UR - http://www.scopus.com/inward/record.url?scp=85058246965&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058246965&partnerID=8YFLogxK

U2 - 10.1145/3240508.3240623

DO - 10.1145/3240508.3240623

M3 - Conference contribution

AN - SCOPUS:85058246965

T3 - MM 2018 - Proceedings of the 2018 ACM Multimedia Conference

SP - 1047

EP - 1055

BT - MM 2018 - Proceedings of the 2018 ACM Multimedia Conference

PB - Association for Computing Machinery, Inc

ER -

Song Y, Zhang X, Tang F, Deussen O, Dong W, Lee T-Y. Photo squarization by deep multi-operator retargeting. In MM 2018 - Proceedings of the 2018 ACM Multimedia Conference. Association for Computing Machinery, Inc. 2018. p. 1047-1055. (MM 2018 - Proceedings of the 2018 ACM Multimedia Conference). https://doi.org/10.1145/3240508.3240623