Photo squarization by deep multi-operator retargeting

Yu Song, Xiaopeng Zhang, Fan Tang, Oliver Deussen, Weiming Dong, Tong Yee Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)


Squared forms of photos are widely used in social media as album covers or thumbnails of image streams. In this study, we realize photo squarization by modeling Retargeting Visual Perception Issues, which reflect human perception preference toward image ratarget-ing. General image retargeting techniques deal with three common issues, namely, salient content, object shape, and scene composition, to preserve the important information of original image. We propose a new way based on multi-operator techniques to investigate human behavior in balancing the three issues. We establish a new dataset and observe human behavior by inviting investigators to retarget images to square manually. We propose a data-driven approach composed of perception and distillation modules by using deep learning techniques to predict human perception preference. The perception part learns the relations among the three issues, and the distillation part transfers the learned relations to a simple but effective network. Our study contributes to deep learning literature by optimizing a network index and lightening its running burden. Experimental results show that photo squarization results generated by the proposed model are consistent with human visual perception results.

Original languageEnglish
Title of host publicationMM 2018 - Proceedings of the 2018 ACM Multimedia Conference
PublisherAssociation for Computing Machinery, Inc
Number of pages9
ISBN (Electronic)9781450356657
Publication statusPublished - 2018 Oct 15
Event26th ACM Multimedia conference, MM 2018 - Seoul, Korea, Republic of
Duration: 2018 Oct 222018 Oct 26

Publication series

NameMM 2018 - Proceedings of the 2018 ACM Multimedia Conference


Other26th ACM Multimedia conference, MM 2018
Country/TerritoryKorea, Republic of

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction


Dive into the research topics of 'Photo squarization by deep multi-operator retargeting'. Together they form a unique fingerprint.

Cite this