TY - JOUR
T1 - Measuring and Predicting Visual Importance of Similar Objects
AU - Kong, Yan
AU - Dong, Weiming
AU - Mei, Xing
AU - Ma, Chongyang
AU - Lee, Tong Yee
AU - Lyu, Siwei
AU - Huang, Feiyue
AU - Zhang, Xiaopeng
N1 - Funding Information:
This work was supported by National Natural Science Foundation of China under nos. 61271430, 61372184, 61331018, and 61471359, by Supporting Program for Sci & Tech Research of China under No. 2015BAH53F00, and by CASIA-Tencent BestImage joint research project. The work of Tong-Yee Lee was supported by the Ministry of Science and Technology with contract nos. MOST-103-2221- E-006-106-MY3 and MOST-104-2221-E-006-044-MY3, Taiwan. The work of Siwei Lyu was supported by National Science Foundation under nos. IIS-0953373 and CCF-1319800. Y. Kong and W. Dong contributed equally to this work.
PY - 2016/12/1
Y1 - 2016/12/1
N2 - Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5,922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.
AB - Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5,922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.
UR - http://www.scopus.com/inward/record.url?scp=84993965231&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84993965231&partnerID=8YFLogxK
U2 - 10.1109/TVCG.2016.2515614
DO - 10.1109/TVCG.2016.2515614
M3 - Article
AN - SCOPUS:84993965231
VL - 22
SP - 2564
EP - 2578
JO - IEEE Transactions on Visualization and Computer Graphics
JF - IEEE Transactions on Visualization and Computer Graphics
SN - 1077-2626
IS - 12
M1 - 7374748
ER -