Measuring and Predicting Visual Importance of Similar Objects

Yan Kong, Weiming Dong, Xing Mei, Chongyang Ma, Tong Yee Lee, Siwei Lyu, Feiyue Huang, Xiaopeng Zhang

Research output: Contribution to journalArticlepeer-review

6 Citations (Scopus)


Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5,922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.

Original languageEnglish
Article number7374748
Pages (from-to)2564-2578
Number of pages15
JournalIEEE Transactions on Visualization and Computer Graphics
Issue number12
Publication statusPublished - 2016 Dec 1

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Measuring and Predicting Visual Importance of Similar Objects'. Together they form a unique fingerprint.

Cite this