Measuring and Predicting Visual Importance of Similar Objects

Yan Kong, Weiming Dong, Xing Mei, Chongyang Ma, Tong Yee Lee, Siwei Lyu, Feiyue Huang, Xiaopeng Zhang

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5,922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.

Original languageEnglish
Article number7374748
Pages (from-to)2564-2578
Number of pages15
JournalIEEE Transactions on Visualization and Computer Graphics
Volume22
Issue number12
DOIs
Publication statusPublished - 2016 Dec 1

Fingerprint

Image understanding
Image compression
Semantics
Color
Chemical analysis

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Cite this

Kong, Yan ; Dong, Weiming ; Mei, Xing ; Ma, Chongyang ; Lee, Tong Yee ; Lyu, Siwei ; Huang, Feiyue ; Zhang, Xiaopeng. / Measuring and Predicting Visual Importance of Similar Objects. In: IEEE Transactions on Visualization and Computer Graphics. 2016 ; Vol. 22, No. 12. pp. 2564-2578.
@article{76b98c711d4d4e48bc6b0b543cf713fe,
title = "Measuring and Predicting Visual Importance of Similar Objects",
abstract = "Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5,922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.",
author = "Yan Kong and Weiming Dong and Xing Mei and Chongyang Ma and Lee, {Tong Yee} and Siwei Lyu and Feiyue Huang and Xiaopeng Zhang",
year = "2016",
month = "12",
day = "1",
doi = "10.1109/TVCG.2016.2515614",
language = "English",
volume = "22",
pages = "2564--2578",
journal = "IEEE Transactions on Visualization and Computer Graphics",
issn = "1077-2626",
publisher = "IEEE Computer Society",
number = "12",

}

Kong, Y, Dong, W, Mei, X, Ma, C, Lee, TY, Lyu, S, Huang, F & Zhang, X 2016, 'Measuring and Predicting Visual Importance of Similar Objects', IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 12, 7374748, pp. 2564-2578. https://doi.org/10.1109/TVCG.2016.2515614

Measuring and Predicting Visual Importance of Similar Objects. / Kong, Yan; Dong, Weiming; Mei, Xing; Ma, Chongyang; Lee, Tong Yee; Lyu, Siwei; Huang, Feiyue; Zhang, Xiaopeng.

In: IEEE Transactions on Visualization and Computer Graphics, Vol. 22, No. 12, 7374748, 01.12.2016, p. 2564-2578.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Measuring and Predicting Visual Importance of Similar Objects

AU - Kong, Yan

AU - Dong, Weiming

AU - Mei, Xing

AU - Ma, Chongyang

AU - Lee, Tong Yee

AU - Lyu, Siwei

AU - Huang, Feiyue

AU - Zhang, Xiaopeng

PY - 2016/12/1

Y1 - 2016/12/1

N2 - Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5,922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.

AB - Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5,922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.

UR - http://www.scopus.com/inward/record.url?scp=84993965231&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84993965231&partnerID=8YFLogxK

U2 - 10.1109/TVCG.2016.2515614

DO - 10.1109/TVCG.2016.2515614

M3 - Article

AN - SCOPUS:84993965231

VL - 22

SP - 2564

EP - 2578

JO - IEEE Transactions on Visualization and Computer Graphics

JF - IEEE Transactions on Visualization and Computer Graphics

SN - 1077-2626

IS - 12

M1 - 7374748

ER -