High-level semantic photographic composition analysis and understanding with deep neural networks

Min Tzu Wu, Tse Yu Pan, Wan Lun Tsai, Hsu-Chan Kuo, Min-Chun Hu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In order to take better photos, it is a fundamental step for the beginners of photography to learn basic photo composition rules. However, there are no tools developed to help beginners analyze the composition rules in given photos. Thus, in this study we developed a system with the capability to identify 12 common composition rules in a photo. It should be noted that some of the 12 common composition rules have not been considered by the previous studies, and this deficit gives this study its significance and appropriateness. In particular, we utilized deep neural networks (DNN) to extract high-level semantic features for facilitating the further analysis of photo composition rules. In order to train the DNN model, our research team constructed a dataset, which is collected from some famous photo websites, such as DPChallenge, Flicker, and Unsplash. All the collected photos were later labelled with 12 composition rules by a wide range of raters recruited from Amazon Mechanical Turk (AMT). Two DNN architectures (AlexNet and GoogLeNet) were then employed to build our system based on the collected dataset. The representative features of each composition rule were further visualized in our system. The results showed the feasibility of the proposed system and revealed the possibility of using this system to assist potential users to improve their photographical skills and expertise.

Original languageEnglish
Title of host publication2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages279-284
Number of pages6
ISBN (Electronic)9781538605608
DOIs
Publication statusPublished - 2017 Sep 5
Event2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017 - Hong Kong, Hong Kong
Duration: 2017 Jul 102017 Jul 14

Publication series

Name2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017

Other

Other2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017
CountryHong Kong
CityHong Kong
Period17-07-1017-07-14

Fingerprint

Semantics
Photocomposition
Chemical analysis
Photography
Network architecture
Websites
Deep neural networks

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Media Technology

Cite this

Wu, M. T., Pan, T. Y., Tsai, W. L., Kuo, H-C., & Hu, M-C. (2017). High-level semantic photographic composition analysis and understanding with deep neural networks. In 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017 (pp. 279-284). [8026247] (2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMEW.2017.8026247
Wu, Min Tzu ; Pan, Tse Yu ; Tsai, Wan Lun ; Kuo, Hsu-Chan ; Hu, Min-Chun. / High-level semantic photographic composition analysis and understanding with deep neural networks. 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 279-284 (2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017).
@inproceedings{11affdc08fab47818682da22d4eb8aea,
title = "High-level semantic photographic composition analysis and understanding with deep neural networks",
abstract = "In order to take better photos, it is a fundamental step for the beginners of photography to learn basic photo composition rules. However, there are no tools developed to help beginners analyze the composition rules in given photos. Thus, in this study we developed a system with the capability to identify 12 common composition rules in a photo. It should be noted that some of the 12 common composition rules have not been considered by the previous studies, and this deficit gives this study its significance and appropriateness. In particular, we utilized deep neural networks (DNN) to extract high-level semantic features for facilitating the further analysis of photo composition rules. In order to train the DNN model, our research team constructed a dataset, which is collected from some famous photo websites, such as DPChallenge, Flicker, and Unsplash. All the collected photos were later labelled with 12 composition rules by a wide range of raters recruited from Amazon Mechanical Turk (AMT). Two DNN architectures (AlexNet and GoogLeNet) were then employed to build our system based on the collected dataset. The representative features of each composition rule were further visualized in our system. The results showed the feasibility of the proposed system and revealed the possibility of using this system to assist potential users to improve their photographical skills and expertise.",
author = "Wu, {Min Tzu} and Pan, {Tse Yu} and Tsai, {Wan Lun} and Hsu-Chan Kuo and Min-Chun Hu",
year = "2017",
month = "9",
day = "5",
doi = "10.1109/ICMEW.2017.8026247",
language = "English",
series = "2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "279--284",
booktitle = "2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017",
address = "United States",

}

Wu, MT, Pan, TY, Tsai, WL, Kuo, H-C & Hu, M-C 2017, High-level semantic photographic composition analysis and understanding with deep neural networks. in 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017., 8026247, 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017, Institute of Electrical and Electronics Engineers Inc., pp. 279-284, 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017, Hong Kong, Hong Kong, 17-07-10. https://doi.org/10.1109/ICMEW.2017.8026247

High-level semantic photographic composition analysis and understanding with deep neural networks. / Wu, Min Tzu; Pan, Tse Yu; Tsai, Wan Lun; Kuo, Hsu-Chan; Hu, Min-Chun.

2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 279-284 8026247 (2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - High-level semantic photographic composition analysis and understanding with deep neural networks

AU - Wu, Min Tzu

AU - Pan, Tse Yu

AU - Tsai, Wan Lun

AU - Kuo, Hsu-Chan

AU - Hu, Min-Chun

PY - 2017/9/5

Y1 - 2017/9/5

N2 - In order to take better photos, it is a fundamental step for the beginners of photography to learn basic photo composition rules. However, there are no tools developed to help beginners analyze the composition rules in given photos. Thus, in this study we developed a system with the capability to identify 12 common composition rules in a photo. It should be noted that some of the 12 common composition rules have not been considered by the previous studies, and this deficit gives this study its significance and appropriateness. In particular, we utilized deep neural networks (DNN) to extract high-level semantic features for facilitating the further analysis of photo composition rules. In order to train the DNN model, our research team constructed a dataset, which is collected from some famous photo websites, such as DPChallenge, Flicker, and Unsplash. All the collected photos were later labelled with 12 composition rules by a wide range of raters recruited from Amazon Mechanical Turk (AMT). Two DNN architectures (AlexNet and GoogLeNet) were then employed to build our system based on the collected dataset. The representative features of each composition rule were further visualized in our system. The results showed the feasibility of the proposed system and revealed the possibility of using this system to assist potential users to improve their photographical skills and expertise.

AB - In order to take better photos, it is a fundamental step for the beginners of photography to learn basic photo composition rules. However, there are no tools developed to help beginners analyze the composition rules in given photos. Thus, in this study we developed a system with the capability to identify 12 common composition rules in a photo. It should be noted that some of the 12 common composition rules have not been considered by the previous studies, and this deficit gives this study its significance and appropriateness. In particular, we utilized deep neural networks (DNN) to extract high-level semantic features for facilitating the further analysis of photo composition rules. In order to train the DNN model, our research team constructed a dataset, which is collected from some famous photo websites, such as DPChallenge, Flicker, and Unsplash. All the collected photos were later labelled with 12 composition rules by a wide range of raters recruited from Amazon Mechanical Turk (AMT). Two DNN architectures (AlexNet and GoogLeNet) were then employed to build our system based on the collected dataset. The representative features of each composition rule were further visualized in our system. The results showed the feasibility of the proposed system and revealed the possibility of using this system to assist potential users to improve their photographical skills and expertise.

UR - http://www.scopus.com/inward/record.url?scp=85031705095&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85031705095&partnerID=8YFLogxK

U2 - 10.1109/ICMEW.2017.8026247

DO - 10.1109/ICMEW.2017.8026247

M3 - Conference contribution

AN - SCOPUS:85031705095

T3 - 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017

SP - 279

EP - 284

BT - 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Wu MT, Pan TY, Tsai WL, Kuo H-C, Hu M-C. High-level semantic photographic composition analysis and understanding with deep neural networks. In 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 279-284. 8026247. (2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017). https://doi.org/10.1109/ICMEW.2017.8026247