Blog Article Summarization with Image-Text Alignment Techniques

Wei Ta Chu, Ming Chih Kao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose an image-text alignment framework to match images with text, and take blog article summarization as the main application. Objects in an image are first detected, from them deep features are extracted and transformed into a space commonly shared with the text. On the other hand, sentences of a blog article are represented as vectors, and are also embedded into the common space. With these processes, cross-modal matching can be achieved. A blog article is then summarized in the representation of images and their matched sentences. In evaluation, we demonstrate the effectiveness of the proposed method, and show that the generated summary makes more sense.

Original languageEnglish
Title of host publicationProceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages244-247
Number of pages4
ISBN (Electronic)9781538629369
DOIs
Publication statusPublished - 2017 Dec 28
Event19th IEEE International Symposium on Multimedia, ISM 2017 - Taichung, Taiwan
Duration: 2017 Dec 112017 Dec 13

Publication series

NameProceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017
Volume2017-January

Conference

Conference19th IEEE International Symposium on Multimedia, ISM 2017
CountryTaiwan
CityTaichung
Period17-12-1117-12-13

Fingerprint

Blogging
Blogs

All Science Journal Classification (ASJC) codes

  • Media Technology
  • Sensory Systems

Cite this

Chu, W. T., & Kao, M. C. (2017). Blog Article Summarization with Image-Text Alignment Techniques. In Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017 (pp. 244-247). (Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017; Vol. 2017-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISM.2017.40
Chu, Wei Ta ; Kao, Ming Chih. / Blog Article Summarization with Image-Text Alignment Techniques. Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 244-247 (Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017).
@inproceedings{53b0fe258af54bd282a7e4e0f4cd069e,
title = "Blog Article Summarization with Image-Text Alignment Techniques",
abstract = "We propose an image-text alignment framework to match images with text, and take blog article summarization as the main application. Objects in an image are first detected, from them deep features are extracted and transformed into a space commonly shared with the text. On the other hand, sentences of a blog article are represented as vectors, and are also embedded into the common space. With these processes, cross-modal matching can be achieved. A blog article is then summarized in the representation of images and their matched sentences. In evaluation, we demonstrate the effectiveness of the proposed method, and show that the generated summary makes more sense.",
author = "Chu, {Wei Ta} and Kao, {Ming Chih}",
year = "2017",
month = "12",
day = "28",
doi = "10.1109/ISM.2017.40",
language = "English",
series = "Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "244--247",
booktitle = "Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017",
address = "United States",

}

Chu, WT & Kao, MC 2017, Blog Article Summarization with Image-Text Alignment Techniques. in Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017. Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017, vol. 2017-January, Institute of Electrical and Electronics Engineers Inc., pp. 244-247, 19th IEEE International Symposium on Multimedia, ISM 2017, Taichung, Taiwan, 17-12-11. https://doi.org/10.1109/ISM.2017.40

Blog Article Summarization with Image-Text Alignment Techniques. / Chu, Wei Ta; Kao, Ming Chih.

Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 244-247 (Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017; Vol. 2017-January).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Blog Article Summarization with Image-Text Alignment Techniques

AU - Chu, Wei Ta

AU - Kao, Ming Chih

PY - 2017/12/28

Y1 - 2017/12/28

N2 - We propose an image-text alignment framework to match images with text, and take blog article summarization as the main application. Objects in an image are first detected, from them deep features are extracted and transformed into a space commonly shared with the text. On the other hand, sentences of a blog article are represented as vectors, and are also embedded into the common space. With these processes, cross-modal matching can be achieved. A blog article is then summarized in the representation of images and their matched sentences. In evaluation, we demonstrate the effectiveness of the proposed method, and show that the generated summary makes more sense.

AB - We propose an image-text alignment framework to match images with text, and take blog article summarization as the main application. Objects in an image are first detected, from them deep features are extracted and transformed into a space commonly shared with the text. On the other hand, sentences of a blog article are represented as vectors, and are also embedded into the common space. With these processes, cross-modal matching can be achieved. A blog article is then summarized in the representation of images and their matched sentences. In evaluation, we demonstrate the effectiveness of the proposed method, and show that the generated summary makes more sense.

UR - http://www.scopus.com/inward/record.url?scp=85045904410&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85045904410&partnerID=8YFLogxK

U2 - 10.1109/ISM.2017.40

DO - 10.1109/ISM.2017.40

M3 - Conference contribution

AN - SCOPUS:85045904410

T3 - Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017

SP - 244

EP - 247

BT - Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Chu WT, Kao MC. Blog Article Summarization with Image-Text Alignment Techniques. In Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 244-247. (Proceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017). https://doi.org/10.1109/ISM.2017.40