TY - JOUR
T1 - Editing by viewing
T2 - Automatic home video summarization by viewing behavior analysis
AU - Peng, Wei Ting
AU - Chu, Wei Ta
AU - Chang, Chia Han
AU - Chou, Chien Nan
AU - Huang, Wei Jia
AU - Chang, Wen Yan
AU - Hung, Yi Ping
N1 - Funding Information:
Manuscript received November 02, 2010; revised February 25, 2011; accepted March 08, 2011. Date of publication March 22, 2011; date of current version May 18, 2011. This work was supported in part by the National Science Council, Taiwan, under Grant NSC 98-2221-E-002-127-MY3 and Grant NSC 98-2221-E-002-128-MY3 and by the Excellent Research Projects of National Taiwan University, under Grant 99R80300. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Zicheng Liu.
PY - 2011/6
Y1 - 2011/6
N2 - In this paper, we propose the Interest Meter (IM), a system making the computer conscious of user's reactions to measure user's interest and thus use it to conduct video summarization. The IM takes account of users' spontaneous reactions when they view videos. To estimate user's viewing interest, quantitative interest measures are devised based on the perspectives of attention and emotion. For estimating attention states, variations of user's eye movement, blink, and head motion are considered. For estimating emotion states, facial expression is recognized as positive or neural emotion. By combining characteristics of attention and emotion by a fuzzy fusion scheme, we transform users' viewing behaviors into quantitative interest scores, determine interesting parts of videos, and finally concatenate them as video summaries. Experimental results show that the proposed concept "editing by viewing" works well and may provide a promising direction to consider the human factor in video summarization.
AB - In this paper, we propose the Interest Meter (IM), a system making the computer conscious of user's reactions to measure user's interest and thus use it to conduct video summarization. The IM takes account of users' spontaneous reactions when they view videos. To estimate user's viewing interest, quantitative interest measures are devised based on the perspectives of attention and emotion. For estimating attention states, variations of user's eye movement, blink, and head motion are considered. For estimating emotion states, facial expression is recognized as positive or neural emotion. By combining characteristics of attention and emotion by a fuzzy fusion scheme, we transform users' viewing behaviors into quantitative interest scores, determine interesting parts of videos, and finally concatenate them as video summaries. Experimental results show that the proposed concept "editing by viewing" works well and may provide a promising direction to consider the human factor in video summarization.
UR - http://www.scopus.com/inward/record.url?scp=79957529980&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=79957529980&partnerID=8YFLogxK
U2 - 10.1109/TMM.2011.2131638
DO - 10.1109/TMM.2011.2131638
M3 - Article
AN - SCOPUS:79957529980
SN - 1520-9210
VL - 13
SP - 539
EP - 550
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
IS - 3
M1 - 5737794
ER -