Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning

Hui Chuan Chu, William Wei Jen Tsai, Min Ju Liao, Yuh-Min Chen

研究成果: Article

1 引文 (Scopus)

摘要

Emotions deeply affect learning achievement. In the case of students with high-functioning autism (HFA), negative emotions such as anxiety and anger can impair the learning process due to the inability of these individuals to control their emotions. Attempts to regulate negative emotions in HFA students once they have occurred, subsequent regulation to HFA students is often ineffective because it is difficult to calm them down. Hence, detecting emotional transitions and providing adaptive emotional regulation strategies in a timely manner to regulate negative emotions can be especially important for students with HFA in an e-learning environment. In this study, a facial expression-based emotion recognition method with transition detection was proposed. An emotion elicitation experiment was performed to collect facial-based landmark signals for the purpose of building classifiers of emotion recognition. The proposed method used sliding window technique and support vector machine (SVM) to build classifiers in order to recognize emotions. For the purpose of determining robust features for emotion recognition, Information Gain (IG) and Chi-square were used for feature evaluations. The effectiveness of classifiers with different parameters of sliding windows was also examined. The experimental results confirmed that the proposed method has sufficient discriminatory capability. The recognition rates for basic emotions and transitional emotions were 99.13 and 92.40%, respectively. Also, through feature selection, training time was accelerated by 4.45 times, and the recognition rates for basic emotions and transitional emotions were 97.97 and 87.49%, respectively. The method was applied in an adaptive e-learning environment for mathematics to demonstrate its application effectiveness.

原文English
頁(從 - 到)2973-2999
頁數27
期刊Soft Computing
22
發行號9
DOIs
出版狀態Published - 2018 五月 1

指紋

Emotion Recognition
Electronic Learning
Students
Classifiers
Support vector machines
Feature extraction
Sliding Window
Classifier
Learning Environment
Emotion
Experiments
Anxiety
Information Gain
Elicitation
Chi-square
Facial Expression
Landmarks
Learning Process
Feature Selection
Support Vector Machine

All Science Journal Classification (ASJC) codes

  • Software
  • Theoretical Computer Science
  • Geometry and Topology

引用此文

Chu, Hui Chuan ; Tsai, William Wei Jen ; Liao, Min Ju ; Chen, Yuh-Min. / Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning. 於: Soft Computing. 2018 ; 卷 22, 編號 9. 頁 2973-2999.
@article{29b863a592b943b98e16a250068338a8,
title = "Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning",
abstract = "Emotions deeply affect learning achievement. In the case of students with high-functioning autism (HFA), negative emotions such as anxiety and anger can impair the learning process due to the inability of these individuals to control their emotions. Attempts to regulate negative emotions in HFA students once they have occurred, subsequent regulation to HFA students is often ineffective because it is difficult to calm them down. Hence, detecting emotional transitions and providing adaptive emotional regulation strategies in a timely manner to regulate negative emotions can be especially important for students with HFA in an e-learning environment. In this study, a facial expression-based emotion recognition method with transition detection was proposed. An emotion elicitation experiment was performed to collect facial-based landmark signals for the purpose of building classifiers of emotion recognition. The proposed method used sliding window technique and support vector machine (SVM) to build classifiers in order to recognize emotions. For the purpose of determining robust features for emotion recognition, Information Gain (IG) and Chi-square were used for feature evaluations. The effectiveness of classifiers with different parameters of sliding windows was also examined. The experimental results confirmed that the proposed method has sufficient discriminatory capability. The recognition rates for basic emotions and transitional emotions were 99.13 and 92.40{\%}, respectively. Also, through feature selection, training time was accelerated by 4.45 times, and the recognition rates for basic emotions and transitional emotions were 97.97 and 87.49{\%}, respectively. The method was applied in an adaptive e-learning environment for mathematics to demonstrate its application effectiveness.",
author = "Chu, {Hui Chuan} and Tsai, {William Wei Jen} and Liao, {Min Ju} and Yuh-Min Chen",
year = "2018",
month = "5",
day = "1",
doi = "10.1007/s00500-017-2549-z",
language = "English",
volume = "22",
pages = "2973--2999",
journal = "Soft Computing",
issn = "1432-7643",
publisher = "Springer Verlag",
number = "9",

}

Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning. / Chu, Hui Chuan; Tsai, William Wei Jen; Liao, Min Ju; Chen, Yuh-Min.

於: Soft Computing, 卷 22, 編號 9, 01.05.2018, p. 2973-2999.

研究成果: Article

TY - JOUR

T1 - Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning

AU - Chu, Hui Chuan

AU - Tsai, William Wei Jen

AU - Liao, Min Ju

AU - Chen, Yuh-Min

PY - 2018/5/1

Y1 - 2018/5/1

N2 - Emotions deeply affect learning achievement. In the case of students with high-functioning autism (HFA), negative emotions such as anxiety and anger can impair the learning process due to the inability of these individuals to control their emotions. Attempts to regulate negative emotions in HFA students once they have occurred, subsequent regulation to HFA students is often ineffective because it is difficult to calm them down. Hence, detecting emotional transitions and providing adaptive emotional regulation strategies in a timely manner to regulate negative emotions can be especially important for students with HFA in an e-learning environment. In this study, a facial expression-based emotion recognition method with transition detection was proposed. An emotion elicitation experiment was performed to collect facial-based landmark signals for the purpose of building classifiers of emotion recognition. The proposed method used sliding window technique and support vector machine (SVM) to build classifiers in order to recognize emotions. For the purpose of determining robust features for emotion recognition, Information Gain (IG) and Chi-square were used for feature evaluations. The effectiveness of classifiers with different parameters of sliding windows was also examined. The experimental results confirmed that the proposed method has sufficient discriminatory capability. The recognition rates for basic emotions and transitional emotions were 99.13 and 92.40%, respectively. Also, through feature selection, training time was accelerated by 4.45 times, and the recognition rates for basic emotions and transitional emotions were 97.97 and 87.49%, respectively. The method was applied in an adaptive e-learning environment for mathematics to demonstrate its application effectiveness.

AB - Emotions deeply affect learning achievement. In the case of students with high-functioning autism (HFA), negative emotions such as anxiety and anger can impair the learning process due to the inability of these individuals to control their emotions. Attempts to regulate negative emotions in HFA students once they have occurred, subsequent regulation to HFA students is often ineffective because it is difficult to calm them down. Hence, detecting emotional transitions and providing adaptive emotional regulation strategies in a timely manner to regulate negative emotions can be especially important for students with HFA in an e-learning environment. In this study, a facial expression-based emotion recognition method with transition detection was proposed. An emotion elicitation experiment was performed to collect facial-based landmark signals for the purpose of building classifiers of emotion recognition. The proposed method used sliding window technique and support vector machine (SVM) to build classifiers in order to recognize emotions. For the purpose of determining robust features for emotion recognition, Information Gain (IG) and Chi-square were used for feature evaluations. The effectiveness of classifiers with different parameters of sliding windows was also examined. The experimental results confirmed that the proposed method has sufficient discriminatory capability. The recognition rates for basic emotions and transitional emotions were 99.13 and 92.40%, respectively. Also, through feature selection, training time was accelerated by 4.45 times, and the recognition rates for basic emotions and transitional emotions were 97.97 and 87.49%, respectively. The method was applied in an adaptive e-learning environment for mathematics to demonstrate its application effectiveness.

UR - http://www.scopus.com/inward/record.url?scp=85017128719&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85017128719&partnerID=8YFLogxK

U2 - 10.1007/s00500-017-2549-z

DO - 10.1007/s00500-017-2549-z

M3 - Article

AN - SCOPUS:85017128719

VL - 22

SP - 2973

EP - 2999

JO - Soft Computing

JF - Soft Computing

SN - 1432-7643

IS - 9

ER -