Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning

Hui Chuan Chu, William Wei Jen Tsai, Min Ju Liao, Yuh Min Chen

Research output: Contribution to journalArticle

2 Citations (Scopus)


Emotions deeply affect learning achievement. In the case of students with high-functioning autism (HFA), negative emotions such as anxiety and anger can impair the learning process due to the inability of these individuals to control their emotions. Attempts to regulate negative emotions in HFA students once they have occurred, subsequent regulation to HFA students is often ineffective because it is difficult to calm them down. Hence, detecting emotional transitions and providing adaptive emotional regulation strategies in a timely manner to regulate negative emotions can be especially important for students with HFA in an e-learning environment. In this study, a facial expression-based emotion recognition method with transition detection was proposed. An emotion elicitation experiment was performed to collect facial-based landmark signals for the purpose of building classifiers of emotion recognition. The proposed method used sliding window technique and support vector machine (SVM) to build classifiers in order to recognize emotions. For the purpose of determining robust features for emotion recognition, Information Gain (IG) and Chi-square were used for feature evaluations. The effectiveness of classifiers with different parameters of sliding windows was also examined. The experimental results confirmed that the proposed method has sufficient discriminatory capability. The recognition rates for basic emotions and transitional emotions were 99.13 and 92.40%, respectively. Also, through feature selection, training time was accelerated by 4.45 times, and the recognition rates for basic emotions and transitional emotions were 97.97 and 87.49%, respectively. The method was applied in an adaptive e-learning environment for mathematics to demonstrate its application effectiveness.

Original languageEnglish
Pages (from-to)2973-2999
Number of pages27
JournalSoft Computing
Issue number9
Publication statusPublished - 2018 May 1


All Science Journal Classification (ASJC) codes

  • Software
  • Theoretical Computer Science
  • Geometry and Topology

Cite this