Speaking effect removal on emotion recognition from facial expressions based on eigenface conversion

Chung Hsien Wu, Wen Li Wei, Jen Chun Lin, Wei Yu Lee

研究成果: Article同行評審

40 引文 斯高帕斯(Scopus)


Speaking effect is a crucial issue that may dramatically degrade performance in emotion recognition from facial expressions. To manage this problem, an eigenface conversion-based approach is proposed to remove speaking effect on facial expressions for improving accuracy of emotion recognition. In the proposed approach, a context-dependent linear conversion function modeled by a statistical Gaussian Mixture Model (GMM) is constructed with parallel data from speaking and non-speaking facial expressions with emotions. To model the speaking effect in more detail, the conversion functions are categorized using a decision tree considering the visual temporal context of the Articulatory Attribute (AA) classes of the corresponding input speech segments. For verification of the identified quadrant of emotional expression on the Arousal-Valence (A-V) emotion plane, which is commonly used to dimensionally define the emotion classes, from the reconstructed facial feature points, an expression template is constructed to represent the feature points of the non-speaking facial expressions for each quadrant. With the verified quadrant, a regression scheme is further employed to estimate the A-V values of the facial expression as a precise point in the A-V emotion plane. Experimental results show that the proposed method outperforms current approaches and demonstrates that removing the speaking effect on facial expression is useful for improving the performance of emotion recognition.

頁(從 - 到)1732-1744
期刊IEEE Transactions on Multimedia
出版狀態Published - 2013

All Science Journal Classification (ASJC) codes

  • 訊號處理
  • 媒體技術
  • 電腦科學應用
  • 電氣與電子工程


深入研究「Speaking effect removal on emotion recognition from facial expressions based on eigenface conversion」主題。共同形成了獨特的指紋。