Cell-Coupled Long Short-Term Memory with L-Skip Fusion Mechanism for Mood Disorder Detection through Elicited Audiovisual Features

Ming Hsiang Su, Chung Hsien Wu, Kun Yi Huang, Tsung Hsien Yang

研究成果: Article同行評審

1 引文 斯高帕斯(Scopus)


In early stages, patients with bipolar disorder are often diagnosed as having unipolar depression in mood disorder diagnosis. Because the long-term monitoring is limited by the delayed detection of mood disorder, an accurate and one-time diagnosis is desirable to avoid delay in appropriate treatment due to misdiagnosis. In this paper, an elicitation-based approach is proposed for realizing a one-time diagnosis by using responses elicited from patients by having them watch six emotion-eliciting videos. After watching each video clip, the conversations, including patient facial expressions and speech responses, between the participant and the clinician conducting the interview were recorded. Next, the hierarchical spectral clustering algorithm was employed to adapt the facial expression and speech response features by using the extended Cohn-Kanade and eNTERFACE databases. A denoizing autoencoder was further applied to extract the bottleneck features of the adapted data. Then, the facial and speech bottleneck features were input into support vector machines to obtain speech emotion profiles (EPs) and the modulation spectrum (MS) of the facial action unit sequence for each elicited response. Finally, a cell-coupled long short-term memory (LSTM) network with an L-skip fusion mechanism was proposed to model the temporal information of all elicited responses and to loosely fuse the EPs and the MS for conducting mood disorder detection. The experimental results revealed that the cell-coupled LSTM with the L-skip fusion mechanism has promising advantages and efficacy for mood disorder detection.

頁(從 - 到)124-135
期刊IEEE Transactions on Neural Networks and Learning Systems
出版狀態Published - 2020 一月

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

指紋 深入研究「Cell-Coupled Long Short-Term Memory with L-Skip Fusion Mechanism for Mood Disorder Detection through Elicited Audiovisual Features」主題。共同形成了獨特的指紋。