Detection of mood disorder using speech emotion profiles and LSTM

Tsung Hsien Yang, Chung-Hsien Wu, Kun Yi Huang, Ming Hsiang Su

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

In mood disorder diagnosis, bipolar disorder (BD) patients are often misdiagnosed as unipolar depression (UD) on initial presentation. It is crucial to establish an accurate distinction between BD and UD to make a correct and early diagnosis, leading to improvements in treatment and course of illness. To deal with this misdiagnosis problem, in this study, we experimented on eliciting subjects' emotions by watching six eliciting emotional video clips. After watching each video clips, their speech responses were collected when they were interviewing with a clinician. In mood disorder detection, speech emotions play an import role to detect manic or depressive symptoms. Therefore, speech emotion profiles (EP) are obtained by using the support vector machine (SVM) which are built via speech features adapted from selected databases using a denoising autoencoder-based method. Finally, a Long Short-Term Memory (LSTM) recurrent neural network is employed to characterize the temporal information of the EPs with respect to six emotional videos. Comparative experiments clearly show the promising advantage and efficacy of the LSTM-based approach for mood disorder detection.

Original languageEnglish
Title of host publicationProceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016
EditorsHsin-Min Wang, Qingzhi Hou, Yuan Wei, Tan Lee, Jianguo Wei, Lei Xie, Hui Feng, Jianwu Dang, Jianwu Dang
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509042937
DOIs
Publication statusPublished - 2017 May 2
Event10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016 - Tianjin, China
Duration: 2016 Oct 172016 Oct 20

Publication series

NameProceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016

Other

Other10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016
CountryChina
CityTianjin
Period16-10-1716-10-20

Fingerprint

speech disorder
mood
video clip
emotion
early diagnosis
neural network
import
Recurrent neural networks
illness
video
Support vector machines
experiment
Long short-term memory
Experiments

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Linguistics and Language

Cite this

Yang, T. H., Wu, C-H., Huang, K. Y., & Su, M. H. (2017). Detection of mood disorder using speech emotion profiles and LSTM. In H-M. Wang, Q. Hou, Y. Wei, T. Lee, J. Wei, L. Xie, H. Feng, J. Dang, ... J. Dang (Eds.), Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016 [7918439] (Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISCSLP.2016.7918439
Yang, Tsung Hsien ; Wu, Chung-Hsien ; Huang, Kun Yi ; Su, Ming Hsiang. / Detection of mood disorder using speech emotion profiles and LSTM. Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016. editor / Hsin-Min Wang ; Qingzhi Hou ; Yuan Wei ; Tan Lee ; Jianguo Wei ; Lei Xie ; Hui Feng ; Jianwu Dang ; Jianwu Dang. Institute of Electrical and Electronics Engineers Inc., 2017. (Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016).
@inproceedings{308be86968d847c6b61a94bf4116bac6,
title = "Detection of mood disorder using speech emotion profiles and LSTM",
abstract = "In mood disorder diagnosis, bipolar disorder (BD) patients are often misdiagnosed as unipolar depression (UD) on initial presentation. It is crucial to establish an accurate distinction between BD and UD to make a correct and early diagnosis, leading to improvements in treatment and course of illness. To deal with this misdiagnosis problem, in this study, we experimented on eliciting subjects' emotions by watching six eliciting emotional video clips. After watching each video clips, their speech responses were collected when they were interviewing with a clinician. In mood disorder detection, speech emotions play an import role to detect manic or depressive symptoms. Therefore, speech emotion profiles (EP) are obtained by using the support vector machine (SVM) which are built via speech features adapted from selected databases using a denoising autoencoder-based method. Finally, a Long Short-Term Memory (LSTM) recurrent neural network is employed to characterize the temporal information of the EPs with respect to six emotional videos. Comparative experiments clearly show the promising advantage and efficacy of the LSTM-based approach for mood disorder detection.",
author = "Yang, {Tsung Hsien} and Chung-Hsien Wu and Huang, {Kun Yi} and Su, {Ming Hsiang}",
year = "2017",
month = "5",
day = "2",
doi = "10.1109/ISCSLP.2016.7918439",
language = "English",
series = "Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
editor = "Hsin-Min Wang and Qingzhi Hou and Yuan Wei and Tan Lee and Jianguo Wei and Lei Xie and Hui Feng and Jianwu Dang and Jianwu Dang",
booktitle = "Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016",
address = "United States",

}

Yang, TH, Wu, C-H, Huang, KY & Su, MH 2017, Detection of mood disorder using speech emotion profiles and LSTM. in H-M Wang, Q Hou, Y Wei, T Lee, J Wei, L Xie, H Feng, J Dang & J Dang (eds), Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016., 7918439, Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016, Institute of Electrical and Electronics Engineers Inc., 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016, Tianjin, China, 16-10-17. https://doi.org/10.1109/ISCSLP.2016.7918439

Detection of mood disorder using speech emotion profiles and LSTM. / Yang, Tsung Hsien; Wu, Chung-Hsien; Huang, Kun Yi; Su, Ming Hsiang.

Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016. ed. / Hsin-Min Wang; Qingzhi Hou; Yuan Wei; Tan Lee; Jianguo Wei; Lei Xie; Hui Feng; Jianwu Dang; Jianwu Dang. Institute of Electrical and Electronics Engineers Inc., 2017. 7918439 (Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Detection of mood disorder using speech emotion profiles and LSTM

AU - Yang, Tsung Hsien

AU - Wu, Chung-Hsien

AU - Huang, Kun Yi

AU - Su, Ming Hsiang

PY - 2017/5/2

Y1 - 2017/5/2

N2 - In mood disorder diagnosis, bipolar disorder (BD) patients are often misdiagnosed as unipolar depression (UD) on initial presentation. It is crucial to establish an accurate distinction between BD and UD to make a correct and early diagnosis, leading to improvements in treatment and course of illness. To deal with this misdiagnosis problem, in this study, we experimented on eliciting subjects' emotions by watching six eliciting emotional video clips. After watching each video clips, their speech responses were collected when they were interviewing with a clinician. In mood disorder detection, speech emotions play an import role to detect manic or depressive symptoms. Therefore, speech emotion profiles (EP) are obtained by using the support vector machine (SVM) which are built via speech features adapted from selected databases using a denoising autoencoder-based method. Finally, a Long Short-Term Memory (LSTM) recurrent neural network is employed to characterize the temporal information of the EPs with respect to six emotional videos. Comparative experiments clearly show the promising advantage and efficacy of the LSTM-based approach for mood disorder detection.

AB - In mood disorder diagnosis, bipolar disorder (BD) patients are often misdiagnosed as unipolar depression (UD) on initial presentation. It is crucial to establish an accurate distinction between BD and UD to make a correct and early diagnosis, leading to improvements in treatment and course of illness. To deal with this misdiagnosis problem, in this study, we experimented on eliciting subjects' emotions by watching six eliciting emotional video clips. After watching each video clips, their speech responses were collected when they were interviewing with a clinician. In mood disorder detection, speech emotions play an import role to detect manic or depressive symptoms. Therefore, speech emotion profiles (EP) are obtained by using the support vector machine (SVM) which are built via speech features adapted from selected databases using a denoising autoencoder-based method. Finally, a Long Short-Term Memory (LSTM) recurrent neural network is employed to characterize the temporal information of the EPs with respect to six emotional videos. Comparative experiments clearly show the promising advantage and efficacy of the LSTM-based approach for mood disorder detection.

UR - http://www.scopus.com/inward/record.url?scp=85020206734&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020206734&partnerID=8YFLogxK

U2 - 10.1109/ISCSLP.2016.7918439

DO - 10.1109/ISCSLP.2016.7918439

M3 - Conference contribution

T3 - Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016

BT - Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016

A2 - Wang, Hsin-Min

A2 - Hou, Qingzhi

A2 - Wei, Yuan

A2 - Lee, Tan

A2 - Wei, Jianguo

A2 - Xie, Lei

A2 - Feng, Hui

A2 - Dang, Jianwu

A2 - Dang, Jianwu

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Yang TH, Wu C-H, Huang KY, Su MH. Detection of mood disorder using speech emotion profiles and LSTM. In Wang H-M, Hou Q, Wei Y, Lee T, Wei J, Xie L, Feng H, Dang J, Dang J, editors, Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016. Institute of Electrical and Electronics Engineers Inc. 2017. 7918439. (Proceedings of 2016 10th International Symposium on Chinese Spoken Language Processing, ISCSLP 2016). https://doi.org/10.1109/ISCSLP.2016.7918439