Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds

Kun Yi Huang, Chung Hsien Wu, Qian Bei Hong, Ming Hsiang Su, Yi Hsuan Chen

研究成果: Conference contribution

95 引文 斯高帕斯(Scopus)

摘要

Speech emotion recognition is becoming increasingly important for many applications. In real-life communication, non-verbal sounds within an utterance also play an important role for people to recognize emotion. In current studies, only few emotion recognition systems considered nonverbal sounds, such as laughter, cries or other emotion interjection, which naturally exists in our daily conversation. In this work, both verbal and nonverbal sounds within an utterance were thus considered for emotion recognition of real-life conversations. Firstly, an SVM-based verbal/nonverbal sound detector was developed. A Prosodic Phrase (PPh) auto-tagger was further employed to extract the verbal/nonverbal segments. For each segment, the emotion and sound features were respectively extracted based on convolutional neural networks (CNNs) and then concatenated to form a CNN-based generic feature vector. Finally, a sequence of CNN-based feature vectors for an entire dialog turn was fed to an attentive long short-term memory (LSTM)-based sequence-to-sequence model to output an emotional sequence as recognition result. Experimental results on the recognition of seven emotional states in the NNIME (The NTHU-NTUA Chinese interactive multimodal emotion corpus) showed that the proposed method achieved a detection accuracy of 52.00% outperforming the traditional methods.

原文English
主出版物標題2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings
發行者Institute of Electrical and Electronics Engineers Inc.
頁面5866-5870
頁數5
ISBN(電子)9781479981311
DOIs
出版狀態Published - 2019 5月
事件44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Brighton, United Kingdom
持續時間: 2019 5月 122019 5月 17

出版系列

名字ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
2019-May
ISSN(列印)1520-6149

Conference

Conference44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019
國家/地區United Kingdom
城市Brighton
期間19-05-1219-05-17

All Science Journal Classification (ASJC) codes

  • 軟體
  • 訊號處理
  • 電氣與電子工程

指紋

深入研究「Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds」主題。共同形成了獨特的指紋。

引用此