Exploiting psychological factors for interaction style recognition in spoken conversation

Wen Li Wei, Chung Hsien Wu, Jen Chun Lin, Han Li

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)


Determining how a speaker is engaged in a conversation is crucial for achieving harmonious interaction between computers and humans. In this study, a fusion approach was developed based on psychological factors to recognize Interaction Style (IS) in spoken conversation, which plays a key role in creating natural dialogue agents. The proposed Fused Cross-Correlation Model (FCCM) provides a unified probabilistic framework to model the relationships among the psychological factors of emotion, personality trait (PT), transient IS, and IS history, for recognizing IS. An emotional arousal-dependent speech recognizer was used to obtain the recognized spoken text for extracting linguistic features to estimate transient likelihood and recognize PT. A temporal course modeling approach and an emotional sub-state language model, based on the temporal phases of an emotional expression, were employed to obtain a better emotion recognition result. The experimental results indicate that the proposed FCCM yields satisfactory results in recognition and also demonstrate that combining psychological factors effectively improves IS recognition accuracy.

Original languageEnglish
Pages (from-to)659-671
Number of pages13
JournalIEEE Transactions on Audio, Speech and Language Processing
Issue number3
Publication statusPublished - 2014 Mar

All Science Journal Classification (ASJC) codes

  • Acoustics and Ultrasonics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Exploiting psychological factors for interaction style recognition in spoken conversation'. Together they form a unique fingerprint.

Cite this