Exploiting turn-taking temporal evolution for personality trait perception in dyadic conversations

Ming Hsiang Su, Chung-Hsien Wu, Yu Ting Zheng

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

In dyadic conversations, turn-taking is a dynamically evolving behavior strongly linked to paralinguistic communication. Turn-taking temporal evolution in a dyadic conversation is inevitable and can be incorporated into a modeling framework for characterizing and recognizing the personality traits (PTs) of two speakers. This study presents an approach to automatically predicting PTs in a dyadic conversation. First, a recurrent neural network (RNN) was used to model the relationship between Big Five Inventory 10 (BFI-10) items and linguistic features of spoken text in each turn of a speaker (speaker turn) to output a BFI-10 profile. The RNN applies a recurrent property to characterize the short-term temporal evolution of a dialog. Second, the coupled hidden Markov model (C-HMM) was employed to model the long-term turn-taking temporal evolution and cross-speaker contextual information for detecting the PTs of two individuals for the entire dialog represented by the BFI-10 profile sequence. TheMandarin Conversational Dialogue Corpus was used for evaluation. The evaluation result shows that an average perception accuracy of 79.66% for the big five traits was achieved using five-fold cross validation. Compared with conventional HMM and support vector machine-based methods, the proposed approach achieved a more favorable performance according to a statistical significance test. The encouraging results confirm the usability of this system for future applications.

Original languageEnglish
Pages (from-to)733-744
Number of pages12
JournalIEEE/ACM Transactions on Audio Speech and Language Processing
Volume24
Issue number4
DOIs
Publication statusPublished - 2016 Apr 1

Fingerprint

personality
conversation
dyadics
Recurrent neural networks
Statistical tests
Hidden Markov models
Linguistics
Support vector machines
Recurrent Neural Networks
linguistics
evaluation
profiles
Communication
communication
Significance Test
Statistical Significance
Evaluation
Statistical test
Cross-validation
output

All Science Journal Classification (ASJC) codes

  • Computer Science (miscellaneous)
  • Acoustics and Ultrasonics
  • Computational Mathematics
  • Electrical and Electronic Engineering

Cite this

@article{dfaffa9b047a4506b3ebaa4bb5ed3f15,
title = "Exploiting turn-taking temporal evolution for personality trait perception in dyadic conversations",
abstract = "In dyadic conversations, turn-taking is a dynamically evolving behavior strongly linked to paralinguistic communication. Turn-taking temporal evolution in a dyadic conversation is inevitable and can be incorporated into a modeling framework for characterizing and recognizing the personality traits (PTs) of two speakers. This study presents an approach to automatically predicting PTs in a dyadic conversation. First, a recurrent neural network (RNN) was used to model the relationship between Big Five Inventory 10 (BFI-10) items and linguistic features of spoken text in each turn of a speaker (speaker turn) to output a BFI-10 profile. The RNN applies a recurrent property to characterize the short-term temporal evolution of a dialog. Second, the coupled hidden Markov model (C-HMM) was employed to model the long-term turn-taking temporal evolution and cross-speaker contextual information for detecting the PTs of two individuals for the entire dialog represented by the BFI-10 profile sequence. TheMandarin Conversational Dialogue Corpus was used for evaluation. The evaluation result shows that an average perception accuracy of 79.66{\%} for the big five traits was achieved using five-fold cross validation. Compared with conventional HMM and support vector machine-based methods, the proposed approach achieved a more favorable performance according to a statistical significance test. The encouraging results confirm the usability of this system for future applications.",
author = "Su, {Ming Hsiang} and Chung-Hsien Wu and Zheng, {Yu Ting}",
year = "2016",
month = "4",
day = "1",
doi = "10.1109/TASLP.2016.2531286",
language = "English",
volume = "24",
pages = "733--744",
journal = "IEEE/ACM Transactions on Speech and Language Processing",
issn = "2329-9290",
publisher = "IEEE Advancing Technology for Humanity",
number = "4",

}

Exploiting turn-taking temporal evolution for personality trait perception in dyadic conversations. / Su, Ming Hsiang; Wu, Chung-Hsien; Zheng, Yu Ting.

In: IEEE/ACM Transactions on Audio Speech and Language Processing, Vol. 24, No. 4, 01.04.2016, p. 733-744.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Exploiting turn-taking temporal evolution for personality trait perception in dyadic conversations

AU - Su, Ming Hsiang

AU - Wu, Chung-Hsien

AU - Zheng, Yu Ting

PY - 2016/4/1

Y1 - 2016/4/1

N2 - In dyadic conversations, turn-taking is a dynamically evolving behavior strongly linked to paralinguistic communication. Turn-taking temporal evolution in a dyadic conversation is inevitable and can be incorporated into a modeling framework for characterizing and recognizing the personality traits (PTs) of two speakers. This study presents an approach to automatically predicting PTs in a dyadic conversation. First, a recurrent neural network (RNN) was used to model the relationship between Big Five Inventory 10 (BFI-10) items and linguistic features of spoken text in each turn of a speaker (speaker turn) to output a BFI-10 profile. The RNN applies a recurrent property to characterize the short-term temporal evolution of a dialog. Second, the coupled hidden Markov model (C-HMM) was employed to model the long-term turn-taking temporal evolution and cross-speaker contextual information for detecting the PTs of two individuals for the entire dialog represented by the BFI-10 profile sequence. TheMandarin Conversational Dialogue Corpus was used for evaluation. The evaluation result shows that an average perception accuracy of 79.66% for the big five traits was achieved using five-fold cross validation. Compared with conventional HMM and support vector machine-based methods, the proposed approach achieved a more favorable performance according to a statistical significance test. The encouraging results confirm the usability of this system for future applications.

AB - In dyadic conversations, turn-taking is a dynamically evolving behavior strongly linked to paralinguistic communication. Turn-taking temporal evolution in a dyadic conversation is inevitable and can be incorporated into a modeling framework for characterizing and recognizing the personality traits (PTs) of two speakers. This study presents an approach to automatically predicting PTs in a dyadic conversation. First, a recurrent neural network (RNN) was used to model the relationship between Big Five Inventory 10 (BFI-10) items and linguistic features of spoken text in each turn of a speaker (speaker turn) to output a BFI-10 profile. The RNN applies a recurrent property to characterize the short-term temporal evolution of a dialog. Second, the coupled hidden Markov model (C-HMM) was employed to model the long-term turn-taking temporal evolution and cross-speaker contextual information for detecting the PTs of two individuals for the entire dialog represented by the BFI-10 profile sequence. TheMandarin Conversational Dialogue Corpus was used for evaluation. The evaluation result shows that an average perception accuracy of 79.66% for the big five traits was achieved using five-fold cross validation. Compared with conventional HMM and support vector machine-based methods, the proposed approach achieved a more favorable performance according to a statistical significance test. The encouraging results confirm the usability of this system for future applications.

UR - http://www.scopus.com/inward/record.url?scp=84962799454&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84962799454&partnerID=8YFLogxK

U2 - 10.1109/TASLP.2016.2531286

DO - 10.1109/TASLP.2016.2531286

M3 - Article

VL - 24

SP - 733

EP - 744

JO - IEEE/ACM Transactions on Speech and Language Processing

JF - IEEE/ACM Transactions on Speech and Language Processing

SN - 2329-9290

IS - 4

ER -