Speech act modeling and verification of spontaneous speech with disfluency in a spoken dialogue system

Chung Hsien Wu, Gwo Lang Yan

Research output: Contribution to journalArticlepeer-review

18 Citations (Scopus)


This work presents an approach to modeling speech acts and verifying spontaneous speech with disfluency in a spoken dialogue system. According to this approach, semantic information, syntactic structure and fragment class of an input utterance are statistically encapsulated in a proposed speech act hidden Markov model (SAHMM) to characterize the speech act. An interpolation mechanism is exploited to re-estimate the state transition probability in SAHMM, to deal with the problem of disfluency in a sparse training corpus. Finally, a Bayesian belief model (BBM), based on latent semantic analysis (LSA), is adopted to verify the potential speech acts and output the final speech act. Experiments were conducted to evaluate the proposed approach using a spoken dialogue system for providing air travel information. A testing database from 25 speakers, with 480 dialogues that include 3038 sentences, was established and used for evaluation. Experimental results show that the proposed approach identifies 95.3% of speech act at a rejection rate of 5%, and the semantic accuracy is 4.2% better than that obtained using a keyword-based system. The proposed strategy also effectively alleviates the disfluency problem in spontaneous speech.

Original languageEnglish
Pages (from-to)330-344
Number of pages15
JournalIEEE Transactions on Speech and Audio Processing
Issue number3
Publication statusPublished - 2005 May

All Science Journal Classification (ASJC) codes

  • Software
  • Acoustics and Ultrasonics
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Speech act modeling and verification of spontaneous speech with disfluency in a spoken dialogue system'. Together they form a unique fingerprint.

Cite this