Personalized spectral and prosody conversion using frame-based codeword distribution and adaptive CRF

Yi Chin Huang, Chung Hsien Wu, Yu Ting Chao

Research output: Contribution to journalArticlepeer-review

15 Citations (Scopus)


This study proposes a voice conversion-based approach to personalized text-to-speech (TTS) synthesis. The conversion functions, trained using a small parallel corpus with source and target speech data, can impose the voice characteristics of a target speaker on an existing synthesizer. Frame alignment between a pair of sentences in the parallel corpus is generally used for training voice conversion functions. However, with incorrect alignment, the resultant conversion functions may generate unacceptable conversion results. Traditional frame alignment using minimal spectral distance between the frame-based feature vectors of the source and the target phone sequences can be imprecise because the voice properties of the source and target phones inherently differ. In the proposed method, feature vectors of the parallel corpus are transformed into codewords in an eigenspace. A more precise frame alignment can be obtained by integrating the codeword occurrence distributions into distance estimation. In addition to the spectral property, a prosodic word/phrase boundary prediction model was constructed using an adaptive conditional random field (CRF) to generate personalized prosodic information. Objective and subjective tests were conducted to evaluate the performance of the proposed approach. The experimental results showed that the proposed voice conversion method, based on distribution-based alignment and prosodic word boundary detection, can improve the speech quality and speaker similarity of the converted speech. Compared to other methods, the evaluation results verified the improved performance of the proposed method.

Original languageEnglish
Article number6269060
Pages (from-to)51-62
Number of pages12
JournalIEEE Transactions on Audio, Speech and Language Processing
Issue number1
Publication statusPublished - 2013

All Science Journal Classification (ASJC) codes

  • Acoustics and Ultrasonics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Personalized spectral and prosody conversion using frame-based codeword distribution and adaptive CRF'. Together they form a unique fingerprint.

Cite this