Personalized spectral and prosody conversion using frame-based codeword distribution and adaptive CRF

Yi Chin Huang, Chung Hsien Wu, Yu Ting Chao

研究成果: Article同行評審

15 引文 斯高帕斯(Scopus)


This study proposes a voice conversion-based approach to personalized text-to-speech (TTS) synthesis. The conversion functions, trained using a small parallel corpus with source and target speech data, can impose the voice characteristics of a target speaker on an existing synthesizer. Frame alignment between a pair of sentences in the parallel corpus is generally used for training voice conversion functions. However, with incorrect alignment, the resultant conversion functions may generate unacceptable conversion results. Traditional frame alignment using minimal spectral distance between the frame-based feature vectors of the source and the target phone sequences can be imprecise because the voice properties of the source and target phones inherently differ. In the proposed method, feature vectors of the parallel corpus are transformed into codewords in an eigenspace. A more precise frame alignment can be obtained by integrating the codeword occurrence distributions into distance estimation. In addition to the spectral property, a prosodic word/phrase boundary prediction model was constructed using an adaptive conditional random field (CRF) to generate personalized prosodic information. Objective and subjective tests were conducted to evaluate the performance of the proposed approach. The experimental results showed that the proposed voice conversion method, based on distribution-based alignment and prosodic word boundary detection, can improve the speech quality and speaker similarity of the converted speech. Compared to other methods, the evaluation results verified the improved performance of the proposed method.

頁(從 - 到)51-62
期刊IEEE Transactions on Audio, Speech and Language Processing
出版狀態Published - 2013

All Science Journal Classification (ASJC) codes

  • 聲學與超音波
  • 電氣與電子工程


深入研究「Personalized spectral and prosody conversion using frame-based codeword distribution and adaptive CRF」主題。共同形成了獨特的指紋。