Mixmodel driven 3D facial synthesis to computer-aided articulation training

Yeou Jiunn Chen, F. C. Liao, Jiunn-Liang Wu, H. M. Yang, Chung-Hsien Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

3D facial animation had been widely used in many multimedia applications and could be applied to help articulation disorders in articulatory training. In this paper, a mixmodel driven 3D facial synthesis including lip and tongue animation is proposed to provide multi-model feedback such as speech signal, lip motion, and tongue motion. Text-to-speech can generate the speech signal of arbitrary text and provide syllable boundaries. Contextual knowledge based phoneme segmentation is applied to estimate the phoneme boundaries in a syllable and the number of 3D facial models can be effectively reduced. Parametric 3D tongue and lip movement models are smoothed by B-spline to eliminate the jerkiness and synthesize the 3D facial with tongue and lip animation. Integrating boundary information, the speech-synching can be easily accomplished. The mult-model feedbacks in 3D facial animation are used to improve the efficiency of articulatory training. The preliminary experimental results show that this method is feasible.

Original languageEnglish
Title of host publication4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008
Pages56-60
Number of pages5
Edition1
DOIs
Publication statusPublished - 2008 Dec 1
Event4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008 - Kuala Lumpur, Malaysia
Duration: 2008 Jun 252008 Jun 28

Publication series

NameIFMBE Proceedings
Number1
Volume21 IFMBE
ISSN (Print)1680-0737

Other

Other4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008
CountryMalaysia
CityKuala Lumpur
Period08-06-2508-06-28

Fingerprint

Animation
Feedback
Splines

All Science Journal Classification (ASJC) codes

  • Bioengineering
  • Biomedical Engineering

Cite this

Chen, Y. J., Liao, F. C., Wu, J-L., Yang, H. M., & Wu, C-H. (2008). Mixmodel driven 3D facial synthesis to computer-aided articulation training. In 4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008 (1 ed., pp. 56-60). (IFMBE Proceedings; Vol. 21 IFMBE, No. 1). https://doi.org/10.1007/978-3-540-69139-6-19
Chen, Yeou Jiunn ; Liao, F. C. ; Wu, Jiunn-Liang ; Yang, H. M. ; Wu, Chung-Hsien. / Mixmodel driven 3D facial synthesis to computer-aided articulation training. 4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008. 1. ed. 2008. pp. 56-60 (IFMBE Proceedings; 1).
@inproceedings{f59542163940419d857263250fb536b1,
title = "Mixmodel driven 3D facial synthesis to computer-aided articulation training",
abstract = "3D facial animation had been widely used in many multimedia applications and could be applied to help articulation disorders in articulatory training. In this paper, a mixmodel driven 3D facial synthesis including lip and tongue animation is proposed to provide multi-model feedback such as speech signal, lip motion, and tongue motion. Text-to-speech can generate the speech signal of arbitrary text and provide syllable boundaries. Contextual knowledge based phoneme segmentation is applied to estimate the phoneme boundaries in a syllable and the number of 3D facial models can be effectively reduced. Parametric 3D tongue and lip movement models are smoothed by B-spline to eliminate the jerkiness and synthesize the 3D facial with tongue and lip animation. Integrating boundary information, the speech-synching can be easily accomplished. The mult-model feedbacks in 3D facial animation are used to improve the efficiency of articulatory training. The preliminary experimental results show that this method is feasible.",
author = "Chen, {Yeou Jiunn} and Liao, {F. C.} and Jiunn-Liang Wu and Yang, {H. M.} and Chung-Hsien Wu",
year = "2008",
month = "12",
day = "1",
doi = "10.1007/978-3-540-69139-6-19",
language = "English",
isbn = "9783540691389",
series = "IFMBE Proceedings",
number = "1",
pages = "56--60",
booktitle = "4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008",
edition = "1",

}

Chen, YJ, Liao, FC, Wu, J-L, Yang, HM & Wu, C-H 2008, Mixmodel driven 3D facial synthesis to computer-aided articulation training. in 4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008. 1 edn, IFMBE Proceedings, no. 1, vol. 21 IFMBE, pp. 56-60, 4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008, Kuala Lumpur, Malaysia, 08-06-25. https://doi.org/10.1007/978-3-540-69139-6-19

Mixmodel driven 3D facial synthesis to computer-aided articulation training. / Chen, Yeou Jiunn; Liao, F. C.; Wu, Jiunn-Liang; Yang, H. M.; Wu, Chung-Hsien.

4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008. 1. ed. 2008. p. 56-60 (IFMBE Proceedings; Vol. 21 IFMBE, No. 1).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Mixmodel driven 3D facial synthesis to computer-aided articulation training

AU - Chen, Yeou Jiunn

AU - Liao, F. C.

AU - Wu, Jiunn-Liang

AU - Yang, H. M.

AU - Wu, Chung-Hsien

PY - 2008/12/1

Y1 - 2008/12/1

N2 - 3D facial animation had been widely used in many multimedia applications and could be applied to help articulation disorders in articulatory training. In this paper, a mixmodel driven 3D facial synthesis including lip and tongue animation is proposed to provide multi-model feedback such as speech signal, lip motion, and tongue motion. Text-to-speech can generate the speech signal of arbitrary text and provide syllable boundaries. Contextual knowledge based phoneme segmentation is applied to estimate the phoneme boundaries in a syllable and the number of 3D facial models can be effectively reduced. Parametric 3D tongue and lip movement models are smoothed by B-spline to eliminate the jerkiness and synthesize the 3D facial with tongue and lip animation. Integrating boundary information, the speech-synching can be easily accomplished. The mult-model feedbacks in 3D facial animation are used to improve the efficiency of articulatory training. The preliminary experimental results show that this method is feasible.

AB - 3D facial animation had been widely used in many multimedia applications and could be applied to help articulation disorders in articulatory training. In this paper, a mixmodel driven 3D facial synthesis including lip and tongue animation is proposed to provide multi-model feedback such as speech signal, lip motion, and tongue motion. Text-to-speech can generate the speech signal of arbitrary text and provide syllable boundaries. Contextual knowledge based phoneme segmentation is applied to estimate the phoneme boundaries in a syllable and the number of 3D facial models can be effectively reduced. Parametric 3D tongue and lip movement models are smoothed by B-spline to eliminate the jerkiness and synthesize the 3D facial with tongue and lip animation. Integrating boundary information, the speech-synching can be easily accomplished. The mult-model feedbacks in 3D facial animation are used to improve the efficiency of articulatory training. The preliminary experimental results show that this method is feasible.

UR - http://www.scopus.com/inward/record.url?scp=78349270482&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78349270482&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-69139-6-19

DO - 10.1007/978-3-540-69139-6-19

M3 - Conference contribution

SN - 9783540691389

T3 - IFMBE Proceedings

SP - 56

EP - 60

BT - 4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008

ER -

Chen YJ, Liao FC, Wu J-L, Yang HM, Wu C-H. Mixmodel driven 3D facial synthesis to computer-aided articulation training. In 4th Kuala Lumpur International Conference on Biomedical Engineering 2008, Biomed 2008. 1 ed. 2008. p. 56-60. (IFMBE Proceedings; 1). https://doi.org/10.1007/978-3-540-69139-6-19