Facial phoneme extraction for Taiwanese sign language recognition

Shi Hou Lin, Pi Fuei Hsieh, Chung Hsien Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)


We have developed a system that recognizes the facial expressions in Taiwanese Sign Language (TSL) using a phoneme-based strategy. A facial expression is decomposed into three facial phonemes of eyebrow, eye, and mouth. A fast method is proposed for locating facial phonemes. The shapes of the phonemes were then matched by the deformable template method, giving feature points representing the corresponding phonemes. The trajectories of the feature points were tracked along the video image sequence and combined to recognize the type of facial expression. The tracking techniques and the feature points have been tailored for facial features in TSL. For example, the template matching methods have been simplified for tracking eyebrows and eyes. The mouth was tracked using the optical flow method, taking lips as homogeneous patches. The experiment has been conducted on 70 image sequences covering seven facial expressions. The average recognition rate is 83.3%.

Original languageEnglish
Title of host publicationAffective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings
PublisherSpringer Verlag
Number of pages8
ISBN (Print)3540296212, 9783540296218
Publication statusPublished - 2005
Event1st International Conference on ffective Computing and Intelligent Interaction, ACII 2005 - Beijing, China
Duration: 2005 Oct 222005 Oct 24

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3784 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other1st International Conference on ffective Computing and Intelligent Interaction, ACII 2005

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Facial phoneme extraction for Taiwanese sign language recognition'. Together they form a unique fingerprint.

Cite this