Facial phoneme extraction for Taiwanese sign language recognition

Shi Hou Lin, Pi Fuei Hsieh, Chung-Hsien Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We have developed a system that recognizes the facial expressions in Taiwanese Sign Language (TSL) using a phoneme-based strategy. A facial expression is decomposed into three facial phonemes of eyebrow, eye, and mouth. A fast method is proposed for locating facial phonemes. The shapes of the phonemes were then matched by the deformable template method, giving feature points representing the corresponding phonemes. The trajectories of the feature points were tracked along the video image sequence and combined to recognize the type of facial expression. The tracking techniques and the feature points have been tailored for facial features in TSL. For example, the template matching methods have been simplified for tracking eyebrows and eyes. The mouth was tracked using the optical flow method, taking lips as homogeneous patches. The experiment has been conducted on 70 image sequences covering seven facial expressions. The average recognition rate is 83.3%.

Original languageEnglish
Title of host publicationAffective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings
Pages187-194
Number of pages8
Publication statusPublished - 2005 Dec 1
Event1st International Conference on ffective Computing and Intelligent Interaction, ACII 2005 - Beijing, China
Duration: 2005 Oct 222005 Oct 24

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3784 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other1st International Conference on ffective Computing and Intelligent Interaction, ACII 2005
CountryChina
CityBeijing
Period05-10-2205-10-24

Fingerprint

Sign Language
Template matching
Optical flows
Facial Expression
Feature Point
Trajectories
Image Sequence
Experiments
Deformable Templates
Template Matching
Optical Flow
Patch
Covering
Trajectory
Experiment

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Lin, S. H., Hsieh, P. F., & Wu, C-H. (2005). Facial phoneme extraction for Taiwanese sign language recognition. In Affective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings (pp. 187-194). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3784 LNCS).
Lin, Shi Hou ; Hsieh, Pi Fuei ; Wu, Chung-Hsien. / Facial phoneme extraction for Taiwanese sign language recognition. Affective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings. 2005. pp. 187-194 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{6d0fdede662c43a7b03698a8d9f380f4,
title = "Facial phoneme extraction for Taiwanese sign language recognition",
abstract = "We have developed a system that recognizes the facial expressions in Taiwanese Sign Language (TSL) using a phoneme-based strategy. A facial expression is decomposed into three facial phonemes of eyebrow, eye, and mouth. A fast method is proposed for locating facial phonemes. The shapes of the phonemes were then matched by the deformable template method, giving feature points representing the corresponding phonemes. The trajectories of the feature points were tracked along the video image sequence and combined to recognize the type of facial expression. The tracking techniques and the feature points have been tailored for facial features in TSL. For example, the template matching methods have been simplified for tracking eyebrows and eyes. The mouth was tracked using the optical flow method, taking lips as homogeneous patches. The experiment has been conducted on 70 image sequences covering seven facial expressions. The average recognition rate is 83.3{\%}.",
author = "Lin, {Shi Hou} and Hsieh, {Pi Fuei} and Chung-Hsien Wu",
year = "2005",
month = "12",
day = "1",
language = "English",
isbn = "3540296212",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "187--194",
booktitle = "Affective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings",

}

Lin, SH, Hsieh, PF & Wu, C-H 2005, Facial phoneme extraction for Taiwanese sign language recognition. in Affective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3784 LNCS, pp. 187-194, 1st International Conference on ffective Computing and Intelligent Interaction, ACII 2005, Beijing, China, 05-10-22.

Facial phoneme extraction for Taiwanese sign language recognition. / Lin, Shi Hou; Hsieh, Pi Fuei; Wu, Chung-Hsien.

Affective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings. 2005. p. 187-194 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3784 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Facial phoneme extraction for Taiwanese sign language recognition

AU - Lin, Shi Hou

AU - Hsieh, Pi Fuei

AU - Wu, Chung-Hsien

PY - 2005/12/1

Y1 - 2005/12/1

N2 - We have developed a system that recognizes the facial expressions in Taiwanese Sign Language (TSL) using a phoneme-based strategy. A facial expression is decomposed into three facial phonemes of eyebrow, eye, and mouth. A fast method is proposed for locating facial phonemes. The shapes of the phonemes were then matched by the deformable template method, giving feature points representing the corresponding phonemes. The trajectories of the feature points were tracked along the video image sequence and combined to recognize the type of facial expression. The tracking techniques and the feature points have been tailored for facial features in TSL. For example, the template matching methods have been simplified for tracking eyebrows and eyes. The mouth was tracked using the optical flow method, taking lips as homogeneous patches. The experiment has been conducted on 70 image sequences covering seven facial expressions. The average recognition rate is 83.3%.

AB - We have developed a system that recognizes the facial expressions in Taiwanese Sign Language (TSL) using a phoneme-based strategy. A facial expression is decomposed into three facial phonemes of eyebrow, eye, and mouth. A fast method is proposed for locating facial phonemes. The shapes of the phonemes were then matched by the deformable template method, giving feature points representing the corresponding phonemes. The trajectories of the feature points were tracked along the video image sequence and combined to recognize the type of facial expression. The tracking techniques and the feature points have been tailored for facial features in TSL. For example, the template matching methods have been simplified for tracking eyebrows and eyes. The mouth was tracked using the optical flow method, taking lips as homogeneous patches. The experiment has been conducted on 70 image sequences covering seven facial expressions. The average recognition rate is 83.3%.

UR - http://www.scopus.com/inward/record.url?scp=33646758220&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33646758220&partnerID=8YFLogxK

M3 - Conference contribution

SN - 3540296212

SN - 9783540296218

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 187

EP - 194

BT - Affective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings

ER -

Lin SH, Hsieh PF, Wu C-H. Facial phoneme extraction for Taiwanese sign language recognition. In Affective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings. 2005. p. 187-194. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).