Multi-modal sign icon retrieval for augmentative communication

Chung Hsien Wu, Yu Hsien Chiu, Kung Wei Cheng

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)


This paper addresses a multi-modal sign icon retrieval and prediction technology for generating sentences from ill-formed Taiwanese sign language (TSL) for people with speech or hearing impairments. The design and development of this PC-based TSL augmented and alternative communication (AAC) system aims to improve the input rate and accuracy of communication aids. This study focuses on 1) developing an effective TSL icon retrieval method, 2) investigating TSL prediction strategies for input rate enhancement, 3) using a predictive sentence template (PST) tree for sentence generation. The proposed system assists people with language disabilities in sentence formation. To evaluate the performance of our approach, a pilot study for clinical evaluation and education training was undertaken. The evaluation results show that the retrieval rate and subjective satisfactory level for sentence generation was significantly improved.

Original languageEnglish
Title of host publicationAdvances in Multimedia Information Processing - PCM 2001 - 2nd IEEE Pacific Rim Conference on Multimedia, Proceedings
EditorsShih-Fu Chang, Heung-Yeung Shum, Mark Liao
PublisherSpringer Verlag
Number of pages8
ISBN (Print)3540426809, 9783540426806
Publication statusPublished - 2001 Jan 1
Event2nd IEEE Pacific-Rim Conference on Multimedia, IEEE-PCM 2001 - Beijing, China
Duration: 2001 Oct 242001 Oct 26

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other2nd IEEE Pacific-Rim Conference on Multimedia, IEEE-PCM 2001

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Multi-modal sign icon retrieval for augmentative communication'. Together they form a unique fingerprint.

Cite this