Natural language processing for Taiwanese sign language to speech conversion

This paper addresses some natural language processing technologies in generating sentences from ill-formed Taiwanese Sign Language (TSL) for people with speech or hearing impairments. The design and development of the PC-based TSL Augmentative and Alternative Communication (AAC) system aims to improve the input rate and accuracy of communication aids. We demonstrate 1) developing an effective TSL virtual keyboard to assist users to efficiently input sign sequences, 2) investigating TSL prediction strategies for input rate enhancement, 3) integrating bottom-up parsing with top-down filtering strategy and variable N-gram language model for sentence generation and selection. The proposed system assists people with language defects in sentence formation and grammatical correction. To evaluate the performance of our approach, a pilot study for clinical evaluation and education training was undertaken. Evaluation results show that the generation rate and subjective satisfactory level for sentence construction have been significantly improved.