Publication

Abstract : This paper presents an application aiding the social integration of the deaf community in India into the mainstream of society. This is achieved by feeding text in English and generating an animated gesture sequence representative of its content. This application consists of three main portions: an interface that allows the user to enter words, a language processing system that converts English text to ISL format and a virtual avatar that acts as an interpreter conveying the information at the interface. These gestures are dynamically animated based on a novel method devised by us in order to map the kinematic data for the corresponding word. The word after translation into ISL will be queried in the database where in lies the notation format for each word. This notation called as LOTS Notation will represent parameters enabling the system to identify features like hand location(L), Hand Orientation (O) in the 3D space, Hand Trajectory movement (T), hand shapes (S) and non-manual components like facial expression. The animation of a sentence fed is thus produced from the sequence of notations which are queued in order of appearance. We are also inserting the movement –epenthesis which is the inter sign transition gesture inorder to avoid jitters in gesturing. More than a million deaf adults and around half a million deaf children in India use the Indian Sign Language (ISL) as a mode of communication. However, this system would serve as an initiative in propelling the Sign Language Communication in the Banking Domain. The low audio dependency in the working domain supports the cause.