Publication

Abstract : The paper proposes the design and implementation of a model that translates live voice or audio recordings of a native Indian regional language (Telugu) to text and further matches it to sign language animations from the pre-defined Graphics Interchange Format (GIF) dataset. The speech is converted to text using Google API. If the text matches with the words in GIF dataset, it generates the respective GIF. A GUI interface is also developed which displays ISL text alphabetically if the given input is outside the range of GIF dataset. Alongside, the performance of four powerful deep learning networks viz. RNN, LSTM, Bi-LSTM, and GRU, has been compared and trained over a Telugu multi-speaker speech dataset for speech recognition. The proposed models yielded accuracies of 51% for RNN, 68% for LSTM, 90% for Bi-LSTM and 88% for GRU. On evaluating the models, Google API-based speech to sign language conversion has been most promising and consistent with precision and recall scores of 94% each and a word error rate of 4.1%.