Indian Sign Language Translation using Deep Learning

Abstract

Indian Sign Language is the language used by specially abled population in the Indian subcontinent to communicate with each other. Unfortunately the general population is not aware of the semantics of Indian Sign Language. In this work we present three deep architectures to translate a given video sequence containing the Indian Sign Language sentence to English Language sentence. We have tried to solve this problem using three approaches. First using an LSTM based Sequence to Sequence model(Seq2Seq), second using an LSTM based Seq2Seq model utilising attention, third using an Indian Sign Language Transformer. These models were evaluated on BLEU scores and the transformer model gave a perfect BLEU score of 1.0 on test data.