End-to-End Two-Handed Sign Language Translation

Abstract

Sign language recognition and translation bridges the gap between hearing-impaired people and ordinary people. Compared with sign language recognition (SLR), continuous sign language translation (CSLT) is closer to people's speaking habits, and has greater practicality. However, there are problems such as insufficient information and short pauses in continuous sentences that are difficult to separate. To solve these problems, this paper gets coarse-grained arm movement, fine-grained finger movement and hand rotation information through the MYO armband; an encoder-decoder model with attention mechanism translates in an end-to-end manner without segmentation. After a series of experiments, the best model was selected, achieving an accuracy of 94.1%.