Gesture recognition for Indonesian Sign Language (BISINDO)

Abstract

Sign language is different from spoken language that emphasizes both the audio and verbal aspects. There are two sign languages applicable in Indonesia, namely Indonesian Signal System (SIBI) and Indonesian Sign Language (BISINDO). SIBI converts spoken Indonesian language into sign language and follows the Indonesian spoken language's grammatical structure complete with prefix and suffix. In contrast to SIBI, BISINDO translates one word from the Indonesian spoken language in accordance with its context followed by an expression showing the ongoing events. We choose BISINDO rather than SIBI in line with the deaf' people's suggestions and requests out there to make BISINDO as an official Indonesian sign language to replace SIBI. This research aims to develop a translator model of BISINDO through computer vision technology such as Microsoft Kinect XBox and machine translation using Hidden-Markov Model (HMM) with optimal number of hidden states. We utilize skeleton data from Kinect sensor for feature extraction. There are four kinds of skeleton features used in this study consisting of the movement of the shoulders, upper arms, forearms, and hands. The experiment results by using this methodology successfully recognize the gesture of BISINDO with an accuracy is around 60%.