Arabic Sign Language Recognition Using Kinect Sensor

Abstract

This study introduces a Real Time System for automatic Arabic sign language recognition system based on Dynamic time warping matching algorithm. The communication between human and machines or between people could done using gestures called sign language. The aim of the sign language recognition is to give an exact and convenient mechanism to transcribe sign gestures into meaningful text or speech so that communication between deaf and hearing society can easily be made. In this study we introduce a translator based on Dynamic Time Warping, where each signed word is coordinating and matching among database, then display the text and the corresponding pronunciation of the income sign. We using the Microsoft's Kinect sensor to catch the sign. We have built our data using a large set of samples for a dictionary of 30 isolated words homemade signs from the Standard Arabic sign language. The system operates in different modes including online, signer-dependent and signer-independent modes. The presented system allows the signer to do signs freely and naturally. Experimental results using real Arabic sign language data collected show that the presented system has higher recognition rate compared with others for all modes. For signer-dependent online case, the system achieves recognition rate of 97.58%. On the other hand, for signer-independent online case, the system achieves a recognition rate of 95.25%.