Gesture Recognition Based on Flexible Data Glove Using Deep Learning Algorithms

Abstract

Gesture recognition based on wearable devices helps to build an intelligent human-computer interaction. However, the sensing units of current gesture acquisition devices are mostly rigid MEMS with poor user experience. Meanwhile, most existing studies directly stack gesture sensing data, ignoring the interaction of gesture signals within the same modal sensing channel and between different modal sensor channels in terms of spatiotemporal characteristics. To address the above problems, we use flexible data glove as gesture capture devices and propose a framework named self-attention temporal-spatial feature fusion for gesture recognition (STFGes) to recognize gestures by integrating multi-sensors data. In addition, we conduct comprehensive experiments to build a dataset that can be used for training and testing. The experimental results show that STFGes achieves 97.02% recognition accuracy for 10 dynamic daily Chinese Sign Language (CSL) and outperforms other methods.