Device-Free and Training-Free Hand Gesture Recognition with Acoustic Signal

Abstract

Hand gesture recognition is an essential Human Computer Interaction (HCI) mechanism for users to control smart devices. While traditional device-based methods support acceptable recognition performance, the recent advance in wireless sensing could enable device-free hand gesture recognition. However, two severe limitations are serious environmental interference and high-cost hardware, which hamper the wide deployment. This paper proposes a novel system TaGesture, which employ the inaudible acoustic signal to realize device-free and training-free hand gesture recognition with a pair of commercial speaker and microphone array. We address unique technical challenges, such as proposing a novel acoustic hand tracking smoothing algorithm with Interaction Multiple Model (IMM) Kalman Filter to address the issue of localization angle ambiguity, and designing a classification algorithm to realize acoustic-based hand gesture recognition without training. Comprehensive experiments are conducted to evaluate TaGesture. Results show that it can achieve a total accuracy of 97.5% for acoustic-based hand gesture recognition, and support the furthest sensing range of up to 3 m.