Towards Multi-Person Gesture Recognition using Commodity Wi-Fi

Abstract

Comparing the recognition of human gestures using cameras, radar, or LiDAR, a WiFi-based gesture recognition system has distinct advantages, such as being low-cost, being device-free, and having much less privacy leakage. Recently, there have been advancements in WiFi-based gesture recognition, but most of the research has primarily focused on single-person gesture recognition. However, in real-world scenarios like e-learning, it is common for multiple individuals to engage in different actions simultaneously. To this end, this paper focuses on multi-person gesture recognition, which presents two major challenges, that is, the recognition accuracy due to the WiFi signal interference, and the processing time for real-time applications. Multi-person gesture recognition is more challenging than single-person scenario due to the interference caused by the superposition of WiFi signals induced by multiple moving individuals. In this paper, we define a concept of super-gesture and propose a WiFi-based Super-Gesture recognition (WiSG) method. Through the decomposition of the super-gesture's DFS spectrogram by Multi-Motion Trajectory algorithm, we extract modified signals of each person. Moreover, a novel feature called Field Motion Velocity is proposed by fully exploiting the advantages of our multiple transmitter-receiver WiFi sensing system. The proposed feature is not significantly affected by domains such as position, orientation, and other factors irrelevant to gestures. As a result, our approach can effectively recognize gestures across different domains. Evaluation results show that the cross-domain recognition accuracy of our WiSG can achieve up to 89% in multi-person scenario. Moreover, our approach can reduce processing time by 20 times against Widar3.0, which satisfies the requirements of most real-time applications.