Deep Spatial–Temporal Model Based Cross-Scene Action Recognition Using Commodity WiFi

Abstract

With the popularization of Internet-of-Things (IoT) systems, passive action recognition on channel state information (CSI) has attracted much attention. Most conventional work under the machine-learning framework utilizes handcrafted features (e.g., statistic features) that are unable to sufficiently describe the sequence data and heavily rely on designers' experiences. Therefore, how to automatically learn abundant spatial-temporal information from CSI data is a topic worthy of study. In this article, we propose a deep learning framework that integrates spatial features learned from the convolutional neural network (CNN) into the temporal model multilayer bidirectional long short-term memory (Bi-LSTM). Specifically, CSI streams are segmented into a series of patches, from which spatial features are extracted by our designed CNN structure. Considering long-term dependencies between adjacent sequences, the fully connected layer of CNN for each patch is taken as the Bi-LSTM sequential input to further capture temporal features. Our model is appealing in that it can simultaneously learn temporal dynamics and convolutional perceptual representations. To the best of our knowledge, this is the first work to explore deep spatial-temporal features for CSI-based action recognition. Furthermore, in order to solve the problem that the trained model fully fails with environmental changes, we use the off-the-shelf model as the pretrained model and fine-tune it in the new scenario. The transfer method is able to realize cross-scene action recognition with low computational consumption and satisfactory accuracy. We carry out experiments on indoor data and the experimental results validate the effectiveness of our algorithm.