Temporal channel reconfiguration multi‐graph convolution network for skeleton‐based action recognition

Abstract

Abstract Skeleton‐based action recognition has received much attention and achieved remarkable achievements in the field of human action recognition. In time series action prediction for different scales, existing methods mainly focus on attention mechanisms to enhance modelling capabilities in spatial dimensions. However, this approach strongly depends on the local information of a single input feature and fails to facilitate the flow of information between channels. To address these issues, the authors propose a novel Temporal Channel Reconfiguration Multi‐Graph Convolution Network (TRMGCN). In the temporal convolution part, the authors designed a module called Temporal Channel Fusion with Guidance (TCFG) to capture important temporal information within channels at different scales and avoid ignoring cross‐spatio‐temporal dependencies among joints. In the graph convolution part, the authors propose Top‐Down Attention Multi‐graph Independent Convolution (TD‐MIG), which uses multi‐graph independent convolution to learn the topological graph feature for different length time series. Top‐down attention is introduced for spatial and channel modulation to facilitate information flow in channels that do not establish topological relationships. Experimental results on the large‐scale datasets NTU‐RGB + D60 and 120, as well as UAV‐Human, demonstrate that TRMGCN exhibits advanced performance and capabilities. Furthermore, experiments on the smaller dataset NW‐UCLA have indicated that the authors’ model possesses strong generalisation abilities.