Multi-Relational Graph Convolutional Networks for Skeleton-Based Action Recognition

Abstract

In motion, the interaction relationship between the human body parts is diversified. However, the existing action recognition methods based on the graph convolution neural networks (GCNs) can only deal with a single relation of skeletons. Even some works describe different relations of the skeleton, the adjacency matrices of different relation graphs are added together. This paper proposes a multi-relational GCNs for action recognition following the idea of describing different relations between entities by knowledge graphs. The natural connection relation, symmetric connection relation, and global connection relation of the human body parts are modeled respectively. The features of the relations are transmitted and integrated through the network, which can improve the representation ability of features. Meanwhile, this paper proposes a two-stream multi-relational graph convolution networks (2S-MRGCNs), which processes the joint flow and the body part flow of skeleton data respectively to represent the action more comprehensively. The experimental results show that the 2S-MRGCNs model proposed in this paper has achieved state-of-the-art results on action recognition in kinetics and NTU-RGB+D datasets.