Traffic Police 3D Gesture Recognition Based on Spatial–Temporal Fully Adaptive Graph Convolutional Network

Abstract

It is critical for autonomous vehicles to recognize traffic police gestures timely and accurately. During the movement of the vehicle, the collected traffic police scales change all the time, in addition, the frequency and amplitude of actions of different traffic police are different. First, we use gesture normalization to fix the traffic police actions at a unified scale and remove the influence of scale changes on traffic police gesture recognition. Meanwhile, a fully adaptive spatial-temporal graph convolution network (FA-STGCN) is proposed to recognize the actions with different amplitude and frequencies. The adaptive spatial graph network can dig the latent joints connection relation of the traffic police under different gestures, which weakens the amplitude impact on the action recognition. The adaptive temporal graph network is composed of the global temporal module and the local temporal module. The global temporal module can obtain the coarse-grained features of the traffic police gestures’ speed and then naturally use the coarse-grained features to guide the local temporal module to adaptively learn the fine-grained temporal features of the traffic police action. The adaptive spatial graph network and the temporal graph network are alternately stacked to finally output accurate traffic police gestures. We thoroughly evaluated our method through intensive experiments, the result shows that our method achieved the best results on public datasets. What’s more, we proofed the effectiveness of each module and verified our methods for moving vehicles for the first time, the performance present meets the vehicle’s practical requirements.