A Joint Global–Local Network for Human Pose Estimation With Millimeter Wave Radar

Abstract

This article proposes a two-branch learning model, namely, the joint global–local network, for human pose estimation (HPE) using millimeter wave radar. The aim of this work is to remediate the ill-posed problems in HPE arising from using the destructive observations with superimposed reflection signals. In the developed two-branch learning model, the global branch takes use of the superimposed signals from the whole human body to reconstruct the coarse pose estimation from a global perspective, and the local branch is responsible for fining the pose estimations with the decomposed signals from individual body parts in a complementary way. In doing this, two branch learning processes will be coordinated with the followed attention-based fusion module in terms of the local and global consistency. It is remarkable that the learning driven by the decomposed signals is motivated by exploiting the spatial-temporal evolution patterns of individual body parts for inferring the corresponding movements, which plays a crucial yet complementary role in the collaboration with the learning driven by the superimposed signals. With the two-branch learning architecture, the proposed method is advantageous in incorporating the local motion constraints from individual body parts into the coarse global estimation from the whole human body, which contributes to reconstructing plausible yet accurate pose estimations with the local and global kinematic consistency. Extensive experiments are presented to demonstrate the effectiveness of the proposed method.