Effective mmWave Radar Object Detection Pretraining Based on Masked Image Modeling

Abstract

With the advancement of environmental perception technology, millimeter-wave (mmWave) radar is emerging as a predominant sensor. While deep learning has facilitated the development of mmWave radar object detection (ROD) techniques, mmWave ROD suffers from datasets because the annotation of mmWave datasets is inherently more complex. Motivated by masked image modeling (MIM), this article proposes a novel pretraining method for ROD to address the limitations posed by datasets. This study conducts masking operations on mmWave radar images from both spatial and temporal perspectives, followed by a straightforward image reconstruction proxy task. To the best of authors' knowledge, our method represents the inaugural application of the MIM self-supervision method to ROD tasks. Additionally, we designed a lightweight self-supervised ROD network (SS-RODNet). Numerous ablation experiments have demonstrated the effectiveness of the proposed method. The pretrained SS-RODNet attains comparable results to the state-of-the-art (SOTA) on CRUW and CARRADA datasets with fewer parameters and floating-point operations per second (FLOPs).