LQCANet: Learnable-Query-Guided Multi-Scale Fusion Network Based on Cross-Attention for Radar Semantic Segmentation

Abstract

Millimeter-wave radar semantic segmentation has proven successful in autonomous driving environment perception tasks. However, relying solely on range-angle (RA) images excludes Doppler information, which is crucial for dynamic target recognition. Despite various fusion method proposals, they consistently encounter issues with interfering RA information and poor fusion results. We introduce a novel learnable-query-guided multi-scale fusion network (LQCANet) for radar semantic segmentation, leveraging learnable-query for effective multi-scale cross-attention fusion. The cross-attention fusion module (CAF) initializes queries randomly, interacting with range-Doppler (RD) and angle-Doppler (AD) information via multi-layer cross-attention. Subsequently, the original RA features integrate with the generated queries to achieve multi-scale feature fusion. This approach prevents interference with RA information and ensures efficient fusion. Additionally, for enhanced feature extraction capabilities, this study introduces the pointwise multi-head self-attention down module (PMD), integrating a convolutional neural network (CNN) and a Transformer to extract both local and global features. Furthermore, pointwise convolution serves as an implicit positional coding method, addressing the limitation that explicit positional coding is not applicable to millimeter-wave radar images. Experiments demonstrate the superior performance of the proposed LQCANet on the CARRADA dataset. In comparison to the state-of-the-art (SOTA) fusion-network TMVA-Net, the LQCANet exhibits a significant improvement of 2.7 in mean intersection over union (mIoU) and 3.4 in mean dice similarity coefficient (mDice), while maintaining a computational complexity of only 27% (27.6 GFLOPs) of TMVA-Net. LQCANet achieves a superior trade-off between detection accuracy and speed, rendering it more suitable for environment perception tasks. Our code is available at https://github.com/Zhuanglong2/LQCANet .