Edge-Supervised Attention-Aware Fusion Network for RGB-T Semantic Segmentation

Abstract

To address the limitations in the efficiency of modality feature fusion in existing RGB-T semantic segmentation methods, which restrict segmentation performance, this paper proposes an edge-supervised attention-aware algorithm to enhance segmentation capabilities. Firstly, we design a feature fusion module incorporating channel and spatial attention mechanisms to achieve effective complementation and enhancement of RGB-T features. Secondly, we introduce an edge-aware refinement module that processes low-level modality features using global and local attention mechanisms, obtaining fine-grained feature information through element-wise multiplication. Building on this, we design a parallel structure of dilated convolutions to extract multi-scale detail information. Additionally, an EdgeHead is introduced after the edge-aware refinement module, with edge supervision applied to further enhance edge detail capture. Finally, the optimized fused features are fed into a decoder to complete the RGB-T semantic segmentation task. Experimental results demonstrate that our algorithm achieves mean Intersection over Union (mIoU) scores of 58.52% and 85.38% on the MFNet and PST900 datasets, respectively, significantly improving the accuracy of RGB-T semantic segmentation.