DiffMamba: semantic diffusion guided feature modeling network for semantic segmentation of remote sensing images

Abstract

With the rapid development of remote sensing technology, the application scope of high-resolution remote sensing images (HR-RSIs) has been continuously expanding. The emergence of convolutional neural networks and Transformer models has significantly enhanced the accuracy of semantic segmentation. However, these methods primarily focus on local feature extraction and long-range dependency modeling of global information, neglecting the spatial correlation of local features, which leads to poor segmentation of small-scale regions. To address this issue, based on Diffusion Model and State Space Model (SSM), we propose a semantic diffusion guided feature modeling network (DiffMamba) for HR-RSI semantic segmentation. DiffMamba uses a hybrid CNNs-Transformer as the encoder structure, and is equipped with the efficient phase sensing module (EPSM), the multi-view transformer module (MVTrans), the semantic diffusion alignment module (SDAM), and the coordinate state space model (CAMamba). EPSM focuses on enhancing local feature representation in the channel dimension, using the phase information of object region features to improve local information interaction and filter out clutter noise interference. MVTrans can observe the spatial location information of the object region from various perspectives to obtain refined global context details. SDAM utilizes the diffusion propagation process to fuse local and global information, alleviating the feature redundancy caused by semantic information differences. CAMamba employs state space transformation to construct the correlation of enhanced local features, and guides the model to achieve feature decoding to obtain refined semantic segmentation results. Extensive experiments on the widely used ISPRS 2-D Semantic Labeling dataset and the 15-Class Gaofen Image dataset confirm the superior efficiency of DiffMamba over several state-of-the-art methods.