Medical Image Fusion with Local-Global Feature Coupling and Cross-Scale Attention

Abstract

To address the insufficient global feature representation in existing deep learning-based multimodal medical image fusion methods, this study proposes a medical image fusion method based on local-global feature coupling and cross-scale attention.The method comprises three parts:encoder, fusion rule, and decoder.In the encoder, parallel Convolutional Neural Network(CNN) and Transformer dual-branch networks are used to extract the local features and global representation of the image, respectively.At different scales, the local features of the CNN branch are embedded into the global feature representation of the Transformer branch through the feature coupling module for combining complementary features;simultaneously, a cross-scale attention module is introduced to effectively utilize multiscale feature representation.The encoder extracts the local, global, and multiscale feature representations of the original images to be fused, fuses the feature representations of different source images through fusion rules, and then inputs them into the decoder to generate the fused image.Experiments show that compared with CBF, PAPCNN, IFCNN, DenseFuse, and U2Fusion methods, the proposed method objectively improves the five evaluation indicators of feature mutual information, spatial frequency, edge information transfer factor, structural similarity, and perceptual image fusion quality by 6.29%, 3.58%, 29.01%, 5.34%, and 5.77%, respectively;subjectively, the fusion images obtained using this method retain clearer texture details and higher contrast, which is convenient for disease diagnosis and treatment.