IGNFusion: An Unsupervised Information Gate Network for Multimodal Medical Image Fusion

Abstract

Multimodal medical image fusion aims to merge saliency and complementary information from different source images to assist in biomedical diagnoses. How to effectively utilize feature information in the encoder is a critical issue. However, many existing medical image fusion methods do not consider the contributions of different convolution blocks. In this paper, we propose an information gate module (IGM) to control the contribution of each encoder feature level to the decoder; it is termed the information gate network for multimodal medical image fusion (IGNFusion). Furthermore, the Siamese multi-scale cross attention fusion module (SMSCAFM) integrates saliency and complementary information from multiple source images. Moreover, to constrain the similarity between the fused image and multiple source images, we introduce a saliency weight (SW). Extensive experiments on ten categories of multimodal medical images (i.e., CT $\& $ MR-T1 (T1 weighted) and PET $\& $ MR-T2 (T2 weighted)) show that our IGNFusion approach achieves significant improvements over 9 state-of-the-art methods.