Infrared and visible image fusion based on dense connectivity and Transformer self-encoding

Abstract

Most of the existing deep learning-based infrared and visible image fusion methods use convolutional neural networks for feature extraction, but the perceptual field of convolutional neural networks becomes smaller as the depth of the convolutional layers increases, and it is difficult to focus on the global features of the source image. To address the above problems, this paper proposes a self-encodering based on the combination of dense connection module and Transformer module, and the Transformer module can extract the non-local interaction relationship of the source image from the entire image, focus more on the global features, and the dense connection module better understands the local features of the original image. In addition, the existing loss function lacks the deep semantic information of the image, and this paper introduces perceptual loss in the loss function to better retain the texture and detail information in the image of the image. After the source image is featured extracted by the self-encoder designed herein, the fused image is obtained by the reconstruction of the decoder. Experimental results show that compared with nine traditional and deep learning-based image fusion methods, the methods in this article have better performance in subjective comparison and objective image quality evaluation.