Infrared and Visible Image Fusion Based on Multiscale Adaptive Transformer

Abstract

In our study, we introduce an innovative Transformer-based approach that utilizes multiscale adaptivity for the fusion of infrared and visible images. First of all, we propose a three-branch network structure to extract multiscale differentiated features of source images, and a cross-modal feature interaction module is designed to realize the information interaction of infrared and visible images. And then, inspired by Swin Transformer, a novel adaptive Transformer fusion network is proposed to fuse multiscale features, which fully considers the global information preservation issue during the fusion process and could better integrate the differential and complementary features of infrared and visible images. Furthermore, we present a cross-correlation loss grounded in correlation coefficients to foster a more robust relationship between the fused output and the original images through cross-correlation. The concluding tests reveal that our method's fusion outcomes adeptly harmonize the complementary attributes of various source images, leading to enhanced visual quality and perception.