Infrared and Visible Image Fusion Based on Autoencoder Network

Abstract

ABSTRACT To overcome the problems of texture information loss and insufficiently prominent targets in existing fusion networks, an information decomposition‐based autoencoder fusion network for infrared and visible images is proposed in this paper. Two salient information encoders with unshared weights and two scene information encoders with shared weights are designed to extract different features from infrared and visible images, respectively. The constraint is added to the loss function in order to ensure the ability of the salient information encoders to extract representative features and the scene information encoder to extract the cross‐modality feature. In addition, by introducing the pre‐trained semantic segmentation networks to guide the network training and constructing a feature saliency‐based fusion strategy, the ability of the fusion network is further enhanced to distinguish between targets and backgrounds. Extensive experiments are carried out on five datasets. Comparison experiments with state‐of‐the‐art fusion networks and ablation experiments indicate that the proposed method can obtain fused images with richer and more comprehensive information and is more robust to challenging factors, such as strong and weak light smoke and fog environments. At the same time, the fused images by our proposed method are more beneficial for downstream tasks such as target detection.