Infrared visible color night vision image fusion based on deep learning

Abstract

In recent decades, with the rapid development of image sensor technology, image acquisition has gradually evolved from a single sensor mode to a multi-sensor mode. The data information obtained by a single sensor is limited, and the use of multi-source data fusion can provide a more accurate understanding of the observation scene. This paper proposes a network structure of infrared visible color night vision image fusion based on deep learning. The network adopts a fusion-encoding-decoding structure for end-to-end learning to achieve the purpose of color night vision image fusion, making the image more in line with human visual effects. The fusion structure contains a multi-scale feature extraction block and a channel attention block, which perform feature extraction on low-resolution infrared images and visible images respectively. The multi-scale feature block can expand the receptive field and avoid losing too much feature information. The channel attention block can improve the sensitivity of the network to channel characteristics. A certain number of convolutional layers and deconvolutional layers are used in the network to realize the encoding and decoding of the feature map to achieve the purpose of restoring the color fusion image. After experimental verification, our method has a good fusion effect, rich colors, and conforms to the human visual effect.