Super-resolution Thermal Generative Adversarial Networks for Infrared Image Enhancement

Abstract

Thermal infrared cameras create IR (infrared) images, thus enabling the recognition of objects regardless of weather, illuminance, or ambient color. With the recent development of deep learning, research interest in image conversion and super-resolution techniques has increased. This paper proposes an algorithm that converts electro-optical (EO) images to IR images using super-resolution techniques based on generative adversarial networks. ThermalWorld data were used as the learning data. Additionally, drones of EO and IR images were added using thermal infrared cameras. The proposed super-resolution technique adapts the loss function and neural network structure to generate a high-resolution IR image. The loss function learns the neural network by utilizing the difference between the actual image and the generated image, thereby generating an image while maintaining the shape of the object on the image. The resolution is further improved by densely connecting the generator neural network structure and removing batch normalization. Finally, the structure of the discriminator is changed, and the stability of learning improved using the spectral norm. Images are generated according to each change item and quantitatively verified through performance indicators of improved image quality. This study analyzes images made with super-resolution techniques by considering the results of performance indicators and discusses the possibility of using IR images; and presents future research directions.