UFN-GAN: An unsupervised generative adversarial network for remote sensing image fusion

Abstract

Different sensors acquire different images in the same area, such as multi-spectral (MS) images and panchromatic (PAN) images. Normally, the MS images possess high spectral resolution but low spatial resolution, while PAN images are opposite in the distribution of spectral and spatial information. Image fusion is a common method to obtain the information of PAN and MS images simultaneously. To generate clearer fusion image with abundant information, we design an unsupervised fusion net based on generative adversarial network (GAN), named UFNGAN for remote sensing image fusion. In our proposed UFNGAN, an adversarial net is designed between our generator and two discriminators to adequately retain the spectral and spatial information of original images without supervision. MS images and PAN images are fused by our generator, which consists of an encoder and a decoder. Our encoder is used to extract deeper feature maps of the original images, and the decoder is applied to rebuild images. Furthermore, the Spatial-Information-Enhancement (SIE) model is utilized to obtain spatial information of MS images for enhancing PAN image, and the Edge-Detection-Registration (EDR) method is applied to register the original images to avoid fused images distortion. At last, experiments are performed on QuickBird and GaoFen-2 datasets.